5,015 karma · joined June 8, 2017
> Users care both about model performance and the expense of using models. There are multiple different types of costs and prices involved in model creation and usage:
> • Training cost: the amount spent by an AI company on compute, labor, and other inputs to create a new model.
> • Inference serving cost: the amount spent by an AI company on datacenters and compute to make a model available to end users.
> • Token price: the amount paid by end users on a per-token basis.
> • End-to-end expense for end users: the amount paid by end users to use a model to complete a task.
> End users are ultimately most affected by the last of these: end-to-end expenses. End-to-end expenses are more relevant than token prices because the number of tokens required to complete a task varies by model. For example, model A might charge half as much per token as model B does but use four times the number of tokens to complete an important piece of work, thus ending up twice as expensive end-to-end.
Hormones are basically messages sent through an animal's body to signal some change should take place. It was discovered that there was a hormone called hCG produced by the human placenta that triggers "you're pregnant" changes in the body. hCG is also present in the urine.
So if you want to detect a hormone, the idea is you inject it into an animal and see if it triggers the relevant changes (since the changes are usually internal, you generally need to kill the animal to check). So you would look for an animal that responds somehow to the hCG hormone, inject urine into it, and check for the response. Mice and rabbits were first used, but it was eventually discovered that certain species of frog that are highly sensitive to hormonal changes made for much simpler and faster testing.
Many people have already said designing for iffy internet helps everyone: this is true for slimming your payload, but not necessarily designing around dropped connections. On a plane or train, you might alternate between no internet and good internet, so you can just retry anything that failed when the connection is back, but a rural connection can be always spotty. And I think the calculus for devs isn't clearly positive when you have to design qualitatively new error handling pathways that many people will never use.
For example, cloning a git repo is non-resumable. Cloning a larger repo can be almost impossible since the probability the connection doesn't drop in the middle falls to zero. The sparse checkout feature has helped a lot here. Cargo also used to be very hard to use on rural internet until sparse registries.