12,786 karma · joined September 1, 2012
Engineering at Known (https://known.com)
Co-founder and tinkerer at Supernotes (https://supernotes.app)
A conversation with 20 turns, 50k tok growth per turn, 1m tok context at end would price out like this:
Fable 5 ($1/M cache reads) ; cache reads 9.5M tok × $1.00 = $9.50 ; cache writes 1M tok × $12.50 = $12.50 ; output 1M tok × $50 = $50.00 ; total = $72.00
Fable 5.1 ($0.25/M cache reads) ; cache reads 9.5M tok × $0.25 = $2.38 ; cache writes 1M tok × $12.50 = $12.50 ; output 1M tok × $50 = $50.00 ; total = $64.88
So yes, cheaper, but not massively.
- if you're gonna order the rest of the bar chart by rank, order your model accordingly.
- if you're gonna highlight a winner in a table of benchmarks, don't highlight your entire model row in the table.
Etc etc
Since most people in Indonesia get their drinking water from these jugs, stating this filter's output in "gallon" terms is just the easiest way to communicate potential savings etc (since people pay per jug, so they think in jug terms, not in any other unit of measure).
That said, effectiveness actually is a frequent shortcut to goodness, in that effectiveness often leads to efficiency and efficiency is one of the best methods of indiscriminate uplifting of people in an otherwise unfair and imbalanced world.
Also I do like Mistral's seemingly newer strategy of focusing on smaller, more fine-tuned models for various use-cases, presumably the result of their large MoE models not competing effectively with the frontier models.