2,808 karma · joined June 13, 2012
But I use the per-token APIs for my usage, not a subscription, so I'm guessing I'm in the minority.
However, from a game theory perspective, when there's a subscription, the model makers are incentivized to maximize problem solving in the minimum amount of tokens. With per-token pricing, the incentive is to maximize problem solving while increasing token usage.
The online activity of the homelab community leans towards those who treat it as an enjoyable hobby as opposed to a pragmatic solution.
I'm on the other side of the spectrum. Devops is (at best) a neutral activity; I personally do it because I strongly dislike companies being able to do a rug-pull. I don't think you'll see setups like mine too often, as there isn't anything to brag about or to show off.
EDIT: after significant prompting, it actually solved it. I think it's the first one to do so in my testing.
It goes the other way too. Given that LLMs are just lossless compression machines, I do sometimes wonder how much better they are at compressing plain text compared to zstd or similar. Should be easy to calculate...
EDIT: lossless when they're used as the probability estimator and paired with something like an arithmetic coder.
The absolute cheapest lumens per dollar COB would be the GVM SD300D, although I highly question the reliability and light quality.
One downside is that the active fan cooling design is questionable - the air goes over the top of the LEDs, and there aren't any dedicated exit holes so the air is just squeezing through the very small gap between the glass and the heatsink. There are also blotches of paint that worsen the TIM contact between the PCB and the heatsink. I used a rotary tool to remove those blotches.
I don't have a problem with collecting, but I'd love for the distinction to be more upfront.
My application has some particularities (important and easy to identify per-token signals) that result in values growing (about 3x to 10x) through layers even in the baseline.
https://news.ycombinator.com/item?id=46302002
https://morethanmoore.substack.com/p/solving-the-problems-of...