4,607 karma · joined February 20, 2007
It should be OpenRouter's responsibility to protect you against it, by regularly benchmarking providers and giving you the control to avoid bad providers. In fact, that's a big opportunity for them, since it justifies their place as a middleman between users and inference providers.
I feel there is still a lot of progress to be made before we can really trust LLM providers.
Once you have started working seriously with these tools, it stops being so easy to switch. What are the open-source alternatives to this? How usable are they for non-technical users?
Another comment I have about this is that unfortunately, there is always an LLM in the loop for each prompt. I feel LLMs should automate themselves away, meaning that for repetitive tasks, user prompts should go directly to deterministic, previously built scripts. This would be both a better user experience (more predictable), and a lot cheaper to operate.
There is the key insight that you don't need to explicitly compute this projection.
The wake up call might be difficult, for the whole country or even the whole world.
A huge waste and completely demoralizing though.
I don't have a lot of confidence for legislative solutions. Although I admire people who keep trying.
Unless someone comes up with a brilliant optimization strategy or new hardware that renders all that inefficient Nvidia crap overnight.
This doesn't mean anything. All LLM output is like that.
That said, I agree that LLMs are terrible at grading stuff, except perhaps if you give them a very detailed evaluation grid.