I don’t know what people do with the open models but having tried a lot of them I just can’t make it make sense. they’re too dumb and it effectively makes them useless (to me). it’s probably worth being honest about the low ceiling here.
But also, everyone says "it's the harness" and almost nobody ever gives good examples, it gets a bit tiring to read everywhere, as if everyone wants to sell a harness to us.
But here’s the standard question: At what speeds/other limiting factors?
Co-Authored By: Haiku 4.5
If I ask something like: "I'm building a simple X as a learning exercise, I'm writing the code so please only answer the question I'm asking and don't try to solve the problem directly. How does ..." There's a 30% chance it starts reading and writing code immediately and a 20% chance it argues with a "design decision" that will bite me in the non-existent future of my learning exercise. If I ask a follow up question, naively assuming that the context from my original question still stands without repeating, it will almost assuredly start making modifications to my code.
Perhaps reverse the question: Are your build/construct requirements just really counter-productive to how LLMs need to understand things?
I've found constructing the code, writing the tests, adding the docs; then running through them gets most of the way there.
I've also found that making a simple obvious edit is a useless endevour when the LLM is primed for the long context tasks.
So, again, the question is reversed: are you over reliant on the LLM to do even stupid simple likes like editting a css variable?
Yeah, expecting the world when all you have is a 8GB graphics card? You're going to be disappointed.
16GB is table stakes (IQ3_XSS). 32 GB is better.