LLMs aren't remotely like compilers or power tools

Just to add - I’m working on something very novel and LLM’s are absolutely useless at them. It’s actually comical to see the outputs - when you work on novel stuff you quickly see what these models actually are.

That mistake was made more than two years ago by whatever experimental version of Google's AI overview (which has strict performance requirement- i.e. needs to answer within a split-second) was up at the time. In other words, it was a very small and primitive system specialised in spitting answers as quickly as possible without a second thought. I hope you realise that basing your assessment of what LLMs can or can't do on that example is not much better than suggesting to put glue on pizza. A mistake that, if we were to adopt your reasoning, would in turn set a hard limit to the analytic skills of all humanity.

> But fundamentally any metaphor will be inadequate, LLMs are too weird. It doesn't make sense to compare a transformer-based deep neural network that autoregressively generates language tokens based on their embedding in a high-dimensional semantic space to really anything that's ever existed.

You might not have read the full post ha

The AI boom is happening because people what to believe that natural language is precise. Not because it’s actually an effective way to interface with everything.

I wonder why he decided to express this thought in natural language though. Should have tried formalising it.

I especially liked table saws and backhoes as examples of _automation_.

Formal languages are called 'formal' because they have rigorously defined semantics. The 'undefined' parts, if anything, reveal that most of the expressible sentences have clearly defined semantics, rather than lack of them.

In addition, the semantics of 'undefined' behaviors is not 'non-deterministic' in the literal sense, since it is, well, undefined.

That doesn't mean the response is correct though. You've just hard-locked the model to one possible response selection.

If you and I do this but we use different seeds, we still have no way to tell whose response will be better for every question or even a class of questions.

There will be responses where the model is outright wrong and we want to change seed or introduce some randomness by raising temperature, in the hope the model answers correctly next time.

Now we're back to running with a random seed and high temperature.

Also, if you want me to believe that LLMs have made you that much more productive, please share some links for useful things you've built recently using them. I'm thrilled if actual things can be made, but I haven't seen it yet.