Hacker News Reader

Comment by sajithdilshan | Hacker News Reader

I feel like the negativity is mainly due to lack of understanding on how LLMs work under the hood. The more you learn about it, the more you realize that it’s just a glorified autocomplete machine and the reason why it looks so capable is the engineering behind prompt and harness.

Unless there is another breakthrough in model training, I don’t see AI taking over anytime soon. However I do agree that’s it has become another tool, the engineers can use to increase their productivity which is a positive thing

View on HN

> glorified autocomplete machine

It is a next token prediction function and it is important to understand the technology accurately based on what it actually is.

What is unique about a next token prediction function though is that every computer program is just a string of instructions. At the theoretical limit a next token prediction function can generate the entire instruction stream (boot loader, OS, application) so a next token prediction function can theoretically generate any computer program, which means that it is a universal predictor for anything that a computer can simulate. Still not AGI/ASI in the woo-woo non-technical interpretations of those terms, but incredibly powerful.

What you’re saying is correct if the model is trained with all the knowledge humanity had, has and ever produce. But at the moment the next token prediction is quite limited to the training data.

Things could change if the model supports re-inforced leaning. That way the LLM would change the weights in real time based on a feedback loop, but again that could vastly improve the quality of the token prediction or completely degrade it as well

The distinction I would make here is that computer code is logical transformations on arbitrary data, not the actual data itself. An LLM can learn the entire space of logical transformation patterns from existing code, and can hallucinate new logical transformations, using a computer as a validator for the logic, so an LLM can create new logic as well as repeat existing patterns, and that logic can be applied to novel input data that the LLM has never seen before.

That’s not how LLMs work at the moment as far as I understand. LLM would not hallucinate any new logical transformation, rather just predict a transformation from its training data.

I understand that there can be many different combinations for all the logical transformations in the training data. But still the number of combinations are finite and I would assume that large number of those combinations would not result in any meaningful outcome.

Best outcome is that it just predicts a new pattern we haven’t discovered (LLM randomly connected the correct dots) one example is protein folding.

> the engineers can use to increase their productivity which is a positive thing

This might be the underlying hidden psychology: being more productive and efficient isn't benefitting the working class. It won't make them more money or make them work less hours, it just means overall more work for equal or less pay or less jobs for programmers outright. It doesn't make things cheaper or better, it just translates into more profit to the owning class. Of course nobody can articulate their anxiety's like that in a hyper capitalist society, but that might be the real reason underneath anti ai sentiment.

That’s not true. If you look at the history of humanity, every invention (internet, automobiles, telephones, vaccines, etc) has increased the productivity of human race and in-return increased the quality of life and lifted millions of people out of abject poverty. Productivity increased through AI is basically the same. Over the time it would elevate the living standards of millions of people.

As for making more money as a software engineer, it’s individual responsibility. You need to change your mindset of being just an employee and think as an entrepreneur. AI has made is so easy to build an MVP in record time and try things out. If you want to make money a nine to five job is not what you should be focusing on.

This is the underlying split on HN. The temporarily embarrassed millionaires, the prophets of capitalism, hyper individualist "entrepreneurs" vs. people who are disillusioned from real world experiences over decades that stripped away all the LinkedIn lunatic fringe of "change your mindset of being just an employee and think as an entrepreneur".

I actually don’t understand what you meant in this comment. This actually looks like some LLM generated slop to be frank.

Reply on news.ycombinator.com