Ask HN: Key bottlenecks to making LLMs smarter?

1 pointtikkun2 comments

What are the main things bottlenecking us from significant improvements in text generation and text understanding capabilities in language models?

I'm trying to figure out how much better GPT-4.5 will be compared to GPT-3.5.

View on HN

The biggest part in my opinion is data. Both quality and quantity.

Then, the deep learning model architecture and the hardware (GPUs).

> I'm trying to figure out how much better GPT-4.5 will be compared to GPT-3.5

It is not possible to give a ratio with certainty. Since it will be trained on more data and probably on better hardware, all we can say is that it will be better.

How close are we to exhausting available training data?

Reply on news.ycombinator.com