Hacker News Reader

Comment by singularity2001 | Hacker News Reader

singularity2001·

Maybe the recently announced 1T Parameter Scientific AI will be trained on mathematical papers as well and picks up reasoning on the fly: https://news.ycombinator.com/item?id=38391923

Or at least maybe it will be a good foundation model to be fine-tuned for reasoning?

View on HN

Picks up reasoning on the fly seems like wishful thinking. On the other hand in nature there also is split into people who get math and those who don’t so maybe there are 2 different ways of bootstrapping brain to be functional and luck is all we need. Anyone knows more on this?

It's just a matter of scale because when you add more floating point numbers magical things happen. The magic is called emergence, as in if you have a big enough computer then it can do anything if you prompt it the right way. As a techno-optimist I believe that thinking is simply very complicated arithmetic and by adding enough numbers we can solve any problem.

Depends a lot on how this scales though. If you need 10x as many parameters to get 1% better results for instance, then even if this works it could be impractical.

You gotta be more techno-optimistic. People are just arithmetic and an existence proof that algorithms can be intelligent. People are just math so it's obviously possible to engineer general artififial intelligence with just math and computers. It's all a matter of adding up the right floating point numbers in the right (partial) order (modulo parallelism).

Very valid point. The last papers I read on the subject conclusively show that all current networks do not hit a limit in scaling though. Bill Gates said that GPT five will not be much better than GPT four so maybe things change

Reply on news.ycombinator.com