Hacker News Reader

Comment by maxdoop | Hacker News Reader

I have fun on these HN chats responding to comments like yours .

It’s just fancy auto complete to you? You honestly can’t see the capability it has and extend it the future?

What’s that saying about “it’s hard to get someone to understand something when their salary depends on their not understanding it”.

View on HN

I feel very frustrated with these takes because instead of grappling with what we're going to do about it (like having a conversation) it's a flat, dismissive denial, and it isn't even grounded in the science, which says that "memory augmented large language models are computationally universal". So at the very least we're dealing with algorithms that can do anything a hand written program can do, except that they've been trained to do it using natural language in extremely flexible ways. I'm having a hard time seeing how "fancy autocomplete" is the right description for this.

I agree 100%.

I don't understand why we can't look at the potential, or even current, capabilities of these LLMs and have a real conversation about how it might impact things.

Yet so many folks here just confidently dismiss it.

"It doesn't even think!" -- OK, define thinking?

"It doesn't create novel ideas!" OK -- what do most devs do every day?

"It is wrong sometimes!" OK -- is it wrong more or less often than an average dev?

In case we forget the amazing predictive capabilities of HN there’s always https://news.ycombinator.com/item?id=9224

«Computationally universal»? Are you quite sure? Hoo boy pass the nitroglycerin hooooo aaah ohh after 80 years Turing was proven right after all ghasp

https://en.wikipedia.org/wiki/Universal_approximation_theore...

Well I don't understand the nitroglycerin reference and of course "computationally universal" doesn't mean "sentient" but the point is when you add the external memory they (Flan-U-PaLM 540B to be specific) have been demonstrated to be capable of simulating a specific, well defined Turing machine without special training. There are some other papers out there arguing this from a theoretical angle too, but this is the one whose title I quoted:

https://arxiv.org/abs/2301.04589

5 thermostats in the right comfiguration can be a universal computer. Approximation, in the computational sense, is relative to some well defined function, which gives the approximation an ontology. The feasability of approximation is not disputed, but the content and nature of the ontology is a battlefield of motivated rhetoric.

Yes, this is arguing semantics but in this particular case, semantics is the whole point.

5 thermostats sounds like a very complex system, by the way!

>> I feel very frustrated with these takes because instead of grappling with what we're going to do about it (like having a conversation) it's a flat, dismissive denial, and it isn't even grounded in the science, which says that "memory augmented large language models are computationally universal"

That's not what "the science says", it's the title of an article that someone put on arxiv.

The article has no theoretical results, just a shoddy empirical demonstration of... something. The author claims that the something is an LLM simulating a Turing machine. But, is it? Really?

Well, here's how the article concludes:

>> Hopefully the reader has been convinced by this point.

"Hopefully" is not how you show computational universality of a neural net architecture. This is how:

https://www.sciencedirect.com/science/article/pii/S002200008...

i.e. with maths. That article you link to is a bunch of hooey.

That's a good paper! Thanks for sharing.

Reply on news.ycombinator.com