Hacker News Reader

Agreed, our whole computing paradigm needs to shift at a fundamental level in order to let AI be 'magic', not just token prediction. Chatbots will provide some linear improvements, but ultimately I very much agree with you and the article that we're trapped in an old mode of thinking.

You might be interested in this series: https://www.youtube.com/@liber-indigo

In the same way that Microsoft and the 'IBM clones' brought us the current computing paradigm built on the desktop metaphor, I believe there will have to be a new OS built on a new metaphor. It's just a question of when those perfect conditions arise for lightning to strike on the founders who can make it happen. And just like Xerox and IBM, the actual core ideas might come from the tech giants (FAANG et al.) but they may not end up being the ones to successfully transition to the new modality.

Feature Request: Can we have dark mode for videos? An AI OS should be able to understand and satisfy such a usecases.

E.g. Scott Aaronson | How Much Math Is Knowable?

https://youtu.be/VplMHWSZf5c

The video slides could be converted into a dark mode for night viewing.

> 3. Learned behavior. It's ironic how even something like ChatGPT (it has hundreds of chats with me) barely knows anything about me & I constantly need to remind it of things.

I've wondered about this. Perhaps the concern is saved data will eventually overwhelm the context window? And so you must judicious in the "background knowledge" about yourself that gets remembered, and this problem is harder than it seems?

Btw, you can ask ChatGPT to "remember this". Ime the feature feels like it doesn't always work, but don't quote me on that.

Yes, but this should be trivially done with an internal `MEMORY` tool the LLM calls. I know that the context can't grow infinitely, but this shouldn't prevent filling the context with relevant info when discussing topic A (even a lazy RAG approach should work).

You are asking for a feature like this. Future advances will help in this.

https://youtu.be/ZUZT4x-detM

What you're describing is just RAG, and it doesn't work that well. (You need a search engine for RAG, and the ideal search engine is an LLM with infinite context. But the only way to scale LLM context is by using RAG. We have infinite recursion here.)

On the tool-invocation point: Something that seems true to me is that LLMs are actually too smart to be good tool-invokers. It may be possible to convince them to invoke a purpose-specific tool rather than trying to do it themselves, but it feels harder than it should be, and weird to be limiting capability.

My thought is: Could the tool-routing layer be a much simpler "old school" NLP model? Then it would never try to do math and end up doing it poorly, because it just doesn't know how to do that. But you could give it a calculator tool and teach it how to pass queries along to that tool. And you could also give it a "send this to a people LLM tool" for anything that doesn't have another more targeted tool registered.

Is anyone doing it this way?

> Is anyone doing it this way?

I'm working on a way of invoking tools mid-tokenizer-stream, which is kind of cool. So for example, the LLM says something like (simplified example) "(lots of thinking)... 1+2=" and then there's a parser (maybe regex, maybe LR, maybe LL(1), etc.) that sees that this is a "math-y thing" and automagically goes to the CALC tool which calculates "3", sticks it in the stream, so the current head is "(lots of thinking)... 1+2=3 " and then the LLM can continue with its thought process.

Cold winds are blowing when people look at LLMs and think "maybe an expert system on top of that?".

I don't think it's "on top"? I think it's an expert system where (at least) one of the experts is an LLM, but it doesn't have to be LLMs from bottom to top.

On the side, under, wherever. The point is, this is just re-inventing past failed attempts at AI.

Except past attempts didn't have the ability to pass on to modern foundation models.

Look, I dunno if this idea makes sense, it's why I posed it as a question rather than a conviction. But I broadly have a sense that when a new technology hits, people are like "let's use it for everything!", and then as it matures, people find more success in interesting it with current approaches, or even trying older ideas but within the context of the new technology.

And it just strikes me that this "routing to tools" thing looks a lot like the part of expert systems that did work pretty well. But now we have the capability to make those tools themselves significantly smarter.

Expert systems are not the problem per se.

The problem is that AI is very often a way of hyping software. "This is a smart product. It is intelligent". It implies lightning in a bottle, a silver bullet. A new things that solves all your problems. But that is never true.

To create useful new stuff, to innovate, in a word, we need domain expertise and a lot of work. The world is full of complex systems and there are no short cuts. Well, there are, but there is always a trade off. You can pass it on (externalities) or you can hide (dishonesty) or you can use a sleight of hand and pretend the upside is so good, it's magical so just don't think about what it costs, ok? But it always costs something.

The promise of "expert systems" back then was creating "AI". It didn't happen. And there was an "AI winter" because people wised up to that shtick.

But then "big data" and "machine learning" collided in a big way. Transformers, "attention is all you need" and then ChatGPT. People got this warm fuzzy feeling inside. These chatbots got impressive, and improved fast! It was quite amazing. It got A LOT of attention and has been driving a lot of investment. It's everywhere now, but it's becoming clear it is falling very short of "AI" once again. The promised land turned out once again to just be someone else's land.

So when people look at this attempt at AI and its limitations, and start wondering "hey what if we did X" and X sounds just like what people were trying when we last thought AI might just be around the corner... Well let's just say I am having a deja vu.

You're just making a totally different point here than is relevant to this thread.

It's fine to have a hobby horse! I certainly have lots of them!

But I'm sorry, it's just not relevant to this thread.

Edit to add: To be clear, it may very well be a good point! It's just not what I was talking about here.

> Something that seems true to me is that LLMs are actually too smart

> I think it's an expert system

I respectfully disagree with the claim that my point is petty and irrelevant in this context.

I didn't say it's petty! I said it's not relevant.

My question at the beginning of the thread was: Assuming people are using a particular pattern, where LLMs are used to parse prompts and route them to purpose-specific tools (which is what the thread I was replying in is about), is it actually a good use of LLMs to implement that routing layer, or mightn't we use a simpler implementation for the routing layer?

Your point seems more akin to questioning whether the entire concept of farming out to tools makes sense. Which is interesting, but just a different discussion.

> It's fine to have a hobby horse!

> I didn't say it's petty!

You did.

And I already showed you made a claim that LLM was AI and that you agree that you were thinking of something akin to expert systems. When I explained why I think this is a signal that we are headed to another AI winter you started deflecting.

I am done with this conversation.

The term "hobby horse" does not imply pettiness.

I don't think I've deflected at all. You're just talking about something orthogonal to the thing I was asking about in this thread, which (predictably) totally derailed the thread into an entirely different discussion.

It's fine! Happens all the time.

Definitely an interesting thought to do this at the tokenizer level!

> 1. A new UX/UI paradigm. Writing prompts is dumb, re-writing prompts is even dumber. Chat interfaces suck.

> 2. "Magic" in the same way that Google felt like magic 25 years ago: a widget/app/thing that knows what you want to do before even you know what you want to do.

and not to "dunk" on you or anything of the sort but that's literally what Descartes seems to be? Another wrapper where I am writing prompts telling the AI what to do.

> and not to "dunk" on you or anything of the sort but that's literally what Descartes seems to be? Another wrapper where I am writing prompts telling the AI what to do.

Not at all, you're totally correct; I'm re-imagining it this year from scratch, it was just a little experiment I was working on (trying to combine OS + AI). Though, to be clear, it's built in rust & it fully runs models locally, so it's not really a ChatGPT wrapper in the "I'm just calling an API" sense.