1,265 karma · joined August 3, 2020
- a predefined document store / document chunk store where every chunk gets a a vector embedding, and a lookup decides what gets pulled into context as to not have to pull whole classes of document, filling it up
- the web search like features in LLM chat interfaces, where they do keyword search, and pull relevant documents into context, but somehow only ephemerally, with the full documents not taking up context in the future of the thread (unsure about this, did I understand it right?) .
with the new models with million + tokens of context windows, some where arguing that we can just throw whole books into the context non-ephemerally, but doesnt that significantly reduce the diversity of possible sources we can include at once if we hard commit to everything staying in context forever? I guess it might help with consistency? But is the mechanism with which we decide what to keep in context not still some kind of RAG, just with larger chunks of whole documents instead of only parts?
I'd be extatic if someone who really knows their stuff could clear this up for me
Although I've also been thinking about the overall role of effort in products, art, or any output really. Necessary effort to produce something is / was at least some indicator of quality that means that the author spent a certain amount of time with the material, and probably didn't want to release something bad if it meant they had to put a certain threshold of effort in anyways. With that gone, of course some people are gonna get their productivity enhanced and use this tool to make even better things, more often. But having to expend even more engery as a consumer to find out whether something is worth it is incredibly hard.
Unless they have a whole seperate model run that does only this at the end every time, so they don't want the main response to do it?
Then again, human 1:1 tutoring is the most effective way to learn, isn't it? In the end it'll probably end up being a balance of reading through texts yourself and still researching broadly so you get an idea about the context around whatever it is you're trying to do, and having a tutor available to walk you through if you don't get it?
What these really need IMO is an integration where they generate just a few anki flashcards per session, or even multiple choice quizzes that you can then review with spaced repetition. I've been doing this manually, but having it integrated would remove another hurdle.
On the other hand, I'm unsure whether we're training ourselves to be lazy with even this, in the sense of "brain atrophy" that's been talked about regarding LLMs. Where I used to need to pull information from several sources and synthesize my own answer by transferring several related topics onto mine, now I get everything pre-chewed, even if in the form of a tutor.
Does anyone know how this is handled with human tutors? Is it just that the time is limited with the human so you by necessity still do some of the "crawl-it-yourself" style?
I've seen a twitter thread where a woman described conditioning her son to find screens boring at an early age by only showing slow train riding footage on screens around the house at all times when the kid was around, which is interesting. I wonder if that'd work well in general or just for that one specific child. It'd probably fall apart as soon as the child gets introduced to the fact that cool dopamine hacking content does exist on screens?