Documentation is still in your Mum's filing cabinet
gerireid.com
gerireid.com
My workplace has a number of multi-hundred page "User Guide" documents that are an absolute chore to read through for anything actually useful to the scenario-of-the-moment because they're so granularly detailed that, even if you find the right section, it might be difficult to relate it to the scenario because there's so much detail that the context gets lost.
If we could get them ingested as an LLM context, and then make a customer-accessible interface, it would be a great value-add for Customers as well as Employees.
I had a few thousand happy customers and would hear from folks whose computing competence had been lifted by the titles I worked on .. that they finally 'got it' when the difference between memory, storage, files and folders was finally clear to them.
It is incredibly frustrating to realize that these apps I wrote in the 80's are still entirely relevant today. Too many times I've been in someones business office, and observed some new generation folks not quite understanding where things are stored, how they're stored, why and when to the use ~/Documents and ~/Downloads and ~/Desktop, and so on. Some folks just put everything on ~/Desktop and wonder why they can't find anything. Some folks thoroughly enjoy being coached through creating their own organization systems, for the first time, at the filesystem.
For the past 30 years I've been printing to PDF every interesting web page I've ever read. I now have a collection of 80,000+ .PDF files, in one folder, a huge collection of my personal knowledge and interests. It is immensely rewarding to "ls -l | grep <some interesting subject>" and get a sorted list .. to see also how I have revisited certain subjects over time ..
And now I'm faced with the issue that I just really want to data-mine this archive, so .. of course .. I'm looking at using an LLM to organize it all. One of the very first things I want it to do is sort everything into folders, by subject, and soft-link files into these hierarchical folders so that I can view the tree as a form of ontology. This is the filesystem, becoming very valuable to me as a user.
Yet, everything the OS vendors seem to be doing lately appears to be to remove the users control over their filesystems. I wish there was as much effort in making the File Browser/Explorer as useful as, say, has been put into making the browser the operating system. Sometimes I think the File Explorer versus Browser dichotomy has been seriously mis-managed by the major players over the past few decades.
I hope we see new paradigms for dealing with ontologies emerge into the mainstream .. else, I suppose, I'll have to build one myself ..
I don't know why folks haven't caught on to this technique - it's so obvious. Plus, it's like having my own little Internet whenever I'm offline .. digging through random selections from the 80,000+ files I've collected is extremely rewarding.
***
A friend of mine has a similar single folder PDF organization system and lately has been trying to better organize it. As I understand it, he's keeping the one folder, but using a reference manager software to track metadata like tags. In contrast, for about 15 years now, I've stored various documents (mostly PDF files) in a folder hierarchy with Zotero for bibliographic data. The PDF filenames match the Zotero citation keys so I can easily switch between the two systems. I'm currently at about 39K PDF files in 4.5K folders with 17K symlinks.
My friend has slowly chugged along, organizing perhaps hundreds of files manually per month. (Maybe he's stopped, I don't know.) It seems to me that transitioning from a one folder approach to something with more context (whether metadata or a folder hierarchy or whatever) is really hard. The best time to add that context is when the file is obtained as I might not remember later. I personally don't trust a LLM to spot the nuances that matter to me when organizing things. A generic organization scheme is of no value to me. I don't know what the goals of your organization effort would be, but this is something to consider.
And there's multitrees, that might or might not help.
https://interactivity.ucsd.edu/articles/In_Process/MultiTree...
Finding, reading outdated, writing, updating, grooming documentation is so much more expensive than just throwing LLM on repository (or multiple ones) with a "Go Fetch" quest.
Sure, it's ephemeral, but with assumptions of $50/h earn and assumption that non-naive piece of internal documentation takes 8h of work time and will be read ~100 times that comes in at $4 at read which I think is much more expensive than straight token-API costs (and probably much much more expensive than subsidized subscription costs).
And this is generous. Looking at Jira's stats from my past work, many long documents in small (but specialized) team were read 10-20 times boosting the (assumption-average-costs) to $20-$50 per read.
But wait, there's more (;-))! That's all assuming that knowledge absorption is 100%, what if only 50% of document is relevant. What if it's 25% etc.
In the end today there might as well be no documentation for code and LLMs could extract it raw from code. Raw - because I don't think indexers/RAGs/compressers are useful. I found that spending time on building such is fruitless: Indices can (and will) go stale just as materialized documents and lower context saturation results in more hallucinations in the end.
(There are out-of-code documents which obviously have no other source than then themselves so there's no other around it, though)