Doug Engelbart’s design for knowledge-based organizations (1992) [pdf]
dougengelbart.org
dougengelbart.org
In reading the pdf linked by Jtsummers, I recall how some of the sales people I supported had 20+ gigabytes of email in their inboxes... which drove me nuts (Exchange Server really didn't cope well with it back then), and now I know why, they were building a hyperlinked database (well, threaded) that they could search through.
Here in HN, we do the same thing over time, building a tree structured database that preserves context and is searchable. The main limitation here is there's only one possible tree, and it's the privileged view.
There's at least a billion dollars of knowledge captured here in HN. Yet, it takes Dang and a strong community to keep that one view rational, due to the corrupting forces of human nature and profit motives from spam and scams.
There's got to be a better way.
Books and movies have long made plots off new ways to help humans learn things and form connections by either 1) changing the presentation format or 2) downloading all the information into the human brain to let it do the sorting. For example, overlying data on a map in 3d using glasses or wiring a human brain into a computer's serial ports.
The main problem moving forward is making the data available. I have a fear that in the future only governments and large companies in bed with governments will have access to this data and use it in unethical ways.
Meanwhile, startups and individuals will be bared from creating 'unlicensed' presentations or accessing the underlying data itself (i.e. running a search engine)
I'd look back to the Memex for the solution to that problem, it stored everything locally, thus links couldn't be broken. Output included all the linked items. It's possible to build a system that stores everything you've seen on the web locally, to prevent things from disappearing.
I think local storage is highly undervalued, and the pendulum will shift back.
Yes. The opportunity to increase our functional representations of 2D knowledge bases is going to be a game changer that will compliment the Cambrian Explosion currently underway with the PKM/TfT use of symantec Graph connections & AI LLMs. Some efforts ongoing towards this [1] already and imho there is so much you can do to interact with data once you open up the presentation model to spatial computing inside a 3D embodied user interface like VR today, Passthrough AR tomorrow, and full AR a little down the line.
Mainly because inferring a different "view" over gigabytes or even terabytes of data (current collections of publicly available "general purpose" datasets can run in the terabytes) is really, really hard and computationally expensive.
And our current data models don't even include satisfactory ways of representing the kind of "common sense" data that would be reflected in a HN comment, an e-mail thread or really any kind of textual source. You can kinda get there by marking "named entities" of sorts but that's the tip of the iceberg, all the structure and semantics is still missing. It's something that's very much being worked on, and has been for decades (with "frame structures" and the like). But it's still hard.
https://news.ycombinator.com/item?id=23666950
Despite ”design for knowledge based organizations” being the title of the article in the PDF, this is perhaps more about approach and capabilities for organizing flow from capture to use to pruning of knowledge, more than the organizations working on knowledge.
There is one aside about the organization itself:
> Engelbart sees every organization as a collection of interacting knowledge domains. He has focused his research on designing support structures for knowledge collection and refinementwithin and across these knowledge domains.
But then that jumps down into a given domain, and goes on to propose a knowledge management approach within that (and each) domain.
This is where it gets interestingly predictive of the recent coalescence in knowledge management tools emerging after the two dark age decades of style over semantics.
The first idea is a concept of bringing knowledge in, working it, and keeping it. He calls this Concurrent Development, Integration, and Application of Knowledge (CODIAK) process.
To make this workable he proposes a time-relevance layered approach very similar to the (perhaps easier to action) “PARA” method adopted by many KM tool users today:
After this, he goes into functional implications, and describes almost to a T capabilities in the latest round of knowledge management tooling “systems” such as, say, Obsidian.md, built on standards (markdown, wikilinks, front matter) and extensibility enabling journaling, querying, views, as first class citizens.
When hypertext markup iterated into a page style description advertorial tool rather than linked semantic knowledge structure, that left an opening for the tools we’re seeing now. Today, Obsidian.md is close to the mark, except, of course, for approachability by casuals. For casuals, consider e.g. Craft.do (https://www.craft.do/).
If you’re doing a comparison before jumping in, it’s worth diffing the tools against Englebart’s take.
I’d argue he’s dead on.
For technical teams, you can hook your obsidian markdown folders to a multi-party savvy file sync system, e.g. git, there are extensions for this. Then it plays nice with, for example, team members using VS Code as well. You can match the markdown, linking, tagging, etc., using VSCode plugins, or by carefully matching options between them, you can have some people in Obsidian and some people in VSCode + Foam.
Using obsidian+git is definitely possible, but it doesn't really get to the seamless collaborative feel that I'm looking for. Looking into Notion again, it seems like they've imported more ideas from Roam et al. and might be worth another look.
> Despite ”design for knowledge based organizations” being the title of the article in the PDF, this is perhaps more about approach and capabilities for organizing flow from capture to use to pruning of knowledge, more than the organizations working on knowledge.
I would agree that without the tools, an organization wouldn't be able to collaborate effectively in the way he advocates for, but I think unlocking group productivity should very much be more the goal here than single-person productivity. From the second page: "Englebert realized that they key to dealing with increasing complexity was human collaboration."
No amount of methodology will let a single person keep up with a team using a similar methodology.
Another key line from me: "Englebert is not advocating that we perform artificial acts with documents by superimposing structure on them. Instead, he advocates that we capture the inherent structure in all forms of human expression in order to make them easier for people to navigate through, view in different ways, and hyperlink."
As you note, there are a lot of similarities in there to Roam Research and those similar systems, especially the cross-linking/blocks type stuff vs just having a single hierarchical document structure. But don't just think of it for your own navigation and recollection, but for others to learn and get up to speed and then offer their perspectives.
But I think it's a big disservice to think about it only from a single-person's POV vs a way of capturing knowledge inside a group.
Think of "git" for software dev too, which is very along the lines of the logging/journaling of changes discussed in the article, but which differs from some previous iterations by being far more multi-user friendly (remember fighting over SVN locks?).
Foam is simply the next evolution - the shared tool is well-understood, we're now getting an augmented local version. And after that, we'll get something that's far more multi-user friendly, too. It's a pendulum that seems to swing back and forth in our industry.
The thing that I hope Foam & Friends will generate is a much better understanding of knowledge "operations", so we can understand the fundamental structures underneath better. (That's what happened with git & signed DAGs as version log. That's what happened with text & CRDTs as a model of shared edits)
And I think wikis are still quite limited in that bit I quoted: "Englebert is not advocating that we perform artificial acts with documents by superimposing structure on them. Instead, he advocates that we capture the inherent structure in all forms of human expression in order to make them easier for people to navigate through, view in different ways, and hyperlink."
They have hyperlinks, sure, but otherwise most pages in wikis have only those views that can be created through operations on a hierarchical table of contents. The structure is imposed by the tech, there, rather than the expression of knowledge itself. The blocks in Roam - https://thinkstack.club/how-to-use-roam-block-reference-menu... - seems like a major iteration on that to me in terms of moving towards units of knowledge as the fundamental element instead of "pages." (It's also worth noting that Foam doesn't do this - https://github.com/foambubble/foam/issues/294 - though apparently Obsidian does.)
And yes, it's closer to Engelbert's vision. We've been getting closer to that (and the Memex) for a long time, and we'll continue to do so. They are distant visions what a future could look like, while we barely understand knowledge work. I mean, the term itself is already an improvement of clarity on the ideas of the memex.
If you compare knowledge work to Mathematics, I'd guess we're shortly before Elements. We're at best about to understand the basic building blocks. Which I find tremendously exciting - we're still at the very beginning of exploration.
In my head they're akin methods, and so in software that idea of "call this from lots of different places, edit it in a centralized way" concept is pretty well established.
In terms of "a single document flowing with a single voice" you can't necessarily do that from assembling an arbitrary collection of blocks like that, but for "good enough for effective in-corp collaboriation"? Sure!
I think that (and the problem of bait-and-switch) still need an answer before it's usable beyond small groups. Maybe small groups is good enough for now. If it goes in any form like software development, we'll at some point split into two camps. One will say YOLO and just implement shared blocks, trusting in good will, and the other group will noodle on foundational principles to get this right. (Group 1 will make the money, group 2 will have the fun. As always ;)
That’s what E’s describing as well.
My point is, he’s NOT describing how to communicate between domains, thus, it’s not really the whole organization, it’s the domains (teams) within an organization.
That explains the fax-mediated comments section. I guess HTML FORMs[0,1] weren't going to hit until late[2] 93...
[0] https://web.mit.edu/kolya/.f/root/net.mit.edu/sipb.mit.edu/u...
[1] View Source confirms still in use by HN
[2] http://1997.webhistory.org/www.lists/www-talk.1993q4/0447.ht...
I am probably overly optimistic that some combination of HTML, linked data, GPT-3 like models for effective summarization of very long texts, better user models (their level of expertise, things they have paid attention to in the past, etc.) will get us part way to Engelbart’s vision.
Our website isn't active yet, but you can follow our LinkedIn if you're interested in updates. We have an MVP release planned for this month. https://www.linkedin.com/company/conceptionary/
This is HN, this is an interesting topic - surely, you have interesting things to say that people here would care about. Recruitment adverts, less so.
Dialogue is the single-threaded brute forcing and tree traversing activity for making decisions. Within a single thread, multiple subthreads do branch off, but then they become decentralized which is not what we want. We want each subthread to be conscious of it’s uniqueness to all other threads globally and for any duplication to be catalogued or ideally merged and then hyperlinked and indexed within the grand scheme. This is difficult when the most granular we can get in terms of categorization is the factual information or simply the topic itself. Places like HN and Reddit do well in allowing you to provide an entire article or website as the context and allow that to be explored, but how would you categorize this information? The real question is how do you make this visible and accessible from a single standpoint: an individual trying to learn something or make a decision. I think search engines like Google have solved this problem well simply by using the heuristic method of merit (backlinks, page rank, keywords) and so to implement this on a miniature scale for an organization would be useful, but the problem is that it would require users within the organization to forgo going to Google and instead use the provided Intranet which initially would be less useful and again may never be as useful as Google. So the solution might be to use Google as an indexing service which currently is not possible.
I think at the end of the day we are ignoring is the incentives. If people were incentivized to construct the information in this way, get some sort of reward, then it would happen naturally. But I don’t even think it is possible for a human to do that, say, store all of HN dialogues globally for example in their head and consider THAT before writing anything otherwise hyperlinking to what is already there.
The main issue is that knowledge is a graph, not a tree, and dialogue is linear. So in order to consume knowledge through text as quickly and efficiently as possible is more of a graph or traveling salesman problem which is very hard to solve, maybe something that Quantum AI can help with. One thing would be to incentivize hyperlinking. Another might be to store information in images using something like a way more advanced DALL-E that allows the human mind to grasp concepts simply through looking at trippy images or videos. To somehow use spatial and visual understanding to up the number of dimensions of what knowledge we can store and download into our brains.