Review: Project Xanadu – The Internet That Might Have Been | Hacker News Reader

Review: Project Xanadu – The Internet That Might Have Been

astralcodexten.com

60 pointspaulpauper32 comments

I knew that crowd during the era of Autodesk ownership.

It was a disaster. There were fundamental problems:

- Nelson had very specific ideas for a database architecture, and they were awful. Links going every which way, all of which had to be consistent at all times. Imagine URLs all having backlinks to all uses of the link. That's Xanadu. You could make this work today, but database architecture of the era was far too primitive. With all that linking, just rendering a page would take a lot of joins.

- Everything is pay per view in Xanadu. What are you going to store? Early thinking was to store the expensive newsletters which investors buy for a few hundred dollars a month. The people involved were way too into microtransactions.

- It was inherently a centralized system, like Lexis/Nexis, or the National Library of Medicine. This would not have scaled, because of the need to keep the backlinks consistent.

- No images. Images didn't fit Nelson's model of text editing. How do you charge for operations on images?

Probably the closest living descendants to Xanadu are the collaborative editing systems where multiple people can work on the same thing at the same time. The text systems for that are well known. There's even a graphical shared editing system, NVidia Omniverse Connector. Nobody uses it. [1]

[1] https://blenderartists.org/t/so-is-anyone-actually-using-omn...

Ted Nelson had really good ideas, but he stood in his own way.

He wanted to build a closed-source system that he owned and controlled. He was a bad project leader and got nowhere. His patents prevented others from using zigzag structures. Decades later, when some people wanted to build an open-source GZigZag, he first said it was okay to use the name, then turned sour on it and prevented them from using it.

If someone wants to do it again, they shouldn't involve Ted Nelson in any way our use anything he has control over, whether a trademark or a patent.

The Curse of Xanadu https://www.wired.com/1995/06/xanadu/

Slightly off topic but I'd like to point the distinction between the contents of that article with the latest stuff on the page side. It's not bad just the same thing you see in every news site.

Wired surely went full corpo didn't they?

Not this again, an overdesigned overcomplicated impractical mess. Something like Wikipedia would never be possible on a xanadu platform because it would be too complicated for anyone to contribute.

Xanadu is just not a realistic model for how collaborative text works in practice, the admin of tracing everything back to everything else would extinguish any useability

Between cookies and sites that track cursor activity, the Web does have backlinks. They're used for monetization, not for collaboration.

Git as used for large open-source projects is probably the closest thing to the Xanadu dream. There is collaboration, and tracking of who did what. Fortunately there is no monetized fractional ownership.

In retrospect, Xanadu's big problem was the lack of a business model.

Gwern has some good comments on Project Xanadu and why it failed: https://gwern.net/xanadu

Mostly, he says, its biggest problem is that what it was trying to do (beyond the sort of thing the web actually ended up doing) just isn't that useful.

(I work at Notion)

Notion has transclusions. In the product we call them “synced blocks” (https://www.notion.com/help/synced-blocks), but the implementation type name is transclusion_container and transclusion_reference.

We also have backlinks. These used to be to prominently feature featured but these days they’re still supported but collapsed by default when viewing a page.

It would be cool to build Xanadu style multi-pane view but users tend to be a bit confused by horizontal scrolling content containers.

I don't use Notion heavily, but I've worked with any number of local editors that support transclusions and backlines. I think this is a wonderful system for individual and organization use. Basically, when all the docs are owned by the same entity, and there's no microtransaction BS, or the misguided idea of perfect traceability of every word on the screen (as opposed to just, you know, copy and paste where appropriate), it's blissful.

But extending that to the whole world's publishing needs? Hah, yeah, no way. I think that these systems, and especially the web, are more vast than Nelson's ideas could possibly have scaled to.

The CrossLine outliner (https://github.com/rochus-keller/crossline/) has transclusion and backlinks as well. Very useful concept indeed.

For a long time I played around with ideas for hypertext that would be fully decentralized, but implement much of what Xanadu was supposed to do (without micropayments, but with personalized filtering). Several years ago I wrote the core of a text editor that would maintain bidirectional links at the correct positions in editable documents. Partly I was thinking about decentralized social media, but mostly I was interested in collective intelligence, i.e. new and better ways for lots of people to coordinate to do intelligent things.

And then LLMs happened. It seems to me that LLMs, with RAG today or giant contexts later, do a lot of what I wanted. Instead of starting with documents from various collaborators, adding links between them, and stitching together new documents with transclusions, you can just chat with an LLM to get summaries, references, and so on. A lot of manual wrangling turns into a conversational collaboration with the machine (and through that, your human collaborators).

Ive seen this abomination batted around every so often as 'the internet that could have been'. Glad it wasn't.

Some of my own critiques.

1. Nelson wanted to institute optional micropayments, like 1 cent or fractions of cents to pay to access content. This would mean that nearly everything would have a paywall.

2. With automated paywalls (to charge and to pay), would lead to scammed content like a infinite scrolling page at 1 cent a page, to not get immediately blocked.

3. The idea was that you could also charge for your content. What would happen is your stuff would get scraped, added to aggregators, and charged more while you get nothing.

4. You pay for seemingly legit content and pay for scams. No way to charge back.

5. With all this micropayments and stuff, would necessitate DRM on all 'pay' content. It would be the only way to stop downloading/archiving/reuploading with micropayments that go to me. I view DRM on everything as a computing hellscape.

6. Nelson's extreme secrecy was what caused his system to never get any traction. Those Mosaic and A-Pachy folks were like 'set up a fresbsd box and make a free website.' None of this goofy money crap.

> This would mean that nearly everything would have a paywall.

Now everything has ads and is SEOed to hell. And everyone used Ad Blockers, so the authors still get nothing.

Ive seen freemium services now, that used to be 'free but ad infested', and 'pay but no ads' - go to 'pay but you still get ads'.

I have no reason to think otherwise if Xanadu did the micropayments scam AND ads. And I would expect some online script would necessitate downloading and paying for ads to decrypt the content to enforce paying to get advertised at.

The scraping and reuploading issue could be solved by some kind of universal global content identification system, integrated into the micropayments system, making sure no matter where certain piece of content is uploaded, the fee would still go to the copyright owner, perhaps with some small percent given to the hosting website. Not saying it would certainly work, but there is a few technologies probably everyone here have heard about that seems like a very good fit for the task.

> 1. Nelson wanted to institute optional micropayments, like 1 cent or fractions of cents to pay to access content. This would mean that nearly everything would have a paywall.

Of course there are obvious problems in his ideas but I think micropayments could have been a better monetization option than the ad and data collection model.

Years ago I held out hope Apple could make something work. I'd be happy to pay for content on the web, if I didn't have to have 50 or 500 different subscriptions.

There's no reason to believe companies wouldn't implement both.

> 1. Nelson wanted to institute optional micropayments, like 1 cent or fractions of cents to pay to access content. This would mean that nearly everything would have a paywall.

Mission accomplished? In real life everything has a paywall. I much prefer a supermarket to the modern internet. I know what I'm getting, I know what I'm paying for it, I can pay cash, and when I walk out it's over. I can see the cost upfront, I can compare different suppliers, I can buy in bulk: I know what I'm getting and what I'm paying.

I hate the "$0" internet.

I think you're not seeing the end result of "everything has micro transactions", and how you can't see the content UNTIL you pay.

In a grocery, you can see the goods or the box. Nutrition labels have various details. You see the price. You can pay or not.

With internet of micro transactions, everything would be gamified to eek out as much money for as little content. You'd have content gatekept behind a paywall, with ads you had to download and watch for a password to unlock. Infinite scroller websites are also infinite money generators. Things in the background would attempt to steal from you by silently paying/downloading content underneath they pay threshold.

Every thing would have their hand out demanding money.

And with this, in order to enforce, you'd have onerous DRM baked in everywhere. Lest scrapers aggregate and create multiple pricepoint sites to target micro transaction levels.

And, in this world, only the rich can access everything.

This view of an alternate internet is a hellscape. At least we can block adverts and disable JavaScript.

I think Ted had a more optimistic vision than that. More like, go to the library and read whatever you want, but with a meter running so you got charged a microscopic amount per word read. It might come to a couple of bucks equivalent if you spent the whole day reading. At one point (I don't know if he departed from this), you couldn't set your own prices. All paid reading was charged at the same amount per byte. As an author you could freely quote anyone else, like transclude a page of their text into yours. The system tracked the transclusions so the other person would get paid for the part that you quoted, and you'd get paid for your own parts. They basically handwaved the question of unauthorized copying (as opposed to their tracked transclusions), at least for a while, by saying that it wasn't allowed but not explaining how enforcement would work.

I knew the Xanadu tech folks pretty well and hung out with them a fair amount. They were capital-L libertarians with the usual belief that they could squash the real world into their ideological framework. I only met Ted himself a couple of times. I think he was less naive, but I don't know how that fit in.

I remember RMS meeting them and getting a big talk about all the stuff they'd implemented over N years. Afterwards he said he could write the same thing in a few weeks. He wasn't interested in the paid-everything vision though.

Massive Nelson fan here. My question right now is knowing that Ted's moved from, "the alt Internet" to a, "really bitchin' writing tool" what's the current state of development? What's Christina Engelbart up to?

If the right people just starting designing software again we could accomplish so much. SV is essentially dead in terms of meaningful innovation. All it would take is developing a better operating system, a better web browser, a better personal computer. The competition has lowered the bar so low. Certainly something could be done. #as #we #may #continue #to #think

I first heard about project Xanadu on Serial Experiments Lain.

This all seems to be the precursor to the recently viral Matthew McConaughey video saying he wants a private LLM, and to think, first of all, we still don't have a great/mainstream tool/product for this, and second of all, it's more than 80 years in the making, is mind boggling...

I read this article earlier today and ended up going down a rabbit hole trying to figure out what Xanadu actually "is" (on a more technical level) but haven't really been able to find much. Does anyone have a good resource for understanding the details?

Think of it as distributed source control on steroids. There's a document, built up by changes. Each change is owned by somebody. To read the document, you have to pay everybody who contributed a change. (That part was always kind of hazy, but there were lots of microtransactions involved.)

You can link between documents, too, like the web, and all links are multi-ended, so you can see who referred to what.

All of this is kept consistent, like an SQL database with atomic transactions.

The original documents are confusing because the concepts needed to build Xanadu didn't really exist then. Nelson also introduced new terminology never used elsewhere, and mostly not implemented. Now we have consistent databases, CRDTs, links, and the Web as a model, plus lots of cheap bandwidth and storage. So we know how to talk about and build this stuff.

And a whole new era of questions. Who owns the weights when an LLM is trained on Xanadu?

You could build it, but no one would come.

A lot of details haven't been explained clearly, as far as I know, but the basic idea is in Nelson's books.

There's no embedded markup. Links and formatting is off to the side in other documents, referencing locations in the original document.

You also have "transclusions," where you say "in document A position A.n, embed part of document B from position B.n to position B.p."

By using transclusions, you can make new versions of documents, without messing up the links and formatting codes even though they aren't embedded and reference by character position. Because you're not editing any original documents, anyone can effectively make changes. You can add a link to a page without even making a new version, you just put a new link in a side document.

In 1984 one of the developers of Xanadu at the time, Chip Morningstar, wrote a technical description to apply for funding.

In typical Xanadu fashion it was kept secret, protected by Dark Magick:

> He who transgresses against the propriety of the Information contained herein shall be Cursed! Woe unto all who reveal the Secrets contained herein for they shall be Hunted unto the Ends of the Universe. They shall be afflicted unto the Tenth Generation with Lawyers. Their Corporate Bodies shall be Broken and cast into the Pit. Their Corporate Veil shall be Pierced, and Liability shall attach to the Malefactors in personem. They shall suffer Ulcers and Migraines and Agonies Unimagined. Yea, Verily, for such shall come to pass against all who would Dare to Test the Powers of Xanadu unto their Doom.

It was believed to have been lost forever but in 2019 he found the only known remaining copy, a printout he had at home buried in old stuff, and posted it with OCR in his blog. He wrote:

> At the time, we regarded all the internal details of how Xanadu worked as deep and dark trade secrets, mostly because in that pre- open source era we were stupid about intellectual property. As a consequence of this foolish secretive stance, it was never widely circulated and subsequently disappeared into the archives, apparently lost for all time. Until today!

I took it and made a cleaned-up augmented HTML edition, with additions like Engelbart’s structural statement numbers which allow linking to individual paragraphs:

https://sentido-labs.com/en/library/201904240732/Xanadu%20Hy...

It includes a glossary of terms and detailed explanation of the content addressing “tumblers”, and is the best technical description of Xanadu that I know of.

https://news.ycombinator.com/item?id=16246665

DonHopkins on Jan 27, 2018 | parent | next [–]

One major problem with Ted's design is that it wasn't simple enough to support multiple compatible implementations.

I'll repeat the James Clark quote from the wonderful DDJ interview that I posted to the other discussion about Ted Nelson:

There's a wonderful DDJ interview with James Clark called "A Triumph of Simplicity: James Clark on Markup Languages and XML" where he explains how a standard has failed if everyone just uses the reference implementation, because the point of a standard is to be crisp and simple enough that many different implementations can interoperate perfectly.

A Triumph of Simplicity: James Clark on Markup Languages and XML:

https://web.archive.org/web/20210323210832/http://www.drdobb...

"The standard has to be sufficiently simple that it makes sense to have multiple implementations." -James Clark

More details and excerpts:

https://news.ycombinator.com/item?id=26217622

>Here's another comment I wrote in the HN discussion from a couple years ago about "Ted Nelson on What Modern Programmers Can Learn from the Past [video] (ieee.org)", in which James Clark talked about his role in the transition from SGML to XML, and the value of standards being sufficiently simple to have multiple interoperable implementations: [...]

there's good "Advent of computing" episode about memex and hyperlinks https://adventofcomputing.libsyn.com/episode-26-memex-and-hy...

> unlike Ted Nelson, Tim Berners-Lee never read about Bush’s memex. He built a system that connected people like never before—but made little effort to facilitate the connection of ideas. There are no trails on the World Wide Web—instead, there are misattributed quotes, dead one-way links, constant plagiarism scandals, and widespread misinformation and mutual distrust. It’s often said that we’re living in a ‘post-truth society’. The words we write and videos we share have become entirely unmoored from the ideas underlying them. Strangely, the Web has facilitated more disconnection than was ever possible before.

> Ted Nelson, in his own oblique and dodgy way, predicted the failure mode we’re now seeing: “This is not a technical issue, but rather moral, aesthetic and conceptual.” We built our global information-sharing system quickly, efficiently, and technically, when we should’ve treated it as a philosophical and aesthetic puzzle as much as a computational one, and built carefully and precisely.

Agreed: nothing is more important than the foundation that you built, especially if it may make or break society.

this article has thrown me into a rabbit holeee.. what a fascinating history. thank you astralcodexten

https://news.ycombinator.com/item?id=16224154

DonHopkins on Jan 24, 2018 | parent | context | favorite | on: Ted Nelson on What Modern Programmers Can Learn fr...

I think his biggest problem is that he refuses to collaborate with other people, or build on top of current technology.

He's had a lot of great important inspirational ideas, but his implementation of those ideas didn't go anywhere, he's angry and bitter, and he hasn't bothered re-implementing them with any of the "inferior technologies" that he rejects.

Back in 1999, project Xanadu released their source code as open source. It was a classic example of "open sourcing" something that was never going to ship otherwise, and that nobody could actually use or improve, just to get some attention ("open source" was a huge fad at the time).

http://www.theregister.co.uk/1999/08/27/web_precursor_xanadu...

>Register believe it or not factoid: Nelson's book Computer Lib was at one point published by Microsoft Press. Oh yes. ®

They originally wrote Xanadu in Smalltalk, then implemented a Smalltalk to C++ compiler, and finally they released the machine generated output of that compiler, which was unreadable and practically useless. It completely missed the point and purpose of "open source software".

I looked at the code when it was released in 1999 and wrote up some initial reactions that Dave Winer asked me to post to his UserLand Frontier discussion group:

http://static.userland.com/userlanddiscussarchive/msg010163....

http://static.userland.com/userlanddiscussarchive/msg010164....

http://static.userland.com/userlanddiscussarchive/msg010165....

http://static.userland.com/userlanddiscussarchive/msg010166....

http://static.userland.com/userlanddiscussarchive/msg010167....

A few excerpts (remember I wrote this in 1999 so some of the examples are dated):

>Sheez. You don't actually believe anybody will be able to do anything useful with all that source code, do you? Take a look at the code. It's mostly uncommented glue gluing glue to glue. Nothing reusable there.

>Have you gotten it running? The documentation included was not very helpful. Is there a web page that tells me how to run Xanadu? Did you have to install Python, and run it in a tty window?

>What would be much more useful, would be some well written design documents and port-mortems, comparisons with current technologies like DHTML, XML, XLink, XPath, HyTime, XSL, etc, and proposals for extending current technologies and using them to capture the good ideas of Xanadu.

>Has Xanadu been used to document its own source code? How does it compare to, say, the browseable cross-referenced mozilla source code? Or Knuth's classic Literate Programming work with TeX?

>Last time I saw Ted Nelson talk (a few years ago at Ted Selker's NPUC workshop at IBM Almaden), he was quite bitter, but he didn't have anything positive to contribute. He talked about how he invented everything before anyone else, but everyone thought he was crazy, and how the world wide web totally sucks, but it's not his fault, if only they would have listened to him. And he verbally attacked a nice guy from Netscape (Martin Haeberli -- Paul's brother) for lame reasons, when there were plenty of other perfectly valid things to rag the poor guy about.

>Don't get me wrong -- I've got my own old worn-out copy of the double sided Dream Machines / Computer Lib, as well as Literary Machines, which I enjoyed and found very inspiring. I first met the Xanadu guys some time ago in the 80's, when they were showing off Xanadu at the MIT AI lab.

>I was a "random turist" high school kid visiting the AI lab on a pilgrimage. That was when I first met Hugh Daniel: this energetic excited big hairy hippie guy in a Xanadu baseball cap with wings, who I worked with later, hacking NeWS. Hugh and I worked together for two different companies porting NeWS to the Mac.

>I "got" the hypertext demo they were showing (presumably the same code they've finally released -- that they were running on an Ann Arbor Ambassador, of course). I thought Xanadu was neat and important, but an obvious idea that had been around in many forms, that a lot of people were working on. It reminded me of the "info" documentation browser in emacs (but it wasn't programmable).

>The fact that Xanadu didn't have a built-in extension language was a disappointment, since extensibility was an essential ingredient to the success of Emacs, HyperCard, Director, and the World Wide Web.

>I would be much more interested in reading about why Xanadu failed, and how it was found to be inadequate, than how great it would have been if only it had taken over the world.

>Anyway, my take on all this hyper-crap is that it's useless without a good scripting language. I think that's why Emacs was so successful, why HyperCard was so important, what made NeWS so interesting, why HyperLook was so powerful, why Director has been so successful, how it's possible for you to read this discussion board served by Frontier, and what made the World Wide Web what it is today: they all had extension languages built into them.

>So what's Xanadu's scripting language story? Later on, in the second version, they obviously recognized the need for an interactive programming language like Smalltalk, for development.

>But a real-world system like the World Wide Web is CONSTANTLY in development (witness all the stupid "under construction" icons), so the Xanadu back and front end developers aren't the only people who need the flexibility that only an extension language can provide. As JavaScript and the World Wide Web have proven, authors (the many people writing web pages) need extension languages at least as much as developers (the few people writing browsers and servers).

>Ideally, an extension language should be designed into the system from day one. JavaScript kind of fits the bill, but was really just nailed onto the side of HTML as an afterthought, and is pretty kludgey compared to how it could have been.

>That's Xanadu's problem too -- it tries to explain the entire universe from creation to collapse in terms of one grand unified theory, when all we need now are some practical techniques for rubbing sticks together to make fire, building shelters over our heads to keep the rain out, and convincing people to be nice and stop killing each other. The grandiose theories of Xanadu were certainly ahead of their time.

>It's the same old story of gross practicality winning out over pure idealism.

>Anyway, my point, as it relates to Xanadu, and is illustrated by COM (which has its own, more down-to-earth set of ideals), is that it's the interfaces, and the ideas and protocols behind them, that are important. Not the implementation. Code is (and should be) throw-away.

>There's nothing wrong with publishing old code for educational purposes, to learn from its successes and mistakes, but don't waste your time trying to make it into something it's not.

[More replies and discussion at the original post:]

https://news.ycombinator.com/item?id=16224154

Reply on news.ycombinator.com