Xanadu was Ted Nelson's proposed version of a hypertext publishing medium that predates the World Wide Web. (I believe he coined the term "hypermedia", but I could be wrong)
It is interesting alternative to "what could have been", and inspired the imagination of technologists in the 70s (I believe).
It featured some unique elements:
- subdivision of "documents" so that sections (sentences, paragraphs) can be folded in from other sources. Somewhat similar to remote CDNs for images, but did chunks of text (e.g. when quoting another article)
- UI: being able to see multiple documents at the same time and how they connect to each other. (Not the same as annotation, but if you can imagine seeing annotations on the side of the screen... something like that)
- greater similarity to the Memex in how information can be interlinked and displayed.
- in built royalty mechanism. (Micro payments)
- files are stored centrally in a docuverse, which is somewhat similar to a block chain in that there is an agreed interpretation of what a document "is" instead of having multiple copies of the same document.
Ultimately, the World aside Web came along and filled the need for a document publishing medium, rendering Xanadu a historical artifact, many of whose ideas have been realized by other projects in other forms.
I read Ted Nelson's autobiography. My impression was he had great visionary scale ideas, but couldn't describe a concrete implementation. I actively sought to understand concretely what Xanadu is and came up empty, a lot of high level words that could mean anything to different people reading them. Nothing concrete, at least that I was capable of understanding in the amount of time I was willing to invest. Nelson's research and vision predates computers and programmers as we know it today, so it makes sense that he wasn't a programmer. Tim Berners Lee was of course a programmer, and wrote in Weaving the Web, "having a working demo is table stakes."
For an overview of the "user side" read his book "Literary Machines".
If you want to understand the technology behind it, consider that there have been two incomplete implementations: Udanax Green and Udanax Gold: https://web.archive.org/web/20090226223053/http://www.udanax... (the domain udanax.com now seems now to be used by someone else).
I don't know the autobiography, but everything I have read by Nelson tells me he had very specific implementation ideas.
With Xanadu he had very clear ideas, for example, about documents that link to each other (even down to the paragraph/word/character level) and are able to preserve their integrity even in the face of edits, and his collaborators were able to come up with quite brilliant technical solutions. Nelson had a lot of working prototypes made up throughout the years.
One crucial problem was that a Xanadu-style system was a closed system; in a distributed world, the same implementation would have had to be run by every single server. It was diametrically opposite in philosophy to what became the World Wide Web, with its "anything goes" URL scheme, and it was a very clear case of "worse is better" winning over a carefully designed, walled-garden ivory-tower kind of solution.
Of course, Xanadu was completely prescient about link integrity. There was a rather sad post on HN recently about an late-1990s book which had lots of URLs to resources, none of which worked.
Did he though? It's been many decades with little if anything to show for it, and Nelson admits to this day he's not a programmer. So it sounds like his ideas are just drawings and equations on paper. That's design, not implementation.
Apparently there's some old demo version, but...
http://xanadu.com/xuspViewer.html
"Unfortunately this made it a very complicated package, eventually too tangled to improve further, and Rob had to get on to other things. "
So one volunteer (Rob) who actually made a working demo some years ago wrote spaghetti code that couldn't scale. That's as close as they can get to working software in decades? Granted the system they propose is difficult to grasp and likely math-heavy, but I don't see it as multiple-decades difficult given the progress computing has made in that time.
The more I read about Nelson and Xanadu, the more it seems like the people backing it have no desire to interact with reality. It's as if they expect their grand ideas, simply by virtue of being grand, will naturally create a viable product on their own somehow, and the "low work" of creating a viable product with a market is somehow beneath them.
I don't mean that they ever had a workable implementation. I mean that it wasn't just cloud talk. They did multiple implementations, of Xanadu itself and other components, some of which was open-sourced.
So it's a lot more than just visionary talk, and a little more than just design.
It might've been the above wired article, but some discussion of his work I vaguely remember was Nelson being bought into by I think autodesk, and Nelson basically managed the project poorly; the team would make headway in some direction over a year, then decide to do a fundamental redesign. Repeatedly. So a fully working product was never built, though some interesting components were
"...he had great visionary scale ideas, but couldn't describe a concrete implementation."
I revisit Xanadu every couple of years. For its time, its "closed" architecture seemed reasonable. How else would one implement persistent two-way links? Once you kinda grasp Xanadu, kludges like pingbacks and link liveness checkers seem ridiculous.
Alas, Ted Nelson, like myself, didn't anticipate, or was slow to fully appreciate, how DNS, HTTP, URL/URI would up-end the world. Whatever else was going on with Nelson, Xanadu, Autodesk, whatever, "the web" tsunami obliterated competing notions.
Animats' comment (below) about git (eg github) being today's closest realization of the Xanadu ideal is insightful.
"its "closed" architecture seemed reasonable. How else would one implement persistent two-way links"
That's a great example of a concrete implementation detail -
unbreakable links means a central link database - that has significant implications in the shape of the vision. For example the only way I can think of to do unbreakable links but also distributed is an immutable public ledger perhaps something like a blockchain. And now we can talk about specifics like what other properties does a blockchain implementation shape? What does a database app with HTML forms and such look like? But did Nelson understand this? Can anyone cite an implementation source that goes into Nelson's analysis of this stuff?
Nelson was not a programmer. However, Mark S. Miller told me that he came up with http://e-drexler.com/d/09/00/AgoricsPapers/agoricpapers.html out of thinking about how to architect the full distributed Xanadu system. I believe Nick Szabo was heavily influenced by Miller's ideas in his first paper on smart contracts: it references Miller and mentions that Szabo did work for the company started up to commercialize those ideas (Agorics). It's not quite a coincidence that you think of blockchains.
I don't know how Xanadu was supposed to work on the inside. I do have a dim vague memory of seeing a mailing-list post many years later in response to someone seeking the earliest prior art on using content hashes for a distributed content-addressable store -- from someone in Xanadu. That's all I got in my memory. I want to say they also mentioned Merkle trees, but I may be confabulating. (I'd be surprised if Miller didn't know Merkle personally -- they both worked with Eric Drexler, and Drexler invented his ent just a few years after Merkle patented his tree.)
Robin Hanson published a paper back then with a non-Xanadu design sketch with similar goals: http://dl.acm.org/citation.cfm?id=43938 but it's paywalled. (Hanson also got involved with Xanadu for a little while.)
This mythos of "working demos" needs to be put to rest.
At least in the form that many people consider "working demos".
Not everyone can, needs to, or wants to code. The ideas a person can express shouldn't be limited by the wood they can whittle. There are many game and ux folks who envision systems w/o having demos. They story board, animate or write Haiku.
Depends on your objectives. If you want people to actually use your stuff, they need to be able to... you know... use it...
Ideas don't take over the world, implementations of them do. Playing around with concepts is all well and good, but don't gripe when you can't demonstrate its usefulness and no one cares.
Look at the impact that Bret Victor has had by not releasing working demos. He communicates in ideas, others are free to implement them. If Bret subscribed to CoGTFO, he would be sucked into a quagmire of maintaining an instance of an idea. By not creating a functional artifact, he forces the universe to make multiple competing copies.
Some ideas need to shown, some can be told. Some people are really lousy at telling stories. Vastly more people can code than think creatively.
I disagree entirely. Execution is everything. If a given person has a vision but doesn't know how to code, they can either learn or hire someone to implement those ideas.
Yes, in interviews etc. he's struct me as an "ideas guy" who got stuck on the same idea for 30 years. You'd think in that time he could have just coded the damn thing.
Here is an interview in which Ted Nelson explains his vision to Werner Herzog: https://www.youtube.com/watch?v=Bqx6li5dbEY The interview is included in 'Lo and Behold, Reveries of the Connected World' documentary. The whole documentary is rather critical of the impact the Internet revolution had on our society, but Herzog seemed to really admire Ted Nelson's alternative vision for the global network.
He had an idea for a sort of world library/scholarly citadel called The Mundaneum - I wonder what he would've made of computers. Sadly, Wikipedia says that the project lost funding from Belgium a bit before they were invaded by Germany, who broke a lot of it.
... and in Xanadu's case, The Right Thing requires the documents to be never deleted. I do not think this is a good example of the Worse Is Better principle, it is more like "you must think of a real world when designing systems"
I agree it's not a perfect metaphor. I do think Xanadu is an extreme case of The Right Thing style of thinking, even if Xanadu would never have actually been Right: the creator still thought of it that way, and IMO that is one of the causes for the failure of the project to materialize.
Honestly, the site has somewhat of a crackpot-vibe. Only major feature I understand from the site is "visible links", i.e. links displayed as actual lines on the screen. This seems more like a gimmick than a ground-breaking fundamental difference from the Web.
Some of the features you mention could perhaps be useful if implemented in a browser extension, e.g. the ability to see/preview linked documents inline. It is useful in IDE's for code navigation, so perhaps also in a browser.
https://www.w3.org/People/Berners-Lee/FAQ.html
Q: Do you have had mixed emotions about "cashing in" on the Web?
A: Not really. It was simply that had the technology been proprietary, and in my total control, it would probably not have taken off. The decision to make the Web an open system was necessary for it to be universal. You can't propose that something be a universal space and at the same time keep control of it.
I wonder if this might be closer to the reason that Xanadu didn't take off than "worse is better". Where is the Xanadu implementation we can hack on, other than an incomplete dump of code here http://udanax.xanadu.com/gold/download/index.html built on a proprietary platform? Why is it that when I look up ZigZag I find an open source project that closed down over patenting concerns http://www.nongnu.org/gzz/ ?
Every once in a while I see references to Xanadu, as a solution that would fix all of the problems with current WWW by making broken links impossible. At some point, I spent some time reading about Xanadu's protocol, and I now think it would never work at all, since it makes some pretty wide assumptions:
Xanadu assumes that documents are stored on append-only structure: they never disappear, and instead of updating the document, one just publishes a new version (with all the text from the previous ones was still available). See for example http://www.xanadu.com.au/ted/XUsurvey/xuDation.html figures 9/10 and then figure 16. Later, there is a reference to cache, but with no details whatsoever: "... content may be cached in many places, as long as it has the same permanent IDs..."
I have no idea how is this supposed work in the real life. Today, the static web hosting is dirt cheap, but back in 90's it was $20/mo for just a few megabytes. In a Xanadu world, what would happen once you no longer pay for service?
The Internet Archive is willing to pay to preserve your content even if you aren't. One could imagine a system where your hosting fee includes an "archive tax" that pays for your content to be archived forever.
In general, I get the impression that Xanadu was designed for a world run by five mainframes and it was never really able to adapt to the Internet.
I think Xanadu is great and it will be one of those things that proves itself in the future. Perhaps whatever replaces the web someday will resemble it.
I bet that someday we will move to a distributed web hosted on something like the blockchain, with redundancy and revision history, and that will begin to resemble Xanadu. It reminds me of sort of a "Star Trek" vision of how things should be, but not how things actually are in this disorganized world.
I think that it's sort of like the Dynabook or early AI or VR, in that it's the correct vision of the ideal result of a technology, but was way too ahead of its time and the technology just wasn't ready.
I wouldn't take everything in the Wired piece as gospel. For example if you read point 3, "Transclusion Misstated" at http://web.archive.org/web/20001101230424/http://www2.educ.k... , it's clear that 1) Ted Nelson understood content-addressable networking long before eg. Bittorrent came about , while 2) Gary Wolf didn't get content-addressable networking even after researching Xanadu, meeting with Nelson himself, and writing a feature article about Xanadu for Wired. So, yes. See also http://web.archive.org/web/20001003011753/http://xanadu.com.... .
"Nelson invented a new way of crossing the street: when arriving at a busy thoroughfare, he would dramatically turn his back on traffic and step with theatrical nonchalance off the sidewalk. Drivers, frightened, would slam on their brakes."
”Eyes On The Road” might've been a reasonable assumption in the day before smartphones, but nowadays I'd advise against literally betting your life on it.
Ted Nelson got the name Xanadu from a short story by Theodore Sturgeon from 1956 called "The Skills of Xanadu". (I asked Ted about it when he visited IBM Research around 2000...)
"The story tells of a representative of the expansionist, imperialistic culture of the planet Kit Carson, and his encounter with the deceptively pastoral culture of Xanadu."
The story is about the social effect of wearable networked computers that can exchange knowledge -- very prescient for 1956!
It's a visionary hypertext project that doesn't really work yet, but the people behind it are happy to dispense snarky quotes about the how the web we all use is a poor substitute because it's a “bizarre structure created by arbitrary initiatives of varied people and it has a terrible programming language”.
The fact that they seem to be so interested in things like "who pays when you mix and match copyrighted content?" instead of "how do we get people to try out our system?" indicates that it's not going to be in widespread production use anytime soon...
I knew that crowd when Autodesk was funding them. They were fanatical libertarians. The solution to everything is a market.
Ted Nelson was focused on text to the exclusion of images. That didn't help. He also had strong ideas about how the internal database should be organized, down to the database implementation level, which didn't help either, because they were 1960s database technology.[1]
The closest thing to Xanadu in wide use today is Github. One central place for all public code. Anything can be changed but the history and old versions remain. Full tracking of who changed what. The ability to create a completely new document from an old one while retaining tracking. There's no micropayment system, of course.
The real conceptual failure was not realizing that the future was ad-supported.
They had some really good people, but did seem to be spinning their wheels.
It was completely separate from Autodesk. Xanadu had offices on California Avenue in Palo Alto; Autodesk was in Marin County. Autodesk was quite good at getting working software out the door, but that didn't reach Xanadu.
Lack of an identified market was a big problem. One market that was considered was financial newsletters. There are lots of expensive newsletters you can buy which have extremely detailed coverage of something important to traders, such as what's happening at the Henry Hub or the Port of Rotterdam or in the Bahamas. These cost $50-$1000 a month. ("Offshore Alert", which covers scams run from tiny island countries, "Platts Oil Letter", which is now a big service, and the Dines Letter, for the gold bugs.) There was no centralized distribution for these, and they had tiny circulations. That didn't go anywhere. All of those are now on the Web as subscription products.
Interesting observation - so there was some consideration of a specific information market rather than the 'Kinko's for information' idea that Nelson has described in the Wired article.
It was more the development process that I was thinking of - the actual production of software, a working system. I take your point that Autodesk's programmers could certainly produce software. I was wondering if the Xanadu project was seen by Autodesk perhaps as a bit of a 'moon-shot', in the sense of low probability of success coupled with great potential.
I was wondering if the Xanadu project was seen by Autodesk perhaps as a bit of a 'moon-shot', in the sense of low probability of success coupled with great potential.
Yes, it was.
Autodesk has done a few of those. They were into virtual reality very early, before 1990. It turned out that doing CAD with gloves and goggles was like trying to paint wearing oven mitts. Precise positioning is tough. So they gave up on that.
Getting into 3D animation worked out better. Autodesk itself did 3D Studio, and gradually acquired other 3D animation companies as they got into trouble. This worked out well, and there was crossover with the CAD business. The really good radioiosity renderer from one of the animation programs was connected to an architectural design program. Now you could place real light fixtures from a library and see the result before building the building and embedding light fixtures in concrete.
Carl Bass, as CEO until last February, was into 3D printing. He thought home 3D printing was going to be a much bigger deal than it turned out to be. So Autodesk came out with several low-end CAD programs that were never very profitable and did a lot of work to make 3D printing easier. Not profitable. Bass is gone.
Autodesk keeps plugging away at boring but useful stuff. Market cap around $25 billion.
(I'm long out of Autodesk, although still a stockholder.)
> It's a visionary hypertext project that doesn't really work yet, but the people behind it are happy to dispense snarky quotes about the how the web we all use is a poor substitute because it's a “bizarre structure created by arbitrary initiatives of varied people and it has a terrible programming language”.
This is a bit like describing Galileo as "the jackass who wouldn't shut up about heliocentrism despite having no answer to the stellar parallax objection". By no means has he ever been right about everything, and he has been his own worst enemy in several ways, but ...
It's a long dream of a better-than-the-Web hypertext ecosystem that predates the Web by 15 years but has never delivered anything actually useful[1]. A great example of why "worse is better"[2]. Even if you do have a grandiose idea that is better than anything that came before or will come soon there's a good chance you will be beaten by simpler systems that can be delivered much faster.
One of the (arguably the) original hypertext visions (AFAIK Ted Nelson, its creator, coined the term "hypertext" in the early 60s).
The main difference to web hypertext as we know it is that it doesn't just link documents together by what we know as "links", but it's bidirectional and with more content inclusion. E.g. if someone takes content from another document and puts it in their article, the text is connected to the source document (instead of them maybe putting a link to the source). And from the source document you can find all documents that used parts of it.
Project Xanadu is Nelson's repeated attempts to make this work, while the world choose the WWW model and is happy enough with that (much to the disdain of Xanadu supporters, who believe the WWW to be a vastly inferior and broken approach). Once HTML took off, there wasn't really a chance to catch up, especially not with the limited resources they have. Lot's of bitterness about that, which also means they haven't really tried to make their ideas work with the web and rather aiming for an uncompromising, pure implementation of their ideas.
Long and short of it, it's a form of bidirectional links (so if you update one side, the other side automatically updates). This prevents link-rot.
The second is putting a payment processor on EVERY query. You want Internet for the Richies- this is how you get it. Of course, there's hand-wavey garbage that says 'everyone would contribute and nobody would owe anyone anything'.
Seriously, IPFS is the best way forward here. Links never change its content (but can still die if noone has the content). And no payment processor is there unless you want to pay other people to pin (read: seed) content.
(To see this post, please pay $0.02)
Edit: Seriously? multiple -1's in a row? I'd love to hear rebuttals if I'm wrong, or a further discussion. If you've read the Xanadu documents, then you'd know that the basis of this is a "Secure Environment and Secure Identification" of computers. AKA: DRM.
Isn't that what we effectively have, with so much of the Internet's value either locked in Facebook's garden or beholden to Google?
I understand where you're coming from with the DRM criticism, but IMO the primary fault in Xanadu's version is that it's an attempt at naïve mom&pop capitalism in a digital world that already is owned by plutocrats wealthy and powerful beyond Rockefellers' dreams.
> Isn't that what we effectively have, with so much of the Internet's value either locked in Facebook's garden or beholden to Google?
No, because you can download the entirety of Wikipedia, including its edit history and discussion.
Also, you can download a not insignificant number of primary sources through both sci-hub and various other sources like Project Gutenberg, IMSLP, etc.
Add to that all the things that swell to prominence on Bittorrent trackers whenever the Streisand effect is in play.
I have a hard time seeing that content disappearing any time soon. But I also have a hard time seeing how that content would have been aggregated in the first place if you had little micropayment barriers between all the things.
When I was much younger and more naive about how the US and capitalism works, I thought that it would be a splendid idea to be able to make money on my thoughts, experiments, and doings. And I thought that it too would be a great idea to be able to leverage others' works, pay them, and build on them.
And now, I do just that. Except there's no money changing hands. That's open source for you. Now, if Xanadu's ideas did come to fruition, then I would expect everything to have a paywall on it. And the worst/regular experience would be pages of pages of aggregate sentences, with each few words being a cent or 2. The pages themselves would be >$1 each to download. Instead of trying to trick browsers and people into spyware - it would be tricking people into automatic slow-reloading links that would cost a few cents... And would continuously reload treating unsuspecting users like a piggy bank.
Instead, the world we have is free by default. Only if something is deemed to cost a lot is there a paywall. Things under that threshold use advertising, which is slowly beginning to fail at the seams. That's because content is expensive, but copying isnt.
(BTW, my points on the main post are still yoyo-ing around 0 and 1. I wish people would respond rather than try to hide my commentary.)
Regarding payment processing etc -- the Xanadu proposal dates from the mid-20th century so it includes a quaint assumption from that era: people expect to be paid for the work they do. What, people are supposed to spend hours typing at a computer for no payment in order to make some billionaire even richer? Who would do that? It's nuts.
No, it's because the rest of us trying to monetize content would be drowned out by the advertisers and scammers and such.
Yeah, we'd make our fistful of dollar bills, and simultaneously owing 5 to 10x that.
At least for the most part, getting hit with malware just means bitcoin miners, DDoS, adverts, or the like. If we were in Xanadu-Universe, then malware would be real cash.
I remember that being featured in the anime Serial Experiments Lain. Here is the Wikipedia article: https://en.wikipedia.org/wiki/Project_Xanadu
Never really understood what it is exactly for though. Which might explain why it never took off...
The Xanadu design of Ted and Roger (and others) is actually quite good. To understand, it does require a lot of reading (and rereading) sources like 'Literary Machines' and the online docs out there. I had to build and run Udanax Green to figure out areas the docs didn't cover.
Now the question is "Is the design feasible to scale?" I'm working toward building a xanalogical system using Udanax Green as a design reference and after 9 months of working on it, I'm convinced the design will scale.
I'm using RDF backend store for the links/endsets. SPARQL/RDF is easily distributed so there is no need to centralize. I'm using IPFS for images.
I'm currently working on a custom browser for viewing, publishing and editing.
Citizen Kane practically invented modern dramatic film.
The techniques Welles developed for Kane were innovative at the time, but are now mundane or clichéd. In terms of writing, framing, lighting, directing, editing, scoring, it was unparalleled.
It's not the best movie in the conventional sense, and maybe not even a particularly good one - I found that the story itself failed to grip me - but it is technically brilliant, and one of the most significant movies in terms of its influence on the art of film-making.
I don't remember the source, but there's a quote to the effect of "I don't think much of Shakespeare's plays, they're just one cliché after another".
Oh, they definitely nailed the drama and suspense. I wasn't panning it for its originality or anything. I just feel like that the script built up, built up, built up and then just.. fizzled into nothing.
Semantic web technologies are not very common today. You couldn't easily implement Xanadu-like content linking in a web browser without using semantic web technologies.
You are aware that Andreas Baader and Ulrike Meinhof were (among others) the founders of the terrorist group RAF (Rote Armee Fraktion - Red Army Faction https://en.wikipedia.org/wiki/Red_Army_Faction), which according to https://de.wikipedia.org/w/index.php?title=Rote_Armee_Frakti... (German article) murdered 33 people and did lots of other crimes? And (same German Wikipedia article) "Baader-Meinhof-Bande" ("Baader-Meinhof gang") or "Baader-Meinhof-Gruppe" ( "Baader-Meinhof group") was its former unofficial name? You are also aware that the RAF was the reason for the "German Autumn" (https://en.wikipedia.org/w/index.php?title=German_Autumn&old...), in which there was a giant witch hunt against potential RAF sympathizers and lots of constitutional rights were abolished, which makes the German Autumn a large turning point in history for anybody who is interested about the history of privacy rights an Germany or is even a privacy advocate?
Ted Nelson's Computer Lib/Dream Machines was absolutely essential to drawing me into the software industry. It's entertaining and inspiring. Everyone should have fun sharing it. That book reached tons of nerds playing with terminals, phones, TV monitors, programmable calculators and modems.
I corresponded briefly with him about this. From my understanding, his goals were:
* micropayments for authors
* immutable revision history
* persistent bi-directional links
* transclusions (quotes connected to the original source) instead of copies
Judging from "Computer Lib / Dream Machines," he obviously knows his stuff. I suspect he just doesn't like programming.
I think his biggest problem was that a lot of his goals were too big for the technology of the time. Even with distributed version control, blockchains, and cloud computing we're not there yet, but we are closer than we were.
It is interesting alternative to "what could have been", and inspired the imagination of technologists in the 70s (I believe).
It featured some unique elements: - subdivision of "documents" so that sections (sentences, paragraphs) can be folded in from other sources. Somewhat similar to remote CDNs for images, but did chunks of text (e.g. when quoting another article) - UI: being able to see multiple documents at the same time and how they connect to each other. (Not the same as annotation, but if you can imagine seeing annotations on the side of the screen... something like that) - greater similarity to the Memex in how information can be interlinked and displayed. - in built royalty mechanism. (Micro payments) - files are stored centrally in a docuverse, which is somewhat similar to a block chain in that there is an agreed interpretation of what a document "is" instead of having multiple copies of the same document.
Ultimately, the World aside Web came along and filled the need for a document publishing medium, rendering Xanadu a historical artifact, many of whose ideas have been realized by other projects in other forms.
Wikipedia entry here: https://en.m.wikipedia.org/wiki/Project_Xanadu
Also worth reading is Literary Machines, since that's an interesting historical read of how people were thinking about uses for computers.