Hacker News Reader

Comment by tptacek | Hacker News Reader

The notion that Urbit believes it can solve all the problems, rather than tackling a single important problem, is one of the things that makes it unserious to me.

The inflexibility of the Internet service model is a serious problem. Poor formal methods for verifying protocols is a serious (albeit not lucrative) problem. Programming languages that make it difficult to express correct programs: serious problem. It is not clear to me how the solution to all three of those problems is the same startup.

View on HN

At a certain scale, it's often a lot easier to build one system that solves all the problems.

Building a building: hard problem. Building 1/3 of a building: impossible problem. Building a cow: hard problem. Building 1/3 of a cow: impossible problem.

Well, some types of buildings are designed to be modular. Cows are not.

I'm also in favor of a single system that solves a bunch of problems than a hodgepodge nightmare. That was the vision behind NLS, the Mother of All Demos, SmallTalk, Plan9 etc. And Doug Engelbart can be found favoring the easy to master but complex at first glance approach https://www.youtube.com/watch?v=VeSgaJt27PM instead of the easy to get started approach of Mac GUIs that won out. The problem arises when the system makes too many weird decisions and people fork it to make their own all encompassing systems. Then it takes forever for crippled versions of these system to be adopted. It doesn't help when even the niche audience is divided. Things that can be disagreed on should be minimized.

So there is no technical need to throw the complex at first glance syntax at new users these days. You could convert it to a more verbose syntax and back again.

Terse syntax looks clean and easy to scan when you use it daily. But what if you don't? Then it's not self documenting, you have to look up commands to read code. If I don't use command line for a few months, I remember the basics like mkdir, ls, cd but I forget the nuances like ls -a and the billion other options. So it becomes a chore of looking things up constantly. Could be fixed with natural language autocomplete for writing code, but not reading. If you considered such use cases, you wouldn't be so sure about the terse syntax.

So many made up words are also asking too much. They'll certainly be worth learning if the system is proven. Haskell has it's own way of doing things, does a lot of things well, but seems to make many things difficult in practice. So it remains a niche that very few learn beyond the surface curiosities. Same for APL, J and K.

And I wrote most of this before seeing https://news.ycombinator.com/item?id=10280348

I mean, this is pithy and all, but it obviously doesn't rebut my point. You could apply the same logic to any disparate collection of engineering problems.

No, you're right, it's a highbrow dismissal. Let me be a little more detailed.

Once you've built a system that's both a protocol and an OS, you realize that putting an abstraction barrier between them is like building a cow by building two halves of the cow, then sewing them together. It's incredibly hard and the resulting cow doesn't work very well.

Take a problem like identity. Is this an OS problem, or a protocol problem? Earlier I mentioned validation of message / content types. Is this an OS problem, or a protocol problem? Is the spinal column part of the cow-frontend, or the cow-backend?

These are just the most obvious examples. For instance, the standard way of building protocols assumes that message processing isn't transactional, and the OS is a dual-level store that loses its mind all the time. If your OS is a single-level store, you can essentially use persistent sessions. Which means you can get exactly-once messaging. Which is a very desirable feature that's impossible to achieve if you assume that the endpoints can lose their minds.

So the appropriate comparison is: build the front-end of a cow, and stitch it to the back end of an alligator, creating the mighty alligow; or, build a whole cow.

Granted, Urbit isn't perfect and I'd hardly call it done, but the whole stack (counting apps!) is only 25K lines of code. So the "whole cow" doesn't seem appalling. The alligow -- I wouldn't even try.

The notion that the OS and the network are essentially the same thing, a series of procedure calls somehow knitted together either with a stack in memory or as a series of serialized frames on a network, strikes me as a very 1980s way of framing computer science. I feel like the last 20 years of network software development have been a repudiation of that idea.

So yeah, I find it frustrating that Urbit insists that an overlay network is "half the cow", and that to have a "whole cow", we must also adopt an excruciatingly idiosyncratic bespoke programming language.

Mostly, for me, this is coming from a place of deep respect for the concept and potential of overlay networks, and of frustration with designs that seem to sabotage that potential in order to make philosophical/political statements.

(That's my only attachment to Urbit as a thing worth discussing; by way of bona fides:

https://hn.algolia.com/?query=author:tptacek%20overlay%20net...)

I actually like that we're having a substantive discussion in a snarky way.

But 1980s? Not to make it personal, but I finished Brown in 1992 and dropped out of Berkeley in 1994 (where I arguably invented ASLR [0], though the basic idea was Larry Peterson's) My field was OS in general and networking in specific, I took Mark Weiser's ubiquitous computing seminar from Mark Weiser, and if there are pieces of paper entitling me to talk about anything, it's '80s networking. "And you, sir, are no '80s networking."

'80s networking as I remember it: at Brown, it was all about TCP/IP versus OSI and crap like that. At Berkeley, the emphasis was more on ATM. And crap like that. Congestion control algorithms, like garbage collection, can be invented indefinitely. Frankly, the only thing more boring and irrelevant than '80s networking: '90s networking. At least in the '80s people still wrote new transport layers and stuff, in the vain delusion that they might get adopted.

And at least in the '90s they still invented their own IETF application-layer protocols that got deployed, or that you could imagine getting deployed. Like that great winner, XMPP. (I spent a particularly depressing afternoon as a fly on the wall in a pre-XMPP working group at some IETF in '97 or '98.) And ACAP? Wherefore art thou, ACAP? And all the calendaring stuff? Various bits of things that were designed to be open networks got fitted into various proprietary protocols of the 2000s.

(Nothing is ever new in CS, and if you're a grad student looking for new ideas about networking, this pattern suggests that the best place to look is '60s and '70s papers. Have fun digging! Or does no one do that these days, either?)

So anyway, we were talking about cows. I get it. You're not interested in cows, only in cow heads. Obviously I share your feelings about cow heads. Love 'em.

So, your problem is: the Internet is full of firewalls and crap. It's a restricted-routing network. This sucks. I remember the Internet when it had no firewalls. You do too. We remember when the Internet was a social network. What is the Internet now? Like we say on the intro page, it's a fing modem, which you use to log in to fing AOL.

So let's create an overlay network with unrestricted routing. Some kind of P2P scheme. (I hope you know Adam Ierymenko's ZeroTier -- he's api on HN. I believe he's basically solved this problem as you define it.)

Great! You have a near-perfect VPN. What else is an overlay network? Have you recreated the Internet as a social network? You know, the Internet that had a distributed Reddit called Usenet -- which, as a digital society, was as far above Reddit as Reddit is above YouTube comments?

Dude, you haven't even come close. All you have is another VPN. Why were those restricted networks put in place? Nobody had a "firewall" when I was at Brown.

To put it another way: what is your overlay network doing? Since you're delegating layer 7 to the OS, you're providing the same basic service as '80s networking, '90s networking, and of course now networking. You're sending datagrams or streams or something between Unix processes on different Unix machines.

For starters, how do you identify these endpoints? A ZeroTier address is an identity in a sense, but it provides no useful information. There's no way to tie it to any identity you're actually interested in.

Who needs a new network for Unix processes to receive data from an effectively infinite set of anonymous untrusted identities?

We already have one of these networks. We call it the Internet. It was a great social network when everyone with an IP address was an institutionally trusted entity. Once that became untrue, we put firewalls on it and it turned into a modem. Anyone can see that this will happen to your overlay network. ZeroTier can be used as a public network or a VPN -- it's a great VPN. With due respect to api, I would focus on that side of the business :-)

But of course there's a bazillion VPNs. You want something different -- you want a cow head. You don't care about the cow body. That's fine. We don't all have to be generalists.

Your theory is: why hasn't someone built a cow head and stitched it onto the alligator body yet? My theory is: (1) if you stitch any head to an alligator and it sticks, it's probably an alligator head; (2) our alligator already has a head, and nothing will be gained by cutting it off and stitching it back on again.

Now, you can see how this process of inferring the rest of the cow goes. You need an identity system, or something, for your network. A PKI. The cow neck. Who holds these identities? Processes on Unix servers? O rly? You're going to infer that person X signed document Y, because person X is connected -- in some way -- to a Unix process with access key to K? Sure, I guess we do that for HTTPS servers, but... for individual human beings?

Ok, you're going to build access to K as a separate component of each application? One keystore per process? Or since it's 2015, per container or whatever? Or the whole computer has access to K? O rly? This is good, this is really good.

So you decide to do what the browser did: create a new opaque layer, above Unix proper and isolated from it, to manage your applications. Imagine if JS "apps" could make system calls through the browser. There'd be no such thing as a Web app. HN would be like a Java applet or something.

An opaque layer (imagine a node that couldn't make system calls, for instance) gives you two wins: it lets you standardize the semantics of a network node precisely, and it lets you run untrusted code with perfect encapsulation. And it offers the even more intriguing possibility of running other peoples' code automagically, which is incredibly useful in a distributed system -- for example, to disseminate protocol validator updates. Thus, "the browser for the server side."

And you go on cow-engineering thus. Until you have the full cow. Then you can drink milk every day and laugh at your strange detour into cow-alligator Frankenstacks.

[0]http://bit.ly/1YGuvcC

It must be evident at this late juncture, Chancellor Yarvin, a full half decade since your Durdenesque doppelganger deemed the denizens of earth prepared for his prolix proclamation of Martian supremacy, that the more Carlylean your Urbit apologetics, the deeper your detractors' disposition for derisory dismissal of your loving labors as the grandiloquent, malformed products of, inter alia, luddite nostalgia for the apocryphal elegance of antediluvian computing, a nocturnal-emission inducing fetish for a fascist technocratic telos, viz automation of global autocracy, or a perniciously persuasive strain of neo-Swiftian satire, which advocates Kafkaesque madness with Kaufmanesque earnestness.

That confuses the hell out of me, because I think your plan is brilliant. What gives, broseph?

My only formal CS-education was at summer camp learning Logo when I was nine. In my early 20s, I taught myself LISP from PG's book purely so I could start wasting a few weekends a year coding an urGAI. The first practical coding I've ever done was this year, when I finally bit the bullet and learned Python and Java.

Is your vision convincing to me because I'm clueless about the harsh realities of programming useful things? Or am I able to discern its splendor because I avoided exposure to the false conventional wisdom that blinds other coders to your insights?

If you put a gun to my head, I'd have to pick door number one. Actually, if you put a gun to my head I'd say Urbit was the best thing since Filmer and then apologize for that over-the-top pastiche. But you see my dilemma.

[Edit: Serious question. You came up with Watt/Hoon 5+ years ago, and you claim it's not much harder to use than Lisp. If that's true, where is all the useful Hoon-coded software?

If I gave you five years to write things in Lisp, you could produce an incredible library of offerings... No?]

Hoon didn't really work properly until 2012 or so. Actually I've paid very little attention to it since 2013.

Most of the time since mid 2012 (me for about a year, me and a few other for two for two more) has gone into writing Arvo, a purely functional operating system. Arvo is not enough like anything else to be seen as anything else but research.

This is a fairly typical timeline for CS research. What's unusual is just the depth of the stack. I would also point out that if you look at the normal cost of developing any sort of operating system, even to the alpha level, the metrics are pretty good.

Where the project looks really unproductive is in objective measures of content production. For instance, Nock took me roughly from 2002 to 2008, which is something like a bit and a half of output per day.

Library gap aside, Lisp has an FFI. If you want to use libpng from Lisp, you just make some C calls. If you want to use libpng from Hoon, you first have to re-implement the functionality of interest in Hoon just to have a sound basis for using the C library (as a drop-in optimization, or "jet").

This obviously makes for slower going, especially near the beginning.

I have added you to the dossier, Mr. Ehrlich - do not fear, this is a good thing.

Duly noted. Feel free to shoot a message to my Gmail.

My address is my first name, followed by the first initial of my last name, then the letter Q.

And now it hits me, for the first time, shamefully: the sense that Urbit is Pynchonesque... it's often said of Pynchon that nostalgia for the 60s is the animating theme of most of his later work, and here it is made explicit: the chain Usenet -> Reddit -> YouTube. (Clearly, I think this is a good thing.)

Do I miss usenet? I have to say I only ever liked it from within emacs, with glowing characters on a black background. Seeing it on a web page, dark on light, is just not the same. Les neiges d'antan

Once upon a time, it was fun to see what you could learn about an IP address that you found in a weblog. Who bothers anymore?

Perhaps it is snowing, in a way that we are too sinful to perceive!

There are plenty of numbers. We could all have one, forever; or until we die, and then we can, like Charlemagne, will them to our children, who will factor them at leisure

Moreover, the idea that Urbit thinks of the network as "a series of procedure calls somehow knitted together" is a very, very gross distortion. (I guess if you think of "'80s networking" as RPC-heavy, you might have sort of a point about that '80s thing. For me, RPC is already in the OS layer.)

Granted, the high-level interface you want to present to the programmer is something like RPC. The two application-level communication paradigms in Urbit are a transactional "poke" and publish-subscribe.

A key difference is that a successful poke (a) contains no return data and (b) is piggybacked on the packet ack. Actually the whole transaction is piggybacked on the packet ack ("single acknowledgment" or "E2E acknowledgment.") Oh yeah, that's another feature that crosses OS/protocol lines.

This lets us produce a particularly non-leaky network abstraction against not procedure/function calls, but Arvo's stacked event calls (if normal events are a lot like GOTOs, Arvo events are more like GOSUBs). So it's not quite RPC. But it's fair to be reminded of RPC.

But at the actual packet layer, a network is a bus for sharing large (but not too large) unsigned integers, which may or may not arrive anywhere. (Network programming is quite a bit easier, by the way, given a bus-width-independent language that can just model packets and blobs as big atoms, then operate on them functionally.)

Effectively, a packet you hear is something that someone said, not something that someone told you to do. Hearing it is learning something. Idempotence at the packet level is crucial, because if someone tells you something twice, it's the same as telling you once. The protocol exists to answer the question: what happens to me if I learn this number?

It's only a short step from here to defining the entire state of an endpoint as a permanently fixed function of the list of numbers you've learned. The computer's state is a function of its packet log. What could be more natural? Why would anyone define a computer in any other way?

But of course, defining a computer as a pure function of its packet history requires you to define its VM and OS as part of defining the protocol.

And a general rule of protocol design is that your chance of achieving a compatible protocol is inversely proportional to the square of the length of the specification. It's also inversely proportional to the extensibility of the protocol.

So you're going to squeeze all of Unix into your RFC? Or even all of JS, bless its soul? And again, you wind up looking for something like Urbit. You may just be interested in the cow head, but you can't get to it without building the whole cow.

>At a certain scale, it's often a lot easier to build one system that solves all the problems.

I have been wondering, are you familiar with Project Xanadu and Ted Nelson's work in general? Project Xanadu took a "whole cow" approach to building a network centered around publishing documents and linking between them. The stated goals for Urbit and Xanadu seem to overlap [1] enough that Xanadu would be easier to implement on top of Urbit than on top of anything else. It could then be Urbit's superior alternative to the World Wide Web for publishing hypertext documents. Last year I emailed Nelson asking whether he knew about Urbit (secretly hoping he might already be working with or for you) but he replied he didn't.

[1] https://en.wikipedia.org/wiki/Project_Xanadu#Original_17_rul...

Without addressing the merits of Urbit, these examples aren't valid. A third of a building is just a building in progress (foundations laid, i-beams installed, etc.) A third of a cow is a calf.

It might be the case that you will get synergistic effects and/or Urbit's vision isn't possible without solving multiple problems. But it could equally be you've bitten off more than you could chew by choosing to work on multiple orthogonal problems.

Thanks. Dividera et Impera. This is the whole idea behind modularity. Solve the network-abstraction problem the best way you can. Then solve the programming language problem the best you can. Do we need a "network programming language"? Maybe, but general purpose programming languages are better for other things, and they too should be able to talk to the network "layer". I probably missed a lot on the article, it was so long to read.

Of course if you have new concepts they need a name of their own. But that only applies if all the new concepts really are new. If they are the same old concepts, use the same old commonly understood name. If they are somewhat different, use a derived name, like "Big Potatoes". They are LIKE potatoes, but they are really big. If they are totally different, try to invent a name that EXPLAINS THEM BY COMBINING the names of previously existing concepts. How many people program in "BrainFuck" and get paid for it? There must be a reason that there are not so many professional brainfuck programmers around. Nevertheless I applaud this project for thinking outside the box, if that is possible they have done it.

A third of a cow is dinner!

Reply on news.ycombinator.com