yarvin9

Well, normally, if you want to decrement a, you write (dec a). But yes, if you want to write your own implementation of decrement, you could do it this way. It's just as bad an idea in Urbit as in any other language.

Nock is a functional assembly language, so it's not meant as a human interface. I think you mean "increment" instead of "decrement." Yes, [4 0 1] is an increment formula in Nock; 4 is the increment operator, 0 dereferences a tree address in the subject, 1 is the root of the tree; so [4 0 1] means "increment the subject," where your subject is 42.

If you know Lisp, you can think of Nock as Lisp without symbols or an environment; instead of having this hardcoded key-value store, the environment, you have a subject which is referenced with tree addresses (1 is the root, n2 is the left child, n2 + 1 is the right child).

Is this horribly arcane? It doesn't seem that way to me, but de gustibus non disputandum.

Not to be all Winnebago Man here, but all the code in Urbit (protocols and language alike) can be replaced and enhanced, except Nock. In fact, the normal way to drive this process is just by merging the new version of the code. It's certainly true, however, that you don't want locked-down components.

Urbit is very happy to be an HTTP client and server. (Alas, HTTP has become the "everything protocol.") It also has its own network protocol, which it prefers for conversations between urbits.

The unreliability of network packet delivery (or more exactly, the unreasonable effectiveness of unreliable networking) is basically the greatest technical idea of the 20th century as far as I'm concerned.

There is actually a significant difference between processing unordered, unreliable packet streams, and ordered, reliable, message streams, with the "universal design." Neat thing about packets: you can drop them. If a packet sends the server into an infinite loop? Or at least, a timer detects an infinite loop? Drop the packet. From the network's perspective, the CPU is simply detecting congestion.

You can refuse to process a message, but that's not a stateless and passive operation; you have to return a failure report. But dropping a bad packet is the most wonderful thing in the world. By definition it doesn't change your server's state. It also doesn't leak information to the sender...

To be slightly clearer, the values here are Urbit "nouns."

A noun is an atom or a cell. An atom is an unsigned integer of any size. A cell is an ordered pair of nouns.

I am fairly confident that many useful sets can be specified as the fixpoint set of a function in this space.

Yeah, that's exactly right. I usually say "normalization."

Xena is right to note that this simple state-transformation function, f(event old-state) -> [actions new-state], is the obvious and eternal way to build a server. It's also worth noting that it's basically Haskell's State monad.

Whatever you call it, it's a design pattern, not a system service. Urbit on the outside, as an "operating function," is defined by the "universal design." But since it's a pattern rather than a service, it will tend to reappear at each layer of a layered system. Urbit on the outside uses this pattern and so do Urbit applications, but these are very different layers of Urbit.

At the application layer, a command to an IRC daemon is a great example. There still seems to be a good deal of boilerplate in the Lua code presented here. The attraction of the "universal design" is the goal of eliminating all, or almost all, boilerplate code in a network server daemon.

How would one go about that? First, be a single-level store, so the daemon is automatically a database. Second, make application messages are transactions with end-to-end acknowledgment, no return value, and exactly-once delivery. The message is automatically deserialized and validated, and passed to the application as a simple typed argument. If there's a problem with the operation, just crash; the result is delivered as a transaction failure with an annotated stack trace. Also, messages should be sent over an encrypted P2P network and authenticated by scarce, memorable identities...

On the other hand, I'm sure the Lua daemon is a lot faster!

Urbit actually wound up with three different concepts which all want the word "type," and can't all have it. Worse, none of these quite matches the PL-theory meaning. So none of them wound up with the word.

One easy way to state a type is to state it as the range of a function. The function accepts all values, and produces some subset of all values; ideally, all values in its range are fixpoints, f(x) == f(f(x)). This is kind of nice because you've defined both a constraint and a validation function. We often want to validate untyped data and make it typed.

On the other hand, this range (Urbit's "span") is not the same concept as the function itself (Urbit's "mold"). Moreover, when you send data over the network, you don't want to send the range or even the function; you want to send the name of the function, which hopefully is bound to something compatible on the other side.

This namespace becomes a third level of type (Urbit's "mark"): like a MIME type, if a MIME type was an executable specification. One of the problems with most PL type systems is that they predate ubiquitous networking; so they're not designed to address the problem of describing external data, eg with MIME types, XML DTDs, protobuf schemas or whatever.

> I'd guess that would greatly depend on what Mochizuki's world allows one to do.

Um, it's abstract math, it's not going to help you build a flying car. Isn't proving a major conjecture enough?

Most of mathematics predates the modern American academic system. This immense body of work wasn't developed by people with a careful eye on the best way to get a good tenure-track job. It was developed by people who did math for one main reason: they were fascinated by math.

I am not a mathematician, but I haven't seen anyone suggest that Mochizuki's world is boring. It's difficult for me to imagine 19th-century mathematicians resisting the opportunity to become students again in a new, promising, and unfamiliar world.

The 21st-century reaction seems to be: I already got that degree, you want me to start over? WTF? Why? What's in it for me? How do I make a name for myself by studying someone else's theory, which might not even be true? And it's a pretty sensible reaction, given the institutions we have.

It somehow fits that this is halfway up the front page, and there are no comments on it.

A good metric for the vitality of an academic field is the level of genuine interest researchers take in the substance of their peers' work. A good proxy for this metric is the use of preprint servers. Math and physics dominate, obviously. Interestingly, these are also the fields which the Soviet scientific system screwed up the least.

Even in math, as the case of Mochizuki (and de Branges) shows, there are limits.

What's the actual tangible reward for investing a year or two of your career in learning Mochizuki's world? Suppose you showed that inter-universal geometry was really, truly, a new major subfield of mathematics? Would math departments all over the world hire the faculty needed to make this subfield a reality? Making you, the reviewer, who didn't do original work but only checked someone else's work, a big shot? Not #1 in IUG, but maybe #2 or #3?

I'm not a math guy, but it's hard to see. And yet, 40 years ago, this might well have been the outcome. Conclusion: maybe we really do live on Trantor.

"They" is fine, though clunky at times. It is important to note that some persons may be triggered by nontraditional gender-neutral pronouns.

I'm not normally a huge fan of gender-neutral pronouns, but you have heard of pseudonymity?

~hidduc-posmeg is an HN user with a very definite gender, but doxing is bad karma and avoiding pronouns is cumbersome. So, why not?

In a system where you choose your own handle, you tend to gender it. But when your "name" is just a number in a phonetic base, it seems like a pity to sacrifice any of its delicious opacity. This obviously shouldn't be read as some sort of taking sides in some culture war or other...

Doh! Yeah, another thing Urbit is ok at is generating HTML. If you were designing a language, especially a language that's basically in some sense a Lisp, in 2015, you'd probably want custom syntax for XML-shaped data, to avoid having to use a separate template syntax...

These are some useful but quite unofficial tutorials written by ~hidduc-posmeg, who's probably asleep right now. It's always pathetic when your users shame you into documentation by reverse-engineering the system and writing their own. (See also: https://github.com/laanwj/urbit-tools)

These are markdown files served from ~hidduc-posmeg's own planet and converted to HTML by urbit, proxied by ~hidduc-posmeg's star ~doznec. Since this system isn't actually supposed to be live yet, hidduc-posmeg.urbit.org is DNSed to doznec.urbit.org; ~doznec talks to ~hidduc-posmeg over Urbit's own network, an encrypted P2P protocol (%ames). Ideally, hidduc-posmeg.urbit.org would DNS to the actual Unix server the planet is running on, for obvious reasons.

This whole thing, like http://www.urbit.org, would fall over in a few seconds of HN traffic if it wasn't behind an nginx cache. However, if you look at your network window you should see a long-poll. If ~hidduc-posmeg edits zir tutorials from vim on zir Unix server and saves, this will drive an update event that'll propagate forward and refresh your browser. You can see this in action in our demos...

A hint: consider the difference between calling web APIs from a cloud appliance controlled by a third party, versus calling web APIs from a personal server controlled by the user.

A new network has no Metcalfe's law effect by definition. So an important early skill is parasitizing existing networks...

Hoon didn't really work properly until 2012 or so. Actually I've paid very little attention to it since 2013.

Most of the time since mid 2012 (me for about a year, me and a few other for two for two more) has gone into writing Arvo, a purely functional operating system. Arvo is not enough like anything else to be seen as anything else but research.

This is a fairly typical timeline for CS research. What's unusual is just the depth of the stack. I would also point out that if you look at the normal cost of developing any sort of operating system, even to the alpha level, the metrics are pretty good.

Where the project looks really unproductive is in objective measures of content production. For instance, Nock took me roughly from 2002 to 2008, which is something like a bit and a half of output per day.

I understand. And it did improve the comment, so I forgive you. But it's just irritating when someone hangs a fat fastball over the plate, you whack it, then find out that the pitch magically turned into a curveball and all you've hit is a ground-ball single.

An XS Nock shirt is a dress for my daughter, but she loves the "code." I guess all kids are different. But you never know what they're ready to learn until you try to teach them.

Moreover, the idea that Urbit thinks of the network as "a series of procedure calls somehow knitted together" is a very, very gross distortion. (I guess if you think of "'80s networking" as RPC-heavy, you might have sort of a point about that '80s thing. For me, RPC is already in the OS layer.)

Granted, the high-level interface you want to present to the programmer is something like RPC. The two application-level communication paradigms in Urbit are a transactional "poke" and publish-subscribe.

A key difference is that a successful poke (a) contains no return data and (b) is piggybacked on the packet ack. Actually the whole transaction is piggybacked on the packet ack ("single acknowledgment" or "E2E acknowledgment.") Oh yeah, that's another feature that crosses OS/protocol lines.

This lets us produce a particularly non-leaky network abstraction against not procedure/function calls, but Arvo's stacked event calls (if normal events are a lot like GOTOs, Arvo events are more like GOSUBs). So it's not quite RPC. But it's fair to be reminded of RPC.

But at the actual packet layer, a network is a bus for sharing large (but not too large) unsigned integers, which may or may not arrive anywhere. (Network programming is quite a bit easier, by the way, given a bus-width-independent language that can just model packets and blobs as big atoms, then operate on them functionally.)

Effectively, a packet you hear is something that someone said, not something that someone told you to do. Hearing it is learning something. Idempotence at the packet level is crucial, because if someone tells you something twice, it's the same as telling you once. The protocol exists to answer the question: what happens to me if I learn this number?

It's only a short step from here to defining the entire state of an endpoint as a permanently fixed function of the list of numbers you've learned. The computer's state is a function of its packet log. What could be more natural? Why would anyone define a computer in any other way?

But of course, defining a computer as a pure function of its packet history requires you to define its VM and OS as part of defining the protocol.

And a general rule of protocol design is that your chance of achieving a compatible protocol is inversely proportional to the square of the length of the specification. It's also inversely proportional to the extensibility of the protocol.

So you're going to squeeze all of Unix into your RFC? Or even all of JS, bless its soul? And again, you wind up looking for something like Urbit. You may just be interested in the cow head, but you can't get to it without building the whole cow.

I actually like that we're having a substantive discussion in a snarky way.

But 1980s? Not to make it personal, but I finished Brown in 1992 and dropped out of Berkeley in 1994 (where I arguably invented ASLR [0], though the basic idea was Larry Peterson's) My field was OS in general and networking in specific, I took Mark Weiser's ubiquitous computing seminar from Mark Weiser, and if there are pieces of paper entitling me to talk about anything, it's '80s networking. "And you, sir, are no '80s networking."

'80s networking as I remember it: at Brown, it was all about TCP/IP versus OSI and crap like that. At Berkeley, the emphasis was more on ATM. And crap like that. Congestion control algorithms, like garbage collection, can be invented indefinitely. Frankly, the only thing more boring and irrelevant than '80s networking: '90s networking. At least in the '80s people still wrote new transport layers and stuff, in the vain delusion that they might get adopted.

And at least in the '90s they still invented their own IETF application-layer protocols that got deployed, or that you could imagine getting deployed. Like that great winner, XMPP. (I spent a particularly depressing afternoon as a fly on the wall in a pre-XMPP working group at some IETF in '97 or '98.) And ACAP? Wherefore art thou, ACAP? And all the calendaring stuff? Various bits of things that were designed to be open networks got fitted into various proprietary protocols of the 2000s.

(Nothing is ever new in CS, and if you're a grad student looking for new ideas about networking, this pattern suggests that the best place to look is '60s and '70s papers. Have fun digging! Or does no one do that these days, either?)

So anyway, we were talking about cows. I get it. You're not interested in cows, only in cow heads. Obviously I share your feelings about cow heads. Love 'em.

So, your problem is: the Internet is full of firewalls and crap. It's a restricted-routing network. This sucks. I remember the Internet when it had no firewalls. You do too. We remember when the Internet was a social network. What is the Internet now? Like we say on the intro page, it's a fing modem, which you use to log in to fing AOL.

So let's create an overlay network with unrestricted routing. Some kind of P2P scheme. (I hope you know Adam Ierymenko's ZeroTier -- he's api on HN. I believe he's basically solved this problem as you define it.)

Great! You have a near-perfect VPN. What else is an overlay network? Have you recreated the Internet as a social network? You know, the Internet that had a distributed Reddit called Usenet -- which, as a digital society, was as far above Reddit as Reddit is above YouTube comments?

Dude, you haven't even come close. All you have is another VPN. Why were those restricted networks put in place? Nobody had a "firewall" when I was at Brown.

To put it another way: what is your overlay network doing? Since you're delegating layer 7 to the OS, you're providing the same basic service as '80s networking, '90s networking, and of course now networking. You're sending datagrams or streams or something between Unix processes on different Unix machines.

For starters, how do you identify these endpoints? A ZeroTier address is an identity in a sense, but it provides no useful information. There's no way to tie it to any identity you're actually interested in.

Who needs a new network for Unix processes to receive data from an effectively infinite set of anonymous untrusted identities?

We already have one of these networks. We call it the Internet. It was a great social network when everyone with an IP address was an institutionally trusted entity. Once that became untrue, we put firewalls on it and it turned into a modem. Anyone can see that this will happen to your overlay network. ZeroTier can be used as a public network or a VPN -- it's a great VPN. With due respect to api, I would focus on that side of the business :-)

But of course there's a bazillion VPNs. You want something different -- you want a cow head. You don't care about the cow body. That's fine. We don't all have to be generalists.

Your theory is: why hasn't someone built a cow head and stitched it onto the alligator body yet? My theory is: (1) if you stitch any head to an alligator and it sticks, it's probably an alligator head; (2) our alligator already has a head, and nothing will be gained by cutting it off and stitching it back on again.

Now, you can see how this process of inferring the rest of the cow goes. You need an identity system, or something, for your network. A PKI. The cow neck. Who holds these identities? Processes on Unix servers? O rly? You're going to infer that person X signed document Y, because person X is connected -- in some way -- to a Unix process with access key to K? Sure, I guess we do that for HTTPS servers, but... for individual human beings?

Ok, you're going to build access to K as a separate component of each application? One keystore per process? Or since it's 2015, per container or whatever? Or the whole computer has access to K? O rly? This is good, this is really good.

So you decide to do what the browser did: create a new opaque layer, above Unix proper and isolated from it, to manage your applications. Imagine if JS "apps" could make system calls through the browser. There'd be no such thing as a Web app. HN would be like a Java applet or something.

An opaque layer (imagine a node that couldn't make system calls, for instance) gives you two wins: it lets you standardize the semantics of a network node precisely, and it lets you run untrusted code with perfect encapsulation. And it offers the even more intriguing possibility of running other peoples' code automagically, which is incredibly useful in a distributed system -- for example, to disseminate protocol validator updates. Thus, "the browser for the server side."

And you go on cow-engineering thus. Until you have the full cow. Then you can drink milk every day and laugh at your strange detour into cow-alligator Frankenstacks.

[0]http://bit.ly/1YGuvcC

And... this is why I keep coming back to HN.

We should get lunch again sometime, Ron...

I obviously find this a very interesting subject. However, I suspect there are better times and places to discuss it...

No, you're right, it's a highbrow dismissal. Let me be a little more detailed.

Once you've built a system that's both a protocol and an OS, you realize that putting an abstraction barrier between them is like building a cow by building two halves of the cow, then sewing them together. It's incredibly hard and the resulting cow doesn't work very well.

Take a problem like identity. Is this an OS problem, or a protocol problem? Earlier I mentioned validation of message / content types. Is this an OS problem, or a protocol problem? Is the spinal column part of the cow-frontend, or the cow-backend?

These are just the most obvious examples. For instance, the standard way of building protocols assumes that message processing isn't transactional, and the OS is a dual-level store that loses its mind all the time. If your OS is a single-level store, you can essentially use persistent sessions. Which means you can get exactly-once messaging. Which is a very desirable feature that's impossible to achieve if you assume that the endpoints can lose their minds.

So the appropriate comparison is: build the front-end of a cow, and stitch it to the back end of an alligator, creating the mighty alligow; or, build a whole cow.

Granted, Urbit isn't perfect and I'd hardly call it done, but the whole stack (counting apps!) is only 25K lines of code. So the "whole cow" doesn't seem appalling. The alligow -- I wouldn't even try.

That's exactly what we want you to think. :-)

Don't worry, we'll put up some ropes...

This is one thing we don't do in Hoon -- there are no user-defined macros, operator overloading, etc.

If you're going to put a lot of energy into binding syntax to sound to semantics, you only want to do it once (per language, at least.)

I read more like a page at a time. Some might accuse me of being a page-at-a-time writer.

Compare the way you perceive "~~" to the way you perceive "++". I hear these as "sig sig" and "lus lus". (Or rather, as "slus," because that's a further Hoon abbreviation, but never mind.) You don't hear the former at all; but you hear the latter as "plus plus," don't you?

This is because "tilde tilde" is so heavy your brain doesn't want to do the work of hearing it out. But overriding that connection doesn't save energy, which is why you do hear "plus plus." Your brain has to think the very complicated little thought, "squiggle I don't want to pronounce." It would much rather have a sound.

It's torture enough to read McGonagall silently. Out loud? Who would try that? It's tantamount to suicide.

Doh! That serves me right for tweaking these right before shipping the whitepaper. Yes, "pel" is ).

It definitely doesn't mean you have to wait for the audio to finish playing. It does mean that your brain thinks the audio. Which has a lot of consequences, including the energy it takes to think...

Most people do read poetry silently, in the same way they read prose (I don't literally subvocalize), and the sound still is everything. Try reading these two poems silently:

http://www.mcgonagall-online.org.uk/gems/the-tay-bridge-disa... http://www.poetryfoundation.org/poem/174183

Loosely, yes, they're both network operating systems. And of course, Plan 9 is brilliant. :-) All the details are different....

There's a lot of folk workarounds for this problem. They're mostly fine for languages that use ASCII lightly. But it'd be a pretty confusing mess if we applied the "pick something in the jargon file" approach to something like the Hoon syntax.

Everyone who's learned our ASCII dictionary, which admittedly is not a whole lot of people, applies it compulsively.

They do say noitatimi is the highest form of yrettalf...

"Subvocalize" has a literal meaning (micro-activation of the vocal cords) which good silent readers avoid. But you're still activating the vocal areas of the brain [0]. If you read much poetry, you'll see that the connection between reading and sound is pretty inseparable.

[0] http://www.ncbi.nlm.nih.gov/pubmed/23223279

At a certain scale, it's often a lot easier to build one system that solves all the problems.

Building a building: hard problem. Building 1/3 of a building: impossible problem. Building a cow: hard problem. Building 1/3 of a cow: impossible problem.