How the Hourglass Won
systemsapproach.org
systemsapproach.org
My problem with IP is that it's not partition friendly. You have to keep your stuff powered on and connected just in case two parties want to have an instantaneous conversation right now, otherwise they can't talk at all. That's not too heavy a burden if you're an ISP, but it's sort of a problem in cases where there are fewer resources available. Like maybe the bits will get there, but they're going to have to wait for the sun to come up and shine on a certain solar panel, or they're going to hitch a ride on a certain delivery truck that's heading that way anyway.
If our software could tolerate reductions in service like this, if its had more modes than up-entirely-and-instantaneous and down-entirely, then there would be fewer incentives for misbehaving governments to attempt to shut it off every now and then, and it would be more likely to be helpful in the event of some kind of infrastructure-damaging disaster.
IP has no notion of connection, and thus no notion of the reliability thereof. That's TCP's job (or what-have-you). I understand the top comment's complaint, but I don't think IP is the problem.
Another case: satellite internet. You can't assume a reliable connection to anything when "beam it into space" is an integral step in the chain of communication. Yet, it works! Whether or not a particular service is reliable is an issue with that service, not the addressing scheme.
IP is an envelope with a to-address and from-address. The upper layer protocols are whatever's inside the envelope - a birthday card, a bill. The lower-layer protocols are USPS, FedEx, the Royal Mail, etc. Blaming IP for partitioning problems is like blaming the envelope for not making it to its destination.
You can't say "get me this data from wherever it might be found" because you have no means of talking about data except as anchored to some custodian machine, despite the fact that the same bits might be cached nearby (perhaps even on your device).
You also can't gossip with nearby devices about locally relevant things like restaurant menus, open/closing hours, weather, etc... you have to instead uniquely identify a machine (which may not be local) and have it re-interpret your locality and tell you who your neighbors are. You end up depending on connectivity to a server in another state maintained by people who you don't know or necessarily trust just to determine whether the grocery store down the street has what you want.
It creates unnecessary choke points, which end up failing haphazardly or being exploited by bad actors.
I don't think you're right about neighbors though. IP does support broadcast to allow you to communicate with nearby machines. Of course, in real networks, this is often disabled beyond some small perimeters because of concerns for overwhelming the bandwidth.
You might think that those layers suck because they are layered on top of low-level protocols. If we just baked everything in from the start, then everything would work more cleanly. That is almost never the case. Those layers usually suck because it is just really hard to do human-level context-dependent whatever. To the extent that they suck for outside reasons, it is usually because the low-level protocols expose a abstraction that mismatches your desired functionality and is too high-level, not one that is too low-level. A lower-level abstraction would give you more flexibility to implement the high-level abstraction with fewer non-essential mismatches.
Baking in these high-level human-scale abstractions down at the very heart of things is how we get complex horrible nonsense like leap seconds which we then need to add even more horrible nonsense like leap second smearing on top to attempt to poorly undo it. It is how you get Microsoft Excel rewriting numbers to dates and spell correcting gene names with no way to turn it off because they baked it in all the way at the bottom.
This is true, but
1) it's an easy problem to solve in many cases (DHCP works great)
2) the exact mechanisms by which an address could be tied to a particular resource are innately dependent on the upper portions of the protocol stack, simply because the very idea of what a "resource" even is must necessarily come from there.
3) the exact mechanisms by which an address could be tied to a particular piece of hardware are necessarily dependent on the lower parts of the stack (MAC addresses, for example)
#2 and #3 illustrate that IP benefits from not solving these issues because doing so would create codependency between IP and the protocols implemented above and below it. Such a situation would defeat the entire purpose of IP, which is to be an application-independent, implementation-independent mover of bits.
Now, of course, if anything is idle for two minutes it's basically dead, and if anything disconnects the connection itself is aborted. So now everything is indeed forced to be instantaneous. (Of course there are kind of reasons for that.. with no timeouts any malicious actor could tie up all a server's port, for example, but still..)
There's no concept of "idle" at the networking (IP, TCP, UDP) level. There are packet response timeouts at the kernel level, but that has always been true.
Some applications (e.g. sshd) also have session idle/timeout settings, but they are just administrative hygiene decisions and usually easy to override. Definitely easy if you control the server, and the ssh client has a KeepAlive config setting if you do not.
> with no timeouts any malicious actor could tie up all a server's port, for example
This might be a misunderstanding. Ports do not get tied up. A server can be overwhelmed with traffic, but that requires packet volume (e.g. DoS attack) and it doesn't matter what ports are used.
Best guess: lack of activity timeouts in a NAT layer somewhere between the two end points that removes the NAT mapping for that given ip/port pairing.
That should be more common in CGNAT or mobile networks than home/office networks. Although I don't know what residential ISPs are up to these days. No doubt some sketchy practices have crept in.
For that matter, it could be a low-RAM (carrier-provided?) NAT gateway (or very high network activity) at the premises, blowing up the NAT table.
Still, could be fixable. If the application protocol has some kind of keepalive heartbeat, or if local hardware can be configured or upgraded.
And as for servers, while you're right of course that server ports don't get tied up, there is typically some memory cost to maintaining an application-level connection, even if idle. I don't know of any common HTTP server for example that, in its default settings, doesn't close connections after some short idle time, for this very reason. More sophisticated attacks, like SlowLoris, have been created for this very reason. A server which doesn't timeout idle connections can easily be DoSed by a handful of compromised clients.
IP doesn't require connection state. If you want resilience in the face of loss you need to add more, like holding the data until a transmission has been acknowledged as received. TCP does that fine for modest packet loss and short outages.
For longer timescales, I don't think you want some sort of store-and-forward baked into the normal network protocols, because meaning decays with time. Imagine I want to watch a movie, but the service is down for a week, so I request it every night. The network stores those 7 requests and delivers them when the movie service comes up again. Do I really want to get 7 copies of a movie I may have given up on? Or even one? That question isn't resolvable at the network layer; it requires application level knowledge. Better to let application logic handle it.
I agree with your concern about government control, but I don't think the IP layer is a place to address it. I think the work around ad-hoc and mesh networks is much more interesting there. That would also drive more resilient applications, but I think that can be built on top of IP just fine.
Or do you need a whole host of different application-level solutions to this problem?
There's a different lens to look at it. There's an old tech talk that got me out of the IP-shaped box a lot of us are trapped in: https://www.youtube.com/watch?v=gqGEMQveoqg
I got Gemini 1.5 Pro to summarize the transcript, which I butchered a bit to fit in a text box:
The Problem:
The Internet is treated as a binary concept: You're either connected or not, leading to issues with mobile devices and seamless connectivity.
The focus on connections (TCP/IP conversations) hinders security: Security is tied to the pipe, not the data, making it difficult to address issues like spam and content integrity.
Inefficient use of resources: Broadcasting and multi-point communication are inefficient as the network is unaware of content, leading to redundant data transmission.
The Proposed Solution:
Data-centric architecture: Data is identified by name, not location. This enables:
Trust and integrity based on data itself, not the source.
Efficient multi-point communication and broadcasting.
Seamless mobility and intermittent connectivity.
Improved security against spam, phishing, and content manipulation.
Key principles:
Data integrity and trust are derived from the data itself through cryptographic signatures and verification.
Immutable content with versioning and updates through supersession.
Data segmentation for efficient transmission and user control.
Challenges:
Designing robust incentive structures for content sharing.
Mitigating risks of malicious content and freeloaders.
Addressing legal and liability concerns.
Currently IP does a good job of isolating the application layer from specifics of the medium (fiber, Ethernet, WiFi, LoRa, carrier pigeons) and provides a way of addressing one or multiple recipients. It works best for links of low latency and high availability.
To my mind, other concerns belong to higher levels. Congestion control belongs to TCP (or SCTP, or some other protocol); same for store-and-forward. Encryption belongs to Wireguard, TLS, etc. Authentication and authorization belongs to levels above.
Equally, higher-level protocols could use other transport than IP. HTTP can run over plain TCP, TLS, SPDY, QUIC or whatnot. Email can use SMTP or, say, UUCP. Git can use TCP or email.
Equally, your interplanetary communication will not speak in IP packets at application level, no more that it would speak in Ethernet frames. Say, a well-made IM / email protocol would be able to seamlessly work with ultrafast immediate-delivery networks and slow store-and-forward links. It would have message-routing mechanism that allows to deliver a message next desk with an IP packet, or another planet with a comm satellite rising above your horizon to pick and store message, then flying over the other side of the planet to deliver it.
I absolutely agree it's not a replacement for IP and that IP has its place. The point rather is to shift one's perspective on what IP is, the implicit constraints it has, and how under different constraints a very different model would be useful and IP would be a lot less useful there. Applications would not be dealing with symbolic aliases for numeric network locations because that wouldn't work.
I don't think IP is going to be outright replaced by other transport-level protocols right at the Internet's "waist", but it can be complemented with other protocols there, still keeping the waist narrow.
IP is pretty much useless here because both the truck (as a carrier) and the people (as sources/sinks), end up with different addresses when they're in different places. You'd want something that addresses content and people such that the data ends up nearest the people interested in it.
It's an example of a protocol which would be in the waist, were it not so thin.
It might not have the authority to broadcast via every access point, so it will likely be very circumscribed, but that’s just a question of the relative rank and authority between the truck operators and access point operators, routing layers, etc… not a question of the technology.
Since even several round trips of handshaking across the Earth takes only a few hundred milliseconds, and the truck presumably spends at least a few seconds in range.
> even several round trips of handshaking across the Earth takes only a few hundred milliseconds
then I'm not sure why you'd bother using the delivery truck as data transport anyhow.
I'm more interested in the case where such infrastructure is either damaged, untrustworthy, or was never there in the first place. If there was a fallback which worked, even if it wasn't shiny, there'd be something to use if the original went bad for some reason.
And in any case how does this relate to the previous comments?
I’m not really sure what the point of this comparison is.
And in any alternate reality I imagine the access point operators will still get to choose who access is granted to, or not?
Of course you could sort of fake locality by piping it through the traditional internet, but the point is to design something (an alternative to IP) that would be resilient even when the traditional internet has failed so using the traditional internet is sort of cheating.
As far as access... I'm not really sure what that means if we're addressing content and not devices. A node isn't going to pick up some data unless it finds that data to be interesting for some reason--likely because it bears a signature from a trusted individual and the cryptography checks out re: its not having been tampered with. If you're assessing trustworthiness based on content there's no need to scrutinize the device you got it from.
Also I apologize I haven't been making a tremendous amount of sense. I think I've maybe had COVID brain for a few days and only just now have I become fully aware of how impaired I am.
And in such an extreme scenario I imagine roughly no adults will care about any ‘content’ at all, food, water, and shelter would overwhelm all other nice to haves.
In this world security looks a bit different. You're not open to remote action by third parties, they don't know where to reach you unless they're already in the room with you. Instead you've got to discover some peers and see if they have any data that you want. Then the game becomes configuring the machines so that data sort of... diffuses in the appropriate direction based on what users are interested in. It would be worse in many ways, but better in a few others.
So suppose the whole neighborhood, and the delivery driver, all subscribe to a topic called "business hours". One neighbor owns a business and recently decided that it's closed on Sunday. So at first there's just one machine in the neighborhood with the updated info, and everybody else has it wrong. But then the driver swoops by, and since they're subscribed to the same topic as the homeowners, their nodes run some topic-specific sync function. The new hours are signed by the owner of the restaurant with a timestamp that's newer than the previous entry, so the sync function updates that entry on the driver's device. Everyone else who runs this protocol and subscribes to this topic has the same experience, with the newer more authoritative data overwriting the older staler data as the driver nears their house. But at no point does the data have a sender or a receiver which are separated by more than a single hop, and we trust or distrust data based on who signed it, not based on where we found it.
I have an application in mind that I think would run well on it, but because our world has crystalized around point-to-point machine-addressed networking, like a million tiny phone calls, it feels like a pretty big lift whereas innovating at other layers in the stack feels much easier--a consequence of the thin waist.
Believe me, this is the least change-resistant option. You don't have to change all the boxes at one end or the other or in the middle to deploy new stuff!
.. unless it's IPv6. Look how well that went. Because it's not compatible, it's effectively a separate hourglass over to the side of the first one.
> IP is that it's not partition friendly
It's simply not at the level to care about that. If there's no link, there's no internetworking. It could perhaps be more router-flap or roaming-network friendly, at some cost.
The thing you want for intermittent operation might be UUCP, or USENET, from the days when links weren't up all the time. Fidonet had similar solutions for the days of expensive telephone links.
https://en.wikipedia.org/wiki/Delay-tolerant_networking
It started as a NASA project for deep-space networks where communications are intermittent, packets take minutes or hours to arrive, and bandwidth is slim. But it's seeing increasing interest for other applications as well.
Anyways, IP is what we have. The cost of changing all of this now is astronomical, so it won't happen.
I think we can get pretty far on just key pairs and hashes.
Basically, modern email designed for disconnected operation. It would work normally on Internet but keep going when disconnected. I was also thinking it would be useful for limited bandwidth like ham radio. I also think that model would be useful for communicating with other planets.
The problem is that it only works for some applications, and applications would need to change to use the new model and deal with the high latency. Which is a big hurdle when mobile and satellite makes Internet more available.
The way I'm imagining it, the applications can each provide a function matching some standardized interface to the network layer. So every time two nodes are near enough to synchronize, they loop through all of the applications they have in common and run the functions for those applications. Maybe some are maintaining a web of trust, others are verifying the content in some way before they decide to store it, who knows--the network layer doesn't care, it just has some node-operator-configured resources to allocate to that app: 1GB of storage and 15 seconds of sync time every 5 minutes or somesuch.
It would be up to the apps to write their sync functions such that old or irrelevant data gets forgotten, new or venerated data gets propagated, etc. Also they'd have to figure out how to make it worth everyone's while to actually dedicate those resources.
You probably wouldn't end up with millions of apps like this--it would only make sense if many people around you were also using the same app--but even just two or three useful ones would probably make it worthwhile to build the lower layers.
E-mail strikes me as one of the more difficult ones to do because it's sort of fundamentally point-to-point, but I could imagine a sync function which gossips about which other nodes that node typically comes across such that whenever two nodes meet in an elevator or at a stoplight or whatever there's a sort of gradient indicating which of the two is more likely to carry a message to its destination.
Data synchronization would be layer on top. I think it would be hard because of latency, and would need to use CRDT or similar. My feeling is that would only work for simple databases, and complicated databases wouldn't be supported.
Because it is so simple, we can add more interfaces and/or routers at any time and expand the network very quickly. By outsourcing responsibility to anything else to other layers, IP can focus on speed.
It'd be really nice for something above IP but below HTTP to gain non-hobbyist traction that separates addressing "who" you want to talk to (an identity) from IP address. So then most applications can work with what they really want to work with - identities - and not network addresses. Existing solutions in typical software mostly rely on HTTP cookies, DNS, and TLS. A lot of existing work on decentralized P2P networks do try or actually solve this problem somewhat with various cryptographic and schemes like DHTs.
> You have to keep your stuff powered on and connected just in case two parties want to have an instantaneous conversation right now, otherwise they can't talk at all.
I mean .. if I want to text you and your phone isn't on, you're physically not capable of receiving it. Something has to capture, save, and provide the message when the other side is available. ISP routers having that capability would be interesting, but not sure it would be a good idea.
Electricity is an hourglass. Coal plants, solar panels, gas turbines, wind turbines, nuclear power plants all produce electricity. This is then consumed by electric cars, computers, washing machines, etc.
LLVM IR is an hourglass. Many compilers and languages produce LLVM IR. This is then converted to many different instructions sets.
I think if you want many-to-many non-coupled relationships, you will end up with some sort of hourglass design eventually.
Details may vary, but that baseline makes it much easier to, for example, have emacs on Windows, Mac, and every flavor of Linux under the sun.
Any market is an hourglass too.
It happens anywhere you have a one group A of things that are necessarily distinct (e.g. software tools) and another group B that is also necessarily distinct (e.g. buried wires and satellites) and people want to be able to use anything from category A with anything from category B.
We could create a mechanism for every single A*B combination, but instead there's gonna be pressure to minimize the effort, and that means a smaller set of intermediate stuff in the middle.
Another example might be that you have a bunch of people with different skills, and a bunch of people with different goods, and then the convergence-point involves money, rather than having a lot of distinct techniques like "how to use carpentry to acquire chickens."
Possibility of fragmentation means that we can not rely on "atomicity" of IP packets. We have to limit ourselves to conservative MTU sizes if we want reasonable performance on possibly unreliable networks. And there are security considerations as well. It would've been great if IP supported a reliable way to do MTU discovery (e.g. by replying with a special "error" packet containing MTU size) or had specified a bigger guaranteed MTU size (ideally, 2^16 bytes).
Finally, ossification of the TCP protocol is a well known problem. Ideally, we would have only IP packets with included source and destination ports, while TCP would be a QUIC-like protocol opaque for "dumb pipe" network hardware.
That’s literally how IP works. The problem is that some middleware boxes block the ICMP error packet that has the MTU so most people don’t see the don’t fragment IPv4 bit. IPv6 gets rid of the bit since there’s not even any fragmentation support (but the error packet remains). To combat this, DPLPMTUD was developed for QUIC where probe packets are used instead to discover the MTU at the cost of global efficiency.
The problem with IP is not the problem with IP, but a problem with middleware that makes deployment of new protocols above IP practically impossible. That's why QUIC rides over UDP rather than being a first class transport protocol.
This allows you to easily detect MTU changes due to changed routing. You can easily determine or re-determine the correct MTU with a single large probe packet. You can feedback application-level packet loss information as part of the "packet too big" message which is useful for not screwing up your flow control. The application can bulk classify and bulk feedback the "lost" packets so you get more precise, complete, and accurate feedback that also costs less bytes. The network hardware can be dumber since it does not need to send feedback messages (for this case) which also results in the pipe being closer to a "dumb pipe" as a whole. Basically the only downside is that you forward "dead" traffic resulting in unnecessary congestion, but you wanted to send that traffic anyways (and would have succeeded if the predicted MTU was correct), so that congestion was already assumed in your traffic pattern.
I guess it may be both, IP packet header could contain a flag depending on which data will be either truncated, or dropped with an error packet sent back.
Truncation is trivial to support at the transport protocol level. UDP, TCP, and literally every other network protocol I can think of already encode (or assume) the "expected" non-truncated size, so you just compare that against the size reported by the hardware descriptors and either drop (which is what already always happens, just earlier in the chain) or the protocol can be enhanced to generate feedback. Protocols literally already do that check (for security reasons to avoid attacker-controlled lengths), so changing network hardware to truncate would likely already work with existing protocol stacks. Any enhancements for smarter behavior than just "drop if truncation detected" would only need to be at the transport protocol/endpoint level, so would be relatively easy to bolt-on without making any other changes to the intervening network hardware and while being compatible with "default drop" behavior for anybody who does not want the extra complexity.
Our forebears have much wisdom to give us, but this might be the most important advice of all.
For those readers young enough to have never used a landline, know that this is bullshit. No VOIP (or cell) link I have ever used today achieves similar latency to a landline call in the US in the mid 90s. Ignoring latency, peak voice quality is far higher than the 8kHz uLaw that was used then, but the 95 percentile quality was much better back then as well.
The flip side is that VOIP is nearly free by the standards of POTS costs in the 90s. This is true for local phone service and even more true as you compare to long-distance and international. Had the Bell heads won out, POTS probably would have come down in real-terms (also a good chance that the quality would have gone up as technology enabled it), but there's no way it would be as cheap as it is today.
The internet is converging on a few major players streaming low quality content , at a higher price, using 100x the resources of cable TV.
We were streaming 500 HDTV channels for $50 / month around 2005. Now similar content would cost you $100 in internet service + $150-$200 in streaming fees. And you need much more sophisticated and numerous ICs for encode-decode.
My point is , you can't judge a mature product against an evolving product . The entire content ecosystem is much worse off. We basically spent trillions of dollars to rebuild the shitty content delivery of cableTV , with worse content.
time spent is converging on social video and streaming video
Full circle
IP video consumption experience is lower quality and requires 100x the resources .
You can embed/tunnel any network transport into another. There is nothing magical about the internet and IP. It is actually being tunneled when you're using a cable modem. WiFi is a horrible hack that encapsulate IP in a very ugly way to make it onto it's wireless tech.
You could have tunnel ATM over IP, I'm pretty sure of it. The depiction seems to me like a flattering extolment of IP.
If you're unable to do that for some reason (e.g. iOS), check out the Kill Sticky bookmarklet, which at least lets you nuke these quickly and with consistent muscle memory. It also gets rid of those trashy sticky headers. https://www.smokingonabike.com/2024/01/20/take-back-your-web...
NoScript also does a good job of preventing this garbage. It can be a bit of effort to use, but I find it worth the effort.
If none of these work, then take that as a signal that the content is not worth viewing, close the tab, and move on with your life.
“Close the tab and move on” is what I've been doing. I'm just suggesting that HackerNews might be a better website if that weren't necessary.
Thanks for the other suggestions. I'm not sure if I can use bookmarklets on mobile (Fennec) but I'll look into it.