Hacker News Reader

Comment by mahyarm | Hacker News Reader

Mesh networking has theoretical limits that guarantee that normal internet will beat it by a mile.

I worked with a sensor network team years ago. I think back then the largest mesh networks were only about 200 nodes (I think it was a French University, but I'd have to go back to our notes). I'm sure it's improved by now.

There are some very difficult limits when you try to route traffic where everyone has the same bandwidth. This is why we have trunk lines and undersea fibre cables.

That is exactly the issue. Either we need deteministic authoritative lookup and routing, or we need a yet undiscovered routing algorithm and network topology.

Or as danjoc proposed, you can just keep the networks small. No need for a backbone on a neighborhood-wide network.

Sure, until I need to contact my family 300 miles away.

Ok, your neighborhood is bigger than mine!

Surely the whole point of ARPAnet was that it was a mesh so that there would always be multiple routes from A to B. This is necessary to make the network fault tolerant. If it isn't a mesh it is not a network just a tree.

the ARPAnet backbone is a mesh. the "last mile" ISP networks are trees. what we have now is a hybrid system. a pure mesh system in those last mile networks is not feasible at scale, unfortunately. traffic congestion, hardware interoperability, and meshnet protocol latency become overwhelming when the number of users rises.

It's likely that the mesh network would be combined with the wired network where if you need to contact someone much further away it will seek them out across what we designate today as the Internet. But mesh networks will fill in the local and regional communications role as time goes on. It's an inevitability as we keep utilizing smart phones, tablets, and laptops.

This is not true. Any network topology, including the present-day Internet, can be considered a subset of a mesh network topology.

And it works because there is capacity where it is needed, and the network is deterministic with authoritative lookup services. Even if everyone had 100/100mbit and we use a mesh network with a relatively efficient topology, with say random walkers to find ressources, it will wouldn't scale well enough.

A mesh network with an incentive system could provide incentive for building out infrastructure where it's needed, without our existing oligopolies over the last mile. In other words - if you were to pay each of the nodes your packets pass through, you'd have competition where absolutely anyone could hook into the network at any point and provide the necessary bandwidth and get paid for it, no permission necessary.

Thats the current internet with a twist, which means wires, which mean land rights, regulations and large telecom competition.

The point is that your last mile can be a proper meshnet, and the backbone operates in real-time competition, and people wanting to compete on parts of the backbone are much more able to - and people wanting to compete in the last mile are able to as well. You could have someone pull fiber from the backbone to somewhere vaguely near you, and depend on the meshnet from there, or you could travel a few kilometers over meshnet hops, or some combination of the two depending on whether a packet relates to something user-interactive. It solves a problem - monopolies in Internet transport - it just doesn't handwave away basic understandings of physics.

'mesh network' popped up here exactly because we want to avoid the fat links in that mesh.

And getting substantial data (including transit) routed through a heap of wifi devices in a city isn't easy. To the next city - hard. Over the big pond - well...

Mesh doesn't need to be limited to Wifi. The only reason purely radio-based mesh networks are difficult is interference. With enough intermediate wired links interacting with the network (i.e. 3 or 4 Wifi hotspots route to an entire apartment high-rise), the problem is trivial, and hierarchies can be generated based on link capacity and topology just like with the Internet (BGP/RIP). This requires updating routing protocols and in some cases, infrastructure, so there's a lot of hysteresis against change (I know; I've tried).

You can order a pizza in 1kb of data and a 10 second latency would be no problem. Pizza web site can be cached.

Current best widely used distributed topology is Kademlia, and it contacts O(log n) nodes in a search. With 1.000.000 nodes that is 20kb of data. With 1.000.000.000 it's 30kb of data. That is fine for simple messaging.

But once have located a large ressource you need, and you start to transfer data, at say 10mbit/s, you use 10mbit/s for every node from source to destination. It doesn't scale with trafic.

I haven't followed the state of the art that close in a few years, so there might have been some improvements, but none that have made much fuss.

There is probably more than one route between two points, in a tightly connected mesh. But I agree, it gets insanely complicated rather quickly.

But you're assuming sending data is 100% unique and arbitrary.

We know with newer algorithms like IPFS that complex datastreams can be broken down into 256kB chunks all each individually hashed. Not only that, but most data is shared traffic. Yourtube, Facebook, software, you name it. Most unique data people create is textural or small, not the 10MB/s canard.

That require a lot of caching. People needs to be villing to dedicate many GB to this cache around the network for it to be effective.

There's 2 levels of caching in IPFS. Persistent and temporary.

Persistent is when you pin something, or add something (which pins it). It's stuck in block storage until you run a command to remove it.

Temporary is when you go to some IPFS site, and download stuff to your cache. This is the same thing as your browser cache. In IPFS, there's a garbage collector that circle around every time you hit 10GB (or whatever you set it to).

What that means, is some content that goes viral ends up being supported by everyone going to said content. I know I don't clean my browser cache every few minutes, or really ever. And if that means that my and others connections are sped up because of that, the better.

I am curious to learn more about this if anyone has a recommendation for a paper or two to read.

This is probably the most widely cited one: http://circuit.ucsd.edu/~massimo/Papers_files/Capacity-Physi...

Do you mean "wireless" and "wired" or is there something specific to the mesh vs trunk topology?

Not sure that that user was referring to; but yes, there is a specific topological problem: Routing tables. With current internet topology, only a small number of nodes need to know how to route to everywhere; most nodes simply forward 99.9% of addresses to their default "upstream" router.

Since the vast majority of nodes are not able to keep global routing tables (both because they lack the storage, and also because they lack the bandwidth needed to keep such tables up to date) you have two choices: 1. A subset of nodes are "supernodes" which know how to get anywhere (aka. our current internet topology), or 2. Addresses carry enough semantic meaning to allow nodes to guess which way to forward them (e.g., "I don't know who this packet is going to, but it's somewhere in San Francisco").

That second option would require a complete renumbering of the Internet, and it's still not clear that it would work; purely geographic routing would result in packets from Plymouth to New York getting stuck at Land's End because they want to go West but the cables all land East of there.

Note: This problem doesn't exist with small mesh networks, because for small networks you can have every node maintain a copy of the entire routing table.

Would it not be possible to build a distributed routing table?

It might be, but you'd need to be smarter than me.

here's a simple (very bad) example of distributed routing.

Addresses are simply bitstrings of some length.

If my address is a=a1a2...an, I need to store the first segment of the shortest path to each r_1=a'1a2...an, r_2=a1a'2...an all the way up to r_n=a1a2...a'2, where a'k != ak (i.e. all the addresses with a hamming distance of exactly 1 from me). I then simply route a message with an address b=b1b2...bn to r_i where i is the first non-zero bit of a xor b.

Maybe the problem is harder than it superficially seems, but assuming the network is not super sparse, sending things in the right geographical direction (GPS is ubiquitous) should be a very good heuristic. Couple it with exact routing information for the "neighborhood" (idk which radius would make sense) of a node, should be able to avoid going down a dead end most of the time.

There is CJDNS (https://github.com/cjdelisle/cjdns/blob/master/doc/Whitepape...) which has distributed routing table and addressing based on public keys.

Traditional tables won't scale.

Have each node know it's position (lat/lon) and the position of it's nearest neighbors. When you send a packet, you send it to a lat/lon, and every node that sees your packet just tries to nudge it a little closer, until it gets there. Then to speed things up on the return trip we can ask each intermediate to add their position to a list on the packet.

Of course, moving nodes present a bit of a problem. Because they will keep attaching and detaching, there returning packets won't know where to reach them. I suppose if you knew your route through space you could send "scout" packets ahead of you, feel out the terrain, and prepend those positions to the list of your packets (so that they can find you when they return).

Dealing with changes to the (static) topology and also rogue nodes (ones that misbehave, drop packets, lying about their position, etc) are interesting problems. I think they can be solved.

I think geography is a red herring.

The addresses must have meaning and hosts must self organise into a subnet based on connectivity, which feels like a np-hard problem... That does not imply that there is no reasonably well working approximation that works more or less as good except for edge cases. (Literally, ha...)

I came to think of molds and how they route networks with just local agents...

No reason a spatial routing algorithm couldn't take connectivity into account. Indeed, this is a pretty straight-forward application of A*.

There is Kademlia, but it still needs to be improved to take geo distance and link quality into account, for it to work on a global scale.

There is Geodemlia, that routes by geographical distance and is based of Kademlia and its routing buckets.

The problem, as with most P2P-networks, is that its not resistant to high network churn.

Yes, it is like thinking you can implement a service like Twitter in a distributed way. You can, but you loose some of its properties like performing analytics over all the messages in real time.

It is important to first apply basic computer science theories before trying to invent a perpetual motion machine.

> You can, but you loose some of its properties like performing analytics over all the messages in real time.

That seems like a feature :) Why do I want my messages to be available for analysis?

> Why do I want my messages to be available for analysis?

So I think you already have a lot of options for that.

The thing is, many of us don't care if Twitter has our messages for analysis. That's their problem, not ours.

Marketing needs shouldn't hamper technical progress.

Citation requested.

Mesh networking just revolves around the idea that computers can be linked. If mesh networks have theoretical limits then the internet has them as well.

Mesh networking is a collection of multiple technologies in its ideal form.

This gets at the heart of the matter: "mesh networking" means a bunch of different things. It started out meaning a network composed of mobile devices with omnidirectional antennas, running an ad-hoc routing protocol. This is probably why OP said it has limits, as such a network will be slow and lose a lot of bandwidth with every hop. These days, it's mostly a phrase used when people are excited about networking technologies.

Mesh networks have some practical limits they are unlikely to overcome, like crossing an ocean, or any sufficiently large unpopulated area. You do need expensive dedicated infrastructure for that.

Nothing about that is a physical limitation of linking computers. That's a limitation of a specific type of linking computers. There is nothing that says people couldn't lay cable down on the Atlanitc to cross the pond. Nothing about that has to do with the concept of the network being made up of linked computers. That has everything to do with how they are linked.

Reply on news.ycombinator.com