NYC Mesh Begins Broadband Expansion in Brooklyn | Hacker News Reader

NYC Mesh Begins Broadband Expansion in Brooklyn

nycmesh.net

279 pointsris88 comments

Anyone working on a Chicago based one? Chicago is quite flat with many taller-than-other buildings scattered throughout that could act as access points.

There's https://www.reddit.com/r/ChicagoMeshnet that gets some posts but it doesn't seem that anyone is active on it.

I've pondered this numerous times, and will gladly contribute what i can. Will check out the sub-Reddit ASAP

I think a chicago mesh net would be awesome! Unfortunately the community around that subreddit seems to have rallied and died a few times. Maybe time for another push?

Chiming in to agree. I was disappointed when I found out the sub wasn't very active when I checked a while back, I'll pop in again.

reddit is not a good place for anyone pushing for decentralization, they'll come for you real quick the moment one government decides you are detrimental to society

"too big to protect" is coming

I'd be down to help with this! Maybe we can start a communication channel? I think there was a Riot channel/room but not active at all. We can try joining that or use a different service (rocket chat? Telegram?)

Here's the Riot room: https://matrix.to/#/#chicagomeshnet:matrix.org

Telegram added location based grouping too we can try.

People may find it hard to believe but Detroit is the leader in mesh networks. They've assisted other cities all over the world in setting them up. I've met these folks and the amount of good they do for the money spent is unbelievable.

https://www.alliedmedia.org/dctp

I do find it hard to believe but thanks for sharing. I know what I am checking out next time I visit Detroit!

Maybe not too hard to believe if you think that one of the main drivers for mesh is the lack of available traditional infrastructure

For people who have visited Detroit, it shouldn't be surprising.

Damn they're growing fast. Last September I tried to setup it up but there was no service because the trees in Tompkins Square Park. Will have to give another looksie.

We have 2 new hubs near Tompkins, one on Ave A and another on 7th St

Has there been any work or thoughts on what to do for long distance city-to-city hops? Like say you wanted to connect the NY Mesh to Boston (not sure if they have one, just an example)?

I live in Phoenix - I don't think we have anything, or if we do, they are all small projects, nothing widespread. We are fairly flat, and there are probably mountains and buildings that could be used for APs (getting permission might be tough, though - because most of them are already used for comms systems - not sure how easy or difficult it would be to piggyback on to a mast).

But we are a good distance away from other metro areas in the state itself (Tuscon and Flagstaff mainly), and connection to say, Los Angeles or San Diego (or even Palm Springs) seems like a virtual impossibility to me.

I'm just curious if anyone has considered how to bridge these long-distance spans in a low cost manner?

I love this project, and I dream of participating in it in my building that is wired for Optimum and Optimum only. It would be a great way to break out of that monopoly.

But I do wonder, how will the rollout of 5G affect projects like this? It's a non-profit so I'll always favor it as a community project, but once Verizon comes knocking with a home 5G router capable of fantastic speeds it'll tempt many, I'm sure.

My cell service on LTE in the US is about as fast as my cable modem. But I can't just cancel my cable internet and tether instead, because every telco performs severe rate-limiting once you use up a bandwidth quota -- there's no truly unlimited data option.

So perhaps you're assuming that 5G will result in unlimited data and the obsolescence of cable internet as a result, but that's not obvious, at least to me.

> So perhaps you're assuming that 5G will result in unlimited data and the obsolescence of cable internet as a result, but that's not obvious, at least to me.

For some reason with every new generation of mobile broadband many people think this is just around the corner but I have yet to find any basis for this belief.

It's actually been happening for awhile now: https://www.telecompetitor.com/pew-smartphone-only-homes-gro...; https://www.pewinternet.org/2015/12/21/home-broadband-2015. Fixed broadband use peaked in 2013, at 70%. Now it's at 65%.

Fun Fact: Some carrier throttling works by throttling packets with an unexpected TTL. Android has a default TTL of 64.

You most certainly shouldn't set the default TTL of a tethered device to 65, because then your carrier may fail apply proper limits to that traffic.

Here in Asia, if my data plan is reached quota, I'll be charged by bandwidth: $x/MB and it's much more expensive than the normal data plans. So I still have full throughput but pay significantly more for it.

I'm curious what severe rate limiting looks like. I have a Sprint unlimited plan and I routinely use about 20-30GB of data/mo and I don't think I've ever been throttled, and if I have I haven't noticed it.

Looks like a totally unusable connection, barely able to load a web page, and here on Tmo it came with an SMS announcing it. It probably took more than 30GB.

There's an interesting org you can become a member of with unlimited 4G data. https://www.calyxinstitute.org/member/map

I'll have to research how this program is funded later, I haven't lived in NY for a few years now but between Lower Manhattan and this section of Brooklyn it seems like richer neighborhoods are getting access first. Do the two supernodes need to be within a certain distance of each other?

Users are expected to 'donate' $20 a month.

https://www.nycmesh.net/join

I'm not sure exactly how that's enforced so make of it what you will.

They also receive donations of roof space and backhaul internet.

Two supernodes are just point to point backhaul antennas combined with a sector (large area) antenna. Distance isn't a problem, line of sight is.

Supernodes are connected by fiber so they can be any distance apart.

The two main areas we have connected are the Lower East Side and Chinatown in Manhattan (mixed income) and in Brooklyn our main hub is in a low-income housing association building, definitely not a "richer neighborhood". It is a priority of ours to get to the neighborhoods that need us the most.

For anyone investigating setting up a local mesh network in their community checkout the Althea project: https://althea.net/

I was interested, until I saw 'blockchain'. Supporting anything that energy-hungry is just flat out.

Not all blockchains are created equal. It is the proof of work algorithms that accompany most blockchains that are extremely energy hungry

In fairness, it appears to use proof-of-stake[1] which has issues of its own, but doesn't use the TWh of electricity that proof-of-work does.

[1]: https://althea.net/whitepaper#blockchain-overview

Many newer blockchains use “Proof of Stake” (PoS) systems that rely on market incentives, instead of "Proof of Work" which is the energy hungry variety. Server owners on PoS systems are called “validators” — not “miners.” They put down a deposit, or “stake” a large amount of cryptocurrency, in exchange for the right to add blocks to the blockchain. Validators are chosen by an algorithm that takes their “stake” into account. Removing the element of competition saves energy and allows each machine in a PoS system to work on one problem at a time, as opposed to a Proof of Work system, in which a plethora of machines are rushing to solve the same problem. Additionally, if a validator fails to behave honestly, they may be removed from the network — which helps keep PoS systems accurate.

Granted, I have no idea what system these guys use - but don't write of the technology just because it has the B word involved.

No blockchain bullshit:

- libremesh - https://libremesh.org (cert is broken :/) (layer 3, they use bmx6 / bmx7 like nycmesh)

- freifunk-gluon: https://gluon.readthedocs.io/en/v2018.2.x/ this is used for more than 100k nodes in germany, it is however layer2 (batman-adv) so beyond 100-200 nodes you'll run into problems but there are solutions.

Shout out to the lonely node on Smith and Douglass! Howdy neighbor.

I believe that's one of the LinkNYC kiosks on Smith St. I wasn't aware they were part of this mesh, although the nycmesh site says they've "become unreliable":

https://docs.nycmesh.net/installs/linknyc/

Is there anything equivalent in SF?! I love the idea of this and would love to get something similar!

Not SF technically but check out People's Open Network in Oakland: https://peoplesopen.net/

Yes SF has one - not well known but you can reach out to Mike McCarthy https://twitter.com/michaelmccarthy

It'd be really helpful if they could come up with some sort of packet of information for renters to give their building owners.

Does https://docs.nycmesh.net/organization/outreach/ have what you’re looking for?

Yes that is exactly it. Didn't see that, thank you.

How does dhcp work in this environment? I assume something like shorter term assignments. That way if you shift DHCP every day without extensive logging you have a good expectation of privacy. We need this in Seattle.

Seattle used to have one: https://en.wikipedia.org/wiki/Seattle_Wireless

Man, I wish they would just expand over the Hudson by a few miles.

Well build a link yourself, that's half the point of a community ISP.

Use the method described here to determine line of sight to an existing node.

https://youtu.be/3eq1Xs-Ahkc?t=126

Then reach out and coordinate. All in all it shouldn't take more than a few hours of actual effort, but probably a few weeks of IRL time.

We have a channel on Slack dedicated to New Jersey, so we are working on it. slack.nycmesh.net #n-newjersey

What's the ping

Each wireless point-to-point hop will add about 10ms.

Try with 1500 byte packets.

  root@OpenWrt:~# ping -s1500 1.1.1.1
  PING 1.1.1.1 (1.1.1.1): 1500 data bytes
  1508 bytes from 1.1.1.1: seq=0 ttl=61 time=8.467 ms
  1508 bytes from 1.1.1.1: seq=1 ttl=61 time=5.832 ms
  1508 bytes from 1.1.1.1: seq=2 ttl=61 time=6.458 ms

Does NYC not have something like SF's monkeybrains.net? This seems sorta-kinda like it, but not?

They are both wireless ISPs. The difference is that NYCmesh is incorporated as a nonprofit.

http://naturalwireless.com/ has a grid all over NYC

How are their residential fees and install process?

Have you used monkeybrains before? I'm trying to find a good ISP in SF and would appreciate any advice!

I used to have it. No complaints from me.

Writing this via MB as I, err, write. No complaints here, either! :)

How's the bandwidth?

100mbps isn't unreasonable to expect out of the gear, but I don't know what they get in practice considering the high RF noise environment.

Just for context, 100mbps is fast. That's like local wired LAN speeds from 15 years ago.

100Mbps was fast 15 years ago. These days it's the bare minimum. And remember that you'll be sharing this capacity with all your neighbors who sign up, and since it's wireless you can't just expand the capacity by adding more cables.

And, as jkilpatr mentioned[0], the latency will make it pretty much useless for a lot of applications (such as real-time multiplayer games, or anything that creates a lot of short-lived TCP connections such as web browsing).

The only way to build out decent Internet infrastructure is to dig. Even if they come from the best intentions then these WISP efforts ultimately just end up poisoning the well.

Then again, maybe it'd be a good thing to keep connectivity at a point just before garbage like Stadia becomes viable.

[0]: https://news.ycombinator.com/item?id=20320949

I have gigabit fiber at home. I also run a WISP.

This is from one of my wisp customers taken literally seconds ago.

  root@OpenWrt:~# ping 1.1.1.1
  PING 1.1.1.1 (1.1.1.1): 56 data bytes
  64 bytes from 1.1.1.1: seq=0 ttl=61 time=7.791 ms
  64 bytes from 1.1.1.1: seq=1 ttl=61 time=7.084 ms
  64 bytes from 1.1.1.1: seq=2 ttl=61 time=6.691 ms

Same server from my gigabit fiber to the apartment unit

  [justin@DESKTOP-UALBV95 ~]$ ping 1.1.1.1
  PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data.
  64 bytes from 1.1.1.1: icmp_seq=1 ttl=56 time=11.4 ms
  64 bytes from 1.1.1.1: icmp_seq=2 ttl=56 time=11.2 ms

The point to point customer antenna added ~6ms of latency the remaining ~2ms is the fiber line to the internet exchange.

tl;dr Wireless point to point is easily competitive to Cable on speed and anything on latency.

> The point to point customer antenna added ~6ms of latency

For one hop. The OP literally has Mesh in the name, so I'd expect a large multiple of that. Especially given that NYC's highrises would likely make it harder to provide direct line of sight.

> Same server from my gigabit fiber to the apartment unit

That doesn't say much unless they're colocated geographically, and the rest of the equipment is equivalent. Especially since 1.1.1.1 is heavily anycasted (like most other public DNS servers).

For the record, my results look like this (also using consumer-grade gigabit fiber):

     teo  ~  ping 1.1.1.1
    PING 1.1.1.1 (1.1.1.1) 56(84) bytes of data.
    64 bytes from 1.1.1.1: icmp_seq=1 ttl=58 time=1.45 ms
    64 bytes from 1.1.1.1: icmp_seq=2 ttl=58 time=1.49 ms
    64 bytes from 1.1.1.1: icmp_seq=3 ttl=58 time=1.47 ms

> tl;dr Wireless point to point is easily competitive to Cable on speed

Very curious about your customer density here. It may be competitive for a few customers, but I suspect that interference from neighbors would cause a sharp drop in urban areas if everyone bought in.

> For one hop. The OP literally has Mesh in the name, so I'd expect a large multiple of that. Especially given that NYC's highrises would likely make it harder to provide direct line of sight.

First off I will note that the goal posts here have been moved. Your original statement was that any wireless effort was doomed. This has now been revised to include many hops and distance to the internet exchange.

NYC mesh uses very traditional WISP equipment and OSPF routing (for the most part, also last I talked to them was a while ago)

If you look at their map they have several two hop nodes.

https://www.nycmesh.net/map

althea.net also runs a multihop wireless network so I can get some samples to discuss right now.

You can find a comprehensive overview of our design here.

https://www.youtube.com/watch?v=G4EKbgShyLw

But to get back on subject Althea has exactly the sort of network your interested in since we're focusing on building a distributed, incentivized, mesh WISP. More than half of our users are behind two or more hops.

Here's a rough sample of some two hop users.

   fd62:e3a0:6100:ae2:2e0a:d8a7:7f25:294f metric  246  RTT  26.976  price  42840
   fd64:30f4:7ea7:5700:f391:6c14:e9c7:6f11 metric  193  RTT  15.750  price  42840
   fd65:9f14:c542:d97e:88dd:13ff:99f8:20d7 metric  239  RTT  29.966  price  42840

Most of these links are degraded by trees and distance, causing higher than ideal latencies. The users are ecstatic though, their only other options are 1mbps DSL or sat internet. No other WISP will enter the area because line of sight is so challenging.

I will also note this is literally the cheapest gear off the shelf. It gets 100mbps to the user but has serious problems with retrying packets it should just drop.

Each hop provides many degrees of freedom, we haven't found a need for more than two hops at all yet.

If we where in an urban area getting a couple of ms would be much easier, interference free 60ghz and shorter distances.

Nullability sounds like a frustrated Comcast exec. The tradeoffs sound reasonable to me on the free alternative.

I run cable with WISP as fallback, their ping times in pfSense gateway monitoring for 1.1.1.1 are within ~5ms of each other.

Point to point can be faster than cable due to fiber/copper running at a fraction of the speed of light. That's why some HFT firms prefer ptp links.

I’ve been lucky to get 2Mbps at places I lived in the last couple years. 100 would be lucky for me.

No way, 100Mbps is fast today for most people.

This 8K youtube video is 5m38s long and 850MiB large: https://www.youtube.com/watch?v=1La4QzGeaaQ

Do the math on that and you need 21Mbps to stream that video without buffering, assuming a uniform bitrate: https://www.wolframalpha.com/input/?i=850MiB+%2F+(5+minutes+...

100Mbps down is fast for most people. What do you think the average user is doing that needs more than 100Mbps down?

> 100Mbps down is fast for most people. What do you think the average user is doing that needs more than 100Mbps down?

For streaming? Sure, you'll be fine as long as you're faster than the content. For downloads (Steam, etc) the difference between 100Mbps and 1Gbps is literally a 10x speedup. If you're waiting for a multi-GB game patch to download then there is a huge mental shift between waiting 20 seconds and 4 minutes.

That said, I mainly interpreted the GP as talking about expectations. You could argue that driving at 30km/h is blazing fast compared to walking, but it's still the slowest car speed that we usually consider worth thinking about.

> "If you're waiting for a multi-GB game patch to download then there is a huge mental shift between waiting 20 seconds and 4 minutes."

It's not exactly a life altering shift though, is it? Four fewer minutes of gameplay time is not, for most people, the 'bare minimum.'

A car that only does 30km/h would be a huge downgrade for any car owner, who would no longer be able to safely drive on any highway. That's a ridiculous comparison. Most users would only perceive any difference between 100Mbps and 1Gbps in uncommon circumstances and the severity of those circumstances would almost always be minimal.

If I'm backing up multiple terabytes to rsync.net, am I going to notice the difference? Oh hell yeah. And ditto if I'm pirating HBO's entire back catalogue. But if I'm bopping around on facebook and youtube? Not a chance. Maybe a few times a month I get a big video game update and have enough time to bake some chicken tendies before it finishes downloading, but a gamer's got to eat anyway doesn't he?

> It's not exactly a life altering shift though, is it? Four fewer minutes of gameplay time is not, for most people, the 'bare minimum.'

It doesn't matter that much in absolute terms, but psychologically it can be the difference between the download finishing while you're complaining about it, or saying "meh, let's do something else instead".

I work from home, family of five and 100 is not enough. all my tv comes over internet. three TVs, often two on at the same time. add kids with tablets, and streaming radio, teleconference and virtual desktop and 100Mbps is struggling. Latency is the tough part, less latency matters for most interactive activities. i work from home: 360Mbps up, 35Mbps down.

> "most people"

I think it's fair to say from your description that your bandwidth demands are greater than average. (Also I think you swapped your up and down, unless you've got a really strange connection.)

opps i did swap, er to test you were reading :-)

i'm not sure my demands will be that unusual, ppl consume what is available. and remote working and EVERYthing over IP is a trend just biting

I have the similar usage to you. Cord cutter, four heavy smartphone/tablet users, remote desktop, etc. About a year ago a badly done patch-panel punch on my part slowed down our internet from gigabit to fast ethernet speeds. (Bad connection on one of the four pairs.) Nobody noticed.

I've got 3 gigs symmetric at my house, with 10-gige in the wall, and Ruckus APs that can do at 500+ mbps if using Apple products. I can't think of a way to use even a fraction of the bandwidth.

How is that an 8K video? It seems like it's only available on YouTube as 1080p (60fps). It was shot originally in 8K sure, but it does not look particularly impressive on my 4K monitor.

The video description says that to show you 8k video, you have to use Chrome.

Check it with `youtube-dl -F`, a 8K stream is available, but may not be presented to you in the browser UI:

    format code  extension  resolution note
    272          webm       7680x4320  4320p60 32524k , vp9, 60fps, video only, 850.48MiB

If you want, you can download it for yourself like this (the +251 part adds an opus @160k audio stream):

  youtube-dl -f 272+251 https://www.youtube.com/watch\?v\=1La4QzGeaaQ

Incidentally some videos also have VERY low bitrate encodes that also aren't shown to most users by default. This particular video goes down to 256x144, but sometimes you see even smaller, particularly with a reduced framerate.

Incentivize the nodes in a meaningful way and performance will quickly improve

Althea.net is such a protocol and it's available for use today.

https://www.youtube.com/watch?v=G4EKbgShyLw

So, back to pay-per-use pricing. Yay?

Flat rate just means that light users subsidize the heavy users.

Flat rate means that the user ends up paying an average 25c/gb in the United States.

5c/gb doesn't sound very bad in that context.

Flat rate is also the biggest contributor to a lot of poor internet service. Providers (cable,dsl) will start selling service at a given speed then oversell it until it's crap. The incentives are all out of whack.

For the 93% of the country without fiber to the door the 2018 FCC report shows that nearly 70% don't see their advertised speed during peak hours.

It would suck for heavy use 'whales' like me, but most consumers would save money since they'd no longer be subsidizing my behavior. The selfish part of me is pretty comfortable with this status quo.

This why staking and network administered rewards are so effective as they dont directly burden the consumer

Transcending the laws of Physics!

where there is a will there is a way

the system doesn't have to be perfectly ungameable

Compared to my location in oz, it's blazingly fast. I'm limited to a max of 1.5mbps (due to long line distance to the nearest adsl exchange). Thankfully, I get 1 bar of 4g, so I use phone tethering to get about 4 times that.

(For context, we have a national ponzi scheme called the nbn which was supposed to bring the internet to rural areas - even more remote than me, but rural still gets nothing, but city areas have had moderate fibre improvements)

Here we have actual unlimited 4G and I used it instead of DSL for two years. Got about 50/25 topping out at 80/30 or so. Was very stable and convenient. Definitely a reasonable setup for rural areas.

This is interesting.

Reply on news.ycombinator.com