NYC Mesh – community-owned network to replace your current internet connection
nycmesh.net
nycmesh.net
We also have a matrix chat room here: https://riot.im/app/#/room/#startyourownisp:matrix.org
Don't want to detract from NYC Mesh or the conversation at all just thought this could be interesting for some of the folks on the thread.
Sadly the "National Broadband Network" would've rendered it obsolete by now
I've also noticed that local municipalities are getting into the ISP business. Beverly Hills, for example, is currently deploying a city-owned/operated fiber-backed ISP: http://www.beverlyhills.org/fiber
NYC Mesh is doing very cool stuff--we wish them the best. We're in touch with a few other community ISPs who are growing organically in competition against big incumbent ISPs. It's inspiring.
Beverly Hills, West Hollywood, and Culver City are all constructing municipal fiber right now. In the case of at least WeHo and Culver, it's aimed primarily at businesses. Some residents may see fiber offered by the incumbents, but we did some math of those offerings and it was $133/mo for 1Gbps. It'd be more helpful for the community if reasonable speeds were more affordable and available, than only some wealthy neighborhoods getting fiber.
Is there a way a lay person (non network-engineer) can somehow get involved with their community to help get this started? I'd love to see something like this get started where I live (Minneapolis) and would be willing to help in any capacity I could.
If you have the passion, but not the IT skill set, be the banner waver. Be the leader of the endeavor. Use your passion to convert others. You'll find your talent.
Forgive my ignorance, I am honestly just curious. I took a look at your test area on your website, and it is filled with homes/condos that are in the millions of dollars. What practical reason would someone that could afford a home that costs this much need to utilize your service?
Is this area essentially exactly that, a test area, and not intended to be the location you deploy in first?
Right around this time we were offered funding from WeTransfer as part of their Net Neutrality campaign, but under the obligation serve their area in Venice. We decided to go for it, and use that as a launching point for a proof of concept network. While not ideal for obvious reasons, it goes a long way towards proving our tech and business model. Our hope is that we can grow out the network towards Culver and CD10, acquire some critical infrastructure there, and then push east across the West Adams/Jefferson Park corridor.
It is a true deployment--we don't plan to uproot the Venice service, but it may end up being an arm of the network as opposed to the core in the future. One of the key goals of our project is to free low income neighborhoods from stone age technology by offering broadband speeds far above DSL for a fair price.
Do you buy any transit at all or rely solely on peering?
Who do you peer with, and how easy has it been to find peers, due to the huge asymmetry in traffic?
Do you have any CDN caches deployed (GGC, Netflix, etc.)?
But, most of the networks that provide the majority of the traffic have pretty open peering policies, and the LA area has a lot of places to peer, and peering places are also usually a reasonable place to get transit so... If you're seeing at least a few gbps from FAANG, they'll peer with you, if you're where they are, and all of them have PoPs in LA, because everyone is there (this is a little circular, LA was highly connected in the PTSN because of population size, and that lead to early internet exchanges and undersea cabling, which leads to more people wanting to be there and so on)
I'd imagine if they have the traffic to justify it, and the space to house it, they'd get the CDN caches. It's a pretty good win, peering is settlement free, but access to the peering point isn't free; keeping traffic from using your ingress capacity is nice (depending on your network architecture anyway)
However that isn't your only option. You can just peer with Netflix, they have an open peering policy.
What do you think about the idea of using the sewer system to lay either fiber or 10+gb copper? Before you think about how gross that is, consider the advancements in drone robotics. You would need to devise a cable-laying robot that could traverse the sewer via remote control. Mount fixtures to the top of the pipes and string the cables through, connecting neighborhoods.
This would require convincing the city to lease you the right-of-way and put up a bond to cover any repairs to damage caused by the cables. But it would solve both the bandwidth problem and the "those are our telephone poles!" Problem. What do you think?
More importantly, it gives us the freedom of building our own infrastructure, since access to public infrastructure is prohibitively expensive--by design, it seems.
That being said, there are companies specializing in installing fiber cables into sewers. They use tracked robots to install steel bands inside the pipes that keep the fiber cable in place. It's not cheap, but it is cheaper than ripping open the streets in a city.
There are also techniques to install fiber cable into water pipes.
Also André Staltz's TEDx Talk that mentions it: https://youtu.be/UjfWAbGfPh0
I really do hope this succeeds and takes off amid the likely scenario that SCOTUS will eventually kill Net Neutrality for good in the next two years. Things like this are the way to go if we want a bit more freedom in terms of surfing the web.
Also, if you're curious about what that looks like today: https://scuttlebutt.nz
Or keep an eye on major repos like the firmware: https://github.com/sudomesh/sudowrt-firmware
There are many places to help. Yes, writing code, but also hands-on stuff like mounting radios on peoples' roofs, grant writing, documentation, etc. Even things like establishing consistency across projects: https://github.com/sudomesh/bugs/issues/34
I've really enjoyed it so far this year as a way of practicing software production skills with other community members and non just inside the context of a for-profit startup.
A lot of pieces are far enough it would also make sense to set up meetups and meshes in SF and Peninsula as well.
Tuesdays. 730pm. Omni Common near Macarthur BART stop in Oakland. Come help!
These days, people use the internet like TV or radio: many consume, few produce.
What about a "community" mesh, where e.g. a small town has websites, email, webservices, gaming hosts etc for shops, businesses, sporting, clubs, council, local news and social media for that town?
Facebook.
Also if you really wanted to only have services available locally why not just do that via access control policy instead of building a second infrastructure?
The only downside for me is that I need to "re-log in" to google once a month. The wife would not be amused. Plus, we are currently paying for 200/20 for about $45 from spectrum.
In all other aspects of technical expertise though it's blowing my previous city, Houston, away. Companies open to candidates based on technical expertise alone (rather than throwing non-B.S. degrees straight into the garbage), lots of great meetups, lots of fellow geeks doing exciting work.
Actually, another thing that surprised me was lack of hacker spaces - Houston seemed to have the same amount of them... one. The only major one I'm aware of here is Noisebridge, same for Houston which has TX/RX.
edit: oh, apparently SF has quite a few more than I realized. So does Houston, but not as many.
https://wiki.hackerspaces.org/San_Francisco https://wiki.hackerspaces.org/Houston
It's not Google Fiber, but it's close.
There are a large-scale effort in SF many years ago... but it's a hard city to build in. You can read a bit of that history here: https://en.wikipedia.org/wiki/San_Francisco_Municipal_Wirele...
It's more of a business product than a consumer product, and it's got limited range, but it's great to know the option even exists.
Meanwhile Brooklyn has Verizon fiber gigabit network and supposedly getting one from Altice too. Two gigabit fiber ISPs in one place would be something quite unique.
https://motherboard.vice.com/en_us/article/kz3xyz/detroit-me...
Source: I live here
I'm strongly considering accepting a job offer for a company in NY since I'm starting to get bored of living in Seattle. I figure moving to NY would mean being surrounded by more people and having more opportunities to collaborate on projects like these.
that being said, get involved and get them in Matrix or IRC, not too difficult to enact change I think people could get in on it with appropriate leadership!
zulip looks interesting to me as well.
I think the best I can do at this point is set up my own server running Matrix (or Zulip, which I haven't really explored) and host my own bridge to ease the transition. It's the classic problem where, now that I actually have money to spend, I don't have free time or infinite energy for side-projects anymore.
Mattermost is nice but it's not federated and I don't think it will catch on as a "public-facing" chatroom option. It's a nice solution inside an organizations that doesn't want to pay for Slack. I haven't used Rocket.chat but it looks similar along those lines. Wire just doesn't seem like the right solution.
Would you also find it to be a problem if they held a meeting and didn't serve fair trade coffee? Should they expel any members with politics you find objectionable? Must they demand that everyone give up their iOS and Android devices too because they're surveillance instruments and running non-free software too?
Here's a good Tumblr post this made me think of:
- [The Grey Tribe — Activism, Accessibility, Setting Good...](https://the-grey-tribe.tumblr.com/post/175038067648/activism...)
- [Great Oxygenation Event - Wikiwand](https://www.wikiwand.com/en/Great_Oxygenation_Event)
all this effort = buying a router and plugging it into your computer.
It sounds like you were sharing your own ISP connection a while ago before this was the case, mightbyte?
"Are you an Internet Service Provider (ISP)? No, we are not an ISP, we are a community that shares ownership of a network. Collectively we have our own internet connection at our “supernode”. Also some of our members share their own internet connections.
Our network peers with other networks at an internet exchange point and provides access to the internet without traditional ISPs. We are a non-profit project of the New York chapter of the Internet Society (ISOC-NY)."
They state in their FAQ they are not an ISP which leaves me to believe they don't buy transit from anyone.
But looking at that peering link there are only 122K prefixes being advertised at DE-CIX NY and NYCMesh only peering with 14 other ASes there.
That would leave large parts of the internet unreachable. Perhaps Hurricane Electric or one their other 14 peers at DE-CIX NY have agreed to be act as transit AS for NYCMesh?
1 - Why aren't they recommending any of the newer routers like the Asus RT-N66U/RT-AC68U? They're well supported by OpenWRT and more available.
2 - To start a similar service in another city, do you need fiber already to be laid? For example, I'm in Pittsburgh and we don't have any fiber anywhere, does that mean it'd be impossible to have a community mesh like this here?
3 - Is there a way to find the "internet backbone" building in your city?
2 - a service like this doesn't rely on much fiber at all, outside of the point(s) where the mesh connects to the rest of the internet. Its almost all point-to-point wireless, with the points eventually converging back on the base stations at datacenters.
3 - the common term for this kind of facility is a "carrier hotel" or "ix" (like internet exchange). A carrier hotel is usually a dedicated-purpose facility where lots of different providers connect their networks together specifically for that purpose, but, hypothetically even a reasonably well-connected commercial datacenter will have at least a couple of carrier connections on site who could be used to create a mesh network base.
1. They are likely using point-to-point/point-to-multi-point directional wireless equipment, designed for longer distance high bandwidth wireless connections where you know the fixed location of the transmitters/receivers and can tune/optimize for that purpose. Those devices you mentioned have hardware designed for omnidirectional local WiFi (broadcast & receive everywhere within x,00 feet). You could have one of those in you're home to provide wifi, and a point to point (with a directional antenna device designed for building-to-building/house-to-house connections) on you're roof to connect to the mesh network. Think of it as a mini satellite dish, pointed to the location from which the signal is being transmitted. Also, using standardized equiptment is more practical to efficiently deploy monitor and manager for a project like this.
2. No. The fiber will be there eventually, and wireless bandwidth is by far slowest link in the chain (the theoretical max speed of 802.11ac is eight 160MHz 256-QAM channels, each of which are capable of 866.7Mbps). Cat6 ethernet cable consisting of 8 conducting wires is capable of providing 10,000Mbps (10-Gigabit) maximum data rate, up to ~100m between powered devices. There are several types of fiber, OM4 is capable of 100,000Mbps over distances up to 150m. These are theoretical speeds, IRL divide by 2 twice and that's a good bencmark target (by 5 for WiFi ;-).
3. Internet Exchange Point https://en.m.wikipedia.org/wiki/Internet_exchange_point
Usually you're looking for 'co-los' (co-locations), buildings that host multi-company/parties server and network equipment and have high bandwidth connections to internet backbone provider networks.
MMWC doesn't have its own fiber but does have access to KINBER's PennREN fiber network, a little-known gem in the commonwealth that happens to run down Fifth Ave and a few other main roads in the city. We're working on rolling out access to it. As we're a donations- and grant-funded non-profit, our rollout will be slower than a commercial operation.
Pittsburgh has a lot of fiber running through it, but most of it is commercial backbone fiber. One data center that I know of had 12 ISPs peering with it, and that was in 2009. Only Verizon FiOS has executed a residential rollout in the city, and you can read more about folks opinions on that in /r/pittsburgh.
I've passed this thread to the MMWC executive team for them to add more detail!
But yes, I'm sure there's cities out there with more options than NYC.
I've lived in 4 apartments in 7 years and never once had a choice.
[I'm on a condo board]
"The network connects directly to the internet backbone, so we do not rely on an ISP."
Their peering policy: https://nycmesh.net/peering/
Is that a physical location somewhere? I've read on the peering link, are they saying there's an "internet ethernet port" at 375 Pearl St, New York, NY that they plugged their whole mesh network into? Where's the ethernet cable go from there?
I'm probably asking a bit too much here (how does internet work lol) but I don't understand how they've bypassed all ISPs... I just figured Comcast, being an ISP, also has a controlling stake in whatever means by which it gets data across the pacific ocean or whatever.
Yes, it is a physical location. Often called 'an exchange', there is usually a building in major cities that most networks connect together at. You rent from the building owner, and they provide a 'drop' which is basically a cable you can connect to your core router in your part of the datacenter you are renting (your 'rack' or 'cage')
From that point, you set up BGP sessions with your 'peers', other networks you have agreements with. Sometimes those agreements are for 'transit' (which means you pay per bit sent across the wire, but you can send all your traffic that way) or they are peers you don't pay (you might have other agreements, like we only send this much traffic or that hte inbound and outbound traffic has to match)
According to that peering link from the above poster, these guys will peer with anyone. They basically want as many peers as possible, and are hoping those peers agree to route as much of the internet as possible.
The way BGP works is that each side of the BGP session tells the other 'i will accept and route traffic for these subnets', for example "i will accept packets destined to any IP address in the 1.2.3.240/24 space" The /24 is what is known as a CIDR, and is a way of writing "all the ips between 1.2.3.0 and 1.2.3.255", 256 IP addresses.
So if the core router on my side gets a packet from my network destined to 1.2.3.123, I could send that packet on to the peer who is advertising that they will accept packets destined to 1.2.3.240/24. You need to have a peer for every IP address to be fully connected to the internet. If you don't, you would normally pay for transit, which will route for all IPs.
However, the backbone nodes require maintenance, so the fee to connect directly is expensive. You can't simply move next to a backbone node and plug in. I presume that backbone fee is the primary reason for the individual's fee for hosting an NYCmesh node.
No, the reason Comcast and other incumbents dominate is that they built out the last mile infrastructure. The backbone costs are a minor expense.
Failing that, any Londoners who are interested in starting something like this?
But if you have any ideas on how to make this profitable, I'm up for it. Email in my profile.
If you guys gather around to chat let me know, I might be interested in participating as well.
There is nothing stopping anyone on NYC Mesh to use cjdns, and some of us do run it for certain services.
Come chat if you’d like, https://phillymesh.net
It would be great to see something like this on the Peninsula
How does it work? Can anyone connect directly to the Internet backbone?
> Our patent pending switching technology ensures data integrity.
I personally have little interest in trading one network which is controlled by a few people for another which will just be controlled by different set of a few people. Even if it is powered by trendy blockchain or whatever.
I am incredibly excited by some of the meshnet work we are seeing, but any of these which are just seeking to position themselves as power brokers on our future communication systems really need to be left on the wayside.
We’ve already seen firsthand what happened when our first iteration was turned into a glorified shopping mall. I’d prefer that we avoid this fate with the second iteration.
That does not describe anything related to a mesh network. It appears that the people running it are joining nodes in a mesh network, but they don't seem to be encouraging you to join in a mesh network.
"All of our network nodes cooperate in the distribution of data, and the network can also function independently in case of emergencies or internet shutdown."
Sounds fairly meshy to me.
You can connect to their "supernodes" which are the backbone, that connect to the Internet proper. You can also connect via other member nodes. So by my understanding of the colloquial term "mesh" for wifi, it is.
I do take issue with their FAQ stating they're not an ISP. They are providing Internet access to the public. They are an ISP, even if their structure and methods are unconventional. Maybe they're avoiding the term due to legalese?
It's an interesting project. I wonder if something like that would work in urban San Antonio... Hm!
internet at cyberfoxfire com
Yes you can then extend it a bit a sort of create a mesh, but it's not really the same as having a mesh network which spreads out evenly from individual access points.
The problem is that in the experience of the similar German "freifunk" project, which tried this with custom router firmware, this really doesn't scale well, even with a custom developed routing protocol (B.A.T.M.A.N). So many Freifunk communities are also choosing this supernode setup as a compromise between "full" mesh and good connectivity.
libremesh is based off openwrt, it needs a web-based dashboard, the firmware seems solid, very interesting.
Usually, at any large distances, you'd try to use fibre optic cables if at all possible.