Ask HN: What would you do with unmetered 10gig? | Hacker News Reader

Ask HN: What would you do with unmetered 10gig?

123 pointslinsomniac176 comments

In the next 3ish months I will have unmetered, symmetric 10gig fiber Internet available at my house. I recently exchanged messages with the first person in town to get the 10gig service, and his reasoning for getting the 10gig service was "it's cool", which I understand...

I'd like to actually be able to justify 10gig service. Ideally, running something on it that appeals to other potential customers of the municipal broadband, to attract new customers to it.

The easy one is to set up a Linux mirror server. In the past I've run a mirror server for ~a decade, so that shouldn't be a huge deal. I like the idea of running a Tor exit node, but worry about the liability of that. Some sort of a block storage for backup service came to mind while reading that Cloudflare Utah post yesterday.

I run production Linux boxes and networks as my day job. But I don't have machine space at my house, so I'm limited in the number of servers I can put in place.

Clever ideas to attract other locals to the Municipal Broadband service?

View on HN

OpenStreetMap is looking for caching servers, those need a lot of outgoing bandwidth https://wiki.openstreetmap.org/wiki/Servers/Tile_CDN

That would be fantastic! To add, the OpenStreetMap Foundation (OSMF) may even be able to provide the hardware.

Please support them, such an important infrastructure that is undervalued.

This is a spectacular idea that I hadn't thought of. Especially since we use OSM tiles at work.

Help the OpenZim.org project run a scraper for their Zimfarm service that archives large sites like Wikipedia, Project Gutenberg, TED talks, etc.

https://farm.openzim.org/about

Or you can run your own internet archiving node with something like ArchiveBox.io or Webrecorder.io and feed it your entire browsing history (including viewed YouTube videos, streamed music, all git repos looked at, etc.).

Alternatively run an IPFS or Bittorrent node, or a Tor snowflake relay, which is a lot less risky than running an exit node: https://snowflake.torproject.org/

And to specifically help users around you, you can then rehost the ZIM archives that you help scrape, and provide a local offline copy of Wikipedia, TED talks, etc to your community so that even if the originals go down, your peers will have access to that content.

See https://www.kiwix.org/en/downloads/kiwix-serve/

I ran an ISP. Most people max their bandwidth for: backups, downloading Steam games, torrenting, and then about a week later their usage drops to about 5-8mbps sustained for Streaming and Videoconferencing.

One question. Why is it that ISPs don't have an on-demand offering for larger bandwidths?

Like if I pay for a 300mbps but I need 1gbps for the day you can charge me 5 bucks extra per day that month.

Are the unit-economics of this that bad? I never understood why ISPs have such crappy/unflexible pricing tiers. Mobile carriers are way ahead on this type of flexible offering and the math seems to work.

I would like something different:

garanteed bandwith + max bandwith

You buy a 1gb line with 50mb guaranteed. That would allow you to use it in the night to a full extend while your ISP doesn't need to provide 1gb all the time

This is effectively what you are already doing. The guaranteed bandwidth just isn’t explicitly defined.

Not exactly. Any sort of SLA will jack the price of your connection way up. We had a 50Mbps connection at the last company I worked for, we couldn't find an isp that charged less than ~$1500 a month for this. This wasn't that long ago either.

Depends on the market. Some regulators enforce minimum performance levels even for consumer connections.

The effort to do this exceeds the potential income.

Mine does that for transfer.

It's a bit cheaper to prepay for an extra 100 Gb than it is to eat the overage chargers. Nothing for speed, though it would be great to get a boost for big videoconferences.

ISPs operate on a similar business model to gyms. They couldn't afford to provide the service if everyone maxed out what they are paying for. Allowing on demand increases of bandwidth would break this business model.

While true in general, the cost of unoversubscribed wholesale bandwidth is rapidly approaching the cost of oversubscribed retail bandwidth.

Wholesale bandwidth is 5 cents per Mbps or less at scale. Not that the ISP pays for all bandwidth, peering and CDN traffic is free.

In other words, even if you max out a 1 Gbps line 24/7, you are only costing the ISP $50 per month.

Add to that that most people are unable to consume even 10 Mbps on average and bandwidth costs become less and less of an issue. An average subscriber will use 2-4 Mbps of bandwidth during primetime, and primetime is all you care about as an ISP.

Nothing in this means that offering bandwidth on demand is impossible or a threat to the ISP business model.

It’s just that the overhead of offering an on-demand product to consumers isn’t worth the time and effort, as you’d be making peanuts while having to deal with a lot of grief.

Bandwidth on demand is a thing for businesses and operators. A good example of this is Megaport, who offers virtual cross connects and transport. Price start from $100 per month plus a varying fee per Mbps, from a few cents to a few dollars per month.

> Add to that that most people are unable to consume even 10 Mbps on average and bandwidth costs become less and less of an issue. An average subscriber will use 2-4 Mbps of bandwidth during primetime, and primetime is all you care about as an ISP

4k Netflix is around 15 Mbps alone, even 1080p is >5 Mbps.

And with games being now > 100 GB downloads these days, I would expect that to contribute meaningfully too to the bandwidth usage even if average person probably doesn't quite download new games daily.

Nope. Subscribers average 2-4 Mbps during primetime, less averaged over the course of a whole month.

Netflix is only about 30% of all Internet bits, and there's a limit to how much Netflix and chill anybody can do.

1080p Netflix is 2-2.5 Mbps in practice because of video compression. You only get 1080p or better with certain browsers too. Many people use Netflix in Chrome or Firefox and only get 720p

I took my numbers from here: https://www.androidauthority.com/how-much-data-does-netflix-...

3 GB / hour to Mbps = 6.667 Mb/s https://www.wolframalpha.com/input/?i=3+GB+%2F+hour+to+Mbps

7 GB / hour to Mbps = 15.56 Mb/s

Wait...netflix in chrome is only 720p!? How do I get 1080?

You have to use Edge or Safari, because DRM.

Or the netflix-1080p extension. It's not a matter of DRM.

Netflix aggressively compresses. HBO on the other hand...

> In other words, even if you max out a 1 Gbps line 24/7, you are only costing the ISP $50 per month.

In bandwidth sure, but the world isn't wired that way. There's a bottleneck somewhere between the ISP and the customer, that can't support all customers on that link running maxed out. In the gym analogy, even if the gym has 1 treadmill per customer, that doesn't help the shortage of weight machines. An ISP may choose to (or choose not to) invest in adding capacity to remove bottlenecks, but that's additional capital outlay that may not be prioritized or budgeted for.

The rationale gets weaker and weaker. Switches get cheaper and cheaper every year and require less and less labor to run.

You can support 5x the bandwidth at 50% of the cost and 20% of the labor than you could in 2005-2010.

The OP is on a brand new FTTH network. There aren't going to be many bottlenecks and any bottlenecks can be cured by inserting a new line card.

And, as I wrote before, in no real life scenario will all 100% of subscriber be running their line at full tilt 24/7.

A 1 Gbps link in meet me room or downtown in a big city is very cheap. 1 Gbps in a residential neighborhood could be the node uplink shared between hundreds of households. Like electricity, most of retail cost is distribution to your home.

In a CATV network, possibly. In a new FTTH network, not likely.

Engineering a non-blocking 1 Gbps network isn't particularly hard or expensive using FTTH.

> ISPs operate on a similar business model to gyms. They couldn't afford to provide the service if everyone maxed out what they are paying for.

My ISP is a "small" provider that focuses on multi-unit urban residential buildings in the midwest. Their business model is to make an agreement with the building management and provide a baseline service to every unit, while offering upgrades and extra services to individual units that want them. They have an incentive to provide far more capacity than the baseline so they have room to offer upgrades.

I pay $25 per month for 250mb symmetrical and am definitely getting those speeds 24/7. I have the option to upgrade to 1G symmetrical for something like $500 per year, but haven't seen the need.

Unless an ISP is serving a very low density customer base, the cost of building out full capacity just isn't prohibitive anymore. My ISP only has around 20,000 customers and they have direct peering with Netflix, Amazon, Apple and others - clearly an ISP doesn't even have to get very large before those cost benefits arrive.

I don't think you fully grasp what operating an ISP is truly like outside of most developed, Western countries. Its absolutely nothing like what you're describing (and blatantly assuming).

For example, Brazil literally has thousands of tiny ISPs, each with, at most, a few hundred customers. They operate on the narrowest of margins, squeaking out close to no profits. They cut corners everywhere. They definitely do not have spare capacity, and over sell by insane percentages.

Another example, Mongolia, where there's basically just one ISP, the government owned Unitel. They have zero incentive to do anything extravagant, or even provide good service. Its a monopoly.

The cost of building out full capacity is very much still a limiting factor in many parts of the world.

ISPs can. I wrote the code that lets them offer such a service, and some internet cafés in Europe do something similar, but they're pay as you go, so you just top up with a different speed tier. (I can not guarantee anyone is still doing this to today though.)

I imagine ISPs do not do this, because of how they sale their service. They want to make it hard for you to get out of the deal you're in, and hopefully in a carrot kind of way where you feel like you're getting a better than normal deal so you don't want to leave.

Giving people the flexibility to change their plan willy-nilly is goes against how most ISPs sell their service. They want to lock you in, not give you free range.

The exception is internet cafés and hotel internet. I could see something like this for cell phone top up plans, but most of those just resell what's available, which means them giving all of the bandwidth they can get.

>Giving people the flexibility to change their plan willy-nilly is goes against how most ISPs sell their service. They want to lock you in, not give you free range.

This seems like a fundamental problem people should be demanding changes to rather than just accepting it as the way it is.

Many, particularly smaller, ISPs pay for bandwidth based on 95%ile utilization. Years ago when I was involved in a few specialized ISPs, say I could fit in 60mbps 95% of the time, and was paying $10/mbps. A day of you using 300Mbps could cost me thousands of dollars.

This is the difference between consumer and business Internet in many cases, and is why so many residential services severely limit the outbound bandwidth: it competes with their extremely profitable business service levels.

(Things may have changed, I haven't been involved with ISPs for 5 years)

The days of $10/Mbps are fortunately behind us. You can get bandwidth for 10c/Mbps or less.

Also a day of 300 Mbps would not have cost you thousands of dollars as you get 36 hours for “free” when using 95th percentile billing.

24 hours of 300Mbps would effective change the 95th %ile to 98.35 %ile, which could be dramatically higher than the 95%ile billing.

As far as $0.1/Mbps, not common around here. Maybe in CA at HE or something, but I can't touch that price around here. Correct me if I wrong, but Amazon's price for a megabit/mo is $55-ish...

Amazon bandwidth pricing is wildly overinflated, so that's not a useful metric to use as a comparison to wholesale bandwidth costs.

At scale means purchasing bandwidth in quantities of 100G. You'd be surprised how straightforward it is to get reasonable wholesale bandwidth rates almost anywhere once you cross a certain monthly spend threshold.

As to the 95th percentile thing, sure, if you've already used up your "free" burst capacity, pushing a further 300 Mbps for a day will push you over the limit.

Amazon, GCP, etc. transit are completely disconnected from reality. Those sub-$0.5/Mbps you can probably get in most big metros across the US and Europe if the connection point is on-net for the ISP in question.

And if you are a residential ISP you can probably handle 80% of your traffic by peering at a few internet exchanges (google,youtube, netflix, etc.)..

Mobile uses a shared channel that needs to be used economically. With wired internet, everyone has their own line. Sure there can be upstream bottlenecks but widening them is cheap, it's the last mile that dominates cost.

So the wired ISP doesn't save too much money when you don't use your BW.

I had an ISP that tried that. At the time 100 Mbps was the top speed offer, I had a 30Mbps package.

Could boost to 100 anytime with one click, but it was ~1 Euro per hour. Used it once lol

They gave up on it, probably because no one used it. Maybe if the price was right.

But technically, it seems easy to implement, at least on some network configs.

But the fixed rate contract for high speeds is likely more profitable for the ISP.

In civilized world (Europe, upc/liberty global) difference between 300Mbit and 1Gbit is $3, per month. Internet economics in US are all about maximizing profits and rubbing nipples https://www.youtube.com/watch?v=vbHqUNl8YFk.

I use my bandwidth to cost people who host "Flash Updater" Trojans money. When you find a site with a downloadable Trojan, just run a download script which downloads in a loop a hundred times simultaneously.

I'm very public about it:

https://twitter.com/AnachronistJohn/status/12343139528652144...

I do hope someone reports me, because if they do, then I'll know who they are. So far, I've cost one scammer close to $2,000 before their links died completely.

Would that be classified as a denial of service attack? (which is apparently a federal crime)

No. I'm not denying or attempting to deny anyone service. I'm simply taking them up on their offer of a free download, but just a million times.

Web sites WANT to be popular. If they get too popular, though, they have to take responsibility for the costs that come with popularity.

If someone wants to claim that the I'm costing them money because their intention wasn't to allow their file to be downloaded millions of times, they'll have to come out of hiding to make the claim. I welcome that.

The real mvp here. Amazing work.

Unfortunately all to often they are hosting these files on a hacked server, or using a stolen card - So your good deed may backfire

Would each new local customer be given 10gig symmetric, or does your loop get 10gig total and your price drops based on # of people on it?

Most people want conferencing that doesn't suck, so that they can Zoom/FaceTime/LTE-over-WiFi/Landline-over-VoIP without having Netflix interrupt their call. But I have that solved on a 50mbit down / 5mbit up connection by using an anti-bufferbloat NAT shaper (Eero SQM, or IQrouter, or homebuilt can all do this).

So if you said "this fiber internet ensures that your calls will stop being interrupted by netflix", that might be a great selling point for people who've experienced this problem on their current Internet, without having to discuss bufferbloat in any significant detail.

(Note that you can still have fairness issues with the wireless router, independent of bufferbloat upstream of it — the above-listed devices apply their fairness stuff not only to the WAN uplink but also to the WiFi downlinks — so 10gbit fiber isn't a complete solution, but it's certainly 95% of the way there.)

The standard service is 1gbps, there is a 5x more expensive 10gig service. Neither one is "local loop speed", but they also do design the service with expected overcommit. I suspect most of the contention is going to be inbound though, so if I'm serving something to the local community there hopefully will be plenty of bandwidth out of my neighborhood.

For 1gbps service, the question would be: Does either the 'modem' linking your home up to the fiber service, or the service network itself, introduce significant high-latency buffering when capacity is exceeded either at your router or at their router?

If the answer is "yes, we introduce significant latency to artificially boost throughput when network capacity is exceeded in either direction", then their 1mbps service is no better than Comcast and is not attractive as a solution to the latency issues of classical providers, and I may as well stick with my 50/5 shaped connection instead of their 1000/1000 connection.

I think you're talking about bufferbloat? If so, I hope the answer is no...

Would you be willing to expound more on bufferbloat? I had never heard of it.

There's a whole wiki for it: https://www.bufferbloat.net/projects/bloat/wiki/Introduction...

To quote: Bufferbloat is the undesirable latency that comes from a router or other network equipment buffering too much data. It is a huge drag on Internet performance created, ironically, by previous attempts to make it work better. The one-sentence summary is “Bloated buffers lead to network-crippling latency spikes.”

I've met a related issue in the field on shared dark fiber links managed by fairly clueless vendors who's switche's buffer configuration led to connections being basically blackholed after a certain number of megabytes of data was sent.

You can donate bandwidth to your community. Open wifi hotspot etc.etc. Depend where you live (neighborhood), but still believe people appreciate. Like library free internet access. You can make people happy. Sharing is caring.

Have to be careful with this or may end up with a no-knock warrant served on your home at 3:00 am for whatever illegal things someone you shared your IP address with.

I really wish there was a way around this, I'd love to offer free wifi, but don't want to deal with illegal content that might flow through it.

I've considered using something like PIA that claims they don't log, but not sure if that would help.

Valid points.

Technically: Pairing this with unmetered VPS (Scaleway) for VPN could be good solution

Hotspot Client -> Hotspot -> Wireguard VPN -> Scaleway VPS

That way the client always has hidden your ip and traffic encrypted.

$3/month for 200/Mb https://www.scaleway.com/en/virtual-instances/development/

Ignoring everything else, this will add a lot of latency, unless OP is located close to Scaleway’s DC.

Fully aware.

Still, high latency internet for free is better than no internet access at all.

Another solution can be registration form and tracking end user private IPs assigned (Ipam) for such any legal issues.

Maaaaybe...

My old company, everyone worked remote and we set up a VPN to our data center as the default route. One of my employees did some benchmarking comparing the VPN to our data center vs. without VPN.

He found that it was faster to use Comcast to our data center, and then using their InterNAP optimized transit. I want to say it was something like 30% faster on average, but I don't have specific numbers and they'd be ~8 years old now anyway.

Scaleway’s data centers are in Europe :)

Even Comcast will be able to beat that.

Couldn't you just block all unencrypted connections? Probably also useful from a user privacy standpoint.

That solves eavesdropping, but you still could have someone that submits threats to .gov entities with your IP and have men in black raid your house at 5 AM.

Route the wifi traffic all through Tor. You can even get this prepackaged for you by using something like https://learn.adafruit.com/onion-pi/overview

This is the only workable solution for giving away WiFi to others. But then no one would use it because every site will captcha you to death if it can even load. It's terrible but the U.S. legal system forces everyone to subscribe to ISPs individually.

That is a really good idea!

There are tons of solutions for semi-open wifi for custmers, e.g. Sonicwall or Meraki firewalls and filter rules, etc. Many of them are Out of the Box solutions too, and are plainly advertised as SMB / coffee shop solutions.

Set em up, turn on filtering, log stuff, turn over any records if/when law enforcement asks.

You could probably hack something together that costs less, but that's adds overhead and wastes time; you need just enough to prove you're doing diligence and keep out of the low hanging fruit. Log everything, and if the fuzz asks for records be able to hand them over.

It's sad that we live in a society where this is a concern, because I really love and appreciate having open wifi when I'm traveling or moving.

I've run an open network to try to normalize it for the past decade, and I hope others are willing to do the same.

How do Starbucks and restaurants deal with this? I'm pretty sure keeping good logs (for a long time) is all they do. Am I wrong?

They have corporate firewalls and/or traffic routed through filtering orgs. Head over to your local starbucks and try to go to questionable sites, is probable you'll be blocked. Chances are good they're logging too, and they have cameras and in a lot of cases credit/debit card info too, so they can correlate things.

Mom and Pop shops are a different story, but most of the places around me are using Meraki boxes -- I recognize the log in prompt.

And broadly speaking, the cops realize that coffeeshops are going to have wifi and aren't explicitly putting up Tor nodes or trying to be an ISP, so they'll cut them slack if they can provide what deets they can. Usually they have insurance and lawyers to CYA on top of this.

This is most likely against the terms of service of the Internet provider.

On the other hand, it might not be against the terms of service if you were to buy a sufficiently expensive symmetric fiber connection. At which point, you would then have become the ISP (with all of the hassle that would likely entail).

Sure. A wholesale Internet connection that allows resale usually goes for a few thousand dollars per month.

Interesting idea, could pair well with our "Little Free Library" in the front yard. Two concerns: Liability for use (as mentioned) and I also don't want to take away from subscribers to the city service by using our open WiFi. Perhaps a time-based limit?

Now, if I was taking away from the incumbents... :-)

I'd have to check into the terms of the service though.

Related: Anyone used those Ubituiti Mesh things? Not the Amplifi, the other mesh. I've toyed with the idea of putting some of those around the neighborhood to provide some sort of wide area wireless, but never pursued it.

TIL "Little free library" thanks to your comment; sounds fantastic!

>Open wifi hotspot etc.etc.

Only do this if you're prepared to talk with law enforcement about child pornography being shared from your IP address.

I believe in Canada activity "from your IP address" is not enough to legally imply you are the one who did it, which is why even the Canadian equivalent of DMCA notices have no teeth here when sent to residential homes. But IANAL, YMMV, etc.

If the police smash down your door, the news labels you a pedophile, and you have to spend $$$$$ on legal costs an eventual acquittal is still a huge loss.

Right, I'm saying afaik in Canada the police don't have a right to smash down your door only based on an IP address. This is different from the US where an IP address is enough probable cause to get a search warrant.

See USA:

- https://www.mjpetro.com/search-and-seizure/search-warrant-at...

vs Canada:

> Receiving a notice does not necessarily mean that you have in fact infringed copyright or that you will be sued for copyright infringement.

> The Notice and Notice regime does not impose any obligations on a subscriber who receives a notice and it does not require the subscriber to contact the copyright owner or the intermediary.

- https://ic.gc.ca/eic/site/oca-bc.nsf/eng/ca02920.html

- https://www.mccarthy.ca/en/insights/blogs/snipits/you-can-st...

- https://apanewslaw.wordpress.com/2014/06/17/the-supreme-cour...

But IANAL, please correct me if I'm wrong here.

For child porn it is still grounds for a search warrant. Which they'll execute and return your stuff after not finding anything (six months later minimum).

Notice and notice is about copyright infringement only. It's just a pretty toothless way to intimidate. If they were actually going to do something then they would use notice and notice to serve you a summons to court. In that case it's either small claims, where you don't need a lawyer and the judge will help you if the other side uses one, or it's full court where the loser pays the all the winner's costs. So as long as you're on the right side of the law, it actually doesn't cost anything if you get sued (no filing fees as the defendant).

Whether or not you're actually liable in this situation is going to have to be answered by a real lawyer.

Thanks, good info.

You mentioned a linux mirror,, A mirror is a just like CDN PoP . Assuming your neighbourhood is not full of linux enthusiasts , if you can run a actual CDN PoP it would be the highest impact for your neighbourhood.

Anything for specific providers like FB, Youtube, Netflix, Prime,Play Store, Apple , Disney + etc or general CDN providers like CloudFlare, MaxCDN etc can change end user experience enough for them to buy in.

It also has the added benefit of saving municipality upstream peering requirements and perhaps money.

Alternatively you can provide application services like OwnCloud, Jitsi for equivalent cloud products, it will be cheaper/faster etc. and target schools/community services, it may help them save on their IT bill as well.

Final idea discord, game servers, streaming servers etc can generate network effect driven interest. i.e. If i want to play with my school friends and few of them are on your servers on the municipal broadband already it is strong incentive for me to get one.

AFAIK you cannot self host a CDN node for any of the services mentioned above.

You might be able to self host an Apple cache.

Perhaps he can convince the municipality to host for the providers he cannot, it will still serve his objective.

You would be surprised on how small some PoPs around the world are. Providers usually come with their own boxes and have specific requirements on space/power/legal ownership etc.

I doubt the municipality will have much luck either.

CDN operators and OTT providers have fairly high traffic requirements before they will qualify you for an on-net appliance.

Netflix, for example, requires a minimum of 5 Gbps of traffic until they will even talk to you.

Perhaps thats is true for Content Providers like Netflix Youtube, for pure play CDN providers adding PoP in new areas bolsters their network and helps in sales a lot, it makes no sense for them to not talk to you at all.

From a security perspective, even assuming those operators already design to not trust the locations they're in, shipping servers to random end users seems like a bad idea.

Oh they don't trust the location at all, in some DCs, I have seen continuous camera feeds, actual cages which are locked etc. this is /was the case when you have to ship your equipment to a co-lo,

As far as "neighborhood full of Linux enthusiasts", the service area is the whole city, though buildout is still in its infancy. Population is ~170K.

Netflix does have a program where you can submit to get them to ship you a CDN box. I might submit that if I do this, but I doubt they want those in a residence, even if it is 10gig connected. Maybe for a year or two until the municipal provider gets their "data center" space up.

10 Gbps connectivity is on the smaller side for a Netflix appliance. They won’t even talk to you unless you are doing 5 Gbps of traffic from them.

A more realistic plan is to have the muni ISP peer with Netflix at the closest IX and take it from there after traffic levels rise over 5 Gbps.

That's probably the plan longer term, but the nearest IX is probably 60 miles of fiber away, and the muni is padding as fast as it can to just build the network.

While they don't have a data center, I presume they do have a place they could put a Netflix box. But that was the closest I know of for running a CDN node. As I said, seems unlikely...

60 miles isn't that far. If you can get some dark fiber you can go the full distance with one hop using off the shelf optics.

Not that you are particularly hard up for connectivity. Fort Collins is on the old Level3 network so you should have no problem getting transport, waves or dark fiber to the Denver IX. Or you could just buy IP transit at the local PoP.

I would have to read your terms of service before I could offer any useful input.

Depending on those ToS, offering services to others could expose you to legal problems.

Yeah, you will definitely want to read the ToS very closely. A lot of home service providers explicit say you cannot host and commercial services, and at least in the past had issues running servers. Back in the day my internet was shut off because I had an ftp server running. Though don't think that is common anymore.

In addition I would assume not 100% uptime, and not enough uptime to run websites for businesses etc. Unless there is a reason to think otherwise. So whatever it is it has to survive downtime. Torrents for example would be OK, but http servers for business not so much (but http servers for hobby sites maybe).

I honestly haven't cared about bandwidth since I hit 50mbps. Having 300mbps, it's fun when I download a new Steam game to watch my hard disk become the bottleneck. But that's about all I get out of it.

I regularly use my full 500mbps between torrents, steam, backing up my VPS servers to local NAS, and my whole family backing up to my NAS over the internet from their 100mbps connections.

I'll admit I am the exception, but with a decent ZFS pool you can easily saturate 500mbps with remote backups alone. I have a bunch of cheap SSDs in a mirror config and get about 2000MB/s write on a good day, so the bottleneck is very solidly on the network side.

How do they backup to NAS using the Internet connection? Is it not local?

Time Machine over VPN (Tailscale/Wireguard), works great. Requires fast ping and good bandwidth though, which we have (<10ms rtt from NYC to MTL! still blows my mind).

Family -> NAS (using Time Machine) NAS -> Backblaze B2 (using Duplicati)

300mbps = 37.5MB/s. That's a pretty slow hard drive.

¯\_(ツ)_/¯ it's a 2TB spinning disk that's a few years old. Been meaning to upgrade to an SSD for this exact reason.

Yeah Steam and torrents are about the only thing using all the bandwidth. Google Drive (free) rarely gets to even 50Mbps. But for a multi person household, gigabit fiber is great

> I like the idea of running a Tor exit node, but worry about the liability of that.

There are also guard nodes and intermediary nodes, which don't cause third party traffic to exit from your house.

> I run production Linux boxes and networks as my day job. But I don't have machine space at my house, so I'm limited in the number of servers I can put in place.

A single virtual machine host can saturate 10G and allows an arbitrary number of virtual machines, limited primarily by memory and storage. Linux KVM is good enough for Google and AWS and it's free. Get some Socket G34 or LGA2011 workstation with 8+ memory slots, fill it with cheap registered DDR3 and you'll be all set.

But realistically, the biggest benefit of 10G to ordinary people isn't being able to transfer 2500TB a month, it's being able to transfer 25GB in 25 seconds instead of hours.

There are people who upload 5GB to YouTube every day. With garbage upload speeds like 10Mbps, that takes more than an hour and a half. With 10Gbps, not only would it only take five seconds, you don't even need to upload it to YouTube because you can host it yourself.

> you don't even need to upload it to YouTube because you can host it yourself.

One of the main value props for YouTube has to be discoverability right? Depends on why you're uploading I suppose.

It's kind of chicken and egg. Everything ends up on YouTube because people don't have the bandwidth to self-host video, so then people go to YouTube to look for videos.

If more of them were self-hosted then the default search method would be one that finds them everywhere. At this point there is more work to be done than only getting people more bandwidth in order to get there from here, but one step at a time.

My primary concern with limited server space is things that are computationally expensive scaling with bandwidth. Serving 10gbps of HTTP Linux mirrors that have internal signatures is different from serving OwnCloud over HTTPS, for example.

Re: ToR. Perhaps it's changed, but back when I ran a routing node (probably a decade ago?) it seemed to be of limited use because of the limited exit nodes, limiting the overall bandwidth. But might be worth looking at again.

Relays help protect privacy. The more independent relays that exist the harder it is to attack the network.

I've had a number of dedicated servers with 10G unmetered bandwidth (and still do have access to one), and its difficult to find good uses for them. A younger, more idealistic me did run a large exit node with support from an organization with legal backing, but this still lead to emails and calls from various countries Anti-Terror police forces. Would not recommend without serious consideration and talking to your ISP.

In a similar spirit, maybe IPFS nodes, or instances of federated communication networks like Mastodon/Matrix/etc for friends and family, or maybe even something like Nextcloud. If you band together with a few people in your community, you could also use each other’s machines for encrypted “off-site” backups (note: you’re all still sort of co-located geographically, so YMMV).

> Clever ideas to attract other locals to the Municipal Broadband service?

If that's the aim, imho it's really difficult. Assuming your neighbourhood isn't made up 99% nerds&geeks, I'd assume most private users are pretty happy if they can watch movies in 4k, download their games on Steam and/or update their gaming console "fast enough". Downloading 50GB already takes only ~30m on my 400MBit/s, and wouldn't feel much different to 1Gb/s. Latency for gaming/video conferencing also isn't an issue these days. Maybe for big families or shared housing (student dorm) this would make sense.

If you wanted to sell me, you'd get me with (1.) reliable and (2.) affordable. Maybe a no-BS pricing scheme instead of "25 US$/month the 24 months and after that 70US$/month" (which virtually every ISP in Germany does, which means I need to waste my time with stupid ISP hopping every other year if I don't want to feel like being ripped off). I just know that - for my usage - there is no practical difference between 1Gb/s and 10Gb/s. At least none that's worth paying more. Sorry, not that useful, but maybe some valuable feedback? :)

On the business end, this is much more interesting. No need to worry about hitting bandwidth limits when video conferencing with customers is pretty nice (we sometimes have that issue and work around it with multiple connections). Also a perk: Faster access to cloud-stored data, or the ability to put data on-premise and still allowing sufficiently fast external access. Of course it must be reliable. But then, your day job is networking, I think you know that kind of business better than I do :)

//edit: Ooooh, post a "Ask HN" and bask in the envy of the internet people =) /s No, seriously, I think promoting offerings like this is a good thing to do, and asking for advice on HN is a good idea. I'd love to see (or maybe even run) a local, no-BS ISP in our municipality.

The service should be providing the reliable/affordable/no-BS portion. It's $60-ish/mo for gigabit symmetric, $300/mo for 10gig, run by the city, all underground infrastructure. The city power infrastructure is super reliable, so they do have some demonstrated ability...

I am interested in knowing the price that you are paying for this 10 Gig unmetered connection.

From another comment it looks like $300 USD per month

https://news.ycombinator.com/item?id=23481803

FWIW, you can get 10Gbps for $50/month in Switzerland [0] and Sweden [1].

[0]: https://fiber.salt.ch/en

[1]: https://bahnhof.se/

>Linux mirror server

Consider adding torrents.

You could also check out running a folding@home node. Though that seems to require a fairly beefy server.

https://www.youtube.com/watch?v=HaMjPs66cTs

Out of curiosity, do they include hardware to actually use it? When Comcast came around here trying to get people to sign up for 2Gbps they required a couple thousand up front and for you to rent a PCIe card for your PC to actually get the speed (since you needed a 10Gbps NIC), which would've been annoying. Unless you have a ton of devices, or are hosting services (does your ISP let you do that?) I don't know what I'd use it for.

Having a Linux mirror within 100 miles has been handy for me in the past though, so if you can spare the storage that would be a good idea.

The hardware to use it is pretty cheap these days, a 10G nic is about $13 on Ebay. A 10G switch is $200-400, and a ~10G router is now $380 from Ubiquti (will only do ~8.5Gbps right now).

They seem to be providing a 10gig copper port on the equipment they drop in, from looking at a photo of the other guys 10gig install... I don't currently have any 10gig, so I'd have to put those parts in place.

I've got 1gbps symmetric unmetered fiber here, and I really don't do much more than when I had 100mbps, or 50mbps.

What I really like is that everyone in the house can go crazy and it doesn't affect me working.

Some comments here suggest donation, which implies that groups of users could utilize what you are not using. But I am used to rationing 250 MB (Verizon plan from 2015) - and there is so much information that can be obtained with this amount of data (mostly text based and some pictures). High quality Youtube videos do not mean as much to me as the audio. A 10 second HD Snap or TikTok does not concern me. Torrenting hundreds of gigs of movies will only occupy your life with sitting and watching movies. Video game downloads are a one time affair (mine have been performed/patched every few months at a public watering hole Starbucks/Gym/Walmart). Aren't you tired of HD Zoom calls with your social circles?

5G wireless everywhere approaching does not feel practical to me. I do not subscribe to conspiracies about direct health detriments (indirect more concerning - people lusting for 4k videos of everything while sitting in a bathroom stall, or in a public place alone instead of natural resolution of the eyes gazing upon real life).

Perhaps high resolution IoT data upload could occupy your infinite bandwidth somehow? This would only be local.

Red flag to me seems to be constant upgrading of "data throughput". Surely data transmission is good, but at some point is it a waste of effort when we can all only take in so much? Youtube audio and 480p video satisfies my curosity...

I'll be honest there is absolutely no need for average or even very technical home users to pay for a 10gig connection right now. If you want to make municipal broadband attractive, the focus should be on providing cheap and reliable 100-250 Mbps connections to a large area.

Just to put some real world perspective on this. Netflix recommends 25Mbps to stream 4k movies.

https://www.usatoday.com/story/tech/columnist/2017/12/10/you...

Metcalfe's law. Every time I have to accommodate someone on the wrong side of the digital divide I wish we had wider broadband deployment. But I know pausing speed upgrades in already served areas won't bring about universal access as the same million dollars spent wiring up lower density areas yields far few connections.

Share bandwidth on I2P? It's like TOR, but with no exit nodes. Many services run on it: IRC, bittorent, anonymous limewire clone, anonymous websites, etc.

Your function would be similar to a TOR relay and you could do that too of course.

Maybe even hosting a Jitsi or Matrix server could be interesting.

Run full blockchain nodes, a bittorrent client, Tor relay (not exit node, so less door knocks)...

Share libgen, sci-hub and run a storj node.

Libgen, personally. I've used it so much, may as well pay it forward.

For those, like me, unfamiliar, Wikipedia says: Library Genesis (Libgen) is a file-sharing website for scholarly journal articles, academic and general-interest books, images, comics, and magazines.

wikipedia unlike google always has the current Library Genesis URL https://libgen.is/

This is funny but looks somewhat useful.

https://news.ycombinator.com/item?id=22790425

Maybe not using it? At that price, you aren't getting a dedicated 10gbps line to anywhere else than your ISP (and still...), afterward it's shared by everyone else. Using that bandwidth fully is not only going to make it not worthwhile to offer this service to people, it's also going to impact the service of others. Use it for what you would be normally using it, don't make up some usage for it, just to use it.

I have a hard time finding uses for my 1 gbps FIOS line. Which is surprising considering the bandwidth I had with my first modem (which is also my username).

> I like the idea of running a Tor exit node, but worry about the liability of that.

Run a Tor relay instead.

> But I don't have machine space at my house, so I'm limited in the number of servers I can put in place.

Also you probably want them to be noiseless and with low energy usage. For example: 10x Raspberry Pi 4 cluster could make use of 10 gigabit and sounds like a fun project.

Isn't it quite difficult to get software to run on a rasp cluster?

I cannot help but ask - So how much does this cost? You mentioned several different plans. Of course I will not get that here, but I will enjoy living vicariously through you and your awesome internet speed. I can hear myself now at work: "I know a guy who gets 10G! And for only $100 a month" Wait for the ohs and ahs.

Sorry to break it to you, but it's $300/mo for 10gig. :-)

That's a steal here in Germany. My company is paying ~500€/mo for 35/35 Mbps...

No, it's 2020. Everyone in germany has 1Gbit. They promised.

My company in the UK was paying the same as you. We asked how much it would cost to get more. We are getting 300M for about 200. When asked why they used to charge us so much, when more cost less. 'If you are too lazy to look up a better price each year, they are too lazy to offer it'. I would engage your service provider, and see if they have a lower price for more.

So I just need 10 neighbors and charge $30 each. Yeah 10 that way I am free Bwah ah.

Yeah that is a lot for home use. I would not mind a little faster than what I have, 50M, you know maybe 100M or 200M. Like you, no idea what I would do with 10G,

Tons of options

Game servers Voice servers IRC Mirrors for software or datasets Custom download to usb service for those in rural areas.

USB? Interesting idea... You've heard of dead drop USB? Perhaps I could add something to our Little Public Library where you could insert a USB stick with a list of URLs, and it would download them to the USB stick...

https://en.wikipedia.org/wiki/USB_dead_drop

Are you connected with the Municipal Broadband service in anyway (I don't mean like technologically, I mean relationship wise)? Are they asking you do come up with some services they can offer customers? If so how about setting up Pi-Hole, then the Municipal Broadband customers can use that as their DNS.

Not connected, though if I were to come up with compelling services, or there were questions about my intentions, I know people involved. The Mayor also used to hire the company I ran to provide technical consulting, so that would probably help.

Well, I guess your first step should be to redo your LAN to actually be able to make use of those 10gig :)

Sure, but I'm not gonna upgrade my LAN unless I have something to put on it. If I have an attractive application, setting up a 10gig switch, router, and pulling some cat7 is the easy part. :-)

I would seriously consider hosting my own startup ideas and side projects at home on a personal server if I had a symmetric 10Gig service. An old server off of Ebay with an SSD raid-1 array is reasonably unlikely to fail, and much cheaper than a VPS.

A decade ago when I had my own Ultima Online server, I would have asked you to host for me!

Go install www.zerotier.com and create a community LAN, then share each others' stuff?

The best reason for this level of service is: You get to stop caring. It's fast enough that everyone you invite over can do _everything_ on hardline and they won't impact each other.

As others have mentioned you could also actually run (personal) servers at home as well.

* Games, with higher bandwidth (audio, video conferencing)

* Live auto-video conferencing the easy P2P way.

* Have a media library of your own and access it while out. This can include private video memories, not just photos. However you could also store and forward images of legitimately purchased HD content* (Assuming you support the private use format shifting doctrines).

* If you're teaching a class from home that video-call could be self hosted, rather than routed through a data center half a country away.

While all this is true, you hit the "not caring" stage well before 10gig. I have a reliable 100 Mbps connection at the moment and do almost everything mentioned above (simultaneously) with zero disruption. Almost every internet problem people have is due to latency, jitter and general unreliability rather than bandwidth.

Depends. A AAA game like Red Dead Redemption 2 can take hours to download with a 100 Mbps connection.

You are going to be throttled by the CDN in any case well before reaching gigabit download speeds.

You can't really download games from PSN at anywhere close to that speed anyway. I have a hardwired gigabit connection to my PS4 and I'm lucky to hit 100Mb download. It's a similar story with Steam, when the stars align it might hit 500Mb for a minute right before finishing, but it usually hovers around 100-200Mb too. It's also much faster than the speeds of rotational hard drives, so unless you keep HD video on SSDs, it's no good for that either.

Beyond speedtests, the only way I can realistically saturate my internet connection is through S3.

I'd have hard time using FastE (100 Mbps); 10Gbps seems like overkill.

Tor node? Seedbox? I'd be hesitant to have either of those at my house or anything with my name on it, though.

> I'd have hard time using FastE (100 Mbps);

You must not get out much :)

Run OpenShift and invite friends to use it for hosting.

You can also contact the-eye.eu who host, among others, a curated list of covid articles. Helping make knowledge available is an interesting cause.

"I like the idea of running a Tor exit node"

Do it. It's the best thing you can do. Thank you in advance.

You can also run a Lightning Network node!

you could run plex and give me access.

I have tried to get Plex up and running with my ~20 TB collection but its library management doesn’t work for me. Fails to load files, categorization is horrible and many other problems. Kodi (XBMC) has perfect “point it at a folder and don’t worry about it” functionality. It just doesn’t have the remote capabilities I’ve found as good as Plex. What am I doing wrong with Plex?

If you don't have your files named correctly, it won't work right. https://support.plex.tv/articles/naming-and-organizing-your-...

It can sometimes be forgiving, but other times it won't find it unless you name the file exactly correct. With a library of that size, you might have to leave it for a day or two to do the initial scrape, as it sometimes does a couple passes to find everything (why, I don't know).

Start by running it with Sonarr and Radarr so that your files are named correctly, then Plex "just works" without any tweaking. Both work well for library file management and renaming, they're not only useful for pirates.

12TB collection here. Throw your media at https://www.tinymediamanager.org/ - then Plex, Emby, whatever will pick up seasons easily.

Actually this isn't as stupid as it sounds. For home use, that's the only thing I can really think of wanting a lot of egress bandwidth. I run a PMS on my connection (400/25), and it's nice that I can let friends or family members watch a movie from it instead of needing to lending them the DVD (and keeping track who has which DVD, getting them back, etc.). My bandwidth means I'm limited to two streams (but then again, I don't buy many movies, I'm more a music person).

Symmetric 10Gb/s are still total overkill, symmetric 1Gb/s would already be pretty good for that usage.

But it's specialized: You need at least some knowledge for the setup and on how to get your BR and DVDs into the server - and then of course a collection of DVD & BR to start with. And, in the age of streaming, this means the person running the PMS is either pretty retro, a home theatre nerd or a pirate. And you probably don't want to taint the offering with "it's mostly useful for piratez" ;-)

Maybe Cloud and heat, use servers as your central house heating system instead of oil, gas,...

Son, let me tell you about the early days of Bitcoin... I left the windows open that winter.

Host a couple of community Vainglory servers to ressurrect the best touch-first MOBA that has is essentially in ICU due to poor management by SEMC / Rogue

reddit.com/r/vainglorygame

Where do you live and how can I get one?

Fort Collins, Colorado. 6+ months into a multi-year build-out. I think the anti-muni-broadband advertisement that the local telco and cable operators took out made HN front page a few years ago...

Who's your provider? Thanks.

> Municipal

So most likely their city/county.

Considering running Tor servers!

Run speed tests.

how have you got that deal?

Reply on news.ycombinator.com