Hacker News Reader

Comment by Nick-W | Hacker News Reader

Hey! I did this too - CenturyLink wanted an insane amount of money to bring fiber to our place, now we service hundreds and we're growing into a major contender in Boulder County - https://ayva.network

View on HN

Just a quick heads up that the homepage video is ~24MB over the wire, even on a phone. That might actually be a challenge if someone's WiFi is down and they're trying to get support over cellular.

(Huge kudos for this project in general)

Thanks! It's actually much less for the bandwidth-constrained, I use adaptive coding. If you have the bandwidth though...

That said, I know our page isn't particularly lightweight anyway, I've been pretty focused on expansion efforts and haven't had much time to update & work on the site.

This page was 9 seconds of white screen before the entire thing loaded at once. I'm on Starlink. Hopefully you get a chance to correct this in the future, as I'm really supportive of projects like yours, but if the page was linked from a top-5 article or something I would have hit the back button already.

Totally - we're not advertising yet and usually only dealing with very local traffic at the moment, didn't expect to get tens of thousands of hits from linking to it from a hackernews comment!

The backend is an azure appservice running on a single low-spec worker at the moment, wasn't planning on enabling scaling before doing a hard-launch but here we are.

I am sorry, I always forget about the 'HN effect'!

It's a great marketing strategy. People without broadband access will truly feel it when they visit.

First Contentful Paint at 12.0 seconds here, and I'm on fiber.

On my phone on Comcast cable it was like 3 seconds for me.

Also was a few seconds of white screen for me. And I'm on fiber

Yes the white page with nothing else mad me think it was broken.

It's currently running on a single azure appworker, you might've hit it during a service autoheal/reset. There's a fair bit of overhead for every connection since it's designed for our subscribers to use and get realtime stats on their connection, was not expecting to get tens of thousands of hits by linking to it in a hacker news comment.

21 seconds from click to video in this old neighborhood (50 Mbps) in Europe.

(not OP)

This nerdsniped me.

10 minutes later, and TIL: MPD is some sort of streamed-MP4 format; dash-mpd-cli is a xplatform Rust utility binary that can download this to an MP4, just given the MPD URL in dev tools.

However I keep getting 1.5 MB and 500 KB for the two videos, no matter window width. Chrome on macOS arm64, 16" MBP.

I'm curious what your environment is, if you don't mind sharing

(also, trivia for audience: last week I saw a tweet that palantir.com was doing over 100 MB worth of videos, and of course, A) they are B) they're poorly compressed, as much as 10x the bitrate they need to be.)

Frontend is full blazor w/hybrid WASM, almost zero JS, all C#. Browser DOM is controlled by the app service in realtime, I plan on using this as a basis for our subscribers to be able to do live traffic & link stats monitoring, among other things.

Similar specs here. Here's a full load in a Chrome guest window with resolution constrained to an iPhone SE: https://imgur.com/a/xRjKWNb

In the left panel, at the end of the video you'll see two important numbers: 28.5MB transferred, 40.7MB resources. "Transferred" is the (compressed) size of everything downloaded over the wire, 40.7MB is the ultimate (uncompressed) size of those.

I don't show it in that video but you can filter by "initiator" to see that the video files are the lion's share of this.

Well... To be fairrrrrrrrrrr

If you used Ayva's fiber internet that video would download instantly =D

"Works for me, couldn't replicate". My place has 10gbps/10gbps service through my network but this is a quick test over 6ghz wifi: https://www.speedtest.net/result/17623249189

I have metronet and it load in a seconds for me, tbf, I came across this thread post HNhug of death, so...

You are doing God's work. Thank you. I wish more people cared about wasteful bandwidth usage.

It's not really the size that matters in this case, because the video is loaded after the page is completed, and in theory, ought not be slowing down the page if at all (my cursory examining of the network tab gives me a sort of confirmation).

However, what is indeed slow is the initial load, and the lack of CDN for their static assets (css etc). When the HN effect started eating their resources, these static assets are what hurt their load time the most.

Page doesn’t even load for me.

Yeah me neither from Australia. Hug of death?

Probably hit it while the appservice was healing, amazed at the amount of traffic it got from being just a comment on hackernews.

I wonder if there's value in a method by which you slow down traffic by region, while keeping/saving resources alive for faster traffic in a different region (that you actually care about). Surge traffic such as HN frontpage will cause massive amounts of traffic to come in from all over the world, but because this service is only relevant to locals, it would suck to have hurt their speed/experience.

I can scale up our worker count as needed, but we keep it at a minimum since our Azure bills are already insane. Fun tidbit, there's a little counter on the bottom right which tracks live app circuits on the site, we were cracking thousands yesterday, normally it's ~5-10 at most. Did NOT anticipate having this kind of attention from a comment, and we were out climbing in Boulder Canyon when the bulk of the traffic hit the site.

Why don't you host it yourself if the kids is so small on a normal day?

Xfinity's website is like that, it barely loads even using their service. I've had it time out several times, and had to start over, just trying to help my mom pay her bill.

I’m curious what the economics are these days - I cofounded a small town ISP in the mid-90’s (think dial-up) and the largest monthly costs was the 24 commercial phone lines. Even though it was a loss, it was a relief to eventually sell to the local phone company 2 years later.

Bad. Our average cost to install service tends to be around $800-$1200, and that's not including overhead of setting up new towers/host sites. Our average cost to deliver service right now is about $80/mo, but the good news is that we're in a solid position to scale up to thousands of subscribers with minimal increase in overhead costs. We do it though because it makes a difference - plus I get random cookies & care packages from people, which is nice.

> I get random cookies & care packages from people

I dunno, (social) economics seem pretty sweet to me.

Yeah, it also feels great - we know everyone now, we provide free service to all of our local fire departments and organizations like the Rocky Mountain Rescue Group, and people rave about us all over social media. I have hundreds of install requests though and only 8 part-time guys though, so it's tough to keep up with the demand and I feel terrible that it sometimes takes us absolutely forever to get someone connected up.

Give your subscribers the option to self install. Give them the tower to point their antenna toward and how to dial in the direction for max gain.

I suspect a many would jump at the opportunity as rural folks tend to be quite resourceful.

I've done this for churches for campus point to point. Don't have enough installers, but about 30% have volunteers that got it done or mostly done. Some we just did the Cat-5 terminations for them.

We did something like that starting up our ISP with fiber installations. The whole village helped digging the trenches, pulling the fiber. The welding we got a little lesson on, and then had a few people doing the fiber welding all over the village with borrowed equipment. This was about 25 years ago, webserver was running on a then ancient 8086 (or 286?) running Linux :-)

Couldn’t have been anything older than a 386 as that was minimum spec even for Linux 1.0.

Seems I have the computers mixed up in my memory, it was definitely a IBM PS/2 Model 70 486. But our computer club were running some 8086 with a version of Linux before that as a router, think you had to compile the core yourself to get it working. Not to mention getting two different network cards running... My memory is fuzzy, wasn't me doing it, just watched the guy do it and bought him lunch :-)

Honestly this is something I need to promote more. We've had a few people self-install which is amazing, and it turns a 4-8 hour install job into a 30 minute alignment one. It's something we can only offer to those who have an unambiguously clear line of sight to one of our access points though, and since we mostly operate in the mountains, trees can often make things very difficult.

> In this sparsely populated rural area, "I have at least two homes where I have to build a half-mile to get to one house," Mauch said, noting that it will cost "over $30,000 for each of those homes to get served."

Does spending 30k per household connected make any sense?

No - and this is our argument when applying for funds, I can deliver 2.5gbps (symmetric speeds) to someone for < $1.5k up to 15km away, and I have a roadmap to eventually hit 10gbps and beyond. Unfortunately we're not "fiber" though, so our projects are automatically deprioritized, even if we're like 5% of the cost.

What kind of transport are you using to hit 2.5Gbps without fibre at that distance?

802.11ad/ay on unlicensed 60ghz, our most economical option is to deploy Ubiquiti Wave Pros. We see real-world 2gbps+ speeds at 15km distances. We have Wave Pro, XG, and XR radios all throughout the network for multigig links, and 95% of our non-business installs are Wave LRs and Nanos. We can do up to 33gbps symmetric on 70ghz licensed bands on a single radio, and I have a number of 10gbps radios, but they're not cheap.

Sounds very optimistic to consistently deliver 2.5Gbps to that size of cell, your backhaul has to be huge.

It's not hard at all, we multipath all traffic across our mesh, we have multiple 10gbps fiber uplinks, and we're bringing up our first 100gbps circuit later this year. Most sites have plenty of capacity and we always try to ensure we have 2-4 paths out of every site. Unlicensed 60ghz is super easy to reuse, we drop 20dB once we're more than 0.2deg off the site, and since we operate in rural mountainous areas, our spectrum at each host location ends up being sparkling clean. Our APs have a 30deg azi/cell width, and we can do up to ~30 attachments on each. Best of all, Wave gear has a backup radio - if I need to take down/upgrade an AP, nobody ever notices, the stations all roam to a nearby AP on a 5ghz backup, and can usually still get around 300-600mbps or so.

How does that perform in heavy rain or snow? I've always been told that when dealing with mmWave, attenuation gets pretty bad when dealing with adverse climate conditions, and Boulder is surely no stranger to snow.

Great - our radios also have a built-in 5ghz backup that is seamless to switch to and causes no interruptions to gaming sessions or anything, but honestly we only see it in use under two scenarios: The radio is literally knocked off the mount and pointing at the ground (and is still getting ~100mbps somehow), or for our long-range links (5km-20km) during really major storms.

The backup is great and shockingly resilient, nobody ever notices when it's in use and it'll usually still get us 300-600mbps of throughput on average. I'd say we typically see about 5-10% of the network revert to the backup radio during a major blizzard/hailstorm, otherwise everything seems to do quite well with normal snow & rain.

I'll understand if you don't want to answer this or legally can't answer it. What does a 100gbps circuit cost a month? How about the 10gbps uplinks also?

We can't, we're under NDAs for all of that stuff :(

Roughly single-digit cents per megabit

> Does spending 30k per household connected make any sense?

Over the lifetime of the structure (which can be upwards of 100 years)? Probably, just like the road, water, sewer, and electrical service to it. It’s another utility.

Water and sewer is probably well and a septic tank/leach field. The electrical service is probably at least that much though.

If the $30k comes from tax dollars that the government earmarked for rural fiber, I guess so?

If it was already affordable to connect those houses there wouldn't have been a need for federal funding (not to say that I think tax dollars should be spent that way or that the program was run well).

Not wanting to make it political, but what are the chances those federal funds are flowing anymore?

Right now ISPs are very concerned about the future of money they've already been promised and pessimistic about the future of such programs. It's possible that this could result in an increased unwillingness to expand into less profitable areas or even to maintain services there.

That said, some ISPs have been known to collect hundreds of billions in taxpayer money on the promise of providing service only to just pocket that money without providing it anyway so for consumers, the real world impact of ISPs losing that money isn't so clear.

Considering that we gave AT&T nearly $400 billion in the 90s to connect everyone in the US to fiber, I certainly hope they're no longer flowing. It was a giant grift. It's taken local fiber ISPs and WISPs (who seem to have been gobbled up immediately) and Starlink to make anything happen.

1: http://irregulators.org/wp-content/uploads/2017/05/BookofBro...

2: A more recent example from 2024: https://www.washingtonpolicy.org/publications/detail/the-42-...

I’m sure Starlink will consume the oxygen. Good news is they can deliver quickly. Bad news is that strategically it’s a fragile and higher risk offering for critical infrastructure.

My area has a similar density and a co-op did it for $80 a month 1 gig fiber to the home. We couldn't even get DSL, and the local telecomms even gathered funds locally earlier to build it, but then just kept the money. I am extremely skeptical of claims that rural areas can't get fiber, fiber is even cheaper to hang than copper is, and yet every rural area has telephone lines and power that were put up many decades ago with even less equipment available.

>local telecomms even gathered funds locally earlier to build it

Nationwide telecomms did that with funds for fiber to the premises as well. I have no idea why these telecomms can take the money and just refuse to provide the services.

It’s really essential to fund or make it feasible for companies to fund broadband. Universal broadband should be a priority.

Rural programs are really successful, until the money runs out. Cities get screwed though — I served on a commission with my city on this topic and the only feasible competition at scale, Verizon, refused to engage unless the city allowed them to pick individual blocks to deploy, which means no servitude for underrepresented people.

Fixed wireless (11/24/60 GHz point-to-point antennas) is pretty good, and the hardware would be a few thousand, but I wonder if there's an issue with the funding where it needs to be actual fiber for some reason.

My first job out of college was at a fixed wireless ISP which was started by a guy with a story just like the OP.

This was 20 years ago now, but the service was very reliable using Motorola radios. Relatively low bandwidth (4-6mbps, not bad for the day) but you could on a good day do that at a few miles out.

Oh man! Wish I had found out about this 3 years ago. I am graduating in May, and I’ve had a terrible experience with Xfinity trying to self-host. CenturyLink doesn’t even service my apartment complex.

p.s self-plug: for our senior year capstone we are working on a secure/private home router firmware. Since you are in this space (tangentially) and local, I would love to chat with you

Anytime! Give me a call or send me a message, we're out at a crag now in Boulder Canyon taking advantage of the weather, so leave me a message if I don't answer and I'll get back to you asap.

Please don’t use C !

“Fully encrypted network with strict privacy policies”

God I wish that was me. Xfinity has a raised middle finger where the privacy policy should go.

I treat our subscribers how I want to be treated. I'm not a business person, I'm an engineer, I care about my privacy, and I love the EFF. Any company who wants to "buy our data" is going to get an emphatic middle finger, and our logging infrastructure is selective and highly amnesic where it needs to be. I mostly log ICMP & network control traffic (OSPF, BGP, etc) because that's the kind of data I do care about which is valuable in tracking down issues or service incidents. Also I always get prior permission and a very specific ~5-15 min time window from someone before we dump/analyze real traffic for a problem they're experiencing.

Not sure it was that insane. The author quotes a cost of over $30,000 to build a half-mile drop. I find that an insane amount of money that a government would pay to connect just one subscriber.

At $80/month breakeven is 30 years out. Assuming no recurring costs. But there's probably positive externalities like increased property values, businesses started, remote work bringing in dollars for the local economy. It could even have positive ROI for the state if it increased tax revenue, or at least help defray the cost.

It's not crazy, but it's longer time horizons than a business would ever care about.

>The author quotes a cost of over $30,000 to build a half-mile drop. I find that an insane amount of money that a government would pay to connect just one subscriber.

That's not much different than what it costs to connect them to power grid or the water and sewage system.

What metric is “insanity” and why do you find that insane?

How did you get the capital and find the time to do this? Is it your full-time gig? I've always fantasized about doing this in my mountain community but it seems spooky

I self-funded. It was about $500k and years of time to get things really going, but we have a dream greenfield deployment with a full-mesh network, our own ASNs & IP resources (couple of /21s & IPv6), and some super high-end network edges that support full multipathing with tons of redundancy throughout the network. It was a labor of love, I'm unpaid, and I'll never see that 500k back, but that's ok. I now have 8 employees and we're growing, fast.

You are my hero. If you have a technical write up anywhere of your story, I would love to read it.

Would love to. Honestly nobody has really shown any kind of technical interest in our network and we've been operating under the radar now for a few years. Now that I have some employees to help out, I might be ok with us becoming a little more known. We have had something between 400-500 install requests purely by word of mouth and without ever advertising anywhere, so I'm a little nervous for us to have a hard-launch, especially since we can technically serve a pretty good chunk of the city of Boulder (which has a population of 100k+).

I legit wish I had known about you a few years ago when writing my thesis. It was about community run broadband internet. I was trying to identify repeatable models for communities who wanted to run their own ISPs to use. This would have been so helpful!

Side note, truly inspirational an something I would love to do in my little village in Ohio.

I was wondering if you wouldn't mind talking more about your thesis? I am interested in rural internet as well for Native American reservations and both opie's and your comments are inspiring and something I definitely want to read.

Drop flyers on campus/doorsteps when students move in, good way to get 1 year contracts if those are worth it for you (might not be). Comcast just walks door to door and hands kids routers on the spot.

We'll probably do a "hard-launch" soon where we'll do a bit of advertising and open up installs within the city, instead of just serving homes up in the mountains. Good news is we don't need contracts, we have exactly 0 turnover and no real competition, it's not hard to beat Starlink and everyone universally hates CenturyLink/Comcast here.

I'm semi rural but 4.1-4.3 miles from 3 towns. I have natural gas and cable internet, but a well and septic.

I was very pleased (before making an offer on the house) to find out about the cable internet as I have worked remote for 11 years. But the ISP is Spectrum, which has historically been a very shady business (like most Cable companies, ISP's).

I also run a Mofi router with a SIM card as a back up network.

Year 1: $52/mo Year 2: $74/mo Year 3: $98/mo

I have made no changes to my account.

This is why we need choices for ISP's.

That's a beautiful thing.

https://startyourownisp.com/

Useful threads:

https://hn.algolia.com/?q=startyourownisp.com

This is yak shaving taken to factorio extremes, now shaving everyone's yak.

how can I do this for my city?

Do you have a few hundred thousand dollars lying around?

This. It's insanely expensive, I'm unpaid and I've sunk about half a million into the network so far and 3+ years of work. We now have enough revenue to keep the lights on if I no longer support the company, but just barely so, and assumes zero growth and minimal support hours. I did this because it genuinely helps our community and the impact is directly visible and notable, but the economics are basically impossible, especially as a startup.

Do you operate it as a nonprofit? Seems like you could be at least seeing some tax benefits probably

Any plans to expand into JeffCo?

Also, this is a highly highly resource-dependent website. Consider a scale back. It'd be a funny tongue-in-cheek thing if you made it super encumbered and say "Our customers can load this page just fine!", but it's counter-intuitive for everyone else haha

I am in France so not exactly in your coverage but I wanted to not that the comparison card (and the coverage one) do not work correctly.

The first information is fine (say, speed) but when I switch to latency the graph does not change (and BTW it's not readable on mobile)

Same for the coverage

This is really cool!

Does long-range ubiquiti just work because you’re in the mountains?

I’m down in much-flatter Columbia South Carolina. Would a similar setup even be physically feasible?

We had a company burying fiber, but their installers almost blew up a gas line during 5 o’clock traffic and it really pissed city council off.

This is what I love in HN.

Someone, somewhere says that they built something for a local community and suddenly Joe from Sydney and Marie from Bordeaux are on the site, discussing its tech stack and comparing the pricing in Wakanda.

Great site.

This page is so large it causes the browser on my phone to crash the tab.

Page never loaded for me on fiber.

I see all your replies to people; have you considered that running a video on a website is just completely unnecessary?

Video background and scroll to animate stuff was cool...in 2013. I think people appreciate a faster loading page with up front information these days.

But you do you I guess! :)

Hell yeah dude.

Reply on news.ycombinator.com