Hacker News Reader

Comment by walrus01 | Hacker News Reader

I would caution against just jumping into this and trying to be a real ISP if your experience is in software development, sysadmin stuff, or something kind of tech industry adjacent but you've never worked for an ISP in a NOC environment, as a junior network engineer or similar for somebody else's medium to large sized ISP. There's a lot of missteps that can be made without experience.

In general I would recommend thinking of starting a small ISP as a depth and breadth of knowledge base you need to develop, similar to how an apprentice electrician is expected to work under some more senior/experienced persons in their field for a number of years before being able to operate independently.

I do not want to discourage anyone but I've seen a myriad of 'oh my god why did they do that' small WISPs with anywhere from 50 to 1500 customers. Going down the wrong network architecture/topology/business plan path can be very costly later on if you're 100% learning as you go.

Overall I think a site like this is a great idea, but the sheer scope of things that would need to be written down in a wiki-like format for everything to start your own ISP, and what systems/subsystems/technologies you will want to be familiar with, could fill thousands of pages if printed in book form. It's a gargantuan task to make a website that has everything needed to start an ISP from scratch, assuming the reader hasn't been involved in somebody else's ISP operations first.

View on HN

Author here. I see your posts often and have a lot of respect for you as someone who obviously has a deeper knowledge of networking than I probably every will. That being said I don't really agree that a person necessarily needs a ton a networking or even any technical background to start a WISP, any more than they necessarily need a ton of experience as a cook to start a restaurant. They definitely need employees or advisors who have that experience, but like you allude to here probably no one can have all of the knowledge to so it themselves upfront.

Also I've also seen a lot of WISPs with a myriad of problems as they grow from a few dozen to a few hundred customers, many of them are able to make that transition, fix their problems, and either grow or be acquired. This seems basically the same as any industry - if everyone waited until they knew exactly what they're doing no one would start anything.

All of this said, starting a WISP isn't for everyone. I talk to a lot of people who are in the early stages of starting an ISP, and often my best, honest advice is that it's not a great idea in their situation.

Does your site have a section that covers the legal steps a person should follow to protect themselves and their small business? i.e. incorporating their business, business insurance specific to a WISP, processes to follow to ensure legal compliance, what data to log and not log, log retention policies, consumer disclosure, frameworks to put in place for law enforcement requests, agreements to have in place for the upstream fiber provider, etc...

There's a little bit of that in there. Surprisingly, in the U.S. at least, there's not much of a gap between being a legal business entity (LLC, corp) and being able to be an ISP. You don't have to log much of anything (unless you're doing voice services as well). You do have to be able to intercept customer traffic without going on site physically in order to comply with law enforcement requests.

But as someone who is not a lawyer and has no business giving legal advice the best I can do is suggest that you talk to a real lawyer.

I think it might be useful to have a section that includes anecdotal experiences of existing WISP's of what actions they had to take in their location to cover themselves, what legal challenges they ran into and any caveats or scenarios to be aware of. Obviously with the disclaimer that it is not legal advise and they should retain their own legal council. It might give folks a better picture of what to expect before they consider the investment.

For what its worth, I really like the idea of people implementing some options for those that might only have access to one ISP. I was locked into Comcast for the longest time and would have loved to see more options available.

Legal requirements can be funky. Back in ~2007 or so I considered doing my own WISP just to get any reliable internet that wasn't making 300 baud modem look nice (put enough latency on telnet and it will look like that).

Turned out that if we wanted to cross a community (smallest administrative region in Poland), to be fully legal we might have had to submit war and disaster preparation plans to Ministry of Defense. For cooperation with military and other services, just in case.

I am not an expert but I've seen this book recommended: Small Time Operator: How to Start Your Own Business, Keep Your Books, Pay Your Taxes, and Stay Out of Trouble

https://www.amazon.com/Small-Time-Operator-Business-Trouble/...

If somebody has the opportunity to work in a role that has 'network engineer' in the title for somebody else's mid sized ISP first, I would encourage them to think of that like free paid training for a couple of years before trying it on their own.

Of course totally dependent on their level of pre-existing knowledge, level of risk with the capital they intend to spend on a startup wisp, financial partners, and what scale/scope of service area they want to do it in. Varies widely with location. And what competition (if any, other than hughesnet/viasat) exists.

I've seen people do some pretty cool non-profit community based WISP things for under 50 houses, if built right it can be quite trouble free.

The tools and equipment to do really basic 100% fiber based GPON network (or active ethernet) are also a lot less costly than they used to be... The things some guys are doing on really shoestring budgets with FTTH aerial drop cables across roofs and low cost GPON things in cities like Dhaka, Bangladesh show that it doesn't have to be super expensive. In a USA/Canada context, right of way and poles and routes are an issue (even after you're got your ROW, go price how much bucket truck/aerial fiber contractors cost, if you don't do it all in house)...

I read somewhere that 40% of restaurants started by people with no previous industry experience go belly up within 5 years? Something like that. I imagine one could easily spend $400,000 setting up a restaurant and working 14 hour days and ultimately not be successful, so doing something like a startup WISP on one's own similar amount of funds (and loans, private party investors) I would think of something similar to that. If one has the funds and the appetite for the risk and is really determined to do it, then go for it!

> I read somewhere that 40% of restaurants started by people with no previous industry experience go belly up within 5 years? Something like that. I imagine one could easily spend $400,000 setting up a restaurant and working 14 hour days and ultimately not be successful

Absolutely agree with this and everything you said here. I think the only thing I see differently is that while someone should have some relevant experience before starting an ISP (or any business) is that it doesn't necessarily have to be networking. For example, some of the biggest challenges I've seen WISPs face as they try to grow are: obtaining lease agreements for wireless relay sites, hiring and training technicians in large numbers, and in logistics around keeping stuff in stock that they need to grow. Someone with experience from a different industry in some of those areas might do just as well as someone with networking experience.

EDIT TO ADD: One of things I find kind of difficult to communicate with people who are interested in it is that it's just another business, and doesn't really have any better or worse odds of succeeding beyond a few years and making money than any other business, including restaurants. People sometimes seem surprised that you can make money at this, but, of course you can! You can also make money running a taco truck or building custom furniture. But you can also lose a lot of money and time.

I also totally agree that a lot of the challenges are not exactly 'network engineer' related, trying to run a small WISP has so many other things that aren't what you would find in CCNA/CCIE study materials or similar. One can feel much more confident in what they're doing once they have a real firm grasp of BGP, OSPF, various types of MPLS, metro ethernet stuff, optical networking, ptp microwave, linux sysadmin stuff for operational support systems and so on, but that's only one piece of the puzzle of many business process related things to cover.

This is not imperative to do in-house. My company (*shameless plug for Astute Internet) has expertise in those areas you describe, and will provide and/or support the edge network infrastructure for small ISP's in exchange for selling them some of those services (we're a mini-carrier and value added reseller) and charging a nominal retainer to be an available path of escalation 24/7. Due to our purchasing volume and industry knowledge, we are able to sell these services for the same or less than what our customers would be able to negotiate on their own, while getting a lot of support from us in the process.

Although this type of knowledge and expertise is rather specialized, we are able to focus on these specific areas without getting involved in the intricacies of the downstream side of our clients' networks, thus avoiding the need for us to have too much client specific or institutional knowledge. This has worked out very well with the handful of eyeball networks we're currently working with in the Pacific Northwest, and we intend to more actively extend this line of business down the coast in the near future.

A big difference is that you’re selling a service only and a service that is really difficult to differentiate yourself in. A big player could swoop in and obliterate you in a second.

Yes and no. Big players will react and offer special deals to people in an area to keep them and lock them in for another year or more if they perceive your ISP as a threat, but if you offer reliable service and good speed and good support, you can eat their lunch.

The problem is that to be competitive you might be looking at years to break even on investment depending on what you offer and where, and how you deliver it. Running fiber on poles to deliver 1gig or 10gig? The buildout is expensive, you have to target dense areas or buildings with lots of units, and even then it's a lot of up-front money and it will be a while before you're out of the red even with a good takeup rate.

How do you mean? Outside most major cities, ISPs tend to monopolize the playing field and there is great demand for cheaper or more personalized service. Having good customer relations would be a major differentiator, and it's not that hard.

Unless you meant obliteration by lobbying, lawsuits or something similar. There's little hope if your competition goes that route.

You can’t compete with large ISPs on price. They can easily undercut your price for a few years until you’re gone. And a lot of people don’t really chose their provider for ‘good customer service’, they chose fast and cheap.

If you’ve set up in an area that a big ISP suddenly sees as profit potential you’re toast.

> I read somewhere that 40% of restaurants started by people with no previous industry experience go belly up within 5 years?

Experience doesn't guarantee success either. I had a neighbor who was running one of the most successful restaurants at the Mall of America who decided he wanted to do his own thing. He bought a local restaurant and completely remodeled it, and it was great. But it never did the kind of business he expected and he had to close it after 6 months and declare bankruptcy. He told me that he put his house up as collateral for a business loan, and the bank told him he'd need to come up with $700K just to be able to continue making mortgage payments - he was forced to move too.

Sounds like his mistake was likely the costly remodel rolled into the business loan that he couldn’t afford.

For those out there wanting to start restaurants: start small! It’s a heck of a lot easier to go from 10-20 tables to buying the lease out of the person next door and expanding.

If you’re small and full because your busy, then you’ll also generate buzz for being “exclusive” and when you expand, people will still flock there, even though there’s plenty of room.

Source: worked at restaurant that did exactly this. Pizza and decent Italian food. Place is hopping all the time and considered the go-to date night and anniversary spot because it’s hip and trendy. They’ve been open for almost 10 years now.

Oh the place definitely needed remodeling. It had been a restaurant for many years and didn't see an ounce of love in that time. But I'm also certain he went overboard and bought the best of everything. Strangely enough it seemed twice as big, even though he didn't add any square feet.

The location isn't cursed, it was taken over by another restaurant that has been open a few years now.

Is AON instead of PON feasible to do for a small ISP or too costly?

You wrote: <<The things some guys are doing on really shoestring budgets with FTTH aerial drop cables across roofs and low cost GPON things in cities like Dhaka, Bangladesh show that it doesn't have to be super expensive.>>

This sounds very, very specific. Do you have a link you can share about it?

What about the customers though? My family used to be chained to a WISP, and we paid $100/month for internet that made dial-up look like a viable competitor. Not to mention, downtime was atrocious (1/10th of the time the service would just be out).

I think the bigger argument for not starting a WISP is that nobody wants to be a guinea pig in your amateur networking project.

(Author here) WISPs are more like local restaurants than big chains. Some local restaurants are terrible. At least with a big chain you know what you're going to get.

That being said, the WISP you had probably just didn't know what they're doing. Which is true of a lot of WISPs (and small restaurant owners!) And is actually in large part what motivated me to write this guide! I'd like to believe that if they followed the guide, and/or gave me a call to help out with the parts they're struggling with, we could fix it. I'm personally involved in running several WISPs right now, and without exception we are providing 100mbps+ service, <20ms latency, and at least 99.9% uptime measured to the customer.

FWIW, and not to undersell the challenges, but I think you're extrapolating too much from a single example (and possibly an old one). The started-tiny-and-local WISP around here is $100 a month for 50/50 and it's been pretty rock solid. While I certainly haven't run a WISP myself, I have set up point to multipoint wireless networks for some local businesses, and with current equipment it's impressive how straight forward and relatively turnkey the basics of the physical link itself have become. It was no problem to have dozens of points over distances of 5-16 miles with solid 100-400 Mbps links. My understanding from far more senior people and reading about real WISPs is that there is a non-linear increase in difficulty/expense over greater distances, scaling up to hundreds/thousands of points, and handling things for multiple independent entities rather then what is effectively a big LAN even if sectioned up with VLANs. There's also completely non-technical challenges to be ready to deal with like compliance departments and legal/law enforcement interaction aspects of when a customer inevitably does something naughty. The path has been tread but it wasn't anything I've had to deal with.

Still, if I was still in an area with dial up speeds and Starlink wasn't a thing, helping jury rig up neighbors with a mini-WISP is a project I'd certainly consider contributing to and believe could be quite reliable and easily beat out DSL at least.

Also speaking of Starlink, I'll be curious if a side aspect of them would be driving phased arrays in general down in price significantly. That'd be a pretty interesting development for terrestrial PtP/PtMP imho, since a lot of the trickiness over long distances is precise aiming. If the antenna merely needed to be aimed with 10-20° and then could perfectly align with the other from there, including real time adjustments for some motion or interference, that'd be another pretty cool improvement. And for the time being at least Starlink has left an opening in that the connections are quite asymmetrical (ie., 200 down but only 30-40 up), which is challenging to work around given their physical and economic constraints. Whereas a WISP can be full symmetric right up to gigabit speeds, assuming their backbone supports it. That could let some remain competitive beyond just density limits.

Is Starlink such a big deal that it would materially affect phased array costs? Cell base stations are already ubiquitous. From my own experience with phased array systems, the big cost is usually in the massive FPGAs that are needed to process all the incoming data in parallel.

>Is Starlink such a big deal that it would materially affect phased array costs?

I guess I think the Starlink terminal is technically one of the coolest and least appreciated aspect of the whole system. Sure it's not a military AESA system but getting that level of phased array into a 100W all weather CPE with a $500 target price they plan to produce tens of millions of is something I don't think has ever been done before, and it definitely could blaze a trail in terms of economies of scale there. SpaceX's rockets and sats seem to get most of the press, and are certainly critical, but the terminal is also a big deal IMO.

I mean, think about it: they've built an end point that will do a point to point link of 500+km to base stations with relative velocities of kilometers per second and can user-imperceptibly swap between multiple ones in real time while maintaining a bandwidth of hundreds of megabits per second. For $500 (granted it's probably costing them at least double/triple that right now, but they aren't making tens of millions yet either). A much more cut down, cheaper version of that could still make a number of terrestrial PtP/PtMP links a great deal easier for random people to take a swing at and get even better results with then now.

Again, I'm certainly no expert at all. But I've kicked the tires on a little of this stuff a bit, recently got my first chance to work with second gen 60 GHz links for example and those feel like a certain amount of work to get just right over multi-km ranges. But if for another $100 suddenly they just handle it themselves as long as they're within 10-20° horizontal/vertical? It'd make some aspects of installs more accessible and cheaper which doesn't seem like a bad thing to look forward to?

OP is of course right that none of this addresses the many other aspects of doing networking.

The phased array wisp thing has been tried before and failed. People seem to think that wisp is somehow using different physics than 4/5G. In reality it's more like Joe's taxi vs Uber.

>The phased array wisp thing has been tried before and failed.

Examples? I don't know of any phased array PtP/PtMP system, at all, in anything like the form factor and pricing of Starlink or something like Ubiquiti's airMAX/airFiber/LTU series operating in the 5, 11, or 24GHz class spectrums. In fact your whole comment makes zero freaking sense, what do you mean "phased array wisp thing"? I don't see how it'd fundamentally be any different then all the very successful WISPs I'm familiar with, or for that matter the LR-LAN stuff that I've done myself. The advantage would be in that one of the trickier bits at longer ranges is getting alignment just right, labor or kit correcting for shifts, and tradeoffs in allocating regulatory max transmission power budget across end points. Skilled personnel can of course learn the alignment part reasonably and an important differentiator for a long time amongst different kit was the kind of tools it had to help with alignment. But it's not nothing and it can take a while, and then there is the rest.

A phased array system like Starlink's that could make its way into $150-500 CPE and hopefully <$5k basestation would be a nice evolution. Only need to get it roughly aimed in the right direction and then the system does the rest with no further human intervention, zero skill required. Basestation could extremely rapidly reallocate high power to various end points which matches consumer load patterns well rather then having to pick 30°/60/90/120/3x30/whatever sector antennas or 4'+ dedicated big boys for real distance.

I have no idea where this "4/5g physics" thing came into your head from. We're not talking physics we're talking economics and barrier to entry.

Pivoting your server or software stack is like changing a floor in a building.

Changing your network stack is like changing the foundations of the building.

Sure you can do it, but the bigger your building, the harder and more expensive it's going to be.

There's a lot of value in learning a lot of these basic mistakes first on someone else's dime.

Sure, you don't need to be a trained chef to start a restaurant, but if you dont know that undercooking pork is going to make your customers sick, you probably should be hiring someone else to do the cooking.

> Changing your network stack is like changing the foundations of the building.

Same thing with data storage. Platform changes are like open heart surgery on data centers.

The WISPs that are started by people with no/little background in NOC operations leave themselves to build, not only insecure, but inefficient networks rather quickly. I started my career on the network side and have worked for a number of ISPs (small and large). And while a WISP creator can make it - they can also just as easily create a bad product unknowingly.

Case in point... I have family that only had one option for Internet service. Long story short a utility owned a WISP and no longer wanted to run it (they ran it OK, but made a number of mistakes when choosing architecture and products to go to market with). One of their employees bought it out and started making worse choices. The owner clearly had never scaled a network nor did this person have any formal training in networks or the protocols he'd have to manage and monitor.

I've submitted no less than a dozen vulnerabilities within their network and have had to install a monitoring stack for said family members to show the owner of the WISP that during peak hours they were dropping a significant percentage of packets, latency was spiking and SLAs were not being remotely met. The owner has refused to acknowledge any and all data I've provided to them, saying that "things are working well". Yet, I have now a years worth of logged data in a very consumable format that shows otherwise.

The reality is I ended up randomly meeting someone who had worked for them and the conversation was such that the owner was furiously trying to figure out how to fix the problems, but had neither the technical skills to do so or the budget to revamp the architectural flaws.

Starting a WISP almost certainly is not for everyone. Getting it working correctly is only one aspect, but keeping an open network secure for your users is not trivial. Not only do you need a working knowledge of multi-user networks but also a decent perspective with regard to the threat models involved. If you can't, minimally, check those two boxes it seems (to me) a bit egregious to expect others to fork over money and trust you for said services.

> I have family that only had one option for Internet service. Long story short

Long story short: nobody cares if they are the only game in town

I think your analogy with restaurants is flawed. There's a reason a lot of newly started restaurants fail.

You most definitely need some form of experience from the restaurant business to succeed as a restaurant owner, unless you are really lucky. Your analogy make it seem as being a chef is the only job that exist in the restaurant business.

Sure, you can succeed without experience, but your chances to do so increases dramatically with experience.

I was also referring to the role of restaurant general manager and owner (Sometimes same person, sometimes not), not just chef.

I realized after the fact that I didn't elaborate on my meaning here, but it seems like you picked up on it.

What I meant was: You should have experience with at least one and preferably several of the important parts of running an ISP before you start one. NOC experience is one of the things that you could already have experience with, but isn't the only one.

> Your analogy make it seem as being a chef is the only job that exist in the restaurant business.

This actually makes my point better than I did. Being a [chef/network technician] isn't the only job that exists in a [restaraunt/wisp].

All that, and: a business where you extract cash from regular people and provide them technical support can be more painful than you ever imagined.

What type of profit potential is there to doing this? Seems like good stable income, but I also see lots of downsides with literally not knowing how anything works (and not being an engineer). There would be an awful lot of new information to learn, so the profit potential would really need to be there.

That would vary wildly by geographical location and whether it's some place that has no WISP at all right now, population density, competition, terrain and tree cover, number of houses, etc. I'm skeptical about the idea of starting a very small WISP right now, one could easily spend $40-50,000 on basic infrastructure for a small area and not be able to compete in nines of reliability/uptime and speeds with Starlink.

Exactly, you need to have access to tall buildings / towers etc with fiber. The cost of putting a couple of antennas on a tower can basically eat up any profit you would get. You are also competing with LTE/5G, and one storm can wipe out half your infrastructure in an afternoon.

Heh, I can't imagine that feeling when you have really no idea what you're doing, you have 50 paying customers, and then all your equipment goes down. This sounds nightmarish unless you are some sort of electrical engineer

As a one-man WISP responsible for everything at the physical level and the logical/IP configuration of the network, monitoring systems, billing systems, customer support systems etc you are also trapped in a personal hell of never being able to go on vacation and on-call 24x7x365.

There are certain sorts of people who are capable and comfortable with doing that and have the motivation to do so, but running a small sized ISP realistically takes 3 to 4 people with differing but complementary skill sets in order for everyone to not burn out. It takes a fair bit of revenue just to handle what would be the reasonable payroll cost for that plus operating costs plus overhead.

My first company was an ISP. There were 4 of us. We were around 19. No business experience. No networking experience. How hard could it be, right?

I configured my first router by calling our provider and saying we'd set things up but something was wrong, and having them echo to me what they did on their side so I could "check" our side - they charged for the setup, but this way we got it for free.

We literally lived in the office for the first year, and one of my most entertaining experiences was answering the phone at 2am on a Sunday morning and the person on the other end expressing surprise that someone answered - they'd just called out of sheer desperation and expected it to ring forever.

Then there was the person who spent three hours terrorising us before it transpired he'd not connected his modem, nor turned it on, or in fact taken it out of its package - something the person talking to him found out call by call, while the rest of us tried to not audibly laugh, because surely nobody could be that stupid? (a "learning experience" - it was how I came to understand why support people ask all those really obvious "stupid" questions.

I can't make up my mind if all the network issues were the worst part, or if the support calls were.

This was 26 years ago, and I have just about gotten over the soul-sucking parts of it by now.

I even contemplated setting up a local ISP again... I don't know why.

> Then there was the person who spent three hours terrorizing us before it transpired he'd not connected his modem, nor turned it on, or in fact taken it out of its package

I worked at a local mom and pop ISP at around the same time, probably a few years later. Those sorts of customer calls were the best (after they were resolved of course.)

* One guy would call at 2 or 3 AM and scream into our voicemail "Internet's DOWN! FIX IT NOW!!!!!". His wife would call the next day and apologize.

* One time, someone called irate that their dialup service wasn't working. He called the number we gave him, and all he heard "was a bunch of ringing and static". After several minutes we figured out he was dialing the modem bank from a telephone and expecting to use the Internet without a computer. He did not own a computer, and was quite upset we had not told him he needed one...

* It took two or three hours one time to figure out why someone couldn't connect. Helpfully, they figured it out themselves -- they were entering "the letter zero, not the number zero".

* A guy called up one time wanting to tell us about an interstellar propulsion device he developed (which also, of course, could cure cancer if the 'magnetic field' was reversed.) I suggested he call the Jet Propulsion Labs and gave him the number.

For me, the support calls were always the worst and I was very glad when the company got big enough that I no longer had to regularly take calls (although the ones I did get were the worst of the worst.)

Great stories, thanks for sharing!

I'm sure JPL loved the referral...

I couldn’t not refer the guy after I spent 15 minutes listening to his…not exactly well-thought-out, but certainly extensively-thought-out idea. Also, it was Friday afternoon and I needed a way to get off the phone. Just hanging up seemed rude.

I once lost internet for a week when our WISP provider's base station went down. We tried calling his company for days, and he only came to fix it when we found his cellphone number and accosted him personally. Thankfully we use Starlink now.

> I would caution against just jumping into this

I don't think that anyone needs caution warnings about starting an ISP... more then likely they know that it isn't like drinking a cup of coffee.

Reply on news.ycombinator.com