Boom, Bubble, Bust: The Fiber Optic Mania [pdf] | Hacker News Reader

Boom, Bubble, Bust: The Fiber Optic Mania [pdf]

internethistory.org

68 pointstim_sw46 comments

I don't get the drama. To me this looks like a normal process in an industry that discovered a groundbreaking technology. A new technology pops up, everybody rushes to innovate, lots of actors do nonsense, some do the right thing, but for many of them you don't know beforehand which is which. So you throw money at all of them. The rush creates some really good ideas and innovations. And a lot of BS and wrong decisions. Within a few years, the successes emerge, everybody else is flushed out of the market.

Today fiber is the network tech. Except for the last mile, everything is fiber. We're at 400G per port, soon 800G, and the market keeps growing and 1.6T is the obvious next step. Normal growth process.

So, I don't really see the takeaway here. What to learn from it? Not to invest in hypes? That's not really how we grow.

Depends on the country, where I live it's pretty common to have fiber to the home. Everything runs on that, internet, cable tv and phone calls.

But in other areas most residential bandwidth increasingly comes from wireless techs. Even in homes with wired service, nearly everyone uses wifi rather than wires, let alone fiber to end machines. The last mile is generally not fiber, nor is it a literal mile.

Indeed many people, including myself are now in the weird backwards world where the WAN is far faster than the LAN.

I have a 3gbps fiber link straight to my house. The only machines in the house that can use anything close to it are the ones that hook up to my USB-C docking station which has a 2.5gbps ethernet port. Everything else is WiFi speeds.

EDIT: This was a sudden change. I live rural and for the last 10 years I only had 10-15mbps speeds top, and most recently only by point to point wireless from a tower 5km away.

The advantage of a fast WAN is you can put it into a capable router (2.5-5-10g) and break it out into multiple clients who can use it simultaneously even if at only 1g each.

This also applies for wireless technology. My phone generally gets a faster connection via 4G, than via WiFi inside my house (the cable connection is fast, but the WiFi is slow). I think because the 4G connection can always see a tower through a window opening, while the WiFi has to go through multiple walls. (No interest in setting up a mesh network as the 4G is plenty fast enough)

This. Only a few years ago Hacker News commenters were crying out for new laws and regulations to subsidize the construction of Google Fiber to every home in America. “It’s a basic human right” they would say.

On the sideline there were a much, much smaller minority who looked at those comments in horror, with the context of knowing that wireless was a serious alternative that wasn’t that far away and was being broadly ignored

The network neutrality discussion belongs in the same bucket. People thought it would be the end of the open internet if there wasn’t a dedicated regulation.

There was dedicated regulation, it was just taken up by states instead of the FCC https://www.techdirt.com/2023/06/12/telecom-industry-ass-kis...

Interesting. I'm not sure "the entire west coast and huge swaths of the midwest and east coast have passed state-level net neutrality laws" is correct if this linked map is to be believed: https://www.naruc.org/nrri/nrri-activities/net-neutrality-tr...

Huge parts of the US apparently do not have such legislation, and it is unclear which states that "proposed" it have put it into law.

I don't see the problem? I have FTTH. I can get 10 Gbps. I don't want wireless, it's an unreliable last resort. If I have the option I'm absolutely going with fiber.

It's also the better long term tech. Fiber has enormous capacity, while wireless is a shared medium impeded by things like walls, and those problems get worse with the increased frequencies needed for more bandwidth.

>> wireless was a serious alternative that wasn’t that far away

But it really isn't. There are hard physics limits on how much data can be transmitted within a given frequency band. Fiber total theoretical bandwidth is is essentially infinite in comparison. All the traffic of the entire internet could probably flow through a single fiber bundle perhaps less than a meter wide. For things like streaming 4k/8k/12k (real 4/8/12k) to multiple devices, wifi will never compete with fixed lines.

> nor is it a literal mile.

The cost problem of the last mile is not for the 2 meters between the fiber router in the home and the living room. The issue with the "last mile" is that it used to be excessively expensive to lay fiber everywhere on the, literally, last mile outside people's home.

I could plug a network card with an SFP port and have actual "fiber to the desktop" for the last three meters at my place but I don't. I'm not sure that these three meters where I run ethernet are called a "last mile" (off by a factor of 530x calling 3 meters compared to a mile) ad mean I don't have FTTH or that the last mile ain't fiber.

> ... most residential bandwidth increasingly comes from wireless techs.

What good does it make to laptops and phones using WiFi if the router is doing 40 Mbps max over DSL? It's once you bump that DSL link to FTTH that suddenly all these wireless devices can enjoy much faster speeds. In all the countries I've lived in people at home used WiFi, even from their phone, instead of 4G because 4G means lots of $$$ / EUR.

I think you're underestimating the gigantic speed boost many people are enjoying thanks to fiber (and not thanks to 4G) now that they're having FTTH.

I recently switched from a supposed 900 Mbps Comcast connection to FTTH, and the difference is astonishing. There's just no substitute for Fiber.

> The cost problem of the last mile is not for the 2 meters between the fiber router in the home and the living room. The issue with the "last mile" is that it used to be excessively expensive to lay fiber everywhere on the, literally, last mile outside people's home.

And yet it was done for electricity and telephones over a century ago. And fibre probably has a longer 'shelf-life' as it does not corrode, so the initial installation is more likely to last longer.

Indeed. I had FTTH in Spain, in Andorra 100% of the (tiny) country has FTTH, I'm in the middle of absolutely nowhere in France atm: rural/seaside (about 45 mins to an hour drive to the closest highway) and yet I've got FTTH.

And at my new place in Luxemburg I've got FTTH too.

Fiber to the home is becoming incredibly common in many countries.

My brother had fiber to the home in 2003 already... In Japan (Tokyo).

> Today fiber is the network tech. Except for the last mile, everything is fiber.

(G)PON, (gigabit) passive optical network, seems to be what everyone is moving to:

* https://en.wikipedia.org/wiki/Passive_optical_network

* https://en.wikipedia.org/wiki/GPON

Even for wireless (LTE/4G, 5G), GPON seems to be used for backhaul:

* https://ieeexplore.ieee.org/document/8650520

* https://www.lightwaveonline.com/5g-mobile/article/14188094/t...

IEEE 802.3ca-2020 allows for 25 GigE per lambda, being able to combine two for 50 GigE; I'm sure there's an ITU equivalent (or will be soon).

* https://www.itu.int/hub/2021/06/new-itu-standards-to-boost-f...

* https://en.wikipedia.org/wiki/Higher_Speed_PON

The main 'semi-hold out' are cablecos, where they are pushing fibre closer and closer to the premises, but generally haven't yet decided to replace co-ax in people's homes yet:

* https://en.wikipedia.org/wiki/DOCSIS

You seem to be agreeing with the author, who notably compares it with railway mania.

> Technologically, fiber communication has been a brilliant success.

> The bubble warped the financial picture. Looking back, Odlyzko compared the millennial bubble to the British rail- way mania of the 1840s ...

> Looking back, the most important lesson from the fiber optic mania may be that the most successful technologi- cal revolutions can be the messiest. The bigger the profit potential, the more manic investors become and the less critical judgement they use.

And the concrete takeaways depend on exactly who you are, but "selling out early" before the bubble pops seems to be the story of the big individual winners in this story.

Yeah the phone companies laid those thick 144-strand bundles before wave division multiplexing. They figured the only way to get more bandwidth was more fibers, right?

WDM came along and caught them with their pants down; now all of a sudden you could just attach different devices to the ends of the fiber and get 16x the bandwidth. No backhoes needed. Then CWDM let you split and combine channels in locations without a reliable power supply -- no electronics, just prisms!

Unfortunately their response -- those who weren't killed off -- was to get the antitrust regulators to look the other way (i'm sure the "national security" card was played) while they bought each other up and carved countries into a checkerboard of regional monopolies.

But hey, if you squint at it from far enough away you can see two colors, so that's not a monopoly, right?

In the meantime though we've started using a substantial fraction of those thick fiber bundles and dark fiber is getting harder and harder to find. Barring another revolution in technology I suspect that at some point we'll be laying even more of those big bundles, or even bigger ones.

> dark fiber is getting harder and harder to find

Only because all of it along any given route now tends to be owned by a single entity. And that entity can simply say "not for sale".

If the market were healthy dark fiber prices would skyrocket instead of availability plummeting.

WDM guy here. Sure, 80 channels are space for lots of 400 Gb/s circuits... But we still fill cables and keep laying more. Though yes, the big 1288 fibers cables aren't common. Also, lambdas don't terminate at every node - so sparse network topology implies that filling all channels on a single link is a merely theoretical situation.

>before wave division multiplexing. ... WDM came along

Would that be Ciena in 1996?

>Yeah the phone companies laid those thick 144-strand bundles before wave division multiplexing. They figured the only way to get more bandwidth was more fibers, right?

I don't know anything about fiber tech but can't they just apply the newer techniques to the already laid out 144 strand bundles?

FYI, this article is from 2016 and discusses the dot com bubble that popped around 2001.

Jeff Hecht is an excellent author.

I was involved in a UK company that strung fibre optics along the high voltage transmission network to quickly build a national network, there was a ridiculous amount of money flowing around hooking up companies, then there wasn't. I got out before it collapsed but you could see it was unsustainable.

Later, due to my experience with the software involved, a colleague and I were tasked with recovering data from another huge firm that collapsed after the boom, walking into a massive empty office with a post it note with a server name and some passwords. Nothing on the db to recover but we had the skills to bypass the security.

FWIW this is happening again in the UK right now with FTTH deployments. There are about 10-20 companies of fairly sizeable scale, with hundreds of millions of billons of GBP of funding laying fibre.

I think this is a good thing for the country (it has forced Openreach & Virgin Media to massively accelerate their FTTP plans); but it is going to be cataclysmic for the investors.

Firstly, the build cost is astronomical and completely inefficient, apparently £1000/home is pretty standard, which is 3-10x what openreach and VM are paying (these altnets have access to all the openreach ducting and poles), due to very poor efficiencies in their build process and massively overpaying for materials and labour.

Secondly, they overbuilding like crazy. I had 5 different FTTH providers at my old London apartment available, with totally separate infrastructure. This is meaning takeup is a fraction of what they expected it to be, as for the most part they expected to have at most one competitor at a property.

Third, they seem to have absolutely no idea how to market and acquire customers, since their main skill seems to be how to pitch infrastructure funds to get capital. Their rollout is very patchy nationwide or even a local area so marketing campaigns are hard to do.

There is going to be a huge wave of consolidation (at best) or bankruptcy in the sector as interest rates have risen and the calculus has totally changed.

I suppose this is what happens when infrastructure stops being funded by the government.

I’ve got 3 current FTTH providers at my flat in London, I wasn’t even aware other companies were about and also offering the service. I guess this’ll be good for consumer costs in the short term before it inevitably implodes.

Nothing new: https://en.wikipedia.org/wiki/Railway_Mania

Isn’t it fun seeing history endlessly repeat itself.

They have the low hanging fruit (large apartment blocks) sewen up and are now spreading out.

Last week I was excited to see FTTH guys cabling up my suburban street and was told it should go live in a month or so. I went onto their website and their register info form didn't work (and still doesn't) due to them using some broken Experian email validator script!

The fortune they're spending on roadworks etc yet their bloody signup form doesn't work!

The follow-up article should be about Google.

Google saw the bandwidth glut (global fiber deployments were over-provisions and under-utilized) and took advantage of this. While earlier starts were in a "bandwidth is expensive" mentality, Google designed in a "bandwidth is cheap" mentality. It gave them a big advantage both in cost of operations and ability to design large scale systems differently than previous competitors had been able to.

There was also a datacenter/colo glut. Google was buying up every datacenter square foot they could find. They could get it at pennies on the dollar just like bandwidth.

This lasted until around 2009-ish when both gluts were exhausted. Google turned to running their own fiber and building their own datacenters just in time.

TL;DR: Google's insight into the economics of global bandwidth and datacenter gluts became an advantage in the early years.

The telco bust was much larger than dotcom bust. I have a couple of charts from the time here: https://blog.eutopian.io/nsa-prisms-commercial-cousin/

>The investors who threw money at optics at the peak of the bubble probably would bankrupt us all if we ever hit a true “technological singularity.”

If we really hit technological singularity, that will be the hail Mary of all bubbles.

conversely, I think we have been in the singularity since the late 1700's. For many thousands of years human manufacturing capability was effectively flat, a few blips and bumps but the advancement was generally steady and linear.

And then in the late 1700's things changed. The graph went hockey stick and has stayed exponential since. However from inside this bubble there is a sort of tech dilatation where everything appears normal and flat and it is only the future that has exponential growth.

Flat? I don't think so. I'm at a coffee shop and looking around me, this is very different from just 20 years ago. People with laptops and smaetphones around me. Just 20 years ago, that same place saw a few CS major nerds with their laptops, everybody else sat with a book or some paper to write on or reading a paper newspaper. Still with a latte, but tech has changed everybody's life dramatically in 20 years. 20 years from now I'm again expecting at least as big a change.

Definitely exponential.

You're agreeing with the comment you replied to. they were saying it was flat until ~1700 or so.

They wrote that it "appears flat". I tried to contradict that.

> However from inside this bubble there is a sort of tech dilatation where everything appears normal and flat and it is only the future that has exponential growth.

Got a friend working for Lumen and it's an overstaffed mess with declining income. He says it's pretty bad internally. They are all wondering if someone like Facebook, Google or Amazon will just snap it up but doubt if it will happen.

Fiber. Dot.coms, housing, crypto, AI, ….

Bad, bad memories of MFNX debacle (metromedia fiber networks). Verizon and Kluge hosed everyone.

I'm not limited by my download/upload speed. Heck, 4G or 5G, whatever is on my phone, is absurdly fast.

I don't see the demand for consumers.

Heck, outside AI, the computer isn't even my bottleneck. Up until ChatGPT/Stable Diffusion was released, I thought computing was near end-game for consumer needs.

640k should be enough for everyone right?

Eh, we knew <1mb was limiting. The fact that I had multiple drives was a sign.

Instant analogy to GPUs...

A similar story is found in the railway bubble bursts in the US. Railways are also incredibly revolutionary technologies that reshaped societies, doesn't mean they are immune from investment cycles.

That being said, given ChatGPT was only released 8 months ago, we are probably very early in the GPU boom, far from bubble yet.

Computing seems pretty important. I have a usecase that can save our company millions of dollars, but unless my company is willing to spend hundreds of thousands on a computer, its not happening. (It may not work, and prototypes still have me conflicted if it will work). We also can't rent servers for this, too confidential.

If we can get prices down 10x-100x, I can even do this for personal/home usecases. I know that sounds extreme/absurd, but given our office is nearly empty with WFH, we can use larger computers and no one would mind.

Reply on news.ycombinator.com