Hacker News Reader

Comment by johansch | Hacker News Reader

That is a key question I have been pondering myself.

One theory of mine (perhaps uninformed; I'm not really a networking expert) is that because of the dynamically configurable nature of their systems, they need to use routers rather than relatively dumb and cheap switches at almost every level - in order to have flexible networking and still maintain isolation between customers.

This could get quite expensive if you have to pay Cisco/Juniper for this. If this is true Google will have quite an edge with their software defined networking here, I would guess.

View on HN

No, they use whitebox switches and software defined networks to control. See https://www.youtube.com/watch?v=n4gOZrUwWmc [Edit: oops, fixed!]

> See https://www.youtube.com/watch?sns=tw&v=z7Cyv3cvIxY

Nike advert?

Lol, sharing multiple youtubes with multiple people at once. Fixed.

That's Google. They have put their cost levels somewhat below Amazon's. Maybe they don't see the need to be 5-10x cheaper than the market leader in traffic costs even if they could...

SDN is changing the model here, and Google is way ahead. In an enteprise, you can use VMWare to do a lot of the stuff you are blowing big bucks on for Cisco/Juniper on and use switches with higher density.

SDN is going to turn the cost structure on its head -- I wouldn't want to be a network guy now, easily 60% of tasks are getting vaporized in the datacenter.

As a network guy, it's a _great_ time to be an experienced network person. The only mature aspect of the ill-defined SDN sphere is OpenFlow and that will only get you so far. Try as they might, controllers like OpenDayLight and the various things that plug into Neutron/OpenStack aren't plug and play for those w/o significant network knowledge.

From my vantage point, it's going to be at least another five years before the cost structure really does turn over on it's head for folks below the hyperscale level

No doubt. Any change is great news for smart people. But the average joe churning out firewall changes and similar are screwed.

Google's really ahead on the networking front, and other cloud providers are following suite. Networking hardware is super cheap now. When you couple that hardware with open source software networking gets cheap.

Large networks like Level3, Cogent, Telia, etc all use big-iron routers (Cisco/Juniper) and will sell you traffic for under $1/Mbps.

They (Level3, Cogent, Telia) don't have millions of ports though...

This is true, but I can't imagine Google/Amazon/Microsoft are using Cisco/Juniper routers at every level of their network.

Hosts play a big part of SDN in that they support the dvswitches along with guest VMs. Not everything is a Cisco/Juniper. Switching hardware is still common in TOR and egress.

VLAN's and Virtual Appliances in the same environment as the guest machines to facilitate routing should allow for scale without costing these virtualization providers too much.

$1/Mbps per what (unit time)?

Per month, usually (billed 95%ile). That price will decrease by quite a bit with more volume.

Thanks. How long has it been roughly $1/Mbps/Month ? Do you know of any sources with historical data?

Average was $0.63/mbit in 2015, and I personally haven't seen lower than $0.40/mbit.

http://drpeering.net/white-papers/Internet-Transit-Pricing-H...

That really depends on volume, location, and provider. For large volumes and cheaper providers (Cogent, HE.net, etc) it's been that way for 2-4 years or more. HE.net will now sell a full 10GbE port for $2600/mo, Cogent isnt too far behind. Sub $0.40/Mbit at >25Gbps volumes in major locations is doable.

Yep, and once you're at the multi-gigabit per second level, the price drops much lower than that pretty quickly.

Indeed - Internet Transit at scale (10 Gigabit+ ports) goes for around $0.63/Mbps at 95th percentile. [1] - for the above quoted 700 Terabytes/month, that works out to $1341/month, if it's evenly spread out on the lower 95th percent of the circuit at around 2.129 Gigabits/second.

[1] http://drpeering.net/white-papers/Internet-Transit-Pricing-H...

pretty sure (at least) AWS builds their own network hardware. I remember reading something a while back that said they found it magnitudes less expensive than buying enterprise hardware, with better performance as they went about the affair as scientifically as you'd expect them to.

Old CoyotePoint routers were just a commodity x86 motherboard with an ancient SSD instead of spinning rust. Junipers use a duo of x86 (routing engine) and ASIC (packet forwarding engine). Cisco has supposedly moved from that architecture to an ARM and ASIC pairing.

The ASIC is just a hardware offload for known routes. Unknown routes, admin work, and Ping packets are handled by the x86/ARM CPU. It's not too different from offloading graphics work to the ASIC on your graphics card, or your mining to your Bitcoin ASIC.

Reply on news.ycombinator.com