Facebook aims to knock Cisco down a peg with open network hardware | Hacker News Reader

Facebook aims to knock Cisco down a peg with open network hardware

arstechnica.com

83 pointsvelodrome30 comments

SDN and OpenFlow are great if you're a company full of programmers like Facebook and Google are. They approach everything as a software problem. For many (most?) other companies the network is just something that has to work and not a core part of their business. You need to be able to hire a network engineer that can come in cold, with a bunch of vendor certifications on their CV, and understand your network. Most places don't want to hire a programmer to sit around and code up new spanning tree algorithms for their network. They just want to plug servers in and have it work. The people that are interested in SDN are running into the limits of what OSPF/BGP/MPLS can do and are going to work around it but many places don't run networks that complex in the first place.

I think there are a lot of gains to be made from changing the industry though, for the first time the hardware is becoming a commodity made by Broadcom and Intel (Fulcrum) just like servers. In the past you bought Ciscos or Junipers to get the custom ASICs that made the thing work, those days are coming to and end. But I wouldn't count the big companies out just yet, they have a massive head start writing software and have huge service organisations and a whole industry full of engineers that have paid to get certified on their hardware and aren't going to want to see the value of that education fall to zero. The winners are going to be the hardware companies that really are software companies at heart - Juniper and Arista are the best examples. Cisco has always loved its hardware and been way behind on software development.

This switch does not require SDN or OpenFlow. The idea is you run whatever OS on it you want that provides the features you need. For instance, you can go buy an OS from Cumulus (mentioned in the article) and run it on an ODM switch and run traditional routing protocols and topologies. It is only a matter of time before other somewhat traditional vendors start selling their OS for commodity hardware as well. Arista will likely be the first.

"They approach everything as a software problem. For many (most?) other companies the network is just something that has to work and not a core part of their business. You need to be able to hire a network engineer that can come in cold, with a bunch of vendor certifications on their CV, and understand your network. Most places don't want to hire a programmer to sit around and code up new spanning tree algorithms for their network. They just want to plug servers in and have it work"

Do you not need network engineers to implement Cisco products?

Yes but they aren't programmers. Configuring routers using vendor tools and standard protocols is a different thing to SDN where all bets are off. It's going to take a different skill set - some people will make the transition and some won't. All of the major network vendors (most notably Cisco) have put a lot of effort into their education programs so that there is a certified pool of talent out there to run their equipment. And all of those people have a vested interest in the status quo. The OpenFlow talks that I've attended have been presented by people that are openly hostile to network engineers which doesn't help the situation.

For those interested, Google has been using OpenFlow for some time now. This presentation by Urs Hölzle is about a year old:

http://www.youtube.com/watch?v=VLHJUfgxEO4

Slides: http://www.opennetsummit.org/archives/apr12/hoelzle-tue-open...

Funny how this didn't make as much waves back then when they announced it. I honestly think it went over everyone's head.

I would love to buy a server that conforms to Open Compute and a rack that conforms to Open Rack. Where can I do that?

Quick google search turned up:

Penguin Computing: http://www.penguincomputing.com/Products/OpenComputeSolution...

Silicon Mechanics: http://www.siliconmechanics.com/c1345

Thanks! Last time I looked, I came up with nothing but the specs on the Open Compute page.

That Silicon Mechanics stuff is obsolete and the Penguin servers don't look like OCP at all.

Former Silicon Mechanics employee here.

While I was working there, we had between zero and no interest whatsoever from anybody in actually seriously trying to purchase anything OpenCompute.

Apparently some board members used to work with some higher-ups at Facebook, so the project is sticking around for a while despite the total lack of interest.

If there's any actual real-world interest in OpenCompute stuff, you HAVE to reach out and ask or it's going to disappear.

Good point, although I suspect people who are willing to reach out are going to the ODMs and people who have interest in clicking the "buy now" button cannot express that interest because there is no "buy now" button.

Penguin just announced a real OCP server today (although still no buy now button): http://www.penguincomputing.com/Products/RackmountedServers/... http://www.theregister.co.uk/2013/05/10/penguin_computing_op...

This is just like the big server vendors (IBM, Dell and HP) losing all their sales to big companies, because they are buying their own designs direct from companies like ZT and Hyve.

What's left, storage systems?

No, they're also working on storage.

I can't imagine that the margins on sales to big companies are that great, considering how much volume they can throw around.

IBM big systems will still be running long after you and I are dead.

Running the same software that was first written before we were born.

Already being worked on really. It's more blob-storage oriented, but take a look at Openstack Swift if you haven't before or Ceph. Lots of other projects out there.

If you don't specifically mean block-storage type software and more open replacements for Netapp/EMC gear, someone else might be the better one to chime in.

For the hardware (storage) part, there is the storage pod from Backblaze:

http://blog.backblaze.com/2013/02/20/180tb-of-good-vibration...

Long overdue.

Cisco is in trouble for a lot of reasons, but this might be the straw that breaks the backplane. Routers and switches make the world go 'round on Tasman Drive...

The timer is running on Chambers' resignation. I don't think he'll be fired -- he's too established and congenial -- but I also don't think he wants to run a company 1/3rd of Cisco's size

Agreed. Plus, this may be the coolest thing going on at Facebook.

People don't buy Cisco just for the box with ports and blinky lights. They buy them for the supportability. As for a business they depend on here network to run and as mprovost mentioned you can take some one cold who has networking experience and If they have half a brain they should be able to fix your network. All Cisco has spawned a whole Eco system of vars who exist throughout the world and can have replacement parts onsite in 4 hours or less. Also Cisco provides code scrubs and a range of other features. So unless some vendor can build these magical switches and provide support they will never be a serious option in any environment.

What is the advantage you get from having a sophisticated software stack on a top of rack switch? I was under the impression that you only really configured vlans and maybe a span port every now and again. Someone care to enlighten me?

It isn't always the software. Having open, controlled, customizable hardware can offer a huge advantage over what you're able to purchase from Cisco/Juniper/etc.

Also, a software switch allows you potentially massively scale the number of ports on a single switch.

In that case, eliminating the complexity of existing control plane software will be a win.

Some people may want to run OSPF or IS-IS, or do some kind of performance analysis, or do ARP/ND/DHCP proxying, or perhaps use switches to aggregate BMC monitoring & control, or who knows what.

I get that you would want all that stuff in a router, but the article is mostly about a new type of top-of-rack switch. My understanding (based on experience that is now a few years old) is that a top-of-rack switch basically only handles layer two switching duties and a separate core router handles the layer three business.

Some people perform routing in the ToR because STP doesn't scale. And TRILL/SPB have been proposed to make Ethernet scale, but then they end up having similar complexity as routing (even using IS-IS).

In many common environments (single-tenant, fairly static configs) it probably doesn't make sense to use an SDN. However, if your environment looks more like EC2's, with lots of VMs/apps/services, lots of tenants, and lots of virtual networks, you need some way create an overlay that provides logical layer 2 switching beyond what simple VLANs can provide. For example, if I want to migrate a running VM from one host to another, it would be nice if the top of rack switches could be made aware of the change as it happens so CAM tables could be updated on the fly. Cisco and VMware do something similar today with the DVS and Nexus 1000v virtual switches, and a lot of the OpenFlow/SDN vendors are hoping to extend that type of dynamic orchestration out to physical switches as well.

> if I want to migrate a running VM from one host to another

If only you could do this on EC2. >_<

It's not putting the complex stack in the switch, it's putting hooks into the switch so that a complex algorithm can run on a generic server and direct the switches function.

For example, lets say you're moving VM's around, or needed more bandwidth between two areas. You could reconfigure the switch all in software much faster by doing it programmatically rather than going through even a standard CLI interface.

Want to design a new L2/3 routing protocol for your specific traffic needs? Now you can.

Think of this as the difference between a graphing calculator and a copy of R, or SciPy. Sure, they both can do the same basic calculations, but the latter has so much more programmability and headroom.

Reply on news.ycombinator.com