Cumulus Linux – first impressions
packetlife.net
packetlife.net
This all seems to be a giant step backwards. JunOS is now the gold standard for its CLI and configuration, due to being highly structured and well organised. Being able to validate configurations before committing them, and automatic rollbacks with commit confirm is a much improved way of doing things over IOS. Given that Juniper hardware can often be bought significantly discounted, the cost savings of white box gear are going to be small, if at all. Thus far, they seem to be sold in such small volumes that comparable Juniper or Cisco gear can easily be purchased for less.
Also, how exactly is Cumulus separating the control plane and data plane? From the look of things, SDN is going to end up being relegated to just easy automation instead, like how Cloud has just been relegated to VMs with flexible billing instead of abstracting underlying infrastructure.
Also, how exactly is Cumulus separating the control plane and data plane?
They're not. Cumulus is kind of anti-SDN.
We're running a gateway router on Vyatta/VyOS and with every single hardware and software change we had some issues. Most were relatively minor (like default settings being not appropriate for our workload), already documented/solved by someone on the 'net. Some had required reading source code and debugging/profiling to see what goes on. Must also admit, some were beyond my understanding and were mysteriously solved with some shamanism and voodoo magic (like randomly tinkering with firmware and driver version combos).
We work closely with all the major SDN vendors for use-cases that are virtualization-centric or cloud-centric. That said, there are use-cases like Hadoop clusters that have very little need for an SDN layer, just very fast L3 IP connectivity.
Cumulus's value is that it's the translation between Linux kernel bridging/routing, and the hardware accelerated routing performed by the Broadcom Trident family. Somebody has to sign Broadcom's agreements and program for it, and Cumulus is doing that cheaper than anybody else as far as I can tell.
Another value is that you can now choose between all sorts of switch vendors, and not be as susceptible to lock-in from that angle.
Retail pricing on a white box 48x 10gB + 4x 40Gb switch is less than $6k, and an annual Cumulus license is $1k. If Juniper can match or beat that, I would love to see it, but I doubt I ever will, because you have to spend days cultivating a relationship with Juniper to even start getting decent pricing. And really, JunOS, despite being nice, puts Juniper at a disadvantage over Linux as the net OS. I'd even pay a small premium to use Linux as it is so much more programmable and makes it so much easier to get new racks up and going.
That said, NBD or sooner hardware replacement plans are a bit weird with Cumulus. I think they expect you to keep onsite spares, which still ends up being cheaper.
I think if Cumulus becomes more popular, we might see some community code that implements validation and confirmed commits and more features. If we decide to go with Cumulus, and we are thinking about it, we might write something like that.
This is how a 1U switch drawing less than 200W can forward 2+ terabits/sec. A server full of 10gig/40gig NICs would be 50x slower and draw way more power.
I prefer JunOS to IOS/NX-OS as well, but both of their CLIs are optimized for managing switches with hand-written (or PERL script written) config files. Most Cumulus customers use automation tools. We've had folks use Chef, Puppet, CFEngine, Ansible, Saltstack, as well as a few mega-scale customers who have home-grown automation tools. The idea is to use whatever you're already using to automate servers to automate the switches; often there is substantial sharing between the server automation scripts and the network automation.
The volume is quite high already, Cumulus Linux alone is managing well over 1 million 10G ports today, and we're not the entire white box OS option. Our software pricing is available on our website: http://cumulusnetworks.com/product/pricing/ and some of our hardware partners publish price (keep in mind, this is web orders of quantity 1): http://whiteboxswitch.com/collections/all-switches
But the biggest cost savings come from automating the management, and not having to buy vendor-locked optics. Compare the prices here (again, quantity 1!): $30 for a SFP+ 1M DAC cable: http://www.fiberyes.com/sfp-cable-cab-10gsfp-p1m-30 $59 for a SFP+ SR optic module: http://www.fiberyes.com/10gsfp-transceiver-axs85-192-m3 to what you're you're paying cisco or Juniper.
We don't really separate the control and data planes. We don't really consider ourselves an SDN company, we enable many approaches to SDN by building very high performance fabrics that you can run your SDN layer on top of. We work closely with VMware NSX (formerly Nicira), Midokura, Nuage, and PLUMgrid. Some of our higher scale customers have their own SDN solution.
<Disclosure: I am co-founder and CTO of Cumulus>
Thanks for the link; I've seen similar pricing on the Quantas before, but only from obscure online retailers, who are often well below prices from more trustworthy sources. Would you recommend whiteboxswitch.com as a reliable source?
I don't think the optics is a fair selling point however, as you can easily purchase 3rd party optics programmed to be Cisco, Juniper, etc. compatible. We currently buy Juniper coded SFP+ SR optics for <$25 from a distributor in China.
whiteboxswitch.com is a fine option, though bm-switch.com seems to have lower web pricing at the moment.
Most of our customers are concerned about support, and thus aren't interested in unsupported optics. I guess as long as Juniper doesn't find out... =)
> The emerging software-defined data center (SDDC) paradigm involves automated control of all network, server, storage and application resources, resulting in a cloud operating system. Unified visibility is essential, enabling the cloud operating system to efficiently allocate resources, detect problems and ensure consistent performance.
Still pretty attractive, and a lot less Buzzword Bingo.
If you're interested in having a substantive technical discussion about Cumulus, drop me a line: nolan@cumulusnetworks.com
You just patch in all the cables straight onto effectively a Hypervisor, and then you generate virtual switches, routers, firewalls, and so on completely via the VM management console.
You're already seeing some of Cisco's smaller competition go this way. It saves physical space, works on standard hardware, and is easier to centrally manage (as you aren't physically moving wires after initial install).
It will be interesting to see if Cisco jumps aboard this train or continues to pretend like the sands underneath it aren't shifting. I'm sure there would be a market for VMs running Cisco's IOS, it is still by far the most popular network operating system.
I agree with this idea in that most of the intelligence will be pushed out towards the edges of the network and overlay networks will make a lot of the physical network invisible to the applications and even the management. However, this does not mean that the networking hardware can all be just dumb devices. First of all, this kind of imperative networking will not scale to large data center implementations and secondly dumb devices will become useless once disconnected from their controller.
"It will be interesting to see if Cisco jumps aboard this train or continues to pretend like the sands underneath it aren't shifting. I'm sure there would be a market for VMs running Cisco's IOS, it is still by far the most popular network operating system."
The way Cisco is getting into the SDN market is by leveraging Application Centric Infrastructure (ACI) to define the complete DC (Compute, Storage and Network) with templates and letting the whole system configure itself. The switches supporting this infrastructure are based on merchant silicon in combination with Cisco ASIC's to provide the most optimal performance at these intelligent edges. Add to that the available open API's and the investment in OpenStack and you have a rock-solid and cost-effective solution for the future of the DC and DevOps.
Cisco ACI: http://cisco.com/go/aci
I would love to talk about it more if you want. These are exciting times. Hit me up on email or IRC.
(Disclaimer: I work for Cisco but these thought are still my own, yada yada)
I am amused when I read things like this, 99% of the Linux savvy folks I know use ifconfig.
That said, ifconfig doesn't support easy addition of multiple IP addresses to a single interface. You have to create 1 alias interface per IP, which gets unwieldy fast.