Linux dominates supercomputers as never before
zdnet.com
zdnet.com
For the young ones in the audience, the supercomputer term has historically been used to describe machines with largely custom hardware from the CPUs up - or at least from the CPUs onwards. Cray vector machines are the canonical example. These were shared-memory / single-system-image machines, software really saw it as one computer instead of a networked cluster.
I was briefly involved with the supercomputing world about four years ago. "Should we use Infiniband?" was no longer considered a non-rhetorical question even then.
While there's a grain of truth in this, in reality, a supercomputer still looks like a room full of refrigerators and not at all like a typical computer or even a server farm.
Some of the computer hardware in there is just commodity hardware but for example the interconnects can't really be described as "souped up network cards", they're in a category of their own.
Yes, the era of the Cray boxes is gone but a supercomputer is still a supercomputer.
This is probably a technological response to the rapid pace of development. It's more efficient this way, instead of developing custom chips and boards for two years and deploying them when the high pace of advance makes them half-obsolete.
After years of being a Windows fanboy -- I look back now and am very glad I made the switch. With Linux -- it's never a case of "you can't do that".
One very minor nit...the PDP-7 where Unix started out cost US$72k+ in 1969 dollars. Nothing compared to the big iron of the day, but very far from a microcomputer (and, really, that market didn't exist until 1972 or so).
Maybe not that much, it next moved to the first model of the PDP-11, which had a max of 56 KiB memory of 8 bit bytes. Then to an 11/45, which could do 64 KiB of code, 56 KiB of data and 8 KiB of stack.
No PDP-11 AFAIK have a separate, dedicated stack space. You're thinking of the 8k dedicated I/O page.
However, as I recall (from the very early '80s when I was using both a 11/44 and 11/45), the MMU had a 8 KiB page size, so the natural thing to do was to divide the data as described, so a stack overflow would trigger a trap. Although after a bit of time with Google, I note the 11/40 manual says all PDP-11s have a location in which to put a stack boundary, so maybe that was a convention which matched what the MMU could do.
About 99% of all devices "just work" out of the box, the other 1% either are very obscure/specialty hardware, or are new hardware. Linux doesn't need a vendor to support Linux like it used to... most generic drivers are built in and work OK (just like the generic drivers built into Windows now-a-days).
Well if you count most GPUs under speciality hardware then I guess you're right, but most people don't. And most linux GPU drivers still pale in comparison to their Windows versions.
Compare that to driver support for things like the fingerprint reader in a Lenovo Thinkpad. I wouldn't want to bet on getting that working at all, unless Lenovo had preinstalled Linux for you. And unlike GPUs, this is likely to be all or nothing, so if you don't get the drivers you can't use that hardware feature at all.
I gave up and I bought a PC (Intel NUC). I was able to install Linux on it, but it failed to boot. I then found a BIOS update, and managed to get it installed (not a trivial task). After that my NUC worked well, except for the IR port, which required additional driver hunting.
Sadly, we're still a long ways from "it just works." The NUC didn't support Linux out of the box, even though it ships without an OS!
Also to my knowledge all modern Chrome devices have Coreboot support which enable "legacy" bootloaders and will boot any distro like a normal laptop. Older models like the C710 have unofficial Coreboot builds which allow the same (and linux driver support on modern distros).
This is a little better then when I was hand-editing x configs back in the 90s but overall I'm disappointed.
Of course, now I'm just using a rMBP, but yeah, I wonder if I've just been very lucky?
In Linux, I always have problems with devices, in one way or another (specifically Ubuntu, but I think it's fairly representing the concept), both for new things, old things, not-so-new things, obscure things, etc. etc.
To complicate the problem, drivers fit in stacks that may interact in unexpectedly broken ways. This means that say, a wireless audio adapter, may work only 50% of the times it's connected. I was using Windows long ago, and I don't remember this category of things happening (I guess because there are only a few Windows).
So, certainly they work "OK", but one needs to be quite flexible on what OK means. I've got two machines which are not so arcane, and I have at least 4, hardware-related (not stack-related), open bugs.
Thing is, Linux is much more enjoyable to use for software development and deployment, development on Windows makes me feel like a second class citizen, the OS to me feels much more oriented towards the exclusive consumer, especially now with the new (metro) tablet like user interface.
This high number is due to kernel programmers writing device drivers, since not so many manufacturers write linux drivers themselves.
If you are thinking of video driver it is a totally different story.
So, in large part, just about all games are now playable on Linux, and more are coming as developers re-release games on Linux, or as many big AAA studios have committed to releasing all future games on linux as well (Crytek, Valve, Unreal, Firaxis Game, etc etc... there are a lot and more coming!)
Also, there are a few components on these supercomputers that you won't find on a typical workstation or cluster machine.
First, the kernel itself is frequently very small, lightweight, and much closer to what you would find on an embedded system than on a traditional desktop computer. That's because once the program on a supercomputer is loaded, the kernel's job has been mostly to get out of the way. This isn't to say that you don't find the Linux kernel on these supercomputers, it just isn't as common as you would think from reading this piece.
Fear not, many pieces of these operating systems are still open source. Here's IBM's fusedOS prototype:https://github.com/ibm-research/fusedos.
Second, many of these computers primarily only run code in C, C++, Fortran, and Python. These tend to be the only major languages in play on the HPC machines, with acceleration frameworks such as OpenMP, OpenCL, and CUDA playing major roles.
Finally, everything is glued together with MPI, a high-level (at least it was in the 90s) abstraction for scientific programming that maps down to very high-performance networks designed to help scientific codes "scale", that is, run effectively when millions of cores are simultaneously engaged.
These are beautiful machines producing important science, and the GNU/Linux operating system plays an incredibly important role in both their implementation and culture.
Most are just slightly spruced up commodity server hardware running Linux. I'm not sure if this is what you're suggesting, but they don't run C/Fortran/whatever on bare metal. They're run by the OS on the compute node just like a normal OS process, except that tasks are dispatched to compute nodes by a central cluster manager. Processes running in a gang communicate via MPI to share data, though coprocessors are also pretty popular as well so you see a lot of communication between the host processor and a coprocessor too. Titan and Tianhe both actually have most of their compute power in the coprocessors (Xeon Phi and Nvidia Tesla, respectively), but they're still arranged in a master-slave arrangement just like if you bought a Phi or Tesla and stuck it in a spare PCI-E slot. They use plain old PCI-E, too. The Cray XT/XE series (a popular model of which Titan is an example) is basically just really nice blades with integrated cooling and a network backbone in a custom cabinet and possibly coprocessors attached to each blade. You could just as easily run Windows XP and play Minesweeper on each blade if you really wanted to, except maybe for some driver issues. The most foreign thing is probably the network backbones, where fabric architectures like Infiniband are popular.
They're also not limited to specific programming languages. In truth, you can run whatever you want if someone has paid the bill for your resource allocation. I watch people run MATLAB on large clusters all the time, which hurts me because it's so damned inefficient. That said, Fortran and C++ comprise the overwhelming majority of large and computationally taxing codes. Just because all that power is there doesn't mean that all of the users take proper advantage of it. One of the larger calculations run on Titan that I know of (Denovo, a nuclear reactor simulation code) didn't even use the GPUs, only the CPUs. Making codes that can take advantage of GPU processing ergo Titan and its predecessor Jaguar has been a major project at the DOE, with libraries like Trilinos being developed to make it easier on scientists, many of whom are only computer programmers as a secondary concern.
The setup you're describing used to be how it was until up to maybe 10 years ago and there are still systems in the top 500 that work like that. Probably some new ones being built, too. But what I've described is what seems to be in fashion these days mostly and the machines I use are all like that. I've heard mumblings about FPGA coprocessors being the Next Big Thing, but we will see.
I can think of only a few categories: desktops, ATM machines, real time, SCADA.
What are the remaining big sectors Linux doesn't yet dominate?
Cisco doesn't use it (IOS is home grown, IOS XR is QNX, NX-OS is Linux), Juniper uses FreeBSD, Arista uses Linux, HP uses (I think) Comware (linux), Intel & Broadcom are pushing Cumulus (linux) on their reference networking platforms, Dell/Force10 is NetBSD based (and older Dell is VXWorks, I think), Pretty sure Brocade is Linux, but they've also used VXWorks. Don't know what Hauwei uses (that wasn't stolen from Cisco, or course).
Not that this is authoritative, but I've talked to a couple of execs at companies who were using OSS platforms for their networking devices (Nokia, Juniper, Force10, Arista) and all of them basically said OpenBSD wasn't a consideration because of the open hostility the people leading the project had towards commercial enterprises. No one was interested in betting the company with the threat of a "DARPA funding" moment hanging over them, much less someone who was more interested in dictating terms and making "statements" than cooperation on business relationships.
[edit for punctuation]
Nevertheless, there are some small companies selling things based on OpenBSD - even though, I'll admit, they are mostly companies started by OpenBSD developpers:
http://www.openbsd.org/products.html
You can also get some consulting & support:
http://www.openbsd.org/support.html
And, of course, there is always the strong possibility that some people are using OpenBSD while keeping it hush-hush and "under the radar" -- maybe even under the radar of their own management.
Make of this what you will.
Because Linux does dominate mobile? Android is Linux and iOS is a *nux. (iOS is a BSD derivative.)
In revenue (app-sales etc) iOS dominates.
Who made such research? I'm not sure if it's even possible to know exactly, unless all vendors who run their own stores publish sales statistics.
They do, thats one of those things that companies that are traded publicly are required to do.
Publicly traded companies are required to disclose revenue, but not required to break it down by source.
Neither Apple or Google break out the revenue from the *Stores (unless they have started doing it very recently). There are third party analytics firms that provide reasonable estimates though[1].
[1] eg http://techcrunch.com/2014/06/23/google-play-quarterly-app-r...
On mobile, Linux is a thin shim of Free software in a proprietary stack.
(Android barely counts as Linux for reasons discussed in my other comment)
Edit: Here is one of the articles: http://phys.org/news/2010-12-air-playstation-3s-supercompute...