Project started to run Erlang on bare metal
kerlnel.org
kerlnel.org
There are major advantages to keeping your grand projects in stealth mode for a while. For one, it's good to wait until you have instructions for even the dumbest ADD teenager to compile and run the thing, or else you'll have to deal with a flood of endless terrible questions. For another, it doesn't exactly inspire confidence to come out and say, "Hey, we want to run Erlang on bare metal, we have README.md, please contribute!"
I fully acknowledge that the creators intended to wait longer and that bandris may have just come across an interesting looking Github repo, but even that proves a point: Your repo should be private until you have something to show. When Linus announced Linux, he already had code running in userspace, with bash and gcc ported!
Edit: I say this having been involved in projects that announced too soon. It's important to establish something concrete first.
The name "Kerlnel" is going to be a problem, because it's so awkward to say out loud. You don't want your name to be difficult for people to say; you want them saying it. "Kerlnel" is a clever pun but that doesn't last beyond the first time you read it.
Guess there could be some gains, but a lot? Enough to bother with?
What about hardware support?
Quite so, hardware is the holy grail. (Alan Kay: "People who are really serious about software should make their own hardware.") I dream of a golden age of experimentation in vertical stacks: specialized hardware designed for specialized classes of application with only so much OS as is needed to support them. Perhaps if the cost of developing hardware falls the way the cost of developing software did, we might see something. Why not an Erlang machine? A Lua machine? A spreadsheet machine?
Enough to bother with?
Order of magnitude is table stakes for interesting, wouldn't you say? Radical experiments demand radical gains. Surely there is room for an order of magnitude if one is willing to sacrifice general-purpose computing.
I think the term 'order of magnitude' has started taking on a connotation of essentially meaning 'a lot'. It's a fair observation, but I hear it bandied about so often that I rarely actually think the parties are in fact using it literally.
My point is that if one is going to build a narrow vertical stack up from specialized hardware, there had better be a 10x advantage over running the application the ordinary way or the experiment becomes a why-bother. Also, the application had better be valuable enough to justify the effort.
This vision of systems design has been alive in the Forth community for a long time – maybe not the "iterating on hardware as part of application development" part, but certainly the specialized vertical stack idea, just in a very austere form. They make the tradeoff of dramatically reducing what the software will do in order to make it feasible to develop that way. That's a tradeoff most of us aren't willing to make. But I have a feeling there are more options if one is talking strictly about servers.
Are general purpose operating systems really that inefficient?
I've thought about "boot into JVM" before, and I think it's enticing for technologists since it's so "clean", but all the projects aiming for this seems to have died from lack of interest (e.g. BEA Virtual JVM/JRockit Virtual Edition).
Edit: Perhaps I should explain where I'm coming from. I work on a high-performance spreadsheet system. One of the things that makes spreadsheets interesting is that their computational model is powerful enough to be valuable, yet not so powerful as to amount to general-purpose computing. Think of a server that doesn't need to do anything but access spreadsheet data, perform spreadsheet calculations, and serve them over the network to some client. Such a server's responsibilities are so specialized that one can't help but wonder how far down the stack one might push them and what one might gain by doing so. I daydream about this sort of thing.
At a previous job I wrote software for a manufacturing company, and it was a real eye-opener to see one of the head engineers there - who had never in his life wrote a program, as we would understand it - modifying the complex ladder logic of a PLC[1] that operated parts of the factory, while I made changes to the software on the controlling PC. I realised that we were doing essentially the same thing, just in completely different spheres of operation.
Another example would be FPGAs, for relatively cheaply one can get a board with such a chip on it, and prototype all sorts of hardware designs essentially by writing software (in VHDL or Verilog). Again I've not done it personally but a friend of mine in smartcard research does this all the time, and doesn't call himself a software developer either even though it's really the same thing, just a different application to the usual general purpose machine.
[1] http://en.wikipedia.org/wiki/Programmable_logic_controller
Targeting xen instead of bare metal sounds better to me. openmirage is doing that with ocaml.
Executives and most managers still had secretaries and dictated letters and memos. The Wang system was revolutionary. A multiuser, networkable word processing system that completely changed the game in terms of the time and effort necessary to produce typewritten documents.
They were supplanted in the 1980s by the more general purpose PC but definitely hold a significant place in the history of business computing.
Or Erlkerlnig, now it's a Goethe reference!
It sounds pretty good out loud if you say it while doing a Stephen Hawking impression.
I fully support making more languages embeddable, but this project doesn't seem to be real. I hope there's more to it than what's listed on their site right now!
Cheers,
Darren
Over this time, my colleague Andrew has been tinkering also, and we seem to have a fair degree of overlap in our end goals, hence kERLnel came about.
It is great to see a lot of excellent discussion, both positive and negative. I think all open discussion is useful, even negative comments have a positive affect in their own way. Although I admit I am surprised that kERLnel was even mentioned here, particularly, as we haven't actually dropped any code publicly yet ;-)
There are other exokernel projects out there that may also be suitable also. In fact, not just exokernels, but also a number of {nano|pico|micro} kernels too, however, the idea of hand coded assembler kind of grabbed me completely for some reason.
For me, the project is not about whether X or faster than Y, or anything like that. It is about what I like doing myself and what I find interesting. I think its great for some many people to comment and have different views, particularly different views to myself - this makes the world a better place!
Cheers,
Darren
It also has the benefit of having actually existing code right now.
Edit: oh, and Erik Quanstrom has started shipping the Nix kernel with his own set of patches in the 9atom distribution, if you can handle the hour+ download time for his ISO. There are some other nice improvements in 9atom, too.
But... wouldn't the effort be better spent improving the existing interpreter? The vague description makes it sound like they're trying to run Erlang without an OS "getting in the way". Looking at the language shootout benchmarks [1], it seems like there's still a lot of room for improvement before the OS would be slowing them down. Operating system services aren't causing Erlang to take 3-25x as long as the C code.
[1] http://benchmarksgame.alioth.debian.org/u32/benchmark.php?te...
Currently, your common options for embedded software are:
* bare-metal with C, C++, or ASM
* load a full OS (i.e. Linux) and run an interpreter to get access to higher-level languages
There are a lot of domains where the problem being solved doesn't require particularly high hardware resources, but requires quite advanced software design. Implementing something like a modern Blu-Ray player's UI and high-bandwidth/low-latency AV in C is rough. You spend a LOT up front on software development costs, and pay more in the maintenance phase. Alternatively, you can save on development and maintenance by running an OS and using high-level languages and existing libraries... but then your board price per unit skyrockets because of the extra processing power, RAM, and flash that you need to support the OS and interpreters.
Access to a modern programming language at a "bare-metal" environment is the holy grain for embedded. It would change the world.
Unfortunately, it has always been practically impossible. And likely still is.
FORTH. Icky, but I thought I'd mention it; very fast bringup time . . . but then it's FORTH.
A commercial RTOS. There are lots of these, some of them are even pretty good.
We went the C-and-ASM route and ported over some USB drivers from another in-house OS. Worked okay. USB is horrible to work with, and that was actually the hardest part to get right on our system.
Also I think this is a good idea in general. With cloud hosting, why run a full Linux OS if you are only going to run a webserver or database shard on it? If the OS could be replaced by a file manager and the runtime code of a single application (with some significant speedup), there are probably a bunch of companies that would be interested.
Room for improvement to do what? Compute n-body simulations. Permutations, or searching a FASTA file, spectral norm? You can't be serious.
The only one of consequence maybe the thread ring Erlang comes 3rd then after Go and Haskell.
As another poster mentioned, going all the way down to the bare metal does not make sense to me.
Or maybe I have lived through too many hardware upfrades to not want to worry about whether the APP will run if we upgrade the RAID controller or something.
Regarding compatibility - my personal long term aspiration is to abstract the HW through LLVM which will eventually allow us to target Xen or other exokernel systems or run more close to the metal by running/developing a true erlang kernel through a mixed ERTS/BMOS (Baremetal OS) code-base.
As mentioned in earlier comments, erlang already has a pretty mean scheduler, and memory management system. Personally as an academic exercise - I'd like to see how this operates at a kernel level and in the future, I'll be spending more time on this. All other existing operating systems I personally believe are just getting in the way of best possible performance.
Finally erlangonxen which you mentioned above is not open source. That does not make sense to me, and it does not give me what I want... Doing this is a very selfish exercise, as it's what I want. Hopefully other people will want (and make sense of) this too, but atm I'm not fussed.
Cheers Andrew
EDIT: To add some context, would it be like http://www.openmirage.org/ ?