Hobby kernel and userspace, built mostly from scratch | Hacker News Reader

Hobby kernel and userspace, built mostly from scratch

github.com

154 pointscarlesfe26 comments

Back when I went to UIUC the SIGOps chapter had a "Roll Your Own Operating System" series (mid-90's)[1]. I'm glad to see they are still doing interesting things.

[1] Still available: http://www.acm.uiuc.edu/sigops/roll_your_own/

Is ToAru-OS a reference to raildex?

I would love that. But "to aru" also actually means something in Japanese, which I would translate like, "there is an". Disclaimer: I definitely am not an expert on Japanese.

It's actually kind of hard to translate concisely, IMO the phrase that best captures the meaning of "ToAru-OS" is "There exists an OS". But it's kind of awkward.

If we used the Raildex translation, ToAru-OS would be "A Certain OS".

Yeah, I think something along the lines of "an existant OS" is an acceptable translation. "To aru" has a subtle connotation implying that it is a standalone thing in its own regard. It's a pretty clever name for a hobby project like this, IMO.

Sorry for going off-topic here to the thread, but ...

As someone studying the language, can you explain what the と portion of the name means? I am aware of the ある (有る) portion, "to exist", and that と is being used here to represent a clause, eg "Xと言った" (X said), "Xと思った" (X thought).

But in this case, there's nothing before とある, so why say と at all in this case? Why not just say あるOS?

I think it's stylistic. The anime series (To aru majutsu no index) uses the same style of "to aru <noun>" instead of "<noun> to aru". It reminds me of Latin, where you can adjust the position of phrases in the sentence for emphasis or rhythm.

Personally, I think of the "to" as emphasizing the specificity of the phrase, if that makes sense. It's not just that "an OS", it's this specific OS that's being referenced by "aru".

Ah, most interesting. Didn't think you were allowed to ever move "X to <verb>" to "to <verb> X", but great to know that this happens. Thank you for the explanation!

According to the FAQ on the wiki, the developper is a fan of one of A Certain Scientific Railgun. https://github.com/klange/toaruos/wiki/FAQs

Anyone with this type of hobby kernel experience, how hard would it be to build the "Ruby on Rails" or "NodeJS" of kernels?

If hard, why so difficult? I'm more referring to a theoretical reason, rather than practical.

Practical Reasons: - drivers are required - C/ASM is hard (ie lack of contributors) - different hardware architectures need to be supported. That part I get.

Why in theory can you not have an operating system framework that allows people to "relatively easily" build a custom kernel (instead of using Linux) the same way something like Go allows me to "relatively easily" build a custom HTTP server (instead of using Apache)?

You can look at OSkit as one attempt to do this. I think the overwhelming reason its not common is that no one wants to build an OS. If they do it's for the educational experience not practical use. This means hobbyists aren't interested in just using a framework.

I am not sure what exactly is meant by "Ruby on Rails of kernel". Probably "an opinionated framework to build your own kernel". In this case OSkit is probably the best answer to the question. Maybe look at micro- and exokernels.

I think another problem is that an OS is not analagous to a website, it's more analagous to a web framework. And it's not like there are any framework building frameworks floating around

I've recently spend some time playing around with the xv6 kernel (more specifically a 64bit port of it), one problem is that the amount of work necessary to get a reasonable clean hardware abstraction for a x86 system is fairly complicated. For example for a full acpi implementation, you either have to write ~50 admittedly short glue functions for ACPICA or implement acpi table reading and an AML interpreter yourself. Unfortunately if you want to use modern features of PCI, there is no way around using acpi.

That being said, there are several such "modular" kernels implemented on top of Xen. For example Mirage is a framework for creating free standing Ocaml applications on top of Xen.

You can using the NetBSD rump kernel[1].

You get all the NetBSD drivers, IP stack etc that you can use, just add memory management, setup and so on, which is not much work, and link your application in to it. It is still under development but the basic stuff is all working well.

[1[ http://rumpkernel.org/

Even if you just want one platform, there is a lot of machinery you need even to boot up, load something from disk, access a USB keyboard and mouse, and display some text. ACPI and UEFI add considerably to this.

Then there's bugs and debugging. If you have a bug in your disk driver you can slowly lose data without noticing. Quite a lot of other bugs will lock your system and you need to debug painfully from whatever logs you can find. Building in a non-C language might make this a bit easier but won't save you from the hardware locking up.

Once you've done this, what have you gained? You have an OS with no userland and no applications. What bits of Linux/BSD do you actually want to replace with something else? The code is fairly modular and OSKit exists to help with this.

The nearest to what you describe is probably retrocomputing; people building their own considerably simpler systems from the ground up. If you want a simpler platform to start on without having to first build the platform, consider Raspberry Pi; the bootloader there effectively just pulls an ARM ELF executable off the SD card and starts executing it. Have a look at https://github.com/dwelch67/raspberrypi

I used to hang on some hobby osdev boards many years ago (losethos was there, when it was still called losethosOS). I never got much further than loading the kernel, setting up protected mode, printing some text and so on, but it was an interesting crowd. The long story short is, building a day-to-day usable general purpose kernel from scratch, alone, is next to impossible.

First have to decouple Go from its runtime before you could build a kernel in it, not to mention everything else that goes w/ writing a kernel. Similarly, you can't use significant parts of C++ when building a kernel w/ it.

It seems like a lot of people are looking at Rust as a system programming language, though... (google rust kernel)

Writing OS's is hard and there are no shortcuts. You should take a look at microkernels and the work Andrew Tanenbaum did with Minix 3 [1] and his modular design as this is the sort of direction you're probably looking for with customisable OS's.

[1] https://www.youtube.com/watch?v=bx3KuE7UjGA

Because building a custom kernel is like building a custom implementation of a language and a runtime.

Even though you can easily implement custom HTTP server you cannot easily implement full custom Go runtime.

He would spend so much time in the UIUC ACM office working on that thing. Pretty impressive for an undergrad ;)

I found the Bim editor interesting; particularity for the simplicity of its implementation.

how does it provide the opengl driver?

"The following external libraries and tools are retrieved during the build process: ... Mesa - Implementation of OpenGL (only the software rasterizer is available)."

From the readme

This is amazing! I've been dying lately feeling an overwhelming sense of disappointment with modern day OSs and their UX. To the point where I've wanted to start to play around with making my own.

Could anybody point me in the right direction? I'm a NodeJS web developer HTML/CSS guy. What would be the shortest/fastest path to me being able to write my own entirely custom UX?

Preferably something I could do in HTML/CSS/JS and be able to run on a desktop/laptop/phone. Thanks.

If you want to create your own complete graphical rendering system you could look into writing a replacement for X11. You could look at the source of tinyx to help you get started. Note that the 'tiny' x source is 5mb of plaintext.

You could also look at building a graphical tool-kit (think GTK/QT) that can be used to render windows and their styling.

Finally, you could look at building a window manager with a bunch of applications which look the way you would like it to. This is probably the most achievable goal, but it also offers you the least flexibility with your rendering.

A couple of other points: - I would definitely recommend targeting a non-mac *nix environment. My understanding of the situation is that on Mac and Windows their is little support for external renderers and basically none for custom window managers. - Most people would consider JS is to be a pretty bad language to write one of these systems in. You will be able to use JS as the end user application programming interface if you like (a la Gnome 3), but you will likely have a tough time writing any of the three systems mentioned above in pure JS. HTML and CSS will likely only be useful for the front-end that you expose to the end users.

Finally,

> What would be the shortest/fastest path to me being able to write my own entirely custom UX?

In the field of hobby OS/systems programming the 'shortest and fastest' route is a very bad attitude to take. Building anything in this area takes time and patience, if you skim on either of these you likely won't get far.

I hope this is helpful -- good luck! Sam

complete graphical rendering system

You can get a lot smaller than 5mb source on that if you're willing to ditch the X protocol. You'd end up with something like RiscOS, Amiga Workbench, PC GEM, or very early MacOS.

> shortest/fastest path to me being able to write my own entirely custom UX

Take your existing windowing environment and language tooling and ask it for a fullscreen framebuffer. Draw in that. You can be up and running in minutes.

> Preferably something I could do in HTML/CSS/JS and be able to run on a desktop/laptop/phone

.. oh. That requires an entire pile of browser machinery and is very far from 'bare metal'. Fullscreen node-webkit?

Hehe, I have talked to Kevin on irc before about his custom compositor. He hangs out on the freenode irc channel #osdev sometimes, iirc. Really nice dude. I am very surprised this hasn't already been submitted here before. :)

Reply on news.ycombinator.com