Fusion – A hobby OS implemented in Nim
github.com
github.com
Q: has the GC of Nim caused any challenges?
(And if not, would you attribute that to Nim unique GC that does NOT “stop-the-world”?)
"Nim gc" on Google indeed puts you at the 1.4 doc page.
ReadMe, Docusaurus, Mintlify, etc?
Keep in mind that I also need to use ptr (as opposed to ref) types in a lot of cases when working at a low level, so there might be a need for some manual memory management as well.
Basically, OSDev is one of the few realms you can't really take shortcuts in. It's kind of like learning Rust, in that you'll have a lot of foundational work until you get some real payoff. Unlike rust, however, there isn't just some cliff where you start getting it; it's a constant uphill trudge.
Learning the boot process of your target architecture, adding core functionality (process scheduling, filesystem/VFS support, IO management, etc), adding driver support, supporting your video device (just getting a basic framebuffer, and then adding each piece after that), supporting USB, supporting CRT and POSIX (if you choose to do so), etc are all herculean tasks of their own.
That being said, it's a super incremental process, so you'll get to watch it grow and expand at each step.
Reading up on the FreeBSD and Linux kernels are good starts. As well as reviewing other hobby OSes such as Serenity, TouruOS, Haiku, etc. And the OSDev wiki is invaluable.
Also, accepting you probably aren't going to build the next big OS or trying to compete with the big dogs is something you'll have to humble yourself with.
Task switching. It's a very intricate process, you really have to understand how to structure everything (especially the task stack) so that you can switch everything from underneath the CPU as if nothing has happened (the CPU is obliviuous to which task is running, including the kernel itself). Add to that switching between user mode and kernel mode (during interrupts and system calls) and it becomes even more challenging.
> Also, any tips if anyone want to write an OS from scratch, aswell?
As @deaddod said, you need to read a lot. Two invaluable resources for me were the Intel Software Development Manuals (SDM), and the osdev wiki. The SDM can be daunting, but it's surprisingly very readable. The osdev wiki has great content, but it can be a hit or miss. I complement them with various blog posts and code on github to really understand a certain topic.
That being said, the most important aspect of this process is to have tons of curiousity and to be passionate about low-level systems programming. I love to learn how things really work at the lowest level. Once you learn it, you'll discover that there's no magic, and that you can write something yourself to make it work.
[0] https://www.intel.com/content/www/us/en/developer/articles/t...
I like languages that disallow null by default (e.g. Rust, OCaml etc) because it seems to be a huge source of errors.
(since they seem very similar at this point, yet Swift is more battle tested)
- it's close to Python in syntax (less noise, more readable)
- has no garbage collector by default (it uses ARC)
- has great C interop
- can be optimized through the C backend compiler
- can target bare-metal with minimal effort
- supports inline assembly
- has great template/macro system (I do use templates, but I haven't had the need for macros yet)
There's probably other reasons, but those are the ones I could think of now. As for why not other languages, I think the only other languages suitable for this kind of work are: C, C++, Rust, and Zig. Here's my take on each:
- C: The mother of all system languages, but outdated with lots of UB gotchas
- C++: I don't like/need OOP, so why pay the prices of C++ complexity and manual memory management
- Rust: I find the language too complicated for my taste. I know it's subjective, but it just doesn't feel right for me. Also writing a kernel involves a lot of unsafe code anyway.
- Zig: I tried Zig and also found its syntax to be a bit too noisy. Also having to worry about allocators in most of the code distracts from the core logic I'm trying to focus on.
Any reasons why you would not recommend Nim?
Or things Nim could improve?
(Nim seems like this unicorn of languages, that's completely overlooked. And I don't understand why it's overlooked)
There are genuine shortcomings, of course. Every language has them.
It is something of a unicorn though. It's so pleasant to use at many levels of the stack from bare-metal to applications. People even use it for games.
If you wear enough hats to appreciate Nim in more than one of its dimensions, your attention is necessarily split enough to not evangelize so loudly for it in any one. People don't want practical, they want provocative.
__MatrixMan__ and SJMG have good points. If it's anything, it wouldn't be a technical issue in the language. It's like a Tesla when it first came out; it was obvious that EVs are the future, but they had less investment, less adoption, not enough charging stations, shorter range, and not a lot of certified shops for repairs. So naturally a lot of people held back until it became mainstream.
The issue is, will it become mainstream one day? Maybe, maybe not. But I'm betting on it myself.
Q2: What would you do different? (and what unexpected positives did you find)?
Yes. It's a pleasant language to work with.
> What would you do different? (and what unexpected positives did you find)?
A couple of things I think need improvements are: (1) better IDE support (especially for JetBrains IDEs), and (2) better support for true sum types and pattern matching[0].
As for unexpected positives, I found that the standard library covers a lot of functionality that I rarely (or ever) need a 3rd party package. Maybe that's because I'm not doing anything exotic.
[0] https://github.com/nim-lang/RFCs/issues/548, https://github.com/nim-lang/RFCs/issues/525
Some day I want to write an RTOS in Nim. I enjoy writing embedded programs in Nim and it’d be fun to make an RTOS.
For this to work properly, user tasks need to be able to respond to completed syscalls async as well. That's why my idea of user tasks is that they should be modeled as state machines, with channel-based events as the core mechanism by which code gets executed in a deterministic manner. The equivalent of signals in Unix (which many find one of the bad aspects of Unix design) would be receiving events on one or more channels for various purposes (e.g. IO completion, timers, abort, interrupt, GUI events, etc.).
Good minds think alike, I guess.
But as a sibling comment says, this is essentially what io_uring is. Read Lord of the io_uring [1] if you want to know more. Polling mode is the key.
The reason I mentioned it after elcritch's RTOS mention is partly that the way the little interpreter has no backward jumps means there are no loops and so no halting problem issue. So, you can still embed conditional error handling logic in system call batches, but the overall call is bounded by the time of its component calls (that is, if they are bounded, anyway...). That might be a useful property for higher levels of the system to guarantee meeting a real-time budget in many situations with very low complexity. I'm not sure if any of this is original with that github repo, but I haven't seen it before in this specific context.
Perhaps the most complete example of "adding a new sys_batch-based syscall" is https://github.com/c-blake/batch/blob/master/examples/total.... which adds a `mapall` that can mmap a whole file or fail trying (at a couple points) for the purpose of just totaling the bytes as an example calculation.
I'm hoping these questions aren't too basic, I have no context whatsoever for understanding this so hope someone can explain.
> Why Nim? It's one of the few languages that allow low-level systems programming with deterministic memory management (garbage collector is optional) with destructors and move semantics. It's also statically typed, which provides greater type safety. It also supports inline assembly, which is a must for OS development. Other options include C, C++, Rust, and Zig. They're great languages, but I chose Nim for its simplicity, elegance, and performance.
As for the overall design goals of Fusion, I have high ambitions, which I list on the same page I referenced. I don't want to build another Unix-like OS; I'd like to experiment with fundamental issues in OS design, such as using a single-address space and capability-based security for protection. Another aspect I'm trying to explore is how processes/tasks are modeled, which I believe should be modeled as state machines with statically-typed channels to communicate between each other (this is not new, it's been done in Singularity OS[1]). There's rudimentary support in the kernel for channels and using them from user space, but it's still early.
[0] https://0xc0ffee.netlify.app/osdev/01-intro.html
[1] https://en.wikipedia.org/wiki/Singularity_(operating_system)
My idea is that processes should compose in a statically typed manner, where opening a channel for reading gives you a Channel[T] to read entities of type T. Two sub-abstractions of Channel[T] would be Source[T] and Sink[T], where they can be used to read and write to any source/sink (including files) as long as there's a registered (de)serializer for T.
Have you played at all with nushell? It's fun in a composing-processes-via-types kind of way, although it's a bit more on-the-surface than what you're describing.
My impression is that much gets done via builtins--you have to start writing nushell plugins for everything if you want to extend the fun to arbitrary programs which nushell knows nothing about. (unless you're happy with json I/O, but you're talking about static typing here).
It sounds like the source/sink type registry that you're describing would solve that problem in a much nicer way.
My idea is that everything in the system should be able to use statically-typed channels, including syscalls and calling into system services. This opens up the possiblity of, for example, composing GUIs in a manner similar to writing a shell script.
I'm rooting for you.
https://www.ethos-os.org/~solworth/petullo14ethosTypes-20140...
As for the ecosystem, yes, it's not as big as Python or Rust, but surprisingly the standard library has most of what people need. I rarely look for 3rd party packages to do something.
That being said, I acknowledge that Nim is on the lesser known languages of the spectrum, but that doesn't take away from its merits as a very promising language that does what it's supposed to do very well.
One thing I think the community should focus on more is IDE support. The VSCode extension is good, but has some rough edges. I also prefer JetBrains IDEs, and the official Nim plugin is very lacking to say the least. I have another side project to create a JetBrains plugin for Nim[1], but I haven't gone far with it yet.
Now I'm playing around more with Nostr (and the Lightning Network)... Nostr libraries for Nim are not as complete or well-maintained as those in Rust, Go, or even Python, etc.
I'm not letting that stop me from using Nim for my projects... I love Nim! But it does mean I have more work to do (and code to maintain). I can make that choice because I'm my own boss and run my own company. But I could see others not making the same choice for rational reasons.
* And dom96 left, unfortunately, because of harassment and abuse, which is another possible reason why Nim isn't as well adopted as Go, Rust, etc. If people want to see Nim succeed more, they also need to focus on improving community safety, too. https://news.ycombinator.com/item?id=38999296
I'd better discuss the deeds.
Is_land == island == IsLaND == is-land
It is bad in team setting, in real world projects.
How it goes now ? Last time I checked the main dev refuse to do anything about against popularity vote In Github.
Otherwise awesome project and documentation Fusion Os
So:
FooBar != fooBar
FooBar == Foobar
Most of the developers in the community are ambivalent about it, because it rarely ever causes problems. If you end up misspelling an identifier, you're nearly always going to get a compile-time error due to static typing anyway.
It isn't a good look that they made the same mistakes as PHP.
I personally don't like that you can have "is_OK" and "isok" and "is_ok" in the same code as three valid different things. Or having "GL_FLOAT" and "GLFloat".
Both options come with tradeoffs. Don't jump so quickly into "that is a mistake" and allow the devs the benefit of the doubt. There is only one rule you need to remember in Nim: only the first character case matters.
As menctioned, a few years ago Selenium had its methods in camelCase. If your code used Selenium, it had to be camelCase and snake_case mixed. When Selenium standarized, it forced everyone to switch to snake_case.
Nim puts great effort in FFI. It means you can easily use C libraries, using their names, even if the case doesn't match, and your code is still coherent. Look at the sample code in https://www.py4j.org/index.html : why do they end with "random.nextInt(10)" in their Python code? Didn't Python had this solved? Not saying that this is the end of the world, but the Nim way is not a mistake either.
Even just reading your foobar example at a glance took a moment for me.
And case insensitivity is also generally frowned upon. To have a language with both sensitivity and insensitivity is the worst of all worlds with none of the benefits.
If you want to understand why at a deeper level I would recommend reading readability or the case insensitivity sections in any programming languages book. Personally, I enjoy Programming Languages, Principles and Practice (Louden & Lambert)
EDIT: Yes, I get it, it doesn't affect YOU. But it doesn't mean it doesn't affect other people. Non-english languages and/or speakers are an easy example. It also eliminates a whole class of human error, and maybe that only affects non-experienced juniors, but they exist too. There are other issues with symbols being case insensitive and string values being case sensitive. If you want a practical example a classic one is HttpsFtpConn vs. HttpSftpConn
All I have personally experienced of case sensitivity is an added layer of friction any time I go to use a REPL for Bash/Python/Javascript/etc or some awful ‘allowercasewords’ gets cemented in place barring a total refactor since you can’t correct files piecemeal.
And case sensitivity in the language doesn’t even help with case sensitivity at the OS level when you’re writing cross platform code =/
Like cardanome said, in practice it's awesome for FFI.
…How? Do you find code more readable when there are two different names that differ only in the capitalization of a non-first letter?
Also since nim is very ninche and used by very little perscentage of the world they haven't encounter much of production scale coding. Well that may be reason nim never get pass weekend hobby projects..
FooBar != fooBar
FooBar == Foobar
That could cause a lot of head ache in large code base ..
It is a pretty amazing feature. Your problem is just imaginary. A consistent case style should always be enforced regardless if you have a case insensitive language or not. There is no real world case where you would want is_land and isLand to exists both in your code and be separate variables.
How it would effect? I haven't touched nim since that decision to keep those insensitivities.