C# for Systems Programming | Hacker News Reader

C# for Systems Programming

github.com

228 pointspjmlp153 comments

Unity has been using C# for low-level code for years, and they have even gone as far as compile C# code into shaders. It's surprisingly flexible, and unsafe low-level code composes well with high-level code quite well.

You can really tell, though, that the language wasn't meant to be used this way. Some common annoyances are:

1) C# is unable to declare fixed-size arrays in structs. There are ways around it, but they are all more unsafe, require code generation or hit the GC. If you've ever written kernel code, you know that fixed-size data members are really common, so this is more annoying that you might think.

2) C# types are either always references (classes) or always values (everything else). There are reference types, but they are restricted so that, for example, the array subscription operator cannot return them. This means that you can't directly operate on values in an array. Once again, there are ways around this, but they involve pointers, which pull in a bunch of the "unsafe" language.

3) The standard library makes a lot of choices that, in hindsight, are extremely unfortunate. Data structures carry generation numbers that increment on various operations and invalidate iterators. This makes a lot of operations that are sublinear in C++ take linear time in C#. You pretty much can't use the C# stdlib for any low-level code.

The projects that successfully use C# in low-level code pretty much all roll their own containers, stdlib and a module that implements black-magic stuff, usually called something like UnsafeUtil. On balance, having written a bunch of lower-level C# code, I'd still slightly prefer it to C++ for (e.g.) talking to the GPU driver, but there's no denying that that's a very low bar to clear. For general kernel code I wouldn't use it.

The 4 languages I would consider for low-level code these days are, in order:

- C/C++ (footguns galore, but lots of existing literature)

- Go (surprisingly easy to control the GC and get it to do what you want)

- Rust (the Rick & Morty of programming languages - not a compliment)

- C# (See above)

Could you expand on this a bit: "- Rust (the Rick & Morty of programming languages - not a compliment)"?

Rick&Morty being a very long going (and excellent) parody version of Back To The Future franchise I'm not sure what qualities this characterization imbues on a programming language and would like to hear your views on the matter (since I found your analysis of C# very much agreeable).

Honestly, I don’t feel qualified to evaluate Rust. At face value, it seems like a good replacement for C++, but I’ve never successfully used it for anything larger than a toy. That’s probably mostly on me.

My comment was referring to the impression that people who like Rust seem to think that you either also accept Rust as the best thing ever, or you don’t get it. I probably shouldn’t have mentioned it, because now the thread will be about Rust.

"people who like Rust seem to think that you either also accept Rust as the best thing ever, or you don’t get it"

I don't think Rust is in any way unique in having people who think that way...

It has got so many people from ruby and js crowds so there will be statistically more of them.

Probably not, but it's a question of degree. The projects I work on don't get monthly requests to drop everything and rewrite them in Haskell/Clojure/Go but we do get them for Rust.

An anecdote: I recently opened a position on my team, and got two applicants on the same day who ignored the project description, and basically said we should hire them to tell us how everything we do could be better if we rewrote multiple projects in Rust.

Of course it's completely possible for a programming language to be both (A) really well-designed and (B) have annoying people among its community. I suspect that's the case here.

Every language that has a reputation of some kind purity in one or more characteristics (Haskell, Lisp, Rust) is bound to attract some people that values purity over practicality.

It's not the case of the rust dev team, which seems very balanced. But I've met a few rust enthusiasts that match the worst of the fancy FP or Typing fan boys.

Those Haskell people do talk a lot about purity.

Of their precious bodily fluids?

Not sure why you're being downvoted. That is a brilliant reference to a brilliant movie.

I personally have language fatigue after playing with various languages over two decades.

The thing I hate about rust is abbreviated keywords like “fn” and “mut” because it sounds like fricken mutt. A function is a function not a fn, and is it really so hard to type “mutable”?

There’s more important things in life to worry about than rewriting codebases in language Z.

An old habit of mine: Avoid symbol-speaking or abbreviation-speaking. I verbalize "fn" as "function" and "mut" as "mutable" the same way I verbalize "std::unique_ptr" as "standard unique pointer" and not "stid unique pooter".

How would you verbalize "=>" ?

Edit: Fat arrow?

"fat arrow" would just be symbol-speaking. Assuming you are referring to the match syntax in rust[1], I verbalize "PATTERN => EXPRESSION" as "on $PATTERN, do $EXPRESSION".

I don't read code in English, even though I'm a native speaker. So its kindof a silly question: this would only come up when walking someone else through the code. I internally read code using data-flow and control-flow graphs. The written abbreviation is just shorthand for a specific type of graph vertex (or edge).

[1] https://doc.rust-lang.org/book/ch18-01-all-the-places-for-pa...

I was more thinking of C# where "=>" is the closest thing I can think of to "function".

There is Func<T> - that's a delegate type.

I verbalise that as "goes to". In the case of, say,

  things.Select(thing => thing.Stuff);

I verbalise the brackets as "thing goes to thing dot stuff".

“Goes to” is what Prof. Elsa Gunter teaches in the PL theory class at Illinois

Something like (x, y) => x + y can be verbalized as "lambda of x and y producing x plus y".

I never understood the push for terseness in newer languages. Are there any IDE's or text editors today that don't do code completion? Code coloring helps too. Usability is improved with full words imo...

I find it hard to believe any language would be so silly as to call functions "functions" in actual code. Fn or fun or gtfo.

Why not skip the keywords altogether and use an ACTUAL λ symbol. Funny thing we have UTF8 everywhere but no one uses it… People are still typing ASCII in markdown files, but there’s Unicode symbols to render tables and whatnot directly in a text editor… but here we are :)

RIIR is a thing. Also, the Rust Evangelism Strikeforce.

I love Rust, but I think it does go over the top, more so than other languages.

Sure, but I think I've heard far more about RIIR and the Rust Evangelism Strikeforce than I've actually seen it in action.

It definitely is a thing, but those people are spending their time being loud and annoying while other people are quietly releasing libraries and shipping products.

Also, give it a few years and the loud and annoying people will have moved on to being loud and annoying about something else.

So it goes.

It's true, Rust is just the most recent inductee into that clique. But it seems to work kind of like the Sith: there are never more than two that are really active at any one time.

Rust is not perfect and will never replace language like c# go or python. But it’s the first language that is a true replacement for c++ because it can do everything c++ can do but in a type-safe and thread-safe way.

It can't do exceptions or inheritance

It is more a C replacement

Is inheritance really that critical a language feature?

Rust also can't overload operator&& and lacks the "macro pre-processor" that was already a bad idea in the 1970s. Absence of specific footguns is not something people will mourn.

To the extent that you should have exceptions in your software, Rust's panic picks that slack up pretty well, and to the extent you should have error handling C++ exceptions were a bad idea and sum types like Result are much better.

Note that although Rust's types don't have inheritance the traits they can implement do. Which is much more likely to be something you actually wanted than say, C++ multiple inheritance.

It can do flow-through error handling and code reuse. Exceptions and inheritance are just two particular ways to achieve those goals.

The world runs (mostly) on C and C++. Those languages will be around, and continue to run the world, for a very very long time.

Well, those languages do tend to attract people who deal in absolutes...

"There are only two kinds of languages: the ones people complain about and the ones nobody uses." -- Bjarne Stroustrup

https://www.stroustrup.com/quotes.html#:~:text=%22There%20ar....

I have a very similar problem. Lisp is so compatible with the way I work, and I'm so productive in it, that I'd rather use it for everything -- even a simple Scheme implementation that lacks things like CL's condition system or "update all instances when a struct or class definition changes" magic. So I have a tendency to be very squarely within Smug Lisp Weenie territory when comparatively discussing PLs.

Well, I personally enjoy using Rust, but Rick & Morty is in my humble opinion awful. Pretty much the poster boy for the "I'm so smart and witty" crowd.

Rust is the good kind of smart. Actual functional solutions to difficult problems. R&M is the cringe-worthy kind of smart. Built specifically to please the viewers personal ego.

Dismissing people who enjoy Rick and Morty just because of an internet meme is kind of intellectually dishonest and falling for the same sort of limited thinking. I assure you the vast majority people just enjoy the show and don't feel superior to you because they enjoy it.

It's really more about how cynical and calculative the entire show is to manipulate the fragile ego of its target audience. And also: this is just my opinion. I'm not presenting it as some immutable truth.

both have fanbases with a very outspoken subset who come across as obnoxious because of their blind, proselytizing devotion without much substance. unfortunately, this subset of the fanbase is the dominantly visible component to outsiders.

It's a reference to the "To be fair, you have to have a very high IQ to understand Rick and Morty" meme. GP was probably referring to RESF style advocacy, which divides the world into smart people who obviously know that Rust is the right thing, and smoothbrains who have stubbornly resisted conversion.

Some of this isn't true anymore. For example, #2 is wrong: You've always been able to get a reference to an array element (both at an IL level, and at a C# level). It used to be somewhat awkward - you would need to pass 'ref arr[x]' to a function - but you can store references in locals now.

Agreed on #3, a lot of the BCL is just miserable in terms of design. There have been improvements lately but I'm still stuck using my own custom containers for various cases.

One good move the BCL and runtime teams have made over time is to just expose more unsafe primitives so you can easily build your own stuff - the Memory and Unsafe classes give you lots of useful tools, Span<T> and Memory<T> let you safely work with memory allocated in various ways, and various BCL stuff has been updated to accept Spans instead of requiring you to use managed arrays.

Thanks, I didn’t know about the expanded use of refs. As the other commenter pointed out, the version of C# used in Unity is out of date by a version or two, and so is my experience.

Considering we're close to C# 10 by now, it's ... a few more versions ;-)

Hard to blame Unity here, though. They have written their own compiler backend that converts CIL to C++ and that was back in the .NET Framework days when performance wasn't that high a priority for language and framework design.

Their transpiler would probably be up to date if they open-sourced it, since plenty of other people would be using it. Instead people are stuck using CoreCLR's AOT backend or other closed transpilers like BRUTE.

Their transpiler also had a lot in common with ones for other targets (I know for a fact that in the past it used to be very similar structurally to JSIL) which probably improves the likelihood that they would have been able to leverage community knowledge and contributions here.

It's understandable though that they want their transpiler and C#->shader compiler to be their secret sauce, though, since they see them as unique advantages over Godot, Unreal, etc.

Can you please explain why #3 is bad for system programing?

Why is Rust the Rick and Morty of programming languages? I've wanted to learn it for ages (mostly because of bevy, but it looks nice itself), but lately I've been hearing not so good things about it

Like the TV series, some of its "fanbase" are a bit obnoxious and self-important. They're a bit like people who'll fight a Middle Ages worth of religious crusades over vi/emacs vs graphical IDEs or *nix vs Windows rather than just using what works best for a specific scenario.

The language itself is really cool, I'm a big fan even if I've not had many chances to write it professionally. It's a really well thought out language, I wouldn't use it in every single scenario but there's a lot of legitimate use-cases for it (especially if you need lots of performance but don't fancy the footguns of C++). The learning curve is quite steep but it's nowhere near as bad as some people make it out to be.

I guess every language comes with its negative connotations. I've heard languages like python and lua referred to as "baby's first language" because they're compartively easy to start out with, C and C++ being for the kind of people who like to talk about how they use it more than they actually use it, the list goes on.

And of course all of them will fight over the specifics (in the case of python, I've been guilty of this when it comes to tabs vs spaces).

I agree although having said I think there's one exception to this rule which is Kotlin, I use it a lot but I've never heard of a dedicated Kotlin hater or evangelist.

The difference is that normally when you make fun of a language like that, it's mostly a joke. You don't normally use it to completely dismiss a whole argument that could actually have real substance.

The only other langauages I see people this with, are VB & PHP (for somewhat different reasons).

I used to love C but it's so hard to accept the possibility I'm going to compile something totally unsafe after using Rust. Do you think Rust's ecosystem isn't going to amount to anything, or that it's just a meme, or what do you intend with the Rick and Morty analogy?

I think Rust kind of aged C.

It happens when some new band push the needle for a genre, its only when this happens that the "modern" stuff start to look old.

C is getting deprecated while C++ will need to catch up somehow (if there's a way).

This is actually a good effect of Rust, making people think more that safety is important and that its possible to have more of that in languages without a GC.

(Sorry for hijacking the question, that was not directed at me, but this is a good debate so i couldn't resist)

Many of those caveats have been improved since C# 7, blame Unity for not keeping up with .NET versions.

https://docs.microsoft.com/en-us/dotnet/csharp/whats-new/csh...

Oh, thank you for that. I’m really happy to hear there’s been movement on this front. Hopefully Unity will get there eventually - I suspect their own solutions to some of these problems might be holding back adoption of more recent .Net versions, at this point.

They claim an upcoming project is to upgrade to .NET 5 (though at this rate I would more expect 6).

yes, just as you imply, don't hold your breath ;)

I imagine they got tied up in all the .Net changes between 2016 and now. I wouldn't be surprised if they were just waiting for Microsoft to pick a path long enough to have a target.

.Net 6 of makes sense because it's supposed to be an LTS version. However, if the release and EOL projections[0] I don't think it actually matters all that much: "LTS releases get free support and patches for 3 years. Current releases get free support and patches for 18 months."

That feels fairly aggressive for a library and language for business applications. I imagine many projects won't be released until the version of .Net used to build it is EOL.

[0]: https://dotnet.microsoft.com/platform/support/policy/dotnet-...

I think all of the issues in #2 might have been resolved with changes to ref keyword and ref-structs added, both in C# 7.

no Zig nor Nim nor D but you include 2 GC'ed languages as suitable for "low-level code"???

Because they have been proven in the battlefield?

https://www.f-secure.com/en/consulting/foundry/usb-armory

https://tinygo.org

https://www.wildernesslabs.co/

You know, it's interesting to me how much people fixate on garbage collection.

Taking Go as example, there are many pitfalls to using it in low-level situations, but the GC is, in practice, never a source of significant issues.

The real problems are:

1) The foreign function interface (CGO) requires a context switch in both directions. (Not because of GC, but because of Go's threading model.) A lot of work ends up being in trying to avoid calls between Go and C in hot loops.

2) Especially talking to the kernel, there are surprising sharp edges in the ABI and C calling convention. I once spent a week trying to populate a bit field correctly. (But other languages would have the same problems matching any C behavior that's left up to the compiler.)

3) The lack of memory fences is a big deal when trying to use kernel-managed ring buffers.

The GC, comparatively, is problem-free for two reasons:

1) A lot of embedded code uses static buffers anyway.

2) The problem of managing memory doesn't go away just because you're in ring0. The Linux VFS, for example, is heavily garbage collected, it just uses a jerry-rigged GC consisting of counting references. I'm sure if I make a categorial statement on this someone on HN will take me to school, but I think a real GC might be a net performance win in some places in the Kernel, like the VFS.

Finally, people often bring up that GC is fine for throughput, but not for latency. Fair enough, but manual reference counting is also not better: someone has to call free at some point, and if you happen to draw the short straw in a blocking system call, it still affects latency. The amortization behavior is probably different, but not in any way that I think matters to most people.

I can't make categorical statements, either. But here's another concrete observation to offer as evidence that things are not so clear cut: in generational GC, heap allocation is typically O(1). A heap allocation is typically O(n), where n is the size of the heap.

Of course that difference also tends to go away if you're using static buffers.

Maybe at small scales like embedded, but Go's GC has proven problematic even with static memory allocation. Some company (Discord?) ran into issues where the large heaps were scanned every 2 minutes because Go refuses to go longer than that w/o a GC check even if it has no reason to believe it needs to.

> Some company (Discord?) ran into issues where the large heaps were scanned every 2 minutes because Go refuses to go longer than that w/o a GC check even if it has no reason to believe it needs to.

https://blog.discord.com/why-discord-is-switching-from-go-to...

"After digging through the Go source code, we learned that Go will force a garbage collection run every 2 minutes at minimum. In other words, if garbage collection has not run for 2 minutes, regardless of heap growth, go will still force a garbage collection."

Discussed here at https://news.ycombinator.com/item?id=22238335

Yup that's what I was thinking of, appreciate you digging up the article detailing my point.

The go team have done amazing work minimising mutation pause times. I'm skeptical I would ever be comfortable using it for code that close to the metal, but if any mark and sweep garbage collected language is going to one day reach a point where that's solely prejudice on my part rather than part prejudice and part technological realism go is probably the most likely candidate.

D is excellent for systems programming

Echoed my thoughts pretty closely here. My thinking has always been "it's a language and a runtime". I found your bullet points interesting and realized it's been a really long time since I've done any serious systems programming...

1) You mention there's a workaround -- I can't recall exactly what it is but I think an attribute added to the array can be used to force this behavior -- that's a language limitation, for sure, in that there's effectively no way using language syntax to declare that (and no way for the compiler to verify that)

2) The latest version of the language blurs the lines here in a few ways -- using ref types allows one to "directly operate on values in an array" without resorting to unsafe.

I think there's really two things to consider in these cases: (1) Can C# handle the requirement at all? (2) Can C# handle it in a way that is compiler verified, or runtime verified without the use of `unsafe`?

The answer to the first question, in my years of working with the language, have always been "Yes". Ultimately, I can find an attribute or fall back to unsafe and do what I need to do without the compiler's help, but the frequency with which I've needed to do so has been "very close to never"[0]. The answer to the second question is often "Yes", too, especially in the most recent versions. The few cases where I have had to drop to `unsafe` for a performance critical piece of code, I have been able to replace with ReadOnlySpan/Span in every case I've encountered, for example.

[0] But I don't write systems software in C#

I absolutely adore Go, finding it to be an imperfect language that nonetheless yields powerful, maintainable solutions. It is the ultimate duct-tape language, allowing one to build robust, performant systems using a massive library and fantastic C interop where necessary. Yet I wouldn't consider it for "low-level" code. Garbage collection alone invalidates it in that regards (religious war: reference counting beats GC every time), in the same way that C# and Java aren't credible low level languages.

You aren't going to see C# or Go used in the Linux kernel.

"Rust (the Rick & Morty of programming languages - not a compliment)"

You mean something that is good to great but that "outsiders" or late adopters feel unexplainably threatened by and thus feel the need to attack it and its admirers? Where ironically the detractors are louder and more numerous than the advocates ever were?

Rust is particularly susceptible to this effect as it has been racking up wins and growing in influence, making people feel insecure about their current skillset/knowledge (always a weird motivator for rhetoric in this industry).

I have only dabbled in Rust a few times. It feels very unintuitive and "write-only" from my 1000 foot, ignorant perspective. But I realize that's on me, and appreciate the focus and purpose of the language.

Rust today seems like a 2013 Tesla Model S. A couple of years ago it was the GM EV1: It used to be largely promise of a future to come with a loads of downsides, where today it's expensive to get started with, has a lot of incompatibilities, but it's superior to most alternatives. Tomorrow it and those like it will be the norm.

Rust is bloody great, the type system is so good. My thesis code has a bunch of type conversions, I managed to emulate 'do' notation (Haskell) of sorts via the type system, but I can now also convert between types really easily. So bloody happy with it.

for example `n.unbind().bind()` can convert to a different type but `inside_n = n.unbind()` will let you work with the data inside the container `n` and after you are finished you can call `inside_n.bind()` to lift it back up to the container type.

> You aren't going to see C# or Go used in the Linux kernel.

Nor C++ or Ada, so what? Linux isn't some kind of golden rule for all OSes on the galaxy.

I am more than happy to do bare metal coding in C# (Meadow) and Go (TinyGo) without having to deal with Linux.

>Nor C++ or Ada, so what?

Volumes could be written as a reply.

>Linux isn't some kind of golden rule for all OSes on the galaxy.

Just about every operating system or major piece of system software is written in C, or limited C++ that is defacto C. Rust has a universe more of a likelihood to supplant those than Go or C#. And in fact is the only modern language making any headway in those cases. We're about two decades past people deploying the trite "Oh well that's just legacy inertia..." argument.

Google didn't write Fuchsia in Go, and hasn't used an iota of it in ChromeOS or Android. Microsoft uses somewhere approximating 0% .NET/C# code in the Windows core. Microsoft research made a toy .NET OS once that, unsurprisingly, went nowhere, just as you can find irrelevant toy example projects pushing the same premise, but they're dead ends.

This is of course meaningless for the vast majority of projects that the residents of HN are working on, where the compromises of Go, C#/.NET, GCs, etc, are massively outweighed by the benefits they bring to those sorts of projects.

>I am more than happy to do bare metal coding in C# (Meadow) and Go (TinyGo) without having to deal with Linux.

Okay?

Waiting for that volume to arrive.

As for the rest it would be a waste of my time on Earth to prove you wrong and change your mind.

Okay?

Yes, that is okay. There is little you could say on this that would be convincing given the every action of Microsoft, Google, and ranks of proven, competent developers who are building so-called low-level systems that power the world. I concur that it would be a "waste of time".

After all, I am willing to waste some minutes with you.

Since IBM i, IBM z, Unisys ClearPath exist, and power the world of insurances and finance, this sentence is obviously false.

> Google didn't write Fuchsia in Go

Indeed, only the TCP/IP stack and volume management tooling, should be easy for you to find out about them.

Then there is this thing called gVisor, it is powering parts of the world that run on top of Google Kubernetes Engine.

Maybe something there for you to learn as well.

> Microsoft uses somewhere approximating 0% .NET/C# code in the Windows core

On the NT kernel, yes, yet good luck using Windows only with the kernel, without the remaining layers, many of which developed in a mix of COM and .NET.

A basic proof how much Windows systems programming experience you have.

Ah then there is this thing called IoT Edge runtime for intelligent devices, written in a mix of Rust and C#.

And before I forget, NVIDIA DRIVE AGX firmware is written in Ada/SPARK.

Bye and learn to grow beyond the Linux bubble.

Not only did Google not write Fucshia in Go, they explicitly made it a forbidden language (listing a litany of reasons to exclude it and any other GC language -- the same reasons that apply to most "low-level" code), they set a goal to migrate any parts imported in. Maybe not the best example.

"A basic proof how much Windows systems programming experience you have."

Which programming language would one write "COM" in? I am so poorly skilled that I must have missed that language. I spent a decade+ writing Windows code, including exposing it through COM/COM+/DCOM, but apparently I was doing it wrong.

And while there is loads of C++ in what we consider the Windows platform, there is vanishingly little .NET.

The "Linux bubble" comment is especially weird. I spend my day in macOS, with most of my day being in XCode slinging Swift and GoLand spinning Go. Somehow I can evaluate the world without trying to mangle it to fit whatever my current primary skills are.

I don't really follow what you two are arguing about - I think it's pretty clear you can in theory write systems-level programs in most of those languages. However if the focus is on Operating Systems, there have been a few formally verified OS's written in Ada/Spark that are worth looking at (if I correctly read these posts as saying that's not possible).

It's kind of all over the place.

The initial point is simply that Go or C# or Java in their normal form aren't "system programming" suitable (which generally means low-level kernels and system services and projects having similar needs. Notepad on Windows isn't system programming), and it's unrealistic for Go to be ranked highly on such a list, much less before Rust which is purpose built for this task.

Could someone use them for system programming? Sure they could. It would be the wrong tool but people makes loads of bad decisions.

The point about Linux and other operating system choices wasn't a rule, it was an observation that the rhetoric often plied on developer boards -- usually where developers want everything to be a nail to their skillset hammer -- has little correlation with the world. Nowhere did I exclude Ada or any other compiled, non-GCd (which Ada never uses when used for anything of any criticality) languages. Anything can make just as good of native code.

> Not only did Google not write Fucshia in Go, they explicitly made it a forbidden language

It's more correct to say that they attempted to write part of Fuchsia in Go, but the results performed so badly that they decided it would need to be rewritten in another language.

Just as Microsoft once made a good faith effort to rewrite Windows in C# (Longhorn) before that effort had to be abandoned.

Longhorn failed due to sabotage from Windows team due to the usual WinDev vs DevDiv politics, hence why since Vista all Longhorn ideas have been redone in COM, and COM followed by WinRT (COM + IInspectable + TLB replaced with .NET metadata).

Midori had no issues powering Asian Bing servers during its life in production.

The late build of Longhorn that leaked back in the day performed so poorly that it took forever just to boot on the computers of the era.

I have no doubt the usual Microsoft politics of the era applied as well.

> Not only did Google not write Fucshia in Go, they explicitly made it a forbidden language (listing a litany of reasons to exclude it and any other GC language -- the same reasons that apply to most "low-level" code), they set a goal to migrate any parts imported in. Maybe not the best example.

It helps when the Go guy on the team is no longer there to argue for it.

> Which programming language would one write "COM" in? ....

VB 6, Delphi, .NET, C++ Builder, C++/CX, C++ with MFC/ATL/WRL, whatever language speaks the COM ABI.

Yeah, maybe you were doing it wrong.

> The "Linux bubble" comment is especially weird. I spend my day in macOS, with most of my day being in XCode slinging Swift and GoLand spinning Go. Somehow I can evaluate the world without trying to mangle it to fit whatever my current primary skills are.

Good for you then, happy future.

"Yeah, maybe you were doing it wrong."

I think you misread my comment which was pointing out that COM is an interfacing standard, and has little to nothing to do with language choices or this conversation. Pointing out that there are multiple languages that can build to it isn't a counterpoint, but actually is the reason I made my comment. It's like saying you "developed the program in JSON over HTTP" in a conversation about programming languages.

Though regardless of that, Microsoft uses C++ for the overwhelming majority of a Windows install (this is trivial to verify). This is meaningless, having nothing to do with the root comment, but somehow this conversation was pulled there.

Windows programs can be written in just about anything. No one ever contested that. "Low-level" code, however, is seldom done in the GC languages. Seldom doesn't mean never, as stated over and over again, and Googling for random fringe counterexamples aren't counterpoints.

> Just about every operating system or major piece of system software is written in C

My friends writing safety critical bare metal industrial control software using Ada would, I suspect, feel this was a somewhat limited perspective.

If you ever do consider writing such volumes, I would recommend getting co-authors to cover the areas you're less familiar with.

Given that I said "just about", ergo not all, I'm not quite sure what the point of your comment is. Pointing out that someone somewhere is using something else, even haughtily adding some ill-considered snark, doesn't seem particularly useful in that context.

It's a big industry. There are loads of exceptions and they still comprise a tiny minority.

"just about" is still a vast overestimate, but you're right - while I did my best to be polite while pointing out the extent to which the wall of ill-considered snark I was replying to was, at best, woefully underinformed, I readily admit that I did still fall somewhat short.

"while I did my best to be polite while pointing out the extent to which the wall of ill-considered snark I was replying to was, at best, woefully underinformed, I readily admit that I did still fall somewhat short."

I was being polite in not openly pointing out that your comment was trolling of absolutely no value or purpose, predicated entirely on your own misreading, but here we are.

I think the main point is that Go (while a very good language for lower level work) is presented as a better alternative to C++ or Rust because its "nicer" but a big reason it is nicer is the lack of memory management.

It's weird to crown Go as better when it is nicer simply by ignoring very low-level work then pretend that the argument of OS development is irrelevant.

A bit of an exaggeration, but its like saying HTML is a better "programming" language than javascript when creating documents, then ignore the fact HTML isn't actually programmable.

Here, OS like development in Go.

https://gvisor.dev/

https://www.f-secure.com/en/consulting/foundry/usb-armory

https://github.com/tinygo-org/tinygo

Small OS projects can be done in any language. Java has an OS built from it. Real Linux and systems projects can only be done with very low level languages.

Discord switched from Go to Rust because of slowness issues, I promise if its too slow for an app its too slow for OS.

If Go wants to be a very low level "systems everywhere" then get rid of the GC. Or alternatively stop ranking it above Rust and C++, when its apples to oranges.

Some people are visionaries, others just believe in what is in front of their eyes.

Out of interest, what have you made with Meadow? ( Didn't knew about it and their kits are sold out)

The same kind of stuff one would use an Arduino for.

> You mean something that is good to great but that "outsiders" or late adopters feel unexplainably threatened by and thus feel the need to attack it and its admirers?

Oh sure, the only explanation is that people feel threatened and insecure; the language is totally perfect and its admirers have never wasted anyone's time or demanded that other people blindly drop everything and start porting to their favorite language.

> Rust today seems like a 2013 Tesla Model S. A couple of years ago it was the GM EV1: It used to be largely promise of a future to come with a loads of downsides, where today it's expensive to get started with, has a lot of incompatibilities, but it's superior to most alternatives. Tomorrow it and those like it will be the norm.

What a beautiful analogy, considering that https://en.m.wikipedia.org/wiki/Electric_car_use_by_country puts electric cars at 4.6% of all cars sold in 2020.

The worst of the RIIR fan boys are intensely annoying, but they annoy the crap out of every rust developer I know just as much.

I've been through enough waves of asshole fan boys for different things now that I can mostly tune them out, but they're basically never the people actually getting things done so even on the days I can't tune them out I try not to let them reflect badly on the project.

Similarly, Rick and Morty is still funny to me even if I do basically have to block any vocal fan of it I encounter on twitter.

I wish I could stop these people. They accomplish the exact opposite of what they're trying to do, making people steer clear of Rust. Rust doesn't need that kind of help in scaring people off. As nice as the language is, it can take some time to warm up to when you're used to not having a compiler second-guess everything you're trying to do.

If it's any consolation, the repeated evidence I've seen that your response here is typical of basically everybody getting real work done in rust does help me to ignore the muppets.

I agree. I have so far stayed clear of Rust because of the apparent arrogance of the Rust community. It hasn’t quite reached the level of arrogance demonstrated by the Lisp community (yet) but it sure seems to want to be a contender.

Calling those idiots "the rust community" is extremely unfair to the rust community and exactly what this thread was pointing out was a mistake.

Please don't be as bad as they are.

> The 4 languages I would consider for low-level code these days are, in order:

What about Zig? https://ziglang.org

First the use-after-delete use case needs to be covered, we already have C, C++ and Objective-C for that.

I don't expect Zig to take off until A.D. 2101.

re #1 - to be fair, even if you have to use `unsafe`, it's not that bad [1]: unsafe struct FixedBufferExample { public fixed int _buffer[1024]; // This is a fixed buffer. }

Sure, you need to be careful, but given that you'd be doing systems programming anyway, you probably would prefer to have explicit control over bounds checking, since the compiler/JIT is (unfortunately) not sophisticated enough yet to elide bounds checking in many cases.

[1] https://thedeveloperblog.com/fixed-buffer

Can you please expand on #3, why is it bad for system programing?

without the standard library data structures, what's the point of using C# for low-level programming? kinda seems like a square-peg-round-hole situation.

Leaving aside that entire operating systems were written in Lisp, I suggest you consider adding Ada to that list. It is very strict, and feels clunky to use by today's standards, but it is highly readable, has a good set of safety features, and an actual spec with a comprehensive conformance test suite. And it's been proven in defense and aeronautical use for decades -- two industries where lives are literally on the line.

Operating Systems has been written in a lot of languages. However the operating systems that people actually use are written in C/C++.

You can declare array length in a constructor. Otherwise you can just use a list and don’t add past a set capacity.

There was a research operating system by Microsoft https://en.wikipedia.org/wiki/Midori_(operating_system), which used a dialect of C#, with some interesting extensions specific to operating system / driver development.

Joe Duffy has blogged about it, excellent read.

http://joeduffyblog.com/2015/11/03/blogging-about-midori/

Also singularity, https://en.wikipedia.org/wiki/Singularity_(operating_system)

Aren't operating systems written in non-garbage-collecting languages?

Do OS written in C# have good performance?

If an OS runs on bare metal, who does the garbage collecting? Ain't this a paradox?

There's nothing inherently contradictory about running on bare metal, and being garbage collected. As long as you can manipulate memory directly, the garbage collector could potentially be implemented in the language itself. See the Boehm[1] GC as an example of a GC implemented in C for C/C++.

[1]: https://www.hboehm.info/gc/

that will need to bootstrap first, and you won't be able to use such GC code in the initialization code for the OS.

GC code using Boehm would not be suitable for device drivers nor realtime code.

I can understand where making it available for higher level systems is desirable.

I could envision the kinds of alternative GC systems that could be inbuilt given the knowledge an OS may have about a process or a sub-system etc. Alternatively, Objective C settled upon reference counting (ARC) and this is superior to GC where it counts.

Where do you envision the GC bootstrapping and how do you see it interacting with virtual memory and your malloc implementation?

> that will need to bootstrap first, and you won't be able to use such GC code in the initialization code for the OS.

It's all highly dependent on the interface between the language and its runtime. Hypothetically, your C# runtime (and by runtime, I'm using it in the same sense that Rust or C++ would use it; I'm not referring to a VM) could provide a function that allocates memory (C#-esque pseudocode):

    public static ulong New(ulong bytes)
    {
        if (gcInitialized)
        {
            return GcNew(bytes);
        }
        else
        {
            return BootstrapHeapNew(bytes);
        }
    }

Until the GC is boostrapped, all allocations are made on a heap, and either manually freed or kept around until the GC takes over deallocation.

> GC code using Boehm would not be suitable for device drivers nor realtime code.

I completely agree, I just provided it as an example of a GC implemented in the same language.

> Where do you envision the GC bootstrapping and how do you see it interacting with virtual memory and your malloc implementation?

I don't see the GC interacting with userland in this case, and instead would only be used for kernel data structures. I don't really think it's a good idea to have GC in a kernel, I only wanted to point out that low-level code and garbage collection are not mutually exclusive.

For that matter, until you bootstrap GC+heap, you could just use statically allocated memory, or even local arrays (via stackalloc). This would preclude using any reference types, but you can do plenty of things with just structs and unmanaged function pointers alone in C# (it's basically the expressive equivalent of C, but with generics thrown in).

> Alternatively, Objective C settled upon reference counting (ARC) and this is superior to GC where it counts.

ARC does nothing special other than automate [myCocoaClass retain] and [myCocoaClass release], a compromise after putting a conservative trancing GC in Objective did not work as expected, given the amount of caveats with GC unaware C code.

As such, Objective-C performance related to heap management remains unchanged.

The kernel obviously.

http://www.projectoberon.com/

No, because there is no reason an OS couldn't have an integrated garbage collector. The garbage collector would have to be written in a non-garbage collected language or using a subset of the language that doesn't allocate on the GC heap, the no-runtime sample here shows that it would be possible to do one in C#. But there is no reason the rest of the OS couldn't use automatic memory management. Linux is a heavy user of refcounted data already, and there is no fundamental difference between that and other garbage collection strategies.

Lisp machines had hardware/OS-based garbage collection ~40 years ago: https://en.m.wikipedia.org/wiki/Lisp_machine

Not necessarily. Lisp and Smalltalk for example were used for OS implementations in the 80s and many later examples followed.

Some past threads:

Blogging about Midori (2015) - https://news.ycombinator.com/item?id=21988694 - Jan 2020 (27 comments)

The Error Model in Midori - https://news.ycombinator.com/item?id=11054912 - Feb 2016 (41 comments)

Midori: A Tale of Three Safeties - https://news.ycombinator.com/item?id=10871222 - Jan 2016 (7 comments)

What happened to Microsoft's Midori operating system project? - https://news.ycombinator.com/item?id=10576697 - Nov 2015 (34 comments)

Blogging about Midori - https://news.ycombinator.com/item?id=10545353 - Nov 2015 (1 comment)

More details on Microsoft's new M# language - https://news.ycombinator.com/item?id=6983649 - Dec 2013 (49 comments)

Microsoft's Midori operating-system skunkworks project soldiers on - https://news.ycombinator.com/item?id=4758071 - Nov 2012 (1 comment)

Goodbye, XP. Hello, Midori - https://news.ycombinator.com/item?id=232554 - June 2008 (1 comment)

Also cosmos: https://www.gocosmos.org/

You don't need any of that to do system programming in C#. Startup time of JIT runtime is not too bad. Similarly, garbage collector is fine if you don't load it too much (neither allocations/second nor bytes/second).

Here's my almost decade-old open source code which works for hours allocating very little (the process gets killed by the OS if exceeds 11MB of RAM): https://github.com/Const-me/SkyFM

Here's somewhat similar modern code, also multimedia related: https://github.com/Const-me/Vrmac/tree/master/VrmacVideo

You seem to have a habit in succeeding to explain what the code is while avoiding to explain what it does.

The first one is streaming audio player for Windows phone 7+ platform. C# was the only language allowed by MS (the native code only arrived in Windows Phone 8), and the OS enforced strict memory limit. You exceed 11MB RAM, the OS kills background agent process, audio stops playing. The networking wasn’t great either, that worked on top of EDGE and early 3G.

The second one is a general-purpose cross-platform graphics and multimedia library. The specific part I linked implement an equivalent of <Video> HTML element on top of Linux kernel APIs like V4L2 (video decoder) and ALSA (audio rendering). All that stuff is C# on top of .NET Core 2.1. It’s running great with JIT and GC, even consumes less memory than VLC player.

If directly consuming low-level Linux kernel APIs and implementing non-trivial APIs on top of them is not a system programming, then I don’t know what is.

Last I checked NativeAOT is not F# friendly, hopefully that's improved recently.

When you last time check it? I think it became better. I recently test F#, so maybe that would be interesting to you https://codevision.medium.com/life-f-and-nativeaot-b65fee97d...

Quote: "Cross-building is possible but requires extra steps, so just use the platform you're running on to avoid errors"

Meanwhile in Delphi you just select the target OS from the list of supported OS'es and just "build and deploy". Why is that every single other IDE that I've tested it's an adventure to create cross-building, to the point where you need to give blood from your fingers to the God of that OS before you finally make a good build.

Mainly because C/C++ libraries/build environments/compilers just don't sit still and continue to evolve/devolve/diverge.

If you have a single language with a single compiler, presumably with a smaller user base, chances are that is less of a problem.

That cross-compiling is still so thorny in 2021 is in my opinion a big failure of software engineering.

The creator of Zig is maintaining a very accessible and immediate cross compilation infrastructure for Zig and C alike (zig cc). It's a secondary thing to him, but I practically use it very often since it just works, in contrast to the other stuff. Unfortunately even zig cc has often issues with less tested platforms like 32 bit ARM

Delphi does not have the problem that it's illegal to redistribute e.g. macOS headers on a non-mac machine according to Xcode license. Likewise, until very recently the MSVC C++ standard library was under a proprietary license which you could not redistribute, only use in your own software. It's a social issue, not a technical one.

Which is quite fun to read for those of us that remember Kylix.

Isn’t there a problem with bootstrapping .NET? Can I truly build from source?

The C# compiler for example is written in C# so you need a functioning .NET runtime in order to compile the compiler and standard library. That doesn't mean you can't build from source though, there aren't any special binary blobs you need to build .NET. For example, a csc (C# compiler) built using the open source Mono runtime will work just as well as one built using an open source build of .NET Core or one built using the .NET Framework binary blobs.

Why can't we just take the existing .NET 6+ SDK product roadmap and add the bare minimum OS-level capabilities such that you could boot directly into the console of the app?

I'm imagining a something like .NET 6 SDK combined with a minimal Linux distribution, and a special dotnet publish command that outputs a vhd/bin file that can be directly booted from disk.

Over time, the .NET stack could consume bits and pieces of what it's hosting layer provides with objectives of totally replacing it, ship of theseus-style.

This isn't relevant enough to the average user to make it onto the SDK roadmap. If community members contributed it I bet it would probably get welcomed into the larger ecosystem though.

FWIW some maintainers have been tuning .NET for Alpine Linux, so if you just want a small container to host .NET apps you should give that a try.

Here if you want something bare metal style, https://www.wildernesslabs.co/

The C# compiler and parts of the supporting .NET runtime are written in C#, so yes there is a bootstrapping issue. Other languages like Rust or even C compilers written in C have this problem.

For initial porting to a new system or processor architecture, the C++ part of CoreCLR (the main runtime for .NET) can be built with CMake and LLVM on the target system. The libraries can be cross-compiled on another system that already supports .NET and copied to the target system. Some Details are here:

https://github.com/dotnet/runtime/blob/main/docs/design/core...

For building from source to satisfy requirements Linux distributions, Microsoft has a system to build from source. My understanding from the last time I closely looked at it is some binary dependencies are decompiled to MSIL (the bytecode used by .NET). Since CoreCLR includes a MSIL assembler written in C++ (ilasm), it can bootstrap using these MSIL sources. But I have not looked at this project closely in a while and it evolved quite a lot while I was watching it. The system for the source build is here:

https://github.com/dotnet/source-build

The "old" compiler which is written in C/C++, I don't think that it's open source though.

The modern compiler - Roslyn - was explicitly re-written in C#, and is currently the main compiler (and has been for about three years now).

Not three. More 8-10

From VS2015 so 7

I cant find any benchmarks comparing AOT to non AOT and ideally C, C++, or Rust. Anyone aware of any?

All benchmark you mentioned(and lot more) https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

Isn't non-AOT C# (and java) "cheating" here, by warming up?

And isn't not needing warm up one of the AOT selling points?

I think in practice PGO for native C/C++ binaries is not that different from a JIT warming up C#/Java code, especially since llvm+clang is so good at optimizing code.

Often, you might want to run servers. If they are JIT compiled, then during warm-up they will be slow(er), and later they will be fast(er). But usually you don't restart servers that often, so they will spend most of their time in "fast mode". It makes sense to me to measure what will be used in practice.

Of course, for command line tools such as grep, it is more realistic to measure the warmup phase as well, as that will reflect real usage.

What makes you think those "non-AOT C# (and java)" have a "warming up" time that isn't included in the run time measurement?

Everything I've ever seen and experienced had informed me that AOT might not be worth it when you look at the whole engineering pie. Especially considering the additional sharp edges it adds around reflection and dynamic types.

Aren't operating systems written in non-garbage-collecting languages?

Do OS written in C# have good performance?

A side effect of history.

Check any of XEROX PARC workstations OSes, namely Smalltalk, Interlisp-D and Mesa/Cedar.

Oberon, EthOS, Oberon-2 and A2 at ETHZ.

TOPAZ at Olivetti.

Singularity and Midori at Microsoft.

Or have some fun digging through Android sources to see outside Linux kernel and ART, what is Java and what is C++.

> Aren't operating systems written in non-garbage-collecting languages?

In practice yes, but there’s no requirement for it.

Two of my favorite people replied to this comment. This was a good comment =)

Reply on news.ycombinator.com