BrainFlood: Runtime code generation via reflection in .NET | Hacker News Reader

BrainFlood: Runtime code generation via reflection in .NET

sbox.game

86 pointsthunderbong25 comments

This is quite something. I suspect this person has a C++ background, because the thing where they're abusing the generic type system to do arithmetic is very much more like C++ template libraries than something you normally see in C#. The advantage seems to be that you can use MakeGenericType() in situations where the normal bytecode Emit() isn't available?

I wouldn't exactly call it "normal", but that's partially because a lot of C# developers still don't know about the relatively new generic maths feature. Those who are familiar regularly "abuse" the system in a similar way because that's one of the major benefits of it. We don't necessarily reimplement basic arithmetic using the type system, but we regularly hoist generic constants into static properties in order to convince the compiler to behave the way we expect. For example: T.PopCount(value: T.AllBitsSet) is a safe and generic way to express the size of a binary integer as a constant.

> For example: T.PopCount(value: T.AllBitsSet) is a safe and generic way to express the size of a binary integer as a constant.

Despite the name, 'Unsafe.SizeOf<T>()` is the safe and preferred way to do this - it is guaranteed to be a compiler constant (in the past you'd say "JIT constant" but you can no longer assume that with the advent of NativeAOT).

It's a bit unfortunate because much like with MemoryMarshal, not all methods are equally unsafe. Some are benign or safe like this one, same are Unsafe.As or, worse, Unsafe.AsPointer.

> Despite the name

Why even have something labelled "Unsafe" which is in fact safe ? Genuine question. What lead to this choice?

The reasoning is generally that using SizeOf implies one is doing unsafe things. Why do you need the size of something in managed code? What would you do with that information?

Not that I necessarily agree with the decision myself, but that's the argument made by others.

I cannot get behind that argument, it seems to me that it ends up needlessly marking work as dangerous and in need of proper review when it's actually not at all remarkable and so you teach people these reviews aren't really important, it's probably just another false positive.

Rust spent lots of effort on making sure these lines are in the right place. The current work to try to "make C++ safer" is likely to ban taking a 32-bit unsigned integer and treating that as a 32-bit IEEE floating point value because it can invoke Undefined Behaviour in their language. Meanwhile in Rust that exact feature is not only safe, it's a pure function named `f32::from_bits` because 0x40000000 literally is 2.0, it's never any other value, there's no actual danger here, you just need to fix your programming language but C++ would rather waste your time on this non-existent problem.

Author here. My background is in rust if anything. Honestly, I prefer to avoid playing these clever games with type systems unless it's really necessary. It just ended up being the perfect tool for this very specific problem. (How do I compile code in a sandbox that exposes a limited reflection API?)

> Practical .NET Optimization Abuse

This is not abuse, it's intended behavior, standard library itself likes to use it a lot internally. That's also how generics work in Rust!

This can be pushed further by passing data by ref's. You can also easily allocate structs in unmanaged heap (aka malloc and free) and then pass them by `ref` without using unsafe (save for construction and alloc/free). Just don't do it for structs which contain gcrefs. C# can lean very heavily into systems programming :)

There have also been a large amount of struct optimizations across the last few versions. So it is not a coincidence that RyuJIT/ILC can trade blows with LLVM (not always but still!).

Edit: Oh god, I saw what this refers to. I take it back. There is likely a better, idiomatic and compiler-friendly way haha.

This post has a lot of similarities to the 'object algebra' paper by Bruno Oliveira.

https://www.cs.utexas.edu/~wcook/Drafts/2012/ecoop2012.pdf

If the OP author has any interest in doing a 1000x service to the C# community, the community needs a translator like you to convert the paper's compiler example into other domain specific examples. I'm pretty sure this paper went nowhere because it lacked a hello world example. something easier to digest and smaller and scope. The classic "dog is an animal and cat is an animal" level of explanation.

Ah, yes, this is delightfully useless! Well, maybe good for stress-testing language tooling (that single huge type expressing the Mandelbrot generator is... something to behold), but mostly just cool-that-this-can-be-done.

Another cool thing that I learned about via this post (as I was wondering about the .scene files in the GitHub repo) is this whole ecosystem:

> S&box is coded in C#. Under the hood, it uses the Source 2 engine (CS2, HL:Alyx, DOTA2) and some of its systems: rendering, resources, physics, and audio

And people say there is no more Good Stuff on the Internet...

This is great and despite what the article says, an actually useful technique. The reason this performs well is the .NET runtime does monomorphization when the generic type parameters are value types (primitive types like int or user-defined structs). In modern .NET you would use static abstract methods interface methods to implement this feature. But if you have to run on the old Windows-only .NET Framework, you need to use the approach used in BrainFlood. See this article [1] for more details. An example of an official .NET library that uses this technique on .NET Framework is TensorPrimitives. See this file [2], the structs are defined at the bottom.

[1]: https://blog.stephencleary.com/2022/10/modern-csharp-techniq...

[2]: https://github.com/dotnet/runtime/blob/main/src/libraries/Sy...

This is fascinating! Code generation is a thing that's solved a couple problems for me recently. The way I do it never feels great. Whenever I want to cache a big dictionary literal, I tend to generate the code as a string and write it to a file. I've yet to find a compelling upgrade that checks all the boxes. (The generated side has run in the kneecapped .net for unity, although the generator doesn't necessarily.)

I'm going to dig into your implementation.

You could source generate this directly into your code instead of dealing with file IO. Newer versions of Unity support source generators:

https://docs.unity3d.com/6000.0/Documentation/Manual/create-...

I am a fan of this particular repository by Jules Jacobs who similarly uses struct based generics (to create immutable collections.)

https://github.com/julesjacobs/ImmutableCollections/discussi...

It has similar "source generator" hallmarks like in OP's post, for example calling a generic constructor which is JITted to a deeply nested struct type.

It's (in my opinion) very hard to reason what the code is doing because it has that flavor of "writing code to generate source to generate code"

It's impressive. And i'm pretty sure it's only real hinderance, why it could not be adopted into the base library, is that the repo needs a phd holder to finish implementing the `IList` interface because deleting is really difficult and theoretical.

Feels like a brainfuck interpreter version of magic(1), a clojureCLR JIT compiler implemented via macro.

1 https://github.com/nasser/magic

> you aren't in a sandboxed jail, you should probably just use System.Reflection.Emit

System.Linq.Expressions is a useful intermediate (that also works in sandboxes). People overlook its compilation power all the time, especially this long after the DLR (System.Dynamic and especially System.Dynamic's IDynamicMetaObject) has been forgotten, but it is a far easier API than System.Reflection.Emit.

I was half-hoping in this article to see a comparison with BF implemented in System.Linq.Expressions.

Yeah, I was totally unaware of this. The namespace appears to be blocked in the sandbox -- probably rightfully so -- but now I'm curious as well. I might have to test it out and write a little addendum.

Edit: Looks like it's slower by a pretty good margin (~3.6 seconds). It's certainly possible I'm doing something stupid. This would be really nice to have for webassembly, where I'm convinced I need to implement some sort of virtual register allocation, and I'm still not entirely sure about control flow.

This is quite cursed, but does show that the .NET runtime system and JIT seems fairly robust even in the presence of this stupidity.

Sounds like an awesome article, but it's unreadable on mobile (cut off on both sides) so I'll have to wait to read it until I get to a PC. Might be worth fixing

Your mobile browser doesn't have reader view?

You shouldn't have to jump through hoops to read an article on a phone. It's 2025; mobile browsers have been mainstream since 2007.

Desktop mode in mobile seems to work fine.

The point is that normal mode doesn't.

Looks fine to me (Fennec 134 on Android).

Maybe it has been fixed, but the article currently looks fine for me on an iPhone.

Reply on news.ycombinator.com