matthewwarren

> I'm impressed with the testing approach.

Likewise, I really liked the SuperPMI tool, it's a very interesting way of testing

> Is anyone aware of similar articles/explanations of the JVM runtime and JIT?

I've recently come across this article about the JVM that discusses internals, http://blog.jamesdbloom.com/JVMInternals.html

Also the slides from this presentation a very good https://jpoint.ru/en/talks/5vqeeeo89yk6q8qmm0y8sg/

> .NET, Java, etc. all sound so 1990-ish.

.NET 1.0 was released in 2002 ;-)

https://en.wikipedia.org/wiki/.NET_Framework_version_history...

> ... is one of my favorite side effects of it being open sourced.

Agree!

I would never have the time or expertise to work all this out just using WinDBG or similar. Having the source, including decent comments and things like the BOTR [0] is amazing for exploring the internals!

[0]: https://github.com/dotnet/coreclr/tree/master/Documentation/...

> Is there any way to do a more fine-grained breakdown of where all that code is coming from?

See https://gitter.im/dotnet/corert?at=5b1af059178709146f21076f for some techniques to do this

I just double checked and you're right, the GC code is pretty small. The 0.49 MB run-time `Test.CoreLib` (http://mattwarren.org/2018/06/07/CoreRT-.NET-Runtime-for-AOT...) includes the GC as well.

> They are big leaves/fruit and there are only about 30(?) or so of them.

Ah, yeah that makes sense, I'd not thought about it like that.

> IIRC, early versions of this included the entire framework, so it's gotten a lot better

Yeah, they're definitely improving things over time

I wondered the same thing, it seems like 'System.Private.CoreLib.dll' is all or nothing, unless of course you write your own version such like 'Test.CoreLib' (https://github.com/dotnet/corert/tree/master/src/Test.CoreLi...)

I don't know if it's technically impossible or just hard to have 'tree-shaking' being done on it, so that you only get the functions/dependencies that are actually called?

Yep, seems like the Mono impl was more mature, you can read a bit more about this decision here https://github.com/aspnet/Blazor/issues/286#issuecomment-373...

Yeah, that's what impressed me as well!

Turns out that even more components could be written in C#, but the ROI just isn't worth it, see https://github.com/dotnet/coreclr/issues/17576 for an interesting discussion

I'd start with the WebAPI sample and it's rd.xml file, see https://github.com/dotnet/corert/blob/master/samples/WebApi

Also take a look at this project https://github.com/shtik/cli/tree/dev/src/slidey, AFAIK it's the project that inspired the video I embedded in the post

> ..using .NET Native to rewrite the CLR in C#

They're already doing this, see https://github.com/dotnet/corert/blob/master/Documentation/i...

Thanks for the link, I see what you mean now

> Is already at the beginning of the article, but should be more visibible how MS Research and Don Syme seen the opportunity to add generics (neeeded for F#) and added that. With design already supporting later addition like variance.

I definitely agree, that's why I put it right at the start of the post :-)

> Does anybody remember the article about their generics implementation in assembly ?

Do you mean http://joeduffyblog.com/2011/10/23/on-generics-and-some-of-t... or https://blogs.msdn.microsoft.com/joelpob/2004/11/17/clr-gene...

They're 2 articles I came across that covered the low-level details of the generic impl.

> It is possible to generate interface definitions in C# on the fly (at run-time),

Huh, I never thought about this restriction, so you can't add a new interface with 'Reflection.Emit', you can only add a class that implements an existing Interface, is that right?

> but unfortunately it requires going outside of the CLR or even the DLR.

By this do you mean writing raw CLR metadata yourself and then somehow injecting it into the running process or something else? Either way, its sounds like an interesting technique, any links to how it's done?

I've heard this argument a few times now, it's an interesting point, having 'reified generics' imposes a tax on any language that wants to use that runtime.

Here's a few other examples of the same point, https://twitter.com/jon_cham/status/969929683587432450 and https://twitter.com/headius/status/958371298975080448

> I started looking at the source diffs and it triggers a panic attack. That's a ton of work.

Don't forget it took several people a few years to do all that! But yeah, I agree, it was an impressive achievement!

This post has a nice investigation into 'Escape Analysis' in Java, https://shipilev.net/jvm-anatomy-park/18-scalar-replacement/

Shows that the Hotspot doesn't handle it in all scenarios:

> But, EA is not ideal: if we cannot statically determine the object is not escaping, we have to assume it does. Complicated control flow may bail earlier. Calling non-inlined — and thus opaque for current analysis — instance method bails. Doing some things that rely on object identity bail, although trivial things like reference comparison with non-escaping objects gets folded efficiently.

> This is not an ideal optimization, but when it works, it works magnificently well. Further improvements in compiler technology might widen the number of cases where EA works well.

AFAIK this trick was never possible in .NET because Substring always 'deep copied' the relevant data into a completely new string

Oohh, thanks for those, you've given me more things to go off and learn about!

Thanks for spotting that, I've just updated the post to have the proper links

Thanks for the link, I'll add your blog to the list

(I guess I'm (sadly) biased towards newer blogs, as I've only been digging into .NET Internals for the last few years.)

> Are there any "awesome lists" on Github that are actually curated like this?

Good question, I've not found one for '.NET Internals', that why I wrote the post.

But in general, it would be nice to know about more 'curated' lists, I've found that giant piles of links stop being useful after a certain size!

If you don't want to watch through the whole thing I wrote a post the gives an outline, see http://mattwarren.org/2017/11/14/Microsoft-and-Open-Source-a...

> I think that the "lack of value types" is probably an overrated difference these days.

Well, seems that the JVM designers disagree, there's been a lot of work put into fixing this, see 'Project Valhalla' for instance

https://www.infoq.com/news/2016/11/valhalla-Implementation-p... http://www.jesperdj.com/2015/10/04/project-valhalla-value-ty...

Thanks for this analogy, I love it!!

> The Book of the Runtime documentation states "The goal of the CLR is to make programming easy"

In case anyone else is interested, that quote comes from https://github.com/dotnet/coreclr/blob/master/Documentation/...

> but I've been told by other people that Microsoft or at least the .NET just generally prefer a simpler approach, without profiling, dynamic compilation, deoptimisation, and other complexity the approach used by the JVM brings. It's much more what-you-see-is-what-you-get from their JIT, rather than the mind-binding things the JVM manages to do with some code, and maybe that makes life easier for their use cases.

Interesting, it's clear after a bit of research that this is the case (i.e. .NET JIT simpler that Hotspot), but it had never occurred to me that it was a conscious decision

I'd always thought that the .NET was just playing catch-up and/or didn't need to do all the things that Hotspot does. (for instance .NET has value types or 'stucts' so some optimisations are not needed because the programmer can just use them instead of classes, if they need the perf boost)

> If it's true, do the new .NET Core internals, allow .NET Core to beat JVM?

I'm hoping that 'Tiered Compilation' will help, but it's early days. We will have to wait until it is 'officially' in .NET Core before it can be properly tested

> you could say that without the BBC ARM would not have existed at all

Huh, I'd not thought about it like that before. Yeah I guess ARM/Acorn does owe the BBC quite a bit!