Also Matt Warren's blog is super cool.
To understand why, we need to talk about 2 separate things:
---
1. Specialization is important for performance and when speaking of .NET, specialization for value types is subsumed in type erasure, but that's not necessarily the case. You can have compiler-driven or even runtime-driven specialization without reification.
In Scala for example we've had the "@specialize" annotation for a long time, with the compiler being able to specialize generic code just fine. It's not perfect, a better implementation eventually happened in Miniboxing [1] but it withered away due to lack of interest and the ongoing work happening in Dotty / Scala 3 and its newer TASTY distribution format, which should make specialization easier to accomplish.
Also there is on-going work to bring value types to the JVM and it's happening: http://mail.openjdk.java.net/pipermail/valhalla-spec-experts...
That said having specialization is pretty cool for fine control of the memory layout and Java developers have to resort to a lot of unsafe hacks for achieving the same thing. But if that cost was paid such that languages like Scala or Haskell could happen on top of the JVM, until they figure out how to do it such that everybody benefits, I think it was a cost worth paying.
Also consider that the lack of specialization forced the JVM 's engineers to get creative in other areas. For example the JVM has always been great at inlining code at runtime, even for megamorphic call sites. And the new GraalVM has super impressive abilities to eliminate boxing at runtime, which works for dynamic languages too: https://www.graalvm.org/
---
2. Reification is in fact about adding info about type parameters at runtime. This aids in using reflection to make a difference between List<int> and List<string>, but people miss the forest from the trees.
As a matter of fact such reflection is only needed because languages like Java or C# have very weak static type systems, compared with other languages in the ML family. With an expressive type system, you never need reflection capabilities.
In Haskell for example the question of whether something is a List<string> or List<int> never, ever happens. In Scala you sometimes need it, but much rarely and you can get it via a compiler-generated `ClassTag`, which is actually a much better approach, because it makes it clear in the signature, this being compile-time reflection. People also like being able to do "new T", however that need completely goes away via proper support for type classes, which both Haskell and Scala have, this being another special purpose band-aid.
And reification is actually a bad feature to have in the runtime, because it makes it hard for languages to support higher-kinded types, or to build dynamic languages.
F# does not do higher-kinded types and is less expressive for that reason than OCaml, Scala or Haskell and the primary reason for why it doesn't do higher-kinded types is because it would have to do type erasure by itself, thus forgoing the performance benefits and the interoperability it has with C#.
Ironically it is support for higher-kinded types in a language that increases its expressive capabilities to the point that you no longer need runtime reflection. In other words ... you're blaming Java for not having a band-aid that happened in C# due to their static type system being basically unsound and thus needing runtime guards and reflection. You quickly get over this once you'll start using a more expressive language ;-)
The majority of the issues and gotchas listed in the generics FAQ for Java do not happen in C# at all. It is liberating.
Reflection is an advanced use case that some people use, but it is rare.
The lack of flourish in. NET overtime more to what happens when a company takes the reins of a stack and later stops working on it.
Microsoft for a while invested and designed JavaScript, Python and Ruby versions for .NET. But when conditions changed, there was no interest to keep the projects going. These are all observations from an outsider at the time the projects were defunded.
Because of the siloed approach to development at the time, and the lack of an external county around those efforts, bootstrapping a community to drive those on their own proved to be very hard. Ruby mostly died, Python is barely surviving.
The mood in the ecosystem went from "we can build these and speed them up" to "this is an ongoing cost, let us rather interop with the real implementations rather than find constant catch up".
Meanwhile, languages that Microsoft did not build did flourish, like PHP
This corporate phenomenon deserves a blog post on its own
But couldn't .NET not being a good target for dynamic languages be one of the reasons to stop working on it?
I keep hearing that .NET is a good platform for other languages, but there doesn't seem to be much empirical evidence for it (at least in dynamic languages).
Scala, Kotlin, Clojure, Eta (Haskell), JRuby are the ones I care about and they've been flourishing.
I'm a pragmatic at heart btw, but pragmatism sometimes leads to ignorance. It's important to see what's available on the other side of the fence without bringing the preconceptions bag with you. Because you can then actually use the metaphorical "best tool for the job".
It's what I'm doing myself, or at least trying and .NET is without a doubt evolving in a direction I like. And I can also write entire paragraphs of what's better on .NET, I just don't find the generics reification to be one of those things.
The implication here is that you never need runtime type checking, which clearly isn't the case in ML family languages. Sum types for example have an abstract representation that must be resolved at runtime (for pattern matching).
C# also doesn't need reflection to resolve types (other than through the virtual route - i.e. inherited method resolution), this is a core feature of the CLR though (via `callvirt` [1] if I remember right) and not a weakness of C#'s type-system. C# bakes the concrete generic types into the generated IL.
[1] https://docs.microsoft.com/en-us/dotnet/api/system.reflectio...
Indeed, but that's plain tagging, which is basically an int and not the same thing.
"callvirt" is basically doing a virtual method lookup in a vtable. It's still at runtime, but we are talking about OOP polymorphism now and how late binding works. Languages with type classes don't need that either, but that's a separate discussion and note that I still like OOP.
> C# also doesn't need reflection to resolve types
No, the developers do, because the language is not expressive enough to express generic pieces of code without losing type info, so you need reflection for constraints (e.g. "new T" or "instanceOf" checks) and runtime guards for downcasting.
Classic example: express a "sum" function that works on any Array<A>.
And note that what F# is doing to solve this particular problem is a hack to workaround both the lack of higher kinded types and of type classes and that does not interoperate with C#, since it requires "inline functions" which are doing type erasure (and it's a big and ugly hack, speaking of ML, because inline functions are no longer values ;-)).
> C# bakes the concrete generic types into the generated IL.
Of course, but that too is part of the problem.
The type system is totally capable of it. The language doesn't make it easy, but the code below is essentially type-classes and class-instances if you squint. It's something the C# team are looking into now anyway, but the main point is the type system doesn't preclude it.
public interface Newable<A>
{
T New();
}
public struct NewableThing : Newable<Thing>
{
Thing New() => new Thing();
}
public class Foo
{
public A Bar<NewableA, A>() where NewableA : struct, Newable<A> =>
default(NewableA).New();
}
public interface Num<A>
{
A Zero();
A Add(A x, A y);
}
public struct NumInt : Num<int>
{
int Zero() => 0;
int Add(int x, int y) => x + y;
}
public struct NumFloat : Num<float>
{
float Zero() => 0;
float Add(float x, float y) => x + y;
}
public static A Sum<NumA, A>(A[] xs) where NumA : struct, Num<A> =>
xs.Fold(default(NumA).Zero(), (s, x) => default(NumA).Add(s, x));
int sumInt = Sum<NumInt, int>(new [] {1,2,3,4});
int sumFlt = Sum<NumFloat, float>(new [] {1.0,2.0,3.0,4.0});
That's a very hand wavey statement. It's the goal of all staticly typed compilers to resolve to concrete types at compile time. How is part of any problem? The CLR has the capability to generate new concrete types from generic types at runtime if required, but obviously tries to do as much of that work up front at compile time.
Next, you can't say that C# has a "weak" type system. It is mainly a strongly and statically typed by design. There are elements of weak typing in C# such as the dynamic keyword, and all C# type safety features can be bypassed if desired. It seems you prefer "weak" to mean "poor" but that is your definition, not the generally accepted one.
Further, it is completely false to indicate something to the effect that C#/CLR can't let other languages flourish on the platform. The CLR stands for "Common Language Runtime" and can run any language you can imagine, including weakly typed languages. It is a plain old finite stack machine at heart. Unlike JVM, the CLR is an ECMA standard and anyone can implement it and .NET sources are not GPL licensed.
Scala used to be available on CLR, and the reason it is not supported is more political than technical. You are again trying to make a technical justification for a political decision. Besides, Scala still only runs on JDK 8 which is funny because Oracle JDK will need a commercial license from 2019 to get patches so you're forced to switch to OpenJDK or pay for a Scala runtime.
The Java community has talked about value types forever, we know that. Good luck unwinding that decision, it is not that easy.
The cost for unsafe hacks to let Scala and Haskell work had more to do about the fact that .NET was not cross platform at that time. That was not the ecosystem impetus for for having Scala run on JVM. It was more because Java as a language is aging and lacks innovation which is well recognized. The cross platform argument against the CLR is gone now, so there is little reason why we shouldn't be able to run Scala and Haskell on CLR instead if you are after those languages.
Also, the point about "forcing JVM engineers to get creative" is not really a solid argument at all. They just have to work around the mistakes they made in the past (no value types + runtime type erasure). We also know for a fact that escape analysis in JVM doesn't really cover that many cases anyway in practice. You need to look at the generated code to tell if it works or not. Besides, it is not something that the CLR cannot implement. On the contrary, there is active effort in CLR towards Object Stack Allocation (you can Google for it).
The general point that reflection is "only" needed because of a "weak static type system" is also not true. You seem to confuse the shape of the API for getting runtime type information with the need for getting such information in the first place. You can try to build a polymorphic serializer in C++ which also has run time type erasure, it is not as easy as in C#. Eventually we need to carry over type information anyway, no matter what language.
Reflection is not about "adding info about type parameters at runtime". It is much more than that. First, we don't "add information" at runtime for existing types, we can inquire about it. This is super useful for many scenarios. Second, we can generate code at runtime as well with reflection, that is, MSIL op codes. This is also super useful.
Your use case of "express sum function over Array<A>" is not the direction C# is heading. We have LINQ for the expressiveness part which Java doesn't have apart from some lookalike hacks. Also, for performance, your sum function is going to run at glacier speed compared with .NET Core 3.0 where we will have SIMD instructions and we can execute dedicated AVX code paths depending on memory alignment, in managed code. This stuff used to be C++ only until now. I don't think your Scala and Haskell implementations stand a chance to perform close to that. And reusing a sum function over a read only span is a matter of a one line call. I like the pureness argument but in practice we need to consider performance as well.
Thanks! I'm glad you like it