How generics were added to .NET
mattwarren.org
mattwarren.org
All the more reason to admire and appreciate MS Research and the development of C# generics. Really amazing accomplishment.
And Linq is one of the single greatest inventions in modern day programming.
What you can argue is that LINQ made this stuff available to more developers (with useful syntax and in a statically typed language, which also must have taken a lot of work), but filter/map/reduce have been around for quite a while.
That obviously doesn't make it less useful or the underlying tech less impressive though.
Linq compiler compiled the whole query into a AST, which you could then convert to a backing query your data store could understand.
To date, I still think Linq was great. I was indifferent to the "sqlized linq", but i liked the lambda quite a lot. It was a nice abstraction and a pretty good language to write queries on.
Disclaimer: I worked on NHibernate Linq integration ~10 years ago.
I've started using it in 2009. I am still using it in my projects, but I am still not sure if NHibernate is functional complete or if the project is nearly dead.
It works fine for me (if it is not worth the effort I am using Dapper.net), but it doesn't seem that there is much active development going on.
I stopped working on nhibernate around 2010-2011, when i started masters. I did only keep in touch with one friend - i am not very sociable person. So I guess short answer is, i am not sure to be honest.
Back in 2008ish, I was thinking it was going to die because Entity framework provided a great support for linq, and it had a great hype. A bunch of enterprises moved away from NHibernate (we also had a lot of legacy from XML configuration - a bunch of internal data structures were referencing xml), so it was clunky. Fabio, Oren (Ayende) did a bunch of improvements, I worked a bunch of shortening initialization times and so on. I think Ayende did a very good job with Linq initially, but there were plenty of edge cases that i remember having to fix :) It was my first opensource project as a core member (second and last was castle project) - so it has a very special place for me :)
I've used the Linq queries in one of my projects, but I like the QueryOver statements more (a little bit too much black magic in the Linq queries).
But I believe that it was a great effort - so let me thank you for that !
Linq with the EF is great for simple queries and updates, anything else and you're in for serious performance problems as soon as the app scales. Better to drop to normal SQL for complicated data loading.
Even MS can't write decent LINQ queries, their ASP.Net identity provider is now the most 'expensive' bit of our app now because they used expensive LINQ queries instead of using raw SQL queries. Admittedly our use-case is abnormal as the specific problem we have is that because of the way users are added their password gets reset almost immediately, as they're invited by an organiser. It also means new users are constantly being added. When the password reset is saved it completely unnecessarily "verifies" the update by making sure the username and email are unique, which means two UPPER()s and CONTAINS()s on string fields. Unlike the old provider there's no usernamelowered field already in the db to avoid this.
We can fix it by over-riding these queries, but it's annoying that the ASP.Net team took a core framework piece that was very performant with fast performing SQL and made it substandard. At least I can look at the code now ;).
I think LINQ gets a lot of flak it doesn't necessarily deserve in complex queries due to people stopping at the black box and assuming black magic. It's a very functional programming paradigm embedded in an otherwise procedural world, and so the skills to debug complex LINQ should be unsurprisingly just a bit different than debugging most else in C#. I don't blame people for stopping at the black box. I just think more people should know that you can do more than stop at the black box.
(Also, it amuses me that your example from ASP.NET identity's changes have nothing to with that it should be using raw SQL queries versus LINQ, and everything to do with denormalization versus the query execution engine. It shouldn't be a huge surprise that Microsoft might trust SQL Server to be able to UPPER() and CONTAINS() fast enough, and the queries rare enough that it isn't necessary to denormalize that information.)
I started a project about 4yrs ago, it was EF vs NH. Went with NH because EF was still pretty immature.
The momentum is pretty clearly toward EF IMO. NH is missing a lot, including proper migration tooling and deep async support. But I don't think that's the real issue. The more foundational problem in my view, is that NH deeply assumes blocking/synchronous database access. I have no idea how NH lazy loading will ever work with async. I imagine you'll have a bunch of FK refs all marked Task<T> that will have to be awaited? In any case, my bet is that EF will get there a lot more quickly than NH.
I would have prefered using DataReader, most often.
When I look at technologies like bitcoin, it's just proof of work/puzzles (a well-understood defense to sybil attacks) + basic crypto primitives like signatures + some of the p2p stuff from bittorrent, etc.
There's real genius in combining the pieces, though. I also spent a few years in serious CS academia so not clear how widely these things (especially puzzles) were known beyond universities.
Plus, while they are enabled by other techniques, those weren't invented in LINQ, either.
LINQ is not about generators...
But those are also more or less just about building ASTs and transforming them, something Lisp has done for decades.
To me, what LINQ offers is a very nice orthogonal DSL that allows programmers to express these thoughts. The underlying constructs are universal, but one could argue that LINQ itself was invented. LINQ was apparently guided by category thinking [0], which makes it fairly elegant, but in terms of expressiveness it is probably similar to SQL. It's all set theory underneath anyway. Unlike SQL however, LINQ does force the SELECT statement to the very end of the query expression, which makes IDE auto-complete work better due to avaiable context. :)
I first had the realization that most data manipulation operations on list-like or table-like objects were actually just glorified set operations (and there was a certain universality to them regardless of query language syntax) when I read this quote [1]: "The turning point was a conversation with Erik Meijer. He told me how categorical thinking guided him in designing LINQ, and explained how category theory brings out a simple relationship between various kinds of databases. The diagram below comes from a paper he wrote on this. It turns out that so-called No-SQL databases are in a categorical sense co-SQL databases."
Don't forget it took several people a few years to do all that! But yeah, I agree, it was an impressive achievement!
I was bored of Java at university, and as the project I was basing my dissertation on was a web-based search engine I decided to use C# and ASP.NET. The professor helping me was fine with it, since he didn't know PHP or Ruby, and he felt he could read C#.
I came across some stumbling blocks with some logic, so I remember posting on a forum somewhere asking for help with setting up a data structure. Someone sent me some LINQ, and I thanked them for the pseudocode. When they pointed out that it was real functioning code, I plopped it into Visual Studio and the whole thing worked. From that moment, I fell in love with the language - ASP.NET WebForms less so, but shortly after this MVC became a thing in the .NET world and I jumped on the bandwagon.
A decade later, and while I've moved onto other languages I still fight C#'s corner, and LINQ/the lambda syntax is one of the first points I raise when people bitch about Micro$oft. The only negatives about working as a .NET developer is being locking into an ecosystem, so if Microsoft and the .NET Foundation can get .NET Core to be feature compatible with .NET Standard then they've got a winner on their hands.
They used to do presentations together with Sun, had the crazy Network Computer idea together with those Java Stations at Sun, ported their installers to Java, were the first RDBMS to allow stored procedures to be written in Java, had their own JVM, IDE and JEE server long before Sun started to get into troubles.
Have kept Java team together, saved Sun labs, turned the Maxime research work into Graal, are making the effort of supporting AOT compilation (tabu at Sun),....
I am a polyglot developer, and while Oracle could be doing a better job, they surely are doing much better than Sun was doing on its later days.
Who knows, if Google hadn't ripped off Sun and they were getting some Android revenue, maybe Sun would still be around.
Or maybe if Sun showed some promise with J2ME and tried to challenge iOS it would have been main iOS competitor instead of Android. But J2ME did not deliver much and the idea to grab some bullshit patents revenue, which Oracle tried, failed.
In copyright case Oracle may still get some money but after belatedly joining CNCF Oracle would not like to be on warpath with Google.
Android fragmentation, OEM customizations, lack of enforced updates and cheapo unresponsive 100 € devices are no better than the devices that gave bad name to J2ME.
https://github.com/mythz/swift-linq-examples
https://github.com/mythz/kotlin-linq-examples
https://github.com/mythz/clojure-linq-examples
https://github.com/mythz/java-linq-examples
https://github.com/mythz/dart-linq-examples
https://github.com/omnibs/elixir-linq-examples
http://templates.servicestack.net/linq/
What? It literally already is, all the time. If anything it's full .NET that lags behind standard often.
> It was only through the total dedication of Microsoft Research, Cambridge during 1998-2004, to doing a complete, high quality implementation in both the CLR (including NGEN, debugging, JIT, AppDomains, concurrent loading and many other aspects), and the C# compiler, that the project proceeded.
I didn't realize that it was less a "must-have" and more of a "research and if-possible" task. I wonder how the .NET framework/languages would have changed if they went with type erasure. Would we even still talking about .NET today?
Windows remains 800lb gorilla on the desktop, and C# is Microsoft's recommended environment on it. It's hardly surprising that .NET remains relevant.
It's a great time to be a .NET developer.
I'm not one bit surprised this is the the guy leading the C# effort's second or third language.
Even popularity is a value, as it automatically provides a community, but it's also a pitfall, as massive popularity inevitably means the average programmer of language X becomes as smart, careful and reliable as the average programmer irrespective of language. And, to put it mildly, that's not a positive evolution. That, above all others, was VB and Delphi's downfall.
On three axes, I would argue VB, Javascript and Delphi did/do incredibly well: ease of development (of small-ish programs), and the deployment story, as well as the empowerment they provided.
On things like consistency, developer productivity, batteries included (Delphi was better on the batteries included front), type system soundness, ... they were somewhat sub-par.
It's just what people value at the time. And of course, it is critical during a career in development to distinguish yourself from the average programmer.
In the short-term, languages with good developer outreach and other factors win. But you don't get things like generics or the .NET TPL without some serious long-term vision. I really do believe well-designed languages win over a long enough timescale.
I'm not necessarily saying it would have failed, but I do wonder :: )
Sure, the language being nicely designed and having some code translation tools helped people switch from VB.
But C# being the “best supported” language in Visual Studio. The productivity gains for developers were too real to ignore, managers saw it, and bought into the Visual Studio ecosystem en masse.
Not just the design, the real implementation too in the existing already complex CLR codebase. And that include lot of areas like NGEN, AppDomains, etc, so is no small feat at all, for a production ready framework already used by lot of developers.
I am programming in .NET Framework from v1.0, the v2.0 (with generics) was a clear cut, so is not something you can add too late because it become pervasive in the framework who leverage it in the design (generics in v2.0, LINQ in v3.0, TPL and Async v4.0) As a note, .NET continue to support non generic collections for backward compatibility, but usage is deprecated.
Really interesting piece of .NET history in https://blogs.msdn.microsoft.com/dsyme/2011/03/15/netc-gener...
I definitely agree, that's why I put it right at the start of the post :-)
Can someone explain why "in-the-VM" generics design is better than the erasure model of Java?
You could look at the head of the list to make that determination, (because the List members still have a concrete type), but that isn't a general purpose solution.
Languages like Scala use manifests to preserve the information but it's a major kludge.
Idiot.
- a List<Thing>
- a List<ThingSuperclass>
- a List<? extends Thing>
- a List<? super ThingSubclass>
- a List<IThing>
- a List<ILiterallyAnyImplementedIFace>
- a List<IHaventForgottenInterfacesCanExtendOthers>
- a List<Object>
- a List<I'm probably forgetting some possibilities>
So yes, it's a boundary, which is significantly more than nothing. But far less useful than it feels like at first.
His two points.
Being able to do type erasure is great because it means your language is coherent. (Don't need to do any nasty kluges in the run time)
And then everyone ratfucks themselves by actually implementing type erasure, and now your language will never have good tooling and thus will die a lonely unmourned death.
He commented that C/C++ has type erasure and it's also a massive kludge (via the elf format). No one will invest that much effort into a new language.
tl;dr: Type erasure, it's great, don't do it.
Imagine wanting to know what the object at address 0x34199920 is. If you have type info[1] you can cross reference the type info with 'generic array' and then do the same with element 2, address 0x34199920 and know it's a 'foo object'.
[1] Say the first 4 bytes is _always_ a type index that the compiler or interpreter generates.
I don't think this is correct. In C# code at runtime, and in the bytecode, you can inspect types for metadata about their generic type params. It is present in the .Net bytecode.
See https://docs.microsoft.com/en-us/dotnet/csharp/programming-g...
> When a generic type or method is compiled into (bytecode), it contains metadata that identifies it as having type parameters.
from https://docs.microsoft.com/en-us/dotnet/csharp/programming-g...
Where erasure can cause issues is at runtime. For example:
X isa List<Int>
Having said that in some respects this is a C# / Java ecumenical matter. Haskell, for example, doesn't bake types into output code so I suppose it erases even more that java, but it supports very rich polymorphic behaviour.
Note that this is not necessarily an issue for very statically-typed languages e.g. if I remember correctly GHC uses erased generics (they don't exist at runtime) which works fine because Haskell has neither pervasive RTTI nor specialisation, so the type erasure has ~no runtime visibility or impact.
The only benefit to the erasure model is it's easier to implement for the runtime.
By that count, so does C.
C has types that are only generic in the case of arrays.
There is a difference.
In Java I can write this function
<T> String getTypeOfArray(T[] arr) {
return arr.getClass().getComponentType().getName();
}
In C I can't write a function with that signature at all because it doesn't support generic functions.
And, in the case of F#'s type providers, it means not having to generate zillions of types when coding against some huge schema. But, on the other hand, it does give you the option to do so.
I'm not sure if F# is leveraging the DLR, but there is a lot of fun that can be had there in generating types on the fly, on par with the most dynamic languages out there. It is too bad that it has been nerfed with the UWP/AOT compilation push.
Yep, I knew about that, but type providers don't necessarily generate generic types. So, I meant something a bit different.
As you know, If Foo<T> is an ordinary generic, then T must be a CLR type, which is to say that it must be code that was somehow turned into a CLR type (usually by a compiler such as C#). But, if Foo<> is a type provider, then arbitrary code of an arbitrary language can go in between the angle brackets (which is why type providers can accept strings). Essentially, each type provider is a compiler, but the code that is generated must be expressed (or "wrapped", if you will) as a CLR type. But these types need not be generic (and usually aren't, I think). So, not having erased types in F# is akin to having a compiler that can't generate loops or subroutines†.
†(Perhaps a better analogy is a compiler that can't eliminate tail-calls, since erasing everything to a certain base type (that F# lets you choose) is kind of like reusing stack frames vis-à-vis the analogy)
Huh, I never thought about this restriction, so you can't add a new interface with 'Reflection.Emit', you can only add a class that implements an existing Interface, is that right?
> but unfortunately it requires going outside of the CLR or even the DLR.
By this do you mean writing raw CLR metadata yourself and then somehow injecting it into the running process or something else? Either way, its sounds like an interesting technique, any links to how it's done?
It was abandoned because interop was hard, and interop was Scala's strategy for success
Here's a few other examples of the same point, https://twitter.com/jon_cham/status/969929683587432450 and https://twitter.com/headius/status/958371298975080448
Ease of programming: your generic type bindings don't disappear at run-time and can be used run-time logic decisions, basically violating Bracha's law (types shouldn't influence run-time behavior), but I found it incredibly useful for meta-programming reasons.
Is there something special that type erasure adds, that you couldn't have Haskell without it? From a layman's perspective it seems that Haskell could be implemented with either an erased or a reified generics model under the hood, without changing the public surface of the language. But is there something that type erasure enables that reified generics does not?
It's not completely orthogonal as erased generics are annoying/problematic when the language does provide reflection/RTTI.
So you've got 4 (reification x reflection) states 3 of which are fine:
* if you have erasure and no reflection (Haskell) you're fine: you don't have runtime types but they don't matter/are inaccessible
* if you have reification and reflection (C#, C++/RTTI) you're fine: you can access runtime types and have them
* if you have reification and no reflection (Rust, C++/noRTTI) you're fine: you can specialise & discard types at runtime
* if you have erasure and reflection (Java) you're fucked: you can access types at runtime, but many aren't here anymore
> From a layman's perspective it seems that Haskell could be implemented with either an erased or a reified generics model under the hood, without changing the public surface of the language. But is there something that type erasure enables that reified generics does not?
A simpler implementation.
- Boxing/unboxing would not be an issue anymore, so you gain performance.
- Runtime types are enforced: List myList = new List<String>(); myList.add(new Object());
In JAVA, this is ok. In c#, it throws.
List myList = new ArrayList<String>(); myList.add(new Object());
this would compile.
It would be nice if mistakes like that were deprecated then eventually fixed, even if over many years/versions.
IList myList = new List<String>(); myList.add(new Object());
Most of the time when reflection is needed I see something like this instead of just a plain list type.
<T> void foo(List<T> list, Class<? extends T> type) { ... }Do you mean http://joeduffyblog.com/2011/10/23/on-generics-and-some-of-t... or https://blogs.msdn.microsoft.com/joelpob/2004/11/17/clr-gene...
They're 2 articles I came across that covered the low-level details of the generic impl.
It's been more than a decade since I last used C#, so excuse me if I recall incorrectly.
.NET chose to implement specialised generics. Many people think that is a better tradeoff.
I thought the Java implementation was almost a workaround to avoid the hard work of a true generics implementation.
The fact that C# generics are understood by all parts of the .NET ecosystem makes it so much better and boosts the language to a much higher level than Java.
C# and Java always seemed to be roughly the same to me until C# got generics, and that boosted C# to a much better plane of existence, which was followed by a great deal of innovation such as LINQ, async-await, lambdas, etc, all of which benefit from C# generics.
>I disagree. I think that C# generics are way better. The C# generics are supported at the virtual machine level, whereas Java just pretends to do generics. Java generic types are unknown to the virtual machine since the compiler just compiles a List<Foo> to a list of objects, leaving the VM ignorant of what it really is.
It is worse than that. Java generics don't work with value types at all thanks to this. A C# List<int> is much closer to a zero-overhead abstraction. It does not box the primitive int values. If Java ever adopts primitive generic types it will break compatibility or introduce massive performance overhead, as touching any non-generic API forces a boxing conversion. Either it negates the reason for type-erasing originally or it eliminates the main benefit of non-boxing value type support (performance).
(I should clarify that Java doesn't have first-class value type support either so this really only applies to primitives)
>I thought the Java implementation was almost a workaround to avoid the hard work of a true generics implementation.
It was a deliberate design decision to retain compatibility between pre-generic and post-generic Java code.
Personally I think that was the wrong tradeoff; there is never a better time to make a breaking change than right now. The cost only ever increases with time. It was also clear back then that Java would exist for far longer with generics than without and that far more code would be written in Java post-generics than pre-generics. The result is everyone who uses Java is stuck with limitations and negative performance impacts forever, rather than accepting some short-term pain.
It is also my personal opinion that source compatibility is what developers actually cared about and Sun should have told the stodgy risk-averse big Java houses to suck it up and get ready for JVM v2. It would have been a good opportunity to fix a few other things in the JVM.
C# had the benefit of hindsight in some ways. I'm not sure if Java demonstrates how open delivers inferior results, how bad leadership can impact a project, or just what happens if you pay attention to what "enterprise" customers claim they want.
Yes, but from a marketing perspective breaking with the past is usually not desired. Especially if your product is currently being adopted and is already partially adopted in both hardware and software (which might have been the case when they implemented generics?).
They did so because they knew the advantages to be massive. And so does everyone else at this point.
That golang decided to forego that wisdom when they started from scratch (and thus create a complex compatibility-story if they were to add it later) is entirely their own fault.
No. Because that would be much easier and was the only obvious answer.
They took a shortcut, hoped nobody noticed and later when people started complaining about this obvious omission in a modern language, tried to weasel their way out of with some “complexity” bullshit and how “people wouldn’t understand”.
So I can fault them for both not including it and then later being disenginious about why.
For this glaring omission the golang designers deserve nothing but ridicule.