Net 9.0 LINQ Performance Improvements | Hacker News Reader

Net 9.0 LINQ Performance Improvements

blog.ndepend.com

203 pointsolvy0216 comments

For me, the most useful part of LINQ is neither the IQueryable syntax tree based extension mechanism, nor the language integrated part (which I dislike), but simply the IEnumerable extensions. Originally known somewhat confusingly as linq to objects. Those allow me to write c# in functional style, keeping the code concise.

The post I submitted refers mostly to optimizations to those extension methods.

This clicked for me after having learned Haskell. It also shares some of Haskell's features/pitfalls, such as laziness.

There are pitfalls, sure, and honestly I wouldn't advise a team having no one somewhat experienced with basic functional idioms (including laziness) to use it. It can lead to obtuse and slow code if used indiscriminately. I try to lead by example myself.

C# has quite a few easy to use things that take a while to understand. In some ways it is a bit of an "experts only" language.

It’s hard to forget what you know and see through eyes of a beginner, but I definitely find C# to be a very “rich” language, in the sense of finding new features and ways of doing things the more I use it and as the incessant releases roll out. I like to think it’s still pretty easy to start with though if one is focused on the fundamentals. Lots of head room for progressively expert use, I guess.

This is how I use LINQ as well. With some non-standard names, it has everything you need! Eric Lippert wrote a great series on monads tying it to LINQ:

https://ericlippert.com/2013/04/02/monads-part-twelve/

Why on earth they named map as s Select and flatmap as SelectMany, I'll never understand. Still vert useful tho.

(I first learned FP with Scala, so the names from there feel the most natural to me, tbh)

Edit: it just occurred to me it has to be SQL inspired or something like that.

Yeah it was intended to be easy to learn to people familiar with SQL. LINQ2SQL and Entity Framework are basically a way to query your database by constructing an abstract representation of your query using strongly typed bindings in your application code. So then the extension method syntax of it was a great way to sneak functional list programming idioms into it. Watching the proliferation of functional style in the .NET ecosystem over the past 15 years has been pretty impressive, IMO.

Indeed, I worked in Scala for a while, dabbled with rust on my free time and am now back to writing C#. Although I prefer the two first, I have to say the ergonomics have improved significantly in C# and there are now nice options for rewriting legacy spaghetti OOP-code.

Map is Select and filter is Where to align with SQL keywords.

This is my preference as well, I love the functional aspects of the LINQ extensions for IEnumerable (and IQueryable)... Just easier to reason with IMO. Not always the most performant option(s) in for example (Entity Framework), but usually a pretty good option most of the time. I also like to use Dapper over EF.

Of course, C# projects tend to have a level of abstractions that are simply absurd to work with. "Enterprise" development makes my cringe more often than not.

> C# projects tend to have a level of abstractions

Such as? I think it really depends on the kind of company you are at (big enterprise, medium size, small company) and the field (web agency, startup, tech company, non-tech company, conglomerate etc). You can have abstractions in Java too.

If you read the test of the sentence you cut off, I specifically mentioned "Enterprise" development. I'm not a fan of most Java shops either.

It's not that you cannot use C# in a relatively light handed way, it's not that it often isn't.

Probably the most painful example in practice was The Enterprise Library Data Access Application Block... Which at the time was all the more painful in early VS were go to definition took you to an interface and never the implementation. There was always a single implementation and never any tests so it was layers of misdirection for absolutely no benefit whatsoever in practice.

I like C#... I don't like a lot of places that use C#. In particular large companies and banking in my experience.

Totally. I couldn't care less if the LINQ syntax had gone tomorrow, but functional composition is so powerful, and easier to maintain too.

I would miss it for `let`, and for multiple `from` clauses. These are things that can be done with extension methods and callbacks, just not as elegantly.

`let` is just a .Select(), isn't it?

A `Select` to a tuple or anonymous class which you have to carry around all the way.

If you look at how a LINQ query gets translated to a method chain (e.g. via SharpLab or let Rider/ReSharper rewrite them) you'll notice that multiple `from`s or `let`s can become quite ugly.

To add on to what you wrote, it amazes me the number of developers that don't understand you can use .Select() to construct named object instances. I'm probably using bad terminology, but something like:

    .Select(x => new Employee { Name = x.Name, DepartmentName = x.Department.Name }).ToList();

How do these developers use .Select() then?

Into anonymous types mostly, where you can't pass the type to another method. Example:

    .Select(x => new { Name = x.Name, DepartmentName = x.Department.Name }).ToList();

Same code as I wrote, except capture in an anonymous type. The documentation practically encourages this.

Oh that's convenient, that's for sure.

There are some queries that are actually easier to compose in LINQ syntax than even in SQL. However, like everyone else, I rarely find myself actually using it. Anything really complex is done in pure SQL and everything else is method syntax.

But it is slightly more expressively powerful than SQL and way easier to follow, if you need it, than the method syntax.

If you like haskell, you might* like some of the other applications of linq's query syntax, like combinatoric parser construction. [1] The query syntax isn't hard-coded to do stuff with IEnumerable, that's just what it does by default. It can be used for pretty much anything. It works a bit like operator overloading.

[1]: https://github.com/acple/ParsecSharp/blob/da8d0cb9ec39e28dd9...

I've only ever used the method syntax for LINQ. I'm not a fan of having another "embedded" language inside my host language, especially since what is returned eventually needs to go back to C#. When I'm not using an ORM like Entity Framework or Dapper, I still prefer to place my data access logic with SQL into a separate abstracted project so it doesn't spill across my application (and can be replaced if I were to require a different RDBMS, although this has only happened to me once in 20 years).

For more junior devs using LINQ, setting them up with a profiler and the debugger I believe helps makes more sense about what is going on behind the scenes. Sometimes it's helpful to have them code using for-loops and standard C# logic, and then compare to how you'd implement in LINQ, to see the positive and negative of both approaches.

I don't understand why the dotnet team doesn't invest more resources and time into tools: doctests and documentation generation, better and faster unit tests that I can write next to the actual code, easy access to the source code, no need to decompile a Dll to see it after pressing F12, a central place for packages and documentation like in pkg.go.dev or docs.rs . Most packages in nuget have no documentation at all, or only in readme format on GitHub, or a short Wiki on this page. Other environments like Rust, Golang, Java, Python are light years ahead when it comes to these aspects.

A part of me jokes that Micro$oft only invested in OpenAI because it's the only sensible way to browse .NET/Nuget package(s) documentation. The other part knows it's dreadfully true.

Small F# community have https://fsprojects.github.io/FSharp.Formatting/ that is a lot better than docfx, it has markdown support, notebook suport, script output embedding.

C# has nothing like that, xml comments and raw documentation that has no value

Sandcastle Helper File Builder has been around forever and started as an internal MS project IIRC, but for some reason few libraries use it.

https://github.com/EWSoftware/SHFB

I used to use this quite a bit after it was introduced, but CHM files are a bit unwieldy for documentation, at least for myself. Having integrated search is nice, but the windowed interface, and being tied to Windows (AFAIK, maybe there are readers for other operating systems) I believe will keep this project from being used. I always found it to be decent for more advanced developers, but felt there was something missing for more junior developers that needed documentation.

I haven't used it in 4-5 years but at the time we were generating html docs with it. IIRC it supports several other output formats.

I agree with you, but I think this is most likely due to how relatively new open source C# is. I'd hazard that most C# being written is still done by enterprises/companies as closed source. I feel this will improve over time, as long as Microsoft continues down the road of being open, and doesn't pull back what they've been doing in the last few recent years.

Some of these features are provided by tools like Resharper, and I wonder if there isn't some kind of agreement (whether written or unspoken) where they don't step on each other's toes. To be honest, most documentation I have seen written in C# projects still makes me reach for the source code due to poor quality. Having lots of autocompleted tooling doesn't help when it comes to reading, only writing, in my experience.

Very true! My first non-work opensource C# project is from 2020 and that seemed strange at the time

https://github.com/unfrl/dug

This is awesome, because most CLI tools for this don't seem to work anymore (at least in my experience). I need to talk to my employer, but I'd like to open source some work I've done for generating static redirects/rewrites in IIS format (still able to be used by open source .NET), as well as a tool that will take those redirects and visit the original URL, then check that a 301 redirect occurred, with the Location header set to the appropriate new URL. I work with a digital marketing agency, and we do a lot of SEO work, including making sure old URLs match up with new URLs. I got pulled into the creation of 301 redirects, so built tooling to take care of it for me.

Here’s a way to write tests next to the code: https://clipperhouse.com/go-test-csharp/

(Whether I recommend it, not sure! I did it and then undid it, with suspicion that tests were taking longer due to, perhaps, worse caching of build artifacts.)

In all their docs they now have links directly to the source for the method you are looking at, for example https://learn.microsoft.com/en-us/dotnet/api/system.string.s...

As for source for nuget packages, it's easy to enable source link to make that happen too, it's just that this is all pretty new so not every package is doing it.

"LINQ Performance improvements" really should read: "our own List<T> implementation performance improvements". Microsoft seem to spend their time improving what they need, rather than spending time on general improvements. LINQ (the syntax, not the method extensions) needs investment. Primarily around lambda allocation and potentially on compile-time reduction of lambdas.

It's time to have value-type local lambdas, or a strategy that doesn't make lambda allocation the overhead that it is. And also there really should be wildcard support (`_`) for LINQ variables by now. Which was been completely ignored when they were brought in for lambdas. It should also be possible to use a lifted-type (like IEnumerable<T>, Option<T>, etc.) as the final item in a LINQ expression, rather than `select ...`. The select adds overhead that isn't needed for certain use-cases and limits things like tail-recursive LINQ expressions.

Libraries like mine that go all-in on LINQ [1], but aren't using `IEnumerable` or `IQueryable`, or any of the LINQ extensions, continually get ignored because MS are focusing purely on improving the performance of their own projects.

A good example is the improved lambda inference. It was only brought forward because ASP.NET Core needed it for its minimal API. It seems like many of the features of the language/framework are driven by their own needs rather than those of the community. The absolute worst thing is the ever expanding set of 'magic methods' (like the LINQ extensions Select, SelectMany, and Where, but also GetAwaiter, and the rest). MS are adding capabilities for themselves (the compiler mostly) rather than do what is really needed and add proper higher-kinded traits to resolve the magic. So everything is weakly typed and only vaguely discoverable by the compiler :/

LINQ is one of those key differentiators between languages, yet it's been wallowing, pretty much untouched since C# 3. I think it's a crying shame that it has basically been ignored since then and, even now, they think LINQ is only useful for iterating lists. And primarily only their own list implementations.

/rant

Don't get me wrong, I appreciate all performance improvements, I'm sure it will help plenty of users. But the focus always seems to be narrowly targeted, which limits the potential.

[1] https://github.com/louthy/language-ext/

If you do have useful feedback, please submit an issue or contribute a PR to dotnet/runtime. That’s how many performance improvements in LINQ come to be, that the post discusses.

7 years ago: https://github.com/dotnet/csharplang/issues/1060

3 years ago: https://github.com/dotnet/csharplang/blob/main/meetings/2021...

I've given up submitting to csharplang now. It seemed like a positive when it was first open-sourced, but the proposals/issues are often just circular discussions that go nowhere. It doesn't feel like (at least from the outside) any proposal by a non-MS employee would make it thru the process unless MS needed it themselves. Which is what I was alluding to in my original post.

I may well be wrong and maybe some proposals have made it through, but it feels like an illusion of community collaboration rather than actual community collaboration imho. I don't expect preferential treatment or anything like that, but it would be good if one of the most standout features of C# got some proper attention after a decade of neglect.

There has been a lot of effort on performance in many areas of the framework and the language. Leveraging that work for LINQ (the language feature) would be very welcome.

C# is already big enough language. This is more about .NET in general and less about C# specifically.

From what I've heard - there is an awareness that the current cost of delegates, while not that problematic anymore, can be improved. Whether this happens via explicit syntax to enable (ref) struct closures and value delegate pattern implementations or via object escape analysis and better devirtualization remains to be seen.

p.s.: I really wish that instead of LanguageExt, companies would have adopted F# instead.

> p.s.: I really wish that instead of LanguageExt, companies would have adopted F# instead.

Ah the standard "why don't you just use F#" line. I have a standard response to that now ;)

https://github.com/louthy/language-ext/wiki/%22Why-don't-you...

Thanks, I did read the reply before but remain skeptical. The reason for this is "strictly-managed" teams are also often resistant to libraries in a similar way. Fully adopting a high-end solution like LanguageExt can be seen as significant commitment. If a business case justifies that, it means that adopting F# could be on the table too. And unlike Haskell, integrating "functional core" into existing .NET solutions is very easy and has low amount of friction, much less so than people tend to assume upon hearing the suggestion.

The C# experience already got rug-pulled with Dev Kit.

I'm not going to spend time working for free on tools that MS charges a subscription to use.

You are confusing devkit, which uses vs license, with base C# extension which is free.

Also

https://news.ycombinator.com/item?id=41884187

https://news.ycombinator.com/item?id=41809351

https://news.ycombinator.com/item?id=41639205

https://news.ycombinator.com/item?id=41384888

Nobody complaining about Dev Kit would be unaware of the base C# extension.

Your library looks very interesting, but I think you might be setting things up ahead of time to be dismissed. There are a large number of using statements, which is not a big deal for those who understand breaking up projects into appropriate pieces for exactly what you need, and keeping boundaries between different concerns. Most developers don't structure their projects the way that you do, and I say that as a positive towards your implementation, but often little things like this can hold back "the most common-denominator" when it comes to developers. I will admit right off the bat that you are far more well-versed in C# and .NET than I am, but I'm coming to this as someone that has worked with lots of junior developers that already struggle with standard LINQ syntax, methods, and (in particular) performance.

You do mention these things in your README, and I appreciate it, as most are set on "selling" their library instead of actually listing what it exceeds at and the purpose. Your comment on being non-idiomatic is another issue that I see (again, talking about people learning and getting used to C#/.NET). Only in my opinion, I feel like Microsoft want to keep their tools and languages following certain practices, and I get that your naming follows more functional programming practices to make them natural to FP. I feel like often, little issues like this are very large to Microsoft when it comes to looking into improvements.

I starred your repo, and am very interested in what you've put together here. On the last few major applications I've built (I normally build content management systems that are highly customized, that I tend to not call apps), I've used a Result<T> type that seems to be roughly the same as Option<T>. I wrote that sentence, and then went back to look at your library, and as much as I thought I knew about FP, I really don't. I think I'm really decent at C# and doing some complex things with it (I mainly use Umbraco CMS, and I've gotten my code to run websites under the minimal recommended requirements of the base Umbraco software). With that said, I read a lot about FP, but still have tons of issues picking it up. F# For Fun And Profit is the closest it's made sense to me. In the end, all this is to say that it's nothing that you're doing wrong. Microsoft is going to target the average or beginner developer, because that is the vast majority of developers that they have in their environment.

I hope that you library can gain some internal improvements, because it looks like you've spent an enormous amount of time on this, and you have enough stars on GitHub to at least indicate that people are actively using your library and getting benefit from it. I apologize if any of this came off as dismissive, I think what you've done here is exciting, and feel like you've set up enough documentation where I can slowly pick up concepts that I'm unfamiliar with.

re: usings. These are the only ones that are necessary:

   using LanguageExt;
   using static LanguageExt.Prelude;

I list all of the others so that people can easily add them to their global-usings. The key is `using static LanguageExt.Prelude`, because static-usings aren't flagged by tooling.

It's a conscious choice not to pander, I don't mind if the C# FP community doesn't include everybody in the C# community, I'm simply presenting an opinionated approach that is close to the 'norms' of FP languages/frameworks rather than trying to fit into the C# conventions. It's intentionally going 'all in' rather than just trying to augment the existing approach with types like Result or LINQ (the extensions). Most of the benefits of pure-FP come from a complete change of approach; I'm saying "leave the previous C# world behind and let's try a different way". Some people really won't like it and that's fine. In a company it needs an advocate that pushes the narrative, if that isn't there, then most won't get past the use of Option, or simple stuff that doesn't move the needle much.

I've been a CTO for 20 years and have mentored many juniors in my time, including in FP. A willing junior with a good mentor will find FP easier than OO once they grok it, because pure FP becomes more intuitive eventually (as it's based on maths, so you get to a point where - if your code compiles - you feel like it must be right, which is very powerful reinforcement for everyone, but especially for juniors).

I appreciate your work, and the stance you take with your library. Trying to cater to too many audiences can often make using a library feel odd or half-baked.

The more C# borrows from F#, the happier I am. I am awaiting for discriminated unions to finally reach C# so I can do domain modelling like a boss. :)

You can largely already pretty easily incorporate DUs using OneOf[0] and Dunet[1].

A practical example of using this: https://chrlschn.dev/blog/2024/07/csharp-discriminated-union...

[0] https://github.com/mcintyre321/OneOf

[1] https://github.com/domn1995/dunet

I'd kill for units of measure too. It makes maintaining any kind of engineering or scientific code so much easier.

In a past project, we used Units.NET. It worked pretty well as far as I could tell. (I was not involved with the gritty domain details.) My colleagues even extended the library where units were missing.

In my experience units of measure in technical apps is only interesting in the UI layer as most apps I’ve worked on have a “base” system of units that all calcs are expecting. What alternate experience have you seen?

Units of measure in F# are useful for more than unit conversion workloads.

They are compile-time type information with no runtime overhead. So if you're using a lot of interrelated arrays, you can make a unit of measure for indices into just a specific type of array. This allows type-level enforcement of never using the wrong index for the wrong array.

Anywhere you might want to keep categories (not category theory) of numeric values distinct from one another without intermixing, units of measure in F# can cover you.

I do find this an interesting thing often said in the .NET community. Yet the question "Why not just use F#" comes up in my head every time someone says this. C# has been playing catchup for many years.

If it is F# that is pushing a lot of innovation in the .NET ecosystem and many years in features ahead (at least it was a few years back) why not reward that effort through its use? We should encourage the behavior we want out of these language development teams after all by voting with our feet. Worked with the Java ecosystem for example - even Java now is improving. It would create a positive feedback loop (like most other innovative tech) where more market would create more engineering effort on it as well.

Reading these forum's over the years it feels like many C# people really want to stick in their camp and wait "just because" - it almost feels "tribal" and their team is "C#". I just don't see this culture in some other language ecosystems - makes me think if F# was in any other ecosystem than .NET it would of thrived a long time ago. That's just my impression though.

>"Why not just use F#"

Because we don't really make choices and the C-level executives think that only C# or Java are worth using since they are the behemoths of the industry.

And I am quite content we don't have to use Java with all that object everything, factory factory factory and plainly insane "design patterns".

It's an open secret and cited many times by officials (eg Hanselman) that F# is the testing playground for C# (and vb.net) features.

Whenever I work in other languages/ecosystems LINQ is the one thing that I really miss. It's just such a nice capability to have available in the standard library. Beautifully designed given the constraints it had to work within.

Relevant section in the annual book-sized post on all performance improvements in .NET 9:

https://devblogs.microsoft.com/dotnet/performance-improvemen...

(for some reason HN would not allow to resubmit it, and the post was left by wayside not reaching frontpage)

Beware, once you get used to using LINQ and usually work in a domain that it shines, you won't want to go back to using anything else.

Do not, my friends, become addicted to LINQ. It will take hold of you, and you will resent its absence.

It’s still less powerful than something like polars

Anyone knows of a comprehensive book/tutorial to learn end to end web development in dotnet. Most I have found are elementary, outdated or poor quality.

I've learned by doing, but here you go.

Books: C# 12 and .NET 8 - Modern Cross-Platform Development Fundamentals - Eighth Edition: Start building websites and services with ASP.NET Core 8, Blazor, and EF Core 8 by Mark J Price

Web API Development with ASP.NET Core 8: Learn techniques, patterns, and tools for building high-performance, robust, and scalable web APIs by Xiaodi Yan

Turorials: series of tutorials on YouTube by IAmTimCorey and Shawn Wildermuth.

That is because all the hot-and-new in .NET web development is Blazor, and it is not really popular outside of Microsoft's Blogosphere (and IMHO never will be and go down the same way as Silverlight did). The "venerable" technologies are still present in .NET 9 and still work, get maintained etc.

Doing web dev in .NET nowadays for me is mostly creating HTTP/JSON/REST APIs and using whatever FE framework you like to interface it (for me that is React or NextJS). The keyword you want to search for is "ASP.NET WebApi" or - more modern - "ASP.NET minimal API".

You could still do .NET MVC server-side rendering using "Razor" (the markup language in ASP.NET MVC - search for "ASP.NET MVC Razor".

Blazor for better and worse can't have the same ending as Silverlight did. In Server Mode it is just Razor + SignalR for HTMX-style frontends that sort of live update/"magic RPC". In Client Mode it is just Razor as a virtual DOM-ish framework running directly in Web Assembly (wasm) in the Browser. At the end of the day Client Blazor is much more like writing front end websites with Elm than writing websites for the Silverlight plugin.

(Hybrid Blazor where you use both server and client possibly in the exact same assembly/project is where most of the worst Blazor mistakes lie, but those aren't the mistakes of Silverlight, those are the mistakes of ASP Classic before .NET where you can be easily confused by new versions of the which bit of this is `runat="server"` and which bit is `runat="client"` and easily and accidentally create wild RPC ravioli/spaghetti code.)

Do you know of any performant, quality examples of Blazor out in the wild? All the examples I've seen have unacceptable UX by modern standards.

https://github.com/Webreaper/Damselfly

> All the examples I've seen have unacceptable UX by modern standards.

What does that have to do with Blazor?

One could easily write outdated UX in React if they so choose.

It's the slowness/sluggishness, not the looks. Most of the time even a simple interaction like clicking a checkbox takes ~1-2 seconds, which probably comes from tracking the state on server.

(I don't mind outdated for .NET stuff, nearly everything from enterprise vendors looks like it just jumped out of WinXP anyway.)

I've never seen that behavior, even in the preview/beta versions. That example is less complicated than the sample project that you get when you create a brand new project from the visual studio template (it's something like increment a counter on the server side via button clicks iirc).

There were definitely some quirks and issues early on but they've done a pretty good job at smoothing that stuff out since it's gone through two LTS version of .NET.

The pipeline for something like that is click->js/wasm websocket->server code->websocket->ui updates. If you're doing something absurd on the back end when a checkbox is clicked, sure 1-2 seconds but that's a developer problem, not a Blazor problem. If you put me in front of a React app, I'd probably do something stupid like that too because I don't have much experience with it.

I suspect your complaint is more related to .NET/Blazor making it easier for less experienced developers to develop a working site/page with a lot of spaghetti where that same developer would struggle to create the same page in another ecosystem. If you compare to equally senior/experienced developers in different frameworks, I suspect you'd see the same basic performance from the two platforms up until a scale that very few of us are actually working in. Blazor can be quite fast if you don't write bad code.

I find a lot of the complaints about .NET to be that it enables developers who write bad code to get things done inefficiently instead of not being productive at all. IMO, having senior developers doing code reviews is pretty essential if you have junior developers on a team but the ability to build something, even if poor, can really accelerate the learning process when a senior person can look at a complete project, point out the issues and show the difference that the corrections make.

Sorry for the rant and sorry if this doesn't really apply to you specifically.

I get where you are coming from, and I'm on the same page. I'm not complaining about Blazor, I'm complaining about not finding any good examples. That's why I'm asking around.

I'm looking for something that will make me "wow, Blazor can be used to build great apps!" or "ok has advantages over the React/Vue etc. equivalent".

Because if there aren't any, in similar thinking with parent, I'm afraid that it won't catch up and it'll go the way of silverlight, so I refuse to invest any time.

The best Blazor app i've seen is nodexr.

https://www.nodexr.net/

https://github.com/Jcparkyn/nodexr

That's what I was looking for, thanks!

> 1-2 seconds

Something is wrong there, probably someone is screwing up form validation in their component framework or something, but IME that’s not a Blazor thing.

Certainly I also can't imagine how you would get strong performance out of Server Blazor in real world/Enterprise conditions. RPC busses are going to RPC and roundtrips are still roundtrips even when SignalR-backed web sockets. Flooding UDP web sockets with HTML chunks isn't necessarily a great idea, even if HTTP3 (QUIC) kind of/mostly succeeds at it, but SignalR as good as it is certainly isn't HTTP3.

On the other hand, Client Blazor has huge initial overhead (it compiles a form of the entire .NET platform in WASM; you can golf it down a bit, but it's still an entire runtime and GC and most of a standard lib depending on what your app does, and its dependencies), but once it is loaded it can be as performant as just about any other Virtual DOM-like (modulo the overhead between DOM/JS and WASM, but that overhead is surprisingly small in current browsers and seems to only get smaller as browsers continue to optimize WASM scenarios).

I don't think I'd recommend it for most consumer-facing websites any time soon, but in my experience Client Blazor is still much better than Angular for performance in an Enterprise/Internal App scenario.

Thanks, that confirms my initial findings.

I’ll look for an opportunity to give client Blazor a try.

Blazor Server in an enterprise (intranet) is extremely fast because the network is extremely fast.

Across the Internet, it's still quite fast if you use Azure SignalR service, which is effectively a globally distributed CDN. Most commercial apps use this service.

Blazor WASM is better than I thought it would be. My company has built a healthcare SaaS app that has ~5k daily users on it and no one has complained about the initial rendering and download time to pull down 25MB on the initial load. This sounds like a lot but with modern broadband, 5G, and CDNs, this takes about a second or two.

>IMHO never will be and go down the same way as Silverlight did

There is a difference: Silverlight was not open-source, whereas Blazor is open-source on both server-side and WebAssembly (WASM).

ASP.net MVC has reached maturity and is notably used in project like NopCommerce.

Also worth looking into are some of the extensions for Razor/HTMX views, which IMO is a way better approach than server-managed Blazor. Jetbrains has a few good videos and really nice dev extensions for this approach.

I also think wasm Blazor is a non-starter as well until WASM GC integration in the browsers gets fully integrated for C# targets.

https://www.youtube.com/watch?v=uS6m37jhdqM

Why is this still a problem within the dotnet world? Why aren't there a dozen popular application frameworks to choose from like there is for Java and most other languages?

In dotnet, it seems everyone uses the Microsoft "blessed" framework, until Microsoft does a full rewrite or loses interest - then everyone has to rewrite their stuff too.

There's no way Microsoft are the only ones capable of producing good library/framework ideas - so what gives?

Because most of .net use is enterprise so people using it go with “no one ever got fired for buying IBM” way of doing things.

If project goes belly up no one can pin it on you choosing something non standard.

So it is a feature of dotnet ecosystem not a bug “doing stuff the MSFT way”

Do you really need that many choices? I always thought that proliferation of back-end implementations in Python, routers in Go and large alternatives in Java was mostly a sign that traditional "default" choices in the ecosystem are subpar.

ASP.NET Core is strong and overall better option when compared to FastAPI, RoR, Spring or Node.js with Express/Nest.js. EF Core with LINQ and safe SQL interpolation is one of if not the best ORMs that exist on the market across all programming languages.

This results in much weaker demand for alternatives, and combined with community that historically erred on the side of "out of box" choices, it's no surprise that there aren't really big viable alternatives to ASP.NET Core. There are for EF Core though - Dapper/DapperAOT and Linq2Db. Aside from that, most community libraries, both C# and F#, instead usually opt to build on top of them, like Fast-Endpoints or Giraffe/Oxpecker.

.NET could use a framework that is what Vert.X is to Spring, but even existing landscape is so much better than almost every other alternative, that it's difficult to complain, except, I suppose, when it comes to Identity and Authn/Authz in general that are more complex than ideal.

p.s.: I'm not saying that all "first-party" libraries are good. ASP.NET Core and EF Core, especially their "core" functionality are great. The surrounding selection of packages however can be hit or miss, and it's where a productive choice is to look for community libraries instead. It's an area that is slowly getting better.

I've recently taken an interest in web development using C#. It seems to me that ASP.NET is THE way for creating web applications in .NET, for better or worse ("for worse" since lack of alternatives sounds a bit suspicious to me...).

Anyway, I came across a podcast featuring the author of "ASP.NET Core in Action", Andrew Lock, and he seems to know his stuff. I haven't read the book yet, but maybe this is the book you are looking for.

1: https://dotnetcore.show/season-6/navigating-the-aspnet-core-...

2: https://www.manning.com/books/asp-net-core-in-action-third-e...

There are alternatives, but most of them have been "consumed" in that ASP.NET does what they do better (most recently "ASP.NET Minimal API" removed a lot of the appeal for many of the smaller alternatives), or "consumed" because "ASP.NET" the brand also includes the low level HTTP machinery at this point and alternatives are increasingly "ASP.NET + some extra framework" which doesn't sound like an alternative, or are focused on small niches and other languages. There's a lot of fun alternatives in F# (Giraffe is the first one to my mind), but obviously that is no longer web development in C#.

Modern Asp.net stole a lot from a little web framework called NancyFx. It really shaped what Asp.net Core is today.

I agree, "Minimal API" owes a lot to Nancy, specifically. There were a few others in that space, but Nancy was definitely the big one that pushed ASP.NET the most.

I really wish suave wasn’t abandoned. We could have such nice things in F#-land but all of our the cool stuff gets dropped in favor of veneers over ms frameworks.

Suave isn’t dead - it’s more “complete”

What is missing? I might make a PR :)

For server-rendered UI, look for resources that uses Razor, ignore Blazor stuff at the beginning.

For .NET Backend/JS Frontend, look for resources that uses minimal APIs. MVC is also good but has a lot of backwards compatibility baggage that led to the creation of minimal APIs.

A bit of the beaten path, but F# with Fable is a very powerful combination.

On the server you can run ASP.Net using Giraffe, which is a Functional Programming layer with comparable performance to C#.

On the front-end, you can write React in an true Functional Programming language.

And of course you can share F# code between frontend and backend.

An architecture I've been using lately is writing a functional core in F# and importing the library in both C# API backends as well as React frontends. As you know, Fable can compile to TS which you can import into your node application.

This means that you can have a team of C# developers writing in a language they are familiar with, a team of node/TS developers writing React and a team of F# developers working on a pure functional core with all of the business logic. Write your validators in F# can you can share the same logic for a form in the UI and an API endpoint on the backend.

In my opinion having type inference, discriminated unions, computation expressions, et al., makes for a very concise and expressive way to model the core logic of an application.

There has to be a better way to do things than this annotation noodle soup. My eyes are bleeding every time I look at modern Net code.

Those attributes correspond to the benchmarking library used in the article. Unit testing & benchmarking code does typically look kind of like a plate of spaghetti.

That said, you would never get a PR through me that does this in the actual business logic. You can use things like AspNetCore without touching a single attribute if you really don't want to.

What .net code are you looking at? I rarely use attributes at all.

> Some more optimizations can happen when the chain ends up with methods like Count(), First(), Last(), ElementAt() or Sum(). For instance, OrderBy(criteria).First() can be optimized to execute as Min(criteria).

As useful as that might be, we should really be writing better code. This is interesting for dynamically generated chains, I suppose. But if these operations are done on bespoke code, this is kind of perverse positive reinforcement. The library itself is recognizing a sub-optimal pattern and correcting for it. I hope there's feedback, at least, that suggests improvements to the underlying code.

I have fond memories of using LINQPad

That's why I guess today I work in Clojure :)

It’s been consistently one of my favorite tools for a good decade. I don’t know how much Microsoft is paying them to keep it Windows-only because that’s the only reason I keep windows around, and I’m sure that goes for a number of important developers.

There’s a Mac version coming (very exciting!)

https://x.com/linqpad/status/1820379112651026587

I'm using NetPad on Linux. Still far away from LinqPad but it's pretty good.

I can relate. I used .Net at the beginning of my career, and that one was one of the few nice things. I enjoyed building in C#, until one day I realized that MSFT is more like a mental prison (apologies for the strong negative word) - mind that it's my personal feeling at that time, many things have changed positively since then, although I moved on already.

Also, I too, use Clojure today - it is very nice.

LINQ is so fucking useful and well designed feature of .NET ecosystem that it is unreal when you gotta use lang which doesnt have such a thing.

C# design team is/was unparalleled

LINQ is largely based on FP stuff, also how Smalltalk collections work.

It is relatively easy to find similar capabilities in most languages nowadays, unless one is stuck on Go, C and similar.

Do you know an equivalent for Linw to EF in kotlin or Java because I have not found it.

Those are linq expressions. They are indeed wonderful. You get an abstract tree from which you can create SQL or API commands to access the data source. I remember in the early days (.NET 3.5?) there were multiple examples of LINQ2X like Linq2Csv, Linq2Rss Linq2Drobox (I'm paraphrasing, I don't remember actual examples, but it was wild).

There is also relinq library which transforms linq expressions into expressions which are easier to understand/use.

Rather quick search, as I am more of a myBatis person,

Java: https://www.jooq.org/

Kotlin: https://www.ktorm.org

Any pointers to such libraries in python?

itertools would be the starting point, unfortunelly Python is rather limited due to the way it only supports one line lambdas.

No it isn't easy to find similar capabitites in java, go, python, ruby.

Maybe you do simulate some of this using meta programming in ruby but its certainly not 'easy to find'.

It certainly is, unless you are talking about SQL like syntax, which is basically syntax sugar for classical FP.

And I explicitly left Go out of my list.

>unless you are talking about SQL like syntax

yes thats what linq is?

https://learn.microsoft.com/en-us/dotnet/csharp/linq/

"Language-Integrated Query (LINQ) is the name for a set of technologies based on the integration of query capabilities directly into the C# language." With LINQ, a query is a first-class language construct, just like classes, methods, and events.

doing this in java is not LINQ imo

  List<Integer> lowNums = filter(toList(numbers), new 
  Predicate<Integer>() {
        @Override
        public boolean apply(Integer n) {
            return n < 5;
        }
    });

To be fair, 99% of linq usage is method chaining syntax, not query syntax

LINQ = Language Integrated Query

Its even in the name. What do you mean by "method chaining is linq" ?

Considering the methods live in the System.Linq namespace, I think the extension methods may also be called LINQ.

"var results = list.Where(x => x.Salary > 12345).Select(x => x.Name).ToList())"

Giant majority of ppl refers to this when talking about LINQ.

But yea, it is LINQ method chaining.

SQL like syntax is LINQ query syntax

> But yea, it is LINQ method chaining.

You mean like fluent interface? https://en.wikipedia.org/wiki/Fluent_interface

What does this have to do with LINQ or C#. I remember doing 'method chaining' in 1990s .

>fluent interface

Various names, same concept.

"fluent interface is an object-oriented API whose design relies extensively on method chaining."

>What does this have to do with LINQ or C#.

Check the name of the namespace where all those APIs like Where, GroupBy, etc. are implemented, it is "System.Linq"

So thats why majority of ppl think about them when talking about LINQ.

Query syntax has like less than 1% of the "market share" versus method chaining style

LINQ is the name for the overall system. LINQ can be written using two different styles:

    // Method syntax
    var evenNumbers = numbers.Where(num => num % 2 == 0).OrderBy(n => n);

    // Query syntax
    var evenNumbers = from num in numbers
        where num % 2 == 0
        orderby num
        select num;

Method syntax and query syntax are both part of LINQ (query syntax is syntactic sugar). .Net developers tend to overwhelmingly prefer method syntax.

These days it would be

    var lowNums = Arrays.stream(numbers).filter(n -> n < 5).toList();

2024's Java is also quite a bit better than 2013's Java.

Which still isn't as nice as LINQ, but this way we've painted the alternative in its best light, not in the light that makes C# look the best.

One of the unfortunate pieces missing from Java's streams is the ability to easily extend them with additional operators. In C# they are all extension methods for the IEnumerable interface so you can add your own methods into the chain but that's not possible in Java.

It is coming, it is called Gatherers, another approach, same result.

I'm a huge fan of Gatherers, lovely API!

Good that there's a solution coming up but it's not as nice as C# IMO. Gatherers are way more verbose both in usage and especially in implementation. In C# they can be written as generator methods which is a very intuitive way to work with streams of data.

i posted that example from a comment that linked to this repo https://github.com/mythz/clojure-linq-examples

my point was that laguange support for sql like sytax is part of what makes LINQ linq. Java niceties is not relevant.

I guess the very fact that LINQ is so many things makes such discussions a bit annoying at times because everyone refers to a different set of features.

Is it the SQL-like query syntax? LINQ to objects? LINQ to SQL? Expression trees in general?

Expression trees and LINQ to SQL/EF/etc. are hard to find elsewhere. The query syntax often doesn't seem to be that big of a deal, especially since not all methods are available there, so pure query syntax often doesn't work anyway.

> yes thats what linq is?

The link that you gave says "LINQ is the name for a set of technologies" which includes the "SQL like syntax".

Includes is not the same as "is".

It isn't the most often used part of LINQ.

sure but you cannot just remove 'sql like syntax' and claim you can do linq in any language.

You can in fact ignore "sql like syntax" and find "similar capabilities" in many other languages. After all, most C# coders ignore the "SQL like syntax" in C# itself.

And when languages imitate features of a different language, they tend to go for the features that people like and use. No-one is going to add "similar capabilities" to the feature that no-one wants in the first place. People who say "C#'s LINQ is awesome!" just aren't talking about "sql like syntax", and people who say "without sql like syntax it's just not on the same level as LINQ" are misguided.

I discovered someone stuff in Java pre-history days.

Because I am feeling nice,

     Arrays.stream(new int[]{1, 2, 3, 4, 5, 6, 10, 23, 45}).filter(n -> n < 5).forEach(System.out::println)

There are easy ways to do some subset of what LINQ does in Java (using annotation processors), in Go (using generators), in Python (using double-underscore methods to capture all the operations the expression is working with at runtime, see SQLAlchemy) and in Ruby.

There isn't a seamless way to do what LINQ does in any of those languages. But if the runtime supports a LISP then you can do more than what LINQ does (Clojure for the JVM, something like zygomys for Go, Hy for Python, and ... well, Ruby for Ruby).

One difference with LINQ is its ubiquity. It works with database, in memory data structures, on disk files. You can use your skills/code across all the system.

It's just built on top of anything that is Iterable. If a language has first class iterator support, they could do something similar.

Mainly first-class functions I think. If you have those, you can just use fold in the core combinators.

Takes a lot more than that, LINQ providers work by accepting a LINQ Expression Syntax tree instead of an opaque function, which allows providers to inspect and traverse the Expression's AST and translate it into the data source it's implementing.

This Expression AST is constructed by the compiler, not something that can be tacked on by a library later.

Yes, but I think the point is practically every high level language can already do this pretty trivially.

If it's scripted you can typically just get a string representation of the function.

If it's Java, JAR inspection/dynamics have been a thing for a long time. And in other languages, they usually directly support metaprogramming (like Rust) and plugging code into the compilation logic.

If it were trivial you'd see LINQ-like providers implemented in "practically every high level language".

Source code of the function means you have to implement the parser/lexer to convert it into a usable AST which is bad for both runtime performance and library size.

Very much doubt this is available in Java, which Java ORM lets you use native Java language expression syntax to query a database?

jOOQ would be one such example, https://www.jooq.org/

Not that I use this, I am a myBatis person in what concerns database access in Java, and Dapper in .NET for that matter, not a big ORM fan.

And case in point most people use LINQ for in-memory datastructures, not the database part.

This is a custom expression language to work within the expressive limitations of the language:

    create.select(BOOK.TITLE)
      .from(BOOK)
      .where(BOOK.PUBLISHED_IN.eq(2011))
      .orderBy(BOOK.TITLE)

If Java supported LINQ you'd be able to use a more intuitive and natural Java expression syntax instead:

    create.from<Book>()
     .where(x -> x.publishedIn == 2011)
     .orderBy(x -> x.title)
     .select(x -> x.title);

Java streams are what you're looking for.

If you insist in telling LINQ === EF, well that isn't what most folks in .NET use System.Linq for.

And back to the ORM thing, jOOQ is one way, there are others, and even if it isn't 1:1 to "from x select whatever" the approach exists.

> If you insist in telling LINQ === EF

I don't use EF, nor have I ever mentioned it.

You're replying to a thread about what it takes to implement a LINQ provider, which was dismissed as every high level language implements it with iterables, then proceed to give non-equivalent examples.

IQueryable<> manipulation has other tools available to it than brute-force iteration, like streams do. Streams may be the closest thing java has, but it's still a fundamentally different thing.

I mean they kind of are. You can find a library in almost every language that transpiles source code ASTs.

They're just not core features.

In Haxe, it's extremely common :) but Haxe is just one other high level language.

Wait what? Am I gonna include a source code parser and AST analyser to my JavaScript library for example, to examine the provided expression source and do this? This reads like the infamous Dropbox comment from when it first got released.

You could also bundle your JS. Or pretend like any number of other solutions like caching parsed ASTs exist instead of being as obtuse as possible, or something idk

Not that I agree it's trivial but even if it was, so what?

This just feels like sour grapes.

Having used it since its inception, I've come to the conclusion that the SQL translator is kind of a misfeature. It creates so many weird bugs and edge-cases and tedium.

I love LINQ, I love having a typesafe ORM as a standard feature of C#, but the convenience of being able to reuse my Pocos and some expressions for both in-memory and in-SQL don't outweigh the downsides.

If I were designing SQL/LINQ today, I'd keep the in-memory record classes and in-database record classes distinct and use some kind of codegen/automapping framework for keeping them synched up. Maybe allow predicate operators to return things other than booleans so we could make `a == b` return some kind of expression tree node.

For ad-hoc queries using anonymous classes? Support defining an interface inline in a generic so you can say

    public T MyQuery<interface {string Firstname{get;set;}; string Lastname{get;set:}} T>();

Like, to elaborate, if you were doing some kind of JSON-based codegen (alternately you could do something where you have a separate hand-written POCO Model assembly and use reflection against it to generate your DbModel classes so it's still Code First). Yes, I know MS tried and abandoned this approach, I used LinqToSQL and EF3.5 and whatnot and suffered all that pain.

like, your master datatable file would be something like

    ```json
    "tables" : [
      "persons" : {
        "dataRecordClass" : "DataRecordsNamespace.DbPerson",
        "objectClass" : "PocosNamespace.Person"
      },
      "columns : {
        "PKID" : {
          "type" = "integer",
          "isPrimaryKey" = true,
          "isAutoGenerated" = true,
        }
        "firstName" : {
          "type" : "nvarchar(255)",
          "allowNull" : true,
        }
        "lastName" : {
          "type" : "nvarchar(255)"
          "allowNull" : false
        }
      }
    ]
    ```

which would generates something like

    ```cs
    public class DataRecordsNamespace.DbPerson : DbRecord {
      public DbPerson() { throw ThisIsAFakeClassException(); }
      public DbInt PKID{
        get => throw ThisIsAFakeClassException();
        set => throw ThisIsAFakeClassException();
      }
      public DbNVarChar {
        get => throw ThisIsAFakeClassException();
        set => throw ThisIsAFakeClassException();
      }
    }

    public partial class PocosNamespace.Person {
      public AutoGenerated<int> PKID{ get; init; }
      public string FirstName { get; set; }
      public string LastName { get; set; }
    }

    public class MyDbModel : DbModel {
      public DbTable<DbPerson> Persons => DoSomeLazyStuff();
    }

    public static class MyDbContextExtensions {
      public static List<Person> Resolve(this DbQuery<DbPerson> dbPersons) 
      {
        //call code to execute the actual query.
      }
    }
    ```

Am I making sense? Then you wouldn't have the problem of "oops I used an untranslateable method or member of Person", because MyDbModel can't have any of those. You'd lose the ability to to switch from whether a query is in-memory or in-database just by removing the ToList(), but I'd argue that's a misfeature, and better-handled by having some kind of InMemory implementation. Like, having DbQuery have a simple `.ToLocalMemory()` function that is a hint that the next part should be done locally instead of in the database would be a better way to do that. Then you could still do

    ```cs
    List<Person> myPersons = connection.MyDbModel
      .Persons
      .DoSomeInDatabaseQueryStuff()
      .ToLocalMemory()
      .DoSomeLocalMemoryStuffToOffloadItFromDatabase()
      .Resolve()
      .DoSomeDotNetStuff()
      .ToList();
    ```

edits: fix some of the HN pseudomarkdown

Guess everyone has their preferred style, I personally avoid code-gen data models like the plague and much prefer code-first libraries.

Here's how you'd do something similar in our OrmLite ORM [1]:

    public class Person
    {
        [AutoIncrement]
        public int Id { get; set; }
        public string? FirstName { get; set; }
        [Required]
        public string LastName { get; set; }
    }

Create Table:

    var db = dbFactory.Open(); // Resolve ADO.NET IDbConnection
    db.CreateTable<Person>();  // Create RDBMS Table from POCO definition

Execute Query:

    // Performs SQL Query on Server that's returned in a List<Person>
    var results = db.Select<Person>(x => x.FirstName.StartsWith("A") && x.LastName == "B");

    // Use LINQ to further transform an In Memory collection
    var to = results.Where(MemoryFilter).OrderBy(MemorySort).ToList();

Everything works off the POCO, no other external tools, manual configuration mapping, or code gen needed.

[1] https://docs.servicestack.net/ormlite/

My problem with this approach is that this falls apart if you write:

    db.Select<Person>(x => Regex.IsMatch(x.FirstName, "^A.*"));

This would fail at run-time instead of compile-time.

That's why I'd rather see the DB classes auto-generated with a mapper to convert them. Having the "master" be POCOs instead of JSON/XML/YAML/whatever isn't something I'm convinced on in either direction, but imho the in-database classes being not real POCOs is the important part because it reduces the the problem of somebody writing Person.MyMethod() and then blowing up because it's not a SQL function.

Isn't this just `.StartsWith("A")`?

How would you perform this regex query with your code generated solution? What would have to be code generated and what would the developer have to write?

As there's a lot more features available in different RDBMS's than what's available in C# expression syntax, you can use SQL Fragments whenever you need to:

    var results = db.Select(db.From<Person>()
        .Where(x => x.LastName == "B")
        .And("FirstName ~ '^A.*'"));

Yes, it's a trivial example. I'm not looking to support it, I'm looking to catch it at compile-time.

if "Person.FirstName" is a string, then that encourages users to use string-operations against it, which will fail if this expression is being translated to SQL for executing in the DB.

if "Person.FirstName" is some other type with no meaningful operations supported on it (which will get converted into a string when the query is executed) then it prevents many many classes of logic errors.

Saw EF now supports custom SQL queries, so been considering that once we've moved to MSSQL (old db server isn't supported by EF).

We're quite accustomed to writing our own SQL select statements and would like to continue doing that to have known performance, but the update, insert and delete statements are a chore to do manually, especially for once you're 4-5 parent child levels deep.

Quick word of advice when dealing with deep parent/child reloationships:

Do not use lazy loading feature. That way lies madness.

We're not doing that where we come from. All child tables have the main id so we can load the data for all child rows with just one query per child table, and we load everything at once.

We were planning on sticking with this, it has worked well so far, but good to know to avoid getting tempted by the alternative.

I've found [0] for clojure, which maps the best IMO, but it also contains links to the same LINQ examples in other languages (java, kotlin, swift, elixir, python, ...).

[0]: https://github.com/mythz/clojure-linq-examples

The difference is many of those are dynamically typed languages. It's still useful, but a lot of the a beauty of LINQ comes from the fact that it is within a statically typed language.

It's surprising that Go didn't ship with it, but given that they just added iterators, it's coming.

Rust has combinators, which is the same thing.

Most new languages are recognizing that functional support (even if they don't consider themselves FP languages) is necessary.

nah it's not coming, functions like maps and filters won't come to go by design, iterators are not only about FP stuff

Go culture is quite clearly against this kind of niceties.

You can do functionality similar to LINQ with chaining as long as you don’t need to call a method with a generic different from the one defined. If you do need that, you’re going to have to do it without chaining. You can still do something similar but it’ll be a lot less elegant than how C# does it.

It’s part of the design philosophy of Go though. They don’t want any magic. It’s similar to why they enforce explicit error handling instead of allowing you to chose between explicit and implicit. They want you to write everything near where it happens and not rely on things you can’t see.

It’s probably the primary reason that Go is either hated or loved. I think it’s philosophy is great, a lot of people don’t. I have written a lot of C# over the years, so I’m a little atypical in that regard, I think most C# developers think Go is fairly inferior and in many regards they are correct. Just not in the ones that matter (come at me!). To elaborate a little on that, Go protects developers from themselves. C# is awesome when it’s written by people who know how it works, when it’s not you’ll get LINQ that runs in memory when it really shouldn’t and so on.

Yes and no, the LINQ syntax being coherent between IEnumerable<> and IQueryable<> hides a lot of good magic.

IEnumerable<> is regular in-memory lambdas/streams, same what you find in many places.

IQueryable<> relies on the LINQ expressions, those CAN be JIT compiled for direct execution, but the fact that they are data-objects is what allows the translation to SQL and execution on the server rather than locally and can give massive gains since processing can be done where the data lives.

For reference, to achieve what IQueryable does with 100% normal code in JavaScript you need something like Qustar that was posted here a month ago.

Regular transform code in JS (Like IEnumerable)

const ids = users.filter(user => user.age<18).map(user => user.id);

IQueryable like to be transformed to the server:

const ids = users.filter(user => user.age.lt(18)).map(user => user.id);

In C# it'd look identical, but in JS or Java this would be achieved via proxy-object hacks (the .lt() function in the filter instead of the < operator and the .id property getter for the user map to send a flag under the hood for the mapping function).

https://github.com/tilyupo/qustar

If you like LINQ, you should really give F# a try.

F# is even better. I only wish performance wasn't hit or miss around sequences/iterator expressions. Hopefully one day it will reach parity with C# through compiler improvements and/or detaching FSharp.Core from conservatively targeting NS2.0 and modernizing it.

I never really got over the order-dependent and fairly slow compile times, but it's been like 10 years since I used it for anything even slightly complex. Is F# better in this regard now, or are there accessible patterns to help deal with that?

Order-dependency is a feature, not a bug, but it does take some getting used to. If you really want to avoid it, you can now declare all functions in a module to be mutually recursive, so you can put them in any order. Cyclic dependencies between files are still prohibited, for good reason. See https://fsharpforfunandprofit.com/posts/cyclic-dependencies/.

The F# compiler is slower than the C# compiler, but it's still more than fast enough for building large applications.

Maybe you've been unlucky but LINQ didn't really seem all that interesting to me using it.

It's pretty well put together but it was very hard to work out the patterns of what it was doing underneath e.g. I could not tell you now how to implement a custom IQueryable (I know where to look but couldn't tell you the rhythms of it) for some database and I am the type of person who usually does go and find that kind of thing out before using something for a "serious" project.

Maybe it's just a microcosm for C# as a whole - very productive language, good design team, quite hobbled by it's weird/philistine upbringings: Bjarne said within C++ there is a simple language trying to escape, in C# you basically have a simple language buried in nouns.

> It's pretty well put together but it was very hard to work out the patterns of what it was doing underneath e.g. I could not tell you now how to implement a custom IQueryable

There's a lot hidden in there, but basically they expect you to use EF. Writing an IQueryable is a similar amount of work to writing a SQL query planner. You get passed a tree of Expression objects.

https://learn.microsoft.com/en-us/archive/blogs/mattwar/linq...

> basically they expect you to use EF. Writing an IQueryable...

I don't agree. I don't feel any expectation to use EF. It would not be relevant anyway to our code.

LINQ is not EF and EF is not LINQ. EF uses LINQ but not vice versa. LINQ is useful without EF.

The LINQ extension methods that we use constantly are on IEnumerable<T> so EF and IQueryable is of no importance to us, but LINQ is used everywhere.

> IQueryable is of no importance to us

I was discussing IQueryable with someone else to whom it was important. In reply to

> I could not tell you now how to implement a custom IQueryable (I know where to look but couldn't tell you the rhythms of it) for some database

And "for some database" the default answer is "use EF" as the intermediary between LINQ queries and the database itself, rather than delving into IQueryable.

LINQ-to-objects is very important and useful but I was talking about something else.

My apologies, that reply of mine was a bit argumentative, without regard to which post I was arguing against.

Yes, I understand that implementing IQueryable is a beast, but let's not pretend that writing a database connectivity layer was a trivial, everyday activity before IQueryable. It's a specialised, tricky task. IQueryable may not make it easier, but it never was "easy" per se. Nor common, or usually necessary. Or something to do as an exercise "before using something for a serious project" as the grandparent post suggests.

YMMV as to how important IQueryable is to your code.

On the other hand, if you can write extension methods over "this IEnumerable<T> source" you can extend LINQ. IQueryable being complex is no barrier to that, in any way.

Back when I was primarily a C# dev, I used OSS lightweight ORMs which had LINQ interfaces. I also frequently used LINQ on in-memory structures. It's fantastic, and I never felt any need to use EF.

That said, C# / .NET shops did have a tendency to mindlessly buy into all sorts of terrible Microsoft enterprisey stuff. That drove me crazy and ultimately is what made me head out for greener pastures.

What are those greener pastures for you and do you still use C#?

It is not rocket science to implement IQueryable but it is not trivial either since the API is optimized towards ease of use rather then ease of implementation. The Select and Where methods support arbitrary C# expression trees, so the implementation have to traverse the tree and throw errors if some expression cannot be translated to the underlying engine.

> Maybe you've been unlucky but LINQ didn't really seem all that interesting to me using it.

Getting good use out of a tool you do not find interesting would mean a person was… unlucky?

Unlucky as in forced to write COBOL prior to C# or whatever

> I could not tell you now how to implement a custom IQueryable

So what? I see LINQ used all the time, and it is almost entirely (extension) methods IEnumerable<T>

Could I implement IEnumerable<T>? I think I did once, as an exercise. It's not that complex. Not that interesting to be able to do it either.

LINQ is useful without EF. LINQ is not EF and EF is not LINQ.

LINQ is a veritable footgun in any large team I find. While it's extremely powerful and really nice, it's so so so easy to blow your toes off if you don't know what you are doing. Some of my favourite screw ups I saw:

* Not understanding when something is evaluated.

* Not understanding the computational complexity of multiple chained operations/aggregates.

* Not understanding the expectation that Single() requires exactly one of something.

* Not understanding how damn hard it is to test LINQ stuff.

>Not understanding the expectation that Single() requires exactly one of something.

Sorry, but idk how it is footgun of LINQ. It is like complaining about 0 or 1 based indexing

>Not understanding how damn hard it is to test LINQ stuff.

Any examples? Because I struggle to see such

How much of that is LINQ-specific and how much is just the cost of using an ORM to build queries rather than typing them out as SQL?

I've never encountered testing problems with LINQ-to-objects.

I sort of agree. I recently had to code splunk a bug with 3 other engineers and we all got to a relatively complex LINQ and of the 4 of us, we all had 4 different interpretations when visually inspecting.

> Not understanding how damn hard it is to test LINQ stuff.

I disagree with this. Just run the LINQ query on a compatible iterable.

Regarding the testing, it's more the sheer multiplicative number of cases you have to consider on a LINQ expression.

> I recently had to code splunk a bug with 3 other engineers and we all got to a relatively complex LINQ and of the 4 of us, we all had 4 different interpretations when visually inspecting.

You can write hard to read code with any framework. Yes it takes effort sometimes to make linq code clear, but you should not give up on it.

> * Not understanding when something is evaluated.

Linq is lazy. .ToList() reifies. there, that's the gist of what you need to know. Not hard.

> Not understanding the expectation that Single() requires exactly one of something.

eh? There are a bunch of these methods, Single, SingleOrDefault, First, FirstOrDefault, Last, LastOrDefault and you can look up and grasp how they differ. It's fairly simple. I don't know what the problem is, outside of learning it.

> Not understanding how damn hard it is to test LINQ stuff.

Hard disagree. LInq chains can be unit tested, unless your Db access is mixed in, which is not a LINQ issue at all, it is a database query testing issue. LINQ code, in itself, is easily unit testable.

Single gives you some guarantees about the returned value. Use First/FirstOrDefault if you don't need those guarantees. You can also provide predicate for FirstOrDefault to select First element that matches your predicate.

> Enumerable.Single Method - Returns a single, specific element of a sequence.

Some overload descriptions:

- Returns the only element of a sequence that satisfies a specified condition, and throws an exception if more than one such element exists.

- Returns the only element of a sequence, and throws an exception if there is not exactly one element in the sequence.

https://learn.microsoft.com/en-us/dotnet/api/system.linq.enu...

> Enumerable.FirstOrDefault Method - Returns the first element of a sequence, or a default value if no element is found.

https://learn.microsoft.com/en-us/dotnet/api/system.linq.enu...

I’ll get started then end up on a rant but..

This is really the thing with the entire .NET stack that’s very hard to communicate. The standard library and framework design are so well thought out relative to anything else out there. More than that, the support within VS is beyond any other dev tool that exists for any other language - it’s not even particularly close. Edit-and-continue comes to mind, which despite how many times people confuse the two is not hot reload, and is wildly more productive and useful.

I remember back close to 20 years ago DHH was espousing Ruby/Rails and that the concept of types at all were a code smell, and thinking “you’re just very wrong, and clearly aren’t familiar with what else is out there”. Eventually a lot of that crowd moved to Node, then to typescript, and came around.

VS Enterprise (expensive as it is) had features 15 years ago that still seem magical when I show them to JS/TS folks now. IntelliTrace is one that comes to mind - there’s nothing remotely close to it’s snapshot debugging that I’ve seen anywhere else, and I’ve really looked.

The big problems with the ecosystem are that the docs are exhaustive but terribly boring, and not well explained from a learning-something-for-the-first-time perspective. They also really expect that everything you do is the Microsoft way, so if you’re trying to interface your code with something like an Avalonia UI, you’re on your own.

The language is absolutely wonderful though, even when used with Rider. The productivity relative to node/typescript is better enough that it crushes my soul having to go back to wrestling tsconfig and imports after working with .NET references for even small changes. So many of the little things I used to take for granted really just work, and work well. It’s just a wonderful piece of work executed over decades, held back by poor community outreach and badly written documentation.

Your experiences are coherent with mine.

Developer experience is far ahead any other technology out there.

Std lib and its API design is world class, I wish cpp had as good stdlib. Tooling is strong, especially debugger

Agree with all of this. I think the biggest problem with popularity around these tools is that they're too good.

When everything just works, you have a lot more time to focus on your problem. Once you can focus on your problem, you may quickly find out you don't actually care about it. Focusing on tools is a great way to hide from this reality.

> IntelliTrace is one that comes to mind - there’s nothing remotely close to it’s snapshot debugging that I’ve seen anywhere else, and I’ve really looked.

https://rr-project.org/

> Edit-and-continue comes to mind, which despite how many times people confuse the two is not hot reload

I'm certainly guilty of this! What's the difference?

I’m simplifying this for brevity but: Hot reload is a concept whereby changes will be saved, if necessary compiled (in the case of compiled/JIT-ed languages), then whatever is pointing at the original source is run again automatically (a web page/an app screen/whatever).

Edit-and-continue allows for changing the code and then updating the output directly in memory without re-compilation or restarting the execution. It sounds similar but in practice it allows for much more rapid iteration and is profoundly more useful. If you’re pretty deep into an application or web app for example (e.g. added to basket -> checkout -> process payment) and are 30 or 40 calls deep in a stack and realise you’ve a simple bug or change to make, you can edit the code in memory, drag the debugger back to the line, re-execute it and move to the next statement. The benefits of this compound really quickly for anything more than trivial scenarios, so much so that I’ll often code directly in a debugging session as it’s just handier to have a full rewindable call stack right there for simple cases where I’ve forgotten a property name or need to correct and XPath or something.

The surprising thing is that this isn’t even new, VS has had this for at least 20 years (and I think 25 or more as i know VB6 had it. Yes I’m old.)

Edit: 27 years ago in VC++5 (1997).

From your description it sounds like in-memory application state is lost with Hot Reload, but I don't think that's true? I admit might be wrong about this, it doesn't apply to Unity which is my main development environment.

Quoting from the docs (emphasis mine): > .NET Hot Reload applies code changes, including changes to stylesheets, to a running app without restarting the app and *without losing app state*

That sounds more like how you described edit-and-continue to me.

Your comment about the docs is the real reason .NET/C#/F# isn't gaining any new users. The dotnet team should actually be embarrassed about this but it's clear they don't care so neither will anyone else. It's 100% quantity (slop) over quality for Microsoft. Their website and guides are terrible and irrelevant for both new and experienced devs.

Modern C# is probably the best general purpose language out there with the best tooling along with the dotnet framework. Too bad the guides and public information all align with the latest trends Microsoft are pushing to appear relevant. Blazor, MAUI, Aspire e.t.c. are all distractions to maintain the appearance of being modern. None of which are production ready or actually good for that matter.

Back to my original point. If you want to create a new web app then you're REALLY pushed to use Blazor, which is confusing, has many flaws, is basically alpha and is just a bad idea in general. For some reason you're shown a laughably simple guide spread over eight pages which could be a single page. You finish the "guide" and so you go to the "documentation". That documentation page is full of buzzwords that confuses new developers and belittles old developers. The end of this page links back to the pathetic guide. It's seriously like this for everything they do. There's tiny little nuggets of information scattered over thousands of useless pages.

I may sound blunt but it's a fantastic technology ruined by terrible management, poor communication and clearly the worst developer relations team any tech company has ever assembled. How can any company with this much money, this much recognition and this great of a technology fumble it so badly. Well... I actually do know why and it's obvious to anyone capable of critical thinking.

This really seems to be the problem - the developer relations seems to be comprised of non-developers.

The docs are clearly not written by engineers and it really shows.

It’s a shame too - MAUI should be excellent. Best-in-class even. They’ve had the most resources and best tech to throw at the problem and are a distant second at best to React Native. (It might see less use than Flutter these days I’ve no idea).

Also having the C# dev kit for VS Code be non-free is just insane. They’re actively giving the market over to node.

As I keep having to repeat here ad-nauseam, DevKit is optional, you only really need base C# extension which is what provides language server, debugger and code completion capabilities. For VSCodium and non VSC-based distrubutions there's also a fork that packages Samsung's NetCoreDbg component instead of vsdbg.

You'd be right to point out that confusing README of these two does not do .NET any justice however.

It's just an API for JIT, basically metaprogramming. It's cool but you can definitely do a similar thing in pretty much every high level language.

With scripting languages, it's all JIT :)

The C# teams progress on this has been slow. Keep in mind the CIL bytecode has had such capabilities for at least 20 years now and only in the past like decade are we seeing more features and optimizations around LINQ and System.Reflection.Emit.

Dynamics were extremely slow in C# and if you look at the CIL generated you see why. It's possible for example to use something like a Haxe anonymous types[1] to optimize Dynamics so that CallSite caching is way more performant.

I am pretty sure in C# the only way to accept an anonymous type is as a dynamic value, so even though the type of the structure is well-defined at compile-time, it will still rely heavily on runtime reflection/DLR with no additional caching beyond what DLR does for any other dynamic type.

Anyways, this leads to niche libraries being built for handling dynamic data like JSON performantly.

Which leads to annoying things like .NET libraries/apps being incompatible (without some adapter) if they use for example, different JSON libraries under the hood. (See [2]).

Problems like these (the lack of actually good JIT/dynamic code support) in my opinion significantly slow down the .NET ecosystems development, that's why it always feels like .NET is just catching up with features other popular languages have.

To be fair though, much of C#'s lag is owed to Microsoft's contribution to .NET being mostly technical debt. Almost everything good that came out of .NET came from open source/non MS teams (like Mono).

[1] - https://haxe.org/manual/types-anonymous-structure.html

[2] - https://learn.microsoft.com/en-us/dotnet/standard/serializat...

> It's cool but you can definitely do a similar thing in pretty much every high level language.

No. When it was release (circa 2007), very few mainstream languages embraced "Code as Data" the way C# did. In Java, there was no way to pass an expression (as an AST) to an SQL library. Which is why LINQ is so much more ergonomic than Hibernate. In C#, you could use language features you're already familiar with (such as "order.id > 100 && order.id < 200") in your queries, whereas Hibernate made you learn the framework's specific patterns (add Criteria etc etc, I don't recall now). Java just wasn't expressive enough for this.

In fact, you couldn't do this even today in a language like say Python or JS. I mean, not without running it through something like babel to get an AST, and having arbitrary rules on what's code and what's data. C# had this in the spec; based on whether it was IQueryable.

> Almost everything good that came out of .NET came from open source/non MS teams (like Mono).

My team adopted Mono very early - like in 2005. Your statement is not true at all. C# and the Framework was a very good spec irrespective of what Open Source / Mono did, and while Mono existing might have accelerated .Net's transition into Open Source, it would have happened anyway due to the definitive swing towards Open Source in the 2000s. Linq-to-SQL, Asp.Net MVC, EF etc didn't come out of Mono.

Have you ever read the source code for Microsoft's ilasm compared to Mono ilasm?

Anyway, EF is cool, but probably every .NET dev has an EF/LINQ performance related horror story (the generated queries are ridiculous).

A self compiling language is more impressive to me than ASP.NET MVC.

And C# is just lacking for what is actually capable in CIL bytecode. Or _was_ when I last used.

There have definitely been improvements, but in my opinion, they have just been kind of slow.

When I think of Microsoft's impact on .NET and it's culture, I think of stuff like SOAP, the SmtpClient, breaking changes in APIs every year and the technical debt left by it, the dogmatic fanboys, etc...

ilasm is for manually writing IL-based programs in text format, a rather rare use-case. How is this related to LINQ?

It's related to MS contribution to .NET which is the subtopic of this particular thread.

I don't see how it is related is relevant to this discussion. Is there a specific point you would like to make?

> probably every .NET dev has an EF/LINQ performance related horror story (the generated queries are ridiculous)

> There have definitely been improvements, but in my opinion, they have just been kind of slow.

> much of C#'s lag is owed to Microsoft's contribution to .NET being mostly technical debt. Almost everything good that came out of .NET came from open source/non MS teams (like Mono).

Do you actively use .NET (any modern target in the last, say, 3-4 years or so)?

> I don't see how it is related is relevant to this discussion.

Because it was brought up as a subtopic in my original comment and someone quote responded directly to it.

C# generics will handle anonymous types just fine. That's what lets you write stuff like `from ... select new { ... } where ...`.

OK, so since I've never actually bothered to look at what LINQ actually is.. there may be others in the same boat: "LINQ allows you to query any enumerable collections such as List<T>, Array, or Dictionary<TKey,TValue>.". Got to admit I still find the LINQ to DB passtru rather opaque even after reading up on it: https://stackoverflow.com/questions/30716776/passing-a-query...

Basically allowing for a uniform way to query data which is reminiscent of a more modern version of SQL (in my optics anyway). Does anything library wise come close to this within the Golang ecosystem?

While LINQ does include a library of extension methods for functional programming with .NET collections (which is great), it also includes "Expression Classes". In a nutshell, this allows a user to pass a single expression lambda to a function, and the function implementor receives the abstract syntax tree for the lambda, not the lambda itself. You can then not only receive and analyze these trees, you can also manually build and compile them. This effectively allows a limited set of runtime macros within .NET.

> Basically allowing for a uniform way to query data which is reminiscent of a more modern version of SQL.

Pretty much. There's the "language integrated" version which looks a lot like SQL:

    var foo = new List<int> { 1, 2, 3, };
    var response = from x in foo where x > 1 select x.ToString();

But that just translates to the method-orientated one which many people prefer

    response = foo.Where(x => x > 1).Select(x => x.ToString());

If instead of querying a List or Dictionary you query a database, using an ORM (usually Entity Framework), that will actually get converted to SQL and run on the DB.

> Basically allowing for a uniform way to query data which is reminiscent of a more modern version of SQL (in my optics anyway)

It's more general and reusable than SQL, so you can map a subset of it to SQL, which is what some object-relational mappers do.

It's a shame, actually, that .NET performance improvements of up to x1000 could still be found after two decades and hundreds of millions spent on development.

Most of the time, it is not because there were too many slow things to be improved, it is mostly because they are adding more facilities to the runtime, enabling other performance improvements.

For example, the ReadOnlySpan type is a recent addition to the runtime and it will allow faster iterations when used. They simply enabled Linq to use it now.

ReadOnlySpan is a breakthrough innovative data structure, consisting of a pointer and a _length, that took Microsoft Corporation two decades to invent.

Well, better late than never.

other languages do not have special pointers that can point to GC object interiors, be transparently addressed with arithmetics and cooperatively updated by GC without pinning, while also allowing to point to stack and unmanaged memory

I see, thanks. Still, x1000 performance improvement is called a bug fix.

I encourage you to read through this section: https://devblogs.microsoft.com/dotnet/performance-improvemen... and go through the pull requests that it references.

The Nx elements figure is a speedup you would expect if you introduce a shortcut to any algorithm that can bypass doing full iteration, and >100x speed-up is what you would expect from going from interface dispatch per element (even if devirtualized behind a guard) to a bespoke vectorized implementation.

Even if you happen to have irrational dislike of .NET, this is still useful learning material and perfectly applies to other languages that expose similar functionality (provided they have sufficiently good type system and generics).

An algorithm that performs a full iteration where a single lookup is sufficient, is a very bad algorithm. An iterator implementation that introduces 100x overhead is a very bad implementation. C++ has types and generics and pipes and is pretty efficient.

C++ is absolutely terrible at this, either UX or performance (or sometimes both), and it's not even close ;)

But those pointers were around since .NET 1.0. Not only that, but things like e.g. ArgIterator were also there! Span could have been there too; it was really a matter of design rather than technical capability.

I think the main reason why C# didn't have this (and other low-level features like plain function pointers) for so long is because the original vision for .NET was that you'd mix and match different languages in your project as needed. So if you needed that kind of stuff, you'd reach out for managed C++ (and later, C++/CLI), while C# was kept deliberately more high-level.

And so once that approach was abandoned, C# needed to catch up to be able to play the "one language that can do it all" role.

'ret T's aka byref pointers aka managed references in their current form is a relatively recent addition. I can't seem to find which exact version but if my memory is correct - around .NET 5 or so, before that e.g. Spans had to use a specially annotated field with [ByRefLike] attribute. There wasn't really such a forefront support in both the language and the runtime to enable the user scenarios and even, if limited, ref escape analysis that are possible today. .NET 9 goes further and allows types that hold byrefs to be generic arguments, so you can now handroll your own "true" span-based LINQ-like code as well.

The ability to have managed references as fields in types dates all the way back to 1.0. What's new is the ability to define custom ref types in verifiable code, but the runtime itself could always do so on a case by case basis - that is how TypedReference, ArgIterator etc have always worked.

Given that it took C++ a similar amount of time to invent the string specific string_view I don't think it's as simple as you're making it out.

ReadOnlySpan is so powerful because it can be used generically and has implicit conversions that allow you to improve the speed of existing methods without breaking backwards compatibility.

It's well designed and that takes thought and time.

C# is Pragmatic [0].

[0] https://news.ycombinator.com/item?id=41761346

Reply on news.ycombinator.com