LINQ one of the most beautiful things in C# after extension methods!
I do not mean to raise the ire of defensive .NET programmers by saying this: as with most things, used right it is a beautiful thing, and there is nothing fundamentally wrong with LINQ. But what it was used for in practice was almost always grotesquely inefficient.
If someone had to make a loop to iterate objects to find a member on every function call, pretty soon they'll realize they should using a System.Collections.Generic.Dictionary for instance, or some other appropriate data structure (e.g. Need sorted data? Then use a sorted tree, for instance). When it's a simple LINQ query, though, it's one seemingly harmless little line, so it tends to get a pass.
Pretty soon profiles were just hundreds (if not thousands) of different cases where LINQ was used as a cure-all, destroying performance under any load at all.
LINQ is not magical. It is doing the same dirty work that you would be doing yourself if you needed to filter / sort / recast, but provides syntactical sugar to do so. The compiler and runtime will happily generate horrifically inefficient code if you request it, with no warning or complaints.
The tldr; is that it is a fantastic set abstraction that is more often than not grossly abused.
I think the real problem is Cargo Cult programming. People are using linq (especially in combination with EF) and not understanding what's actually going on. So they end up getting things like n+1 selects on collections with thousands of items. They don't understand delayed evaluation. Or how sorting effects a Where() call.
The problem is programmers who think they can take some code from stack overflow and use it without outstanding it. That problem is not unique to LINQ or .NET.
It absolutely isn't unique to it -- powerful features allow programmers to shoot themselves in the feet, and this is true since we began this profession.
When dealing with larger applications and datasets it has the potential of easily imposing enormous costs, of the sort that absolutely eclipse most other anti-patterns. Appending on an immutable string through recursive function calls suddenly looks like child's play.
The truth is that in most apps, these inefficiencies simply don't matter, and to all of the "just learn how to use it right!" replies, I would say that odds favor that those people aren't using it right at all. But in the end it just doesn't matter that much because small amounts of data, low demands, etc, just makes it an irrelevant cost. Most apps are running with an excess of CPU resources relative to the demands on the app, so it just doesn't matter.
Many .NET developers have never run Visual Studio's profiler. They don't even know what the costs are, but we're in a world where 1500ms to load a basic form is okay.
When you deal with very large scale financial data, as I do, however, your world is a very different world. You don't just have a scrum where you try to teach people that LINQ isn't magic (which of course we did). The risk factors become much higher.
LINQ isn't just about hitting a database either, it enables a more declarative / functional approach to development in general, which brings huge benefits in terms of parallelisation / robustness / testability.
It seems absurd that you would block your team from using it.
Edit: Oh and btw, you're not the only person who has to deal with "large scale data". Many of us do and manage just fine with LINQ.
And yes, those milliseconds matter. And it's very strange that I noted that after repeatedly encountering performance issues with LINQ, you say that we should have oversight. Do you understand that such is oversight? That the rules and exceptions came about because the recurring issue with LINQ (particularly LINQ to Objects) being an enormous performance issue?
It doesn't matter to you, probably because you don't even know what the cost is [and that isn't meant to be at all a slam. For most software the cost simply doesn't matter]. And that's fine. But save the continual exclamatives (which you always add right after defensively down-arrowing).
Though I have to chuckle at the notion that LINQ helps with testability, or robustness for that matter.
No, you're exaggerating your position to try and justify it. There is zero substance in what you're saying. All I see is someone who appears to have made an irrational decision about a technology because he personally doesn't like it. There's no fundamental reason why LINQ to Objects should be any slower than say a foreach over a set. You can get is a mess with either approach. If you're not prepared to mentor your team, or do code reviews to check for problematic usage then it's a fault of yours, not the technology.
> It doesn't matter to you, probably because you don't even know what the cost is [and that isn't meant to be at all a slam. For most software the cost simply doesn't matter].
Of course it's meant to be a slam, you have no knowledge of my experience of optimisation or my deeper understanding of how frameworks like LINQ works, so instead you assert that I "don't even know what the cost is". I have spent a large part of my career doing 'to the metal' optimisation, so I will ignore that comment.
> Though I have to chuckle at the notion that LINQ helps with testability, or robustness for that matter.
If you don't understand why imperative and functional code are different then you won't understand why LINQ is inherently more robust and testable.
There's no fundamental reason why LINQ to Objects should be any faster than a foreach over a set. Which is precisely the point. LINQ has an amazing way of hiding details in a manner that allows for massively inefficient code to look completely innocuous. For throw-away, one-off "command line utility to convert A to B" type code, it is absolutely golden. For long-term use code that may be called millions of times, it can be disastrous.
You have no interest in a reasoned discussion, and your argument style is best described as defensive. That is all. Have a nice day.
I'm out of this thread, it's not adding to the discussion.
This, in a nutshell, is your argument. And it's an incredibly poor, out of place argument that is founded on ignorance, and does absolutely no service to .NET.
Performance in most modern applications is achieved through proper algorithms and data structures. LINQ is often a hasty short-cut around those, but it offers the illusion of elegance such that many people (such as, apparently, you. It's all functional-like and such, so it has to be good, right?), are blissfully unaware.
Again, a lot of the time that's perfectly fine and causes no real harm -- similar to the way regular expressions might be a costly but powerful catch all -- but in other cases it is not good. That is the case with large scale financial software. If you have a different experience, good for, but simply saying "incredible!" over and over again does absolutely nothing to make your case.
I find it incredible when I see comments like this. What you should have done is instigated a mentoring programme for the programmers who aren't yet up to speed. LINQ is one of the best things to happen to the language and you just go and kill it.
Also, check out LinqOptimizer -- it analyzes your LINQ queries and can optimize them for better performance (and it's free!): https://github.com/nessos/LinqOptimizer
For example, all queries that are generated for database calls via LINQ are reviewed and analyzed to make sure its what is expected.
You are right, LINQ is not magical and was never meant to be. I don't think using LINQ is an automatic ticket to leave your brain at the door.
In your case, it's easier to blame the tech and stop using it than to take responsibility for training the team to be better developers.
I would also imagine that on this team the use of var was also restricted?
LINQ is really one of the best things in .NET. Just like any tool, you need to learn it before using it properly.
Believe what you want. I will say again that of the three "I can't believe it!" responses thus far, I'll bet zero have ever even run their profiler. Ignorance, as they say, is bliss, and in many cases is perfectly fine.