Hacker News Reader

(Note: When I say LINQ I am referring to the functional style it encourages, not the query syntax. The query syntax is nice, but it's just a trivial syntactic transformation.)

Correct me if I'm wrong, but the world is moving towards functional programming (i.e. LINQ) not away from it. Personally, I find LINQ far, far easier to read, write, and analyze. (On the other hand, I understand the deferred semantics and watch for warning signs like enumerating a sequence more than once.)

Honestly, a C# company avoiding LINQ sounds to me like the canary in the mineshaft telling you the company has programmers falling behind the times and doing things the hard way.

I'm not sure what you're actually arguing for or against. There is nothing at all wrong with functional programming, and such had nothing whatsoever to do with my comment. That LINQ happens to use some functional techniques doesn't make my criticism a criticism of fP.

Functional languages use the concepts you're criticizing even more extensively than LINQ does.

What are the "concepts I'm criticizing"? I suspect that you have absolutely no idea what the context of my comment was, because your diversion on the topic of functional programming is just grossly out of place.

LINQ encourages the belief that set operations are free, such that you no longer have to concern yourself with concepts like memoization or appropriate structures. This has nothing to do with functional programming. Literally at all.

You've actually been pretty vague about the concepts you're criticizing, except for "LINQ". Now I can see you were alluding to just the deferred execution part (the "lack of memoization").

Basically, you're saying it's too easy to accidentally perform a computation twice by iterating through a sequence twice. Recurse on that problem and you get an exponential blowup.

That is actually a problem mostly unique to LINQ and it is really important that a programmer understand the deferred semantics. Some tools, e.g. ReSharper, will detect multiple enumerations and warn you about it. Judicious use of ToArray/ToList solves most issues with defer-splosion.

I haven't been vague whatsoever.

In your first post, everything is "linq is bad" in generic terms except for these two bits:

> "it falls apart in real-world persistent code with considered algorithms and storage"

> "it makes it so easy to do [grouping / sorting / etc] that developers start to resort to that as a catch-all magical solution that is costless because how much could a line or two of code cost"

With hindsight I know your issue is with LINQ's deferred non-memoized execution. It's really easy to enumerate a sequence twice, but every place you do that might add a factor of 2 to the runtime and so there's a danger of exponential blowup that must be avoided. But that's not what I see when I read the above.

The above reads as complaints about easy composability and worries about code monkeys failing to consider what the code they're writing will do. Those are standard "functional / high-level programming is bad" complaints. With hindsight I know that's not what you meant, but that's what it reads like.

I'm definitely in agreement with the last point. However, I originally said that good C# code contains a lot of LINQ, not that C# code with a lot of LINQ is necessarily good. Deferred execution is something that confuses people, but frankly, it's a concept they need to learn. It's been in the language since yield return got added in 2.0.

LINQ's deferred execution was the right choice for performance, but it's the hardest one. You've got to know when to call .ToList(). I'm not denying that I've seen people evaluate the same expensive list 100 times, use a join when precomputing a Dictionary would have been much faster, close a connection before the result is actually evaluated. But I've never seen C++ programmers say you can make mistakes with pointers, so don't them.

Clojure's lazy sequences are guaraanteed to evaluate once, but that comes at the expense of storage. In particular, a query like the one in the original blog post will evaluate multiple intermediate lists that then need to be thrown away. And indeed, they've introduced reducers to address this, which behaves more like LINQ.

Don't forget about Memoize(). Often it can be a better choice than ToList().

In particular, a query like the one in the original blog post will evaluate multiple intermediate lists that then need to be thrown away.

Why not fuse the operations, if the values are immutable?

Well, there's two ways to do that in Clojure: write your own list processing (usually regarded as bad style) or use reducers. Both are appropriate in specific instances, but the fact remains that the lazy seq code above is the idiomatic way of doing it in Clojure. So the _default_ way of doing it is slower than LINQ's default way of doing things.

It wouldn't be hard to make LINQ behave like Clojure, either.

I get what you're saying, I understand it, and I agree with the pitfalls. But I really really really don't agree that one should avoid LINQ. Sure if you don't understand how it works, and how it impact the performance, you should use it less. But don't try and drag the language down because you find it difficult. LINQ is easily one of the best things that have happened to a mainstream language in a long time, and if used right it produce highly maintainable code, that is very easy to read and understand. A lot of data manipulation really is done on sets, and LINQ makes it a breeze to do.

But don't try and drag the language down because you find it difficult.

Oh give me a break. My exact complaint about LINQ is the opposite -- that it makes expensive operations seem simple, which with all innovations in language means that mediocre programmers will start abusing it in all codebases.

LINQ has a place. To say that one should be using it "all the time", however, is exactly as I said -- a canary in a coal mine. The only time I've ever come across the extensive use of LINQ it has been in horrific code.

So I'm guess you haven't done any programming with the reactive extensions yet?

And why would you guess that? Because I don't consider LINQ of Rx observers being a magic cure-all?

In fact, LINQ on Rx is a symptom of the classic LINQ issue -- don't have subscriber or source filtering, or coherent, useful events -- simply broadcast everything and through the magic, free filtering of LINQ all is solved in the consumer.

Wouldn't it be (O(n) + O(n) + ...) at the worst? And if it's on lazy sequences, which I thought it was, then composing a .Select and an .Aggregate should only involve a single pass through the sequence.

I'm not talking about individual LINQ queries, which themselves can be perfectly fine: Microsoft is pretty smart, so if you're doing basic LINQ for objects grouping and sorting and selecting, they're going to use decent algorithms given the structures used. There is nothing surprising that the submitted usage shows very similar performance, as in the end both cases are doing essentially the same thing.

The problem is that it makes it so conveniently easy to do brute-force tactics that....oh the horrible things I've seen...code gets littered with LINQ doing naive queries repeatedly over massive sets of data. Of course you need good coders and good code audits, but LINQ, I think, gives a unsupported sense of comfort that one is making good code (where if people had to code these as loops, it would become very evident very early on that maybe they should rethink their approach).