Hacker News Reader

Comment by gameswithgo | Hacker News Reader

One of the tips is to avoid Linq, which many .NET developers are hesitant to do. I made a library that lets you use Linq style convenience functions without a performance hit in many cases:

https://github.com/jackmott/LinqFaster

View on HN

I feel I need to clarify this - my post is aimed specifically at optimizing code for high performance. Many of those tips will reduce readability of the code, which may make it harder to determine its correctness.

I'm a big fan of LINQ and I use it in my own code, just not in the high-perf bits. LINQ is great for writing code which has "obviously no deficiencies" in terms of correctness. Without it your code may end up having "no obvious deficiencies".

I also like LinqFaster, LinqAF and these other libraries/tools which can make LINQ usable in more domains.

So your library speeds up Array<T>/T[] and List<T> Linq operations? That's quite interesting, though usually in my experiences in Linq performance optimization the problem is usually prematurely using the wrong data type for your Linq operation.

Given the immediate next bullet point in the article is about foreach operations over List<T>, that would be my top expectation why they were seeing so much allocation in Linq. ToList() allocates a lot, and to many projects I've seen reify everything with ToList() way too often in Linq usage. I've argued before that List<T> is very rarely the appropriate data structure for a lot of Linq work, and personally consider ToList() harmful. A successful strategy I've used to cleaning up Linq performance in projects is to simply start by remove all ToList() calls entirely and work to move API signatures to use smarter, more appropriate data types than List<T> and IList<T> everywhere.

You just need to be careful not to allow any subsequent modifications of the original collection when doing that (assuming you're using the returned LINQ result), or you'll introduce runtime errors. Typically, my use of ToList or ToArray means I'm guarding against future runtime errors due to collection modification at the expense of memory allocation.

It sounds like you know this already, but I like to think I just saved someone from introducing a runtime error after following your advice without understanding the consequences.

I just often find I need to remind people that there are other data types to consider reifying to beyond List/Array. While I'd probably suggest removing ToList() from Linq as List<T> really does seem to be the wrong data type more often than not, but for whatever reason so many people find it the "easiest" way to deal with Linq. I feel like ToLookup() and ToDictionary() should be more people's best friends when working with Linq. Too often I find bad searches and joins that become so much easier/more performant if some middle product was appropriately stored in a Dictionary<K, V> or ILookup<K, V>.

Also with Linq against IQueryable sources, so often ToList/ToArray is used where AsEnumerable would be better. Understanding those monad boundaries is hard, and AsEnumerable doesn't always sound self-explanatory to people when they need to cross those boundaries. (I've thought before that some simple IDE highlighting of which bits of Linq are against IQueryable and which against IEnumerable might help some people think better in Linq.)

People abuse Linq a lot though; enormously complex queries over very large datasets without really knowing what you are doing. When people need .Each, some will just do .ToList().Each(. Etc. I found a bigger issue even with abuse/overuse) (or use at all really) of dynamic. I wish there was a way of to ban it at compile time.

I will challenge every use of dynamic (and var, for that matter), unless it's used in the very few appropriate cases.

`var` is entirely syntactic sugar (compile-time type inference) and there's no runtime cost associated with it.

I find it makes most (but not all) code more readable, particularly given that we have great IDEs/tooling in the .NET world.

I agree, by using var you need to name your variable better therefore making your code more readable. Rather than relying on the interface/class definition to explain to someone why you used "obj".

To var or not to var is purely a "tabs 'n' spaces" debate, but doesn't affect performance.

It will affect the performance of someone trying to read your code.

People are still afraid of `var`?

What's wrong with any "use at all" of dynamic? It's just another type like object that has runtime reflection built-in and works well for interop, and without much performance problems.

>without much performance problems.

Just depends on the kind of thing you are working on. For many people you could use dynamic all over, no problem. For many people that would be a disaster. (super high throughput server, things that need super low latency, games, rendering, libraries that could be used in any of those domains)

From personal experience it also gets abused by people looking for ways to shortcut writing actual classes

Exactly: I am usually writing backends for financial systems which need high throughput and low latency; dynamic is just a massive performance hit. And almost never needed; people usually use it because lazy in my experience.

It is just ‘faster’ for people to dump JSON into dynamic and write against that than it is to properly create structured classes. In the end ofcourse the unittests validate the JSON, the code is slow and harder to use and refactor; you would have saved time and stress actually just writing classes with proper typing.

Also worth mentioning: https://github.com/kevin-montrose/LinqAF

Focused on reducing the allocations which makes linq heavy

In that vein, Nessons has a Stream Fusion library: https://github.com/nessos/Streams

For lightweight threads I recommend https://github.com/Hopac which is an implementation of SML's John Reppy's Concurrent ML on .NET.

This looks really interesting - are the benchmarks in the GitHub repo against .NET Framework? With all the performance work that's been done in .NET Core, it might be interesting to compare it with .NET Core.

Also, I noticed there haven't been any commits for a while - is this more or less considered 'complete'?

Yes it is more or less considered complete. I may do a pass over it to see if it can leverage or support span<t> better sometime soon.

Performance deltas should be similar on .NET Core.

Reply on news.ycombinator.com