LINQ and Learning to Be Declarative | Hacker News Reader

LINQ and Learning to Be Declarative

nickstambaugh.dev

59 pointssieep57 comments

I'm the author of language-ext [1] a pure functional framework for C# and have been pushing the declarative programming thing in C# for over decade. C# is a great language for declarative programming and LINQ is awesome. You need to constrain yourself (and your team) so you don't fall into bad habits, but it can really pay dividends in terms of code stability/maintainability.

I have a couple of samples that I think might surprise you if you think C# must look like Java...

* A game of pontoon [2] - I create a Game monad which is an alias for a monad-transformer stack of StateT > OptionT > IO. This shows how terse you can get, where the code almost turns into a narrative.

* Newsletter sender [3] (which I use for my blog) [4] - This generalises over any monad as long as the trait requirements are met. This is about as declarative as you can get in C#. It's certainly pushing the language, but is still elegant (in my eyes anyway).

[1] https://github.com/louthy/language-ext

[2] https://github.com/louthy/language-ext/blob/main/Samples/Car...

[3] https://github.com/louthy/language-ext/blob/main/Samples/New...

[4] https://paullouth.com

Your documentation appears to be a bit broken. When I try to click on pipes I get a 404. Just to let you know.

Thanks for that, think I've fixed them all. But it is 1 am, so the chances I've missed something isn't zero!

The API docs are all auto-generated [1] so sometimes my repo markdowns get out of sync when I refactor!

[1] https://louthy.github.io/language-ext

The example with lambdas should be written on multiple lines, too. At the same time, you can leverage the => syntax to avoid braces and end up with five lines:

    List<Product> GetExclusiveProducts(List<Product> source)
      => source
        .Where(p => p.ProductTitle == "iPhone")
        .OrderBy(p => p.TypeOfPhone)
        .ToList();

(You could join the first two lines, but I think that’s ugly for multi-line expressions.)

Also, less lines is not a good argument for SQL-style syntax vs method-call syntax. The good argument is that the SQL-style syntax is limited to only a few basic operations, when there are many more useful methods available.

Another reason is that this does not compile:

    List<Product> GetExclusiveProducts(List<Product> source)
    {
      return from p in source
             where p.ProductTitle == "iPhone"
             orderby p.TypeOfPhone
             select p;
    }

This method returns IOrderedEnumerable<Product>, not a list. To fix it, you would need to either change the return type, or go outside of the SQL-style syntax and call the ToList method:

    return (from ... select p).ToList();

There are some things that are harder to write in extension methods too. Such as `let` declarations. Or multiple from clauses referencing iterating variables from multiple levels. I use both.

I'm largely with you on this... I prefer the extension methods myself most of the time.

Also adjacently worth mentioning that I tend to use Dapper instead of EF, so there's even less use of this type of expression all around unless it's actual SQL or assembled SQL via a query builder.

The second SQL-like version is far more readable. There is just less "syntax noise" and it's far more "declarative" by definition.

The author stating that the lambda is "better" because it's less lines is also silly. It's been a while since I've written C#, but pretty sure the SQL-like version can be formatted to a single line as well:

List<Product> GetExclusiveProducts(List<Product> source) => (from p in source where p.ProductTitle == "iPhone" orderby p.TypeOfPhone select p).ToList();

You can format it on one line if you want. Please don't do that, though. Personally I prefer the method syntax with one method per line, it reads more linear than the query syntax.

Agree and disagree. I prefer the second approach, but I wouldn't format either as a single line.

Never understood why a lot of programmers are so obsessed with reducing lines of code at the expense of all else.

What's important to me is that I can understand the intent of the code, that I can reason about how that code will be executed at runtime, and that I can easily debug the code if needed.

Of course, there's no need to go full enterprise Java. Never go full enterprise Java.

But having ten lines of clear code that's easily debuggable and which runtime characteristics is predictable is much better than one dense line that's difficult to untangle, or a few that's hard to predict what will do etc.

Not to mention it's much easier to put a breakpoint on one of those specific lines

Years ago on the Ruby Rogues podcast, someone mentioned they aim to do just one operation per line. I’ve aimed to do that with ny TS/JS ever since. My files are longer but they’re usually easier to read.

Over all, peoples prefer writing with lambda, somes use the SQL-like, but there is a major preference towards lambdas.

I love LINQ (the object method syntax not the keyword syntax) and used it extensively in a production application in ~2015-2017. It is easier in many cases to write and read than manual loops.

In performance-critical code at the time it had to be avoided due to allocations and poor performance compared to loop-based implementations.

However, the new versions of .NET have been reducing this penalty [1], to the point where it might make sense to try LINQ first if it's more concise/clear then only rewrite after profiling!

[1] https://devblogs.microsoft.com/dotnet/performance-improvemen...

> Functional programming isn’t an afterthought in C#, it’s effective and you should learn it if you haven’t already.

I think C# is the best functional programming language because you always have access to a procedural code safety valve if the situation calls for it.

100% purity down the entire vertical is a very strong anti-pattern. You want to focus on putting the functional code where it is most likely to be wrong or cause trouble (your business logic). Worrying about making the underlying infrastructure functional is where I start to zone out.

Does F# care if a DLL it references was coded in a functional style?

> 100% purity down the entire vertical is a very strong anti-pattern.

It really isn't. The benefit of pure all the way down is that later you can replace bits with something more performant if necessary. But starting out with the idea that some bits should never be pure just means none of it is. The beauty of pure functional programming is that its very compositional nature means that you can replace a component without having a detrimental effect to everything its composed with: as long as you maintain referential transparency for that component.

What we gain from imposing the pure functional constraints on ourselves is genuine composition. If we compose two pure functions into a new function, that resulting function will also be pure. This is the pure functional programming super power that leads to fewer bugs, easier refactoring, easier optimisation, faster feature addition, improved code clarity, parallelisation for free, and reduced cognitive load. Opting out for arbitrary reasons at certain stages of the vertical threatens all of that.

All of this!

"Pure functional" isn't a style choice. The judges won't hold up big placards with 10 on them because you executed your business logic in style.

It's a contract that your code will give the same output given the same input. The contract goes both ways: your ability to supply a caller with functional code is made easier by your callees supplying you with functional code.

Someone decides that that's draconian and says "what's the worse that can happen?" and starts mutating in a library dependency. Well now your backtracking parser may or may not be able to backtrack. Your transactional code may or may not be able to be rolled back. Your multi-threaded code may or may not be free of races. `true || f()` no longer means the same thing as `true`. You might need to start scaffolding all your unit tests with @Before and @After to setup and teardown state, and give up running them in parallel.

Maybe you really, really, really need to fire the missiles at the bottom of the call chain. Fine, just mark that method as 'red' so I get a compiler error when I try to serve up 'blue' code to my callers.

As long as you're showing off the syntax (matter of taste) on blog posts you might as well get some mileage out of the semantics (guarantees).

The problem is this tends to collide hard with things which are both stateful and mandatory ubiquitous, like logging.

"Functional core, imperative shell" is a great tradeoff position though.

>> Does F# care if a DLL it references was coded in a functional style

Deeper problem: it can't know. It can only assume. I'd have to check how the loader works but it may be the case that "first call to a function in an external DLL" is not stateless (and can error!) because it triggers the linker.

State can also be managed using the pure functional paradigm (using recursion to provide an updated state as an argument or using a State monad, for example).

Logging is a side-effect and should be treated as such by using a declarative side-effect type, like an IO monad, or could be tracked for later persistence with a Writer monad.

The oft repeated “Functional core, imperative shell” is just nonsense when you try to do it for real. It’s one of those dogmatic statements that gets dragged out in these types of discussions and adds nothing. This is why effect monads are so useful: there’s no need to create arbitrary boundaries to have effectful and pure code.

Alan Turing already proved the equivalence between lambda calculus and the Turing Machine: anything imperative can be done functionally. There really is no need to create arbitrary boundaries.

In languages like C# it may be pragmatic to do so in certain circumstances, like making ASP.NET request-handlers invoke pure functional computations and return the result of the invocation. This is pragmatic/preferable to writing a competitor to ASP.NET. Outside of situations like that I don’t see any value to an ‘imperative shell’, whatever that actually means.

If logging is non-functional, then you can't log in a functional core.

Luckily you can log functionally.

There are a lot of patterns you get when you can be 100% certain of something. For certain functional languages, you could actually safely assume immutable patterns and such.

I really love C# but I will also say that 100% purity is a super power that C# will never have (can't have 'em all).

C# isn't unique in that, and it also doesn't offer the tools required to help avoid having to dip into procedural code.

Here's things Clojure, for example, offers:

1. Structural sharing which makes you less likely to have to dig in to procedural code.

2. The transient function which takes an immutable data structure and makes it... not immutable... you can now do your procedural code, then switch back into immutability land with persistent!.

3. If the above fails, you can dip into Java.

So yeah, I vehemently disagree with C# being the best. I could kinda see it if you said F# (type safety), but C# itself? Uhhh.

I like to return IEnumerable instead of List<>:

    IEnumerable<Product> GetExclusiveProducts(List<Product> source) =>
        source
            .Where(p => p.ProductTitle == "iPhone")
            .OrderBy(p => p.TypeOfPhone);

That way the user can decide if they want a List or Array or Set or whatever, and you can also add additional queries to this

Also better to pass IEnumerable to the function instead of List, for the same reasons

Also I forget the syntax but you can use an extension method to add it to linq I think

Given the use case I’d consider IGrouping<PhoneType,Product>. My point being LINQ is a wide subject and most code I see barely touches the surface.

The only problem with IEnumerable's open-endedness is that it's so open-ended. It makes no implicit guarantees about order, finiteness, side effects, speed/efficiency, idempotency etc. It's easy to assume those things until you accidentally find a situation where one or more are not in your favor.

This is generally better yes, especially if it's going to end up inside another LINQ operation - you can avoid materializing the list.

Cases where it isn't:

- risk of multiple execution (there is a CA warning for this)

- when returning data protected by a mutex, you should always materialize a copy of the returned list/array rather than return an enumerable

It really depends. IEnumerable could have a lambda that leaks the world and makes a web call on every iteration for all the caller knows. You know iterating a List will be fairly banal.

So from an API perspective, I think returning IEnumerables can end up being too cute. The caller has to deal with this unknown thing.

In general you should be clear about what you return and loose about what you accept. If your whole API is returning IEnumerables and some are expensive to iterate and others are not its actually less clear what I'm getting. And this isn't to say List is always the answer either.

If you're going to write code snippets, please make sure they compile!

The final example has signature List<Product> but tries to return IOrderedEnumerable<Product>.

I recognise that this is very much a taster rather than even a full introduction, so the author didn't want to explain IOrderedEnumerable<T>, but please when writing blogs, run through your code examples and make sure they compile.

This means that your audience can follow along.

( It just needs a .ToList() on the end. )

Thanks for the heads up, must've slipped my mind. Edit: fixed

Isn’t that also true for the second to last example with the query syntax code?

Once you start with LINQ you basically see it everywhere. It makes you think different about structuring your code, which is a good thing. It makes you think about immutability and pure functions, leading to more robust code.

But it’s also a fine line, when to use it and when not to. While LINQ is easy to read and make sense of, it is far from being easy to debug. Especially on inexperienced teams I tend to limit my LINQ usage and try to write more “debuggable” code. But that’s my approach, I would love to hear other peoples thoughts on it.

I think the final version is far worse than the second version. This is simpler in what way ?

Fwiw in the final iteration I would still place each clause on a seperate line aligned by their leading '.' because I find it much easier to scan.

Agreed. That's how I would write it in a professional codebase. I added it to show the transformation from separate functions -> LINQ one-liner. Cheers!

Having things on one line is not an upside. Just think of all the Bash one-liners that are write once edit never. Doubly so in what's essentially a tutorial.

My intention wasn’t to promote one-liners as a best practice, but to illustrate how declarative style can evolve from imperative code.

In production, I agree that clarity should come first.

> Your coworkers and QA will thank you for learning LINQ and ditching the imperative methods that plague your Python brain.

This is a very unfortunate joke: Python has list (and generator) comprehension expression for a long time (2.3?) which are similar to LINQ. At some point in the history many languages stole useful expressions from other paradigms.

Let’s joke on BASIC, it always works.

List comprehension is pretty good, but I prefer LINQ method-style because it's executed left-to-right, whereas I keep having to look up the order of Python.

> This is a very unfortunate joke: Python has list (and generator) comprehension expression for a long time (2.3?) which are similar to LINQ.

I love Python, its my main daily driver, both at work and by preference for most of my personal coding, but Python comprehensions and genexps are much more limited than LINQ language level query syntax (Scala’s visually-similar construct is more like LINQ in capabilities) and Python—purely because of core and stdlib convention which also drive convention for the ecosystem, not actual structural features—lacks anything like the method syntax as a common API (unlike, say, Ruby).

EDIT: Thinking about it a little bit, though, it should be possible in theory to implement LINQ in Python without language level changes (including providing something close to but not quite as clean as the language level query syntax[0]) as a library via creative use of inspect.getsource and ast.parse, both for providing the query syntax and for building the underlying expression tree functionality around which providers are built (support for future python versions would require implementing translation layers for the ASTs and rejecting unsupported new constructs). Conceptually, this is similar to how a lot of embedded DSLs in Python for numeric JIT, compiling GPU kernels, etc., from (subsets of) normal Python coded are done.

[0] existing comprehension/genexp syntax looks similar, but relies on simple iteration, not pushing code execution out to a provider which may be doing something very different behind the scenes, like mapping "if..." clauses into SQL WHERE clauses for a database query.

> with a one-liner for the actual logic

Looking at that 'one-liner' I get strong perl vibes, or is it chills?..

I worked in a C# shop for a bit, but never wrote LINQ and saw very little of it.

However, in my Java days, I've very much enjoyed using streams. Looking back at LINQ now, it seems like a nice DSL around streams (which probably exist in C# land, too).

Interested in commentary around this!

You worked in a C# shop and never wrote LINQ? How is that possible? That's like driving a manual car and never shifting out of first gear.

In retrospect, I imagine you’re right!

LINQ definitely helped me get my Unity gamedev sideproject get completed faster

I only have ever used LINQ in the context of enterprise. What are your thoughts on using it in game dev?

Very useful when you need to do logic on Iterables. Feels almost like python.

For example, in a game you may have a situation where you maintain a runtime list of objects with a certain property but you want to filter by some other property

Interesting. I remember a gripe being the performance of LINQ wasn't great even for a webapp years back. I'm assuming its negligible nowadays or did you notice less frames when the LINQ runs?

I don't know why LINQ would have any better or worse performance than a standard loop for most tasks. Abusing LINQs will of course lead to degraded performance. I've found if you want to insert more logic into your loop, sometimes you have to switch batch to good 'ol loops. I'm sure folks abuse the convenience factor to write more loops rather than modify existing ones which can lead to degraded performance.

For my game it's irrelevant because most linq loops are only triggered by events so there's no loops running on every frame.

For example, lets say you have state machine based AI, and a scripted event triggers tells an enemy starship to change allegiance. The enemy starship then scans a runtime set of enemy factions using LINQ that are within 3 km to identify new targets to attack.

That makes a lot of sense. Thanks for the response!

I think the future of programming will be mostly declarative and parametric since most professional software is redundant.

I’m going to reserve a thread on this post for folks who want to share horror stories trying to implement their own LINQ providers.

Why?

Writing your own LINQ provider is a very niche activity done by people who want to translate or “transpile” C# expression trees into something else.

It is fundamentally a difficult endeavor because you’re trying to construct a mapping between two languages AND you’re trying to do it in a way that produces efficient target code/query AND you’re trying to do that in a way that has reasonable runtime efficiency.

Granted, on top of that, I’m sure LINQ provider SDKs probably add their own complexity, but this isn’t an activity that C# developers typically encourage.

Not a horror story. I like how marten worked with postgres and wanted something similar for sqlite, so I made a library that stores data as json, and translates linq to sql using json query functions. It wasn't very fast, but it was fun experience. For next attempt (once I have more time) I will probably include source generators to precompile queries and skip if not all, then most translation at runtime.

"But what if I told you we could make it simpler"

... and then the author proceeds to presenting clunky, unreadable fluent syntax

That was a good joke for the afternoon.

Are you saying that you’re unable to read a fluent syntax that’s been supported by most mainstream programming languages for the past 10 years?

Or are you suggesting that the majority of programmers struggle to read and understand fluent method chaining?

I don’t have a dog in this fight because this blog post is very novice oriented. I’m just genuinely confused why you think it’s unreadable or “clunky”. What is it about the fluent example that you find clunky?

Can you show me how you would write it? I don't see how this contributes to the conversation unless you show me how or why

These patterns are commonplace for many years now in many languages.

The intention was to show how declarative code can come from imperative with a 'true' one-liner.

i think his point was that the actual LINQ (language integrated query) is already superior to transposing it back to the functional fluent version. i largely agree, especially when you're doing joins and group by.

i've worked with people in the past who refused to allow any actual LINQ in the codebase (use resharper to convert it to fluent!) even though it essentially became obfuscated.

Reply on news.ycombinator.com