Exercises for understanding Haskell Lenses (2019)
williamyaoh.com
williamyaoh.com
Its payoff doesn't seem to justify the cognitive overhead (concepts, operators, etc) that it brings.
I'd highly recommend that newcomers to Haskell avoid Lens in their own code for as long as possible. The juice really isn't worth the squeeze.
Based on the people who have amount of "C++ experts" I've interviewed over the years who don't know much about template metaprogramming or weird rules about initialization and class layout, yet have shipped meaningful code while avoiding most of the weird parts of the language.
Even languages like Python can get weird, such as default arg initialization:
def add_item_to_list(item, list_to_add=[]):
list_to_add.append(item)
return item
a = add_item_to_list(1)
a.append(3)
b = add_item_to_list(2)
This is my go-to Python 1st-round question, and many prolific Python coders don't get through this.
From what little I've learned of Haskell so far, it seems like one can eschew the weird PL-heavy parts of the language and just be productive (with the tradeoffs the language contains, e.g. GC).
It seems to me that it's one of those things you've either encountered (and go "well, that's weird") or... haven't encountered (and go "well, that's weird" upon having it explained). Have pop quizzes about the sharp edges of various languages proved useful as a filtering mechanism?
It is scary that software engineers want to be considered professionals, and yet don't want to be held responsible for understanding the tools they use.
Would you trust a bridge designer who didn't really understand the intricacies of concrete? Not me, no thank you.
If you're coming into a role saying "I'm very familiar with Python" and you don't know one of the most famous 'gotchas', that says something.
If you're coming into a role and saying "Hey, I'm not so familiar with Python but I have experience in other languages and I'd love to learn it as part of this role" I think that question is not appropriate.
It's pretty easy to just never bump into certain corners of languages -- even if you use said language extensively. I've used Python for, like, 8(ish) years professionally. I maintain a few moderately popular open source libraries. I honestly don't recall exactly what happens when you do the `list_to_add=[]` thing, because these days I pretty much never pass in a list with the purpose of having it be mutated by something else. I know is that I did try it once, a long time ago, and that it does something unexpected. But I couldn't tell ya more than that just by looking at in.
Color me dismayed to discover that I'm actually a hack fraud.
What I said elsewhere still stands. Yes... this kind of thing is the difference between 'works well' and 'exposes private user data to other users'. I'm glad you don't use such constructs, but realistically, given python's closure rules and when data with interior mutability are captured, this situation arises more than just in this particular situation. And like I said... it can easily lead to exposing private information to other users.
Is it fair to expect a candidate claiming fluency in language X to back that claim up? Sure. But I don't think the general thrust of your comment is very productive.
it would be extremely hard for a Python developer not to encounter this core property of the language (call by assignment), unless the candidate has just graduated from a random 1-month bootcamp, which may be the purpose of discovering of with the first-round question.
a = add_item_to_list(1)
a.append(3)
b = add_item_to_list(2)I should not try to HN comment while running out the door to catch my bus...
Trick question it's a type error?
`struct ^. field1 . field2` to get stuff
and
`struct # field1 .~ blah` to set stuff.
It's syntactical memorization on par with any other language. A bit strange, but just syntax.
Lenses themselves are awesome. The "bad", naive, not-general type is really simple.
``` newtype Lens a b = Lens {get :: (a -> b), set :: (a -> b -> a)} ```
So simple that modifiers can be mechanically generated for your data types. And suddenly, you can express super-duper complex modifications without all the obnoxious boilerplate it'd otherwise take.
Say you have two data structures
data Struct1 = Struct1 { someStuff: [ Struct2 ] } data Struct2 = Struct2 { otherStuff: HM.HashMap Text Struct3 } data Struct3 = Struct3 { name: Text }
And you have a list `[Struct1]` and want a `[Text]` of the `names` of Struct3. How do you do this in an imperative language? Manually accessing the fields, for loops, etc.
The van Laarhoven formulation of lens and ekmett's lens library optiocs in Haskell makes this trivial.
`myList ^.. someStuff . each . otherStuff . each . name`
Now, what if you wanted to modify all the names and prefix them with 'id-'? Again.. trivial.
`myList # someStuff . each . otherStuff . each . name %~ ("id-" <>)`
Your formulation is just field accessors. Not composable, and ultimately no better than just using the generated field accessors haskell provides.
If you take Haskell and remove all the types (e.g. Scheme or JavaScript), you can have all the same abstractions, with basically the same syntax and operational behavior. But then of course the operational behavior is just applying/composing functions in the end... And because there are no higher-kinded types to deal with, category-theoretic language is less likely to sneak in.
I say this as someone who has used Haskell for years and studied category theory / PL theory as well. And I like all of those things, just think it's sometimes not really necessary to get the mental model at all and be able to use those tools.
I think that some code duplication is also not always the worst thing. Just because strings can be "combined" through concatenation and numbers can also be "combined" through addition does not necessarily mean you need one general notion/function to capture both, especially because if you take that to an extreme then you end up with some function that "sqoogles the byamyams" (i.e. the abstraction has become so general that the only way people really understand/use it is by looking at its concrete instantiations).
This isn't generally true, as such a language would have to be lazy by default and the property is more important than having all the types, because certain abstractions that can be expressed "naturally" in Haskell is a byproduct of non-strict semantics. Also it's the property that sometimes helps avoiding worst-case complexity of evaluations - something that eager languages have to live with at all times, no matter the abstractions.
There's a few examples here: https://augustss.blogspot.com/2011/05/more-points-for-lazy-e...
However, it was hardly ever used in practice because it isn't polymorphic and, perhaps more importantly, it can only be used for accessing a single item from a record.
The reason for the explosion of interest in "optical" libraries in the 2010s is because of the development of other optics that can simultaneous access multiple fields (in order), access a functor full of fields (unordered), access field that maybe doesn't exist, pattern match on a variant, zip an indexed set of fields, etc. And all of this is done is both a composable, and transparent way.
Haskell's Traversable functors existed in 2008, and while your lens type can also be traversed, you have to explicitly convert it to a traversable to use it and, or compose it that way.
I'm sorry you feel the need to understand profunctors in order to use optics. Ideally the system would abstract that all way. Unfortunately in Haskell, some of these components leak out (how much varies from implementation to implementation) because this abstraction uses constraints and polymorphism in a very novel way. It's kinda amazing that Haskell's type system accepts it at all.
The alternative seems to involve a proliferation of casting of optics upon every use, requiring either chaining of casts or writing n^2 casting functions from between every related pair of optics.
[0]https://web.archive.org/web/20071023064034/http://luqui.org/...
Everyone has their own tastes. Someone unfamiliar with the notation of ordinary algebra might argue for "you will have to find two numbers that the difference between the two is 10 (that is, so much as is our number) & that we make the product of these two quantities, the one multiplied by the other, exactly 1, that is, the cube of the third part of the variable" (https://www.maa.org/press/periodicals/convergence/how-tartag...) as easier to understand than "find u and v such that u - v = 10 and u v = 1", but I think most modern readers would agree that people uncomfortable with the algebra are better served by learning how to read the latter than by sticking with the former. (And, I think, also that it doesn't help either to keep the variables but replace the symbolic operations by words: `(and (eq (subtract u v) 10) (eq (mult u v) 1))`, in pseudo-Lisp.)
In contrast, most Haskell notation I've seen is either an ad-hoc invention for some library, or it is an ASCII version of notation in a niche domain like category theory. Even Haskell's >>= operator for flatMap/bind seems to be an invention, as far as I can tell the equivalent concept in CT is Kleisli composition, denoted by a sharp sign and the regular composition operator (as far as Wikipedia shows - I'm not formally trained in CT).
Additionally, people rarely if ever give a proper name to this notation in Haskell, making it completely impenetrable to even represent the formulas in your mind. How am I supposed to read `user1 ^. name` ? When I see `∇⨯f` I know how to read it (del cross f, or nabla cross f, or curl f) because that was an explicit part of how I was taught the operation (and note that it is not an arbitrary digraph, it can really be computed as the cross product of the pseudo-vector nabla and f), but Haskell tutorials and documentation completely skip this step, in my experience.
(Edit: I'd also be curious about `x += y` and `x << y` in C.)
Same with your C example. It's doing the exact same thing in C, except the operators (* and .) are separated from each other.
-> is a digraph. It means the same as `*.` I don't usually pronounce it, I just think of it as itself. If I have to say it to myself mentally, I say "sub". I'm not sure I've ever tried to say it aloud to a coworker; if I did, I might have said "arrow" or something.
You do have to learn these things, just like you have to learn all the other operators. But simiones still has a point - at least the math-based operators are much more widely known and understood than the category-based ones.
For example, '>>=' (commonly called bind in haskell) is well-specified. Anything I see it used with is going to be a Monad, and thus follow certain laws. Examining the imports, I can immediately tell what any operator is.
In C++? Forget about it. The 'left-shift' operator which is supposed to shift bits to the left, can somehow also print things to standard output. In what world can the terminal be bit-shifted left? In fact, we understand this because no one reads 'std::cout << "Hello world"' as 'shift std::cout left by "Hello world", because such a thing is non-sense, whereas '1 << 2' is '1 shifted two bits to the left'.
EDIT: And then, when you add in external libraries, it gets worse. `<<` can be used for creating lexers and parsers in boost if I recall correctly. Completely lawless, and, when you survey the ecosystem, also dangerous. So many bugs in C++ and such due to this.
I perfer `rec^.field` with operators in lens for the same reason I prefer `ptr^.field` in Pascal over hypothetically writing `ptr DEREFERENCE ACCESS field` or `ACCESS(DEREFERENCE(ptr),field). It lets me hide what is essentially just plumbing-like-grammer behind operators in order to let me focus of the parts of the program related to the business logic at hand, namely `rec`, `ptr` and `field`. Otherwise the plumbing tends to drown out the more important parts of what is going on.
Second of all, even within C, these operators are ubiquitous. You will use . and -> and * to deref and & to take the address, and = for assignment, and ++ or -- for increment/decrement, and ==, &&, ||, ! for logical operators, and the <operator>= notation for applying a change to a variable, and [] for subscripting, in almost every program you right in C. Shortening a very common operation to a symbol is much more easily acceptable than shortening a more rarely used operation, or one that's specific to a library.
Basically, the status quo in programming as I see it is this: it's OK to use symbols instead of words for (a) syntax in your language (C curly braces, dots), or (b) ubiquitous constructs that are used virtually all over any program (+=, *), or (c) if they are already well-known symbols in other popular languages, or in non-programming fields (such as the near-universal regex syntax). Even (a) and (b) benefit a whole lot from familiarity with other languages, as should be expected. They also still require proper names - which the C standard for example gives for every operator, as do C tutorials.
> Worse still, C++ operators are completely unintellible without context.
Um, compared to what? The C++ operators require less context to understand than the Haskell lens operators by a large amount, so this seems like a really odd criticism.
Your last two paragraphs... yeah. Operator overloading has been used for some, uh, unusual uses, even by the standards committee.
This is my exact point. The C++ operators require much more context.
Suppose you see:
`v << i`
in c++. What is the type of v? What is i? Is it an int? Or an output stream? Or something else entirely! I have no idea. By the C++ standard it could be anything. In fact, it is heavily dependent on what's in scope, and indeed, several different things could overload it depending on what's in scope. Very confusing.
In haskell,
'x ^. i'
tells you everything. 'x' is any type, but 'i' is mandated to relate to 'x' in that 'i' is an optic targeting 'x' and returning some 'subfield' of x. In particular, 'i' is 'Getter a b', where 'a' is the type of 'x'. That is absolute. There is no other interpretation. Haskell type class overlap rules prevent any module imports from confusing the matter.
I'll agree that overloading << to do output is a WTF moment. But at least I can hire programmers off the street who know what output is. In contrast, that last paragraph of yours looks like you were carrying around a bowl full of random jargon, tripped, and spilled it all over your keyboard.
For your C++ example, I have to know the type of v and i (and that in C++, it's possible to overload operators). For your example, I have to know what an "optic" is, what a "Getter" is, what Haskell thinks a "subfield" is (and that means that 'x' can't be any type, but has to be the kind of type that contains a subfield), and I suspect several other background things that I don't even know what they are.
C++ makes you know a bit more about the types (though even in Haskell, you had to know that x had subfields). Haskell makes you know much more concept-level context.
The jargon isn't random, it's professional. Have you seen technical documentation for plumbing works in construction projects? They've got this glossary to memorize: https://wpjheating.co.uk/the-complete-plumbing-glossary/
It's scary that this sort of criticism is considered appropriate on hacker news, which is supposedly a forum for professional software developers. Very sad. Everything I mentioned should be taught in 2nd or 3rd year computer science course (namely, everything I mentioned was about type systems, which is a standard topic most university programs will cover)
If the concepts are the hard part to learn, then operators and technical speech are the least of your worries, but a mandatory part on the path to acquiring the art. The most important thing math has shown me, is that notation and precision of speech take a huge part in making things clear. Yet people get heart attacks when this is applied to informatics.
I still think this appeal to familiarity is dangerous. You take your knowledge about C++ as obvious and standard, and disparage knowledge about Haskell as a bowlful of jargon. But no-one comes into the world finding C++ intuitive, either; it has to be taught. And teaching C++ without referring to operator overloading, or with other attempts to obscure the way its practitioners actually talk about and practice it, can only be supported so far. Similarly here—I suspect more of the people who code with lenses in Haskell use the symbolic operators than the 'word' operators. Though I have no data to back that up, obviously it's how the writer of this tutorial thinks about them. So why shouldn't they use the format that makes sense to them, and that they hope to teach people to use?
However, I was curious - is it not possible in Haskell for two different libraries to introduce the same operator with different meanings, and still be used in the same program (though perhaps not in the same file)? That would be an unfortunate limitation all on its own, and one that mathematical notation certainly doesn't share.
In general, re-using symbols for similar purposes is also an extremely commonly used tool in mathematics, for the exact reason that constantly introducing completely new symbols is not worth the mental hassle. For example, when defining the "dot product" and "cross product" operations for vectors, entirely new symbols could have been introduced, but it was preferred to reuse common multiplication symbols (hence the names) as there is some relation to the real number multiplication operation.
It is possible, of course, just like it's possible for two different libraries to have a function of the same name. But it's always possible to determine statically (by looking at the import list, etc.) which of the instances the usage refers to.
As for <iostream> overloading the left shift operator to use for writing to an ostream, that is widely regarded as a terrible idea, and such overloading is generally frowned upon in the community. In general, <iostream> was created before C++ was even standardized, and well before good practices became established, and doesn't reflect at all more common usage of the language (see also the fact that they don't throw exceptions by default).
The only other somewhat widely used example I know where operators are overloaded with entirely different meanings than their standard usage is indeed in the boost::spirit parser generator library, where they are trying to approximate the syntax of typical parser definitions using C++ operators, with at best mixed success. And they don't just overload <<, they overload all the operators with completely different meanings - such as overloading the unary plus (+x) operator to mean "token appears 1 or more times" and even the structure dereference operator (x) to mean "token appears 0 or more times" and so on. Still, I don't think too many people are crazy enough to mix boost::spirit with regular C++ in the same file.
Note also that operator overloading is not commonly supported in other C-family languages, typically because* they are trying to avoid C++'s usage of it.
user1->name : "user1 arrow name"
(*user1).name : "value of user1 dot name"
x += y : "x plus equals y" or "x increases by y"
x << y : "x left shifted by y"; the C++ overload for output streams is a terrible one that also doesn't have a name.
> In contrast, most Haskell notation I've seen is either an ad-hoc invention for some library, or it is an ASCII version of notation in a niche domain like category theory.
But that's my point! When it was introduced, none of the symbology of school algebra satisfied condition (1) or (2); it was ad hoc (perhaps supported by some reasoning—as Reade's for the equal sign being two parallel lines, "than which no two things are more equal"—or perhaps not), and was found fully as abstruse and obfuscatory as you find symbolic operators in Haskell. Even Arabic numerals were thought a device for lies and deception, compared to good honest Roman numerals, when first introduced.) If we hadn't adopted those operators anyway, then we'd still be writing out all our equations in words as Tartaglia did, and our mathematics would be the poorer for it; had we stopped our notation earlier, we would still, as in pre-Arabic numeral times, send our children to special advanced schools to learn multiplication. New notation when first introduced is confusing and strange, but, once made commonplace, enables new thought, so that what was a niche domain becomes commonplace.
(For that matter, I'd disagree about (2). For example, there is a symbol called the vinculum that is used for many purposes in mathematics (https://en.wikipedia.org/wiki/Vinculum_(symbol)). Probably almost everyone has used that symbol for one of its meanings, but I'd argue that very likely almost no-one knows its name.)
If the Haskell community or the Lenses community wants to do the hard work of teaching everyone their new symbols, go right ahead. Until that work is done though, I would advice everyone to avoid using these symbols.
In regards to the vinculum, it's true that I've never heard that name, but I learned different names for the different uses - not in English though, as I took mathematics in Romanian, and some of the terminology and even notation is not the same (for example, repeated fractions are denoted using parentheses, not a vinculum - as in 1/30 = 0.0(3)).
Then how, if I might ask, is a symbol going to become ubiquitous, if it is not to be used while it is still obscure?
(and (eq (subtract u v) 10) (eq (mult u v) 1)) could be:
(u `subtract` v `eq` 10) `and` (u `mult` v `eq` 1)
> (u `subtract` v `eq` 10) `and` (u `mult` v `eq` 1)
True. I used Lisp because I think it's reasonably common to use words for arithmetic operations in many Lisp dialects, but not in Haskell. But it's not the prefix notation to which I was objecting, but the idea of spelling out every operation when we've got a notation specifically designed to facilitate rapid thought and computation without that.
> The basic operators here are ^., .~, and %~. If you’re not a fan of funny-looking operators, Control.Lens provides more helpfully-named functions called view, set, and over (a.k.a. mapping); these three operators are just aliases for the named functions, respectively. For these exercises, we’ll be using the operators.
Optics don't just save you a few keystrokes. Instead, they make tedious code that you would never write trivial. And more importantly, they're composable. So it's impossible to appreciate their effectiveness without using them on a real-world app. For example, I've worked in an app with three layers, each with its record types, and without the lens library, the pain of converting between records would have been insufferable. Instead, most developers would couple the three layers together to avoid the problem, which makes the codebase brittle.
But I quickly ended up in dependency or type hell when I tried using them. I hope that people will take inspiration from the concept in the future.
Wouldn’t know about the dependency issues, but the newer ‘optics’ library [https://hackage.haskell.org/package/optics] has much simpler type signatures and greatly improved error messages. Personally I don’t like it, since it hides the implementation details too much for my taste, but it seems to be becoming pretty popular in the Haskell community.
A lot of the time when I couldn't completely untangle some type definition in lens library, I'd just go with gut feeling or guess what it does by analogy with stuff I already understood, and surprisingly often it compiled and worked like I expected.
I found Hoogle very useful for this guesswork.
You will need to attain a certain degree of fluency in Haskell before you have the problem that deep access into structures or any of the other problems lens can solve [1] is a clearly-separated problem that rates among your biggest.
Until you reach that point, I advise not to touch lens. By the time that is your biggest problem, it is likely that your other Haskell experiences will make it not too frightening, but it's not a very good gateway into Haskell.
[1]: One of my favorite uses, though not a common one, is that you can use it to abstract a pair (thing to operate on, how to operate on it) in a way where both the elements are still fully composable, where the lens is sitting in the second spot of that tuple, and whatever is operating on that only needs the type of the "how to operate on it" part, it is oblivious to the type of the "thing to operate on". I used it to build a game engine where the engine did not have a hard-coded concept of "player 1 goes, then player 2 goes, then player 1 goes", etc.; instead, the game logic itself decided that and all the engine got was a combination of what structs to manipulate and what to perform on that struct. (The engine itself did networking and converted things to and from JSON, while letting the pluggable game logic do everything else.) An interesting use for lenses beyond just "better record syntax" as I started passing the lenses themselves around in my code as first-class values. But probably ultimately an even worse way to get into Haskell than using lenses as record syntax improvements.... my point here is just that lenses do have uses beyond record accessing, that's just their headline usage.
But the language allows defining your own (infix) operators, so the core language designers weren't entirely innocent either. The abundance of custom operators in Haskell kind of made me appreciate Lisp and Forth a little more!
There are of course many good things about having your language designed by mathematicians, too.
No, this is not the true motivation. Human attention spans on symbols, the more succinct the code is, the more meaning it can encompass in the same number of characters. Writing abstracted enough code often requires inventing new languages to hide the complexity behind abstractions.
Software development in Haskell is particularly focused on DSLs, lens language being one of them.
^.. is a relatively simple operator though, it just abstracts away traversals.
Human attention may pick out symbols more easily, but human comprehension benefits from things that actually have meaning. Otherwise you end up with a soup of ASCII symbols that very few can parse since every library in Haskell seems hellbent on inventing new and useless ascii combinations for function names and operators.
Even the article goes (emphasis mine):
--- start quote ---
The basic operators here are ^., .~, and %~. If you’re not a fan of funny-looking operators, Control.Lens provides more helpfully-named functions called view, set, and over (a.k.a. mapping)
--- end quote ---
And then proceeds to use the ASCII soup instead of the more helpful function names.
It doesn't help in the least that there are multiple get operations ^. ^? ^.. Or this idiocy: "Here, a single < before the operator signifies that it returns the updated value, while a double << signifies that it returns the old value. This gets a little weird with the monoidal operator; <<>~ returns the updated value, not the old one". Bonus points. What does this do: <<<>~
Because "why have useful names when you can pretend your fake intellectual superiority by using confusing ascii symbol combinations"
> Software development in Haskell is particularly focused on DSLs, lens language being one of them.
The domain specific language for lenses is view, set, traverse, update etc. Not whatever this is.
The word you are looking for is "script"; though that is rather ambiguous in the context of programming. I suppose "writing system" is good enough.
And now we have entirely expressed the problem: someone tried to make their code more readable by adding a donation-specific writing system! Just break out your handy Rosetta stone, and you'll be fine, right?
I could make it even more succinct by LZW compressing it. That would not improve comprehension, though...
[EDIT] Nor do Perl libraries tend to contribute to or encourage line-noise style.
Perl is IMHO worse because it relies much more on global mutable context. For example, the diamond operator <> behaves differently depending on whether the magic global variable @ARGV is set.
By contrast, Haskell is typed and purely expression based, so that at least in principle you can pull apart expressions, examine them (and test them!) individually and then understand how they combine.
And Perl is not line noise. It's a very nice language.
> This isn’t meant as a lens tutorial. Prerequisites are some basic knowledge of how to use Haskell, GHCI, and familiarity of the idea of lenses, if not the specifics.
With that out of the way, it's kind of disappointing that the article isn't a tutorial. As it is, it just lists the lens operators, gives you alternate, but still non-obvious names for them, and foists some exercises upon you. It could have easily given one short example for each operator without adding any significant tedium for those who already have an idea of what they do, and then it would have been a perfectly fine tutorial.
So here's some examples, (Hoogle has some nice examples too; thanks Hoogle):
person = Person {_name = "Alice", _title = "Dr"}
-- get field with magic lens
person ^. title == "Dr"
-- create new object with new field value
person & name .~ "Bob"
== person & (name .~ "Bob")
== (name .~ "Bob") person
== Person {_name = "Bob", _title = "Dr"}
-- create new object with function applied to field value
person & name %~ lowercase
== (name %~ lowercase) person
== Person {_name = "alice", _title = "Dr"}