Worst practices should be hard (2016) | Hacker News Reader

Worst practices should be hard (2016)

haskellforall.com

67 pointstempodox31 comments

I love what little Haskell I’ve managed to pick up. That being said, I’ve come across bugs and poor practices in published libraries just the same as in any other language I’ve ever used.

That is to say, “Nothing is idiot proof given a sufficiently talented idiot.”

I think analyzing languages through the lens of incentives ("what is this language pushing me towards/away from?") is a great idea. But if Haskell's incentives make it well-suited for programming in the large, why is it so rare to see large companies using Haskell?

One explanation is: maybe Haskell is great for programming in the large, but is not very "accessible" to the average programmer. That is, as long as your team is competent, you can build really large, long-lived projects in Haskell more effectively than in most languages. However, if your languages requires an above-average level of competence, can you really claim that it's suited for "programming in the large?" To me, "PITL" means maintaining a huge codebase across decades, with dozens or hundreds of programmers coming and going. I'm also inclined to discount this explanation because the Haskell community often claims that Haskell isn't really that difficult to learn.

Another possible explanation is that people just haven't realized that Haskell is a great PITL language yet. But Haskell has been around for quite a while now; why didn't it take this long for C++ or Java or Go or Rust to become mainstream PITL languages?

I think the real reason is that Haskell actually isn't a great PITL language. Clearly it checks some of the most important boxes: type-safety, non-nullability, immutability, etc. But these are outweighed by other aspects -- most significantly, the "excessive abstraction" mentioned by the author. As admitted in the article, "abstraction is too fun!" Haskell's powerful type system, operator overloading, language extensions, etc. are all double-edged swords in this way, and in terms of PITL their costs far, far outweigh their benefits.

I'd really like to see more languages that make PITL their top priority. Focusing on "safety" or "robustness" is too narrow-minded; PITL affects almost every aspect of a language: the syntax, the type system, the tooling, even the compilation speed! AFAIK, Go is the only modern language to make PITL their "north star," and it has succeeded quite well -- but there's certainly still a lot about Go that could be improved. I think we'll see more "PITL-first" languages in the near future, probably incorporating ideas from Go, Haskell, Rust, and others.

> Go is the only modern language to make PITL.

I'd say elixir hits this really well. We're managing a huge project with lots of painful, legacy code that should probably be chopped off, but it's not godawful to maintain. I'm currently in a situation where we have some code hasn't been improved, so every night while europe sleeps I console into the container, hot-swap one module with some copy-pasted code, trigger a sync, and then kick the container so the sync code gets reset. Given our limited dev resources (there is a dev on staff that would be the fastest to fix it without having to learn a ton of business logic context but he's being resourced to other higher priorities now) to solve this problem, It's not the worst (this is honestly one of the least painful 'options' for managing this situation)... I mean, it will be awful if I have to do this for two months, but at that point, that's not a PL problem, that's an org problem.

I always hear Haskell people focus on reuse in terms of mathematical abstractions, like a reusable way of composing three functions in a a pattern that's "Really everywhere if you know where to look" or something, while abstraction as OOP people understand it takes a backseat.

To me, that seems hard to interpret any other way than "To use Haskell to maximum benefit you have to be very smart and able to see deeply hidden patterns with ease".

As an outsider, nothing about it seems like it was designed to take 1000 random new average graduates and put them on a project and get good results.

Java looks nasty and ugly but its hard to argue with how well it seems to work for large desktop apps. JS works amazingly well just because every individual project is trivial, the whole ecosystem is based on frictionless npm reuse.

Haskell looks like it has potential, and studies seem to show it reduces bugs, but I have no real way to evaluate it. I don't see thousands of people making million line apps with huge teams of mediocre developers.

I can't really try it myself because... I'm not about to spend 2 years working on a personal megaproject, learning the details of 2 dozen libraries,etc, just to evaluate it.

So... how am I supposed to know how well Haskell does in the contexts I'm actually interested in?

I could learn it, and build some toy app... but that's not going to tell me how it scales to a 10+ year monolith that is primarily about manipulating state.

Rust seems like the best choice for the low level stuff these days. I don't know what the true best for high level is, but Typescript and Python+MyPy rarely disappoint.

Almost every large project starts as a small project.

Arguably, all of the actually-successful ones did. The only projects I have heard of that started large are rewrites of another project, in another language, that did start off small. It seems possible there are a few run under rigorous engineering management, I imagine likely only in aerospace.

So, if you want a language to be good for large projects, and to get it actually ever used for them, it has to also be good for writing small projects, and good at adapting to them becoming large over time.

C++ targets that role: you can write a useful 500-line program, and also maintain a million-line system, all in the same C++. Rust is making a go at that space. Common Lisp and Ada wanted a piece of that pie. Scheme, by contrast, explicitly chose not to. C is fantasmagorically unsuited for writing big systems, but is shoehorned into the role anyway. Java was imagined to be targeting the role, but its design was so crabbed as to be almost as bad at it as C; but it is gradually being grown into the role.

I note in passing that PITL is unfortunately close to PITA. Maybe another term of art would be a better choice.

> why didn't it take this long for C++ or Java or Go or Rust to become mainstream PITL languages?

Java and Go have had corporate behemoths sponsoring them, some of the most powerful companies the world has ever seen. Rust has also had corporate backing of a sort, but I wouldn't call it a mainstream language anyway. C++, I don't know.

A Go+Haskell=NewLang could be really interesting... or really unusable.

There was a project that started down this road but was abandoned https://oden-lang.github.io

I absolutely agree that programming in the large is about the time dimension, or, as Google puts it [1], "Software engineering is programming over time." But I do think the author recognizes that - their argument is about the fact that despite Haskell giving you less short-term productivity it also discourages worst practices that will impede future programmers.

However, that still leaves us with the truth you found, that you don't see this in practice. There should really not need to be any "convincing." And it's not that people aren't allowed to use it: we all know of stories of an old application that is inexplicably business-critical, isn't in a language the company otherwise uses because someone got away writing it in a language they happened to want to try, and hasn't been rewritten, and that language is basically never Haskell.

I've been pondering Google's observation and I think my conclusion is that software engineering is the art of making good APIs for future programmers. The core part is still that software engineering is the sort of programming where someone (possibly future-you) will base their work on your program, and therefore the value of the program is in how it is written and not just whether it gets today's job done. But the future programmer can always choose not to base their work on your code and do the same work you did, and the goal is that you should be making their job easier. And, in turn, the future programmer may or may not be doing software engineering, which is to say, they may just wish to consume your base code and solve that day's problem without making things better for yet another future programmer. (This is common when a project either temporarily or permanently does not expect much future development; you need to keep it running and respond to slight changes in business requirements, but it's not worth doing the engineering work to generalize your task.)

In this view, it's actually important that your language facilitate both best practices and worst practices! If the language forces you to write good code all the time, then it's very hard to make things easier for future programmers, especially future programmers who are not trying to do software engineering. If it takes me five days to solve a problem well in Haskell because I can't do any worst practices, but adapting it to next month's problem also takes me five days because I still can't do any worst practices, I didn't do any software engineering - I didn't make my job any easier.

One of the interesting things about Python and Ruby is that they're languages that very much optimize for worst practices (short, untyped scripts with no abstractions and dynamic variable definitions and so forth), but they also happen to give you facilities for best practices if you want. Both of them have grown very good optional type checkers in the last few years, mostly written by large tech companies that have been programming in the very large in those languages, in both time and space, and needed to deal with their code. Both of them have optional linters. Both have facilities for good abstractions like optional record types - but both also give you plain old maps. And so forth. (By contrast, consider shell, which is fantastic for programming, that is, solving immediate problems, but completely unsalvageable for software engineering, that is, building APIs for others. There is no good way to bring order to hundreds of thousands of lines of shell, even if you wanted to.)

So, the Python/Ruby software engineer can build APIs for the future Python/Ruby programmer by using best practices in the long-term core they write, but the future programmer is absolutely free to use worst practices to solve their immediate problem if indeed they're solving an immediate problem. Or they can contribute to the long-term core using best practices. That seems like the happy medium here. And it's also very interesting that, as far as I know, they never set out to do this explicitly; they just happened to have just enough facilities for abstraction that over time people built more as they found themselves needing them.

Forgot the reference! [1] https://abseil.io/resources/swe-book Software Engineering At Google: Lessons Learned from Programming Over Time

this is complete elitism. languages do not “ requires an above-average level of competence” it might just have a longer learning curve. projects that are maintainable such as “huge codebase across decades, with dozens or hundreds of programmers coming and going” aren’t a special kind of program. these are just the successful ones

the distinction is between “hobby/academia” and “industry”

> languages do not “ requires an above-average level of competence” it might just have a longer learning curve.

Isn't that literally the same thing ?

I have mixed feelings about this.

I suspect Haskell has enough base-level complexity to be productive in it that it gets in the way of its adoption.

Contrast that to say writing something to serve http requests in Go. The number of complex concepts or idioms in related library pieces is comparatively very low. [0]

I’ve found the complexity of the solution should match the complexity of the problem being solved. If the solution has equivalent number of moving pieces and choices as the problem space requires, then it’s been made “as simple as possible, not simpler”. This is where some languages (tools) are better suited to a problem than others. Don’t reach for imperative patterns when css declaratively does what you want and has matured enough to have boiled down the very good ways of doing it.

I like Joe Armstrong’s approach of writing and then iterating a solution (a method, part of a feature, whole feature) until the extra dead material falls off.

So like I said, I have mixed feelings about this. It’s a shame to have to add artificial complexity.

[0]: Go and it’s libraries are admittedly not a perfect example. I find Go leans a little past the edge in trying to make things simpler than they actually are. But it’s not too far off.

The company I work for uses Haskell and Go. I can immediately read and comprehend what’s going on in any of the Go repositories in a few seconds. By contrast, every Haskell repo ends up becoming a DSL — which is a pleasure to write and refactor for the original authors but impossible for anyone else to understand.

“Best practices should be easy” is the corollary which, unfortunately, we are a long ways off from with Haskell itself, lending programmers to continue to use Python, etc. Looking forward to the day we figure out how to make Haskellesque ideas easefully available in future programming language X.

The end goal of a software project isnt just to write some code it's to write an effective application that's valuable to the business (aka it's supportable and iterable and all that businessy stuff). It's very easy to write some code in Python but very hard to know that it's correct without huge amounts of testing. And if you have changes happening on an application it's much harder to know your change hasn't affected something else when your main route is writing huge amounts of tests (that can also break).

A language that falls over at compile time when it's passing wrong types means your potential avenues for mistakes are already hugely narrowed down to business logic. If your application functionality is kept on course more then you're less likely to descend into chaos and find yourself paying a load of expensive Devs to chase down stupid bugs and debugging to keep things working rather than doing valuable "new" work.

Another benefit is if you can refactor with confidence you can expand functionality easier and not find yourself written into a corner by prohibitive Dev time costs.

The same incentives that would make a programmer chose "worst practices" in language X would also cause them to chose language X over language Y in the first place.

Saying "you should pick language Y because you won't have discipline in the future" is presupposing that i have discipline right now.

That's part of the advantage, chasing away devs who like worst practices. That makes libraries more trustworthy.

I really dislike this software development meme: that you should care more about why one tool is superior to the other than just knowing how to use any of them well.

If you hired Larry Haun and his brother to frame a new house, they'd have it done in a day or two with nothing but nails, a hammer, a SkilSaw, a framing square, a chalk line and some chalk. If you hired some random contractor, it'd probably take six guys a month, go way over budget, the job would be a mess, and they'd have used 15 different tools.

Larry is better not because his tools are better, or because one tool emphasizes how to use it the right way. Larry just knows his tools and knows how to do the job well with them. And that isn't to say that using any tool at all is fine; an unsharpened rip saw will take a hell of a lot longer to use than Larry's SkilSaw. But the outcome of the project still depends way more on Larry's skill than his SkilSaw.

This kind of discussion is more relevant to inheriting projects than constructing them from scratch.

To take an example I’ve inherited several Ruby on Rails apps from novices at big cos. The language takes the perspective that everything should be easy, so apps get rickety fast without diligent programmers. Which is a good goal but not something you will always have control over.

I think this presumes that working on projects built by novices is something we have to resign ourselves to. We could demand real engineering licenses for software, real apprenticeships, and a common 'zoning code'. But we have to push for it and not accept the status quo.

Software aside, I just checked out Larry Haun's videos -- very cool!

Kudos to the author for calling out Haskell's String—but the problems run deeper than defaults. For example: if you're not aware of strictness, you can easily wring poor performance out of the "good" types like Text and ByteString. And even in cases where you want laziness—streaming IO, parsing, and so on—you still can't use those, and need a stream processing library like conduit to handle resources and errors reasonably.

In comparison: does lack of monads, operator overloading, and so on make go simpler than Haskell? Sure—but, having written quite a bit of both: it really helps that go knocked this shit outta the park with their string type and related designs ([]byte, io.Reader/Writer…).

(Though, in Haskell's defense: go's designers include the inventors of the now-ubiquitous utf8, and Haskell…predates utf8's existence by 2 years.)

I've never been a fan of asserting that what the language you're using is defined for is full of good practices. For example, we all know declaration-site mutability causes lots of problems, but immutable-everywhere is not the best solution to it, only the most brute-force. The /best/ solution, something the fun sort of programmer knew well before Rust proved it, is use-site mutability. You can't get anywhere near the same performance with immutable-everywhere without a runtime (or complex rope-model library) specifically designed for it, and it matches the intuitive model anyway (I will never understand the Haskell-aligned habit of asserting that functional matches how you think about code, while pointing at an IO system that involves a function taking the observable universe and returning a modified copy of it).

> the Haskell-aligned habit of asserting that functional matches how you think about code, while pointing at an IO system that involves a function taking the observable universe and returning a modified copy of it)

The idea is that most code is pure, so we accept a slightly inferior model for code that does IO, in exchange for a significantly superior model for pure code.

And the idea is generally wrong; you can mangle code into a pure shape but the conceptual model does usually involve mutation.

My favorite example is Pandas, which makes iterating a dataframe hard on purpose to encourage using vectorization.

I was explaining this exact quirk of pandas to someone recently!

But I think a better goal is to make bad code patterns unduly tedious and good patterns very pleasant. There’s some cool idioms for library design in that direction that I’ve had fun doing

This is why most mainstream languages are increasingly coming around to the benefits of FP. I would love to use Haskell at work but Kotlin + Arrow gets 90% of the way there while staying familiar and easier to learn.

Looking for companies to support this claim, I think there could be a case made for Elixir/Erlang. WhatsApp, for example. Are there Haskell examples I’m not aware of?

It is used in financial tech sometimes. Although this is misleading as financial tech has a penchant for weird functional shit like OCaml, F#, K, Julia, etc...

Unlike a video game, you aren't transitioning state all the time. Proposition-based testing is also very convenient in Haskell with say QuickCheck, good for dealing with money.

Reply on news.ycombinator.com