Hacker News Reader

Comment by kccqzy | Hacker News Reader

So?

I tested my Python app on a list with ten elements and it works fine. But in production I unexpectedly received a list with a million elements! It crashes! Or more likely, it repeatedly receives a short list but still runs out of memory due to a memory leak I hadn't considered.

Python doesn't prevent a bad software engineer from writing bad code. Try to sell that.

View on HN

Lazy eval opens the floodgates for complex space leaks. Suppose you have a complicated algorithm that runs until it reaches mathematical convergence. In a lazy language like Haskell, those cycles of iteration/recursion can accumulate thunks and crash. Thunk accumulation and crashing can depend on input that you can't control or know in advance.

How do I know this? Because our big Haskell app died this way on live customer data and became a fiasco for my employer. We dumped Haskell and never looked back.

I don't doubt your experience, but in my experience judicious use of bang patterns and StrictData are enough to prevent those issues. It's no different from debugging and avoiding memory leaks in other languages. It just takes different skill sets that are more difficult to find.

Bang and StrictData are fine locally but things get enormously worse when that data flows into deeply-layered libraries that I don't own.

Then, I need to understand and modify the evaluation behavior of a deeply-layered external software stack, including its crazy type-level magic, all before a looming deadline.

Seriously, how is this good for my sanity or career?

What’s your product if you don't mind sharing? Curious.

The machinery is large-scale ML and discrete optimization over large/complex data structures, designed for a specific vertical industry. These algorithms run for hours and lazy eval was an engineering disaster. Our team had experienced Haskell developers who fought complicated space leaks night and day.

0/10 would not recommend.

And what are you using now?

The heavy numerical work is in julia, python, R, and a little custom c++ where necessary. Team productivity has gone way up.

Reply on news.ycombinator.com