Hacker News Reader

Comment by leononame | Hacker News Reader

This one definitely hits close to home. By empirical measure, static typing reduces bugs, but maybe it's just me. I definitely like the comfort of trusting my types.

My only experience with dynamic typing is really node, which improves a bit with typescript. But I'm still extremely wary of it because it doesn't give you runtime guarantees. It irks me a lot that you can just do JSON.parse, declare any type on it and call it a day. I firmly believe that static typing increases code readability and catches low-hanging fruit, but I'm going to have to sieve through that research it seems.

View on HN

> By empirical measure, static typing reduces bugs

As the article says, this does not appear to be actually true when you go and check.

Well, how is this measured? My experience working over 3 decades with large teams also says this is so (static typing does prevent bugs) but that is not science. How do they measure this 'when you go and check'? I am really curious as I don't even understand how people make large systems without static typing and all massively complex systems that I personally have worked with that run for decades transacting billions$ etc without flaws are all statically typed.

We don't seem to be able to measure it, despite people trying several wacky approaches.

Either that means it's hard to measure, or it means the effect is not actually there.

Either way, this means that we cannot say that static typing reduces bugs.

> all massively complex systems that I personally have worked with that run for decades transacting billions$ etc without flaws are all statically typed

Did you try building the same systems without static typing as well and see what the difference was?

Or are you just saying that you built systems with static typing and they were successful? That tells you nothing about static typing except that it doesn't prevent successful programs, which is a very weak thing to be able to claim!

Lots of people, like you, think static typing is very important for developing software, but when someone challenges this and says 'can you actually show that?' they have never been able to. At some point you need to reconsider if it's actually the case.

I do reconsider it all the time; we only have short lives and I cannot build a lot large systems in my life. So that is why I asked if how it is measured. I see that my teams are more effective with static typing, but that also is my management as I, as I said, cannot really imagine writing large systems in not statically typed systems. I write a lot of scheme and k and bash and wrote a lot of tcl and Perl but never could get to scale like I can with statically compiled languages.

I lean more to dependently typed languages than dynamic as I simply saw only misery, but, again, this is not saying anything, it is just my experience. I am a bit afraid though, because there is no clear measurement, it is just anyone’s experience.

Edit; which actually would be fine; if it works for you and your team, company and business goals…

> I see that my teams are more effective with static typing

If you think you really can see evidence for it then I'd encourage you to write up a paper and submit it for peer review. I guess when you sat down to write it you'd suddenly realise that you don't actually have any hard evidence.

> cannot really imagine writing large systems in not statically typed systems

Ok but lack of imagination is not science. Some people cannot imagine a spherical Earth, but that doesn't make it untrue.

For example I work on a system in a dynamically typed language (Ruby) that successfully handles tens of billions a year, so we know that it is possible. (We are adding optional static typing to it, but it was written without it.)

> Ok but lack of imagination is not science.

Sure, but that’s what I said in the first place. It all seems there is no evidence, not even empirical for either way. I was looking for any, if there was. MS or whatnot must have something no?

> I was looking for any, if there was.

I would say in summary that no, we don't have any evidence. Evidence that people have tried to present has been found to be flawed in peer review.

> We are adding optional static typing to it,

Why, if not for reducing bugs?

> Why, if not for reducing bugs?

Maintainability, tooling, and some people would argue for reducing bugs - but I'd challenge them in the same way I challenge you - can they prove it? And I would guess that they cannot.

If the person I was replying to was saying that they could not imagine a large system without types, and well they don't have to since I gave them a real example.

I agree it seems hard to prove; disappointed not more tried. Especially large companies that have skin in the game (MS with ts and c#/f#, Google with go and dart, oracle with Java, Mozilla with rust). Guess it’ll be a religious argument for some time to come and I will steer clear.

Something else I find rather likely is that different people are working most effectively with different methodologies (a believe that is grounded in the repeated experience of being shocked at how other people program and still be effective). So it is entirely plausible there is a self-selection bias that people that work best with strong types don't work on projects with weak types and vice verse. I guess its really hard to control for that effect when you want to look at big projects, since people need to be willing to work on that for a long time.

> can they prove it? And I would guess that they cannot.

Of course not, to do that you’d need a way to detect all bugs in a program.

If we had an effective way to that, we’d be using to remove all of the bugs from our programs instead of looking for ways to avoid bugs.

Perhaps it's like the LHC. Static typing reduces bugs above a certain project size, but we haven't been able to perform a study at large enough project sizes. Now if only we could get a bunch of governments all over the world on the case...

(only slightly joking)

A lot of the studies have been problematic because they look at toy problems (like leetcode problems). Static typing really starts to show its value in large projects. One huge project I'm aware of that uses a lot of python is the Sims 4 which uses it for a lot of the game engine and mods.

Your argument is analogous to the one used by homeopaths and those believing in telekinesis. "There is an effect but it disappears in a laboratory setting, but it's still there!"

All effects disappear if you don’t use strong enough measurement techniques

The article mentions the literature review is from 2014.

Many older statically typed languages (Java/C/etc.) don’t enforce null safety. Tony Hoare called it his ‘billion dollar mistake’.

https://en.wikipedia.org/wiki/Tony_Hoare#Apologies_and_retra...

Thankfully more modern type systems (Swift/Kotlin/F#/Rust/TypeScript/etc.) now do, hugely reducing the likelihood of runtime errors.

As a mostly dynamic language developer I never saw the benefit of Java-like static typing but more modern type systems seem much more useful.

> hugely reducing the likelihood of runtime errors

You say this like it's a fact... but again nobody has been able to show this in a proper scientific study.

Considering how many null pointer bugs I've seen, I think that this is a case where absence of evidence is not evidence of absence (at least for null safety).

There's no evidence of the absence of the easter bunny, but we are happy that he probably doesn't exist since there's an absence of evidence.

Write up your experience with null safety as a paper and submit it, because this would be a world-first if you can demonstrate it.

I’m not really sure what you’re arguing. You seem to be saying that since we can’t prove that one approach is better than the other, we should just assume they’re all equal.

We do have proofs that certain classes of bugs are impossible given a certain type system.

Almost all of the null pointer errors I've seen came from not fully thinking things through. I can see the same mistakes happening with non-nullable types because someone got lazy and passed a default value or something. You get the benefit of the compiler shouting at you for not initializing a variable, but that won't necessarily protect you from not thinking every situation through / lazily passing default values / making poor assumptions.

I'm with GP, I'd really like evidence of this before people continue shouting things which aren't proven.

But the languages that are null-safe all have Option, Optional or Maybe.

So if I do need a null value, I can make the field optional and the type checker will help me figure it all out.

I don't think a scientific study is needed. There exist different classes of bugs. The stronger static typing, the more classes of bugs become impossible to make. You will not see a NullPointerException in Haskell.

The only way in which strong static and dynamic typing could produce the same number of bugs would be if strong static typing resulted in introducing other bugs, ones which wouldn't be introduced in a dynamically typed language. Proving that would probably require a scientific study ;)

I am curious for why people say this as in my experience when writing a function in a dynamic language that takes a variable as a parameter:

The function will generally only work on a subset of types for given variable.

If I don't check the type of the variable in the function, the function will not behave as you might expect, e.g. silently fail or crash.

If I do check for every possible type for a given variable:

I may not have a good way of handling certain types being passed in, I may be forced to either log something out, create a run time crash or even have the function silently fail. All 3 are bad run time behaviours.

If I am checking for every type in the functions, then using static typing would cause the failure at compile time so the bugs could never exist, but also being significantly less verbose than the dynamic language equivalent.

> I am curious for why people say this

I'm saying it because it's a fact. I'm not giving you an opinion - it's a falsifiable fact that you can verify for yourself - we have as an industry not been able to give any good evidence for static typing reducing bugs that has stood up to peer review.

You're presenting arguments for why you think there should be evidence... but when people look there isn't actually any evidence. Maybe your arguments are not sound for some reason that we don't understand, or maybe we are unable to measure the effect.

Well, this paper found that a conservative underestimate of 15% of bugs would be saved through static typing:

https://www.microsoft.com/en-us/research/wp-content/uploads/...

I'm not sure what evidence your fact is based on, feel free to present some evidence for it.

> I'm not sure what evidence your fact is based on, feel free to present some evidence for it.

The linked meta-study in the article we're commenting on.

https://danluu.com/empirical-pl/

"under the specific set of circumstances described in the studies, any effect, if it exists at all, is small"

And this question has generated the most rebuttals and retractions I've ever seen for flawed studies in computer science - it's notorious.

Remember this?

https://dl.acm.org/doi/10.1145/3340571

> Well, this paper found that a conservative underestimate of 15% of bugs would be saved through static typing

Yeah does look that one shows a larger effect and has not been rebutted or retracted. One positive measurement against many negative measurements.

It does seem to me that you are conflating lack of evidence with non-existence though.

We haven't measured an effect but that doesn't mean that we can't reason about this in other ways. The lived experience of people that do this thing for a living can be an important resource. You could argue that we are getting in to soft social science here, but is there value in what the collective of trades people think about their tools?

We didn't know the science behind steal for a long time, but we still figured out how to make it and that it holds an edge really well.

Empirical measurement is not the only way to discover value.

Fwiw a similar pattern is seen in cooking, where lots of chefs have reasoned ideas of how things work based on tradition and lived hands on experience, yet are scientifically disproven.

Lived experience works to some extent, eliminating failures and making progress, like with your steal example, but that doesn’t mean it can pick optimal options from a list of options that all work to some reasonable extent; aka dynamic typing does also build software.

> The function will generally only work on a subset of types for given variable.

This is true for many functions defined in statically typed languages too. Just because your function says it works with an integer, doesn't mean it's necessarily going to work with _any_ integer (think `1/n`). Very often the type used isn't narrow enough.

Modern dynamic languages understand this, see for example Clojure's spec[1].

[1] https://clojure.org/about/spec

I like libraries and languages that create a bunch of types which are essentially just renaming existing types. So you'll have a type PostID which is just an int but the language won't let you give a PostID to a function that takes UserID even though its both just integers.

Interesting. I wonder what about software development that we do that you can prove scientifically then.

Does planning a project mean faster delivery?

Does thinking about architecture up front reduce refactors and technical debt.

Does estimating lead to faster software development?

There is a lot we have to decide without evidence in how we develop software.

Maybe we just pick the things that feel right and make us happy.

> Does planning a project mean faster delivery?

Does that plan include the client changing their mind 10 times before delivering? Or finding out that some approach doesn't work and that pivoting is needed halfway through?

Likely depends on how smart the planning is.

> Does estimating lead to faster software development?

Unlikely, because engineers are then busy ass-pulling useless time figures over and over instead of actually working on the project.

All of these points have high random factors attached to them regardless, so you'd need a pretty big sample to say what generally works best.

Yes I agree with you. My point is that almost everything we do in management if software teams and architecture decisions is a guess.

We have to carry on regardless and make decisions.

The nuance is what you do, why you do it and is there cultism or cargo cultism to the decisions.

> It irks me a lot that you can just do JSON.parse, declare any type on it and call it a day

You should look at zod [0], which validates data with inferred types based on the validation having passed, so you do have the guarantee of the type being correct at runtime.

[0] https://zod.dev

Thorough understanding of the problem and thorough testing prevent bugs. Static typing allows you to more succinctly express your thorough understanding of a problem, but it does not fix the root problem.

Quality of life benefits are where static typing (and in some cases performance) really shines.

Reply on news.ycombinator.com