To type or not to type: quantifying detectable bugs in JavaScript | Hacker News Reader

To type or not to type: quantifying detectable bugs in JavaScript

blog.acolyer.org

142 pointsff_149 comments

As we have recently adopted TypeScript, this is a nice affirmation, but preventing bugs is only part of the benefit and I'm not sure it's the most important part.

TypeScript, in conjunction with a code editor that supports it, significantly improves the code editing experience. Sometimes I still have to chase down documentation, examples, etc. to figure out how to use something, but it happens far less often. In general, I feel like I can code faster and with more confidence.

I assume all this is also true of Flow, though I've never used it.

The killer apps of static types are code completion, documentation, modelling rather than safety and performance. This makes TypeScript's unsoundness much more acceptable, though many in the type systems community still have a difficult time grocking this.

Pretty sure the first code completion programs were in dynamic languages. With many environments, you literally asked the object what methods it had. Same for documentation.

The first code completion system was in Alice Pascal circa 1986. Statically typed.

After that, it appears in production for the first time in VS 97, still using static type information. It wasn't present in smalltalk IDEs before then, or LISP ones.

I'll confess this surprises me, but not overly so. I'll try to find what I was thinking of. Primarily, I thought you could do live reflection in lisp machines. Which is basically this.

The primitives to do something isn't equivalent to doing it. So lisp machines had live reflection, but they didn't have the UX in place to surface the experience that we know today as code completion, they essentially didn't know that this feature existed or would be useful. As we know, UX is very important.

A similar situation happens today when we argue about live programming. Smalltalk had the infrastructure in place (hot code replace, fix and continue) to do much of it, but never provided the UX for what we would recognize as a live programming experience. Defining the experience is as important, if not more, than being able to realize it.

I'm not sure I appreciate the distinction here. I am not claiming that the original toolsets were the same as modern ones. In large, I would expect that was a limitation in memory and extra resources. Similarly, I expect the completion you are referencing is a far cry from modern completion. I remember early completion that I had access to was limited to available methods only. Was the referenced one more sophisticated?

That is, I was really only trying to reference the primitives. Because, well, progress. :)

Alice Pascal's code completion isn't really sophisticated, just convenient. They needed it because it was a syntactic editor and typing things was inconvenient. It turned out to be a good idea (for exploration) outside of its original context (save on typing)...but it's discovery was basically an accident.

I should add that I do appreciate being corrected here! I can't edit the parent post anymore to indicate that my claim was likely to be interpreted in a false way, unfortunately. Hopefully interested folks read down the thread.

I would think that primitive symbol completion (similar to Alice Pascal) was already available on the Lisp Machine a bit earlier.

See for example the command 'List Completions' in the CADR sources of 1980:

https://common-lisp.net/svn/mit-cadr/tags/system-46/nzwei/co...

"Give a menu of possible completions for string so far"

and

'Completion Apropos'

"Do apropos within the completions of what has been typed"

The editor of a Symbolics Lisp Machine would already sectionize a buffer and would know what is defined there. It also kept track what was changed in various buffers, what compilation errors were related to which definitions, etc.

Basic completion would work over symbols in a buffer or what is available in packages in the running Lisp.

Note that the Symbolics had a COMPLETE key on the keyboard and in a Lisp editor buffer it would run COM-COMPLETE-DEFINITION-NAME 'Attempt to complete the definition-name of the symbol under point'-

Mid 80s Symbolics introduced the Presentation System, which recorded all I/O (including graphics) and recorded which objects were displayed as what type.

Thus when one interacted with the machine, it knew what were classes, methods, functions, etc. and it could also reconstruct it from the textual display. If one would for example use an editor command which needed a class, then when typing, completion and searching was limited to classes and only classes on screen would be mouse sensitive. Also if one would right-click on an object, there would be only the commands for that class or presentation type in the popup menu. Similar for all kinds of dialog menus. The system knows which types are acceptable and knew which objects could be reused, or which to search.

This is not really 'completion', but puts the classes (or the presentation types) directly into the user interface - based on the dynamic classes and presentation types of the objects displayed.

A lot of interaction in Dynamic Windows goes through listeners and command loops, which understand input contexts and accceptable presentation types. The main listener does that, but also all other applications had such a command loop. When you interact in a REPL with Dynamic Windows, completion wasn't really that necessary, since the UI had access to the actual objects (and not just a textual representation).

From a UI standpoint I as a developer would write typed functions (aka commands) which then would be invoked via gestures (keystrokes, commands, mouse gestures, ...). Developing an application always then would involve writing presentation types and presentation methods for classes (and other things), which could then also be used in a listener/repl in a programming sessions. Let's say we wanted to implement a calendar, then one would implement presentation types for days, weeks, months, years, persons, rooms, events, tags, etc., provide visual and textual representations for them. Then while programming, we could play around with those objects in the listener. It would know which commands are applicable for the things on the screen - no completion necessary. Still if a person/event/room/tag/... would be requested, it could complete or search through the known persons/events/rooms/tag/....

Much of the UI was developed with Flavors and later with CLOS , such that there were lots of classes, lots of messages/generic functions/methods, ... Naturally this worked only over the things loaded into the current Lisp world. Code or other objects on disk was not available for this kind of interaction - though some stuff worked also with pre-computed tag tables. But to know about classes, methods, presentation types and presentations, they usually had to be present in the running world (which could be saved and restarted later).

Is apropos really code completion? It is neither using static nor dynamic type information, it is just using a local and/or global namespace. It is horrible for the browsing that code completion is mostly used for.

If I remember correctly, clos had a strange naming connection that included the type in the symbol name. I guess that would make apropos sufficient, right?

> If I remember correctly, clos had a strange naming connection that included the type in the symbol name

CLOS is built around multiple dispatch, so there isn't any type to go in the method name if you wanted; methods don't belong to a class, they just specialise generic functions onto any number of their arguments. What lispm described goes quite a ways beyond apropos, though.

Browsing is done with other tools, for example the Flavor Examiner or special editor commands.

Completion can know that something would be a Flavor message and search only in those or that something has to have a function binding.

But what it all kind of hinders, is that Lisp is a verb-objects language and not an object-verb-parameters language, where the object is an instance of a class and where the class is also a namespace. One is supposed to choose the operation first. In Lisp namespaces are packages (and not classes), thus the first thing is to choose the namespace of the operation, and then complete over the available operations in that namespace. Also names for operations tend to be long, so typing graphics:d-p and complete that would likely find the draw-point and the draw-polygon functions, since those would be exported from the package. Often there would be short nicknames for a package, say, gr for graphics, so typing gr:d-p COMPLETE would find the matching operations.

A drawing function will likely be named, for example, graphics:draw-polygon and take a graphics-stream and a list of points, thus find it quickly via some completion should not be that difficult. In the original Flavors it would be like (send graphics-stream :draw-polygon list-of-points) ...

But again, the action in the editor is only part of the story, because much of the development action will take place in the listener, where one interacts with actual objects.

Lisp, with SLIME (Lisp IDE tool for Emacs), gives you coding completion with a dynamic language. This isn't exclusive for dynamic languages.

However i don't want to debate "static" vs "dynamic". Static typing has advantages, it's simply that the advantages I see are performance (execution speed) ones. As mentioned above, i don't buy the idea that simple type errors are a serious speed bump in development speed.

SLIME wasn't released until 2003.

As an active promoter of live programming, I'm totally enthusiastic about code completion using dynamic type information. It just didn't happen before static types. Code completion on dynamic execution context is a bit challenging because you need to surface execution context somehow (which is a problem live programming focuses on).

it looks like guile's readline came in 97, with tab completion added in 98

Tab completion is often a small subset of code completion. When I think of tab completion I think of it being able to complete something you start typing based on what it knows to exist, but rarely does it understand much context.

Code completion meanwhile knows context much better.

right, context did exist at that point but seems limited to filenames, and symbols afaict, obviously though you can obtain much more context in a statically typed environment

It is this 'obviously' that I am challenging. It might seem easier to do it for statically typed languages, but it is not something that is only possible there.

Note, I am decidedly not claiming it is easier in a dynamic language. Just making the claim that it is possible in both.

I suppose yes, I could see such a thing working in the style of "A unified approach to solving seven programming problems" perhaps 'naively' would be the right word

>SLIME wasn't released until 2003.

SLIME replicates the same functionality of the Lisp programming environment of the late 1970s, so most of the features have been available for Lisp long, long ago.

Sure? And what were those features? As far as I know, most lisp IDEs had feature sets similar to enclave, which meant simple string-based code completion most of the time.

Code completion goes beyond auto-completing method names on objects/modules. Either way this shouldn't be some tribal us vs them thing, both static/dynamic have well documented utility for different use-cases.

But if we're going to make a generalization on which is easier or most capable to build complex editor integration around I'd say statically typed languages is the winner here.

Modern code completion does, yes. Just as static analysis goes beyond type checking of a language. The computing resources necessary to support modern tools is beyond what older tools had.

I am not trying to make this an "us versus them." To the contrary, I'm trying to point out both toolsets have many of the same advantages.

I will even concede that static languages do seem to have the better IDEs today. I assert that is as much a by product of money spent on development. Not a foregone conclusion of language design.

To me the big advantage of static typing is refactoring. I am dealing with a mid size JS codebase right now. It's well written but it's still really hard to refactor because you never know what code may break. In C# or C++ I can make a change and the compiler tells me what breaks. I haven't used TypeScript but it seems it will also make refatoring easier.

At this point I've done about equal amounts of programming in JavaScript and TypeScript. Yes, that's a huge difference. Without that one difference, the languages seem really similar, especially since ES2015.

I'm sure someone might chime in and say that good tests will cover that, but client-side JavaScript code can be difficult to test properly, so I'd rather just get the type checking for free.

I am deeply convinced the killer application of static types is polymorphism. Types create ways of organizing your code that are simply not available for dynamic languages.

Unless, of course, you get out of your way writing dispatchers, as is usual in Python. But those are long, repetitive, bug-prone and can not really be made generic.

You can have dynamic types that give you something similar. Overloading is much more difficult (though not impossible, few dynamic languages allow overloading outside of the receiver; e.g. In Dylan).

You hit the nail on the head. The biggest save for well structured teams would have to be revisiting tickets/speed of development. Having a compiler tell you: this field is missing, this field is wrong, a cannot be b saves a ton of time in terms of development validation (manual or automated).

As someone who grew up on Python and Javascript, I was never really that sold on typing, until I refactored something by changing a field on the model and Flow showed me all the places in the code that needed to be updated.

Granted, other more mature type systems/IDE will automatically refactor, but this was a really big obvious win for me :)

The title is, provocatively, "to type or not to type."

The top of the article makes it clear that it really, really, really will answer whether we should type or not type. From the top:

>This is a terrific piece of work with immediate practical applications for many project teams. Is it worth the extra effort to add static type annotations to a JavaScript project? Should I use Facebook’s Flow or Microsoft’s TypeScript if so? Will they really catch bugs that would otherwise have made it to master?

>TL;DR: both Flow and TypeScript are pretty good, and conservatively either of them can prevent about 15% of the bugs that end up in committed code.

In this comment I will address wherher this is really enough to deliver on that promise:

Why is bugs a metric instead of bugs per programming hour, or hours of programming including fixing bugs, with or without typescript?

Typescript adds types (and requires programmers to keep them in mind.)

I think it's hardly controversial to claim that typed programs have a lower bug count because type errors are caught. It is also not controversial to claim that programming time is slowed down, because of the need to think of types.

The question is: how much?

If a programming language slowed everyone down by 5x but resulted in 75% fewer bugs, few people would choose it. Most people would choose bugs.

If a programmer language was 100 times faster and only resulted in 5x as many bugs (so, instead of 100 times base rate, 500 times base rate) then I and practically everyone else would always choose it for almost everything. After all if it's 100x more productive you can take 80x productivity gain and sacrifice 20% of your programming wall time to spend on debugging.

So real results around speed and bug count, as well as the insidiousness of bugs, are crucial.

If the bug count were not increased in comparing a and b, but the second language had showstopping bugs that took 100x longer to find and fix, nobody would choose b.

Bugs aren't just a number.

The article quantifies the time involved.

Specifically, Table II gives mean times of 231.4 s (Flow) resp. 306.8 (TypeScript). So if you spend on average more than 5 minutes debugging errors that static typing could have prevented, you're probably better off just writing the annotations.

> It is also not controversial to claim that programming time is slowed down, because of the need to think of types.

I'd dispute that. I think the types provide me a notation to think about something that I'd already want to think about, so I can actually program faster with them than without.

(Very much agree with your overall point: what matters is not how many bugs we have but how much business value we deliver overall)

> Typescript adds types (and requires programmers to keep them in mind.)

Programs without TypeScript also require programmers to keep types in mind - it's just that those in those programs there's no way for a computer to verify that the programmer got the types right, and no unified way for a person to write down the types involved at any given point in their program.

> Programs without TypeScript also require programmers to keep types in mind

Not types, as such, but usually generics.

JS is hard to talk about in this context because of it's weak typing and desire to type cast just about anything. So the comment may well be true for JS, but I wouldn't be certain of it, it certainly wasn't the case when I was working with CoffeeScript to avoid the common pitfalls of JS.

But, for most dynamic languages, if you'll allow to stretch things a bit, I don't need to understand precisely what I'm feeding into the function.

Instead of asking "is it a string?" or "is it a vector?" or anything else along those lines, usually I just need to ask "is it iterable?", if so, good enough for me.

This can ease refactoring, when you need to change to a new type, because if the interface provides the same mechanism, you don't need to change anything. If it doesn't, then it may be possible to provide the interface, rather than modifying the function.

---

Things can get more complex than this, however.

Most people think of dynamic and weakly typed languages as the same.

However, there are several strongly typed, dynamic languages. Often times they would not allow you to pass the wrong type to standard functions.

Some dynamic languages have guards and contracts to ensure types - and some of those guards and contracts are... "compile time"... for lack of a better term.

That gives you optional typing, that you can add as you feel the need to tighten up your code, whilst still allowing you to be as dynamic as you feel appropriate.

> The question is: how much?

The article over on https://games.greggman.com/game/dynamic-typing-static-typing... attempts to answer that and comes to a conclusion that is different from the study in question.

So the article you just linked ("Greggman") was interesting and began roughly along the lines of the questions I expressed in my comment.

Sadly, JavaScript is not among the languages Greggman compared (didn't see it in the charts or on the page), let alone both JavaScript and a language that adds types to it.

That's what we're really interested in, isn't it. And as the original article we're discussing ("To type or not to type") promises, that's what would have:

>immediate practical applications for many project teams. Is it worth the extra effort to add static type annotations to a JavaScript project? Should I use Facebook’s Flow or Microsoft’s TypeScript if so?

So if these languages had simply been included in Greggman, it really would have gotten at the essence of my questions. Unfortunately they weren't studied.

I would certainly dispute that programming time is necessarily slowed down. Ironically, I find myself thinking much harder about types in a dynamic language, precisely because there's no compiler there to do the rote work for me. Lately I've been using pandas dataframes quite a bit, and the number of hours I've wasted due to problems with types is mindboggling.

Flow has features that allow for IDE-like editing, but I'll be dammed if I ever managed to get it to work.

My team is in the process of switching a React app to TypeScript and an interesting issue popped up just today.

I was refactoring some component to be able to implement a new feature. At some point of the process I managed to break the component so that it would not render, only complained about an object not being a proper React component and that I was probably trying to render a bunch of items that should be put in an array. The error was reported by something deep inside React so I couldn't tell from the stack trace where the issue was. After trying to find the problematic bit by trial and error (lots of undo/redo there) I finally decided to just move the whole file to TypeScript. 20 minutes later I was done and the error was gone. I don't even know what I did to fix it but obviously I had to change quite a few things here and there to get the file to compile cleanly with decent typing and one of those type errors must have been the key.

Type systems aren't just about adding static type annotations to your otherwise dynamically typed code to catch a few bugs.

If you embrace the type system and start modeling your domain types with the natural invariants in mind you start seeing the real benefit. Once you enforce the invariants of your data in the types you'll notice entire categories of bugs will impossible to introduce. Abstraction barriers between different parts of your codebase will become more clear and impossible to break. Otherwise complicated architectures will become easy to express.

Types change everything.

> If you embrace the type system and start modeling your domain types with the natural invariants in mind you start seeing the real benefit.

Except that few type systems have the capability to do this in a reasonable way that doesn't lead to horribly contorted code full of type-induced damage.

Just look at the difficulties that people have trying to express code at a higher level of abstraction like partial application, currying, generic composition, etc (as in fantasy land or ramda) in typescript.

If you're acclimatised to working in a specific type system, you'll probably never notice the areas where it stops you legitimately abstracting.

> entire categories of bugs will impossible to introduce.

This is true.

> Otherwise complicated architectures will become easy to express.

This, I think, is false. In fact many complicated architectures (depending on your type system) will become impossible to express, and to some extent this is a feature rather than a bug.

"Impossible to express" seems a bit much, but you might have to write your own Any type.

There's a great talk on this subject by Richard Feldman from Elm Conf. I haven't used typescript so I'm not sure if the techniques translate well. https://www.youtube.com/watch?v=IcgmSRJHu_8

It's great to see the wealth of data we have thanks to open-source being used for such studies, and we should have more. More concretely, I can see why actual benefit could be either greater or smaller than the reported result:

Greater: One of the advantages of types is that they impose a certain discipline and organization on the code. This global quality may have a significant impact that isn't measured in this study, which focuses on a very local effect, and so the actual effect may be significantly greater.

Smaller: Not all bugs are created equal, and there's actually a huge variability in both the effort bugs require to fix and in their impact on total product quality. If the 15% reduction is mostly in "cheap" bugs, then the actual effect may be significantly smaller.

We should try to create a taxonomy of bugs, classified by kind, domain, project size, cost to fix and impact. This would help us get a better picture of the overall effect of various techniques.

This premise of many bugs being "type errors" is overblown and certainly very recent, for i've never heard such a claim on the internet until only recently (1 or two years ago). And the world was using dynamic languages for decades on the internet (old ASP, old Actionscript, old JS, PHP4 anyone?)

If the 15% of your bugs are simple type errors, then I'd suspect the quality and experience of your development team.

The big white elephant in the room that javascript users don't want to acknowledge.

Mind you, static type checks are nice stuff, but to claim that 15% of bugs are type bugs, is another thing altogether.

I speak from experience, having managed development teams for years. Junior coders need (and deserve) to be trained first doing stuff without hurry to allow for mistake, not used immediately or thrown into critical (or rushed) projects.

It follows from Curry-Howard isomorphism.

Can you say more clearly what it is that you're saying follows from the Curry-Howard Correspondence?

My understanding is that the proof that your program is isomorphic to is not necessarily a proof of anything interesting in the domain of your program, which means that the practical consequences of Curry-Howard for day to day programmers is pretty limited. This is very definitely the case for most type systems practically used today (including the one that typescript uses).

It means that you encode some of your logic as types which can be statically checked. Ergo your logic can be statically checked.

> I've never heard such a claim on the internet until only recently.

Knowledge progresses. You can choose to come along or be left behind.

> If the 15% of your bugs are simple type errors, then I'd suspect the quality and experience of your development team.

This is an empirical study, carefully constructed to be conservative in its estimation of the achievable benefits of what even these limited type systems could achieve in practice. No amount of sophistry about what can be called a type bug can alter that. If you think the code sample was unrepresentative, perhaps you could arrange for the code of the projects you have managed to be analysed by the same methods.

>Knowledge progresses. You can choose to come along or be left behind.

Dynamic weakly typed languages have been in use on the internet for more than 10 years and i've listed them above, so we're not talking about new stuff or new paradigms.

If you, with your overtly smug comment above, want to imply that you are not "left behind" in terms of "knowledge" of the state of the art in programming, you shouldn't be clinging to javascript transpilers but using truly modern tools like Haskell, Clojure/Clojurescript, Racket, Julia and Lisp/Parenscript.

Because knowledge progresses, and you shouldn't be left behind.

Indeed I have moved on - I'm not using Javascript transpilers.

Maybe it was an unreasonably smug reply, for which I apologize, and I hope it hasn't distracted from the main point that the value of this empirical study is not diminished by specious arguments about what constitutes a type error.

>the value of this empirical study

The problem is the sample that this study is looking at. As i wrote above, GitHub is simply a big container of code for literally everybody, so judging kind of bugs produced by looking at what it's inside the Github repositories is akin to judging how many gramatical errors book authors make by taking a look at all the MS word documents stored in Dropbox.

PS: No worries, and thank you for being friendly. :)

That's a fair point, and I would like to see this sort of study extended to professional code, and more complete type systems. Unfortunately, there are only a few languages and type systems in which this sort of comparison can be made.

I am, by temperament, inclined to to believe that typing can improve quality, but so far, empirical studies have not made much of a case for that, and I put empirical evidence above even my own opinions.

Depends on what you mean by type errors. For example, NPEs in Java are technically a "type error", in that your code is mishandling data due to insufficiently specified types, and I'd have no trouble believing that 15% of Java bugs are NPEs or otherwise null-related.

If you extend the definition of type errors to include nulling issues, I'd have no trouble believing that TS reduces the bug rate by that much or more, since TypeScript lets you have non-nullable types.

Of course, you could argue that Java devs are just as lacking in quality and experience as JS/TS devs, but that starts to sound suspiciously like No True Scotsman/"real programmers do it in x86 assembly".

> If the 15% of your bugs are simple type errors, then I'd suspect the quality and experience of your development team.

IME everyone writes bugs, but simple type bugs rarely make it past testing if you have full test coverage. It would be interesting to know what the typical test coverage in the relevant projects looks like.

One problem here is that Javascript eats a lot of type errors silently.

  $ node
  > 1+[]
  '1'
  > 1-[]
  1
  > 1*null
  0
  > 1*undefined
  NaN
  > x = {}
  {}
  > 1+x.a
  NaN

This is not something that other dynamically typed languages necessarily do.

This is not something that other dynamically typed languages necessarily do

Yes, I agree, and this is because Javascript is very weakly typed, and sadly, no tool like Typescript will go so far in counteracting this, because the runtime itself (JS) is still weakly typed.

So... no real substitute for good old runtime testing, i'd say.

Except that a strong static type system wouldn't allow you to write code like this in the first place. You wouldn't hit these issues at run-time because you caught them at compile-time.

Say you have an API endpoint:

    router.post('/users', (req, res) => {
      const body = req.body as MyEndpointInterface
    })

If someone posts to that endpoint with the wrong interface, you're still going to have problems. I'm not sure of any language that would catch such a thing at compile-time and not need some sort of test.

Before I even read the article, I wondered how may of the bugs detected would be due to JavaScript's implicit type coercion rather than specifically dynamic vs. static typing issues. Sadly the article doesn't seems to say, but I thought it telling that the one example they gave:

  function addNumbers(x, y) {
    return x + y
  }
  console.log(addNumbers(3, "0"))

...of a bug that type annotations can detect has nothing to do with dynamic typing! As you point out there are plenty of dynamically typed languages (e.g. Python) where this would raise an exception. Now you could make an argument that in a static vs. dynamic context yo'd be talking about a compile-time vs. a run-time error, but strictly speaking, the bug here (a function returning a nonsensical value) has nothing to do w/ the kind of type checking performed. Static/dynamic typing and strong/weak typing are orthogonal issues and it's a pet peeve of mind when people sort of muddle them together as they do in this article. FWIW, I think there are decent arguments to be made on both sides of static/dynamic divide, but weakly-typed languages have always just felt like a terrible idea to me that allows careless thinking about data types to creep in all to easily.

Javascript is written around the expectation that developers are flawless, having total knowledge of their codebase and never making any mistake. Looks like there exist people that share that expectation, that's news to me.

> And the world was using dynamic languages for decades on the internet (old ASP, old Actionscript, old JS, PHP4 anyone?)

Are you claiming that that code has a low bug density? Those languages are famous for low quality code.

Please recall that my claim is for simple type errors. PHP SQL injection attacks, for example, were caused not by type errors but simply by a lack of enough education on how to safely use SQL (i.e. by using prepared statements, etc.)

No javascript or PHP system will prevent those. And that's my point, the really serious, time-consuming, havoc-creating bugs are the ones no static type check will save you from.

> PHP SQL injection attacks, for example, were caused not by type errors

Great example: SQL injection are catchable with types. Instead of using "String", whatever risks getting SQL injection attacks should only accept "EscapedString".

Besides flavio81's comment about SQL injection, Django has a very similar system for avoiding XSS too, where strings are automatically quoted when displaying, and only SafeString objects can create HTML tags.

The study showed that average programs on Github really do have these errors. Maybe not on your teams, but they are definitely out there.

Also, I am not sure how you could know whether these bugs have happened on you teams unless you were specifically classifying bugs by root cause? When the cause of a bug is a simple type error, programmers don't generally tell their manager, they just say it's fixed.

The true beauty of TS is that you can use it however you want. You want your TS file to be vanilla JS go ahead. You want to use the type system go ahead. There doesn't seem to be much of a downside to me except for the developers that falsely think a type system will slow them down.

I'm going to echo this. Ever since I've been working with JS (8+ years) type errors have only been a small handful of the bugs I've seen. Even 15% seem too high.

> This premise of many bugs being "type errors" is overblown and certainly very recent, for i've never heard such a claim on the internet until only recently (1 or two years ago).

Not true, I've been seeing it since at least 2004, and making it from not so long after that.

> the world was using dynamic languages for decades on the internet (old ASP, old Actionscript, old JS, PHP4 anyone?)

All of which were notoriously buggy.

> If the 15% of your bugs are simple type errors, then I'd suspect the quality and experience of your development team.

I struggle to think of a bug that isn't a type error. Almost all bugs^ boil down to "the programmer thought this was this, but it was actually that". As you gain experience with using static typing, you see how you can use it to eliminate all your bugs, and the ones that slip through turn out to be cases where you should have made a type distinction but didn't.

^ implementation bugs - I'm not counting cases where the program does what was asked for but that turns out to not be what was wanted

I think that when you have a type system that catches all bugs the type system itself is probably Turing complete. Writing the types in such a system may very well be programming.

The more advanced systems are already too hard to use too be helpful for common business people which means they don't help much in reducing bugs from misunderstanding the specs. Just my $0.02.

> I think that when you have a type system that catches all bugs the type system itself is probably Turing complete. Writing the types in such a system may very well be programming.

It is programming (though the best way to do it is in a system that's distinctively not Turing-complete - look at what Idris does), but it's all verified at compile time.

> The more advanced systems are already too hard to use too be helpful for common business people which means they don't help much in reducing bugs from misunderstanding the specs

Not my experience. The type systems make it much easier to express your actual business logic in straight-through code that looks like what it's meant to do, shunting off the awkward cross-cutting concerns to be resolved ambiently in a way that's safe, but doesn't get in the way right now.

> This premise of many bugs being "type errors" is overblown and certainly very recent

It really depends on whether your language is strongly typed or weakly typed. If it is strongly typed, then the runtime will catch type errors better, for instance if you try to add a string to a integer, it will yield an error. With Javascript it will yield nothing so bugs with accumulate and sometimes silently until the source of the bug becomes difficult to trace. Same for object properties. Let's say you expect foo.prop and you write foo.props . A strongly typed language would complain, JavaScript won't.

Some will argue :"Well duh, that's what testing is for". To an certain extent, and it might not catch type coercion bugs. Furthermore code bases with 100% code coverage are rare these days ...

> I speak from experience, having managed development teams for years. Junior coders need (and deserve) to be trained first doing stuff without hurry to allow for mistake, not used immediately or thrown into critical (or rushed) projects.

Juniors needs good tools in order to eliminate potential bugs. Static typing is one of that tool.

>Juniors needs good tools in order to eliminate potential bugs. Static typing is one of that tool.

Yes, I agree fully with you, it is excellent for training. My only gripe is against that "15%" claim and its implications.

I stumble on randomly getting integer/float/string in surprising places at least once per week in JS. Store integer in localstorage, you will randomly receive float back in chrome.

You speak from experience, the paper speaks from 95% confidence level statistical significance from a seemingly well-designed and very conservative study. Who should I trust...?

>from a seemingly well-designed and very conservative study

I'd dispute that. As i mentioned above, the sample is "javascript github projects". Github is used, at least in my country, often as scratchpad for bootcamp studies (in Javascript of course). This means in a given month, 200+ repositories created of code made by total beginners. Is this representative of your development team?

Yes, but consider this: how many scratchpad GitHub projects have actual bugfix PRs that are linked to filed and resolved bug tickets? The projects in this study were selected exactly on this basis.

Further, the authors realise that GitHub projects may be biased:

> GitHub may not be representative of projects, since proprietary projects tend not to use it.

And, they overcome this bias by sampling, from all GitHub projects:

> To construct a list of bugs we could uniformly sample, we took a snapshot of all publicly available JavaScript projects on GitHub, with their closed issue reports. We uniformly selected a closed and linked issue, using the procedure described above and stopped sampling when we reached 400 bugs.

When you sample from all available JavaScript GitHub projects, you have a very high probability of overcoming representativeness bias.

The next 5% of bugs can be taken care of by a handful of goodl tslint rules.

- missing awaits

- using 'this' in the wrong context

- shadowing variables

Those errors make for valid code that's wrong in many cases.

it doesn't catch type coercion, missing properties, ... a whole class of bugs that don't yield runtime errors.

And it prevents 100% of the time wasted on testing something in a browser (or generally at runtime) only for it to fail for some trivial type mismatch.

These things don't typically make it as bugs into production code but are a big time sink with dynamic languages.

I would really like a "strict mode" on my browser developer tools that would cause any problem in Javascript to stop everything on the page and write a huge "Hey, your code can not run because it had a parsing error at this line!" at the console.

Eslint and a Webpack dev server would have you covered in that regard.

As someone whose background is mostly in statically typed languages, who has been using Ruby and Rails for the past year, the biggest issue for me is when refactoring/making big changes to central parts of the app, the many repercussions of these changes often take a long time to track down, and subtle bugs are often introduced. By comparison, similar changes in a statically typed language are are usually much less scary, and less prone to having subtle breakage going unnoticed. A big test suite helps, but it's not a replacement for all the help a static type system will give you.

I'm in the same boat. Mostly C# for 15 years, and now Rails. I've seen bugs in production that were non-trivial to track down that a static type checker would have caught. Refactoring is tough, not just because of the lack of static typing, but because of implicit dependencies, crazy and inconsistent ways to handle and invoke method messages, etc. In short,I'm not a Ruby fan...

I feel like I'm on a completely different wavelength with those who offer up "full test coverage" as a desirable alternative to a type system's assistance. Or a desirable situation period.

I'm all for tests as a tool, but not full coverage as its own goal. One of my favorite tools is Visual Studio Code. As far as I can determine from the github repo it is very far away from the full coverage end of the spectrum. And yet, it works pretty darn good. Maybe I just haven't found the unit test hoard, but it appears they have taken a more strategic, moderate approach.

I'm a big TypeScript fan but at times it just doesn't work & needs to be ignored. Like all things, you get to a point when you know what rules are okay to ignore. You can't do default props in React very nicely for example. Functional programming like pipes can also be a pain.

That said, TypeScript (and Flow) offers other huge benefits than the 15% of bugs mentioned.

* Improved documentation

* Better code hinting in the editor & better editor experience

* Teaches better JavaScript & prevents browser bugs. Some browsers are very forgiving when you incorrectly use JavaScript with DOM elements. Others are not.

Types slow down beginners a lot but that's a good thing. When learning a new language, a type system can act like a pair programmer or a teacher helping guide you in learning the types of the language. Each type has things you can & cannot do. People who learned JavaScript using JQuery & then try to type JavaScript without JQuery run into many of these issues. After using types for a while, the time investment is very minimal especially compared with having to come back & find the error later.

Doesn't seem overly tricky? https://stackoverflow.com/questions/37282159/default-propert...

I've done that method & maybe I've missed something but I get a ton of "object is possibly undefined" warnings then.

I actually use Typescript to speed up development. Having the type-system looking over your shoulder is great for reducing the code-test cycle.

The TC39 has to step up their game and introduce static typing to JavaScript.

I don't buy the bugs argument, but I firmly believe it would make the development experience considerably better. Not only because of tooling, but specially because code would be more way more expressive.

Also in my almost 20 or so years writing Javascript and other dynamic languages I've never once changed the type of a variable. Changing a var from string to int or object is just wrong.

> ... I've never once changed the type of a variable.

I only do that for functions that accept multiple types (accept liberally and all that), but internally convert them into a canonical representation. For example, if a plain string is to be treated the same as an object with a name property, I will have code like

  if(typeof(x) === "string") {
    x = {name: x}
  }

I like how Rust just lets you declare a new variable with the same name but different type, which then shadows the previous declaration for the rest of the scope.

In those cases I usually create a new function depending on the type.

So for example I have a "main" function that does something with a date and expects a date object:

    doSomething(date)

And then if I want to pass a string I do something like:

    doSomethingString(date) {
        doSomething(new Date(date))
    }

You've never taken a value from a web input field that was supposed to be a number? You've never used parseInt?

I assume they meant if they had to do that, they would assign the result to a new var instead of overwriting the existing one.

Yes that's what I meant.

Casting is common in any language.

The author over looked flow-typed which I believe is an analog of DefinitelyTyped.

This seems sorta silly. Better tooling prevents more bugs. That is the Crux of the claim. Yes. It should.

Better static analysis is almost certainly going to prevent bugs. Type checking is the fashionable static analysis nowdays.

My bed with it is that it requires rewriting. Which, itself, will introduce bugs. Or just be expensive. Using other tools are likely to give similar benefits. With the side benefit of keeping the working code you have.

TypeScript and Flow are both designed to work with the way people write JavaScript. The DefinitelyTyped project adds TypeScript types for existing JavaScript code without touching the existing code.

You don't rewrite your code to use these systems. You add annotations, which should not be introducing bugs.

That is good and I'm interested in that workflow. I've yet to work with a team that didn't decide they had to rewrite it all. :(

This was definitely a first class design goal for Flow and TypeScript and, I'm certain, a non-trivial part of their success.

I feel there should be a counter-article along the lines of "Unit testing and static code analysis conservatively prevent 80% of the bugs". I've used both Flow and Typescript and imho they're both frequently more trouble than they're worth. They catch the simplest of newbie bugs that are rare in modular, linted, unit-tested codebases. There are static code analyzers (eg. tern.js) that provide hinting without requiring a transpiler. With Flow and Typescript you have transpiler overhead, but when it comes to places you might need them the most - checking and sanitizing data interchange that's so common in modular, service-oriented architecture - they fall flat. Flow or Typescript could have been more useful if the annotations truly were annotations in the form of comments, unfortunately they went the transpiler way.

>They catch the simplest of newbie bugs that are rare in modular, linted, unit-tested codebases.

This can't be repeated often enough.

FWIW, Flow allows you to put the annotations in comments, but the syntax for the type annotations is generally more pleasant to work with.

I have found the types to be useful in a large (for JS) codebase with more than a handful of developers. The types help catch bugs, especially when refactoring, even with a decent test suite and linting. Plus, using React and GraphQL, we get static types all the way from data coming from the server through to the UI.

Nice. It's too bad they didn't support that initially. It was issue #3 on their repo, incidentally. I might still be using it if it had been implemented sooner.

> They catch the simplest of newbie bugs that are rare in modular, linted, unit-tested codebases.

They catch 15% of the bugs in Javascript projects on GitHub.

Why do people insist on restricting their static code analysis into heuristic rules when there are so many perfectly good complete rules to be checked with a type system?

>They catch 15% of the bugs in Javascript projects on GitHub.

Using JS projects in GitHub is a really misleading sample. GitHub is used by all kinds of purposes, for example most coding bootcamps and institutes here tell the students to upload their code to GitHub. You have, for each institution, about 400 Javascript GitHub repositories each year, full of code written by true beginners.

GitHub is simply a big container of code for literally everybody, so judging kind of bugs produced by looking at what it's inside the Github repositories is akin to judging how many gramatical errors book authors make by taking a look at all the MS word documents stored in Dropbox.

The study only includes projects where people file bugs and fix them. That excludes most true beginners since they won't use the issue tracker at all.

> Why do people insist on restricting their static code analysis

Because it's trivially easy to show cases where a static type system rejects useful and correct code.

Yes, static type systems do "eliminate whole classes of errors", but they also eliminate whole classes of correct code! Is the freedom from (some, not all!) runtime type errors really worth the loss of expressive power?

Well, the answer is of course "it depends" on specific circumstances: sometimes it's worth it, sometimes it's not and most of the time the very choice (ie. should I write in statically or dynamically typed language in this case?) is not important at all because tons of other choices have much more of an impact on whether the project succeeds or not.

Unit testing is not mutually exclusive with static typing. Normal statically typed projects have unit tests too.

The biggest advantage of static typing, in my opinion, is that simple refactorings like "rename" or "move method from one class to another class" or "change what method does and then modify all places that call it" are done routinely and very often. Where renaming something in javascript is careful change you do when fully fresh, rested and focused, it is one quick right click when you are lazy and too tired to do real work in statically typed code. Means difference between rarely done and often done.

The other big advantage is when I am learning somebody elses code, all info about who calls who and how is right there.

>The biggest advantage of static typing(...) or "move method from one class to another class"

You are posting about static typing advantages. Please note that the use case you mention would not need too much refactoring in languages like Common Lisp (dynamic language) or Julia (dynamic), because of multimethods and because of the way methods are called. Methods -and method signatures- aren't constrained inside class definitions, they are separated.

JavaScript does not have multimethods and methods are called on objects. It sounds like methods must be purely functional and not use any instance fields.

The use case I had in mind was that I delete method from one class and create it in another. Then I go through all red in workspace and modify what is needed - sometimes just call the other place, other times remove the call entirely, yet other times conclude that method needs to go back because that two special places that don't have access to the other class instance.

Similar with changing signatures - I add/remove parameters and then go through all reds and decide how to fix them case by case.

The kind of refactoring you need to do depends on the language, but it seems like multimethods will have their own issues. I don't have experience with this, but suppose you decide to split a multimethod into two separate operations because on second thought, they shouldn't have the same name after all? How do you fix all the callers?

> The biggest advantage of static typing, in my opinion, is that simple refactorings like "rename" or "move method from one class to another class" or "change what method does and then modify all places that call it" are done routinely and very often.

Note that this is very doable in a dynamic language environment like Smalltalk or Lisp: you're constantly interacting with the running system, and so can easily rename a function or method, update all source locations, update the class of live instances (e.g. UPDATE-INSTANCE-FOR-REDEFINED-CLASS[0]) and even change the class of live instance (e.g. UPDATE-INSTANCE-FOR-DIFFERENT-CLASS[1]).

> The other big advantage is when I am learning somebody elses code, all info about who calls who and how is right there.

Again, a live environment is capable of tracking all callers & callees. It's pretty awesome, and honestly the only way I'd prefer to develop.

[0] http://clhs.lisp.se/Body/f_upda_1.htm [1] http://clhs.lisp.se/Body/f_update.htm

> I feel there should be a counter-article along the lines of "Unit testing and static code analysis conservatively prevent 80% of the bugs".

We need to talk about costs though. If a type system lets you write 15% fewer unit tests, it's worth it for that alone.

> There are static code analyzers (eg. tern.js) that provide hinting without requiring a transpiler

A static analyser is already a type system. You need a way to hint to the analyser, at which point you've got a notation for types in your language as well (maybe a limited one). You want the analyser to run on every build anyway, so a transpiler is no more effort; if you make the analyser optional developers will skip it, and will introduce bugs as a result, so it's better to simply not have the option.

> With Flow and Typescript you have transpiler overhead, but when it comes to places you might need them the most - checking and sanitizing data interchange that's so common in modular, service-oriented architecture - they fall flat.

You need to use types on the interfaces too. Use something like thrift.

Alright, costs. I think it's not well advised to use a type system as reason not to write unit tests. It seems to be a common misconception that static types are a replacement for testing. I think the real question is can you afford not to write unit tests that exercise functions with invalid arguments. I think it's still worthwhile because, like I said, you can have code where all the type constraints are satisfied but become violated by ingested data. You probably shouldn't assume that every user of your code is going to use thrift or graphql or whatever.

I think it's a stretch to say a static analyzer is "a type system". It doesn't obviate runtime type checking and implicit type conversion. But yes, absolutely you should run it on every build.

> I think it's not well advised to use a type system as reason not to write unit tests. It seems to be a common misconception that static types are a replacement for testing.

Of course they are: whatever your acceptable defect rate, both types and tests are ways to spend effort towards achieving it. The more defects you can eliminate via one, the less you need to catch via the other.

> I think it's still worthwhile because, like I said, you can have code where all the type constraints are satisfied but become violated by ingested data.

Most languages won't allow that to propagate through the system, and if you get an error at the boundary then it's obvious what the error is. (Admittedly a compile-to-Javascript language is the example here and may be the exception).

Fundamentally you decide how likely certain classes of errors are and look at the cost/benefit of the various measures available to you. In my experience types can, when you commit to them and work with them, reduce the defect rate very close to zero, and more cheaply than any other measure; I only feel the need to supplement them with tests very occasionally, and only for the very core parts of the code where defects would be most expensive.

> I think it's a stretch to say a static analyzer is "a type system". It doesn't obviate runtime type checking and implicit type conversion.

In terms of the mechanics of what it does, it's a type system. I don't see any value in having two type systems for the same code - type systems work better if they're part of the language so that all of the tools understand the same types the same way - so I prefer to use a language where the language type system covers all the use cases that a static analyzer might be helpful for.

My vision of it is Flow and TS and stuff prevent 15% of bugs that the beginner and intermediate level developers typically make, but engineers on the advanced and higher levels avoid these bugs by just writing a good code in any language they use.

True, but a lot of those people would rather use a safer language to prevent those bugs than write preventative code every single time, or remember to replace built in language features with their own functions every time.

So it helps developers that need it and does not get in the way of developers that don't ? Then its a win without any downsides.

That's an assumption for a best case scenario vs optimizing for the worst. I rather optimize for the worst, especially in a team; and furthermore a company with growth.

I've been writing js professionally for more than ten years, and I totally love flow. Sure, it doesn't happen that often that I feel like I've caught with it a bug I wouldn't have without it (not even sure how you can tell one from the other, when you correct a type check error), but the most notable effect is the release of tension : I don't have to double check everything I write, I can rely on the type system to report obvious problems.

If anything, I think flow does not go far enough, compared to compiled languages with type systems. The fact that it allows to mix typed and non typed code is error prone, and there are annoying edge cases, like the spread variables not being correctly checked for (this can cascade quickly, when passing properties in react). I would love to see types as a core part of the language, with native browser tooling.

Thanks, that's interesting experience!

Similar experience with typescript, the win is not having to make room in headspace for things the machine can handle.

No amount of good code will prevent a typo in a property name that's only encountered on the edge case of an edge case. It's arguably that 100% test coverage could catch such a bug but that's neither always possible nor cost effective. Static type checking is for catching exactly that sort of thing.

Even advanced and higher level developers can easily make the same mistakes as beginners or intermediate given that the code base is big enough and there are many people working on it. The probablility of that is lower but not 0. As an example you can look at quotes from MS guys, they said that it was benefitial for them to create TypeScript. And from my experience many MS engeneers are at very good level.

Oh yes, the old “I’m too good to need help from no stinking compiler” argument.

I never mentioned my level, didn't I? :-) In fact our team is going to switch to TS and I just don't have any other option but start using it. I am OK with that, I guess.

I dunno man, it sounds more like an intervention than a switch.

The best comment in this thread!))

Won't most code bases be written by teams consisting of some or even most beginner and intermediate level developers?

My impression is that most companies experience a shortage of talent such that they take in greener members with each passing year. Ideally they should have the best conditions to turn out good quality code before they get to an advanced level. The tooling plays a big part.

Revolutionary complement: if your definition of talent excludes junior developers, there won't be any seniors 10 years later. Every single senior was junior few years ago.

Moreover, companies are biased more against older people then young. Which gives us few yes when we are hiring magically appearing seniors - after they got experience (elsewhere) and before they are old.

To not make type errors you not only have to be a perfect typist and completely engaged all the time, you also need to be an expert in the part of the system you're developing.

Don't you just need to to actually use/test your code? Don't you need to do this in a statically typed language as well?

Often you don't. When you're really leveraging the type system, if it compiles it'll be correct. You might need to test the actual business logic, but realistically that ends up being a small proportion of the code you write. People don't like talking about how type systems let you write fewer tests, perhaps because it makes them sound like cowboy coders, but honestly it's a huge advantage.

Not only can you write more precise tests (because you don't have to worry about run time crashes from type errors or logic errors from type mismatches) but your tests themselves are also type checked.

I wouldn't consider engineers who think they "just write good code" and won't make those mistakes "advanced" or on a higher level. They have yet to learn one of the most important lessons.

>but engineers on the advanced and higher levels avoid these bugs by just writing a good code in any language they use.

Belief that one can just "write good code" and you don't need types/tooling/linting etc is a sure sign of a still inexperienced developer.

Actual senior developers are much more humble and use all the help they can find.

>Actual senior developers are much more humble and use all the help they can find.

sure, this is raison d'etre of package managers like NPM.

Christ, absolutely. No more hunting down obscure libraries through the Internet and guessing which version the author had in mind.

Or which 3 packages with the same name from spooky abandonware looking websites.

Typing systems for dynamic languages shouldn't be viewed as crutches for the inexperienced dev - experienced devs should seek them out when the ROI is there.

If any level of your tech stack offers you guarantees, use them so that you have less code to write. Less code is less opportunity for bugs and faster development.

I am wondering what level downvoters have :-)

"I think you're arrogant in suggesting that only beginner and intermediate programmers make these mistakes" level.

Yes, advanced programmers make these mistakes too, and they find tools like this helpful in catching them. That's why Microsoft created TypeScript. That's why Facebook created Flow. Facebook and Microsoft are two companies I wouldn't call beginner or intermediate.

1. Never I suggested that advanced programmer don't make mistakes. They do. Beginners just make more and/or more expensive mistakes.

2. Never I suggested that MS and FB are beginners.

This is an issue on which self-serving opinion, on both sides, predominates, so this study is very welcome. If you think it is flawed, do your own study.

> writing a good code

AND

using good code.

If a "Expert" have the option between use something that solve more problems better than other, is NOT a expert to not use it. Is just masochist.

Agreed, re-using a good code is one of the most important skills.

Engineers on the advanced levels are concerned about lightning fast and safe refactoring and resilience against spaghettification when passing on their code to other team members or outsiders of unoptimisticly unpredictable skill sets.

Reply on news.ycombinator.com