Hacker News Reader

Comment by slashdev | Hacker News Reader

I used to be a huge dynamic languages fan, with python being my favorite language over the majority of my career. Then I worked on a large python project of ~100k LOC with a team of ten. That's when I realized that writing code faster isn't the problem. Reading it is, making changes to code someone else wrote is, and refactoring across dozens of modules is a problem. Static languages help a lot with all three. I still love dynamic languages for small tasks, but I'd rather use a static language like Go which still keeps much of the dynamic feel, and helps catch my mistakes at compile time. I actually just set it to watch the project directory for changes and recompile automatically in a terminal on another screen (actually to run the unit tests, which includes compiling.) That's constant feedback and comparable to dynamic languages speed for edit-compile-test runs. But the best part is the unit tests. They run so much faster with Go that I don't get distracted waiting for them to finish, and that keeps me in the zone and much more productive.

View on HN

Agreed. I often find that most people who are very enthusiastic about dynamic typing have only seen it in action on solo or small projects.

Add complexity or people and dynamic typing quickly crumbles.

I made this point in an earlier comment, but I'm a firm believer that if the speed at which you can physically type code is the bottleneck, then you are going to eventually have much bigger problems. Hastily written code will require more time to fix than it will cost you to slow down and think about a good design, regardless of language.

To represent the case for dynamic languages as "the speed at which you can physically type code" is a straw man.

I agree, I think there are many more issues to consider. However, the OP seemed to be focusing on that particular argument, hence my comment. A few examples:

"The results were that the people using the dynamic version of the language got stuff done much quicker"

"it took less time to find those errors [in the statically typed language] than it did to write the type safe code in the first place [in the dynamically typed language]."

"[programs written in]" dynamic languages took significantly less effort to create (less time)"

If you can code quickly, then the effort required to refactor previously written bad code is lower. I.e it is not that you can type faster that matters, it is that you can fit more change iterations in the same time frame. And many times multiple change iterations are required because you can't see beforehand whether a particular refactoring will be a net positive or negative.

Strongly typed languages beat dynamic languages hands down for refactoring. The ability to know immediately what broke without having to run all your code paths dominates any time savings gained by typing faster.

I'm a fan of strongly typed languages with good type inference (and therefore less typing), but even in verbose languages like Java this is true about refactoring.

The effort of writing (typing) code is directly (at least) linearly proportional with the effort of reading said code (including the time when you're thinking in your head before you typed it in). And by "effort", I'm strictly referring to the physical aspect of reading the text, disregard the comprehension part.

As many advocates of static typing emphasize the importance of reading/ refactoring in large code base, this would be a huge factor as well.

I do agree with the arguments here literally, but I think they sound much more forbidding than they should be.

There are plenty of big projects around that could broken into a collection of small, independent projects without any loss.

I mean, unless you live under a rock you have to realize that there are a lot of complex non-trivial apps that run in languages like Python, JavaScript, and Ruby.

Sure, but that doesn't prove anything besides the fact you can write apps in any language.

The question is more: would writing these apps in a statically typed languages have been "better" along certain axes (easier to maintain, easier to write, easier to evolve, easier to debug, faster to ship, easier to recruit for, easier to ramp up newcomers on the team, etc...).

You said that it "quickly crumbles". What exactly does that mean? Sounds like you are softening your stance now, so why even so something so definitive?

Because automatic refactoring tools are not available for dynamically typed languages, programmers are more hesitant refactorings. So the technical debt accumulates and the code gets worse because of that death spiral.

That's what I mean when I said that dynamically typed languages crumble under large code bases.

Also in Erlang and Clojure. I think one of the most reliable software systems of all time was written in Erlang.

http://www.ericsson.com/news/141204-inside-erlang-creator-jo...

Erlang is popularly used with an external static type checker, the Dialyzer.

I disagree, I warmed to dynamic typing specifically due to the experience of working with 2-3 million lines of C++ code. The problem with static typing at that size is that you very frequent impedance mismatch. A lot of the code you write ends up being about translating from one type to another which is almost identical.

You also get dependency problems you don't have with dynamic languages because you aren't required to import type definitions to use an object from somewhere else. Your code easily get tightly coupled. You end up with a nightmare of compile times.

The couplings get so intricate over time that it becomes and almost intractable problem to figure out how to take it apart.

That is much easier in a dynamic language. You can tackle problem far more independently from each other because dynamic languages don't impose as much dependencies between types.

> You also get dependency problems you don't have with dynamic languages because you aren't required to import type definitions to use an object from somewhere else

The dependency is still there, you are using that object in your code.

Dynamically typed language gives you the worst of both worlds: same dependencies and zero discoverability.

> You can tackle problem far more independently from each other because dynamic languages don't impose as much dependencies between types.

This doesn't make sense. The code is exactly the same except that you don't have type annotations with dynamically typed languages. In other words, you can't tell exactly how the code you're looking at interacts with the rest of the code base.

Which makes it impossible to untangle when it grows and also for automated tools to figure out what these dependencies are.

> The code is exactly the same except that you don't have type annotations with dynamically typed languages.

No they are not the same.

Here's some key differences:

- If at runtime an execution doesn't even reach the code, then it doesn't matter at all for that particular execution whether the types were "correct". With static typing you need to get it right for all possible executions before you can even compile, regardless of how unlikely or unimportant those executions are. In practice, having to get it all right for all executions before you can even run one is a massive productivity loss in a lot of scenarios.

- Not only might a "type" of something in a dynamic language not be expressible statically (at least in majority of static languages) in the first place, dynamic code is never dependent on any particular type. You might get an unintended "type" in and the code can still work. There is a dependency on infinite amount of unexpressible "types".

You are completely ignoring the point of dynamic typing and you are using it as if it were static typing without annotations, by your own admission. But that's your problem. Similarly I could also use statically typed language as if it were dynamic and then claim that static typing is exactly the same as dynamic except more verbose.

> You are completely ignoring the point of dynamic typing and you are using it as if it were static typing without annotations

That's exactly what it is.

What is your definition of a dynamically typed language?

So I just told you about a few things that are physically impossible in static languages, therefore dynamic language cannot be a static language subset. For example, you first need to prove that it's possible to tell a static language that "for now, for this execution, I don't care about these 3150935031 trillion code paths being correct, just these 123 code paths, so just compile and do it". Without that you are just trolling.

    x = 3
    print(x)
    x = "hello"
    print(x)

Not possible with static typing and type inference (which is static typing without annotation) but done all the time with dynamic typing. (Though still not a good idea, in my mind, but to each their own.)

I'd rather use a static language like Go which still keeps much of the dynamic feel, and helps catch my mistakes at compile time.

Right on. It's the "dynamic feel" which is really the key. It's the ability to iterate quickly, the access to anything in the runtime, and the ease of building software tools. It's an environment that makes you powerful by enabling everything and staying out of the way. Whether that's provided by "dynamic" typing is just an implementation detail.

Unused imports and variables being errors is pretty squarely counter to that philosophy.

I don't think Go represents the "static typing, but lenient" philosophy particularly better than C#, for example, does. Go seems to focus on simple typing, not lenient typing. There's a big difference between the two.

I never said Go was perfect. Go needs to have much better debugging for my tastes.

I am never bothered by unused variables when I'm doing exploratory programming in Go because if the variable is still unused, I haven't written enough code to get to the point of running it for testing.

I used to be bothered by the import thing, but then I installed goimports, and now Go is actually better than other languages for imports. Just write "foo.Bar" in your code and goimports will automatically add "baz.io/foo/" to the imports section on save.

How about optional type systems? I feel like in Go I have to use something (like an interface) to achieve the feel of dynamic typing but it feels much worse than having optional types or no types at all.

100k lines of Python? I find it hard to believe that this project is properly organized. Rarely do you see 100k lines of Python solving a single problem, instead a project will be divided into many components, some of which can be reused in other projects. There shouldn't be any difference working on 1k line or 1 million-line code bases as long as the code is decoupled. Testing each part in isolation helps a lot (as opposed to running tests for a big monolithic ball of mud), as it ensures contributors are not introducing crazy dependencies.

I'm not against types at all, but don't think they solve problems related to big code bases.

They don't /solve/ problems but they sure as hell help in solving them. No need to throw everything out just because it isn't a silver bullet.

Is there ever a project of that size that's properly organized? We did our best under the constraints we had. There was always a long wishlist of things we would change if we had the time, and some of them we were able to do.

> But the best part is the unit tests. They run so much faster with Go that I don't get distracted waiting for them to finish, and that keeps me in the zone and much more productive.

You're describing a problem specific to dynamic languages with slow implementations (Ruby, Python). Languages like JS that have optimized multi-tiered JITs such as V8 will essentially run your unit tests just as fast.

Is that really the case for unittests? (this will be different for actual run in production) Apart from some things shared across the whole codebase (loggers, db access, etc) most of the code is unlikely to ever get "hot" enough for the optimised JIT to kick in.

If the code isn't hot enough to leave the interpreter into tier 1, then the engine has judged that interpretation is likely to be faster than non-optimized compilation plus machine code execution. An AOT compiler with no interpreter (such as Go) only has the latter option. So, assuming the JIT is tuned properly, the resulting turnaround time will be faster than AOT.

But your comment wasn't about Go, it was about "slow dynamic" (python, ruby) vs V8. I'm saying that's just "non-optimized" vs "very likely non-optimized", so multi-tier JIT shouldn't affect much of the unittest runs.

Even following that logic about JIT vs AOT in a single run, the python vs v8 equivalent would be that: you have non-optimized code vs non-optimized code which still has to keep track of execution counters, therefore the one without counters should be faster.

Regarding unittesting Go vs V8... I'd really like to see a big project where it could be compared - unfortunately that's not realistic.

Oh, I see. Apologies for misinterpreting you.

In the case of, say, CPython vs. V8, it may well be a tossup. But I highly suspect that V8 would usually win even for unit tests because it'd be able to optimize functions that many unit tests share. For example, it could optimize the test harness itself (important when it's a nontrivial test harness like Cucumber or whatever), it could optimize the string search/regex functions, and so on.

You'd think so, but in reality if you take something like a big Java codebase and disable the compilers, then run the unit tests, they are always waaaaaay slower.

The "shared across the whole codebase" stuff tends to be much bigger than you'd think. For instance the collections libraries, string manipulation, any core code or data structures in your own codebase, etc, all get compiled very fast. You don't have to run something very often for it to become eligible for compilation.

While that's true about the usage I'm not sure it applies in that comparison. All primitive collections, string manipulation, simple ops, etc. are already written in non-interpreted language and compiled AOT in Python. I expect they're also native in V8 and Ruby, although I don't know that codebase.

I'm not sure about "core code". I mean, the context is "unit tests", not "functional tests". If you have hot core code in your unit tests, then something smells... Hot helpers, and cross-cutting utilities, sure, and I mentioned them.

Several flavours of Lisp have been used for building very big pieces of software (e.g. Emacs, Orbitz). Same for some Erlang products. I think this depends on the architecture, documentation, etc.

Dynamic typing allows for loose protocols, which is sometimes very advantageous when you don't know exactly the software you are building. PG explains this much better.

Nonetheless, I'd love to see more optional static typing systems on top of Lisp.

You cannot program effectively unless you understand the syntax and semantics of the data representations you are using, and when you do understand those things, perhaps you are unlikely to make type errors in the first place.

I meant the domain of the problem, so I was saying dynamic typing is advantageous for exploratory programming and modeling. Whereas static typing is easier to use if you have pre-planned all your programs.

I wasn't intending to disagree with you; my intent was to suggest a possible explanation for why static typing may be less helpful in practice than some people (myself included) would have guessed. To put it another way, programmers have to be aware of their data's types regardless of whether they are using a statically or dynamically typed language. If this is correct, then we can assume that the developers of successful large projects have this ability, regardless of the language used in that project.

From my experience, a statically-typed language also lets me think upfront about the structure of the data that a program will manipulate. Getting your data structured properly from the start would result in straightforward algorithms and less future refactoring, which is certainly more efficient than throwing away and rewriting code every few months.

Which is great, for projects where that kind of upfront waterfall design is sensible.

That's not most projects.

It has nothing to do with upfront waterfall design specifically. Making choices about how you represent information is an integral and unavoidable part of algorithm design at any level of detail.

There are no projects for which a certain amount of thinking before you write is ill-advised.

Indeed, and personally I find that having the types around also makes it much safer to do the inevitable rounds of refactoring once you figure out what you're really building. (Especially in a language like Haskell where most of the type information is actually inferred from context. It requires absolutely minimal type annotation.)

EDIT: Maybe it's just my bias, but it seems to me that most of the "dynamic" crowd just haven't tried a truly "agile"[1] statically typed language like Haskell or O'Caml.

[1] Hey, if they want to abuse the connotations of out-of-context words to describe the paradigm, so can I. :)

Absolutely this. Most programs are really all about data. I build my data structures first. Always. If someone can't tell me approximately what data I'm working with, I won't work on that project. Data first. Then the data drives an interface. When you have those two pieces of the puzzle figured out, then we can start designing the code that solves the problems needed to get the data from the source to the destination.

I'm sure there are domains I'm not experienced in where that Dev model doesn't work. Like games or embedded systems and stuff. But for building web applications or statistical platforms, that's my workflow because if you don't get the data right, you're fucked.

I think the data-first approach sometimes leads to a problem of: "We saw we could, but never asked if we should."

The particular scenario I have in mind relates to a company with an internal system for managing work from proposal to invoice. There was a lot of data there, but it was never exactly clear what parts are used by what processes, what rules are invariant, and aspects of it existed purely for historical reasons. Sometimes you could implement a feature for Important Manager #1 only to discover that Important Manager #2 doesn't want it to apply to his employees. Then you'd put in special-case-code, and the dance continued.

In a situation like that, I'd far rather focus first on what the goal and process should be, and then use that occasion to refactor the stored data to accurately represent the evolving model for "how we do business".

I have no words for how broken this is.

You don't refactor data. Data is. Refactor your code.

Data isn't a model for "how you do business."

Data should be a model for what is. How you do business is an interpretation of the underlying data.

I deeply disagree with your approach, but I hope to do so with respect.

I think you may be overstimating the quality of data I'm griping about. Some of the tables are mostly-unused metadata about optional key-value pairs stored in other tables. Some of the key-value pairs are queried constantly but still haven't become "real columns" because nobody wants to rock the boat.

So instead there are dozens of extra joins and queries going on, and looking at CREATE TABLE definitions helps you find only about 60% of the data-points you might be interested in. In some places entities are related not by a link-table or foreign-key, but by having a similar prefix in a text-value. (So WHERE clauses contain substring checks.)

I believe one of the many contributing causes is that people tried to store their data ASAP before they knew exactly how they wanted to use it, and then the next time they assure themselves the data is technically available without stopping to consider whether it's available in the right way.

I agree. The first question I ask is "What is the data?" and the second, just as important question, I ask is "How is it used?"

This is obviously not true. Many (if not most) projects in the Node.js community for example, are made up of hundreds of thousands of lines of dynamically-typed code (mostly npm modules written by other people) and these projects are usually very easy to manage (many Node.js developers wont even realize that they have so much code running behind the scenes because third-party Node.js modules are taken for granted). The secret is breaking everything up into well-defined modules.

I suppose the same can be said about Python and other dynamically typed languages.

Sure, and windows adds ~40 million lines of code to most .Net projects. For comparison just the Linux kernel is ~12M lines of code.

It's not that dynamic languages force bad designs, it's that they promote bad designs. Breaking a project into lots of tiny chunks is one way to manage complexity, but it also adds a lot of overhead as components can't assume much about the other side of there API. A good tradeoff for an OS, not that great for a FPS.

That said, there are times where dynamic languages allow you to get by with far less code.

> I'd rather use a static language like Go which still keeps much of the dynamic feel

If I'm going to use a language for the purpose of having a type system that's supposed to keep me out of trouble, it's not going to be something like Go, in which I'm going to spend time fighting (or end-running around) the type system where it's not expressive/flexible enough, but I'm not going to get much in the way of useful help except on the most basic level.

There are diminishing returns with the type system. Having a super expressive/complex type system like Haskell can be good, but it also adds a lot to the overhead in keeping the type system happy. It can distract you from actually solving the problem at hand. I think Go strikes a happy balance between the two extremes of dynamic and super-powerful type systems.

I may be biased, but seriously? Go is like on the very very super-primitive end of that scale. Only something like C is more primitive. Java has a more powerful type system! (Standard) Haskell isn't even near the most powerful extreme.

Go is nowhere near the middle on a scale between dynamic and super-powerful type systems.

I personally almost feel that using Go and super-powerful type system in the same sentence is disingenuous.

Even the person that gave the talk would have agreed with you. That was his gut feeling as well. Then he did and look at actual research. The whole point of the talk is that the actual evidence doesn't back up your or his gut feelings.

This is why I find the gradual typing paradigm so interesting! You begin by writing your program in an untyped language, allowing for easier iteration and more exploratory programming. As you codebase grows, you transition modules over to static typing.

A particularly interesting approach, I think, is Sam Tobin-Hochstadt's typed racket. https://docs.racket-lang.org/ts-guide/

Agreed 100%. This is actually the reason that I have been picking up go lately, you get some of the more lenient syntax of python, with static typing guarantees.

Syntactically, Go is less lenient than Java, C#, or most other freeform languages, due to, for example, automatic semicolon insertion.

Maybe lenient isnt the best description. It is much more concise without losing static typing.

Type inference like this: myvar := myFunc() makes coding feel like its a dynamic language

I'm surprised that people only seem to be considering Go as an upgrade to static typing, from languages like Python or Ruby.

Kotlin has a very lightweight syntax with lots of type inference as well, but also has many features that people often seem to miss in Go. It also has transparent access to all Java libraries, which is a huge number, which is a big benefit. It seems like a natural path for people who liked dynamic languages. It's much less well known, of course.

https://kotlinlang.org/

Well there are a few reasons that I decided to pick up go rather than something like kotlin.

1. No jvm needed. This may not be a benefit if you need to access java libraries, i just like that you can compile a binary. 2. Great concurrency support 3. Seems to be more popular. This may sound like a poor reason to choose a language, but if its more popular there would be better documentation, libraries, ide support, etc

That's usually a good heuristic. But Kotlin has better IDEs, tools, debuggers, profilers etc than Go. Partly because it's made by a developer tools company (they make many of the top IDEs), and partly because it can leverage all the tools made for Java.

WRT not needing a JVM, I don't perceive much advantage to avoiding one unless you need small downloads i.e. desktop apps. But Go isn't suitable for that use case really anyway.

I've found that if you have strict static explicit extensible typing and a lot of patience, you can get very nice results. It takes a long time to get the ball rolling if you're writing C++ with a lot of enums and structs (and no pointers or templates, except those the STL uses internally); but once you've built your infrastructure, everything else becomes much easier.

Yes, types are documentation; needed in large projects.

I think the pro-type inference crowd really misses this point.

Types have two jobs: improve performance and force documentation of expectations. Of the two jobs, the latter is more important because you can hack the performance issue with JIT and whatnot.

Extreme type inference risks losing the whole point of static typing. If you make a function, it should have its types documented, period. Either the type expectations a function has are trivial, in which case you may as well just write them out, or they are complicated, in which case you absolutely need to document them. Type inference is great for function invocation and okay as typeclass/generic system, but in a large project, you should never create a function or method without specifying the types you expect to handle.

Reply on news.ycombinator.com