Hacker News Reader

Comment by dsign | Hacker News Reader

Duly noted the desire of Hack to have more "typing". It seems that as soon as projects growth beyond a certain size, everybody starts wanting to add that.

I wonder what would be the optimal lifecycle for long projects. Obviously, "start with dynamic language X until you reach 10^n lines of code, then fork X and add types to it" doesn't work for smaller companies :-(

View on HN

I feel like dynamically types programming languages had a huge hype period, and then all the startups we saw write in their dynamic languages later ended up adding types, because maintainability is simply SO much easier when you have a good type system and the compiler can help you. I say that coming from a road starting at PHP, then moving to Python, JavaScript, Java, Haskell.

Funnily enough, strongly statically typed languages are not bad at all for the early prototyping stage. In fact, I would argue that they are much better, but you have to switch your approach from slapping some functions out, to instead slapping out some ADTs, that form the basis of your app.

A lot of good advice for Haskell is given in this thread https://www.reddit.com/r/haskell/comments/7sr8k7/fast_protot....

One important thing to note though, is that I honestly don’t think this will work well, unless the language has at least type inference and support for ADTs, newtypes (or, tagged types) and pattern matching at the minimum. Less than that, and you end up fighting the type system, instead of letting it be your help and guide.

> and then all the startups we saw write in their dynamic languages later ended up adding types, because maintainability

Maybe because they weren't startups anymore?

Honestly I don't understand the deal with type errors in simple web apps : every input is a string which is parsed into the correct type. After this, with good naming and conventions operations should be obvious, e.g. why would you try to do a mathematical operation between a customer's name and it's account balance?

To me it looks like these days the hype is copying whatever big names are doing regarding maintainability without having nearly the same constraints (static typing, huge frameworks, ...).

> why would you try to do a mathematical operation between a customer's name and it's account balance

that's a straw man to js typing, and never a problem in practice. What's more often happening, is that you change the structure of some object in your store somewhere, that's passed around a lot. And you forget to update the usage one of those places and stuff break runtime.

My experience in going from being very insistent on static typing and moving to Ruby (after lots of hesitation) is that you run into lots of type errors, chances are you also have uncaught logic errors in code your test suite isn't exercising. E.g. the above example would presumable be caught if you actually tested all the code acting on the account balance, even without a single test to explicitly test the types.

That goes irrespective of if those type errors are caught at compile time or runtime - if those errors aren't caught by your test suite, odds are you have a problem that static typing will not fix but writing more tests will. In either case my response to type errors these days is to consider it a bug in the test coverage.

(I'm sure some will argue that typing in languages like Haskell avoids this more than others, and that might be true, but languages like Haskell are too hard to work with for most people to ever get mainstream traction; I'd love to see more experimentation in making more advanced type systems more accessible, though)

It is true that all type errors can also be caught by additional tests. But is this the most efficient strategy? I would think that letting the compiler find certain classes of errors should be cheaper than writing and maintaining tests for these error classes.

Indeed, as these are error classes unit tests aren't even very good at.

It can't be repeated enough: Types are universal quantifiers, tests are existential quantifiers, you'll realistically never get even close to the guarantees of a type system with tests alone.

If your unit tests are on a level with your static type system, you have basically rebuilt the static type system in your unit tests.

(And pitting unit tests against a static type system is a false dichotomy, it's good to have both.)

My point was that you rarely need to write test to catch type errors. You write tests to determine if a unit of code works as expected. In doing so you tend to exercise the conditions that cause type errors as a side effect.

Let's say foo() internally calls bar(). If I pass a given valid input to foo() it passes the wrong type to bar(). If I pass another valid input to foo() it passes the wrong value but of the right type to bar().

If you ensure full test coverage of foo() you will catch both. Type checking in your compiler will only cover the first case, so you still need the same test coverage anyway to have confidence in the code.

Full test coverage of foo() is hard to prove. Have you exercised every argument possibility, every exception that could be thrown and every global/class/instance var reference? Working with Ruby most of my career, my experience is probably not.

A good type system is much more thourough and let's your tests focus on behavior.

That's what you have coverage checkers for.

And you don't need full coverage. Aiming for full coverage is a folly - you aim to cover the API surfaces you're actually using. If you still get lots of runtime errors something is very wrong with how you test your code.

> A good type system is much more thourough and let's your tests focus on behavior.

The problem is no languages I'd be willing to use have a good type system that actually covers much.

> The problem is no languages I'd be willing to use have a good type system that actually covers much.

This is basically down to opinion, and I can't argue with it. I've collaborated on Ruby projects for about 10 years, I love the language, but my experience says a large code base, with test coverage, regularly breaks on things that most type systems wouldn't allow.

Dependencies often cause subtle breaks like this, even in big, common ones.

On top of that, this requires paranoia to avoid breaking callers. With a good type system, you can largely discard that paranoia.

"What's more often happening, is that you change the structure of some object in your store somewhere, that's passed around a lot. And you forget to update the usage one of those places and stuff break runtime."

This is the big reason I tend to even prototype a system with a statically typed language, if it's going to take more than about a week. Dynamically typed languages have the characteristic where they seize up after a week or so of development, it becomes impossible to fit the full graph of who is calling who in your head anymore, and from that point on, you tend very strongly to do defensive code changes rather than good ones, e.g. "this function used to take a string, but now it takes a dict with three parameters, but I'll still keep the code for if it gets passed a string because maybe something is still doing that". Then when the time finally comes to do a big refactor, you're often darned near rewriting from scratch.

I've lost track of the number of times I've refactored something successfully in a static code base of several months age by just continually running the compiler until it's done complaining, and how often I start a refactoring that seems to be going really well until the compiler points me at some bit of code and I realize that what I'm doing right now is actually fundamentally flawed in some way I'd forgotten about. In a dynamic language you tend to just power through those and put in sub-optimal solutions.

Plus, at least in my experience, if you do this consistently, it often is practical, feasible, and correct to just "ship the prototype", because whereas with dynamic code I would have a messy prototype with half the methods taking an unclear combination of values, with the static type-based process I've written something quite production-ready, with cleanly-defined interfaces.

> After this, with good naming and conventions operations should be obvious

Why on earth rely on people for this when you have a machine available for you, that can automate all this? A good type system is invaluable for both on-boarding of new people, refactoring code, and maintaining a codebase.

Why would you not want to make sure you could never call the wrong REST endpoint[0] for example? Etc, etc...

[0] https://haskell-servant.github.io

> e.g. why would you try to do a mathematical operation between a customer's name and it's account balance?

Types like `int`, `float` and `string` are rather impoverished. Much more common mistakes are:

- One the numerical side, trying to perform arithmetic with different units of measure ( https://en.wikipedia.org/wiki/Mars_Climate_Orbiter ) or with different representation sizes ( https://en.wikipedia.org/wiki/Cluster_(spacecraft)#Launch_fa... ). These sorts of errors are trivial for machines to identify, if we let them (e.g. https://docs.microsoft.com/en-us/dotnet/fsharp/language-refe... )

- On the textual side, type errors are widespread and cause a large proportion of security vulnerabilities ( https://www.owasp.org/index.php/Top_10_2007-Injection_Flaws ). Again, they're trivial for machines to identify, if we let them (e.g. http://blog.moertel.com/posts/2006-10-18-a-type-based-soluti... )

"One important thing to note though, is that I honestly don’t think this will work well, unless the language has at least type inference and support for ADTs"

I think this is spot on. That was my journey as well. I started using Python in around 2000 because I didn't had to write type of every local variable and declare every little structure. (For example, you want to return multiple values in a tuple? Better declare it as structure or object.) For a long time, Python was my favorite language.

Eventually, I came to Haskell and it became a new favorite. I think the "hype" of the dynamic languages subsided largely because of the two important developments you mentioned (which are now in some form a standard in any new language that is being developed).

I still use -defer-typed-holes or even -fdefer-type-errors when writing haskell fairly frequently, though.

i personally started to find dynamically/optionally typed languages troublesome to read -- i have to do type deduction in my head, which just feel burdensome.

as simple and easy as python feels without static typing, a handful of files of python is my tolerance level before i start to hate it

Yep. A lack of typing puts an upper bound on the complexity of the code that effectively limits the line-count and relegates the language to 'glue' status. The reason is simple: typing functions as guard rails and guard rails are what keep you safe on the road when you are driving for 1000's of km. They have no function if you just putter about town at low speed.

High line count projects almost always gravitate towards languages with such safeguards built in because otherwise the accumulation of subtle errors leads to too many runtime issues, and typing takes care of whole classes of bugs.

IMO any mechanism that shifts that time of bug catching to development rather than operations should be preferred, ditto for any mechanism that forces a program to end when an undefined situation is detected, the closer to the root cause the better.

>A lack of typing puts an upper bound on the complexity of the code that effectively limits the line-count and relegates the language to 'glue' status.

I disagree. I work(ed) at two of the more famous Python shops: Yelp and Reddit. These companies have many millions lines of Python code combined. Also FB was mostly a PHP shop until they started Hack in 2014, 2 years after they IPO'ed.

Testing and monitoring can be used to combat type errors. Runtime type errors still happen occasionally, but honestly these are some of the easiest bugs to squash.

I think you might be conflating static typing with strong typing when you infer that Python has "lack of typing." Python has a dynamic, but strong type system.

I've coded professionally in many other languages (C, Java, Rust). I think C and Java have the worst ROI when it comes to type systems. In my opinion if you're going to use a type system on a new project today, then use something that provides stronger guarantees like Rust or TypeScript (and lowers the burden by inferring or deducting many types).

Python isn't perfect, but it's still the first tool I reach for anything web, ops or data science related.

> Testing and monitoring can be used to combat type errors.

I agree, though IMO this is another upper-bound. You can only test and monitor so much before resources and performance start becoming a bigger issue.

I think companies can definitely go without types if they have solid documentation and a good review system to both prevent bugs and help with onboarding.

What is the upside of having no types, though? We're out of C land and modern advanced typed languages offer generics, type inference, interfaces and inheritance. You can pretty much do whatever you need without ugly casting or compromising your data structures.

Dynamic typing solves things you have in C, not so much in C#.

In my experience: Far less up-front specification, far fewer names/concepts, fewer lines of code, far fewer of those "oh, this should work, but how do I wrestle the type system into understanding it?".

Now, there's a continuum here.

When designing the main data store for a project, you want to iterate on that and get it just right.

Or if you're working on a library to be used by lots of people, it's definitely nice to be able to specify things that lets the compiler help people get their code right.

On the other hand, in almost all the applications I've worked on, there's also a much larger fraction of absolutely necessary and often complex code where each little module is working on some details. There the data structure doesn't really matter so what JSON provides (untyped arrays + simple objects/dictionaries) results in much less cognitive overhead than a fully specified and typed thing. A few black-box tests both documents and checks functionality, and also ensures that the thing keeps working after a refactor.

Of course, YMMV.

Surely the most common argument was that dynamically typed languages need less lines of code to do the same thing, combine that with the time it takes to write code is roughly proportional to the size in lines, and the size of the unit tests similarly proportional, and it becomes a pretty hard argument to refute.

For a long while you were comparing languages line Python to something like Java, and it was pretty much a no brainer - the argument was right. But nowadays if you compare something like Javascript to Typescript, it's not near so clear. Type inference is really biting into the size advantage line dynamic typing had over static typing.

The other thing is the claim 1 line of statically typed code of takes as long to write as 1 line of dynamically typed code. It's true as stated, but change it to "correct line of code" and it's not true. You don't have to write tests for type declarations, and statically typed code has less bugs after it's accepted by the compiler / interpreter, so while the writing takes the same amount of time the testing and fixing takes less for statically typed languages.

The final advantage of dynamically typed languages is statically typed languages just need more head space - just ask someone who has used Rust. Granted Rust with it's explosion of types caused by lifetimes is a bit of an extreme case, but it's still true for most languages. But it's not so true for the Typescript vs Javascript comparison, and I suspect really careful standard library design might make this mostly go away too.

I wouldn't think that you run into type errors often with a mature code base. But that's because you have to deal with this problem sooner or later. In a statically typed language, it is quite obvious what fits where. From my (albeit limited) experience with js, python and ruby, you have to invest a lot of effort into documenting and testing things like function parameters, which would be completely unnecessary in a language with static typing, even if that language has a boring, Java like OO type system.

When you test expected function behaviours, sufficient type testing tends to fall out of that pretty much for free - if your test suite is testing function parameters for the sake of testing function parameters and not for the sake of verifying it gets the right output, then the test suite is deficient.

And bricks will fly if you add enough engine to them. It's all a matter of resources and what is the optimal path. That FB was a success even though it was built in PHP is not a testimony for PHP, it is a testimony for the engineers at Facebook who were able to pull that off despite PHPs obvious shortcomings. The fact that they built their own in-house compiler for the language speaks volumes in this respect.

Yelp and Reddit also have above average resources to throw at the problems they try to solve.

Python is the first tool of choice for quick solutions to immediate problems for me as well, but that does not make me blind to its obvious shortcomings.

You’re completely discounting that PHP has qualities that may have helped early Facebook grow. In fact one the engineers did a talk about exactly that.

The language that you start in may not be the most appropriate tool in the longer term. Facebook grew so fast they did not have enough air to switch, hence their incredible investment in making PHP better.

Look, I've built I don't know how many websites in PHP, and my company still earns a lot of money with a tool I threw together in a day or two in it so I have nothing but admiration for what it can do. But you'll be the last person to hear me argue that there aren't better tools for the majority of the jobs at hand. We are all looking forward to the day that it can be retired, but in the meantime we will definitely keep using it and we will continue to pray to the PHP gods. It is a very productive little language that has a ton of warts and some serious issues, but it does have a niche and it is useful.

That's a fair point, I'm sure there are ways to build things that have the benefits of PHP today that may not have been as abundant back when Facebook was starting out.

Yes, in my experience on coding PHP & Java is that everything is so cumbersome in Java with little gain coding wise (there are of course other benefits with Java)

Hard to move code around, comment out code, prototyping, create samples of data structures etc.

What is important on a large project is enforcing contracts, if it is on function calls, HTTP calls, database calls etc.

With function calls you have type hinting in many dynamic languages, with that you can catch most type errors & avoid writing some tests.

How the types work within the function body is quite irrelevant.

And statically typed languages have recognized this by adding var, auto & what not.

> Hard to move code around, comment out code, prototyping, create samples of data structures etc.

With some IDE skill, this is easy in Java, but then developers get used to this way of developing caused in no small part because of the language and think that features of the language (like static types) enable and entail the IDE features (auto-refactoring, intellisense, code generation), rather than the IDE features being developed partly in response to the limitations of the language.

Dynamic languages often have a tooling problem, at least from the perspective of Java developers, but often you don't even need those heavy IDE features to get equivalent or better work done and so tooling suffers or isn't evangelized much. A coworker complained he can't just find callers of a JS function (unless within the same JS file) with eclipse; I pointed out that vim trivially does this after generating ctags and demoed it... Still, in lots of code bases that aren't in Java, I find myself needing to know "who calls this" not as often. Besides, old school find | xargs grep or ag work to find that info usually too. Even better, if it's a Web Service name, a string-based tool will find the usages on both the client in JS and on the server in not-JS.

Meanwhile there have been dynamic languages like Common Lisp that have IDE features like debugging built-in to the standard language itself, there are strong types that implementations can and do use to produce optimized machine code (not just correctness -- and you can verify an optimization with 'DISASSEMBLE, another standard function), modern implementations like SBCL have at-compile-time warnings for type issues or typos or whatever, and ancient tooling (slime) has things to tell you who-called-what, etc.

I work at Pivotal, arguably one of the most test-obsessed companies in the world.

Tests and types are not direct substitutes. My view these days is that types become more valuable as a codebase grows. I have worked on Ruby on Rails codebases where, quite frankly, I found myself wishing it out loud was Java on Spring Boot. (Many of my peers in Labs are flatly enthusiastic about Kotlin on Spring Boot, FWIW)

In the same genus as the "just use tests" argument is "business logic only belongs in the app, not the database". Yeah, well, it turns out that no amount of testing is a substitute for constraints and transactions.

> A lack of typing puts an upper bound on the complexity of the code that effectively limits the line-count and relegates the language to 'glue' status.

That's an opinion, not a fact.

> IMO any mechanism that shifts that time of bug catching to development rather than operations should be preferred

That's probably true of an existing product, but that's not the only concern in product development.

Likewise. I come from a mostly C background (embedded), but now Python is especially popular in my close circles. I liked it at first, but over the past five years I grew to despise it for anything more than a single file, 300 loc program.

In stark contrast, I’ve been playing around with Unity/C#/Visual Studio and my productivity with this tool stack is through the roof. There, it is rare that I create code which doesn’t fill its intended purpose the first time around. It’s all so easy, and InteliSense makes the quite verbose typing not felt. The tooling and safety is magnificent.

Types are documentation that the compiler (or interpreter) forces you to write and keep consistent, and that your editor can easily understand (:

The optimal lifecycle is to start with the same language you'll finish with. You can rewrite and refactor as much as you want, but you never need to rewrite the entire thing from scratch in one shot and get it right.

The funny thing is, with microservices it shouldn't really matter what language each service is written in as long as the services can talk to each other, but most companies end up using the same language for everything because it's easier to hire people who know the same language and can work on multiple projects.

Also because if you use the same language it's easier to re-use code and interface definitions.

Yeah, going down this path usually leads to a tightly coupled, monolithic-in-practice system of "microservices" which are effectively developed and operated like the previous, monolithic system was, but with added overhead (both runtime and dev-time) on top for a layer that fakes loose coupling, with the sole actual reason for existence of this layer being that it allows people to call this abomination of an architecture "microservice-based".

The software architecture equivalent of Security Theater... Decoupling Theater

We started with two languages on our microservice backend. It is definitely an up-front investment. But it pays off in several ways:

- our service infrastructure is really language-agnostic.

- our design discussions are on an abstraction level above the language level. (of course there are language-specific implementation strategy discussions, too.)

- while it is a bit harder to find suitable developers, I think we attract a very valuable subset: people who are not tied to their favourite language, and don't shy away from learning new things.

But PHP today is a typed language. One of the new features of PHP 7.1 are Nullable Return Types http://php.net/manual/en/migration71.new-features.php

How does it compare to TypeScript?

I met a few PHP devs who were hating on JS, because PHP was mature now and had static typing, etc.

A similar strategy on a smaller scale could be gradual typing, for example Typescript. For me personally, clojure + spec seems very inviting.

Yeah, I'm really excited to see people exploring gradual typing. Like dsign, I've noticed that I really like flexible languages like Ruby and Python for early work, but once something is big enough, I start to miss the things heavier languages give. I'm sure it will take us a decade to work it out, but it'll be fun getting there.

Indeed, this seems to be the best way for a small code base that gradually grows in scope and criticality. It allows rapid development early on and better stability later in the project.

I'm really glad that Python is making good progress on this front.

I think this might be a historical accident. What typed language could ppl use? Java, C++ mostly? They’re (for me) both a horrible mess. I think ppl will start using these “new” typed languages from get go: TypeScript (or Flow+JS), Hack, there’s likely more space for these. A high level, both functional and imperative, expressively-typed with type-inference language is going to dominate mid-term. Or still put differently, everyone’s learning from Haskell while not using Haskell.

Type systems have improved greatly in the last decades. There are now many statically typed languages that have type inferrence, generics, and other goodies. Likewise many dynamically typed languages have added support for type annotations (typescript, python, and now apparently php). The resulting code is more or less comparable in terms of verbosity and readability. IMHO the minor overhead of documenting your types is well worth the improved readability and resilience against whole categories of bugs that should not be allowed to exist in any software system. There are very few good reasons to not want that.

The trend seems in any case for people to have tools in their pipeline that transpile and verify code. E.g. most javascript code gets shipped in minified form. If you are doing that, you might as well run a transpiler, some sanity checks like linters, code analyzers and type checkers. Many people do exactly that. Those type of tools benefit a lot from having more typing information available. Hence the popularity of things like typescript and recent additions to ecmascript of things like classes.

Once you have that, the step to a full fledged statically typed languages is the logical next step. It seems the hack people have reached that point.

I think this goes for a lot of problems in society, not just programming. We fix this and that until it's too complex to overview, then we virtualize or put a black box over it. When that doesn't work any more, we rewrite from scratch, discovering the same bugs and problems we had the first time around.

Recent xkcd visualization: https://xkcd.com/2044/

But the open source code smaller companies produce is working for the 10^n fork crowd I assure you. Choose the best tool for the job and build for throwawayability.

Reply on news.ycombinator.com