Stripe is building a Ruby typechecker | Hacker News Reader

Stripe is building a Ruby typechecker

medium.com

"Why don't they just use (insert static-typed system here) instead?" someone will inevitably ask.

"Because converting a large, complex set of living codebases from one language to another is a non-trivial task that requires boiling any number of oceans. This approach doesn't have that problem." comes the wiser, if less interesting, response.

This seems like boiling oceans as well. It's hard to guess which oceans are bigger or harder to boil. Clearly Stripe thinks this is the easier way to go. That wouldn't have been my instinct. Maybe they made a really good estimate for how it will be though. I'd be very curious to see their full analysis!

You're absolutely right! Building probably was a lot like boiling an ocean. Grafting static typing onto a very dynamic language is a massive and complex undertaking.

That said, it's one item that can be built greenfield. In some senses, that makes it easier to apply to other codebases once the tool is in a working state. It also means one thing to work on, rather than N, which is much easier to plan for.

You're completely right that this is an ocean to boil, and it's rarely easy to guess the size of any given one. Yet it's possible that the naive intuition - when boiling oceans, prefer fewer - might be reliable in this case.

Grafting a complete and safe static typing system onto a very dynamic language is a massive and complex one.

But writing an advisory type checker (that just warns) with a specific code base as a starting point is a very different thing. Biggest question to me is if they support duck-typing. Secondly, if I was to do type checking for Ruby I'd start with parsing yard-style annotations, as a lot of libraries etc. come with that as part of generating their documentation (and if not, writing them means you're now writing comments a lot of tools can generate documentation from.

You're right. An advisory system isn't incredible difficult or intimidatingly large and complex, and doing it for a small set of codebases is much easier than solving it as a general problem.

But uh, I didn't want to tell the commenter that they weren't thinking it through and were being utterly betrayed by their instincts. It's hard to get people to listen to that kind of thing. So I opted for telling them that they're right, implying that they have the opportunity to become more right, and reminding them of how right they are.

It's only massive if you are doing it wrong. All examples I found so far were doing it wrong. For cperl (perl5 with classes and types) I did it right instead. It took about half a year, with most parts improving the rest of the system and signatures. Ruby already has signatures, so thats out if the way. Python eg did it completely wrong, PHP is doing it right.

The goal is catch type violations with optionally declared types and constant literals, and help the compiler and optimizer to specialize ops on types, and do array and loop bounds checks, essentially moving runtime checks to compile-time checks. A good type checker should make overall code faster and safer, what you see out there are rudimentary and not scalable efforts to make it safer, but not faster.

It certainly looks like ocean boiling if you look at all the horribly bad efforts out there, comparable to pypy, unladen swallow, perl6, ... But if you look at proper type checkers and optimizers it's trivial, really.

This might be as hard, but it's not as risky as rewriting the whole thing.

This approach, at worst, will just fail.

Rewriting the whole thing can have real big consequences if not done carefully and takes a real long time to bear fruit.

I'm one of those people. I'm not sure I'd keep loving Ruby as much as I do now if the majority of Ruby developers end up using the type checker and I'll also have to. I like Ruby in part because it's strongly typed but I don't have to lose time writing types. That was very welcome coming from C and Java in 2006. I prefer to lose time when I pass a type that I shouldn't have passed. It doesn't happen often (not every year) vs at every single method definition.

To understand if this project could have a future outside Stripe, how popular is Python 3.6's optional static type declaration?

> I like Ruby in part because it's strongly typed but I don't have to lose time writing types.

That’s why type inference exists. Static typing doesn’t imply annotating everything manually with types. Granted, most static type systems require (or at least encourage) this but it’s not necessary everywhere (or indeed for most usage). Annotating public interfaces with types is, I’d argue strongly, already best practice everywhere (in languages that don’t support it in the language, it’s done in documentation).

> To understand if this project could have a future outside Stripe, how popular is Python 3.6's optional static type declaration?

Or rather: How popular are TypeScript or Flow? Very.

Ruby has the advantage of 'duck typing', due to its strong OOP foundations. Not quite sure what Python has that reduces the demand for static type declaration.

I think part of the popularity of TypeScript could be down to a large proportion of javascript coders that have a background/preference for static typing. It could also be that certain classes of bugs are more prone to happen in JavaScript or it's harder to debug when bugs do happen?

Ruby has been my first professional language and I've sticked with it for a decade, but I never really understood duck typing before switching to Go, with its interfaces (a type implements an interface if it implements its methods, so we can use interface as function parameter type).

I don't get why we talk of duck typing about ruby : if I pass as parameter an object that does not quack like a duck, nothing will prevent me to do it and it will generate an exception when quacking. Did I miss something obvious?

The obvious part is that Ruby does not have a distinction between "compile time" and "runtime". As such, if you want to catch those cases, you need tests.

Duck-typing can be boiled down to: The type of a parameter to a function is defined by the methods that will be called on it, not by its class.

E.g. if you have:

    def foo source
      dosomething(source.shift)
    end

Then absent additional restrictions inferred from requirements of dosomething(), the class of "source" is irrelevant. The relevant type information is whether or not "source" responds to "shift".

A proper Ruby-ish type checker will need to be able to handle that, or it'll push people to write non-idiomatic Ruby.

I've digged a bit on the subject (better late than never), it seems that duck typing is seen in ruby as a guideline, rather than a language feature. Basically, it's about recommending to use `#respond_to?` rather than `#is_a?`, `#kind_of?` or `Class#===` in end user code.

It's a language feature in as much as the absence of static typing and the ability to check for methods is what makes it possible.

But yes, it it a guideline, and Ruby code that violates it will generally be seen as not being idiomatic Ruby.

Oh, I see, given the interpreter is written in strongly typed language, allowing to go without static typing is a feature by itself, and allowing to test for methods presence allows to do duck typing, thus it allows to say that ruby implements duck typing.

It makes more sense than talking of duck typing explicit usage by end users, given it's quite rare we assert proper received parameters extensively.

EDIT : which makes me realize we have a machine language without typing (binary), on top of which we use a language implementing strong typing (C), used to build a language without strong typing (ruby), on top of which Stripe is building strong typing. Making a definitive decision is hard :)

Nitpick: C is weakly typed, Ruby is strong.

C is statically typed, Ruby is dynamic.

Strong typing is about objects having a definite type. In Ruby, you cannot cast a String to a Fixnum, an exception will be raised. But in C you may cast anything to anything really, and your program will believe you, whether it could possibly make sense or not...

I thought "strong typing" and "static typing" were interchangeable. I stand corrected, thanks.

I wonder if this would be a good opportunity to use a typing system that's more like Go's, where you don't need to explicitly declare that a class implements an interface, but instead any class that has the right methods will implement the right interfaces automatically.

(In your example the method could be:

    def foo(source: Shiftable<?>)
      dosomething(source.shift)
    end

where `Shiftable` is an interface that's something like

    iface Shiftable<T>
      .shift(): T
    end
)

> A proper Ruby-ish type checker will need to be able to handle that, or it'll push people to write non-idiomatic Ruby.

It probably won't be idiomatic (maybe just add interfaces?) but I think duck-typing is overrated anyway. Just because an object responds to a method doesn't mean it's going to do anything like what you expect. To take a contrived example, Array.Shift and Keyboard.Shift are probably unrelated methods.

You don't have that guarantee with interfaces either - you're trusting that the developer isn't lying to you. My experience is that I've more often run into developers that have artificially barred me from doing what I wanted by checking for a given class, than that I've accidentally passed in something that satisfies a given type but does something different.

> You don't have that guarantee with interfaces either

True, but you can give the interfaces/contracts proper names (or namespaces). To use the example in the parent comment, `Array` and `Keyboard` wouldn’t implement the same contract even if both have a `Shift` method.

I guess now that I think about it Scala does have a strongly-typed duck-typing thing, but it's kind of silly and rarely used.

JavaScript and Python also have "duck typing."

I'm not a typing person, similar to pmontra. Any programming language where print("Hello world") is longer than print("Hello world") bothers me.

And most of my JavaScript is untyped, because (and apparently this is common, someone did some stats on it a while ago) typing errors aren't a great source of errors.

However I love optional typing. Being able to build a thing quickly, and then add those guarantees if/when you feel they're needed is great. My current codebase is about 80 modules. Nearly all are untyped JS. The database code is TypeScript because that's where I feel type handling would add the greatest benefit - not that I have had typing errors, just that I like the additional peace of mind.

> typing errors aren't a great source of errors.

That’s not true. On the contrary, typing accounts for a substantial fraction of all errors (and refactoring could transform even more logic errors into catchable type errors). The best research (best methodology, sample size, by far) on the subject [1] puts the lower bound for the fraction of type errors in JavaScript at 15%.

[1] https://paperpile.com/app/p/3070170e-933e-0956-82ec-879fb924...

Not doing a URL hunt as I'm at work, but as mentioned in the comment you're replying to (literally after you chopped it off), there was a large study showing the opposite on HN a while back.

> there was a large study showing the opposite on HN a while back.

Yes, sorry: I didn’t mean to ignore that part of your answer but the research I’ve linked to essentially supersedes everything else in that area. Prior studies were riddled with inadequacies in controlling for confounders and outright methodological errors. I’m assuming you’re referring to [1] and while that was an impressive study in its own right, their classification of bug sources is indirect and, ultimately, simply didn’t allow any assertions regarding the prevalence of type errors.

[1] https://paperpile.com/app/p/54ee3fdd-a6ef-0767-add5-55b60419...

No problem! Thanks for clarifying.

You'll have to justify this: "Prior studies were riddled with inadequacies in controlling for confounders and outright methodological errors."

Your recent study is pay walled, so I can only go by the abstract, but it doesn't sound any more rigorous to me.

I also find its comparing very different things. It says bugs in release, but it sounds like they consider commits to be releases, which would be very unfortunate.

Its also specifically targeting JavaScript, which prior study already showed to be one of the poorest performers in terms of defects (i.e., it tends to have more). For example, it would be interesting to know how many bugs were due to weak typing.

I'm also curious, adding the typed annotations should be a blind activity to be rigorous. Was it? Or did they knew there was a bug and what the bug was then they added the type info?

I also feel they're doing a one way comparison, having to add types can introduce other forms of bugs, but since this study is targeted, I find it to suffer a little from confirmation bias. Like type checker detects type errors, is a bit of a no brainer conclusion. I understand they mean this in the sense that 15% of bugs were type errors. But that doesn't mean TypeScript code would have 15% less bugs, because they did it one way only. It could introduce other bugs in the process.

Would have been interesting too to analyse how long it took for the bug fix commit to be made over the introduction of the bug. So you would know the real cost of them. Which is important, since annotating adds dev time, but so does big fixing. Would be nice to have time data.

Finally, the prior study demonstrate the same outcome. That static tends to have lower defect rate then dynamic and functional less then imperative. The difference is in the outliers. Clojure and Ruby were both outliers that outdid almost all static languages.

So actually, it would be great to reapply your study to them. Clojure has an optional type checker already so it could be a good one. It was also the strongest outlier.

> You'll have to justify this: "Prior studies were riddled with inadequacies in controlling for confounders and outright methodological errors."

That topic has filled whole blog posts. But in a nutshell, no previous study was able to compare directly the effect of typing disciplines due to confounders that they couldn’t control for in their experimental setup. For instance, virtually all previous studies compared different programming languages, which obviously differ by more than just their typing discipline. Many studies had very small sample sizes due to using human test subjects to write sample programs. These test subjects were almost exclusively students without real-world experience — or, in some case, any prior experience in the respective programming languages. Thus, many studies — from the outset — tested beginner-friendliness of languages rather than everyday use. Fair enough, but not the same as type checking benefits. Furthermore, most of these studies, for the same reason, were restricted to testing on artificial, academic, small toy programs rather than real-world applications.

The few studies that looked at big real-world data sets (the biggest and most rigorous being linked above) didn’t even look at typing discipline as an individual factor — again because they couldn’t regress it out as a single factor.

> Your recent study is pay walled

The link I posted has the full text PDF (that’s the whole reason for me to share Paperpile links rather than the original or DOI). See also an academic discussion [1] of the manuscript.

> Its also specifically targeting JavaScript

Which is explained in the paper: no other constellation allows to compare the effect of added static type checks as directly as JavaScript/TypeScript/Flow. That’s because (a) these languages only differ in their type checker, and are gradually typed, i.e. types can be added to just a subset of the program, which enabled the study methodology. And (b) there’s an extensive database of real-world code to analyse.

> For example, it would be interesting to know how many bugs were due to weak typing.

Fair enough. I’d expect the effect to be (much) less pronounced in a language with stronger type guarantees. Then again, the 15% number is absolutely an underestimate to begin with (see the paper and the blog post [1]).

> adding the typed annotations should be a blind activity to be rigorous

That’s not at all obvious, and from experience I disagree. Thinking about types is automatic neither in static nor in dynamic languages. Once you’ve figured out the correct type, yes, it’s a blind activity … but that’s a pretty empty statement.

> having to add types can introduce other forms of bugs

Again, this claim is far from obvious, beyond the trivial “if I make this type an `int` even though it should be a `string` then that’s a bug”. Fair enough, but adding the type annotation and performing a type check merely reveals this bug. The bug itself was present in the programmer’s flawed assumptions about the invariants in the code. Your assertion is equivalent to saying “compiling or interpreting the code [rather than writing it on a piece of paper and never touching it] can introduce bugs”. — The bugs are already in the code, we just didn’t detect them.

> Would have been interesting too to analyse how long it took for the bug fix commit

They only considered bugs they could fix within a highly constrained time frame, and only looking at local code around the bug. Hence, again, why the 15% is an extreme underestimate. Furthermore,

> So you would know the real cost of them.

The study calculated the token cost of adding types (and provide a justification for this metric). In a nutshell, the cost is negligible, especially in languages with strong type inference capabilities (Flow outperforms TypeScript here).

> Clojure and Ruby were both outliers that outdid almost all static languages.

Right, and I’d expect the same to still hold. But it shows one of the crucial shortcomings of previous studies: they primarily did not examine the effect of static vs dynamic typing. Rather, they examined the effect of different programming languages, to which typing is just one contributor amongst many. It should come as no surprise to experienced programmers that some languages (regardless of static vs dynamic discipline) vastly outperform others. For me, this is all the more reason to be excited about the headline topic: If done well, statically type checked Ruby could be an amazing language, with the added benefit of static typing at (as shown) virtually no cost.

[1] https://blog.acolyer.org/2017/09/19/to-type-or-not-to-type-q...

> If done well, statically type checked Ruby could be an amazing language, with the added benefit of static typing at (as shown) virtually no cost.

I think this is where I feel the jump in causation to be too high. No one knows why Ruby and Clojure match up to Scala and Haskell. The prior study couldn't isolate the impact of a particular feature, but it did a pretty good job, at least similar to your study in finding overall defect rates.

Ruby with types, if we take your conclusion, should end up even outperforming Haskell by almost 10% in overall defect rates. That just doesn't sound believable to me.

I fear types are not independent variables. They affect other variables. And that's where I argue that they might in fact come at some cost which we still do not understand.

I still strongly feel they should have added types without knowing the bug. There's many ways to type data, those choices do matter. There's code that can't be typed precisely, or requires considerable efforts to type, there's even code that can not be typed at all, and must be rewritten differently to be typed.

Now I'm not going to argue JavaScript doesn't benefit immensely from static type checking. What I argue is that doesn't mean every language will see benefits from static types.

That said, I'm a big fan of gradient typing, especially when it can add runtime contracts over boundaries between typed and untyped. But, I'm not sure how successful they would be in Ruby. They've failed in Clojure, while no real analyses was performed, the community rolled back most adoption of it due to their impression of it not providing any value while adding extra effort. Obviously, the Clojure gradient typing system isn't as well maintained as Flow and TypeScript, so maybe that played a role.

In my experience I lose less time specifying that an argument is a string in Go than I do adding a test to handle the possibility that nil gets passed in Ruby.

amen to that.

i appreciate type checking so much more now than i used to, and i find that i disliked it most when i needed it most (i.e., i blanched at it more when i was a worse programmer, even more likely to make type errors than i am now).

there are categories of checks in application code (as well as whole categories of simple unit tests) that i just wouldn't have to write with a static type checker. additionally, as other comments have pointed out, it'd make it easier to automatically refactor things.

i started coming around on it with groovy a number of years ago (after sorta hating the type checker in java/c/c++ for years before that). i'd find myself writing a bunch of untyped code, but then i'd also find myself going back and specifying types as i wrote tests, because it was often easier to just specify the type than to write a test for the possibility of the wrong type coming in.

as someone who mostly writes ruby code these days, i'd love to see an optional type checker get added.

You switched from Java to Apache Groovy to lose the static type checking, then later stuck types in your Groovy code to make it act like Java. Sounds like Groovy's value-add to the ecosystem was providing the syntax for doing that without writing `Object` everywhere.

> I don't have to lose time writing types

In C#, I write type names when I declare a type, when I create instance of a type, when I declare method argument or return type (which you would probably do even in a dynamic language as a comment - you don't want people to have to go through your code to figure it out themselves, right?) and when I declare a collection of that type. Out of all of these, only the last one seems as "wasting time", because if I declare a collection and instantly use it on the next line, the reader can probably infer what type I meant to use - but in all other cases, not writing types would make the reader's job harder, not easier.

Don't the same principles apply when you're working in a dynamic language?

am i mistaken, or do generics specifically eliminate that last case...?

Generics allow such a case - without generics I wouldn't declare what type collections are because they would be just collections of objects.

It's not just about preventing errors; it's also about making maintenance work much easier.

> "Because converting a large, complex set of living codebases from one language to another is a non-trivial task that requires boiling any number of oceans. This approach doesn't have that problem."

Sometimes, if a baby is really, really ugly, it's OK to throw it out with the bathwater.

You're absolutely right! But just because it's OK doesn't make it pragmatic engineering or good project planning.

> boiling any number of oceans

How many lines of code are we talking about here?

How many services do you think a big company like Stripe has? Let's be exceptionally conservative and assume fifty, with a minimum of a million LoC between them.

All of them need to keep working for the business to keep working.

Ok, but they also have a lot of developers.

They do! And most of them are considered valuable employees because they can produce features the business wants in acceptable timeframes.

It's perhaps possible that the business and management side of the house at Stripe might not want to give that up for a while so all of those developers can re-implement every service. Given that historically something like half of all rewrite projects fail, there's some non-trivial level of risk there.

Though I understand if some think Stripe can afford it.

So it might (or not) be the best option for them at this point, but on the other hand, is it another case that shows a static-typed language is a better option for developing complex systems? IMHO, it is.

I think this shows that static typing becomes more desirable and a better investment as size and complexity of a codebase - and company - grow. This might not be the same as evidence that starting with static typing is preferable.

To your earlier point, larger systems are harder to migrate. This suggests that starting off with an investment in static type systems is more tenable as a long-term strategy.

All other things being equal, that's probably true!

If only all other things were equal...

That's the catch, yes. All other things are very rarely equal. There are almost always other factors worth paying attention to when forming a long-term strategy. A handful that spring to mind are availability of talent, software runtime performance, schedule impact of static vs dynamic typing, and expected runway of the business.

Producing a statically typed system that runs slowly and correctly after the business has folded is not always more valuable than producing a buggy one in time to start producing revenue.

It doesn't come even close to that. There are works that attempt to tackle this issue, if your interested I would look at those

I think making Ruby typed is a cool endeavor. It's just that it's a bit hard to justify from a business perspective..

It's not like changing to typed Ruby wouldn't affect the code bases. They'll end up having something that will resemble Ruby, but likely won't be quite Ruby. It's unlikely they could automate it. They'll have to rewrite a lot of code and typically, when you do that you also want to refactor obvious problems, so you end up rewriting big parts. And that's in addition to the cost of trying to make Ruby typed.

Or you could leave the code base as it is and extract functionality out and rewrite it in something more manageable.

I think it becomes a reasonable business pitch if you look at it as a tool that significantly reduces the defect rate and improves on-time delivery.

i work on pytype [https://github.com/google/pytype], a similar project for python, and it has definitely made business sense. it lets developers code in the language they find most productive (largely due to its dynamic nature), and adds back some of the benefits of static type checking. in particular, it has caught a lot of bugs that would otherwise only have shown up in production, and perhaps not until some rare code path was hit.

Not yet open-sourced, but this is definitely planned.

You can try it out there though: https://sorbet.run

By the way, a number of RubyKaigi 2018 talks were dealing with type checking (and the other were about improving performance & memory usage).

I had the "pleasure" of creating and integrating a parser for a custom DSL into ACE. I've created the parser in ~3 days; it took another 3 to make it work with ACE.

If it was up to me, I'd chose monaco editor (as the guys from mozilla did it with their webassembly editor)

(author of the demo site here)

I just was looking for a quick editor for the demo, and ace seemed to work well for this. Thanks for the pointer to manaco, I'll look at that for next time.

Using an editor in the browser won't be the final product. We're planning on integrating into your editor of choice instead.

Thank you so much for all the interest! We're flattered and excited to see all the discussion about the project.

Try it out: https://sorbet.run

If you would like to get in touch with us about anything, please email us at sorbet@stripe.com.

In the presentation at RubyKaigi (which will be available online soon) we explicitly mentioned we'd like to chat with folks trying to scale Ruby into the millions of lines of code, or folks also working on similar typechecking projects. Of course feel free to email even if you aren't in those groups, but I wanted to get your attention if you were.

One thing that springs to mind: Does it handle duck-typing? In other words, can I specify "any object that responds to methods x and y" as the expected type of a variable?

Because if treats classes as equivalent to types, then to me it encourages non-idiomatic Ruby.

We only have the published examples to go on, but unfortunately they all seem to assume that type means class.

If that’s borne out I could never use this library. It’d be the antithesis of duck typing and object messaging.

So you know when people ask "why does X startup have X,000 developers? what do they do all day for a single app[0]. Here is your answer:

"Technical details: - 9 month of work by 3 people; - real thing, runs over all code of #Stripe"

https://twitter.com/darkdimius/status/1002103748875902978

Now let's look at this from a financial perspective:

So let's say average $175k salary per employee + benefits and you're looking at easily north of $600k to do this.

I'm not saying it's "right" or "wrong", but thats just what companies at this scale do because they create development cultures and subsequent financial controls that allow for indirect production software development to happen.

[0]for the record, Stripe is much more complicated than just a "single app"

Any reason why existing typecheckers for Ruby didn't make the cut? There are projects like RDL [1] which provide similar functionality. I know for certain that people from Stripe were taking a look at it last year from the issues they had raised.

Disclosure: I am a grad student, recently started working on RDL.

[1]: https://github.com/plum-umd/rdl

I'm sure they will come up with some plausible reason like the project having incompatible goals, or embracing a different design architecture than the one they wanted.

Truthfully, these 'prestige' projects are done for reasons other than actually improving the development standards. They probably didn't use RDL because it was more fun to write their own then figure out someone else's thing.

For all we know they may not have even had a real need for a new typechecker. Stripe could have a valued engineer who seemed bored so engineering management gave him or her an intellectualy interesting project to do - even if it wasn't that important to the business.

Sure, that's possible, but we don't really have enough information to say either way.

I think they mentioned they will present more information in an up coming conference. And it makes this TypeChecker possibly the most battle tested solution out of all the competing ones.

It would seem your cynicism was unfounded https://news.ycombinator.com/item?id=17226332

What's the next step for RDL?

It seems like Hummingbird is much more powerful than Sorbet, but it's hard to tell without them publishing a paper yet.

There are a bunch of interesting stuff coming along. I don't know if you have seen the Refinement types in Ruby paper based on RDL recently published in VMCAI [1], this opens the window to verification of Ruby code.

[1]: https://arxiv.org/abs/1711.09281

Thanks I added that to rubybib.org - please encourage your colleagues to create PRs when they publish papers so the whole community can find Ruby research more easily.

Sorbet appears to be a language extension rather than a library, similar to TypeScript or Flow in the JavaScript ecosystem. I’m guessing that one of their goals was ease of use for type annotations. I’ve had a quick look at RDL and while I quite like its syntax it is somewhat more cumbersome than having type annotations “inline”, as it were. Furthermore, the annotations in RDL are themselves stringly typed, which comes with its own issues.

Hi and thank you for working on RDL! We do love the project and spoke with Jeff Foster a few times last year.

We did use your standard library annotations for Sorbet and have many fixes to them sitting in your pull request queue: https://github.com/plum-umd/rdl/pull/68 https://github.com/plum-umd/rdl/pull/72 https://github.com/plum-umd/rdl/pull/57 . We stopped submitting more since these weren't being upstreamed.

Before we started our project, I evaluated rolling out RDL instead of building our own typechecker. Trust me, I would MUCH rather use an existing project than have to build our own, but sadly it just didn't scale to our millions of lines of code.

I'm more than happy to chat about the details of why we didn't use RDL if you'd like to email me at sorbet@stripe.com. Thanks again for your great project, we're standing on the shoulders of giants.

You are embarking onto a very interesting, but tough project due to the extremely dynamic nature of Ruby!

Have you considered the property-based QuickCheck approach? Instead of building a type checker, you just annotate your code with properties (preconditions, postconditions and class invariants). You can generate tests to verify your code and also inject runtime checks to fail early in case of violations.

In my experience, this tends to work much better on dynamic languages and it also scales to larger codebases with heavy usage of dynamic idioms.

Oh, I am not sure how these went unmerged! We should do something about the PRs.

Will shoot you an email to discuss more.

Quick plug for RDL. It's pretty great. There are a couple issues with Rails, but for standard Ruby code and modules it's a much better process to read, write, and maintain code with RDL than without

This looks really nice... i wish it was just part of the language to be able to write ruby with type information e.g. instead of having a separate more verbose sig method that's called before each method signature...

``` class Foo extend T::Helpers

  sig(
      # Both positional and named parameters are referred to by name.
      # You must declare all parameters (and the return value below).
      foobar: Integer,
      widget: String
  )
  .returns(Symbol)
  def foo(foobar, widget: nil)
    (foobar.to_s + widget).to_sym
  end

end ```

I'd love it if we could write this as :

``` class Foo

  extend T::Helpers

  def foo(foobar:Integer, widget:String:nil):Symbol

    (foobar.to_s + widget).to_sym

  end

end

```

Not sure the best way to handle default values... e.g. widget:String:nil is kind of awkward... but still i'd be better this way then the additional method call 'sig'

The second version would be difficult, maybe impossible currently due to keyword arguments, since:

    def foo(foobar:Integer)
    end

is already valid, and I think the lexer might struggle with the ambiguity; but I'm not 100% certain.

Edit: I seemingly missed the last line of your comment where you state something similar.

If foo:SomeType is problematic, let’s just use foo::SomeType instead. I agree with the parent that inline types are much nicer and more intuitive than he sig based approach.

For some reason that reminds me of Pascals interface/implementation split...

For those not in the know Pascal units had to declare the function (procedure) prototypes before implementing them.

(one of the authors here)

We'd love that too! We're chatting with the Ruby folks to see if anything like that is feasible. We experimented with lots of other syntaxes (comments, yarddoc, .rbi files, monkey-patching stdlib classes to be callable) and the `sig` syntax seems to be the easiest for folks to use inside Stripe.

After writing sigs for the past few months, I don't dislike it as much as I thought I would. You get the syntax highlighting and autocomplete of your editor, while `sig` and `T` are short enough to not feel like too much boilerplate.

We're very open to other suggestions if you have any for other syntax suggestions. Just email us at sorbet@stripe.com.

Tomdoc?

I don't know, as a long-time Rubyist, I actually prefer Sorbet's fluent interface with `sig(...).return(...)` to these (albeit more typical) type annotations. Reminds me of RSpec.

I know the editor is not the news, and it's used just for demo, but more info about it: currently it's an ACE editor strapped to a wasm that performs the checking. The integration looks more like a quick poc since it's not using an webworker for performing the checks nor does it use the editor's markers for errors.

Is Crystal still a thing? I thought it sought to keep Ruby ease but statically for speed and to reduce bugs?

Definitely still a thing, but development looks like it's slowed a bit. Maybe the Manas team is overwhelmed? 550 open issues and 130 open PRs. Nice language though, hope things improve!

Yes, I've enjoyed using it and there are several well written frameworks like Kemal, Amber and Lucky for apps.

And it's really useful for CLIs and APIs.

Crystal is progressing well, but they don't have the resources of a lab full of incredible engineers thrown at it.

It's definitely a threat though. The Amber Framework is very familiar to people who know Phoenix or Rails.

Personally, coming from Ruby, Crystal has a slightly steeper learning curve. However, I feel that magic spark deep down now that I've spent more time with it and I'm more aware of what inference can do and how to use it for my benefit.

Little safety things that seem irritating at first, but then you put the pieces together and understand them a bit more.

T | Nil is a culprit here. It seems obnoxious, but then you realize, I can just chain methods in a case statement, I don't have to check for nil over and over, just in the topmost part of whatever code it is. It's not how I normally think, and my instinct was wrong based on my initial understanding, but now I'm much more comfortable with it.

As for tooling, Crystal is rather weak. This too will come with time.

Crystal is a completely different language with a different architecture and object model that only superficially looks like Ruby. Porting code from Ruby to Crystal is somewhat easier than, say, porting the same code to Python, but is still an act of porting. You can't use any kind of automated drop-in process to rewrite Ruby code to Crystal. They're just too different.

A good type system for Ruby is very much needed. Matz himself has stated that a type system is very much on the horizon for Ruby. Can't wait.

With Ruby getting an AST, and potentially a type system in the future, I think that ruby 3 could lend itself to bring transpiled to crystal relatively easily, depending on exactly how much type information there is, how difficult / possible it is to correlate that with the AST.

Scala 3 is doing the typed AST thing, so hell, why not?

Syntax is only the first obstacle. You also have the object model and the standard library to deal with as well.

You're absolutely right! Any Turing-complete language can be transpiled to any other.

That said, you can't always expect sensible or human-brain-friendly results if you're moving between two languages that work very different.

Why must you keep telling people "You're absolutely right!" It seems insincere and condescending at best. If you really think you know better than the person you're responding to, use evidence and logic, not cheap conversational ploys.

I do need to mix up the verbiage more.

In practical terms, people respond much better to having their egos stroked than they do to being told they're wrong. Evidence and logic work much better when someone doesn't feel like their ego is at stake. Telling someone they're right before pointing out that they could be more right is one way to do this. Telling people that they're wrong is more likely to provoke a defensive reaction and a closed mind than it is genuine thoughtfulness and consideration. We've all seen people refuse logic, reason, and evidence because they feel personally attacked, I suspect.

This is a lesson I gleaned from the (in)famous "How To Make Friends And Influence People". It's not wrong, it's just cynical.

So of course it's insincere. Human interaction is greased with little insincerities. It's how we deal with egos.

You're not supposed to acknowledge insincerity. You're supposed to make up another reason that merges the sincere and the insincere viewpoint together.

Yup. But the Dale Carnegie stuff is exhausting.

I'm sincere in that I want people to listen to my points without getting their egos in the way.

The only place this works the way you think it does is in your mind. Any normal person is going to see that and register it as goofy and ingratiating. Like Spock or some alien that thinks it's cracked the human code when really it's nowhere even near the uncanny valley.

I understand why you think that.

This may surprise you, but I really wish my experience agreed with you. I really wish most people saw immediately through insincerity and ego-stroking and empty complements. Indeed, some do.

Most of the time, it's disturbingly effective. My data to date suggests that most people won't look too closely when you feed them kind words to hide the others.

Yes definitely, however -> gets better response and cooperation than -> Well, actually

Just my two cents, not parent btw

> I think that ruby 3 could lend itself to bring transpiled to crystal relatively easily

I think for this to be tractable you'd basically have to implement a complete Ruby interpreter in Crystal and then transpire the Ruby program to bytecode to be run by the Crystal interpreter. So not really a meaningful or useful transpilation any more, and certainly not fast.

Even basic things like method dispatch do not have the same semantics in Crystal as in Ruby, so almost nothing could be tarnspiled 1-to-1.

We don't know very much about what ruby 3 types will look like or what other information we'll have.

Even a small subset of ruby being transpiled would be a useful thing for some developers.

You're much more aware of some of the constraints here than I am, as Oracle doesn't pay me to hack ruby for a living.

With a typed AST, there'd be enough information there to build a foundation for a rb2cr tool. I didn't say the two languages are best friends and it'll be a breeze.

Most of the ruby code I want to rescue isn't littered with string-based define_method nonsense. Most of ruby I think worth saving at all is outside of rails and exists in various tools or maybe some metasploit modules or stuff like that. The pieces of code that utilize the grossest dynamic elements of ruby, like define method or objectspace, that stuff doesn't need to survive.

Even getting 50% of a codebase in ruby compiled to reasonable crystal is a much better foundation than previously purported successors (elixir, scala, swift) can do.

If ruby gets a typed ast, I'll try and write a simple transform tool for the simplest ruby.

> Most of the ruby code I want to rescue isn't littered with string-based define_method nonsense.

I see what you are saying - but this is where the issue is I think. You may not write code that does sophisticated metaprogramming, but the gems you use are probably fundamentally based on it. Even just requiring some of the standard library uses a surprising amount of of metaprogramming. For example you can't load something as basic as 'fileutils' without doing a lot of metaprogramming.

The entire Ruby ecosystem is built on metaprogramming.

That's where the boundary for transpilation will lie then, but I don't consider that to be a good enough reason to justify not trying to write a tool like that.

I really appreciate your thought on this! I look up to your work a lot. Sometimes I wish I had spent my career with compiler a instead of disassemblers...

Yes, there will be a lot of ruby that can never by crystallized. Yes that kinda sucks. So yes it's very good to explore optional or gradual for incremental refactors and modernization. But if even 1% of ruby code CAN be crystallized, then it's worth it to do it. Maybe not for you, but my time isn't nearly as expensive as yours must be, so I understand your reasoning here I think.

I don't mean to be adversarial here.

Part of the reason it might be coming across as a slightly negative reaction is that I'm passionate about implementing Ruby exactly as it is. If people want to write in Crystal that's great it can be fast. But if people want to write in Ruby, doing all sorts of metaprogramming, or that's the code they actually have today and need to run in order to keep their business going, then I want to make that just as fast for them automatically. Without telling them to use a subset, or not to use awkward features. I want people to bring me their insane code and I'll find a way to make it fast for them. That's the challenge I'm enjoying at the moment.

If people are happy to use a subset of Ruby then yes it could be transpiled. But a simple subset of Ruby should work great in the new JIT compilers we're getting anyway, without using Crystal.

I used to be happy with Ruby, but learning Scala and then Crystal really changed my mind about where the line ought to be drawn wrt to expressiveness and safety.

I don't and have never used the vast majority of Ruby's metaprogramming features, but I'm the odd duckling in an equation where probably 95% or more Ruby programmers are Rails programmers, and I am not. I mean, I can, but I don't. That factor makes less sympathetic to the majority of Ruby users who won't have any stake in a project like what I'm thinking about. And that's ok. And either way, if they want to make use of all those features then they will or maybe already have realized that Ruby already does what they want and Ruby3's care for backwards compat surely will cater to that majority in order to keep them on board. Metaprogramming I think is the primary style of Ruby, and that's OK.

The Crystal team has essentially reprogrammed the way I think about how it should feel to write good OO code. Scala did the same thing for a time, but just the sheer pain of SBT drove me away. With Crystal, Having parametric modules is an incredible advantage. I can write mixins essentially like traits that specialize on some new type in my program.

I have to consider how my methods end up typing out, and that changes the way I think about how I'm going to get from here to there. With Ruby, I really like that left-to-right idea of chaining until I arrive at the structure I want, and the more Crystal I write the more I realized how irritating it was to hunt down Ruby bugs by:

whatever.tap{ |o| puts "#{o} is a: #{o.class} here" }

or even deeper with responds_to? or whatever I'm trying to figure out. With crystal, all I have to do is look at the stacktrace, it is getting a nil where it shouldn't be at SomeClass#some_method on line 230842349, and I immediately can work on the bug at the source of it, instead of the tedious extra step of finding it.

Being able to make Ruby do magic was never part of the equation that held me a romantic captive to Ruby, it was always the succinctness. Elixir came close, but Crystal gets me all the way there.

If nothing else, maybe I will learn something new trying to find a subset of ruby3 that I can transpile.

I haven't read much about the new JIT, but if you're happy about it, maybe I should stop procrastinating and find out about it.

The type system is the least of your problems. Ruby's type system is fairly simple and straight-forward.

The problem is that the ruby execution model is fundamentally incompatible with ahead-of-time compilation without making a lot of decisions about what occurs before compilation and what is deferred until later. E.g. a lot of Ruby programs execute code to determine which files to "require" (e.g. iterating over all files in a directory is a common pattern). In some cases that is a convenience for developers. In others it's meant to e.g. provide "plugin" abilities at runtime. In both cases the included code can totally change the behaviour of large parts of your code base.

Those decisions are possible to make, but you must make them and you probably can't make them satisfactorily without providing a mechanism for the developers to specify intended behaviour. If your decision is to defer everything until after compilation, you'll basically end up JIT-compiling most of the code at runtime, and all type information from before that point is suspect.

If larger companies (eg. Stripe) would put more effort into improving Crystal instead of improving Ruby, Crystal would absolutely blow Ruby out of the water. Instead we get improvements to Ruby because larger companies have larger codebases and don't want to refactor everything all at once. They'd rather slowly add type annotations to their Ruby codebase until eventually it's all done. I get it, it's better from a business perspective short term. In the long term, Crystal would be able to run at least hundreds of times faster than Ruby.

There's absolutely no chance that a company like Stripe would spend resources improving a language like Crystal, or even think about porting their current code base to it. If porting was what they wanted to do, Crystal would only be slightly higher than Ancient Greek on their list of potential languages. Not saying I don't like Crystal, but it makes ZERO business sense for Stripe, and they are after all a business, not a programming language improvement charity.

There's so much unanswered here, does it handle parametric polymorphism? Does it allow sum/product types? Is there inference? Can you bound generic types?

IMO it would be a complete waste of time to add a type system that only prevented the most trivial of type errors: mismatching Integer and String.

Interesting choice, considering that Stripe is a massive Scala user.

Is it really cheaper to write type checking for one of the most dynamic languages ever, than port to their other language? Seems to be strong evidence against the "stack doesn't really matter" viewpoint.

It's substantially less risky, less invasive, and easier to do in parallel with other ongoing feature development than a port to another language would be. It's much easier to get incremental value for the investment if you're keeping the same language and codebase than if you're trying to replace everything all at once.

Yes it is. Stripe can do this with three (badass) engineers while the hundreds of other engineers continue to build their applications in Ruby. Stripe's Ruby code base is far larger than their Scala codebase, with each being used for the types of uses that play to their strengths. Rewriting millions of lines of Ruby into another language would more or less involve pausing all application development for one to two years, which simply is not feasible.

This way, a small team of skilled engineers make this typer work, and guide its general adoption, getting the codebase well typed over the course of years, all the while Stripe continues to grow its product suite and further conquer the world of e-commerce.

Twitter did a huge Scala rewrite. There are many lessons to be taken from their lack of product velocity while doing so that the Stripe team took onboard when making this decision.

ok this is going to sound snarky but it's an honest question: Why not just use Crystal[1]?

It's basically just compiled ruby with static types, and it can infer types correctly in most cases without you explicitly noting which they are.

The best argument for this (instead of Crystal) is keeping access to ruby gems.

[1]: https://crystal-lang.org/

My 100k LOC code base is already written in ruby though. I have to rewrite everything?

yeah. that's a pretty compelling reason, but on the other hand, there's a lot of value to be had from modularizing your codebase and microservices so... you could do all future dev in a new language, and if you wanted, gradually convert your existing modules. Sadly, few places think far enough ahead to see the values of modularity (even ignoring language switching) and end up with giant monolithic codebases so language switching just isn't an option without a complete rewrite.

While you have some good points, I'm not sure monolith and modularity are mutually exclusive. Some places I'm sure think some places think far ahead, and determine that the cost of microservices isn't worth the later benefit while they're trying to bootstrap a project.

Is Crystal a superset of Ruby where any valid Ruby project is a valid Crystal project -- just lacking type definition? If Crystal can do that then I think adoption of Crystal would benefit greatly.

> Is Crystal a superset of Ruby where any valid Ruby project is a valid Crystal project

No. So the idea pretty much stops there.

As @chrisseaton said, no, BUT some ruby is valid Crystal. I think it would be more accurate to say that a subset of Crystal is valid Ruby, and a subset of Ruby is valid Crystal.

That's correct. You can run your Crystal programs with the Ruby interpreter, but you can't compile your Ruby to Crystal (simple scripts usually work, but bigger things that use use gems probably won't).

You can't run a lot of Crystal code in Ruby, either. They implement keyword arguments completely differently (Crystal does it more like Python, where all arguments can be specified by name or position) and of course Ruby doesn't support Crystal syntax relating to types. Crystal also has a tuple type with a literal syntax ({1, 2, 3}) that isn't valid Ruby. But otherwise it's fairly close.

Thank you for the correction, I was not aware of this.

For me it would have the same things that make Typescript appealing:

- Full access to a large and mature ecosystem (one that Crystal doesn't quite have)

- The ability to gradually add typing to an existing Ruby codebase. No need to rewrite an entire application. Just gradually add types to new code and during refactors.

If I was starting a new project I would definitely be considering Crystal, but I still feel like something like this would be great. Typescript has been a godsend for my frontend development life, and I would love to have something like this for my backend Ruby development.

I've been working with Lucky and working with a bit of Crystal - it's still not quite there yet. When we were working on a recent release of Lucky we found a kinda breaking in the compiler that the team fixed pretty quickly but it still shows just how new it is. I'm optimistic for 1.0 though

That said, Flow style gradual/optional typing in Ruby would be awesome

It’s not just compiled Ruby, but a completely different language with a superficially similar syntax. You can’t really port anything.

This Ruby typechecker is a similar approach to Facebook's "Hack" language, incrementally upgrading their legacy (PHP) codebase to a new statically-typed language instead of rewriting everything in a new language.

https://code.facebook.com/posts/264544830379293/hack-a-new-p...

Putting this out here in case if anyone finds it useful - I like a lot of good ideas from Contracts.rb such as the type signature looking more expressive (personal taste - looks Haskell/Elm ish) and support of Maybe's. Granted it's not static but def some good ideas to steal from.

https://egonschiele.github.io/contracts.ruby/

Typical Rails codebases already enjoy a reasonable layer of 'typing' via ActiveRecord coercions, validations, and possibly some system that declares/documents the types/structure of your REST API.

Also, I advocate keyword arguments (especially 'required' ones), which tend to kill 80% of the use case of typing for dynlangs ("what's the type of argument x?")

A meaningful test suite would hammer the last nail in the coffin.

Is the idea that you would take your existing code and add typesafety to it? A statically typed language is faster because it does type checking at compile time and does not need to do "if string, ok else throw exception", so adding a bunch of type checking to a dynamic language adds the development overhead without the benefit. I suppose there is a false economy in that statement as defects caught early save time vs late defect discovery, which in my experience, is very common due to type issues in dynamically typed languages.

I personally prefer working with statically typed languages but I've been working with elixir for the last 8 months or so and do not feel very compelled by the use of dialyze (type checker) in the dynamically typed language - you get used to the idioms and try to keep the code type-y but it seems unidiomatic to take type safety too far in a dynamically typed language when you don't get the performance benefits. You cover the things you're likely to get wrong but embrace it for the sake of development speed otherwise. For a new project the right question to ask is "why would we use this instead of a statically typed language." I suppose for an existing project the question this is trying to solve is "how can we reduce bugs now that we've made it to market."

I was at RubyKaigi. This presentation was one of the best delivered, so kudos to the Stripe team for all the work put into it.

A question I didn't get to ask while there is -- will there be some sort of type sharing system introduced?

I've been a user of both Typescript and Flow (both great tools), but saw Typescript's popularity soar because of community written types.

> A question I didn't get to ask while there is -- will there be some sort of type sharing system introduced?

Yes. This was one of goals of our talks. We wanted implementors of type systems for ruby to start collaborating, in particular on a repository of typed shims.

The details of how this would work out might be different. Typescript and Flow change syntax and thus they cannot be included inline in arbitrary JavaScript libraries. We're intentionally compatible with Ruby syntax. Thus we would like this "repository of types for libraries" to only contain types intermittently, until types have been accepted to upstream of respective library.

> We wanted implementors of type systems for ruby to start collaborating, in particular on a repository of typed shims.

I see. Do you see this being a mono-repo of types that's under a Sorbet org or something Sorbet looks for in a gem if provided? Opening issues for a project's types with Typescript has been somewhat painful since many live in one repo [1].

1 - https://github.com/DefinitelyTyped/DefinitelyTyped

Is there any discussion on why Stripe chose to write so much code in Ruby? I would have thought for financial software where subtle bugs can have a huge impact you would want to opt for a language with static checks. Or maybe only a small part of the codebase requires this?

I have to say, after using Stripe for a few years - I'm amazed at how well it works. In fact, I can even create a "refer a friend" system on a rails app in some 15 lines of code[1].

I'm excited if they can bring the same simplicity to their additions to languages and frameworks as well.

[1] https://blog.projectpiglet.com/2018/03/refer-friend-using-st...

Interesting! Anyone with insight into the ruby core team know if Ruby may add typing (optional or mandatory) in the future?

Matz has talked about it as an opt-in. I think his opinion is to not force people to adopt it if they won't enjoy it or their productivity will be hampered by it.

Was at the Kaigi and on the opening keynote Matz talked a bit about it. He doesn't really support adding it since he feels like in 20/30+ years, advances in technology will make writing types "obsolete".

It's a weird argument. If it's not necessary in 30 years (I doubt it because some of the type inference issues are simply undecidable, and that in much simpler languages already), then the language can just opt to ignore type annotations or merely check their consistency. Having them is great either way for the benefit of documentation.

My take is he doesn't really like (or "get") types and feels protective of his language, which is fair. I think Python made an interesting choice here, i.e. provide syntax for types without semantics. The drawback there is potential fragmentation on type system semantics.

So, Sorbet is to Ruby what Dialyzer/Dialyxir is for Erlang/Elixir right?

Are they using basically the same approach?

Having done some legacy Ruby work I'll say this is sorely needed.

Is this analogous to mypy?

It is a similar idea, yes.

I think the most fascinating thing that Ruby 3 could approach is an approach similar to Dart 1's optional types. That way a fast unchecked mode could be available for developing, and REPL usage, and then a separate switch could be flicked for building a release.

Seeing the Ruby devs take compatibility so seriously is such a confidence builder for me, either way. Ruby is such a joy to program in, and for certain projects, Crystal fits the "square enough to go in a square-ish hole" role, but I really miss some of the ease-of-use that Ruby has to offer.

require 'Something'

Something.new.methods.sort

I use that every single day. Crystal can't do that. Yet.

Little things like that make Ruby such a human-friendly language.

Now that we've learned our collective lesson that naive dynamic types are too slow for big projects, I am glad to see Ruby-thinkers begin to explore the type system space and search for something that will benefit all Ruby hackers.

I suspect that if we arrive at the point where Ruby has a gradual or optional type system available, that refactoring some of the old Ruby tools will be very pleasant. If or when that day comes, I know I'll devote a chunk of my own time to try and catch the tools back up, and the rest will follow if only there are hackers who want to hack badly enough.

A lot of people that love Ruby and dislike Crystal seem to dislike Crystal for reasons like what you mentioned. I'm pretty sure that you can get a sorted list of methods in Crystal (not with the same syntax unfortunately) at compile time in macros. I'm not sure why you would ever want to dynamically get a sorted list of methods at runtime... unless you're adding methods at runtime.

I use it in the repl for classes I'm not familiar with. I use Ruby for duct taping a lot.

People created dynamic languages for the exact reason of not having to define types.

Now the trend is to bolt on type checking to dynamic languages because people don’t want dynamic behavior. Most developers I’ve encountered who seek to add types to dynamic languages will never truly understand dynamic languages, closures, concurrency passing or functional programming for that matter.

If you want a statically typed language, use one. Use one that was designed for deterministic behavior and static correctness.

If you have a huge codebase in Ruby that is full of bugs and is unpredictable, don’t blame the language or lack of types.

There are plenty of ruby codebases that are bug free and work well without static typing. How do they exist?

> There are plenty of ruby codebases that are bug free and work well without static typing. How do they exist?

This is a bad question. Refactoring Rails is AFAIK a nightmare. Given enough devs it can probably be done, however it might be painful.

> Most developers I’ve encountered who seek to add types to dynamic languages will never truly understand dynamic languages, closures, concurrency passing or functional programming for that matter.

I don't see how seeking to add types precludes also understanding "closures, concurrency passing or functional programming", especially since these are in no way foreign concepts in statically typed languages.

> If you want a statically typed language, use one. Use one that was designed for deterministic behavior and static correctness.

I definitely see what you're trying to say, but as I understand it, these projects to "add types" often come on later, when the project is so big, that it is simply less time consuming to just develop a type checker, than it is to rewrite your entire codebase in a new language.

My argument is that you can always implement your own breed of type checking either via convention (with appropriate enforcement via static analysis or linting tools if so desired without having to run the code), or you can do type checking at run time -- most dynamic languages have a facility to interrogate a variable to determine its contents and you can implement any number of tests on it as you see fit to validate it.

In practice the difference in guarantees between a static type checking system and a runtime one are so vast that I think they're mostly incomparable. Runtime checking is like doing most of the work of modelling a program with static types but achieving a fraction of the benefit.

But let's say you have a function that takes an integer and goes completely bonkers when you pass in a string. Why would you not want to prevent that?

I'm not sure how well you understand functional programming either if you think it needs dynamic typing. The king of functional programming languages (Haskell) is statically typed.

The king of functional languages is not Haskell but Lisp in my opinion, and Lisp is dynamically typed. With Common Lisp, there is no difference between runtime and compile time: it's one in the same.

> People created dynamic languages for the exact reason of not having to define types.

Not having to define types is not the biggest merit of dynamic languages.

The biggest benefit is to be able to do things that you couldn’t do if you had a type system. You can do some crazy stuff with Ruby’s meta programming.

How many problems out there are really apropriately solved by runtime metaprogramming?

I don't understand the appeal of static typing. It always feels like I'm adding overhead without receiving any benefits. I've worked in some fairly large dynamically typed codebases and have never run into issues with type errors. Static typing does not alleviate the need to test your code, which is a more effective way of reducing bugs in your system than annotating your methods/functions.

Not sure I've ever heard anyone say static typing removes the need for testing.

Types are self-documenting. Onboarding new devs in a large static codebase will be 10x easier than a dynamic one.

Coding is actually faster in statically typed projects - autocomplete works every single time and you never have to dig into another source file to see what objects/methods you have.

And, most modern type systems (Typescript, Swift, etc) don't have much overhead at all with their impressive type inference.

I would disagree with the statement that coding is faster in statically typed projects. The ease of onboarding new devs is solely based on the code quality of the project. Well written programs are easy to understand regardless of typing. Terribly written programs are opaque regardless of typing.

Sure, any code base can be horrible to learn, but two equally well written code bases, types can help significantly. When your new employee can be confident that changes something in one place, won't invisibly break something in another place, they are more prone to start making changes.

Vimal Kumaar touches on how it has helped them get employees contributing to the code a lot quicker, after they moved to PureScript https://www.youtube.com/watch?v=HLEwYghBjo8 (I think around 24+ minutes in, but I recommend the whole talk).

If I have to connect a couple classes in separate files with a few objects/stuff inside of them, I'd bet good money that a statically typed language will provide a faster coding experience. Especially if you have leveraging third party libraries. Working in huge libs in JS (Sequelize comes to mind) is a horrible development experience that requires you to have the documentation open.

Those Scotsmen are pretty untrue, huh?

The claim is pretty facially silly at a glance besides, though. You literally don't know the fields and methods, to say nothing of what they actually give you, on a Ruby object without opening either the code or pulling up Pry and doing `ls`. Which I have done, often enough, to fail rather stunningly to miss doing so when in Kotlin or when using TypeScript. Checked that one? Good news--now you get to drill down into the next one, and the next one. I hope that isn't coming from another library, too, 'cause jumping from library to library in Ruby or Python is a chore. And even if not? God help you if you're using ActiveRecord, I hope you enjoy scanning your database migrations or describing tables just to learn method names.

It is a bad scene. And don't get me wrong; I like Ruby quite a lot. But I won't make excuses for it.

Personally, with the actual tooling available to modern systems, I write TypeScript much, much faster--we're talking probably twice as fast--than I do even ES6, which I otherwise quite like. And TypeScript can't even assert whether something is a whole number or not, unlike better languages. It has replaced Ruby for me on the strength of that alone.

It's a double edged sword - onboard some devs on "advanced" scala code and let me know how the 10x is working for ya. I also feel that sometimes types cause additional congitive load - you have to mentally model the types and what they do. OTOH I'm pretty sure most people can onboard a js project that's well written.

I think exploratory, debug-based onboarding can be easier with poorly written static lang codebases that dynamic, but then again depends on what convetions are used. I don't care much about autocomplete unless I have to type ReallyLongClassNamesThatAreSupposedToBeMeaningful; besides coding is mostly about avoiding errors, not typing faster.

In the end, whatever floats your boat but dyn langs surely have their place.

I've heard the "mentally model" argument a few times, and tbh I don't quite understand it. You have to model the object regardless, either with types or tons of extra checks to make sure that object has what you need. I've seen so much javascript code do exactly that because the methods had no idea what they were going to be given. The contract of static typing eases that load, I think.

Ya for sure. Without getting a debugger into the application code, it sometimes seems completely impossible to look at a piece of dynamic code you're not already familiar with and decide if it's going to work. I've found this especially tough during code reviews for new stuff that make heavy use of abstractions, even when the abstractions are good.

Personally I feel that my mind is registering what each type does, kinda like building a mental dictionary. Dynlangs also have types but in terms of reading code it feels you can just skim through and skip the details. I guess there is a difference in the cognitive load of reading code and writing code.

When using a dynamic language you need to know what properties are on an object right? So you can work with them, access them, whatever. So just write it down so everyone else knows too. There, that was easy.

If that’s the case, then why not have the compiler check them as well?

Yes, that's exactly my point. Right them down in your program so that the compiler can check them rather than holding them in your head.

Sorry, I misunderstood your comment.

At this point I don’t think anyone sane is seriously advocating for the adoption of Scala.

What alternatives are there to what Scala provides? What newer language would a 'sane' person advocate?

Unless you really need it for a platform like Spark where there aren't many other options, pretty much any other statically typed language is likely to be more productive in the long run, once you account for the costs of Scala's complexity. Java, Kotlin, C#, whatever. And if you don't need or want the benefits of statically typed languages, then the same is also true for most major dynamic languages. It's difficult to be more specific than that because I don't think there's a very compelling case to use Scala to begin with.

Actually the only language I used autocomplete was Java, mainly because it's almost impossible to work in Java without a IDE. However sometimes I turned autocomplete off because those long menus are a pain to see. I don't use a real autocomplete in Emacs with Ruby, Python, Elixir, JavaScript. I use pabbrev which remembers the words I type. It's usually enough not to make typos and it helps to remember the methods/functions.

What languages have you worked with before, with static types?

If it's something like Java, C, C++, etc, then I can see your point, but if you pick any of the languages with type inference (like Elm, Haskell, PureScript, OCaml, F#, etc), then you rarely actually have to write types.

Working with the type system is a different way of working. It's more beneficial to see the compiler as a helping friend, instead of an adversary that is just there to complain.

Some interesting reads might be:

- http://elm-lang.org/blog/compilers-as-assistants

- https://matthew.brecknell.net/post/hole-driven-haskell/

- http://bitemyapp.com/posts/2017-09-23-please-stop-using-type...

Personally, I honestly hate every time I don't have a time system to rely on. I don't make any false pretenses that I have the whole program in my head 100% of the time. I don't want to have to think about things that a computer can solve for me, hence I like my compiler to do as much work as possible.

As for unit testing, a type system is never supposed to remove the need for unit testing, and if you see anyone promote a language because of this, they are probably (hopefully) over simplifying. What it does do though, is significantly cut down the need for certain types of tests, since you can model so much more logic via types (ADTs, Union types, etc), and get it checked by the compiler.

As an aside: I simply do not believe you've never run into a type error, unless you've only written projects that 100 lines of code, and even then it's quite easy to run into type errors.

The languages that I've used with static typing are Java and Objective-C so perhaps those languages have negatively impacted my perception of type systems. I can do more research on other languages to see what they have to offer.

I used to run into type errors when I first started programming but I've learned how to eliminate them by changing my programming style. I add a lot of constraints to how I program, and follow a system of naming conventions that make it obvious what type something is.

I take a lot of pride in my work and try my best to build maintainable systems that can be extended easily or run forever without changes. I typically write around 250K-350K lines of code per year in mission critical and highly regulated industries. If you want to learn how to write systems that have zero defects research companies that are known for quality design and apply their principles to your own work. I would suggest Netflix, and Toyota as the two companies that I would emulate.

> I add a lot of constraints to how I program, and follow a system of naming conventions that make it obvious what type something is.

And you find it unbelievable that someone would prefer that the compiler simply enforce these conventions?

I don't find it unbelievable, I just personally don't get the appeal because you have to do that work when you name things anyway. e.g. I would never expect a variable named "first_name" to be anything other than a string. It's also fairly easy to handle most type conversions so I would rather be permissive in what I accept (to a reasonable point) than be rigid.

Well what are the properties of config_options or app_user or objects like these? Is it even documented, or am I forced to scan the entire method trying to figure out?

Typically I would initialize an AppUser object and print out its default representation. You can quickly find info on an object by doing something like:

    AppUser.new 
      => prints all properties of the object with nil values 

    AppUser.instance_methods - ActiveRecord::Base.instance_methods 
    => prints all unique instance methods on that class

values

config_options is extremely generic but I would expect that it is a Hash of key/value pairs. Given the fact that it has options in the name you are likely going to have to look it up regardless as you have multiple choices when using it. I'd expect some documentation if it is truly configurable, or you can transparently look at the method signature if it uses keyword args. (say for a library that handles formatting currency)

Quickly reading other people's code is a definite skill. While it can be a pain to do, it is worthwhile since the code is the only real source of truth.

Yeah, of course there are ways of doing it. But if you define types you can in a second be looking at all the possible values. The tools help you be more productive and make fewer mistakes.

Well, what kind of projects? And what order of magnitude big?

Whether dynamic type errors become an issue depends a lot on how often all codepaths are run.

If what you're running is a batch processing script, it's likely to crash out as quickly as a type-checking compiler, and then static types have no direct appeal.

On the other hand, if it's a long-running server with a lot of finite state machines(maybe it's running a multiplayer game or something) it's really hard to reach type errors without an extensive manual test.

This becomes a relatively bigger issue as the codebase gets bigger, again depending on the type of code. If the main way the system expands is by handling a more diverse set of data types, dynamic types allow less code to be written, but also make it harder to verify.

In practice what I tend to see happening is that a glue layer appears that needs dynamic behavior, while the codepaths underneath that benefit more from static typing.

The usefulness really depends on the team. If you're working with another engineer or two and they're all senior engineers that you've coded with for a while, then it's probably not necessary since you'll understand each other.

However, if you work with a junior team or you have to inherit a project, then I've found static typing to be extremely useful. One might argue that code review is suppose to solve for that, and while you can catch some of the problems, you won't catch all.

It's definitely helped to make refactoring faster. Specs will help to some degree, but static typing will traverse through every function to ensure the parameter you're passing is as expected.

No more is this suppose to be a nil, a blank value? You can specifically define what you expect. While some people hate optionals with Swift, I've found it to be an interesting concept.

I started working with TypeScript over vanilla JS a year or so ago. It's completely changed the way I work. Having autocomplete, pointing out silly errors (forgetting to return a value, forgetting to set a prop on a React component, forgetting to add a member to an object) before I refresh the page, and being able to pick up old code without having to reread through it to figure out what the hell it actually spits back out is a godsend. The latter has sped up my productivity significantly; being able to get the output from a library, hover over the variable, and see the exact shape of the output saves an incredible amount of time when doing mundane day-to-day tasks.

You've gotten a lot of responses but let me throw one more idea at you: bad or mediocre code bases. Or old codebases that have the scars of years of tweaks, fixes, and hacks. Thats where I think static types shine the most.

Comparing my Python and Java experience, jumping into the middle of a codebase and correctly untangling a few particularly problematic spots or reworking and refactoring packages without introducing errors is dramatically easier with static typing.

It's feels like the difference between untangling actual wet noodles vs solving a sliding block puzzle. It's just more mechanical.

Even BEFORE running tests there is a lot of confidence that you haven't overlooked something. Hell, you can't catch typos in python without 100% code coverage. The sanity bar is just that much higher before you even get to unit tests.

For me, static typing actually makes it easier and faster to write code. It’s like someone looking continuously over my shoulder for mistakes.

I am the least efficient when I write code in JS.

On the other hand with Ruby, and meta programming, I can do stuff that would be impossible to express in the most popular type systems.

Reply on news.ycombinator.com