Static Typing for Ruby: Adopting Sorbet at Scale
shopify.engineering
shopify.engineering
> sig {params(name: String, id: String).returns(Integer)}
My ruby isn't quite good enough to parse the language constructs that make up that line, but it's not pretty. I guess....sig is a class method which takes a block....and params is a class method (Added to BasicObject or something, perhaps?) to which you can pass any number of keyword params, and which returns an object that has a return method, to which you can pass any object type. I guess I don't know why you need to call sig at all, as opposed to just params( args ).returns( type ). I also have no idea how the sig call gets associated with the method that follows it.
I just wonder if there was a cleaner way to phrase this that's still syntactically viable.
Sig also turns into a no-op if you have runtime verification turned off, which is another good reason not to call params right away, because (in ruby) you can ignore everything in the block if the block is not called, sort of like a debug macro in C, but you can't do that with a method that is called - it must evaluate its parameters.
It is, more or less, an implementation of this idea: https://yehudakatz.com/2009/07/11/python-decorators-in-ruby/
sig(name: String, id: String).returns Integer
I think Ruby was an excellent replacement for Lua to do ML, once Torch ran out of steam due to LuaJIT memory limitations. However, historically, MRI Ruby was slow and problematic.
Personally, I prefer Ruby to Python. Ruby is very close to Smalltalk, with some ideas from Perl and Lisp.
Why problematic?
On the slowness, I would agree that Ruby is slower than many languages, but as an interpreted language this is -up to a certain point- by design, or at least an accepted part of the trade-off one accepts when he takes a programming language. Moreover, Ruby's natural competitor is Python, not -say- Clojure; and I wouldn't that Ruby is significantly slower than Python.
I use Ruby for everything I can, Python for ML, and R for data analysis. I just find it easier to not swim upstream and force ruby onto all of those contexts when the other 2 languages have loads of inertia behind them in those spaces. It's unfortunate.
https://kvitajakub.github.io/2016/03/08/luajit-memory-limita...
But while I personally enjoy it more than Python, Python will always be the one I would recommend to anyone for work or learning-- it's so much more straightforward and predictable. The "one right way to do things" mentality that pervades python makes things so consistent and intuitive. Ruby, on the other hand, gives you 10+ ways to form a loop. The paradox of choice and all.
> In fact, Ruby, to me, so much of the enjoyment in Ruby is these incredible subtleties, of how many different ways you can structure a conditional. Like, Ruby has, I don't know even the count, there's gotta be 60 different ways you can say `if something`, right? And it is in those 60 different ways that I find half the enjoyment of writing Ruby. Like, it was one of those things where I knew, very early on, that Python was not a language for me because it said, right in the manifesto, there should be preferably one and only one way to do things. Ruby has the exact opposite approach, there should be preferably ten thousand subtle different ways of doing things, that will allow you to write that particular conditional, with just the right emphasis, do you write it in the front, do you put it at the back, is it multi-line, is it single line? Like, there's so much variety and it's in that variety that I find poetry. And it is the poetry of writing Ruby code, of making those subtle distinctions where, at the end, you can like, "Ehh, should we move it around" like, where I just go like, giggles, right? Like, this where like we talked about that big smile, right? So much of that big smile comes from, not just like solving the problem, but solving it in a poetic way.
https://pragtob.wordpress.com/2020/08/24/the-great-rubykon-b...
https://speakerdeck.com/eregon/running-rack-and-rails-faster...
It's frankly easier for me to rewrite a small service in Rust, and have native typing (plus thread safety, no race conditions, easy parallelism, etc).
While I'm best at Ruby, and have used it professionally for well over a decade - it's wearing on me.
You'll also notice the three selling points on the home page:
- It's fast (it has to be, else it wouldn't work in large existing codebases)
- It's IDE ready (so that a dev tooling team can go to their organization and say "if we adopt this, the IDE features will make our engineers more productive")
- It's gradual (so that typed and untyped code lives side by side)
If you have the luxury of starting from scratch, there are far better ways to ensure 100% type coverage from day 1. Unfortunately, companies like Shopify and Stripe don't have that luxury, and are contending with hundreds of developers who maintain millions of lines of Ruby code.
Also, maybe I missed it, but Sorbet has a very fleshed out plugin for VSCode but that's about it. How was the experience getting people who don't use VSCode to integrate it into their workflow?
Quoting from the blogpost:
> - Developers get used to Sorbet syntax over time
> ...
> Our main observation is that developers enjoy Sorbet more as the typing coverage increases.
On the editor integration, the funny thing is that VSCode is the only editor that needs a special extension to integrate with Sorbet. The way Sorbet supports editors is via the built-in Language Server Protocol (LSP) mode which can be used with any LSP plugin for any editor. I know people who are using it with Vim, Sublime Text, etc. Integration is, in a nutshell, running `srb tc --lsp` as a subcommand and piping data in/out via stdin/out. Our tool Spoom actually uses the same LSP mode to provide extra developer tools and analysis on top of Sorbet.
Story that Ufuk shares is common.
We saw this process repeat in many companies, Stripe, Shopify and many others - folks are initially bothered with verbosity of `sig` syntax, but as the company starts using it in practice stop being bothered pretty quikcly. It happens even faster now that IDE integration allows to auto-complete entire signature and can in most cases correctly guess the types of arguments & result type. You rarely type `sig`s yourself.
The common internal asks at Stripe are about even better IDE support, new features & etc and that's where most of investments are. Well, and and obviously performance. Sorbet was already very fast and we intend to keep it fast as our huge codebase grows.
I was unsure exactly which bits of Sorbet rbi were still required with tapioca https://github.com/Shopify/tapioca/issues/114
We are currently using `sorbet-rails` but it appears Tapioca would be a replacement for that as well, is that correct?
The DSL generators are not 100% complete to fully replace `sorbet-rails` right now, but we are preparing a 1.0 release of the gem that should be able to do that.
> but you don't get any `sorbet-typed` RBIs, which, right now is not a concern for us.
Ahh ok! I think this might have been a bit I missed, cause I was commonly trying to do both which likely caused issues for me.
> The DSL generators are not 100% complete to fully replace `sorbet-rails` right now, but we are preparing a 1.0 release of the gem that should be able to do that.
Will keep my eyes open, thanks!
(Also: Thank you for sorbet! It's the biggest reason I continue to use Ruby for my personal projects.)
1: https://github.com/chanzuckerberg/sorbet-rails/issues/104
This is indeed a problem in our codebase as well, and so far our team has been suggesting that people add shim (i.e. manual) RBI definitions for the methods that they find are missing from the types they've expected them on. This is a good stop-gap measure to solve a problem that is not very common with an easy solution to implement.
That's surprising since I would have figured most enterprise shops are on JRuby
How? Do they track all of their devs commands?
We do the same thing where I work. If you're working on a developer productivity team, the service whose SLA you're responsible for is the dev tools, so it's critical to know how long they take, which commands are run most frequently, etc.
In fact, Sorbet, has this built in: you can give it an assortment of `--metrics-...` [1] and `--statsd-...` [2] flags that configure Sorbet to talk to a statsd service directly:
[1] https://sorbet.org/docs/metrics
[2] https://sorbet.org/docs/metrics#reporting-metrics-directly-t...
Just like in C, you have trivial, nontrivial, complex apps, then there's still a long way before production OS kernels at the extreme.
Many advanced libraries don't use metaprogramming much, and even if they do, it can be often made either more explicit or shifted to compile time.
Edit: The downvotes are interesting since the portability claim comes from my experience doing just that.
Having said that, such optimizations are indeed possible and we are already thinking about how we can use the type information we have to that effect. There are still many things that need to line up for that to be a thing, so we are not expecting any new developments on this soon. But this is a very exciting direction for our Ruby Infrastructure team at Shopify.
Nope, Crystal is a statically-typed language with distinctly different than Ruby semantics that features Ruby-ish syntax.
Sorbet is an optional static type system for Ruby.
They aren't trying to do the same thing. One is trying to appeal to the aesthetic preferences of developers who like Ruby with static typing, the other is trying to enhance the Ruby ecosystem with static typing. There's only a superficial similarity between these things.
Only reason why it's still alive is Rails, I don't see much hope for it longer term, Crystal and Elixir does IMO.
Using Ruby/Sorbet screams using JS/Flow to me, instead of using a language with types built in.
Can you expand on why you think this is?
Is there any language or ecosystem that can protect against a rogue employee?
Doesn't sound like a Ruby problem.
This is a thing?
It's your own dumb fault if you expose the web-console or similar on production.