A few words on Ruby's type annotations state
zverok.space
zverok.space
In most of the larger programs I've worked in the value of static typing is immense. In terms of my productivity, the only programming language feature that's possibly had a bigger positive impact is garbage collection.
Runtime metaprogramming is very cool, but I've yet to work in a codebase where I felt that the value of it was greater than the value I get from good static analysis.
One way to think of static types is that they're a debugger that simultaneously debugs every possible run of the program. And on top of that, you also get code navigation and better performance. It's hard for me to imagine a sufficiently great metaprogramming feature that would be worth giving that up.
All this to say, runtime metaprogramming doesn't seem inherently more capable compared to what's possible in the more “static compiler” languages; rather, it seems like an escape hatch for when you can't generate the code you want at compile time.
For that matter, Haskell can do exactly the same thing, and as you note its type system is even more sophisticated than Rust’s.
Depending on how dynamic your problem is this might be required and then “traditional compile-time” macros would not be enough.
It's by definition more powerful though, in that the runtime language is (hopefully) more powerful than the type description language.
I'm not arguing that it isn't preferable to produce (de)serialisation code statically. Just that by definition it's more capable as it can do everything the runtime can.
15 years of ruby and all that metaprogramming is literally the most painful part.
Funny, because I used to love it.
I think there is a serious disconnect about what is "metaprogramming."
For example, I recently wrote a function that takes an arbitration nested object and a list of paths through that object (e.g. "path.to.object"), and does things with the data found at those paths. For (1), I could write a type that, given an object type, would return all the possible allowed paths as types, basically preventing the function from being used correctly (and it already has helped me catch a few typos here and there). So now func({foo:...}, "bar") is disallowed directly at the type system.
But working with those exact types inside the function would have been too complicated, so I also did (2): inside the function, I just treated as the strings as normal strings (which, to the runtime, they are!) and did normal JavaScript metaprogramming to get arbitrary keys from arbitrary objects. I then have less type safety inside the function, but I can test that part more, while still leaving the caller of the function with the exact correct types.
In fairness, there are other ways to solve the same problem in other languages, with similar type safety. And JavaScript natively is not that high up on the metaprogramming scale. But I'm always impressed in Typescript how much a powerful type system and metaprogramming can dovetail into each other.
I've played with this idea a bit in Solargraph (especially in regards to Rails) and while the results are promising, the barrier to entry and amount of labour involved is pretty high.
def for_environment(environment)
environment =
case environment
in String => code unless code in UUID_RE
return none # no support for filtering via environment codes
in UUID_RE => id
return none unless env = Environment.find_by(id:)
env
in Environment => env
env
in nil
nil
end
...
end
I think defp looks and feels very Ruby-like compared to the alternatives (which have the problems you discussed).And Elixir is great, so I can appreciate the inspiration.
Your example method is a bit tricky, because it rebinds an argument and then presumably uses it in the elided bit (the "..."). If the case statement was the entire method body, given proposed syntactic rule it could be written as:
def for_environment
in String => code unless code in UUID_RE
none # no support for filtering via environment codes
in UUID_RE => id
Environment.find_by(id:) || none
in Environment => env
env
in nil
nil
end
To me this "rhymes" nicely with the ability to write `rescue` blocks at the method level: def foo(params)
send_request(params)
rescue => e
raise if critical_error?(e)
end
---That is of course a pretty rough approximation, and the real rules would probably be a bit more complex[0], but it definitely seems achievable and possibly worth doing even without taking the type-annotation aspect into account.
[0]: for a start, patterns would need to be extended to allow binding the block argument in a manner equivalent to `def foo(&block)`
For example, with overloading,
defp for_environment(Environment => environment)
case
when environment.isolated?
where(environment:)
when environment.shared?
where(environment: [nil, *environment])
else
none
end
end
defp for_environment(nil)
where(environment: nil)
end
defp for_environment(String => code unless code in UUID_RE)
none # no support for filtering via environment codes
end
defp for_environment(UUID_RE => id)
return none unless environment = Environment.find_by(id:)
# calls the pattern matched method for Environment
for_environment(environment)
end
Which would be equivalent to (using syntax from OP), defp for_environment
in String => code unless code in UUID_RE
none # no support for filtering via environment codes
in UUID_RE => id
return none unless environment = Environment.find_by(id:)
case
when environment.nil?
where(environment: nil)
when environment.isolated?
where(environment:)
when environment.shared?
where(environment: [nil, *environment])
else
none
end
in Environment => environment
case
when environment.isolated?
where(environment:)
when environment.shared?
where(environment: [nil, *environment])
else
none
end
in nil
where(environment: nil)
end
Which would be equivalent to, def for_environment(environment)
environment =
case environment
in String => code unless code in UUID_RE
return none # no support for filtering via environment codes
in UUID_RE => id
return none unless env = Environment.find_by(id:)
env
in Environment => env
env
in nil
nil
end
case
when environment.nil?
where(environment: nil)
when environment.isolated?
where(environment:)
when environment.shared?
where(environment: [nil, *environment])
else
none
end
end
In this case, I feel like defp with overloading could really clean up the code (defp without overloading, not so much). You no longer have to add the input transformation to your mental stack when groking the method.I also don't really see a problem with allowing &block like a normal def.
> An integral part of this informality is relying on Matz’s taste and intuition for everything that affects the language’s core.
I think a more defined process would mean a better future for Ruby and Ruby developers.
Also, this is subjective but I don't like the syntax.
> Sorbet is 100% compatible with Ruby. It type checks normal method definitions, and introduces backwards-compatible syntax for method signatures.
I would have preferred if they had introduced a compile step the way typescript does, and provided a TS-like syntax. I find the current version hard to read.
The sheer propagation of JS might have something to do with the big push to have some kind of typing. From my own experience, if I see ruby I know I can either re-write it or find an alternative in TS/JS.
The ubiquity of JS makes it more accessible, but I'm still trying to find reasons why one would choose Ruby.. I'm always a 'right tool for the job' but I don't know what the niche is.
Edit : My pedant in me :
> compile step the way typescript doe
Typescript transpiles to JS. I don't 'believe' there is a compilation step.
> Typescript transpiles to JS. I don't 'believe' there is a compilation step.
This is a common misconception. Transpiling is not something distinct from compiling. "Transpiler" is just a trendy name for a certain subset of compilers. Just because it compiles to another "high level" language doesn't mean it's not a compiler.
Every "transpiler" is a compiler.
Sources:
On BIX in the 1980s, when the only implementation of C++ was Cfront, which translated to C, I asked Bjarne Stroustrup if it was a preprocessor. He told me quite emphatically, "No, Cfront is a compiler." (I don't think the term "transpiler" was in common use at that time.)
The Wikipedia article on Cfront agrees:
> Cfront was the original compiler for C++ (then known as "C with Classes") from around 1983, which converted C++ to C
https://en.wikipedia.org/wiki/Cfront
More recently, and relevant to this discussion, the TypeScript team specifically calls tsc a compiler:
> Let’s get acquainted with our new friend tsc, the TypeScript compiler.
https://www.typescriptlang.org/docs/handbook/2/basic-types.h...
In fact, if you search that page for "pil", you will find nine references to "compile" and none for "transpile".
I've always seen them as different, but as you lay out, they're all doing the same thing in the end.. Thanks for making me a little more aware!
Does Sorbet offer something comparable? That would make adoption easier.
Yes, RBI files.
I do agree with your first point that the lack of adoption of Sorbet among library maintainers negates some of the benefit of the type system (compared to TS). Seeing all those `T.untyped`'s in generated RBI files is a little scary :D.
TS has been around a lot longer than Sorbet and even moreso a lot longer than RBS.
When I was playing around with ruby for any significant sized projects, I found it became unmaintainable. Granted I was most definitely using it wrong, but apart from that, I didn't see the appeal.
1. 2. 3. 4. 5. standard library
6. consistency with OOP *
7. "everything is English language". I find good Ruby readable just like books **
* Ruby is the best translation of "everything is an object" imho
** evil Ruby is the Perl side of the moon, but it's easy to ignore it
> When I was playing around with ruby for any significant sized projects, I found it became unmaintainable
Ruby requires tons of discipline, as it has obscure meta-programming powers that are meant to be used for DSL libraries, while usually Ruby beginners spam them understating their make your codebase unmaintainable.
I have inarguably written some of the absolutely most elegant solutions of my entire career in it. But without good discipline, experience, and understanding you can absolutely make a horrifying mess of unrivaled proportion.
Unfortunately when you work on real world projects where most devs are relatively junior (or otherwise inexperienced with Ruby) or on projects where there have been multiple cooks who didn’t necessarily share the same underlying mindset about the project, you end up with more of the latter than the former.
But for small, consistent teams of experienced Ruby engineers? It can be incredible.
Last year I spent 10 months alone on a Rails app. I decided to buy in. Do things the Ruby/Rails way. Read books on how to think about OO in Ruby. TDD everything.
I can’t quite say what changed. But now Ruby is my favourite language.
It is almost encourages you to write clean code but it doesn’t force you to. It lets you make mistakes. It treats you like a grown up.
TS, Java, etc. They treat you like a child who can’t be trusted to do things right.
Yes, this means you can end up with some horrifying god awful Ruby code. But it also means you can end up with some truly beautiful abstractions.
I think the very things that prevent you from creating unmaintainable messes are the things that prevent you from writing incredible pieces of software.
That’s probably why they miss Ruby
[Edit]: Added a missing word
I feel similarly re: Java and Ruby. But I also love Rust which relies on static types and one of the strictest compilers in existence.
But there’s something I can’t quite put my finger on where—yeah—Java and Golang (in my mind) force me into a boxed-in world where I’m not trusted to make good decisions about abstractions, but Rust and Ruby encourage me to do so.
A slightly better framing is they languages within stricter static types put up guardrails. They’re meant to keep you safe. Sometimes they also make it hard for you from going where you want to go.
Don’t get me wrong i like static languages too. TS is one of my favourites. My time with Java was good. Just trying to give a different perspective on Ruby
Practical Object Oriented Design in Ruby by Sandi. She has a lot of great talks on YouTube as well.
This is totally Java, but TypeScript? Practically every error can be turned off in the config or in a comment, and the type system is a veritable arsenal of powerful footguns.
I actually can't argue with that. Thanks for a great brief breakdown.
That's the big nail in the coffin. Except for RBS and Steep, I don't know of any that does/did static type checking.
And yup, Sorbet's static type check is very partial to the point they recommend enabling runtime type checking.
Also in its usage, Sorbet needs to _evaluate_ files. Too bad if one of them was a script that included `FileUtils.rm_rf`. Ok I'm going overboard (maybe?), but Sorbet is not stable in face of code that has side effects when the file contents is evaluated.
Are you saying this in the context of static analysis or runtime analysis? I’m pretty sure Sorbet does not need to eval files for static analysis.
> Ok I'm going overboard (maybe?)
You are going overboard. I like both TS and Sorbet. Neither type system will protect you from running malicious code on your computer.
cat > foo.rb <<'EOF'
module Foo
File.open('oops', 'wb') { |f| f << "hello\n" }
end
EOF
srb rbi init
ls -1 oops # => oops
It's not even about malicious code, it's about blanket eval'ing a whole tree of rb files, which may contain anything from mistakes to legit but side-effectful code.Neither of those have such side effects:
rbs prototype rb foo.rb
rbs prototype rbi foo.rb
typeprof foo.rbLanguages with dependent types can do even more. Of course that is much more complicated and at some point you can only check the type at runtime.
But the idea that static type checking is worse than runtime type checking because it can't do all the checks is idiotic. You're throwing out the static benefits for no reason.
You should use static types as much as possible, and runtime checking where that isn't enough.
Nowadays, I almost exclusively write things in TypeScript, and yes, the type system there brings so much peace of mind.
Doing a community-driven approach like DefinitelyTyped for third-party libs in Ruby seems like it'd be much harder than JS. The culture around metaprogramming and complex overloads seems like it'd be insanely difficult to type.
I think static typing is a very fundamental part of language design that affects the entire language. To make static typing ergonomic without massive boilerplate, you also need concepts like generics, co and contra-variance, type inference, and other things that dynamically typed languages never have to really concern themselves with.
When you try to bolt it on afterwards, I think you can end up with a lot of foot guns and corner cases and a false sense of security.
If this "magic" dictionary was sourced from a schemeless api or database, it was impossible to figure out the schema of this dict without running the code in production.
Anecdata: adding typing (via Sorbet, then RBS) to multiple projects over the years uncovered many corner case (and not-so-corner case but non obvious) bugs, some subtle but critical, before they ended up crashing production.
I can vouch with real life experience that it is useful. RBS+Steep is so useful I inadvertently found myself† beginning to write type-first, describing my code in .rbs files then filling in the implementation. Thinking in types has helped me uncovered issues right at the idea stage when I could have produced a smart^Wfatally flawed implementation that would still work thanks to Ruby's dynamism (mind you, Ruby's dynamism is great, it is merely a tool to be wielded at appropriate times)
† Which is kinda useful as a "ok this thing works" rule of thumb in my book
Sure there're holes in the system, but the utility is significant, especially relative to the cost.
And then the expressiveness of the type system there is also really pragmatic. You can't natively express union types in Java and many other static languages without writing your own Either monad implementation. Instead, you often wind up with code that has objects with a hodge-podge of field sets that are nullable.
It's nothing short of miraculous how well the null-safety situation is in TypeScript. It's built on the back of community-driven types in DefinitelyTyped where the maintainers of libs often aren't the ones writing and keeping types up to date. Nowadays, types are becoming more and more first-party, but it still amazes me how we got to this point.
> The “this can be typed, this can’t” approach would most probably lead to a schism unseen before: splitting of the community into those who prefer type-annotated code and consequently became reluctant to any of the most expressive and dynamic features—and everyone else.
I'm an active user of Sorbet at work, and I can see this happening even with my own opinions. On the one hand, I don't want to see the Ruby ecosystem fractured more, but on the other, I've seen clear benefits from helping add static typing to my team's project. I didn't really know much about Steep at the time, so I pushed for Sorbet. My first preference was actually Ruby 3 types, but when I looked into implementing them, I found they provided much weaker guarantees than Sorbet. The tooling also wasn't there.
The false sense of security they give is not a problem. As with testing, this is not a black or white answer, having more is better than having nothing. You could also argue they are more strict by allowing more richer types like unions and structural typing, letting you strictify things that you in C# or Java would have solved with non type safe Dictionary<String,Object>, instanceof(), overloads galore or layers upon layers of AbstractFactories.
Unfortunately this article itself needs some editing. I stuck through to the end, but it was not enjoyably lucid.
> I honestly tried to perform it alongside the reader: since the early drafts of the article, my expectations were that I’d come up with some coherent idea about a possible Ruby typing syntax. My attempt resulted in a deeper understanding of the level of design complexity making it hardly achievable.
Plus one for trying it out, I guess. The temptation is there, because Ruby is so flexible. "It works so well for DSLs, maybe it can be expanded to include typing!" The trouble is, the added syntax rules to support typing will never stay in the sweet spot between "enough in the foreground to assist coding" and "enough in the background to not distract from the coding flow." You either live without it, as in Ruby, or learn to mentally parse over it, as in other languages with the Func(String s, Int i) syntax.
For myself, I'm fine with the typing being in a separate .rbs file. Especially if my IDE supports it well.
We type[0] by having one separate .rbs file per .rb file. Works really well with an editor's vertical splits: type outline on one side, code on the other. That, or use something like vim-projectionist[1].
We have a static typing guide for contributors[2], and I'm also accreting common errors and issues behind the scenes to produce a 'rbs+steep by example' to ease folks in.
[0]: (WIP: there's a huge codebase to type, but we're progressively getting there) https://github.com/DataDog/dd-trace-rb/tree/master/sig
[1]: https://github.com/tpope/vim-projectionist
[2]: https://github.com/DataDog/dd-trace-rb/blob/master/docs/Stat...
Possibly! But likely the reason is this:
> I am writing this on my phone, in a barrack that houses some 200+ of my brothers-in-arms in the Ukrainian army’s training camp; I use short periods of rest between training, mostly at night and on Sundays.
I won't expect a lot of text-polishing opportunities in conditions like that. I'd be grateful for any coherent and insightful output, like the article.
# Python's current type hints
def readlines(name: str, chomp: bool = False):
vs # Ruby's potentially valid type hints that don't exist today
def readlines(String name, Boolean chomp: false)
I know with Ruby you often want to put more important things first, because it deserves to be the center of attention but in the above case I think the 2nd one is so much more readable at a glance. There's less syntactic noise with the Ruby example because it avoids the equals sign and excess colons. You could also make a case when calling a function both the type and name of the param itself are really important. If you think of this as a user experience exercise, someone calling your function is the user. As a user I very much like the idea of knowing what I need to pass in and "what I need to pass in" is a combination of a descriptive name and potential type.At a fundamental level, imperative or partially imperative languages are nice because you can typically build upon what you know to do new things in incremental steps.
For example if you know how to make a variable and a loop you can combine your knowledge to make a nested loop and mutate a variable without learning any new concepts.
The Ruby example feels like you're incrementally adding something new. I don't really need to learn anything new to understand I'm adding types. It's also very clean looking. The Python one looks like a different language. I mean, I know both Python and Ruby and use them regularly but `chomp: bool = False` just looks icky to me. It's like some weird hybrid Python / Ruby combo and takes a few seconds to understand.
Having types in separate .rbs files has proven to me to be perfectly usable, and in fact better in many ways.
Sorbet's DSL of having `sig { }` present is the worst kind of annotation, as it needs to pollute `Kernel` with `#sig`, even when you disable runtime checking.
This is a no-go if you develop anything else than an app (e.g a gem) as you certainly don't want to push Sorbet†† as a dependency to your gem consumers, not the least because `sorbet-static`†† is still only available for `x86_64`. In general Sorbet seems to be designed to type apps, gems being an afterthought.
† https://github.com/jubnzv/virtual-types.nvim
†† or include a monkyepatch that defines an empty `Kernel#sig`
Personally if we do bring types, I would like to see an optional typescript like definition
def foo(:String x) -> String
or similar.
That aside, I'm curious where specifically you found Sorbet lacking?
def [] in Integer => index
# ...
end
def [] in Integer => start, Integer => length
# ...
end
def [] in Range => range
# ...
endThe more you think about it the less arguments there are for "just use Crystal".