Comment by jbackus | Hacker News Reader

Comment by jbackus | Hacker News Reader

Ruby's parser is notoriously complex; if I remember correctly, only a few members of the core team even know how to maintain it without introducing regressions.

The craziest part of this is that Ruby does not provide a full featured Ruby parser, so its entire static and dynamic analysis ecosystem depends on a (actually very high quality) 3rd party parser, begrudgingly maintained by someone who (AFAIK) doesn't even write Ruby anymore: https://github.com/whitequark/parser

When I see new language features like this, I think of how Ruby's entire tooling ecosystem depends on the dramatically underfunded (and therefore primarily goodwill) efforts of high output maintainers like whitequark and a few others. Ruby's highly dynamic and untyped nature means these tools are all the non-runtime guarantees you can get, basically. Epitome of digital infrastructure.

Consider asking your company to fund some of these people:

* https://github.com/whitequark (maintains parser)

* https://github.com/sponsors/bbatsov (maintains RuboCop)

* https://github.com/sponsors/mbj (maintains unparser and mutant)

---

As context, I know this stuff intimately because I used to contribute heavily to most static and dynamic analysis tools in the Ruby ecosystem (https://github.com/backus?tab=overview&from=2017-12-01&to=20...) and used to track new ruby changes really closely: https://cognitohq.com/new-features-in-ruby-2-4/

I helped out with parser as well for a while, though it was mostly documentation, triaging issues, that sort of thing. I think I still have push access to the Gem, as at the time the project had a bus factor of 1 (= whitequark).

Anyway, I remember just how frustrated whitequark would get every time CRuby decided to make some random syntax change. I have a lot of respect for the current maintainer(s) for putting up with the ever growing complexity of the Ruby syntax. I hope Ruby stops changing the syntax on a regular basis, but I doubt this will happen any time soon.

Yeah, I don't remember any of the Ruby tooling ecosystem maintainers having nice things to say about the various syntax changes they've added over the years. I don't blame them.

These changes are always justified with a hand wave and a reference to "developer happiness" and ergonomics. I think that justification is seen as insulting when all the static/dynamic analysis tools obviously actually deliver a lot of value to Ruby devs, and yet the basic tooling to support this ecosystem (robustly parsing, traversing, annotating, rewriting, and unparsing an AST) is not provided by the language.

`parser` is indeed a fantastic gem. @whitequark is the initial author but today it is maintained by @iliabylich

Is Ripper dead? I attempted to do some minor work on Ripper and it was profoundly painful. My explanation for Ripper boiled down to a domain specific language in parse.y comments that gets translated into C via a very confusing Ruby script, which manually calls Ruby VM functions, that wait for it, are dynamically defined. This constructs a parse tree. If Ripper is still getting maintained, I'm seriously impressed.

Yeah. Been years since I even played with it, but even in 2016 I remember Ripper being completely unusable, failing to parse plenty of normal Ruby syntax. The Parser gem also maintains a separate parser for every Ruby version, so you can a program using Ruby 2.5 but tell it to parse code according to Ruby 2.2 syntax. It's also just way better built, tested, and has other nice features like tools for rewriting ASTs.

Reply on news.ycombinator.com