Mastering Ruby Code Navigation: Ruby LSP Enhancements in the First Half of 2024 | Hacker News Reader

Mastering Ruby Code Navigation: Ruby LSP Enhancements in the First Half of 2024

railsatscale.com

140 pointsksec67 comments

Ruby has a lot going for it, but as other commenters point out, the metaprogramming nightmares of the language have held it back 10-30 years behind modern language ecosystems, depending on the feature you're looking at. Celebrating "jump to source definition" (sometimes working) for such a mature language is a symptom of the nature of the language. Sometimes insane dynamic freedom is really useful, but it comes with heavy drawbacks.

As someone who uses Ruby as well as plenty of other languages, I agree that it can feel awkward not to have "go to source" work reliably, but in the greater scheme of things, it's not a big drain on time to do the occasional full text search or look something up through programmatic introspection. In other areas that might really slow you down or become blocking, Ruby's ecosystem still seems OK to me. I'm thinking package management, compilation or transpilation, availability of battle-tested libraries etc.

I guess it depends on scale as well. The bigger the codebase, the nicer it is to find implementations and references automatically, and the bigger the company, the more likely they are to have good workarounds for the shortcomings of various language ecosystems or even dedicated dev experience teams, in which case Ruby's potential runtime intricacies might really start to weigh it down. Then again, there seems to be some movement away from that all-too-dynamic stuff e.g. in the rails codebase.

> it's not a big drain on time to do the occasional full text search or look something up through programmatic introspection

Okay, this won't probably be 100% on topic, but in my experience how well these methods work is inversely proportional to the size of the codebase and also quite badly in some cases depending on how the code is written.

The other day I was working on a Java enterprise codebase (monolith, ~300K LoC) and I shouldn't have had that many issues navigating the codebase, however someone made a validator that resolves the logic to a specific class at runtime. So you'd basically have validator.validate(Object someObject). Suddenly if I wanted to find all of the places where validator.validate(MyObject myObject) is called, I could not do that easily and with naming like validator.validate(entity) all over the place, text search didn't help much either. They had an okay type system but chose not to use it, just because they wanted to pass in arbitrary objects, without regard for how easy finding usages will be in the future.

My point is, that you can probably write code that's easy or hard to navigate in any language (within reason), but I'll gladly take whatever tools I can get in any stack out there!

> however someone made a validator that resolves the logic to a specific class at runtime. So you'd basically have validator.validate(Object someObject).

The pains of not having multiple dispatch.

I don't think metaprogramming is the key point here, though it doesn't help with this kind of issue.

That is, yesterday I spent the afternoon trying to follow a maze of signal observers in a Vue3 project. After a moment, not finding why the redirect to login page was nowhere to be found in the signal obsevers of the concerned transition, it finally reveled to be linked to a more broad route hook mechanism that the framework provides.

No meta programming is involved, be it in signal observers or route hook.

Regarding metaprogramming, Crystal keep only the parts that are straightfoward to deal with in static analysis, from what I grasped when skiming its elevator speech.

On Ruby side the great stuffs I like in it are more linked to the homogeneous approach it follows. Almost everything is an object and something like `42.extend(:custom•module).original•action` is possible due to that. It's not 100% pure object though, reserved keywords like `if` can not be used as objects, that is `if.class` is not valid.

I’m very curious how a language that’s not yet 29 years old can be held back 30 years.

Makes perfect sense to me, it just means the language was already a year behind when it launched.

Which languages were ahead at that point? Haskell started in 1990, so was it only a year ahead after 5 years in existence but since then it has gone 30 years ahead and Ruby hasn’t moved forward at all, or is has it gone 30 years into Ruby’s future after another 30 years?

Or is it that the original comment showed no knowledge of programming language history? It’s a tough choice!

In emacs, there's robe mode which I found to work very well. It keeps a Ruby process running with your code loaded in it.

I wonder why no one's written a Ruby LSP with this approach rather than relying on static analysis.

https://github.com/dgutov/robe

The ruby-lsp project made by Shopify has (or had...) a rails plugin that worked this way. It adds some routes to your server during development and uses them to introspect the running process.

The fact IntelliSense and jump-to-source are supported at a very superficial level goes to show the heavy drawbacks of dynamic types.

Sorbet may fix things, but at that point, just use a language with more mature tooling around types, like Python or TypeScript.

Dynamic types offer dubious marginal benefits but bring tons of downsides. The demonstrations in this article reflect that.

I like coding in Ruby a lot more than Python or Typescript. From using the Ruby LSP in the article daily (though I am not on the team) I can say it works quite good and I have very few issues with navigating Ruby source. If I were very worried about types, going to Python or Typescript seems like not a very good solution. Python barely has more type support than Ruby, and Ruby is making a lot of gains in this area. Might as well go to an even more expressive and powerful language like F#, Haskell, Ocaml, Scala, or similar.

> Python barely has more type support than Ruby

That's not true. mypy and Pyright (the main type checkers) are very ahead of Ruby's Sorbet, and Python actually has proper ergonomic syntax to work with types, which Ruby doesn't. It's also seeing constant development to bring more powerful typing and syntax.

Also, typing has become part of the Python culture, which means most of the big libraries have type hints. It doesn't seem like that is the case with Ruby, where type hints aren't as prevalent. Even the core maintainers don't like the idea and want to keep it as dynamic as possible.

I've had the hardest time getting mypy to work well in vscode. Trying pyright with neovim now, curious how it goes.

pyright just requires the Microsoft Python extension. In my experience once you have that it works instantly.

I don't know Pyright, but is it still Python? It's probably fair to bring it as Sorbet is mentioned, if course. But you can't use Sorbet using a pure ruby interpreter, if I'm getting it correctly, and I guess it's the same for Pyright. It's that so?

Mypy/pyright aren’t interpreters. They’re basically just linters that you run against Python code to see any typing issues. I don’t mean that dismissively,they’re great tools.

Pyright and mypy are typecheckers, completely orthogonal to the interpreter. They work like linters that statically perform type checking. By default, Python's type hints are ignored at runtime (i.e. you can do `x: str = 2`, and the runtime doesn't care), but there are tools to perform runtime checks (e.g. beartype) and libraries that make use of types as metadata (e.g. dataclasses, Pydantic, FastAPI).

Instead of Pyright use Based Pyright for anything that isn't Code, and in Code (not Codium) you can use Pylance instead.

https://github.com/DetachHead/basedpyright

Ruby has practically no support in a basic tool like VS Code. (Love Ruby btw)

I've tried the VS Code(ium) Shopify Ruby LSP plugin, and it's just BAD.

Have you tried the 'solargraph' VS Code(ium) plugin? I've found it to be pretty good. If you haven't do give it a try, but do know that you need to build the YARD documentation for installed "gems" for Solargraph to not be dogshit.

As described here [0], you run 'yard gems' to build said documentation. Do note that the method of having this documentation always be generated doesn't seem to work with "gems" installed with Bundler. Very frustrating, that.

[0] <https://solargraph.org/guides/yard>

Thanks, will eventually give it a try.

Or the fact that people continue to do a lot of development in these languages would suggest that the benefits are more than marginal, and the lack of a few editor features is not such a terrible hindrance.

Strongly typed languages have a higher barrier of entry and require an engineering mindset. That's anecdotal but if I think of exceptionally competent people I've worked with on JS projects, all of them have spent time building and advocated for properly typed code bases.

The other camp "just hates it" because it "slows them down", as it seems they spend most of their time fighting the types but never get to the point where you get that huge return on investment.

I don't know, the ergonomics of the type system is not the same in all languages. A tool chain that report early useful feedbacks in a non cryptic sentences will certainly gains adoption quickly.

Unfortunately most of the time the result is at best needlessly cryptic, with zero thought about simplicity of use. Ruby has a reputation of taking care of this topic.

I've been working with types and malloc for years in C, then enter Java. No need to malloc anymore and everything worked. Great, goodbye C. Then enter Ruby, no need to write types anymore and everything worked. Great, goodbye Java.

That's the great picture. Looking into the details I've been working with Perl, JavaScript, Python plus many other less common languages. I always had a preference for languages that hide complexity away from me.

Code completion really doesn't matter much to me. I've been working for maybe ten years with an emacs package that remembered words and attempted to autocomplete with the most used suffix. It worked surprisingly well.

In my professional experience, types are a godsend for large, and/or long-running projects that have been worked on by many people over the years. They reduce complexity by informing you up-front of the shape of the data and/or objects that a function demands and produces.

If the type-checking system is decent, they also automatically catch a whole class of problems that will only show up at runtime, and may take years to surface. (Ask me about the error-handing code that detonated because it contained a typo in a Ruby class name, which was only discovered three years after it was introduced... when that code was actually exercised during an... exciting outage of a system it depended on.)

> types are a godsend for large, and/or long-running projects

Agreed. But that doesn't mean that every language needs to be statically-typed, which seems to be where we're heading nowadays.

IMO large and/or long-running projects should be written in languages with sound static type systems, not scripting languages with types tacked on. Conversely, I often work on projects which are neither large nor long-running - for those, a dynamically-typed scripting language works perfectly well.

> a typo in a Ruby class name, which was only discovered three years after it was introduced

So the code containing this typo was never tested? That's asking for trouble even if you have static typing.

> So the code containing this typo was never tested?

The code absolutely was tested. However, (obviously) not every possible path through the code was tested.

Given a long-enough timeline, you will NEVER remember to test every little thing. Given sufficient code complexity, it can be next to impossible to actually exercise every code path with hand-written tests.

That's one of the troubles with large projects written scripting languages like Ruby... you have to write an assload of tests to replace what you get for free in languages (even "loosely"-typed languages like Erlang) that have pre-runtime type-checking (whether or not it's provided by a compiler).

> Conversely, I often work on projects which are neither large nor long-running - for those, a dynamically-typed scripting language works perfectly well.

Oh yeah, for small things such languages are A-OK. I 1000% agree with that. The big problem (that you may never encounter because I bet that you're smarter than the average unending parade of Project Managers) is how often small projects are forced into becoming big projects, and then huge projects as time goes on.

I’d add to this that there’s a good reason the testing culture is so strong in ruby: you absolutely need to write tests for every last little 2-line method to make sure you did it right. With no compilation or type checking step, there’s no other way to know if your code is even valid.

Which means that IME a huge number of tests in my ruby code were doing things that a type checker does automatically: make sure that passing nil makes it return nil, making sure that passing an array does the right thing vs a string, etc etc etc.

I have very vivid memories of working in a large rails code base and having no idea whether my change would work until I actually exercised the code and realized I was completely messing up the contract that some method expected, so I had to slow down and write tests to validate my assumptions.

Fast forward to now, I work in a large Rust code base, and it seems like 99% of the time, if my code compiles, it works flawlessly the first time. So I concentrate on writing integration/functional tests, which give me a lot more bang for my buck than having to write hundreds of little unit tests. (I still write some unit tests, but the need for it is much less… the vast majority of things you’d write tests for in Ruby are things the type and borrow checkers handle for you in Rust.)

> ...there’s a good reason the testing culture is so strong in ruby:

"Strong" as in "You easily burn 10x more time writing tests as you do code... and not because it's difficult to think through how to write good tests"? If so, yes.

"Good"? Hell, no! That's a bad reason.

> ...a huge number of tests in my ruby code were doing things that a type checker does automatically...

The folks I work with demand that we don't write these tests, so they don't get written. Guess how often code detonates in production because of things a typechecker would have caught... despite the enormous volume of test code.

To be crystal clear, I totally agree with your statements in this comment. I started my "career" with C++ and I'm so damn glad I did. Had I started with Ruby and Rails, I would have come to the conclusion I was far too damn stupid for this field and left to become a lawyer.

“Good” in this context didn’t mean “this is a good situation”, but rather “if you’re using ruby, it would be very bad if you didn’t write tests”, and “bad if you don’t” can be roughly reworded as “good if you do”, at least if we’re presupposing that you have to be writing Ruby.

> Strongly typed languages have a higher barrier of entry

Agreed

> the other camp "just hates it" because it "slows them down"

I’ve no doubt there are some that fall into this category, but not everyone, not by a long shot.

The problem is that Rails is the super power that keeps Ruby alive (in my personal opinion). I’d love to use a Typescript solution that has the power and opinionated style of Rails, but there’s nothing comparable. I think AdonisJS wants to be that, but it’s not battle-tested enough for me to feel comfortable relying on it for production projects.

It's really the other way around, Ruby is the superpower that makes Rails what it is. There are very few (if any) web frameworks comparable to Rails because Rails is only possible due to the dynamism and expressiveness provided by Ruby.

Can you provide some examples? At least from my point of view, Rails’s power comes from its opinionated convention over configuration approach and its batteries-included philosophy.

People talk about the DSLs and conciseness, but frankly those don’t make/break Rails for me.

I’m curious to know what part of Rails wouldn’t be possible in something like Javascript.

In terms of convention over configuration, Ruby's metaprogramming is crucial for the name-based class conventions that make Rails so convenient. For instance, if you have a class User with the attribute name, at runtime Rails dynamically adds methods to the User class, allowing you to do things like User.find_by_name("Alice"). Rails uses Ruby's metaprogramming capabilities to accomplish this.

The same is true for defining associations between models. Consider the following model:

   class User < ApplicationRecord
     belongs_to :organization
     scope :active, -> { where(active: true) }
   end

You mentioned that DSLs aren't important, but the DSL for defining associations is extremely convenient. Adding the belongs_to line defines numerous methods on the User class, allowing you to do things like User.last.organization. The scope line dynamically defines the active method, allowing you to call User.active, which executes the code in the provided block. There are also DSLs for defining routes and migrations, which are extremely important when using Rails.

Ruby's metaprogramming is also utilized in Rails controllers to load the correct code. For example, the index method in the UsersController knows to look for the views/users/index.html file because it can reflect on the method and the class to determine where to load the correct code using Rails' autoloader, Zeitwerk.

Ruby's open classes and monkey patching (something that should be done with extreme caution) allow ActiveSupport to add extremely useful methods to Ruby's base classes. This enables conveniences such as Date.today - 10.days or DateTime.now.beginning_of_day.

These are just a few examples that I can think of off the top of my head, but nearly every major convenience in Rails relies on the flexibility provided by Ruby. While it might be possible to reproduce many aspects of Rails in JavaScript, I don't believe you could replicate all of it with the same readability and intuitiveness that Rails offers.

>In terms of convention over configuration, Ruby's metaprogramming is crucial for the name-based class conventions that make Rails so convenient. For instance, if you have a class User with the attribute name, at runtime Rails dynamically adds methods to the User class, allowing you to do things like User.find_by_name("Alice"). Rails uses Ruby's metaprogramming capabilities to accomplish this.

This can be accomplished in statically typed languages too. For example, Java has an ORM called Cayenne that generates models based on database tables. You can get similar functionality and ensure you're not accidentally calling something like `find_by_nam`

ActiveRecord doesn't generate models based on database tables, it uses conventions on table and attribute names along with Ruby's metaprogramming capabilities to define behaviour on models automatically. From my understanding, Cayenne requires manual mapping of the database to your models, which is done via XML files that are either handwritten or generated by their GUI application and have to be regenerated on every database change. That's a far cry from the ease of use of ActiveRecord. How you value that ease of use versus your desire for static typing is up to you, but they’re not comparable from a functionality point of view.

And like I said above while you might be able to recreate some of Rails' functionality in other languages, you wouldn't be able to replicate all of it with the same fluidity and readability that you have in Rails

Not everyone is at a point where they are greenfield picking a language for a new project. Sorbet was built by a large organization (Stripe IIRC) and is used effectively by organizations with large Rails codebases. I think Sorbet is a great way to maintain velocity on a large Ruby codebase after the initial velocity benefits of dynamic typing have ceased and the dynamic typing is actually a drain on velocity.

There is no strong evidence to back up your claims, just opinions.

> Sorbet may fix things

Sorbet is very slow for large codebases. I keep checking it from time to time, but the biggest service I'm dealing with just fails to run through the initialisation. And that's ignoring the idea that someone will have to clean up the result. Not holding my breath for it to be functional in LSP situations.

How large is large here? I'm using sorbet on fairly big project and it takes like 3 seconds to scan the entire codebase with "srb tc", and with their LSP in editor the diagnostics refresh almost immediately before I stop typing. That is, without even using Sorbet's cache

I was tired of feeling slowed down by grepping in large Ruby code, and the artificial boundary gems create requiring a `bundle open` to keep digging. I created my own Ruby LSP to solve these problems: https://github.com/pheen/fuzzy_ruby_server

I think it works well, is stable, and I would still reach for it over Shopify's LSP personally. Feel free to use it! Though as a heads up, I don't work in Ruby much anymore unfortunately so I'm not actively working on it or accepting issues.

I used to work at a Ruby shop. As a language, Ruby can be elegant despite the overall weirdness, and some of my coworkers were really productive with it.

On the other hand, the Ruby community seems stuck in the 2000s, with some of the self inflicted Ruby pains and attitude being fairly similar to what some Python devs were doing in the late Python 2 era.

These days, I still like Ruby and I would enjoy building some little project with it, but I would rather make a living using other stuff.

Curious if you have any examples of these pains? I'm not familiar with the late Python 2 era but I've been writing Ruby for a long time and wonder if I'm doing some of these as well.

Yep, it suffers from hipsteritis and inability to adapt consistently and sensibly to prefer the greater net good, and instead randomly prefers awkward, more troublesome workarounds. It's too much like a Perl 2.0 in some respects.

How does this compare to the JetBrains solution?

Ruby is nice, I just find the DX to be far behind the other languages. Like code completion is barely working.

Copilot is pretty good now. It will be interesting to see how much classical code completion matters in the long run and which languages will work best with AI.

A solution without ai would be nice

Why?

I have to provide a reason why I am not interested in integrating ai into my workspace?

I'm aware of the limitations of a dynamic language in terms of the looking up references and implementations of types/functions.

For everyone commenting about those and who haven't, I suggest trying RubyMine to set expectations on what is possible first.

> I suggest trying RubyMine to set expectations on what is possible first.

In my professional experience with large, very long-running Ruby projects, RubyMine will at least once per session (and usually much more than once) give you search results that are no better (and sometimes worse) than if you'd just run `grep` on the codebase. (e.g. "I'm pretty sure ONE of these five hundred 'run' functions is the one you want! Oh, by the way, I also am showing you results from every .gem file I know about!")

It also has a bias for presenting results quickly and allowing you to make queries quickly... so quickly that it often returns partial results to queries with absolutely no outward indication that the results are incomplete.

Given what it seems that the other Ruby "Intellisense" plugins are having to do to well-understand real-world Ruby projects (especially those who use Rails and friends), I'm sure that a godawful lot of manual work and special-case knowledge has gone into RubyMine's Intellisense. But, it's still not nearly enough to be reliable on large projects.

I thought they have a little indicator that says it's a partial result set?

> I'm pretty sure ONE of these five hundred 'run' functions

Yes. Same thing with all unannotated dynamic languages. Like Smalltalk has a bazillion `#value` and `#value:`s, but there were options to scope it down. You can do that in RubyMine, but I forgot if you can do it by class/gems without manually creating a scope.

Just saying that such navigation isn't impossible with Ruby.

> I thought they have a little indicator that says it's a partial result set?

There might be one (one would think there WOULD be one), but if it does exist, it's so small and unobtrusive that I've never noticed it. My coworkers who use RubyMine every damn day don't seem to notice it either. If it exists, it needs to get much more obvious, IMO.

> ...but I forgot if you can do it by class/gems without manually creating a scope.

I know that my RubyMine-loving coworkers definitely don't do whatever this is and just power through the worse-than-grep results.

> Just saying that such navigation isn't impossible with Ruby.

Fully-automatic trouble-free navigation of sufficiently-large/complex projects is pretty much impossible with Ruby. (At least in a Ruby project that lacks comprehensive, correct type annotations.) I'm just saying that folks who might take RubyMine for a test drive on a small or toy project will be unlikely to get an accurate demonstration of its limitations.

(In my professional experience, RubyMine is not THAT much better than the 'solargraph' VSCode plugin.)

> but I forgot if you can do it by class/gems without manually creating a scope.

You can choose and edit scopes with these 2 buttons — https://imgur.com/a/jZute6v

Ah! I've definitely seen my coworkers use that dialog, so I fully expect that they're aware of (and use) that button.

Rubymine is the only editor I’ve ever used that gets the code navigation consistently right without any config. You can even navigate to dependency implementations. It’s invaluable with understanding really large code bases.

Excited to try this to see if it measures up.

I started a small service recently in rails. I'm forcing sorbet with typechecking (you can still set things to false like tests which sometimes are a pain to get to pass). It feels almost like a typed language with LSP jump to etc. It's getting there.

How do you like using sorbet? Does it feel natural? I’m also running service on rails and considering trying to integrate sorbet but worry about the syntax

Not the OP but I started working with Sorbet in a Rails codebase 6 months ago.

Learning it is easy and using it is also easy if you stay away from: 1. Meta programming 2. Methods that are returning different results based on some conditions.

the second case it can be handled by Sorbet but it makes the type more complex.

I find this a nice side effect of Sorbet: - the moment I start fighting the types or feeling to add many T.any it is a signal to me that I should split my methods/objects

Some things that I found useful:

- I started to like T.enum, T.struct and somehow I am feeling them missing now in a normal Rails project.

- I also like T.let as it helps with object shapes and memoization.

- It also solves the inheritance and there is no need to discuss what exception to throw when you want to define a method that should be implemented in

- I would recommend the gem sorbet-results that adds a simple typed monad

- It helps a lot refactoring

To get the full benefits always try to make your files with typed:strict and of course use tapioca with Rails

Thanks for the opinion! I work with type annotated Python day-to-day so I'm excited to have that sort of type safety with a tool like Sorbet, sounds like I should check it out sometime soon.

One drawback of Ruby code completion is that it tends to require partial execution of code that may contain side-effects, and this is inherently dangerous.

Writing my own Ruby code is fine but debugging others code is a PITA.

Possibly off topic:

"Look up the source" in 2024 is a symptom of a wider problem. Ruby was innovative in the 1990s but it is starting to fall behind the times.

Those of us who are old enough to remember the euphoria that greeted the release of Ruby on Rails in 2004 are mostly surprised to see Ruby's current, limited market share. There was a moment when it seemed like Ruby was going to sweep the whole world of software. But that didn't happen. The mutability that made Ruby so fun also makes it difficult to maintain Ruby codebases over the long-term.

For awhile Ruby seemed relevant in fields beyond Web development, such as devops. I recall 12 years ago a startup tried to recruit me, they were going to revolutionize devops by building automations for Chef. But the metaprogramming in Ruby made it less than ideal for devops work. That startup failed. In most places I now work, a combination of Ansible and Terraform dominate, save where the company uses the constellation of tools surrounding Kubernetes to manage everything. The places that use Chef have been using Chef for 15 years, no new company chooses Chef today.

There was a stretch after 2004 when Ruby was trying to break into new areas. GTK bindings for Ruby were developed and some people experimented with desktop Ruby. But that never took off.

Many of us who loved the metaprogramming in Ruby, but wanted a better structure for that metaprogramming, moved on to other languages such as Clojure.

Some of us loved Ruby because it allowed easier polymorphism than Java did -- after all, you could use higher order functions to produce functions, but without having all the ceremony of building a Factory. But folks who just wanted the polymorphism of functions returning functions eventually moved to NodeJS.

Ruby always felt fluid and magical in a way that made it seem vaguely unsuitable for data science, and eventually Python came to dominate data science.

The Ruby community witnessed the conflict between Rails and Merb, and their eventual union, and eventually Rails became a comfortable framework for quickly generating an API. But for anything that needs high levels of write throughput, no one would chose Ruby, and so other languages have taken some of the market share.

Even on the Web, some big companies (Twitter, AirBnB) either gave up on it completely, or now only use it for templating the frontend, or offering an API to support the frontend. For backend processing, many companies have moved away from it.

In short, Ruby has been losing market share for awhile. In some fields, such as devops, where it used to be competitive, it has almost completely disappeared. It survives only in Web development, and only because of Rails.

None of this can be very inspiring for Ruby developers.

Reply on news.ycombinator.com