Ruby 3.1.0 Preview 1 | Hacker News Reader

Ruby 3.1.0 Preview 1

ruby-lang.org

220 pointsriffraff54 comments

I'd like to see Ruby deprecate some of the rarely used standard library items.

For example, does Rinda and GDBM need to be in the standard lib? Maybe they're better suited as stand-alone gems?

Similarly, I think it should stop bundling some gems.

I see they are moving some gems from 'default' to 'bundled', which I guess is some sort of deprecation or demotion -- that's great!

I'd also like to see them move development onto Github, that should make it easier for more people to participate.

That's been slowly happening [1]. For instance, with Ruby 3.0, net-telnet and xmlrpc were removed from the stdlib and now have to be installed as standalone gems.

[1] https://stdgems.org/removed/

>I'd also like to see them move development onto Github, that should make it easier for more people to participate.

This has been discussed a few times in the past 5 years. But some member refused to use Github due to it not being open source.

Not sure if that is still the case but you could certainly try and ask again.

GitHub is becoming a bloated mess however... if it aint broke don't fix it.

Looks like dbm/gdbm were removed from the ruby repo, just not documented in the release notes:

https://github.com/ruby/ruby/pull/4525

https://github.com/ruby/ruby/pull/4526

https://github.com/ruby/ruby/pull/4619

> I'd also like to see them move development onto Github, that should make it easier for more people to participate.

From: https://www.ruby-lang.org/en/community/ruby-core/#following-...

> The current primary repository of the latest Ruby source code is git.ruby-lang.org/ruby.git. There is also a mirror on GitHub. Usually, please use this mirror.

Enhanced error log messages are always welcome on the ground :)

Reminded me of the work done in elm to explicitly improve their error reporting by deeply considering the user experience [1]

Repo for the newly included error_highlight gem is interesting [2], anyone have more background on this ?

1. Elm “the perfect big report”

https://elm-lang.org/news/the-perfect-bug-report

2. Error_highlight gem repo: https://github.com/ruby/error_highlight

Unrelated to this but i guess the comment reply time expired to your question before I could answer about the dead physicist. No not without doxing myself. I did some asking for you(and me i suppose) though and most likely what he was working on was something to do with decoherence of entangled qbits for quantum computing.

Good to see IDE support for TypeProf type hints[1]. I've been doing a lot of work in statically typed languages lately, and spoiled by how good IDE integration is with the compiler. Ruby has been moving towards static types for a while, but the lack of IDE support has always been the dealbreaker for me to go back to Ruby. The ecosystem needs more tooling like this.

[1] https://github.com/ruby/typeprof/blob/master/doc/ide.md

I'm not sure that the new "valueless" hash syntax is an improvement. From my experience with Typescript, it's a common source of bugs.

But I'm sure that people had similar complaints when compact hash syntax was introduced in Ruby 1.9, and now nobody even thinks about it. So I'm sure I'll survive.

I'm in the opposite camp - I love the short object syntax in JS and the way it mirrors destructuring. I'm very happy to see something similar come to Ruby! But, that said, I think that the Ruby version is ugly compared to JS and the extra colons are noisy. I assume there's some reason why the parser can't read `{ x, y }`, as ruby is notoriously hard to add syntax to.

Count me in as well, very happy to see this addition.

> I assume there's some reason why the parser can't read `{ x, y }`, as ruby is notoriously hard to add syntax to.

Something in the parser using the colon after the first identifier to disambiguate between hashes and blocks, probably.

Was just playing around locally and found one of the undescribed Irb improvements is a drop-down with autocompletion suggestions nice!

I haven't had a chance to try yet, but I'm hoping some of the irb regressions that began around 2.7 are fixed. Try pasting an array or hash with 1000 or so elements. In 2.6.x it's instantaneous, in 2.7.x it can take minutes--

"foo(x:, y:) is a syntax sugar of foo(x: x, y: y)"

This syntax sugar is a nightmare to me. Very confusing and for what? To save 2 keystrokes?

It's only two keystrokes if your parameters are single characters.

  def some_method(with_longer_parameters: with_longer_parameters)
    do_something(with_longer_parameters)
  end

  def some_method(with_longer_parameters:)
    do_something(with_longer_parameters)
  end

Is it to spare keystrokes in method definition, or rather, in method invocation?

Your second definition reads that `with_longer_parameters` is a keyword argument with no default.

Still, you've illustrated the keystroke savings with your choice of parameter names :-)

Yeah, should be invocation, there's a nice example on the feature request (https://bugs.ruby-lang.org/issues/14579):

  def login(username: ENV["USER"], password:)
    p(username:, password:)
  end

  login(password: "xxx") #=> {:username=>"shugo", :password=>"xxx"}

I don't find it that confusing. Parameter names are usually longer than one character.

I highly doubt that any such feature is introduced to save keystrokes.

If you want to save keystrokes, Ruby is probably not your language. And you also really need to invest some time in your editor/IDE.

The sugar is that it removes redundancy and adds DRY:

> Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.

Now, the value for DRY is minimal here: `referer_acceptence_header: referrer_acceptance_header` is quite easy to spot (and there too: if these are your bugs, you probably need to invest some time in your editor/IDE), but avoiding it alltogether in the language is always nice.

While yes, parameter names are usually longer than 1 character, I'm super disappointed that Ruby has introduced this. I hate hate HATE this in JS. Sure, it's easier to write, but my brain always has to do a small double-take as to what I'm looking at while reading as it looks almost identical to destructuring.

Also, how is this going to work in function heads? `def foo(bar:)` means that `bar` is a required keyword. Is it going to have to be `def foo({bar:})`? In that case, it's going to come off as a potential style error to many people.

My bad, the example illustrates in a function head. So how are require keyword args going to work now?

Oh wait, it's function definition vs callsite (duh). This is still unnecessary confusion.

I really wish there was a command to `un-sugar` a script - especially intelligently built into editors.

That way - if you're new to a language and don't want to deal with all the cognitive overhead of syntactic sugar (or if you just find some particular feature really complicated) - you could "un-see" it.

Keyword args are real nice for maintaining API compatibility but they're commonly passed down through the call stack resulting in a lot of:

"foo(arg, my_keyword1: my_keyword1, my_keyword2: my_keyword2, my_keyword3: my_keyword3)"

that now becomes:

"foo(arg, my_keyword1:, my_keyword2:, my_keyword3:)"

that is just a bit less for your eyes to have to digest and parse, although i can see leaving the trailing colon off as being a common bug.

sometimes you can just double splat both the signature and the wrapped invocation, but if you need to peel off kwargs in the wrapper method then you're stuck passing them all back in, or else you fall back to pulling kwargs out of a hash (plus you lose some level of auto-documentation in the method signature that way).

it'd be kind of nice to have a literal which could be passed into the method invocation that would expand out to all the kwargs that were passed, while still being explicit in the definition.

> To save 2 keystrokes?

A lot more keystrokes if you are using meaningful argument and/or hash key names, but the big benefit is, IMO, readability (at least, that's the big benefit I see with the very similar syntax in JS.)

I just don't see the readability arguments. I think it hurts readability in a big way, especially in JS where it can be mistaken for destructuring. I've seen firsthand people who defend this syntax get confused. Sure, it's brief confusion, but if anything introduces any kind of confusion at all, I don't see the benefit. `foo: foo` is verbose, sure, but I don't know how one could argue that it isn't readable. It's about as clear as you can get.

Can anyone share their experiences using YJIT or RBS in Rails?

I’m curious how the impact affects development, deployment, etc.

> I’m curious how the impact affects development, deployment, etc.

YJIT is pretty much transparent in production, if not it's likely a bug.

When we tried MJIT in production to compare it against YJIT, it causes lots of request timeouts on deploy, because the JIT warmup would take 10 to 20 minutes and it's much slower during that phase.

But YJIT warms ups extremely fast and with a much lower overhead, it's seemless on deploy.

The only thing you may need to tweak is `--yjit-exec-mem-size`, it defaults to `--yjit-exec-mem-size=256` (MB) which is not quite enough for larger apps.

As for development, it would work, but with code reloading enabled, you'd likely exhaust the executable memory allocation pretty fast, because for now YJIT doesn't GC generated code [0]. It will come soon, hopefully before the 3.1.0 release, but that's one of the reason why it's not enabled by default.

[0] https://github.com/Shopify/yjit/issues/87

Is it 256 mb for the entire app server or is it multiplied by each Unicorn/Puma process?

Each process, a small part of it might make it into CoW during boot, but that won't make a big difference.

If you don't have the free RAM for it and you app is small enough, you can try lower values.

Also note that currently YJIT fully initialize that memory to make it easier to debug, so it's not even virtual. That's another thing that is on the roadmap to be improved soon, and ultimately you'll only pay for the part that YJIT really use even if 250MB is allocated.

But overall all JITs trade increased memory usage for faster execution, you have to store that generated code and metadata somewhere.

Do u guys expect it to be on by default eventually, at least for Rails apps?

Yes. Basically the main reason it isn't the default now is the still rough executable memory management.

Other than that it's been rock solid and offers decent speedup across a wild range of benchmarks and real world applications.

So my personal expectation is for it to be enabled by default in 3.2, but I might be wrong.

Disclaimer: I'm on the YJIT team.

In general it's not a big change. You should only turn on YJIT for long-running jobs -- the prod Rails server, plus probably background workers if you have them. YJIT does nothing unless you turn it on with the --yjit flag.

YJIT's going to affect memory usage, so it'll change your optimal number of processes and threads for your Rails server - play with it for your app specifically, because the percent speedup and mem expansion vary a lot from app to app. YJIT works fine with Puma and Webrick. I don't think anybody's tried it seriously with less-common servers like Falcon or Thin, but I'd expect it to work -- file a bug if it doesn't, because it should.

YJIT does speed up Rails - we're seeing about a 20%-25% speedup on little "hello, world" Rails apps (see: https://speed.yjit.org/), and about 11% on Discourse for a single thread. We don't have good multithreaded or multiprocess numbers yet, but it's in the works. YJIT scales with multiple threads/processes just like existing CRuby.

(All speed numbers are accurate as of right now, but may change over time.)

I would probably not use it in dev mode. While YJIT has pretty good warmup numbers, Rails throws away all existing application code for every request in dev mode. That's going to make YJIT a lot less useful. Play with it -- maybe I'm wrong for your app, especially if you use a lot of non-reloaded code (e.g. methods inside gems.) But I wouldn't expect great results in the development RAILS_ENV.

For deployment the short version is: add --yjit as a command line parameter in production mode and (probably) for your background workers. You can do this with "export RUBYOPT='--yjit'" or use your local preferred way.

Where I give weaselly-sounding qualifiers like "probably," that's because it can depend on your config. If you have plenty of memory but are often CPU-bound, YJIT is usually good. If you have really limited memory and your server CPUs are mostly idle, YJIT is usually bad. YJIT is also currently x86-only and runs on Mac and Linux but not Windows.

There has been a lot of historical demand for a Ruby config for servers: something that uses more memory to get faster operations, and that is optimised for long-running processes. YJIT is aimed directly at that. Non-JITted CRuby is mostly the opposite: fast startup, modest memory requirements, doesn't get significantly faster over time.

How one would check if the YJIT was correctly enabled?

    $ ruby -v
    ruby 3.1.0dev (2021-11-08T09:35:22Z master 7cc4e147fc) [x86_64-darwin21]
    $ ruby --yjit -v
    ruby 3.1.0dev (2021-11-08T09:35:22Z master 7cc4e147fc) +YJIT [x86_64-darwin21]
    $ ruby --yjit -e 'p RubyVM::YJIT.enabled?'
    true
    $ ruby  -e 'p RubyVM::YJIT.enabled?'
    false

You can call `RubyVM::YJIT.enabled?` to check.

> YJIT works fine with Puma and Webrick. I don't think anybody's tried it seriously with less-common servers like Falcon or Thin, but I'd expect it to work -- file a bug if it doesn't, because it should.

Have you tried it with Unicorn?

Yes, it work fine with Unicorn, YJIT is currently serving a small portion of Shopify storefront traffic, and that app runs with Unicorn.

Of course since each of the unicorn process will generate its own executable code, the memory usage difference with Puma is even bigger, and copy on write can't help here.

Im out of my depth here but why can't they share the JITed code?

Because JITed code will inline things like the address of some specific objects (typically constants) etc. To share code between processes you'd need to ensure all these references are exactly the same in each process.

Yeah, I was trying to come up with an idea for this. We try to eagerly load as much as we can before fork, so most of the memory is shared between the forked unicorns, but this won't work with YJIT right?

Ideally we would be able to eventually do the forks of a "mature" unicorn child an hour or so after all is JITed...

Beside the operational nightmare that it would be to deploy with such a strategy, it wouldn't work long term.

JITed code can be invalidated and recompiled, so your forks would still drift over time.

I'm a big proponent of unicorn (and forking setups in general) for various operational reasons, but I think JIT might be the last nail in the coffin.

can we expect for better numbers on the official 3.1 release, or we need to wait for 3.2 version?

Swapping out a JIT must be an unimaginably complex operation.

It's not swapped, both are available for now.

Not really. Separate out your JIT into well defined components (compiling a method, installing the compiled version, invalidating the compiled version, deoptimising, etc.) and make sure your JITs use those common interfaces. The JVM manages to support multiple JITs (including Graal which is written in Java) and multiple GCs through this sort of approach.

No news on Ractors improvements :'(, maybe next year

Those error log messages will save me time--super excited about that!

Very confused by the example of the "pin" operator given here - what is it for, and what is the improvement in this release?

Definitely not a clear description, but I believe it stands for "pattern matching in". Ruby's new pattern matching uses a sytax of

case {{ value }} in {{ pattern }} {{ code }} ... end

the new feature apparently allows expressions, rather than just local variables, in the pattern spec.

They probably should have included the link to the feature ticket: https://bugs.ruby-lang.org/issues/17411

https://docs.ruby-lang.org/en/master/doc/syntax/pattern_matc...

Nice, compiling it now. Will be fun to play with YJIT and try the new features.

Reply on news.ycombinator.com