Ruby “Thread Contention” Is Simply GVL Queuing
island94.org
island94.org
I’ve made peace with python, accepted that I need to use it, but I hate it. Death to for loops.
Already in early 2000's CERN was scripting their builds, Fortran and C++ tooling with Python.
Their Grid Computing tutorials, predating what we now call Cloud, already used Python.
If you just use it as a glue language to call out to other things, prepare data, and iterate over results: it's pretty clear and concise.
(I do personally like the reflection-magic in a lot of Ruby apps, so I might be overlooking something that feels normal to me, but is some very weird behaviour to anyone else!)
Ruby does well with metaprogramming magic because most of the languages own keywords and concepts can be rewritten at runtime. They’re implemented AS Ruby code.
I would say it’s logically consistent because there isn’t as many « special » things. Python’s def keyword is special. You can’t make your own, nor can you hook into it. In Ruby, that’s just a message being passed to the current scope’s object. You can do anything you want to it. Ruby plays by a pretty consistent rule book.
I'm sure there are minor exceptions or arguable cases but that's also the case with Ruby. Most infamously with Proc.
Classes are objects in python, usually instances of class called type or a subclass of type(usefulfor controlling class creation), this is called the metaclass. A constructor call MyObject() is basically just the __call__ method of the metaclass.
Modules are also objects usually of type types.ModuleClass or a subclass although really you can stick any object in sys.modules dictionary and have it act like a module and import it.
Sure it doesn't use messages instead attributes whether field or methods both go through the same lookup method __getattribute__ which goes up the class inheritance chain in an OO manner. There's also __getattr__ for more method_missing like access instead of overriding every object attribute access.
A function or lambda is an instance of function class although I don't think you can't modify or override the function class I think you can hook it to log it.
You can also do all sorts of stuff by hooking module import and rewriting the ast. Or more easily you can decorate/wrap the functions in another function object.
It's true that you can't modify c based classes but you can do a lot of magic in python, people usually just don't because unnecessary magic is frowned on. But sometimes it's useful like the way pytest rewrites assert a==[1,2,3] to print a nice error message with both variables https://github.com/pytest-dev/pytest/blob/main/src/_pytest/a...
There's was recently a cool article on things to you can do with python decorators
https://news.ycombinator.com/item?id=42918846
Basically the python function is created as a regular python function but then passed to a jit decorator
@something.jit def some_func(): ....
Which takes the source code and uses it as a dsl for something else compiles it say something to run in the GPU and assigns it the name of the original function. Technically you haven't hooked creation of of the func just replaced it with a wrapper or in this case a related function that doesn't call the original but it does the job.
Unless you're talking Rails, which indeed uses a lot of metaprogramming under the hood and is quite different than programming in basic Ruby.
I’ve been writing Ruby for almost ten years now, and I honestly have no idea what these refer to.
2. Basically Ruby code seems to favour magically generating methods and variables based on other strings, which makes them impossible to search for.
Other languages sometimes do that too, e.g. ill-advised __dict__ Python tricks, or with macros in C++ or Rust. But it's definitely worse in Ruby, and it's a common complaint (search for "magic" in these comments).
I literally have given up following Gitlab's code before. That never happens in better languages, e.g. I can easily follow VSCode's codebase (Typescript) or gitlab-runner (Go).
You might be talking about Rails here and not Ruby per se.
In Asciidoctor, which is unfortunately written in Ruby, where is `convert_inline_anchor` called from?
https://github.com/search?q=repo%3Aasciidoctor%2Fasciidoctor...
Nowhere apparently!
Until you eventually find this monstrosity:
https://github.com/asciidoctor/asciidoctor/blob/8d3f6230c964...
Are you referring to Ruby, or specifically Rails?
It's like seeing a c++ codebase riddled with macros. Is c++ directly responsible for the madness? Probably not, but the tooling allowed it.
And Ruby takes a double whammy on this front because Rails was really what drove the popularity. Hard to describe the frustration of hitting a very large Rails codebase and literally not even being able to find the definition for a class because it's got a fairly generic name and is getting auto-loaded in the bowels of the codebase by someone who thought they were clever like 7 years ago.
It's like a special version of DLL hell. Or all the pains of global window/self vars coming from script tags in JS, but at least that gave some breadcrumbs, and the ecosystem is generally moving away from it even if ESM is still painful in a lot of ways.
Either way - I don't like hidden things. I'd much rather see the sausage get made then try to save a couple lines of typing, and the older I get, the more I judge languages on the simple metric of "How well does text-based search work for finding relevant code?". Rails performs really badly on that metric.
In Ruby, you always use the global name (with caps) that normally matches the library with possibly some nesting. (exceptions exists, but its also possible to add globals in Python)
Unless you are talking about include, but thats for mixins, which are snippets of reusable coee you can add to you class.
It doesn't feel at all as dirty as `from blah import *`.
The problem with DSLs is that you now have a different language for each project you have to maintain.
Each time you come across some little DSL that shortens the original writers work by a few lines (at most), you have to read it carefully, and in depth, to understand what was intended.
I'm turned off by Ruby for this reason alone. If I want the ability to throw little DSLs all over my code, Lisp does it better and more readably.
I'm not saying this to attack the language, just pointing my current mental picture to ask: what's the magic everybody loves? I know there is something there because it's an almost unanimous view among ruby devs, so I want to get it.
Ruby makes message passing first class. That just changes how you think about programs. In exchange you give up passing functions so our anonymous functions are our blocks (actually just another object that can receive messages). So you don't `list(something)` you `something.list` and that lets you change what `.list` does depending on who `something` is very easily.
Ruby's defining feature is that the line between language author and program author is razer thin. You can completely change the language yourself by extending core classes. This is why we have `1.day` in Rails even though its not supported in the main language. DHH (author of Rails) could add that without consulting Matz (author of Ruby). So lots of stuff gets prioritized to make developers happy because its easy to add.
In Ruby the messages you receive are passed up the chain of ancestors. Your ancestors are a linked list of classes going back to a root class like `Object`. You can modify this linked list at will with mixins and inheritance with complete control (should I go before or after this other receiver).
Ruby's REPL and debugging experience is amazing. I often keep one tab with an `irb` or `rails console` open to sketch things while developing the code for it elsewhere. I'm also always inside a debugger to figure things out. When I'm in Rust or Python I'm met with a very different environment.
You’re probably talking about the ability to reopen (or “monkey patch”) classes (which is why defining the same class again can’t be an error). That’s a powerful tool if you use it carefully, especially for debugging or hacking on someone else’s code.
> blocks feel like a more cumbersome/limited version of lambdas
Yes, that’s basically what they are. You could totally write JS-like code and pass even multiple lambdas to methods. However, the most common use case — passing a single important lambda (e.g. to a “map” or “each” method) — gets special treatment and its own language feature.
> what's the magic everybody loves
For me it’s the “batteries included” factor. Ruby can do a lot out of the box and most of the interfaces are well designed with tons of useful methods with descriptive names. It feels like someone cares about my experience and is trying to anticipate what I might actually need.
Also the object oriented mechanisms are easy to understand and don’t feel weirdly bolted on like in Python.
Python’s “new gil” (it’s some 15 years old...) uses a similar scheme, with a much lower timeout (5ms? 10?), but it still suffers from this sort of contentions, and things get worse as the number of CPU threads increase because they can hand the gil off to one another bypassing the io thread.
I don't imagine the overhead will be too bad though, so it may decrease latency while keeping throughput essentially the same.
Of course you'll want to benchmark on your specific load. For example, at my day job it'd make no difference because we don't use threads, each Passenger worker has just one thread and handles one request at a time.
Well, _scaling_ is queuing. But other things: cache hit rates, data packing efficiency, efficient use of hardware intrinsics, etc, don't relate to queuing.
* An arrival rate of requests * A processing time for each request * A corresponding queue length when a new request comes in before the current one is done being processed.
So anything that affects processing time affects queueing. Cache hit rates, data packing, etc., all affect processing time, but processing time is in the context of a queue.
Algorithmic complexity affects the processing time of each request / task. Knowing the latency distribution is crucial in understanding the queueing characteristics of the system, and the algorithmic complexity will greatly affect that distribution.