Hacker News Reader

Comment by btown | Hacker News Reader

> Rails has started to show its age amid with the current wave of AI-powered applications. It struggles with LLM text streaming, parallel processing in Ruby, and lacks strong typing for AI coding tools. Despite these constraints, it remains effective.

A plug for Django + gevent in this context! You have the Python type system, and while it's inferior to TypeScript's in many ways, it's far more ubiquitous than Ruby's Sorbet. For streaming and any kind of IO-bound parallelism, gevent's monkey-patches cause every blocking operation to become a event-loop yield... so you can stream many concurrent responses at a time, with a simple generator. CPU-bound parallelism doesn't have a great story here, but that's less relevant for web applications - and if you're simultaneously iterating on ML models and a web backend, they'd likely run on separate machines anyways, and you can write both in Python without context-switching as a developer.

View on HN

Django shouldn't even require gevent - Django's ASGI support has been baking for a few releases now and supports async views which should be well suited to proxying streams from LLMs etc.

Relevant:

- https://fly.io/django-beats/running-tasks-concurrently-in-dj...

- https://blog.pecar.me/django-streaming-responses

(Reminds me I should try that out properly myself.)

then you have to rewrite your whole app to use asyncio keywords and colored ORM methods. A gevent monkey patch, or eventually nogil concurrency makes a lot more practical sense.

You don't have to rewrite your whole app - you can continue using the regular stuff in synchronous view functions, then have a few small async views for your LLM streaming pieces.

I've never quite gotten comfortable with gevent patches, but that's more because I don't personally understand them or what their edge cases might be than a commentary on their reliability.

Last time I checked, simply running Django through the ASGI interface dealt you a baseline perf hit.

It's pretty minimal: https://docs.djangoproject.com/en/5.1/topics/async/#performa...

> When running in a mode that does not match the view (e.g. an async view under WSGI, or a traditional sync view under ASGI), Django must emulate the other call style to allow your code to run. This context-switch causes a small performance penalty of around a millisecond.

Gevent IMO is more of a having-discipline footgun than a fundamental one: it makes itself so easy to use, without sprinkling asyncs in your day-to-day code, that you can forget that it has the same problem as explicit Python asyncio… namely, that if you burn CPU in a tight loop without any IO or logging or anything else, you’ll never yield to the event loop, and will block all concurrency until you’re done.

Detecting this might require external health checks and the like. And runaway regexes can trigger this too, not just tight loops. But those are also problems that asyncio has!

The only other one is that you’ll want to think about forking models with servers like gunicorn, and patch at the right time - but this is a one-time thing to figure out per project, and documentation is quite good now.

Just move to Elixir. Phoenix is Rails-like enough and the platform is excellent for parallelisation, clustering in specific hardware and so on.

Normally "switch languages" isn't great advice, but in this case I think it's worth considering. I have heard people coming from Django and Rails background describe Elixir as "a love child between python and ruby". Personally I love it

But does Elixir come with a whole scientific computing ecosystem?

Not to the same degree that Python does (then again no other general-purpose language does!), but it does have the start of one and it's fairly cohesive.

https://github.com/elixir-nx

Fortran? R? C? C++? Even Java may occasionally make a good showing here (depending on what you are doing).

Having seen... things... unless it's written by people with the right skillset (and with funding and the right environment), that it exists doesn't mean you should use it (and the phrase "it's a trap" comes to mind sadly). https://scicomp.stackexchange.com/a/10923/1437 applies (and note I still wouldn't call Julia mainstream yet), so while I'm not saying people shouldn't try, the phrase "don't roll your own crypto" applies just as much to the numeric and scientific computing fields.

I'm not super tuned into the scientific computing ecosystem, so not sure if this is what you mean. But maybe? Elixir's Numerical Elixir projects seem very relevant for scientific computing. Check 'em out: https://github.com/elixir-nx

Edit: Hah! aloha2436 beat me to the answer. Sorry for the repetition.

Must your statistical computing ecosystem comingle with your web interface?

You can split it off and have your Python code be an API you call, but now you have at least two languages involved (Python+Elixir, plus JS somewhere, plus the possible mix of C/C++/Fortran/Rust(maybe?)). Given Ruby on Rails was mentioned, just using Django seems similarly like the least risky thing to do (this all assumes you are doing numerical stuff, not just a standard CRUD app).

To add to what others mentioned, there’s also https://github.com/livebook-dev/pythonx which embeds a Python interpreter into your elixir program.

And if you want to make the move, I know a great resource: http://PhoenixOnRails.com

</shamelessselfplug>

I personally think Elixir is a great language, but the jump from ruby to functional programming is big enough that I'm not sure it's useful general advice.

Also, the size of the elixir community and the libraries available is completely dwarfed by rails. Elixir, Phoenix, all the core stuff is really high quality, but in many cases you might doing more work that you could have just pulled from a gem in Ruby. It's unfortunate IMO. It's an underrated language.

Very much this.

I think the community tends to overestimate the ecosystem’s maturity which is one of the big things holding it back, both because it blinds the community to areas that need improvement and leads to bigger shocks when newcomers do unexpectedly run into the rough edges.

This is true, but AI is closing the gap.

"Take this Ruby gem and write the equivalent functionality in Elixir" is something that LLMs are very good at.

That's over simplifying things ... no, complex gems cannot simply be transformed to Elixir with ChatGPT. You'd have to have an expert in both languages fixing all the bugs.

Do you have some examples?

And the switch is rather easy. I've been writing elixir for nearly 10 years, rails before that, and have overseen the "conversion" of several engineers from one to the other.

Generally I'd say any senior rails dev, given the right task, can write decent elixir code on their first day. There are a lot fewer foot guns in elixir and Phoenix, and so other than language ergonomics (a simple rule that doesn't stretch far but works at the beginning is use pipe instead of dot), there's minimal barriers

Honest question from someone working on a non-negligible Rails codebase: what would be my gains, were I to switch to Elixir?

I've watched Elixir with much interest from afar, I even recently cracked open the book on it, but I feel like my biggest pain points with Ruby are performance and lack of gradual typing (and consequent lack of static analysis, painful refactoring, etc), and it doesn't really seem like Elixir has much to offer on those. What does Elixir solve, that Ruby struggles on?

Performance of what, exactly? Hard to beat the concurrency model and performance under load of elixir.

Elixir is gaining set theoretic type system, so you are showing up at the right time. https://hexdocs.pm/elixir/main/gradual-set-theoretic-types.h...

> Performance of what, exactly? Hard to beat the concurrency model and performance under load of elixir.

The performance of my crummy web apps. My understanding is that even something like ASP.NET or Spring is significantly more performant than either Rails or Phoenix, but I'd be very happy to be corrected if this isn't the case.

I appreciate the BEAM and its actor model are well adapted to be resilient under load, which is awesome. But if that load is substantially greater than it would be with an alternative stack, that seems like it mitigates the concurrency advantage. I genuinely don't know, though, which is why I'm asking.

> Elixir is gaining set theoretic type system, so you are showing up at the right time. https://hexdocs.pm/elixir/main/gradual-set-theoretic-types.h...

Neat! Seems clever. Looks like it's very early days, though.

There's three reasons to choose elixir or perhaps any technology

The community and it's values, because you enjoy it, because the technology fits your use case. Most web apps fit. 1 and 2 are personal and I'd take a 25% pay cut to not spend my days in ASP or Spring, no offense to those who enjoy it.

Some of the big performance wins don’t come from the raw compute speed of Erlang/Elixir.

Phoenix has significantly faster templates than Rails by compiling templates and leveraging Erlang's IO Lists. So you will basically never think about caching a template in Phoenix.

Most of the Phoenix “magic” is just code/configuration in your app and gets resolved at compile time, unlike Rails with layers and layers of objects to resolve at every call.

Generally Phoenix requires way less RAM than Rails and can serve like orders of magnitude more users on the same hardware compared to rails.

The core Elixir and Phoenix libraries are polished and quite good, but the ecosystem overall is pretty far behind Rails in terms of maturity. It’s manageable but you’ll end up doing more things yourself. For things like API wrappers that can actually be an advantage but others it’s just annoying.

ASP.NET and Springboot seem to only have theoretical performance, I’m not sure I’ve ever seen it in practice. Rust and Go are better contenders IMO.

My general experience is Phoenix is way faster than Rails and most similar backends and has good to great developer experience. (But not quite excellent yet)

Go might be another option worth considering if you’re open to Java and C#

Thank you, I really, really appreciate the thoughtful answer.

I've written APIs in Rust, they were performant but the dev experience is needlessly painful, even after years of experience using the language. I'm now using Rails for a major user-facing project, and while the dev experience is all sunshine and flowers, I can't shake the feeling that every line I write is instant tech debt. Refactoring the simplest Rails-favoured Ruby code is a thousand times more painful than refactoring even the most sophisticated system in Rust. I yearn for some kind of sensible mid-point.

Elixir seems extremely neat, but I've been blocked from seriously exploring it by (a) a sense that it may not be more any more performant than Ruby, so why give up the convenience of the latter, and (b) not having seen any obvious improvement on Ruby's hostility to gradual typing / overuse of runtime metaprogramming, which is by far my biggest pain point. I'm chuffed to hear that the performance is indeed better, that the magic in Phoenix happens at compile time, and that gradual types are being taken seriously by the language leadership.

I guess I'll have to take Phoenix for a spin.

Performance is a complex story. Elixir is very good at massive amounts of parallel computing, and uses this to handle each request (or socket) in it's own contained manner, which nets you some simplicity in designing systems and scaling. However it's not very good at single threaded off-the-block performance (but neither is Ruby)

Typing is coming, some is already here, and if you're impatient you can use dialyzer to get half way there. But in my experience you need it less than you'd think. Elixir primitives are rather expressive, and due to the functional nature there really isn't any "new" data structure that isn't easy to dive into. And with each function head being easy to pattern match in, you can dictate the shape of data flow through your application much as you'd do in a typed language.

The ide story isn't great, but it's getting better. There are a few LSPs out there, and an official one coming soon. And I'd say all of them beat solargraph handily

But most of all I'd say that, since it's a bit more strict, elixir saves you and your coworkers from yourselves and each other. There are still several ways to do something, like Ruby, but going out of your way to write very cutesy code that the next programmer will loathe is more difficult. Not impossible, but harder. And with the rock stable concurrency systems in place, a lot of the impetus to come up with clever solutions isn't there

Many thanks, I very much appreciate your thoughtful answer. I've added some of the context for my question here: https://news.ycombinator.com/item?id=43144841

Re types, I think what I really want is just comprehensive static analysis. Coming from Rust, where I can immediately pull up every single call site for a given function (or use of a struct, etc.), I find refactoring Ruby/Rails code to be comparatively very painful. It is several orders of magnitude more laborious and time-consuming than it should be, and I just don't find the justification for that cost convincing - I'd trade every last bit of Rails' constant runtime metaprogramming cleverness for more static analysis.

What I like about Rails is its batteries-included nature, but I honestly could do without those batteries materialising via auto-magic includes and intercepted method calls. I appreciate that that's just the culture of Ruby though, so I don't expect Rails to ever change.

The lack of cutesiness in Elixir sounds lovely. I don't know if functional approaches could make up for a lack of typing for me; I think I'd need to try it. I've used and enjoyed Haskell, but of course that's very strongly typed.

Elixir has had some static analysis for a long time, you can use a command to see all call sites of a module and function. It's useful, and most of the LSPs use (or used to use) it. The newer versions also are adding various hooks to the compiler, to allow for better tooling

As for magic, elixir can have what looks like magic, but upon closer look it's nothing more than macros, which are generally pretty easy to decipher and follow. It has minimal automatic code generation, what it has is mostly the form of generators you'd run once to add things like migrations

I have the same experience with typing in Elixir. It's hard to explain without experiencing it yourself, but the dynamic typing just doesn't feel like as big of a deal as it might in other languages. Elixir's guardrails (such as pattern matching in function heads, which you mentioned) get you most of the benefits - and you still get the convenience and simplicity of a dynamic language. It's a great balance.

I'm looking forward to the upcoming gradual type system - it can only be an improvement - but I would still encourage people to try Elixir now, and let go of their preconceptions about static typing.

Wrote about it more here: https://arrowsmithlabs.com/blog/you-might-not-need-gradual-t...

Thanks for your article, George, I enjoyed it.

I cracked open an Elixir book last night, and with the benefit of a few chapters, I can see how Elixir's pattern matching can obviate some of the issues I have with purely dynamic, Rails style programming.

I also note that there appears to already be more static typing going on than I realised. In your add_comment/2 code, for instance, you focus on the {published: true} pattern matching. That is very neat, but what stands out more to me is that all clauses of that function require BlogPost, a struct type.

Am I right in thinking that every instance of BlogPost type must be constructed explicitly in your code? I.e. that every possible instance of BlogPost in the code base is knowable at compile time, along with its entire life cycle?

Or does Elixir partake of the horror that is duck typing, where any conforming untyped map of indeterminate provenance will pass the guard check for a BlogPost?

> Would you like another undefined method exception on that NilClass?

*thousand yard stare*

> Am I right in thinking that every instance of BlogPost type must be constructed explicitly in your code? I.e. that every possible instance of BlogPost in the code base is knowable at compile time, along with its entire life cycle?

Almost. You can create a struct dynamically at runtime like this:

    struct(BlogPost, %{title: "The Title"})
    # => %BlogPost{title: "The Title"}

… but you rarely need to. In fact I'm not sure I've ever used `struct/2` in real life. 99.9% of all the structs you ever create will have their types known at compile-time.

> any conforming untyped map of indeterminate provenance will pass the guard check for a BlogPost?

Nope. In the `add_comment/2` example, If I pass anything except a %BlogPost{} to that function, I'll get an error.

I appreciate the explanation, you're a champ. :)

While I remain haunted by thoughts of someone e.g. deserialising a YAML file into a map, which then sneaks in some __struct__ key and squeaks past the guard clause, I also appreciate this seems fairly unlikely in practice. I think I'm just traumatised by Rails. It sounds like the culture around Elixir eschews excessive cutesiness, though. Promising!

Don't let your experience with Ruby colour your opinion of Elixir. Elixir's similarity to Ruby (and Phoenix's to Rails) is hugely overstated.

I wrote a long Reddit post about that[0], but for the tl;dr you can ask José Valim himself: https://news.ycombinator.com/item?id=36604054

[0] https://old.reddit.com/r/elixir/comments/1ipyi81/why_should_...

Don't take my word for it but IIRC structs are implemented as maps with a __struct__ key with the struct name, and that's used to implement checks and balances at different levels of compilation, linting and so on.

In practice I find that I hardly ever need to think about things like this. A few times I've done macro expansion to peek under the hood and figure something out but that's partially Lisp damage, I could probably just as well have read some documentation.

Thanks! You are completely correct: https://hexdocs.pm/elixir/main/structs.html#structs-are-bare....

The JVM now has virtual threads so Elixir's days as the only game in town for massive parallel computing may be numbered.

> You have the Python type system, and while it's inferior to TypeScript's in many ways, it's far more ubiquitous than Ruby's Sorbet.

I'm a big fan of Ruby, but God I wish it had good, in-line type hinting. Sorbet annotations are too noisy and the whole thing feels very awkwardly bolted on, while RBS' use of external files make it a non-starter.

Do you mean Ruby lacks syntactic support for adding type annotations inline in your programs?

I am one of the authors of RDL (https://github.com/tupl-tufts/rdl) a research project that looked at type systems for Ruby before it became mainstream. We went for strings that looked nice, but were parsed into a type signature. Sorbet, on the other hand, uses Ruby values in a DSL to define types. We were of the impression that many of our core ideas were absorbed by other projects and Sorbet and RBS has pretty much mainstream. What is missing to get usable gradual types in Ruby?

My point isn't technical per se, my point is more about the UX of actually trying to use gradual typing in a flesh and blood Ruby project.

Sorbet type annotations are noisy, verbose, and are much less easy to parse at a glance than an equivalent typesig in other languages. Sorbet itself feels... hefty. Incorporating Sorbet in an existing project seems like a substantial investment. RBS files are nuts from a DRY perspective, and generating them from e.g. RDoc is a second rate experience.

More broadly, the extensive use of runtime metaprogramming in Ruby gems severely limits static analysis in practice, and there seems to be a strong cultural resistance to gradual typing even where it would be possible and make sense, which I would - at least in part - attribute to the cumbersome UX of RBS/Sorbet, cf. something like Python's gradual typing.

Gradual typing isn't technically impossible in Ruby, it just feels... unwelcome.

None of my customers ever asked for type definitions in Ruby (nor in Python.) I'm pretty happy of the choice of hiding types under the carpet of a separate file. I think they made it deliberately because Ruby's core team didn't like type definitions but had to cave to the recent fashion. It will swing back but I think that this is a slow pendulum. Talking about myself I picked Ruby 20 years ago exactly because I didn't have to type types so I'm not a fan of the projects you are working at, but I don't even oppose them. I just wish I'm never forced to define types.

I for one really like RBS being external files, it keeps the Ruby side of things uncluttered.

When I do need types inline I believe it is the editor's job to show them dynamically, e.g via facilities like tooltips, autocompletion, or vim inlay hints and virtual text, which can apply to much more than just signatures near method definitions. Types are much more useful where the code is used than where it is defined.

I follow a 1:1 lib/.rb - sig/.rbs convention and have projection+ files to jump from one to the other instantly.

And since the syntax of RBS is so close to Ruby I found myself accidentally writing things type-first then using that as a template to write the actual code.

Of note, if you crawl soutaro's repo (author of steep) you'll find a prototype of inline RBS.

+ used by vim projectionist and vscode projection extension

If you want something more similar to Next.JS but in the python world, now you have https://fastht.ml/, which also has a big performance benefit over Django. Hahaha, same as Next.JS over Rails, because it is much more bare bones. But I would say that fasthtml has the advantage of being super easy to integrate more AI libraries from the python world.

now that was a crazy rabbit hole

Reply on news.ycombinator.com