Hacker News Reader

Comment by danpalmer | Hacker News Reader

HTMX is great for rapid prototyping, but Rust has never struck me as suitable for rapid prototyping. There are some nice abstractions here than reduce the amount of code for sure, but that's only one aspect of rapid prototyping. Rust still has a high learning curve, compiles relatively slowly, and still takes a lot more code to achieve things than the equivalent Python or even JavaScript/TypeScript.

Shuttle looks pretty cool as a platform, but it seems that it has decided that because Rust is a good language for a platform (makes sense!) that it must be good for usage on the platform. I really don't think I agree with this.

In my experience, Django + Heroku (or the Heroku competitor du jour) is a convincing rapid prototyping setup for larger sites. And things like RunKit or Glitch are a convincing setup for smaller builds. I'm not sure whether Shuttle adds anything over these options unless the requirement is to use Rust.

View on HN

Rust has:

- serde; this beats both cpp and Python in terms of not spending a second of dev time on implementing serialisation deserialisation unless you're doing something funky. I would argue that this is one of the cases where it's much harder to handle it in Python than in rust

- other proc-macros allowing you to quickly map your structs to sql or whatever else

- lsp that works with zero configuration and beats many languages (again, cpp and Python included) because of rust's strictness, and rust-analyser is an absolute pleasure to work with

- cargo tooling and zero-pain dependency management; high quality of packages in general

Once you get past the initial learning curve and get comfortable with the language, these factors (and quite a few more not directly related to the language itself) eventually become more and more important for prototyping. (all subjective of course)

> serde

they have very many formats implemented. Which format is zero penalty, so I can use it and not write my own logic?

> they have very many formats implemented

It's not "they", it's whoever wants to implement serde support for their own thing, can do it (so, many 3rd party crates have a "serde" feature that can be toggled on/off).

> Which format is zero penalty

This one is tough to answer unless you provide details re: what's your in-process data format, what's your target serialization format and whether it's binary or not, what's the objective function (serialization speed, deserialization speed, storage efficiency), and what exactly you mean by 'zero penalty' (which penalty?).

// note that there's also alternative frameworks worth mentioning like rkyv and borsh-rs, each optimizing their own objective function

Python has pydantic [1] and msgspec [2], both of which are pretty similar to serde in terms of features, and pydantic 2 has even been rewritten in Rust.

[1] https://pydantic.dev [2] https://jcristharif.com/msgspec/

Which crate does map structures into SQL?

sqlx [1] can definitely do this.

[1] https://docs.rs/sqlx/latest/sqlx/

> it seems that it has decided that because Rust is a good language for a platform (makes sense!) that it must be good for usage on the platform.

This isn't true - or at least it wasn't our thought process when starting to build Shuttle. Yes Rust made a ton of sense as the language with which to build the platform, but the point of Shuttle was always about providing a great developer experience to the end user. _That's_ why we chose Rust for the 'front end' of the platform.

Our position was that the amazing type system combined with generics, metaprogramming (macros) and excellent compiler errors would enable us to build something truly special when it comes to how devs build on Shuttle.

That's fair about it not being your thought process, but do you think your familiarity with Rust may have affected how you assessed the suitability?

Your view on Rust matches my views on what makes Rust a great language, but I don't feel like they make Rust a fast to develop language.

- Type systems are great, but pay off more as a codebase grows. Rapid prototyping skews towards small codebases. Type systems do have a non-trivial cost, and that has outsized impact on smaller codebases.

- Generics are nice, but small codebases have less need for them.

- Metaprogramming and macros are nice and allow building new abstractions, but smaller codebases need fewer abstractions. They also add to the learning curve, as they come with their own learning process.

- Compiler error messages are indeed helpful in moving faster, but while Python's error messages aren't great, I hit fewer of them because the language is more flexible, more things "just work" and fewer things are errors. Same for JS, etc.

Correctness and safety are great, and I think Rust gets these much faster than other languages, but memory safety is a big benefit of Rust that is entirely unnecessary for almost all rapid prototyping. Speed is also a benefit, but is again rarely needed for prototyping and early stage projects.

I'd love to understand more why you feel that Rust is faster, and not just for Rust developers.

I also have done rapid prototyping in Python and Rust, and have started only using Rust for such projects.

- Tooling and packaging are just better. I find myself fighting Python sometimes when setting up testing with pytest, or packaging with poetry/pdm. This overhead is eliminated with Cargo.

- I like to write small PoCs for complicated data models, to show how they interact before integrating the concepts into a larger system. I use to do this in Python, but switched to Rust. Python's poor typehinting makes it very hard to know when the concept I'm implementing actually doesn't work. Instead I have to write tons of unit tests to validate it, which ends up taking much longer than using the typesystem as a form of unit testing, and then writing a few logic validating tests.

- Large refactorings are much easier in Rust. This is absolutely critical for rapid prototyping, because it's common to rip out large systems. The typesystem tends to point me to every area of the codebase I need to worry about. I've started considering it a code smell if related functionality isn't evident through the type system.

- At least in my experience, it's easier to organically grow a codebase. I feel comfortable using a ton of `unwrap`s and `todo!`s in my codebase, because it's easy to grep for them and retrofit proper error handling. I can change a function's type to include a `Result` and the typesystem will direct me to every location I need to update to properly implement error handling. Most languages I work with I feel I must do everything roughly correctly from the beginning. Rust makes me comfortable hacking stuff together, because there's a clean way to slowly remove the tech debt.

Hopefully that gives you some idea why Rust can actually be an amazing prototyping language. It can be hard to see when you're still learning the language, but once you have, you rarely run into weird lifetime compiler errors. It really is a language that teaches the programmer.

> Tooling and packaging are just better. I find myself fighting Python sometimes when setting up testing with pytest, or packaging with poetry/pdm. This overhead is eliminated with Cargo.

Python's package / project management story is seriously abysmal compared to its competitors. It's crazy that for a language with as much adoption the best we have is poetry. Adding and isolating project dependencies should not be a difficult thing to do, it should be first class in any serious language like it is in node, ruby, elixir, rust, etc etc etc.

I think they are making steps with Pipfiles, but yeah it's definitely not great. I had to use Python for a hackathon a while ago and it was probably one of the worst experiences I had trying to get set up with my group.

Thanks for the input, and I completely agree in the value of all of these aspects, just not so much for prototyping.

Tooling and packaging disappear with a sufficiently good prototyping environment (and Cargo doesn't completely eliminate this in my experience anyway), Rust's type system means also working with the borrow checker which is not useful for data modelling (outside of safety concerns), large refactorings aren't necessary in small prototypes (even as things change).

You're right that it's easier to grow a codebase organically from a good starting point, and Rust will offer that, but I'm not sure that's the right focus for rapid prototyping. The aim is not about setting solid foundations, it's about testing ideas.

Rust is nice, don't get me wrong. I don't disagree with any of these points. I just think that they're all in the mode of improving "traditional" backend development, not in the mode of making rapid prototyping as good as it can be.

Re data modeling: If you’re using copilot or something similar I’ve found it really fast to drop in a comment with a sample of the data and then quickly crank out the appropriate types. It doesn’t get them all perfect in one shot usually but it reduces the overhead enough to make it effectively free when considering the safety and refactoring benefits.

Also rust offers you out of the box tests, so you can write tests to prove your code does what you'd expect, before finish the whole prototype, it helps a lot versus most languages, most of the time you can just run `cargo watch -q -c -x test` and start hacking.

Many other languages have similar easy ways of running tests on file changes.

For example, in (Node)JS, you can just do `npx jest -o` if you are using Jest testing library, and with Scala you would usually use `~ test` in sbt shell.

This is likely just a me problem, but I always found Jest extremely annoying to set up with React and TypeScript - particularly with attempting to follow the documentation for jest and then failing spectacularly (in spite of using React Testing Library, jsdom + jest).

Cypress on the other hand I found to be really, really good

Cypress and Jest aren't really operating at the same level though. Cypress competes more with Playwright (functional, E2E tests), and Jest competes more with Vitest, Chai, Mocha, Jasmine for unit tests (though it can also do much more).

Personally I prefer Vitest to Jest... if it will work for the codebase I'm in.

vitest, like everything to do with vite, just works. The API is almost identical to Jest so you can swap over easily.

That's not the same thing, I think he's referring to the fact that in Rust you can keep unit tests and code in the same file, and strip them away at build time. That's not a thing in the js world (and most other languages), where tests must be in a different file and you are forced to export functions to test them, even if you wanted to keep them private in that file.

To be honest you might be able to use just the built in node test runner (deno and bun also have their own test runners), but IMHO are not as ergonomic as Rust, in Rust you can test private functions in the same file, and that let's you prototype faster, you write a simple function and a test (even when the prototype is not fully functional).

In JS land you could do the same I think, but you would need to export all methods that you want to test and they are in other files, and to make things worse you need to setup Typescript to make your IDE more useful and also write less tests, I would argue that in that case vite/vitest will be more helpful because the tests runs with the same config as the bundler, but still, it does not work out of the box as Rust, there might be other languages that have also good ergonomics as Rust in that regard, but none of the languages that I use besides Rust do that.

How are Rust tests any more suitable for rapid prototyping than Python tests? If writing tests can even be considered "rapid prototyping".

You cut down a chunk of tests by not needing to figure out what type something is. You can also use assert! and assert_eq! in your main code without even using tests which also cuts down on tests, where applicable.

You can also write unit tests in the same files as your main code. Eventually you'll want some kind of test suite when your codebase gets big enough, but it works well enough that you don't need to if you don't want to (for a small codebase).

Frameworks like Axum also of course support testing - for instance this crate lets you quickly mock up your server without having to do much: https://docs.rs/axum-test/latest/axum_test/

> You cut down a chunk of tests by not needing to figure out what type something is

I have literally never written such a test in a rapid prototyping context.

When you are prototyping you change code here and there, so a type system helps in that regard and I consider type system as a kind of test. So by having that built in, you need to write less tests, but AFAIK python has an optional type system, not sure how that works, but I suppose it would help in that regard.

If find that exactly the situation where a type system is intrusive, not helpful.

Python has had tests and a cli that discovers them since 3.2 https://docs.python.org/3/library/unittest.html

>>Test Discovery¶ >> >>New in version 3.2.

It's invoked by:

>>python -m unittest

Edit:

I guess if you're argument is for the watcher...great but I use `entr` https://github.com/eradman/entr when I want that feature and it just works across languages but I've not used continuous testing for ages.

Not every project has to be suitable for every team of developers. Rust has a good number of people that both like it and are also deeply familiar with it, so it makes sense someone would build something that caters to them. Rust can be great for rapid prototyping when everyone on the project is already a Rust expert.

So is rapid if you’re rapid.

> but Rust has never struck me as suitable for rapid prototyping

Having Rust a lot, and coming from JavaScript, I'd say it depend at what stage of prototyping you want to stop:

If you want to make a one-shot implementation of a quick demo without ever iterating on, then, yes, Rust is going to slow you down.

If your prototyping effort involves lots of iterations because you're learning in the process (learning about the use-case, the market and so on), then Rust quickly pays of compared to JavaScript or say Python because refactoring is straightforward and you're almost never going to break things in the process. Most of the benefit comes from static typing alone, but Rust type system allows you to express things in a more robust way than C# or Java for instance, leading to an even more streamlined iteration cycle (but it's not a game changer either).

But the place where I've found Rust to shine, with no match whatsoever, is going from the stable prototype to the production ready stuff: in addition to the ability to get the best achievable performance on your hardware, I find the error handling to be the killer feature in that scenario: when prototyping, you use unwrap everywhere without paying attention (and it basically behaves as in a language exceptions where you don't catch them) but then you just ctrl-F `unwrap` and you can deal exhaustively with all possible error scenario with very limited effort. Ensuring after the fact that every possible exception gets handled properly is a much harder task.

Totally agree with this.

I can spin up a generic Axum backend service within probably about half an hour to an hour that contains most of the things I need (auth, CRUD, similar things) once I know what I need, configuration and business logic depending. I think the key phrasing is primarily "when you're familiar with Rust" - if you're not familiar, then it will definitely be a slog.

Certain things that would require you to manually implement a complex trait (for example, Send + Sync + 'static plus whatever else) would probably slow you down though, but presumably by that time you probably already know what you're doing.

Do you have a goto library for auth in Axum? Maybe I just don’t implement it often enough but auth in Axum usually takes me an hour at least even if I’ve implemented it somewhat recently.

>> The place where I've found Rust to shine, with no match whatsoever, is going from the stable prototype to the production ready stuff

Love it or loathe it, spring boot is going to win that race every time.

A kind of sweet thing about modern spring is it can be pretty easy to rip it out when you've arrived at your feature complete state. So if you wanted to, you could get rid of reflection without losing JIT for example (i think Graal is getting JIT eventually, which would be the other way to force yourself rid of reflection today, so maybe this becomes moot in future).

But either way, if you approach your spring boot development with this exit strategy in mind, you can reap the rewards of removing a big project dependency to own over time leading to cheaper TCO long term once the project has reached feature complete state. It'll even likely be slightly faster if you're ripping out all those classes from the classloader, removing all that reflection and proxies etc. - but you can do all this while scoring the benefits of rapid iteration & extensive code reuse / building on the shoulders of giants, at the start of the project by exploiting spring and especially by exploiting boot.

Could you do something like that with say Axum? Maybe. Probably not.

I mean that's quite a niche idea spurred on by your next comment i've quoted below but even if you're not interested in eeking out the last say in performance and cloud memory footprint costs then spring boot will still more than likely be absolutely fine without doing any of this. And you'll be done sooner too of course.

As for production, does that mean stuff like robust and safe, automated scrubbing measures for handling developer access to sensitive prod memory dumps for diagnostics? Or maybe just mature observability tools - it's like bringing the "tokio console" knife to a "JMC & JFR" gun fight.

I guess my TL;DR here is just: Java. But it could equally be C# or maybe even something else. It's definitely not "with no match whatsoever".

>> in addition to the ability to get the best achievable performance on your hardware

I've ended up spewing so much above i don't feel like going much further and it was actually this part that made me reply.

What you're saying seems unlikely - you're claiming not just fast but the best possible performance on given hardware. So we're talking really annoying things to own like repurposing #[repr(C)] not for FFI but to allow explicit ordering of struct fields to maximise memory throughput. We can do that kind of stuff in Java too (heck most languages allow something like this), java has some unsafe off-heap facilities which can be (ab)used for extreme cases - although there's potentially much nicer and safer APIs in the latest Java which i haven't looked at yet - https://openjdk.org/jeps/442

Best achievable performance usually means for one part of your process at the cost of another code path - when you get down to the most extreme levels, it becomes trade offs. If you want to go that far in java, you can. It's not like you're getting extreme performance for nothing by going another way. E.g. rustc won't magically recognise every time i could use vector instructions, you have to write your code careful of branching and dependencies, be friendly to the compiler for it to do its magic. But of course, we have vectorisation (and even some auto-vectorisation) in java too.

> HTMX is great for rapid prototyping, but Rust has never struck me as suitable for rapid prototyping. There are some nice abstractions here than reduce the amount of code for sure, but that's only one aspect of rapid prototyping. Rust still has a high learning curve, compiles relatively slowly, and still takes a lot more code to achieve things than the equivalent Python or even JavaScript/TypeScript.

In order to support the borrow checker, Rust has a richer type system that allows programmers to encode far more of the business domain than most other languages. Short of logic bugs, the vast majority of my Rust code is in the "if it compiles, it works" bucket which makes it great for rapid prototyping despite the slow compile times. I have to spend a lot less time and mental bandwidth on testing and defensive coding and Rust's version of the public/private distinction o makes it great for wrapping the data model behind types that can only be used correctly - like making sure a user id can only come from a User model type instantiated by the database code and not an arbitrary integer.

My favorite part though is that web frameworks like Rocket make it really easy to manage all of the magic going on behind the scenes using trait definitions, in contrast to mega-frameworks like Django. (Rocket was created by the Flask author iirc)

The first framework I used was Rocket!

I'm sad that the development pace for it has slowed down extremely significantly though.

> (Rocket was created by the Flask author iirc)

unrelated individuals (Flask, Armin Ronacher; Rocket, Sergio Benitez)

you're right though that Armin is writing Rust these days though!

Rust's richer type system provides relatively little value over other contemporary type systems like those of Python, TypeScript, and C#, while not providing anything remotely resembling an "interactive" development experience (no REPL, incredibly slow compilation, not hotpatching), and imposing the extremely burdensome and completely unnecessary burden of manual lifetime management.

Until these flaws are fixed, Rust is utterly unsuitable for rapid prototyping.

When I wrote C++ full time I frequently prototyped algorithms in Rust.

Prototyping in C++ is awful: it's so easy to accidentally trigger UB and then you're wasting time debugging some memory corruption instead of exploring your problem.

Rust is close enough to C++ that you can get a good idea of equivalent C++ performance, and it isn't too hard to translate the prototype when you're finished. An exception being porting Serde (de)serialization over.

It really boils down to personal preference.

I never considered Django (or anything python related, eg. FastAPI) to be something good for prototyping because the package manager is a nightmare, deployment is clunky, I often hit pain points or limits in Django.

Node.js before the whole require/import mess was neat and simple. All the libraries were working nicely, no problems with npm. Nowadays it's a hit or miss experience. I'm using bun these days to prototype and if you restrict yourself to bun stuff you're good and fast and zero hassle.

Rust, contrarily to JS (or TS, the land of any) or Python, has serious safety belts. This means for prototyping it won't be as fast, but you'll sleep well at night.

In my experience, if you're not a beginner you'll be able to be fast and iterate quickly with Rust as well, albeit it's not the same. On the other side, once you're done with your prototype I feel way more confident in my Rust prototypes, compared to my JS ones.

So, does it really slow you down?

The Shuttle boilerplate is nice, but it doesn't really add that much, the equivalent setup in Axum is not that much of a bother (I tend to use a axum + sqlx template I built for new projects).

Shuttle the platform is nice, albeit I remember being annoyed by the limits of the free plan (I know, I know, I am cheapskate but I don't want all that nice marketing money go to waste), I generally deploy on fly.io or some more cost effective VPS.

Reply on news.ycombinator.com