Why Not Rewrite It in Rust? (2016) | Hacker News Reader

Why Not Rewrite It in Rust? (2016)

transitiontech.ca

69 pointsbishala77 comments

Writing a few lines of "why not rewriting it in XXX" is more a sign of inexperience or immaturity in my humble opinion.

Maybe it is a good idea, but if you are going to propose to a dev team a massive rewrite that will take months or years, you should at least have the decency to analyze the code base, do a small POC, profile some part and point out where the improvement could happen. (And if you've discovered something that could be improved, you are more likely to convince people to rewrite only this part than everything)

Otherwise, it seems a bit cheap, and more something that someone repeats without necessarily understanding deeply because it is in the zeitgeist. It is probably also a bit of magical thinking, if we rewrite all our current problem will go away.

Programmers, especially naive ones, tend to think asymptotically. We focus on what would be the most theoretically ideal and write off pithy concerns like "time" and "effort" as implementation details. I'm guilty of this myself.

Rust is this unusual situation where a new language is in many ways a strict improvement over a very widely-used preexisting language. That is extremely rare. So I think that's where the religious fervor comes from.

Would the world be a better place if Tor and the Linux Kernel and every C++ project out there had originally been written in Rust (setting aside the fact that this was obviously impossible)? Sure, maybe. But those projects also benefit from decades of refinement and bug fixes. Rewriting them means losing all of that. As an idealistic programmer it's tempting to write factors like legacy refinement off into the margins, but they're actually very significant. A piece of software's value doesn't solely reside in its preconceived design.

Incidentally, Tor has some Rust in it these days. I haven't heard from them for a while, so I don't know how the project is going...

Tor is probably one of the better cases because of its stringent security requirements, but there remains an oft-ignored tradeoff of losing all that refinement work (and the fact that you can't just flip a switch to make it happen). It's a cost/benefit decision.

Totally; that's why it's not a wholesale re-write, it's slowly replacing old code bit by bit, or writing new code.

Which is the only real way to migrate any codebase of significant size to a new language.

> those projects also benefit from decades of refinement and bug fixes. Rewriting them means losing all of that.

Not my experience at all, having spent a significant chunk of my career doing rewrites for scalability. Having the existing codebase makes it very easy to benefit from the accumulated knowledge, while at the same time doing a pass through that can catch a lot of outright errors. If you can rewrite a project incrementally while keeping it working (and for C->Rust, you can), then the arguments against rewriting your code don't apply, and I'd actually consider full rewrites a useful exercise even if you weren't rewriting to a better language.

My experience as well. The key point here is the incremental part. This works especially well if you're adding regression tests incrementally as well. This necessitates a certain modularity & testability in the legacy project though, which often isn't the case. There's a certain threshold in technical debt (depending on the size of the project), at which point a complete rewrite may be better.

For most of the other projects Working Effectively With Legacy Code by Michael Feathers contains a good approach for slowly modularizing and testing the system, before exchanging parts.

I agree. I made rewrites all the time (and from lang to lang, to architecture to another). Rewrites are a net plus for me.

What is important is that $new_lang or $new_arch MUST help to cut in big ways problems and/or code.

To say something minimal, Rust eliminate (except when interfacing with $old) NULLs. Also, pattern matching make a lot of problems go away, FOREVER.

This mean, that is like have another developer also helping in cut work and at the same time increase quality.

This is not limited to compilers. For example, I work for business apps and use RDBMS a lot. Use a RDBMS well? It cut work and increase quality.

Rewriting queries, schemas and using indexes, views and ANY trick the RDBMS allow you (the idea "not use a DB at full because 'data independence' is poison if not VERY well justified) help by spades.

So:

- Naive rewriting = VERY BAD

- Move from $old to $new WITHOUT the help of a "improved" architecture, feature, tooling or concept = WASTED

- Move from $old to $new with better? TOTAL WIN.

P.D: I think move from too similar Langs probably will waste stuff. You see much more bang from your buck when move from imperative to functional, than from imperative to imperative. Not because functional is superior, but because it force to bring power (use of inmutable and algebraic types, for example) that before was dependent on discipline...

Maybe. But even with the old code base for reference, it's easy to not see that those weird lines right there are written the way they are to handle two different corner cases (and of course there's no helpful comments to explain it). This is especially true if you're rewriting from $MORE_OPAQUE_LANGUAGE to $CLEARER_LANGUAGE. There can be a lot of not obvious things going on in the opaque stuff, and you can drop things you need in the rewrite unless you are very careful.

As carlmr points out, tests can help a lot... if you've got tests for those corner cases. They can be new tests, created as part of the rewrite - if anyone remembers that they are needed, or if anyone can see the corner cases either in the problem space or in the code. (This argues for doing the rewrite while the original authors are still around, preferably with them as part of the rewrite team.)

Right. Put differently: any sufficiently long-lived codebase contains institutional knowledge. Knowledge that goes deeper than just the explicit interface boundaries that you're planning to re-implement. That doesn't mean rewrites are a bad idea, just a disruptive one. They'll introduce new fragility for a while before they start being a net gain. Code has to be broken-in, like a new pair of shoes.

> There can be a lot of not obvious things going on in the opaque stuff, and you can drop things you need in the rewrite unless you are very careful.

If the knowledge is so opaque as to be nonobvious from the code in front of you, then it's already an accident waiting to happen: someone making normal maintenance changes to that code (not a rewrite) would be equally likely to break those properties.

If you are not an important member of the dev team you shouldn't propose it at all. Even if you have a magical process that could transform a known maintainable mess into maintainable rust you don't get to talk about it to the dev team.

There will always be inexperienced people who just learned about "cool language X" and go on a crusade everywhere they go about why Y is not written in X with obvious disregard on how difficult (and often useless?) rewriting a whole project is.

I remember seeing a post on a Qt mailing list asking if there were plan to rewrite Qt in Erlang.

At least it usually come from a good place and they tend to be polite about it. Better than a lot of open-source freeloader getting angry because B is still not implemented in A ...

Qt in Erlang is a good one.

I feel like all the language comparisons are largely missing the point, at least for me. I don't write (mostly orthodox) C++ because it's a great language. In fact, I think it's a terrible language, hampered by a myriad of systemic issues that will never be addressed and seem to be getting worse over time.

However, I also don't write code in a vacuum and other things matter more to me than language features. First-class GPGPU support, a mature ecosystem (including some of the highest quality, most efficient code that exists), vastly superior tooling, multiple established compilers, good C compatibility, the list goes on. The purported advantages of Rust, such as the trait system (which is legitimately great), the static safety guarantees (which I personally don't care about at all) and others pale in comparison.

If you really like Rust, you don't address these things by obnoxiously evangelizing language features. You address them by building up the ecosystem, improving the tooling, making the language more pleasant to use and eventually advocating for important vendors to include first class support for Rust in their products. Then, and only then, might this kind of in your face evangelism make some sense, if we ever get there - and the way towards it is paved with patience and lots of hard work, not opening GitHub issues.

If you had a code base in Rust would you consider rewriting it in a language with First-class GPGPU support, a mature ecosystem (including some of the highest quality, most efficient code that exists), vastly superior tooling and multiple established compilers?

Depends on the codebase. If one of those things could qualitatively improve my codebase - for instance, by introducing GPGPU support and speeding everything up by an order of magnitude - I very well might. Ultimately, it's a cost-benefit calculation that has to be done for every individual case, it's not something you can answer in the abstract. None of the language features of Rust can offer a similar level of advantage, except perhaps in cases where (limited) static guarantees of certain kinds of memory safety are of critical importance.

If my code runs slow because of lacks in one of the things you list I'll probably rewrite that part of my code. Note that I wouldn't rewrite everything, at least not at first.

In almost all cases though, the that fact that my codebase is working means the ecosystem, tooling and compilers are good enough as is and so I won't have a compelling reason to rewrite. I can get great programmers productive in Rust faster than the boss can figure out that neither I nor the new guy knew anything about rust on the first day.

Typically the only time the factors you mention come into consideration is a brand new project. I will generally know before I write the first line of code if I need GPGPU support or not. I ecosystem, tools, and existence of compilers are just a few of trade offs I will consider.

My humble opinion: it is almost never worth rewriting a large codebase in a new language. The only exceptions are if (a) the old language is so old that it lacks an active community and is no longer being improved, or if (b) the old system needs so many changes that you might as well rewrite it anyway.

It's not worth rewriting it until someone does and somehow manages to steal your customers with it.

The old system needs a big change; it's flaky and riddled with security flaws because the old language encourages undefined behavior. C was a big improvement on assembly, but it's also the leading reason everything we rely on fails frequently.

I agree! But the cost of such a move across the industry would be incredibly high, probably taking decades of highly expensive investment. Is it really worth the cost for all projects?

"rewrite in X" request is a comon sign of immaturity. I'm guilty of it but eventually I learned that A) rewrites are rarely worth it B) it's disrespectful to ask others to scrap their code and rewrite it in my favorite language

One of my favorite apps at the moment seems like it could benefit from a rewrite, in part because core pieces have long standing bugs that may only be solved by someone doing the very deep dive a rewrite would entail. I thought long term it might be nice if it were written in Rust. But rather than suggest RIIR I considered writing a component of it in Rust myself. That would be educational for me and would bring something new to the "Rust ecosystem". Having written similar code before (in C++) I realized just how huge a task rewriting just a core component would be. It could take a decade or more to rewrite an entire program in any language even with all devs working on it.

Look at Servo. Hey, they started building a browser in Rust! And some of those components are useful in Firefox. But Servo itself might never become a fully functional browser. And that's a project from the folks that created Rust and Firefox. So maybe a response to RIIR could be "Yeah, when is Servo going to be done?"

I know RIIR appears to offend a lot of C/C++ devs, and I understand why. I do however, think RIISSL (Rewrite It In Some Safe Language) is a real meaningful thing.

The reason Rust is the figure head for this is because I've not seen other languages with any amount of popularity take "safety" to the degree Rust has. I imagine there are others, I recently saw bounded array types in Ada and thought it was cool, but Rust definitely has the momentum.

This isn't to say that I'm justifying RIIR. I'm justifying RIISSL. To which currently, there aren't many.

In the future I hope to see many safe languages. I love Rust, but I'm super excited for the next language that blows Rust out of the water on safety and usability. I hope RIISSL is a thing in the future, and I hope there are more options so that Rust doesn't get so much unneeded hate over RIIR :)

I really do love Rust though.

edit: I should amend, RIISSL is only applicable where safety matters. I could definitely see arguments where all common unix tooling matters, as even find could have security implications. Yet, `find` compared to SSL implementations are vastly different for RIISSL.

> I know RIIR appears to offend a lot of C/C++ devs, and I understand why. I do however, think RIISSL (Rewrite It In Some Safe Language) is a real meaningful thing.

It is not meaningful for open source projects where RIIR or even RIISSL types jump in and start offering unsolicited advice to contributors. These folks can fork and/or make some POC etc if they seriously believe it is a net win.

In fact, the amount of successful penetration of RIIR is a direct measure of the wisdom of doing it.

I.e., if it were a good idea, there would be no need for an advocacy. People would just do it. If the ecosystem is ever mature enough, we will know by the counting the number of projects that switch.

But every day, there is the choice: should I improve the product for users, or potter with the build system? Because language choice is really a build system detail, to users.

Depends, would you not jump in and advocate a project fix fundamental flaws if it affected your production security?

Which don't get me wrong, this is likely not advocatable (is that a word?) to something like Find. I'm not defending that. However C bugs in SSL/Apache/etc are terrifying. A person jumping in and advocating RIISSL is similar (but obviously different) to jumping in and advocating for critical fixes, or perhaps even security audits.

I really enjoy articles like these. There is something special and provocative about a highly sarcastic article about programming, like a comedian that can make light of a serious topic. Maybe it doesn't solve anything but it does bring to light another view you might have not seen.

Unrelated but reading this reminded me of the classic article [Programming Sucks](https://www.stilldrinking.org/programming-sucks)

I have some toy search technology (indexer and query processor). I started in C and did a series of rewrites, learning and changing something significant each time.

On version 6 I switched to Rust (having only just started learning the language) and it feels pretty unlikely I'd switch back.

The rookie mistake isn't to want a rewrite. It's to ridicule something you haven't tried in earnest.

Nobody is ridiculing Rust. They are ridiculing flippant requests by non-contributors to rewrite non-trivial software in another language.

As a D user, I am looking at Rust with curiosity, but haven't yet seen any reasons that would make me switch.

I think Rust is such a well designed language. But as someone coming from more high-level languages I sometimes feel a bit lost due to not getting as much help from the IDE as I'm used to for C# in Visual Studio and Java in IntelliJ. For Rust I've only tried IntelliJ with the Rust plugin, which seems like the best choice for what I'm after.

Rust may hopefully get better autocompletions if they work on their Language Server implementation. C# compiler was explicitly rewritten so that it is IDE friendly and I fully agree with this decision. That said even good, old Java has excellent tooling support that I do miss from all these hip languages (such as Rust).

The compiler team is planning on following in C#’s footsteps with the rust-analyzer project.

I can only recommend trying it out. It works comparatively well. The problem with RLS is that it always needs working code to compile in between and it always loses state. I think it's a design issue at the core of RLS. rust-analyzer does this a lot better and faster, but is still in an early stage.

That would be great! One thing that makes working with a language pleasant is good tooling.

VS Code support is good too. https://areweideyet.com/

Such tables are tricky. What does it mean to have syntax highlighting? For languages like C, it means that keywords, numbers and strings are colored differently. But for languages like Java/C# it's not enough, expectations are different. For such languages you expect class names to be colored differently than variable names, and IDEs for other languages rarely provide that.

The level of IDE integration has little to do with whether a language is 'high-level'. In fact, the more abstract a language, the more likely it is to be difficult to do complicated analysis.

Why don't you rewrite D in Rust?

Because 'R' as a name is already taken. Even if it weren't, it would lead to too many bad pirate jokes.

Call it "Dust" instead of "R".

Call it 'IronDust', put it on the CLR, and have all of the meta jokes.

Sum types (and therefore a nice way to do e.g. validation or error handling). A better OO model (particularly when it comes to secondary concerns like serialization or comparison). Safe scoped resource management (with memory-safety without GC as a secondary benefit). A more standard/better-understood way of doing what D calls "compile-time reflection" (if I understand what D's doing there correctly).

Can you produce a string via compile-time functions and compile that string as a Rust fragment? That was incredibly flexible tool in D. The only thing that comes close is Lisp macroses.

Procedural macros take a list of tokens, and you give a list of tokens back, and they're then compiled.

https://doc.rust-lang.org/stable/book/ch19-06-macros.html#ho... (starts with derive, then moves on to attributes. Function-like ones aren't stable yet)

Thanks, that's similar, also I would argue that for inexperienced developer just producing desired text would be a simpler solution (but dangerous).

If you use the `quote` crate[1], it's pretty much identical in terms of developer experience.

[1] https://github.com/dtolnay/quote

IMO the fully general version of that is too flexible to be maintainable; it's better to have more constrained/standardised functionality that lets you solve the problems you'd need it for. Generally there are two kinds of things that you would want to be doing at compile time: producing code to handle the fields of a structure in a generic way (which you can do with frunk), or producing delegating wrappers to handle cross-cutting concerns (which you can do up to a point via the combination of built in delegation and first-class functions, though the lack of HKT does make some cases impossible).

a reminder that "rewrite it in rust" is not usually a ground up rewrite, it's an effort to translate the c into a safer language as you go, it doesn't just burn it down, salt the earth and start over. Have a look at remacs [1] to see how this works in practice.

1: https://github.com/remacs/remacs

And it will all be well taken if RIIR folks show things they have done themselves instead of telling others what they need to do.

Exactly, stop evangelizing and start doing.

I've dabbled with Rust a few times for hobby projects, but what always turns me off is how frustrating it is to work with strings.

If you haven't already, check out the Rust By Example section on this:

https://doc.rust-lang.org/rust-by-example/std/str.html

The frustrating part was concatenation, it was surprisingly annoying to combine a few strings and an int into a single string.

The fastest and easiest way is to use format!.

  let cat = format!("{}{}{}", s1, s2, n);

That doesn't seem too bad (Though it's still no C# with the + operator), I'll check that out next time I'm in the Rust neighborhood. Thanks.

No problem!

I personally only tend to use + when there's two or maybe three things at most, in any language, and then switch to an API like this if it exists for more complex things, no matter the language. I find it much easier to read. YMMV.

(And you can use + in Rust, but you'd have to call .to_string() on the integer first)

They should do string interpolation like C# since C# 5. Really easy to use, type safe and avoids a lot of mistakes.

It is possible to do string interpolation in Rust with a macro:

https://docs.rs/interpolate/0.2.3/interpolate/

steve, think I sent you an email before and you responded, but just wanted to say again the Rust Book is a work of art. Since I read it and emailed you, I completed a fairly non-trivial Grad School project entirely in rust. I took a lot of the shortcuts, but the rust book was an excellent primer.

Best intro documentation of any language I have tried to learn recently.

EDIT for those who havent seen it: https://doc.rust-lang.org/book/

Thank you! <3

When thinking of rewriting anything, the first thing we should ask is "why rewrite?", definitely not "why 'not' rewrite?", despite using the same language it is already implemented or using another one.

This is somewhat reminiscent of people who complain endlessly when a piece of software is still 32-bit, for purely nebulous reasons. People want things to feel clean and new, or they get an icky feeling that never seems to go away.

(2016)

...yet timeless.

I was tinkering with crosscompiling Rust to arm/aarch64 just yesterday. Rust has a ways to go before it can catch up to C/C++ for cross compiling.

Edit: I should explain a bit. C has had the concept of cross compiling baked into it for decades. It's a well established, easy path. By dint of C, C++ comes along for the ride. Lots of linker is magic required for bare metal C programming. Weird flags required for arch-linux-gnueabi cross compiling. Rust is very new and just doesn't have that history yet. It's an opportunity for someone who has a lot more spare time than myself. :-)

> It's a well established, easy path.

i do kernelspace development in C for embedded devices.

i would take exception at "easy". i assume you're saying that because you're able to use linaro's arm toolchains out of the box, straight from your distro's repositories. and you're right, that is mostly easy.

but as soon as you need more than simply compiling an executable to run on top of an incumbent kernel, since gcc toolchains are tied to kernel version, things get hard. unnecessarily hard. recompiling the compiler shouldn't be necessary. llvm-based compilers like clang address this in a more sane way, but aren't as mature as gcc.

and build systems for C don't support cross-compiling very well either. project based on cmake and autotools will usually work with a little tweaking, but if a project just uses makefiles you'll end up re-writing the build system 90% of the time.

rust's cross-compiling isn't quite there yet, i agree, but the language's (and the compiler's) design choices set the stage for a much better cross-compiling process than you get from gcc.

We should send emails to linus and ask him to rewrite linux in rust

I made a comment about this a year ago that still applies: https://news.ycombinator.com/item?id=16547890

... that was a much more calm and complimentary response than i've grown to expect from linus.

Everyone seems to act like the only option is to rewrite everything or nothing. Isn't possible to have rust and C talk to each other, and progressively rewrite the most relevant parts of the code in rust?

Yes, and was part of the core design of the language; that's how it's worked with Firefox.

You could make a case for sharing a wasm-style byte array with a C subprocess, but Rust objects won't be safe if C can address them.

Now now, let's be nice. We were all obnoxious at one point or another in our lives.

TLDR - there is too much code to rewrite despite the daily horrors of life-threatening and altering software bugs. Not that I think Rust will solve everything

Reply on news.ycombinator.com