Building a Cloud Database from Scratch: Why We Moved from C++ to Rust (2022)
risingwave-labs.com
risingwave-labs.com
Finally somebody understands this.
Rust is not difficult because of lifetimes, it just gets in the way of freely prototyping what you want.
This situation is slowly improving with the compiler getting better and better.
Then there is some annoying macro usage. Some Rust code looks truly alien.
I'm not a rust programmer, but I guess that's an issue if you come from a dynamic language, not from c++.
With Go and Rust I have to make a dedicated function somewhere and then have it be called after starting the program. Ain't exactly rocket science but the difference in time to do it and the convenience is still very stark.
I love working with Rust for the stability and speed it gives me but the value proposition and daily coding flow are VERY different compared to a dynamic language. With Elixir I am mostly just brain-dumping and stuff happens extremely quickly and fluidly, with Rust I kind of sigh and accept that the next 10 minutes I'll just be writing and writing. Might even zone out and make a dumb mistake because I have to spend more keystrokes and more time to do something I'll do in a minute in Elixir.
Maybe it's time to look for snippets support in my editor. Or start asking ChatGPT for them.
I don’t think being able to brain dump code is a good thing and it leads to an unmanageable mess when you have a large codebase. Types are essential as far as I’m concerned, but I constantly have to get into arguments with the dynamic type fans because their “flow” Is being restricted.
Brain-dumping is simply a way to iterate and figure out what works and what doesn't, quickly. After I do it I take the requisite time to emulate exhaustive pattern matching, and add tests that cover the functionality well.
My problem with Rust, and I love the language a lot, is exactly this: I want to quickly find what would work. After that I'm very happy to take the proper time to go things rigorously.
And please don’t say tests solve that because they don’t. They help but they don’t solve it.
Dynamic languages are very valuable when you can afford to iterate and gradually find out the golden path. For the better or the worse, many businesses are of that type.
That being said, I just ended an Elixir contract and absolutely want to code Go or Rust for money again.
I tend to think more than write code, so usually my first go is reasonably close to what I need. But there are always edge cases you just never think about. I'd rather have a strong debugger than a repl imo
You are unfortunately very correct (at least in my 21 years of experience as well). I too feel annoyance when I know that I made a piece of code work well but (a) every now and then I am truly wrong and (b) various pieces of the system in the code interact in sometimes unexpected ways, unveiling inputs to your code that you haven't foreseen.
So even though it often takes a heavy and annoyed sigh out of me, I still roll up my sleeves and add the tests because I've shot myself in the foot too many times, and ignoring past experience is just being a stupido.
As for debugger/REPL, they are not orthogonal; you can have both. I'd more contrast debugger with tests themselves -- both are ways to go step by step through a process that you know is faulty somewhere.
REPL to me is just a way to more quickly sketch a v1.0 of a piece of code, nothing else.
https://depth-first.com/articles/2020/09/21/interactive-rust...
https://github.com/google/evcxr
The other option is to use the playground, https://play.rust-lang.org/?version=stable&mode=debug&editio...
with {:ok, id} <- get_id(),
{:ok, val} <- get_val(id)
do
show_success(val)
else
{:error, e} ->
# We reach this if either get_id or get_val fails
show_error(e)
endSyntax is easy enough. Like all programming languages, you'll get used to it in a weekend and won't even notice it afterwards.
Elixir is absurdly productive: very terse code, very self-explanatory stdlib API, transparent concurrency / parallelism, and in-OS-process high availability and mostly-self-healing hierarchy of green threads and supervisors observing them.
People on HN and Reddit really love to roll their eyes at stuff that's getting popular, and likely will avoid the technology out of a misplaced spite, just because a lot of people are talking about it. It's a weird phenomena.
(Not saying you're doing it, I just kinda got carried away here.)
foo.bar(baz)
bar(foo, baz)
I'll have to check how does NeoVim fare with these things.
Is it meaningfully harder than c++ in this regard?
I'd have an easier time prototyping in Rust than C++, and I've been writing C++ for 10 years and Rust for a little over a year.
However, prototype in something like Ruby (or even TypeScript, and some people mentioned Elixir) is in a different universe compared to C++ or Rust.
Like, it's not even a "fair fight" to compare it meaningfully.
I find that modern C++ "kitchen sink" approach to features makes it good for rapid prototyping, a bit like Perl on the dynamic side. It won't force you into a specific approach, pick the one that gets you there the fastest. Use raw pointers and malloc(), or use fancy smart pointers and RAII, your choice, you can also do both, you have exceptions, you have goto, you can write(), printf() and cout, you have classes and lambdas and (multiple) inheritance and traits. More options are better for rapid prototyping.
Also, C++ is the most popular language for competitive programming, you can't get more "rapid" than that.
Now that I say that, I guess someone could write an "unsound helper functions" library that helps you write code full of UB conveniently. It could be an interesting thought experiment. But I don't think the community would be very happy about it...
I've seen way too many half baked attempts at making prototypes fast that do end up in complete rewrites.
Go tell that to a C++ programmer!
;)
If it fits the borrow checker well (like in CLI apps, stateless servers, data transformation) then the borrow checker fits beautifully and seamlessly.
In other settings (complex turn-based games, some compilers, GUI), the borrow checker can cause some artificial complexity and prototyping slowdowns compared to other paradigms.
In fact I like Rust for prototyping precisely because I don't yet have tests, and the extra compiler diligence frees me to focus on the prototype.
I don't think that the parent comment implied prototyping in dynamic languages but rather in statically compiled ones.
FWIW in C or C++ you don't have to do as much code gymnastics to satisfy the compiler so consequently prototyping is both faster and easier when compared to Rust.
When I open something up to read it, to try and understand it, to dig in and fix something broken… having everything be exposed in the explicitly written code… is vastly more useful to me than trying to mentally interpret a language as I read it and remember the rules, the meta-programming rules… and all the implicit stuff that can affect how the program will behave.
Lot of programming languages aren’t good at this explicitness, and I’ll be honest, it’s a genuine struggle for me picking up the languages like this, only made worse when the documentation is very prose like and fails to convey how anything actually works, eschewing that in favour of, at length, demonstrating the feel of a programming language through countless demonstration example that lack mechanical “what is this doing under the hood” explanation.
Rust may influence us into patterns that are better in some (likely even most) situations, but not always. Sometimes the situation calls for other approaches, and that's okay.
Though, people tend to gloss over that C++20 is decently comparable to Rust in terms of semantics and safety, the issue is very few C++ codebases are modern and the footguns are still lurking for the careless.
That said, Rust iterators are very nice and the async story completely mocks the C++20 coroutine nonsense.
* unreadable coding style: This is not a C++ problem. * memory leak: Memory leak is an old C++ problem, since C++11 there is no reason for not using smart pointers.
The only point that could be attributed to C++ is the segfaults perhaps, due to its lack of safety. But your other two points made me skeptical about Rust bringing you much benefit. I am specially concerned about throwing away months worth of code. Rewriting everything from scratch is what newbies love to do and will most likely get you in the same mess you were in. Try to learn with the error of others [1]. And btw, the problem is usually who writes the code and not which language is used.
[1] https://www.joelonsoftware.com/2000/04/06/things-you-should-...
In C++, threading was my only way to execute parallel tasks, Python 3 showed me a about coroutines, which I've really started to love.
So in Rust I'm experimenting a lot with Tokio right now, and I'm trying to duplicate what I've made in Python. I often think about how good this compiler is that it won't let me do things which I clearly would have done in C++ either to cut corners or, more importantly, out of lack of understanding the consequences.
I was creating this WebSocket server which would serve as a bridge to a MQTT broker, all in async code, all in one big file. I was constantly fighting with the compiler about reusing variables or handing them over to tasks.
When I then attempted to refactor the code by splitting it into several files, the core of the server in main.rs, some WebSocket-related stuff in websocket.rs, all WebSocket-stuff related to the communication with the clients into websocket_handler.rs and all MQTT stuff in mqtt.rs, all my fighting against the compiler fell together. In my inner eye I could see how the variables were handed over to which file during execution, and that this was the reason why I was no longer allowed to use them in the previous file. So much logic, so clear, and the compiler forced me to do it that way.
Then there's the ease of integrating external libraries, which, at least in Windows back then used to be non-trivial in C++.
That said, most code isn’t a hot path. Performance doesn’t matter in most programs. And if you’re a veteran C++ programmer, none of this will come as a surprise.
Allocating a new String/Vec every single time you do something is killer for performance, but if you do it once up front then clear the data structure for the next use it should be performant enough.
That's funny. I've been going mostly the other direction. I'm avoiding mutable structures whenever possible. I have a much easier time reasoning about stuff when I know that things aren't going to change mid-life.
Also, that is exactly what Rust's borrow-checker is there for. It doesn't allow you to mutably borrow the same data structure multiple times.
Slower than V8, definitely possible. Slower than Go, very unlikely, since it doesn't have a generational or compacting gc.
I received a review comment some time ago about changing a line of code in order to spare one allocation, for a call that was executed _once_ in the whole lifetime of a program. That was not the only time I've observed this attitude.
I think starting with being handy with profiling and only after measurement, thinking about solutions, rather than starting with the idea that certain patterns are harmful per se.
There are many reasons for not using smart pointers, first amongst them for me being performance.
Smart pointers do allow you to remove a large class of memory bugs from your code (albeit not memory leaks) in the same way that Rust's safe references do. However C++ smart pointers impose a run time performance penalty on your code, whereas in Rust the checks happen at compile time leading to better runtime performance.
i'm not sure if this is the case. unique_ptr doesn't really do much other than use the type system to ensure that only one instance of the pointed-to value exists at once. as far as i understand, it doesn't strictly do anything at runtime.
For shared_ptr, the atomic ref counting can cause surprising overhead due to contention in multi-threaded scenarios, even if just accessing the pointer, depending on how the shared_ptr is passed through functions...
Much nicer than hacky ifdefs.
This is exactly what I need. For both, C++ and Rust smart pointers.
Can you share any more info on how to set this up?
https://sourceware.org/gdb/onlinedocs/gdb/Skipping-Over-Func...
https://www.youtube.com/watch?v=rHIkrotSwcc
At 18 min.
Technically it is correct, it causes an extremly light overhead due to the inability of the compiler to optimise a part of it.
In practice, even in HPC, I never notice it to have an impact ever.
And as an advise for any new C++ programmers: Please JUST use unique_ptr, everywhere, all the time...
Trying to manage the lifetime manually with new/delete in modern C++ is not worth the cost in 99.99% of the case. If the impact of unique_ptr is noticeable in your program, you are very likely in a realm where you should not even doing memory allocation anyway.
What the "overhead" of unique_ptr is usually attached to is the inability for a compiler to pass the unique_ptr as a value through a register but will instead have to use the stack (memory). And that even doesn't apply in the general case but _only_ for unique_ptrs holding an object with non-trivial copy-constructors or non-trivial destructors. This is due to the platform ABI and not the C++ compiler limitation.
Anyway, calling that an overhead is a far stretch and almost purely theoretical unless someone is able to measure the negative effect of such code transformation in the real-world codebase. And I say this as not particularly heavy user of smart pointers.
Also, many other codegen transformations will not fit into a very limited amount of ABI registers so should we argue about not using those as well?
In my opinion, this was only a "campaign" of Google trying to use a unique_ptr as a leverage to persuade the committee to accept their break-the-world ABI suggestion. We know how it all went.
Ye well I knew the overhead was really small.
But even if you had to, what implications would it have, if you care to explain?
Stack overflow says "delete" would check for null before deallocation: https://stackoverflow.com/questions/4190703/is-it-safe-to-de...
Happy to be educated
That said, if this really had been an overhead, virtually every language out there would suffer from it, including Rust. Every language out there at some point needs to call into the libc.
Also, even if you chose to use some libc for system calls, there is no reason to use malloc()/free(). You only need mmap() or similar, and then your own language runtime can write their own user-space memory manager.
As a side-note, on Windows, libc isn't even a system library. The official way to execute system calls is using win32, and things like malloc()/free() are simply wrappers around HeapAlloc() and HeapFree().
PS: small overhead is technically still overhead; yet, I am not saying you should worry about it
Overhead is almost purely theoretical and it cannot be avoided and is not anything common or specific to unique_ptr's.
Unreadable code is not a C++ specific problem, but does Rust make it easier to write readable code? Probably. Same for memory leaks and other similar problems.
The C++ programmers that love over-complicated, read-only code would be right at home in Rust. In fact they’re probably working on it right now :-)
Smart pointers lure you into a dark alley where each tiny object lives in its own tiny heap allocation, and before you know it you have so much memory management sprinkled decentralized all over your code base that the memory management overhead becomes a performance problem, but at the same time it's too late to do anything about it because it would mean rewriting everything (this is then usually when the desperate search for a silver bullet like a "high performance memory allocator" starts).
(not that Rust is necessarily better in that regard when people start to work around borrow checker restrictions with Box and Rc...)
If this is starting to change then it's a good thing.
Note that while smart pointers ease pain in the short term, there are subtle complications that arise from the constraints that they introduce. One is performance, in some cases the cost of generalized memory management can be too much. Memory fragmentation can be an issue too. RAII the mechanism is not free but requires compatible code and containers. I'm sure there are lots of projects that have broken under the added constraints introduce by such smart classes.
And while smart pointers are orthogonal to systematic memory management, if you get the management right the use of smart pointers can easily become a net negative.
"It is complicated so let's see if there is a simpler approach".
I can see it repeated very often, but can you point me to any evidence for such claim, based on modern low-pause GCs and real data (not simulation)?
I find low pause GCs have pretty substantial overhead and have way lower allocation throughput than old stop-the-world GCs. And the difference between manual memory allocation wasn't that big either (still within 2x).
This was my first thought, but then I realised newbies would have started with Rust. :)
In the first few months of a big build a huge amount of time and effort goes into the foundation: design, POCs, setting up environments, CI, testing infrastructure…
My guess is that they weren’t very deep in production code, and the fact that they didn’t fall for the sunk cost fallacy speaks highly for them.
I've a feeling it's easier to write decent Rust than decent C++.
The post contains comments about how difficult they found it to keep coding style consistent and language feature use consistent. This suggests to me an inexperienced team without the skills to manage a project of the size and complexity they need.
Now whether with the right team they could produce a better solution in C++ or Rust, i've no idea, but if this is the team they have, then it sounds like Rust might be the right choice for them.
As the system grows, you realize all that was wrong with your previous version, and can write a new better one.
That doesn't mean you need to switch to another language to do it though.
In particular for a product like this which is itself targeting developers.
Your point that a rewrite is very risky is well made regardless though.
I would argue that it is, after a few years of Rust. My style in C++ was formed by my exposure to other people's code. I guess this is true for most people.
Certainly, in C++, as well as in any language, Rust included, there are a myriad of ways to express yourself when solving a problem. Is my C++ coding style unreadable? I don't think so. But I used that language for 30+ years. Let's say I only used it for five years -- what then? I would have much less exposure to other people's code and hence to well written/readable code.
Rust has rustfmt (a code formatter) and clippy (a very opinionated linter). And they get used lots. Most Rust CI now includes `cargo fmt --check` and `cargo clippy` runs than need to come out clean.
This goes a long way in making code canonical. Clippy's amazing suggestions also made me a much better Rust developer in a much shorter time.
Are rustfmt & clippy part of "Rust the language"? No. But in a way they are since there is nothing else they compete with for opinion on how Rust code should be formatted and written.
When you onboard new people and they have adapted the pov that this makes lots of sense, it goes a long way in avoiding code being produced that is difficult to read for others.
And because of this, I do indeed think that it is a problem of a programming language, in 2023, if the default tooling does not include such utilities. And a strong incentive through the community, pretty much all the available learning materials and repos in the wild, encouraging their use.
I.e. often there is a mix of styles, indentation/line wrap preferences and even the occasional tab in a codebase that otherwise uses spaces. Etc. etc. And that's just the formatting part. Linters seem to be mostly used manually by experienced developers but rarely are part of the commit pipeline or even if they are, their suggestions enforced.
Your mileage may vary. I mostly look at code from a very specific industry. But I can't help but notice these things now, because of my exposure to Rust.
C++11 did not invent smart pointers, pretty much every C++ project had already been using them long before they were added to the standard. They help with leaks to some extent, but are not a panacea, I can do a memory leak with shared_ptr in just a few lines of code. I've seen this line of reasoning many times before and it needs to stop.
Sadly there is, lack of security culture and plenty of devs won't allow them on a PR.
Wow. Genuinely surprised.
Rust is naturally safer than C++, however there are still many domains where a bit of yak shaving is required before actually coding what one cares about.
So it boils down to where to spend resources, and as note, I am definitly not a fan of Orthodox C++ ideas, rather having C++ improve its safety story as much as possible.
Orthodox C++ can be safely ignored by any sane developer.
What both of those had in common was a single strong tech lead that would veto code with no argument if it didn’t meet standards.
I’ve yet to work on Rust but I’ve seen horrific and buggy code in C++, C, Java, Kotlin, C# and of course JavaScript and Python.
- Their product is still relatively new (7 months of coding isn't that much) and not even released,
- they picked a technology specifically designed for the problems they've identified on their existing codebase.
- they completed the rewrite (and weren't stuck having to maintain the old codebase), and are happy with the result.
The risk is losing the current user base.
For this particular software: No users, no market penetration, no risk at all doing a rewrite.
In C++, you need to use a plain pointer or a unique pointer. The former makes the function leak when passed a pointer to heap.
The latter requires a heap allocation and free for the struct contents.
And, again, people talk about it as if it was a real thing and they knew how to measure it...
Also, C++ and Rust are very similar in appearance. So, I struggle to see how jumping ship here would be a big win.
If they get it fully working and stable I agree modules should go a long way to getting rid of it (entirely if you are willing to wrap old header files in modules yourself).
All my hobby coding in C++ uses modules nowadays.
There is plenty of C++ tooling for GPGPU, game engines, IDE, GUI programming, HPC,... that Rust still lacks.
In what concerns cargo, vcpkg and Conan are good enough, and support binary libraries, no need to compile everything from scratch.
The camp that thinks Rust is just unnecessary fuss because "you can do it all" in C++ already. They pride themselves with knowing C++ esoterica, and don't like that Rust lowers the entry barrier to writing similar software. They want an exclusive club. They also see ownership as a nuisance. They know you can be equally reckless in Rust too, but they don't like that Rust is so "in your face" about it, too.
The camp that is afraid Rust will marginalise C++ and hence push them towards undesirable (even if high paying) jobs.
There's the camp that's jaded by seeing too many fads come and go, and probably just burnt out at some point. So, they just want to work with what's there and be left alone. They don't want to learn something that they know (incorrectly or not isn't for me to judge here) will be dead in the water soon. And they'll have wasted their energy on it for naught. Or in short, they see it as an unwelcome distraction. This camp also includes "but it has no ISO specification for aviation/etc" too.
I think all these are wrong.
I don't think C++ will be marginalised, it just won't be the default choice for certain domains.
I don't think the extra tooling is a nuisance, if anything in my experience it unburdens the mind from non-domain concerns.
And I also don't think it's a fad, given how productive the language makes people feel, and the fact the Rust Foundation is very serious about the longevity of the project, and that it has many big-player backers.
There's also obviously others too; people come in about 8 billion flavours.
The original statement and the one in parens are directly at odds with each other. If every C++ dev requires years of experience and studying to use correctly, it follows that they have left years worth of code that is not done correctly in their wake. Also, no other (non-esoteric) language requires this high a barrier to write correctly.
> if you can use it relatively well you can write safe code without much trouble
And yet there isn't a single large C++ project that doesn't suffer from violations of memory safety - the one class of bugs that we know how to eliminate completely, if using a proper runtime system.
Do we? Why exactly do we need sanitizers in Rust then?
My biggest issue with most Rust posts is that they set up strawman C++ arguments (that are usually just C code targeting a C++ compiler). I'm sold on all the idea behind Rust and making sure that programmers can't just easily drop down to insecure parts of a language - I just don't think we should be shaming good C++ codebases because we want Rust to "win."
Well, the syntax is alien and new, it doesn't do OO the way 9 out of 10 working developers expect it to, almost all C++ popular design patterns have to be rejected, many of the claims("fearless concurrency") are exaggerated and the Rust evangelists are really really toxic when referring to C++ or C, often simply claiming that fewer bugs is worth the tradeoff from switching to Rust.
I literally saw a comment on HN this past week complaining that OSes are not switching to Rust, almost completely ignoring the fact that any rewrite of a 3 decades old codebase is going to introduce more bugs than it is going to prevent, even if you have perfect memory safety (which you won't, in an OS).
In all honesty, Rust would have been adopted orders of magnitudes faster had Rust evangelists been welcoming, and not accused of being defensive or hostile.
I mean, look through this very page full of comments, and be honest, how many of the comments talking about C++ developers are welcoming to C++ developers?
The majority of Rust programmers I know were C++ programmers in the past. Many, including myself, use both professionally.
Does this, to you, look like a response that is welcoming to C++ devs? I gave careful and impassive reasons, and you strike back with literal ad hominem attacks.
This sort of reply is what makes Rust, as a community, look shallow-minded and toxic.
That jumble of ideas is a good example of the arrogance directed not only at C++ programmers, but also Golang and C.
- Smart pointers
- auto and const references all over the place
- RAII
- Pure data structs
- Free functions
- A little bit of templating but not too crazy
Modern C++ reads a bit like JavaScript strangely!
I didn't say it was.
I didn't say anything about best practice, only about familiarity.
I yield to your comment.
I like the language. I really don't want to deal with their community.
There isn't another live to live until 30 "again", and then be within the Rust culture.
Rust has, by analogy, stapled it's "Ninety-five Theses to the door" of C++ and those C++atholics are afraid their way of life is ending before they do.
False BS
>"those C++atholics are afraid their way of life is ending before they do"
We are into insults area now.
>It’s basically a compiler enforced set of C++ best practices
Its not, its a whole different programming paradigm where the compiler dictates how you write your software. You dont let your creative facilities to drive you or whats best for your domain - you just obey the language rules...
> But as more and more engineers joined us, some shortcomings of C++ came to bite us: unreadable coding style, memory leak, segmentation fault, and more.
One advantage of Rust of C/C++ that is underreported is the that Rust makes it easier to operate a team with diverse levels of experience/expertise. One or two senior engineers set up the tone, design and coding standards. Junior folks try to follow but if they screw up Rust offers more checks and guard rails limiting the blast radius. Many bugs simply do not make past the borrow checker and type system. This also helps when churn and attrition happens in the team. I witnessed quite a few horror stories when aspiring but not-yet competent C++ coders wrecked havoc in C++ code-base when seniors were busy with something else.
- Seasoned devs but they never heard of clang tidy/format - Memory leaks on a new db codebase where each component should have clear ownership of data or pass to the next pipeline/stage - you dont even need smart poirters here, just a half decent design. - I am not even going to mention sanitizers, probably rocket science for them
This is especially true if you are trying to use those libraries in multi-threaded environment.
C++ has been around for... what?.. 40 odd years? And any non-trivial project still links with C libraries.
I don't see this problem somehow going away in Rust w/o some revolutionary changes in operating systems, which is the major supplier of decrepit but unavoidable libraries you have to link with in order to get anything done.
The problem was not the seasoned core of developers, but the fresh hires.
Now, this would be an interesting problem. Just as Python has PEP8 and some other programmable coding guidelines, we should have different sets of modern programmable C++ coding guidelines, which can be checked by clangd.
And I mean modern, logical, Rust challenging, C++20 coding guidelines. Something this particular team would follow. Nonsense like Orthodox C++ doesn't cut it.
C++ and Rust are simply not competing on the same level. You don't need a complicated "enterprise" building farm/cluster to build your project, dependencies are managed by the guys who also write your compiler, there is a growing community that actually listen to your feedback and you don't need to wait 40 years to get networking support into the standard library. What C++ can offer? Languages like Rust, Golang and Zig all want their users to focus on their applications, while C++ just wants you to focus on C++. OO was a cool concept back in the 90s, but that was 30 years ago when you have 8MBytes of RAM if you are lucky - who wants to model every program using OO in 2023?
It has never been a better time to depreciate the C++ language and codebases written in C++. Museum is a much better place for C++.
First of all, that blog post - as many Rust-themed blogs from start-ups - is a submarine article attempting to increase the notoriety of their product. Not enough to disqualify it, but a hint that it won’t necessarily be technically solid.
And indeed, on a technical level it’s superficial and doesn’t give any code examples to substantiate its claims. Code is critically important, because as another recent blog showed, some people don’t know why it’s a bad idea to pass non-zero-terminated char buffers to C string functions and then turn this into a big story about the dangers of C++.
The story also has some red flags, such as not being able to enforce coding standards. Code review is a near-mandatory quality control and teaching method that easily handles this. Clang-format and clang-tidy take care of the low-hanging fruit.
Having segfaults is a really bad sign. So is not mentioning the words address sanitizer, valgrind or similar.
Having memory leaks in 2021 is incomprehensible. This is an amateur mistake of the worst kind in modern C++. It means they’re writing C.
I don’t have time to dig more in depth, but there’s a disturbing lack of critical thinking and skepticism in these comments.
Embarrassing for pointing out that OO is dead and it is not acceptable to ask users to wait almost 40 years to get networking support into the standard library? No, C++ standard committee members should be embarrassed, not those disappointed users.
> that blog post - as many Rust-themed blogs from start-ups - is a submarine article attempting to increase the notoriety of their product.
As any article published by an commercial entity, it is always about increasing its economic benefits. Personally, I don't know anyone from the company but their blog is just one more success story of depreciating C++ and I applaud for that.
> Having memory leaks in 2021 is incomprehensible. This is an amateur mistake of the worst kind in modern C++.
Sorry but I stopped reading after this. I've been writing software for the past 20 years, worked for the biggest names in industrial, had my PhD in CS about 15 years ago, I write code on almost daily basis (Linux kernel modules, database kernel, infrastructure kind of code). I make the above mentioned amateur mistakes pretty often.
Not saying I am highly skilled, just saying if I can make such mistakes after working 20 years in the field, it wouldn't be rare for other people especially those young developers.
btw, I recently had a nasty memory leak bug in golang that took me a couple days to reproduce and debug, yes, in a GC based language.
This one hurts, but so true =)
Although Rust is an improvement in many ways, I actually think it is becoming more C++ like with every release. I think it has a good chance of over-taking C++'s complexity, even if it has a much better user story around build & package/module/library management.
My entirely egotistical opinion is that the maintainers of the language have to take a very good and critical look of the current status quo and start systematically killing off any complexity they can.
Personally I don't want C++ 2.0. Tire people enough and they'll just move to another language again. I know I have, several times in my career, and will do so again if I have to.
For what it's worth, the community, library authors, and most importantly the language design teams are all very aware of this. And there's efforts being made to make all these pain points things more ergonomics and approachable.
I think it just a slower process than we'd like. I think in part it's so slow might be due to how the decision making and work is structured; it's very open, but also slow.
Stuff like GATs took ages to design and implement, and yeet and parts of async are similar. These are done very carefully and they also let decisions marinate for a while before further action.
It's not a critique, since I think it's good and wouldn't know a recommendation to speed it up.
So, it's getting better, but slowly
Perhaps I'm worried I'm just not smart enough to be a Rust programmer.
Off topic: I also find the Rust community unwelcoming and very clique-ish, I get the very strong "you're not apart of the cool gang" vibes and I'm too old to care to navigate this.
Rust's Discord however is AMAZINGLY friendly. I've been contacted in DMs by maintainers of very prominent and popular libraries by outlining a problem that I had, in detail.
And finally, r/rust I witnessed first-hand being extremely toxic at times.
It really depends where you're looking.
And I 100% agree that their Discord is great. Also recommend it.
I've seen lots of activity on beginner and intermediate questions though.
As well as just chatting to people, or helping someone with an interesting problem.
I've had a really difficult problem ignored on Discord, but I also think the threading UI on there isn't great for lots of concurrent and unrelated discussions. I just found it frustrating to use. So, maybe some questions fall through the cracks?
Managed to resolve it in the end though. However, having typed all this, I realise a wiser thing would've been to ask on StackOverflow instead lol.
The thing about GAT is that it was never meant to be a feature. It just removes a non-obvious limitation from the language. I remember when learning Rust, I expected "GAT syntax" to just work, and was surprised when it didn't.
So, for me it makes the language less arcane.
> Perhaps I'm worried I'm just not smart enough to be a Rust programmer.
I think everyone can learn anything. It's just a matter of finding the right angle to understand a concept. If something hasn't click, force yourself to use those parts until they do.
> Off topic: I also find the Rust community unwelcoming and very clique-ish, I get the very strong "you're not apart of the cool gang" vibes and I'm too old to care to navigate this.
Ah maybe. I've not experienced this personally, but I've seen others make similar claims. It's generally aways very welcoming from what I've seen. There's the occasional smartass on their Discord, but that's just... Discord Things(TM). People ignore them, and if they're obnoxious they get called out and asked to chill out a bit.
Out of curiosity, what are your main interaction channels/places with the community? Maybe you just accidentally navigated into ones that don't suit you personally
I arrived at the same conclusion. I have faith in Rust's maintainers and I know things can't happen as quickly as I'd like them to. I am still human and can rage when something doesn't go my way -- and it looks easy to me as a side observer -- but I don't judge and I applaud the team's dedication and rigor.
You are expected to use one specific compiler. And there's very little (apparent) interest in actually writing a spec, so that other people can actually try to implement their own compilers with a working borrow checker, compilers capable of compiling rustc without sacrificing soundness (so far the compilers I've seen are lacking borrow checker, for example).
My only remaining hope rests on the Ferrocene people.
So I can see why Rust is being used for user applications, but until that monoculture issue is fixed, I don't really see it replacing C, C++, much less SPARK, as a language for critical systems.
It is my sincere hope that having Rust in the Linux kernel, and in GCC, results in Rust getting an actual spec, so that we can finally know what is expected compiler behavior and what is a bug in rustc, other than "what the rustc from this specific repository does".
Like, for example, Rust 2018 edition is frozen (AFAIK), so that would be a good target for a baseline spec. Could be a more recent edition, even; it doesn't really matter as long as it's something.
Cargo sure does, but support for rust outside of cargo is much, much worse than equivalent support for C and C++. It doesn't matter for self-contained projects like simple Makefile ones, but at large scales, moving everything to cargo doesn't seem reasonable right now. Maybe eventually?
On the flip side, Rust makes the usual cause very easy, with the ability to to pull in code with a single line in a config file (I would argue this is probably a double edged sword - I've some some Rust projects that look like they're a javascript project with a thousand dependencies). I think the "simple" case of compiling Rust without cargo is more complicated and has a higher friction.
Perhaps this isn't fair, complaining about Rust's build system when you're off the beaten path because it's difficult with loose footing. C++ doesn't really have a beaten path (Well since C++17 it's slowly become CMake), that's why all the routes are so well trodden...
A lot of folks use Conan or vcpkg for pure C++ dev.
C++ is also becoming better with every release, and C++20 is a huge improvement.
In all honesty, code written in C++20 feels just like code I have written in Python and/or JavaScript. And 20–40 times faster.
> C/C++ is undoubtedly one of the most popular programming languages for building database systems. Most well-known database systems, including MySQL, PostgreSQL, Oracle, and IBM Db2, are created in C/C++.
Considering C and C++ to be the same language is quite a stretch. They are fundamentally different.
C code will not accept C++. A C compiler will not support C++ (unless it's explicitly designed as a C++ compiler too). A C-only programmer will be as lost on a C++ codebase as on a Rust, Go or Java codebase.
What can’t it do.
And like Linux it’s written in C. Not that means much.
“Never bet against Postgres”.
[1] https://en.wikipedia.org/wiki/PostgreSQL#Release_history
Why not combine Postgres for OLTP workloads alongside another database for fast analytics/time series, optimised to deal with high throughput ingestion and low latency queries?
NB: I am one of the co-founder of an open-source time series database (questdb)
And it's still too easy to shoot yourself in the foot, by deadlocking, or using up all the server's resources with a query that looks very similar to one that was fine.
Almost the same here, have tried just about everything that comes along, and sooner or later I end up swapping out the database de jour and go right back to sql server in almost all use cases (for the type of projects I do).
I feel like db's are pretty much a solved problem, and can't imagine trying to build and market a brand new one these days. Almost as hard as trying to come up with a new operating system and convincing people to switch.
I wish good luck to anyone that tries, but talk about pushing a boulder up a mountain; definitely doesn't seem like the easy path to success.
There was also Informix, SYBASE and DB2, but those hardly matter nowadays.
Fail over without downtime or operator intervention.
Async I/O.
Stop larping as graybeards, and try to actually do something instead of endlessly bike shedding your cargo cult.
This really looks like a case of not seasoned developers throwing crap at the wall and seeing what sticks. I doubt their Rust implementation will go any better.
About Rust... I'm open-minded, let's see what happens.
Forget unique pointer or smart pointers. People had been coding in C++ far before that without fearing memory leaks by following RAII principals.
Second to it is, probably, concurrency. RAII isn't going to save you here either.
Then there's also input processing where you may run into a situation that causes resources not being de-allocated because you never anticipated a particular shape of the input.
RAII works for very local and very well-defined things, it's not a good answer to dynamic situations or resources with complicated ownership.
It’s about a team of allegedly experienced C++ developers at a start-up that spent seven months building a project only to spend another two months rewriting it in Rust and then a few more months moving from async to Tokio. But those lost months have not been in vain, because they have 1600 stars on Github. They actually bragged about their Github stars.
It seems the Rust euphoria has reached such levels that people have stopped thinking critically both about the inexplicably bad business decisions and the near-zero technical content described in the article.
I bet most candidates to C++ job offers end up discovering the hardly reality of existing code.
How many styles do we now have on C#11, Java 20, Python 3.11, PHP 8.2,....?
This is why static analysis and linters are so relevant for keeping teams into a specific way of doing things, it is not like we can always justify start from scratch into simpler languages.
I couldn't agree more with this comment. As great as the community is, there's a lot of projects that are just not maintained. I wish this was different.
At first I even thought it was an argument against Rust, not in favor...
Coroutines in Rust are a native language feature and their API is part of `std`, but the executors indeed come from community libraries.
Tokio is one option, but as long as you don't contextually fork (for parallelism) you can in theory run the same coroutines on any other executor, like the ones from async-std. Spawning a sibling task (as opposed to mixed child task polling, which can be macroed inline) unfortunately isn't available through the common API so far, which is where the bulk of the mentioned incompatibilities stems from.
In other words, it seems Rust co-routines at the level of Tokio are not as performant as C++ native Co-routines [2] and further down the thread, it's mentioned that once generators are stabilized, it might help build lower cost and performant stackless coroutines in Rust.[3]
[1] https://github.com/rust-lang/rust/issues/43122#issuecomment-...
[2] https://github.com/rust-lang/rust/issues/43122#issuecomment-...
[3] https://github.com/rust-lang/rust/issues/43122#issuecomment-...
If you'd like to do modular data processing instead then the current best Rust approach is using `Iterator`s[1] and possibly the `rayon`[2] crate for safe parallelisation.
There currently is no coroutine feature that maps nicely onto `Iterator`s, or rather that's indeed expected to be the unstable generators feature, as it should cover that use case without overhead. (Async-coroutines map onto `Future`[3] instead.)
You can think of Rust's `async` and generator use cases vaguely like the ones of JS's async and generator functions. The latter will function without any runtime or trampoline if you advance it in a loop, as far as I can tell, but they're unwieldy if what you want to yield on are only waits on external events. (`Waker`s[4] have similar consumer semantics to JS's or C#'s continuation passing, even if they are only indirect memory management handles.)
It also often makes sense to use both of these at the same time to produce something like an `AsyncIterator`[5], but I expect that feature to be quite some ways away in Rust, at least in a seamlessly mixing fashion.[6]
- - -
I'm actually pretty curious which will turn out faster in practice between C++'s coroutines and Rust's generators, since their memory management is so different.
The Rust coroutines (that is: including `Future`s) are generally stackless already, but also by default heapless, as their entire memory is abstracted as opaque instance with anonymous type that is returned to the caller by value. Boxing them is an explicit operation, and it's equally possible to pin them on the current stack/in the current instance instead. (That's how `.await` works; the child is embedded into the outer `Future` directly.)
There are some pitfalls related to this, since that instance's size is determined by the largest `.await`/yield site, but the tooling should (eventually) be able to warn on unexpectedly large `.await`/yield locations like it currently warns on unexpectedly large enum variants.[7]
(I'll borrow your footnote style.)
[1] https://doc.rust-lang.org/stable/core/iter/trait.Iterator.ht...
[2] https://crates.io/crates/rayon
[3] https://doc.rust-lang.org/stable/core/future/trait.Future.ht...
[4] https://doc.rust-lang.org/stable/core/task/struct.Waker.html
[5] https://doc.rust-lang.org/stable/core/async_iter/trait.Async...
[6] then comparable to JS, I'd assume, where `yield` and `await` aren't necessarily paired: <https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...>
[7] https://rust-lang.github.io/rust-clippy/stable/index.html#la...
Also seastar comes with coroutine supports[1], and incidentally it was written for a very high performance C++ database implementation.
Rust's coroutines currently have practical optimisation issues related to instance size (because the compiler isn't smart enough yet in a few places), and the more powerful generator API very much isn't ready yet, but they give you considerably more control over the memory management. It's easy to stack- or pool-allocate coroutines there if necessary for performance, while in C++ you wouldn't be able to use the native language feature where frequent or unmonitored heap allocations from fine-grained coroutine use would be a problem. (Apparently this makes them unsuitable for AAA game development, for example.)
Library-provided coroutines may be more versatile.
Rust coroutines evaluate to opaque value types instead of handles, for comparison, so the caller gets to decide what to do with them. Currently they're a bit unwieldy though, until naming (aliasing) their types becomes possible. The unstable documentation for that is here: https://rust-lang.github.io/impl-trait-initiative/explainer/...
(Edit: Sorry, I'd grabbed the wrong second link. Fixed.)
Seven months, 10+ team but let’s average that over time to 10. Let‘s take 20 days/month and we arrive at roughly 200LOC per day. C++, database server code. Interesting…
With Cargo I can build code that just runs. With Python it is always a gamble. Different arch? You need to install different packages to compile the C/C++/Fortran code if the library author did not care about WHL. Starting up an application always a gamble, do we have all the deps? Did we miss some?
And so on. With Rust + Cargo I have confidence that the executable runs. Yes, the compilation is an extra step, but I would have that trade every single time for extra safety and reliability.
It sounds like you'd be better off with higher-level Haskell for these tasks.
Have you got a link to that request?
They certainly pay better, upfront, than a startup does.
I would like to know what the core authors of Materialize or InfluxDB-IOX have to say about using Rust for building a database. They've found the dead bodies, if there are any.
I would love to hear how you explained this to your investors.
Oooff.
Yes
So I totally get why people want to make better database software, but at the same time, I don't understand how startups think the "move fast break things" approach will work here.
The value of a buggy, incomplete, poorly documented database is 0, and the value of a database system that actually works is infinite. It's totally discrete.
How this is not going to end up like every other database startup? Funding gets cut eventually because they can't demonstrate traction, but they can't demonstrate traction until the database product is actually way way better than the old dogs. Which will take 10 years if not 20.
MongoDB is the only new database that managed to break through, and it was a single-threaded document store without proper ACID properties. For a database, it was simple, but it would still regularly eat your data. Today, something like MongoDB would never get traction.
I wish these rising wave guys luck, but I don't see any indication they understand the obstacles they're facing.
Snowflake, Cassandra, DuckDB, Clickhouse, Redis, InfluxDB, DynamoDB, BigQuery etc.
There are many databases that have broken through they just are far more niche focused.
> but it would still regularly eat your data
No it didn't which is why it became so popular. The whole fsync saga was always overblown because (a) every client that shipped set it a safe default anyway and (b) it was fixed almost immediately.
I know that PostgreSQL had "problems" with fsync(), but they never ended. They just accepted defeat, but so would everyone else who relies on filesystems to store their data. It's even more ingrained than that. The way hardware works, and, especially, the communication protocols around it (eg. SCSI or NVMe) are structured, fsync() is always going to be a problem.
The mongodb organization repeatedly poo-pooed the issues with data loss, and only after repeated public exposure of the issues did they do anything about it. I don't know why people aggressively deny this when anybody can just google it and judge for themselves.
All of the clients set safe defaults and the documentation was very clear about default fsync behaviour.