This is impressive. But IMHO, anyone starting today should just use Rust from the beginning, and avoid having to endure all of the problems for years and then taking on a big HN-worthy post about 4M lines.
Sure theres "enough for the world", if you sum everybody, but i bet that even a company of the size of Dropbox would have a hard time hiring everyone they need if the complexity of the language is the same as Rust or C++.
For instance, if you read the Go manifesto to why the language was created, you can read between the lines that C++ getting in the way where enginners were fighting hard to deal with language complexity.
So the same way as Java before them, they created a language for the "average programmer". And you can see how the language is very pragmatic and how they fight to no put more complexity on the language.
By the way, i know its a personal opinion, but i think Rust will not be a great fit for the cloud backend exactlya because of this kind of thing. It will be hard to create codebase as complex as the ones in Java, because there will be much less man-hours available.
Maybe if generations are getting smarter, IDK, but needing to rely on a big head-count of C++ or Rust enginners for any company, i bet they will have a hard time to fill all the positions they need.
Theres much more C++ enginners than Rust right now, and companies have a hard time hiring them (and i bet that for Rust it will be no different giving its complexity).
My understanding was that the push from C++ to Go was largely motivated by long C++ compile/linking times. Go is designed to compile large code-bases much faster than C++. I've experienced this myself, it's pretty frustrating to wait 2-5 minutes for a C++ app to compile / link just so you can debug it for 30 seconds and start all over again.
That's quaint. When I first started working in C++ professionally, we were at +16 hours for a clean build and link of just one of our systems (circa 2004). Over the years, reduced the amount of code (without sacrificing functionality), shared common code (object model) between client and server (went from about 2M LOC to about 250k LOC), reduced the number of external dependencies, introduced precompiled headers on both windows and Linux, and a clean build was down to about 30 minutes. An incremental build and link could be less than 30 seconds, depending on the scope of the change. But, yeah, C++ leaves a lot to be desired in terms of edit, build, test, repeat cycles.
I mean, how could they know their link times would be that much better?
I know that Pike et al. worked on compilers for Inferno, and the Ken´s C compiler used some tricks to know beforehand they could make it compile and link very fast, but they couldnt know if the language they were designing would be as fast (to compile) as C or Limbo.
They could say confidently: "we could do better", but they would need more pragmatic motives, even to convince Google to support that effort.
And i bet Google was struggling with C++ enginnering, specially within the newcomers on the team.
I think, both things are true, but developer ergonomics play a very important factor into the why of a language and where it fits in the scheme of things.
Sure, you can hire new talent that knows Rust, but how long for them to pick up the relevant domain knowledge? How long for the existing talent that has the domain knowledge to become equivalently proficient in their current tech stack?
Plus, with a rewrite, you risk falling ill to second system syndrome.
Economically and from a practical engineering standpoint, it seems to me that Dropbox is taking a very pragmatic approach. Gradually make working with the exist code easier rather than throwing the baby out with the bath water.
I'm not hating on Rust, but this fervor to rewrite everything in Rust is obnoxious and often short sighted, in my opinion. If you're talking brand new, green field development, by all means, it should be considered.
I'd agree with you re Dropbox, a relatively "enlightened" modern software company. But they are the exception, not the rule. Most companies, big and small hire the cheapest developers they can find. I'm not talking about silicon valley tech companies, but think e-commerce and corporate sites (Nordstrom, General Motors) and old school companies (e.g., Goldman Sachs, power utilities).
These companies don't care about the "right tool for the job", they care about the right people for the job. If the "people" are adept at antiquated technologies, then they'll still pass muster.
shouldn't be a factor for a company *at that scale*
Rust crates are still full of the type of bugs that Python libraries killed years ago. Garbage collection overall really does make things easier. And Java still does concurrency better than practically everything else out there.
If you are doing systems programming, go for Rust instead of C++. Maybe even Rust instead of C. But, I would reach for Python, Ruby, Kotlin, Clojure or even modern Java before Rust for most non-Javascript projects.
As for systems programming, I would argue that Rust still isn't an alternative to C++ in several deployment scenarios that many of us care about, although it should eventually get there.
If you're doing concurrency, you should look at how java.util.concurrent does it. And, then, if you aren't doing it that way, you should start asking really hard questions.
Could you be more specific about what triggered such an enthusiastic description?
Take a good look at things like ConcurrentLinkedDeque or ConcurrentSkipListSet and then look at what guarantees they do and do not make and think about the rationale behind those decisions.
And all you have to do in java is create the collection, hand the reference to multiple threads, and IT JUST WORKS like a normal Java collection.
You do NOT want to be using semaphores and locks directly if you have java.util.concurrent at your disposal. Look at the "Concurrent Collections" section:
Most concurrent Collection implementations (including most Queues) also differ from the usual java.util
conventions in that their Iterators and Spliterators provide weakly consistent rather than fast-fail traversal:
they may proceed concurrently with other operations
they will never throw ConcurrentModificationException
they are guaranteed to traverse elements as they existed upon construction exactly once, and may (but are not guaranteed to) reflect any modifications subsequent to construction.
That library is something I sorely miss when I'm operating in another language.
You should also dive into the code itself. Sometimes you'll look at a function and marvel at how much work it really takes to get something right. And sometimes you'll look at a function and wonder how it could possibly be so simple.
Is the typical implementation of those days structures relying on many small scale mutexes to avoid contention, or RCU techniques ?
I think there are alternative paths when contention starts becoming too high.
It's been a while, but the code is the real reference. You can go back and look at the mailing list archives as well.
That's an interesting claim. What do you mean specifically?
I rarely file bugs on Python packages no matter how obscure my setup is.
Now that Rust stable is actually reasonably solid, I just ignore crates that demand nightly.
And if you started 5 years ago, you're now likely spending half of your time over the next few months/years porting your code from Python 2 to 3. What a disaster.
I've mostly used the Java, C++ and Go in my career, and it's pretty amazing to me that I can write write safe, readable code (like Go) on top of high-performance libraries (like C++) and step through code on a prod machine with a remote debugger, get live performance metrics with a single CLI command, etc.