When Rust Makes Sense, or The State of Typed Languages
m50d.github.io
m50d.github.io
As an SRE/SysAdmin I struggle to believe that C or C++ can be written safely. It is very common that I see software being emergency patched because of memory unsafety from very experienced engineers in ubiquitous software.
Why would I need the performance/efficiency of C in the first place? Well, I regularly write software that runs on thousands to tens of thousands of machines in production. This software runs below the service. I feel a certain sense of pride in minimizing my observer effect on the system. With Rust, compared to Python, I can write software that has more functionality with a fraction of the memory and CPU utilization.
Another thing that has become more appealing to me is the strong static type system. Not having a formal background in CS I fell in love with Python for how it enabled me to move quickly, and Python is still my go-to language for most things, but after years of supporting large production sites plus running my own code in production I'm a bit tired of runtime exceptions. We have great exception logging and there are entire classes of bugs we'd just never see with Rust's type system and error handling.
Essentially, Rust gives me new super powers and the borrow checker is my sidekick making sure I don't shoot myself in the foot.
I'll admit that OCaml is something that I've looked at before but it didn't have the staying power for me. It's likely just because it was too different at the time compared to what I was used to.
Take a look at Herb Sutter's CppCon talk this year:
https://www.youtube.com/watch?v=hEx5DNLWGgA
Reinventing the wheel isn't entirely necessary. Though if you're a SysAdmin and you write your codebase in Rust.. that's some nice job security. ;)
I think C++ has a lot of awesome stuff coming out but I find Rust more pleasant to code in. Also having package/dependency management has been great compared to using the old versions of libraries that come with distros or tracking down all the transitive dependencies of software. Rust is just a more delightful experience out of the box.
> Though if you're a SysAdmin and you write your codebase in Rust.. that's some nice job security. ;)
I definitely understand this but I don't think it's true and will be less and less true over time. It is becoming very common in the SRE/SysAdmin industry that strong coding skills is a requirement. When I started 10 years ago that was very different but it's becoming harder and harder to find jobs as a Linux grunt. The SREs I work with care about software engineering best practices, code review, testing, continuous integration, etc.
Also, even if that were true, I'd much rather have people with a weaker understanding contribute to Rust where the compiler has your back. If you haven't used Rust you might be surprised how good the error messages are. Once you spend a bit of time understanding the basics the compiler is super helpful.
And I object to using Rust (or other production-level but "edge" languages) as being "job security". If the underlying software matters enough that using Rust has significant benefits, then it matters enough that employees spend a few days learning a bit of Rust. This fear of languages as if learning a language was this huge mountain to scale in hiring is just silly. The actual code and environment probably outweighs the language used significantly.
The real exception to this is if you're running a codebase that has essentially non-programmers adding bits here and there. Somewhere where you know that the individual contributors will find learning a language to be a higher barrier than contributing new pieces of code. (So either a rather trivial system, or contributors that have an unusually low capacity.)
Learning a language in days is a fairytale, the fact that you're so cavalier about this point makes it seem that you're more interested in doing advocacy than having an honest discussion.
So in the context of a conversation about the costs of training and long-term maintainability, well, you should probably write all your stuff in Java, but if you can't, then you're better off selecting Rust over C++. I can teach Rust to anyone that knows Python, and once you know Rust you're at least familiar with the most useful subset of C++.
I gather from this post that you seem to use a risky strategy for your development - let juniors loose on a problem and hope that Rust/Java will save them. Or is that just a straw man? In any case, all juniors should get coaching, CI with static analysis and code reviews are likewise highly recommended.
Finally, I can't understand how someone can recommend Rust over C++ for long-term maintainability with a straight face.
The first stable version of Rust was announced four months ago, let's talk again in 10 years and see how your code from 2015 is doing.
Therefore I think it's quite fine to say it takes a few days to learn a bit of a language, enough to start contributing. Enough that it's not "job security" to pick something that provides a real benefit.
- If you enforce memory safety to all of your code, the number of the libraries that you can use will be severely limited. Imagine if you can only use C++14 libraries. How many C++14 libraries are out there? And I'm not sure if there will be many libraries written in safe C++, at least for the next few years. The adoption rate for newer C++ standards (C++11, C++14) was generally not satisfactory.
- If you only enforce memory safety to your own code, then you could use the existing ecosystem. But in that case, your program will still segfault because of the libraries you're using. Admittedly this is also a concern of Rust when you use C libraries in Rust. The more safe code you use, the safer your program is. The problem is, if you choose this route, there's not much ecosystem-ish advantage in the C++'s side. You will also have to "reinvent the wheel" as you would have to do in Rust.
This problem all boils to this: adding guaranteed memory safety to the unsafe language is a breaking change. And the problems occurred by breaking changes are very hard to solve.
These are valid concerns, but you're being very dismissive of the benefits
I think confining errors to libraries is generally a good strategy. In effect you are guaranteeing that you're not going to introduce any new unsafe code.
It might uncover errors in the existing code.. but that's a much smaller issue and one that fades with time (esp if you're talking about libraries that a lot of people are using)
Is this "good enough" for a brand new aircraft-control/heart-lung-machine system? Maybe not. Maybe then you should start from the ground up using either Rust or C++/GSL entirely. But I think for most use cases this is sufficient and brings huge immediate benefits now. And of course with time libraries will be updated to include safety guarantees (which for the most part seem like it wouldn't involve a ton of work)
This is more of an organic process that basically in some form preserves the millions of man hours people have invested in C++ and less of a clean slate solution. As I've said before, you gotta make a living now in the current reality with the current codebases and the current tools. You can't just judge languages in a vaccuum.
This is probably the reason why our opinions vary. I believe rewriting existing libraries to be safe would be a tremendous amount of work. This estimation comes from my experiences with Rust. Safety guarantee hugely affects the interfaces of the libraries. You cannot "just" gradually increase safety of your library. It is basically a redesign. At least with what Rust currently provides this was my experience. And the method suggested by Herb Sutter seems to be very similar to Rust's. So I expect the same churn to happen in the C++ ecosystem, and it would effectively split the ecosystem into two: safe ones and unsafe ones.
Let me rephrase myself again: if adding safety to C++ in a gradual manner was that easy, Rust would have not been invented in the first place. The Mozilla guys could just use C++. There's a reason why Rust had to exist, despite all the "reinventing the wheel".
Don't diminish it, b/c the presented solution is non obvious/trivial. If it was, it would have been done ages ago.
So either A - The Mozilla folks may have not thought of this solution
B - At the time of Rust's development C++ was stagnant and not evolving as it is right now. There wasn't a lot of hope for fixing it at the time. It's thanks to the work of several people that the C++ standards committee now is a fast moving organization.
If Mozilla were considering starting the Rust project in 2016 I don't think they would have gone ahead with it
As to how much the newer features help, time will tell...
There's plenty of evidence it can, for certain values of "program". redis comes to mind as a product with a specific goal and implementation, and well managed design process that makes that happen.
The problem is when projects grow to have dozens of developers, of varying skill, pumping in features in a continual battle to stay up to date with standards - like a web browser. Then, I think the possibility of secure C is shown to approach 0.
Did you consider either of those? Were your reasons for preferring Rust similar to what I put in the article?
let (|>) x f = f x
In fact, even the + and - operators are implemented in F# and shouldn't count as part of the language (just very commonly used bits of the stdlib). I fear that perhaps not enough intros to F# focus on the relatively simple underlying language, and that everything else on top is just an application of said language.
With the type system, it's actually easy to figure out the operators, based on the type alone. Looking at |> for instance, we see a type of:
'a -> ('a -> 'b) -> 'b
But it's a valuable comment as it illustrates that the simplicity is not being marketed correctly and turning away potential users.
In Haskell you get correct by default and have to work for speed. In C your program is fast by default, but wrong, and you have to work for correctness.
OCaml and Haskell have somewhat odd syntax, for someone who is coming from C like languages or even Python. You need to learn a bunch of functional programming concepts to use them effectively. You are still limited in what you can do, such as writing a native extension module for your existing Python code base. And they don't really help on one of the major selling points of Rust, which is to be able to write safe multi-threaded code with shared, mutable data structures. While there may be lots of good high-level parallelism available, at some point for certain problems you really just want to be able to use a shared, mutable data structure, but doing so in almost any language but Rust makes it very easy to make a mistake and have aliased, mutable references to it that can invalidate all of your logic, even if it isn't actually memory unsafe.
It can. You just need to avoid those performance optimizations that were en vogue in the 90s/early00s. Luckily modern computers (and the more modern C++ standards) can handle value semantics just fine and you don't need to put pointers into vectors anymore because directly putting the data there would be too costly, etc.
If you only use the new (C++11 up) smart pointers (avoiding new/delete) you've essentially achieved 80% of Rust's safety model. Add rigorous application of const correctness and you're good to go. Rust still adds a few important things but until its eco system is matured (and there is real tooling) modern C++ is a pretty good alternative.
auto p = std::make_unique<int>(5);
auto q = std::move(p);
std::cout << *p << std::endl;
fn main() {
let p = Box::new(5);
let q = p;
println!("{}", p);
}
error: use of moved value: `p` [E0382]
println!("{}", p);
^
note: in this expansion of format_args!
note: in this expansion of print! (defined in <std macros>)
note: in this expansion of println! (defined in <std macros>)
help: run `rustc --explain E0382` to see a detailed explanation
note: `p` moved here because it has type `Box<i32>`, which is moved
by default
let q = p;
^
help: if you would like to borrow the value instead, use a `ref`
binding as shown:
let ref q = p;
error: aborting due to previous error
It doesn’t make sense to talk about C and C++ together like that and the idea that it would possibly maybe make sense to talk about two fairly different languages the same way is very much the root cause why some people consider it impossible to write “safe” C++. C and C++ probably have as much in common as C and Rust (both of these have pointers, so they must be the same) or as Haskell and Java (both of these have types/classes, so they must be the same).
I only mention those two together as the previous options for low level, no runtime languages. Outside of Rust both C and C++ were independent options I've considered investing more deeply in.
Wherever you look in the C/C++ world you find this legacy (POSIX, OpenGL, OpenMAX, OpenMP, OpenCL, on and on) so no matter how determined you are to try to match Rust with `safe' C++ you get dragged back into C. The term "C/C++" is used and understood because it is 100% appropriate, and it will continue to be appropriate as long as unsafe C/C++ interfaces proliferate.
Obviously Rust can't completely solve this problem either, but Rust at least provides a mechanism to isolate the C legacy parts behind tested and supposedly `safe' API interfaces, leaving the developer exclusively within his type+memory+data race safe environment. That is a superior model.
Try writing a fast A* (without any pointers and memory management, easily done in C++), maybe a travelling salesman problem -- simple 2-opt heuristic (easily done without memory management). If you find these suitable for libraries, and not something you would write on your own, there's not a single fast matrix library implementation in Java or Haskell. If something that "simple" can't be done, then, in my case, I cannot use these languages as easily as I can use C++.
A simple principle, a simple idea of copy and move semantics enabled me to write C++ code without worrying much about memory at all. When allocations become a problem, I just move the container out of that heavy loop, or change my malloc implementation with a simple flag, and it's still 10x if not 100x faster than what Haskell/Scala/Java could accomplish in the same amount of time it took me to write the C++ code.
Having said that, I can easily agree with the remainder of the post that the existing languages/ecosystems aren’t alternatives, and that statement does not change if you correct Scala’s “great performance” to “mediocre performance”.
If you needed to squeeze the last bits out of your hardware you would have to waste more time. That's my main complaint. As verbose as C++ is I still can't make stuff perform as best as it could in Haskell/Scala/Java without spending more time (much more than it would take me to fix in C++).
All of the languages are nice to work with but for some reason the more you distance yourself from the data representation in your memory (with references, boxes etc.) larger the time you have to spend to makes things fast, and all of those abstractions that saved you time, now turn out to be the overhead you don't want, and all of those abstractions now need to go away.
I guess value types might change the thing for Java and other JVM languages. Can't wait to see how successful it'll be.
The function's certainly there. I don't think it will automatically collapse an AB+C operation (though there's no reason that can't be implemented), but you wouldn't get that in C++ either (or rather, the template-fu needed to achieve it there would be harder than doing it in Scala).
> All of the languages are nice to work with but for some reason the more you distance yourself from the data representation in your memory (with references, boxes etc.) larger the time you have to spend to makes things fast, and all of those abstractions that saved you time, now turn out to be the overhead you don't want, and all of those abstractions now need to go away.
Yes and no. I'll grant that it's very rare to find a language that's better than another in every conceivable circumstance. But there's no law that abstractions have to be expensive (I guess that's part of the point of Rust). And even when they do come with a cost, computers get cheaper and programmers get more expensive.
The high performance part of your Haskell code might not look much like idiomatic Haskell in the end, but you can write abstractions around that. (The holy grail is high performance idiomatic code. If you befriend the compiler just right, it's possible for many problems.)
Text.Bytestring is a good example in Haskell land.
Despite that, fusion is beautiful and happens without all of that verboseness that makes the same thing in C++ work.
Rust comes up a bit short, I think, when looking at some functional styles. Function composition doesn't look very easy in Rust, so a lot of the concise code you might expect in a functional language is much more work to express (no custom operators either). Not that I blame them: having no-overhead (no GC, but also no spurious code/allocations in general) yet fully safe as a core design goal is a huge constraint that no one else has had success under.
Perhaps it is because they are not looking at it for the same use cases, and more as something to augment their toolkit. Rust makes it possible to be a decent systems programmer without having to spend a decade of acquiring the war wounds to become a great C or C++ dev. In the past, the only real option for those devs if they wanted decent performance was Java, and it's really hard to get Java to talk to Ruby or Python, and it's also hard to understand what the JVM is doing under the hood. Rust is now a possible alternative.
Note the articles aren't listed in order. Rust for Python Programmers is the latest, written four months ago.
Languages like Go are tricky for Python extensions because they include their own runtimes, which means you have 2 runtimes with garbage collectors to deal with.
In a more "fuzzy" sense, I think there's also a philosophical appeal in the way Rust promotes human-friendliness over theoretical purity. Python promotes one correct and logical (to humans) way of doing things and I feel Rust is designed with the same ambition, even if the requirements of low-level memory management necessarily make it more complicated.
Python + Go is hardly any better than just Python. Go is easy to learn but the reward is small.
Python + Rust, otoh, increases the types of problems you can solve and can dramatically improve your skills as a software developer.
I think the advantages of Go lie in making it easy to reason about performance/operational behaviour (which Rust and OCaml have, but is perhaps missing from Haskell), and in having a well-supported syntactic metamodel (scala.meta may ultimately become such a thing). With some recent emphasis on code generation it almost resembles the lisp traditional.
But I can't see Go itself working out in the long term. The concurrency model is too ad-hoc, and the language too tightly coupled to it. And in a language that's already gone to the trouble of a static type system, the cost:benefit for generics is insanely good.
I am not looking at Rust merely for replacing C for native extension libraries.
I'm looking at Rust for having a good static type system; I'm getting very sick of dealing with "hashly-typed" data in Python, or all those places where I forget a yield and so never actually block on a deferred, causing errors at runtime rather than just not compiling (sometimes type errors, if I try and access that value, but sometimes just things executing out of order, which are much harder to debug).
I'm looking into Rust for having a good multi-threading story. I've been doing plenty of event-driven programming in Python (using Twisted), which is great for non-blocking network I/O, but there are other ways to block on long computations, disk I/O, blocking in native libraries, and so on. At some point, you really do need good multi-threading, and Python falls fairly short in that department. You can hobble by, but it's not pretty.
Go doesn't solve either of these problems. It has slightly more static typing than Python, but without generics, it's pretty limited. It doesn't really do anything to prevent nasty race conditions with shared data structures in multithreaded code. Sure, if you are disciplined and use only message passing of immutable data structures, you can avoid some problems, but part of the point is that when programming in the large, with multiple people working on something over multiple years, it's easy for some mistakes or bad assumptions to creep in an wind up causing problems.
I've taken a look at Go, and it never really appealed to me as a direction to move in. If I want message passing concurrency in a GC'd language with a runtime, I can use Erlang/Elixir. If I want just simple, pretty code without much multithreading, I can use Python. If I want raw performance and low level access, I can use C or C++.
Rust actually does offer advantages over all of these. It does have it's complexity, so for quick and dirty scripting tasks I'm likely to stick with Python. But especially once Rust gets an easier to use non-blocking I/O story (since event based non-blocking is great for networking), I'm going to be seriously considering it simply as a replacement for Python for any large project that will be worked on by several people over several years, just due to the additional safety and static guarantees it offers.
In this comment I just discussed Go, since that's what the parent comment mentioned.
Haskell and OCaml are both great languages, but to a lot of people, they come off as much more "academic" or "ivory tower," and require a lot more effort to learn to use effectively for people who are not used to functional programming.
Couple that with the fact that they still use GC, so are a lot harder to get consistent low-latency performance as well as being a lot more difficult to use for any applications where you need to embed them in other applications, and they just don't offer enough of a reason to move to them over the current combination of "Python for high-level stuff, C or C++ for low-level, performance sensitive stuff".
Rust, on the other hand, promises to be effective for the low-level, performance sensitive stuff, is good for embedding in other languages, offers the additional benefit of giving statically checked memory safe multithreading with shared mutable data structures, and provides a powerful static type system for effective programming in the large.
I think the problem is that people who are used to C++ and Python approach Haskell as if it were just another language, in that category of languages.
You need you learn some new stuff in order to really understand Haskell, but that's hardly a fault. I actually think non-programmers would have an easier time learning Haskell than experienced programmers. There really isn't anything inherently difficult in Haskell. It just requires a different approach.
It can be annoying to write Haskell programs sometimes, because it's so hard to get it to just compile. But that's the entire point: the fact that the compiler can do so much when we restrict ourselves to pure functions. It also forces us to figure out exactly what our program needs to do, before we can implement it, rather than build on top of proof-of-concepts, as is typical with Python.
Note that the pronoun "it" in your sentence could easily have antecedents such as "assembly language" or "Brainf*".
[1] https://sympa.inria.fr/sympa/arc/caml-list/2015-05/msg00034....
- Haskell: While pervasive laziness does have its merits, I believe it is not the best choice in general, especially for performance-wise. Having to use streaming libraries to do even a simple I/O is just ridiculous. Its syntax was also a big obstacle to me, but thankfully I overcame it. But I'm not sure I can also persuade my colleagues to do the same thing.
- Scala: If I have a JVM requirement, Scala would be a really good fit. (Un)fortunately I don't have such a limit, so what Scala provides are less appealing to me. It also seemed way too gigantic to me at a glance, probably to be interoperable with Java. But I've not tried it deeply.
- F#: I've only heard good things about that, and it was quite a great experience learning through its great IDE. The problem is that even though MS's recent moves I'm still not completely sure if I could retrofit myself to the .NET ecosystem. But the future is bright.
- OCaml: OCaml always felt a bit "obsolete" to me. AFAIK it even didn't have multi-core support until very recently (no parallelism, concurrency only), and they seem to have no resources for a decent Windows support. It has also a funny syntax. I definitely agree Rust's syntax is ugly at best, but OCaml's is... I just can't describe it. Hopefully I can get used to it someday, like I did to Haskell.
And I met Rust and instantly fell in love, even though it requires me to do manual memory management. While manual memory management makes a certain data structure harder to implement, what I realized is that for my daily job, especially for "boring" works (e.g. CRUD apps), it is OK to use manual memory management. If there's a need, I can always borrow libraries from others to accomplish my job. And it is fast by default, less need to optimize the code by myself, as seen in Haskell.
That said, what I have as a "best" language in my mind is, probably a strict version of Haskell. Yeah, that would be Idris, like the author suggests. I really hope that Idris will take off in the near future.
Incidentally, the work on the GC is on its way.
I heard the name was a play on words on a one-of-error concerning Trust.
Is this intentional? If so, what purpose does it serve?
It's a different markup. One that many people find easier and more natural to write. Of course, different people have different opinions on that.
For people who find writing and reading markdown nicer than HTML it makes sense to use markdown as the base format that you write everything in, and then convert to different formats as needed (html, pdf, whatever).
For you, HTML may play that role. Personally I find HTML ugly to read and write.
To summarize: markup is better to read+write. But the difference is almost zero. Anything other than trivial markup is a mess; exceptional layout often impossible (and I value the ability to creatively use layout to get my point across). It doesn't support wysiwym editing (very well). It isn't trivially statically hostable - and if you try it's fragile and flashes uglily. Many flavors exist that are slightly different.
I prefer flexibility, consistency and simplicity over micro-optimizing source readability.
I also like the idea that the "source" is right there on the page rather than a separate "API" - the dream of XHTML+XSLT. When I originally set it up I wanted the page to be readable using Telnet - which I think it is, more so than one rendered on the server would be. I couldn't tell you now why that seemed important though.