The Path to Rust
thesquareplanet.com
thesquareplanet.com
The first week was madness. I'm fairly experienced developer and the Rust compiler hit me to my fingers constantly. Usually the only way out was to write the whole part again with a different architecture. On the second week I was done with my tasks, very comfortable with the language and just enjoying the tooling. Also I was relying on tests way less because of the compiler, even less than with Scala. If it compiles, it has a big chance of working. Cargo is awesome. Let me repeat: Cargo is awesome. Also I like how I can write code with Vim again and even though for some things I need to read the Rust source, it is pretty easy to get answers if you're in trouble. In the end I wrote some integration tests with Python and I'm quite happy with them.
Now I want to write more Rust.
1: I mean, it's madness if you want to talk to it from the outside. If you stick with C++, this is "merely" flamewar material (take the Boost devs and the C++ committee on one side, and game devs like Mike Acton, Jonathan Blow, or Casey Muratori on the other side).
Some of them do, or have C equivalents, but many of them don't. Once you get into "there's only one library that does it and it's one guy's graduate thesis" territory, often the only option is on the JVM.
Those libraries aren't tried and tested. They can be reimplemented. And in some contexts, they shouldn't be trusted as a black box.
Not saying you have to like Go, we all have different preferences. But FYI, the gc has gotten much better in the latest releases. I don't think you would be bothered by gc pauses. (my understanding is that it' both faster and more "spread out")
Consider a widely deployed library written in C -- say, zlib or libjpeg or SQLite or openssl. Could you rewrite those in Go or Haskell or Scala? No. Because nobody wants a big surprise when linking a harmless C library all of a sudden brings in an entire GC that's starting extra threads, etc.
In other words, Rust is the first practical language in a long time that could be used for a new library that might be used in millions of different applications written in dozens of languages.
Also RAM has a cost, for garbage collected service you need to reserve some extra RAM. Scaling these processes add an extra memory overhead we're not so eager to take.
For reference, the Go 1.5 GC propaganda:
> Go 1.5, the first glimpse of this future, achieves GC latencies well below the 10 millisecond goal we set a year ago.
Blog post: https://blog.golang.org/go15gc
Slide: https://talks.golang.org/2015/go-gc.pdf
Talk: https://www.youtube.com/watch?v=aiv1JOfMjm0
Edit: Meanwhile, over in Java land they measure stop the world pauses in seconds, not milliseconds:
http://stackoverflow.com/questions/15696585/long-gc-pauses-i...
https://blogs.oracle.com/poonam/entry/troubleshooting_long_g...
Also most GC'd languages don't give you strong control over data locality(by nature of everything being a reference) so you pay in cache misses which are not cheap.
It's also worth pointing out 10ms is the maximum, not the minimum. There's plenty of workloads where you're not going to see pauses anywhere near that long. It's certainly not impossible to write a 3D game in Go and get 60 fps with high reliability, especially with the amount of stuff nowadays that actually runs on the GPU. You're not going to be able to do this for your next AAA game, but I wouldn't see a huge reason that an "indie" game couldn't use Go for that. (Probably not my first choice, all else being equal with library support personally I'd suggest Rust for this for a lot of other reasons (the type of concurrency you often get in games is going to be well supported with Rust's borrow checker architecture), but contrary to some discussion I would say it is still on the table.)
If I add a cyclic reference here will that make GCs longer? Maybe I should have a free list and reuse these objects (after all they're responsible for most of my allocations)?
As soon as you're thinking like that as you write each line of code you've lost all of the benefits of the language being high-level, and you'd be better of controlling memory manually.
Maybe not impossible, but highly highly improbable. If you say that then you have no idea what it takes to run 60fps constantly. It is HARD, VERY HARD.
But it's hard to write a 3D game in Go and get 60 fps with certainty.
It's not that hard to write something in Go where you pay two milliseconds every 5 minutes or something, or less than that. Again, let me reiterate, 10ms per sweep is the max, not the min. Plus it sounds like people think that Go somehow guarantees that you're going to pay this on every frame or something, rather than potentially separated by seconds or minutes.
As I said, Go probably isn't my first choice for a game anyhow, but people are grossly overstating how bad it is. Games may be high performance, but they also do things like run huge swathes of the game on a relatively slow scripting language like Lua or some Lisp variant or something. It's not like the network servers that are Go's core design motivation are completely insensitive to performance or latency either.
(Plus to be honest I reject the premise that all games must be 60 fps on the grounds that they already aren't. I already disclaimed AAA vs. Indie. But it's still not a bad discussion.)
Like I said in another part of the thread, different tools for different domains. You certainly can get a consistent 60 FPS on any consoles, which are also much less forgiving about techniques that would be perfectly fine on a PC.
Generally you're much better off using a scripting language like Lua for the places where you want the advantages GC brings but scoping it so you can limit the damage it does to your frame time.
http://www.cs.technion.ac.il/~erez/Papers/real-time-pldi.pdf
The issue with GC is it's not deterministic. There's a whole nother aspect that has to do with free heap space. Quite a few GCs need some multiple of working set free to do the compact phase. If they don't have it then GC times start to spiral out of control.
On an embedded device(or modern game console) you'll be lucky to have 5-10mb free. On PSP we used to have only 8mb total since 24/32mb went to video+audio. We still ran Lua because we could constrain it to a 400kb block and we knew it would never outgrow it.
Just like everything in software there's different tools for different domain spaces. Otherwise someone would just write one piece of software that fits every problem and we'd all be out of a job.
The more common strategy I see in CompSci for your use-case is to have a mix of memory pools, GC, and safe manual. Ada had memory pools and I'm sure you get the concept. Safe manual is when static analysis shows a delete can happen without safety consequences. So, that can be unmanaged. Then, what's left is handled by the concurrent, real-time GC. In typical applications, that's a small fraction of memory.
The issue is with using GC based language for areas where you need high throughput and low latency(there's the whole cache miss thing which GC exacerbates).
And unless your RC algorithm is more clever than most that's going to be stop-the-world.
My current results is that Go performance can be really good (C++ level) if I try to avoid allocations as much as possible. With a more sloppy implementation that e.g. did not reuse buffer objects ([]byte) between multiple network packets the performance dropped significantly to about 1/3 of the C++ implementation, and the GC time dominated.
Fortunatly it's quite easy to avoid garbage in Go, as we have value types there and the synchronous programming style means lots of stuff can be put on the stack and there's not so much gargabe due to captured objects in closures and similar stuff. All in all I'm quite confident that with a mediocre amount of optimization Go can be suitable for lots of application types. Although I would not necessarily try to use it for low-latency audio processing if something else (that produces garbage) is running in the same process.
(I am kind of interested in Rust, but not going to touch it until it gets proper HKT support - you have to duplicate so much code otherwise)
Comparison: I have one app written with Scala and consuming 350 megabytes per Mesos task. The similar-sized Rust app is using 8-10 megabytes of RAM. When you have 1000 instances running during a peak, this escalates fast.
I don't see how making the type system stricter makes the language harder to learn. Maybe that's because I know another relatively paranoid type system (Ocaml), but still.
A type system that rejects your code is like a teacher looking at a proof you just wrote, and tells you "this doesn't even make sense, and here's why". It may be frustrating, but this kind of feedback loop is tighter than what you would get from a REPL.
And you do see partial progress: the type errors change and occur further in the source code as you correct your program. Each error is an opportunity to fix a typo or a misconception. The distinction between a broken prototype that doesn't even compile and a working program isn't binary: when you correct a type error, your program is less broken, even though it doesn't compile yet.
Not only that: the Rust compiler, for many cases, will suggest calling it with a ‘--explain’ argument to get an explanation of what’s causing the error and, usually, how to solve it. A very friendly feature.
It's confusing, as hell.
Let me try to explain a little bit. I'm a average dev, and have been for ~7 years or so. My languages have typically been dynamic, but the past 2 years has been heavily Go.
I picked up the majority of Rust, as well as basic borrow checker usage, in a matter of hours. But that's where the fun stopped. Generic syntax beat me up (mostly related to Static dispatch and Type Assertions). Lifetime syntaxes are currently kicking my ass as well. The problem as i see it, seems to be that i am a more.. "dig in" developer. And frankly, that does not seem to fly here. I've read a lot of the rust docs, but rather than start to finish, it's been in response to questions i have. This seems to be problematic for rust, as important concepts can be left in the dust.
A good example of this is Static vs Dynamic Dispatch is rust. Heavy trait usage combined with using Static Dispatch incorrectly resulted in code not designed for what i wanted, and lots of seemingly random bugs. Ie, `foo = Foo::new()` would fail, but `foo = Foo::new(); foo.bad(baz);` would succeed.
Currently i'm trying to figure out how to either move a variable out of a block, or, more importantly define a lifetime on `Ok(r) => r,` so that the result can escape the block.
Anyway. None of this is the fault of rust exactly, and again, i love this language. I'm switching because Go hasn't offered the full safety i want. But man, it has been quite the hellish experience to get started.
> how to to either move a variable out of a block,
let x = {
let s = String::new();
// stuff happens with s
s
};
> more importantly define a lifetime on `Ok(r) => r,` so that the result can escape the block.
As you can expect, line 13 is trying to borrow a value that does not live long enough for `req`. Unfortunately, hyper::client::Client's req.body() requires a &str it seems. Somehow i need to pass it a &str (which i would normally do via &String).
Do i need to move s:String out of the block? Do i need to specify a lifetime? These are the sort of "What do i even search for!?" moments i run into heh. Granted, less so these days.. thankfully.
Note that `&mut Some(ref c)` looks pretty terrible, i'm experimenting with an API i'm writing.
Without compiling myself, it can be a bit tricky, but...
Yes, so the issue here is, your String will go out of scope at the end if the if let, on 14. But you're trying to store a reference to it in something that lives longer: the req will continue to live afterwards. So yes, you need to move s outside of the block somehow; there are a few different ways of doing this. What I'd try first is something like this: https://gist.github.com/steveklabnik/54cf7a4a522cd1a7c6e0130...
Now that the binding lives in the outer scope, it will live longer than the `if let`, you're basically moving it out. I'm trusting the compiler's control flow analysis here; it should let you do this, given that you only use s after you've assigned to it.
I was trying to be clever, and use explicitly say that `s` should live for the lifetime of the parent function. Any idea if that is possible in this case?
edit: Sidenote, i appreciate that you hang in Rust Beginners, i didn't mean to imply that people weren't being helpful. Just that, it can be difficult to find a solution at times.
On the plus side, i managed to help (i hope haha) a guy in #rust-beginners today, so hopefully i've paid it forward a bit. Appreciate your help!
> Any idea if that is possible in this case?
fn body<B: Into<Body<'a>>>(self, body: B) -> RequestBuilder<'a>
The RequestBuilder struct tries to be clever: it just keeps a pointer to what you want the body to be rather than taking ownership of it or copying it. So when you say that "req = req.body(&s)" you've given the request a pointer to your string, but that string is freed inside that if-block. I don't see an obvious fix right now, but I'll update this is I see something.
Fwiw, https://news.ycombinator.com/item?id=11780196 has the answer to my example problem. I'm also inquiring about using lifetimes to solve the problem.
I know, I know, no non-trivial checker can be both sound and complete. But if too many useful programs end up being rejected because of that gap, the type system must be changed.
Overall, I'm happy to accept the tradeoffs of the borrow checker. I have to deal with a learning curve and the occasional workaround. But I get fast, correct and expressive code. And even though I like to think of myself as being extremely careful about pointer ownership, the borrow checker still catches real problems in my code that would be a nightmare to debug, especially once threads get involved.
Knowing when to use borrowing over owned copies I think is vital as well. Once I started planning out what was going to own what my code became a lot simpler and I spent less time fighting the borrow checker in general.
Hopefully as rust evolves so will the tooling and the borrow checker. It seems like you could build some cool visual aids and/or debugging info into IDEs for novices who are new. --explain already sort of does this.
A strategy that I have reverted to in Rust is to use #[test]'s to mark my progress... I might only write one short function, verify it works with a #[test] then move on. It's almost TDD, but not quite. I started this practice back in my Java days, but have found it particularly beneficial with Rust.
I've been trying to port a personal project to Rust recently.
Sometimes Rust's type system is too limited to understand why something is safe. Here are a couple examples:
* in C++ code, you can have a class in which one field has a reference/pointer into another which came earlier in the declaration order (and thus will be constructed first and destructed last). This is often a useful thing to do (one example: https://users.rust-lang.org/t/struct-containing-reference-to...) You can't do that in Rust. They have to be separate instance variables on some thread. If there's no thread running to own it, you have to use referencing counting (Arc or Rc) or unsafe blocks. Or maybe instead of keeping a reference, have all calls take the outer struct as a context argument and use some sort of struct/lambda which knows how to find the thing you're referencing given that (this is what I'm trying in my code). Someone proposed a language change for "self-borrowing structs" (https://mail.mozilla.org/pipermail/rust-dev/2014-February/00...) but it didn't go anywhere as far as I can tell.
* in C++ code, you can loop over one instance variable and then call a private method on self which mutates a different instance variable. In rust, you'll get errors about self being partially borrowed. I think you have to restructure the other method to not take self, which probably means grouping things into child structs. Basically Rust doesn't look across functions boundaries to decide if something is safe so it has to consider this an error even if it isn't for the particular method you're calling.
Fundamentally, I think the choice is between these three options:
* use a garbage-collected language and not have to be explicit about these details. There's no possibility of buffer overflows or use-after-free errors but you have to pay the runtime overhead of the garbage collector. Go is clear about the costs involved: pauses up to 10 ms, 25% of all CPU cycles, and 50% of RAM (see http://golang.org/s/go14gc). Other GCed languages likely have similar costs even if they aren't stated as clearly.
* use an unsafe language like C/C++, enforce these things manually in your head and with comments, and occasionally have security problems when you screw up.
* use a safe-but-explicit language like Rust/Swift and have to "show your work" to the compiler quite a bit more, finding a different way if safety is too hard to prove.
AFAIK Swift is semantically in the garbage-collected pile, it uses (statically injected) reference counting.
I believe Chris Lattner has noted they'd left the door open for something like affine types (ownership) in the future, but that's not really their goal at the moment.
For partial borrows, there has been some work on it (https://github.com/rust-lang/rfcs/issues/1215), but I agree that this is something that's missing. That said, I very rarely run into this, and there's usually some fairly obvious restructuring I can do to make it work out.
I don't see how that could be true. It means using a separate heap allocation for each referenced piece (although the owning_ref thing steveklabnik mentioned might minimize that), a bit of extra RAM for the counter, and a bit of bookkeeping. I'm not saying the overhead is huge, but how could it be zero?
fwiw, I finished this section of my code, and the context approach I mentioned worked out well for me. It was just a bit of a puzzle to find a way Rust would like.
I'm pretty happy with Rust so far even though I've had to restructure parts of an apparently-working program to fit its model. My program is now more obviously correct, benchmarks are pretty good so far (though I wish profile-driven optimization were supported/mature: https://unhandledexpression.com/2016/04/14/using-llvm-pgo-in...), and the open source library situation seems better than C/C++ for what I'm doing and actively improving where C/C++ is stagnant.
let idx = args
// iterate over our arguments
.iter()
// open each file
.map(|fname| (fname.as_str(), fs::File::open(fname.as_str())))
// check for errors
.map(|(fname, f)| {
f.and_then(|f| Ok((fname, f)))
.expect(&format!("input file {} could not be opened", fname))
})
// make a buffered reader
.map(|(fname, f)| (fname, io::BufReader::new(f)))
// for each file
.flat_map(|(f, file)| {
file
// read the lines
.lines()
// split into words
.flat_map(|line| {
line.unwrap().split_whitespace()
.map(|w| w.to_string()).collect::<Vec<_>>().into_iter()
})
// prune duplicates
.collect::<HashSet<_>>()
.into_iter()
// and emit inverted index entry
.map(move |word| (word, f))
})
.fold(HashMap::new(), |mut idx, (word, f)| {
// absorb all entries into a vector of file names per word
idx.entry(word)
.or_insert(Vec::new())
.push(f);
let mut idx = HashMap::new();
for fname in &args {
let f = match fs::File::open(fname) {
Ok(f) => f,
Err(e) => panic!("input file {} could not be opened: {}", fname, e),
};
let f = io::BufReader::new(f);
let mut words = HashSet::new();
for line in f.lines() {
for w in line.unwrap().split_whitespace() {
if words.insert(w.to_string()) { // new word seen
idx.entry(w.to_string()).or_insert(Vec::new()).push(fname);
}
}
}
}
(for {
fname <- args
f = fs::File::open(fname).orElse(
e => panic!("input file {} could not be opened: {}", fname, e))
r = io::BufReader::new(f)
w <- f.lines().flatMap {
line => line.unwrap().split_whitespace()
} .distinct
} yield Map(fname -> Vec(w.toString))).sum
(I thought I knew Rust, but I haven't used it for a year, and its usage has changed a lot even if the core language hasn't.)
I write C++ for most of my day job, and this would not pass code review because of readability problems in my team.
The two loops contain a single statement and it is trivial to see what's going on. It is also easier to extend. And it is still using lazy iterators (I assume) where it makes sense, i.e. in the split_whitespace call.
- The error handling should really have been refactored. `try!` would make this easier. (I haven't used it since it is in the `main` function and the code was a direct replacement.)
- Error during reading words is not accurately handled. Again, `try!` would make this easier.
- `words` and `idx` look coupled to each other, which ideally shouldn't have been.
- `words.insert` is quite an opaque method; not everyone is sure if it returns true on a duplicate key or not. I've added a comment but frankly I'm not satisfied of that. <deleted> One alternative is to use `words.entry` instead, which gives a named enum variant. </deleted> Oops, `HashSet` does not have `entry`...
- Ultimately, the body of the outermost loop should go to a function.
That said, do you really think that `line.unwrap().split_whitespace().map(|w| w.to_string()).collect::<Vec<_>>().into_iter()` is a chunk of code which can be read at a glance? It at least has to be named (like T-R's example). Functional approach means that you can split functions and individually review them; the original code, IMHO, didn't.
The nice thing about functions like "map", "filter", "fold", "flatmap", etc., is that they describe intent - when you see a "map", you know it's not aggregating things, just applying a function: "map" has a distinct purpose from "fold" (and depending on how pure things are, you may not even be allowed to do anything crazy).
Aside from that, the higher-order functions have pretty intuitive algebraic laws for refactoring (like with "compose" mentioned in my other comment) - It's not clear how you'd factor code out of a nested loop without understanding the whole thing, whereas "flatmap" (a.k.a. "bind" for the list monad) has laws for refactoring it.
> It's not clear how you'd factor code out of a nested loop without understanding the whole thing, [...]
You are right, loops are particularly hard to refactor. I still argue that my code is better (barring any future expansion) because it fits within handful lines; you cannot easily understand 100 lines of code with the cyclomatic complexity of 1, but you can often easily understand 10 lines of code with the cyclomatic complexity of 8 (e.g. triple loops). The size of code, syntax and contextual information matters as much as algebraic laws.
let mut words = HashSet::new();
for line in file.lines() {
let line_words = line.unwrap().split_whitespace();
words.extend(
line_words.map(|s| s.to_string())
);
}
words.into_iter().map(move |word| (word, f))Retrofitting better error handling in the single expression version requires changing the shape of the whole thing, editing the expression throughout.
let words = fn (f, file) => {
file.lines().flat_map( |line| {
line.unwrap()
.split_whitespace()
.map(|w| w.to_string())
.collect::<Vec<_>>()
.into_iter()
})
.collect::<HashSet<_>>() // prune duplicates
.into_iter()
.map(move |word| (word, f)) // and emit inverted index entry
}
let idx = args.iter()
.map(|fname | (fname.as_str(), fs::File::open(fname.as_str()))) //+ (str, fd)
.map(|(fname, f)| { f.and_then(|f| Ok((fname, f))).expect(&format!("input file {} could not be opened", fname)) })
.map(|(fname, f)| (fname, io::BufReader::new(f))) //+ (str, buf)
.flat_map(words) //+ {str}
// absorb all entries into a vector of file names per word
.fold(HashMap::new(), |mut idx, (word, f)| { idx
.entry(word)
.or_insert(Vec::new())
.push(f);
}); //+ {word:[filename]} let open_file = |fname| (fname.as_str(), fs::File::open(fname.as_str()) );
let check_err = |(fname, f)| {
f.and_then(|f| Ok((fname, f)))
.expect(&format!("input file {} could not be opened", fname)) };
let to_buff_reader = |(fname, f)| (fname, io::BufReader::new(f));
let line_to_words = |line| {
line.unwrap()
.split_whitespace()
.map(|w| w.to_string())
.collect::<Vec<_>>().into_iter() };
let get_file_words = |(f, file)| {
file.lines()
.flat_map(line_to_words)
.collect::<HashSet<_>>().into_iter() // prune duplicates
.map(move |word| (word, f)) }; // emit inverted index entry
let add_to_index = |mut idx, (word, f)| {
idx.entry(word).or_insert(Vec::new()).push(f);
idx };
let idx = args.iter()
.map( compose(to_buff_reader, check_err, open_file) ) //[1]
.flat_map(get_file_words)
.fold(HashMap::new(), add_to_index);
Rust itself does not have a compose function. But a closure works fine, so `|fname| to_buff_reader(check_err(open_file(fname)))` would work. If you want to go further, a nightly Rust allows for this kind of construction:
#![feature(unboxed_closures, fn_traits)] // that's why we cannot use stable (yet)
struct Composed<F, G>(pub F, pub G);
impl<F, G, T, U, V> FnOnce<T> for Composed<F, G>
where F: FnOnce<T, Output=U>, G: FnOnce<(U,), Output=V> {
type Output = V;
extern "rust-call" fn call_once(self, args: T) -> V { self.1(self.0.call_once(args)) }
}
impl<F, G, T, U, V> FnMut<T> for Composed<F, G>
where F: FnMut<T, Output=U>, G: FnMut<(U,), Output=V> {
extern "rust-call" fn call_mut(&mut self, args: T) -> V { self.1(self.0.call_mut(args)) }
}
impl<F, G, T, U, V> Fn<T> for Composed<F, G>
where F: Fn<T, Output=U>, G: Fn<(U,), Output=V> {
extern "rust-call" fn call(&self, args: T) -> V { self.1(self.0.call(args)) }
}
fn main() {
println!("{}", Composed(|x| x+3, |y| y*4)(5)); // 32
}Ah, yes it would. Clearly I'm a bit too tired to be writing code, if I've overlooked function application. I suppose it doesn't have to be point-free. =)
Pretty cool, though - thanks for the info.
However, its overloading a custom type to behave like a higher order function, which is a fairly complex piece of code. Most uses of generics are much simpler.
Some people seem to fall back into it a bit when they discover point-free style (and that seems to make up a lot of what you see in mixed-paradigm code). I don't think it'd be controversial to say it's bad practice in any paradigm, so the association of it with FP is kind of like judging web programming by late 90's beginner PHP code (which, at one point in time, did describe a lot of web programming, but it was never good, and we'd like to put that behind us).
I opened the external example he linked which has solutions in a number of languages. (https://www.rosettacode.org/wiki/Inverted_index) I could understand the C version of the code pretty well at a glance. Granted it was long but it looks like the intuitive way I'd write the code. I acknowledge my bias here.
I looked at some other languages on the site and both the the C# example and D example look much more readable then the rust code (and I've not used either of those languages either I think they are intended for "system" programming as well) the code as provided in those languages seemed more intuitive.
That said, there's certainly a point to be made (and indeed other commenters have made it already) that this isn't code you would want to maintain. For production-level code, it'd be broken down more, so that individual pieces could be reviewed and tested in isolation. The code example here was to demonstrate the expressivity of the language more so than the One True Way of doing it.
This looks like code from the 'I found out about FP, let's apply it everywhere'-stage. For instance, if the iteration over arguments was done with a regular for...in loop, the filename would be visible in its scope and the rest of the code could be simplified due to not having to pass the filename everywhere.
Also, I don't think this demonstrates expressivity well, because the purported functional construct uses unwrap/expect (it panics). The expressivity of Rust allows you to have the whole expression to have Result has its type fairly easily.
(Sorry for ranting a bit, but I think that Not only is this very readable, will tick off people. Which is sad, because it could be changed into something which is more readable but still expressive.)
The functional code I give in the article is quite radically different from what you would/could do in those languages, and I would argue that it does show that Rust provides some interesting and flexible mechanisms that add to the expressivity you have as a programmer. Sure, it's overused in the example, but the exaggeration at least has a chance of communicating the point. Maybe the argument is that I should have used a simpler (yet still not trivial) example instead, but I couldn't think of one at the time.
Re the "Not only is this very readable" part, I completely agree. I could change it to something less, hmm, bold, but I'm not sure it would make that much of a difference at this point. Suggestions are welcome.
It went thought the code identifying code patterns and goals, and then making them into separate functions or data structures as appropriate.
Rust does not force you into the functional style, and usually I use imperative style when writing Rust code, except when the task is so simple that writing e.g. a for loop would be just noise. For example, if I just wanted to convert every string in a vector to uppercase, I'd probably just write "list.iter(|string| string.to_uppercase()).collect()". But if I were, say, hashing every string in the vector, I'd probably use a for loop over an iterator.
There are enough similarities with ML languages, and I have used so much FP concepts all the way back to Caml Light, my first FP language, that I would just naturally follow that path when writing Rust.
In Smalltalk it was just natural to mix both styles.
I use the same strategy in "dynamic" languages like Python and JavaScript, that support both an imperative and functional style.
https://gist.github.com/danieldk/3bd3b84c1a7c8bc8c902314a488...
(Compiles and seems to work properly on a small document collection.)
> .iter()
hmmm. Not far off the old
> i++ // add one to i
I feel obliged to point out that this is false. Rust prevents _data races_, but not _race conditions_. You can read more in the Rustonomicon here: https://doc.rust-lang.org/nomicon/races.html
A couple of other beginner-friendly resources would include:
- An alternative introduction to Rust [1]
- 24 days of Rust [2]
- CIS 198: Rust Programming [3]
[1] http://words.steveklabnik.com/a-new-introduction-to-rust
If the whole OS stack can be implemented in the language, ignoring the Assembly help that even C requires, it is a systems language regardless if it has a GC for heap management or not.
It is also a descendant of Oberon, which was used very much successfully at ETHZ during the 90's.
Every time I have this argument I wish some PhD student bothers to rewrite Native Oberon in Go.
Also you should educate yourself, a small list of GC enabled systems programming language is:
Lisp, Algol-68RS, Mesa/Cedar, Modula-2+, Modula-3, Oberon, Oberon-2, Active Oberon, Component Pascal, Sing#, SystemC#, D.
There are plenty of ACM, SIGPLAN, digital archive papers from the OS written in those languages.
Far as Go, it's designed to recreate the Oberon-2 experience of Pike. It's basically a modified Oberon. If Oberon can do OS's, then Go can do OS's with some modifications to runtime or something.
https://github.com/jjyr/bootgo
http://wiki.osdev.org/Go_Bare_Bones
Although the Wiki author doesn't seem to understand how to use the unsafe.Pointer type, and makes use of external Assembly instead, but the bootgo does it properly.
I just sent the Github author an email suggesting it as his next project. Plus a little praise for giving us the needed ammo. :)
One could also interpret is as "Rust is quickly becoming my favorite language for all systems work [...], and has __also__ largely replaced both Go, Python, and C/C++ in my day-to-day."
Rapidly? Where?
Even Microsoft is now contributing to the eco-system due to their collaboration with Docker. Also Joe Duffy (from MSR Midori) tweets about his Go experiences.
Personally I prefer other languages, but I am surely happy to see more Go and less unsafe languages being used.
Other examples of companies using Go for network services: Dropbox (most backend), CloudFlare (Railgun and other things), Uber (geofencing), Bitly (messaging with NSQ), Disqus (realtime commenting), DigitalOcean, Heroku, SoundCloud, Dailymotion.
> The decision to use Go was deliberate, because we needed something that had lower latency than Java (where garbage collection pauses are an issue) and is more productive for developers than C, while also handling tens of thousands of client connections. Go fits this space well.
Source: http://techblog.netflix.com/2016/05/application-data-caching...
Discussion: https://www.reddit.com/r/golang/comments/4l0fv2/the_netflix_...
Python doesn't offer much against languages that compile to native code, have type inference (if strong typed) and REPL.
The only reason for me, would be if I am required to make use of a framework or library that requires me to use Python.
In this context, for example, I don't see a reason for a world government to impose one definition of "systems programming" worldwide by military force (this is what it would take!). Just like in almost everything in human language, you get it from the context. You should try to teach a computer to recognize human speech including the meaning, then you'll realize that almost all of it relies on context and pre-existing knowledge. "Precision" comes from the interpretation - human language communication is the nightmare of people who love functional programming, it's full of hidden state and context and active interpretation.
Which is why it's so easy for this to happen: http://dilbert.com/strip/2015-06-07 If someone wants to argue, there is no way to provide a water-tight human-language text that "Dick from the Internet" can't attack. There always is a way to mis-interpret human language.
http://people.inf.ethz.ch/wirth/ProjectOberon/index.html
All the way from the hardware Verilog to the GUI.
This is the 2013 re-edition of the initial project. Later versions of the operating system with its 3D Gadgets framework were quite nice to use.
It doesn't get more systems programming than that.
http://vbn.aau.dk/ws/files/58355126/main.pdf
So, one can do Oberon all thd way down until we need custom cells. ;)
Another popular, reasonable definition for "systems language" is "something one could write an OS in".
Perhaps it could be amended "Primarily used for writing programs without a GUI."
>>Unfortunately, higher-level languages are often not a great fit for systems code. Systems code is often performance critical (e.g., kernels, databases), so the developer wants predictable performance, and tight control over memory allocation/de-allocation and data layout. This can be hard to achieve in higher-level languages or when using a garbage collector.
In a strict ANSI C compliant compiler, signal handling is implemented in Assembly.
A systems programming language is one one can be used to write a full OS stack, regardless how heap management is done and how much Assembly help is needed.
Off the top of my head, Time and SIMD are the only two first-party efforts that are missing from there.
Things which will, at this rate, probably never be added -- either because there isn't a "standard" solution to these problems or because it doesn't seem worth it:
* async io
* web framework/http
* numerical hierarchies and bignums
* linear algebra / stats
* GUIs
* "human" time / calendars
Bignums in general are a bit too open-ended, but I could see bigints at least being added someday.
And as for async io, I think it's the same story as with HTTP: imaginably included contingent on implementation maturity and user demand.
I apologize for how glib my remark above was, it didn't add much to the conversation, and was more a knee jerk response to the notion that "Systems Language" is now, or ever has been, well defined.
I brought up the handling of signals, because process management is, in practice, a pretty big deal in what I would consider systems programming. But I don't think it's important to all systems programming. Much like I don't think avoiding garbage collection is necessary for all systems programming, though it's clearly an issue for some applications.
What I do find odd is the notion that it is necessary to not have garbage collection, but not having a solid story behind handling signals from the OS receives a pass.
Rust can handle signals, but its implementation is very platform specific (even the currently recommended crate lacks Windows support), and ultimately calls out the FFI. So it seems that saying Rust has support for signals is like saying Go supports manual memory management because you can call C.malloc and C.free...
* try!() Is pretty annoying
* Working effectively with C in non-lexical ways seems to involve some unstable libraries and still requires nightly rust
* Macros are safer, but can't do some things that C macros can. For instance, they are hygienic, which means you can't conjure up new identifiers. For that, you need a syntax plugin, which is very powerful but the APIs aren't stable yet. This goes to the previous point.
* A few annoyances, like warning when you don't use a struct field as "dead code". If I'm interfacing with C I probably need that struct field whether the rust compiler sees it or not, but I don't want to disable all dead code warnings for that.
I think you can silence the dead code warnings by either marking the field `pub` or prefixing it with an underscore.
However, without an ergonomic way to create state machines (i.e. generators), it's hard to use at all in the intended fashion (small per-connection state instead of large boxed closures or coroutine/thread stacks).
I ask because a lot of extremely smart people I know like Rust.
Great tooling: Libraries can be written, published and used quickly. Code can be easily rebuilt on upcoming Rust versions without uninstalling anything. Integrated testing and documentation generation. Cross-compilation.
Truly cross-platform. As an example, Rust even supports both GNU and MSVC targets on Windows, that means Rust libraries can be linked into the C++ projects compiled with MINGW or MSVC. All the standard library features are cross-platform, unless namespaced otherwise.
Rust linear type system requires variables be either uniquely mutable or immutable, but not both. This solves whole class of resource management problems (memory being the most important) in the simplest possible way. This compile-time guarantee also works for all references to data, which become thin pointers at runtime. This, together with Send-able and Sync-able types, also ensures no data races.
Language is up-to-date. No-nulls, pattern matching, closures, generics, attributes will feel familiar to programmers coming from various languages. Language had many iterations and changes, and every bit of it was designed with care. Right now it may be a bit explicit, but sugar can always be added later if it appears worth it.
Stable release cycle, stability guarantee. We know when new beta will be renamed to stable and we know what's in it! That means there is time to check all ecosystem for possible breakage, even though historically Rust introduced hardly any breaking changes since 1.0 a year ago.
This is the right choice if you want the language to be "a better C" but it means that there is none of the Run-Time Type Information that we are used to in C# (or java).
The RTTI plays a small but key role in e.g. how a DI container works: The container works out what parameters a constructor needs at runtime and instantiates those recursively; and in how a mocking framework works: Generating a class on the fly with the right interface declaration. Even XUnit/NUnit work by using RTTI: The testing framework looks for all classes/methods with the right attribute metadata, and runs them.
It's easily fast enough for how we use C# and Java, but it's not zero-cost.
There are unit testing tools in Rust, but they cannot work the same way - IIRC, there is compiler support for them. So, While Rust as a good chance at being "A better C", don't expect it to do all the things that C# does involving types at runtime. A different goal meant a different language design.
There are other interesting things in Rust like the Macro system that might help do similar things, but it's really not going to work like in C#
I occasionally do this even in languages with good DI frameworks like Java when I don't want to pull in a Guice or Spring dependency. Even "manual" DI like this is so much better than not doing DI at all.
What issues in Go do this sentence refer to?
Guess you would normally run your tests with it, and maybe compile a dev / beta / RC version with race detection enabled?
Never used it in prod, but it did find a problem for me.
But obviously for any other language (C,Go,etc) you can use a compiler + static analysis + dynamic analysis to have the same confidence.
What I like with Rust is that implicitly you know that every one has to run the equivalent of analysis tool to release/deploy the code.
Not quite. There's certain types of issue which its type system won't detect - you can, in fact, easily get into a race condition, just not what's known as a data race - and there's no promise that what you've written actually does what you expect it to do, just that it'll do something without use-after-free and other memory-related bugs, and if you've written your program well it's more likely than e.g. Python that it does what you intended (in my personal experience).
Getting closer to "if it compiles, it works", but still not entirely there, there'd be the likes of Idris and other systems with proofs embedded into their type systems - but even then you still have to verify that the proof matches what you're actually trying to do in the real world.
Sometimes it's used too much by people coming from other language, deploying those patterns to Go