Fearless concurrency with Rust
blog.rust-lang.org
blog.rust-lang.org
This is what Go should have done. Then Go's "share by communicating, not by sharing" would be real, not PR. Go code often passes references over channels, which results in shared memory between two threads with no locking. In Rust, when you do that, you pass ownership of the referenced object. The sender can no longer access it, and there's no possibility of a race condition. A successor to Go with Rust ownership semantics has real potential. Go has a simpler type system than Rust, and seems to be well matched to the back-end parts of web-based systems.
However, in the intervening time, Rust has diverged farther from Go. It no longer has green threads nor emphasizes message-passing based concurrency. In addition, Go was ready to use much sooner; by having a much simpler type system, and less ambitious concurrency story, Go has been ready to use and build up a library ecosystem for many years now, while Rust is only just now hitting stability to the point where you can write code and not have it break if you don't constantly keep it up to date with the latest nightly.
So, I think that they are both valid strategies. There has been lots of good code written in languages which don't provide as many safety guarantees or as strong a type system as Rust has; it's not an absolute requirement for all code to be written with such guarantees.
By staying simpler and not trying to solve as many problems, Go has been ready for production use for quite a lot longer, and I think that there are many things for which it is still simpler and easier to use.
That said, I do prefer Rust now that it's actually hitting a stable and usable state. I wish it had been ready sooner, but that's the price you pay for needing a lot of time on iterating on how to make these stricter semantics actually usable in practice.
Google has much deeper pockets than Mozilla to build a language and infrastructure.
It has never been true so there's nothing to forget.
The number of full-time Rust people paid by Mozilla is roughly the same as number of Go people paid by Google.
Mozilla is a wealthy company and while Google is even more wealthy, it doesn't mean that it spends all its money on Go. Both v8 and Dart are staffed with more people and Dart is not taking off the way Go is.
People are trying to rationalize away the explosive popularity of Go with all sorts explanations, the "it's all because it's Google" being one of them.
The simple explanation is: people use Go because it's a good language. That's the big factor in Go's success.
Go fits a niche and is good enough, for now. But Go as a language is seriously flawed, and its flaws will become more and more obvious as the language gets more popular. People can certainly understand Go made the choice of "minimalism". But minimalism doesn't mean the language shouldn't have features people really need. The infamous "You don't need that in Go" sentence will not fly that much longer,especially given how the Go team and community tries to answer Go short comings. The biggest insult to intelligence is that "go generate" feature.It's just embarrassing.
Go for me is a missed opportunity. I still like it but I keep a close eye on alternatives that don't have their core library riddled with "interface{}" like types.
Because there is a need for something simpler and safer than C,that is easily accessible to "script kiddies",that is fast,that has easy concurrency, that compiles to machine code,that has type safety,that isn't tied to windows, in order for people to produce executables that can easily be distributed or concurrent servers.
Whatever language succeeds in that niche will be the biggest language of the next 10/20 years. Mark my words.
Actually, there is no need for that. Type safety has nothing to do with software being more reliable or more secure. There is, however, a need in language creators, who understand that large standard library with static binaries and cross compilation are super important in a fast moving world where many different operating systems live together on many different architectures.
It is true that type safety doesn't necessarily mean that software is more reliable and secure (it depends on the type system, the semantics of the language, and a host of other things), but saying it has "nothing" to do with it is incorrect. In particular, preventing undefined behavior certainly has an effect on security, whether or not it's through a static type system. As pcwalton has mentioned on several occasions, something like 50% of critical CVEs in Firefox exploited attack vectors that would have been prevented by Rust's type system.
Bugs have nothing to do with formal properties of the language. They are the result of people's thinking process. And type system may just as well be the thing someone was dealing with while making his next CVE-worthy mistake.
Even if we assume that people write buggy code at about the same rate in C++ and Rust, as long as the vast majority of Rust code is not inside an unsafe block, Rust still reaps this benefit, because bugs in safe Rust shouldn't cause memory unsafety.
Finally, yes, bugs do have something to do with formal properties of a language. Really. They do. You can prove that software matches its specification, or fulfills certain properties. People have. SeL4 is a thing. Relying on a static type checker means you only need to trust the static type checker to be correct in order to enforce the appropriate properties, not all code written in the language, and "the Rust type checker" is a much smaller kernel to trust, and will be more widely exercised and heavily scrutinized, than "all other Rust code ever written." This is literally the entire point of static type systems.
I apologize, but this is going to be my last response to you, because I don't think we can have a productive conversation about this.
You are missing my point. It doesn't guarantee to not cause other bugs. And saying that something eliminates some type of bugs is completely misleading, because this is not what matters. It matters to not cause other bugs as well as eliminate some bugs. Which is impossible to guarantee with formal methods. And which is why current academic approach to languages cannot bring us anything, until the whole system changes.
Psychological approach to programming language design is what has to happen. Anything else is broken.
If you said this to me in an interview I would politely end the interview there and you would not be hired. A high school student with no programming knowledge at least can be taught. If you're confident in what you just said, you're both ignorant and unteachable.
I really can think of no more scathing indictment of Go than this is the thinking of a typical Go user.
>The simple explanation is: people use Javascript because it's a good language. That's the big factor in Javascript's success.
>The simple explanation is: people use PHP because it's a good language. That's the big factor in PHP's success.
Doesn't feel right, does it?
Google and corporate sponsors are incredibly important in building the ecosystem of programming language. Traction may be the single major defining characteristic of language success and one can certainly buy traction with money and influence.
Having someone like Google, Mozilla, Apple, Microsoft etc behind it makes a big difference.
(But google's name certainly helped.)
I know he's the least publically visible of the three, but IIRC godoc and gofmt are mostly his babies. After getting used to those I think I miss them more in other languages than I miss the concurrency.
Rob Pike is explicit about early Go development; everything that went into Go was agreed upon by Robert, Ken, and Rob. If all three didn't agree it didn't go in.
I have written a fair bit of Go code. But I will switch to Rust without a second though once the ecosystem takes of. The lack of simple things like generics or algebraic data types in Go is jarring.
I think that in hindsight, this may be true, but it took a _very_ long time to figure it out. It took _years_ of iteration before we got to where we are today, and the previous work in this space, which used regions but not borrowing, didn't go further because of the ergonomic issues.
In retrospect, hard things often seem obvious, but that's because the work was already done!
And while the core idea is correct, holy shit there's so many details that the Rust team has had to deal with and iterate over and reduce until it's nice. I suppose this is why the CLR and everyone just punt on the whole issue.
It certainly had a big influence on Rust, we're fans of the work.
I had an undergrad senior thesis in 2006 that futzed around with regions and borrowing, and I know I read some papers on their interactions, but I'm not immediately finding anything chasing references.
edit: Of course, I don't mean to diminish the accomplishments of Rust, which had turned these moving parts into a working language. In particular, Rust makes it easy to ignore these issues most of the time.
To put it in startup terms: other people working on your idea only validates it, it's not a threat. Execution is ultimately what matters. I prefer letting academia blaze the intellectual trail, and then have a number of people implement it after. :)
Perhaps some day we will see a successor language that follows the philosophy of Go using the new stuff we understand thanks to Rust.
Well-understood like code generators (go generate)? Were generics not well-understood enough, so they decided to go with something that's well-understood to be a poor solution to the problem?
Perhaps by programmers (and the designers/maintainers of certain mainstream languages...), but there are some very strong models out there; we've had well-behaved parametric polymorphism à la Hindley-Milner for decades and Haskell's typeclasses are in my opinion a very well-thought-out approach to ad-hoc polymorphism (and which the more ambitious C++0x version of Concepts in some ways resembled).
C++'s templates are like the untyped lambda calculus of the type-level world; they're perfectly capable of doing most anything you'd want from them, but using them effectively requires a lot of boilerplate and discipline (and as for std::allocator_traits et al., I think that's more an issue of library design than the design of templates).
I've never found subtype polymorphism particularly convincing beyond the case where independent single classes implement meaningful interfaces, and that's basically like a member function–oriented version of typeclasses.
Now, all this said, I haven't actually used Go, so I don't know how well the rest of the language would interact with these ML/Haskell-style systems, but I imagine at least basic parametric polymorphism would be relatively unobstructive.
I definitely agree with the sentiment about subtype polymorphism.
template <class X> foo {};
template <template<class> class F> struct bar { F<int> f; };
bar<foo> b;
I haven't investigated it but variadic template template arguments might help there, though.
They do:
template <template <typename...> class Container, typename T>
Container <T> fin (T n) {
Container <T> set;
for (T i = 0; i < n; ++i)
set.push_back (i);
return set;
}
int main () {
for (int i : fin <std::vector> (10))
std::cout << i << std::endl;
}If the only answer to the issue they raise is "you don't need that in Go", "Use go generate" or "Go isn't for you", Go is going to get a lot of bad rap that will stop the adoption of the language.
If the Go team thinks the language doesn't need generics they are either out of touch(their rights,they don't know owe us anything) or arrogant. Either way, Go will not be the successful language it could have been.
One should never have to resort to interface{} to code anything in Go. Developers want type safety, not write type assertions everywhere. The fact that 1/ Go has generics but you cant create yours(map/slices/arrays) 2/ the core lib is full of these interface{} functions proves the language has a big problem.
If Rust had been missing an important feature like generics, the Servo people would have been telling the compiler people "you're being silly, of course we need generics!". And because Servo is the official test-case project for Rust, the compiler people would have listened.
Go has nothing like this. There's no formal effort to ensure practicality.
It's hard to design a language. Everybody is always trying to give their conflicting opinions. Many language designers react by turning inwards and ignoring everybody else.
The Go team might not understand generics, but that doesn't mean generics aren't well-understood.
This is my core problem with Go: they clearly have no idea what a decent programming language would look like and they're getting undeserved attention becauase of their previous (admittedly impressive) work.
I think Go has gotten fairly decent traction because there are roles for which real programmers find it reasonably well suited compared to existing alternatives, and I think that, not the previous work of the creators, is what drives attention to it.
OTOH, I think Go's period in the sun is going to be a lot shorter than, e.g., Rust's unless we see a Go 2.x that addresses some of the "not in the Go 1.x timeline" things with Go fairly soon.
1. Generics aren't templates, and are in fact a solution to this problem. They don't expand at compile time. 2. The long compilation times are indicative of deeper problems: an over complicated code base that should be refactored down, and lack of modularity which would allow partial compilation (only compile the changed modules). 3. Removing generics creates new problems with increased bugs and duplicated code. It's trading compile-time for development time, which has historically rarely been a good trade. 4. Even if this is a legitimate concern for Google (which I still don't think it is), it's not a legitimate concern for 99.9% of projects out there. How many projects actually have millions of lines of code that can't be factored down and modularized? If this is really Go's target then they should admit it's a highly specialized language and not a general-purpose programming language, and that most people shouldn't be using it.
Calling these goals "subjective" as if this justifies them is just a smarter-sounding way of saying "That's just, like, your opinion, man." Opinions can be wrong, and the opinions of the Go team are wrong.
Even OCaml, whose approach to parametric polymorphism is worlds simpler than any of the monstrosities mentioned above, just changed its approach by adding GADTs, which I still don’t understand fully despite the best efforts of generous HN commenters.
So I think that, given that Golang/Issue9 decided not to include inheritance, linear types, exceptions, or overloading (even of operators!), in large part because they aren’t well-understood, it’s entirely unsurprising that it decided not to include generics, either.
Yes, that is exactly what I said in the post you are responding to.
> I think it’s fair to say that we are only beginning to understand the benefits and drawbacks of that approach to code generation, which is why every time I recompile the Java application I’m currently working on, I get erroneous warnings telling my that my varargs are causing heap pollution in my generic static methods. (Which I can suppress, but instead I’m planning to delete those methods.)
Just because you don't understand something, doesn't mean that nobody does?
Many of the ideas in go were not well understood and had only been demonstrated in research languages prior to go. Its concurrency model and associated primitives, and the automatic interface system were in roughly the same state then as Rust's new ideas are now.
That said the ideas in Rust were not as well understood 5 years ago and I doubt that the go creators could have incorporated them well at the time. It took Rust 5 years of experimentation to come to what it is now (and its still got a number of additions to go!).
Its interface system is an implementation of structural typing, which may be even older.
You could maybe argue that neither had "industrial-strength" implementations, but that seems like a stretch.
to Go's authors the CSP model was very well understood through three different implementations: Alef, Limbo, and Plan 9's libthread.
Also Newsqueak.
> Go code often passes references over channels, which results in shared memory between two threads with no locking
Having a complex type system would cut into compile times (an explicit primary design goal, to the point where the Go compiler must be written to read each file exactly once[0], no more).
As for the code you're referring to, Go is not designed to be a language which prevents you from shooting yourself in the foot with provable code. It's designed to be a language which makes it reasonably easy to be sure you haven't, as long as you follow the general idioms and best practices. What you're describing here is definitely not one - I can think of a few instances in which pointers might be reasonably passed over a channel, but they're few and far between.
Also, I would think it's pretty obvious that once you've sent something over a channel you shouldn't try and write to it anymore[1]. Do you have an example of the code you're referring to?
[0] well, technically "at most once", since some files can be skipped entirely.
[1] I'd need to think about this, but I don't think it'd be difficult to detect this statically[2] at compile-time and enforce that stack-allocated rvalues into channels are never used again in the same scope. It's definitely possible to extend `go vet` to handle this, and it may even be possible to write this in as a compiler error in a future version of Go.
[2] Incidentally, one of the reasons that it's so easy to do reliable static analysis on Go code (compared to other languages) is that the grammar is incredibly simple - it's almost entirely context-free, which is very rare among non-Lisps. Having a more complex type system usually requires at least some additional syntax to along with this, which means you'd have to start sacrificing this design goal as well in order to create a more elaborate type system.
Same in Rust. Thanks to the module system, the Rust compiler never rereads a file more than once.
> [1] I'd need to think about this, but I don't think it'd be difficult to detect this statically[2] at compile-time and enforce that stack-allocated rvalues into channels are never used again in the same scope. It's definitely possible to extend `go vet` to handle this, and it may even be possible to write this in as a compiler error in a future version of Go.
That trivially fails to solve the problem, due to aliasing.
> [2] Incidentally, one of the reasons that it's so easy to do reliable static analysis on Go code (compared to other languages) is that the grammar is incredibly simple - it's almost entirely context-free, which is very rare among non-Lisps. Having a more complex type system usually requires at least some additional syntax to along with this, which means you'd have to start sacrificing this design goal as well in order to create a more elaborate type system.
Rust's grammar is also context-free, as far as I know.
Your proposed static analysis is not reliable. The "more complex" type system in Rust exists precisely so that we can do more reliable static analysis.
I believe bytestrings are context-sensitive :( I don't remember the exact details here, so I could be wrong. But other than that...
Are you sure about this? When you use a generic function accepting and/or returning values of type T, I guess the compiler has to generate a version of the function for each instance of T. But the compiler cannot know all the possible use of the generic function without having walked through the whole code first, which implies at least two passes. How does it work?
No I don't think so.
> Same in Rust. Thanks to the module system, the Rust compiler never rereads a file more than once.
As I said right at the beginning of my post, I'm not trying to compare Rust and Go directly, because I don't think that's meaningful. I'm explaining why these particular features would be difficult to incorporate into Go. Note that I never said that Rust reads a file more than once, or that Rust's grammar is not context-free. In fact, the word "Rust" doesn't appear anywhere in my comment at all except in that very first paragraph.
I love talking about PLT and would otherwise be interested in having a discussion about static analysis and hearing why you think it would not solve the problem, but I have to say, it's both frustrating and discouraging to post an in-depth response and then get downvoted twice, with the only reply being one which very clearly ignores the very first line of my entire response.
If it's actually not that hard, go implement a checker for Go that does check that statically-allocated rvalues are never used again, and post it here. You'll probably shoot to the top of Hacker News, it'd be a nice open-source project for the resume, and it'd provide a very useful tool for the Go community.
Stack allocation isn't in the semantics of Go, so that's a pretty weird thing to use a basis of a static analysis. It's also not sound because of interfaces or closures. You would want something more like "fully by-value data with no pointers in it, no interfaces, no closures".
It's perfectly reasonable to compare any two programming languages, especially where their use cases overlap.
> As for the code you're referring to, Go is not designed to be a language which prevents you from shooting yourself in the foot with provable code. It's designed to be a language which makes it reasonably easy to be sure you haven't, as long as you follow the general idioms and best practices. What you're describing here is definitely not one - I can think of a few instances in which pointers might be reasonably passed over a channel, but they're few and far between.
Absolutely sure is better than reasonable sure.
But everyone keep trying to though, that's kind of interesting.
The reason I am guessing is because Go billed itself as a "systems programming langauge". It did, go find the original announcement video (or was it a press release). That is what it said.
Later after replacing C idea didn't quite materialize. They clarified that they mean "systems programming language" to mean something else actually. Now if I said that or some other anonymous user, ok, fine, people get confused, don't know enough etc etc. But I don't believe Rob Pike doesn't know what a systems programming language is.
Anyway, I am not saying one way or another but just illustrating that everyone doing the comparison is not completely insane.
Now they way I see Go is more like a Python++ or Ruby++. All those nice concise languages, + concurrency, + speed, + some type safety, + easy static binary deployment. So I agree with you that Rust and Go should not be compared.
Internally what I've seen @ Google is it seems to be used here and there for services where Google teams have typically used C++ (but other companies might use Java.)
And yes, some places where Python has been used, Go makes a decent replacement.
Not my cup of tea, but I can see its niche. It's not the same niche as Rust. At least not right now.
Almost all languages are different and thus have different design goals. That doesn't mean it should be off-limits to compare them.
Then why does everyone insist on comparing Rust and C++? Objectively Rust is much closer to Go than to C++.
That's just dumb. Rust and Go were, in fact, contemporarily designed to similar constraints and with similar goals. That they made significantly different design choices is an interesting point that should be discussed and not swept under the rug just to win internet points.
I think this is absolutely true, for what it's worth.
You're saying, I think, that the Go implementation and runtime looks more similar to Java and that C and Rust are clearly in the same family (in the sense of being only losely coupled to a runtime environment). That's the way a language implementer might look at the question, I guess. It's certainly not the only one.
I'm not sure what you mean by this, but when I hear "managed heap" I think of heap memory managed by a garbage collector. This is not a feature of Rust.
> emphasis on the use of functional constructs
Rob Pike wrote (unavoidably, because of the lack of generics) crippled versions of map and reduce for Go and declared that the almightly for loop was superior. I don't think an emphasis on the use of functional constructs follows from this.
> rejection of lots of previously-thought-to-be-essential OOP features
Yep, this is an important similarity between Go and Rust. Rust brings traits, which are (extremely) similar to Haskell's typeclasses, into the mix as well.
> built-in concurrency based on a rejection of the traditional primitives
Sure, but the differences between the languages is crucial here: Go builds safe(ish) concurrency primitives into the language; Rust does not, but provides language features powerful enough to build memory-safe concurrency primitives in the standard library. And, perhaps surprisingly, mutexen and the like are actually encouraged precisely because Rust can make them safer.
> general de-emphasis of metaprogramming constructs like templates and macros
I don't agree that Rust de-emphasizes these. Generic programming is strongly encouraged, and indeed Rust's generics are implemented very similarly to C++ templates. Rust also strongly encourages the use of macros for cleaning up repetitive blocks of code, both inside of and outside of function bodies. And very powerful metaprogramming is on the way (already available in nightlies) in the form of compiler plugins, which allow arbitrary code execution at compile time.
> broad design emphasis on preventing common programming mistakes.
I think this is a feature, or at least an intended one, of every higher-level programming language :-)
There's been a great community and core team effort to design small, orthogonal language features (or extensions to existing features) that can be used to regain the performance + code reuse benefits one gets from using inheritance, without all the associated warts. The DOM/Servo problem is a tough one and it's going to be very interesting to see if Rust can solve it without resorting to the blunt instrument of inheritance.
Here's an example from last year of such an attempt, in RFC form: https://github.com/rust-lang/rfcs/pull/250
At least in Rust, type checking takes basically no time compared to things like optimization passes.
Oh give this lame argument a rest already. They're both supposedly general-purpose programming languages. Rust shoots for a little lower level than Go, but they can certainly be compared, and the comparisons are valid.
> Having a complex type system would cut into compile times (an explicit primary design goal, to the point where the Go compiler must be written to read each file exactly once[0], no more).
I'm not sure how this is a defense of Go: yes, that's a primary design goal; it's also a terrible design goal. Trading an adequate type system for a short-term gain in compile time is an amateurish mistake.
Compilation time can be mitigated on even the largest projects by only rebuilding changed modules. If you're getting to the point that this isn't working, then maybe it's time to start working on reducing the size of your code.
I've worked on projects that were 500,000 lines of code and compile time was sometimes an issue, but not nearly as large an issue as bugs which would have been caught by an adequate type system (large C#/Java codebases pre-generics).
> As for the code you're referring to, Go is not designed to be a language which prevents you from shooting yourself in the foot with provable code. It's designed to be a language which makes it reasonably easy to be sure you haven't, as long as you follow the general idioms and best practices. What you're describing here is definitely not one - I can think of a few instances in which pointers might be reasonably passed over a channel, but they're few and far between.
We've tried this approach before, many, many times, and the result has been high-profile bugs.
The fact is, these "idioms and best practices" will not be followed perfectly on projects of any reasonable size if they are not enforced by code. Why would you not enforce them with code? And if you're designing a language, why not design the language in such a way as to allow code enforcing best practices? Like, a type system?
> Also, I would think it's pretty obvious that once you've sent something over a channel you shouldn't try and write to it anymore[1].
Yes, just like it's pretty obvious that when you free memory you shouldn't write to it any more. We've never had any issues with that, now have we?
> It's definitely possible to extend `go vet` to handle this, and it may even be possible to write this in as a compiler error in a future version of Go.
I didn't know about `go vet` and I wish I didn't.
So, to avoid a second pass over the code during compilation, you add a second pass over the code during `go vet` that has less capabilities. Good thinking. I'll add that onto the list of other stupid ideas that have been tried decades ago and didn't work but are finding new life in Go.
> Incidentally, one of the reasons that it's so easy to do reliable static analysis on Go code (compared to other languages) is that the grammar is incredibly simple - it's almost entirely context-free, which is very rare among non-Lisps. Having a more complex type system usually requires at least some additional syntax to along with this, which means you'd have to start sacrificing this design goal as well in order to create a more elaborate type system.
Yes, adding syntax for static type checking will definitely make static analysis like type checking harder. Are you fucking kidding me?
Compiler shall not be an obstacle to a man. We tried enforcing things before, many times. Things got abandoned.
For the former group, error checking just gets in their way. Their job is not to make the software perfect, it's only to make it satisfy one person's need, to replace something that previously wasn't computerized with something that was. Oftentimes, it's not even clear what that "something" is - it's pointless to write something that perfectly conforms to the spec if the spec is wrong. So they like languages like Python, Lisp, Ruby, Smalltalk, things that are highly dynamic and let you explore a design space quickly without getting in your way. These languages give you tools to do things; they don't give you tools to prevent you from doing things.
The second group works in larger teams, with larger requirements and greater complexity, and a significant part of their job description is dealing with bugs. If a significant part of the job description is dealing with bugs, it makes sense to use machines to automate checking for them. And so they like languages like Rust, C++, Haskell, Ocaml, occasionally Go or Java.
The two groups do very little work in common (indeed, most of the time they can't stand to work in the opposing culture), but they come together on programming message boards, which don't distinguish between the two roles, and hence we get endless debates.
You describe a system where you have to keep everything a program is doing that's relevant in your head at once, and when you're forced out of that state, it's catastrophic. You seem to be assuming that's the only way to get productive work done while programming. I happen to know it's not.
If a language has a sufficiently good type system, it's possible to use the compiler as a mental force multiplier. You no longer need to track everything in your head. You just keep track of minimal local concerns and write a first pass. The compiler tells you how that fails to work with surrounding subsystems, and you examine each point of interaction and make it work. There is no time when you need the entire system in your head. The compiler keeps track of the system as a whole, ensuring that each individual part fits together correctly. The end result is confidence in your changes without having to understand everything at once.
So why cram everything into your brain at once? Human brains are notoriously fallible. The more work you outsource to the compiler, the less work your brain has to do and the more effectively that work gets done.
We all agree that a null dereference is a bad thing at runtime. I see no advantage for me as a programmer to be allowed to introduce null dereferences into my code as a side effect of "getting things to work" if then when the code runs it doesn't work right. This increases my cognitive load as a programmer, it does not decrease it.
I would argue that you don't think about the compiler anymore when using a language like haskell than you do when using Python. But you do get more assurances about your program after ghc produces a binary than after Python has finished creating a .pyc -- and that is a win for the programmer.
The questions, to my mind, are
1. How easy it is to opt out?
2. How often do you have to opt out?
3. How easy it is to write well-typed expressions?
4. What guarantees does a program being well-typed provide?
For example, you almost never have to opt out of the type system in a dynamic language, but the static type system is very basic. In a language like Rust, you opt out semi-frequently (unsafe isn't common but it's certainly used more often than, say, JNI), and it can be hard to type some valid programs, but opting out is simple and the type system provides very strong guarantees. In a language like C, you never have to opt out of the type system, and the annotation burden for a well-typed program is minimal, but the type system is unsound--being well-typed guarantees essentially nothing in C.
All languages fall somewhere along this spectrum, including Go. It's just a question of what tradeoffs you're willing to make.
We tried static code generators before, as well as linters and static code analysis on untyped code, and those have pretty well been proven to be ineffective. All of which are supposedly "new innovations" in Go. So if you want to defend Go that's not an approach you can really take.
The interesting thing is that, thanks to Rust's ownership system, it is easy to discover when handles to a future value go out of scope. This allows the library to reuse internal resources to make the overhead of using the library as light as possible.
For example, a Stream is modeled as a sequence of futures (more specifically a future of the next value and a new stream representing the rest of the values, kind of like lazy-seq in clojure but async), but instead of allocating one future for each iteration of the stream, the same internal structure is reused while still being able to expose this API.
But Rust being so low level to not care about IO is Awesome :)
The discussion points out something else that too often receives too little attention. That is, the quality of documentation is extremely relevant to product success. Sadly, software is more often than not poorly documented, not only frustrating for potential users, but an unnecessary impediment to uptake of the program.
It appears the Rust developers are serious about describing how Rust works. That's a very favorable sign of their commitment to the project and speaks volumes about dedication to grow the Rust community.
Also, are there any plans to integrate tutorial-style guides similar to those from Rust for Rubyists into the Rust book? There's something incredibly fun and real about learning by example. It also would give the reader a chance to read some idiomatic code.
Which has a bunch of examples that utilize most of Rust's features and are runnable/editable in the browser.
And there was a comment on /r/rust[1] that made a good point about the documentation: It's better for searching for known-unknowns rather than unknown-unknowns. Since the community is still relatively small and idiomatic Rust is still being defined, it's hard to find those known-unknowns.
That being said, I've found that the Rust user forum[2] is very helpful and generally the community is full of people who understand the language way better than I do and who are willing to help.
[1]: https://www.reddit.com/r/rust/comments/2zu3eo/what_is_rust_b...
I'd been waiting for beta to drop to do a re-organization of the TOC of the book. You can see it on nightly here: http://doc.rust-lang.org/nightly/book/
I've carved out a whole section, "Effective Rust", specifically for this kind of thing. I'm looking for a better name than "Effective Rust", since that's a very common book title, though. Don't want to be too greedy with that namespace!
We do also have Rust by Example, but I admittedly don't give it as much love. Once the book is done...
We do link it from http://doc.rust-lang.org/ , but I guess that still says community...
It actually took me a while to find the link there just now. I don't think "External Documentation" really imparts the right meaning. Perhaps something like "Other Guides and Resources" would make more sense?
This has bitten me so much that I avoid fine-grained locking when possible and instead use a single global lock everywhere, and have new threads start in the locked position, and then only selectively unlock the bits of code that I can prove are thread-safe, or adding fine-grained locks only where absolutely necessary as shown by benchmarking.
This is the difference between a data race and a race condition. We can't generally prevent deadlocks, I would imagine that's (like all race conditions) is an unsolvable problem at the language level. Maybe some PhD will prove me right or wrong, though...
In general, problems of "guarantee property X" can almost always be solved with a sufficiently advanced type system, but at the cost of (a) inventing and using a sufficiently advanced type system and (b) disallowing some perfectly good programs.
While this may be true for certain restricted systems, this claim is misleading. Session types, as invented by K. Honda and refined by numerous others, guarantee deadlock and race freedom only within any given session. Session initiation (basically messages to replicated inputs) can deadlock and race in many (most?) session typing systems. The decision not to guarantee the absence of deadlock at session initiation points is what makes session types so simple.
It is possible to restrict session types so that they are completely deadlock-free and race-free, but -- as far as we know -- that guarantee comes at the heavy price of making the typing system completely inexpressive, so you cannot type many natural forms of concurrent programming. Alternatively you can give up on simplicity and make the typing system very complicated (e.g. the system pioneered in "Strong Normalisation in the Pi-Calculus" by Honda et al, and its successors). The former is a real problem, as witnessed by the continuing inability of the session typing crowd to get a handle on message passing in Erlang.
Finding a system that is expressive, simple and guarantees deadlock-freedom is a major open research problem (and I suspect infeasible).
I'm not sure what the equivalent in my world of "session initiation" is. If by "replicated inputs" you mean what in linear logic is expressed by the "!" connective (often called "replication"), it's definitely possible to have ! without deadlocks or any observable nondeterminism. See Caires & Pfenning, "Session Types as Intuitionistic Linear Propositions", for example.
I already listed a few of the limitations of this approach (limited process topologies, and no `select()` analogue). I'm not sure whether these qualify as "completely inexpressive", but I do think the lack of any `select()` analogue is pretty limiting in practice. I'm not sure how hard it might be to remove that limitation, but it wouldn't surprise me if that were a major sticking point. Still, overall I think I'm more optimistic about the possibility of capturing most reasonable forms of concurrent programming in a deadlock & race-free manner.
Yes, replicated input is !x(v1).P in pi-calculus, and Milner was directly inspired by linear logic's replication.
> Still, overall I think I'm more optimistic about the possibility of capturing most reasonable forms of concurrent programming in a deadlock & race-free manner.
I don't think you can, or indeed want to excise all forms of race condition from concurrent programs, for if you do that, there would be little concurrency left. You could only do deterministic concurrency, which is not what you want in many cases, e.g when you use concurrency to mask latency. For example how would you define a lock or mutex if all races are forbidden? You need race conditions, but you want them in homeopathic doses, you want them constrained and tightly controlled.
I also doubt that concurrent programming patterns like dynamic deadlock detection and removal are amenable to a type-theoretic analysis in linear-logic based systems such as Caires & Pfenning's.
Rust safety guarantees are based around a few rules involving references, ownership, lifetimes and borrowing. None of them forbid the creation of mutexes or anything else which you will find in Rust.
'unsafe' Rust code shouldn't break these rules any more than the 'safe' code does, the only difference is that the compiler lets you do something which it doesn't know is safe. It's up to you to design the code such that it is.
If code marked as 'unsafe' does something which actually violates one of the assumptions on which the Rust compiler works, then the safety of the whole program is jeopardised.
(There was actually a bit of bike-shedding surrounding possibly renaming the 'unsafe' keyword to better reflect these facts, which included suggestions like 'trustme' and 'yolo', but a satisfactory term was not found)
Of course, what you say is perfectly true, the Rust language itself has no threads or concurrency, and hence is automatically dead-lock free.
I guess what I was trying to get at is that 'race conditions' can fall under the same kinds of errors as logic errors. This is the way that people usually attack the phrase "if it compiles, it works."
I would define a race condition as "an error due to nondeterministic concurrency". If you ban nondeterminism, for example, race conditions are impossible. In practice, nobody wants to ban nondeterminism, so races creep in. There might be smarter ways to eliminate races (ways of only allowing benign nondeterminism), though. So I'm not sure that practical languages without race conditions are an unachievable goal, just a difficult one. Give it another 20 years and we'll see where we're at. :)
(Although, come to think of it, you can probably work around that with some cleverness; e.g., you could provide the option to access either the data protected by the Mutex or the next Mutex in the sequence through &mut; whichever you chose, you couldn't access the other until you were through with the children. You still have to keep the mutexes structurally in a linked list, though, which is obviously not ideal for all situations. And you still need to deal with the fact that the reference to the first Mutex must be an &-reference, hence can be replicated indefinitely, which allows you to subvert the ordering guarantees; but this was already a problem, and I think there are some clever ways around this using rank-2 lifetimes).
Yesterday I wrote a macro to build simple actors[1], without a care for robustness (this was just a rough draft and I didn't expect it to even work so quickly - amazingly it did!). Now I'm looking at how to fill it out and make it generally useful and get this: I didn't even try to handle failure yet, but it already responsibly joins the thread and deallocates its resources as soon as the actor can no longer receive messages. Automatically! As a natural consequence of Rust's memory model and type system. Despite the fact that Rust knows nothing about concurrency at the language level.
> the same tools that make Rust safe also help you tackle concurrency head-on.
The tools that make Rust safe, in this instance, being its ownership type system. Why on earth should that be the case? Why would ownership types make concurrency easier? The post gives plenty of in-depth answers to this question, but to my mind it misses the bigger picture.
The big-picture reason why ownership types and concurrency match so well is the Curry-Howard correspondence.
Ownership types are known as linear[1] types in the PL theory literature. Linear types are so-called because they correspond to linear logic, a logic sometimes described as "the logic of resources" rather than "a logic of truth". In linear logic, instead of reasoning about eternal truths like "1 + 1 = 2", you reason about resources, like "I have an apple and a banana". If you also happen to know that "from an apple, I can make applesauce", then linear logic lets you reason that you can obtain the state "I have applesauce, and a banana" (but no longer an apple).
However! Linear logic has another interpretation, a Curry-Howard interpretation. You can also view it as a type system for a concurrent language based on message-passing, where your types describe the protocols message-channels obey. "I have an apple and a banana" corresponds to a channel which will first produce an apple, then a banana, then close. This is known as the session-types interpretation[2].
At first glance it may seem that this interpretation of linear logic and Rust's interpretation are unconnected. I doubt it; in logic, everything connects to everything else. It's no coincidence, for example, that ownership (linearity) is what you need in order to send mutable values across a channel without copying. There is a deeper structure waiting here to be discovered, and I for one am deeply excited about it.
[1] In Rust's case, technically they are affine types. An affine type is a linear type whose values can be freely dropped, i.e. deallocated.
[2] Session types have been around for a long time, but only fairly recently was the connection to linear logic discovered.
Links/papers on session types:
1."Propositions as Sessions" by Wadler (http://homepages.inf.ed.ac.uk/wadler/papers/propositions-as-...)
2. Caíres and Pfenning, "Session types as intuitionistic linear propositions" (http://www.cs.cmu.edu/~fp/papers/concur10.pdf)
3. Pfenning's published works page (http://www.cs.cmu.edu/~fp/publications.html).
[0] https://github.com/Munksgaard/rust-sessions [1] https://github.com/Manishearth/humpty_dumpty
"First, the `linear` attribute as described here does not create true `linear` types: when unwinding past a `linear` type, the `linear` attribute will be ignored, and a `Finalize` trait could be invoked. Supporting unwinding means that Rust's linear types would in effect still be affine."
http://internals.rust-lang.org/t/std-fs-tempdir-ignoring-err...
Edit: yep, it was mentioned in the discussion here. http://internals.rust-lang.org/t/issues-in-new-i-o/1658/4?u=...
Another (experimental) language that uses linear types to facilitate concurrency/parallelism is LinearML: https://github.com/pikatchu/LinearML
I'm working in a toy language focused in data (relational), on top of F#.
But if you're building on top of F# you have a static type system at your disposal. Usually, languages that incorporate linear types and first-class garbage collection treat linear types as a separate kind, and allow them to interact with standard types only in limited ways. See, for example, https://github.com/idris-lang/Idris-dev/wiki/Uniqueness-Type.... I believe http://protz.github.io/mezzo/ is also exploring this design space.
It will start as a interpreter, but probably the challenge is the ergonomics of it: How make it "easier" to use... probably limit it to only handle resources? (ie: files, handles, etc, like the IDisposable in .net)
(search for { contra godel, direct logic }) Can't find it but he quotes a really interesting exchange between Wittgenstein and Turing (who was attending his class) where W in his inimitable style just waves away paradoxes and says "so what?"
I love Clojure, but this would be a completely different beast, a crazed DRAGON probably.
PS: I want a Dragon
EriPascal, an ancestor of Erlang, used runtime ownership for passing mutable binaries around. Once you send the binary in a message it is a runtime error to use it again afterwards.
Like, in the "Disaster averted" example the mutex section, how is it detecting that "error: `guard` does not live long enough"? I see that access returns a borrowed mutable T. But, how does the compiler know that that's borrowed from guard instead of a hypothetical second or third parameter or from some other (global?) state altogether?
Edit: Or maybe if a function returns a borrow, then the borrow cannot outlive the scope of the function?
fn access<'a, T: Send>(guard: &'a mut MutexGuard<T>) -> &'a mut T;
EDIT FOR CLARITY: in particular, you're saying that if you take a "sublease" from something you've already borrowed, that sublease is valid for no longer than the original lease. So for example if you borrow a field out of a borrowed structure, the access to the field can last no longer than you had access to the original structure. In this case, we're saying that access to the data lasts no longer than the lock is held.
I'd recommend http://blog.skylight.io/rust-means-never-having-to-close-a-s... if you want to get a bit more of a feel for how this works.
The details about the shorthand are here: https://github.com/rust-lang/rfcs/pull/141
static safety guarantees can come with auto pointers (unique_ptr in particular). This is why I said lthread_cpp & C++11.
Statically or at runtime guarantees don't buy me much. Regarding mutations, const and copy/move constructors give you want u want. Generally, all you need to do is have 1 thread create an std::unique_ptr<Class> object and pass it to the channel for other thread to pick it up.