Narrowing the notion of a runtime in Rust
github.com
github.com
There's a compile-time optimization which does many of the things people want to do with such threads. If you have two tasks communicating over a channel or queue, only one task writes to the queue, only one task reads from the queue, and the maximum length of the queue is 1, the code can be compiled as a coroutine. Go may get that optimization at some point; it's been discussed.
(Things that have been learned about Go goroutines: 1) the above case is moderately common, 2) for "select", the most common case is N=1 alternatives, followed by N=2. N>2 is rare. Go has special case code for N=1, and the case for N>1 is very general and does a lot of dynamic allocation. A special case for N=2, which in practice is usually read-or-timeout/abort, would be useful.)
The main reason that it's taking a while is they want to get it right. There are a number of breaking changes in the pipeline that they want to get finalised, to improve the language. After that, they still have plenty of features in mind for post 1.0, but these will be added as to not break 1.0 code.
For example, I'd been looking at doing Rust development for Android; currently native (i.e. C/C++ apps) on Android get run through a Java wrapper; starting a Rust process with the Rust runtime active is incredibly complex. With this change, it should be trivial to have a basic Android activity that just calls a Rust main() and have everything go from there.
The cheapest "large memory" part on Mouser for the ARM arch with an A/D is a Freescale MKL02Z8VFG4 8K flash / 4k ram and a 12 bit A/D is 1.12 qty 1, just above 50 cents qty 500.
This particular likely doesn't affect running Rust on an Arduino at all, as what it is doing is removing the runtime for std, which contains many of the OS abstractions. You don't have an OS on an Arduino however, so std isn't particularly helpful there. Several months ago they already did what was necessary for that, factoring out much of the completely OS independent functionality into a "core" library (http://doc.rust-lang.org/core/), so you can opt out of using std and use "core" instead, plus add a little bit of platform specific glue, to run Rust on bare metal.
But the last commit to that fork was just 10 months ago though... might be tempted to have a look during the holidays.
Will be hard to find an ARM equivalent for the ultra low power MSPs though.
The small, deeply embedded scale usually runs bare-metal or a small RTOS, while the next tier up often runs Linux. Now the small scale ARM is basically on the fast side (32bit proc) the very low end embedded cpus class. But it's often selected for everything but the most cost or power sensitive applications which tend to go for very small, power sipping 8bit cpus (e.g. MSP430s)
Just for fun, I picked a semi-random example of ARM mcu from digikey: EFM32ZG110F32[1]. Based on the datasheet it consumes about 8mW when running on full 24MHz speed. Meanwhile the AVR chip used in eg. Arduino Uno[2] consumes about 10mW when running at 8MHz. Of course at these ultra-low power levels there are quite a large amount of variability depending exactly what you are doing and how well your code is made.
[1] http://www.digikey.com/product-detail/en/EFM32ZG110F32-QFN24...
[2] Atmega 328p http://www.atmel.com/devices/atmega328p.aspx
So I doubt many people will be motivated to work on this.
https://github.com/rust-lang/rust/blob/6bdce25e155d846bb9252...
(And one of those isn't a Lovecraft quote, it's a Majora's Mask quote.)
Seeing the diff in the test cases should show how the API has changed, but I'm having a hard time reconciling the description of the changes with how the tests have changed.
There are several occurrences of changes like this:
- spawn(move|| {
+ Thread::spawn(move|| {
tx.send(i);
- })
+ }).detach()
Can anyone shed some light?
Threads unlike tasks must be detached if they are too be allowed to outlive their parent.
This was quite an exciting idea at the time. However, in practice, it just didn't pan out: the compromises required to paper over the differences between 1:1 threading and M:N threading managed to almost entirely obliterate the respective advantages of both models.
As a result, Rust is in the process of removing the "tasks" abstraction and replacing it with a bog-standard 1:1 threading model (though it still benefits from all the typical memory-safe and concurrency-safe abstractions that Rust otherwise provides). All users of concurrency in the stdlib will now be using native threads, and it will be left to outside implementors to experiment with green threading libraries (see https://github.com/carllerche/mio for one example of such).
One of the consequences of this change is that the `std::task` module has been replaced with `std::thread`. Formerly, every Rust program imported the function `std::task::spawn` to use for spawning tasks. With the removal of `std::task`, the newly analogous function from `std::thread` has not been added to the prelude, and is therefore not imported by default in every Rust program. It could be added if such a thing were desired (you can see remarks to this effect in the comments on the PR itself (https://github.com/rust-lang/rust/pull/19654#issuecomment-66... and https://github.com/rust-lang/rust/pull/19654#issuecomment-66... )).
As a core contributor, you will retort that Rust has different priorities than, say, Erlang, which is built around the concept of providing N:M processes (green threads) as a language primitive. My answer to that is, why not improve on Erlang, instead of settling for improving on C?
And just because Rust has a focus on pragmatism doesn't mean that it's not blazing new territory. Neither Ada nor Cyclone can boast such extensively useful statically-safe references. And figuring out how to do unboxed closures memory-safely in a language without a garbage collector is, as far as I can tell, entirely novel (though you'd have to ask a PL researcher to be certain). I expect aspects of Rust to show up in all future systems programming languages, including future revisions of C++ (as well as brand-new efforts such as M#).
And if you really need to M:N behaviour, it is still available as a library.
This change was necessary to keep Rust as a general use systems programming language.
M:N threading decouples degree of concurrency from degree of parallelism, which is not desirable only for networking. And the old way wasn't about forcing M:N threading, it was about providing a common interface for concurrency independent of the relation of concurrency to parallelism in the backend implementation.
I can understand the idea that that abstraction may have been too expensive, but I can't get behind the idea that 1:1 threading as the only thing supported in the standard library somehow provides more freedom for developers.
Long term, we can have libraries that provide M:N threading with minimal overhead; and also libraries that provide 1:1 & M:N together like the old std (with the associated overhead).
That is, the programmer is getting more flexibility, just not right now (especially with Cargo, which makes it particularly easy to use external libraries). The team is trying to take a longer term view focusing std on safe interfaces to the OS primitives, which can then be used to build fancier/more abstract libraries more easily.
Having done some things with libgreen threads myself, they really didn't give anything over proper threads (especially without segmented stacks). So I am curious how this will evolve in the future.
No, native threads have a much higher context switch overhead. The downside of green threads is the lack of parallelism, you can only use one CPU core. This is why decent languages multiplex green threads over OS threads (M:N). The mistake rust made was trying to make that distinction transparent.
Some of the actual context switch in native threads is performed by the CPU, where as with green threads it is not. On top of this, the whole stack is copied upon a context switch.
[1] http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.8.9...
>Some of the actual context switch in native threads is performed by the CPU, where as with green threads it is not
Exactly. Native threads have a context switch, green threads don't.
Native threads are natively supported by the OS, so managing them requires calls to the OS, the main reason why it's slow. However, it comes with a few advantages. The main advantage of native threads is that they are preemptive - they can be scheduled based on ticks (i.e. number of microseconds spent running the thread), even if they are in the middle of a loop. Also, they can be run on different CPUs, the OS takes care of all CPU flags/registers/etc., they are actual OS structures, ...
http://msdn.microsoft.com/en-us/library/windows/desktop/dd62...
http://www.linuxplumbersconf.org/2013/ocw/proposals/1653 https://www.youtube.com/watch?v=KXuZi9aeGTw
I would love to read a longer article about this. M:N has quite an allure, but it never seems to reach mainstream. Erlang and Go provide it and they seem to be happy. Both languages do not focus on performance, though.
M:N is great for maximising throughput, at the expense of fair scheduling. If you want any kind of fair scheduling, make sure you 'yield' often. Hope you're using a language that helps you with this.
If you want to avoid deadlock, don't get stuck in any infinite loops and avoid OS thread synchronisation, e.g. in third-party libraries. Oh, and if you're running on a NUMA architecture, make sure you resume tasks on the same thread that suspended it, unless you like needlessly consuming your processor interconnect's bandwidth.
On the plus side, it's a lot easier to write concurrent algorithms with green threads than when you have to farm work out to thread pools. It's even easier if your language was designed for it and can help you avoid the common pitfalls above. But if you can use a thread pool easily, you probably shouldn't be looking at M:N threading.
It is really great to see Rust drops the mixture of green and native threads, and reduce the runtime to minimum. It will make Rust stand out in many environment that no other languages would work well (besides C).
Last I heard, more advanced support hinged on Emscripten upgrading their version of LLVM, which was much more ancient than the bleeding-edge version that Rust uses.
Isn't this fucking awesome?
Here's a dummy repo from steveklabnik to demonstrate the process: https://github.com/steveklabnik/rust_example (though the change in the OP is so now that I doubt this repo has yet been updated)
In fact, as we approach the alpha, I expect several frantic last-minute breaking changes as people attempt to beat the buzzer.
That allows them to tell people that, yes, they can depend on the fact that the language and API will not change from, say, 1.2 to 1.3. And that they will continue to support it for the foreseeable future. But I imagine that eventually they will want to make breaking changes, as they certainly won't get everything completely right with 1.0. But that will be clearly marked as an entirely new version.
More info:
Which lets us relive the fun times of eg. Dlang 2.0 release or Py3k
But in an attempt to answer your question, I see Rust development as an application of simulated annealing (https://en.wikipedia.org/wiki/Simulated_annealing). In all honesty, we could probably spend another two years toying around and seeking a "more optimal" solution... but I personally agree with the devs that we have reached a point where the value of seeking backwards-incompatible solutions no longer justifies the time spent unable to cultivate a growing stable of third-party libraries.
That said, Rust won't be standing still post-1.0. Not even close. There are lots of language improvements that will be arriving backwards-compatibly in the coming year (e.g. improvements to dynamically-sized types, improvements to closure type inference, various improvements to the trait system like negative bounds, and so on.). Longer-term, I expect Rust to look into a design for higher-kinded types, which could eventually allow for generic monads in the vein of Haskell (though this should be considered a very far-future goal, perhaps even worthy of a Rust 2.0).
Pushing it out before we think it is ready just because some people want to use it will result in stabilising a suboptimal language and so trade-off the long-term experience for the 5 years (or 5 decades?) for some short-term gratification.
Thanks to libthread being orthogonal to processes, it turned out to be easily portable.
[1] http://man.cat-v.org/plan_9/2/thread
[2] http://man.cat-v.org/plan_9/2/fork -- this is what linux' clone(3) syscall was modeled after.
Do you want GC or not? Do you want lightweight threads or not? What level of runtime requirements do you want?
Its finally converging on something sane, but its taken a while.
I don't want to make it seem like I'm coming down too hard on Rust however, and I'm excited to see how everything works out once version 1.0 is released.
Do you have any alternatives to implementing a strong type system in order to give certain well-defined safety guarantees for anyone using the language? Any other alternative theories or implementation strategies that would be better?
> I'm more partial to the Go style of language design, which focused on making it easy to do the things the designers did in practice.
The designers/implementers wrote the first non-bootstrapped compiler in OCaml, a functional language. How's that for "things the designers did in practice"?
As far as doing what they were doing in systems programming in practice: the point of the project is to actually get away from their then heavy use of unsafe languages like C++. But then they took the concepts of smart pointers, ownership etc. (presumably inspired by best practices in C/++) and tried to make it bulletproof to use.
The other two are really part of the same coin. Having green threads involves having a runtime that has to be started. Deciding to remove the runtime means removing the green threads.