Yehuda Katz and Steve Klabnik Are Joining the Rust Core Team
blog.rust-lang.org
blog.rust-lang.org
We've been using Rust in production for awhile now, specifically because of its combo of native speed and memory safety[1]. This bending of traditional tradeoffs has let us implement features that would have been otherwise impossible.
Interfacing with Ruby via its C APIs, we are able to do some pretty crazy stuff, like sample memory allocations in production with imperceptible overhead[2]. Most importantly, we can do it without fear of segfaulting our customers' servers.
It's really great to see people other than low-level bitbangers being added to the core team. In this case, having a production user and someone focused on new users demonstrates the Rust team's commitment to building a functionally diverse community.
Lastly, I think opening up the core team to a wide coalition of companies is the right way to build a robust, long-lived open source project. Yehuda talked a little bit about this in his Indie OSS talk[2]. There are several "pathogens" that can affect open source projects, and the more diverse the distribution of power, the more likely you are to be immune to these pathogens.
1: http://blog.skylight.io/bending-the-curve-writing-safe-fast-...
Could you talk more about what sort of features these are?
The closest I can get to a single "secret" is aggressively delegating. What I mean by this is delegating before you feel comfortable delegating. The only way I know of to get everything done that I want to get done is to set the vision for something, do some initial implementation work, and find people who share the vision to help as early as possible. The primary reason I end up working on so many things is that things are way more connected than you might expect, and the full picture of something like Ember involves both day-to-day work on Ember and advocacy around a whole host of related technologies. I'm very grateful for the number of groups that have welcomed my participation.
I got involved in Rust largely because it was the shortest path to a low-overhead, non-crashy agent for Skylight, even counting the work we had to do to keep up with ongoing development over the past year. When getting involved in a technology so early, I tend to get really invested, and try to find holes that I know how to fill and fill them. That's what happened with Cargo and a whole bunch of other areas in Rust.
Whether this perception is right or wrong, there are a number of prominent individuals within the open source community who are widely seen as rapidly jumping between trendy or hyped projects as they arise, to benefit from the exposure that this involvement can bring.
I'm not passing any judgment, mind you. I personally don't care which projects you choose to get involved with. However, I do know that other people have noticed certain trends around how certain open source developers move among projects, and this does negatively affect the impressions these people have of these open source developers and the projects they get involved with.
Likewise, it is illogical for faulting a developer for possessing that same drive when it comes to improving emerging technologies.
Squandering one's skills to appease some anti-progress group is silly.
Furthermore, if you continue to bring something to the bandwagon, people will continue asking you to join them.
It would probably be much easier for them to stick to one thing but that wouldn't fully utilize them as a resource.
In this case, I think you've got the causality mixed up. Projects often gain prominence thanks to the work Yehuda puts into them, as he's particularly good at articulating their strengths to a wider audience. It's not that he's bandwagon hopping—it's that he's helping to create the bandwagon. ;)
In the cases of TAG and TC39, I saw an opportunity to take low-hype, low-bandwagon organizations and revitalize their missions and purpose. In particular, bringing on web developers as active participants and the follow-on effects of having them join GitHub and modern practices (slowly), helped increase their profile and stature. In both cases, I hardly did most of the work, but I did spend a lot of time articulating a vision for these organizations as ones that could be far more effective by involving more practitioners. I think it has worked.
In the cases of Ember and Handlebars, I saw something missing in the ecosystem and built my own tools. In both cases, the tools were hardly instant-winners, and I had to spend a ton of time recruiting fellow-minded collaborators who shared a vision for the future. If I was in it for the bandwagon-hopping, I would have tried to join Mustache or Angular, and not spend years to build up my own, relatively small-in-comparison ecosystems.
My real MO is to try to envision a better future for something related to the web or my product, and then either find existing projects that already share a part of that vision or create them if they do not. My involvement in many different projects is because of the fact that big-picture ideas involve improvements to multiple technologies.
That just might be the best piece of advice I've heard in a long time.
—Linus Torvalds (2000-05-02)
It also helps with humility: if you're constantly working on things that are too hard for you to do, it's hard to build up an unhealthy belief in your own abilities. And humility will keep you open to unexpected learnings from collaborators and people working on related things.
Much of the software our company use has been shaped, at least in part, by Yehuda. I hope we'll grow the company, so that at some point we'll be able to reciprocate.
The rust community is lucky to have these two. I recently went through the rust guide and it's terrific! And as a happy rubygems user, I'm glad to see crates following a similar path.
> What exactly does this change entail, then?
The core team is listed at https://github.com/rust-lang/rust/wiki/Note-core-team . Basically, like most projects, it just adds weight to our opinions in decisions that are made, and that we're sure to be involved in making those decisions. In general, we try to get rough consensus for major decisions from the team + community.
Also, both Ember and Rust are critical for my business (Skylight), and keeping them both strong and effective are key to my business interests ;)
I'm looking forward to dumping Go in favor of Rust once it's competitive in terms of libraries/stdlib. I've been building some big stuff in Go recently (mostly around financial market simulation) and the language shortcomings have hurt a lot. I've been using Go since January 2013 as my primary language, but everything I've seen of and tried with Rust is amazing.
I'm not an expert in Go, so feel free to correct me there. Additionally, IIRC Rust wanted to take the same route like GO regarding interfaces and generics, but Niko made an argument that it is wrong turn to make.
> IIRC Rust wanted to take the same route like GO
> regarding interfaces and generics, but Niko made an
> argument that it is wrong turn to make.
This is a mistaken conception, as Rust has had something approximating generics since the very beginning (though they have changed drastically since the outset), and traits have always been explicit rather than implicit (and it was pcwalton who initially conceived traits, not nmatsakis (and traits as well have changed drastically since their inception (the bottom line is that pretty much everything about Rust has changed drastically since its inception))).TL;DR embedding small libraries with high-performance or low-memory requirements into programs written in high-level languages.
For example, you might implement a few parts of your rails app with rust to make them faster.
This means that, in theory, one could make a pretty good web framework in Rust that would be both quite fast and ergonomic. That said, writing those abstractions will take time, so we'll probably see lower-level HTTP libraries, followed by Express-style abstractions, followed further on by full-stack solutions like Rails.
The earlier parts of the stack are coming along quite well, so give it a few years and I think we'll have a pretty good story to tell. In the meantime, you can use Rust today very effectively as a language that you embed inside a high-level language, and I think a sizable chunk of the Rust community will use Rust for those kinds of embedding use-cases in perpetuity.
To be honest, if you think about your Rails controllers/models, 1. There isn't really all that much there(fat-models/controllers are pretty slim when you're language was built for web browsers/operating systems). 2. Think about your priorities. I can guarantee its not performance or safety. You want to build a content-rich/feature-full application. Basically, prioritize iteration speed, at all costs. No matter how good the Rust frameworks get, I don't think Rust will be the "iteration speed" language.
Meanwhile, I think, in the spirit of wycats' comment, image if ActiveRecord, Unicorn, Rack, the template renderers, etc, were all built with Rust, but still used through a Ruby interface. That would be quite the performance improvement for Rails and it would be interesting to compare its performance characteristics to a fully Rust web framework/application.
Related: Would Rust be a bad choice for backend web development (ie, an alternative for Ruby/Python) and why or why not?
A rust programmer must be aware of the complexity of memory management. Although they are not forced to use all of it it, the standard library does expose enough of it, to force every rust programmer to be comfortable with the complexity of it.
In return they get the opportunity to write code that can compete with c/c++ in terms of performance, and control over latency, while still retaining safety.
>Would Rust be a bad choice for backend web development (ie, an alternative for Ruby/Python) and why or why not?
Depends on how much you care about maturity and what timescale the code has to run in production.
Rust code will be much faster than equivalent Ruby/Python code, but the memory management of it, will be more complicated. I would argue Go offers a better trade-off in performance, complexity and maturity for those types of projects, even though Rust is a much more interesting language.
So, if you can afford to not micro optimize your allocation strategies and just use a garbage collector, i would always choose to do so. Even with a language design as pretty as that of Rust.
It confused me a bit initially.
http://benchmarksgame.alioth.debian.org/u64q/compare.php?lan...
Is that not representative, or is the idea that Rust speeds the learning curve with its similarity to Ruby?
(2) The Haskell code is nowhere near idiomatic.
It is still useful to know that Haskell can be quite fast if necessary, but it likely does not reflect what your actual experience using Haskell would be.
This isn't meant as a personal attack firstly. Secondly, I've heard this said more than a few times lately and am generally curious:
Do you know which Haskell examples are idiomatic, which aren't, and why?
I feel like in the case of some it's just parroting someone elses argument.
fannkuch-redux pidigits binary-trees
binary-trees uses artificial strictness for the sake of the benchmark.
Nine of the ten Haskell implementations are demonstrations of writing C in Haskell (without reaching C's performance).
They are just not bounds checked. That is not "writing C in Haskell".
>binary-trees uses artificial strictness for the sake of the benchmark
What on earth is "artificial strictness"? Forcing evaluation when you need something to be evaluated immediately is not artificial, not unidiomatic, and not "writing C in Haskell".
>Nine of the ten Haskell implementations are demonstrations of writing C in Haskell
And end up shorter than rust? The idea that writing lower level code is "writing C in haskell" is nonsense, the ability to write more verbose but faster code is not harmful, it is useful. But the fact that lower level haskell is still shorter than rust and yet you want to act like it is some sort of horrible thing makes it hard to believe that you are really concerned about the horrors of writing slightly more verbose code.
I'm not certain the point of writing unsafe code in Haskell when there are better tools for the job. Haskell is fast enough as is and using it to hammer screws just reeks of wrong tool for the job.
The point of it is to show the performance differences, but that doesn't exclude a program like --
http://benchmarksgame.alioth.debian.org/u64q/program.php?tes...
-- or like --
http://benchmarksgame.alioth.debian.org/u64q/program.php?tes...
etc
The reason for people to bother adding "idiomatic" versions, is that other people really do work through looking at the source code "on an individual basis, one at a time."
Please stop bringing up the shootout (esp when haskell is mentioned) and dragging everyone through this conversation again.
Benchmarks are great to debate but this one is causing more harm than good; let's talk about other benchmarks.
I didn't!
Your accusation is completely wrong!
(And it hasn't been called that for over 7 years.)
That's the point I'm making. People constantly dismiss the results they don't like as "unidiomatic" and embrace they ones they do as "idiomatic".
>is that other people really do work through looking at the source code "on an individual basis, one at a time."
Except that people don't bother adding them, so your reasoning for why isn't relevant. If you want to claim the shootout is useful for this, you need to allow for grouping benchmarks together to be presented against the "standard" ones. Let Bob and Sally submit "Bob and Sally version of Foo benchmarks" and let me compare them against Foo and against other languages.
I already provided links to two programs and you're still saying that people don't bother adding them.
Pidigits:
$ patscc -I/home/cody/sources/ATS-Postiats-contrib/contrib -pipe -O3 -fomit-frame-pointer -march=native pidigits.dats -o bin/pidigits.ats_run -lgmp
$ gcc -pipe -Wall -O3 -fomit-frame-pointer -march=native pidigits.c -o bin/pidigits.gcc_run -lgmp
$ time ./bin/pidigits.gcc_run 10000 > /dev/null
real 0m0.969s
user 0m0.963s
sys 0m0.004s
$ time ./bin/pidigits.ats_run 10000 > /dev/null
real 0m0.972s
user 0m0.968s
sys 0m0.004s
$ ls -larth bin/pidigits.*
-rwxrwxr-x 1 cody cody 15K Dec 13 18:59 bin/pidigits.ats_run
-rwxrwxr-x 1 cody cody 14K Dec 13 18:59 bin/pidigits.gcc_run
k-nucleotide: $ g++ -c -pipe -O3 -fomit-frame-pointer -march=native -std=c++0x k-nucleotide_gpp3.c++
$ $(PATSCC) -DATS_MEMALLOC_LIBC -pipe -O3 -fomit-frame-pointer -march=native -std=c99 k-nucleotide.dats
$ patscc -DATS_MEMALLOC_LIBC -pipe -O3 -fomit-frame-pointer -march=native -std=c99 k-nucleotide.dats
$ time ./k-nucleotide_gpp3 < ~/Downloads/knucleotide-input.txt
real 0m0.177s
user 0m0.300s
sys 0m0.055s
$ time ./k-nucleotide < ~/Downloads/knucleotide-input.txt
real 0m0.056s
user 0m0.036s
sys 0m0.020s
$ ls -larth k-nucleotide k-nucleotide_gpp3
-rwxrwxr-x 1 cody cody 38K Dec 13 19:07 k-nucleotide
-rwxrwxr-x 1 cody cody 96K Dec 13 19:07 k-nucleotide_gpp3
[0]: https://github.com/githwxi/ATS-Postiats-contrib/tree/master/...[1]: https://alioth.debian.org/forum/forum.php?thread_id=14942&fo...
[2]: https://groups.google.com/forum/#!topic/ats-lang-users/QdwKp...
Using strict evaluation when appropriate is most certainly idiomatic Haskell code.
> Nine of the ten Haskell implementations are demonstrations of writing C in Haskell (without reaching C's performance).
This (C in Haskell accusation) is debatable, though I'm not sure there's much to be gained even given a totally successful discussion where we both communicate our points 100% effectively.
I posit that using Haskell in such a way is a disservice to the language. If you want to code in such a way, why use Haskell?
Haskell is plenty fast as is and doesn't need to resort to unsafe code. What's the point?
Furthermore, even if you write nearly everything in the IO monad Haskell you'll gain a lot from the type system and from easily peeling out small, meaningful pure segments.
No, the point was that it is "far from idiomatic". Did you read the post I was replying to?
In "Real World Haskell" they advise you to forgo of arrays, and even has a chapter "Life without arrays", saying "arrays and hash tables are often used as collections indexed by key, and in Haskell we use trees for that purpose".
How is it not? That is a complete non-sequitur and makes no sense. What sort of definition of functional programming are you using that you think it restricts what data types you are allowed to use? Do you think you aren't allowed to use lists in C?
>In "Real World Haskell" they advise you to forgo of arrays
You might want to read it instead of misrepresenting something out of context. You might also wish to consider that many of the haskell shootout submissions came from the authors of RWH.
The normal definition of functional programming, in which not all data structures used in imperative programming are idiomatic (or even considered "functional").
For starters, mutable data structures would be considered not purely functional. E.g:
"Most books on data structures assume an imperative language such as C or C++. However, data structures for these languages do not always translate well to functional languages such as Standard ML, Haskell, or Scheme. This book describes data structures from the point of view of functional languages, with examples, and presents design techniques that allow programmers to develop their own functional data structures" [1]
[1] http://www.cambridge.org/us/academic/subjects/computer-scien...
I can't find any reference anywhere to any sort of definition like that. Or even one that mentions restricting data structures or memory layout at all. The only "normal" definitions I can find are: "functions are first class" which is a pretty weak definition and doesn't really exclude much these days, and: "the return values of functions depend only on the functions arguments" which is what everyone in the haskell world thinks of as "functional programming". Neither of these definitions exclude arrays.
>For starters, mutable data structures would be considered not purely functional
A data structure can not be functional or imperative. Only what you do with it. Mutation is perfectly fine if it is localized (remember the function's return value needs to only depend on its arguments, nothing more).
It usually is. The reality is that like most memes like this, there's a grain of truth at the center, but most of the people repeating it don't know where the truth ends and the exaggeration begins.
Regex-dna is using low level ByteString operations to squeeze out a bit of extra performance. This makes it quite a bit longer and would rarely be done in practice since the gain is small.
Fasta again does some low level ByteString operations, but not nearly as significant as regex-dna. Not that bad.
Others like fannkuch-redux are just using "unsafe" vector functions. All that is doing is leaving out bounds checks. That makes an miniscule difference in performance most of the time, but the way the shootout works there's only incentive to completely optimize for speed, so you get 1% optimizations you wouldn't normally see. I'd hardly call it unidiomatic though, you just normally use "read" instead of "unsafeRead", etc. In some cases I would say the unsafe calls are idiomatic. If you are going to be doing an operation repeatedly in a loop, doing the bounds checking once before looping and using the non-bounds checked calls in the loop is pretty standard.
Pidigits is normal haskell (although you could argue the lack of spacing isn't idiomatic, but that hardly matters).
I stopped there since the rest are getting enough slower than rust, but looking them over there's still nothing outrageous, just more low level ByteString usage. Overall none of them are that far from idiomatic, none of them do anything particularly crazy, and I think the speed gains from doing most of the optimizations they are doing are fairly small. I find it particularly odd that people act like the ability to write lower level code that is faster is a problem with haskell. Especially given that the "low level" haskell versions are often still shorter and no harder to read than the "idiomatic" rust versions.
I love Haskell, but I still don't think it occupies the same space as Rust.
My opinion is that Haskell can overlap, but something that occupies the same space as rust would be Ivory[0] or ATS[1].
0: http://hackage.haskell.org/package/ivory 1: http://www.ats-lang.org/
It is in virtually every language on the shootout. That's one of the reasons the shootout is so terrible. A regex benchmark doesn't tell you much of anything unless the task itself is to write the regex engine.
>I love Haskell, but I still don't think it occupies the same space as Rust.
Neither do I. I am simply correcting this weird meme that "haskell is really slow and the shootout shows you have to write insane code to be fast" when the whole point of the shootout is to write the fastest code you can, the haskell code is pretty ordinary and still significantly better than most other languages, and an "idiomatic" version is not 10 times slower, it is 10% slower.
And yet, the programs don't all perform the same.
One regex task doesn't tell you much of anything because so much can be different with a different task. Then again, people are usually surprised by V8 and Irregexp.
The GHC version has been updated so many times since the last programs were contributed, that I hope the code would look better if it was written to use the latest greatest Haskell.
Because they don't all use pcre. Duh?
>The GHC version has been updated so many times since the last programs were contributed, that I hope the code would look better if it was written to use the latest greatest Haskell.
Why on earth would the code change because of a new compiler release? Do people rewrite the C one every time a new GCC is released? And the code looks fine, what "better" do you want?
"Duh?" because you haven't looked at the code?
http://benchmarksgame.alioth.debian.org/u64q/program.php?tes...
http://benchmarksgame.alioth.debian.org/u64q/program.php?tes...
>>Why on earth would the code change because of a new compiler release?<<
New GHC releases (and libraries) have in the past provided new ways to write the code.
>>what "better" do you want?<<
Any "better" that 6 or 7 releases has provided.
Me: "Not all use pcre" You: "Two that use pcre"
What on earth is that supposed to show?
>Any "better" that 6 or 7 releases has provided.
You have some weird ideas about haskell/ghc.
Those two that both use pcre don't perform the same.
>>You have some weird ideas about haskell/ghc.<<
If I have, I got them from seeing programs being re-written when new stuff became available.
Isn't "idiomatic" defined by what you "normally" use?
I mean: it's not like the specific problem domain needed a "less commonly used function" (e.g. you need a function to do X, where X is something that you rarely need to perform).
Rather it's: "we need to code A, but we will use less commonly used functions instead of what we'd normally use for A, just to get more performance".
Plus, it's not like you merely exchange function f with g, while all other factors stay the same. The choice of those "less commonly used functions" also affects other aspects of the program's design (e.g. making it into a more imperative style, going for unsafe, opting for mutability, etc).
So the interesting question to call it idiomatic or not, for me, is:
"If ultimate performance wasn't a factor, would a Haskell programmer write this program in the same way?"
Yes, it is exactly like that. That was the point. It is literally changing out the name of a function. It has absolutely no effect on the rest of the code. Which is why the code is idiomatic by any reasonable definition.
Why wouldn't he?
With a little familiarity with programming at large and Haskell syntax can learn to discern the two cases even if you're not a Haskell programmer.