Why I think Rust is the "language of the future" for systems programming | Hacker News Reader

Why I think Rust is the "language of the future" for systems programming

winningraceconditions.blogspot.com

243 pointspcwalton181 comments

Out of mild concern over the title (not that it ought to be changed, TFA doesn't really have a meaningful title), I'd like to preemptively defuse any potential flame war.

Go and Rust are not really competing. There may be some overlap in domain, but they occupy different niches and will likely appeal to different crowds. Go will appeal more to people who prefer its focus on conceptual simplicity and its "opinionated" nature (very much like Python). Rust will appeal more to people who prefer "functional" trimmings (algebraic datatypes, pattern matching, liberal use of higher-order functions, Scheme-style macros) as well as folks fed up with C++ who aren't willing to give up complete control over memory and performance characteristics. The fact that both languages are built around a similar concurrency model is just convergent evolution.

It's tempting to cast Go vs. Rust as Google vs. Mozilla and/or Chrome vs. Firefox, but there's no practical reason that both languages cannot peacefully coexist and thrive.

Does Go not have higher order functions and pattern matching? If not, are they on the roadmap?

I'm sure that Go has higher-order functions, I just mean that Rust encourages you to use higher-order functions all the time. For example: in Rust, `for` loops desugar to higher-order functions (it's very neat, actually). Furthermore, whereas in Go you spawn a goroutine with the `go` keyword, spawning a task in Rust is done by handing a function (usually a Ruby-esque closure) to the `task::spawn` library function. (Note also that "higher-order functions" are not the same as "first-class functions".)

Go does not have pattern matching, to my knowledge, though without algebraic datatypes I don't think it's really a big deal.

Go does not have pattern matching. There are type assertions, but obviously that's not the same thing.

Go has higher order functions in some sense, and the standard libraries use them for some things. But I wouldn't say they're nearly as ubiquitous as they are in Ruby or your average functional language. My read is that Go is pretty firmly grounded in imperative-land.

Regardless, since Go 1 came out relatively recently, I doubt they're going to make substantial changes to the language anytime soon. Obvs I don't speak for them, etc etc.

Go has full closures and first-class functions, and their use is as idiomatic in Go as it is in Javascript; for instance, they're the mechanism by which you implement filtering in the "http" package.

Go has first class functions yes, but without the support for generics/parametric polymorphism, you can't write "higher order functions" for polymorphic functional constructs like map/filter/reduce etc.

The two languages aren't really in the same space in the first place. Go is a simpler language that leans more heavily on garbage collection. Rust is a more complex language that can be safely used without the GC at all.

Go is a great language -- I greatly admire its simplicity -- and for its domain it's fantastic. Rust is in a different domain: low-level systems programming in which abstractions must be zero-cost and control over the machine is crucial.

Go actually provides what I believe to be zero cost abstractions that map straight to the c memory model, making it very easy to use for systems programming.

If you want a fixed size array of bytes, for for it. Zero overhead. If you want a slightly smarter list you can use a go slice, which is also pretty low overhead. I've personally verified that if I create a struct in go it takes up the exact same amount of memory as the same c structure.

As a concrete example, I recently built an interface to kernel crypto apis using pure go (no cgo). There's no magic to go data structures, you can define structs in go and pass pointers directly to syscalls (through the syscall package) and everything just works. Dealing with large arrays of binary data is similarly straightforward. So go does give you complete control over memory layout if you choose to use it.

The elephant in the room is that go does have garbage collection, and that ain't free. Practically you can minimize it's impact by managing your own memory. In fact that's what Brad Fitzpatrick is doing with his go based memcached implementation.

It all boils down to how you define systems programming. I guess if you mean is go suitable to write a kernel in, the answer is probably no, (but it would sure be fun to try). If systems programming requires having the complete ability to interact with the operating system and all syscalls + tight control over memory layout, then maybe go is the way to go.

https://github.com/shanemhansen/gocryptodev/

[edit]

with respect to python and garbage collection, did you know you can actually turn it off? If you have no cycles in your program you can turn off gc and let the ref counter clean up all your data, similar to Objective-C's ARC.

http://docs.python.org/library/gc.html

FWIW, Rust's arrays and structs are also laid out in memory exactly as C's are.

There is a lot more to zero-cost abstraction than memory layout. If you do manual memory management in Go, then you get no safety. It is also difficult to use the standard library with manual memory management. Furthermore, interfaces in Go are not zero-cost; you incur virtual dispatch whenever you use them.

The article is about doing safe manual memory management. That's something that's unique to Rust.

> The elephant in the room is that go does have garbage collection, and that ain't free.

Since HN is Lisp country, it has to be said that having GC isn't an "elephant in the room" - though a slow GC implementation may be.

> If you want a fixed size array of bytes, for for it. Zero overhead.

There's the overhead to check the bounds on operations in that array, which you don't have in C.

Simplicity is absolutely the most crucial aspect of a language used for successful commercial application in my space (15-30 mil annual). That's what I like about Go. Don't know much about Rust, but if it's being designed by a small group of engineers who think oop is still awesome, then successful commercial software developed in Rust will be just as costly to maintain as the 500k to 5 mil line codebases I run across today, written in currently available languages.

Speaking in general:

You get all this wonderful language functionality, and I promise you that you will write some shitty code at some point just to keep up with the daily pressures presented by a successful software package (clients screaming, salespeople breathing down your neck, etc), I don't care how good of an engineer you think you are. What's really funny is that, because everything has gotten so much more complicated (dev tools, business, cost of bugs going up, how to make money with software, whatever), the successful software I work on today is not making any more money than the successful software I worked on back in the late 80's running under DOS. Back then, the oop free tools and the target os were simple enough that we really could become SME's on our domains as well as on the dev tools we used to write code for those domains.

C# and java are definitely not the solution for new applications which are expected to generate revenue for the next 10 years. If Rust is just trying to be smarter at what these languages already do, I'm not interested. Something has got to come along which simplifies development, especially concurrency, and allows programmers of varying skill levels to write productive code in. Then after 5 mil loc, programmers coming and going on a project, an app doesn't end up looking like a fragile mess.

Rust is not particularly object-oriented; idiomatic Rust code leans more toward the functional style as opposed to the OO style.

The language is designed to make it harder to write buggy code at the cost of some amount of language complexity. Whether this is an acceptable tradeoff will depend on your particular situation.

>C# and java are definitely not the solution for new applications which are expected to generate revenue for the next 10 years.

That doesn't even make sense. Both C# and Java (and JVM, CLR languages) will be used in new billion generating applications in the next 10 years and more.

I should have said "generate competitive revenue." Sorry about that.

If you're still using C# or java in 10 years time for a commercial app, the only reason you will not be at a competitive disadvantage is because all the other competitors in your space are using the same languages.

> programming in which abstractions must be zero-cost and control over the machine is crucial

How well does Rust do in this zero (or low) cost department? In C++ STL there's the same goal, but it compiles to code very far from what you would write in assembly without abstractions.

Eg. how are code compactness and data layout optimizations?

Not sure what you mean here. Rust tends to be pretty dependent on inlining optimizations, just as C++ is. As for code compactness, the way we do task failure leads to significant code bloat at the moment (though that's on track to be fixed, and the code is off to the side and not executed unless you actually fail). As for memory layout, the rules follow C; struct fields are laid out in the order you specify them in the code, and so on.

For what it's worth, I like Rust because I like Python. Python does strive to have one "obvious way to do it", but that ends up meaning it has a very colorful toolbox full of different ways to solve different problems, much like Rust. Generators, context managers, metaclasses, decorators, and descriptors are all very different mechanisms, but they all work together well.

Hell, I keep discovering that Rust has already implemented language features I'd independently thought up half-baked versions of.

And not to encourage a flamewar, but I believe there's a gigantic unspoken niche that Rust and Go will be fighting over: people like me who stay away from systems programming because it's a pain in the ass. C is tedious, C++ is bozotic, D is obscure—so the lowest I've bothered to go for a while is Cython. But now there's Rust, and I'm genuinely enjoying it, and I see other people who've mainly been sticking to Ruby or Python who are enjoying Go. This will be interesting to watch. And regardless of what I think of Go, I'm glad there's finally some activity in this space again. :)

I have not had time yet to look into rust more closely, but do you know if it would be able to export a Rust module in a dlopen-able library, accessible from C ? Writing python extensions in Rust instead of C would be pretty exciting

Ha, I was just thinking about how Rust bindings to Python might look.

I haven't actually tried this, but since Rust libraries just compile to .so's and it's possible to declare a Rust function with a C signature, I bet it'd be a simple exercise to write a CPython-compatible extension module.

"I haven't actually tried this"

Please remedy this immediately. I'll even provide a highly-imaginative name: "Prusty".

Apparently it won't work yet! Tragic. In the meantime, I can deliberate on whether or not to call it FOXDIE.

Surely "Pyrite"?

Awesome :)

For those with even less of a chemistry background than I have: rust is iron oxide, and pyrite is iron sulfide. It's a good analogy because oxygen and sulfur are in the same group (column) in the periodic table, so they have some chemical similarities.

There was a music software language called Pyrite. It ran inside of Max graphic programming environment. It then evolved into SuperCollider which is now one of the most used music programming languages in experimental and academic music. Http://github.com/supercollider

But that pyrite hasn't been active for a long time.

Awesome.

Prusty comes closes to "Prusti" which is the bulgarian word for "fingers", and "prust" meaning "finger", but also "soil"

This is not quite possible yet, but will be at some point.

Rust does currently depend on a language runtime which expects to control the execution of all Rust code (in particular managing the task scheduler), and the runtime does not have an embedding story yet. Even with an embeddable runtime though, the process would be more involved than loading a library through `dlopen` and executing a function.

As part of the effort to rewrite the remaining bits of C++ runtime code in Rust (almost all of Rust is written in Rust), there are further plans to make Rust code runnable without an underlying runtime and without split stacks. After that it will be feasible to write code in Rust and just call it like any C function.

Rust has three different ways of allocating memory. And you can't pass things allocated one way to functions that expect something allocated the other way.

A Python replacement, this ain't. More like C++ on quaaludes.

No, Rust has two ways of allocating memory (on the task heap or on the exchange heap), and most functions take borrowed pointers, which accept both kinds of memory.

(Edit, re below reply: It depends how you define "allocation"; it's either two or three. I wasn't considering the stack as allocation, but you're right that Rust takes the traditional stack/heap distinction and expands it to stack/exchange heap/task heap. We should update the tutorial to make it clear that borrowed pointers work for the stack as well as both heaps.)

I agree that the language is not designed to be a Python replacement, however. No language is suitable for every task.

from http://dl.rust-lang.org/doc/tutorial.html

"7.3 What to be aware of

Rust has three "realms" in which objects can be allocated: the stack, the local heap, and the exchange heap. These realms have corresponding pointer types: the borrowed pointer (&T), the shared box (@T), and the unique box (~T). These three sigils will appear repeatedly as we explore the language. Learning the appropriate role of each is key to using Rust effectively."

It's 3 different types, not 2.

Rust statically prevents leaks and double-frees, warns about implicit copies of large structures, and has GC built in if I want to use it. It's certainly more familiar to a Python dev than C++. The whole trait system is even like a built-in and vastly simpler zope.interface.

Would be interesting to read a comparison with D. And no, "obscure" is not a meaningful description when comparing it to a language still at v0.3.

I have used D but no Rust yet. They look superficially similar.

On the Rust side, algebraic data types, an AST macro system, saner tuples, non-nullable and immutability by default. I'm a bit skeptic about the different pointer types and how they will interact with meta-programming.

On the D side, many good things like unittest(), best-in-class overloading support, compile-time everything, etc.

http://laser.inf.ethz.ch/2012/slides/Alexandrescu/2-D%20cour...

That's fair. I'm conjecturing somewhere into the future: D never really took off, but I'm hoping Rust will.

Of course they compete. People aren't born with a gene that puts them into exactly one of the camps you describe. These ideas compete and so do the languages, provided they can be used for largely the same tasks, which seems to be the case.

I'm a die-hard C guy. My motto for years has been "you can pry pointers and address spaces from my cold, dead hands."

Of the new languages I've seen lately, Rust is my favorite. I love how it gives me better ways to express things I actually want to say without imposing GC on me.

But even so, I can't see myself actually using it for much, because writing in a language other than C means buying in to that language's runtime. Buying into one language's runtime means that your code won't play nice with other languages' runtimes.

If I write a library in Rust, how can I expose my types and algorithms to Ruby, Python, Lua, etc? How will Rust Tasks play with Python threads? What if I use a Rust Pipe to send a Python value between tasks? How do I keep Rust from doing a GC pass while I'm holding the Python GIL? etc. etc.

Programming Languages by their nature want to be at the center of your world. If you buy into their abstractions, everything works nicely. But if you try to mash two of them together in a single process, you start to suffer from the fact that their abstractions overlap and don't interoperate at all.

If you're only writing an application (ie. not a library) and never want to embed other languages into your application, then this might be ok. But I'm more interested in writing shared functionality that is useful across languages. Why should the whole stack of parsers, crypto, compression, etc. have to be written separately in each language? Life is too short to do some great work that is only usable by one language community -- computing is so big and changes so much that one-language-only functionality is at best limiting your market and at worst dooming your code to obsolescence when the next big language comes around.

So as much as I instinctively like Rust, I think I'll be sticking with C.

I'm not sure if this will alleviate all of your issues, but eventually the goal is to allow Rust programs to forgo the runtime entirely.

An old comment from pcwalton (sorry, I've lost the link):

"I'd like to see a 'runtime-less Rust' myself, because it'd be great if we could implement the Rust runtime in Rust. This might be useful for other things too, such as drivers or libraries to be embedded into other software. (The latter is obviously of interest to us at Mozilla.) Rust programs compiled in this mode would disable the task system, would be vulnerable to stack overflow (although we might be able to mitigate that with guard pages), and would require extra work to avoid leaks, but would be able to run without a runtime.

"If anyone is interested in this project, I'd be happy to talk more about it -- we have a ton of stuff on our plate at the moment, so we aren't working on it right now, but I'd be thrilled if anyone was interested and could help."

See also the remark by brson further down in this thread: http://news.ycombinator.com/item?id=4578044

"If write a library in Rust, how can I expose my types and algorithms to Ruby, Python, Lua, etc?"

The same way you expose them in C. Rust and C are compatible at the binary level.

"How will Rust Tasks play with Python threads?"

More or less the same way C setcontext()/swapcontext() workalikes play with Python threads. We probably want a runtime-less Rust to allow users who aren't using tasks at all to just omit the whole system, though. The language itself knows nothing about tasks; they're purely part of the runtime library.

"What if I use a Rust Pipe to send a Python value between tasks?"

Should work as you expect. In Servo we're already sending Objective-C values (which require special APIs to perform the memory management) from task to task over pipes.

"How do I keep Rust from doing a GC pass while I'm holding the Python GIL?"

By not using the GC. It's easy to tell when you aren't using the GC; you can just avoid @ types, and there is a warning you can turn on to enforce no GC use. This sort of thing is the reason why we support manual memory management.

"If you're only writing an application (ie. not a library) and never want to embed other languages into your application, then this might be ok. But I'm more interested in writing shared functionality that is useful across languages."

We're already successfully embedding a JavaScript engine into a pure Rust program for Servo. JavaScript can access Rust types and vice versa.

"Why should the whole stack of parsers, crypto, compression, etc. have to be written separately in each language?"

I agree completely that this is undesirable. That's why Rust doesn't do this. For your three examples, Rust and Servo link to C libraries: Hubbub for HTML parsing, NSS for crypto, and zlib for compression.

Thanks a lot for the info!

> We probably want a runtime-less Rust to allow users who aren't using tasks at all to just omit the whole system, though.

I'd be really interested in reading more about this if/when it is available. I'm also interested in whether you'd ever ship the tasks library standalone, such that you can interoperate with Rust tasks from C (obviously you'd have to follow certain rules to maintain the integrity/safety of the runtime).

> I agree completely that this is undesirable. That's why Rust doesn't do this. For your three examples, Rust and Servo link to C libraries: Hubbub for HTML parsing, NSS for crypto, and zlib for compression.

Understood, but I'm interested in the story for the guy who is writing those libraries.

You'd want a runtime-less Rust if/when it becomes available (there are no immediate plans for this, but we'd be happy to help). We use native OS dynamic libraries, so once it's possible to build Rust code without a runtime it should be pretty easy to call Rust from a pure C host app.

I just wanted to say, I totally understand where you're coming from here. I often end up writing libraries in C explicitly because that's the easiest way to hit as many targets as possible and not impose any unexpected runtimes on other languages, even if I know it would be easier to write in another language. So, I would say that not enough language designers think this way, and it's really refreshing to see you describing this point of view.

I've often thought that a useful exercise would be to come up with a language that adds just a bit more power than C but stays within the boundaries of the C runtime, just for this purpose.

However, if new languages like Rust can deliver this, all the better! One language that I once looked at with this in mind was Clay[1], which seemed to basically be C with more safety and generics. However, it doesn't seem to be as nicely supported as Rust, and perhaps Rust's static analysis and concurrency support will be more powerful.

[1] http://claylabs.com/clay/

> The language itself knows nothing about tasks; they're purely part of the runtime library.

How are they implemented? Do you provide access to low level concurrency constructs such as CAS, threads etc.?

Perhaps the language Clay (http://claylabs.com/clay/) might fit your tastes better?

I'm a C die-hard as well.

Tell you what I'm doing lately, that makes me less and less interested in the new-school elite, and just generally enjoying a language 'enigma' moment: putting Lua in everything.

That is, I mean to say, Rust is neat and all but we can do it all in Lua, pretty well, too. Table types are fabulous, kids!

And being able to take any collection of C libraries, and glue them together into 'ones own runtime' with a common language framework/boundary, means a lot.

So, yes, of course.. Lua isn't fast, it has weaknesses, there is luaJit and so on: its not about Lua. Its about the fact that you can choose whatever languages are out there, get competent at their adept usage, and win.

I believe that Rust will be a systems programming language when there is a bootable Rust OS that boots to an editor, a shell, a means of interacting with devices, and a Rust compiler sitting there, waiting for input .. until then, it sure is fun to watch the new school language paradigms get all twisty and bent in the rush to school someone on how systems 'should be programmed, instead of the way they are now'. Research in this field is a terrifically interesting social cache.

Has anyone done a RustOS? Is there one? I know of a couple LuaOS projects, and it sure seems to me like what was done with Linux on Android could be done to Linux, again, with Lua .. Rust .. and so on. Could be very cute times ahead, in the distribution mecca .. GoboLinux, but instead 100% booted to Lua? Hmm .. bogglethink .

Is this intentionally snide? You appear to be implying that nobody should try to solve problems, because we already know how to work around them.

No, really, I'm not being snide. I honestly believe that Rust is of great interest, as is the effort and front of new language exploration. However, there is an intrinsic quality of language: it is infinitely useful, or not useful, depending on one thing only: the user.

The whole idea of modern systems languages like Rust or D is actually to avoid the need of using Python, Ruby etc. The language covers all your needs in a nice coherent package from top to bottom.

His use case is writing libraries that can be re-used broadly.

> The language covers all your needs

This is exactly the "one language" myopia that I was arguing against. Anyone who tries to sell me on a language or programming system that asks for complete buy-in and offers no interoperability story is immediately suspect in my book.

(FWIW, I don't think the Rust guys actually sell this line, so I'm arguing against you, not them).

No one will ever invent "the last language you'll ever need." One reason for this is that different languages excel at different things. For example, Rust is not as easy to sandbox as Python, Ruby, etc. It's also not as simple to write quick scripts in.

Another reason is that you'll never convince everyone in the world to use the same language. Even if you convinced them that they should use the same language (which is impossible), there are billions of lines of code in use that is written in different languages. If you can't interoperate with them, you are a rock, you are an island.

There's nothing "modern" about the idea that "we're going to make the last language you'll ever need."

Grandparent has a point, and it's not quite as outlandish as you make it out to be.

Consider that a lot of Python developers write most of their code in Python, then rewrite critical parts in C to speed it up. Python is not a speed demon, and that becomes particularly problematic with tight loops or heavy math—but writing entire applications in C is error-prone and tedious.

Rust won't outright replace both of those, but it does offer a nice middle ground: it's safe, it has some higher-level constructs, it does away with some boilerplate, it builds fairly easily, and it still compiles essentially to machine code. Writing quick shell-like scripts or entire applications in a systems language is now a viable option for lazy jerks like me, even if it's not always the best one.

> Writing quick shell-like scripts or entire applications in a systems language is now a viable option for lazy jerks like me, even if it's not always the best one.

I agree completely. I love Rust for this reason too. That's not what the grandparent said, or what I'm arguing against.

Grandparent was dismissing my concern about language interoperability because new languages should cover "all your needs." This attitude is closed-minded and dangerous.

A language that covers lots of needs is great. A language that is so useful that you find yourself needing fewer total languages for your project is great.

But the moment you use that as a reason to stop caring about interoperability between languages, to argue that it doesn't matter whether your language can interface with others, that's the moment when I stop paying attention to what you are designing.

Rust is what I had hoped Go would be.

Google employs some of the brightest computer science minds in the world and turns out stuff like Go and Dart, which seem to be more aimed at enterprise Java programmers rather than computer scientists or programming enthusiasts.

Maybe because that's what Google needs its own programmers to learn right now. Although, they should make a language that's meant to eventually replace Java/Dalvik for Android as well, and a language that's very easy to use and learn.

Yeah, absolutely. Go and Dart really both fit needs of working with large software and giant scales. As I think they've both said independently, conservative design was considered a design bonus. You can see why as it gets teams up-to-speed quickly, is less off-putting to project managers, and is more likely to benefit from experience surrounding how to write effective, readable code forms.

Stuff like Rust is awesome, but no-one yet knows what an Effective Rust book might look like, and you really need that sort of solidified design understanding to work. Code is write once, read hundreds of times, so that exciting wild west of "we don't have idioms yet" simply won't fly at a big company.

One could automatically translate the Java API part of Android into Go? Also, it should be possible to write a VM for Go by extending Dalvik? I suspect that this territory is claimed by Dart, however.

Neither Mozilla nor Google are particularly keen to faff around designing languages for the hell of it. :)

Google needed to ease the burden of hours-long Java/C++ compile times on massive projects. Hence Go.

Mozilla needed a language that was as fast as C++, but safer and trivially parallelizable. Hence Rust.

Beyond these goals, the fact that the rest of the world is excited for these languages is just gravy.

Well, that makes Mozilla's language goals a lot sexier to a lot of us i believe =P

Seems weird. Google especially is a company that relies so much on programmer productivity & getting things right. Getting them wrong typically means lost revenue.

I mean they spend crazy amounts of money on all kinds of projects that have no directly viable monetization scheme, yet for some reason you think it would be weird for them to explore programming languages? Odd.

You attribute too much to these entities called "Google" and "Mozilla". They're made of people.

No executive asks for a new programming language. However at some enlightened organizations, they are willing to let hackers explore radical approaches.

At some point, in order for them to become "official" projects, the hackers have to align the language with the organization's goals. But the imprimatur of their creators is unmistakable.

In any case, I doubt there is any plan to move everything in Google to Go. They're willing to let the creators and some enthusiasts play around with it, maybe deploy a few apps internally. But for Rust to be a success, Mozilla's going to have to use it extensively in their main product.

"No executive asks for a new programming language."

Not that I think you're wrong, but I think the histories of Javascript and Ada stand as exceptions. :)

Hm, you're right. There's also the scripting languages in Office, which were definitely management-driven. They share something with JavaScript - a strategy to differentiate an already dominant platform from competitors.

That said, managers usually have little say in defining what the actual language is going to be like or what its other applications will be, because so much has to be deferred to the language designers. Even though Netscape's execs specifically requested a Java-like language, they got something rather different.

Visual basic. C#. Powershell. F#. Javascript. Erlang. Fortran. Objective-C.

All languages developed intentionally by corporations.

Additionally, there have been languages developed by committees, such as ALGOL, COBOL, CPL, Haskell, Ada, and many others.

Yeah, I admit my comment was not all that well thought out.

I was reacting to the usual dumbass theory that $BIG_ORGANIZATION is executing some master plan which explains all of their actions. It explains how they filter and rank ideas, but it doesn't explain why those particular ideas happened to be floating around in their organization.

When it comes to things like new frameworks and languages, in my experience, it's usually some employee going off on their own. And the reason why the language is the way it is has more to do with that person's obsessions and interests. They later justify it to the organization by producing results and getting peer buy-in. FLOW-MATIC (ancestor of COBOL), Perl, Java, Sawzall, Go, and Rust seem to be like this.

The other pattern is when companies intentionally make a language to compete, or lock-in a developer platform. Now this move really is predictable, and any MBA can draw you a 2x2 matrix showing why Microsoft had to create C#. Likewise VB, PowerShell, F#, and JavaScript. It doesn't mean they are bad languages. But they are often platform specific since their whole purpose is to corral developers into one camp.

I don't think Haskell counts at all. That was an academic project amalgamating other academic projects.

I don't know enough about the others you mention. Maybe this isn't a useful way to think about programming languages, but I was just exploring the idea by rambling about it. ;)

Objective-C was developed mostly by one guy (Brad Cox) back in the '80s. He was inspired by Smalltalk. He didn't do it on behalf of any corporation that I know of, though he and a partner did try to commercialize it later.

I understand C++ project can take a long time to compile. Does Java project really take hours to compile?

If any project takes hours to compile, it needs to seriously be refactored.

Rob Pike, on justifying the development of Go:

"today's server programs comprise tens of millions of lines of code, are worked on by hundreds or even thousands of programmers, and are updated literally every day. To make matters worse, build times, even on large compilation clusters, have stretched to many minutes, even hours."

http://splashcon.org/2012/program/404

99% sure that remark is in regard to C++. If it's about java, then it's just not true...

You've obviously never compiled java at google :-)

I think that's because they have a lots of Java code to compile. When there are a lot of GO codes to compile, it will take hours as well. I doubt GO's compile speed is significantly faster than Java. Both of them don't have the dependency problem plaguing C++. It would be interesting to compare the two.

I'm pretty sure that go's compile speed is significantly faster than java. It's hard to compare though because java compilation is not the same thing. There is the bytecode in the class files and on the fly compilation in the vm. As well as some stuff that happens when the bytecode is loaded and run.

A lot of go's compile speed comes from how simple it's syntax is besides the gains it gets from avoiding C++ dependency issues. Java's syntax while less complex than C++ is still complex compared to go's. And it won't have go's parse time savings.

Go's syntax may look simpler, but from a how many tokens do I need to look ahead parser perspective, I don't think it's much simpler than Java. Go does have a hand tuned parser written in C, vs the java parser written in java and perhaps not tuned for speed, but there's no reason a fast java parser cannot be written.

Java has jars and classes and dynamic calling. You don't compile everything into one big lump, nor you have to recompile millions of lines of code for one change in a file.

If Go wanted to compete on compile speed, then dynamic loading would be a better idea than having a simplistic compiler that compiles fast into one monolithic executable.

The main reason the Go compiler is fast, is because it is simple and doesn't do many passes and optimizations.

Which is like advertising the fact that your company makes a fast car, but forgetting to mention that it can only carry a driver if he weights 10kg.

Nothing you've said contradicts my statement concerning compiling java at Google.

Go doesn't need to rely on dynamic loading because compiling a library is fast and statically linking the libraries is fast. The result is a monolithic executable true but it's still fast without dynamic loading. Go doesn't compile all the source that goes into a binary every time. Libraries get compiled into an archive and don't have to be compiled again unless they change. In that way it has the same benefit as jars for compilation speed.

He never specifically said it's C++ nor Java. I think he's talking about C++. C++'s slow compile problem has to do with the repeated inclusion of header files for every source file, which can become a FileNum^2 recompilation problem. Though pre-compiled header files and refactoring into pre-built libraries should reduce the compile time. Java simply does not have the slow compile problem due to its simple package/class per dir/file, where a compiled class won't be recompiled again.

You know, I've worked on some pretty large C++ projects, and the only time I've seen an hour long compile time, it was due to some include hell that was pretty trivially optimized.

I'm assuming you'd rather see them focus on declarative programming, because it seems to be the big thing among enthusiasts and hobbyists.

Google works with software projects with planned lifetime of decades. It's far better to rely on imperative programming concepts which are tested and trusted with experience grown from what, 50's or so.

Sure, declarative programming is a nice toy, but that's all. I'm yet to see major companies investing millions, if not billions of capital on a system written in declarative languages. We all "know" functional programming "is the future", but yet nobody trusts their money and time on them.

I still wonder if there are major commercial projects started with say Python or Ruby, or JavaScript which have planned lifetimes to 2030 and beyond.

I'm yet to see major companies investing millions, if not billions of capital on a system written in declarative languages. We all "know" functional programming "is the future", but yet nobody trusts their money and time on them.

SQL is declarative.

Erlang is kinda-sorta functional, and was developed specifically to run expensive high-uptime telecom systems.

...both of which are domain specific languages, or developed as such.

I really think C++ is going to gnaw "market share" from C in any low-level domains and fight back Go and other competitors in high-level systems programming domains thanks to it's recent C++11 standard and upcoming standard library extensions which being the transition of making it much more on-par with other modern languages. Bjarne Stroustrup(the language creator) has also hinted for much faster standardization process than last time around(when it took some 13 years to come up with a new standard), and would hope for a 5 year cycle instead, so perhaps C++16 is coming up next. Oh, and the standards committee is already working on it.

The only major setback I see with C++11 in comparison to many other languages aimed for large-scale software development is it's very small standard library which completely lacks support for modern technologies such as hierarchical filesystems, sockets, XML and HTTP and so on. They are coming up later on, though. The core language is very solid and doesn't lack anything special. Yet, it's not any more "bloated" or complex by it's standard(in terms of page/line/word count) than say C# 3.0 or Java 7.

tl;dr: I think many people think C++ is "old" or even dead and we need something more modern, yet now C++ evolves faster than ever and tries to meet that demand.

Yes, truly the only thing holding C++ back has been lack of features. Have a link: http://yosefk.com/c++fqa/

Nice, but you're 4 years late, sir! Everything can be viewed in a bad light if it's wanted to and we both know it. So how about actually talking about the bigger picture, and problems with programming languages and the actual art of creating large-scale software rather than trying to be smart?

C++0x has some useful features, but it doesn't address Yossi's complaints about C++, which are at a more fundamental level.

Yossi, by his own words, is a whiner. The fundamental reason behind is attitude towards C++ is to stir up controversy, rather than to provide objective criticism. If his attitude was sincere, I'm sure he wouldn't have spent so much effort in mastering the language.

ad hominem.

...which is completely fine when is not used as an attempt to debunk the argument, but show that the information is biased.

Any form of critique towards a biased source can always be countered with claims of it being an ad hominem argument - which it strictly speaking usually is - but that alone does not make the critique false or the original source any more reliable.

As an example, many of the points Yossi makes are subjective and opinionated and as such FQA is nowhere near an unbiased and reliable source of problems within the language. Nobody who actually wants to be serious about the subject cites FQA as a whole. At best, individual points he makes with which many agree with. Including Bjarne himself.

Any form of critique towards a biased source can always be countered with claims of it being an ad hominem argument...

Well, any critique which consists solely of personal attacks can be countered that way. It's super effective!

There are plenty of companies who have spent millions on developing their websites, written in a variety of the languages you list as 'toy' languages, or even PHP. Whether they would have been better to use other languages is impossible to say of course, but these languages are being used for significant projects. The use of the pejorative 'toy' is not very useful, and likely to evoke emotional reactions.

Go was inspired by a talk on the features of C++11 [1] - in despair at the addition of yet more features; the origins of the language and the philosophy behind it might make interesting reading if you use C++, because it was done directly in reaction to the (as they saw it) baroque edifice of C++.

Though I agree that people should not judge languages by popularity, novelty, or coolness (though it seems many people do), that doesn't mean you shouldn't be open to looking at new languages, if only to inform how you use your favourite one. Go for example has been inspired by the languages you deride, or their precursors (Python, Ruby etc), and in fact has taken much from them in syntax and in the popularity of closures etc.

I do agree with your other point though that this is an excellent area for Google to focus on, and something which will benefit a lot of domains which currently don't use scripting languages (systems programming, many large server systems, desktop apps, mobile apps).

[1] http://commandcenter.blogspot.co.uk/2012/06/less-is-exponent...

I'm aware of what and why Go came to be, and I can easily relate to the reasons(I've read the blog post). I am by no means saying C or C++ as languages are perfect, nothing is. However, as I mentioned in my another reply to sanderdj, there's huge investment in effort for the abstractions the two languages build. It's very hard to tackle the problem of existing codebase for example, as that requires building stuff from ground up(that happens later on), or the fact that there are far more competent C or C++ programmers than there are Go or Rust programmers(and this isn't likely to change for the next 10-15 years or so, I'd bet), or the fact that there exists practically no libraries for these languages(ports are relatively fast though, few years max.), portability is a major concern too, tools too as well as language standard and educational resources(books, uni grade education, best practices over years, ...) won't exist for years...

So while I think that C and C++ will phase out eventually even in the domain of systems programming, I really don't see Go or Rust doing that for the foreseeable future, if ever. It would be really great to build trustworthy(non-leaky, efficient and secure) abstractions on top of languages like Rust and Go, but how reliable would it be? When would it be feasible? Who takes the first steps? When will the mainstream follow? Are these languages going to be relevant after 20 years?

Oh, and I'm not deriding other languages in their specific domains(I'm a Ruby fan, not so much into Python), but rather the fact that I've been under the impression that many people think that soon C and C++ will phase out and become obsolete(or that they have already done so), we go higher in abstractions, Python and Haskell are the future and computers are so fast that nobody has to care about optimizing their code. Then reality and post-PC era strikes back, battery life and software power-efficiency play higher role than ever...

My experience with Go and especially Rust is quite limited, but it would be really great if I could do my stuff(right now games and path tracing rendering, render times from 12 hours upwards) in either them instead of C++. I really like the ideas behind the languages and they fit my core domain just as well as C and C++ do. Though the thing is, there are no tools. No libraries. No books. Nothing. There's simply no way to jump over, even if the possible performance penalty would be tolerable. And this isn't likely to be changing much for the better for upcoming years, if even then. Perhaps when the languages get an actual standard... It's sad in a way even, but can't escape the facts which make them not viable for many domains.

> No books. Nothing.

No books?

Go in Action; Evan Shaw; Manning; http://www.manning.com/shaw/

An Introduction to Programming in Go; Caleb Doxsey; http://www.golang-book.com

Network Programming with Go; Jan Newmarch; Free e-book; http://jan.newmarch.name/golang/

Programming in Go: Creating Applications for the 21st Century; Mark Summerfield; Addison-Wesley Professional; http://qtrac.eu/gobook.html

The Way To Go: A Thorough Introduction To The Go Programming Language; Ivo Balbaert; iUniverse

The Go Programming Language Phrasebook; David Chisnall; Addison-Wesley Professional

The Go Programming Language (upcoming); Brian Ketelsen; The Pragmatic Bookshelf; http://pragprog.com/book/bkgo/the-go-programming-language

Learning Go; Miek Gieben; Free e-book; http://www.miek.nl/projects/learninggo/index.html

Systemprogrammierung in Google Go: Grundlagen, Skalierbarkeit, Performanz, Sicherheit; Frank Müller; Dpunkt Verlag; http://dpunkt.de/buecher/3449.html

Programmierung in Google Go: Einstieg, Beispiele und professionelle Anwendung; Rainer Feike, Steffen Blass; Addison-Wesley

Remove the books that are mostly too general introductions for novice programmers (the syntax etc). Remove the poorly edited, self published free ebooks. Remove the german language books.

Now divide by the number of C++ books published in the last decade.

You got something like 0.05?

Thanks for the cogent response, not sure why I was modded down, but never mind. I've been doing a few small experiments in Go just to see what it's like and have loved the compilation time, lack of cruft like header files, and the simplicity. The standard library for Go reminds me of Ruby in many ways as it mostly works exactly as you'd expect with few surprises, and it covers a surprisingly broad range.

I agree that hyperbole on the future of computing is overblown (I wonder if Rust now feels it has to compete with Go for hype!), but there are a lot of things in say Go which are not as nice in C, unicode strings, loops, slices, etc. So it's not entirely pointless to start again with all we have learned since 1969. All of the little niggles in C can be worked around, but overall Go feels like a cleaned up version of C which took a few ideas from C++ and other OOP languages and left the rest out as unnecessary. Given the problems vast hierarchies of classes sometimes cause, and the huge and growing complexity of C++, I can sympathise with that point of view.

To say that there are no tools or libraries is exaggerating slightly, there are tools for Go (see http://golang.org/pkg/), and books, just not as many, which is quite normal for a young language which has just hit 1.0. For Rust there will be far fewer as it is even less developed, but that's to be expected. As you say if Go or Rust ever do become popular it will be over decades, not within a year or two.

JLS7 is ~600 pages; the latest draft I could find (because apparently the actual standard isn't available online, a problem in itself) for C++11 was over 1200.

I very much doubt a version bump will stem C++'s decline; while it contains plenty of improvements there are no really radical changes. What are the changes that you think will make people who chose C or go over C++03 switch to C++11?

Take a look at Herb Sutter's talk at Microsoft's GoingNative2012 conference: http://channel9.msdn.com/Events/GoingNative/GoingNative-2012...

I really don't see where someone would pick C over C++ when starting a new project if not either due to 1) lack of (good enough) C++ implementation or 2) lack of confidence in using C++(this is personally a problem for me, as I've been programming in C much more than C++ and still feel unfamiliar with lots of things, the least not being the object oriented paradigm). Of course personal taste may differ, and this is understandable; I'd prefer more compact language but right now the domain I work in has no alternatives.

From the outside, C++ appears to be such a mess that I actually cannot tell whether C++11 made it better or worse.

I don't think systems programming is going anywhere—I mainly stick to Python, but would like something lower-level available that I actually enjoy writing. The thought of having nothing better than C++ for the next however many years is terrifying.

> From the outside, C++ appears to be such a mess

Implying that you don't exactly even know the language, if it really is such a mess you think it is. I bet a bunch it really isn't.

> I actually cannot tell whether C++1 made it better or worse.

This really just shows the fact that you are clueless about C++ and what C++11 brings to the table. Whatever you think has no relevance. Not because your opinion would be less valuable, but because you can't be objective about something you don't understand. You just follow what you hear from others - often times those who are in the same situation. Hooray for circle jerking over how C++ sucks. This leads nobody nowhere.

This is a major problem with C++, but it's mainly a problem for people who don't write any C++.

So what does C++11 bring to the table? To mention some:

Safety - Standardized smart pointers, nullptr, better type-safety

Performance - Rvalue references, move semantics, constant expressions with constexpr

Concurrency - Standard facilities for threading, async, futures

Language features - lambda expressions, support for UTF8/16/32, uniform initialization

Libraries - std::chrono for various time handling facilities, standard random number engines and generators, hash tables, regular expressions, tuples

There's also plans to introduce more libraries to C++ next year, and evolve the language through libraries rather than the core language specification.

Many of the problems people face with C++ can be avoided, and people know it. Many of the problems in existing code bases are fault of poor design. (Blaming C++ for multiple inheritance because the designer shot themselves to the foot by not being competent at designing software is a prime example. It's not like the same functionality couldn't have been implemented via policy-based or component-based design for example. Easier to blame the tool than the user, I get that.)

Of course, those who use the language just avoid them and those who don't use the language keep bitching about them. Strange.

Wait, wait.

> Safety - Standardized smart pointers, nullptr, better type-safety

> Of course, those who use the language just avoid them and those who don't use the language keep bitching about them. Strange.

Dumb pointers are built into the language. They just look like an asterisk.

Smart pointers are, at best, something like `unique_ptr<...>`.

So C++11 continues to have the error-prone variant as the default, and you have to import a stdlib thing and put some ugly incantation on every pointer in your entire program to finally get memory safety. But you expect those who use the language to "just avoid" broken features like dumb pointers -- the features that have dedicated syntax, the features that take 92% less typing and reading effort, the features that have been around for twenty-nine years.

Your argument is that C++11 is great because they tacked on a bunch of stuff to fix the core language, and obviously I should just ignore the core language because the other facilities are so great. In fact, you deride anyone who uses the core language as merely not understanding C++ well enough or creating "poor design"—even though multiple inheritance has built-in syntax and component-based design does not. (Rust has no MI, and its traits look like they'll map quite well to components. Hm!)

You are blaming the user for not using the tool in a way it was clearly not designed, because that other way happens to work better. Maybe it's not just me. Maybe, just maybe, this tool sucks and you're so used to it that you can't believe anyone else would possibly not want to use it.

I'm interested in Rust because it lets me do systems programming without having to avoid the entire core syntax because it's all broken, without having to worry that whatever I'm doing might be bug-prone but only hanging around because C did it, and without having to deal with rhetoric like this implying that anyone who doesn't think favorite tool X is the height of perfection must just not understand it well enough. I've had my fill of that crap from the PHP crowd.

What exactly are you arguing about?

> So C++11 continues to have the error-prone variant as the default

Yes, and rightly so. What would happen to the existing code if the "default" pointers became smart pointers? Would they be unique? shared? weak? Also backwards compatibility with C would be gone, stopping people from using C++ as a better C with improved functionality. Of course, everything would break and some people would possibly even rejoice!

> Your argument is that C++11 is great

No. My argument is that C++11 is an improvement and that there's nothing better in the market which could be used where C++ is being used in, without sacrificing portability, efficiency, productivity and existing investment in skill and knowledge. As I've mentioned in various replies, I'd be more than happy to move onto language which let's me do what I'm doing with C++ - unfortunately no such language exists. C# makes me tied to Microsoft's platform(Mono is bullshit, don't even bother). Java ties me to Oracle and Java's own platform built upon JVM, compromises efficiency, forces programming paradigm and exposes to whatever problems the Java implementation may have now and in the future. D seems to be like C++ with some changes, less tools, portability, performance and libraries available. Needless to say Python and Ruby aren't even considered. Rust is very unstable, features come and go, tools don't exist, books don't exist, perhaps the whole language won't exist in 5 years. Go seems more mature and seemingly the most potent candidate, although with various drawbacks(no support for libraries, whatever written in Go can't be included with other languages).

> In fact you deride anyone who uses the core language as merely not understanding C++ well enough or creating "poor design"

Anyone who plays with fire and burns themselves while doing so can do nothing but blame themselves, I'm sure you agree. There's no denying the fact that using C++ correctly is actually hard. It's not so much because of the language itself(I'm not saying language hasn't got problems) than the ways it's being taught. For example, if you Google for C++ tutorials, you see things like naked new and delete, using explicit resource management where RAII would've been sufficient, not taking advantage of STL, ... Take a modern book dedicated towards learning C++11 from the ground up(e.g. C++ Primer, 5th edition) which actually avoids lots of problems and uses the language in a safe, efficient way. I'm sure anyone learning the language that way instead of from books dating back to mid 90's, from people who still consider C++ as "C with Classes" actually avoids lots of the problems you've mentioned. I assume I don't have to mention about actually designing software and the complexity of it regardless of the language. It's very loosely tied to the language itself, rather than the core concepts, requirements and benefits behind the design that it's clear that the designer is to be blamed for any issues.

> You are blaming the user for not using the tool in a way it was clearly not designed, because that other way happens to work better. Maybe it's not just me. Maybe, just maybe, this tool sucks and you're so used to it that you can't believe anyone else would possibly not want to use it.

The "tool" is flexible. In many cases as the programming language and practices evolve, we find better ways to do things and I think this is a good thing. This happens with every language. How would a Python programmer from 1997 compare to what we have today?

I don't consider the language to be perfect, but it does what it's intended to do the best what there's available. Feel free to provide alternative languages for systems programming in the domain in which C and C++ work in. Alternatively I wouldn't mind alternatives even for the domain in which I work in(3D simulations, games, computer graphics).

> and without having to deal with rhetoric like this implying that anyone who doesn't think favorite tool X is the height of perfection must just not understand it well enough.

A language does not have to be the height of perfection to be the best tool to do the job with. As it should be quite clear by now, I don't consider Go or Rust to be viable alternatives for C++ in many domains for the upcoming years. We both know why.

Meerkat telescope project has lifespan of at least 20 years and it's control-and-monitoring software is being written in Python.

This is to answer your direct question only because I fail to understand how your paragraphs fit together.

Hedge fund trading code may not have "2030 and beyond" lifetime, but it's certainly something major companies are investing a good bit of money into. Also tends to be in OCaml...

Or rather: some hedge fund (of thousands all over the world) uses OCalml (Jane Street, IIRC).

That does not equate to "hedge fund trading code tends to be in Ocaml".

It's not just Jane Street at this point. Jane Street has been using OCaml for a while, and others heve been picking it up too...

My understanding is that a significant number of quants use Haskell.

You seem to be using a definition of "declarative" that I'm unfamiliar with. The functional languages you target in the second to last paragraph are very different from the more imperative languages in the final paragraph, and I wouldn't consider either group to be "declarative". I'm left wondering what concepts you're attempting to criticize.

Having said all that, yes, I believe major commercial projects are started in those languages, though I have no idea if anybody plans them into the 2030s.

Functional languages are declarative. The point is that there are two kinds of languages; languages which people bitch about and languages which nobody uses.

Sure, I can mention tons of commercial projects which are started in those languages, and which use them at least partly(for example Python for scripting interface). However, I really doubt none of the languages provide a solid ground to build upon(even if we weren't talking about systems programming and it's future). Consider C and C++ for example; in what language is the OS you are using written in? The compiler it's compiled with? The libraries you use? The web browser? The JavaScript interpreter? You get the point. The fundamental systems we use are built upon these two languages, and I think it's a big problem people would want to address(myself included), because these languages are nowhere near perfect. There just aren't better alternatives for that domain, considering:

1) existing codebase 2) libraries 3) tools 4) efficiency 5) portability 6) programmer availability, resources and education 7) language reliability and future proofness(e.g. backwards compatibility, vendor and thus platform independence).

These are real problems when it comes to building reliable abstractions(that's the whole essence of systems programming, isn't it?), and thus far C and C++ have been the only languages which have been able to provide all of those the best.

When it comes to the future of computing from a hardware level, things may(or may not) seem a bit better; for example utilizing GPUs for general computing, using low-power ARM/x86 cores for parallelism. There are foundations for these things in C(and now C++ too) and libraries to help programming these things. How many years will it take to get a reliable Go or Rust support for programming a vendor-independent GPU? What's the overhead compared to say C or C++? The alleviating thing here is that GPUs and ARM cores aren't CISC machines so compilers and thus software is highly responsible for the performance, so with a good compiler you can get away with using less efficient implementation language, and get gradual improvements too.

"How many years will it take to get a reliable Go or Rust support for programming a vendor-independent GPU? What's the overhead compared to say C or C++?"

We're already using Rust for OpenGL ES 2.0-based graphics programming.

I didn't expect that, I am almost surprised! Though, I was more after using Rust or Go for general purpose GPU computing via OpenCL for example. There seems to be some form of project for binding, but I certainly would not put my money on some random person doing the bindings. Very experimental and unreliable stuff.

This is a hard problem, because it involves mindshare and resources; even a perfect language would take years to gain a serious foothold. But it'll never be solved if nobody tries.

I'm glad someone's trying.

This is definitely true, and the exact point I'm building my arguments upon. Are Rust and Go better languages than C and C++ for solving lots of problems, in many, many cases yes! But we can't use them. How frustrating is that!?

Of course there's a difference between being able to use a language and when mainstream starts picking it up. Considering for example C++, it wasn't only until mid 90's when it really took off, a bit more than 20 years after it was named C++ from C with Classes. Perhaps Go and Rust have their place in the stack sometime in 2030.

What do Go and Dart have to do with each other, besides their origin at one of the largest tech companies in the world?

Have you written much Go? What do you think of it in practice?

As languages, nothing. In design, their conservative nature.

Go is not a conservative language. A conservative language would be something like D, which is mostly a clone of C++ with a few extra features tossed in.

Go is a highly opinionated language with at least three big new ideas:

* goroutines for concurrency ("Don't communicate by sharing memory; share memory by communicating")

* a new type system which is based on structural subtyping (some people have called this static duck typing).

* use of return codes rather than exceptions for routine error conditions ("We don't want to encourage the conflation of errors and exceptions that occur in languages such as Java." See https://plus.google.com/116810148281701144465/posts/iqAiKAwP... )

Just those three things are enough to make Go code look very from what came before. And I could go on-- things like the way go does namespaces and scoping are also very different.

It's a different philosophy that leads to much better code, in my opinion at least, not dozens of gee-whiz features, which seems to be what the languages of the 1990s gave us and are continuing to give us.

> We don't want to encourage the conflation of errors and exceptions that occur in languages such as Java.

I'm not getting this argument:

    checked exception == error
    runtime exception == panic

Except these big new ideas have been around for decades, some of them for nearly half a century.

You're right that Go is not conservative, though, but retro, trying to mitigate the damage done by C and C++.

The ideas of Go are new to the mainstream, and at the end of the day, that's what matters. It's a programming language, not a PhD thesis.

None of those three "big new ideas" are big or new. Go is incredibly conservative, primitive even. It takes a few tiny things like those you mentioned from languages that have had them for decades, but leaves out so much important stuff that the language is barely useful at all.

I am a computer scientist that has performed research using Go because I love working with Go. I absolutely despise Java. So I'd say that your assessment is a bit off.

I also look forward to trying Rust as well. Having algebraic data types makes me giddy.

Three different types of pointers? Perhaps that is going a bit far?

> If you've a sharp eye, you're wondering what that "~" is that I snuck in on the type of the closure for the child task. That's actually a pointer type, of which Rust has three (none of which can be null, by the way):

> ~T is a unique pointer to a T. It points to memory allocated in the send heap, which means data inside of unique pointers can be sent between tasks. You can copy unique pointers, but only by deeply copying (otherwise they wouldn't be unique!) (and by default, they are "non-implicitly-copyable", so the compiler will issue warnings if you copy them without writing the "copy" keyword).

> @T is a managed pointer to a T. Currently, these are reference-counted and cycle-collected (they may be full-on GCed in the future). Copying one increments the reference count, so multiple managed pointers can point to the same data. These are allocated on a per-task private heap, and cannot be sent between tasks.

> &T is a borrowed pointer to a T. It can point to the inside of arbitrary data structures - on the stack, inside ~ or @ pointers, etc. Rust has a static analysis, called the "borrow checker", that ensures that borrowed pointers must not outlive the scope of the pointed-to data (i.e., it is impossible for rust programs to have a use-after-free).

It's not too daunting. `&` is your go-to pointer for most scenarios. `~` pointers are for anything you want to send to a different task. `@` pointers are for things you want to be garbage-collected.

Fantastic summary of the pointers. It's less daunting now.

Do you have reasons why it's going too far, or is that just an emotional reaction? Consider that these pointer types map exactly to the pointer-type templates provide by C++11: unique_ptr, shared_ptr and weak_ptr: http://en.cppreference.com/w/cpp/memory

In the interest of starting discussion and not just stating facts, I will take the position that I think Rust's adoption of these concepts into the language is a Good Idea. Many pointer-based codes have these pointer types, but they are enforced through convention alone. C++11's templates are a step in the right direction, as they make explicit what convention assumes, but the compiler does not know about them. Yes, because they are template classes, there are certain things one can and cannot do with, say, a unique_ptr, but the compiler cannot do any analysis on the usage, because the concept has no representation in the language semantics. (Ben references a blog post by Niko Matsakis about some of the analysis Rust does: http://smallcultfollowing.com/babysteps/blog/2012/07/19/yet-...)

I agree they're good to put into the language semantics, but as a side point on C++11, couldn't the compiler actually assume a bunch of things about those templates, since their behavior is specified? GCC, at least, assumes many things about even C stdlib functions, because it "knows" what they do, so rather than treating them as just regular functions, it can make stronger assumptions about them when optimizing, and/or generate code to implement their functionality internally, rather than linking the library implementation.

You can, but the difficulty, I think, is that this clean separation of pointer types exists along with naked pointers, including void*. That is, C++11 still carries the legacy of C's free-for-all, so the kinds of guarantees that one could get from a C++ compiler - even with baked in knowledge of unique_ptr, shared_ptr and weak_ptr - are going to be limited.

A lot of the productivity gain in higher-level languages comes from not having to manage memory. Having three different incompatible, differently allocated pointer types to fool around with slows you down. The creators of Rust are hoping that the speedup from not having to do global garbage collection in most scenarios will balance out the loss of productivity.

Personally, I'm skeptical. Azul showed us that pauseless GC was possible, even for Java. Android and .NET showed us that even without pauseless GC, the performance of GC'ed languages was adequate to build attractive user interfaecs. A lot of Rust's design choices mean that it's not really suitable as a web development or scripting language, and those two communities have shown themselves the most receptive to new languages.

Mozilla has tried to rewrite their core product in a different language before. Look up "Javagator." It did not end well. But, who knows. Maybe this time really will be different.

It's not impossible to build attractive user interfaces with garbage collection, but it is significantly harder, both for users and developers of the system. Android system apps go to great effort to avoid using the GC during critical animations, while in iOS this is much easier. Implementing a good garbage collector is a lot of work, and it's very difficult to tune; even the JVM, which has had an enormous amount of man-hours put into it, struggles with GC pressure under some loads.

Additionally, the applications we want Rust to be suitable for aren't competing with GC'd languages. They are competing against applications written in C++, with manual memory management and no automatic storage reclamation. They will be put head-to-head in benchmarks with them. We don't take performance regressions, and relying on global GC would be a significant risk. The Javagator is, in fact, an excellent example here!

(As an aside, the pointer types are not all incompatible; all pointers can become &T, and the vast majority of functions take & pointers.)

It's a little misleading to say that GC makes "attractive user interfaces... significantly harder." For any business application, for example, using GC is a no-brainer. That's why I mentioned .NET in my response.

Android games have been known to struggle with GC sometimes. However, garbage collection has gotten a LOT better on Android over the years. I wrote a small game in Android 1.1, back when GC pauses were anywhere from 100ms-250ms, and avoiding allocations was VERY important. Nowadays, most pauses are less than 5 ms, and the NDK is always available if you want to write native code and avoid GC altogether.

In general, the game industry seems to be moving towards a model where you have "engines" and "level packs." The physics, graphics, etc. engines are developed by a small group of people in C or C++, and the level packs are in whatever scripty language you want.

The development of the web reflects a similar split. You have the C/C++ engine, and the HTML/CSS/ECMAScript content. C and C++ have low programmer productivity and a lot of odd quirks, but nobody cares because the bulk of the development has moved higher up the stack. So attempts to replace C or C++ get a "if it's not broke, don't fix it" type response.

This is one reason why golang, for example, has seen more users come in from the scripting and web development community than from the C++ community. C++ projects tend to be old, conservative projects that are at the bottom of some giant stack-- like Google's infrastructure, for example.

Rust has some interesting ideas. The typestate concept in particular is interesting. I also do kind of get what they're going for with the 3 different memory types. I guess we'll see how it all works out.

Some tasks - such as implementing kernels - are poorly suited for garbage collection. And while there do exist garbage collectors that can be used in hard real-time scenarios (http://queue.acm.org/detail.cfm?id=1217268), that's not the norm.

I think there will always be a place, at the lowest levels, for manual memory management. Providing abstractions to make that as safe and manageable as possible is a Good Thing.

Some tasks - such as implementing kernels - are poorly suited for garbage collection

Right, but we have C for that. I also doubt that Rust would be suitable for hard-realtime applications, due to the fact that it has garbage collection. Even task-local garbage collection is still garbage collection, and not what you want for hard-realtime applications.

Of course there's a bunch of different definitions of "hard realtime" floating around, and rust would probably be suitable for some of them. I fully expect that someone will reply to this with something like, "Well MY definition of hard realtime is playing movies, and Rust is suitable for that." :) But when I think of hard realtime, I think of applications where you must have absolute determinism.

Your definition of hard real time is the accepted definition; playing movies and such is soft real time.

Anyway, about C: that's the point. People are trying to create safer systems programming languages that can replace C.

I don't think Go is intended to replace C. It was intended to replace C++.

C++ was intended to replace C, so it's intended to replace both.

Let's not judge the intentions of the Golang authors by what the intentions of the C++ authors were.

Golang is a higher level language than C. It will not be suitable for non-GC'ed environments, and the Golang authors are perfectly well aware of that. It's ok to make a language that solves many problems really well, without trying to solve every problem in the world in one bloated language.

I see Rust has ADTs and strong static typing, how much is enforced at compile time?

One thing I love about OCaml is its exhaustiveness checking. I've been doing a lot of Erlang and it feels so dangerous without the compiler telling me if I've missed a pattern, or if a clause might return the wrong type. (Dialyzer and typespecs help somewhat, but aren't nearly as nice.)

If I could have that sense of safety with Rust, I would be persuaded to investigate it sooner rather than later.

Short answer is "all of it"[1]. 'match' statements in rust also exhaustive. I believe ADTs are used and checked exactly the same way as in ocaml (I've used SML and Haskell and they are the same as there).

[1] not to say dynamic failure is totally gone from the language -- but it is if you avoid both writing 'fail' and using library functions that can fail. A common example is option::get(Option<T>) -> T, which fails if the optional value is None -- it leaves a sour taste in my mouth whenever I use it, but I still do sometimes.

Has anybody heard if they've finalized a spec yet, or at least come close? I've thought about getting into Rust from time to time, but the lack of a finalized spec and/or stable reference implementation has always scared me off.

We're getting close. Rust 0.4 (hopefully to be released next week, assuming no more schedule slips) will be quite close to final syntax-wise. Although not all the features will be implemented, your code will be mostly backwards-compatible.

If you'd like to try out the language, I recommend working off git master rather than 0.3; the language has been dramatically simplified in Rust 0.4.

Excellent, thanks for the update! :)

There is no finalized spec (and the spec has actually fallen quite far behind for the moment), nor do I expect anything about Rust to be called 'final' for a while yet.

The upcoming 0.4 release (any day now) makes a big push to get a lot of the syntax pieces in place, but still has many rough edges. Beyond that the standard library is still an incomplete and inconsistent stew of different styles.

Rust is still firmly alpha software, but should stabilize significantly in the next year.

Rust feels more like Perl 6 (always on the horizon) or Ada (the do everything language), while Go already is already specified, being improved, and out in production.

I don't think it's like that. I think they're just developing it through usage.

And it's not like it's been ten years yet.

> I don't think it's like that. I think they're just developing it through usage.

What's shipping with it?

I have a bit of the same impression that the other poster has, that it's always in development without something stable. I hope that gets dispelled sooner or later, because, for better or worse, fuzzy marketing type things like that matter.

The compiler is written in Rust, as is Mozilla's experimental browser engine called Servo.

Your impression is completely accurate; the language is in development and is not ready to use yet. It shouldn't be used in any shipping products. There's nothing to "dispel" here until the 1.0 syntax is is finalized; fortunately that should happen in months rather than years.

From a pure marketing perspective, sure, maybe it would be better to finish more of the design and implementation before talking about it publicly. But Mozilla's pretty committed to working in the open, and that has its own benefits. For example, significant parts of the Rust implementation were created by contributors who were not paid Mozilla staff, including a Google employee (in her spare time).

(Disclosure: I'm a Mozilla employee but not part of the Rust team; I've contributed a little bit to the Rust compiler in my free time.)

Fair enough. Like I said, it was just an impression; a vague feeling, so I'll be curious to see what they come up with when it comes out.

Ada is also in production for a few decades now.

In nine months, Rust has gone from version 0.1 (its first public release) to version 0.4 (expected next week). Much of the work in that time has involved removing (!!!) features from the language and polishing what's left. Version 1.0 is anticipated in the first half of next year.

If you really feel the need to compare with Go, measure the time from Go's first public release (2009) to its first production release (2012).

Oh, come on. Rust started as an open-source project in 2010 (with the release of the preliminary design and OCaml compiler that Graydon had written as a hobby project). At that time, work started on the self-hosting compiler, leading to its initial 0.1 release eight months ago, and proceeding with releases every few months and steady progress toward the remaining goals on the 1.0 roadmap [1]. Just because Go was developed earlier doesn't mean that Rust isn't getting done. And it's not noticeably more complex than it was in the initial design; in some ways it's simpler.

(By contrast, Perl 6 has been in development for over a decade, with a half-dozen implementations in various stages of completion.)

[1] https://github.com/mozilla/rust/wiki/Note-development-roadma...

I've attended a few Go meetups here in SF already, and I haven't even seen any Rust meetups on the radar.

At these Go meetups, companies are presenting what they're using Go for in production already, with great success. Go is not difficult to program in, Go is functional, Go is performant, Go is developer friendly.

That is not to say that Go doesn't have warts, it does (32-bit garbage collecting anyone? or the slow regular expression engine?), but people from Ruby, people from Python, and people from JavaScript are gravitating toward Go and _enjoying_ it.

Go also has excellent stewardship in the form of Pike et al.

In my mind, at least, the gap between Rust and Go is pretty big at this point. This is why Rust feels like vaporware (especially compared to Go).

People like to compare Go and Rust, and I think the comparison is pretty natural. The difference is that Go already has "boots on the ground."

If Rust releases something awesome in the next few months, the gap between Go and Rust could rapidly close (depending on developer adoption). But, in my mind, the longer Rust takes to iron out the quirks of it's syntax (not implementation, syntax!), the wider that gap will get.

"In my mind, at least, the gap between Rust and Go is pretty big at this point."

It's the same gap it's always been -- a few years.

When Go was announced it had a spec, a significantly larger standard library than Rust, two compilers (and a parser in the standard library), and much, much better documentation. While the language had changed until the Go 1 release from a few months ago, it had changed less in two years than Rust changed in a few months. Go also introduced a tool that would automatically rewrite your old code to adapt to the new language changes.

All that means is that Go was announced much later in the development process than Rust was. If Rust had been developed like Go, it wouldn't have been made public yet. But that's not the way Mozilla does things.

I don't know if this is a fair criticism. Rust has not been stuck in eternal development for many years like Perl 6. Rust is less mature than Go just because it started (in earnest) later and it's a little more ambitious. Go was immature not that long ago.

which is especially important in the multicore world where heaps must be protected by a global lock

The article seems nice, but this is misleading at best, and flat out false at worst.

Concurrent memory allocation has been around for many years, and gasp is perfectly usable with C.

I wondered how long it would take for somebody to call me out for that. It seemed like a sketchy claim when I wrote it (it wouldn't be too complicated to make the @-heap lock-free in rust, for starters), but in retrospect, I shouldn't have said it at all. Thanks for pointing that out.

I tried to think of an alternate justification, but couldn't come up with a non-awkward way of saying it, so just redacted that part entirely. I'll say here, though, a few reasons I think avoiding malloc/GC is important:

- pcwalton has mentioned his desire to be able to reason about when the garbage collector runs, so that he can statically guarantee that the graphics rendering task in servo will never take a big pause to do a GC.

- In this-must-never-fail environments, such as writing kernel device drivers, using dynamic allocation has to be avoided (and, if i'm not mistaken, even more likely to fail when done in an "atomic section" using kmalloc(GFP_ATOMIC)). Rust in its current state is not suitable for kernel-level development, but it's not far off; and it's critical to be able to reason about memory allocation for such applications.

From the end of the last post in the series (http://winningraceconditions.blogspot.com/2012/09/rust-4-typ...):

    let arc = RWARC(some_data);
    for num_cpus().times {
        let arc2 = clone(&arc);
        do task::spawn |move arc2| {
            process_data(arc2); // might read, write, whatever
        }
    }
    let modified_data = unwrap(move arc); // blocks on all child tasks at once
    // do more of the algorithm, etc.

This is a nice fork/join pattern, allowing implementations with minimal copying and reasonable safety. If this is indicative of design decisions made by the creators of Rust, I think it looks pretty good.

I don't know if it is the language of the future, but Rust is definitely high on my "let's reprogram Singularity/Plan9/Friends"-and-actually-use-it list

I'd certainly join and attempt to help any project started in that direction, at least.

Makes me wonder if anyone is working on a llvm-to-rust compiler.

What would that be used for?

Checking correctness of the rust->llvm compiler, obviously :P

It's interesting that you mention Singularity, as it turns out that Singularity has been a major influence on Rust. For example, the current communication is very similar to the channel contracts used in Singularity.

Well I'm a big fan of many concepts of Singularity. And I like rust. I guess that's no coincidence ;-)

How do you implement closures entirely on the stack? What about the case when a function returns a closure into an external translation unit? How do you cover the bound parameters? You can't do call-time lambda lifting if the bound parameters are out of scope at the call site, you need to put something on the heap and GC it. Same with compile-time lambda lifting (producing a chain of bind1st stubs on the executable heap)

My mistake. I meant to write "can SOMETIMES be entirely on the stack" -- fixed that just now.

Functions indeed cannot return closures without heap-allocating them. (It might be possible in specific obscure cases, but definitely not in general.) However, if you wish to curry a function foo(a,b), you can do that in the caller (instead of at the function definition itself) by writing:

let curried_foo = |b| foo(a,b);

That is then stack-allocated and can refer to 'a' in the same frame, and can be passed to other functions you might call from there (but not returned upwards, indeed).

I'm not an expert, but I believe that Rust will allocate closures depending on the data that they close over. So if you reference a stack-allocated variable from within your closure, it gets put on the stack. If you reference a heap-allocated variable from within your closure, it gets put on the heap.

I could be dreadfully wrong, though.

No, where a closure is allocated depends on the type of the closure. The type of the closure is inferred just like any other type. It has nothing to do with what it closes over (although what it the closure is allowed to close over depends on its type).

>Stack-allocated data lets you often avoid dynamic allocation overhead and garbage collection (even closures can sometimes be entirely on the stack).

There's an entire website named after the results of over-using the stack. Smart pointers and move semantics in C++11 completely smoke GC and memory management everywhere else.

While I'm sure Rust is a nice language and I applaud its innovation,there's already a perfectly good wheel.

Smart pointers and move semantics in C++11 completely smoke GC and memory management everywhere else.

Aside from having to enforce proper pointer usage at runtime, whereas Rust does it at compile time.

Good article!

Sorry to be nitpicky but the font size/line-height makes it really hard to read. Perhaps make both a bit bigger for optimal reading?

Probably this would help :) http://viewtext.org/article?url=http%3A%2F%2Fwinningracecond...

I just found this Firefox add-on today and I'm completely in love:

https://addons.mozilla.org/en-US/firefox/addon/clearly/

Here's what appears to be the Chrome version:

https://chrome.google.com/webstore/detail/iooicodkiihhpojmme...

Makes it ridiculously easier to read basically everything on the web.

I like the Readability add-on and bookmarklet:

http://www.readability.com/apps

Rust looks very interesting. I downloaded it and tried it out on Windows but got "libgcc dll missing" error. They really need to statically link in the MingGW runtime to make it easier to run.

There is a bug on doing that, but keep in mind that bundling libgcc would only make Rust error out with a more informative error. Because Rust uses the system linker, you need MinGW installed to use Rust.

Someone ought to mention the similarities between Rust and Scala. These are two really great languages, IMHO.

Rust indeed looks very promising. I hope it'll pick up usage beyond Mozilla internal projects.

Rust and Go provide me with more day-to-day pleasure than... well... a sadly large number of things. This is a good resource for learning more Rust, thanks.

why didn't they call it YAPL?

Another language that adopts a different syntax for the sake of adopting a different syntax. If you are going to ask me to adopt a different syntax, you'd better have a good reason for it, because I can do everything I need to do with the languages I already have.

You can do more in assembly language than you can in any other language. That doesn't make it a good language to program in except in very special cases.

Reply on news.ycombinator.com