The Rust module system is too confusing
withoutboats.github.io
withoutboats.github.io
Are the keywords idiosyncratic? Maybe. Can a new user get it working in an hour and learn it in 2 days? Yes.
It would be cool if there were a polyglot build system that you could learn once and have the same expectations in every language. But there are downsides to this proposal (not least of which is the fact that when someone names a polyglot build tool, it's often NPM). I doubt the module system is the #1 or even the #5 thing people struggle with when learning rust.
Sadly not yet.
My VC++ builds are still faster, because they can enjoy using binary libraries across projects, something that cargo doesn't yet support.
Also VS 2015 and the new VS 2017 do have quite a few improvements regarding the linker, incremental builder and C++ modules support.
https://blogs.msdn.microsoft.com/vcblog/2015/12/03/c-modules...
https://blogs.msdn.microsoft.com/vcblog/2016/02/11/compiler-...
There were also CppCon talks from both Google and Microsoft, but I cannot search for them now.
This isn't anywhere near the top issue that I (or anyone else I know of) encounter with build performance in Rust. Dependent library builds are very parallelizable, so they tend to build quickly. Moreover, the build performance that matters in the day-to-day cycle is mostly rebuild performance, and the binary artifacts are cached in this case.
We could spend time adding a global cache of build artifacts, I guess, but all the time we'd spend working on this wouldn't impact my development velocity much at all. Incremental compilation, on the other hand, would.
It takes me more than one hour on a core duo with 8 GB + HDD, while I occasionally see the same crates being compiled and get to do something else.
I could probably buy a new computer, as this one isn't something that one can easily parallelize, but why should I when my C++ builds are actually fast enough?
I am just an opinated Rust dabbler, please take my complaint as contributing feedback about what I see as one possible show stopper for Windows developers to adopt Rust, vs our current .NET + C++ workflow, in terms of productivity.
Thanks for Rust.
I'm not being sarcastic here, Rust seems to be a great language (definitely on my list to learn), I'm really just observing. Looking at the success and history of programming languages it seems to me that many if not most professional programmers want some obscurity, a desire that is often disguised as an apparent need for brevity which in reality plays no role with modern editors.
Maybe this makes sense, after all it's better to have a language that an incompetent boss or a marketing department cannot just pick up easily to 'roll their own solution'.
At the same time, implicit-block type procedural languages like Visualbasic and Realbasic/Xojo are popular, but they do not get much love from professional programmers, even though they are maybe the least obscure among all imperative mainstream languages. LISPs are also an interesting case, S-expressions are among the least complicated and easiest to grasp syntax and CommonLisp offers practically everything a programmer could ever wish for (including multiple dispatch) but it never became very popular.
But maybe I'm wrong. For example, advanced Lisp macrology is definitely obscure, so that should count in favor of Lisp being more popular according to me 'hypothesis'. Haskell syntax is fairly easy, but monads are certainly obscure, yet Haskell does not seem very popular. Likewise, Python does not seem to be very obscure, it certainly isn't syntax-wise, and it's still very popular.
So I don't know, perhaps my assessment was premature. It almost certainly was. ;-) I've just always had this impression that many programmers prefer a certain amount of special trickery and shortcuts over clarity. But perhaps it's the availability of libraries what counts most in the end.
Wirth has a nice paper where he writes about the error of not doing any PR and thinking "build and they will come" would just work.
https://www.inf.ethz.ch/personal/wirth/Articles/Modula-Obero...
Check the "Conclusions and Reflections" chapter.
I was able to learn C reasonably well as a child precisely because of how straightforward it is. You can reserve a block of memory. You can free it. You can get the address of a value. You can get the value from an address. It works in an extremely straightforward way, unlike most languages. To a child, programming in C feels like the computer is doing exactly what it was told to do and nothing else. It feels like you understand what is going on, even though your understanding is a very simplified model of what is actually going on.
Modern languages like C#, Javascript, Ruby... They are much harder to understand than C. The first time I saw a closure, I was completely dumbfounded. This doesn't make sense, a function is accessing a local variable of another function that has since been popped off the stack! What sorcery is this?? Even today things often confuse me, when I saw C# async/await for the first time I just couldn't understand how it can possibly work until I spent some time reading about it and I had been programming for decades by then including a decade in C#.
That's only weird to you because C doesn't do it.
For someone starting in one of those languages, it simply makes sense that I can use any variable in scope. (Stack allocation? What is this sorcery you speak of?)
Because stack allocation has nothing to do with C, C is not explicit about doing it (except alloca) just like most other languages, and other languages use it just like C does, including C# with its closures and async/await. It even has stackalloc.
In any non-systems language you don't need to know (and knowing it doesn't help you with anything) this stack sorcery you're talking about.
You can mess with the stack in C, you can also mess with the stack in C#. But you don't have to, and normally you don't. You use it in a completely transparent manner, not caring where the compiler decides to put your data.
Here is where it's less transparent. In a language like javascript there is no concept of a stack so what's in the quote doesn't make any sense.
When it comes to stack allocation, in C this is the compiler's decision and in JS this is the execution engine's decision.
In JS, that quote makes perfect sense, and the answer is: the engine checks what can be safely put on the stack, and the stuff that is closed over is stored on the heap to avoid losing it when the function returns.
In C, a variable's lifetime is limited to the current function call. The natural implementation of C is to track both function calls and variables on stack structure with statically determined offsets. Thus, C has stack-allocated variables.
Since a Clojure variable's lifetime is not limited to the current function call, C's implementation details aren't relevant. Stack-allocated variables are a bizarre notion.
In reality it comes from trying to apply how that language works in normal cases (in the absence of those features), and the features requiring special compiler magic to work.
Yes if variable lifetime is as you describe in Clojure then that sentence doesn't make sense for Clojure.
As I've already told someone else, it is unlikely that learning C at some point in your life (not as my first language, not as the language I use professionaly, not even in my top 3 most used languages) permanently cripples your ability to understand other languages.
This is an issue with learning a new language when you've already got your mind molded to a different one.
JS programmers have the exact opposite issue when learning C. Why can't we do this? Why must we worry about scopes for the stack? Wtf is the stack? I wonder what happens if I try to use this variabSEGMENTATION FAULT.
These languages generally are dealt with differently in one's mind, and trying to apply the mental model from language 1 to language 2 usually makes it feel harder than it really is.
I really could say the same about "it feels like the computer is doing exactly what you told it to do" about the other languages you listed. A crucial part is that you don't actually need to understand what the computer is doing under the hood to use it, especially in a higher level language. C is more like "you were forced to talk to the computer in its own language", whereas with JS/C#/etc you don't have to know what addresses and memory are. You just tell the computer what tasks you want done, and how to model them, and it does them.
So I'm very skeptical that the other modern languages you listed are harder to understand than C. They're just different.
So unless learning C somehow causes permanent disability I will have to respectfully disagree.
C sort of does cause a disability (not really a disability, this is a useful skill!) in the sense that now you're actually thinking about how a language feature works in the hardware, which is not something you necessarily need to do for higher level languages. The moment you ignore all this it's likely that the language will make sense again. Of course, like I said, this is all subjective. The solution to "I can't use async/await because I don't understand how they work" is basically that you pretend that they're magic, and try to figure out the internals on the side. You need not understand how things work under the hood to use them, though it is an extremely valuable skill to know what C#/JS/etc end up doing at the lower level.
It's a sort of "ignorance is bliss" situation.
It varies from person-to-person though. It's not specific to programming either. When I was a physics student I personally would be easily able to define layers of abstraction in my mind and work within one layer, even if I didn't understand another -- you ignore how the other layer works and assume that it is magic (while simultaneously working to understand the other layer). One of my friends could not do this -- he had to understand rigorously how the theorems worked all the way down. As you can imagine, this required a lot of effort. It meant that he always had a thorough understanding of the topic at hand, but often meant that he spent a lot more time to understand something and had no middle ground to choose.
For me, this has carried over to programming. If I'm dealing with a language like C or C++ I will try to understand how everything works on the metal. When I first learned about C99 VLAs I spent an hour mucking with a debugger and inspecting the stack. When I realized that C++ had function-local initialized statics I spent many hours digging through C++ spec/implementations to understand how it worked (http://manishearth.github.io/blog/2015/06/26/adventures-in-s...). These are things at the same abstraction level as C/++ which I must understand to be able to use them.
But when I'm dealing with languages like JS, I can operate perfectly fine without knowing how they work. Indeed, when I first learned about generators, I was able to use them for quite a while before I learned how they work. I still want to know how things work (and generally dig into it), but it doesn't block me from proceeding if I don't.
This is not how everyone approaches languages. Like I said, it is subjective.
And everything function parameter is passed as value. In many 'advanced' languages, some things are passed by value, other by reference depending on the type, sometimes automatically, sometimes manually, it took me years to figure that out.
Same things for loops.
And I still have nightmares about some languages memory models: what is allocated where? what kind of allocation was performed on the object that this function returned, do I have to free it manually or will it disappear when I leave the scope? which scope? is it on the stack or is it the GC that takes care of it? when does it take care of it? should I call it manually?
That doesn't mean all technologies that have the necessary properties to allow some people to appear super human necessarily become popular. It just makes these technologies more sticky if they manage to somehow stay afloat for long enough.
I'm not sure whether or not these obscure sides of technologies are generally a bad thing. If a language provides opportunity for those who really understand it very deeply to come up with much better solutions, then that is a good thing. I would say this is the case with maths for instance, or rather with particular fields within maths.
But there are other much less favorable examples where it feels that these supposedly smart solutions shouldn't be necessary in the first place and getting even trivial things consistently right requires memorizing book length lists of rules and exceptions. CSS and C++ come to mind.
I feel that what the general populace would require, if you wish to see programming languages more widespread, would be an ease of interface akin to a quality WYSIWYG editor, which companies have been working on for awhile. I would love to use such an editor myself, instead allocating my new cognitive savings for problem solving.
The gulf between WYSIWYG and the least-obscured / fewest constructs / best ergonomics / best developer UX / etc language still seems rather immense, even if you're thinking about Scratch. In Scratch you're dragging and dropping statements of logic, whereas in a WYSIWYG editor you're dragging and dropping elements of the page as you actually want it to look. The latter is much more declarative, and in range of what small business people want. I would argue that dragging and dropping for loops in Scratch really only helps for those who have keyboard difficulties. It's still a for loop.
I'd also add as a side point that obscurantism doesn't always mean bad ergonomics or bad UX. Sometimes it protects cognitive resources, and other times, when you are forced to dig, the bet for cognitive savings fails.
I only knew(and liked) C reasonably well before Rust. And nothing felt obscure when I started learning it.
I can only remember not getting what `||` meant (in the context of defining a closure without args). The positional meaning of `||` and `&&` is the only thing, I can recall right now, that can be considered obscure syntactically (for C developers at least) . They should have gone with literal `and`/`or` for the operators IMHO.
// prints 'true'
println!("{}", true||(||true)()||(||true)());
// prints '()'
println!("{:?}", (||||||||||())()()()()());
That's true. But put your self in the mind of a C developer looking at Rust code for the first time:
if a || b {
println!("True");
}
Then:
thread::spawn(|| println!("Hello from a thread!"););
----
IIRC, there are also cases where you have to write `& &` instead of `&&` to not confuse the compiler. That's a design/practical issue.
Both those issues would have been avoided if literal `and`/`or` were used.
I find it interesting how the only thing that momentarily confused me, as a C developer, about Rust syntax, was caused by Rust authors not wanting to syntactically deviate too much from C.
- 'use' statements are relative to the root of the crate, not the current module. To work around this, you can write 'use self::name' or 'use super::name' for relative imports. Annoying but no big deal.
- If you import a symbol into module 'a', it doesn't also get added to 'a::b'. I don't know why I keep expecting this.
Once I learned these two things, Rust imports were easy.
I'm not sure I agree with withoutboats' proposals. His chnages would make several obvious things work the first time, but at the expense of taking a simple, easy to explain rule and replacing it with something implicit and mysterious.
The bigger Rust learning curve issue is making friends with the borrow checker. I like the borrow checker. It has my back, even when I'm trying to write fast code that would be recklessly abusive of pointers in C. But I can't deny that it took me a week or two to make friends with the borrow checker.
Which is ironic, considering the premise is that the module system is "too confusing". Having never used Rust, the current rules - as explained by the author himself - seemed straightforward. For the proposed new rules, I'm confronted with two long paragraphs.
I didn't bother to read them. If the intent is to make the system less confusing, there should be a way to explain the new rules as tersely as the old ones.
That being said, I do feel that the union of a confusing and simple system can often be a more confusing system so I'm wary of this proposal reducing complexity.
I'm writing an application. Inside main.rs I can write
extern crate foo;
use foo::Bar;
extern crate foo;
use self::foo::Bar;
My typical use case is importing definitions, for example an error type or enum from the crate, let's say something from tokio_core.
I'd like to use this enum both in main and in my module; from my (limited) experience, you need to extern crate tokio_core both in main and the module, then use the definition. In this case the syntax is different for somewhat obscure reasons.
I made a simple example with just two files: (In the meanwhile I realized that my first example was wrong :) https://github.com/lucaotta/rust_modules
Let's say that after my refactor, I've "moved" the crate import into the module since "logically" it belongs there. In this case the compilation fails if I don't use self::. But previously it was working in main.rs! And I have no idea why...
I always leave "extern crate" in the crate root; then it all Just Works.
One thing that might be confusing is that `main.rs` is the entry file of binary crates, while `lib.rs` is the entry file of library crates, and a project can contain both a binary and a library crate. The binary crate then does need to `extern crate` the library crate.
We've been working on a new version of the book, if you have time, I'd love if you would take a look at the new chapter on modules [1] and file PRs or issues on the repo [2] if it's still confusing or leaving you with unresolved questions!
[1] - http://rust-lang.github.io/book/ch07-00-modules.html [2] - https://github.com/rust-lang/book
I really wish your own modules would require some relative import and be scoped within `mylib`. Eg you would do `use mylib::image::MyImage` and `use image::OtherImage` instead of `extern crate image as image_crate; use image::MyImage; use image_crate::OtherImage`.
So you'd end up with `use image::MyImage` and `use self::MyImage`?
The problem with this is that I'm not likely to remember how modules work a few months after finishing a Rust project. I reckon core aspects of a language should be so simple that they don't need any work memorising them.
Although I still don't quite get how macro importing works. It feels like when you #[macro_use] on an extern crate you get all the macros implicitly everywhere without having to issue any use statements. This seems weird and I have yet to try to find an explanation of it. But I've got modules all over my Advent of Code solutions which use macros from nom without having any nom-related use statements.
http://rust-lang.github.io/book/ch07-00-modules.html is the second edition of the book's chapter on it, I think it's much better.
> This seems weird
Yes, that is how it works. Macros aren't in a namespace. Yes, it's weird. It's an artifact of history; eventually, it will be fixed.
- `extern crate` is logically a crate-internal magic `pub mod` or `pub use`
- You should usually put your `extern crate`s in the crate root
- Putting these two together, you can refer to crates as if they were toplevel modules.
This is one of the very common confusions I've seen.
It is better, yes, although I think it takes a while getting to the point in a few places.
Docs are something I'm interested in, so you might hear more from me through appropriate channels about this and that.
> Yes, that is how it works. Macros aren't in a namespace. Yes, it's weird. It's an artifact of history; eventually, it will be fixed.
I'm glad that I both got it right and that it's not necessarily going to stay that way. I don't know how much macros are going to proliferate in the future, but I don't want to find myself with macro naming collisions. I can certainly imagine some of nom's causing issues with other crates, and having to define a new crate within my tree just to contain them would be irritating. I assume that would work as a workaround, anyway.
Basically, today you define macros with macro_rules!. This shipped in Rust 1.0 because we did not have the ability to invent and implement a more proper macro system, and didn't want to hold the whole language up on one thing. In the future, you'll write macros with a "macro" keyword, and macro_rules! will be deprecated, and "macro" macros will support namespacing, etc.
Naive question: What the hell's the difference? Is a crate not a module? IMO, there shouldn't be a difference.
(EDIT for clarity)
In a nest module within the crate, you can use names defined in the modules containing you, transitively, but in a crate you naturally can't use names defined in other crates linking to you. It's really not the same kind of relationship.
Telling him about the typo was meant as a courtesy to him, because I appreciate his post and want to help strengthen it in this small way.
1) For imports, the namespace hierarchy should have a one-to-one correspondence with the file and directory hierarchy.
2) For publishing code, instead of modules and Cargo, there would be one huge worldwide repository. You'd be able to freely import anything marked public, and the build system would cooperate with version control to make things work seamlessly.
Unfortunately it's probably too late for Rust to adopt such ideas. I wish some new language used them, though.
The other part is to have a set of conventions. Do I put "extern crate" in the root module or in the submodule that uses it? How do I order my use statements? Do I import just the module or each item in it? When the community settles on a standard way, IDEs can follow it.
That said I am a bit confused at the confusion. Rust is better than C++ in this regard, in my opinion, even if the system is a tad slower to build still.
The main difference is that in this language a module always is one file, which consists of 2 parts. 1. INTERFACE (public) 2. IMPLEMENTATION (private) The way it is organized makes compilations also very fast.
USES only uses the public parts of the module. A LIBRARY is like crate in rust.
Sadly, this module system does not use private classes/types in public functions. But that would not be hard to add.
I have no idea what Rust tried to accomplish. Being used to Object pascal many other module systems seem primitive.
You forgot to mention, cyclic dependencies were allowed on implementation part.
Also later on, Delphi introduced packages, which are a set of units, with just some of them being public. Similar to the upcoming Java 9 package model.