Rust extension traits, greppability and IDEs
eli.thegreenplace.net
eli.thegreenplace.net
> [...]
> It's entirely possible that using a language like Rust without a sophisticated IDE is madness, and I'm somewhat stuck in the past. But I have to say, I do lament the loss of greppability.
Seems more like Eli has completely missed an important and integral feature of the rust ecosystem: `cargo doc`.
While the stdlib sadly remains out of it (long-standing RFC 2324[0]), cargo doc will otherwise generate the documentation including all dependencies by default.
This means you can in fact get this information by "checking the documentation", just the right one: a global search in the project's `cargo doc` will quickly tell you where the method comes from[1], without the need for any IDE or complicated integration (though I don't think it works with text browsers as it uses javascript, for terminal / console contexts maybe try `rusty-man`, I don't know how good it is tho).
I couldn't imagine working on a Rust project without having its `cargo doc` permanently open in a tab, having the unified offline documentation for all your dependencies is just way too useful, I run `cargo doc` almost as often as I run `cargo check` (despite that being mostly useless for binary crates...)
[0] https://github.com/rust-lang/rfcs/issues/2324
[1] just tried it on a local project, the first methods it surfaces are 4 methods from byteorder (on ByteOrder, WriteBytesEx, BigEndian, and LittleEndian) and one from serde_json (on Formatter).
Visual Age, JBuilder and Visual Café still took a while to come into being.
I find dealing with a browser tab to be a bit awkward, especially if I'm working on a Rust project using Wasm and actively testing in a browser and having to flip between docs and my app. Having a dedicated documentation viewing app that I can show/hide with a keyboard shortcut just saves a second or so every time I need to pull up docs. And the integration with IDEs can sometimes mean I don't even have to leave my editor to get the answer I need. It also integrates documentation for std/core with docs for crates I'm using giving me a single search of everything that my Rust code can use. The only downside compared to cargo's docs is that it doesn't document my own code that I'm working on, but it's much easier to remember the details of that code.
The cherry on top is that it isn't limited to Rust. If I'm working on something with an FFI dependency, I can have the same documentation workflow for C/C++ documentation. If I'm working on a Wasm project, I can pull up HTML/CSS documentation the exact same way. All my documentation lives in the same place under the same keyboard shortcut.
rustdoc does support a hoogle-like syntax (in a rustdoc page click on the question mark next to the search field for shortcut and "search tricks"), sadly it's not very good.
Rust does add a few wrinkles due to its "type policy" being less regular though e.g. how you handle `self`, as well as the various types of references.
For instance let's say you have
impl Foo {
fn foo(&self, p: &Path) -> usize
}
I'm actually aware of cargo doc - it's really useful, indeed. It's still note quite "greppability" though; I can open `cargo doc`. I can just open the browser and google "rust write_u16" and immediately find it. I can keep VS Code open with my project and search for it there. But neither of these is greppability :)
The point is that Rust is behind Go on convenient documentation access, which is ironic seeing that Rust requires documentation access much more than Go.
go doc also accesses all documentation for packages used in a project, making it do quite a lot more than grep.
Just in case this needs an ELI5: it queries project packages to print the relevant documentation for any accessible component: “go doc container/list.List” - bar any typos from a phone keyboard - prints all documentation for the stdlib linked list. In case Rust had a decent doc tool it would be the equivalent of “rust doc std::collections::LinkedList”.
(And no, that the Go tool needs to do less work to find all relevant documentation than a Rust equivalent is not relevant to the discussion.)
Which has nothing to do with the article, or my comments. The entire issue of the comment is that Eli has a symbol whose origin they don’t know. Where the symbol comes from is 75% of the question.
Of course if you want the documentation of a symbol whose full qualification you know `go doc` gives you that, that’s what I wrote above.
But it’s not TFA’s problem, and thus not the problem anyone’s trying to solve here.
Web documentation is easy to produce and is acceptable to many. All other kinds of documentation can also be converted into web documentation, therefore you do not have to spend extra effort for it.
Terminal documentation is more difficult due to the limitations of displaying things in the terminal, therefore it requires more effort.
I certainly agree that those who prefer terminals are a subset of developers - despite this missing the point of terminal-friendly documentation also means generally reusable and integration friendly, unlike a full web application - but terminal documentation is objectively orders of magnitude less work than writing web app for showing documentation.
Terminal documentation just means plan text. You print it, that’s it. Making it pretty involves indents and line wrapping. Any source formatting is just ignored.
Text based interfaces and terminal applications came first exactly because they are simpler than web browsers and applications.
We're not talking about a "TUI" app here - a terminal doc tool just dumps plaintext and terminates.
Plus, while one can dump plain text to HTML files it's still easier to just print the text.
Rust takes forever to build too if you've never built the project before, but most people also build that ahead of time too. Repeated doc usage and repeated builds/checks don't require the "full build"
Also there's a tool (forgot the name now) that can render them in the terminal similar to manpages.
Note that there is no problem with the web version being available, it’s useful at times.
I suppose it would be nice if rust's docs supported a full text search.
Edit: tpying is hard
It's probably mentioned in that issue, but I'm sure I read recently they want to crate-ise `std`; at which point presumably `cargo doc` would show it like anything else, wouldn't even know the difference? (That was even the motivation iirc, just for a different part of cargo, not doc.)
But ad-hoc polymorphism is just oh so useful.
I'm... not sure what you'd want to happen when there are literally 5 methods called write_u16, how is a general-purpose search to know which one you're looking for without more information? If you want contextual matching... use an IDE (or hook rust-analyser into your editor of preference).
(also there are way more than 5 matches in total as there are methods like `write_u16_into` which would also match at a lower rank).
So simple ctrl-f in the file would do (or the call site would identify it).
foo().bar().baz()
If `baz()` is a member of a trait, however, that trait must be in-scope.
I think instead we should focus on continuing to push the tooling forward. Language servers were an important step in this direction, freeing us from comprehensive IDEs, but I think that's just the tip of the iceberg. More recently we have Tree Sitter, and GitHub at least can do some basic semantic analysis of code right in their UI. How can we make it even easier to build tools that understand a language? And what new interfaces can we put on top of them? What about a CLI that lets you grep code at an AST level, maybe showing the types of the expressions it yields? What about a declarative format like TextMate that lets you describe the basics of how a type system is wired, so you don't have to craft a whole language server by hand?
Maybe there's also something to be said for designing languages to be "tooling-friendly", even if that doesn't mean "as plain-text". Lowering the bar for writing tooling that can have a semantic understanding of the syntax and types, etc.
There's lots to be explored here, in my opinion
Which diff/merge tools support language servers currently?
i.e. how do you thoroughly code-review changes without switching between a decent diff program and an IDE continually?
Yeah, sure Rich Editors support the "basics" of diffing, but for complex (i.e. 3-way) merges, I find them pretty limiting: stuff like Beyond Compare, p4diff are more useful in those situations IMO.
I'm not familiar with those other tools, but it's not hard to imagine they or similar tools could one day add rich language integration using language servers, tree-sitter, or something else
edit: here's the GitHub thread https://github.com/rust-analyzer/rust-analyzer/issues/4558
Not ideal, but still way better than just not doing anything.
For rust-analyzer it's blocked on improvements in Chalk.
There is no way to add methods to a type in Python that mypy understands. You have to create a new type. This leads to code that is very difficult to extend and pushes you towards inheritance.
I talk a bit about this here: https://insanitybit.github.io/2020/07/19/intersection-types-...
For me, this became a huge problem when I was building a plugin system in Python. I wanted types to only own their own logic, but I also wanted types to be able to tell other types about themselves. For example, we have a Process and a File. I want to be able to go from Process to File with only one of those knowing about the other. Without extensions you are forced to create ugly workarounds.
Also, I've never had a problem with intellij and extension traits. I kind of wonder why you'd stick with a tool that's a bad experience.
>>> class A:
... def foo(self):
... print('foo')
...
>>> class B:
... def bar(self):
... print('bar')
...
>>> A.bar = B.bar
>>> A().bar()
bar
rust-analyzer supports vim/emacs as well. in vim i can put my cursor over a symbol and get all the information id get from an ide via a quick shortcut. a feature-rich vimrc for rust can be seen here: https://github.com/jonhoo/configs/blob/master/editor/.config...
> You check the documentation of that crate and indeed, you find the write_u16 method there; phew.
Like, all wild card impl. traits are in the documentation, including such which are dereferenced.
EDIT: Just to be clear if you pull in other code, including potential traits, you need to open the documentation specific to your crate (cargo doc).
But the real message here is if you're relying on regular expressions to implement IDE functions, plugins, language syntax checking, auto completion and so on, you're going to have a bad time. Period. This is the real weakness of VS Code IMO.
An IDE that operates on the syntax of the language is infinitely better than one that relies on regular expressions and simply treating source files as "text".
This is one reason I will always use Jetbrains IDEs given any choice because I know I can hover over a symbol and it'll tell me where it comes from. "Go to definition" will just work.
As soon as you start designing your language practices around the limitations of what regexes can do you're going to have an even worse time. For example, in Hack (@FB) we couldn't use namespaces or trait aliasing. You then had people asking "why are people creating all these abstract final classes with static methods in them?" when the answer is obvious: because they can't use namespaces).
My experience using C++ with VS Code was just horrible because the IDE was unreliable when it came to pointing out syntax or type errors. So you could waste your time compiling something that doesn't compile. I've never had this issue with CLion, for example.
https://marketplace.visualstudio.com/items?itemName=rust-lan...
Instead you should use rust-analyzer which will soon become the official rust extension.
https://marketplace.visualstudio.com/items?itemName=matklad....
What you really want is to install the rust-analyzer plugin, not the Rust plugin.
I find this comment very confusing. Neither plugin was developed by Microsoft.
The "rust" extension generally comes up first because it's older and has significantly more downloads. That being said, it's true that imho everyone should use the new one.
People usually complain about Nim's import behaviour which is roughly an `import *` in other languages, making it hard to immediately tell where things come from. Personally, I find it fantastic because it removes a lot of boilerplate. In case of conflicts, the compiler throws an error, forcing you to qualify your import.
Now in Rust, you explicitly need to import the trait in order to use it because there _might_ be naming conflicts. Here, the 1% case spoils the 99% case.
fn magic_num(&self) -> usize {
return if self.name.len() == 0 { 2 } else { 33 };
}
fn magic_num(&self) -> usize {
if self.name.len() == 0 { 2 } else { 33 }
}I guess here it doesn't matter as much since it's a simple enough cases, but I'm just talking in the general sense.
With that said, I think is_empty probably has very slightly smaller overhead when it comes to figuring out the author's intent. Because, well, that was the intent. Length checking against 0 is merely a method of that intent.
Also, this "intent decoding" while reading code (no pun intended) adds up in mental cost, IMO. So, it wears down the the reader/reviewer attention more quickly.
> Some structures can answer .is_empty() much faster than calculating their length. So it is good to get into the habit of using .is_empty(), and having it is cheap. Besides, it makes the intent clearer than a manual comparison in some contexts.
It also has a lint for implementing both methods if you’ve only defined one, to help perpetuate the convention.
I'd even say any return is actually one too many.
If !validation_rule1 { return default/err }
If !validation_rule2 { return default/err }
Work…
If !validation_rule3 { return default/err }
Work…
return result/success
That pattern is addressed by chained not nested conditionals, which can then be wrapped in a match, i.e.:
if !validation1 { err1 }
else if !validation1 { err2 }
.
.
.
else {
...work...;
success_result
}
match validators.find_map(|v| v(data)) {
Some(err) => err,
None => { ...work...; success_result }
}
I missed the intermediate work steps in your post when I initially wrote the response. That can be done a number of ways, e.g., breaking things up into to chained steps (each of which might use the above pattern) that mix validation and work and return a Result type with either the data for the next step to consume or an error.
(validation_rule1')
.and_then(validation_rule2')
.and_then(work1)
.and_then(validation_rule3')
.and_then(work)What the author wrote is perfectly valid, fine code. Please stop insisting everyone does something one way or another when there's no clear advantage here to doing so except for the lack of single bloody keyword.
For many of these formatting/stylistic lints, there are valid arguments to be made for an alternative choice. I personally often disagree with one or both tools. I still follow them.
The huge benefit here is that most Rust codebases look identical. I can open up burntsushi's regex crate and see code that is formatted and styled exactly like mine. It makes reading new code a lot easier.
/me raising hand
I've used all the major IDEs on real projects for years at a time each, and so done years (often on the same projects) with just a quality editor and a set of external tools, and I honestly slightly prefer the latter.
I'm effective both ways, but my observation is that my work is of better quality when working in a non-IDE environment.
I fully understand that, as with most such things, says as much or more about me than about IDEs.
I think a large element is that, not having auto-complete, I find myself reading more docs (increasing system familiarity) and noticing unergonomic interfaces sooner.
The one feature I do miss is language aware navigation, but that loss is somewhat balanced by the lack of disruption when said navigation inevitably breaks, due to non-compiling code or working with new language features not yet supported by the tools.
* Vim or VSCode - essentially zero plugins in both cases - format on save using google-java-format - syntax highlighting turned off in Vim, minimized in VSCode - code folding disabled - quickfix mode in vim and error display/navigatin in VSCode set up * maven, gradle, or ant to build * browser open to JavaDocs for stdlib and any library I happen to be using (though I work very hard to minimize dependencies) * JMC, VisualVM, JitWatch, async-profiler, etc. for profiling
I do occasionally still fire up IDEA to use the debugger or other tools, but rarely.
As for renaming methods/variables, for the latter, scope will almost always be small, so rename is usually trivial. For methods, it's almost always easy enough to rename, rebuild, then fix the errors.
I also typically use Vim for C programing, though there I often use cscope and/or ctags to somewhat smooth out navigation.
Again, I do not have a problem with good IDEs, and still occasionally use them. I just don't feel a need for them, and often prefer working in a plain editor.
I can't find it now, but I remember an RFC that proposes a solution to this, although in the opposite direction (i.e., more traits in the standard prelude). It would be interesting to see a `use` variant in a future edition of the language that allows users to bring in specific parts of the trait's contract, e.g. `use std::io::Read::read_exact`.
Or better yet, the development experience of using Toga in Mesa/Cedar (1987).
http://toastytech.com/guis/cedar.html
https://www.youtube.com/watch?v=z_dt7NG38V4
Or Smalltalk, Interlisp-D or Lisp Machines for that matter.
The time a language must be usable with Notepad alone is long past due.
One concrete issue I ran into is that when cleaning up a list of imports, I can't use Ctrl+F to find out which ones are unused, since I can accidentally remove a trait which is never used by name but only imported for its methods. And I can't count on rust-analyzer's unused import detection working 100% of the time. And it's bad that some methods (for example the infamous `Read` or two `Write` traits) are inaccessible unless you add a trait import (which rust-analyzer sometimes knows how to add when I press Ctrl+.).
To me, the overarching value of Rust is locality of reasoning, and clearly indicating nonlocal reasoning in explicitly-readable and machine-checkable ways (eg. the `Cell` family of types). Trait methods are nonlocal reasoning, and they aren't clearly demarcated (a trait import behaves as`use module::Trait::*` but isn't written that way in the source code). If you're opposed to wildcard module name imports, you should be opposed to wildcard trait method imports.
IDEs for other languages provide plenty of capabilities to code navigation, no need to go Ctrl+F.
onestly, as an old-jeezer, I still don't understand the hate on IDEs.
Who in their right mind has the courage to tackle ANY project without an IDE? I constantly see my collegues insist to use vscode without plugins, then fork or COMMANDLINE GIT (ugh..) then external tool for viewing jsons, and explicitly use postman and whatever tool...
and then fail to understand the very basic fact that an integrated collections of tools works better for the end user.
...I... I don'k know...
Unfortunately I had to develop on a remote machine (because compiling rust in my local machine is too slow). Jetbrains solution for remote development just didn't work for my use case (the remote was a M1 ARM running MacOS, not supported by jetbrains gateway). So I switched to vscode. Vscode editor is great, the support for remote development is also very good. Unfortunately rust-analyzer is not as good as jetbrains's rust plugin.
This is a problem with IDEs: you're either all in or you're out. Language servers did help decoupling language support from the actual window where you type stuff, but still not everything is in a language server (e.g. jetbrains rust plugin is not a reusable language server)
I used MS-DOS since version 3.3 and my first UNIX in 1993.
XEmacs became my refuge as there were hardly anything resembling the PC, Amiga, Mac,... IDEs.
So 30 years later we have IDEs everywhere we get this anti-IDE attitude, some kind of CLI revivalism.
Two things that can improve this are preferring Trait::method(obj) notation and importing traits explicitly rather than relying on crate preludes.
> preferring Trait::method(obj)
… vastly improved my understanding of trait usage.
I think your suggestions do a good job addressing that; could be a thing for a style guide.
FYI, Sourcegraph supports LSIF generated by rust-analyzer. For example, https://sourcegraph.com/github.com/matrix-org/matrix-rust-sd... shows lock method belongs to tokio. It should be able to show origin of write_u16 as well (In the precise mode, not search based).
Gitlab can also consume LSIF, and github should do it someday, IMO, their current way doesn't scale. In most languages you should implement a full compiler front end in order to provide a 100% correct goto definition.
Perhaps the general design principle should be to tend towards one extreme or the other, rather than end up with a language that needs an IDE but doesn’t leverage it fully.
For example for this code:
trait Foo {
fn foo(&self) {}
}
struct Bar;
impl Foo for Bar {}
mod test {
use super::Bar;
fn test() {
Bar.foo();
}
}
22 | Bar.foo();
| ^^^ method not found in `Bar`
|
= help: items from traits can only be used if the trait is in scope
help: the following trait is implemented but not in scope; perhaps add a `use` for it:
|
20 | use crate::Foo;
|- I'm already using 'cargo check' as my linter (rustc doesn't understand modules, seemingly).
- My setup works for JavaScript/React/Svelte, SQL, C/C++, Python, and Go.
- It can't be the case that (Neo)Vim + Ale [0] is arcane. Also the link in "Tools" from rust-lang.org goes to the rust.vim repo, which defaults to Syntastic, which I'm pretty sure has the same issue.
I don't necessarily think I should extrapolate from my experience to a broader "Rust is different for little/no gain" critique I've made in the past, but I sure want to haha. And to kind of anticipate a little, I wouldn't blame Ale here because--again--it works for everything else.
But anyway, yeah I've mostly given up on linting and I just run 'cargo check' in a terminal buffer now. It's probably something as simple as adding a command-line argument to... idk cargo or rustc to make the additional "help" format a little more amenable to Ale? Maybe I'll take a crack at it when I get some time.
I did find this blog post, which seems to suggest there may be a bit of extra config required to get rust-analyzer going with ALE: https://petermalmgren.com/rc-batch-day-9/
Good luck!
One of the things I've really enjoyed about the Free Software ecosystem is choice, like mostly before Rails the attitude was "we don't presume to know the best anything, here are some options." I'm not saying there aren't tradeoffs (complexity, bitrot, 7 half-baked options vs. 1 incredible one), only that I kind of low-key resent the "opinionated" software that's out there. I guess I don't really see it as opinionated (like I think Rails' convention over configuration was actually a pretty good idea and exemplifies the "execute" phase of the explore/execute cycle of tooling), I see it as "flashy and targets the 80% use-case to get mindshare".
To be clear, I don't think rust-analyzer is that opinionated or flashy or lazy. I think it's awesome. But Rust's linting story is essentially "why would you use anything besides rust-analyzer, language servers are awesome and have no downsides." Which like, yeah they do.
[0]: https://github.com/dense-analysis/ale/blob/master/doc/ale.tx...
And yeah rust-analyzer is definitely the happy path, but if you were trying to get by without a language server (not going to lie on a very large rust project rust-analyzer’s RAM usage can get up there), I think that `cargo check` output properly parsed and displayed in your linter would probably get you a huge portion of what you need. Between that and rustfmt, I think the only thing you’d be missing would be go to definition and that kind of thing (but you can set that up with a ctags-style thing or just replace with fuzzy grep).
In fact, before I figured out how to get rust-analyzer working in emacs, I used a setup where `cargo check` ran on save, and the output showed up in flycheck. From what you said about ALE, it sounds like you could set it up the same way. The main thing would be making sure there’s a way to get the FULL output of `cargo check`, probably not inline because there tends to be a lot of it. But e.g. with emacs I get error underlines that show the first part of the error, and then I can expand that in a popup window, or run cargo check, which opens cargo check in another buffer, with the output parsed so that if I press enter on an error, it brings me to the location in the code.
All of which is to say that it is doable! Maybe it would just take some extra ALE configuration, since the defaults are probably oriented towards people who are using rust-analyzer.
I use rust-analyzer in emacs, but it’s available in any editor that supports language servers.
With my setup, I’ll start typing a method, get a completion for the trait method I want with the trait indicated, and select the completion. The trait is then automatically imported and the method is completed.
`import Foo.Bar ()`
Which looks like it's just importing nothing at all (`()`) but is actually pulling in some traits and nothing else.
You don't see this often because "orphan instances" (instances defined in a file besides the location of the trait definition or the type definition) are discouraged, but they do occur.
I had also thought that there was a reason one could need that pattern to import instances even in the absence of orphans, but on reflection I think I was mistaken.
Edit: After thinking about it. This only really helps inside a given codebase.
What would stop someone from creating a trait that looks similar to an existing function name, but that does something bad?