Rust 1.32 released
blog.rust-lang.org
blog.rust-lang.org
The default removal of jemalloc should greatly cut down on binary size as well, if I understand correctly.
I've dabbled in Rust, but haven't had a major reason to use it for anything yet. I'm looking forward to when that time comes.
In Rust, everyone is a print debugger. The only thing that really goes wrong in normal code (once it compiles) is "why is this value not what I expect?". Dropping down into GDB is way overkill.
Thank you for sharing your hacks!
Anyway, I'm super curious what someone is doing with Rust and Zelda. How do I learn more without Twitter?
Looks like this might be it? and then maybe I'm just missing out on images and videos that are only in some ephemeral tweet? https://github.com/CryZe/WindWakerBetaQuest
[1] Things like custom menus https://twitter.com/CryZe107/status/1026355408343126017
Playing around with physics https://twitter.com/CryZe107/status/991389826002931714
Implementing Super Mario Odyssey-style snow https://twitter.com/CryZe107/status/991104446812819456
Rendering the Rust logo into 3d namespace https://twitter.com/CryZe107/status/990644091963756544
You write tests? Consider selecting variables of interest (and printing them to STDERR when debug mode is on) like one of many test.
Looking at memory and all the variables has its place, but as you said only "here and there" - because when you have to do that, you have already lost: you are looking for a needle in a (hay)stack, and will lose much more time that just eyeballing the variables of interest you selected before.
The best description I've heard for master-level debugging is that it's a process of narrowing down the problem space as cheaply as possible. Your brain is telling you that based on everything you 'know' about the code, the right answer should come out. If the wrong answer is coming out, something you 'know' is wrong.
After the most obvious failure mode doesn't reveal the problem, your next check may not be the second most obvious failure. Instead you're multiplying the cost of verifying an assumption times the likelihood it's correct times the 'area' of the problem space it eliminates. Checking things like "is it plugged in?" sounds stupid but brings down the worst-case resolution time by hours.
Long story short, let's say I'm sitting in an interactive debugger looking at a stack frame, expecting that a particular variable has the wrong value, but it's fine. The cheapest thing for me to do next is to look at all of the neighbors of the suspicious value, and those in the caller and on the return. With println, pretty much every subsequent check costs the same amount as the first one. And if there's no short path from starting the app to running the scenario, that cost could be pretty high.
If you believe that you have a high success rate on your first couple of guesses, then println works great for you. But what if you're wrong? Have you ever tracked how many attempts it usually takes you? Or are you too wrapped up in the execution to step back and think about how you could do better next time?
Also, I want to be clear that I'm not telling anybody how to debug, as long as you aren't making that choice for your whole team. Don't choose tools or code conventions that break interactive debugging because "println was fine for grandpa so it's good enough for me!" That's a big ol' case of Chesterton's Fence.
Having experienced the higher plane of fully integrated IDE / run / debugging with arbitrary expression evaluation, conditional breakpoints, etc, I can't even imagine how anyone could work with "print debugging".
I find that in most cases it's easier for me to figure out what's going on, because I can quickly scan a log of how different variables changed over time, instead of having to step through one step at a time.
* Rust code can have memory errors + undefined behavior, because Rust code can say "unsafe". Plenty of real projects use "unsafe". (Alternate reason: because the compiler has soundness bugs.)
* Memory errors + undefined behavior aren't the only reasons people like debuggers. Consider: there are plenty of other memory-safe (GCed) languages in which people find debuggers useful (such as Java). "The only thing that really goes wrong in normal code (once it compiles) is 'why is this value not what I expect?'" is arguably true there as well.
And, for the record, gdb works decently well with Rust code. Not perfectly (yet) but well enough to be useful. I have tried it (although I'm more of a printf debugger myself).
It's incredibly useful to limit the regions of unsafety and use them to build reusable, well-tested safe abstractions, but it's a mistake to confuse that with eliminating unsafe entirely or ignore the possibility there could still be errors within them.
I'm willing to bet that the vast majority of Rust code (outside of std) is safe. I've written unsafe once ever, in years of writing rust.
I agree that it's unfair to generalize that debuggers have no use in rust, but it's fair to generalize and say that most rust developers do not experience segfaults, or other memory corruption issues that often call for a more advanced approach to debugging.
It's very much about project choice. I immediately ran into unsafe trying to test some functions marked extern. Then again writing toy VMs and GC algos.
You may not use the debugger often, but it's there if you need/want it, which is an important message that I think is lost with "all Rust programmers are print debuggers".
Congrats on only using unsafe once in years. That's pretty neat.
Don't get me wrong, they're handy but I find them much more useful for stepping flow than root-causing errors.
Also if you're dealing with race conditions the only way to safely root-cause to to stash away data somewhere in mem and print it later as flushes/fences/etc change behavior. Debuggers make that even worse.
Love my debuggers for behavior issues but each tool has it's place.
I'm not sure how Rust's support is here, but in my experience it's the exact opposite. Debuggers with var-watch or conditional breakpoints can do this (and a heck of a lot more) on the fly, and that's almost always faster than re-compiling and running. Even at the extreme-worst case, you can be a print-debugger with a debugger without needing to rebuild each time, just re-run.
X86 is pretty orderly so you usually don't see that class of bugs until you start getting on other architectures but when you do man is it nasty. C/C++ volatile comes to mind particularly. MSCV makes it atomic and fenced which isn't the case pretty much anywhere else.
Also debuggers don't help you with the 2nd/3rd order effects when you need to trace something that's falling over across 5-6 different systems. With print based debugging I can format + graph that stuff much faster than a debugger can show me.
Like I said, different tools for different uses. It's just important to know the right tool so that everything doesn't look like a nail.
Yes, that is definitely true. But so does calling a printing func that does IO, since it often involves system-wide locks - I'm sure many here have encountered bugs that go away when print statements are added. But debuggers are definitely more invasive / have stronger side effects, and have no workaround, yea.
Multiple systems: sorta. Past (legitimately shallow) multi-process debugging that I've done has been pretty easy IMO, you just add a conditional breakpoint on the IPC you want and then enable the breakpoints you care about. Only slightly more complicated than multi-thread since the source isn't all in one UI. Printing is language agnostic tho, so it's at least a viable fallback in all cases, which does make it a lot more common.
---
To be clear, I'm not saying there's never a need for in-bin "debugging" with prints, data collection of some kind, etc. You can do stuff that's infeasible from the outside, it'll always have some place, and some languages/ecosystems give you no option. Just that it's far later than most people encounter, when a sophisticated debugger exists. E.g. printf debugging in Java that I encounter is usually due to a lack of understanding of what the debugger can do, not for any real benefit.
> the only way to safely root-cause to to stash away data somewhere in mem and print it later as flushes/fences/etc change behavior.
Dude, I literally called that out in the root post ;).
I think a lot of debugger vs printf-style debugging is a matter of preference and familiarity. I'm used to debugging embedded or distributed systems where debugger support is not so great, so I've gotten used to other techniques (including printf-style stuff). But a lot of people love using debuggers, and I find it elitist to tell them they're wrong.
You'd be impressed with the power of formatted printf + excel. Solved some fun issues like quaternion interpolation normalization via graphing and the like.
1.) Ability to see the value of every variable in scope without needing to decide a-priori which variables are worth looking at.
2.) Ability to traverse the call-stack and identify at what point a computation went wrong without having to instrument every single call & variable.
3.) Ability to interactively try out new code within the context of a stack frame. When I find a bug, oftentimes I'll try 3-4 new approaches just by entering watch expressions until I find an algorithm that works well on the data. This would take 3-4 full runs without the debugger.
4.) Ability to set conditional breakpoints and skip all the data that's working properly, only stopping on one particular record. When your loops regularly have 100k iterations before they fail on one single iteration, that's a lot of log output to sift though (or a lot of unnecessary loop counters & if-statements) for a rarely-encountered case.
The comparison with Java is interesting. With Java, I have often found that errors occur in a rather non-local fashion, due to dynamic code loading, confusing inheritance trees, and ubiquitous mutations and what have you. Maybe I'm not actually calling the function I thought I was, maybe because I have actually received a subclass of my expected class. Print-debugging is often too narrow to highlight the cause. In such a situation, I would fire up the debugger and inspect the general state of the application (which Java makes relatively easy to do).
In contrast, in Rust things tend to happen in a very constrained fashion. You can't randomly mutate things, you can't (without considerable effort) make complicated graph structures where everything can touch everything else. With the occasional exception of highly generic code, your call sites and function arguments are exactly what you expect. So I can rely on print-debugging to quickly find the cause of my problem.
Incidentally the same is true with Haskell, moreso even, except due to laziness the evaluation order can be harder to ascertain - debug statements can appear in a strange order (or not at all).
You are being overly-pedantic in your interpretation of the comment you are responding to. It isn't claiming that we have absolutely no undefined behaviour or memory errors in rust. The point is that undefined behaviour and memory errors are rare in Rust development, so tools intended to help find memory errors are just a lot less useful.
Your second point is spot on.
Perhaps everyone being a print debugger in Rust is less a compliment to the language, but a criticism of the tooling. I absolutely adore Rust, but understand there are still some vast gaps in the tooling.
Maybe that is a missing niche of the market; a debugging protocol similar to the language protocol used in VSCode (RLP in Rust provides this).
Then the IDE could integrate with any language and debug it, regardless of the details on how the language functions. And it can provide a better UI than GDB (which isn't a high bar, it's more like trying to dig down to find the bar because GDB UI is horrid)
An issue that Rust has had to overcome is that it produces bloated binaries by default. Hello World type applications are hundreds of kilobytes long, because they ship jemalloc and libunwind. Removing one of those helps.
Weakness is basically binary size and code complexity.
I think the big selling point of jemalloc when it appeared was that it was much better for multithreaded applications. But since then glibc has improved a lot in this area, and nowadays has a similiar design with per-thread pools etc.
Does any other language have a similar feature?
The dbg! macro is definitely inspired by https://hackage.haskell.org/package/base-4.12.0.0/docs/Debug....
I haven't seen a debug function that returns the value again anywhere else.
https://github.com/arclanguage/anarki/blob/master/arc.arc#L1...
Edit: Yes, it's in Common Lisp too, so it goes back at least to the 1980s and probably further.
http://www.lispworks.com/documentation/HyperSpec/Body/f_wr_p...
.tap { |x| puts x }
values.map { |x| x + 2 }.select { |x| x > 3 }
[0, 1, 2, 3].map { |x| x + 2 }.tap { |x| puts x }.select { |x| x > 3 }
Also, if I want to pause at a point and step from there, just drop `import ipdb; ipdb.set_trace()` at the line I want to set a breakpoint.
With good support, a couple of trace points, even on a live running instance is all that is needed, without any extra recompiles.
More devs should learn about JTags, IntelliTrace, DTrace, ....
No need for manually writing printf-debugging and recompiling all the time, when the debugger can do that for you.
Prints a YAML dump of "my str", followed by file/line number information, then returns its argument so it can be embedded in expressions:
$ perl -MXXX -E 'say uc(WWW("my str"))'
--- my str
...
at -e line 1
MY STR
dbg!(n * factorial(n - 1))
[src/main.rs:5] n * factorial(n - 1) = 2
[src/main.rs:5] n * factorial(n - 1) = 6
[src/main.rs:5] n * factorial(n - 1) = 24
...
So, I just looked it up, and I found it. Actually, Damien wrote two. One that he updated from 2014 to 2016, and one he started in 2017 and has maintained to the present. I have no idea the reason for that.
Yep leave it Damien. I remember he did something similar for a test module where it would show the expressions that were evaluated in the test failure output.
Except you often don't want a full execution, you often just want a partial execution where you suspect the problem arises. I can assure you that a good UI debugger is extremely helpful. Command line debuggers less so.
This extra work pays off pretty quickly, though. When I have a bug, I find a selection of tests that works with that code, add a couple of printf-equivalents and then rerun the tests. Usually I can spot the error in a couple of minutes. Being an older programmer (I worked professionally for over 10 years before Beck published the first XP book), I'm very comfortable with using a debugger. However since I started doing TDD, I have never once used one. It's just a lot faster to printf debug.
The way I've explained it before is that it's like having an automated debugger. The tests are just code paths that you would dig into if you were debugging. The expectations in the tests are simply watch points. You run the code and look at the results, only you don't have to single step it -- it just runs in a couple of seconds and gives you the results.
You may think that the overhead of writing tests would be higher than the amount saved with debugging and if it were only debugging, I think that would be true. However, one of the things I've found over the years is that I'm actually dramatically faster writing code with tests compared to writing it without tests (keep in mind that I've got nearly 20 years of TDD experience -- yes... I started that early on). I'm pretty good at it.
The main advantage is that when you are writing code without tests, usually you sketch together a solution and then you run the app and see if it works. Sometimes it does pretty much what you want, but usually you discover some problems. You use a debugger, or you just modify the code and see what happens. Depending on the system, you often have to get out of the context of what you are doing, re-run the app, enter a whole bunch of information, etc, etc. It takes time. Debuggers that can update information on the fly are great time savers, but you still have to do a lot of contextual work.
It takes me some extra time to write tests, but running them is super quick (as long as you aren't writing bad tests). Usually I insist that I can run the relevant tests in less than 2 seconds. Ideally I like the entire suite to run in less than a minute, though convincing my peers to adhere to these numbers is often difficult. That 2 seconds is important, though. It's the amount of time it takes your brain to notice that something is taking a long time. If it's less than 2 seconds (and run whenever you save the file), usually you will barely notice it.
In that way, I've got better focus and can stay in the zone of the code, rather than repeatedly setting up my manual testing and looking at what it is doing. Overall, it's a pretty big productivity improvement for me. YMMV.
> [src/main.rs:4] x = 5
How to differentiate the value of 'x' for a given thread/instance/whatever? Don't add the info by hand to the debug message.
edit: typo
That just means your debugger has a prohibitively high cost to use. If it takes more than 2 clicks to launch a full debugging session of your project, you need a new IDE.
With printf debugging, you can put print statements everywhere you think something might be wrong, run the program, and quickly scan through the log to see if anything doesn't match your expectations.
With print-debugging if you find a bug you have to stop your app, insert print lines, recompile, redeploy, relaunch, click through your app to reach buggy location and then scan through log. This really feels like stone-age once you've ever used a IDE.
Sometimes, print debugging is the only practical way to fix a bug. For example, very rare bugs which can only be reproduced by running many instances of the code for a long time, or situations where attaching a debugger is not feasible (as in live services).
Never used Alpine containers, so no idea how it works with DTrace / SystemTap.
However a quick web search revealed the following right away, surely there are other results available.
"Systemtap for CoreOS Container Linux"
https://medium.com/makingtuenti/systemtap-for-coreos-contain...
"App Trace Roll: Users Guide" . For Red Hat Linux clusters
https://docs.huihoo.com/rocksclusters/app-trace/4.1/index.ht...
I would hope at some point you would get to the point where you move past this, once you get to Enterprise scale print debugging stuff would be slow and laborious.
Do you mean someone could get confused because they think 1 byte has endianness? Well then maybe they shouldn't touch these functions since they don't understand what is endianness.
It's weird that they have these methods on a u8 though. Probably just comes automatically because it's an integer type so they prefer to have a consistent API?
It seems to come from a macro used to define those primitive int types: https://doc.rust-lang.org/stable/src/core/num/mod.rs.html#38...
It isn't unheard of for 1 byte to have endianness. To be specific, we can call it "bit endianness". It matters when serializing bits to go over a wire, such as when bit-banging I2C or SPI.
An array doesn't normally have endianness either. I guess you could have a programming language that does the indexing backwards, with 0 at the end of the array. I've never heard of such a thing existing.
Compare this with a endianness functions typically used in C. We get ntohl for example. The size is specified by the "l", the argument is of type "long", and the return value is of type "long". No bytes are involved in that interface.
Compare with what the Linux kernel uses. Again, no bytes are anywhere to be seen. Functions like cpu_to_le32 take and return 32-bit values.
It looks like Rust is doing things Python-style, which is a scary thought. Instead of providing distinct ways to change endianness and to interpret data as integers, the functionality is crammed into one interface.
Except it's not. These specific methods aren't some grand interface to change endianness. These are convenience routines for converting between bytes and integers, which is a not altogether uncommon thing to do. I certainly do it a lot.
The existence of these byte-to-integer conversion methods does not imply the non-existence of other methods to convert endianness within integers. Indeed, those methods have existed since Rust 1.0: `swap_bytes`, `from_be`, `from_le`, `to_be`, and `to_le` are all methods defined on {integer type} that return {integer type}.
The fact that we typically refer to byte ordering doesn't mean the others are not also endianness.
I read somewhere in all the discussions that Carl was hesitant to go all-in on futures 0.3 (those are definitely landing in std, right?) in tokio before some additional ergonomics had landed, possibly some language feature among them but I may misremember...
Do you know the list of things and possibly their tracking issues? Futures in std, async/await, etc, something something more? Would love to be able to keep track and follow along :)
> (those are definitely landing in std, right?)
Yes.
> Do you know the list of things and possibly their tracking issues?
I know you may not have your fingers in this particular flower pot, so the "you" was directed at the entire Rust community. However, while we're here you still get a big personal thanks -- your Rust for Rubyists got me hooked those years back and the docs/book (1st ed and then 2nd ed with Carol) work you've done since is nothing short of amazing -- and with these latest developments I hope you find a great place to continue your work <3
Show me serious embedded device mfgs using Rust. Actually using. Not some testimonial on the Rust site. The real truth is no company is going to use it until there is serious use will happen until there is IDE support. Segger is a company large enough to be taken serious and unlike Keil or IAR small enough to get it done.
Great things start with hobbiest wierdos. IMO... once you stop looking for things to be offended by you can get to work, son.
You probably won’t get a 2x boost from it. But you might notice some improvements. It’s worth trying if you can test it quickly.
Software best practices are like flossing: good in the long term, but probably not crucial to keeping your bite.
Don't get me wrong, dbg macro isn't one of them. Heavy user of print debugger here.
I'm also quite happy with the uniform path work that's landed. Rust now has easily one of the best namespace management's I've used in a programming language.
Cheers!
The URL template for normal rust installers is:
* https://static.rust-lang.org/dist/rust-1.32.0-{TARGET-TRIPPLE}.{EXT}
* https://static.rust-lang.org/dist/rust-1.32.0-{TARGET-TRIPPLE}.{EXT}.asc
* https://static.rust-lang.org/dist/rust-std-1.32.0-{TARGET-TRIPPLE}.{EXT}
* https://static.rust-lang.org/dist/rust-std-1.32.0-{TARGET-TRIPPLE}.{EXT}.asc
The file extension for auxiliary target installers is `.tar.gz` (or `.tar.xz`) for all targets including Windows.
Note: Due to a known bug, browsing the complete list of all installers is not available on https://static.rust-lang.org. It is however still possible to access dated repositories via the following URL template:
https://static.rust-lang.org/dist/YYYY-MM-DD/
Installers for the current stable release of rust can be browsed at https://static.rust-lang.org/dist/2019-01-17/
Cheers!
Yes, for multi-target (eg lib + bin, ..) it can be a little more work to look it up.
But the new system is SO much more convenient to use overall.
To give a concrete example, please look at this [1] Cargo.toml and this [2] example and try to figure out the crates its using. I've highlighted them in Cargo.toml, to show how irregular the use is [3].
We don't have test-only or binary only dependencies yet. They would make this a bit easier.
I want to focus on the code I'm writing, not onto figuring out which crates I need to include. On the bright side though, the system is still better than what C++ got which is not knowing from which header a particular symbol came from :).
[1]: https://github.com/djc/quinn/blob/e555d11a430b9760d149a27659...
[2]: https://github.com/djc/quinn/blob/e555d11a430b9760d149a27659...
[3]: https://gist.github.com/est31/7cb33e8a6b63c8798381bdb5e91a14...
I also wish we could specify /examples or /tests specific dependencies in Cargo.toml.
A question here is though if such a complex structure is so rare that making it easier for the 90%+ of crates is worth it, while inconveniencing the complex cases.
x = foo();
invoke_debugger(); // An interactive console appears here.
y = bar(x)I'd expect something like this: https://github.com/krixano/Lydrige to have it easy in this regard - it's a statically-typed but interpreted language.
Well, the trouble is knowing if you need to spawn a debugger. On Windows, there's a nice API for checking if you're being debugged (IsDebuggerPresent). On Linux, you have to read /proc/self/status to see if there is somebody running ptrace on you. On other Unixes, well, the only trick is trying to spawn another process to ptrace you and hope it works.
It is possible, just tricky.
If you have JIT debug enabled, you will be prompted to attach your debugger to the process when hitting such a breakpoint.
Mostly, if I want a break somewhere I do this:
*(int*)0 = 1;
It's easier than adding a __asm(int3) because it's cross platform. On windows for example it's not possible to perform inline assembly in x64 builds, and compiler intrinsics are not portable.
> literal mactches against literals of any type;
... of course it is. Noted for next time ;)