What a better Rust would look like
kerkour.com
kerkour.com
The case sensitive visibility/privacy sound terrible to me. I dislike having to type pub as much as I do, but that seems worse for readability and clarity on top of throwing a wrench into the established rustfmt naming system.
About "Memory management", they already made some progress with non-lexical lifetimes in the past which did indeed improve the end user experience. I think "Polonius" aims to reduce the friction further, but I haven't looked into the specific improvements it might bring in user experience.
That's an optimistic way of thinking about. I feel hesitant to pull in libraries because you don't know when or why the community is going decide to change. Then your that fool using that old code, stuck with sudden tech debt.
It also makes stuff like stackoverflow weird to use where you get different answers depending on the year. I think the biggest problem is with libraries that try to make things more ergnomic. Learning about pin and how to use it is weird, everyone keeps recommending some higher level library.
Instead, I think every language should have a list of batteries-suggested libraries, with a roughly 2 year turnover period. This way you get a community suggested group of the common functionality needed to be productive. It won't change so frequently that you're constantly looking up function names, but it also won't get sclerotic, and can move from old grungy libraries to new libraries when necessary. Of course, you can always just stick with an old "edition" of the community libs if you've got a legacy app that you don't want to migrate.
It's wildly inefficient (mostly due to reflection), not type safe, and somewhat surprising. Guess the output https://go.dev/play/p/lSyiaN3IYOR
The author also mentioned "Images manipulation" as a thing that should exist in the standard library.
We actually know how that one went from go too: There's the stdlib image/draw (https://pkg.go.dev/image/draw). But generally, it's recommended to use the non-stdlib golang.org/x/image/draw package instead these days (https://pkg.go.dev/golang.org/x/image/draw)
For backwards compatibility reasons, the stdlib couldn't change, and it's actually far more confusing as a result.
I take a much different lesson than the author of this post about what lessons to take from Go's "large stdlib" experiment.
Personally, i think that this is simply a problem of not being able to throw the entire thing away, learn from the mistakes and make a better iteration.
I'm all for "batteries included" anything - be it a standard library for a programming language, an IDE with its own component libraries like Lazarus, a web development framework like Angular that's less fragmented than React or maybe even an operating system that has a lot of the software that you'll want to use preinstalled (or available on a DVD or a similar medium directly).
It's just that making sure that those batteries are good is impossible without iteration. Having to do backwards compatibility essentially makes that sort of iteration impossible. So what's the solution here?
Have a clearly defined end of life for the current version and a new major version in the future. Python 2 might have had a lot of warts, but Python 3 would clearly be better! One could say the same for Python 4, Python 5 and so on. The same goes for Go, Rust and any other high level language - just look at Java 8 and the newer iterations. You just need to learn to throw old things away instead of keeping them around in some half-alive state.
I say that as someone who's currently maintaining a Java 8 monolith that cannot be updated to anything more recent. If your software gets so large to the point where it cannot be migrated/rewritten in a manageable way, you've made a mistake. If you cannot feasibly migrate it over, then consider whether you even need it in the first place (e.g. build systems that you do less).
There are sub-optimal cases where that simply isn't possible: the Linux kernel comes to mind, as does a lot of other large legacy software that's slowly trudging along on the backs of tens of thousands of developer years. But somehow you don't see many new/existing systems using FORTRAN or COBOL and that's all for the better.
Go moved "image/draw" into "golang.org/x/image/draw". This was a good solution because it meant you could update the compiler without having to worry about also updating your image drawing stuff and that possibly breaking.
I wish they did that with http, tls, and all the other stuff that isn't really part of the language itself.
As is, I usually have to wait months to get compiler benefits (like being able to use generics, or smaller binary sizes, or fixes to bugs) because the compiler update also comes with breaking changes to the http package (like the http2 change) or tls or whatever.
My lessons is that there should be 3 tiers of libraries:
1. Compiler-versioned libraries. Things like io, syscalls, reflection, and things who's implementation depends heavily on the compiler internals. Breaking changes are never made without extreme circumstances.
2. Official, separately versioned, standard libraries. Things like golang.org/x/ or the idea behind the rust-lang-nursery. This is for stuff like http, logging, tls, crypto algorithms, image manipulation, high-level filesystem APIs, other protocols and abstractions. It is supported to use an old version of these libraries with a new compiler forever, and the compiler will never change libraries in 1 in such a way that they could break any old version of 2. You should update these whenever you can, but they may make breaking changes, and may be versioned as such.
3. The rest of the packages, stuff the community writes. These should prefer to use semver, and should prefer to use libraries from 1 and 2, but no promises are made.
I think having these 3 tiers is ideal, especially because it makes it easier to update the compiler (it will never break you), ensures perpetual support for older versions of official non-stdlib-but-still-core libraries (may make major releases that break, but the old one builds and works forever).
This is already pretty close to how C works actually, and it's great. Tier 1 is the actual language spec itself, linking libraries, etc. Tier 2 is libc. Tier 3 is the rest of the world. I really don't have to worry about how my pthread_create call in glibc will break when I update gcc because that separation is well maintained, and backwards compatibility is assured.
Python and go are both a clear step back, where I update the compiler to get some feature, and then my http server stops working even though I had no intention of updating it.
Hmm, honestly, this is a really great take. A thin OS base, with the "batteries" being optional. Or at least being to use specific versions, bits and pieces from various points in time as needed.
Of course, this could turn into an Eldritch mess, but then again, it might also lead to working software over not being able to update the stuff you want to update because the rest would break otherwise.
Overall, there's probably a lot of consideration to be made about the details, but that seems workable. Stability might still sometimes be an issue (e.g. should the language internals change, a la Python 2 to 3), though i'm not sure whether that can even be addressed all that well, at least in the higher abstraction level languages (with separate runtimes like CLR, JVM etc.).
I still think it would be good to have a rust installer where you get a set of "almost standard" libraries pre-installed. So that in companies where you have to get everything approved, you could get this package approved.
Don't understand and don't support initiatives that want to stuff it full of actual applications and stuff (like http server, async runtime) etc. Just use I don't know, something else at that point.
A large official language SDK has its own downsides too though. For my money, the .NET base class library is the best “batteries included” platform out there. It has certainly had its mis-steps over the years though. How many projects that went all-in on the original ASP.NET ( now called Webforms ) are now totally stranded on a technology island that the broad .NET community is racing away from? How about the graveyard of UI frameworks for .NET? WCF? Using the “official” solutions did not help guarantee any longevity for these technologies ( though they are mostly still “supported” ).
It is hard really to say that one approach is superior. The Rust tool chain at least makes the distributes solution easier. For some stuff at least. The lack of a baked in library might make things like use in the Linux kernel harder in some ways.
I think the larger standard library of a language like Go is appealing but Rust is a community effort and doesn't have Google scale resources to maintain a large standard library.
I particularly abhor Go style uppercase for public visibility and much prefer Rust's explicit syntax. In addition, i find Rust's module system very enjoyable and prefer it to any other language i have used once i got familiar with it.
Wanting to ship without violations does not mean you are not interested in getting your algorithm working, first. Often enough, you will abandon your laboriously borrow-safed code without shipping it, because you figured out a better way. But! you had to borrow-safe the new way, too, before you could even try it out.
Rust has already penetrated a good fraction of places that will tolerate it as it is now. To find wider usage, the user experience will have to get much better. Without finding enormously wider usage, it will fizzle. It will have fizzled: you only know after it is too late.
Fizzling is absolutely the normal fate of any new language. It takes absolutely phenomenal adoption to generate a lasting ecosystem. Even phenomenal adoption is not always enough. We once thought Ruby had legs, and Ada.
The problem is that the borrow checker isn't just a verifier of correctness. It actually impacts the semantics of a program.
For example, in javascript if you open a TCP connection and let your connection handle go out of scope, the connection will stay open. And as a result, your program will sort of just hang. In Rust, when you open a connection and your handle goes out of scope, the socket is closed automatically. How does the compiler know when your program should close the socket? It knows based on the borrow checker's analysis of your program.
If your program confuses the borrow checker, that means that the compiler can't figure out when it should close that TCP connection on your behalf. This isn't the sort of thing you can just turn off in debug mode, because programs should act the same in debug and release modes. A program which doesn't close the TCP connection is a different program from one which does.
There are ways around this - you can use types like Box, String, Rc, Cell, etc. And then just .clone() all over the place. But your code will run slowly. And thats still way more annoying to use compared to other languages. I don't know any other language which makes you write "Box" to put an object on the heap.
I hear the criticism. It is legitimately really hard to learn how to write rust code in a way that makes the borrow checker happy, especially at first. I'm still avoiding async rust after getting my feet burned by it a couple of years ago. As another commentator has mentioned, rust makes you experience all the pain up front. Its a lot of pain for new programmers to the language - probably too much.
I genuinely believe rust will never achieve the popularity of languages like Python, Go or Javascript. Its just so much harder to learn. I don't think there's anything wrong with that. Most programmers don't need to write systems software. I'd rather rust keeps focuses hard on what its good at: situations where performance and correctness are more important than development speed. We have plenty of good programming languages for quickly prototyping things out. There's nothing wrong with using them when they're the right tools.
But sometimes you have other bugs you know about that you want to fix first. Or you want to try out something else, first. As long as your builds report violations, you will know you aren't done. You might suspect one or other warning is why something mysteriously doesn't work, and try tackling that. Or you know already, and have bigger fish to fry.
Before you are done, and ship, you will certainly have fixed everything, because your release build had to complete without violations. You are strictly better off making your own work schedule choices.
I would like for Rust to be used more in places where C is today. But that can't happen if it does not get wider acceptance.
Haskell has type holes which cause a runtime error when executed as they cannot be evaluated which is an interesting approach (it also has other uses, such as figuring out types).
Hm. Or can you? It might be possible to make a special "leaky" rust compiler mode where:
- Your program is only allowed to be single-threaded
- Nothing ever gets dropped. (All allocated memory leaks). And all variables get put on the heap.
- Mutable references can alias
That might be enough to make rust's lifetimes irrelevant. But I'm not convinced the juice is worth the squeeze.
If you don't mind segfaults, you could make it useful by copying C's semantics instead - but you'd need to manually free memory to avoid memory leaks. Or if you also don't care about performance and rust's ffi, you could put a tracing GC in there. But if you want that, why not just use Zig or Go?
The program that is coded all in one go before ever compiling, and then never altered after, is rare as unicorns.
Any working program being altered will have almost all its borrows neatly lined up.
What? I never said that. Its bit rich to say I've missed the point, then immediately misinterpret me.
> Any working program being altered will have almost all its borrows neatly lined up.
Sure. My question is: When the lifetimes don't make sense, what should the compiler do if its expected to produce running code all the same? Should it leak memory? Should it produce code with undefined behaviour - like use-after-free errors and race conditions?
You seem to want code thats been explicitly marked as invalid by the borrow checker to still run. The rust compiler isn't designed to do that, and I'm trying to think through how the compiler could make invalid code run at all. What should even happen if you try to use a variable after handing over ownership?
No program is going to ship with whatever bug it allows in, so who cares? The overwhelming majority of your undefined behaviors have no effect on what the person is interested in, right at that moment. They will get around to your crisis on their own time.
You are just insisting that your crisis be vanquished before they are allowed to look into theirs.
What you are proposing is to Rust to have two languages inside the same compiler. I guarantee you people would simply put debug builds in production if they could "turnoff" the borrow checker like you are proposing.
I am not suggesting anything even vaguely like what you are saying.
The compiler should emit whatever the code it sees says to emit. If it neglects to arrange to close TCP connections, they won't be closed. So what? I am not interested in TCP connections right now, or I would patch up that bit, in response to the warning the compiler spit out. I have other things to worry about right then.
The release build will insist everything be taken care of, so there is no "other language". You are just permitted to get there by a route you choose, in place the route Cargo would impose now.
That said, I do think Rust could add something like Haskell's type holes where instead of just randomly leaking memory the code has undefined behavior and will simply crash the program if executed.
The choice of which pointer type to use, e.g. a reference vs. Rc vs. Box, does impact semantics. And your broader point is well taken: since that choice affects semantics, it's really something you have to consider as you're designing your data structures; it doesn't really make sense to design the program first and tack on memory management at the end, like the parent is suggesting.
But that's not the borrow checker. For comparison, C++ has very similar behavior around smart pointers, but no borrow checker. Also, destructors are lexically scoped: a variable's destructor will be run when it goes out of scope, no earlier. The borrow checker used to be lexically scoped as well, but it's gotten smarter over time [1], and a borrow can now end in the middle of a block, if the borrow checker sees that the borrowed data is no longer used afterwards. Yet this has intentionally not been extended to include types with destructors, on the grounds that Rust code sometimes relies on destructor timing for correctness (in ways that the borrow checker can't see), so running destructors early could cause breakage, which would be hard for code authors to spot by just looking at the code. This same lack of predictability is considered acceptable for the borrow checker precisely because it doesn't affect runtime semantics.
[1] https://stackoverflow.com/questions/50251487/what-are-non-le...
This is a great idea! It made me realize why I still choose Python for prototyping. (I had been wondering...)
I absolutely love Rust's guarantees and prefer its memory management over GC. But nothing beats no memory management! Make everything Arc, make everything static---I don't care! My program will only live long enough to print "OK" and reach the next TODO, anyway.
> If your program confuses the borrow checker, that means that the compiler can't figure out when it should close that TCP connection on your behalf.
That's quite okay. "Process end" is as good a time as any, in this case.
> This isn't the sort of thing you can just turn off in debug mode, because programs should act the same in debug and release modes.
I don't know why programs with and without overflow and bounds checking are the "same" while programs that close unused sockets at different times are not, but it's okay---we don't have to call it "debug" mode. "Development" or "prototyping" mode both sound fine to me.
> There are ways around this - you can use types like Box, String, Rc, Cell, etc. And then just .clone() all over the place. But your code will run slowly. And thats still way more annoying to use compared to other languages. I don't know any other language which makes you write "Box" to put an object on the heap.
I think this is precisely the point. In other languages we can just create objects without worrying about where they should be stored. This is good because most of the objects -w-e- some people create don't get stored anywhere, anyway. They don't make it to the first release.
--
And yes, I realize I'm describing what is, from some angles, a completely different language. But I think one of the original discussion points is "why Rust-like languages keep popping up"? This is my take.
I didn't know where to squeeze this in, but the biggest annoyance in my (quite limited) time with Rust was: refactoring functions. I like breaking up functions into smaller chunks: `do_xy() { ... do_x(); ... do_y(); }` especially where a block of code could really benefit from being under its own name. Developing in Rust, I often found myself looking up internal types for the parameters/return type, and sometimes fighting the borrow checker. (I'm not sure type inference would be a good solution to the first, and I believe the second is mostly due to lexical lifetimes?)
I hope Rust never does this, as I think it would kill the best thing about the language. There's almost no point in using Rust if you're not interested in leveraging the borrow checker to write safer code. Just write C++ instead.
The last thing the Rust community needs is to be flooded by new users who don't bring with them a culture of writing safe code. If you give users the ability to just turn off the borrow checker, they'll leave it off and ship code in debug mode.
Many users find the borrow checker perplexing only because they can't understand why the code they write day in and day out in other languages gets rejected by Rust. Well the answer is likely because it's unsafe in some subtle way, and the borrow checker caught it before you even had an inkling there was an issue. Best to get used to writing safer, well-architected systems first, and then you'll have an easier time satisfying the borrow checker.
I don't mind, personally, if Rust fizzles, but it seems like you ought to care.
It just needs to be good enough for people not bothering to change.
Personally I think the understated way in which Rust now maintains a death grip on the WASM tool chain and tooling is the answer. Let the web devs have their JS. We can have everything `<canvas>` and below.
The docs don’t advocate this approach. Presumably because they favour correctness and understanding over blind hacks. I get why the docs are the way they are but I don’t think it’s worth the blood sacrifice necessary to stay on Mr. Borrow Checker’s Wild Ride.
Note that the above advice is no longer a 100% hammer-fits-all solution because futures and async and pin have unleashed brand new eldritch horrors upon us.
Technically those methods don't break free of the borrow checker; they satisfy the borrow checker by not borrowing.
Exactly! And remember, it's not always getting a grip on the language. Sometimes it's getting a grip on the problem or one of the possible solutions.
Rust absolutely can still fizzle, and certainly will be found to have done, in time, without major improvements in adoption rate. If you have better ideas for that, propose those. The more the better.
Spurious objections to ways to improve adoption favor other languages.
You can try to take your “fizzle” comments to /r/rust. See how many people care about Rust being on its deathbed due to not adopting your misguided suggestion.
It tells you what’s wrong, you fix it, you move on. It’s no different than forgetting a semicolon in Java.
Nothing needs to be undone after. Either those 4 tools get you a valid program or you made a logic mistake you would’ve made in any other language. Or in Rust sans borrow checker. I don’t see how that’s an issue.
I don’t see how my methods are spurious objections or how I’ve spuriously objected to anything. My methods are tested, tried, and proven to work on a sample of n=1.
All that’s left is government funding and we can start mass human trials next month.
My few experiences with Rust made me realise that I’d become used to dropping into a Python shell to sort of explore and actually experience what was happening. Even in toy programs in Rust, I had to change my approach and also ensure I wrote one small, working chunk of code at a time. (Rather than roughly sketching out bigger sections, for example.)
I quite liked it, but changing habits isn’t easy and now I’m wondering if maybe it will limit the wider adoption of Rust.
This is an interesting idea. I wonder what class of borrowck problems could be ignored like. Some of them can't be ignored, I think, because they would allow catastrophic things like use-after-free. But are there other violations that aren't as serious?
I use todo!() a ton when building as well. Sketching out data types and signatures before diving into implementation details.
While other languages with RC based GC mechanisms take care of managing the references automatically, even the C++ std ones do it, in Rust you have to manually increment the references with calls to clone().
So then we land in Rust code golf, where one tries to make the code as much as possible with bare bones borrow checker.
I don't see how this would work, Rust is already move by default, the problem is how to get std::shared_ptr() like convenience where copy member functions are called by the compiler, with the side effect of taking care of the counters implicitly.
In Rust, copying is explicit, thus manually typing the copy()/clone() calls.
struct X {
std::unique_ptr<XImpl> p;
};My initial remark was regarding the ergonomics of reference counting, naturally for values greater than 1.
The reason is exactly that, ergonomics and developer productivity.
A reason usually taken into account when picking programming languages for a project, if multiple options are available.
I don't want to comment on Go's visibility here, but is it that much different than how in Rust the lack of a semi-colon means you're returning (or the presence of a semi-colon means it's an expression instead of a statement, which has tripped me up many more times than the inverse)?
Which has interesting consequences, e.g. for non-ASCII identifiers where there is no concept of lower- or uppercase: https://go.dev/doc/faq#unicode_identifiers
Whether or not this bothers you is a different question.
Edit: Found some prior discussion here https://users.rust-lang.org/t/why-no-lifetime-elision-for-st...
Surprisingly, such a thing is actually possible. We can use borrow-checker-like static analysis layered on top of shared mutability (such as RC). It involves selectively making certain sub-graphs ("regions") immutable, thus getting the borrow checker's zero-cost memory safety, without the architectural restrictions the borrow checker normally imposes.
The core insight is that a borrow checker doesn't need every object to be completely affine, all it needs is some notion of what objects are reachable from what others.
This has recently been prototyped in the research languages from Forty2 ("capsules") [0] and from Milano, Turcotti, Myers [1].
Pony's "iso" [2] has shown that it can work remarkably well in production, and Vale [3] is making it work with single ownership.
In my (biased!) opinion, it's one of the most promising areas in language research. If we can get the speed and safety benefits of Rust while being more ergonomic and intuitive, I'd consider that a big step forward for the field. An exciting time to be in language design!
[1] https://www.cs.cornell.edu/andru/papers/gallifrey-types/gall...
[2] https://tutorial.ponylang.io/reference-capabilities/referenc...
Edit: along with the authors own blog explaining their take: https://kerkour.com/supply-chain-attacks-and-backdoored-depe....
The short version is, it's better to have a centralised (though proxyable, and opt-outable) central store of hashes of module content, but then provide flexibility over distribution. The concern is that a central package manager adds complexity and opportunity for security vulnerabilities without adding much value.
My reduced list includes only:
* Strong typesafe wrapper types. We had a few RFCs, always dropped. This would actually change how I program, and would increase typesafety a lot. I don't even need fancy wrapping with the ability to select which traits to expose and which not, just wrap+automatic expose would change my life a lot
* Let me compile to shared libraries. I get it, all-static has its pros. But refusing to support shared libraries is asinine. It's fine if I have to always use the same rust compiler version! just give me the possibility!
I compile Rust to shared .dll and .so libs all the time. Is this what you're looking for?
--crate-type=dylib, #[crate_type = "dylib"] - A dynamic Rust library will be produced. This is different from the lib output type in that this forces dynamic library generation. The resulting dynamic library can be used as a dependency for other libraries and/or executables. This output type will create *.so files on Linux, *.dylib files on macOS, and *.dll files on Windows.
I meant compiling a program and linking the crates as shared libraries. Kinda like a normal C/C++ program, and how the whole linux wolrd works.
...and I guess even the strong newtype wrapper is actually there with `struct Newtype(type);`, but I end up having way too many `.0` everywhere that I really dislike
I'll try not to post before my morning coffee next time :p
Okay I see. You can kind of get this from the libloading crate.
https://docs.rs/libloading/latest/libloading/
It works but it’s not great.
Similar to Python's _foo. I don't like it because if you want to change the visibility you must change the name at all current use-sites.
I'm open to other suggestions for how to effectively communicate the public interface without maintaining some separate documentation.
Article> It's the fruit of my experience working with C, Python, TypeScript, Go, and Rust and my frustrations with the latter.
I don't really see the influences from anything other than Go here.
Don't get me wrong, I love both Go and Rust, for very different reasons.
No it really, really shouldn’t.
``` pub fn horror()->Result{Ok(Result(mut &self))} ```
``` pub fn better()->self ```