The Rust I wanted had no future
graydon2.dreamwidth.org
graydon2.dreamwidth.org
That said, I still hate async with a passion, it makes the language more complex and not very elegant (i.e. function coloring). And now that I know how it works behind the scene (thanks to Jon Gjengset [1]), it feels so complicated and hacky, a mediocre very high level concept that someone managed to implement as a zero-cost abstraction. Impressive, but still a bad idea.
I'm sure the pro of having a BDFL instead of a committee is being able to follow a singular vision, instead of trying to appease members by adding the fad du jour which might stray a little too far from the original vision. Too many chefs in the kitchen and all.
I stopped paying attention to the language around 2012-2013 or so when the direction clearly steered towards the latter.
You may want to check https://austral-lang.org/ out.
OCaml is / was short for Objective Caml. Hence the name used to have an apostrophe. At some point that was dropped.
It's even hard real time capable, which makes it potentially viable in scenarios like games where GC traditionally hasn't been.
Rust's async doesn't have the limitation author described – you can spawn and block both (Tokio has some limits there, but that's Tokyo's choice, not language limitation), making "color" largely irrelevant.
The second point was that both should have identical syntax, which Rust deliberately chose not to, because from Rusts perspective that would be too much implicit magic.
Async without function coloring means being able to call `async fn add(a, b: usize) -> usize` from anywhere and having an usize back, whether you're calling from a regular or async function. If you need slightly different logic because of async, you got function coloring.
In fact, a hypothetical async without coloring functionality would not even need the `async` keyword at all, as all functions would be effectively the same, so you could choose to call one asynchronously or not with no particular ceremony. Is this even possible to do without compiler magic?
So the two camps I've seen on solving it is either
a) blackbox it and set some rules for implementers, inevitably leaving to some hidden and very nasty bugs on the user end or bottlnecks for advanced users, or
b) give implementers the full tools to setup, launch, and synchronize themselves, which may end up being slower than a single threaded solution if the user isn't adept already with parallel programming.
You inevitably always have bits of a) creep in, even for the most explicit solutions
The insight that function coloring gives you is terribly important. Roman Elizarov (of Jetbrains, author of Kotlin and lead on Kotlin Coroutines) agrees too: https://elizarov.medium.com/how-do-you-color-your-functions-...
async functions without coloring means that the only warning you'll ever get that `calculate2Plus2()` actually ends up running a distributed BigQuery and writing the end result to disk, printing it to stdout() and parsing that result to give it back to you is... hopefully, documentation is up to date and you read it?
Async function coloring is not a problem. Async function coloring is a solution to "software developers are awful and will do awful things without any warning". If `calculate2Plus2` did not exhibit the write to disk behaviour in v1.0.0, but bumping to v1.0.1 does, I'd really want a warning that it does at least, and ideally a compiler error. The proper solution to function coloring is to have a pleasant API to interop between both worlds so that, at worst it's just a dozen characters more to say "yep, I really want to block here".
So the whole sync/async distinction doesn't really make much sense outside the context of single threaded UI toolkits, where indeed you have to read the docs anyway because doing some very CPU intensive work on the UI thread will block it even if no "blocking" APIs are ever invoked (and what is or is not blocking is somewhat arbitrary anyway in many cases).
Alternately, an effect system can communicate "touches disk", can make functions which are polymorphic to effects (e.g. map touches the disk if the mapping function touches the disk), and can distinguish different effects, unlike async/sync being anything-or-nothing. IMO async is more of an implementation detail (as "may suspend/yield" is not a very precise or interesting effect), whereas effects would communicate the properties that ohgodplsno wants to know.
https://dev.epicgames.com/documentation/en-us/uefn/specifier...
converges/diverges, computes, varies, reads, writes, transacts, no_rollback.
Blocking has meaning in a lot more contexts, and being a consultant on JVM related topics, you should know that: it's the entire purpose of Project Loom, and Loom doesn't entirely get rid of colour coding either explicitly for that purpose. Loom wasn't made because the guys at Oracle have a deep love for JavaFX, but rather takes into account the server world, where you really want to know that you're going into another context, another computer, etc. The only time where the existence of async doesn't make sense is if your entire language and ecosystem expects everything to already be distributed. In which case, you've just switched the default color to async.
Finally, you chose to read async as the current JS/C# abomination implementation, but most of the sensible languages have implemented it as an effect: Kotlin has suspend funs, they don't return a Promise, but they tell you two things: they're going to touch something like the disk or the network, and if you really want to have them in a non-suspend context, you can either get a Deferred<T> out of them (and find another thread to run it on, and handle synchronization yourself), or run them on the current thread (and block everything).
Kotlin suspend funs do not tell you they're about to touch network disk, that's the reason they use "suspend" and not "async". Suspend funs can also use generators (with yield) in which case no I/O is happening but they are nonetheless suspending.
So this is why blocking as a concept isn't a great one, IMO. The Kotlin designers were right to not use the word in their implementation of coroutines. Where Loom discusses blocking a bit, it's not defined as being about blocking, it's about being able to scale up threads to way beyond what was previously possible. It just so happens that the primary reason you'd want to do that is if you have lots of threads that spend lots of time waiting for things, but that doesn't automatically require network or disk access. For example you can use threads that spend all their time in Thread.sleep if you were writing an agent simulation.
Wasn’t basically all of the JDK’s file APIs rewritten to io_uring-like calls to support Loom? I have thought that IO was definitely something that Loom handled.
You're conflating too many orthogonal things.
It's not merely because of the performance of touches disk/network that async was used for those cases, it's because that waiting is not because you're held up by the language doing calculations. That isn't the case with a function like you describe.
Marking functions async when they aren't yielding just to signal that they might be slow is a bizzare idea. That's not what async is and it doesn't bring any real benefit, it's just abusing the notion (and limiting the contexts where you can run those functions). You'll still be using libraries which wont follow this strange idea, and you should know their performance characteristics.
Blocking code exists in all major languages, including JS. In a single thread context, having something "async" wont help you at all, if it calls anything blocking, which can be something as common as JSON.parse
>If it suddenly expects a disk write and I'm running on a ROM, I'm going to have a hard time. If it suddenly tries to contact google.com and I'm on an airgapped machine, I'm going to have a hard time.
All of those have nothing to do with async, and what async is created and used for.
What you want is something like Haskell's IO "tainting", a (side) effects system, or something to that (no pun intended) effect.
As a design choice of OCaml’s eio - an async runtime, you must pass around a dependency to all your IO functions. With the “net” object passed visible in my function type signature, I can tell some network work may occur. The first benefit is we get synchronous Go-like code.
In case of java there really is no compiler magic even, “just” runtime magic.
With Rust's async you have to worry about a different worker thread picking up the work after a context switch, which makes things complicated. Not so with Java.
Its a lovely design though.
To my mind it's very much a balancing act between "low power to the developer, high power to the language" and "high power to the developer, low power to the language" all up and down the stack from software / hardware to "consumer / framework".
TL;DR - he says that "doParallel" and "doConcurrently" are separate operations with distinct semantics that designers of programs must care about and that conflating the two (especially the common "doConcurrently-and-often-but-not-always-in-parallel") is one of the most common causes of bugs in programs that need to make progress in multiple threads of execution.
To schedule tasks properly, you need to know who is waiting on them as early as possible. With a promise, you only know when you get to the "wait()" call, which is too late.
The correct solution is called structured concurrency.
The issue is that you want to be able to save the entire thread stack cheaply so that you can switch to another task at an async yield point, and then go back to your previous stack frame. You want to do this without spawning system threads.
Say you have an f -> g -> h call stack that blocks on IO. If all the functions are in fact async state machines (or some other kind of callbacks) your thread stack will look like this:
Executor.loop() -> Executor.run(h) -> h.await()
That is, your functions are really objects that are queued up and taken as needed by the executor. If h has a yield point, you can put it back on the queue, go back to the loop and pick up some other work. Later you can do the above again and await h to again up to the next yield point.
Now consider the case where the middle function g is NOT async. In that case the call from f to g will be a normal function call that gets put on your call stack. In a language where this is allowed, you'll still have to wait or execute h and you'll get a call stack that looks like,
Executor.loop() -> Executor.run(f) -> f.await() -> g() -> h.awaitUntilCompletion()
(where our awaitUntilCompletion is minimally just calling h.await in a loop until h is finished). At this point, you are stuck: you can't yield from g() because it is a normal function with a normal stack frame, so your thread has to wait till g() finishes before it can be used for anything else. You are basically blocking a thread on h. At this point you might spawn a new thread to keep your thread pool count up (which is expensive and memory hungry) or just accept the forced synchronous blocking (which reduces throughput).
If this happens over and over again, you either end up running out of memory or end up running code in a synchronous fashion but with worse performance (because of all the state machines). This is why functions are usually coloured, so that you don't get yourself in this accursed state by accident. AFAIK the alternative is to allocate all of your call stacks in the heap so that you can switch them in and out of kernel threads, which is what fibers are. This requires a complicated runtime with a scheduler such as in Loom,
https://github.com/mirage/ocaml-cohttp/blob/16e991ec1f7e5f0c...
Performing these effects is similar to throwing exceptions up the callstack where whichever ancestor handles the IO work, then resumes the child with IO work done in hand.
- what the type of res is (is it a string? A buffer? Async string? Something else?)
- how this code does not block: what is happening in parallel? At which point will it block?
I would like to be excited about OCaml’s algebraic effects, but right now I don’t really get it.
- It's implementing non-blocking I/O using effect handlers. The complexity is not exposed to the library user, which is actually the whole point. If you want to dive into the concurrency library (Eio) and study how it works, that's very doable, just like any concurrency library in any other language.
There's really not that much to get–OCaml will do async I/O without having to colour functions, just like Go, Erlang/Elixir, and soon Java. It's not something sexy like monads that excite people with mysterious FP concepts, it's just a lot of hard work that went into simplifying how developers code at scale.
When reading ocaml on _text_ like from GitHub, I personally accept that I won’t need to know the type and underlying data structures just yet unless I read and have open the .mli file.
I’m excited first and foremost because we can have async code feel synchronous like Go.
How does the code run? IO runtime is defined in userland, like Rust’s Tokio. That’s the “Eio_main.run” part. Underneath, eio spins up and manages processes in a non blocking manner, even scheduling work.
Since effects are essentially resumable exceptions, this particular http request throws an effect to the runtime. The logic and this particular process pauses (okay I may be wrong here). Logic flow now continues somewhere in our runtime level (the effect handler), where all the async IO work happens. The runtime resumes the child process and returns the http result back to the child via a “delimited continuation”.
IMO, this is even worse than function coloring.
Rust has basically 2 executor libraries, tokio and async-std. It seems to me like tokio is solidifying as the executor of choice and it's only a matter of time before that design is baked into the std library.
std::async is mostly "good enough" for ordinary cases where you only want to fire-and-forget and have no control over the underlying dispatching and scheduling algorithms. If you want something tweaked towards your use-case, and hence actual high-performance executor library, std::async is not gonna cut it. And that's where the "C++ executors" proposal come in.
I think designing and evolving any living programming language is just one of the hardest problems out there.
Incredible blog post indeed, was awesome to read it.
His dislike for functional constructs didn't do Python any favour, but now that he's gone I see Python is adding the kitchen sink as well. I haven't kept up with the language since that pattern matching proposal.
If they're all terrible, then the language won't appeal to anyone and will die off naturally.
On the other hand if everyone gets to add their favorite terrible idea, there may still be enough non-terrible parts to appeal to enough people to keep momentum.
One of the most popular and succesful languages, and a major force in AI innovation?
The transition was slow (Python 2 was supported in parallel to 3 for a long period of time so there was more than enough time to migrate), relatively easy to do, and brought huge benefits to the ecosystem.
It could have been done and forgotten in 2 years if some community members had not been dicks about it.
JS devs have been transpiling for years now and it works.
Python should have released a conversion tool that could automatically convert most stuff and identify the stuff it couldn't convert. This kind of directed upgrade would have made the process much easier for developers to actually accomplish.
Sure. The point is that a community can make things go poorly, with or without a BDFL.
Here's the timeline the way I think about it:
- 2008-2012 Python 3 becoming usable (byte formatting, six library)
- 2012-2016 Libraries (Django, etc) becoming compatible
- 2016-today Applications (Trac, Ansible, Chrome, etc) becoming compatible.
It took a lot of time for the ecosystem to be ready. Yes, there was a lot of stubbornness, but that's reality.
There are many things that could have been done to make the transition go smoother.
There's a lesson in there to learn. Its unfortunate that the people who need to learn it most likely won't.
(That said, JS is actually a very versatile and almost-great language, getting better all the time)
https://peps.python.org/pep-0020/
Things like "There should be one-- and preferably only one --obvious way to do it." just was such a breath of fresh air in language design.
Function coloring seems to come up a lot in these discussions, but I don't see a better way without providing a runtime. Could you propose an alternative approach to async, without sacrificing ability to write zero-overhead, high-performance bare metal systems?
template<template<typename> class MyContainer>
int getFirstInt(const MyContainer<int>& myContainer) {
return myContainer[0];
}
They're not expressive enough to implement a monadic async library, as far as I'm aware, so they don't fulfill the role of proper HKTs in this context.
>minus `template`’s duck-typiness so you’d have to spell out what contract the container must adhere to.
Do you think it's consistent that the T in MyContainer<T> can be "duck typed" but not the MyContainer?
For an example of GATs, you can see the following:
https://play.rust-lang.org/?version=stable&mode=debug&editio...
AIUI you need GC in order to do that. There's a reason why Rust's async/await support needs compiler magic.
(I'd like to see standard interfaces for "pluggable" garbage-collected arenas in Rust, but this will need to wait until after Allocators/Storages get fully stabilized.)
I really wish people stopped using this concept, especially in the context of Rust because the `async fn`/“regular function” split is strictly equivalent to the `try_something()`/`something()` split (the first one being failible and returning a `Result` in case of failure). `Result`s and `Option`s are coloring the stack in exactly the same way a `Future` does (and `async` is pure syntactic sugar on top of future).
So, someone may like exceptions and green threads more than `Result` and `async` (and this is a completely valid PoV, even though I personnaly like the explicitness better), but thinking `async` is somehow special is just a conceptual mistake.
BTW, in the original blog post the “red” functions were actually functions with a callback parameter, which is actually very different.
Tons of stuff in std and many other crates don't use async. A lot of stuff is harder and more code to do async. A lot of it is much harder to read.
That's the difference.
Well, there are dozens[0] of incompatible error types in std, two of them being just called `Error`[1][2].
Everytime you create a function that use more one of the different error types from std (for instance, an I/O error and a TryFrom-related error) you have to create a custom Error enum to accommodate the different kinds of errors from std…
> community agreement to do things that way
And the community agreement is e̶r̶r̶o̶r̶-̶c̶h̶a̶i̶n̶, ̶f̶a̶i̶l̶u̶r̶e̶, ̶ s̶n̶a̶f̶u̶, this_error, unless it's for a binary crate and then it's ̶a̶n̶y̶h̶o̶w̶, ̶e̶y̶r̶e̶, color-eyre.
The error ecosystem is much more fragmented than the async one (even though I think this_error is here to last, so I don't have to migrate once again…). In the async world, there's pretty much tokio (but I think it would actually be better if it interoperated better with other runtimes so alternatives could emerge)
> A lot of it is much harder to read.
Ah yes, fut.await is fundamentally harder to read than res?, right?
Overall, you vastly underestimate the friction that the `Result`-based error handling adds. I still think it's worth it, and it's been slowly getting better (did I say I loved this_error?), but it remained one of the biggest point of friction in Rust over the 8 years (time flies) I've been using it. Async is comparatively less disrupting (mostly because it only appears in the call stacks where you're doing I/O, whereas errors are ubiquitous).
[0]: https://doc.rust-lang.org/std/error/trait.Error.html?search=...
A nice talk about this: https://www.reddit.com/r/programming/comments/da141r/ron_pre...
More concretely, threaded code has nothing to say about whether or not its synchronous or asynchronous, and asynchronous code has nothing to say about whether or not it's evaluated in parallel.
That is an artificial distinction you made up. I have written many apps and desktop applications, all of which where using 3-10 threads, some of them doing "parallel" work meaning the parallel computation of divide and conquer algorithms, some were doing concurrent work.
I tried async often. I threw it away every single time, because it "infects" the entire codebase. I suspect the only reason async exists is because the Javascript event-loop is single-threaded and there are no blocking primitives in JS. If you don't have access to blocking data-handoff between threads, you can choose between callback hell or async/await.
Please take a look at the video I posted earlier.
2, 4, even 256 sockets exchanging data can be worked with concurrently on a single thread and gain performance vs blocking and waiting for the first socket to finish. There's no parallelism, since they're never actively reading/writing at the same time, but they're concurrent because they exist and operate in overlapping timeframes.
Running two independent algorithms could mimic this - run part of algo A, then part of B, then A, etc. It's not useful for performance, though. To be useful you require parallelism - you have to have the algorithms executing at the same time, using multiple threads.
On async itself - it's not perfect, but honestly, there are contexts where it makes a lot of sense. I work on a lot of non-blocking C code - the entire programs are basically epoll and timer driven. As a result, the high level coordination is just callback hell. Async is very nice in comparison.
Yes, you can spin up threads to do everything with blocking, but non-blocking I/O came around specifically because the threads add overhead and kind of suck. It's worth noting that having threads can also infect the codebase. You either have to carefully manage mutexes, or you only communicate with channels and have to worry about keeping things updated and in sync. Sometimes this works great with minimal communication between threads. However, if you have one socket per thread and the sockets are all triggering actions that mess with the same data... it's not so great.
Besides IO Completion Ports for async IO has been the fastest way to do heavy IO on Windows, and it was introduced years before JavaScript even was a thing.
And that's just one example. So hardly think JavaScript was the main driver there.
Rust Channel have the nice additional that they don't panic, and don't have all the weird Go Channel Axioms[1] that you have to wrap your head around to get things to work correctly[2].
For what it's worth, I don't think Java Virtual Threads (a.k.a. project loom) has implemented an explicit concept of channels, although I guess BlockingQueue could work as a channel.
There is a big issue with bringing CSP to languages like Java 21 or Go, that use so-called "colorless concurrency" or "stackful coroutines". CSP losses a lot of its modeling power when you combin shared mutable state and pre-emptive (or invisible) parallelism. And this is exactly what both of these languages do.
"Colored" abstractions like async/await are more cumbersome (especially in Rust), but they let you know exactly at which points of execution your stackless coroutine may suspend, and you can handle synchronization accordingly.
Those functional concepts are cumbersome, but threads ("green", "kernel", "virtual", "goroutines" or otherwise) are not the panacea their proponents claim them to be. I had to debug way too many data races in my life to consider them perfect. Rust still has a lot to improve on its async front (no async traits, async closures or async iterators and the async runtime confusion), it can at least promise me that I won't get any data races.
[1] https://dave.cheney.net/2014/03/19/channel-axioms [2] https://www.jtolio.com/2016/03/go-channels-are-bad-and-you-s...
But not any other kind of race condition.
Also, data races are “safe” in java. Plus you can rely on immutable data for those parts. Parallelism will always be hard, there is no model for shared mutability that would be easy.
Being sync or async is essentially a property of the attention of the caller, not of the action itself. Is "eating a donut" a sync or async action? If I'm focusing all my attention on it, essentially putting all tasks aside (after) - then it's a synchronous action. If I'm reading a book/watching a video/walking/etc, while eating – it's an async action.
How does "func EatADonut() async {}" aka "eating a donut is an inherently async action" even make sense to people?
But consider a network request. The vast majority of the time is not spent in the CPU.
Right?
s/eat a donut/make a network request/
Is "network request" a synchronous or asynchronous activity? It depends whether your code blocks and wait's until response (or timeout) or continues executing and handling it when it comes. It's property of the "attention" of the caller, not of activity itself.
Something that pegs the CPU to 100% because it's doing intense processing isn't a good candidate for async. Similarly, some code can cause issues when written in a non-streaming fashion. Take the following example (in python):
x = [x for x in range(1_000_000_000)]
y = (x for x in range(1_000_000_000))
In other words, "async" is a label on a box of donuts that implies (though doesn't ensure, people can of course still do bad things) that the donut won't explode if you look away from it.
Even if this is an original logic, why language is deciding for me what is considered heavy or not? What if I'm fully aware that I'm doing heavy processing and I want it to be happening in the background. What if I'm writing HFT software and every call is heavy for me. Language is not a right level of abstraction of marking "heaviness" of the code.
It really just doesn't make sense. Why stop there and not start marking functions with how many times per second they can be called? Like you can call "light" function 100 trillion times per second and OOM the system, so let's mark it "func light() async 2_times_per_second {}". It only can be called from functions that have lower per second label.
Plus, if Python cares so much about not OOM-ing the system I would start by not requiring 32-bit int to occupy 28 bytes in the first place.
Well, I'm not that deeply versed into any language's history, but I imagine any 15+ year old language had a point where they needed asyncrhonous functionality but had to write around legacy code. I don't think Javascript would have been written the way it was in the 90's if asynchronous operations were a mandatory priority.
So it's probably a lot more "this is how we hack this in without creating Python 3.0". Relatively elegant to make it an optional part of the language that you explicitly choose to delve into, instead of a core feature that would have broken thousands of sites behind the scenes if you made it "right".
>why language is deciding for me what is considered heavy or not?
because we decided decades ago that we didn't like using fork(), nor creating/destroying processes ourselves. problems that would inevitably need multiple solutions when supporting multiple platforms because these aren't language features so much as OS calls made in a language that you may or may not be writing in.
Remember at the end of the day that languages are abstractions on top of an OS. Any interesting ideas for problems involving more than a single process space that your OS makes for you is bound by what the OS's API lets you do. It's arbituary but also based on historical problems to wrangle.
So we haven't had enough problems where we need to bound a function by how much it is called. We have for how to interact with all the above OS/language problems.
Regarding stickiness of function colors - it never happens when async/await is used correctly (in the same principle as IO monad isn't sticky).
The question is how you translate mental models from real world to the code, and async/await fails here spectacularly. It's just weirdly unnatural thinking about sequencial processes. It requires tremendous amount of cognitive gymnastics just to reason about simple things.
Of course it does, read it as "beware, something blocking down the road".
If you can EatADonout without blocking, please do, but want it or not that's a different implementation, one that doesn't block and the signature it's telling you so.
We're so used to sync and having hidden blocking operations. I wonder if in an alternate universe the first languages considered the blocking/async nature of operations and then some newer languages considered hiding this information into seemingly sync functions would produce similar but opposite outrage against it.
Implementation is the same. In both cases it's the same set of CPU instructions, but async/await languages create artifical division, forcing developer to think othervise.
So, let me explian my reasoning. Code starts with a developer's mental model of a problem and behavior of the system and then translating it into the code. The more straightforward this translation, the more readable the code. Code is a second degree map of problem domain so to speak ("real" world -> mental model -> code).
Like if you want to add two values, the simplest form of code would be "add(2,2)" which is pretty straightforward. If the code forces you to do some mental gymnastics (i.e. "2 2 addOnlyEvenNumbers") - that's less straightforward, less clear and less readable.
In the same vein, if you want to execute some function ("EatADonut" or "MakeHTTPCall") – you may care or not care about blocking and waiting for results. But it's your call. So it makes sense to give you two options to run this function. Go has simplest possible solution – "eatADonut()" vs "go eatADonut()". It doesn't matter what is a "default" here – it could be "eatADonut()" (go to background) vs "sync eatADount()". What matters that "eating a donut" is just a set of instructions to the CPU, and it's up to caller to decide how you want to execute it in terms of concurrency.
Now, "async/await" approach turned this ownership of "synchronicity" around. Now function is deciding how it should be called. Mental model of "actions" now needs to be translated into "actions being async or sync for the purpose of fitting into this language concept". Which is cognitively expensive for no added benefit.
Sure you can rationalize it, and get used to it as to any other absurd design, but it still adds unneeded complexity to the code, makes it less readable and less clear.
It's actually a very different set of CPU instructions. the function EatADonut is the same set, but "async" means the kernel needs to take time out of its execution to do some action as small as accessing an open thread, or as large as "gain access into a completely different piece of hardware" before putting that set of instructions onto that different thread. Not the program, the actual scheduling process between your program and the OS you are executing on. Then it needs a way to to get that result and sync it back onto whatever thread spawned it and access the result.
It is in fact a huge action, so marking it with Async is basically a very explicit warning.
>Which is cognitively expensive for no added benefit.
On the contrary, I can't even begin to imagine the amount of compiler optimizations it saves on as well to have that be explicit in code. I'm sure Go has to do all that on the fly while a colored language gets to allocate all those potential processes before the program runs. It's only no added benefit if you dont care about performance. But to be frank, you probably don't need more than a single thread if your problem isn't bounded by perormance. Parallel programming is all about getting something done faster after all.
You have to think of it from a higher level.
Nobody knows if eatdonut is strictly async but everyone knows all IO calls are async. So something like socket.send would be async, this is obvious.
Then anything that calls socket.send would then in turn become async itself. The async sort of spreads around like a virus. Any function that uses other async functions gets polluted with the async marker.
So in this sense, you should think of it like this. If eatADonut is async it means it touches IO. It sends data somewhere else to some other thing. It also means it's not unit testable.
In a nut shell This is the intuition you should have about async:
Async Tells You about the properties of eatADonut. eatADonut doesn't tell you anything about whether or not it's async.
Under this intuition, EatADonut, by good design principles should NOT be async. This is what it should be:
fn eatAdonut() -> ResultOfEatADonut
async fn sendDataToIO(data: Data) -> ()
async parentCaller() -> () {
let data = eatAdonut();
sendDataToIO(data.serialize()).await;
()
}
IO calls should be very general and small functions while your other functions should be pure and unit testable. This is the overall intuition you should have about async.
Believe it or not, this specific aspect of async is actually an improvement over golang style concurrency. Not an overall improvement, it's still debatable which style is better, but I'm saying the async marker is actually a feature that other styles of implementing concurrency don't offer.
Basically, async encodes the concept of IO and function purity into the type system. It allows the type checker and You to tell which functions touch IO and which functions are pure and unit testable.
People think unit testability is some kind of marginal benefit because you can cover testing with integration tests which are basically tests that happen across IO. It's more than that. The ultimate dream of programming something as if it's a bunch of modular lego blocks that you can elegantly place together is most fully realized with pure functions. Async enables more of this style of modularity in Rust (but not without some cost which is why it's debatable whether or not it's better than go style concurrency).
Do you mean on a hardware level? Because otherwise, hell no.
If I’m writing a binary file parser that buffers some data and processes it, I sure as hell want to wait for IO, there is no other meaningful way forward in most cases. It is also not a small isolated part, but a tight loop bouncing between CPU and IO.
Async is a function of the caller.
I mean in the context of async await. The smallest primitive that is properly async is an IO function.
There are functions, you can call them sync or async if they handle IO or UI or they will get a necessary data later.
I still don't understand why I can't fetch a URL from top level javascript. Also I don't understand why zig async and await passes the control flow that seemingly total arbitrary way. The naive approach (put async calls in a queue, and periodically check if they are completed or can be executed) seems fast, deterministic, and good enough in every way?
Okay, maybe zig needs the speed, and can't just stop the execution of the sync code time to time to do something else, but why javascript? Maybe javascript async is build upon regular promises and regular objects, instead as a proper language element with proper support in the javascript engines? I don't know. Anyway async as is used with colors is totally against the picture that the words "async" and "await" suggest.
> Maybe javascript async is build upon regular promises and regular objects, instead as a proper language element with proper support in the javascript engines?
Yes, they are promises.
> The async and await keywords enable asynchronous, promise-based behavior to be written in a cleaner style, avoiding the need to explicitly configure promise chains.
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...
> The naive approach (put async calls in a queue, and periodically check if they are completed or can be executed) seems fast, deterministic, and good enough in every way? Okay, maybe zig needs the speed, and can't just stop the execution of the sync code time to time to do something else, but why javascript?
You're describing preemptive multitasking. You can create separate execution threads pretty easily in most languages (even super old C code), but that's not why Javascript and Zig have the 'async' keyword. The difficult part isn't the asynchronous execution; the hard part is handling the result from asynchronous code (i.e. the 'await' part). Promises are a simple mental model for organizing how pieces of code depend on the results of other pieces of code. The C# documentation does a good job explaining this:
https://learn.microsoft.com/en-us/dotnet/csharp/asynchronous...
Note: From Zig's documentation it sounds like 'async' is cooperative multitasking. This is single-threaded execution. It is "concurrent" code, but it isn't executing in "parallel".
Concurrency is difficult because it is explicitly resource management. You don't need concurrency for calculating the correct answer, you only need it for managing time.
Sending a letter and getting a reply is an inherently async action; you can stare at the mailbox all day but you probably don't want to. Waiting in line at the bank is inherently sync; if you try to do something else and then come back they'll make you start over again.
Yet, it's up to me to decide, not to the mailbox.
Caller can use this information to invoke 10000 IO bound tasks in parallel, whereas cpu bound tasks are limited to number of cores and may want to be throttled. In your analogy, I can eat a donut while simultaneously listening to music, but I can not eat a donut while simultaneously eating fish soup.
Now where it get complicated is when a function does both io wait and heavy cpu processing. Now neither color of the function match.
Forcing this upon the caller to invoke such functions in special ways is the strange part. Taken to the absurd you ought to have special syntax for every resource constraint in the computer, memory bound, disk bound, wifi bound, and so on. It would be better if the runtime could figure this out by itself and parallelize accordingly.
The single threaded aspect of async functions does have certain other benefits, like not having to care as much about muxes when synchronizing state.
Other possible semantics would be to start multiple tasks and wait for them all — this is where structured concurrency comes in, giving a good analogue with gotos.
Async-await is fundamentally an optimization, not about semantics only. Languages without runtimes simply can’t reason about it without language support, but managed ones could do so as every IO call goes through it — scheduling multiple tasks to a single core is now possible.
Without it when you want to do asynchronous stuff you either block on it (not good since JS is single-threaded), eg "let toto = asynchronousFunction(); doSomething(toto)" and nothing can be done during the time asynchronousFunction is waiting on IO or something (in a browser environment that means part of your website stops responding to user input); or pass as an argument to the asynchronous function a function that will be executed by the asynchronous function once it "wakes up" (usually called a "callback"), and while it's waiting, the JS runtime can execute other stuff, eg "asynchronousFunction((toto => doSomething(toto)))".
Callbacks were I think from the start in JS. After a while promises were introduced, which compared to callbacks avoids nesting, and probably have some other advantages that I don't know about. Still, they are method chaining and not "regular code". For example regular try/catch won't work as usual, you have to use .catch(). Even later async/await got introduced, which allow you to write asynchronous code as if it was regular code, with a few exceptions (for example you can only use await in an async function, top-level await took a while to land on Node, stuff like that).
I don't think using real world human actions helps with understanding any of that. Async functions make sense in JS because again, JS is single threaded and if you do a blocking call (like asking the kernel to read a file, or making an HTTP request and waiting for the response), the event loop is blocked. From Node.js documentation's on the synchronous function in the fs module:
> The synchronous APIs perform all operations synchronously, blocking the event loop until the operation completes or fails.
I feel like I need to point out that Donald “Structured programming with GO TO statements” Knuth included an example of coroutines in the first volume of The art of computer programming, the first edition, dated 1968. In assembly language for an accumulator machine. With a box of scraps!^W^W^W^W^WThat is to say, that C and most other languages have made coroutines awkward and thus virtually unused in the past three decades or so does not mean they are particularly novel or high-level.
Granted, Knuth used coroutines in a simulation and not for I/O, so he did not need that much of a scheduler, but still.
It's just an abstraction, it's not zero-(runtime)-cost. It might be the "lowest possible cost", still nonzero.
I continue to find the "function coloring" argument misses the point unless you're arguing from a developer experience perspective. Why should two fundamentally different things look and function the same? Want this the ultimate pitfall of early RPC implementations where everything looks synchronous?
In Rust, a lot of the friction is due to how async functions/futures fundamentally differ in model of execution and how that interplays with the rest of the language. Other languages get to hand-wave a lot of these problems away with a GC. (It certainly could be less of a pain nonetheless.)
- Futures don't execute until polled, can partially execute if something isn't ready (and would block), can partially execute and be arbitrary cancelled and dropped. There is no equivalent in sync functions unless you make them co-routines.
- Since you can "pause" futures if something would block, you need a place to store the function's state for when it's resumed. Now you must be consider if the async function's state is `Send` if the future wants to move to a different thread to continue executing -- which is why you see `pin!` used. Sync functions don't care about this since you always run to completion on the same thread, and the stack is ephemeral.
- Likewise, the `Future` returned by the async function is going to need to encapsulate it's state that it saves. In the general case, this is a compiler generated anonymous struct that changes if any state saved across `.await` changes, hence the opaque `impl Future`. This is why you see `BoxedFuture` a lot to abstract this away at the expense of an allocation. Ideally, the new associated types with lifetimes can avoid this with traits.
So if all functions were co-routines (i.e. functions that can be resumed and re-entered) they would all have the same "color". But all you really did was "lift" all sync functions to be "async" functions with no internal await points.
(IMHO, if the C# team back in the day decided to implement full blown co-routines into the language instead of just `async/await` as a compiler trick, I think many other projects would have followed suit with a more general co-routine solution instead of treating `async/await` as this special thing which is just a specific compiler generated implementation of co-routines.)
async in Rust is a popularity issue even more than a technical issue.
A whopping amount of people who are coming to Rust are doing so because they want a good ecosystem for implementing network service servers. Rust/cargo/crates hits the sweet spot for them.
I am with you that I loathe async/await in Rust. However, I also have to acknowledge that without async/await, Rust is a vastly less popular language.
All the other uses of systems programming are simply dwarfed by number of people building network services. That's just the sad reality.
Trust me, you do not want to put this stuff into the compiler. It's not just that it's cheating for perf, but it's confusing to users (no source code to read how these work) and frustrating that they can't write their own. Ultimately what this really means is that these things are indeed written in some language--the compiler's IR. JavaScript actually has a ton of this kind of things and every JS engine has gone through multiple generations of "what language do we write Array.sort in!?". In V8, these intrinsics are written in a DSL because doing them in asm, a special dialect of C++, or one of two compiler IRs ended up being more trouble than its worth.
You want a clear separation between what is language and what is library.
The fact that JS has about 3 of them (is it only 3? I'm not sure of this), with different interfaces, and non-usual behavior is what creates problem there. But other languages quite successfully make their own containers either different enough or similar enough that it's not a problem.
Map
Set
WeakMap
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guid...
V8 has over 35,000 lines of Torque, the DSL for describing these kinds of built-ins. Big chunks of that are dealing with Array methods like splice, sort, from, foreach, etc; and then there is stuff like BigInt, arguments object, a zillion methods on strings.
> Almost all high-level languages have builtin containers.
I don't think that's really true. Maybe for Python and Ruby, and other dynamically typed languages. I dunno about Swift. But it's not really true for Java or C#. Those languages have their collection/containers libraries written in themselves. Java has other problems separating the language from the libraries, but it's not due to collections.
Personally I'm happy Rust took the C++ route, it makes things more interesting, but I can see his point.
Part of the reason was Rust's desire to show itself as a direct competitor w.r.t performance, I think.
Say you use a vector in C++. You push one element and pop two. In Rust that's a None. In C++ the answer is UB (in my case 43).
That prioritisation applies to the whole ISO language, not only to the standard library.
What I'm talking about is that Rust's Option is very cheap in all the cases where it can be very cheap, which makes this whole design feature more affordable. C++ eventually grew std::optional which is not powerful enough for this work and yet is also bigger and slower. They could have done better, but safety wasn't a priority.
Unlike C, which the only way to be safe is not to touch it at all.
As for performance, ISO doesn't implement compilers, there are many ways to improve performance while keeping the semantics in line with the standard.
If the compiler vendors decide to focus elsewhere is another matter, a bit like there are languages as complex as Rust and compile faster, because that has been a concern, whereas rustc developers have their concerns elsewhere.
Even your dynamic example won't tell me calling pop/push x/y times respectively will fail if x > y.
I have to trigger it somehow.
I haven't written the Language of Kings in a while: does GCC -Wall truly enable all warnings yet?
clang has a -Weverything and it isn't supported.
The argument is that Rust forces you and in C++ you can forget/the compiler can do what it wants
It's like Yaml parsers, they had "load_yaml" which is unsafe and "safe_load_yaml" which is the secure option. Imagine no surprise when Rubyists went for shorter safe looking method and got their servers pwned.
Depends what you mean by explicit.
Look at it like this:
pub fn main() {
let mut vec : Vec<i32> = vec![];
let x = vec.pop();
println!("{:?}",x); // prints: None
}
vec.pop().expect("I want a value")
if let Some(x) = vec.pop() {
println!("{:?}",x);
} else {
println!("EMPTY"); // prints: Empty
}
Types like Result will issue warning that you didn't handle the cases.
I looked at your example and played a bit with it and yes I agree with you - compiler does help in this case.
My old text:
So instead of checking if vector is not empty you check that the return result is not empty. I do not see much difference. If Rust compiler would choke when "else" clause in your example is not present I would understand your point about compiler preventing improper access.
> If Rust compiler would choke when "else" clause in your example is not present
The compiler won't choke, but it will stop you from accessing the value.
It doesn't matter if you omit the `else` clause or not, the type system ensures that you can't access invalid values.
Here's a bit of an example based off of @Ygg2's code: https://play.rust-lang.org/?version=stable&mode=debug&editio...
Hacker News will try and discourage really quick back and forth comments. To do so, the reply button is hidden. However, if you click on the timestamp, to go to the comment's own page, you can reply there.
Rust does that with the combination of "sum types" (aka "enums with data") and pattern matching. An `Option<T>` is either `Some(T)` or `None`. Matching on an option with the pattern `Some(value)` creates a new syntactic scope where the value is accessible. This scope is only entered if the `Option` is actually non-empty.
All ways of getting the value from the option are ultimately shorthand for a match in that way. For example, option.unwrap() will either get the value if it there, or panic. Option.unwrap_or(x) will either get the value of it there, or use x instead.
In practice this is x100 less error prone than C++. Source: was burned by vector.front() on empty vectors more than once. In C++, UB. In rust, usually the "empty case" is considered when first writing the code, or at least caught during review (unwraps tend to be very visible in reviews)
I'm not a C++ expert but here is my understanding. Vector is essentially a tuple of (dynamic_array_address: ptr, size: size_t, capacity: size_t). To pop a value from vector you just decrements size. So what happens when you have empty vec? Your size is 0, and you're substracing 1, which causes undefined behavior.
Correct way is to check BEFORE you pop_back().
In Rust, any number of invocation of pop() will not result in undefined behavior. You can ignore the value, or you can check it, but it doesn't expose you to UB just for slightly misusing a vector.
Absolutely and this is exactly what I do. I always check the containers before removing elements.
>"In Rust, any number of invocation of pop() will not result in undefined behavior. You can ignore the value, or you can check it, but it doesn't expose you to UB just for slightly misusing a vector."
I do not understand "slightly misusing" part. I assume in Rust it would bomb if pop() returns "none" and you try for example add said "none" to some number.
That's the thing. It wouldn't. Worse you can do is make whole program panic, with well defined stack trace and message.
It will never return non-sense and pretend all is ok.
https://godbolt.org/z/vYcMhE9h7
> Error: attempt to access an element in an empty container.
It is also what attracted me into C++ coming from Turbo Pascal and Turbo Basic, back in the early 1990's.
Although C++ culture could be much better towards safety, it is definitely better than whatever WG14 is doing, or C has brought into the picture for the last 50 years.
Also anyone that just copy pastes C like code into C++, is the kind of developer that will be using unsafe{} all over the place, on the languages that have them.
C developers like telling themselves that only people with bounded rationality make security critical mistakes. All the skilled C developers have ascended beyond the mortal realm and would never let themselves be chained up with crutches for the weak like affine types or overflow/bounds checking.
Yes you could. But I do not think " cultural" is the right term for not putting all your code in an `unsafe` block
It's not. It's a safety phenomenon. See Java, C# etc.
Java doesn't have huge focus on it, but managed to get it right.
As for your dynamic arguments request, have fun,
In a runtime example I can run it with tests and it would behave fine if both values are same or first arg is bigger, in Rust's case it would behave valid for ANY combination of arguments.
Coming from Java the C++ stuff is jarringly unsafe.
Still, if you want to contribute to OpenJDK internals, they will only be taking pull requests in C++, not Rust.
So better learn how to build those seatbelts and airbags, for all the projects we rely on, including Rust's own reference implementation, that aren't going to rewrite their code from C++ into something else.
> It's easier to work with than C++, but that's fairly faint praise.
So, while Rust positions itself as a C++ alternative with all the complexities that C++ developers love, it may become less attractive for other use cases. For instance, I find it highly questionable to develop a web app in Rust...
Rust as a C++ replacement serves a real unmet need in the marketplace.
Knowing what we know now, we should’ve gone with TypeScript and just written our own Odata filter on top of it, but live and learn.
If I was in a position where I could pick and chose languages, and not worry about how not having TypeScript in most things will mean our best front-end developer can never go on vacation because he’s sort of our only front-end developer, I wouldn’t mind using Rust for web-backends.
Rust GUI is obviously not a great experience. At least not yet. But it’s not that bad either. I think it’s mostly the case of how JavaScript is just so good at it and seeing such a fast pace of improvements because the entire world uses it for most GUIs these days, that it’s just hard for anything else to compete. I mean, look at stuff like Flutter or Blazor, they are backed by Microsoft and Google and they’re vastly inferior choices for most use cases compared to simply building things in React, ReactNative or even electron, and that’s not because I have some wild love for JavaScript, it’s because it’s seeing rapid improvements they dwarf it’s competition simply by being used by a lot of people.
I wish Rust would have someone like Facebook pick it up and build a frontend framework for it, but I think that is just too unlikely for you to bet on, and you certainly wouldn’t want to do it yourself, even as open source because then that would probably be your entire job.
On the flip-side, the packages that handle basic back-end web stuff for Enterprise use are rock solid in Rust. Which is impressive, at least to me, considering it’s young age. I have no idea why, but maybe some serious players are contributing to it because they use it themselves. There isn’t a “Django” or Ruby+Rails for Rust, but if what you’re building is a lot of smaller APIs with various transport methods and data access in an federated authentication scenario then Rust is surprisingly mature for the web. It’s primary disadvantage being that your TypeScript and Rust developers won’t be able to cover for each other (which is why we didn’t poc with Java).
We have yew, and my personal favourite: leptos (https://leptos.dev/)
I mean who'd want to and why bother trying to support that use case? Basically none of Rust's value proposition exists for a web app while nearly every single one of the downsides do.
The single biggest problem I've run into is that there's no real good story for a web app framework (especially since everyone's gone crazy over async). Rocket is perhaps a bit too magical for rust folks and it's been abandoned, but I rather liked it. I've warmed up to axum, but all this async stuff still rubs me the wrong way.
And, yes, compile times still suck.
The number one syntactical beef I've is operators. e.g. == vs ===, ""+number.
any of which offer everything you've outlined
Mono is mostly only relevant if you need platform specific capabilities that wont be covered in .NET.
I'm talking about things like exhaustive "switches", built-in, ENFORCED handling of "missing" values (null, undefined), and finally useful error handling.
I actually don't need any of the baremetal features of Rust. It's just that most of it's "zero-cost" abstractions are still far superior to those of other languages.
(Some dislike it, but Java’s checked exceptions are exact homologous of result types)
Outside of that, yeah they just differ in very low-level aspects that most developers don't even know about.
Do crates have namespaces that ownership is verified for?
I am constantly baffled that NPM, PyPi, crates.io etc. didn't copy this idea for those last two reasons. In my mind, it's not quite best in class without it.
[0]: https://central.sonatype.org/publish/requirements/coordinate...
[1]: https://central.sonatype.org/faq/how-to-set-txt-record/
[2]: https://central.sonatype.org/publish/requirements/coordinate...
[3]: https://internals.rust-lang.org/t/pre-rfc-formal-squatting-p...
[4]: https://blog.sonatype.com/this-week-in-malware-may-13th-edit...
cough C#. You also get LINQ. And a fairly heavy amount of web frameworks in ASP.NET / Razor.
The pieces and features i wanted exist but tying them together was not always straight forward.
From the top of my head I used:
- actix-web
- tracing (logging)
- sea-orm
- config from yaml files and a custom implementation to allow profiles.
- utoipa (for openapi and swagger ui)
- thiserror
In a professional context I would not have used Rust for this project, but it was quite fun to explore it's possibilities and rich crate ecosystem.
I write my web apps in C++. They tend to be little bit more than just query database / update database. They're exposed as JSON based RPC and can be accessed by JS front end from browser or third party systems we interact with. The performance is stellar and the code size is not much different comparatively to using PHP / Ruby / Python / your_pet_goes_here.
But we already have plenty of options for this use case (.NET, JVM, Go, Python, Ruby, node.js, PHP, Erlang, etc. etc.). Very mature ecosystems that solve a lot of different edge cases.
There are very few C++ alternatives worth mentioning, none as mature as Rust in terms of adoption/tooling.
If you need to write a webapp that would require C/C++ kind of memory handling/performance then Rust would be the ideal candidate I think.
I don't know what kind of tradeoff matrix makes you use the Rust graydon describes over the existing options.
For example my main issue with D (but TBH that's something I've given up on actively tracking 10 years ago probably) is that by including GC in the runtime/stdlib - it basically painted itself as a poorly supported competitor to C# rather than a C++ replacement.
Indeed in software engineering there is no silver bullet nor a perfect language for everybody :-)
Basically, I view Grayson as a leader who set the tone for Rust being a language that was willing to take ambitious swings on cutting edge features. But I don't think he would have been the person to eventually make the cuts and compromises necessary to hew the language into a cohesive, mainstream language. Rust ending up as a replacement C++ helped it not only determine which features to keep and which rules to follow, but also helped it create the right pitch for developers to use it.
This does lead to a larger question about BDFLs. Perhaps, like CEOs, the BDFL you want when you're starting a language is not the BDFL you want when you're maturing a language, or maintaining a language. Especially around feature selection, in the beginning it may pay off to add a lot of features based on user feedback, but later on it may be better to push back more. And from a psychological standpoint, I have wondered about the pressure of being a BDFL. Grayson has been open about stepping down partially due to reaching his limits, and I suspect other BDFLs have thought about it too. The job sounds exhausting and thankless. At a certain point, wouldn't you want to leave and start a new project? And wouldn't we want the person who had success once to give it another shot?
If anything, it's more a C replacement than a C++ replacement. It will take some market share from both of course (and other languages to a lesser extent too), but functionality-wise, it just isn't (currently, at least) practically able to replace some C++ use cases.
Relative pointers are possible, depending on what you mean. Making this safe (e.g. preventing users of that type from breaking the relative addressing) is done via the Pin type.
Which means you can't track this. Tracking moves is fundamental here.
> Relative pointers are possible, depending on what you mean.
Pretty sure they're not possible in the sense I mean, for the same reason as above - you need custom moves for this. I'm referring to a pointer (not an offset; a pointer) that automatically adjusts itself when copied or moved. So that it always points somewhere N blocks before/after itself.
These are just two examples, to get the point across that Rust actually lacks some capabilities (since somehow that surprises people). You can find more.
I mean, arguably. If it is fundamental for your hypothetical use case, then sure, but this is not required for a lot of use cases, like smart pointers.
Nobody claimed otherwise. The question was what things Rust can't do, not what it can do.
I don't think anyone who still uses C today and hasn't lived in a cave for the last 20 years would be very interested in Rust since it's just not at all like using C. There is the rare Bryan Cantrill who for some reason was seemingly unaware that other languages existed from 1990 until 2018 but I think it's safe to say most other people who primarily use C would not prefer the leap to something like Rust.
For the people who use C++, certainly they're already used to a language that wants to dominate, so Rust should be fine. In terms of features, apparently it doesn't hold up, but because it's like C++ I'm sure they're just a couple of years away from adding those things too, and then the language can continue being more important than the actual data transformations the programs are supposed to be doing.
> but because it's like C++ I'm sure they're just a couple of years away from adding those things too
If they're smart (which they are), I'm sure they'll eventually cave and add some of the missing stuff, no matter how much they want to believe these features are unnecessary. Just like how C is finally coming around and adding generics and all that. The particular capabilities I mentioned here wouldn't be impossible to add, and they can probably achieve some of them better than C++ did. But I do think there will remain use cases that Rust will fail to accommodate.
If there would be something just like Go, but with a bit more powerful typesystem like Rust has (Option<T> instead of `err != nil`, and so on), and a simplified ML-like language instead of an imperative one... that would be my dream.
For whatever reason, C# devs seem incredibly resistant to even just looking at F#.
VS Tooling lacking versus C#/VB, no support for code generators, no support for Roslyn, no support for GUI frameworks, no support for EF tooling, many .NET vendors don't support projects if using F#, community likes to create their own wrappers instead of embracing standard .NET projects, ....
Rust is imperative too, though.
Yes. You have two choices: Interface implemented once, or virtual on all your public members.
I personally think Interface is the sane choice.
Would be nice if the .NET devs let us mock POCO's though...
I kinda like the language (its what I want basically) but the operational aspects are what I actually need and want first and foremost.
I think what it comes down to is that the creator/BDFL of Nim has an eclectic set of things that he simply doesn't care about and will never add to the language even though they are basically table stakes nowadays.
It's tooling isn't great though. I think its syntax is a lot nicer than OCaml's for example.
There is this wonderful language that is up and coming called Roc that look promising.
https://www.youtube.com/watch?v=6qzWm_eoUXM
language examples here -> https://www.roc-lang.org/tutorial
Disclaimer: I'm the author of said language :)
I'm curious how references in your language work. I see the very small example, but it doesn't explain much.
Some questions in that regard: Is `&T` a type? Can you store it in a structure, or return it? Can you have a reference to a reference? If you can have a function `f(&T, &T) -> &T`, how do you distinguish whether the reference it returns lives as long as the first or second argument? If references can't be stored in structs, how do you do non-owned iterators, or string slices?
> Is `&T` a type?
Inko's syntax for references is `ref T` for immutable references/borrows, and `mut T` for mutable ones. Unlike Rust, you can't implement methods/traits _only_ for references, instead you can only implement them for the underlying "base" type. So `impl ToString for String { ... }` is valid, but `impl ToString for ref String { ... }` isn't.
> Can you store it in a structure, or return it?
Yes.
> Can you have a reference to a reference?
No, `ref ref T` is "collapsed" into just `ref T`, and the language has no notion of pointers and pointer-pointers.
> If you can have a function `f(&T, &T) -> &T`, how do you distinguish whether the reference it returns lives as long as the first or second argument
Inko doesn't have a borrow checker, so it doesn't. Instead it relies on runtime reference counting to prevent dropping of values that still have references to them. Over time I hope to implement more compile-time analysis to reduce this cost as much as possible, but borrow checking/lifetime analysis like Rust isn't something Inko will have.
Or to put it differently, I want the compiler to catch say 80-90% of the obvious "this ref outlives its pointee" errors without complicated borrow checkers. For the remaining 10-20% the runtime check should suffice.
Not to be snarky (for once) but
- More powerful type system: that goes against the whole implementation culture behind Go and their wider philosophy
- ML-inspired instead of imperative: goes even more counter to the above
I have never tried Go and I will probably never care to try it, but I have never seen a language which manages to both be (1) simple in the Go-sense and (2) look remotely anything like an ML language.
Didn't they implement generics already?
https://go.dev/blog/why-generics
It seems like their implementation has enough power to do this in most cases, although "Option" is not built in.
A good example of this trade-off in Java is Lombok. A very handy library that legitimately avoids a ton of boilerplate, but it also absolutely tanks your build time. In a real system, a large one, your team is better off just getting good enough with their editor that they can generate the hateful boilerplate, and leave Lombok out. Because you'll be paying for Lombok all the time, and only need it a small fraction of the time. There are hundreds, thousands of these conveniences that are deeply tempting but should be avoided, in every build. The problem is that the programmers become attached to these little nicities and actively resist giving them up, even though they are so costly.
They are not almost instant once your project grows to a certain size, which is still well within the bounds of a realistic single company’s project (I work on Materialize which is all in rust).
Yes
> shared generics
I don't know what that is. Some googling suggests that it's a nightly-only feature, whereas we use the stable compiler. If it's something that can be done in stable, I'd appreciate a pointer.
> lld (or mold ideally)
$ echo $RUSTFLAGS
-C link-arg=-fuse-ld=lld -C debuginfo=0I think the next innovation in statically typed languages should be to somehow break the build-test-debug cycle. We basically have the same interface to programming as dynamically typed languages, despite having considerably more information available because of the types. This should be exploitable somehow to give typed languages more advantages, and change the very nature of the loop.
I've spent of hours interactively debugging Rust code, crawling through tracing output etc, and hours waiting for Rust code to link with mold because I added an eprintln somewhere, so I'm not convinced.
I love writing Rust and think it makes my life easier, but there's this stockholm syndrome about compile times. It sucks, and it's not the type system or borrowchecker's fault.
What do you mean?
How is that Rust's fault?
I wanted to delombok to save 30s off of each build, but the rest of the team(s) refused to let it go. I thought, and still think, that was a short-sighted mistake.
I don't think this is actually true right now and for a while.
>In a real system, a large one, your team is better off just getting good enough with their editor that they can generate the hateful boilerplate
Which you can get by delomboking if you really need to.
I know that the main point is about governance and how having a BDFL would have led to a completely different language but I really would have preferred the Graydon-BDFL-Rust to what we have today.
Very interesting article, worth a read.
> Traits. I generally don't like traits / typeclasses. To my eyes they're too type-directed, too global, too much like programming and debugging a distributed set of invisible inference rules, and produce too much coupling between libraries in terms of what is or isn't allowed to maintain coherence.
Very much this!
> I wanted (and got part way into building) a first class module system in the ML tradition. Many team members objected because these systems are more verbose, often painfully so, and I lost the argument. But I still don't like the result, and I would probably have backed the experiment out "if I'd been BDFL".
I seen this the ML first class module system mentioned in a few different contexts and want to pick it up at some point.
Being a relative rust noob compared to other languages, I always felt traits were a super power when compared to eg interfaces in Java/C++ and flexible but with useful constraints compared to the structural typing nature of Typescript interfaces
Which types grow new trait methods is dependent on trait implementations, which can be generic and apply to an unbounded number of types, based on complex matching rules, and figuring out requires reading every impl of that trait to see if any match a given type, or asking the compiler/IDE. I want to explore languages which explicitly select a trait implementation at the call site (like a Heap<int, int_greater> type which uses int_greater::cmp() to implement a min/max heap), or naming the trait (but not picking an impl) at the trait method call site (like Write::write_all(stream, "hello world") or (stream as Write).write_all("hello world")). I think Zig takes a similar direction.
(some_integer).div_ceil(&2)
^^^^^^^^
= warning: once this associated item is added to the standard library, the ambiguity may cause an error or change in behavior!
= note: for more information, see issue #48919 <https://github.com/rust-lang/rust/issues/48919>
= help: call with fully qualified syntax `num::Integer::div_ceil(...)` to keep using the current method
= note: `#[warn(unstable_name_collisions)]` on by default
For example, in the real world, there are multiple ways to order strings (called collation), but since there can be only one implementation of (String, Ord) type-trait pair, one ordering is canonical. That may be bearable, but having canonical (String, Hash) implementation is not. What if you want to use (say) faster CityHash instead of canonical MurmurHash? So Rust resorts to things like BuildHasher, to get back multiple implementations.
Because it is global/anti-modular, it interferes with separate compilation, and it is one of reasons why Rust is slow to compile.
Then why would one use trait instead of module? Since there can be multiple implementations in module, you need to specify. So module is more verbose. In my (and Graydon's) opinion, a bit more verbosity is worth it for modularity, but many people disagreed.
This is another theme: Graydon is okay with verbosity, boilerplate, and being bureaucratic. In my (and Graydon's) opinion, programming is work that is secretarial, not artistic, so it is unimportant whether code is ugly or not. You may think current Rust is ugly, but no, it is the way it is because lots of people really cared about Rust code being pretty. If Graydon was a BDFL, Rust would be even more ugly, and in my opinion, as a result, would be a better programming language.
Recently I've seen a lot of programmers I respect consider them a weaker or failed alternative to typeclasses, basically a dead-end. And I had been reluctantly coming to the conclusion that I must be wrong in some way I'm not able to fully perceive yet. Seeing such a notable PL designer come out on their side makes me feel less like a fool.
Canonical instances goes against modularity but something like modular implicits for ML-family languages could improve the terse was a lot to something closer to Haskell levels. You still miss out on the other advantage rust has: the dot operator is great because it allows identifiers to contain less context (they don’t need to have long global names or be hidden away in some tree of modules: you can have .map mean different things for different types) and gives a massive hint to autocomplete for which things to suggest (I don’t know how you even signal to autocomplete in a language like ML that you would like a function to operate on the following value so please suggest things with the right type).
But I guess enough syntactic sugar might make things palatable.
Selecting and enforcing usage of a subset of C++ is a way to deal with that mess which companies frequently use.
Sure, but that's why Carbon and Cppfront are a thing now. These two actively remove stuff from C++.
We'll probably see this happen in the Rust community as well once the newer Crab language becomes established. Then we'll have to rewrite everything in Crab, but it will probably be a lot simpler than the Rust rewrite.
Besides, the C++ committee does, in fact, remove stuff from the language to clean the mess, just not without due care.
I feel if the Haskell committee could simply pick one of the alternative preludes, and gather some packages into a new standard library, and announce it all as a Haskell 2.0 or Haskell prime or whatever it could really reinvigorate the community. But what committee would be motivated to do such a thing, bearing the responsibility if it went wrong?
I would really like to be able to specify what prelude I want at the .cabal file. AFAIK, that's one change that wouldn't break anything.
(Anyway, Haskell is one case of a language asking for a fork. Maybe it takes the form of Idris or some other derived language having some sudden growth instead, but you are right, the committee currently isn't as fast as the community.)
The bigger problem is that strings are broken: by default they are linked lists of Characters. You can use better strings (like Data.Text) and you even get to use them as literals in your source, but the language ecosystem of libraries mostly assumes that you are using the default strings. And converting back and forth is annoying.
Because of all the existing libraries, it's harder to route around [Char] by yourself.
Rust as an alternative to C++ does, for me, involve all of the weird magical nonsense that you kind of need to get any of this working. The extreme use of generics to build out DSLs to get things working. General libraries being very hard to write, but still possible, to get alright ergonomics for usage itself. And yeah... the zero-cost abstraction thing.
I think Graydon-Rust would have also been very interesting, but it sounds unappealing to me, person who wants "C++ but nicer".
But to his point... saying "you could have much faster compile times" is very tempting! Just, especially when it's messing around with Rust "for fun", the ergonomics sound pretty unfun.
So if you want to try something like that, just look at Go :)
StandardML uses a far superior Hindley-Milner type system. It has pattern matching. It has Option types for good error handling. It doesn't have bad features (for GC'd languages) like direct pointers and slices. It has immutability by default. CML even offers a better take on channels too. Modules keep interfaces more standardized (superior for big projects IMO). And to top it all off, SML is easier to learn than Go. It's also as fast as Go (despite the compilers being a side project).
Go's only advantage is more extensive libraries, but that would be fixable in SML with just a fraction of the money Google spent on Go.
But it also turns out the very same isn't as neurotically attached to his 'likes' as most of us are:
> The Rust I Wanted probably had no future, or at least not one anywhere near as good as The Rust We Got. The fact that there was any path that achieved the level of success the language has seen so far is frankly miraculous. Don't jinx it by imagining I would have done any better!
He understands that his likes are just contingent facts about a single mammal, not truths. This is a hard-won understanding. Most people never reach it.
Yep. That just goes to show that you shouldn’t put individuals on pedestals as if they are superheroes. More things than we usually think are in fact team efforts.
> I know that the main point is about governance and how having a BDFL would have led to a completely different language
That isn’t the only main point.
> > The Rust I Wanted probably had no future, or at least not one anywhere near as good as The Rust We Got.
If you would be fine with a niche language then that “Rust” would have worked for you. But if you also wanted a language with wide industry backin etc.—maybe not so much.
It really is an incredibly language and ecosystem, in large part, because of its performance potential.
To be clear, the only real options in this space were arguably C, C++, and maybe in some circles D in my mind. C++ and C by far had the mind share.
Had Rust gone the way Graydon wanted I don't think Rust would be so interesting in the OS and Embedded space. This is a space that it turns out is really ripe for change.
Embedded application are growing more connected, and more complex all the time. Security is a serious concern perhaps followed by or proceeded by performance depending on who you ask. Rust checks so many boxes off in this space its really hard to argue that it isn't a better solution.
Would you rather write a little embedded http server on an IoT device in C, C++, or Rust? What about an embedded networking stack? What about a mesh network stack? I know the answer I'd have every time for this myself.
,,A lot of people in the Rust community think "zero cost abstraction" is a core promise of the language. I would never have pitched this and still, personally, don't think it's good''
If the language makes compromises in performance, it's not a real C++ competitor anymore.
Some things are not about what people ,,like'', but that we need a language that is safe and can compete with C/C++ in performance for systems level programming, as most security problems in the world come from C/C++ memory management.
If it's significantly slower than C++, Mozilla couldn't have picked it up to replace C++ code base, as there was a huge competition in performance between browsers.
The trick is to find the right place to compromise so the cost is minimal overall even if it isn't zero.
In the case of automatically upgrading from int to bignum, or green threads, going lower level would be much harder.
I just wish Rust stayed focused on being the best systems level programming language (for example finishing the SIMD package, getting into stable Rust, getting language level GPU integration similar to CUDA / OpenCL).
PyToch, and now the newer GGML for example is still written in C++ for example
I still like the changes being made in the language, just not the focus.
I do agree though, the cost of the check is so small that it doesn't matter in most cases.
Think about it: we have dozens upon dozens of Hoare-esque “sacrifice some performance for ergonomics” languages out there. But what are you gonna do when you need performance that the language is too inflexible for? Hmm... perhaps write some of your app in C...?
If it weren’t for Rust and similar languages, we would never have a hope of moving on to more modern languages for those “can’t do this in my $mainlang due to performance” problems. How would yet another language that (according to Hoare):
> and I would have traded lots and lots of small constant performancee costs for simpler or more robust versions of many abstractions.
have helped with that? It wouldn’t! You would have still been stuck with having to bind to C, C++, assembly, or whatever else sufficiently “zero-cost abstraction” language.
"Complex grammar. I've become somewhat infamous about wanting to keep the language LL(1) but the fact is that today one can't parse Rust very easily, much less pretty-print (thus auto-format) it, and this is an actual (and fairly frequent) source of problems. It's easier to work with than C++, but that's fairly faint praise. I lost almost every argument about this, from the angle brackets for type parameters to the pattern-binding ambiguity to the semicolon and brace rules to ... ugh I don't even want to get into it. The grammar is not what I wanted. Sorry."
Personally I find rust to be not easy at all for humans to read, and so it's interesting that it's also hard for parsers to parse. Not sure what was optimized for in the design.
This machine perfect neatness comes from lots of small places in the syntax where Rust decided let's make the syntax permissive so that it's easy to autogenerate code with some macro or to make good diffs.
Then a systematic implementation of trailing commas and allowing leading/trailing separators in some places. The result is quite sterile and to me is no longer organically nicely readable.
tldr: rustfmt has no soul
I also found this bit interesting: "It's easier to work with than C++, but that's fairly faint praise".
I see this kind of thing in my own personal projects all the time. I'm thinking "oh it would be really cool if I built X" when in reality most of the time users just want really simple stuff.
Being easier to work in than C++ might be faint praise, but it's probably the biggest draw of Rust for me. I don't want to touch C++ with a 10ft pole, but I love using Rust.
I find Rust basically unusable -- at the level of abstraction I want to write code, basic definitions break line limits.
Rust seems to be a repetition of C++'s mistake: a language which conspires you to pretend it's another. There are now nearly as many Rusts as C++s.
If I return to any domains where Rust would be relevant, I'd probably now opt for Zig or equivalent.
via https://github.com/RustPython/RustPython/blob/main/jit/src/l...
Have a look at real-world rust repos that are more than simple application code.
Personally, I feel like I'm being visually assaulted.
use rustpython_vm.. builtins.PyDictRef, py_compile, py_serde, scope.Scope
use rustpython_vm.. InitParameter, Interpreter, PySettings, VirtualMachine
use rustpython_vm.pyobject.. ItemProtocol, PyObjectRef, PyResult
use logic.. ProgramError, ProgramResult
setup_scope :: (vm: ref VirtualMachine) -> PyDictRef =
let code = vm.new_code_object(run py_compile(
file = "stdlib/rumblelib.py",
module_name = "rumblelib"
))
let attrs = vm.ctx.new_dict()
let run : () -> PyResult[None] = fn
return? attrs.set_item("__name__", vm.ctx.new_str(own "<robot>"), vm)
return? vm.run_code_obj(code, Scope.with_builtins(PyNone, clone attrs, vm))
let sys_modules: PyDictRef =
v unwrap vm.get_attribute(vm.sys_module.clone(), "modules")
.downcast()
.ok()
.expect("sys.modules should be dict")
return? sys_modules.set_item("rumblelib", clone attrs as object, vm)
return ok(None)
vm.unwrap_pyresult(run)
return attrs
py_to_serde(type T) ::
(py: ref PyObjectRef, vm: ref VirtualMachine) -> ProgramResult[T]
let val = return? py_serde.serialize(vm, py, serde_json.value.Serializer)
let out = return? serde_json.from_value(val)
return ok(out)Some of your potential syntax changes seem less readable but they might look nicer to you such as changing `attrs.clone().into_object()` into `clone attrs as object`. You probably come from a python background I would assume but the rust version is easier to parse and understand but maybe to you not visually appealing.
Having an explicit return argument I could understand. I found this weird in the beginning but this is something I got accustomed.
Not sure why square brackets should be favored instead of angle brackets. The imports in the Rust version are also very explicit and fine I think.
Replacing `()` with `None` seems silly, same with replacing `Ok` with `ok`.
Maybe the `py_to_serde` arguments of the function definition spanning 4 lines could be considered unastethically pleasing. This is something I noticed quite a lot with Rust code. Fmt also seems to break to multiple lines quite fastly when writing function or method definitions.
I replaced () with `None` (& with ref, ! with run, etc.) because I want code to read like english (ie., literate) all other things being equal. I dont find `()` "pays the cost" of illegibility by syntactical convenience.
Changing "operator-like methods" to operator words would have a radical impact on the parsing precedence and "overall look" of the lang. ... so `clone ...` would be more readable if those changes were made. The advantage of many keyword primitives is that you dont keep reusing the same syntax for everything... syntax is there to support expression. I think more is better.
`Ok()` to `ok()` was more a broad notational philosophy of basic type constructors being unobtrusive.
Scala has gone from C-like (v2) to python-like (v3) syntaxes --- and I think the ML-ish whitespace, python-ish "beat poetry" approach is mostly just better.
I buy some arguments that symbolic redundancy can help (ie., both indent and use symbols)... but Rust's syntax philosophy is clearly to "keep adding symbols", and I dislike it.
C uses symbols, but I do think C is quite beautiful mostly -- because it's so simple. Rust is creaking under the weight of its size and symbolic choices
When writing code you are pattern matching and certain special symbols such as `()` over `None` and `attrs.clone().into_object()` over `clone attrs as object` are just faster to detect at an instance.
Using more english prose becomes a word salad that is harder to parse than using special symbols which convey meaning. Certainly more familiarity with a programming language and it's syntax elements will help you your pattern matching mechanism and allows you to understand code faster. In my opinion Smalltalk gets it quite well in this regard.
I also would rather prefer lisp language syntax, or apl.
You seem to want contradictory things — if you don’t need that level of control just use any of the litany of high level languages with nice syntaxes. I don’t think we can eat this cake anytime.
This makes brining in the "full power of dynamic languages" almost trivial and extremely performant.
Delete half of rust's symbols, get rid of its macro system, rewrite it until lifetimes are inferred /or/ allocators are explicitly chosen, etc. etc.
I don't want to feel visually assaulted when writing the type signature of the sort of function common in a dynamic language. Here `mypy` is also treasonously guilty.
Consider, eg., zigs "function which returns a type at compile-time" is an example of where blindingly-obvious syntax retains its blindingly-obviousness because of compile-time eval... ie., we dont need a "second syntax" to program the compiler.
This "two syntaxes, one for the runtime and one for the compiler" approach -- has swamped Rust as it aims for greater expressiveness. Not least, because it has a third syntax: one for unsafe.
Safe, Sound, Complete 99.999% of the time; a joy to use 100% of the time
What are you referencing, concretely? For example, "unsafe" doesn't add or remove syntax.
And you'll find about "4 languages" all mixed together.
Most "syntax" in AOT, statically typed languages does not directly generate machine code, but it does directly impact what machine code is generated. So there's not a clear distinction in practice.
For example, a lot of the syntax is used to control method selection and verification - that's not a "different" syntax or language by most folks' definitions.
Or below, let's invent a language where there's (in my sense) "one syntax for everything",
Eg., consider something like,
const SimpleTrait = trait with:
val name
def GenericApiTrait(type T) =
return new trait with:
def response : self -> str
const MyAPI = GenericApiTrait(SimpleTrait(new class)) with:
val name = "World"
def response : self -> str = f"Hello $name!"
# later on, in the app,
def calc(x, y) : int, int -> int =
return x + y
# ie., the **same** syntax as that earlier which targets compile-time
Whereas in Rust, the syntax for returning values and "quantifying over types" is radically different.
This is the historical approach (C++, C# .. almost all langs) -- but not one modern innovative langs follow.
I think with as much desire for novelty rust has, cramming it all into its "compile-time syntax" has hobbled it.
A simple uniform language for both c-time and r-time could be used
One of the reasons that something like unified syntax is not preferred is because there are some important distinctions between runtime and comptime, and a lisp-like syntax doesn't translate well because lisp doesn't have a meaningful distinction between the two.
For example, it's desirable for some folks to have the "comptime" work (declaring types, imports, function signatures, etc) to have a purely declarative syntax. And languages like Rust use declarative syntax to express things about types, interfaces, and their constraints. Contrast that with the logic of the program which is more convenient for a programmer to express in the imperative style.
Feel free to do this. I predict you will end up with a language that appeals to nobody but yourself
I imagine it has more users than just me.
My version would, of course, be Nim++
Show us an extant language that does this, then.
Partial Evaluation (automatic) was a very unsolved problem, last I looked.
The fact that it is a law-level language best fit for system programing came much later. Late enough that there are people here and there still trying to use it for the original use-cases.
PHP show that a language only needs to get that one thing right.
Rust have found its niche. Graydons vision seem to be a more elegant language which would compromize on the points which actully make Rust succesful.
This is a wonderful comment, and puts into words something I've been thinking for a long time. The best general-purpose languages always seem to start out with a strong niche, then grow from there.
> (Swift at least traps in release by default -- I wish Rust had chosen to).
I enable it in release on serious projects:
[profile.release]
overflow-checks = trueIt's very annoying , but I use it in code where this is critical.
incorrect crypto code. If overflow is intentional, it should be annotated as such in operations, will generate similar assembly and won't panic. If it isn't intentional, then the code was bad to begin with.
Anyone writing these things today in C or C++ already understands object lifetimes and Rust just adds a static checker for them.
In such projects churning out lines of code is not the bottleneck, ease of development should not be prioritized over long term maintainability.
Why on earth would you try to rewrite python CRUD apps in Rust?
As someone who only used Rust casually - understanding object lifetimes and knowing how to encode this in Rust type system is not the same thing. Not to mention that Rust can't statically prove some things that are valid (eg. cyclic references).
I've certainly seen seasoned C++ programmers saying this. Of course, a few of them of say they understand object lifetimes so well, they they don't need the static checker!
> Why on earth would you try to rewrite python CRUD apps in Rust?
This is one of the great mysteries of our times. I do think the enthusiasm for using Rust for web apps and such (a) is misplaced and (b) has been a drag on Rust developing into a better C++ replacement.
There is a difference between understanding lifetimes and being able to keep track of them. I have a lot of objects in my more than 10 million lines of code. Most of them have simple lifetimes that are easy to track, but a few for reasons (which may or may not be valid - often the reasons are it was built in C++98 and updating to modern lifetimes is hard when it is used all over) have complex lifetimes that are tedious to track. It isn't that I can't, it is that I get bored/make mistakes and the static analyzer wouldn't (Or course C++ can't be statically analyzed, but if it could the static analyzer wouldn't fail for the same reasons I fail)
That's why we're stuck with lifetimes as function contracts.
Because Rust has a lot of incredibly helpful features that make bigger systems far less of a pain to maintain. I work in Java/Kotlin, and my entirely gut-based estimate is that 66% of problems wouldn't happen in Rust.
My big favorites are: - Sane, well-defined, enforced, opt-out error handling - Sane, well-defined, enforced, opt-out handling of "missing" values - Exhaustive switching - WITH usable ADTs (enums) for encoding valid state
None of the web languages I know have all of that.
But even Java is pretty much there: checked exceptions are available (and imo superior), but Optional is also an.. option. Kotlin/scala can handle nulls, but java can also statically analyse every usage with annotations.
switch expressions are exhaustive in java over enums and sealed ADTs. (Rust has a misnomer, their enums are ADTs, java has both real enums and ADTs now).
To give you a few examples:
- = instead of == for equality would have been the natural choice
- := for assignment is similar enough to what is used in math for definition, so that languages like Pascal use it
- <> for inequality is something SQL got right
Smaller things that bug me are the ubiquity of the double colon (::) and the weird mixture of snake case and camel case conventions.
And not to leave the wrong impression, I think Rust got many things very right. My personal highlights are:
- -> for the return value
- concise keywords like `fn`
- `where` for constraints
In general more Algol/Pascal and Haskell - less BCPL and C/C++.
I think its cppfront that is taking the approach of `:=` being a declaration with the type being inferred (ie shorthand for `: Type =`). Reading up on that has made me the most ok with applying this to functions (which I see coming up more these days) but I think i still prefer functions having a more distinct look as I process them differently when reading. Now, cppfront's approach to types I think is bonkers, making critical details hard to find except maybe through convention.
https://github.com/hsutter/cppfront
> <> for inequality is something SQL got right
Maybe I'm not recognizing the biases of my own learning background but this never reads right to me vs "not equal" / `!=`.
> - concise keywords like `fn`
In other discussions, it sounded like Graydon had an upper limit of 4 characters for keywords
https://www.reddit.com/r/rust/comments/13oemrg/question_abou...
For me, I had a "whoosh" moment for `fn` and always thought it a weird abbreviation, completely overlooking "fn" keys on laptops.
I don't understand the subtleties here: Is tail calls an optimization the compiler can do irrespective of the language? Or is there something that prevents this and requires the compiler to use stack here? Is there any visible effect to the programmer from supporting tail call or not, other than performance and stack depth?
How does not supporting them compete with C++ performance?
If rust says they don't support it, does it mean they don't even allow the compiler to do it?
Obviously the syntax of the language itself already supports calling the function itself, and whether tail call optimization is supported or not doesn't affect the visible result afaik (except in case of stack overflows / performance)
Or like xmtp or mail or whatever, you generally write your client assuming you are going to talk to any spec compliment server, which can stop you from using some extensions if you aren't sure they will be supported.
Possible is not the same as guaranteed.
Some constructs cannot be optimized for tail recursion. Languages with tail calls have guidelines of how to must write your code to ensure the tail call happens - often a seemingly small code change is the difference between tail call optimization happening or blowing up the stack.
Second, in at least some cases where the optimizer can apply tail calls you need to be guaranteed it will happen. Nothing stops a C++ optimizer from applying tail call optimization (I don't know if any do, but it is allowed in some cases), but the language doesn't require it, so even if your optimizer supports it you can never know that it happens - and more importantly you cannot be sure that after changing the code or upgrading your compiler you will still get it. Thus even if your optimizer supports tail code optimization you dare not do deep recursion.
If your language doesn't have support for tail calls you cannot do deep recursion with confidence. If your language does, then you can do deep recursion so long as you follow the rules of the language. (whatever those are - I'm not up on the latest research here, so I don't know the state of modern tail recursive languages are.)
Now, 256MB is pretty huge, but we probably want way more memory kept for heap storage and similar things (because managing lifetimes purely on the stack will be hard and might require a lot of copying of data), so it will be less than that. Now, you may ask, "Why not start with small stacks and make them bigger as needed?" It's a good question, but since our stacks are contiguous areas of memory and we don't know what's in them, we still have to space them out in our memory map allowing for as much growth as is needed.
We might solve some of these problems by introducing segmented stacks, but this is one of those problems that crosses so many language, runtime, and OS boundaries that it's been hard to do in the general case, and it feels like gently nudging people towards writing code that can be tail-optimised is easier, just as it's easier to push people to use async than it is to provide systems and APIs that would allow for blocking code and a huge number of threads.
The problem is adding TCO to a language that was born without it. It won't break existing programs but you'll be able to run a program using TCO only with a specific compiler or interpreter from a given version. It could catch up if the main implementation of the language does it, but often we care about compatibility with the alternative implementations. Think about Ruby and Python, MRI and CPython and all the other implementations of the language. If I'm not wrong, MRI added TCO under a runtime configuration variable but JRuby doesn't support it.
Therefore _yes_ the compiler can always do it, but it may involve patching the called function to do some work. Splicing code into it and/or changing the calling convention. That's difficult to do for unknown caller/caller pairs, e.g. function pointers or separate compilation to machine code without enough metadata to patch it later. It runs a bit close to "sufficiently smart" compiler which usually means doable in theory but unlikely in practice.
If you pick global designs to make them easier - probably most notably having the callee clean up the stack frame from the caller - and that makes other things slower, then you've given the competition an edge.
Clang now has tail call annotations for C++, which works mostly because the compiler can reject them in the cases where it hasn't implemented the lowering. Rust can probably have it in the same approximate circumstances as C++.
I’ve always seen safety and lifetimes and borrowing as the main value prop, so I was surprised to see he was sort of aiming at an ML without GC, rather than a C++ that doesn’t blow up.
For example rust in it's current design relies quite a bit on the combo of: generics monomorphisation + inlining + dead code elimination. But this reliance doesn't play well with an stable ABI. Similar features like generic associated types and similar do not make the story easier. Like an rust stable ABI likely would only support `dyn Trait` and no generics or "on the fly" provide a `dyn` variant for generic method where possible. But even that isn't really good enough for a lot of use cases in a lot of different ways. Additionally a ton of important rust things are not `dyn` compatible at all.
Even some of the "easy" to solve things aren't that easy for non-technical reasons. E.g. a lot (but not all) of `impl Into<T>`, `impl AsRef<T>`, `impl Borrow<T>` etc. cases are best handled for a stable ABI context by aplying the single function of the trait (`.into()`,`.as_ref()`, etc.) _before_ calling the function and only having an ABI stable version for that. The side of which traits qualify for this is easy (you annotated the traits) the "when not apply it even if it seems valid" part isn't that easy not for technical reasons but for communication/documentation/avoiding unexpected outcomes reasons.
I recently revisited it for Advent of Code (where part of the challenge was getting the compiler itself to even build with even semi-modern tooling), and a lot of the above features still have value.
What is the rust experience wrt to inlining? Can every expression be inlined or only selected ones? How can you know what got inlined in some expression? Do you have to manually annotate every single function call you have to inline or is there a more general command?
Stdlib containers are all in the generics category.
Is this a problem is rust? Is too much/too little marked inline? What if you really need to inline some function from some library that was not marked inline by the author?
If you implement these things in the compiler (open code means emit the implementation inline as you go, can also emit calls to the compiler runtime which is roughly similar to library code that the compiler ships and knows lots about) then users need to hack the compiler to change them.
However, if the structures are in the compiler, and you've done things like encode them directly in the AST, the compiler has a better chance of emitting useful diagnostics for them and of optimising them at the semantic level of the container.
C++ goes with library code supported by compiler intrinsics, and a common developer experience is compilation errors referring to iterators some distance into the library code. It also can't sanely do things like call reserve on a vector outside of a loop, because by the time it's ready to optimise things it's holding raw pointers with mangled names, not a hashmap instance.
Conventional wisdom is to put containers in the stdlib. D has some support in the compiler. I'm starting to think this is one where conventional wisdom has got it wrong.
Overall, a really interesting article. Though I like today’s Rust, I do think I would prefer the trade-offs made by the alt-Rust outlined here.
Perhaps it’s just my own personal preference, but I think there is a strong bias in users towards what they are already familiar with, and it’s hard to break away from those without a BDFl or similar position of authority who can impose their vision.
I think whether you view that as benevolent or not (I would, because I agree with lots of de facto development standards being detrimental to language quality, but I understand that that is my opinion and not hard fact) is in the eye of the beholder.
Makes me wonder if things like Linux, or C++, were historical anomalies, the stars aligned. How many good projects fail because of 'loosing arguments' that should have been won, or the community didn't form, etc... a million things..
I maintain the actor model is probably the most theoretically perfect concurrency and distributed computing model. The holy grail. We just don't have the right hardware for it and it's extremely limited by addressability issues with current technology.
So I don't really find this surprising, nor disagreeable. It's just not a model that Works Well at present.
You just have to be aware that actors aren't going to magically get you more CPU cores — i.e. you can have as many IO-bound actors as you want; but a CPU-bound actor (done correctly, such that "gets out of the way" of actor scheduling) is just a regular CPU-bound preemptive OS thread; and you can only realistically have as many of those as you have CPU cores in your machine, before you start experiencing highly degraded performance.
Most systems don't need more than 100 (different) CPU-saturating things to happen at a time. If you do, actors won't save you... but nothing else will, either. You'll need to scale horizontally. (At which point, the actor model becomes very useful from another perspective — that of transparent distribution of messages between nodes.)
But tbh, there's a reason that, even on the systems Erlang was originally designed for and is "idiomatic" for — those being telecom packet switches — the Erlang software only performed the role of the control plane. There was also a data plane in each of those boxes — some kind of FPGA or ASIC — designed specifically for the job of applying a list of active routing rules at each input port, such that packets on that port would be unwrapped, maybe filtered, route-matched to an output port, maybe buffered to combine with other packets, then re-wrapped and emitted at said output port. The job of the Erlang code was to listen for signals bubbled up from the data plane, in order to build complex state-machines, do accounting, etc., resulting in change commands being pushed back down into the data plane ruleset.
Actors are great for keeping a bit of local state in order to make decisions, and coordinating with other actors and their local state to make more complex decisions.
Actors implemented naively — where all actors are uniformly green threads — are not so great for doing the same thing over and over, at scale. Telling N actors to do the same thing is no substitute for a DSP, nor for a tensor core.
But this isn't an indictment of the actor model, nor of current hardware, but rather of the current state-of-the-art in actor-model languages. You could totally create an actor-model programming language where actor-pools that are doing SIMD are transparently "promoted" during compilation into GPU shaders / FPGA gate-networks / etc. Nobody's done it, but there's nothing stopping anyone. (Anyone interested in trying this could probably do a proof-of-concept on top of Elixir's Nx library.)
Which is precisely why I stated that they're the best model theoretically that simply do not work on our current technology.
I don't think a language solves things. It's a fundamental shortcoming of our technologies, routing topologies, etc.
Both Swift and Rust both veered away from their original champions. The champions helped by focusing the problem and providing a technical skeleton, but the need (the pain of C/C++/Objective-C) was both intense and complex, so the community was stronger than the BDFL model.
Interestingly, Swift has seen Rust forge ahead on a number of fronts, but is quietly adopting the best of Rust, and soon interoperating with C/C++ will be frictionless. The ties to Apple are being loosened, with a more portable stdlib and a Foundation library that subsets the legacy Apple Foundation instead of dragging Apple API's into other platforms. If/since Apple is to rewrite its systems in Swift, Swift will likely evolve into the best language for migrating off C/C++.
Compare Graydon, von Rossum, or Chris Lattner to Java's Mark Reinhold. Mark has been quietly at the helm of Java since 1997, navigating: the Oracle and open-source transitions, partners ranging from IBM to broad developer communities, continuous VM updates that kept Java relevant, and the quick pace of recent language/library upgrades: lambdas (method and field handles), vector processing and FFI, native...
I remember that one. The change was shortly after I started fooling with Rust and was major. Major as in it broke all the code that I'd written to that point.
"Async/await. I wanted a standard green-thread runtime with growable stacks -- essentially just "coroutines that escape, when you need them too"."
I remember that one, too; it was one of the things that drew me to the language---I was imagining something more like Pony (https://www.ponylang.io/).
"The Rust I Wanted probably had no future, or at least not one anywhere near as good as The Rust We Got."
Almost certainly true. But The Rust We Got is A Better C++, which was never appealing to me because I never liked C++ anyway.
Rescript lets you skip most of those, plus the (overblown but also real) weird syntax. I like ocaml and use it for projects still but if you just want to play around with what's unique and strong about it, rescript is what I would recommend right now.
My interest back then was in a higher level language with type inference and a modern ML-like type system but that could be used for systems programming, especially in database, virtual machine, and even operating system dev.
These days I work pretty much full-time in Rust, and I think Rust as it is today delivers on some of that promise, but not all. I feel like the language's borrowing and ownership checking are pretty brilliant but really begin to become a pain when dealing with nested and interrelated trees of objects and iterators (like if building a compiler or query evaluator, etc.), and resorting to Arc/Rc/RefCell, etc. feels awkward.
I'm not sure if the language Graydon talks about here would have been better for that or not.
But I'm also happy we have Rust, because it's an improvement over what else is out there, and I hope the community gets through its growing pains.
Realizing this, I thoroughly feel the need for a “Rust, the good parts” doctrine.
A good portion of use-cases could be successfully implemented with a small subset of language. The small subset doesn’t need to be any more complicated than Go. And in doing so, we’d be reducing the entry barrier for masses and encouraging wider adoption.
Edited: For clarity
Also, what is the identity crisis that JS is having?
>"Hello, you've been (semi-randomly) selected to take a CAPTCHA to validate your requests. Please complete it below and hit the button!"
...pops up and the button doesn't actually work. Truly one of the worst blog hosts out there if you actually want everyone to be able to read what you write.
One of the things I like about Delphi is that it has some powerful types as compiler intrinsics. Sets, strings (the compiler-generated code does call into RTL methods for things like finding substrings, but the string type itself), and so forth are all compiler-generated.
I would like to see more, in fact: I think a map type would be a great inbuilt addition. (What I'd really like is compiler stubs so you could link in your own implementation. Whatever is linked in, it's then heavily optimised by the linker to be inlined etc as appropriate.)
Graydon wanted something else even before Rust 1.0 was released. He wanted an OCaml-like language with modern Go/Erlang-inspired higher level concurrency abstractions.
- Explicit lifetimes are what make rust what it is. It would have surely failed had they not been introduced.
- I disagree about having a first-class module system instead of traits. Coherence and implicit instance resolution are a core value of Rust. `Send / Sync` are key examples of that.
- Green threads probably wouldn't been viable for rust, given the kind of programs it's targeting.
- Pretty much everything else however, I agree with.
I see the ongoing attempts to add linear types for low level coding, alongside automatic resource management more future proof.
At least in terms of his personal preferences. I don't think it's true in terms of market share, given the massive success of Rust.
Isn't that a matter of self selection? A language which developed around those priorities would have a community sharing those priorities now.
The point is if that community would be as large as the current one, larger, smaller.
Any ideas what he meant here?
What it BDFL?
"Benevolent dictator for life (BDFL) is a title given to a small number of open-source software development leaders, typically project founders who retain the final say in disputes or arguments within the community. The phrase originated in 1995 with reference to Guido van Rossum, creator of the Python programming language."
What alternatives are there?
Does anyone have any insights on the particular issues with actor-based concurrency vs. "direct parallelism like threads or locks" that he might be thinking about here?
Interestingly like Graydon suggests this and 'tis something D has, you can only have `ref` for function parameters. This is something the users sometimes complain about, but I guess it has positives.
Silently using invalid value is the worst choice.
And every case it is wrong
Financial math is not hard when you know how
Use integer types not horrific cludges like "decimal"
Just because IBM did it does not make it right. It is wrong
Wow, so one of the principal creators understands that that's what it is, not a replacement for everything under the sun.
And it's a GREAT c++ alternative, but ffs please stop telling me to use it for everything from webdev to embedded.
I never understood that one either. There is always only one "solution" that will make your program compile, so why not just let the compiler figure it out?
I did not mention lifetimes, but the principle is the exact same: the lifetime annotations are an API promise, and so inferring them means that a change in the body of the function would change the promise, breaking other code.
TFA loses me here. I rather like the Fn/FnMut/FnOnce business, though yeah, closures of dynamic extent are very limited unless you Box them to make them of indefinite extent... and so the whole language lacks that character that functional languages with GCs have, but it's still functional, just functional with a straight-jacket.
Earlier in TFA there's a mention of exterior iteration as in generators, and I want to point out that while generators are very nice, they are not a substitute for closures. Icon, for example, had iterators and first-class co-routines ("co-expressions"), but no closures, and so where one needed closures one had to use co-routines (costly!). It's true that with generators one needs closures less than without generators, but still, closures are very important.
> Traits. I generally don't like traits / typeclasses. To my eyes they're too type-directed, too global, too much like programming and debugging a distributed set of invisible inference rules, and produce too much coupling between libraries in terms of what is or isn't allowed to maintain coherence. I wanted (and got part way into building) a first class module system in the ML tradition. Many team members objected because these systems are more verbose, often painfully so, and I lost the argument. But I still don't like the result, and I would probably have backed the experiment out "if I'd been BDFL".
This loses me too.
It's great then that Rust didn't have a BDFL! :)
> Underpowered existentials. The dyn Trait mechanism in Rust allows runtime and heterogeneous polymorphism, a.k.a. Existentials. These types are useful for many reasons: both solving heterogeneous representation cases and also selectively backing-off from monomorphization or inlining (when you have those) in order to favour code size / compile time or allow dynamic linking or runtime extension. Early Rust tried to use these extensively (see also "first-class modules") and actually had an intermediate-rigidity type called an obj that was always a sort of Cecil-like runtime-extensible existential glued to a self-type record that allowed method-by-method overriding at runtime (almost like a prototype-OO system). Today's Rust strongly discourages the use of any such dynamic dispatch, a feedback loop arising from both technical limitations placed on them and a library ecosystem that's taken that as a sign never to use them.
This, on the other hand, is a brilliant observation and I agree with it as with much else in TFA.
> Tail calls. I actually wanted them! I think they're great. And I got argued into not having them because the project in general got argued into the position of "compete to win with C++ on performance" and so I wound up writing a sad post rejecting them which is one of the saddest things ever written on the subject. It remains true with Rust's priorities today, I doubt it'll ever be possible across crates (maybe maybe within), but as with stack iterators IMO they're a great primitive to have in a language, in this case for writing simple and composable state machines, and "if I were BDFL" I probably would have pointed the language in a direction that kept them. Early Rust had them, LLVM mostly made us drop them, and C++ performance obsession kept them consigned to WONTFIX bug status.
Oh dear. TCO is essential, IMO. I understand that it may not be possible in cross-crate cases, but still, TCO is very important.
1. He hasn’t been involved in the language for a long time
2. His vision for the language was completely different compared to how the remaining developers ended up designing it. So if I ended up liking Rust under his “BDFL”ing then it would be for completely different reasons compared to why I like (and dislike) Rust today