But Rust works badly with mmapped (memory-mapped) files, as the article notes. So in C you could load (and save!) stuff almost instantly, whereas in Rust you still have to de-serialize the input stream.
Rust "works badly with memory mapped files" doesn't mean, "Rust can't use memory mapped files." It means, "it is difficult to reconcile Rust's safety story with memory maps." ripgrep for example uses memory maps because they are faster sometimes, and its safety contract[3] is a bit strained. But it works.
[1] - https://github.com/BurntSushi/fst/
[2] - https://github.com/BurntSushi/imdb-rename
[3] - https://docs.rs/grep-searcher/0.1.7/grep_searcher/struct.Mma...
And in C, you have to design your data structures to be mmap friendly anyway. Same deal in Rust.
But this is moving the goal posts. This thread started with "you can't do this." But you can. And I have. Multiple times. And I showed you how.
So your code operates directly on a block of raw bytes? I can see how that can work with mmap without much problems.
My argument was more about structured data (created using the type system), which is a level higher than raw bytes.
Correct. It's a finite state machine. The docs of the crate give links to papers if you want to drill down.
> My argument was more about structured data (created using the type system), which is a level higher than raw bytes.
Yes. You should be able to do in Rust whatever you would do in C. You can tag your types with `repr(C)` to get a consistent memory layout equivalent to whatever C does. But when you memory map stuff like this, you need to take at least all the same precautions as you would in C. That is, you need to build your data structures to be mmap friendly. The most obvious thing that is problematic for mmap structures like this that is otherwise easy to do is pointer indirection.
With that said, this technique is not common in Rust because it requires `unsafe` to do it. And when you use `unsafe`, you want to be sure that it's justified.
This is all really besides the point. You'd have the same problems if you read a file into heap memory. The main problem in Rust land with memory maps is that they don't fit into Rust's safety story in an obvious way. But this in and of itself doesn't make them inaccessible to you. It just makes it harder to reason about safety.
You can mmap files in Rust just fine, but it’s generally as dangerous as it is in C.
Rust has plenty of situations where you do unsafe things but wrap that in safe APIs. If you’re returning regions of that mmapped file, for example, a lifetime can be associated to those references to ensure that those are valid for the duration of the file being mmapped in the program.
It can be used to ensure that if you need to write back to that mmapped file (inside the same program) that there are no existing references to it, because those would be invalid after an update to the file. You need to do the same in C, but there are no guardrails you can build in C to make that same assurance.
I've never heard of this trick. And my first reaction is "That would be a nightmare of memory unsafety if I did it in C++"
What's it used for? IPC?
I know some very skilled C++ and Rust developers who can pull it off. If you're at that skill level, Rust is not going to get in your way because you're just going to use unsafe and throw some sanitizers and fuzzers at it. I wouldn't trust myself to implement it.
It has a very specific meaning in Rust: the user can cause memory unsafety if they make a mistake.
> I think a better word is “error-prone.”
The issue with the connotation there is that it's not about the rate of problems, it's about them going from "impossible" to "possible."
“A commission attributed the primary cause to general poor software design and development practices rather than single-out specific coding errors. In particular, the software was designed so that it was realistically impossible to test it in a clean automated way.“
Ergo, concurrency doesn’t kill people, people do.
Dangerous means dangerous. It's not up for interpretation.
Languages have multiple, very different words, for exactly this reason.
most rust folks who use mmap don't mark the region as Celled, which means they risk UB in the form of incorrect behavior because the compiler assumes that the memory region is untouchable outside the single Rust program, and that's not true
(it's also not true generally b/c /dev/mem and /proc/pid/mem exist, but it's beyond Rust's scope that the OS allows intrusion like that)
In other words, mmaped files are hard, and Rust points this out. C just provides you with the footgun.
C has this issue as well, but Rust's compiler/borrow checker is particularly strong at this kind of analysis, so it's potentially bitten even harder.