Rust lifetimes: Getting away with things that would be reckless in C++
randomhacks.net
randomhacks.net
template<typename T>
std::pair<std::string, std::vector<std::string_view>>
tokenize_string(T &&str);
Which isn't to say the Rust solution isn't totally cool. Being able to easily check this class of errors at compile time is probably a lot nicer than needing to learn all the relatively complicated parts that would go into a easy to use / safe /efficient / still slightly weird C++ solution.
Which is to say, yes, you can obviously write C++ code that works. But you run the risk that one tiny mistake, or a change weeks, months, or years later, causes memory issues. Being able to completely rule out this class of error at compile-time is really amazingly useful.
Of course you can jump through more hoops, by encapsulating the string, string_view data in a proper class. But when there's tradeoffs between safety and convenience, there's a point where people will understandingly choose convenience.
Really I think this is the biggest problem with Rust. The stuff broken about C++ isn't really its memory model, it's simply its complexity. And every time I look at what's happening in Rust I see the same kind of complexity being introduced at every turn. I mean seriously: "named lifetime parameters"? And the syntax is a bare single quote? That's a joke, right? No one but Rust nerds is going to grok this, and real code needs to me maintained by mediocrities.
Frankly Go seems to have this "don't introduce new nonsense" problem under much better control, so if I need to pick a horse in the race, it's that one, not Rust.
Memory safety bugs are particularly bad bugs, as they represent a program that is completely off the hooks, allowing an attacker to take control of the program/computer by inserting shell code or to read private keys directly out of memory (for example). Even if you're not in a situation with aggressors, memory safety problems can lead to the program scribbling all over memory resulting in random crashes and misbehaviour, and these are so hard to diagnose since there's spooky action at a distance, and they can easily be timing sensitive/heisenbugs.
The memory model of C/C++ is definitely broken, look at all the security flaws in programs in those languages.
> Frankly Go seems to have this "don't introduce new nonsense" problem under much better control, so if I need to pick a horse in the race, it's that one, not Rust.
If you're doing something were Go is the appropriate tool for the job, that's sensible... but, if you're trying to write a library to be called from other languages, or run on embedded devices, or have precise control over memory layout (including the heap), languages like C, C++ and Rust will likely be much easier.
Yes, a proper typing system involves more visible moving parts than a broken typing system. But a proper typing system is vastly more powerful. C++ may not have lifetimes, but every time I touch C++ I have to be really careful that I don't accidentally introduce memory or threading bugs, because they're both really easy to do. Someone who's programmed C++ for a long time might not even realize it, but then again, the vast majority of C++ code in this world almost certainly contains numerous memory and threading bugs.
And why the comparison to Go? Go doesn't live in the same world as Rust. It's not a systems programming language. At this point it's mostly a glorified web server application language. And if you're going to pick Go because of the lack of memory lifetime issues, you may as well pick any other garbage-collected language out there.
And btw: the idea of calling something a "systems programming language" that can't even make a system call is laughable. Can I mmap a buffer to pass to a hardware video codec or make an ioctl to an audio driver in rust? Guess which language I can do that in? If "Go doesn't live in the same world as Rust", then Rust isn't in the same galaxy as C/C++.
If you quibble that it's not in the standard library, note that we are talking about the actual languages here, not their standard libraries (that crate could easily be pulled into the official distribution if it was deemed sensible to do so).
Therefore it is clearly in your interest to use languages that remove various classes of bugs. No language claims to be able to prevent bugs. But if your compiler can prevent you from making silly mistakes, that's a win.
> I'm saying I see Rust falling down the same "cool new ideas" rabbit hole as C++, and Haskell, and Erlang, and Common Lisp, and... Of those languages only one has achieved any notable success.
What bizarre argument that is. Haskell is a research language, what else could it be but be about "cool new ideas"? And it does effectively remove different classes of bugs, though like most functional languages, it's a managed language, and thus not terribly suitable for low-level programming. Erlang was designed to solve a certain class of problems, and it clearly solves these issues better than C++. Not everything and everybody needs to solve low-level issues or use inline assembly.
Rust can use unsafe code (guarded by unsafe blocks), but this code has to expose a sane interface. So you can do pointer trickery that is otherwise impossible, but instead of having an entirely unsafe program, locations with dangerous code are clearly outlined.
Serious question though: has anyone ever done this? I mean, are there kernel header import utilities I can use to get the flag definitions and parameter structs, etc...? You can sneer and downvote me all you want, but it seems that the clear truth is that Rust has not been used as a "systems programming language" in the only regime where IMHO that term has any meaning.
Basically: drilling a hole through the compiler to the assembler (which I'll state again is done in Rust as yet more complicated syntax users need to learn and not as, say, a native library interface) is a valuable tool, but it does not a syscall interface make.
(I didn't downvote you.)
Rust has been used to make a toy kernel (as in press key, get message printed and/or change the color of the message). And this was several months ago. I'm pretty sure that makes it a decent systems programming language.
As others have pointed out people have written linux kernel modules, kernels and all sorts of other low-level things in Rust.
(Rust's inline assembly interface is essentially "write in ASM"... If you're truly needing to go that far down yourself, then knowing assembly seems very reasonable. However, this shouldn't be needed much, just use libraries that do it internally.)
So C and C++ are laughable for systems programming, because the language standard doesn't cover system calls.
The APIs for systems calls and support for inline assembly are language extensions.
Your argument applies equally well to sandboxing, for example--why go to all the trouble to lock down a complicated Windows sandbox if installing malicious software is only one kind of security vulnerability? The amount of subtlety in a Windows sandbox is incredible (all it takes is one leaked privileged file handle), and there are plenty of ways that security vulnerabilities can happen that aren't thwarted by a sandbox. But nobody is arguing that sandboxing is pointless, because it massively improves security.
This is the first rust code I've looked at, and I grok this. As someone currently doing C dev in my day job, and who's done a lot of C++ historically, ability to enforce lifetimes sounds outstanding - particularly as it probably combines marvelously with RAII.
This would wonderfully, and it means it's impossible to bypass the lock. Which is something I really wish I had in other languages; just a few days ago I finally fixed a subtle intermittent crash in a C++ program that was caused by taking the wrong lock when touching shared state.
Yes. Lifetimes are hard. Yes, it requires lots of compiler wrestling. However Rust compiler gives pretty good hints what lifetimes should be added and deduces lifetimes in few simple cases. Additionally you can opt out if if you are willing to allocate more memory.
template<typename T>
std::pair<T, std::vector<std::string_view>>
tokenize_string(T &&str);
Yes, returning indexes is also an excellent solution in this particular case, because it forces the programmer to keep the underlying string around. And you probably can save memory by using a pair<uint32,uint32>.
Still, as several other people have pointed out, the truly idiomatic C++ solutions here are things like sub-ranges and string views. And these all use pointers internally. I deliberately chose to use an pair of pointers in place of these idiomatic types because it was easier to explain what was happening, not because I think C++ programmers should mess around with bare pointers.
What I like about the Rust version is that I can work directly with low-level types in high-performance code, and the compiler still has my back.
(Furthermore, there is nothing "C" or wrong with regular pointers. Their only "flaw" is that they can't manage an object, but they're still the semantical way to refer to one.)
I never implied that pointers are bad, just that this particular usage of pointers is bad, and as such should not be used as an example of why we would want something else over C++. If I ever saw a function with a signature like the one proposed I'd be extremely suspicious of what was going on.
Note that this would happen in the same way if the function returned pairs of iterators. It is a choice that depends on the documented and intended usage of a function. If you feel your tokenizer will be used on temporaries (I fail to see how that would ever be useful though) you could overload the function to also take universal references or change the return parameters so that nobody can get hurt. Otherwise you return whatever delimiters you are comfortable with, with the assumption that they will not be used in some way. All code can be broken if you actively try to, so it makes no sense to protect against everything.
Maybe not "regular pointers", but returning iterators is very common (all throughout the STL, for one thing.) Those iterators are part of the public interface, and they provide no real guarantees over bare pointers.
I don't have a horse in this race, though. I think the "nicest" solution in abstract is for `tokenize` to be a token-generator* that takes a character generator argument. If you want to store the tokens yourself you can copy them, if you're happy to stream-process them then you can do that, and if you want a reference into the original string storage you can go fly a kite.
*: Where a "token" is probably another character generator.
That's not Rust's philosophy. Rust is memory-safe, period.
But yes, Rust is way, way, way better in this regard.
You would. I wouldn't because I don't know any better. How many others are there out there that don't know any better? If you code by yourself, fine, but if you work with a team you are gonna need a higher level of training, so you could teach people like me to catch the problem. Are you always going to be disciplined enough to do code reviews that are in depth enough to capture fishiness in the type signature?
With Rust, if it compiles and passes unit tests, the code review is already 90% done.
Yes, that's their only flaw. A flaw which has caused countless bugs and will continue to do so in the future. This is why C++ provides so many utilities designed to keep you from using them in 99% of cases. But yeah, that's their only flaw.
In C++98 maybe ...
http://en.cppreference.com/w/cpp/header/experimental/string_...
If you are concerned about memory safety, the only appropriate choices are to copy the string and have the string_view objects share ownership, or to use the primitive index range idea and punt the problem back to the programmer.
1/ could you build the same unsafe behiavor in Rust if you wanted to by not specifying lifetime constraints ?
2/ If yes, shouldn't lifetime constraints be mandatory ?
http://play.rust-lang.org/?run=1&code=enum%20Token%20{%0A%20...
The tokenization function itself doesn't have to explicitly specify lifetimes like it does in the blog post. It would also work without them and Rust would still ensure that you don't end up with a use-after-free.
TBH I don't find this to be that useful -- it covers up potential mistakes, and of course means that code breaks when compiled with the older version of OCaml which didn't do this.
Edit: I should note that this doesn't break type safety.
It has some unexpected properties: it can sometimes produce dangling pointers because it knows the target data will never be used, even though a GC would treat it as live. And on the other hand it will often promote data to an excessively long-lived region (because region lifetimes have to be nested). So current versions of the MLKit use GC as well as region inference.
But at the end of the day, the Rust compiler will never allow a reference to outlive the original object.
fn test_parse_unsafe() {
let v = {
let text = "The cat".to_string();
tokenize_string3(text.as_slice())
}; // `text` destroyed here
assert_eq!(vec![Word("The"), Other(" "), Word("cat")], v);
}
fn test_parse_unsafe() {
let v = {
let text = "The cat".to_string();
tokenize_string3(text.as_slice())
};
assert_eq!(vec![Word("The"), Other(" "), Word("cat")], v);
} // `text` destroyed here
This could lead to "memory leaks", where a destructor is implicitly postponed to a higher scope, but I don't think it would be much of a problem in practice (the promotion would only be through simple scopes, not through loops, and maybe not through `if`s). In fact, there's an yet-to-be-implemented accepted RFC covering this[1] (there's no guarantee that it will be implemented though, just that the idea is mostly sound).
[1]: https://github.com/rust-lang/rfcs/blob/master/active/0031-be...
I believe that it's safe to promote through an if, although obviously not through a general loop.
On a two-way if statement, then a given storage location is either set on zero, one or both branches. If it is set on neither branch then the if statement is irrelevant and can be ignored. If it is set on one branch, then either it had an original value and hence can be treated as being set on both branches, or it must be destroyed within the branch of the if (no null pointers - think about it until it is clear that the type system guarantees this). Hence we are only interested in cases which are isomorphic with the location being set on both branches.
We can treat this as a phi node following the if: there is one output value, which has been created in one of two different ways. In this case we don't know statically which value has been constructed, but we do know statically how and when to destroy it regardless of which one we get, because both branches have the same type and storage location. We don't actually need to know where it came from.
Any obvious problems? I think it works...
I.e. one branch could be created by .as_slice() on a String, the other could be created by referencing a global, e.g.
let s = if cond {
let some_string = create_it();
some_string.as_slice()
} else {
"literals are always-valid &str's"
};
That seems like a reasonable case to raise a type error. That defines the cases quite neatly: if it's temporary on both branches then it can work, and if it has different lifetimes then the values can't be merged and should be rejected. If the programmer really meant for this to work then they need to copy the global, and copies should be written explicitly.
However, it's easily possible to have the &str come from temporaries of different types in the two branches. This would restrict the static destruction case to only working through an `if` when the "parent" values have exactly the same types; which doesnt seem nearly as valuable and possibly not worth the effort.
Although this starts getting really messy.
On a related note, the approach I mentioned is actually mentioned in that RFC PR:
> Store drop-flags for fragments of state on stack out-of-band
It would mean that you would have to wrap the incoming string in a class, and probably add the tokenize_string method to that class. Then you would also have to wrap the results vector in a class that then addrefs the original string wrapper class.
But after that, handing out pointers to the contents of the string would be no problem as the results class would addref the string class and then release it when done ensuring that the string wrapper class remains alive as long as the results object has not gone out of scope.
Of course Rust's approach of alerting you when your code path causes dangling pointers is also interesting, but I wonder how that would work if you were to link against a static library that handed out references to internal objects like that - could the compiler see the scoping problem?
So what stops you from returning a shared pointer in case of get_input_string? Then take over that ownership and use it. It's still a potential problem that v is logically disconnected from lifetime of that pointer, but at least you could avoid the problem you described.
(1) It has at least three HTTP packages that I'm aware of (not finished ones, admittedly).
(2) While perhaps for you lacking an HTTP package is a dealbreaker, many of the areas Rust is targetting (such as embedded) don't require it at all.
(3) Why should people stop talking about a language because it hasn't been released yet, anyway? I'm not sure I follow the logic here.
"Usable" is an ironic way to spell "hackable".
You mentioned the words "news" and "use". Does the H in HN still stand for something? Maybe HN stands for "HTTP News"?
Managed languages solve a large set of problems, but introduce two big problems as I see it:
- Cause nondeterministic overhead which is particularly problematic for kernels, games, and software with heaps over, say, 10 gigabytes.
- Limit ability to safely pass objects outside their ecosystem. How many libraries written in managed code are worth using outside of the language they are written in?
Strong typed languages with automatic memory management, having native compilers and used for writing OS, exist since Mesa/Cedar (70's at Xerox PARC)
Back when C was UNIX only, no one talked about C ABI. The only ABI that mattered was the OS ABI.
Amoeba, OS/400, Lisp Machines, Oberon are a few examples of having multiple languages interoperate via the OS ABI..
Regarding the interoperability, it really depends at what level OS, GC or RC services are available.
If just on the language runtime, yes it complicates the distribution of libraries. As it raises the issues how many copies of the runtime one gets and version compatibility issues.
If the OS offers the services, then any language on that specific OS, can enjoy exposing libraries that interoperate with GC or RC.
COM (now WinRT) is a possible example of such OS services.
With the overhead of GC...
Now to get back what was lost in terms of safety, new languages are required, as the old ones are considered legacy by the average Joe/Jane coder.