Well, Zig's native string type support, or lack thereof, is also a questionable choice. It's not like Zig did all the right choices and only Go and Rust made questionable ones.
Rust: `&str`, 'some bytes, and a length', 'Constructing a non-UTF-8 string slice is not immediate undefined behavior, but any function called on a string slice may assume that it is valid UTF-8, which means that a non-UTF-8 string slice can lead to undefined behavior down the road.' - also `String` and `OsString`.
edit: apparently Rust enforces that &str cannot be created with invalid UTF8, at the additional performance cost of runtime checks, so it has stronger guarantees than other languages like Go/Java/Zig/etc (as usual)
Go: `string`, 'in effect a read-only slice of bytes', 'a string holds arbitrary bytes. It is not required to hold Unicode text, UTF-8 text, or any other predefined format. As far as the content of a string is concerned, it is exactly equivalent to a slice of bytes.'
Zig: `[]const u8`, 'some bytes, and a length', 'Zig has no concept of strings', 'by convention parameters that are "strings" are expected to be UTF-8 encoded slices of u8.', 'Generally, you can use UTF-8 and not worry about whether something is a string.' - and the stdlib has functions for working with unicode and strings.
If you're constructing invalid `str` you already have severely fucked up. And dog is eating your HDD.
Presumably it'd be better to know this immediately on initialization of the variable rather than at some point in the program that actually expects valid utf8—god forbid you transmit or persist the data before this happens.
Rust’s string constructors (for both str and String) check that the data contains strictly valid UTF8.
You can opt out of this check in performance sensitive code - but doing so is considered unsafe. In safe rust it’s impossible to construct a str / String which contains invalid utf8.
[0] https://doc.rust-lang.org/std/primitive.str.html#invariant
[0]: https://crates.io/crates/bstr
The idea of using unsafety here is to have a way to tell the Rust compiler: "Trust me, I promise that I only fill the strings with valid UTF-8 data. You don't need to insert code to check UTF-8 validity." And the Rust compiler answers: "Okay. I accept that but if something goes wrong I can't catch that."
This allows avoiding repeated validations of a byte slice for UTF-8 validity, for example.
I agree that bstr is what you want unless you specifically understand this detail and know why it is the way it is.
I admit that a library invariant is allowed to be violated temporarily under some strict preconditions. One example are bank account money transfers, where the credit and debit don't happen exactly simultaneously. Something similar could happen in a library.
But for language UB this doesn't work. Why? Is it because of compile-time optimizations possibly deleting code restoring the invariant?
Sorry - I think my comment wasn't clear. The point of from_utf8_unchecked isn't to create strings with invalid UTF8. The point is that if you know that your string has already been validated, you can skip re-validating it.
Because these are different types with different semantics.
Why should there be one string type? Languages have to interface with the real world, where string data could be anything, but programs still need to be written for the common cases... hence, &str and String.
Last time I looked I don't think it did. It looks like there is a unicode namespace now, but it is very primative. It doesn't have a simple way to iterate over codepoints, much less graphemes. Nor does it have anything to inspect the category a codepoint belongs to.
It doesn't have graphemes (neither does Go or Rust), or other advanced unicode support. Instead that is provided by ziglyph[0]
A lot of the Rust string API lives in str, not String.
If you're working on bytes string in Zig the status of them is unknown, so you have to always validate even if it's just Utf8. In Rust you can omit Utf8 validation for bytes from a String.
While Rust doesn't goes as far as I would have want it's miles better than what Zig and C are doing - the programming equivalent of "Trust me bro!".
And then everybody proceeds to write their own String library anyway.
"The only winning move is not to play."
This allows Rust to have all kinds of string flavors (fixed-len or NUL-terminated or growable, on stack or heap or in ROM, with SSO, with CoW, interned or refcounted, atomically or not, and so on), but they all coerce to the basic &str, so they have the same basic methods, and are compatible with most functions that just want a string without caring how it's allocated.
You can use your own weird string type if you want, but usually you're not forced to convert or copy it to use it with standard library functions or 3rd party dependencies.
In rust Strings are usually passed in to functions as a &str, and returned from functions as String. It makes sense once you’re used to it, but I think philosophically Rust is much closer to C++ than C. Rust is missing the elegant minimalism of C or Zig.
This has nothing to do with strings though - how is &Vec<T> different from &[T] or &Box<[T]>? You need to understand ownership, references, and unsized types.
Sort of. It certainly affects strings, and you can't really use strings without understanding the difference.
My point is that its yet another thing in the big bucket of junk you need to learn in order to use rust effectively. Rust is the only language I know of that has separate types for owned strings and a borrowed strings. Its a powerful concept - and its fast and efficient for the computer. But its not "free". The cost is that it makes it harder to learn rust.
While we're on the topic - it also really doesn't help that the names of these types in the standard library are super idiosyncratic. String is to Vec<u8> as &str is to &[u8]. You just have to memorise that. How about Path and PathBuf? One of them is like String, and one is like &str. I swear I have to look it up every single time.
I think the investment is worthwhile - I adore rust. But as I said, rust feels more like a better C++ than a better C. All the pain is front loaded, with the assumption that you'll eventually write enough software in rust to make the investment worthwhile.
Simpler languages like Python or Go get out of your way immediately and let you be productive. I prefer rust, but there's definitely something to be said for that attitude.
Since Rust prefers to have verbosity and correctness over ease, it makes sense to have multiple types to reflect the different requirements of the underlying data.
I sympathize with some of the std library verbosity when it comes to strings (and paths) but I don't think it's really a big criticism of why Rust's learning curve is steep. You have the same issues in C++, and doubly so in C because it doesn't help you at all and when you screw up the program crashes. When you use strings in an unmanaged language and need to manipulate them, you do need to understand the underlying consequences of how those strings are defined and stored. Comparing to managed languages like Python and Go is a bit unfair, because they're working in a different domain.
Python is a great example of what happens when you try and hide complexity too far from the programmer - the python3 ecosystem fracture took millions of dollars and over a decade to settle - because of the inherent complexity in string representations.
> Rust is the only language I know of that has separate types for owned strings and a borrowed strings.
C++ and Objective-C (kinda) have different types for owned and referenced strings. C++ in particular has the same requirements as Rust, where referenced strings need to be able to refer to literals and it's not acceptable to make the default allocate.
Ownership is hard to learn, and it is a barrier to learning Rust. However, don't confuse it with C++. Rust has two+ string types on purpose, not because of carrying 1970's legacy.
Is this true? It was (is!) certainly true for C, but C has an especially emaciated expectation for string processing primitives. Any runtime developed after like 1995 that I can think of has fixed this by providing a sane string implementation people generally agree upon.
Rust and Go both don't have a builtin package for grapheme iteration - and many naively (and incorrectly) think that a 'Go unicode rune == "character"' in Go. I assume the same happens with Rust `char` type.
If you care about unicode-aware string sorting (you should), rather than the naive string sorting the Go and Rust standard libraries provide out of the box... then you probably want a proper Unicode library.
I think the only language that gets Unicode 'right' out of the box is Swift, as it actually provides grapheme iterators, Locale awareness, etc. - but it comes at the cost of the language being tied to the (ever-moving) Unicode standard.
I mean unicode is incredibly complex, but it doesn't even seem like there's a consensus outside of swift's string implementation of what a grapheme even is.
(Granted, this might support the above concept that people can't even agree on what a string is, but unicode code points seems like a reasonable baseline to expect from a modern language. That said, rust doesn't even include unicode normalization in the standard library, although the common crate for it seems like a reasonable solution.)
Linguistically it's easy, graphemes are the squiggles people actually draw, as distinct from how a machine encodes them. Of course since people aren't a single individual with just one consistent opinion that does mean there's room for nuance - maybe some people think this is two separate squiggles.
In recent years, application support for this has greatly improved.
I think this might (at least partially) be why Rust's stdlib doesn't have this. If it did, then support for it would be tied to Rust's release schedule and which version of Rust you're using. Granted it is every six weeks, and it's usually trivial to update, but that's still a connection that could be an issue.
By having this be in a separate library it means that it can update as and when it needs to, and it's not inherently connected to a specific release of Rust.
Let's just assume a String is UTF-8 to make things "simple".
Is a String mutable or not? Should mutable and immutable Strings have the same underlying structure? If mutable, is a String extensible or not? Can a String be sliced into another String? Can those slices be shared? Should you walk across codepoints or characters (which could be multiple codepoints due to combining)? If you want to insert a codepoint in the middle of a String, what are the performance guarantees?
I can go on and on ...
"String" really has to be a library as there are simply far too many permutations once you step away from "Shove ASCII to tty".
That said, I personally would balk at willing adopting any runtime that didn't enable iterating over a sequence of unicode code points, whether they be stored as utf8 or some 16-bit form, from a string of bytes in 2024 unless I were guaranteed to avoid having to deal with text processing of free-form human input.
https://github.com/U8String/U8String
With that said, implementing it is not easy, especially in terms of ensuring it is as fast as built-in string or faster (because the project goal is to be faster than built-in string and Go implementation, and match or outperform Rust's String where applicable).
I can't imagine something like that being possible in e.g. Java or most other high-level languages - they simply do not expose appropriate low-level primitives when you need a "proper" string type.
It’s supposed to be “I have NOT seen”.