Hacker News Reader

Comment by nicklecompte | Hacker News Reader

In particular: strings aren’t actually simple arrays of characters in Rust like they are in C, but there is an underlying array on the heap, and the notion of “slicing” that array of characters still makes sense semantically.

View on HN

Slicing strings is a difficult concept when dealing with Unicode because slice indices and lengths can come in multiple flavors, but the only flavor that makes sense (grapheme or grapheme cluster) is difficult to count.

>strings aren’t actually simple arrays of characters in Rust like they are in C, but there is an underlying array on the heap

Not a Rust expert, but to my understanding, 'string' is an array of characters, not necessarily living in the heap.

The object 'String' (with capital S) might be, but with &str I can have a constant array of characters that is not in the heap and not even in the stack: it's in the code (.text code segment[1]?, or EEPROM or flash, for embedded folks).

If I understood it correctly, &str is a slice with a pointer to that piece of code. Of course being const, it cannot be changed. I can copy it into a String and manipulate it (append, insert, etc.), and that's where the heap is used.

&str, as an hypothetical struct, it may be in the stack, initialized in-scope with (probably) a pointer to somewhere in .text plus some bytes for length or other information.

[1] https://en.wikipedia.org/wiki/Code_segment

&str is not necessarily a static string literal; it's a slice of any string, living anywhere (including on the heap inside a String). Technically speaking, it can't be mutated because it's a non-mutable reference (like any other non-mutable reference). &mut str is a thing that can exist, and is mutable, it's just that you can't get one of those for a static (literal) string that lives in the code.

Thanks. Yes, of course it can point to something in the heap. My comment was related to GP's "but there is an underlying array on the heap" and I wanted to say that &str is not always necessarily in the heap. I missed that part.

> strings aren’t actually simple arrays of characters in Rust

Are you talking about them being a container with a pointer to the actual array, and also a size and etc?

In C, strings can hold invalid unicode. However, in Rust a str is guaranteed to be valid utf-8.

For added confusion, Rust has a `char` type which is actually 32-bits. You can create arrays of them, but the resulting string would be in utf-32 and thus incompatible with the normal `str` type.

    let s = "hello world".to_string();
    for ch in s.chars() {
        print!({},ch);
    }

will iterate through a string character by character. That's the most common use of the "char" type - one at a a time, not arrays of them.

Although the proper grapheme form is:

    use unicode_segmentation::UnicodeSegmentation; // 1.7.1
    let s = "hello world".to_string();
    for gr in UnicodeSegmentation::graphemes(s.as_str(), true) {
        print!("{}", gr)
    }

This will handle accented characters and emoji modifiers. A line break in the middle of a grapheme will mess up output.

By the way, open season for proposing new emoji starts tomorrow.[1]

[1] http://blog.unicode.org/2020/09/emoji-150-submissions-re-ope...

Thanks for the code snippet, however (kind of a Rust newbie here)

  $ cargo install unicode_segmentation (chokes)

  $ cargo install unicode-segmentation (seems to work)

  and in Cargo.toml
  [dependencies]
  unicode-segmentation = "1.7.1" (seems to work)

  yet in the code, its:
  use unicode_segmentation::UnicodeSegmentation;

  Why couldn't they be consistent using a dash vs. an underscore ?

Identifiers in Rust can't have dashes (rustc will never rely on whitespace for proper tokenization). It is arguable that because of that crates should not have dashes in their name. cargo was changed to automatically convert dashes to underscores for convenience because some dislike underscores in names. I personally think this is an unnecessary source of confusion and now I would make all tools (mainly cargo) treat them interchangeably.

> rustc will never rely on whitespace for proper tokenization

Huh? That seems clearly untrue, eg `fnfoo` vs `fn foo` or `x && y` vs `x & &y` or `x<<shift_or_type > ::foo` vs `x < <shift_or_type>::foo`? Presumably for some or all of those, one version ends up being a error (eg bitwise and with a pointer from `x & &y` probably doesn't work), but that's not at the level of tokenization.

Given the context, I think they simply mean that a-b is interpreted the same as a - b.

It can't be interpreted as a single identifier.

You don’t cargo install libraries; the cargo.toml line is the right thing.

You can’t use -s in Rust identifiers, so they need to be normalized to _ to be referred to in code.

this is way too common in Python world too - packages (crates) use dashes (they look nicer?), but the packaged library is named with underscores, because `-` is math operation and can't be in a name

char is usually some single byte of data. Characters can be multiple bytes. Slicing a string on character boundaries is more coarse grained than slicing on byte boundaries.

char in Rust is 32bits, so it has a 1:1 mapping to Unicode glyphs. You might also want to care about grapheme clusters, but those are not part of the stdlib.

Unicode codepoints; glyphs may take multiple codepoints.

You are, of course, correct.

Right - the point is that &str isn't syntactic sugar / alias / etc for &[u8] and it would be confusing to have a notation that suggested otherwise.

Reply on news.ycombinator.com