This is actually in Go, but the issue is the same.
Suppose you have a struct like this:
struct foo {
struct bar elem
} s;
If you know the address of `s`, you just calculate the address of 'elem' from it and read the contents; a single memory read, all the data together cache-wise. Suppose on the other hand you have a struct like this:
struct foo {
struct bar *elemptr;
} s;
If you know the address of `s`, you have to first read `elemptr`, and only then read the value of `elem`. That's an extra memory fetch, and probably from a different part of the memory than `elem` is from. Copying on modern processors is very fast, and the resulting copy will be "hot" in your cache. So conventional wisdom I've heard is that unless `struct bar` is quite large (I've heard people say hundreds of bytes), it's probably faster to just copy the whole structure around than to copy the pointer to it around and dereference it.
Caveat: I haven't run the numbers myself, but I've heard it from several independent sources; including, for instance, Apple's book on Swift.