The heart of what bothers me is you have to deference addresses...

I don't understand why you keep saying that. They're not addresses.

From the language side of things, they're first class scalar entities just like strings and numbers. From Perl 5 they have nothing to do with memory addresses. (You might as well suggest that a nested data structure in any language without pointers is "dereferencing addresses", or that accessing object attributes is "dereferencing addresses".)

You can't write in Perl something like:

    my $reference = 'ARRAY(0x1234abcd)';

... and expect to access the array at that point in memory even if you stringified an array reference and saw that its stringification included that hex address. References are not pointers. They don't dereference addresses.

From the internals side, they're SVs, just like all other scalars in Perl 5. An SV is a C structure. Yes, the internals use pointers, but so does any other virtual machine implementation.

(Okay, you can write code like I said above, but to make that work correctly, you have to write C to do it, because C allows direct access to memory by address.)