Are you familiar with the distinction between "pass by reference" and "pass by value"? If you call everything which behaves in the former way a pointer, I think you confuse an implementation strategy with a language construct and you lose an important feature of C pointers.
What you see in Perl (and Ruby) is a unique identifier that happens to be a memory address. It could be anything else, but the memory address is an identifier that's essentially free to generate. That's it. It's otherwise irrelevant. You have to write code in a language which has pointers to do anything with that information, and even then you have to cast it to a real pointer to do so.
We could discuss Perl 5's dereferencing syntax (it's ugly, no argument there) but that's a syntax issue and not an implementation concern.