As an aside, C's conflation of arrays and pointers was done because they're very closely related, and you can get very similar results to a C pointer-to-pointer by wrapping an object in a one-element array. In C, ^x is equivalent to x[0], and Java doesn't have ^x, so you can just use x[0].
More generally, though, a C programmer is virtually never thinking of 'the address 54fc892d': unless they're doing really low-level stuff, they won't think of it like that any more than a Java programmer, even using pointers all day, just like the Java programmer is; that's what I don't get about your point. The C programmer will think of ^x (or alternatively x[0]) as a handle on the string "foo", the same way a good Java programmer would. Some programmers (Java or otherwise) might confuse a handle on "foo" with "foo" itself, but they'll be bitten by that when they try to write a swap function or mutate a string literal or whatever.
Java's reference types are restricted pointers that you can only dereference using . or array indexing, you can't use array indexing on any arbitrary reference (it has to be of the right type, but C pointers were already strongly typed), you can only do arithmetic on them via indexing, and you're not allowed to refer to the non-pointer type. None of those differences make them not pointers, to me.