In my intro-to-java-class, indirection was one of the first things we learnt when we learnt about objects: variables hold a reference to an object, not the object itself. I don't understand why pointers are somehow a great insight that is exclusive to lower-level programmers.
And if a java person ended up trying to make a method that swaps two primitive values, they'd have to learn about the distinction between passing a primitive value into a method as opposed to passing a reference to an object.