In the general case, you can make a hologram that looks like anything from any angle (like play a movie as you walk past) - given a particular viewing distance, because at a different viewing distances you will see mixtures of different "frames" due to parallax.
I can't offhand reason out what "viewing distance distortion" would do in the specific case of making a 2d image "follow" you (might even cancel out!), but in any case it's not a big deal if the viewing distance is much larger than the hologram size.
Interestingly, there's a traditional optical solution to this problem as well: beer pump logos, which are often displayed behind a very convex lens which partially corrects for parallax distortion.
(Edit: you will not, however, be able to correct for rotation. The hologram can't possibly "know" what orientation your head is in, only where it is.)