Actual 3D-sight might include us seeing the inside of objects, what's behind them, etc.
We're aided in converting our 2x 2D sight into 3D mental models through the use of color and parallax. In projections on images, it's easy to just use color and get rid of parallax, and still get sort of close.
Suppose we tried to make a 3D projection of a 4D world, we don't actually get any significant new tools. We're still going to be limited by having to use parallax and color. This doesn't allow us to add enough information to add that 4D projection into 3D as neatly as the 3D into 2D works. At best you could try to distribute the information somehow. For example by giving information about the third dimension through brightness and the fourth dimension through hue. It certainly wouldn't work nearly as well though.