No, I mean which way is the photon traveling - left or right? (Relative to a line pointing "down".)The photon is traveling radially outward; that means it is traveling in a direction opposite to a line pointing "down". At least, that's the way it's traveling spatially. In spacetime, a photon at the horizon is traveling along a curve of constant radial coordinate r (and constant angular coordinates theta, phi if you include them). A photon inside the horizon is traveling along a curve of decreasing radial coordinate r; inside the horizon even outgoing null curves (the worldlines of outgoing light beams) have decreasing r.
Emit a photon from just inside the event horizon, traveling directly away from "down".
What does the photon do?
According to an observer that is falling inward just inside the event horizon, the photon moves radially outward at the speed of light.
Does it go into orbit?
No. As you say, it has no angular momentum, but that's not the primary reason; the primary reason is that there are no "orbits" inside the horizon. In fact, there are no "orbits" inside a radius of 3/2 the horizon radius; at that radius, a photon can orbit the hole in a circular orbit.
Does it hit the black hole?
Eventually, yes.
(How did it turn around without ever curving?)
It didn't. The spacetime itself is curved inside the horizon to such an extent that even a photon traveling radially outward ends up hitting the singularity.
Once again, your intuitions about how "space" works and how things "travel in space" break down inside the horizon. You are thinking of a point with radius r < 2M, i.e., just inside the horizon, as a "place in space". It isn't. It's more correct to think of it as a "moment of time". The horizon itself is also not a place in space; it's more correct to think of it as an outgoing light beam.
Does it just travel forever thinking it's moving away from the black hole, but not actually going anywhere?
This is one way (but probably not the best way) to visualize what a photon exactly at the horizon does; it is moving radially outward at the speed of light, but because of the curvature of spacetime at the horizon it stays at the horizon.
(i.e. redshift into nothingness, since it's constantly fighting gravity)
"Redshift" is relative; you have to specify what observer is receiving the photon and measuring its frequency.
Tell me what it does from the POV of the photon, not the black hole.
There is no such thing as "the POV of the photon"; photons don't have "rest frames" in the usual sense of that term. I said above what the photon does from the POV of an observer falling into the hole.