Photons radiate in all directions, but doesn't every photon go in a straight line?
However, because EM diffraction is observable at macroscopic scales, the "straight line" thing really is confusing. Growing up, I always wondered how radio waves transmitted through walls and assumed they just went through them when in fact they go around.
I think this explaination is right:
"In electromagnetic wave, it is the electric and magnetic field vector (or amplitude) that is oscillating." (not the position)
from here: http://www.physicsforums.com/archive/index.php/t-186751.html