Most likely it didn’t hurt the neck directly but rather did not provide the normal tension reducing benefit that hot water does. On top of that your muscles do spasm a bit when you are cold which could possibly build more pressure which over a period of days/weeks would result in pretty bad stiffness.

I think I had a slightly stiff neck after 9 hours of sleeping, then jumping into the cold water made me jerk unnaturally, giving me neck spasms.