That might be the case... I mounted a heavy U-lock on the front forks of a mountain bike, made steering very heavy. Felt pretty similar to a heavy wheel to me.

Of course this is all subjective (my observation, and the parent's downhilling observation).

Definitely easier to steer lighter wheels regardless of speed.