The closest observation point is three miles from the launch pads, so yes, the VAB (Vehicle Assembly Building; it's insanely large) is probably around three miles as well.

Why the three mile distance? Back during the Apollo program, engineers determined that if the Saturn V were to explode on the launch pad, the resulting fireball would be around four or file miles in diameter. I think it's a similar radius for the Shuttle as well.