No boreholes are perfectly straight and vertical. In most cases, you're twisting the drillstring at the top, so it naturally "wants" to corkscrew. Furthermore, anisotropy in the rock causes things to deflect (deviate in drilling terminology). Usually you don't know exactly where the well went until you run a directional survey afterwards (basically, you lower a gyroscope down the hole).

At the scale of a well, the steel drillstring is more like cooked spaghetti. You don't "push" down, you use the weight of fluid (drilling mud) for pressure. The drillstring is only meant to transmit torque. It's easy for it to bend over long distances. (You can even turn things entirely around and use fluid pressure to turn the bit, in which case the drillstring is basically a steel hose that's stored coiled up.)