How about we have satellites orbiting asteroids for a piggy back ride and then getting off the orbit when they are near a point of interest? My guess is the gravity would be too feeble.
You would need to first match the asteroid's velocity, and by then you do not really need the asteroid. But something very similar to your idea is used constantly for interplanetary travel - gravity assist which uses other bodies' gravity to accelerate and change direction.