If you just take a magic balloon up to a certain height, once you're up there and let go fo the balloon - you just start falling back down.
You need to have a lot of sideways velocity in order to actually get into orbit, to be revolving around whatever body you're trying to get off of.
(It didn't used to make sense to me, either.)
But no, it's not that you would be pulled into the sun. You would simply be in a solar orbit just like the earth, unless you continued accelerating and reached solar escape velocity (~600km/s). You can think of gravity like successively larger "wells" which you must escape from if you want to go anywhere. You can't just point a spaceship at a planet and go there.
Disclaimer: You can, you just need so much power it’s like flying with a helicopter from your bedroom to the kitchen, absolutely wasteful.
http://parkersolarprobe.jhuapl.edu/The-Mission/index.php 'Journey to the Sun.'
That's for escaping from the surface of the Sun. At Earth's distance from the Sun, the velocity needed to escape from the Sun's gravity is only 42 km/s.
What the GP said is that if when you reach Earth's escape velocity, there is still a long way to go until you have Sun's escape velocity (so you are on a solar orbit), and after you reach that, there is a huge distance until Milk-Way's escape velocity, and an even huger way until you escape the Local Group.
That's not what he said. Escape velocity is higher than orbital velocity (as ythn pointed out upthread). And escape velocity from Earth is still slower than Earth's orbital velocity around the Sun, which in turn is lower than escape velocity from the Sun's gravity at Earth's distance from the Sun.
We do this all the time with probes to other parts of the solar system. Manned missions (Apollo) used a small number of earth orbits before doing the burn to get to the moon primarily to ensure that systems checked out before making the bigger step.
Unless you hit the ground, or a mountain, or a plane, or the moon, etc.
Launching a rocket to orbit is like starting from the bottom of the well, and spinning around the slope really fast until the nickel is returned to your hand.
The gist is that gravity pulls you. To go to space you have to escape gravity. Even if you go thousands of miles above earth, gravity will still pull you. To counteract gravity, you need to go really fast sideways.
When you spin a ball attached to a string, the faster you spin, the more the ball pulls the string away from your hand. If you spin it really really fast, the string breaks eventually and the ball flies away.
The string is gravity.
Except you can't, so speed is very definitely not irrelevant.
To give an example, if you provided propulsion to move an object at 1 mile per hour, you could turn off all propulsion and the object would have escaped Earth's gravity once it was 4 * 10^12 km away. This is, of course, an extraordinary waste and probably practically impossible.
I provided a very specific computational counter-example to your statement. You’ll need to start there if you wish to compose a serious reply.
I’ll restate the counter-example for your convenience. One can provide just enough continuous thrust to move an object at a fixed unchanging speed of 1mph for a long time (about 283 million years), at which point it will have reached a distance where the escape velocity is only 1mph. Once it has surpassed that distance, one can disable the thrust and the object will have completely escaped Earth’s gravity without ever having moved at more than 1mph. It will then asymptotically slow down to 0 mph over an infinite amount of time.
There is ample introductory material on Newtonian physics that you could learn from, only a google search away. It’s disrespectful of my time to assume I must be on the hook to handhold you through it.
Imagine that I have a spacecraft with imaginable engine which is able to indefinitely provide thrust enough for this spacecraft to move straight up from the planet’s surface at the constant speed of 1 mph.
Will this spacecraft leave the planet in indefinite amount of time or not? If not - what is going to stop it?
https://www.quora.com/Escape-velocity-is-supposed-to-be-24-0...
Multiple answers explain that there isn't a fixed minimum change in velocity required for an object to escape Earth's gravity.
https://www.quora.com/Does-rocket-always-need-the-escape-spe...
The first answer here explains the same thing, using basically the same example (1 m/s instead of 1 mph).
https://space.stackexchange.com/questions/4688/couldnt-i-esc...
The answers here overwhelmingly reiterate and support the same example in this thread.
For instance, if you had a rocket with a fixed delta-V and it used all of its energy almost instantaneously, say it escapes from Earth. But if you lower its thrust so that it is not accelerating upward (lower than the force of gravity), it could use the same delta-V without leaving the ground.
The key insight revealed in the term 'escape velocity' is that it is independent of the mass of the body. It is also independent of how quickly it acquires that velocity.
http://www.physlink.com/Education/AskExperts/ae158.cfm
Escape velocity is the inverse of terminal velocity of an object falling from infinite height (discounting air resistance).