True. But gravity is a conservative field (again, neglecting radiation), meaning that the energy that goes into raising something against gravity, you can get back by lowering it to the original position again.
> And gravity is not a force. Gravity is the curvature of space-time.
If we're not in "overly pedantic argumentation" mode, then I'd say, go climb some stairs. Feel that? That's a force. But if we are in "overly pedantic argumentation" mode, then yes, gravity is the curvature of space-time in General Relativity. That may not be the final word, though. What is gravity in Loop Quantum Gravity?
When you'd say "go climb some stairs" what you feel are the stairs. And you should also feel like a bit of an idiot, in my humble opinion.
Take two objects and place them apart to float stationary( the line they make should be orthogonal to the center of the field ). Even though they started completely stationary, they will slowly start to move towards each other, as an unknown force is acting upon them. In reality they are falling towards a common center.
I guess this doesn't work on dimensionless points, so the your comment is correct as far as mathematics are concerned.
Forces can indeed be modeled as a curvature in space-time. However, there is a bit more to GR than just that, which is why it took Einstein years to go from SR to GR.