> The former is with respect to the inertial frame of the faller(s), the latter seemingly with respect to the inertial frame of some shared gravitational field.
Fair, and I'm not sure I follow that, either. I'm not sure how an entire field can really have a reference frame in the first place.
> though at the moment I only get it as a "negative" argument: we are being accelerated upwards/outwards with respect to the field's reference frame, because otherwise we would be moving along our natural inertial path.
That pretty succinctly captures my understanding.
> It is still a bit puzzling that if the ground disappears we appear to accelerate, whereas my current reading of what you wrote seems to imply that we would immediately return to our "inertial path".
Yes, that's the sticky bit! I think "appear" is load-bearing here, because what are you measuring relative to? The surface of the Earth is not an inertial reference frame (according to general relativity, it's accelerating!).
It might be easier to think about if we use centrifugal force -- the fictitious force you feel when being swung in a circle from some central point. From an inertial reference frame, you have some velocity in the tangential direction, and a rope (say) is pulling on you in the radial direction. This force accumulates with the existing velocity over time to produce a circular path.
From your perspective, however, you feel a force pushing you outwards, which is resisted by the rope you're attached to. You would feel that, if the rope was cut, you'd be "pulled" out into space (radially away). But this is because you're not aware of -- you don't sense -- the tangential velocity you possess, because it's canceled by the choice of a frame that's moving with you.
The change to an inertial frame introduces an extra velocity term, and this in turn allows us to replace the unattributable outward pull of the centrifugal force with a force merely attributed to the tension of the rope you're attached to. (In fact, this might be a good intuition for what an "inertial frame" even is: it's a frame in which all forces are equal and opposite to other forces. Gravity and the centrifugal force are unbalanced in this sense.)
The gravitational field (curvature of spacetime) plays the same role that the hidden velocity does. In an Earth surface frame, the feeling of being pulled "down" is attributed vaguely to the fact that there's a bunch of mass "down there". In an inertial frame, the feeling of being pulled "down" is attributed to the ground pushing "up" at you.
In both scenarios, we can find a frame involving an extra term that lets us redescribe the phenomenon we observe in terms of forces attributable to physical entities.
The "Fictitious forces" Wikipedia page has some pretty good schematic animations for the centrifugal force, FWIW.
[0] https://en.wikipedia.org/wiki/Fictitious_force