Let's start with a graviton as a gauge boson, mediating the gravitational interaction similarly to how the photon is a gauge boson mediating the electromagnetic interaction.
First, let's start with "gauge". We can use as an analogy an air pressure gauge, one that measures relative air pressures. Let's take it to a place with a standard pressure (say, in conditions which are effectively STP) and tune our pressure gauge so that it reads "0" at that air pressure. As we wander to and fro reading our calibrated gauge, we'll see pressures that are zero, positive or negative. If we climb a tall hill we'll see a negative reading. If the temperature drops we'll see a positive reading.
If we contrive things so that we can take a reading of a generalized pressure with our pressure gauge everywhere in the universe at all times, we can construct a (classical) gauge field. In deep space, the gauge will read strongly negative. At the bottom of the ocean, or deep in Jupiter's atmosphere, or at the core of the sun it will read strongly positive. In the early universe, it'll be strongly positive; in the far far distant future away from the black hole that will dominate our patch of de Sitter vacuum, it will read strongly negative. We'll get zero values in some places, like near the Earth's surface through a lot of Earth's history, or in the upper reaches of Jupiter's atmosphere through a lot of its history.
Our choice of "0" is not ideal, because "0" is only rarely the value at any point in our gauge field that permeates all of spacetime. Instead we should set "0" as the value in extragalactic space, because then "0"s will dominate the field (indeed it is possible that all readings will then be non-negative). In effect, when we set our "0" at STP we normalized the gauge field; when we decided instead to set our "0" in extragalactic space, we renormalized it. We could obtain an ideal renormalization if we could sample the whole of spacetime and find the lowest reading of our pressure gauge, but we can certainly get rid of practically all negative values by taking far fewer samples in regions where we think the lowest readings might be.
Once we have settled on a decent normalization, we could look at the propagation of nonzeros and study their statistics. If they follow the Bose-Einstein statistics, we'd call them "bosons". If they follow the Fermi-Dirac statistics, we'd call them "fermions". If they follow some other statistics, we'd assign them yet another name. (Our choice of generalized "air pressure" probably follows some odd statistics.)
Perturbative quantum gravity works something like this. [2]
We have a background spacetime with a metric; we have a gauge that measures the deviation from this metric. It'll be "0" at every point where the arrangement of stress-energy exactly matches the metric, and nonzero elsewhere. We are interested in modelling the gravitational interaction as the arrangement of nonzero values in our field. Patterns of nonzeros around gravitationally interacting matter themselves evolve (under a suitable decomposition of spacetime into 3+1 space and time) and interact like (classical) waves (made up of many molecules), and upon some study we can determine that these waves in our gauge field form patterns that strongly suggest they have a rotational symmetry of two, which we expect on theoretical grounds too because the metric is a rank-2 tensor field so particles representing the (change in the) metric field should be spin 2.
Conveniently, in a quantum gauge group theory, a particle with spin 2 is attractive of a particle with the same charge and repulsive of a particle with the opposite charge. (Compare with spin 1, where same-charges repel and opposite-charges attract). [4] So we can identify the nonzero numbers in our metric gauge field with gravitons. This is amenable to study with perturbation theory.
Unfortunately General Relativity is a non-linear theory and in our perturbatively quantized gravity, when you have a lot of high-energy gravitons they spawn more gravitons. We would want to apply Wilson's thinking on renormalization and reset our gauge to "0" in a cluster of these high-energy gravitons by finding some suitable ground value in the cluster. This is extremely successful up to a point [1], but as the energies of the gravitons increases we have to take more measurements to find a suitable ground value, and eventually we have to take an infinite number of measurements to find one. This is what is meant when you read "gravity is perturbatively non-renormalizable".
There are, as you suggest with ("... space-time curvature ..."), other values related to the gravitational interaction that we can turn into a quantum field [3], but most suffer a highly similar fate: in some conditions we have to do an infinite amount of work to make our field values sensible and match observables.
Finally the gauge field that we built on the metric is fully relativistic and generally covariant, so it works with any system of coordinates, choice of units, slicing of spacetime into 3+1, etc. that we want, up to diffeomorphisms (we have to remember that we chose a static background spacetime). So even though it gives us useless readings in some regions of a spacetime containing strong gravity, perturbative quantum gravity is a useful and standard tool. However, it is not considered a candidate for a fundamental theory (barring some unforessen advancement in renormalization theory) rather than an effective theory and moreover by implication it undercuts General Relativity's claim to be a fundamental theory too.
- --
[1] Relativists tend to define "strong gravity" at this point, since we get correct results from renormalization at any energy lower than it. Strong gravity only appears very close to gravitational singularities (and in the case of black holes, that means well inside the horizon). If we are using the path-integral formalism then we'd find that we have "strong gravity" in this sense in every Feynman diagram containing at least one loop of gravitons.
[2] https://arxiv.org/abs/gr-qc/0206071
[3] https://www.wikiwand.com/en/Canonical_quantum_gravity
[4] One might ask, "is there oppositely-gravitationally-charged matter anywhere"? It's an OK question, and people have discussed it seriously. Sabine Hossenfelder has touched on this a few times on her blog, including http://backreaction.blogspot.com/2017/04/why-doesnt-anti-mat... although while the photon has no electromagnetic charge, the graviton itself (in perturbative quantum gravity) has gravitational charge (this reflects the non-linearity of General Relativity).