A separate comment for this:
> there doesn't appear to be such a thing as pressure that exists in the universe independent of the particles within it
This is the thing about fixing a gauge: you've decided here that the "standard" pressure is vacuum. That's perfectly fine. But it's also perfectly fine to set the 0 point higher, and talk about negative pressures.
We do that with temperature, for instance; we set the 0 point of our temperature gauges and talk about positive (or positive and negative for e.g. celsius and fahernheit) temperatures. We do that with calendars, for instance, talking about years before or after some 0 point. We do that unthinkingly when we talk about "a quarter past [some hour]" or "a quarter to [some hour]", for instance.
Gauging is done -- often implicitly -- on a huge range of physical quantities. We can even formalize the process into a gauge theory, wherein we encode such choices into a gauge field, which can have its own dynamics. (As an example, let's make a field throughout the Earth's atmosphere where every point has a value which would be reported by a calibrated barometric pressure sensor; we can choose one particular reading as a surface and call it "FL10" and fly airplanes in that surface/layer. Depending on latitude and local weather conditions, "FL10" will coincide or deviate from the 10000 feet above the WGS84 surface reported by GPS equipment at the same point. Further depending on surface features, a radar altimeter may report anything from ~ 10000 feet to "oops you've just crashed into a mountain". Moreover a pressure altimeter can slide "QNH" value around, essentially channging the zero point; so our pressure gauge field depends on how we measure, as well (different observers will report different values at the same point!). Moreover, very fast moving pressure altimeters (naively) may report different values compared to those that are slowly moving low in the subsonic regime. However, by and large, if two airplanes are near one another with their pressure altimeters corresponding to FL10, they have to be careful not to collide! So our pressure gauge field does have its uses in predicting physical events like collisions, even if there's nothing truly special, universal or even locally constant about FL10 or any similar surface.
Note that when we slide around zero points, the physical differences don't change. You're still suffering fever symptoms whether we tell you your temperature in degrees Rankine or kelvins-above-normal-body-temperature. The time in seconds from one northern winter solstice to the next is essentially the same whether you use the Gregorian, Julian, Chinese, or any other calendar to fix the day, month and year.
When we fix these zero positions and label the distances from them in degrees or years or volts or whatever, we are essentially laying down a set of coordinates that are at least locally valid. This lets us concentrate on local interactions without carrying around additional burdens from considering how how our local zero and gauge-markings might not be appropriate elsewhere in spacetime (or to a different culture, or whatever), and generally we can convert from one such fixing to another.
Although it is tempting in the pressure case to say that the zero is universal and so only the choice of units (pascals, pounds per square inch, whatever) is idiosyncratic, let's think about how to represent pressure without making any such choices.
In General Relativity pressure is encoded in the stress-energy tensor as the doubly-spatial components (T_{11} = T_{22} = T_{33} = p) where 1 2 and 3 are the three space-like axes and p is the total pressure. T_{12} is the flux of 1-momentum in the 2 direction, T_{13} is the flux of 1-momentum in the 3 direction, so T_{11} is the flux of 1-momentum in the 1 direction. If we make the 1 axis "left" and "right", then we can think of T_11 as "the flux of momentum from the left going towards the right through a point". That's just pressure pushing on something to the right (or a tension pulling on something to the left!).
We can assign any sort of label to the 1 2 and 3 directions as long as they're orthogonal. If 1 is left-right then 2 could be forward-backward and 3 could be above-below. Or we can say 1 is x-axis, 2 is y-axis, 3 is z-axis in Cartesian coordinates. Or we can say 1 is radial distance, 2 is azimuthal angle and 3 is polar angle in spherical coordinates. And so on. We can also set down units, whether they're metres or feet or light-seconds or anything. And finally, we can set down some origin for this system of coordinates. Whatever such choices we make, we quantify the relative momentum flux, but we don't actually change it.
That there is a flux at all is invariant in the face of these choices of coordinate and gauge. (The more precise statement is that the divergence of the total non-gravitational stress-energy vanishes, so one could in principle concoct an observer that sees the pressure-tension as some other form of stress-energy).
But your question is essentially, "what if there is nothing to the right to push on"? Well, if there is a flux of left-right momentum in the right direction, but nothing to the right, then for a truly pointlike object, there's now less stress-energy in that object and a bit of stress-energy to its right. Conversely, there is now some stress-energy to the right of the pointlike particle that has our test particle as a neighbour to its left. Depending on the gauge theory of the particle, we can now consider what becomes of the two neighbours: do they drift apart? Do they rejoin? In the former case, our gauge field can carry away the momentum to infinity; in fact, that's exactly what photons do: they are gauge bosons, massless, but not momentum-less. In the latter case, our gauge field might pull the momentum back inside the particle, essentially how asymptotic freedom works for objects that feel the strong nuclear force. And indeed the strong nuclear force is again representable as a gauge field: the gluon (massless, but not momentum-free). Photon pressure is even testable with e.g. solar sails. Gluon tension (tension being the inverse of pressure) is seen in hadronization.
One molecule of nitrogen is surprisingly busy with tension and pressure being carried around locally by gauge bosons!
Finally, depending on how deep you mean by "dig deeper into the math", you (or someone else reading this, including me at some point in the future :-) ) might like Terry Tao's take on this sort of thing: https://terrytao.wordpress.com/2008/09/27/what-is-a-gauge/ If nothing else, his approach to "explain[ing] what a gauge theory [is]" there is a bit different from mine here. :-)