Observations of the missing baryons in the warm–hot intergalactic medium

No, while the observations were of oxygen, this doesn't imply that the gas is primarily made of oxygen. The issue with the warm-hot IGM is that it is a strange temperature where virtually nothing emits much light. Highly ionized oxygen (in this case, only one electron out of the original eight remain bound) is one of the few things that does. Here, based on X-ray observations they have estimated the density of this oxygen, and then estimated the overall density of the IGM based on the ratio of oxygen to hydrogen.

I guess (IANAP) that's because hydrogen is easier to observe as part of interstellar gas within a galaxy as opposed to the thinner intergalactic filaments which is also further away and doesn't get illuminated by stars or stellar remnants.

So the baryonic matter in galaxies is already accounted for to a larger extent than the filaments.