I would expect diversity to decrease with diffusion rate, so e.g. diversity in soil or mud would be lower than diversity in water.
There are more unique species on mountains or islands because it is hard for those species to leave and take over new territory. On the contrary, something on continental land or ocean can expand huge distances if it has new adaptation.
Just my amateur speculation, but I would imagine underground caves are very isolated from other caves even relatively close in terms of distance.
> even at depths where the energy supply is orders of magnitude lower than enjoyed by organisms in habitats that see the sun
> Something unexpected that caught Ruff’s attention was how total diversity went up with depth. This was surprising because less energy is available at deeper levels of the subsurface. For archaea, diversity went up with the increase in depth in terrestrial environments but not marine environments. The same happened with bacteria, except in marine instead of terrestrial environments.
Also a additional "wild" theory (from me, wield as in not properly researched/fact based):
Mono cultures or quasi monopolization of resources of one species rarely survive long term in nature (as in either they kill themself or them or the situation changes enough to remove the "monopoly". And if you go very deep underground you have increasing slower interchange/mixing between localities, and in general live is slower acting (to preserve energy). So over fit species (which risk themself due to monopolization) might in such bioms die out or fix themself before they can widely spread.