Would a way to test this hypothesis be to switch garbage-collecting algorithm?
G1GC should be fine, so enable GC logging, analyse them using:
https://www.tagtraum.com/gcviewer.html
or
You can log GC for long time with rotation, something like this https://dzone.com/articles/try-to-avoid-xxusegclogfilerotati...
For the GC analysing you will be looking for tenured generation, so must add this flag: -XX:+PrintTenuringDistribution
You would be looking for GC major and GC evacuation times. Major GCs are STW and take more time, so overall the goal is to eliminate them as much as possible.
I usually find it very important to have charts of heap usage. Overall heap allocated (all regions) vs complete heap size. The same for non-heap area.