[1]: I prefer saying Java platform rather than JVM, because a program written in Kotlin shares over 90% of its infrastructure code with a program written in Java (not including the OS), and the JVM constitutes only about 20% of that.
When I was developing a VR game, my team noticed that some ultra-sensitive people started to get sick when we would drop around five frames (the vive displays 90, so this is only a ~5% drop). There was more to it (dropping a few frames every few minutes was usually fine, but consistently dropping frames every second or two is an issue).
If GC only causes occasional (maybe every 2 minutes) loss of a frame or two it should be no problem. If you watch benchmark FPS traces that count every frame delay you'll see that occasional stutters happen in basically every game.
Loss of a single frame just isn't noticeable
Streaming/Loading assets in the main render thread is a very 2005 thing to do.
And some games much later didn't. I recall needing to fire off a shot on each new gun I picked up in Far Cry to be sure I wouldn't get a stutter later at a more critical moment.