This is not guaranteed at all:

> Note that the fail-fast behavior of an iterator cannot be guaranteed as it is, generally speaking, impossible to make any hard guarantees in the presence of unsynchronized concurrent modification. Fail-fast iterators throw ConcurrentModificationException on a best-effort basis. Therefore, it would be wrong to write a program that depended on this exception for its correctness: the fail-fast behavior of iterators should be used only to detect bugs.

Source: https://docs.oracle.com/javase/8/docs/api/java/util/Vector.h...

This is conceptually similar to the race detector in Go, which is also a debugging tool, but without strict guarantees.

Java might throw a ConcurrentModificationException. The docs are quite clear that it's not guaranteed and best-effort only.