Probably the fact that S3 is eventually consistent and IO is not immediately visible to other clients?

I'd suggest that OP use something like https://aws.amazon.com/fsx/lustre/ if they really need to mount S3 as a filesystem (do read the documentation regarding distributed consistency though), but other than that using AWS solutions in general (Athena, Presto on EMR, Spark on EMR) will tend to use specialized S3 committers such as https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-spar... and are quite good at minimizing this problem.

Other than that, read the S3 docs for this behavior.