I suspect you're right that an adult will be able to achieve "nearer" phonetic fluency in a language that shares more phonemes with their first language than they would in one that has little phonetic overlap with it. So a Spanish speaker will have an easier time becoming fluent in Italian than would an English speaker, etc. (I'm surprised by your assertion that Japanese has large phoneme overlap with English, but I don't know much about it - it just seems surprising to me.)

I'd be curious to see a visualization of the overlapping phonemes between languages, like a large Venn diagram kinda thing (or maybe just a graph). I think this data is probably available and a visualization wouldn't be too difficult to create - maybe it already exists but I couldn't find a comprehensive one with a brief search.