"Marginally ROI positive" works in a ZIRP environment. These are huge capital investments; they need to at least clear treasury return hurdles and importantly provide attractive returns.
I am fundamentally skeptical of "scaling inference". Margins are not defensible in the market segment OpenAI is in.