> I see no reason why inference can't have a ~60-80% margin for quite some time.

Same as anything with low margin: Viable competition and low switching costs.

Many sysadmins lazily do 100% AWS because they don't know any alternatives. CFO's might enforce using more economical inference providers if the cost is even 10% less.