It's not a 20m / 4% floor, the fines will be discretionary and most importantly are likely to be incremental.
EU regulatory authorities do not usually operate in a "shut your service off for non-compliance" mode: assuming they receive a tip-off about your business not being compliant with GDPR
1) they send you a notice and ask for proof of compliance. If you produce evidence, the case is likely closed.
2) assuming you can't produce that evidence, they will give you a warning and a deadline by which to become compliant. A small fine may be levied at this time
3) if the deadline passes without you having done anything, they will eventually get round to giving you a new date, issue a sterner warning and increase the fine for non compliance.
To get a $20m fine you'd have to be a massive corporation and be caught repeatedly flaunting the regulation in an outrageous and damaging way (for the consumer); any idea of closing off your small service / website / whatever to EU users because of the perceived risk of a fine is a massive overreaction.