Fastmail business account: https://www.fastmail.fm/help/receive/domains.html
Both options are very inexpensive and if the client still wants cheaper I would probably refer them to someone else for advice.
Edit: I should say that of course they need to have a recourse to stop providing services to people who don't pay. However, giving maybe 30 to 90 days to at least download your mail after your payment is late would be nicer than going "remember years of email you have saved here? Say goodbye to it all".
This was a while ago, so my memory may be failing me, but I'm fairly certain we had an alternate "recovery" address set up - just no one thought to send an invoice there, or call the phone number on file, or whatever... it was a "hey, you show no signs of having received the invoice, so we assume you're a PITA deadbeat and we're going to perma-delete the account for you!"
Moral of the story (from my perspective) - if a customer isn't responding to your invoice, it might just be that they don't have it... try to reach on them a side channel before cutting off service!
You can set up a 'Personailty' address to match the alias, & select that when sending the reply. There may be a way to always use a specific personality with certain addresses, but I haven't used the feature in so long I don't remember.