1) DDOS attack is detected 2) Attacking IP addresses are sent automated DDOS abuse notifications 3) ISP, like your credit card company when it's machine learning algos detect fraud ask for human back channel verification like SMS. 4) ISP notifies user of suspected bot on one of their devices. The onus is put on the user to run a secure network and remove or fix offending devices.
This system could work well for residential connections at least.
It could be implemented similarly to the way spam is handled by the internet, bad neighborhoods and networks that don't self police are treated as second class citizens.