What's frustrating is that DNS is returning an address, instead of just failing, and so macos is caching that value (though it might be cloudflare doing that).
sudo dscacheutil -flushcache; sudo killall -HUP mDNSResponder
dnsmasq has a really useful feature for dealing with this: --bogus-nxdomain