What if ESNI/ECH is being used?
I could just visit 1.2.3.4 to reach the captive portal, but my question was more about "how can they allow based on SNI if that is encrypted too"?
Regardless of what DNS server is used, my connection requests to websites (whether to a real messaging site, or a faked xray one, or whatever) might be using ECH, so how could they allow the request in that case? I don't think they can control whether those messaging services are using ECH either.
Just blocking DoH would probably be enough to force clients to fall back to regular DNS. This happens automatically for a captive portal with a whitelist. You could probably take care of clients with a cache by just dropping any connection without a readable SNI. I guess it depends on implementation, but I'd wager most clients would fall back to a non-ECH connection anyway (since it doesn't really add anything - you could just sniff DNS to figure out what domain the client is trying to connect to).
I feel like a simple IP block/whitelist for the messaging service would be so much easier than all this SNI business. There's nothing stopping them from verifying specific SNI certs against the real ones anyway if you went the xray route.