You can but the issue is that if the caller _thinks_ it can reach it over IPv6 it'll never send the IPv4 unicast packet which means the server won't wake.