Iaas provider is not going away,paying for the cost of SDN now for features that doesn't even exists yet, is insane.
SDN isn't required for k8s, what is required is that each Pod (group of containers) get it's own IP address, and that the IP address is routeable in the cluster. In many cases, the easiest way to achieve this is via an SDN, but it is also achievable by programming traditional routers.
The reason for wanting an IP address per pod is that it eliminates the need for port mangling, which dramatically simplifies wiring applications together.
And ports are different between different replicas of your service, since they're chosen at random during scheduling.
It also makes ACLs and QoS harder to define for the network, since you don't have a clean network identity (e.g IP Address) for each application.