Interesting.

Reading that I'm not sure the problem is that SRV lookups are slower, but that forcing clients to do a SRV lookup, wait, and then a A/AAAA queries is slower:

If I own a popular client and I switch to mandatory SRV lookup, and don't issue A and AAAA queries until the SRV lookup fails, then for the services and paths where SRV lookup doesn't work, user experience will be vastly degraded.[1]

However,it appears that making SRV optional remains a possibility:

Note that none of what I've said necessarily applies to optionally doing SRV and using it when the results are available.

[1] http://lists.w3.org/Archives/Public/ietf-http-wg/2014JanMar/...