Thats not really loading it from the remote site is it, its more the url bar lying to the end user? What happens if you copy the url or enter it directly?
Entering the URL directly will make an HTTPS request to the URL's origin server, as it always has. There will be no AMP Cache involved in that request, but it will work just the same.