Only if you want to squeeze the best performance out of the web - for instance if you were serving up JavaScript that is different for each request you could have it pass through CloudFront. You see this in the widget and ad business quite a bit.
Cloudfront doesn't replicate your keys out to each edge each time you invalidate. It only retrieves them from the origin as a result of a cache miss. If TTL was 0 you would miss each request, making each request slower than serving directly from the origin.
how could this possibly be faster than a direct connection of the client to your origin server?
I don't think it would be. Also, if you're constructing purpose-built javascript the best solution might be to merge that information into the url scheme so that you can cache the results.