I don't know about the US, but herearound the sky and what flies there is massively regulated.
And those are regulations you don't want to fuck around with.
You may be, for example, flying over military controlled airspace. I don't see it that anybody can just jump into his flying car and just fuzz around at her or his discretion.
There's a reason why flight plans are filed for every flight.