You're absolutely right. There's definitely real limits, vertically and horizontally, to how much density is physically possible.
As an upper limit, we have good historical examples density of around 1.2 million people per square kilometer being workable. I don't think any part of the US today comes close to that - what data I can find says that the most densely populated part of the US is at 66,000 people per square km. That's multiple orders of magnitude different.
With that in mind, I think it's possible that we haven't yet hit any real physical or space-based limits in American cities. Only political ones.