Legacy. The US designed air transportation and had an airport in every city before any thought was even given to the concept of transit. The airports are designed to get people to and from the United States, not for people on a hop between other countries.
Yes, this is a competitive disadvantage. I don't think there is a lot of money around to solve this problem. I'm sure if, say, the Port Authority wanted to have a sterile transit area in JFK, they could probably convince the federal government to allow it. But they don't care. Nobody with money cares. Therein lies the problem.