There's quite simply no easy fix here. The best schools (and yes, that includes many public schools) are teaching a variety of skills that help the students whether they're going on to college, a trade school, or straight into work. And I suspect that those schools would be competitive or even beat Japan and Finland.
But just look at the conditions of some schools. --Detroit is making the news for schools that are falling apart and infested with rats and worse. When the government won't even provide funds to provide a clean, safe space for students to learn, just how much can you expect from the teachers and students?
Most of us don't feel good about going through increasingly intrusive screening at airports. Imagine how students at some of the schools in New York feel having to stand in long lines and go through metal detectors every day before class. Do you really expect them to respect an environment when they're put through that?
In Kansas they're trying to defund the state's Supreme Court because they ruled that the funding of public schools was inadequate and distributed unfairly to the point of being unconstitutional.
Look at the Republican candidates all crowing about how they'll shut down the Department of Education if they're elected. Don't get me wrong, like all government agencies, some reform is needed, but there are plenty of examples to show why the department is necessary!
There are other factors in play as well, of course - as much as people would like to pretend otherwise, there are a lot of people living in poverty in the US, and students living in poverty consistently do worse in school than middle and upper class students.
So really, there are a LOT of reasons for the US scoring this way, and unfortunately our country is not doing a good job of addressing these reasons.