I think the thing most people are missing is that the lack of delays is due to having a system that has few delays :-) When I lived near London, my morning train into London had the following performance record: 75% of trains arrived within 10 minutes of schedule, 95% of trains that
weren't involved in a cancellation arrived within 10 minutes of schedule. "Involved in a cancellation" means that either the train was cancelled, or was late because a previous cancellation made it late. So this means that more than 20% of the trains on the line were late directly due to cancellations (which is a horrible statistic -- my line was one of the worst).
But even when you think of that terrible statistic, when the next train comes -- there is not enough room. Because the platforms are not long enough and/or the signalling equipment can't handle it, they can only run trains of a certain length. When a train is cancelled, then it causes mayhem down the line as people cram to get on the trains to get to work. If you have 5 trains an hour, you only have to get to a delay of 12 minutes before you need to cancel a train. It all snowballs.
Basically, what I'm saying is that it's not like Japan doesn't have all the problems you've stated, it's that they realise that if they don't deal with the problems quickly the entire system breaks down. As others have said, some of it is cultural (Japan is a rule following culture so if you put up signs saying "This is the rule", then by and large people will follow it). But other things are just that the train companies realise that it is more than worth their while to pay for solutions to small problems in order to avoid the snowball effect. For example:
- There is a guard on the platform almost all the time. My dad actually dropped his bag on the tracks once and the guard quickly and efficiently retrieved it.
- They have equipment to inspect and clear the tracks. They use it regularly. Japan does not have large herds of animals in general, so animals are less of a problem it seems. But the shinkansen has very large fences all along the length and the track is regularly patrolled. My mother in law lives near the shinkansen tracks and I've seen them patrol the tracks several times a day.
- Virtually every street crossing is controlled. This makes a big difference.
- People with disabilities are looked after by guards at the station, not the conductor. When you go through the gates, you can ask for a wheelchair. The guard will go and get one. They will accompany you to the platform and make sure you get on the train. They will call ahead to your destination and a guard will meet you to take you off the train. Also, it's important to understand that the platforms in Japan are almost universally perfect so getting on and off the train is very easy.
- Technical breakdowns are rare because they spend money on maintenance. However, in the UK, I recall hearing that many of the problems are due to people stealing copper wire from the switching equipment to sell as scrap copper. Again, this is a cultural thing -- that kind of crime is just incredibly rare in Japan.
Trains are relatively expensive (about the same as the UK), but the main answer to your question is that they are managed really well. They use the money efficiently to stop small problems becoming big problems.