The late night pubs do exist, sure, but a lot of them stay open to capture any foot traffic from people missing their last train around midnight. Most don't serve full menus at that hour and it's usually a skeleton staff (short of, say, a HUB or something... but those aren't going to be open past 2, short of Roppongi & co generally).
Japan (really Tokyo, I guess) stopped being truly 24-hours post 2011 earthquake. After the power issues around the country, I noticed a lot of places never returned to their same livelihood. It's a shame, was fun while it lasted.
I'm mostly commenting at this point because the misconception of Japan being this constantly-on always-good-food crazy ass heaven is really tiring to see these days.