the nomikai is part of the work, where the social status is being reinforced (pouring drinks while deeply bowing to the boss, getting forced to drink, the speeches, the clapping, etc)
Most people are more than likely getting off at regular hours and going home or out for dinner and drinks (whether you consider that work or not -- they arent being paid). Otherwise the bars and trains wouldnt be packed full of people at 6pm and happy hour wouldnt exist.