There aren't that many leaving simultaneously - plus they tend to be slotted into the same group of platforms.
As somebody who commuted for a long time at peak rush hour back and forth from Penn Station to D.C., I can tell you that following people who "seem to know where they're going" would not be a reliable strategy.