Once people see others walking towards a platform (particularly if they just checked something on their phones), they tend to do the same.
As somebody who commuted for a long time at peak rush hour back and forth from Penn Station to D.C., I can tell you that following people who "seem to know where they're going" would not be a reliable strategy.