One problem is that the first person is often female, not white or foreign.
Another is the demotivational effect when the first person eventually does find out they are underpaid versus colleagues or the market - and they generally do. And then they will leave.
It gets worse when the lower paid person is clearly outperforming the higher paid person, or where the higher paid person is clearly under performing.
And finally it's just not the right thing to treat people unfairly versus their peers. If the lower paid person is being underpaid then quietly fix the issue.