1. You ignored inflation. Your argument gives a justification for taxing the real gain, but current CGT taxes the nominal gain, which is generally greater. So the difference between the two is taxed twice.
2. Let's do some math. For simplicity let's consider a world with no CGT at all, only income tax.
Say I earn $10k income and pay 40% income tax, leaving me with $6k. Then I invest it and gain 400% on my investment, so I have $30k.
If I had not been subject to income tax, I would have taken home the entire $10k I earned; I would have invested it and gained 400%, leaving me with $50k.
But in fact I only have $30k. Since, out of my total (admittedly hypothetical) tax-free profit of $50k, I have lost $20k owing to the imposition of a 40% income tax, how can you say my capital gains have not already been taxed?
As with everything, it comes down to an argument about what's "fair".