I don't think we are arguing over 50-100 points here or there. It's the general percentiles of performance.
A kid with a 1200 SAT is not going to do as well in a challenging college as a kid with a 1500.
I say this as a 1400ish SAT kid from a nowhere small town where I was top 5% of class and then found myself in a selective engineering program where I was at the 50% level at best. Had I been admitted to say an MIT, I would have probably flunked out. Had a 50% level kid from my hometown gone to my college, they would have flunked out.
SAT allows admissions to compare people like me with kids from well known elite, selective and private high schools. College admissions counselors are not going to know the 20,000+ high schools in the country and how their GPAs compare. It's as close to a clean comparable metric as you are going to get, with a margin of error.
Even a native speaker of English might not get a full score on the TOEFL or the IELTS if they are unaware of the format etc. That doesn't mean the TOEFL isn't fundamentally a strong test of the knowledge of English.
Another way of looking at it: Athletes need training and coaching, but that doesn't mean any random guy paired with a coach will become a top athlete.
That it’s possible to prepare for the SAT to score better doesn’t seem intrinsically opposed to its purpose.