My reputation/referrals have allowed me to skip parts of the process but never the whole thing (I've skipped phone screenings, for example). I haven't been in the industry as long as you (~6 years) which might have something to do it. Also, you may be well known whereas I am not.
As an extreme example, you aren't going to get a job at Google without spending several hours coding at the whiteboard.
Why is this? Well first of all, most candidates can't actually program. Asking them to weeds out a lot of the chaff. Of course there are people who obviously won't have trouble with the simple questions. But if, on general knowledge and someone's say-so, it is obvious that they meet your bar, then the odds are good that they are better than what your company expects. If you're happy to always be better than those around you, this is fine. But if you want to land in a more demanding environment, it won't be obvious to anyone, including you, that you meet the standard. And so they will need you to demonstrate somehow that you do.
If they don't, that is a red flag. It either means that they are not selective enough to guarantee decent co-workers, or you're so clearly above the bar that you're not going to have co-workers in your league.