Yes, I also usually only read the comments. Only if I'm convinced (by the discussion) that the link is worthwhile, will I click on it.
The linked article itself is also allowing others to influence your judgement and opinion, especially if it's a news article.
There are usually lots of valuable features contained in comments that are quick/easy to use.
It's a compromise for time and also a small measure against tab-explosions.