Yes, I think this is the biggest issue. Since many websites are funded by advertising, they do not want people to be able to extract data from their pages, they want people to "visit" the site and view the ads.

Also, rights. People "own" things like sports fixture lists... they don't want you extracting that data without paying to use it.

The semantic web could be perfect technically, but it was never going to apply to content that people were attempting to "monetise"... which seems to be most of the web's content.