First, the software technology scales very poorly unless the size of your data model is trivial. The semantic web is a type of graph search problem with all of the hard algorithm and data structure challenges that entails. Most interesting semantic web applications require data models operating at scales where existing graph platforms have pathological performance characteristics. The gap between capability and requirements is several orders of magnitude. You can't throw hardware at it; when I was first hired to do CS research on scaling the semantic web, I was working on literal supercomputers.
Second, the human element of the semantic web doesn't scale. Semantics are contextual and subjective. As the number of people contributing to the data model increases, the consistency and quality of the data model decreases. This becomes an extremely expensive global coordination problem at scale that may be unsolvable. It is broadly recognized that you need a single, global arbiter of semantics in any scalable system just to maintain semantic consistency, which basically means strong AI.
If we have the technology to solve these two problems, I suspect the Semantic Web will no longer be interesting.