If built without server-side rendering, they're often obscenely slow. Browse the Modern Web™ on 2G and it's rage-inducing. A comment I made when this same question came up recently: https://news.ycombinator.com/item?id=13212789
If built poorly, all of the browser features you've come to rely on break. Like the back button. And links. And scrolling.
Does your website need to be accessible? Better remember to build that in too.
How about SEO? AMP? Browser history?
Most of this stuff works out of the box with traditional HTML and server-side rendered pages. So even though it's 2017, I think the question you should still ask yourself is, do you plan to re-implement all of this functionality? If not, 10MB of the hottest new JS framework may not be for you...
(And this is coming from someone who builds SPAs for clients for a living.)