I think a good middle ground might be to do a first pass with a "stupid" crawler, and then re-visit the sites where you were blocked or that just contained a bunch of javascript with a headless browser.
If Google can crawl it, then you can too. And while Google doesn't use a headless browser (or at least I assume they don't) they absolutely do execute javascript before loading the content of the page. And they execute the click event handlers on every link/button and when we use "history.pushState()" to change the URL Google considers that a new page.
You're just going to get a loading spinner with no content if you do a dumb crawl (I disagree with that and think we should be running a headless browser server side to execute javascript and generate the initial page content for all our pages... but so far management hasn't prioritised that change... instead they just keep telling us to make our client side javascript run faster... imagine if there was no javascript to execute at all? At least none before first contentfull paint)
This is true for some SPAs, but not all SPAs, and there's not really any way of telling which is which.
I don't personally attempt to crawl SPAs because it's not the sort of content I want to index.
That being said, with sites like HN, Reddit, LinkedIn, Twitter, news outlets, etc. the lines between “document” and “application” get blurred. In some ways they’ve built a micro-application that hosts documents. Content can be user submitted in-browser. Content can be “engaged with” in browser. Some handle this blurring better than others. HN is an example IMO of getting it right where nearly everything that should be addressable (like comments) can be linked to. Others not so much.
(As an aside, I love marginalia!)
It's simply not feasible to treat them as any other website if you wanna index their data.