Their proof-of-concept only reads memory from the browser it's running within, right? Violating browser sandboxing, yes, but does that mean it would also (theoretically) be able to access the memory of other programs?
And while that alone might seem pretty innocuous, think of things that are kept in memory for a specific tab. This could make XSS attacks able to grab HTTP-Only cookies, or be able to read some autofill data or potentially password autofill information. I don't know enough about the architecture of those things to know if they are kept in the same process, but if they are it's possible.
That being said, I didn't read any specific reason why that JavaScript PoC couldn't be expanded to work on other processes just like the rest of the Spectre attack. And personally I'm assuming it's only a matter of time before it is figured out.
You can enable (experimental) Chrome process separation here:
chrome://flags/#enable-site-per-process