I'm afraid I still don't understand you.
> JavaScript appears to not be a possible attack vector. So now we're just worried about already had access to run a native binary.
All the major browsers have pushed patches, to close this, because it was a gaping hole. You're only not worried, because they've already fixed it. (Well, when I say major browsers, I haven't seen or heard anything from Microsoft's Chakra engine.)
Take note that it involved killing off SharedArrayBuffer in JS, at least for now.
> but obviously if a rogue binary had to sit running for days with only a part success rate
That's misreading the situation.
A tight loop is required, yes, with partial failure, but it is not so low as taking days. The original implementation [0], read at around 10KB/s. Whilst that isn't fast if you're somehow trying to read gigabytes of data into cache, and then attack it, it's more than enough to read passwords out of data.
> So now we're just worried about already had access to run a native binary.
Now we're worried about any unpatched JavaScript engine, like perhaps Chakra, which would make all Edge users vulnerable.
You also don't need access to an executable binary. You could also target any DLL that gets dynamically linked on Windows. In fact, the example Windows implementation hooks into ntdll.dll, which has a known location on all versions of Windows, which allows it to bypass ASLR.
[0] https://spectreattack.com/spectre.pdf