Virtual DOM came about since it offered a simplistic top down view = fn(state) model without terrible performance. Other top down renderers were terribly inefficient and this built on that. It was never innately faster than targeted direct DOM manipulation. It was just compared to other approaches that were innately built on diffing as well. And things like reading from the DOM can cause reflows and other terrible performance bottlenecks.

Fine-grained reactivity existed back then and was more performant for updates. Always was. Just had its own issues since pre-MobX we didn't see implementations in JavaScript which provide glitchfree execution guarantees. So Virtual DOM was a great invention but I think it was misrepresented early on. That's what got me to start working on Solid. I knew the performance was there without a VDOM from day one. I'd seen it. So when Knockout started waning in popularity 2015/2016 I started working on a replacement.