Milliseconds absolutely matter. Every React render cycle (the entire relevant subtree from top to bottom because again, it's all synchronous) that takes more than 16 milliseconds presents to the user as a dropped frame. In fact that 16ms also has to include any DOM updates and the resulting browser reflow, which you have much less control over and are usually more expensive than your JS, so really the window is smaller than that.

Also, many of the optimizations I listed above can make code less readable in small ways. Many of them are things you should not do eagerly (premature optimization is the root of all evil), and should only go back and do once you've identified a specific problem. If an across-the-board speed increase prevents them from ever becoming problems, that's a win.

If you're thinking I'm a JS-hater, you're wrong. I think JS is a good language and I love using it where it's appropriate. But there are some usecases that benefit from a faster technology, and it's absolutely bonkers to try and argue that that technology shouldn't exist because "you can still make something slow with it if you really try".