I think the whole benefit of a static website generator in general is to avoid the "I copy/paste the same header/footer on each" step, especially copy-pasting the menu/header, as your menu can change as you add new pages / blog posts.
I can see good benefits of having a way to do static site generation with the React ecosystem if you're already familiar with it.