We did the same thing, then migrated back to React-Router. We had issues getting relative urls and redirects to work, examples were hard to come by, and generally the experience was just poorer than React-Router.
I did the same as you, initially reach router seemed like a great alternative, but it really doesn't feel finished, updates are few and far inbetween, and there is a large backlog of open issues / PRs.