For anyone using iterm2, copy mode (https://iterm2.com/documentation-copymode.html) was a big help in reducing my mouse use.