How about using the already provided git provided diff-highlight tool ?

git config core.pager /usr/share/git/diff-highlight/diff-highlight