I'd never done anything much with pam before but found it relatively easy to get PrivacyIDEA working on CentOS, it's even easier if you're using Ubuntu. https://www.privacyidea.org/ and Cornelius is really helpful.