2FA is already a hassle for users. Now you want to make them do math too? This is not a solution. Just don't use SMS at all. Google Authenticator is a better solution than yours.
Security-wise, having a secret user math function seems more secure than the Google app. I can give reasons why if needed.