Low demand.
Licensing costs from third party authentication providers.
Increased support costs.
Early adopter status of the technology.
Skepticism as to utility (to wit, malware that captures token keys or output).
Integration with non-interactive logins, APIs, mobile software.
Don't get me wrong, I like two factor auth, but there are lots of practical reasons it's not universal.
I also have found that carrying around cash is a lot more secure than using my debit card. My debit card has been copied, and money stolen from my account, 5 times. To date I have never had my wallet physically taken from me nor lost any money that way. Cash is compatible with every system I interface with in-person and it carries no service charge. This seems backwards to me. Banks really need to get with the program.