So with this, you need to be somewhat paranoid, but not totally paranoid.
Those same organizations would likely be looking at PKI-based smart cards that they issue themselves over something like this, though.
The reason that "upper end of secure machines" have disabled USB ports is because they are organization-owned machines that are issued to untrusted employees (often in organizations where all employees are untrusted in the relevant sense). But in the case of first-party machines (e.g., personally owned machines) where the user is similarly security-conscious, that factor doesn't exist. So, really, all you need to be is a security-conscious individual that uses your own computer for things where you have security concerns. (Or, as an organization, be one where the threat profile you concerned about addressing is more external than internal.)