there is practically no attack vector in using infura.

Infura can't fake your can't fake your cryptographic signature. Worst case, they censor txs from your address at which point you can run your own node.