I'm not sure where the confusion is. OP mentioned that his Linux system can already do routing. The purpose of this card is to remove that load from the computer. The manufacture suggests it can do up to 100Gbps which isn't trivial.
To me, this suggests that it's packet-rate limited, and if so, it can really only be counted on to do 1500/9000 or ~16.6Gb/s with standard frames.
It uses another CPU to do that. GPU is fundamentally different, high memory bandwidth, embarrassingly parallel, virtually no branches, and what not. That's just using a different CPU to do more CPU, and using the same OS the host already runs.
Then it requires its own security maintenance (+training) and patches.
IOW the customer has full control of the host, but the cloud provider manages the smartnic. Incidentally, this is exactly what AWS does with their ENA adapters designed by... (ex-?)Anapurna Lab they bought some years ago (:
Pretty much all modern NICs are already using separate hardware to reduce the load on the main CPU. I.e. using a different CPU to do more CPU.
Without that you're looking at sacrificing a whole core or two just to handle 1Gbps, nevermind 10+.