In lala-economic-theory-land yes. But this is not companies competing for the favors of customers. This is nation states competing for the favors of companies and that is a totally different matter.
In the real world you need taxes to operate a country and with big business being able to relocate their capital and their profits at a moments notice corporations have a substantial advantage over nation states (which are by definition somewhat immobile).
So the 'inefficiency' that you seek to eliminate is the oil required to keep the machine running. Note that a company couldn't care less if your country roads fall into disrepair, healthcare sucks and education starts to fall behind. It won't show up in the quarterly reports for a long time and by the time it does they can easily move on to another place.
Having a very low corporate tax is only possible if your country does not require major infrastructure and that's one of the reasons why most tax havens are either nearly un-inhabitable (Cyprus, Panama) or extremely small (Monaco, Lichtenstein, Vaduz, Luxemburg).
Ireland is the exception and it only succeeded there for reasons that the UK will find hard to emulate.