No. You read about money sitting in the Caribbean because the US tax system is broken and tries to tax money made anywhere in the world (this makes no sense, and no other country does this).
BUT .... that doesn't mean they avoid the US taxes! For Google the money ends up sitting in Bermuda: not a place noted for its abundance of cheap colo space or computer scientists. Obviously it isn't going to stay there because there's nothing to spend it on. Instead the money sits offshore until something more useful is found.
What US companies want to happen is that the US Govt puts in place a "tax holiday" in which they temporarily stop being idiots and use the same tax rules as the rest of the world, thus allowing those companies to move the money earned abroad back into the USA without it being taxed twice (once by the country where it was earned and once by Uncle Sam). This has happened before so they think maybe it'll happen again. Then they can spend it on building fancy offices and hiring splurges, all of which yields taxable income inside the USA.
What actually tends to happen is that politicians, even the ones that acknowledge the sense of this, don't like being seen to give big companies a tax break. So they prevaricate and eventually the money gets spent abroad in places like Europe or Asia, where double taxation treaties work properly. At that point the money gets taxed via payroll taxes, employee income taxes, VAT etc. Own goal for America.
Still ... the money doesn't sit there because they want to avoid US taxes. They generally want to avoid double taxation, and then spend the money saved inside the USA.