Edit: Hah, I just realized that congressional sessions open with prayer as well. Not sure what other countries does this?
Likewise, the US prayer is non-denominational (it typically is monotheistic though). Ireland, Canada, South Africa, and the UK also have parliamentary prayers.
UK currency often features the letters "D.G.", which are the initials to a latin phrase meaning "by the grace of God", but other European currency references to God have ended with the switch to the Euro.
The US certainly has above average entanglement of religiosity and governance, but hardly in a sense that makes it a theocracy. Politicians talking about faith and God is a very different thing from, eg, the country being run by the pope.