A blockchain decentralizes trust, you can always solve the same problem a blockchain tackles by introducing a central point of trust. The question is if this choice is acceptable, more resilient, cheaper...
In the end you do trust the code, a computer running the code or some currency exchange.
There's both centralized and decentralized exchanges available.
There are a lot of banks, with different interests that usually compete with each other and work under regulations that have been discussed and approved after reaching consensus
There are several different central banks, that work together under regulations discussed and approved because they reached consensus
The "decentralised" exchange already exists