Uh, no.
Paying with cash is exchanging a promise that someone else, usually the central bank, will pay the bearer on demand the value of the promissory note.
Even if you could actually exchange the note for its equivalent in precious metals — which you can't — you're still just exchanging a token without intrinsic value. Its only value is conferred entirely by the monetary whims of the government.
Governments can, and have, on multiple occasions, completely disregarded sensible monetary policy and started printing money when they didn't have enough of it, totally destroying the value of people's cash in hand through hyperinflation. This is something you have literally no defense against.
When paying with some other method, we need to make a clear distinction between debit and credit.
If I pay with a credit card, I am asking someone else, usually a bank, to lend me the money to pay. Obviously, in such a situation, they have the right to refuse, albeit only in accordance with any pre-existing legal agreements between us.
If I pay with a debit card, I am asking one my debtors, usually a bank, to repay part of the debt they owe me to the person of my choosing. They may choose to be somewhat awkward about this, but they fundamentally cannot choose to disregard the debt they owe me, so long as they wish to remain solvent. Nor do they, in any meaningful sense, have the ability to grant or refuse "permission" to pay. They owe me money and, subject to the relevant bankruptcy laws, any pre-existing legal agreements between us, reasonable anti-fraud measures, etc., they have to pay.
None of this has anything to do with the issue of "freedom". It is simply a matter of the way financial transactions work in modern society. The amount of money in the economy hugely exceeds the amount of cash in circulation. Most money exists in the form of credit and debt. Your money exists in the form of credit and debt.
There is nothing scary or unfree about having money in the form of a debt the bank owes you, compared to having it as promissory notes from the government, or as legal deeds to property, or precious metals with a market value. Each and every store of value is entirely reliant upon the continued proper functioning of the market, the legal system and the government.
Money in the form a bank deposit is probably the safest way to store moderate amounts of money. As a physical entity, cash can be damaged, lost, stolen, etc. And the only person affected by this is you, meaning you have no common cause with any other person to recover your losses. A bank's stability and existence is underwritten by the livelihoods of all its customers, employees and investors.