You're probably right, but it doesn't always feel that way when you have serial numbers on every piece of currency. And coins are so uselessly low value today that they're hard to use.
You can also track serial numbers when businesses return daily cash collections to their bank.
Especially the biggest denominations won't be passed back to other customers in the meantime, but rather sent back to the bank early.