Yeah, most of us absolutely are not getting our money back.
The importers pay the tariffs, and they might get a refund, but it's unlikely they can distribute the money back to the people who they passed the price increase onto.
Imagine I imported 1 ton of rice and paid the tariff. Then I split that ton of rice into 2000 one pound bags and sold them to two super markets, with a higher price accounting for the tariff. Then one super market decided to absorb the price from their margins and sell it at the same price as before to avoid price shocks. Can I track down the other 1000 purchasers who paid a higher price? Is it even worth it?