They did say that!
Look at gdp numbers or other economic stats. British voters aren't worse off because of brexit. They're told they are and probably some of them believe it, but it's not true, you can see that in the data.
Anyway during the campaign Remain politicians said there would be economic impact. Leave said it won't be as bad as is claimed (they were right), but even if there is an impact it's worth it to regain control over other things. That was their argument. In a vote with 30 million+ voters you can find people who will say anything, but there was no way anyone could miss that message.