It's about your individual liberties limiting someone else's personal liberties. Nothing collective about it. But in public discourse, it is always the liberties that powerful people benefit from disproportionately that are framed as "good". I.e. when there was a concerted push by black liberationists in the US to form armed militias, gun laws were tightened. Now that gun ownership is interpreted mostly as a right valued by disgruntled white people, it is expanded.
Same with equating of control over money and liberty. We live in an age of almost unparalleled wealth accumulation in the hands of the few. I'm sitting on my local council and I can tell you that if we have to increase the fees for school lunches due to low tax revenues, there is nothing "collective" about the ramifications. It will directly and forcefully impact a relatively small number of individual children, whose parents will have significantly (for them) less disposable income as a result, severely limiting their liberty to afford their children a decent start to life.
It is beside the point if I think that someone can or can't make good decisions about the use of their own money. Even if I assume that they'd make much more profit if they invest it on their own, I'd still argue that a healthy society should be based around the principle of solidarity and a wholistic view of individual freedoms, and not just advance the advantage of a select few that have the means to push for their favorite liberties to be prioritized over everyone else's.