I think its always been about money and time, effectively this is what happens:
- A lack of people able to make sustainable time commits to the cause means even with a fair amount of money, you could lose out in numbers (voting or otherwise)
- A lack of funding to sustain the cause, even if you have members that can make the time commitments, and importantly, not enough money to have professional lobbyists working your cause singularly
Other lobbying organizations, typically with corporate funds, don't have either of those problems. Corporate friendly organizations like the one's Amazon employs (or employs lobbyists directly, not an uncommon practice either) have hefty sums of money to keep lobbying, 24/7.
Some of this could be fixed with reform, of course, at least on paper. For instance, removing term limits allows representatives to bulk up on expertise on issues, which is actually makes them more resistant to outside influence, all things being equal.
The other thing that has gutted this is organizations that do exist are either always scrambling for money, like the EFF, ACLU etc (yes they have money, but compared to corporate lobby organizations, they have a finite amount of donors they can "call action" to, without hitting a wall), or have been gutted in other ways. One forgotten aspect about labor unions, is they did act as an effective lobbying force in politics for workers rights and other causes[0], sometimes not even labor related directly.
Even on a local scale, you have incumbents who can fall back to their local network of people, and challengers have to build this over and over again. Thats the fundamental problem.
I think systematic reform would be the actual long term solution, but its hard to change a system that is primarily concerned with perpetuating itself.
[0]: https://exhibitions.lib.umd.edu/unions/political/environment