The electorate will do what it wants. If people keep electing terrible reps, term limits don’t stop that, it just stops that one particular bad rep and replaces them with another. Similarly, we have some awesome Michigan House members who would be fantastic again, but they’re term-limited and have to run for something else to continue to serve. It’s sad.
I think there's a strong argument that many representatives don't necessarily start awful, but are driven to awfulness in the pursuit of keeping themselves in office. Like you allude to, most voters are grossly misinformed or vote irrationally. And this irrationality and misinformation is very effectively exploited by special interests and monied groups. Consequently the desires of these special interests and monied groups start to become more important than the interests of society when a politician is primarily concerned about staying in office.