I think reasonable people are worried about exactly that. Enraged violent people managed to get a few meters away from a room full of Congresspeople validating an election, in support of their charismatic guy who believes much worse.
Q-Anon has believers currently sitting in Congress, and is predicted to gain more seats this year[1].
1: https://www.businessinsider.com/the-36-qanon-supporters-runn...