They mind because they are completely outnumbered. Policing only works if you can point out a small minority and say “these people are causing harm to the majority of us”. With something like fireworks (or cannabis protests in many places) the police are severely outnumbered and they don’t have enough room in their jails anyway.