Most security you'd encounter in India doesn't even carry sidearms, just Lathi sticks or batons.
At a guess you've seen the police with rifles at high value terrorist targets - airports, major stations, national tourist attractions/monuments. Such locations will have police with SMGs or assault rifles in most other places too, including places like the UK where police are rarely armed otherwise...