I think there should be a serious punishment but lengthy jail time may not be the best answer. I would fully support some period of house arrest, lengthy community service, and whopping big fines.
In the one incident I'm aware of an operator crashed his drone at the Seattle Pride Parade, striking a woman in the head. He received a $500 fine and a 30 day jail sentence. The city attorney was aiming for 90 days of jail time.