Yup, its definitely a dicey issue. 5-10 mph on a skateboard is definitely a courteous and safe speed.. The problem is that 12-20 mph is also attainable and unfortunately not everyone is as courteous as you. It's the bad apples that spoil the bunch.

Yes, I do extend this to rollerbladers and scooters. Segues have a maximum speed of 12.5 mph and restrict their own downhill movement. I'd say they're sidewalk safe. Same with wheelchairs.