Ask HN: How do crowdsourcing apps protect against bad data?

I've never worked on an application that uses crowdsourcing to manage common user data, and lately I've been curious about how one would go about implementing it.

It seems like a "too simple" approach would be vulnerable to trolls and user error.

With an example like Waze, I imagine there might be some threshold of reports that must be reached before a change is made, but that might not work as well in other applications that have less users or have data that changes less frequently.

There are other approaches I can think of, but I'd like to hear from people who have actually worked on such systems. What makes one of these systems work well, and what kind of data would make such a system a poor fit?