Caller ID spoofing is a true misfeature. The ability should be removed, or consumers should have out-of-band access to the real number in order to be able to at least blacklist it.
Letting the caller set the caller ID is the only way someone calling you from Comcast about your bill can have Comcast show up on the ID. Most large companies like that don't own the numbers they call from, or the call centers -- they outsource both inbound and outbound phone support and sales. Typically to multiple phone center companies at the same time, who all have to call "as" Comcast, and ramp up or scale down with more or less phone numbers as needed. They'll use autodialers too, with real people rather than recordings, to minimize the delay between one outbound call ending and there being another person for that now-available rep to talk to.
So the information is there. However it is worth a lot of money to the phone company and they sometimes resell that information to others who repackage it. They also in turn don't always give you this information even when you pay for caller id which is similar but not the same. Originators can block paid caller id, I have never seen a case where you can block ANI subs
The SS7 interconnection partners usually go through extensive tests before allowing you to hand over signaling traffic via SS7, but this is not so much the case for SIP interconnects, where we're lacking a bit of clear standards (however working groups like http://www.sipforum.org/sipconnect exist and are taken more seriously nowadays).
If you are allowed to do "CLIP no screening" - which means you can set arbitrary caller ids in the user-provided part, the terminating system (the hop delivering it to the called party) is still able to check both fields, so this could be a way to pin down the real calling party, even if it "spoofs" its caller id.
Harmful to whom? Twilio?