I haven't used Plivo, but IIRC, with Twilio, you had to verify you owned the number you used in the Caller ID field, either by buying a DID from them or by entering a confirmation code sent to that number via a call and/or SMS.
As you know, this is a preventive measure against spoofing; however there are some edge cases where allowing custom caller IDs might make sense. (e.g. when you want to have people you called get back to a certain number—custom phone line numbers at a large organization.)