Indeed - Twilio is awesome purely because it allows you to integrate with the phone networks, which are basically a legacy system, naturally within your applications.
I've not really played with Asterisk. How is it to configure, and what has your experience been like? We outsource the PBX side of our system to a SIP provider called Voipfone (http://voipfone.co.uk) but it would be appealing if we could do this in-house and have even more flexibility.