> you often have to have your own experts anyway.

+1 on this

IME the most successful outsourcing arrangements revolve around scale of work, not substituting an entire department. Keeping the amount of in-house experts lean and having them manage outsourced resources. So it is more of a tool for someone internal who has the needs organized rather than a dependency on a 3rd party to know what is best for you