In case of smaller sites page load time would increase, because they still had to contact an ad network to retrieve the ads and assemble the page with ads on server side.
It would only improve load time for big sites which have their own ad serving and a server farm on the backend to quickly retrieve ads without having to connect to an external ad network.