This totally works, but does have some drawbacks:
* High friction. The advertiser needs to seek out sites, send emails back-and-forth, agree on pricing, etc.
* Cheating. How does the advertiser know that their ad is really being shown? How do they know that the publisher really has lots of visitors, and isn't just sending them back traffic?
* Lower payments. Unless you're writing about something where people spend a lot of money (insurance bloggers, etc) you'll likely make more money if you use something that supports personalization.
The way this works out is big publishers (all the brands you've heard of) make direct deals with big advertisers, but (a) they still make a substantial portion of their money by first checking whether there's a high bid from a network and (b) they integrate various vendors to verify the traffic is legitimate.
(Disclosure: I work on ads at Google, speaking only for myself)