It's also a strong guarantee that if the package is built any different than the upstream default, you know exactly what changes are being made, since they happen on your machine.
1) I don't want bandwidth costs
2) It's much harder to get contribution for a binary system (I expect). If you're installing foo and it doesn't work for some reason you probably won't want to install Xcode and Homebrew-dev in order to try and fix it. When you're installing from source you're already a `brew edit foo && git push` away from helping fix it.
It's also worth noting that most packages don't have dependencies in Homebrew. This is because we don't duplicate what is already there and OS X comes with quite a lot of the common low level libraries. So compiling foo takes significantly less time than it does with Macports.
And of course now with the switch to Snow Leopard we've got the 64-bit issue as well.
Yes, it does take a while, but these days it doesn't take that long anymore.
OS X is unique among modern unixes in that absolutely nobody distributes app bundles that have external dependencies on non-system libraries (except for a few that are too freetarded to bundle ffmpeg).
4 different architectures (PPC/32b, PPC/64, Intel/32 and Intel/64) are probably hell on a binary-only package manager. Source-only is simpler and lighter on the architecture side, though it's much heavier on the client one.
Fink does binary packages though, I think.
fwiw, i use fink on mac os and it supports binary packages. many of the ports in its stable branch are available as binary packages.