The problem comes when you start wanting to use things that aren't on Macports, and are only on homebrew (or maybe fink). But homebrew won't work with macports! So either you have to install one package or the other manually, or you have to install both macports and homebrew, which means that not only are you dealing with an unsupported setup, but you're also dealing with using two separate package managers for the same operating system.
Oh, and don't forget that anything installed via the App Store is completely independent of all of the above.
As a Linux user, I find this concept mind-boggling. I don't have time to keep track of a thousand different package managers. I just want one that handles everything.