I generally use Bitnami's stacks. There's one for LAMP too.
Benefits: Fairly latest versions. Everything in one directory (I usually install them under /opt). No effect on system packages. No need to add PPAs for new versions.
Drawbacks: Can't update an existing installation; have to install a new version of the stack in another location and migrate.