You don't HAVE to build from source:

   brew install node

And I think there is a deb package as well. But, I think I remember someone on the core team recommending to just build from source for now. I have not been doing that and it has been fine though.