Raphael really goes deep on the browser compat side of things. It supports IE6, falls back to VML. As a result, it's quite a bit heavier than SVG.js.
Raphael also seems to be a bit more about abstracting SVG away from you, where SVG.js is more direct manipulation of the SVG document.