Rendering is slow, buggy, and results in choppy scrolling. The annotations feels out of place UI-wise, but often also content-wise.
Unfortunately, http://fermatslibrary.com/ is down at the moment so I can't comment with an informed opinion.
But there are two killer features that most annotation software fail to supply:
1. print-ability. Physical copies are important.
2. Accurate rendering (related to 1). There's a pretty high barrier to overcome before someone else's inline notes are more valuable than knowing that I'm looking at exactly what the authors intended for me to look at.
That said, the scientific PDF annotation space is ripe for disruption. IDK how meaningful that market is (directly or indirectly), but someone could definitely step in and own it with something that's even an iota better than current options.
I don't know what kind of annotation software you usually use, but I have yet to annotate a PDF and lose the ability of 1 and/or 2.
As for crowd-annotation services, I can only think of genius and medium. Genius does not preserve original format, but that is obviously unreasonable to expect (they're songs!). Medium is its own format, so rendering is accurate by definition.
I've mostly dabbled with preview, Adobe reader, Zotero and Mendeley. You can in all do whatever advance shit you'd like in 3rd party software and copy-paste.
Reading academic papers requires focus and concentration. While I like the idea of comments on papers, I found that site too full of petty distractions to serve that purpose.