Our experience was that backbone was very helpful in laying out code and providing clean event triggering, but that a lot of it was simply too slow to use. We started out with a View per card, but that was impossibly slow with 100+ cards (and we need to support ~1000+). We ended up with just one view for the Card section, and a view each for the sidebar and options sections. That made it much faster, but also took away many of the benefits of backbone.
We also ran into significant speed problems with model gets and sets. Initializing the data for 200 cards (~10 attributes per card) took hundreds of milliseconds on Chrome and other browsers. We ended up using plain javascript objects for the hottest parts of that code.
Overall to us, backbone was probably worth it, but also cost us many long days of performance debugging, and we're still not 100% happy with it. For our next project we may try something else.