I'm going through the same thing. I just got a small application up and running using Backbone.js yesterday, but it still feels like a more-complex application could get messy.
I did have one question regarding redundancy - if I take advantage of something like validation with Backbone, I still need to do the same validation on the server-side for any data that I POST or PUT back to the server since I can't trust what comes from the client. Is the only gain from doing the client-side validation in Backbone the faster feedback to the user without requiring a round-trip to the server?
I'm currently trying to decide between Backbone.js, Angular.js, Ember.js or sticking closer to what I know with a jquery pjax solution.