I absolutely love Backbone, but the biggest issue for me has been dealing with complex logic in the templates. Most of the time now my render() method just builds the darn elements right in there.
And when performance is a concern, you can't beat string interpolation within CoffeeScript heredoc strings.