FWIW, I'm not an expert on Ember.js or Backbone; just someone trying his hand at hacking the Ember.js code to work more effectively with collections (since some of the use cases I have would be impossible using the current methods of bindings and collections).
How large is large enough to cause problems?
There are definite workarounds such as setting up paging--but the simple approach of using Ember.js bindings to bind a collection to a template will fall to pieces as that collection grows.
About 10 months ago, Strobe was trumpeting the "Ember" model for views (auto-updating templates, heavy nesting, bindings everywhere) and a lot of SproutCore developers began to use it.
The bloom is definitely off of that rose. At the latest user group meeting three weeks ago, many developers spoke up about having to remove all of their template view code because the performance was absolutely terrible.
At this point, "Ember-style" template views have been relegated to a separate opt-in library, are only being recommend for lightweight read-only data, and the recommendation for developers to use them is being removed from SproutCore's documentation.