Furthermore, it allows for easier data portability- when you can 'collect' data as you go and have API calls that much faster, it's okay to make more expressive UI decisions.
Not to mention, at the end of the day I like having a separation between my frontend and backend. Frontend testing from Rails or Django always feels a little dirty, a little off, and in many cases is even a discouraged practice("If you need to test your view, you should be using a helper" logic). I also like not ever having to make model method choices based on what my frontend desires might be- the concerns can be completely separate.
And lastly, I frankly can do a lot more interesting things much faster in Ember than I can in Rails from a UI standpoint. Ember turned my favorite part of software development, UI design, into the joy it used to be when I was just using pad and paper.
In a client side application,if the app is slow,the users wont be happy about it.
If it takes for ever to load,the users wont be happy about it.
On the server there is always caching so that requests never hit the app itself.
On the client,if the js code is slow ,for whatever reason,you cant hide it to the user.
What you say might be true on the server,these "features" definetly have to be balanced with performances on the client.That's why sometimes it make sense to use React over AngularJS,for instance,because Angular is known to be slow.