The architecture is very clean. And you get a full third-party API "for free" instead of having to bolt it on afterwards (and maybe missing some stuff that can only be done in the web client).
Not only that, by the statelessness of REST you get really good scalability: Each request is independent of each other request, so you should be able to just add frontend- and backend-servers as you see fit without having to worry about global state.
Still, after having done just that (build a web application that is nothing but a JS frontend to a REST API - quite like the new twitter) with http://tempalias.com (source is available), I can tell you that it's not all rainbows and unicorns to misuse that saying once more.
One thing is Sammy, the client-side framework I used. It's quite easy to get lost in one big heap of application that's very hard to maintain. That is totally my mistake though and with a bit of additional experience and some help of require.js or something, I could fix that.
The other thing is UI. People are still used to the browsers page-loading-paradigm. In that case, the browsers themselves provide status update during requests. With AJAX, you'll have to do your own loading indicators and every site does it differently and no site as fine-grained as browsers (Connected, Waiting for reply, Loading... etc).
While the granularity of notifications can be fixed (best is to leave it as a single notification, but respond really quickly), you still don't solve the problem that it looks different on every site. This is a general problem of pure-AJAX sites.
Newtwitter suffers from this. Lately it's quite slow for me and I'm constantly wondering whether some JS has crashed or whether it is just slow.
And finally, sometimes, you have to do really convoluted solutions for problems which could ever-so-easily be solved by just being able to send a few dynamically generated bytes.
Of course, you could cave in and chose the impure solution, but I didn't want to which caused me some headache in my case, like not having access to the hostname of the <script src>'d bookmarklet runner script (https://github.com/pilif/tempalias/blob/master/public/bookma...) which I now have to pass in on bookmarklet invocation.
To be honest though, over time, we'll learn to cope with such issues (and browsers might get a bit better in notifying users about AJAX connections going on) and at that point the advantages could really get into play.
After tempalias which was a personal fun-project, we decided to reuse this rest-only architecture for a project my company is currently working on, fully prepared to deal with the issues outlined here because, really, we believe that the benefits might well outweigh the issues.