Instead of (or in addition to) considering remoteStorage, one could also go back to "normal" WebDAV, if such a thing exists, and completely forget about the CalDAV/CardDAV servers.
I'd say WebDAV is more mature, it has litmus[1] for testing implementations. remoteStorage has as far as I know only one production instance running at 5apps [2]. The benefits of using the RS protocol are mostly due to the CORS headers (which could be implemented easily for WebDAV) and the use of OAuth/Bearer, for which a PR exists for SabreDAV [3]. One thing missing from WebDAV is the (implicit) mapping of OAuth scopes to ACLs, which should not be too difficult to implement... The discovery, by depending on Webfinger, is also not one of my favorites. I'd prefer something like OAuth authorization server discovery [4].
I mean, I am not opposed to using remoteStorage and love JSON as much as the next guy, but it just doesn't bring (in my opinion) many benefits and loses interop with existing WebDAV clients for no good reason...
[0] https://datatracker.ietf.org/doc/draft-dejong-remotestorage/
[1] http://www.webdav.org/neon/litmus/