Ask HN: How to do a SaaS with client side encryption, yet a Web API?

I want to make an open source "Basket In" software (the concept from GTD) so I can quickly capture offline text, voice and pics on the go, then get them synced on all my devices to sort them out the next morning.

However, I don't want the server to know what I, or any user really, is capturing, so I will encrypt it on the client side.

The problem is, I also want to provide a Web API, so that people can write their own client for it.

The server, however, can only send encrypted data to new clients. How do I solve this problem? Do I just share the encryption key with each client, including 3rd party site? What if a user suddenly distrust a client? Do I need to encrypt again the whole content with a new key?