Making Google’s CalDAV and CardDAV APIs available for everyone | Hacker News Reader

Making Google’s CalDAV and CardDAV APIs available for everyone

googledevelopers.blogspot.com

253 pointsZikes81 comments

While this is great and I sincerely congratulate Google for it, I still find the wording slightly disturbing. They say they were going to close down the API because only "a few large developers" were using it. Then that since they've heard from developers about a lot of "use cases" they've decided to change their mind.

For me, openness is a relatively unconditional principle. You do it because it is a virtue in and of itself. To hear that we need to spell out specific use cases to justify something being open seems to indicate that Google slightly misses the point. Google shouldn't have to like our reasons for wanting things open. Google should have to justify why they are closed. Indeed, it's in the most unfriendly situations when the openness matters (ie: we want to get our data out of Google and move away from Google services).

So while I'm happy to hear this, and I know I'm picking nits here, it still doesn't quite feel like the "old" Google that really made a principle out of these things.

> While this is great and I sincerely congratulate Google for it, I still find the wording slightly disturbing. They say they were going to close down the API because only "a few large developers" were using it.

Incorrect. They said they were going to make it a whitelisted, by-request, partner-only API for that reason, rather than that they were going to close it down. There is a fairly significant difference.

> For me, openness is a relatively unconditional principle. You do it because it is a virtue in and of itself.

In the real world, few things are unconditional, and while Google has certainly claimed openness as one of its values, Google has never claimed that using open standards is an unconditional principal that is never weighed against other concerns.

I'm thinking they've learnt from mistakes in their past where they've put in a concerted amount of effort to open source something only then to find out nobody wants to use it. Open sourcing it to their current standard requires code cleanup, documentation etc.

Google should have to justify why they are closed.

This seems like a rather entitled position to take. By what reasoning is Google, a private, for-profit entity, obligated to make anything of this sort available to you?

I congratulate the team involved in this. I don't think this signals any change of course for Google. Perhaps corporate will notice this and tell them to shut it off again, because blablabla google+ blablabla social blablabla. Whatever.

This is pretty outstanding on Google's part. Thank you to whoever made this happen.

If you hear people saying that Google closed CalDAV (an open standard for calendar info), please point them to the blog post at http://googledevelopers.blogspot.com/2013/06/making-googles-... to let them know CalDAV is remaining open and that Google is opening up CardDAV access (used for contact info) to everyone too. I'm glad that Google is doing this.

This is so awesome to see. I really want to thank the people involved in this decision, do you know who those might be Matt?

I'm sure Googlers will point them to this thread, so feel free to thank them here :)

There were a bunch of people, including regular Googlers from around the company who asked about this.

Personally, I'm most grateful to the core group on the team that listened to both external and internal feedback and then went back to see how they could make things better. I understand and respect if those folks want their privacy, but I thought it was really great to see them not only change their mind, but ask about how to push more on openness in this area. At this point, Google supports open standards for calendar, contacts, and email. It's great to see CardDAV open up to everyone.

Matt, do you have any insight into whether or not there's even any internal discussion within Google about revisiting the XMPP issue regarding Hangouts, and/or publish a spec for their new protocol?

Of course I totally understand if you either don't know, or aren't allowed to comment. Just curious if it's something that's even a point of discussion or if everybody considers the issue closed, done, over, fini...

It's safe to assume that for anything that's even mildly controversial externally, there's a ton of internal discussion as well. Usually the internal discussion is even more passionate than the external discussion.

I'm an advocate for the idea that Google wants to compete on a level playing field, which leads directly to the fact that I believe Google should also support open standards and interoperability. That means that any time we aren't doing that, we need to have a really, really good reason for it. That said, after a ton of discussion about XMPP internally, I understand the team's decision and their reasons.

Thanks for sharing. It's good to know that there is (probably) at least some discussion about this stuff. If anything, maybe you could give them a little nudge in the direction of releasing a detailed spec (and a reference implementation) of the new Hangout protocol? :-)

  | Google is opening up CardDAV

CardDAV was accessible, but it was not documented. Some Googling could get you some archived mailing list threads where people posted the 'secret' URLs[1]. Even then, I couldn't get responses to all of the requests defined in the RFC. I remember getting a listing of supported request types from the server, but not all of them worked. Also "All Contacts" was not accessible. "My Contacts" was the only address book accessible to CardDAV.

For reference, I've put the stuff I was playing around with on Github: https://github.com/bsandrow/scratchpad-google-carddav

[1] I say secret because those mailing list posts were the only references I could find to them, so I don't know how they were obtained.

That's a great decision on their part. However I really hope they reconsider the removal of ActiveSync support aka Google Sync. Totally anecdotal, but my wife is not a fan of the official Gmail iOS client which is their current solution for push email, and IMAP works OK with the stock Mail app for her except for the lack of push. I'd be OK with paying for Google Apps account to get ActiveSync back, except that you can't do that with a Gmail address.

I thought the reason for the activesync/exchange shutoff is that it had to be licensed. aka that google paid microsoft for being able to use it? If so maybe it isn't worth it due to a licensing increase or just not used by many people (for some value, google is crazy about numbers guiding decisions).

Gmail's IMAP supports the IMAP push extensions. I don't know if Apple's Mail app supports them or uses them, but just because it's IMAP doesn't mean it isn't push.

Thanks, I didn't even know there was such as thing as P-IMAP. So I guess it's up to Apple to add support for this to the Mail app huh?

I think the iPhone does support it.

OS X's mail client supports IMAP push as well.

I'm glad Google made CardDAV available and decided not to restrict CalDAV, at least for now. However, I wonder how long it will continue to support these standards, especially for users of the free services. Supporting these standards might not be directly in Google's best interest, since the more thoroughly Google supports task-specific protocols such as CardDAV, CalDAV, and XMPP, the fewer opportunities it has to present ads. So it may still be good for users to think about moving to services where the user is actually the customer.

In all cases the user is the customer. Some customers pay with real currency (e.g. those who have enterprise accounts), others pay by loaning a few thousand pixel-seconds to advertisers.

It's very much in Google's interest to encourage task-specific protocols, because they can be attached to subscriptions, and subscriptions are denominated in real currency. If Jane Smith knows that GAE has good support for syncing her calendar to her iPhone, she's more likely to use it as her company's email/calendar/documents provider.

Could you be more specific? AFAIK there are no ads in Google Talk/Hangouts, Calendar and Contacts. Google is known for making the vast majority of its money from ads, but definitely not for putting ads in all of their products.

Even if they did show add in these products, how does that relate to supporting standards as alternative means of accessing your data? Like in Gmail, which shows ads in the web interface but also supports POP and IMAP.

While this is nice, it would be great to see native CalDAV/CardDAV support in Android without having to use non-free third party adapters that are not even feature complete [1] [2].

[1] https://play.google.com/store/apps/details?id=org.dmfs.calda...

[2] https://play.google.com/store/apps/details?id=org.dmfs.cardd...

I'm not quite sure why I'm getting heavily downvoted. Does anyone else find it odd that Google is supporting CalDAV/CardDAV on their servers but not on their operating system?

Or am I being downvoted for being critical of Google?

I think it's a pretty fair criticism.

Your comment is of topical interest, but it doesn't seem to be terribly relevant to the announcement at hand. I suspect that's where the downvotes came from, though I've not personally voted either way.

What is the use case?

For service provided by Google, use the supplied sync adapter. Yes, it is not CalDAV/CardDAV, but a whole google account package.

For third-party services, use adapters supplied by these third parties. The third party adapters have the ability to be first class citizens in Android, but to make them so is up to their authors.

The use case is exactly what you have described: third party services. If one wants to run a Calendar service, open standards allow them to use a published standard protocol (CalDAV in this case) that clients can use regardless of platform. Most desktop calendar software (and even iOS AFAIK) support CalDAV and it seems that Google has cleverly omitted CalDAV support from their mobile OS, locking Android users into using Google Calendar with no other option.

That being said, there's nothing stopping someone from writing a proper CalDAV/CardDAV adapter and releasing it on the Play Store. But with all of the claims of Google supporting open standards I was highly surprised to see that Google neglected to implement CalDAV/CarDAV in Android.

Seems like a crazy high bar to hold for "supporting open standards". Google made the APIs available and allow you to provide your own client on their platform with minimal to no interference. They just don't do one auxiliary part that would amount to not much more than making competing with Google Calendar slightly easier. Competition doesn't mesh with the kumbaya, sunshine and happiness tone of open source, but it is fairly strident stance to say companies have to go out of their way to hurt their own products to be open.

I do not think that supporting a standard protocol for calendars and contacts that all other OSes use is a "crazy high bar". I would expect that a mobile operating system whose primary purpose is greatly dependent on contacts and calendar to support the same standard that other OSes use.

In that case, I guess that we can praise Apple for meeting the "crazy high bar" for "supporting open standards" since Apple decided to include support for the most widely used open protocol for calendar and contacts in their mobile OS.

I don't keep up much with the Microsoft space, but aren't they supporting Cal/CardDAV now too on their windows 8 phones?

If so it is a bit of a weird thing for Android not supporting those protocols to be considered a 'crazy high bar' of effort. If the other smartphone OS's support them, it is somewhat curious that Google doesn't as well. Even when they support it server side. Strikes me a bit too much like a Hotel California situation.

Microsoft has CalDav/CardDav support coming for WP8 in the next update [1]. iOS has had CalDAV/CardDAV support to boot (AFAIK).

I'm not sure why posters ITT are acting as if I am making demands to Google. In my original post I simply stated that "it would be nice" to see native CalDAV/CardDAV support in Android. But apparently making suggestions for a better product by providing the same support for open standards that other OSes provide is "a childish perspective" according to a Google employee [2]. I guess we need to keep in mind that a client and a server are barely even related [2]. TIL.

[1] http://www.engadget.com/2013/05/14/windows-phone-8-summer-up...

[2] https://news.ycombinator.com/item?id=5827461

Neither Apple nor Microsoft allow you to plug-in your own sync adapter. What if you do not want to use CalDAV/CardDAV, but your own protocol?

For example, Samsung, Asus and Sony do exactly that. They provide their own sync adapters. Maybe that's more important to them (and their customers) than generic CalDAV/CardDAV.

Cool didn't know it was next update for windows 8, but still a good sign.

I do find the dichotomy of supporting "open" protocols to their service, but not from their operating system a bit odd. Though in that respect Apple isn't much different regarding imessage/facetime/etc...

I guess I would rather have my device/os support open standards over the service I might use them upon.

That's because one is a service, and one is an open operating system. If a service doesn't support a protocol, you can't use that protocol with it, so there is demand that the service support the open protocols that most clients have implemented. If an open OS doesn't have built-in support for a protocol, you can choose to add it. It's apples and oranges. It's like saying your laptop doesn't support a particular printer because you have to install a driver to print.

For an OS, it makes more sense to err on the side of supporting fewer protocols by default and allowing users to clients to add the ones they need than bloating it up with all possible protocols you ever might want to connect to on any server, anywhere.

I think the key point is that while the Contacts and Calendar applications are typically pre-installed on an Android device, they are still applications and not a part of the operating system itself. Any third-party developer could create a competing application that would have equal status on a device, while supporting CalDAV and CardDAV as well.

Trying to build CalDAV and WebDAV into the Android OS wouldn't work, because they would be dependent upon those applications, which would then make them essential OS applications.

Contacs ("People") and Calendar are part of AOSP, althrough most vendors do not ship them.

It does not matter, as neither AOSP nor vendor-specific Contacts or Calendar applications contain any protocol code. These frontends use contact and calendar specific content providers, which in turn provide plugin based API for sync adapters. Google's own sync adapter is such plugin, as well as Facebook's or even Microsoft's (for Skype contacts).

It is true, that there are no CalDAV/CardDAV sync adapters on the devices. However, because not everyone is interested in one, it is perfectly fine to install them from Play Store for those interested.

Perhaps you could refer to a "standard" CalDav client interface included with Windows? Or .Net? I can't think of any OS that provides a standard library for this... Maybe Linux has a few, but not sure that it's in the box... same for OSX.

OSX: iCal Linux: Evolution/Sunbird/Lightning (Thunderbird plugin) I'm not sure if Outlook is still shipped with Windows but IIRC I remember Outlook supporting CalDav in the past.

I do realize that all of these run in userspace and technically are not a part of the Operating System but they are provided with the distribution/purchase.

Those are CalDav clients... they aren't using a library exposed by the OS for custom apps... they each have their own CalDav client library.

Because you're acting like it's Google's responsibility to provide a client in addition to a server. This is not the case. In fact, the two are barely related.

Or perhaps because it's kinda silly to say, "Oh, you gave me one thing? Now give me another thing!" It's just . . . a childish perspective? I'm really not sure how to describe this.

I'm not acting as if it is Google's responsibility to support any protocols at all. It's just curious that Google would implement a protocol on their servers but not on their operating system. Am I demanding anything? No. I am blaming anyone? Not at all.

The personal attacks based on your perception of my comment are frankly unprofessional and completely unnecessary.

I didn't make any personal attack against you. I answered your question about why you were getting downvoted by describing what your comment sounded like to me. If you didn't want to know what people thought about what you said, you shouldn't have asked.

Not the OP; but you do say "It's just . . . a childish perspective" - slightly hard to not construe that as a personal attack.

But I suppose you have a different standard for that sort of thing.

Is it not in Google's motivation to make Android the best phone platform it can be? The lack of native CalDav sync irritates me (the app/plugin listed works fairly poorly, from my experience), as well their horrid excuse of what they call "Exchange" support on Android. If you aren't using GMail, they make it very clear you are a second class citizen on Android. Part of the reason I left it after using it for a month.

user: jamesaguilar about: Staff software engineer at Google.

Apparently not.

It's one thing to allow people access to their Calendar and contact information in Google's systems via an open protocol.

It's another to provide an implementation of that protocol on a different system. Why should they? They use their own protocol for this task. They don't provide an XMPP client either. Let others fill that gap, though I understand your frustration that current candidates are insufficient.

The developer says it will be "open sourced" once it reaches 1.0

The following is from the FAQ[0]

    Why don't you make it open source right now?

    Because it's not ready yet. Some parts of the source code are quite messy, it would be too embarrassing to publish it ;-)

Silly.

[0]: dmfs.org/wiki/index.php?title=CalDAV-Sync_FAQ

Good decision. Can Google also revert their decision to drop XMPP federation with their Hangouts now?

This has been beaten to death already. Hangouts supports way more functionality than XMPP calls for. In order to create a cohesive product, you can't have half your features not work with 3rd party clients that people might use. I think a better plea would be for Google to release Hangouts as a protocol.

> Hangouts supports way more functionality than XMPP calls for.

While federating with XMPP they can expose only a subset.

> I think a better plea would be for Google to release Hangouts as a protocol.

I'm only for it. Which doesn't preclude the option above. Additionally, Google can extend XMPP if it lacks what they need.

I would love to see classic Google Talk return, with status settings/messages, the ability to see if contacts are online, etc., however I'm not so sure extending XMPP is the way to go.

It's my understanding that XMPP is an exceptionally verbose protocol, built on XML. It's great for interoperability, but there are better formats available now, including JSON and msgpack, which would be much more ideal for mobile IM.

XML can be compressed: http://xmpp.org/extensions/xep-0138.html

JSON isn't less verbose by the way. It also lacks namespaces, so I doubt it can help for a complex and extensible protocol.

True, there are loads of libraries to support XMPP on various platforms though.

I wonder how hard it would be to rework XMPP into a JSON encoding? Offhand, it seems like the fundamental vocabulary could remain the same, with just the syntactical details changing over from XML to JSON.

Edit: Interestingly enough, there is a (apparently satirical) XEP for this:

http://xmpp.org/extensions/xep-0295.html

How are you going to translate XML namespaces which are heavily used in XMPP into JSON? The satirical example shows that it's not going to look good ;)

It is agreeable. I am waiting for them to first merge Google Voice to Hangouts. XMPP also needs to up its game and support these extra functionalities.

I think a better plea would be for Google to release Hangouts as a protocol.

I really hope they do this. And, truth be told, I'll be a little bit (but not totally) surprised if they don't. Google have, despite some recent criticism, been fairly good about supporting open standards and protocols in the past.

  > In order to create a cohesive product, you can't have
  > half your features not work with 3rd party clients that
  > people might use.

Why not? Having clients with a range of functionality is how the entire open-standards world works. Somehow, the world wide web continues to work just fine even though it's possible for people to use any browser they want.

If a user has a third-party client, and it doesn't support some fancy feature that the first-party client supports, that doesn't seem to be a big problem. I don't think a significant number of people would care if using video conferences required the official client.

The web browser analogy is a poor one. There are countless standards in place for the reproduction of content on the web, and these days most popular browsers are quite good at supporting those standards.

When it comes to a commercial product like Google Hangouts, allowing users to use unsupported third-party clients is a big risk. People will conflate the client and the service; problems with the client will damage the reputation of the service, and so on.

  > There are countless standards in place for the
  > reproduction of content on the web, and these days most
  > popular browsers are quite good at supporting those
  > standards.

In many areas of the world, IE is the most popular browser by a huge margin. Even in America, IE still has a 40% market share[1]. That's a huge number of users which will not be able to use modern web standards.

[1] http://clicky.com/marketshare/us/web-browsers/

  > When it comes to a commercial product like Google
  > Hangouts, allowing users to use unsupported third-party
  > clients is a big risk. People will conflate the client
  > and the service; problems with the client will damage
  > the reputation of the service, and so on.

This position flies in the face of both experience and common sense.

Any serious commercial site is will support ancient, crufty, poorly-functioning browsers. Hell, if I want to order something from Amazon in w3m or lynx, I bet it would work. A customer with a terrible browser is still a customer, and maybe they are too busy managing their investment portfolio to be bothered upgrading their old Windows 95 desktop.

And nobody is going to sign up for Hangouts, miss the official client, and accidentally download Pidgin instead. If a person signs in with a third-party client, they know exactly what they're doing. They're not going to go post on the support forums saying "help, I can't find the videoconference button".

Requiring all users to install some proprietary application out of a desire for a "coherent experience" or to "preserve the service's reputation" is user-hostile behavior.

E-mail analogy is better. Imagine a service, which only allows using one e-mail client and doesn't allow sending e-mails to anyone outside itself. I wouldn't want to use such service.

Interestingly, such a system (e-mail like system which only supports first-party clients and can only send to and receive from other users of the same service) is a common feature of web discussion fora and social networking sites.

You might not want it, but such services haven't exactly failed in the market.

Your example isn't good, since forum messaging has limited scope, while e-mail is generic. Instant messaging can have limited scope in other scenarios (let's say you chatting with some support representative for some service). It can be generic as well. Here we are discussing generic instant messaging. Therefore it's proper to compare it to generic e-mail. When it's artificially limited by the service walls, it's unacceptable.

such services haven't exactly failed in the market.

Indeed, services like AIM, Skype, Whatsapp and etc. have many users, and are presented as generic instant messaging services, yet they don't allow their users to communicate with others. It's somewhat surprising to me, that people are ready to accept that fact. For some reason they'd see cutting their e-mail off from other servers as unacceptable, but the fact that they are confined within the network of proprietary IM clients doesn't bother them, even though because of that they need to use many clients or protocols to communicate with different networks having an account on each.

> Froum messaging has limited scope. E-mail is generic.

Making a system closed by boundaries automatically gives it limited scope, but forum messaging isn't any more limited in scope than results from those closed boundaries. Ditto with social network-based closed mail-like messaging.

See above. Social network messaging is presented as generic. So I won't compare it with forum messaging.

At this point, I'm not even sure what your point is, since you said my examples were bad, and now seem to be saying that your initial criticism doesn't apply to one of my two examples, and that you wouldn't compare the two examples I gave with each other.

My point is, that there are scenarios when closed messaging networks are acceptable, and there are scenarios when they aren't, but despite of that, people tolerate them in case of IM, which allows various messaging services to get away with it. I said that I really don't understand why that happens, since in case of e-mail they don't tolerate it.

Hangouts, FB messaging and etc. are generic cases, not the ones which should be acceptable as confined environments (like your example of the forum messaging).

No-one ever cheered for "embrace, extend, extinguish" when Microsoft did it, and no-one will do it for Google either.

Does this mean there will be a way to sync Google contacts on iOS again?

That seems to be the implication of the new CardDAV API.

You could already do this in iOS by going into Settings -> Mail,Contacts, Calendars -> Add Account -> Other -> Add CardDAV and entering your Google account user/pass.

This makes no difference whatsoever.

The blog post says that both APIs will be getting OAuth 2.0, though. If iOS doesn't support that protocol for CardDAV, doesn't that mean that this may actually break it?

I wouldn't be surprised if they require OAuth2, but I'm guessing they'll grandfather in clients with lots of users (like Apple), until they can switch over.

I've suspected for a while that their issue with CalDAV was security. Currently you need to store a plaintext copy of the user's password to do caldav. The only copy of my google password in my iOS keychain is for caldav. Since Google Authenticator stores its key in the same keychain, both auth factors are there.

I'd kinda wished they'd propose fixes for caldav, rather than just drop it. Now it sounds like they're doing that.

I know that Google Authenticator supports application-specific passwords for applications that don't support OAuth 2.0, but I'm not familiar with Google's use of application-specific passwords outside of the context of Google Authenticator.

Perhaps that could be enabled for all users regardless of 2FA activation, and be made available for use with these and other applicable APIs?

Actually, application-specific passwords have nothing to do with Google Authenticator. Authenticator is only for the two-factor code. Application-specific passwords are just individual passwords you can set up for anything that doesn't support the two-factor authenticator login.

Just so you don't have to share the same password with multiple products.

Good point. I wish the "application-specific" passwords could be tied to a particular service. (I think they just act as a global password.) But I should switch my caldav config over to one anyway.

This is good news, I guess, but I'll play the pessimist and ask: "How long will this commitment hold?

Good move. Now please clarify what will happen with XMPP in Hangouts!

Not merely a reinstatement: OAuth 2.0 as well.

Reply on news.ycombinator.com