New APIs to power customer engagement in Direct Messages | Hacker News Reader

New APIs to power customer engagement in Direct Messages

blog.twitter.com

112 pointstechnofide70 comments

A big history lesson and a big WARNING for anyone who is thinking of building anything with Twitter's APIs. Once this company, long ago, encouraged developers to use their APIs to build products and utilities.

Developers heeded the call and flocked to the company's APIs. They built new clients and innovative solutions to make Twitter truly useful. Twitter grew fast with the developer's help and the developer's themselves did well. Many raised millions of dollars in VC funding to make vibrant companies around the Twitter ecosystem. Things were going well.

Then Twitter became greedy. Twitter decided to destroy the very companies that grew Twitter's user base. They changed their TOS monthly outlawing various product categories. They banned clients. They put restrictive quotas on their APIs to starve out companies so they could reap their revenue for themselves. Hundreds of products fell. Millions of developer hours were lost. The Twitter ecosystem shrank and so did Twitter's user growth.

So if you're thinking about building anything with this company then be warned. If anything you've built is remotely popular, then it is only a matter of time before they come to pillage whatever you've built.

Many things that make Twitter special were invented by the developers you mention:

http://furbo.org/2011/03/11/twitterrific-firsts/

> First use of “tweet” to describe an update

> First use of a bird icon.

> First to support replies and conversations (in collaboration with Twitter engineering.)

I think that's why this discussion thread is so negative, Twitter benefited immensely from the work of third party developers and then told them to go pound sand.

This is why I coined the phrase "Those who don't know history are doomed to repeat it" j/k

So having a big release and advertising these 3rd party tools could be completely motivated by a need for someone else's innovation? Or are there businesses left standing who are innovating and successful with their services? They've worked great for me in the past, but I've never done anything beyond using them for OAuth or showing a hashtag list of tweets.

As much as I agree with you, I wonder if the changes in leadership at the company since then might mean a different direction. Reading their blog posts, it seems like a "reboot" of sorts.

I am in the camp that still thinks Twitter is an important tool: both as a communication medium, as well as a competitor to Facebook/Snapchat/etc. Whether or not this vision for the API platform plays out as history repeating itself, or as a way for Twitter to continue to grow - remains to be seen. But I definitely root for them to try and try again when they fail.

To OPs point, definitely tread lightly at first, and don't bet your whole company on Twitter's APIs. I'm excited to see where this goes.

This! And I should know. My Twitter to Facebook client. http://2fb.me was crippled by Twitter for trying to distribute tweets. The main purpose of Twitter and a Twitter api is distribution or so I thought. Facebook on the other hand has been a great friend. And the Facebook page gets a few new followers everyday even though the app has been reporpused. Facebook.com/toTwitter

I agree with your sentiment, but it applies to a lot of popular companies, especially debt financed companies that don't charge their users. When they stick their hand in the VC cookie jar, the necrotic hand on the other side, grabs them and pulls them in.

Twitter also did something that I find particularly interesting: they somehow convinced developers that you needed them to bless your app and provide APIs to build a client that accesses their service, even though those credentials must be distributed to the public with your client. To people like me who are old enough to have dealt with AOL, the idea that developers are willing to buy into this fiction is absolutely insane: if the official Twitter client can do it, then so can your client, and there is nothing Twitter can do to stop you that doesn't also break their client (which they are generally less willing to do than change their API on short notice!).

As far as I can tell the only thing that enforces this is what should amount to an antitrust level of collusion between the mega-companies to restrict access to products (via services such as the Apple App Store) that do things that the other mega-companies dislike, as it is in all of their best interests to maintain this same fiction. The reality is that client-side API keys are totally meaningless and make no sense: in an attempt to believe they make sense some developers then start to believe other crazy things like "if I compile a token into my program other people can't read it as it is compiled" or to insist "there must be some way to encrypt my binary to make sure that other people can't get my API key", but this kind of broken thought only happens due to a broken premise :/.

(That said, it isn't 100% clear to me that if someone released a true Twitter client Apple would remove it, so if someone disagrees with that premise I am all ears: that is simply the only theoretical mechanism I can see that would seem to enforce this, and I feel like I have heard of this happening before, but AFAIK the random apps in the App Store for abusing Tindr are all based on reverse engineering the true API for the service, but it is also possible that Apple cares less about helping Tindr or that Tindr has a different internal mentality and culture surrounding this kind of thing; or it could even be the case that these now have been removed and I haven't noticed yet as I don't really pay much attention to track them, or even that I am wrong and they are somehow more indirect in their implementation.)

Once I considered writing a Twitter client and the only reasonable option was to implement a proxy that would hold the API key... Hopefully I ended up not wasting time building this!

> As far as I can tell the only thing that enforces this is what should amount to an antitrust level of collusion between the mega-companies to restrict access to products (via services such as the Apple App Store) that do things that the other mega-companies dislike

Actually, at least in the US you could be having legal problems... you're circumventing access controls, which may or may not be a "hacking crime". Also, by using the Twitter API you consent to its Terms and Conditions, so they have legal grounds to sue you (for example via trademark law; you most likely will advertise that your app supports Twitter login, but then Twitter can sue you as you advertise Twitter compatibility while violating their T&C).

It's IIRC never been tested in court so it remains an effective FUD, even without the Apple/Google store banhammer.

Where would someone even see these terms of service unless they have already bought into the fiction? The terms of service on usage of the official Twitter client could apply (saying you can't reverse engineer it), but if you are reverse engineering something for straightforward interoperability there is a lot of precedent, including for extremely similar services (such as the AOL Instant Messenger Client); I think they would be hard pressed to do anything more than ban you, just you personally, from Twitter.

It's the US government I'd be afraid of. Other governments even extradite people like Kim Schmitz into the US, who never even set foot into the US, for copyright infringement. Their justification is "he had used US servers".

Who says that this cannot be done with a 3rd party client developer? The precedent is there, circumstances may be seen similar, all it needs is a clueless DA and judge and boom you have a load of problems on your head.

I wonder if a clean UX could be constructed around users signing up for their own API key and providing that to the client.

It's not necessarily that API consumers feel they must be blessed, it's that it's less hassle. And less likely to land them in a courtroom for for violating API terms of service. "Stealing" someone else's API key sounds like a great way to prove intent to do so - not good!

Baked-in client keys still have some non-security uses: The ability to control rate limiting and help trace bugs/performance issues back to their original "well behaved" clients. It also adds some minor friction for badly behaved ones (more so if everyone's obfuscating it in their clients.) EDIT: Some API docs request that you self-report about your client in e.g. your User-Agent string for similar reasons, when no "blessing" is desired.

Server-side keys don't actually magically solve any problems anyways. You're just shifting the problem from "how does Twitter prevent API abuse by unauthorized clients" to "how do I prevent API abuse by unauthorized clients". You may be in a better position to solve that problem post-shift (e.g. "I'm okay with unofficial clients - but authorized user must individually auth against their paid account!) but you might not ("I just wrote my own twitter client - I'm in no better position to prevent API abuse than twitter.")

> That said, it isn't 100% clear to me that if someone released a true Twitter client Apple would remove it

The way I see it working is Twitter's lawyers sue the author of the unauthorized user of their APIs for intentional ToS violations & circumventing protections (where all those wonderful anti-hacking laws come into play) when they ignore the cease and desist - and either nip the problem at the source when they show up to court, or get a default judgement and a court order they can deliver to Apple.

It's a meatspace solution instead of a digital one, but it works.

@replies and hashtags (one of the most successful features of twitter) came from third party apps,

I have heard rumblings about this from a few old salty dogs, but I never heard such a detailed explanation of what actually happened with Twitter + developers.

While this API does seem exciting, I'm way more hesitant to create anything given the company's history.

Who really trusts Twitter at this point for anything developer related?

Not just talking about their API. Twitter never knew what they were doing, and that used to be fine, but the difference now is people finally figured out that Twitter doesn't know what they're doing.

They keep releasing all these new features that are all over the place (AND worse than before). They should really stop, step back, and think about only a few core fundamental problems and work on those instead.

Thanks for the feedback. Can you be specific about the API features that you're referring to that are "all over the place"?

I'm personally really excited about this, and this is probably the most cohesive a strategy I've seen for the API platform. We're very focused on fundamentals right now.

From reading his comment it seems like his primary concern is the issue of trust. He goes on, "Not just talking about their API...". So I think the features he is referring to are user facing features, not API features.

One really obvious one: Polls.

Polls were introduced in October 2015. There is still no public API to create them, view them, or vote in them -- for users using third-party clients, polls are simply invisible.

That's frustrating, I realise. It's probably the most common feature I'm asked about regularly (alongside group Direct Messages). Polls are complicated to add, as there are three aspects (creating, voting, and then seeing the results), and they depend on cards, which are also not currently part of the API. It would be good if we at least added the information that a poll is attached to a Tweet to the Tweet JSON object, even if we can't do all the other things right away. I'd suggest keeping an eye on the roadmap for where we're going, but can't promise those things right now.

Andy - do you have any more information on this:

"A plan to replace the public statuses/filter, statuses/sample, and search/tweets endpoints with a streamlined API that provides increased access when rate limits are reached."

Some kind of pay-as-you-play model? Appreciate any help - building a product to help smaller businesses leverage the power of Twitter and would like to see how, in return, can help Twitter.

Nothing specific to announce at the moment, but yes, that's the direction that we refer to in the blog post - a new developer portal that would enable you to scale via tiered pricing in the future (but that remains free to use for standard access).

do the current rate limits constitute standard access?

Yes, that's correct at the moment.

No i think you guys on the API team did a great job really.

I was just criticizing about the company itself. It's great that you the API team is doing your best to tackle this trust issue but I think this needs to be dealt with on a company level. Otherwise it's just you guys working really hard to see it not pay off.

Ask via a Twitter Poll, and we'll answer via Tweetbot.

As someone who uses Twitter's APIs heavily, this is both encouraging, and slightly terrifying.

We draw down a lot of data from Twitter. Obviously, we always want more data, so we got in contact with GNIP to see what we could afford (which, in itself took a long time). As it turns out its incredibly expensive, and as a very early startup, we couldn't afford any of their plans. We had no option but to fall back to their standard, free APIs and make do.

There are plenty of people who would happily pay good amounts of money for access to more Twitter data - there really are a million uses for it - however Twitter's current prices are far too high for anything other than a VC funded Silicon Valley startup to afford. I hate to think how many potential startups and cool projects have been killed off instantly simply because Twitter's prices are insane. You can get real-time global stock market tick data for a year for less than Twitter charge per month for access to the decahose.

So, I'm glad to see Twitter being more open about their future plans, and really happy to see they're moving towards a more self-service paid API for those than want and can afford it. I just hope they make it affordable and don't kill off too much in the process. The last thing they need is to upset a lot of developers, again.

Twitter's API has always been something they've not leveraged enough. All they had to do was keep it open, find a way of serving adds through 3rd party clients, and I suspect there would have been an explosion of good clients that could have made Twitter much easier to use for people who just can't figure Twitter out. Twitter shouldn't be complicated, but it is, and by trying to hold onto the brand as tightly as they have, they've prevented good developers from making easy to use clients that could have brought in users.

I really want Twitter to succeed. I hope this is the start of a turnaround for Twitter.

In the market for Twitter data that we were in at ScraperWiki, it wasn't the price per Tweet that was too much, but the minimum spend and the difficulty of getting access to the data at all.

https://scraperwiki.com/2014/08/the-story-of-getting-twitter...

That aside, our treatment as potential partners was really bad. There just wasn't a process for new ideas to be brought to Twitter management attention. I wouldn't risk using their API for a business again.

Well said, both Gnip and datasift costs are way too much for the startup companies.

And the funny thing about datasift is you can not sign up for the monthly plan and they charge you annually which starts from $60,000 for their basic plan.

This is a good move, in a series of good moves recently, by the slowly-becoming-irrelevant service.

I want to get excited by this API change, I really do. But I can't help but quash my own shower-thoughts about what I could do with this API, because what's to say they let their 3rd-party devs down again?[0]

[0] - http://www.theverge.com/2012/8/23/3263481/twitter-api-third-...

They're aiming for the young or the forgetful developer, I assume? Twitter shutting down the APIs a few years ago wasn't generally considered the best way to get commitment from application developers.

Maybe I'm misremembering, but wasn't that while Dorsey was gone? Maybe this is a new management direction?

Twitter recently sold Fabric to Google. https://techcrunch.com/2017/01/18/google-twitter-fabric/

I never used Fabric even though it was extremely useful exactly because I knew Twitter will pull something like this.

It's really not any single person's fault. The problem goes really way back when they made the decision to become a media instead of platform. Back then it could have become anything but now it's already been decided what Twitter is, so it's near impossible to change the direction. That's why little things like this is just waste of time and money.

"shutting down the APIs" - you mean, requiring authentication and adding rate-limiting? which meant that we could get on top of the fail whale and control platform stability?

The APIs have been open and usable all the time.

Imposing a token limit on APIs with no recourse to lift it is effectively the same as shutting them down.

Maybe he meant adding arbitrary token count limits out of the blue that forced countless 3rd party apps to close shop once they became successful? Or the fact that new features like polls are absent in the public APIs, years after they have been introduced?

I'm sure you're very excited about these new APIs for #brand #engagement, but don't pretend that Twitter has been a great steward of its public API over the last few years.

I'm not pretending - I've been responsible for the public-facing APIs (since 2014). We've launched mute, accessibility text for images, multi image and video upload, etc etc. We haven't added every single feature, but we've done what we have done in a very methodical and careful way that ensures platform stability. Polls are a complicated issue because they depend on cards, which are also not currently part of the API. By publishing a roadmap for the first time, we're providing visibility into what's coming. Thanks for the feedback!

In retrospect, my words were probably a bit harsh, sorry about that Andy. It's never a nice feeling having one's hard work shittalked by strangers on the Internet.

A roadmap is definitely highly appreciated. Something I'd love to hear from Twitter is a definitive statement on 3rd party clients: are they a welcome and supported part of the Twitter platform, or is their existence a historical accident that have no role in the strategy going forward?

(I'm aware that such a statement is probably not in Twitter's interest to make, but one can have dreams, right?)

Fool me once, shame on you. Fool me twice? No thanks.

>These APIs are designed to help businesses use the entire Twitter platform

If the only focus is on "customer experiences", why would I want to use this network as a regular user? I don't want endless brands and marketing.

More features for SMM and similar "almost spam" activities. I miss days when I was able to post tweets by simple curl command and third-party clients for reading Twitter were not banned.

I'm a little confused about this: I see there are better DM apis which I guess is the big push here, but what is interesting to me is the new activity webhook api.

Does that mean that a 3rd-party app will need a server to handle activity notifications to replicate what twitter does on their activity page?

Until they increase their API rate limits, this change is next to useless.

We increased the rate limits for a range of the API endpoints in the fall last year, in fact. The updates we announced today should also offer more and better data to developers as they scale.

You still cannot release a twitter client that doesn't get maxed at 100,000 users. Why bother implementing ideas when it will probably meet with the same problem due to our success.

Well, the platform is not just about client apps, people use Twitter's data and features for all kinds of things. You're welcome to build on the platform if you've got a good idea.

Well, my good idea was a client app, so no, I'm (and I suspect a good group of other) not welcome anymore.

Frankly, it should be a lot more about client apps and new ways to use the information in a personal manner instead of creating an API that looks for all the world like something I can use to automate my support staff on twitter and fire a bunch of people.

Building a useful client is a way to build on the platform. It's giving the users an alternative to use your platform...

Twitter is so afraid of losing their users that it's becoming a self-fulfilling prophecy.

Maybe it would be useful and clarify the landscape if the client and data/reporting API channels were separated.

Two things I really want:

* Tweet count on a URL

* Read a tweet feed without auth

Also - the HN title is super misleading. The actual title is: "New APIs to power the future of customer engagement in Direct Messages".

Fool me once, shame on me. Fool me twice...

...

Can't get fooled again.

It would be a good thing if, with this announcement, Twitter made a commitment for how this would have a support for the future.

Like "this will be available for x months/years, you can make x requests per minute/hour/day, you can buy a plan offering x for $x dollars" and so on.

Yes, that can limit some better offer in the future? Perhaps. But that make things more stable, and people would commit more to twitter.

Can you change your terms in the future? Of course. But for newer customers. The ones that sign up right now will have that promises kept. Good for everybody.

That's a really good piece of feedback that I'll be sure to pass along. In terms of plans and pricing, we're not yet announcing those details, but stay tuned as the new APIs we reference in the two blog posts today mature. We'll continue to offer free access to the APIs as well.

Will the free Search API have lower rate limits than they are currently?

I cannot say definitively, but I don't believe there are plans to lower any limits. Additionally, given that the existing standard Search API has not been significantly enhanced in several years and the replacement is going to be based on the enterprise offerings, I'd expect the quality of the data to be a lot better than what you receive on the search endpoint today.

:) thanks for considering it. Now I'll just wait for my 2 cents :)

I really hope Twitter goes bankrupt and we all switch to an open alternative. That, or become an NGO.

Ah, customer service at Twitter had a good run. Back to the bots!

Yeah, thanks, but no thanks.

Regarding Twitter's history of killing APIs and 3rd-party dev; is there a bulletproof approach to setting up such an API/service with the guarantee that it will not be killed/ruined, short of business closure? A one-sided contract, so-to-speak?

We're deprecating a legacy OpenID connect implementation and rebuilding the entire user onboarding experience. Part of this requires rebuilding the authentication layer. This is the first time I used their APIs. Instead of sticking with the heard, twitter decided to stick to their OAuth1 implementation instead of adopting the standard today - OpenID. What's worse is that they've come with the ingenious OAuth Echo, which requires you to ship your mobile applications credentials embedded in the code, virtually public to anyone who decides to poke around.

Omitting the obvious remarks on Twitter and APIs, this is clearly a good move. Most companies have a twitter handle and the public has already absorbed that fact, so it just seems more natural to offer a bot through twitter DM service than, let's say, telegram. I wonder what took them so long, considering that bots have also been natural to the twitter ecosystem for many years.

This can increase their growth and possibly, if a company bot goes over their future non specified limits, they can cash in. Sorry, could not help myself :)

Bummer that it's all webhooks. They're pretty fragile. It'd be much nicer if Twitter had some kind of streaming API that you could connect to with a sequence number and get any messages that you might have missed, or at least any that they still have.

Basically, Kafka over Websockets would be ideal.

In the early stages where we are now, we're finding the new architecture pretty robust. There's an acknowledgement back on our side with webhooks responses, so we can build features that ensure your apps never miss an activity.

Hey, looks like it will finally be possible to send pictures in DMs from 3rd party clients... can someone please confirm?

I wouldn't hold my breath. It's entirely possible that the new DM APIs will be restricted to approved clients or accounts.

I believe Tweetings for Android has been updated to do so already.

Twitter's swan song?

I'm sure this API will be great news to those people who are harassed with hate speech every time they use Twitter. Since Twitter has an economic incentive to ignore that problem (if they do fix it, then their "user engagement" takes a nosedive) I don't think their future is particularly rosy.

Reply on news.ycombinator.com