Twitter shut down Emojitracker's access
medium.com
medium.com
In the past, Twitter had little formality associated with granting elevated access. It was very much, "Hey, I want to do this cool thing!" "Okay, sure, here you go." As Twitter's platform business has matured to include many businesses building billions of dollars worth of social media monitoring and other tools, we've formalized the process to becoming a Twitter Official Partner (partners.twitter.com). Over the last six months, we've started contacting all of those API key holders with elevated access and asked them to clarify what they're doing. In some cases, we know the business and business owner and reach out personally. In other cases, like this one, the business is listed in our systems as "N/A", so we send the template mail.
In the email, we encourage people who believe their app is within the bounds of acceptable use cases on Twitter to contact us directly, and provide a link to do so. The owner of this app elected to blog publicly about the situation before contacting us, which is unfortunate. We have contacted the owner of this app and hope to resolve this situation, as we do with hundreds of other developers on our platform. We do occasionally provide exceptions for apps that are non-commercial (not-for-profit, no ads, etc.)
Note also that in this instance, the notice is NOT about shutting down this app. It merely inquires about why the developer needs elevated access, something that is typically reserved for our business partners.
None of this is related to continuing to use the Twitter API or our commitment to enabling developers to build on our platform (Fabric, Gnip, our Ads platform, and the Twitter API).
It did not ask him to clarify what he is doing, it did not say anything about bounds of acceptable use cases or what they might be, it did not encourage him to contact Twitter directly, or provide a link to do so. It did _not_ "inquire about why the developer needs elevated access", there is no such inquiry in the email. It _did_ say elevated access was going away.
It said he could use the free API with rate limits, or the commercial Gnip API.
It kind of half-heartedly suggested he could "reply to this email" if he had "questions". From the email, there was no reason to think elevated access might still be available, it pretty clearly said it would _not_ be, so I'm not surprised he didn't have any questions -- the email was quite clear (at saying something pretty different than you are saying).
So when you describe the email that goes out -- it does not seem to describe the one he received, according to him in the post. Are you talking about a different email? That he did not receive, or that you think he should have received in addition? Or do you actually think that email somehow communicates what you describe above? (It really really does not, which is why I think you must be thinking of a different email he did not receive or post, or you were internally misinformed about what the email was going to say).
With the email he actually pasted into the OP it is not surprising that he simply publicly notified his userbase and other interested parties that the service would be going away -- what else do you do when your upstream provider tells you the service is going away? This is all very standard and professional.
But, if you are saying that you do continue providing free elevated access to certain projects that seem worthwhile and meet some 'boudns of acceptable use', then that is nice, cool, I'm glad Twitter is doing this. (Maybe you should have told him that in the email though! And it would be great to actually advertise that fact, and what the bounds of acceptable use are, and how someone can get in touch with you to request access.)
That's not entirely correct. Under option 2 after mentioning it's commercial side, it says:
There are also other solutions that offer varying levels of access as well as historical search.
It's not prominent (likely because they only want it pursued in special cases), and it's easy to miss, but it is there.
> If they want all these special access cases handled through Gnip, whether they are discounted or free, that's their call.
There's nothing in the mail suggesting they "want special access cases handled through Gnip".
In any case, it's all speculation, there was never a dialogue, and the HN title as it current stands is probably inflaming this more than it should be. Their access was not shut down (and the blog post's title and wording does does not say it is shut down, but that they believe they will be shut down). An email was sent, it was poorly worded, and the recipient overreacted (or at least prematurely reacted). The responsible thing to do would have been to contact Twitter or the sender and discuss the needs and goals of the project, and see what they could offer.
Immediately writing a public blog post is not negotiating in good faith.
Yet this seems to have been quite effective, given that the head of Developer Relations at Twitter is replying to this story on HN..
Public shaming on twitter+blog is 50/50. It often gets the attention of an engineer and sometimes even a founder, and when it does the response is often reasonable. Why would you ever try anything else?
I've had a crazy technical issue with AWS that our account representative proved useless for about but a tweet got it fixed. Also see news articles about people suffering the most ridiculous treatment from comcast/timewarner/verizon/at&t until they get in the news and everything is finally sorted out. This is a totally standard thing you should already be familiar with. The problem is that these big companies get too many queries from crazies and people who have no idea what they're doing, they're inundated with stupid support requests. So a legitimate support request will never be noticed by anyone who knows anything. So you need a sort of public vetting process. You need the "shaming" part to express the priority in large companies where business people have all the power.
I'm not sure there's a better way, that's just how it is.
Going public means getting in front of people that care more about customers or has more time to do so. It feels a little like skipping the line though.
I think it boils down to this, if the email said "Hey, we noticed you have special access, and we are trying to clean those up. You should either use the regular API with rate limits or our commercial offerings. Let me know if you have any questions." then I think the appropriate first step would be to contact Twitter and explain your situation. Since the actual letter contains that and just more information about possible options, I don't see why it should be viewed any differently.
> Hey, we noticed you have special access, and we are trying to clean those up. You should either use the regular API with rate limits or our commercial offerings. Your special access will be terminated on Thursday, April 21st. Let me know if you have any questions.
On receiving an e-mail of that nature, I would very likely let my audience know that I was shutting down the service on Thursday, April 21st just as the blog post author did.
I would not assume "Let me know if you have questions" meant "Let me know if you'd like to clarify your situation and request an exemption from this policy". I would assume it meant I could inquire about the details and process of the shutdown, or ask questions about the Gnip service or other options that they discussed in the e-mail.
Immediately going public can backfire, depending on the specifics of your case. Now there are possibly competing interests at play, the public pressure to let them continue, and possibly some anger over at Twitter for not being given the smallest benefit of a doubt (when their project works because of the good grace your company exhibited in the first place) or even asked before playing hardball. If I was the person responsible for making the call at Twitter, I would probably acquiesce, but I would want to tell this person to go get bent.
If they don't wish to communicate that message, the message should end with at the very least something like "We don't want to terminate a popular service in error, so please contact us today to tell us more about it."
Did everyone miss this part of the blog post?
> I’ve replied to Twitter asking for an exception, but I honestly don’t expect to receive one—and even if I were to, it wouldn’t help other developers who I believe deserve to get the same level of opportunity as me.
> In the email, we encourage people who believe their app is within the bounds of acceptable use cases on Twitter to contact us directly, and provide a link to do so. The owner of this app elected to blog publicly about the situation before contacting us, which is unfortunate. We have contacted the owner of this app and hope to resolve this situation, as we do with hundreds of other developers on our platform. We do occasionally provide exceptions for apps that are non-commercial (not-for-profit, no ads, etc.)
If this was the intent, the template email was kind of poorly drafted, because that's not what it says at all.
It says multiple times, without qualification, that they're going to be cut off — "this type of elevation will no longer be available," "your current elevated rate limits will no longer be available," "we will remove this elevation from your account," and "we encourage you to evaluate alternative options." At no point does it say that keeping the elevated access is an option.
So I mean, can you really blame the owner of this app for reading these absolute statements as being absolute and believing their app was cut off, rather than seeing it as some kind of sideways inquiry as to whether they think their use is acceptable?
As many others have pointed out, the e-mail from Twitter in the blog post absolutely does not state this. I add my voice to the chorus of others asking if there is an alternate e-mail, or if you and Twitter truly believe the e-mail in the blog post ever made this statement.
> The owner of this app elected to blog publicly about the situation before contacting us, which is unfortunate.
Again, joining in the chorus of voices, I want to ask why it was unfortunate that the developer made a public statement that his service would be ending? In the correspondence with him it was clearly stated "On Thursday, April 21st we will remove this elevation from your account" (contradicting your claim that you "asked them to clarify what they're doing"). I would argue that the developer has the responsibility to notify their audience that the service is going to be shut down at that time.
Why is a public response anything but appropriate given the content of the e-mail that Twitter sent to him? If the situation you describe here is truthful, then don't push blame onto the external developer when the e-mail didn't come close to communicating what the Twitter team thought it did.
Every 6 months you guys make a new change that kills a bunch of websites who were silly enough to rely on your APIs. Then you give an explanation like the one above: something about needing to grow the platform, and pushing the responsibility for the problem back on the businesses.
When you killed the tweet count a few months ago, the story was exactly the same - something about depreciating it in order to build a more dependable platform, and people shouldn't have relied on it in the first place. It's clear you guys just wanted to take control of your data.
As a social network, I love Twitter. But I would never waste any time building on your platform. At every step you leave a trail of dead businesses in your wake.
If a contract isn't feasible, at least make sure it's in the other side's best interest to keep the situation going. Getting free access when it costs the other side (in managing access, serving requests, handling support, etc, no matter how small) is not something I would feel comfortable basing a business off of.
The difference is that the person is not getting anything that's not already offered to the general public. It's unfair to use the court system to enforce a company's exclusionary political decision that "everyone can access our website EXCEPT YOU", at least barring any actual contractual terms (not generic Terms of Use). The change I propose would not force anyone to accommodate a certain API change or provide any special functionality. It would simply prevent non-neutral access rules from being enforced via legal vectors.
Twitter would still be able to charge for access to the APIs they charge for now. They'd simply be unable to use the court system to compel someone to stop accessing the data that they have no copyright interest in and which they serve up to the world for free. I'm not sure if Twitter has tried to do this yet, but it's the normal step companies take once a consumer develops the ability to evade their IP blocks.
To be honest, the law already can be interpreted this way. The problem is that it often isn't. Companies have been able to convince non-technical judges that concepts like trespass to chattels are applicable any time someone is talking to their server. We need language in the law that will clarify the matter to prevent big companies from squashing small innovators that they find inconvenient or threatening instead of leaving it up to judge roulette.
Of course it's ideal to have a contract with a company that guarantees access to the data stream for a reasonable chunk of time, but the reality is that unless you're already a big shot, platform vendors like Twitter aren't going to give you the time of day for something like that.
If we take away the ability of companies to selectively allow access to data that's available for free to the general public, it gives these guys a fighting chance. The web is a publication platform. It's equally absurd to say "I'm publishing this novel and everyone except kbenson can read it" as it is to say "I'm publishing this data to the general public and everyone except kbenson can access it". You can try to stop someone from accessing your data, but certainly no court is going to consider it reasonable to assist a company in preventing kbenson from reading a widely published manuscript. Data access is substantially the same thing, and should be treated the same way. Keeping non-disruptive entrepreneurs on the same footing as everyone else makes innovation more secure.
> It's understood that a company could stop offering some data that a company is dependent on altogether, but I think it's substantially less likely than a company threatening to terminate access to data that already exists (which is what happened here).
I'm not sure sure. There's regular griping on HN about Google cancelling services, even though they are generally beta services, because people expect them to offered. Some of these people griping are people that built products on Google APIs that were discontinued.
> Of course it's ideal to have a contract with a company that guarantees access to the data stream for a reasonable chunk of time, but the reality is that unless you're already a big shot, platform vendors like Twitter aren't going to give you the time of day for something like that.
A contract? Maybe not. But if they have a clearly defined deprecation policy, that's a start. And if you are paying them, well that's a lot better, since they are incentivized to keep it going because of your (and hopefully others') money.
As for the rest (access to public API), I'm not really interested in arguing it, as we likely agree more than we disagree. :) The only caveat is that direct consumers of the API often don't behave like regular public users, and that may put a strain on the system and degrade performance for everyone, causing real problems and the need to spend real money to fix if you still want to allow unfettered public use.
I believe that sites can implement technical measures to "enforce" a fairly normal request profile. Third-party consumers will have to comply with that because it will be technically impossible not to do so. Beyond that, there is no need for a legal method to address access. If the data source is concerned about said consumers going through a code path that is more intense on the backend, they will accommodate with a lighter API. :)
Doesn't that make some projects and uses, where it needs more than what is regularly allowed, impossible? I think the natural response to that would be "well, we'll just charge them (something/more) and give them special access beyond our normal restrictions", and we're back at square one. Enforcing everyone gets the exact same access is essentially communist theory, not mean in an inflammatory way, but in that it really does enforce some level of mediocrity on everyone (to be fair, as insurance some bad actors makeing things untenable for the rest) in comparison to an approach that allows some companies to pay for for extra access to account for the extra burden which may result in new products that people want.
I'm for making publicly available data available to use as wanted, but I'm not really for enforcing arbitrary limits that prevent whole classes of useful products that people generally might want in the name of being fair, especially when "being fair" might mean contradictory things depending on the side you approach from. The fact that the data is generated by the users, and ownership may or may not reside with users or the aggregator, and current law may not be the best with regard to that just makes it all the more complex. :/
I don't think we should force companies not to have a paid API, that's all well and good. What I want to remove is their ability to stop scrapers from gathering the data from the generic public interface just because they don't like the way the scraper is using the information.
As it pertains to this specific story, I think Twitter was well within their rights and norms to try to move a special cased user to some more traditional offering of their, or to at least make them justify their usage. I also think it's well within Emojitracker's rights and is acceptable to try to make a public stink if they think they are getting a raw deal. The only thing that I think leaves a bad taste in my mouth is that he did so prior to attempting a good faith negotiation with Twitter.
I don't see any encouragement or really any path to recourse in the email. It reads as though the decision is final. Email linked from blog post: https://cdn-images-1.medium.com/max/2000/1*MoLgPNLiQwQ8s-Y1D...
My interpretation of the email is essentially: we used to offer this for free, and we're now offering a commercial alternative. You have three options: use the paid option, use the free API that does not work for your use case, or shut down.
If I'm missing something obvious please let me know.
1. Build a successful product
2. Twitter discontinues dev-friendly policies, product gets shuttered
3. Later, tweet doubts about Twitter's dev "commitment"
4. Tweet gets removed
5. Read yet another horror story about working with Twitter
6. Twitter guys claims that Twitter is "committed"
7. Turns out, previously mentioned horror story tells a different story
8. Rinse, repeat?
As Twitter's platform business has matured to include many businesses building billions of dollars worth of social media monitoring and other tools, we've formalized the process to becoming a Twitter Official Partner (partners.twitter.com).
Simply because you're an even bigger company now that means implicitly that you need to change your processes, perhaps you could explain the reasoning of what was going wrong, previously.
I think a lot of developers would question whether or not you guys even have such a commitment at all. Twitter does NOT have a good history in this regard. If anything, you guys need to bend over backwards, sideways, upside down, and any other way you can think of, to treat developers well and woo them. I mean, somebody needs to go full Ballmer and start screaming "Developers, developers, developers!!!". [1]
Seriously, you guys are so behind the curve in terms of developer perception that "good enough" isn't even close to good enough. You should probably be reaching out to developers, offering to fly them to Twitter HQ, and treating them to steak and lobster, etc. And that's just to get back in the good graces of the development community.
Developers do not compete with Twitter, developers make the Twitter ecosystem larger and stronger and better. Twitter needs to embrace that mindset and go all in on becoming a desirable platform to develop for.
As for myself, our app offers limited Twitter integration, but we're damn sure not going to spend a lot of time and energy on Twitter given the history and what-not.
My advice? Kill Gnip, make it trivially easy to access the firehose, commit to adhering to open standards[2][3][4][5] across the board, and put developers front and center. If you want to charge for firehose access, fine, but kill any notion of some complicated application / approval process. Make it as simple as using the API now plus checking a checkbox and plugging in a credit card number, whatever. It's 2016 for crying out loud, there's no reason I should need to fill out a form and have a sales representative call me for something like that.
[1]: https://www.youtube.com/watch?v=Vhh_GeBPOhs
[3]: http://www.foaf-project.org/
[4]: https://www.w3.org/2008/09/msnws/papers/W3C_FOSN_Position_Pa...
Sure there is. The reason is that they want to guess how much money you'll be willing to pay and have your sales rep charge you that.
Not really. This follows the coloring book pretty well:
Step 1. Inflexable corporate rhetoric and/or positions expressed privately.
Step 2. Other party blogs about a scenario that puts egg on their face.
Step 3. "Nah man, we're flexable! Why didn't you just talk to us first?!"
The rest of your explanation is just specious, as pointed out by other people, and also reflects poorly on your company.
Really? Where in the email does it ask anything of that sort? The only "unfortunate" part that I see is that this person exposed the false rhetoric and lack of flexibility that Twitter has shown time and time again. Apologize and fix it, stop trying to whitewash this.
If what you just said is what it the email meant to communicate then it fails at every level. Please don't try to shift the blame for Twitters poor communication to the recipient because they "elected to blog publicly about the situation".
The "situation" is entirely created by Twitter.
Twitter has a (recent) history of terrible communication with developers and it would not surprise anyone if what seemed to be the case WAS the case. If it is NOT the case, as you say, then you really need to improve your company communication.
It is directly related.
Twitter desperately needs to decide whether you are a platform or a closed business. If you're a platform your job should be encouraging any and all innovation on there, and have clear pricing tiers which do not result in sudden jumps from free to large charges or access cut off and directly reflect the cost to you of serving the API. You should never be in a situation where you're emailing to cut off access (and yes this notice is shutting it down).
Successful products built on twitter should be celebrated, not forced off the platform.
Try the app. Talk to the developer.
Template mail should not exist.
Hi Prashant,
I want to pay $ 100 per month to extend the rate limit of the free API. Is this possible?
Thanks,
wslh
I think it's because Twitter is actually a successful, fairly open ecosystem that attracts all kinds of use.
What seems like ham-handed or draconian decision making is just Twitter misunderstanding its role as infrastructure provider.
What we all implicitly want from Twitter is for it to embody the values of net neutrality. If someone can build something cool on top of Twitter, why not let it happen?
Twitter's investors have pressured the company to monetize, and most of the bad platform decisions seem to be tied to those efforts.
Twitter's folly is assuming that usage patterns are "bad". It should view all usage in a content-neutral manner and provide and support APIs to let the ecosystem grow.
It is the hubris of caring too much about content and content engagement metrics that is driving this folly.
Twitter is a very simple broadcast graph. This is incredibly powerful, and while it's not quite as lucrative to own as Facebook (with users habituated to seeing the same boring stuff from the same boring people every day and clicking on sponsored content), it's far more powerful and is likely to change society more than other social media platforms.
All this is why Twitter should pivot and become a decentralized infrastructure... of course this will never happen. I'm predicting that Twitter will die a slow death as it tries to copy Facebook's revenue generation approach. It is also copying Facebook's real name initiative by caring too much about content attribution and verified accounts (which users don't care about).
Eventually we may see a decentralized broadcast graph, but until then it makes sense to expect Twitter to behave like a cognitively impaired dictator with respect to its ecosystem.
I think most of this is driven by intense cost-cutting measures and intensely misguided focus on revenue metrics in terms of content and engagement. Twitter is not a product, it's an infrastructure.
True, but for a platform business there has to be a correct recognition of the relevant time horizon. Amazon's profitability numbers in its first decade illustrate this.
Twitter makes decisions that people on the internet grumble about and make blog posts about and yes, even post on twitter about, but they still keep using twitter, which is all twitter cares about.
But they need to decide how they want to make their money and grow in parallel. Is it ads? Celebrities accounts? Is it selling API access?
If I'm Twitter CEO I would stick with "developer friendly" approach in order to re-start Twitter growth.
In short, my opinion is that companies needs to be nice with developers and cherish them all the way until the company is very very successful. I guess Twitter executives think that they made it and "developer friendly" stage is behind them.
Twitter and Facebook may be Big Corp today, but that wasn't always the case, and a lot of the initial growth that helped them get into the mainstream was due to hobbyists and independent devs that made cool little apps like EmojiTracker that would catch the attention of early adopters that then became the initial users of the platform.
The sad thing is that while Facebook became the generic one-size-fits-all "mom and dad" network (I can't even remember the last time I even used one of their 'apps' or whatever), Twitter really had the potential to evolve into something beyond that, if only it cultivated better relationships with their devs and encouraged people to build on and experiment with the platform.
Oh well. Lets keep an eye out for the next thing, I suppose!
Because Twitter doesn't "exist to make money" it also "exists to be Twitter." I'd prefer to never again hear that something "exists to make money", which is quite facile.
EDIT: there's a bit of a war between downvoters and upvoters on this comment. Let me break it down for you. The only way to agree that "twitter exists to make money" is by agreeing that the mission statement of every company on Earth should be, "Make money." Not much of a mission statement is it?
I fully agree with you, except on this point. With all the shitstorming, the mis-attribution of quotes, massive and massive amounts of fake accounts etc. going on, the Verified by Twitter program is a very good thing especially as it does not force you to publically post your real name.
However, it could be expanded for greater reach.
If Twitter is a neutral platform, a fake account is not distinguishable from a real one and it doesn't matter. It takes an incredible amount of work to make a fake account seem like a real person.
By creating content leaderboards that do not remove fraudulent network effects, Twitter has created an incentive for fake account creation. Its own ad-sales model benefits from this too, since many of the platform's top line numbers include fraudulent engagement data.
Also, Twitter has not made data/tooling available for third party providers (like SimplyMeasured, Radian6, etc.) to do meaningful fraud analysis, so companies are paying for social media engagement numbers that include bots, sock puppets, etc.
But my larger point is that as a network Twitter should not care about spam and should instead let others determine what content is "good" vs "bad" for their particular use case.
I think these sentences contradict each other? Or maybe I'm misreading something.
Anyway, last December, after the release of The Force Awakens, someone created the account @AdamDriver2015 which was so convincing that even @VeryLonelyLuke quote-tweeted them. They were suspended after a few days, and one Twitter user claims it's because the owner of that account was creeping on other Twitter users under the guise of being the actor.
https://twitter.com/jennanamika/status/684176364438601728
It's very easy to make a fake account seem like a real person.
> By creating content leaderboards that do not remove fraudulent network effects
Do you count "retweets" here? That's a large part of the problem, and it seems hard to change that without fundamentally changing what Twitter is.
> third party providers (like SimplyMeasured, Radian6, etc.) to do meaningful fraud analysis
... hm, are you focusing on the aggregate affect of fraudulent accounts? It seems to me like that's a different problem from individual fraudulent accounts, and that verified accounts are legitimately good at the latter and aren't intended to deal with the former. In particular, all the bot checks are irrelevant for an individual fraudulent account, since there is a real human behind the account, just the wrong human.
The first statement is conditional on Twitter behaving like a neutral platform, which it does not. The second is meant to indicate that there is a cost associated with creating fake accounts, which is a factor in the economics of the system.
> It's very easy to make a fake account seem like a real person.
Why is this a bad thing? If casual Twitter users find the content entertaining, the fake account benefits the ecosystem.
> Do you count "retweets" here?
Yes. No need to ban retweets, just to show stats that indicate which are likely from retweet networks.
> ... hm, are you focusing on the aggregate affect of fraudulent accounts?
Yes because that is what drives the money flows. People create fraudulent accounts of both kinds to generate revenue. Someone somewhere is paying for the campaign that included thousands of fake retweets, etc. I saw a network graph recently that showed that such networks are easily identifiable with simple network analysis.
Now, we could argue that we shouldn't really care about the concerns of celebrities, and that fan/celebrity interactions aren't what Twitter is for. But it's clear that they drive a lot of the traffic there, and Twitter encourages it. The verification system is one tool for making this experience better.
Honestly, I wouldn't be surprised if they had zero fake retweets. Just have a plausible-sounding name (done) and no competing account that's actually by the person in question (done), and reply to some relevant tweets from high-profile accounts, so you show up in the single-tweet view. At that point, some fraction of real humans will find out about the account and retweet you. No analysis will differentiate this from real humans, because everyone involved is a real human.
It seems difficult to argue that @AdamDriver2015 benefitted the ecosystem. Those who found the content entertaining did so because they thought it was genuine. Especially if it's true that the person behind the account was creeping on people in DMs, this seems like a very indefensible claim—but even if not, it's not advantageous to the ecosystem, unless Twitter is actively participating in making people believe the fake content is genuine.
When the program is used to delegitimize public figures who dissent from the left wing hegemony on the site[0], then it has lost all legitimacy. Verifying accounts now implies tacit approval of their message, which is not something Twitter should enter into lightly.
[0]http://www.businessinsider.com/milo-yiannopoulos-nero-unveri...
1. Promoted tweeks - frankly, I find this kind of annoying, but I do understand.
2. Limiting high bandwidth streams to paid/commercial accounts, I'm not sure what they offer for NPOs, but I'd be surprised if they weren't lenient in their pricing structure.
Frankly, I understand it... At this point they aren't even breaking even iirc, so they need to make cuts to some deep cost centers that aren't facing end users.
TBH, I think they should also put methods in place of detecting bot networks, and freezing out those accounts.
I realize that, I'm just not sure the investors have the right long-term view about the value Twitter offers.
Twitter, as it stands, is very useful to people in the media who use it as a free sentiment analysis and polling service during election cycles, and a free firehouse for unqualified reports that they can follow up on. However, Twitter doesn't receive remuneration on any of this value.
I am quite curious to poke your brain a little on this. I agree that the concept of Twitter-as-an-infrastructure could be amazing. I also agree that what they're doing is definitely taking massive steps away from that ideal (Whatever Twitter's initial lofty goals were, I would classify its current niche to be some sort of celebrity-gossip/outrage machine).
I'm having trouble articulating exactly what the correct thing for them to do would be, though? What sort of API access would you envision that they can provide in order to help developers grow the infrastructure in the best way possible?
You mention that Twitter could become a decentralized infrastructure. What exactly do you mean by that, and how would it work?
GNU Social [0] is a good stepping stone. Decentralized Twitter with different hosted instances joined together so everyone on the network can communicate. You're free to take your data and move it to any other instance you want or even host your own.
While I agree with the spirit of the idea, most people don't seem to care about the benefits that self-hosting would provide, and even for the ones that do, well, it's just such a hassle. And for the majority of the world that isn't particularly clued up on how tech works? Yeah forget about it.
Maybe that's where the web will go eventually, and I would see it as an improvement over what we have now. But the current reality is that mainstream adoption is centred around large, monolithic services and personally I am more interested in figuring out what is it that these services provide and how I can improve on that in the way that people already want, rather than trying to convince them why they should care about being in control of their own data.
So when I think about "decentralized infrastructure", I don't think about self-hosted servers, what I'm imagining is closer to, say, google app store. Sure, it's still owned by a single corporation, but anyone can develop almost anything for it that then can be used by anyone else.
Imagine what a hybrid between twitter and an app store could look like. Giving people new tools to integrate social apps around the web in totally new ways. Developing new ways of interacting with each other, rather than every news and blog site having the same facebook like button and the same "twitter feed" box and the same comment section.
That's what I want to see. Not "it's like twitter but YOU control your data". Having ownership of your own data is important to have alongside any product that you develop, but (as far as we can see empirically) you can't make that your sole "killer feature" and expect mass adoption.
Something like twtxt[1], except trivially easy to host, and searchable, and (the hard part) with all of Twitter's users.
Given how much Twitter is struggling already, is this really the time to be alienating more of its community?
Personally, I've had a really nifty side project that I've been working on since last year that I wanted to launch on Twitter's platform, but as of a month ago I've all but given up on actively developing it. All the people that I know on Twitter with whom I wanted to share it have either left already, or are only sticking around because as-of-yet there's nowhere else to go that does what Twitter does.
So I'm curious to hear some perspectives on this: clearly the direction that Twitter has taken is making a lot of devs unhappy, but what would be the right thing to do? What sort of API access would people ideally want to see it provide? What would be the ideal best-case scenario necessary for the devs to want to start coming back and putting their time into building the ecosystem?
Twitter offers several free endpoints for developers, including search, streaming, a raw sample of tweets, and the like. They all have rate limits and the like attached, but they're real, free, access to their data.
For people who want bigger, broader, deeper access, they sell access to that data. That's been the case for the last few years.
In this case, Emojitracker unfortunately falls into the cracks in between, where they need deeper access to do what they want, but it's not a commercial enterprise.
I agree, it would be nice of Twitter to continue giving Emojitracker free access, but are they obligated to do that? They are a business, and providing access to millions of Tweets for free seems like an unfair obligation to ask them to continue indefinitely.
Right. The point here is that the decision might be stupid. It's fodder for exactly this type of a story, while Twitter is pushing for improved relations with developers and trying determine its future as a going concern.
Of course they aren't obligated to do continue to provide free access, in the same sense that I'm not obligated to use their platform. But it might be in their best interest (in both cases).
For all the acclaim that Mr. Dorsey gets as a CEO, Twitter is a great idea but a basket case of a public company.
But continuing to provide this noncommercial API access for free indefinitely is exactly what I think they should do and I'm a little disappointed that they don't agree. Twitter isn't wrong here, they're just assholes.
Because "special access" means free in this case.
Do you realize you are arguing that a customer should have free access forever, while others customers pay? Do you know what type of liability that is for any business?
But I think that a rich, robust ecosystem would flourish more around a platform that provided free API access to noncommercial projects.
Of course, it's conceivable that this API access costs twitter a significant amount of money - it is a lot of data, after all. Even a low cost could be written off as advertising within the platform. "Look at the cool stuff we could build on this API, let's make a commercial product and pay twitter for access!"
This. Also, if API access costs Twitter a significant amount of money, they shouldn't be in the platform business and shut down their API entirely.
While not indefinitely, it would certainly help the case to just public a cost-benefit analysis of how much Emojitracker's access was costing Twitter (though not in terms of "lost revenue" based on what Emojitracker would have been paying for that access).
Can I pay $ 100 extra per month for more access?
To get the data emojitracker has would be tens of thousands a month or more.
But no, I doubt you can pony up $100/mo and get a meaningful amount of data.
It is a serious challenge that the data has some value, but that value is hard to extract usefully as a revenue stream. The emojitracker folks won't pay much, but offering it to them at a small rate vs a high rate to some other service has its own issues.
But since emojii tracker isn't real time, could they use a month old firehose? 6 months old? At some point the value of the firehose access falls off for "mainstream" users of that data, does it fall to a point where there is marginal value in giving free access to others on it? Even for small dollars?
Clearly as a public company Twitter has to do what it can to get good value for the assets it has created. That is a hard thing to do.
Twitter has "users" but those users are not the "customers". Twitter created a watering hole so that it could interest nearby predators in access to where the herbivores would be on hot days.
To be counter-pedantic (is that a word?) If there were anywhere else for Emojitracker to get their data, then I would agree with you that Twitter did not create it.
Flip that with Snapchat who 100% do know their users and converting them into customers.
http://adexchanger.com/data-driven-thinking/is-snapchat-on-t...
> “Developers took to our service from day one and [gave] Twitter a much more global reach."
Twitter would not be where Twitter is today without outside developers building their company for them, through the creation of tools that users actually want to use to interface with Twitter.
Twitter as a standalone product does not have a future. Every change they make that turns more developers away from the platform continues to reduce the number of users globally who are using Twitter in some fashion. It's a downward spiral. Twitter's demise is inevitable if they continue down this path. Developers should assist them by abandoning the platform in droves. There won't be much of a platform left to work with anyway, so why bother? Let Twitter be the next Yahoo.
It was bad. He posted something that made him look like an idiot or a liar. I'm not saying he is either one of those, but the post he wrote makes almost no sense when compared to the e-mail presented in the post. It is like he didn't even read it. If he was trying to say that they never sent that e-mail, then that point wasn't clear. I'm sure he had the best of intentions but it really blew up in his face.
They've been telling us that they planned to shut down free elevated access down for well over a year now, so this doesn't come as a huge surprise. Twitter seems to do regular audits like this, and at least for our academic work, we've had to tell them what we're working on etc. every 6 months or so to keep elevated access.
Having said that, looking at the volume of data that Emojitracker is pulling down, it doesn't seem like it would be too difficult to spread the tracking over 2 or 3 separate connections. The UserStream API, whilst not designed for server-to-server connections, would probably allow Emojitracker to spread the ~900 emoji they track over 3 streaming connections using only a single account.
I mean, isn't this the case anyway? Few people have "elevated access to the Streaming API that Emojitracker depends on in order to operate at its high volume"
And then they do this?
Also: is the service being removed colloquially known as the "Twitter Firehose", or something else? I recently implemented a feature that uses the standard rate-limited API. I went through a lot of pain to get the most out of the rate limits. And people would always ask me "why don't you just get all the tweets through the Streaming API?". I wonder if I had done that, how screwed I would be now.
- "Twitter Developer" access - which is what you find here: https://dev.twitter.com/ - that includes a streaming API that is relatively limited in terms of volume of data and operators that are allowed, but is free and open
- Gnip Twitter access - this is commercial access that would include the Firehose option (though that's not very common). But basically this is a paid offering that offers much more functionality than the developer platform.
In this specific case, the author is just losing elevated access to the streaming API:
https://dev.twitter.com/streaming/reference/post/statuses/fi...
I'm guessing they had extra keywords or multiple connections available so they could track those keywords. Nothing to do with the firehose or anything else. Just a user who had an exception to the official policy that's being taken away.
- Twitter Firehose - full stream of all Tweets, commercial
- Twitter Decahose - 10% sample of Tweets for analysis, commercial
- Twitter Powertrack - real-time streaming of posts based on query (country, language, keyword, etc.), commercial
- Twitter Full-archive API - search interface for entire corpus of Tweets, commercial
- Twitter Streaming API - a mini-powertrack of posts based on basic keyword searches (up to 400 keywords), free
- Twitter Streaming Sample - a small sampling of Tweets for statistical analysis, free
There's a couple more esoteric product offerings from a data perspective, but these are the relevant ones. I think it's a pretty broad breadth of different options, with different functionality at the free and commercial options.
Basically live tracks tweets that use emoji, so emoji are ranked by frequency.
When you are using someone else's platform, they still make the rules... this line in particular: "it wouldn’t help other developers who I believe deserve to get the same level of opportunity as me." ... nobody "deserves" access to any API... period.
I don't understand the entitlement in so many tech projects, who get upset when a 3rd party they are freely using changes their service... you were never entitled to it in the first place, you merely benefited from its existence and should be appreciative of what you were allowed to do instead of feel entitled to it.
(When Emojitracker originally received elevated access 3 years ago, the project was not famous or well known at all, it was just a slightly more formal version of Me saying: "hey check this out, I think this might be a cool hack but I need access to 900 keywords instead of 400, that ok?" and Twitter: "yeah cool, no problem dude.")
You should never expect a level playing field from private infrastructure.
If anything this is more proof for the need of a public access Twitter-like service using public key crypto for self-signed authentication.
They sought out and begged for developers to use their platform. Then they started killing the apps one by one.
Your comment while "wise" in hindsight is thoughtless, arrogant and tone deaf to the work involved in creating an actual product that users use.
There is a large difference between entitlement and disappointment.
The OP was not being thoughtless, arrogant or tone deaf, he was just being honest. If you feel disappointed it is because you made a poor decision. You should learn from your mistakes instead of shouting at the person who points out the tragic results of your actions.
There are plenty of open source and open platform projects out there. BitTorrent, Bitcoin, HTTP, OpenPGP... pick one and start building!
You should feel entitled to your individual rights in whatever republic you live in. You should not feel entitled to tell a private company how to run their business. Doing otherwise will always lead to disappointment.
The right thing to do is consider this a lesson well learned, shut up, eat crow, and get on with your life like an adult, safe in the knowledge that you won't make the same mistake again.
Generally speaking the vast majority of people in this forum have a shockingly bad understanding of how our government, legal, accounting and financial systems work. That infrastructure isn't going anywhere and companies like Twitter exist only because of these formal systems. Learning the details of the relationships between these systems is your responsibility. You can easily be taken advantage of without a knowledgable understanding of the rules of the game. This is no one's fault but your own.
When twitter was just shutting things down and making it clear they were going to make money off the thing, I wasnt surprised nor was I happy, they had clearly decided to monetize what other people were doing themselves, but worse.
Since then twitter has not done anything that I would consider engaging or of note, and from the articles I read on HN, in a crunch on how the heck to monetize their platform and are coming back to the dev community trying to win mindshare.
You can't have it both ways.
How is this any different? Honest question.
EmojiTracker appears to be essentially frivolous and puts stress on Twitter's platform for no good reason.
GitHub and Twitter both provided a free service and then modified it for a specific user without warning. I would say throttling CocoaPod affected more "legit" users than "shutting down" EmojiTracker did. (I put shutting down in quotes because it seems EmojiTracker still has limited access to their API?)
How come Twitter gets called out as unconditionally Evil, yet GitHub was just looking after itself?
Take a look at Known (https://withknown.com/) and Woodwind (https://woodwind.xyz/): one to have an actual website, the second to have a stream of content you want. Done.
Forget silos.
In essence I think the product spoke to the nature of the whole issue. The tweets left Twitter to seed Facebook with good interesting, connected data. Twitter would know little about the context while Facebook would get enriched context from the tweets.
Twitter doesn't want to be a dumb pipe to make apps smarter and better. They want to be the bucket like Facebook.
I know ... It's tremendous amount of data going on there, so I just don't want to imagine how hard it is to scale it and offer that to everyone.
Nevertheless it's so sad that they break promises.
Twitter nowadays is a bot driven, marketing platform ( with some sort of journalism touch ). I still remember the days that I was excited when I got a new follower. Now I think it's just about "hashtags".