New developer requirements to protect our platform
blog.twitter.com
blog.twitter.com
https://www.eff.org/deeplinks/2018/01/ninth-circuit-doubles-...
For a straightforward counterexample to your argument about TOS enforceability, see 3taps. Or Facebook v. Power Ventures.
That's actually not true, if it's public information, then you're free to grab it. See below:
https://www.theverge.com/2017/8/15/16148250/microsoft-linked...
By reverse engineering the API, sure they can shut you down, but a lawsuit? Hardly.
https://www.ca9.uscourts.gov/media/view_video.php?pk_vid=000...
The 5-minute spans around minute 20 and around minute 53 are particularly interesting. Even HiQ agrees that they have no right to ignore TOS terms pertaining to things behind login screens.
Er, since when? Auernheimer/weev was sentenced to 4 years for "hacking" AT&T by changing a URL parameter to access other customers' info. It was questioned in the appeal decision that let him go, but the ruling was on narrow (venue) grounds.
https://www.wired.com/2014/04/att-hacker-conviction-vacated/
Fines and jail sentences are things only the government can impose. I'm not talking about what the government might do to someone violating Twitter's user agreements (I doubt they'd do much).
The overall gist of your argument is really hard to understand. Twitter makes a service available conditioned on an agreement they have with their users. That agreement is binding. The legal issues here seem pretty straightforward.
It's possible that we're talking about different actors in this story. You might be thinking I'm saying Twitter is likely to sue individual users who use third-party clients. I don't think that will ever happen. On the other hand, if you're the kind of organization Twitter is describing in this bulletin, making potentially millions of dollars by abusing the Twitter APIs to spam or deceive users, I have no problem at all seeing how Twitter might seek recourse in civil court. It's their site, and access is conditioned on their rules.
Then you have the practical part. Would you really base your project on reverse engineering and hacking a third party service? It will end up being an arms race. Everytime they change something you'll race to hack it again. In the meantime your service will be inoperable. If your user base is in the tens of thousands that will kill you. I can't think of any reliable service that every couple of months or so would go offline even for a few hours until the developer finds a way to circumvent whatever countermeasure Twitter came up with.
So you play nice and follow their rules. Even if they suck, which by the way is the main reason why Twitter lost all the love of developers and has stagnated for the last years.
Depends on what the service is. youtube-dl faces this issue but is some of the best software I have ever used.
That's some hefty praise, do you mind explaining why it's so great?
They (any service with a large user base across many devices) are at a serious disadvantage of not being 100% in control of app upgrades across the legion of devices and app versions supported. They are just as likely to break a legitimate user's app as yours (even higher, imo). I don't think the technical war is nearly as difficult as you imagine. On the legal side, you might be right depending on the geographical location of the dev.
In regard to the original question, I think we'll begin to see a shift in that direction as these services tighten the screws. Like youtube-dl, Youtube++ app is a relief to anyone who watches much YT on a phone/tablet. It's easily sideloadable even on iOS and actually puts the user first with regard to permanently settable playback speed and ability to hide algorithmic suggestions among many other tweaks. It's based on a year+ old version of the official app and is still working fine even after being DMCAd. So you know they got Google's attention, but haven't had to update the app in almost a year. I also see these gray area tools as an important check on the power of the platforms to not abuse their dominance. Like jailbreaking was before Apple started incorporating many of the more popular tweaks into iOS itself.
Yes they are? Alternative apps use the official API, their internal services use their internal API's, which are also likely versioned. So if the internal DM API has an upgrade, the website can change to the new version while the mobile app is using the previous version.
youtube-dl and Youtube++ are already solved problems. The fact that you have to sideload it means the user market is tiny and at this point I highly doubt YT even cares anymore.
> I also see these gray area tools as an important check on the power of the platforms to not abuse their dominance.
This is the weirdest thing you've said in a whole lot of weird things. YT proved how powerful it was via demonetization of so many creators videos. Nobody could do anything about it, primarily because the real power behind YT is that creators don't have an alternative, not users.
I think we might be speaking about different things. My original point was that reverse-engineering Twitter's private APIs (used by the first-party app) is not as difficult to do or maintain as the parent suggested. My reasoning was that for the foreseeable future there will be legitimate users on non-updated clients, forcing Twitter to choose between disabling those clients or letting your 'cloning' of those apps' API calls continue to function.
> This is the weirdest thing you've said in a whole lot of weird things. YT proved how powerful it was via demonetization of so many creators videos. Nobody could do anything about it, primarily because the real power behind YT is that creators don't have an alternative, not users.
Maybe I wasn't clear. I'm not speaking about creators-vs-YT, but consumers-vs-YT in terms of anti-user restrictions in the YT app and website. Issues that grey area tools alleviate and therefore provide the only effective check on YT's ability to take them further. Just as abusive ads and tracking has caused a jump in adblock usage. The harder it gets to enjoy YT content through the first-party app, the more people will take the time to discover alternatives.
The concept generalizes to Twitter or YT or any platform (or creative industry) that achieves ubiquity or monopoly. Even if you disagree, maybe it sounds less 'weird'.
It's Twitter's API, and Twitter's servers.
"Has the law changed in some noticeable way to make reverse engineering the official Twitter client and extracting their first party keys for purposes of interoperability suddenly illegal?"
If you don't have permission to use someone else's stuff, you don't use it. We were all taught that in kindergarten.
"We used to do this all the time"
The fact that you abused other's web servers back then doesn't mean that it was right then or now.
The bits and pieces of an API I did piece together (from older protocols), were stunningly unreliable. I am sure they were left in only to frustrate people trying to do exactly what I was.
I have a few twitter accounts:
one where I follow people I know and devs and it's very noisy. It's nearly unusable, it's full of political rants, even from well known devs, angry tweets, just not very useful. I don't read it much.
The other twitter account I follow people who write funny tweets, and that account is a goldmine. I love it. I like almost every tweet. Nobody follows this account, it's just like a read only twitter and it's amazing.
I have a third twitter account for local news, I follow the local fire department, police, local news, local businesses, and it's also super great.
Twitter's android app makes it easy to switch between accounts, it's basically just like different feeds.
Most bot farms have a LOT of dev accounts, multiple apps, ways to automatically create an app, etc.
This is not a solution against bots.
When maintaining an army of bots becomes more expensive than the amount of money they are making from it, they will stop doing it.
There's an upper bound to "bot complexity"; that being the complexity you want to impose on a normal user. For many sites (e.g. instagram) it's often easier to just "pretend to be a normal user" and hit it from a headless browser. You lose some functionality, but at least for my needs, that's always been a sufficient fallback.
As such, I've always been of the opinion that the primary function of API limitations is to punish/limit/extract more money from the good participants.
API monitoring stops bad actors; doesn't matter much if you extract the creds from a binary blob if the usage pattern begins to deviate from the norm, invalidating the creds and forcing a creds cycle.
API access is now effectively for ancillary or supporting tools.
Not knocking it, it makes sense from a platform integrity perspective. It is what it is.
The suggestion is doing this for Twitter too, so instead of using an API key you would log in through an app and it would scrape the data it needs.
[0] probably doesn't exist, just a hypothetical example
They write that they can lift this requirements for certain apps. But any new 3rd party twitter client is pretty much toast.
Unless they take the time to go through the review process and get a rate limit increase, in which case this theoretically should improve the app quality overtime as only those willing to put in the work will make it through.
Everything has shifted back to the walled garden approach. Twitter had destroyed a wide ecosystem of applications that promoted their brand several years ago via a similar approach. FB has done (and may be again doing) the same.
The message remains clear: don't build your business on another business without contracts and guarantees. When the utility of having you as a promotional instrument wears off, they will dump you.
Or even with them: https://techcrunch.com/2018/06/21/twitter-smytes-customers/
These two tweets really show just how WTF this was
> A vendor notified us of their acquisition at 6am this morning and shut down their APIs 30 minutes later, creating a production outage for npm (package publishes and user registrations). The sheer unprofessionalism of this is blowing my mind.
> It takes weeks to negotiate and sign an acquisition. You didn't find out at 6am. You couldn't give us a week? Even a couple of hours to take your service out of our critical path and avoid an outage? Fucking shocking behavior.
Full paragraph explanation for each permission you're requesting. (meh)
Video of the app working.
Only can use personal account for testing/polling API.
You only require approval from twitter and a product website if you have higher usage than that, which is what would prevent you from using that workaround.
> Beginning today, anyone who wants access to Twitter’s APIs should apply for a developer account using the new developer portal at developer.twitter.com.
Existing API keyholders:
> Eventually, all developers with existing access to our APIs will be required to complete a developer account application in order to maintain their apps.
I know that fixing/changing/upgrading things at scale is much harder that what most people think, but man... they have done nothing to fix these type of issues, and don't tell me is not something easy to fix.
The other day a journo complained on twitter about the problem of dealing with crappy messages: https://twitter.com/sarahfrier/status/1020512187322789888
So in less than an hour, I had a prototype to help to filter those type of messages: https://www.dropbox.com/s/z914olzgn68v660/Screenshot%202018-...
If a random dev can do this, why they can't fix it?
I like twitter, and I'm genuinely curious why some of these seemingly easy to fix problems are not taken care.
But is trivial to return a secondary, already parsed array. I haven't used the API in a while and was just seeing if I could achieve that.
Want to join that space? Gotta be lucky with review/limit raise request I guess. It's possible they just want services to start small and act legitimate to raise the bar for "bad apples" and will raise the limits easily afterwards, we'll have to see.
But I'm more and more thinking that Twitter and other platforms should consider marketing automation to be a form of spam. Once you start questioning that there isn't a human behind the human account you give up on human interaction.
So yeah, maybe there's some vanity metric that marketing automation props up for companies like Twitter. But underneath that vanity metric, the real quality of the site is being diminished.
Compare with email spam countermeasures making it more difficult to set up a email server that other email providers will accept mail from. Or malware resulting in the rise of app signing and app stores.
My last hope is that the "review process" they talked about could really differentiate bad apps/bots and interesting bots and let them work. But I honestly doubt it will happen.
This wasn't intended to be an editorial, it was intended to provide context. I believe that it's not clear from the title alone as you've included it here that it was Twitter. Yes, it's on the twitter blog platform, but so are many other things that aren't actually from twitter.
Personally, I spent some time trying to make sure the submitted title was more accurate and informative than the one on the post. I freely acknowledge that it's your call. I think you got this wrong.
Perhaps I should've put:
Twitter introduces “new developer requirements to protect our platform”
To be honest, I do what I think is right, and I don't care to second-guess the mods. But I'm slowly being trained not to care at all.
... since the original title was neither misleading ...
To me the title as it stands is misleading. That's why I wanted to make it clearer. <fx: shrug />
But seriously, your call. You thought it was editorialisation, intentional or not, and you changed it. I think the existing title is less useful or informative, but it's not my final decision. I've long felt that the blind reversion of submitted titles to the title of the article is misplaced and over-zealous, but as a moderator on other platforms, I know that every decision requires work, so not actually making a decision is a huge saving in time, effort, energy, and will.
(Several edits have been made as I try to make my points clearer. Not sure I've succeeded, but I have no more time.)