Twitter effectively killing JSONP too | Hacker News Reader

Twitter effectively killing JSONP too

blog.dansingerman.com

66 pointsdansingerman27 comments

This doesn't mean killing JSONP dead, it just means that you'll need a server-side script to generate oauth-authenticated URLs first.

This is what I do in one application right now, and it works really well, meaning that for the calls in question, the rate limit applies to the user's IP and not our server's.

I've considered doing this as well. I currently have a Twitter app that's 100% browser Javascript and it works great, but only with unsigned requests. I hate to add any server component but a tiny server that just generates OAuth signatures would work. Then let the user's browser make the requests to api.twitter.com directly, saving me bandwidth.

Orthogonal to the OAuth issue, I wouldn't mind if they replaced JSONP with a JSON response with appropriate CORS headers. JSONP is badness.

I've thought about that sort of approach. Can you describe in a bit more detail how you do that, especially how you ensure it is secure?

It's not a strict answer, and I'm not the one being asked, but about doing "that" while having at least JSON alone, you can proxy the queries and wrap answers by yourself. E.g. in nginx you can use HttpEchoModule:

http://wiki.nginx.org/HttpEchoModule

http://www.gabrielweinberg.com/blog/2011/07/nginx-json-hacks...

Wouldn't those URLs be easy to steal, though?

Require a cookie (browser will hopefully not send that from other sites), check referer, use HTTPS. It's not going to completely stop someone stealing your bandwidth, but it's better than nothing.

The lesson is not "do not build on someone else's platform", but "be careful whose platform you build on". There have been signs for quite some time now that twitter was not particularly supportive of things built on them.

Is there any way to know if one party is more reliable than other? Four years back, was there any way to know that Twitter would limit their APIs and change their terms and conditions the way they have done? If not, isn't the lesson here "do not build on someone else's platform"?

Either that, or "sell your company within four years".

Four years ago, Twitter was headed in a very different direction (and was under different leadership, too). So I'm not sure that it would have been correct to predict that Twitter, as of four years ago, would end up as it is today, since much of that is due to things that happened in the interim.

That said, the writing has been on the wall for quite some time, even if not quite four years.

I remember thinking that something like this was inevitable last year (circa July 2011), and I wasn't even a developer on Twitter's platform, so there were probably signs much earlier than that too.

They "sunsetted" @Anywhere so they can focus on embedded tweets, a more focused attempt to provide badges/widgets for browsers. So really I doubt they want browser apps calling their API directly, but rather to use their embed timelines.

I realise that's a tiny subset of what's possible with the JSONP API, but that's my guess about why they won't support JSONP.

And, if they do shut it down, it sucks. I've run http://listoftweets.com for several years and made various other Twitter mashups using their JSONP API.

I'm wondering (a) if people are going to switch to screen scraping as a way to avoid the new API restrictions, and (b) how much Twitter could do to thwart such attempts.

My (uneducated) guess is that it might play out like other DRM wars. Twitter could come up with new ways to scramble their HTML/CSS/JS to make it harder to automatically scrape... until scrapers evolved to deciper the new patterns, ad infinitum.

If they're willing to flip off search engines and do all page rendering in JS, there's no limit to what they could do to make a scraper's life hell.

It'd be relatively simple to generate non-sensible fonts (where code points don't map as expected) where the browser still renders legible tweets but scrapers get unintelligible gibberish. The special font would essentially be a one-time pad, generate new font mappings every few requests and your HTML is essentially uncrackable to scrapers that don't have the horsepower to OCR every tweet (at which point you may as well pay for Gnip anyway).

True, but it would be much simpler to hack the Twitter native applications to do the dirty work for you, depending on your purposes.

That, or just use the oauth_secret from the official Twitter applications; those were cracked ages ago, and even if they changed them (thereby breaking all their old applications), they're super-easy to crack again.

Hardening the webpage would still decimate the amount of people with the skill/motivation to pry out tweets. HTML scraping is much more accessible than poking around native, source-less apps. (And if it was still a problem, the same tactics as above would work equally well in native apps.)

>If they're willing to flip off search engines and do all page rendering in JS

Isn't that a pretty big if?

They will be killing a lot of third party plugin too, a good example would be the wordpress plugins, a lot of them use the jsonp stuff.

Seems weird to mee, but hey, anything to make money in the end I guess

Well, goodbye twitter widgets on lots of websites, I guess. (deviantART does a JSONP widget on user pages, lots of wordpress plugins do it, etc...)

Somehow I feel we're really not losing that much.....

Does this mean that "http://api.twitter.com/1/statuses/user_timeline/<user... will be deprecated? Or is this talking about authentication?

They deprecated RSS the other day. Even if that's not deprecated yet, you should assume it will be at some point. They seem to be slowly removing access to everything.

My understanding is that the entire API ver 1 is being depreciated in 6 months, so yes. That endpoint will still exists in ver 1.1, but requires authentication, and is subject to the rate limits associated with that.

I would like to know this info as well.

Do all calls to the Twitter API require authentication now? It used to be that some calls (like search) did not require authentication, and I would assume those would still work with JSONP?

Nope, all REST calls now require authentication: https://dev.twitter.com/docs/api/1.1/overview#Authentication... (and search is now formally part of the REST API as of v1.1)

Good. JSONP is stupid anyway.

Care to elaborate, or are you just baselessly venting against acronyms?

Reply on news.ycombinator.com