Offer HN: Twitter API Help | Hacker News Reader

Offer HN: Twitter API Help

37 pointsabraham25 comments

I will answer any burning questions about the Twitter API. Issues you might be having, OAuth, rate limiting, you name it I can probably answer it.

What makes me qualified to answer Twitter API questions? I am the all time top poster on the developer group: http://groups.google.com/group/twitter-development-talk/about and my TwitterOAuth library is the 4th most watched PHP repo on GitHub: https://github.com/languages/PHP

Edit: Feel free to contact me through other means as well: the@abrah.am or @abraham

View on HN

Is there an established (or perhaps undocumented) way to retrieve an entire conversation from a single tweet?

The ugly way a coworker and I ended up using was to just see if the single tweet was in response to an ID, request that ID, see if that was in response to an ID, and so on until it either ended or we cut it off prematurely after a certain conversation length. This method also doesn't allow you to see any future messages within that conversation.

For traveling up the chain your method is the best. For getting replies to a tweet there is an undocumented API [1] that was implemented with #newtwitter that returns some responses.

You could also use the third-party API that Twitoaster provides. Twitoaster will generally only work if the user who created tweet that started the conversation is a Twitoaster user.

[1] http://app.apigee.com/console/apigee-console-snapshots-12833... [2] http://twitoaster.com/api/

I am trying to access twitter via RSS feeds to display them on the home page of my site. I have tried using ASP, but I get errors, it seems the amount of times I can update that feed are limited. When I use Google's JQuery, I don't seem to have the same problem. Is it a limitation of using ASP (classic not .NET) to pull Twitter feeds? Or is there a server limit to the amount of times their RSS URL can be polled?

Twitter API has rate limits. I think it is 360 req/hour per IP if you are not white-listed. So if you poll using server-side code you are likely to hit the rate-limit. You can cache results to avoid this.

RSS feeds are subject to the REST API rate limit [1] of 150 requests/hour unauthenticated and 350 requests/hour for user authenticated calls. The reason you don't run into the issue with jQuery is because the rate hit counts against the visitors IP rather then your servers. Two easy ways to help alleviate the issue 1) make a bot account force authentication [2] for all calls 2) proxy your request through apigee.com which has a whitelisted IP with more requests/hour.

[1] http://dev.twitter.com/pages/rate-limiting#rest [2] http://dev.twitter.com/pages/oauth_single_token

Thanks folks, much appreciated! :)

I am trying to set up a Twitter bog for a blog powered by ExpressionEngine.

At the moment, the blog titles are formatted by EE's own Markdown - the difference between it and Markdown is that em-dashes are made with "--".

I could use a service that retrieves the RSS information about the blog titles and post their formatted versions (with proper curly quotes, dashes, etc.), but I prefer to manage everything myself.

My question is, what is generally the way to write a Twitter bot for a blog that publishes the post title and (shorturl) link similar to what @daringfireball does, and how do I preserve the formatted title text?

Complete Twitter API novice, but I've been trying to get into doing some projects with it, but the available resources haven't helped me much.

Thanks a bunch for offering your help to everyone at HN.

The basic steps will be: 1) Hook into the post publish action in EE. 2) get the final URL and shorten it. 3) Format the tweet. 4) Post to Twitter.

1) This is EE specific and I know nothing about EE so you are on you own.

2) You can roll your own URL shortener if you want or use one with an API like http://bit.ly. http://code.google.com/p/bitly-api/wiki/ApiDocumentation

3) Format the title however you want. Probably by taking the pre-formated title text, running it through EE's Markdown function, and concatenating the shortened URL to the end.

4) Finally a POST request to Twitter to create the status: http://dev.twitter.com/doc/post/statuses/update. For this I recommend finding a OAuth library for EE as rolling your own will take time and likely won't conform the the OAuth specification. Since it will be a single account there is a shortcut to getting access tokens for the account: http://dev.twitter.com/pages/oauth_single_token.

I have problems with authentication. I am using python and tweepy. My issue is, after the "Allow | Deny" process twitter is supposed to redirect the user to the callback url with an oauth_verifier variable in the url. Right now I am only getting a token which I already have passed earlier. Thanks in advance

What's the best way of handling failwhales that appear in the API? Sometimes they appear with the right HTTP response code, but at other times they don't - most helpful. Right now, I'm checking responses for the name of the failwhale image - is this what you do as well?

The Twitter API docs (http://dev.twitter.com) has HTTP error codes for each Twitter API. There is Twitter Search API, Twitter REST API and Twitter Streaming API. So though the meaning of the error code might differ slightly generally it is the same. For example, 200 is always successful request and 404 is always resource not found.

Are you seriously checking for failwhale's image name in API response?

For successful API calls that return a status code other then 200 I would just ignore them. There are no API methods that will cause havoc by being called twice (other then maybe sending a duplicate DM.)

For failed API calls that return a 200 status code validate the response. For example if you are getting a user then verify that there is a screen_name and id present. If they are present in the response you are fairly safe to assume the call actually succeeded.

Could you please tell if the Twitter API has better support for certain programming languages than others? Specifically, is there any advantage to using Ruby/RoR over PHP for a project that interacts heavily with Twitter? Thank you!

Some of older API methods seem more natural to rails developers since they were basically just returning xml/json from existing routes. But the REST API is platform/language agnostic so use the tools that you know and work best for you project.

what is the best way to get whitelisted for more than 360/hour API calls? I'm trying to do some network analysis and data mining and am held up by the API limit. I've applied about 4-5 times and have been denied each time.

Why did you get denied? If you don't know, ask @episod/Taylor to tell you.

My understanding is Twitter has basically stopped approving whitelistings. Unless you are in the 1% that has a genuine need they won't give it to you. Have you looked into using the Streaming API? http://dev.twitter.com/pages/streaming_api

The likelihood of rejection might not be as dire as I previously stated but you can judge for yourself:

http://groups.google.com/group/twitter-development-talk/msg/...

I do in fact use the streaming api but that doesn't help with things I need, such as getting screen names based on a twitter id, and getting lists of followers for thousands of people at a time, in order to do some graph processing.

How are you getting user_ids without having the companion screen_name? You can get some extra API calls by proxying through http://apigee.com/. You will likely just have to delay some of the processing to stay withing the rate limit.

You received overt rejections? So far I've just heard nothing back.

how does one handle rate-limiting : I mean what is the back off method that Twitter approves ?

I have an app that pulls data from the Twitter streaming API, filtering on a keyword. Suppose, I start getting rate-limited errors (I guess any response code other than 200, means that you are getting rate limited).

Can I still keep pinging Twitter - and when the 60 minutes are up, will the tweets start coming in again ? Or can I sleep for exactly 65 minutes and hope that everything will be fine again ?

You are confusing the Streaming API with the REST API a bit. The Streaming API rate limiting notices [1] for statuses are delivered in the stream and once the limitation period expires the matches will start automatically being delivered again. This generally means your keywords are too generic or common.

If you are getting a response other then 200 check for the coresponding reason on Twitter's list [2].

[1] http://dev.twitter.com/pages/streaming_api_concepts#filter-l... [2] http://dev.twitter.com/pages/streaming_api_response_codes

That's a nice offer, and I don't want to alienate you at all, but is this really a necessary topic for the Hacker News frontpage? Wouldn't this kind of Q&A be better-suited to email, Stack Overflow, IRC, or the Google Group?

What makes this post less deserving of the front page then the other Offer HNs? I am more then willing to help people out through any method of contact.

Reply on news.ycombinator.com