Tweetping, Twitter traffic in real time | Hacker News Reader

Tweetping, Twitter traffic in real time

tweetping.net

98 pointspzeups38 comments

Check out the Twitter activity in realtime

This project is based on Nodejs w/ Socket.io, Processing.js and Backbone.js

Designed and realized by Franck Ernewein.

Open articleView on HN

How are they getting the data? Twitter's APIs seem so limited these days.

Data come from the Twitter Streaming API https://dev.twitter.com/docs/streaming-apis

Is the site getting the firehose stream though? How do you get that level of access?

They're probably using the 'sample' firehose, which is not protected and still serves a few tweets per second.

The free version gives access to 1% [1] of all tweets, which seems about right for this.

[1] https://dev.twitter.com/discussions/2752

that looks like a lot more data than a few tweets per second though.

It struck me seeing this how rare it is now to see any kind of Twitter experiment. Complete 180 from a few years back.

Yep. I was part of the Web Ecology Project in 2009-2010 and we were able to get pretty awesome access to Twitter at the time. I had a good dialog on a weekly basis it seemed with their API team and they were super responsive and supportive of us. Although, whenever I asked what we needed to pay for full Firehose (which some companies were getting) I couldn't get an answer from them, and I think they only wanted to talk with huge companies.

Then that all stopped. No more whitelisted IPs for search, no more new whitelisted accounts for API calls, and then they stripped all my whitelisted accounts of their 20k/hr API calls and put them all back at the standard 150/hr or whatever.

And what's worse is the network is bigger than ever. Instead of scaling API access up with network size, they scaled it down. This is really a shame as I think Twitter is terribly interesting to with it from an academic standpoint, but unless you have huge money to shell out for API calls or direct access you're screwed.

There's so much potential information there, and some very neat things you can do with it. But for me at least, their new API policies completely turned me off of making cool things with it. Not that the API isn't still powerful.

I'm surprised at the amount of traffic coming from Indonesia. Can anyone offer some insight?

Indonesia has 4th biggest population in the world. Also, 6th most active country on Twitter (2010).

Very cool! Mind providing some background on how you've rendered the world map and determine tweet coordinates? Are you mapping the tweets' latitude and longitude through some projection formula?

Ya I'd like to know also

It's only geolocated tweets. No projection, just a render with Nodejs w/ Socket.io, Processing.js and Backbone.js !

This was really cool. The office is having internet connectivity problems right now, and after we fixed it, this page loaded, and for a brief moment, I felt like a hacker from the movies :D

Curious about the CPU usage of canvas apps on older systems, I ran this in FF and Chrome. (I'd classify the system I'm on at the moment as a mid-tier business PC from 2008.) FF maintained a CPU usage mostly in the 33-40% range and Chrome in the 18-23% range. CPU usage would spike up to 75-90% in FF, whereas the largest spike in Chrome was only 40%.

It's just one data point, but I was impressed with how much more efficient Chrome was than FF.

That's true! http://blog.lightjs.org/2013/01/svg-canvas-divs/

Chrome is 62% faster than Firefox, Safari 85% and IE10 67%

This is pretty awesome, but I would like to be able to dig in deeper. After looking at it for a couple minutes, there's really nothing more I can do.

I made a project that is similar - shows a bit more information except its only for the US and you need to search a specific word/hashtag. http://tweetpleth.herokuapp.com/

Incredibly well done and excellent execution. However, there is a noticeable delay/lag that totally took me out of the user experience. Even though the concept was really cool, I wouldn't mind if everything was slowed down a bit so I get what's actually going on and there was a bit less lag.

I just realized that they don't have twitter in Australia!! ;)

I think europa should be europe, for correctness's sake

A pause button would be nice. Also, parts of Australia light up with activity on the map, but strangely it isn't listed on the dashboard.

I believe the regions are defined as the blocks that become visible when you hover over the content title. The Asia one doesn't include Australia.

Are you sure? the webpage showed me these:

----

The following error was encountered while trying to retrieve the URL: http://tweetping.net/

Connection to 91.121.175.82 failed.

The system returned: (111) Connection refused

The remote host or network may be down. Please try the request again.

Your cache administrator is webmaster.

it's back!

Is it just me, or is it ironic that this site is currently unavailable?

It's just you. What does visualizing tweets have to do with high availability?

would be nice to have a website for "tweetping traffic in real time(as it gets hit by HN)"

the author is working on it, more you are, more he will improve perf.

Nice. I like trendsmap as well. www.trendsmap.com

It's only on geolocated tweets ! Awesome.

that means the less popular locations get even less exposure, as there are plenty of places in the less developed world of which Twitter location (desktop) is unaware of.

That is fun. A bit scary, but fun.

will be back soon.

back soon.

nice work.

nice work!

iam love twitter and right now iam watching a show called #domian and iam looking for that hashtag and there are per minute about 30 tweets and i dont see them on your site. iam from germany.

or did i just overlooked something?

Reply on news.ycombinator.com