Show HN: Twitter graph monitoring over time, SocialGrapple | Hacker News Reader

Show HN: Twitter graph monitoring over time, SocialGrapple

50 pointsshazow22 comments

Ever wanted to track who unfollowed you last month? Or who followed your competitor last month? Or how your social graph is growing over time? SocialGrapple does this.

I've been working on SocialGrapple for a few months (since stepping down from imoveyou.com, YC2010S), this week I got the core product to the point where I'm proud of it so I want to show it off: http://socialgrapple.com/

I wrote up a blog post with some screenshots that will get you an idea of what SocialGrapple looks like:

"SocialGrapple snapshot: @limedaring's social graph growth after her HackerNews publicity" http://shazow.posterous.com/socialgrapple-snapshot-limedarings-social-gra

Screenshots: http://i.imgur.com/OzTIb.png http://i.imgur.com/tghcT.png

The gist of it is: I pull the account's social graph every night (soon to be every hour) and do a diff from the last known state, then create an event log based from that. Then I do a bunch of graph operations like intersection between following/followers to determine stalkers/stalking/mutual, annotate the feed with metadata useful for skimming/ordering/filtering, intersecting two accounts to see how growth relates to somebody else, etc.

Backstory: I built Tweepsect some years ago (http://tweepsect.com/) which lets you intersect your followers/following as it currently is. It's still one of my most-used projects, and a lot of people were asking for more like support for larger accounts, data over time, charts (a lot of people use Webnumbr to track their follower count over time), track who unfollowed them (qwitter and similar stopped working months ago). This is where SocialGrapple comes in, with an angle on competitive intelligence too.

I would love some feedback. What would it take to get you as a paying customer?

Clickable links –

Blog post: http://shazow.posterous.com/socialgrapple-snapshot-limedarin...

SocialGrapple: http://socialgrapple.com/ (Make sure you have a look at the blog post first)

Screenshots: http://i.imgur.com/OzTIb.png http://i.imgur.com/tghcT.png

This is great and something I see as an immensely valuable tool for social measurement. It'd be nice to integrate Facebook into the mix as well, showing fan counts and competitive analysis.

As a potential customer, I see this as valuable as a free resource for basic insight, but a paid monthly service for more extensive analysis.

I've been looking at Facebook and originally all my language was service-agnostic in the event that I toss Facebook in there, but now I'm kind of backing away from it.

1. Facebook doesn't give good access to the more granular stuff, mostly just superficial counts. Especially if you don't own the page/account. 2. Facebook already does pretty decent analytics. Is there a big thing they're missing?

http://i.imgur.com/OzTIb.png

This screenshot is why I really love using SocialGrapple for my account – in the quick overview of what happened yesterday to my account, you can see that I followed one person who followed me that day ("mutual" tag on it), and who the most important accounts are by follower count (x2 means twice as many followers as following, and a better ratio usually means a more important person).

I love Twitter and using it to meet new people, but when 100+ people follow me in one day, it was really awesome to have SG to help me parse the data.

</unsolicited review>

Got this error when trying to sign up for tweepsect:

Traceback (most recent call last): File "/base/python_runtime/python_lib/versions/1/google/appengine/ext/webapp/__init__.py", line 515, in __call__ handler.get(*groups) File "/base/data/home/apps/jsoauth/2.346920482575371317/main.py", line 127, in get token, secret = client.get_credentials(oauth_token, oauth_secret, oauth_verifier) File "/base/data/home/apps/jsoauth/2.346920482575371317/oauth.py", line 150, in get_credentials return self._extract_credentials(response) File "/base/data/home/apps/jsoauth/2.346920482575371317/oauth.py", line 165, in _extract_credentials raise OAuthException("Problem talking to the service") OAuthException: Problem talking to the service

Hrm this happens sometimes when the Twitter API fails in a weird way. Try again?

One of the cool things of Tweepsect is it runs 99% in-browser JS (including all the API calls). There's only a small piece of the OAuth that had to be done server-side which lives on AppEngine. I need to add better error handling to that someday. :)

Just curious: how do you deal with the increasing limits of the Twitter API? I noticed you don't require me to log in with my Twitter account (which is great), so it seems you are not using user's request quotas, but your own.

Indeed, I tried to make the experience as non-intrusive as possible. One of the biggest complaints from Tweepsect was that people didn't want to OAuth to try it out. Especially if I play the angle of "use SocialGrapple for corporate intelligence" I can imagine users may be weary to give away their true credentials, which is fine.

I applied to have my @SocialGrapple account whitelisted 6+ months ago (a bit pre-emptive, but not enough), but they never got back to me and then last month they announced there will be no more whitelisting. This sucks.

Right now I just try to use the API as efficiently as possible and spread jobs out to avoid hitting the limits. In the future, I might need to distribute the load across multiple auths or request the user to sign in.

I tried to make the pricing in-line with the parts that are "expensive" to me, both in terms of limited API calls and data storage/performance. I hope this is transparent enough to the users and doesn't get in the way.

I have looked at using gnip (Twitter's official bulk data reseller) but the pricing structure is way out of my league (starting at $500/mo). Can't blame them though, since they dedicate an EC2 instance to each customer. Pretty cool scaling model.

(From the superficial complaint dept.):

I assume that your using http://html5boilerplate.com/ to some extent, along with the default favicon and icon.png that it ships with. I'd recommend replacing that with an original icon or at least one that's less identifiable. Great project nonetheless.

Ooof, you're right. Good catch! Keep em' coming, making a list.

Would you consider sharing the technology stack that you used to create this application?

I love the visuals. How perfect for a brand manager to show that doing X can result in Y.

Thank you!

The stack is Pylons (behind uwsgi+nginx), PostgreSQL, and my own task scheduler[1]. Right now it's hosted on Linode. Will be moving to DotCloud eventually. :)

[1] http://github.com/shazow/turnip

Thank you for the details!!!

I really like the followers-over-time graph.

Question: is there a way to embed the graph into another "dashboarding" tool? I.e. through an API or even a simple URL?

Data export via API is on the list. It'll be done as soon as someone actually needs it. :D

I'm hesitant to pay since I can't see any examples of what I'd get. A short trial would be great.

Yea, you're right. Check out the link to the blog post? That has a pretty good insight on what the premium meat looks like. I'm working on better non-user facing stuff as I write this. :) Takatakataka

I got distracted after the first link to the product :) This is right up my alley, so I'm definitely going to try things out and hopefully have good reasons to pay.

Sweet! Send me a note with what I need to do to get you as a customer. :) My email is in the footer when you're logged in.

I agree with johns, as great as your explanation and screenshots were, I'd really love a 15-30 day trial just to give it a go and find a way to work it into my work flow.

Just curious, do you have email alerts and notifications sent out through the app? If so, how?

Would love to chat about that.

Email notifications aren't launched yet but I'm preparing to test start testing it with a small of users. Want in on it?

I can't afford to give out many trials right now, but what I am going to do soon is give all the free plans access to a specific detailed analytics account (probably mine, @shazow) to give people an idea of what it's like. For now, I'm counting on that $10 isn't a huge loss risk.

I'd love to chat too, hit me up at andrey@socialgrapple.com

Wow this tool is just great and something I've always wanted.

Will be using it extensively for poaching clients from competitors. Also will love to see the effectiveness of competitors and my PR campaigns. Can't wait!

I have already signed up for a premium account!

Reply on news.ycombinator.com