Ask HN: Hitting a wall with Twitter | Hacker News Reader

Ask HN: Hitting a wall with Twitter

21 pointsmittermayr22 comments

Hey guys, I am desperate. I have been running a Twitter Analytics service for a few years now and it's gotten way too popular. I have companies like the NASA, Firefox, UFC, Disney, WWF, Subway, Playboy, UrbanOutfitters, Tate, Medium, Jamie Foxx, Queen Latifah, Tim Ferriss and even @ev and many many more using and/or paying for my service.

It's constantly breaking down due to Twitter's strict REST API limits (something everyone knows and has to work around with), but I receive e-mails from people daily willing to pay more, and more, and more if I can find a way to keep the service up and running.

As of now, it's all coming down to site-streams, which is only enabled to whitelisted accounts. They would solve all the problems, I could make the service real-time and make a lot of money before all those guys decide to go with Adobe or Sprinkler, the big terrible ugly things. This is my biggest shot taking off and I feel like the window is about to close very shortly.

I do not sit in San Francisco, I have been unable to meet Twitter folks in person outside of SF so far. It seems a lot of the partnerships are built locally, in SF, through networks, which makes a lot of sense, but completely shuts it down for us.

I've been trying to reach out through every possible venue, partner programs, beta programs, mailing lists, been an active and helping member on the API forums and have hit the wall now, not knowing what to do next.

I have the technology and technical intelligence, the infrastructure, I have lots of paying customers, I haven't even started marketing the software yet (word of mouth), and all I fail at is getting permission to proceed professionally from Twitter.

Anyone, please, any ideas? This is a last resort call.

Naive/obvious questions:

Have you

a) done everything listed at https://dev.twitter.com/docs/streaming-apis/streams/site#App... ?

b) considered getting on a plane to SF, spending a few weeks there, and network like hell, in an attempt to meet face-to-face with someone at Twitter? You said you're desperate, so all options are on the table, though I suspect I'll get shot down for suggesting this.

a) yes, everything b) this is on my list of 'if all else fails' but as you know, networking is a bit like dating. it's a whole lot easier if you're not desperate ;)

You should absolutely be on a plane to SF. As you said yourself, this is your shot. Don't blow it by sitting at home wishing people would fly out to meet you.

100% agree. Fly to SF, knock down every door, connect with every twitter employee possible on Linkedin, and find out who you need to talk to.

Take up your clients' offer, who want to pay more. Cull the ones who don't want to. You'll end up with less clients but be much more profitable.

I would, and prices will be going up soon anyways, but the problem is: I can't deliver a service for those large customers, they have millions of followers and events happening in their account, the data they require is there, just not through regular non-whitelisted means. I've met folks who got whitelisted years ago, for 'just asking', but this has all been heavily cut down now.

I assume this is your product: http://start.fruji.com/pricing.html

One immediate thought: Your prices are ridicuously too low. By like a magnitude. Of like 10 times. $25/year? Heck, $25/month would be too cheap. Increase your prices, and your problem is solved because you'll have fewer accounts to get all the data you need.

Pricing is super low currently because I can't deliver much on a promise, sometimes, the service / servers go down for days and there is no dashboard available for the customer. I would feel terrible charging a lot of money for a service that has constant outages. But prices are going up considerably soon, I've got all the plans laid out, just need to catch up with the technology.

Is your service down because of you, or because Twitter's API goes down intermittently?

And, before someone suggests the obvious, yes, I've considered reaching out to some users of my service that are active investors, founders or business partners of Twitter. But I want to respect their privacy if at all possible and knowing the story behind Twitter, most of the original folks have moved on, it's a different business up there now (great background info in Biz's latest book, worth the read).

Have you applied to become a Twitter product partner? I don't see you listed at: https://business.twitter.com/partners/list/certified-product...

Yes, 5 times. There is no feedback mechanism, unfortunately, so I wasn't sure whether my application was missing certain minimium requirements.

Happy to intro you to someone at Twitter. My email's in my profile.

Have you considered using services like Gnip or Datashift who resell Twitter's firehose?

It would mean rewriting your service but it would offer long term viability.

yes, have considered it, but from first enquiries it seems they are charging rates that are simply impossible to fund at this point, well in the tens of thousands of dollars. I might be able to bring the rates in line with incoming revenue, but this one's a very scary approach, not being vc funded.

also, gnip doesn't seem to have actual real-time access to follow/unfollow events and only access to 1/10th of the public tweet stream, offering a 'sample' of all tweets (which is what the non-whitelisted accounts also have).

you do realize that sample 10% nonsense is simply untrue if you have a way to get the stream for the X,000,000 most popular accounts on a daily basis right (it has to be because getting streams for a specific account has to have all the tweets, not 10% of them)? Forget the bots, just focus on the active users.

I'm not familiar enough with them, or your product (obv), to know, but would user streams help in place of site streams?

partially yes - but this would bring us close to a policy violation and potential blocking. user streams should only be access from a twitter client app (mobile/desktop/web) but explicitely not in a server-to-server type of scenario (see docs). as soon as a lot of our major users are being hooked into the user stream mechanism, we may experience IP blocks or policy related shutdowns. it's a scary thing, they have all that power and we try really hard to stick to the rules here.

Have you reached out to your customers yet? @ev has obvious ties and Tim Ferriss was an early investor.

ev is pretty much out and focused on different things (not on the best terms with Twitter I believe) I would assume from what I've gather recently and through Biz Stone etc. He seems like a really nice and smart guy, but I feel this is not the right alley.

Sounds to me like you are looking for excuses not to solve your own problem...IF you have a means of reaching out to the guy, do so , whats the worst that can happen?

Reply on news.ycombinator.com