Show HN: Make your AngularJS, EmberJS, or BackboneJS website crawlable by Google | Hacker News Reader

Show HN: Make your AngularJS, EmberJS, or BackboneJS website crawlable by Google

13 pointschaddeshon22 comments

http://www.BromBone.com

I posted “Tell HN: My Web App has 13 Users” 3 months ago (https://news.ycombinator.com/item?id=5386966). Over time, it grew to 150 (free) users, but retention was pretty much zero. I was trying to cast a wide net and offer a generic “headless browser as a service”. It turned out I had make a service that was a little helpful to many people, but didn't really solve anyone's problem completely.

I finally took the advise offered in that thread. I focused on a problem I understood, and solved it completely instead of just offering a helpful tool. Instead of a “hosted headless browser” BromBone is now a complete solution for people who want their javascript websites indexed by Google.

BromBone.com - The javascript SEO problem has been solved.

Google can't index javascript driven webpages. The accepted solutions have been: 1. Run your own PhantomJS server to render snapshots and serve the snapshots to Google (a real pain). 2. Accept that your pages won't be listed on Google.

With BromBone you don't have to either. You just have to make a small change to your .htaccess file.

BromBone crawls your websites, runs your javascript in a web browser, and saves the rendered page. When Google crawls your site, proxy the requests to BromBone, and we'll send the rendered snapshot to Google.

Smile. Your javascript driven website is on Google.

You have me wondering about the feasibility of offering this as a CloudFlare app.

If I was able to throw up a little ajax/client side JS driven app at a domain, throw CloudFlare over it (like I usually do for most smaller domains I own) and then have it all happen without even needing to change that .htaccess file, then your onto something even better.

I also think the pricing model may need some work. Per page seems like it could cut you off from some potential customers. For instance, I'm already thinking of a blog engine I wanted to try out more thoroughly by making it the one i use on my new blog but wasnt prepared to loose "google-ability". I Would like to be able to use a service like this but when I do my mental math it doesn't add up. I wanted to use a blog engine that will produce a large number of pages by pulling in third party service activity into my site. So its not 100% what I want, but damn its tempting.

If your pricing it to cover costs, might I suggest looking to optimize your stack to bring down those costs down?. If you can cut the price and make it more of a 'yeah for $X its easier than doing it myself' you'll get more takers. Right now it feels that your current lowest price per month is too high for a lot of devs to decide the $ is a throwaway expense to save them time. Id hazard, below $25, and probably with an order of magnitude more pages.

All that said... I may still try it out. It looks interesting.

Thanks for your suggestion about CloudFlare. I am looking into it.

Perhaps you could use crawl frequency in your pricing. For example daily (or even hourly) crawls for the top tier plan.

Most people probably don't have that many pages so I'm thinking the existing pricing tiers might not work well?

And maybe offer a free plan that only crawls your site 3x per year or something really slow like that.

I almost did that. Ultimately I decided to take an "it just works" philosophy to crawl frequency (unless you need realtime).

Would you segment the number of pages differently? I think some sites with a lot of user generated content might actually have a ton of pages. But it is hard to get real numbers.

Hmm I still like the crawl frequency as a price differentiator personally.

Other ideas might be a service level guarantee for the high priced plans. And included email support for setup.

Maybe Patio11 will chime in with some ideas for your pricing plans? Everything I know I got from reading his comments/blogs.

(I think he searches for mentions of his username, so the bat signal is officially turned on :-)

They're undepriced, but I think the first problem is going to be going from 0 users to 10 users (bang down doors) rather than extracting the maximum proper value from those first ten users. The optimal pricing strategy will still lead to him having 0 customers until he figures out who needs this and how to reach them repeatedly.

You're right. Getting these first ten is SO hard.

After I have ten customers, I will have much more information to base my prices and differentiators on.

I think number of pages is a good metric for pricing plans. Maybe the numbers could use some tweaking (e.g. something between 200 and 50k), but this seems like a good differentiator among people and how much money they have to spend.

It also seems like a good way to divide people by their scale. You want a metric that means something to a customer, and one that they will bump into eventually.

What you could do to add value to the upper plans, is also offer hourly scans/on-demand scans/scans invokable by an API for the big customers.

Clickable: http://www.brombone.com

Do you think there would be any demand for a separate service that makes it easy to screen scrape javascript heavy sites? (might be legal issues there though?)

(Sorry for all the comments, this concept intrigues me)

I don't know. That was brought up in my previous posting, and I've had some people email me about it. It's hard for me to tell. I'm just not in that community enough to understand their needs.

I'm afraid that every scraping project is just enough different that it would be hard to write something that offered value to a lot of people.

That's probably a good point. I guess if someone wants to screenscape a dynamic JS heavy page they'd probably want to interact with the page.

Marketing idea: How about you offer a free version but insert a link back to brombone in the crawable/rendered version?

As long as you were very upfront about it, it might not be too offensive.

Wouldn't that blacklist his own site?

Why would it? I guess I don't know a lot about SEO.

Me neither :) But my line of thinking was that lots of sites that have nothing in common pointing to the same site... must mean that that site isn't isn't very relevant

(Now that I'm trying to explain it I see my mistake: it could just mean that the site is very general, like Facebook.)

Some people think that embedding a link to your site in a web product/plugin you distribute can lead to bad things, especially if it is not a nofollow link and your plugin is used by nefarious sites. http://www.jitbit.com/news/184-how-google-authorship-and-pen... has a bit about the Penguin algorithm and this effect.

That sounds really useful! Will google penalize for this though?

This is actually the method that Google recommends for these AJAX powered websites. https://developers.google.com/webmasters/ajax-crawling/

I am just letting people out source the hard work. Since the rendered page will be proxied from BromBone, there will be no way to tell that the same it rendered the file instead of the normal webserver.

That's very useful info; I had no idea. I thought the Google crawlers were basically headless Chrome bots so I wouldn't have expected this.

Do you mention this on your site? Seems like a good endorsement, "This is the correct way to get indexed, as per Google!"

I have a section about that. I hope people will read it.

Reply on news.ycombinator.com