GoogleBot as QA tester | Hacker News Reader

GoogleBot as QA tester

statsheet.com

28 pointsjohnbb14 comments

I always "wget -r --header 'Cookie: login-credentials-if-neccessary' | grep 500" my staging site before sending it to production.

I also use hoptoadapp.com for error reporting.

Edit: For clarification, this isn't the only thing I do, but it's a helpful sanity check.

I agree that this method might work for statsheet.com, at least in its current incarnation, but it's a bit of a special case. Just because you can rely on spot-checking and only reacting to critical errors when you're strapped for resources doesn't mean it's a good model.

It's like writing a post talking about how you've eaten beans and rice for the past four months. Maybe you needed to do it that way to keep your company afloat, but that doesn't mean that the post constitutes advice.

When bootstrapping a startup a lot of exceptional circumstances occur. Not having resources to test a massive web of pages and data is one of them. The last paragraph of the post makes this clear.

Why doesn't it constitute advice? There are lots of entrepreneurs that are strapped for resources. I don't find this as an "exceptional circumstance". My point was simply that it is ok if you don't have 100% test coverage or any automated test coverage. I think that is a waste of resources in a very small startup in many cases.

Just because you can rely on spot-checking and only reacting to critical errors when you're strapped for resources doesn't mean it's a good model.

I agree. The proposed system can track critical errors and syntax errors, but it can't really keep track of logic errors. The stats may not cause any errors but Googlebot can't tell if they are accurate or not.

Checking for 500's seems like a pretty minimal test suite. There are plenty of other errors that could occur.

Sure, but the point is that when resources are very low you may need to get by with the minimal amount of testing required.

The options are either to spend a bunch of time trying to squash every potential bug, or know that some bugs will get through but at least users will benefit from having a bunch of new features to play with.

This only ensure the pages don't explode, you still have no idea that you are showing NASCAR stats on your NBA homepage...

Crawl and Fuzz locally:

http://github.com/relevance/tarantula

Actually I do because I'm the one that developed those pages. Why would I put NASCAR stats on a NBA page? ;-)

Also, as a last resort users will tell me if something is off.

I think you should reconsider those statements.

Just because you wrote it doesn't mean it's doing what you think it is.

And users really can not be counted on to report errors. Even if they do, the average bug report reads something like "So I put in your address into the google, but when i got there nothing came up! I tried over and over, but every time I clicked on the google, nothing happened! I asked my brother's boss' kid, who's a real whiz with computers, and he told me I should 'eat my cookies' or something, but I'm lactose intolerant!!"

You are missing the point. At a small startup with limited resources you have to make tradeoffs. The tradeoff I've settled on is more functionality in place of 100% quality (if there is such a thing). I think people early on waste a lot of time trying to nail every single bug when some bugs just don't matter in the grand scheme.

Also, I'm not a junior programmer that needs someone looking over my shoulder to make sure I've implemented a spec correctly. Saying it doesn't do what I think it is doing is like saying that I may not exist even though I think I exist.

I have seen many senior programmers who think their code is doing X when it is really doing Y. I've seen a very small number of programmers whose code is always doing what they think it is doing. Every member of the latter group of programmers spends considerable energy double-checking themselves.

Therefore if you're taking short-cuts, the possibility that your code is doing something other than what you think it is is realistic, not ludicrous. Now it may be appropriate for you to take those shortcuts. But you shouldn't take them without being honest with yourself about the risk.

But what do you consider a short-cut? How much testing is enough? Doesn't the situation dictate what is enough and what is really a short-cut? What needs to be tested at a startup IS NOT the same as what should be tested inside a large corporation.

And what "risk" are you referring to? As I've stated, I decided to side with "more features" than "100% test coverage". Sure there is "risk" that there are bugs, but a) there are bugs regardless of how much testing you do and b) I've decided I'd lose users because I don't have features that stand out above the competition rather than because a minority of those features have minor bugs. So to me, the bigger risk is not developing more features.

PEOPLE. downvote != disagree.

Reply on news.ycombinator.com