How Secret Used Automated Testing for Their Android Launch | Hacker News Reader

How Secret Used Automated Testing for Their Android Launch

blog.testmunk.com

38 pointsweitingliu17 comments

This is not terribly difficult to get running locally (at a smaller scale). Square has a tool called Spoon[0] that makes it really easy to run a test suite across a bunch of devices.

On a recent project, we had 5-7 devices plugged into a spare box that would run a test suite on every push. It worked out great and the ability to capture screenshots was really helpful for the design team to help spot strange UI issues. More details on the setup here: http://www.sep.com/sep-blog/2014/01/06/android-robotium-and-...

But if you need larger device coverage, Testmunk looks pretty awesome :)

[0]: https://github.com/square/spoon

I appreciate TestMunk's goal to make automated mobile testing easier and more accessible, and I truly hope they succeed in that, but for the love of testing, please do not view that example feature file at the bottom as an example of proper Gherkin.

It is riddled with implementation details and brittle explicit waits. To be fair, they did clearly state: "we generally recommend against fixed waits" but then please do not put this forth as an example to be copied. Scenarios should describe the behavior from a user perspective and leave out implementation details. You should not have to change the your Gherkin no matter how much the underlying implementation changes so long as the expected behavior remains the same. Not to mention the misuse of the Given/When/Then keywords and the "I logout" steps -- that is what before/after hooks are for.

I also recognize TestMunk points out that this feature utilizes standard steps to get your first test cases going and to get some screenshots. They also advocate for the page object pattern, which is good, but it might be helpful to be clear that this is not how you'd actually want to write your Gherkin, whose true value comes when written with/for product owners, free of implementation details.

Can someone explain to me the appeal of Gherkin and other such tools? I don't see why the example on that page are anything but code that wants to "look better", and just ends up introducing another thing to maintain in your application.

How is writing a step in English and writing the English to translate it to code better than just writing the exact same steps in code to begin with?

Gherkin/Cucumber's big thing is that it lets the people who can't read code (i.e. project managers, suits, and customers) understand what's being tested. Usually along the lines of:

"As a user, I want to buy things, so I should be able to put things in the cart and get an accurate total price"

"As a user, I want to view the site in my own language, given I have come from a German network address, I should get the German language site"

This frees you from having to explain the finer points of cookies, or geoip, or any number of concepts to people who do not operate on a level that close to the bare metal.

The translation isn't really error prone either and isn't hard to implement from the programmer's side. Say that language example above? The actual test would be written like:

    As a user,
    I want to vew the site in my own language
    Given I have come from a German network address,
    I should get the german language site.

Those last two things are recognized via a verbatim regex (basically /^I have come from a German network address$/),

and the actual implementation code does something based on that, say, setting a test IP to a known German netblock, and the last line would actually load the site and look for the right localized strings.

So they're rather closely coupled together. If you change the wording of the test for whatever reason, the old code won't work anymore (actually, I think Cucumber will tell you that there was no test found), necessitating someone to actually verify that the test as written actually tests what the new requirements are.

Thank you for the explanation. This doesn't sound like it really has any advantages over something like Dokku...

> "As a user, I want to view the site in my own language, given I have come from a German network address, I should get the German language site"

I hate this philosophy so so much. If I wanted the German-language site I'd ask for it.

It's somewhat difficult to "ask for it" in a way that's accessible for a large number of people. Either way, you're presuming something that may or may not be true.

It's safest to handle the main use case, that is, people using the most prevalent language of the region, and implement a language chooser for everyone else.

> you're presuming something that may or may not be true

What's that? I'm presuming that there's more than one language available, but so does the initial example.

I mean no matter what you do, you're defaulting to a language that the user of your site may not speak.

I don't consider it "unsafe", or problematic in any way, that http://en.wikipedia.org/ gets you English wikipedia even if you access it from a German IP, whereas http://de.wikipedia.org/ gets you German wikipedia even if you access it from an Australian IP. There are plenty of websites that will not give you the page you request no matter what you ask for, if they think you should be going to a different page. That's idiotic.

Secret did NOT get $35M in a Kickstarter campaign, it was venture capital. Or was that a joke I didn't get?

Looks like it was edited on the sly:

http://webcache.googleusercontent.com/search?q=cache:http://...

Better link, since Google will eventually update their cache: https://archive.is/RAKfE

(Yes, I get the recursive irony of archiving a Google cache result).

Given I wait for 5 seconds

Then I press view with id "splash_signin_text"

Then I wait for 2 seconds

Then I clear input field number 1

Then I wait

...

Looks like someone never bothered understanding what Given/When/Then are for.

I think that describes most people. What's wrong with this?

I find the text hard to read on my screen. Does it have to be so gray?

It was like a mobile phones orchestra. Amazeballs.

Reply on news.ycombinator.com