Ask HN: Web Dev Testing Best Practices | Hacker News Reader

Ask HN: Web Dev Testing Best Practices

63 pointsgrumps23 comments

So I'm making the jump from a 150K employee company to being employee #18 at a web design agency. Part of my role, will be: -Defining/Documenting how designs should work (aka setting up Requirements Managment) -Process and Development definition(SDLC definition, refinement, deployment) -Setting up and running QA (essentially, from what I know it's tested by the Dev team)

What I'd like to know, is how is everyone testing there webapps. Most things are built with Drupal or Sitecore (.NET).

View on HN

For testing the front-end, take a look at Selenium: http://seleniumhq.org/

It's a bit of a pig to get used to initially, but I guarantee the first time you run a test and find a newly-introduced bug, it'll make your day - and your team's, for that matter.

How's their mobile support?

I'll second that.

Furthermore, Sauce Labs offer servers running lots of different OS/browser combinations that you can run your Selenium tests on.

In other words:

- we write our Selenium tests in Python (assert that when you click, X happens)

- they get run on our continuous integration server every time we push to GitHub

- the Selenium tests fire up IE 8 on Windows on a Sauce Labs server and run themselves

- the output gets piped back to us as part of our unit testing suite, emailing us if any of the tests fail

Well, we almost have it working as well as that :) Either way, we've been really impressed by Sauce Labs so far.

I work at a web design agency about that size. Good luck finding any budget for testing. Clients don't care, and if you budget for it most will go with a lower bid somewhere else.

As someone else said, it's basically account people clicking around, you fix stuff, they click more until they feel comfortable presenting it to the client. Then you find out that some bigwig at the client has a 5yr old blackberry where the layout doesn't work right, and that becomes your focus of your debugging for a week.

I'm reminded of the time that I built an ordering system, and had to rebuild part of it before launch because it didn't work on the client's CEO's webtv. Yes... webtv. Given that he was the only one using it, and he wasn't going to be ordering from himself, I tried bravely to suggest that we not rebuild the front end to work with webtv, but it was not to be. :(

If you'll forgive the self-promotion, you might like to check out my web app, Testpad (https://ontestpad.com), as a lightweight tool for organising manual testing. It's basically structured checklists rather than formal test case management, and has been doing well at agencies both big and small. Because it's lightweight and flexible (think cross between Excel, Notepad and OmniOutliner) you can start out as simple as you like and iterate the complexity as and when it's appropriate.

And to answer your question: I am of course using it to test itself! Plus some selenium for basic automation, but without going overboard as the app is very javascript-heavy and interactive, which limits how much can be usefully/cost-effectively achieved with automation.

Strive to test on three levels:

1 - Unit tests, low-level method testing w/ mocked or no external dependencies (such as db)

2 - Integration tests, test against external dependencies, such as db, apis

3 - System tests, front-end testing that covers all parts of the system using a tool such as Selenium

Terminology varies and differentiation of levels, as well, but this can serve as a good guideline.

You'll need to have everything automated and integrated into a build system where you can reliably kick off a build and see the results.

A note on Selenium: any non-trivial setup is going to require a fair number of resources, both in time and environments. Also, you will have to dedicate time to maintaining tests and your UI changes. However, the pay-off is quite large when you can run and end-to-end set of tests on your application and know that you didn't break something.

edit: formatting

I would recommend using the Selenium WebDriver browser automation framework (there are C# .NET bindings). This can be used in combination with a BDD approach using SpecFlow, which is also for .NET, or Cucumber for Ruby.

Here are two useful resources where you can learn more about these and successful automation approaches.

* The Secret Ninja Cucumber Scrolls (http://cuke4ninja.com/toc.html)

* Ideas for efficient BDD with SpecFlow through examples (http://ndcoslo.oktaset.com/t-4861) a very recent presentation, from NDC 2012, where the SpecFlow author goes through a couple of testing approaches.

Learn about the "Page Object" model approach to automated web testing. It's important for creating reusable, encapsulated tests that can evolve with the system under test. This is vital for advancing past the initial, small automated tests to much larger suites whilst still keeping them maintainable.

I tend to believe that QA should focus equally on function and experience. Consequently, I'd:

1. Unit test both front and back end code. This thread is full of great examples for tools that will make that process easier.

2. Use a tool (I like jmeter) to frequently test how your project performs. You would be surprised how often small changes result in massive slowdowns, especially under load.

3. Bring basic usability testing into the QA fold. Web dev is extraordinarily client facing - projects live and die according to the whims of clients. Consequently, if you bring things like simple hallway testing into the development flow, your whole team will recognize how important it is to make things elegant and simple to use.

I like to think of test coverage as a grid.

On the horizontal layers you have unit testing, integration testing, acceptance testing - which are tests at various levels of abstraction throughout your code base.

I then like to test the vertical slices tiers of the application - usually views, controllers, models, database etc. I like to see unit tests, integration tests, and 'acceptance tests' focusing in on each of those tiers to build confidence in the coverage of your testing.

Automation is absolutely key, and getting those automated tests into your continuous build and release process is definitely worthwhile.

For backend testing, I like to write tests using nose in python, using the awesome Requests library to drive the web interaction. Because Requests handles cookies well (for whole sessions), and allows all http methods, and has great header, file, post params, etc support, it really makes driving webapps simple. A couple of functions to get a session into appropriate state (logged in with user or whatnot) and you have easy setup fixtures -- so you can try things in isolation quite well.

Interesting article about unit testing as it relates to API development: http://blog.seatsync.com/2012/06/unit-testing-as-path-of-lea...

Unit tests of lower level back-end functionality is always a good idea, but I find it actually makes API development simpler because it frees you from the problems and complications inherent in testing via your target client.

Disclosure: I wrote the article.

One of the problems I really wanted to tackle was ensure consistency in web rendering across browsers and over time. Basically, automating the process of "does this page look right?" That culminated in my startup (Mogotest) and the concept of Web Consistency Testing. Neither are right for everyone, but check them out. Feedback is always appreciated, too, even if it's about how I use Bootstrap :-)

Most agencies that size do very little testing. Typically any testing is manual and done by a mix of developers and account managers. Not saying that you have to operate that way, but be aware that automated testing is a relatively foreign concept for most agencies.

I try to write unit tests for internally developed libraries. For some projects, Selenium tests will be written as well.

Drupal has its own version of SimpleTest for testing code:

http://drupal.org/simpletest/

Core ships with a lot of tests and most of the larger contributed modules do too. They're pretty helpful to run frequently to make sure you haven't broken anything but they won't test your configuration or any of your front end stuff.

I'm more concerned with testing less commonly used modules, and the front end. I'm newer to the web dev word, but not new to SW development and testing.

Take a look at Adobe Shadow (good writeup here: http://www.webmonkey.com/2012/03/adobe-shadow-simplifies-mob...)

It'll let you refresh changes on all your devices at once.

For JavaScript unit testing I have been pleased with Siesta so far (http://bryntum.com/products/siesta/). The standard version can be automated using Selenium or PhantomJS.

Individuals and interactions over processes and tools. Working software over comprehensive documentation. Customer collaboration over contract negotiation. Responding to change over following a plan.

Learn to use the dev tools in Chrome.

Here's a great start: http://jtaby.com/2012/04/23/modern-web-development-part-1.ht...

Just wanted to thank everyone for their input.

So from what I gather, eveyrone is a fan of selenium. I do recall it being brought up during my interviews.

Does anyone have a favorite resource they like? i.e: eBook, blog etc

When you type the words "web design agency", it makes me think that the right answer is "do not test". The projects more often than not are just not big enough to justify it.

A friend of mine introduced me to CasperJS and I love it. It's built on top of PhantomJS.

Reply on news.ycombinator.com