Jeffrey Greenberg on Technology Strategy

Testing javascript heavy, web frontends remains a hard problem in software development. Everyone wants to do it in order to be agile, but few if none can really do an ideal job of it. Some key reasons they want to automate are:

• allow developers to quickly smoke test their work before they commit it
• support extensive, comprehensive functional testing

The reasons it’s hard to do are twofold 1. the frontend in a new product development is always in flux, and 2. there’s no great tool for quickly creating robust test scripts that can be automated. In particular, while there are some recorders for the gestures on the frontend, they usually fail in certain ways that demand one to further generate custom test scripts for that tool to run. It’s extra work that often becomes outdated on the very next build.

Generally, good agile practice demands lots of test code to support quick and reliable bug finding and fixing early in the development cycle, and not sometime later. So good teams will make some compromises to their agile test strategy to get as close as is reasonable to ideal without it becoming non-agile. The typical strategies are:

• create unit tests for all infrastructure, including going up into the javascript client as far as possible, but not automating user interaction
• create testable javacript widgets (look and feel) that can be instantiated into test pages that can be quickly looked at by a developer or tester

Some tools that help automated javascript and RIA testing are:

• http://windmill.osafoundation.org/trac/wiki/BookChapter-1-Install
• http://windmill.osafoundation.org/trac/wiki/BookChapter-4-3-JavascriptTests

• Provides multi-platform testing and recording
• Downside: recorder doesn’t handle all circumstances so programming is required, and test scripts will end up with dependencies on DOM structure

• Dojo is a javascript library with testable widgets, and a testing infrastructure for scripts
• Provides an experimental testing infrastructure, DOH, with a recorder

All three provide recorders. though Selenium is probably the most mature. They all suffer from issues in recording where the auto-generated scripts can often fail, require knowledge of XPATH to sort things out, and are not easy to incorporate into development processes so these scripts are maintained easy.

The key thing here is that it’s tough to incorporate these tools, which are themselves less than ideal, into workable processes that developers want to do and that support the flexibilty product designers need on the frontend. The upside is that if a working compromise can be found, the development will ultimately go faster & cheaper, because issues can be found sooner.

The long and the short of it is that if you want to do an agile process, it takes time and planning but the results can pay off. In the end, simplicity on the frontend pays off not only with users (typically) but also in development costs and time.