Just reading through an article on bucket testing over at Luke Wroblewski's blog - Functioning Form (thanks to Pabini @ UXmatters for the link). Bucket testing is the process wherein you test two versions of the same thing in parallel (page, form, design element etc) by channeling part of your site traffic through one version or the other and analysing the results (note).
Luke raises the issue that, with advances in the ease with which bucket testing can be undertaken, organisations are performing tests on increasingly isolated design elements. For example, the use of text colours in particular areas of a page. However, looking at the results of a very granular test, and then adjusting the design accordingly, does not result in an optimised design.
Any student of mathematics, finance, economics etc will tell you that the 'optimal' solution to a set of equations or conditions is rarely the combination of the optimal solution of each equation. This carries directly into the design of a page or form or screen of a Web site/application. It is invariably pointless carrying out a test only on a specific element and then adjusting the design based on those results without also verifying that the change produces a more optimal solution to the whole.
Luke puts it like this:
"A cohesive integration and layout of all the elements within an interface design is what enables effective communication with end users." [emphasis mine]
In other words (not that he needs my help), the optimal solution to a design problem is not the stitching together of individually-optimised components.
Note: I'm in the process of writing a post/article on how you can carry out this style of analysis with some degree of statistical rigour.
Posts
1 comment:
Lots of dot-com businesses do this all the time. I've seen it called A-B testing. I once had (or am I imagining this?) a client who had up to 20 different home page designs in circulation at once, and they'd tweak the mix of them in reaction to the site's monthly traffic, sales, and conversion numbers. Much in the way a investment portfolio manager will tweak the composition and balance between aggressive and safer investments, the site's managers would tweak the randomization to serve up the right combination to generate the most money.
Post a Comment