The role of 'value' in Web site design

The following three-part series of articles was published on the Australian edition of ZDNet's Builder site in February, 2004. These articles have since been syndicated internationally, to the UK and China (in Chinese).

Some of the concepts - and particularly the way they've been articulated - lack polish. However, the issue of business value of IA continues to attract interest, so I thought my own site should at least include a reference.

'Value-based approach to Web site design'

Part I
Part II
Part III

Feedback, comments on this are most welcome.

A statistical sidebar

The project from early 2005 that got me back into the saddle, statistically speaking, involved a large amount of methodological research, data collection, analysis and report writing. It's the single largest client project I've undertaken and resulted in a 20,000-word report.

The nice part of it was the range of analysis techniques I was able to employ, from simple descriptive and summary statistics (means, standard deviations, counts etc) to cross-tabulations and on to multi-variate analysis using k-means clustering, principal components and MANOVA tests.

To be honest I think the client on the receiving end of the report was a little overwhelmed by that part of the document, which is why I had buried it in an appendix.

Anyway, it blew the cobwebs off my statistical calculator, textbooks and probability tables!!

A little statistics with your Web site

I've recently re-discovered my fascination with things mathematical and particularly those statistical. I have a degree in Applied Mathematics, majoring in applied statistics, so it's not idle tinkering. But after graduating in '94 I've tended off towards more humanistic and social topics of study - archaeology, electronic commerce (masters degree in 2001) and business (MBA in 2004).

A project I undertook earlier this year (I'll write about the work in a later post) got me back in touch with my numerical side. That interest seems to have stuck for the time being, and I've just about completed a new project that carries on the theme.

Most people will be familiar with the idea of comparing two things to see whether they've changed. On a Web site we might want to see the effect that a change has had on the number of page views and so we count up the page views before the change (or calculate an average) and count them up after the change (or calculate the average) and compare the two. If one's higher than the other then we agree that there's been a change and congratulate ourselves.

You'll see it all the time. "Last week saw an 7% increase in traffic to the Web site" or "Today's sales are up 3% on yesterdays'".

The problem with this type of comparison is that it ignores the fact that almost every natural process has some random fluctuation to it. If we fail to take that fluctuation into account then we can mistakenly assign a causal relationship when in reality the observed difference is completely random. For example, average daily page views might vary by as much as 6%. So a 7% increase for the week isn't very special at all. In fact, it's hardly even noteworthy.

So, for the past few days I've been working with one of our Web application developers to put together a simple statistical test for 'significant change' in one of our clients' Web sites. This client spends a fair amount on advertising as a means of driving Web traffic, so I figured it would be useful for them to be able to determine - correctly - whether or not the advertising campaign was having a real impact.

The test we're using is a standard Mann-Whitney rank sum significance test. Since I can expect to have more than 20 sample points in either the "pre-" or the "post" sample, we've set the test up to use the approximation to the normal distribution.

There's a very nice lecture/book summary on the statistical theory available here: http://faculty.vassar.edu/lowry/ch11a.html

The analysis program is being built into the administration system (content management etc) for the Web site, and we should be able to start carrying out analysis in the next few weeks (we need to collect some data first).

I'm feeling a little tickled at the idea of bringing statistical rigour to a Web site; it can only help (I think) make the work we do more seriously considered.