Thursday, June 29, 2006

Oz-IA Retreat coming together

The organisation of the Oz-IA retreat being held in the first weekend in October is coming along nicely. The venue (Mercure Hotel just adjacent to Central Railway Station in Sydney, Australia) has been lined up; the speaker list is taking shape; and the level of interest is building.

The retreat is planned to be a semi-formal series of practical sessions on information architecture and related topics, aimed at practicing IA's and those within the broader industry interested in expanding their knowledge of the theory and practice. There'll be case study presentations; detailed how-to sessions; and general 'where are we going' discussions. And there'll be opportunities a-plenty to meet and mingle with peers and uber-IAs from Australia and around the world.

More information will be forthcoming in the next few weeks & months, but pencil in those dates (conveniently following on from this year's Web Directions conference). It should be a cracker!

Wednesday, June 28, 2006

Senior managers shouldn't care about the Web

Just reading through Gerry McGovern's piece titled "Senior managers: you can't keep ignoring the Web" and, like many of Gerry's articles, I agree with the practical up-shot of the argument, but not the basic premise. That is, whilst I would agree that, in practice, a senior management team should include at least one executive whose responsibility it is to oversee the direction and operation of the company's Web presence(s), I disagree that they should be doing so because of some inherent special quality of the Web.

A senior management post in an organisation of any size should be driven by the desire to realise the strategic objectives of the firm. Typically, achieving these objectives will require activities that are well suited to the Web. I say 'typically' because this is not always the case. And so a senior manager who spends energy on a Web presence where that presence isn't directly contributing to the achievement of those strategic objectives is wasting their time, resources, and potentially damaging the performance of the company instead of helping it.

Senior managers - actually, anyone for that matter - can't afford to be enarmoured of a technology to the point where they blindly implement initiatives without regard for actual benefit. This is relevant for the Web just as much as it is for an IT project, or a TV campaign, or a product release. They must retain their focus on the strategic objectives of the company and be open-minded enough to be able to select, implement, and operate the best (effective, efficient) initiatives towards those goals. In practical terms this will often include some form of Web presence.

As a side note, I think the history lesson in the conceptual framework of corporate Web sites is now fairly out of date. New companies no longer implement organisation-centric Web sites with anywhere near the prevalence that we saw 5-10 years ago. Instead, marketing teams and Web agencies are embracing customer-centric philosophies and representing themselves accordingly; and providing services that are similarly centred on the needs of the target customers. Sadly, the organisation-centric Web presence lives on, and probably will do for some time, but there is a definite shift towards customer-centricity occurring throughout the business world.

Monday, June 26, 2006

Jakob's latest Alertbox - How many users to test?

Jakob Nielsen's latest Alertbox, titled "Quantitative Studies: How Many Users to Test?" looks at the improvement in confidence intervals and margins of error to be had through increasing the number of users tested when measuring usability metrics. [Note: this is in contrast to his recommendation to test 5 users when looking qualitatively at usability.]

The article draws on large numbers of usability tests carried out by the NNg and provides some interesting points of note:
i) Usability metrics tend to follow a Normal (or Guassian) distribution - which makes the statistical analysis that much more convenient;
ii) User time-on-task performance tends to show a standard deviation of 52% of the mean;
iii) 20 users offers a reasonable test sample size for most usability metrics.

A couple of counter-points worth keeping in mind:
i) Although the finding that usability metrics tend to follow a Normal distribution is useful, most statistical analysis techniques include methods whereby the Normal distribution is not a requirement. This is particularly the case when performing quantitative analysis on non-parametric data (e.g. ranks, categorisations or counts);
ii) Always calculate the standard deviation and margin of error based on the data that you've collected. Whilst NNg's insight provides a useful starting point for deciding the number of test subjects, you need to go through the process of calculation sd and e for your data set;
iii) NNg use a 90% confidence interval as their baseline for determining the recommended number of test subjects: sometimes this level of confidence is insufficient, and so a greater number of test subjects would be required. (I tend to use 95% CI myself, particularly if the implementation cost is high.)
iv) Confidence intervals form part of the general set of summary statistics about a data set (along with mean, variance etc). They describe a characteristic of a particular sample, which allows some inference as to the nature of the general population - they don't provide a comparison between two populations. For example, is a time-on-task CI of 3 mins +/- 30secs better or worse than a time-on-task CI of 3:10mins +/- 15 secs?

Finally, this article feels like a response to the JUS article cited here previously. In particular, a response to Lewis and Sauro's references to use of 5 test subjects for usability studies, which may itself have been influenced by a previous Alertbox article.

Monday, June 19, 2006

Statistics without tears

That's the title of an interesting introductory book on Statistics that I'm reading through at the moment. Written by Derek Rowntree and published by Penguin (ISBN: 0-14-013632-0), the book provides an introduction to the theory and practice of statistical analysis without (much) recourse to formulae, calculations, charts, graphs or tables of figures.

Aimed at people that have to deal with statistics and statistical analysis, but who could go their lives seeing without a Gaussian distribution and not view it a loss, the 184-page text takes the reader on a fairly painless journey through descriptive and inferential statistcs; their meaning, use, and calculation.

If you've ever found yourself struggling to make headway into the topic of statistics then this book may offer you an olive branch.

Alternate titles for the book were: Statistics without calculations; Statistics for the innumerate; Statistics in words & pictures; The underlying ideas of statistcs; or How to think statistically.

PS: I'm reading this in preparation for a presentation on statistical analysis of usability and user research data to a largely non-mathematical audience later this year.

Tuesday, June 13, 2006

Australian football comes of age?


I spent my childhood surrounded by kids of European and South American backgrounds, being taunted by them about Australia's (and, by implication, Australians') complete absence from the soccer World Cup. Whenever it rolled around the Italians, Croatians, Brazilians, Uruguayans, Chileans etc would cheer on their national sides, and laugh at our failure (again) to even qualify.

Earlier this morning Australia's Socceroos played their first World Cup game since '74. Behind for a good portion of the match they put in a tremendous effort in the last 20 mins, finally scoring in the 84th minute of the game. A second goal in the 89th minute put us in front; and a sealer in the 92nd minute made it comfortable.

It may be an overstatement to say we've come of age, but we've certainly proven we belong.

Monday, June 12, 2006

Very OT: Socceroos World Cup appearance

A little bit excited in the lead-up to the Socceroos first appearance in a football World Cup since 1974. The Aussies take the field in about 90mins to meet Japan.

In their last appearance, also in Germany, the Socceroos failed to win a game - or score a goal - so I'm hopeful we'll see a much improved performance this time around.

Anyway, we'll know in about three hours whether this World Cup will be a different story.