Multi-variate testing ready to burst forth....oh reeeaallly!?

Reading this just now and I'm subsequently bracing myself for a spate of useless statistical analysis from the field of Web analytics. My experience with the application of multi-variate testing goes back a decade and includes the fields of Statistics, archaeology, marketing and more recently, information architecture. Time and time again I see multi-variate testing wasted through a complete lack of multi-variate analysis.

Folks, it isn't enough to calculate the mean of several variables and pat yourself on the back for your multi-dimensional approach to research. Unless you're going to perform analysis that creates a correlation between variables you are wasting your time. Even something as simple as cross-tabulation will provide you with insights not available through standard summary statistics on a single variable - despite calculating them for a series of variables.

For example:
Out of 100 users...
65 found the interface easy to understand, 25 found it confusing, 10 found in frustrating
50 were able to locate the information they required, 35 were unable to find the information, 15 found the information but didn't recognise it.

So, does that mean that a majority of users find the interface easy to understand and were able to locate the information they required?...

What if I told you that, of the 50 users able to locate their information, 25 of them were the ones that found the interface confusing? How about if, of the 65 that found the interface easy to understand, 35 of them were unable to locate their information?

Anyway, it bugs the bejeesus out of me when I see this sort of thing.

And if you're thinking this guy may not be representative of the standard within the Web analytics fraternity, pick up a book - any book - on the subject, and I challenge you to locate the analysis that goes beyond this style of simplistic, superficial level.

If you find one let me know. I'll even buy a copy.