4 seconds - Part II

OK, Akamai seem to have recovered from their little glitch & I now have a copy of the report. The detail of the report paints a very different picture from what's being reported in the press release and the media. This should serve as a lesson for you all not to rely on the media for your research - track down the primary source for the article and read it (carefully) for yourself.

Some questions I have for the report authors, journalists & PR people at the various companies:

• on the headline finding of the report - that "Four seconds is the maximum length of time an average online shopper will wait for a Web page to load before potentially abandoning a retail site." - the report data indicates a very different picture: 80% of dial-up users, and 68% of broadband users will wait longer than 4 seconds before leaving a Web site. Looking at the detailed data from the report, I can't see any way to arrive at an 'average' measure of 4 seconds. Neither the mean, mode or median values for the data come out at 4 seconds. The lowest figure I can arrive at is at least 5 seconds, and possibly quite a bit higher.
• Broadband users start to consider abandonment after less than 1 second in some cases (1% of broadband respondents). Dial-up users show a little more patience, starting to abandon the site after 1-2 secs of waiting (3%). Wouldn't this be a better way to report the findings? "Online shoppings start abandoning sites after 1 second of waiting"
  
• From the original report: "Roughly 75% of online shoppers who experience a site freezing or crashing, that is too slow to render, or that involves a convoluted checkout process would no longer buy from that site". The respondents were asked to indicate whether they would be 'less likely to buy from the retailer again online' - not whether they definitely would not. This is a significant difference in interpretation of the data, and leads to the sort of attention-seeking articles shown below.
  

How the article was reported elsewhere:
Slashdot: "Of course we all want webpages to load as fast as possible, but now research has finally shown it: four seconds loading time is the maximum threshold for websurfers. Akamai and JupiterResearch have conducted a study among 1,000 online shoppers and have found, among other results, that one third of respondents have, at one point, left a shopping website because of the overall 'poor experience.' 75% of them do not intend ever to come back to this website again. Online shopper loyalty also increases as loading time of webpages decreases. Will this study finally show developers of shopping websites the importance of the performance of their websites?"

What's wrong here:

• four seconds is not the maximum threshold for websurfers. 80% of dial-up & 68% of broadband users indicated they would typically wait longer than this for a page to load;
• 75% of respondents indicated they would be less likely to return to the site; not that they had no intention of returning;
• The report does not correlate an increase in online shopper loyalty to a decrease in webpage loading time: it indicated the converse (slower loading time correlates to decreased loyalty), which is not necessarily the same thing.
  

InformationWeek: "The survey found that more than one-third of online shoppers abandoned sites entirely whenever they suffered a poor experience. Some 75% of the online shoppers polled said they wouldn't be likely to use the sites in question after they had a poor shopping experience."

What's wrong:

• More than one third of dissatisfied online shoppers who also abandoned a site did so due to load times, errors or crashes. The percentage of abandonments due to all sources of dissatisfaction ('poor experience') was not reported, but is also presumably higher than one third.
  
• Again with the 75% of online shoppers (see above)

Sydney Morning Herald: "According to a new report on consumer behaviour, four seconds is the longest that online shoppers are prepared to wait for a site to load before backing out of the transaction."

What's wrong:

• Again, 68% of broadband and 80% of dial-up users indicated in their survey response they would wait longer than 4 seconds for a page to load;
• Slow loading times was a source of dissatisfaction for 33% of respondents;
• 18% indicated they had abandoned a transaction due to the slow page loading on the site.

Somewhat interestingly, the biggest recorded factor affecting the likelihood that a dissatisfied online shopper would also shop with that retailer off-line was the convoluted or confusing checkout process.

4 Seconds - part I

I've just read an article on the Sydney Morning Herald site claiming: "According to a new report on consumer behaviour, four seconds is the longest that online shoppers are prepared to wait for a site to load before backing out of the transaction."

This is followed about two sentences later by: "It found that the average shopper will abandon an online store if forced to wait more than four seconds for pages to load."

So, either 4 seconds is the longest time shoppers are willing to wait - representing the extreme upper fringe; or it's what the 'average' user is prepared to wait before abandoning a process. Since these are two very different things, I'm currently trying to get my hands on a copy of the full report available from the report's sponsor Akamai (they're in the business of providing 'Content delivery, Application Performance Management, and Streaming Media Services').

After filling in the obligatory form on the Akamai site so that I can lay hands on the free copy of the report, I'm currently just getting the following:

"Service Unavailable - DNS failure
The server is temporarily unable to service your request. Please try again later.

Reference #11.95088790.1163111838.a9fda45"

I'll provide more information on this whole saga when Akamai corrects this little fault and gives me access to the report.

User research: subjectivity and objectivity in practice

My latest article - and the first article in my new column - went live at UXMatters yesterday. Have a read of the article and let me know what you think.

Since Oz-IA and other recent events...

Since the Oz IA retreat things have been rather hectic around here. We're in the final stages of developing a major Web-based business application that utilises AJAX throughout the interface to provide a more responsive experience for the users. My work on the project commenced in March 2004 with a high-level conceptual architecture and has progressed through various stages since then. Four more weeks should hopefully see us finished with the main body of the application, leaving integration into other systems as the last stage.

I've also been busily working on writing a column for UXMatters, which I hope you'll see in the near, near future. The column's about user research generally and starts with a review of some recent debate about the merits of user research; I'm hoping it isn't too abstract for the readers.

A couple of recent projects have finished off well, and been well-received. Our site for Tourism Queensland - Queensland Holidays - won an award recently, for which we were all pretty happy.

On a more personal note, the summer is almost upon us and with it a new season of international cricket!! I'm looking forward to the Ashes series, and if you look carefully you'll see me parked on a seat in the Ladies Stand for the duration.

I'm not sure how many of you may have attended my presentation at the Oz-IA conference, but the feedback I've received so far has been largely positive, including this from Zef in NZ who says I hurt his brain - but in a good way.

More soon...