Another Use for Browser Event Capturing Technology
Judy Cantor, jcantor@att.com
Michael Etgen, etgen@att.com
AT&T
Introduction
As usability experts, we are always trying to find ways to demonstrate the benefits of usability engineering. Especially in industry, design and more traditional engineering are funded far more often than usability evaluation and iteration. Usability experts moan and groan that time, money, and other considerations block our ability to do our jobs properly, especially in web time.
We are starting to take a vastly different approach. Rather than accepting defeat at the hands of time and money constraints, we are beginning to identify techniques and create tools that can be used to assess usability in the fast paced limited funds environment. At Human Factors and the Web ‘99, we reported on a technology that currently exists in most browsers. The way that browsers display information is through "events" and these events can be captured and recorded. For our purposes as usability experts, these events include virtually any interaction that a user can have with the content of a page. By creating a little JavaScript tool, events like Mouseovers, clicked links, menu pull downs, double clicks, etc. can all be recorded and categorized by page name. When wrapped in a structured usability test, this technique can provide a rich source of data in a virtual usability test that is quick and easy. (We will demonstrate this technique live at an upcoming 1999 HFES ITG Demo session.)
In this article, we report on another interesting use of browser events. We were asked to work on a web application that was difficult to use. We recommended and implemented an entirely new User Interface design. The redesign was recommended in order to provide more usable information about how to navigate through the application, to better exploit the UI real estate, and to more closely resemble one of a family of web services. When the initial design was reviewed, one of the concerns was that the resulting UI might negatively impact the download time of the Home Page. We were addingThe new design incorporated additional links, additional text, and graphics. To meet the download time requirements and still produce a UI that improved usability, we worked on simplifying the art work. All graphics were reduced to a single color, all dithering was removed, and some graphics were replaced with smaller, quicker-to-download versions. We avoided fancy graphics, animations, and the like, and focused on designing a UI that was functional, easier to use, but still attractive.
Soon after the new design was implemented, we began to get reports that the new version seemed to take longer than the original, and the blame was falling squarely in our laps. These reports were based on subjective impressions rather than objective data., Nnonetheless, they were from engineers who were familiar with the different application versions, and whose concerns were taken very seriously.
So we designed a simple experiment using browser events. The goal was to test whether the change in UI design itself, negatively impacted the download time of the home pages. One important problem was that there were a lot of differences between the 2 versions of the application. The newer version included new features and a re-architecting of some of the back end, all of which can impact the speed of loading a page. In order to determine whether the UI produced download time differences, it had to be disentangled from the backend processes and servers. If, once we make the Home Pages "stand alone", we find download differences between versions, then those differences could be attributed to the UI.
Experimental DesignThe home pages for the 2 versions were downloaded to a local PC client, and each was detached from it’s servers (in the same way). The goal was to identify how long it takes to load each page and compare them, independent of back end processing and server related issues. All trials were run on Netscape 4.05.
Download Time Calculations
The download time calculations were reflectproduced by "onUnload()" and "onLoad()" browser event handlers. This application had a login page before the home page was loaded. We recorded 2 timestamps. The first was when the login page (all components) "UNLOADED". The second was when the home page "LOADED" (all components). The time between T1 and T2 is the time it takes the home page to download. All calculations were in milliseconds as provided by the client machine system clock. Trials were run under a couple of conditions to more closely resemble the actual user experiences.
To encourage first-time visitors to stay at your site, include information or services that are helpful to the visitor. You can do this by creating a user profile and performing a needs analysis and a task analysis. Provide simple, attractive graphics, short download times (Nielsen, 1999), and obvious, straightforward navigation (Fischler, 1998). To speed downloads, don't use animations. Limit your use of graphics. Keep the total file size of a screen to 20 kilobytes or less (Sullivan, 1998).
Experiment 1: Cache Cleared each Trial
In this case, each time a trial was run, the cache was first cleared. With regard to the user experience, this is similar analogous to when the home page is loaded for the first time. It is not however, the best user model. Under most circumstances, after the very first time that TM is loaded, there will be information in cache.
Experiment 2: Cache Not Cleared
This experiment most closely resembles the average user experience. The cache is allowed to accumulate. Netscape is re-opened for each trial, but the cache is not cleared. The very first trial is eliminated from the calculations since it would reflect the same information as clearing the cache.
Results
There were 30 trials in each experiment, 15 for each version of the application.
Experiment 1: Cleared Cache
On average, the difference between versions was less than 2 tenths of a one second, and was well below what would be expected for human perceptibility. Statically, the variation between these versions is not more than what would be expected by chance, t(28) = 1.1, p> .05.
Experiment 2: Cache Not Cleared
This experiment showed results that were similar to experiment 1. The average difference in download times between versions was 183.33ms, the variation between which is not more than what would be expected by chance, t(28) = .ooo70007, p> .05.
Summary
The experiments, completed in one day, supported our assumption that the newly embellished UI did not impact page download time. Improvements in other sorts of usability will have to be tested with a usability evaluation on human performance. New features, changes in back-end processes and server traffic could all delay the download of pages, and it will be difficult to target any actual cause. Of course, it is first necessary to establish that such differences exist at all, a problem that we will leave to the system performance testers.
We again demonstrate the benefits of using new or existing technologies to collect data that is quick and easy, and clearly beneficial data on the user experience. Such techniques do not impact development time or cost much money, can potentially help the user, and improve our own credibility as user experience experts.
© Internet Technical Group
Last update: August 18, 1999
URL: http://www.sandia.gov/itg/newsletter/jun99/event_capturing.html
hosted by Sandia National Labs
Disclaimer: Neither Sandia Corporation, the United States Government, nor any agency thereof, nor any of their employees makes any warranty, express or implied, or assumes any legal liability or responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately-owned rights. Reference herein to any specific commercial product, process, or service by trade name, trademark, manufacturer, or otherwise does not necessarily constitute or imply its endorsement, recommendation, or favoring by Sandia Corporation, the United States Government, or any agency thereof. The views and opinions expressed herein do not necessarily state or reflect those of Sandia Corporation, the United States Government or any agency thereof.