In the next sections we will introduce the main approaches that are used in remote web usability evaluation or that can be found in the literature.
We will refer to these approaches as "client application", "web site" and
"proxy", according to the definition introduced by Baravalle and Lanfranchi (2003) in order to categorize them.
7.3.1 Client Application
The web browser approach is based on the development of a specific client side application that will log the user while is surfing the web (see Figure 4).
We can distinguish two main kinds of client applications:
• general purpose logging applications;
• web logging applications based on web browser customisation.
Remote Web Usability Testing
Figure 4 – Client application approach
General purpose applications are used to monitor the user behaviour, and provide a record of the user experience and/or quantitative data about the experience, which can be used to evaluate the user experience. General purpose applications usually are able to provide to the researcher a movie of the user interaction and/or quantitative data about the experience, as log of mouse movements, page visits and time used for operations.
Less commonly, applications can require the client user to have a web cam; in this case it can be recorded not only the behaviour but even the user expressions.
Clearly, applications providing a video record of user interaction can be very useful for research activity on user behaviour.
Advantages of this approach include the possibility to see exactly user actions. Disadvantages of the approach include that researchers must review all the tests to process the data and, like in standard tests, this can be time consuming. Beside that, these kinds of applications (Noldus Observer to name one of the most used) have usually high costs and due to their requirements are usually used in an usability laboratory.
Remote Web Usability Testing
Ad hoc client applications are commonly built upon a web browser that can be customized or developed from a browser whose source code is available.
The first approach was followed by Usable Tools usability browser (2001), that is built upon Microsoft Internet Explorer and works in conjunction with a logger, that stores and processes the data that the browser is transparently sending. Usable Tools stopped the development of the project at the end of year 2002.
The second approach is followed by Uzilla, that developed a custom version of Mozilla4 with analogue functions.
The main disadvantage of these two approaches is that the users are forced to install and/or to use a particular client, some hardware and/or software platform.
In fact, it the test is performed in an usability laboratory, users will attend to the test with an environment that is partially (or totally) different from the environment they are used to.
Regarding the software needed for these tests, we have to at least consider the operating system, the web browser and installed plugins;
regarding the hardware we have to at least consider the system technology and the performance of the system configuration.
These variables can affect the kind of experience of an user browsing a web site, and are not considered in a laboratory environment.
If, instead, the test is performed remotely, users can be tested in their environment, but there are some drawbacks. Users will have to install the logging application by themselves, which is not always a fast and easy task; moreover, if a custom browser is used, users are again not tested
4 Mozilla is an open source web browser based on the same engine of Netscape Navigator.
Remote Web Usability Testing
with their usual environment. While most of Internet users are browsing using some version of Internet Explorer (95%)5, only 62% of users use the most common browser, Internet Explorer 6. It means that there is not at all a common environment shared by web users.
7.3.2 Web Site
The web site approach is based on a web application which is running in the web site under investigation and that monitors the user behaviour (see Figure 5).
Figure 5 – Web site approach
The application will be responsible for logging the user behaviour while he is browsing the site.
First of all, it is important to mention that web site logging is possible only if a server side logging application is available. Analysing the technology, we can distinguish two different alternatives:
• local server side application;
• remote server side application.
For local server side application we intend an application that is running in the web site under investigation. This kind of application requires the web
5 Statistics from Browser News web site, http://www.upsdell.com/BrowserNews/stat.htm, retrieved 18, August 2003.
Remote Web Usability Testing
site under investigation to support the technology used by the system. As today, several technologies can be used to develop this kind of system, but most of the web servers do not support or at least do not support properly all the technologies. Adopting this kind of system assumes that the site under investigation is supporting the server side technology used by the system.
For remote server side application we intend an application which it is not installed in the local server. The web site under investigation is modified in order to collect data and contact the remote server, that is responsible for storing and eventually processing the data. If the web site modifications include only client side code (ECMAScript code is in fact the only alternative) the researcher can use the system with any kind of web site, overcoming the problems described earlier. The researcher needs only to modify the pages that need to be tested, and these pages will interact with the remote application, that stores the data. Modifying all the pages is not difficult using server side includes (SSI) technology, that is available in most server side programming languages. This kind of approach is adopted, among the others, by Paternò, Paganelli and Santoro (2001).
The drawback of this approach is that requires a full access to the web site code and so it can be used only in few cases, as most of the web sites will not allow external researchers to modify their source code. While the approach is very interesting for testing web sites with the cooperation of the site maintainers, it can not be used in different circumstances.
7.3.3 Proxy
The proxy approach is based on an application that lies in between the client and the server and is responsible of monitoring the navigation (see Figure 6).
Remote Web Usability Testing
Figure 6 – Proxy approach
The main advantage of this approach is that the testing users do not need to install anything on their computer and that the researcher does not need to have access to the web site code.
Our research aims to contribute to these issues by developing a software system, called OpenWebSurvey, based on the proxy approach, able to record, store, share and process data for web usability analysis.
OpenWebSurvey shares the same proxy approach of WebQuilt (Heer, Hong, Waterson, & Landay, 2001), but the two implementations are based on different technologies and they have different goals. OpenWebSurvey gathers data about user environment and characteristics (e.g., browser name, browser version, operating system, language), allows user identification through a login process and, in a future perspective, aims to collect also client-side data (e.g., mouse movements, mouse clicks).
OpenWebSurvey allows researchers to use its server installation, allowing also non-skilled users to undergo an usability test, without having to install the proxy on a dedicated server.
An alpha release of OpenWebSurvey is being tested at Turin University, Department of Computer Science. Chapter VIII will discuss OpenWebSurvey implementation in detail.