Questionnaires are written lists of questions that the researcher distributes to test users. Questionnaires differ from surveys in that they are written lists, not ad hoc interviews, and require the users to fill out the questionnaire and return it back.
This technique can be used at any stage of development, depending on the questions that are asked in the questionnaire. Often, questionnaires
Web Usability Inquiry
are used after products are shipped to assess customer satisfaction with the web site. Such questionnaires often identify usability issues that should have been caught in-house before the web site was released.
Questionnaires are often submitted remotely: they can be used to gather user answers and personal data without effort, or they can be used in conjunction with remote web usability testing tools, as we will describe in Chapter IX.
Web Usability Testing
C h a p t e r I V
Web Usability Testing
Over the last few years the number of web sites and of Internet users has increased: Internet has become a mass medium.
Actually the Internet is plenty of web sites, some created for information purposes, others for marketing purposes, but all them need to attract users and distribute information or products.
The risk of building a non usable web site is the risk of losing users, with all the economical and ethical consequences that this may convey.
Consequently one of the main issues in creating a web site is how to make it usable for every typology of user. And, in general, the Internet users are characterized by different socio-cultural backgrounds and skills.
The methodologies proposed for web usability testing are strictly related to HCI and experimental psychology principles, and derive from approaches to traditional usability that can be found in the literature.
All these approaches share the same methodology structure, that can be split in different phases:
• define the testing purposes;
• define the objectives of the test;
• define the profile of test users;
• define the experimental design;
• define the tasks that the users should perform;
• specify the test apparatus;
• identify the required personnel;
• prepare the subject for the test.
Web Usability Testing
First of all, it is necessary to know clearly which are the goals of the research, and to be able to translate them into a specification of objectives for the test.
Defining the testing purposes implies understanding which are the characteristics required for the web site and which are its functional requirements. Then it is possible to define in detail the goals of the research: a broad goal like "is this web site usable?" is not enough. The goals should be focused on the topics and issues that are relevant for the project.
To achieve this, it is very important to identify the target of the users to test, according to the testing purposes. The users can be novices, or experts, male or female, young or old, or belonging to different socio-cultural or ethnics groups.
One of the most debated issues in usability testing is the number of users to test to have relevant results, but this issue will be analysed in Chapter V.
The fourth phase regards how the test will be structured and conducted to eliminate non-interesting variables from the analysis.
Then there should be an accurate phase of design of the test, in which the researchers specify the tasks that the subjects will be asked to perform during the test. The definition of the tasks should be made considering which are the tasks the users normally perform when they are using the web site and which are the testing purposes. A task should be defined in terms of:
• actions to be performed;
• meaning of a completed task;
• meaning of a non-completed task.
Web Usability Testing
The test scenario should also be considered, setting up all the instruments and aids that the subjects will need, paying attention to not distract or influence the users with instruments different from the usual ones. For example, using a "cool", new browser instead of a common one could not be a good idea.
According to the selected approach it is also important to decide how many and which kind of personnel is needed. If the test is a laboratory experiment, maybe a test administrator and one or more observer might be necessary as well.
Selecting the sample user between the population is a very difficult task, since it requires to decide the number of users necessary to represent the population with the correct mix of experience, skills, and demographic characteristics.
It is possible to recruit the subjects from fellow employees, from family and friends, to enlist temporary employment agencies and market research firms to get people, to put out an ad on the Internet, or in newspapers, to contact user groups and industry organizations. Of course it is easier to find the subjects if the target users belong to common user groups, while it could be harder to find particular user populations, such as blind persons belonging to a particular social cultural and ethnic group.
The last phase is very important as it regards the preparation of the subjects for the test. The subjects should be set at ease before performing the test, explaining them the organizational details and the purposes of the test, stressing that an usability test aims to evaluate a web site, not its user.
After completing the test, the data should be analysed to produce quantitative and qualitative results.
Web Usability Testing
In this phase it is important to identify which are the major problems, to summarize the performance data, to evaluate them performing statistical analysis and then to summarize the preference data.
An ideal timeline for a web usability testing is to follow the life-cycle of the web site development: in the initial stages it is important to test the various versions of the web site, maybe against the competitor’s products to find out which are the major design issues. Then in the middle stages it is important to validate and refine the design, while in the later stages the purposes should be to verify that the web site meets the requirements and the functional specifications.
In the following sections, we will analyse the different techniques adopted for web usability testing, while the following table summarizes when the different methods should be applied with respect to the design phases outlined in Chapter 2.3 .
Web Usability Testing
Gearing-up Initial design
Iterative development
System monitoring Thinking
aloud X X X X
Co-discovery
X X X X
Question
asking X X X X
Performance Measurement
X X X
Server log
analysis X
Activity
recording X X
Remote
testing X X
Table 2 - Web usability testing methods: development phases