This paper argues that the model of searching for information on the Web as used by many existing search engines does not meet the needs of Internet-naive (but PC-literate) users. This is based on two user trials carried out in the summer of 1995 with people who had not encountered the Internet before. Results show that potential users need at least some understanding of basic Internet concepts in order to carry out successful searches. Without this understanding, potential customers are likely to be discouraged from taking up Internet services after initial bad experiences.
The Internet's potential as a major revenue earner can only be realised if users can find exactly what they want on the Internet quickly, accurately, and with little effort. The development of tools which support the finding of relevant material within a few mouse clicks and key strokes is becoming increasingly critical, given the unprecedented rate at which the Web is growing and the rising numbers of novice users.
Two user trials were carried out during August and October 1995 to collect information to support the construction of navigation tools for ordinary users (rather than 'netsurfers'). The trials aimed to understand how people currently search for information in their everyday lives, and then to incorporate these processes into a prototype search engine in order to ensure its usability and acceptability.
All participants in both trials were deliberately not Internet users. Fourteen members of the public of mixed ages, demographics, and little computing experience took part in the first trial. In order to determine whether the results also applied to PC-literate customers, the second trial deliberately targeted eighteen customers who used computers on a regular basis - either adult males between the ages of 20 and 45, or teenagers of either sex. The results from both trials were remarkably similar, which suggests that familiarity with PCs does not affect the ease with which novice users can find information on the Internet.
The procedure for both trials was the same. Participants were introduced to the Internet and briefed about its nature. They were asked about the latest product or information they had searched for 'in the real world', and encouraged to discuss the approach they had taken. They then attempted to replicate that search using a variety of Internet search engines such as Lycos, Webcrawler, and Yahoo.
The Internet as a Global Resource
Most participants from both trials had heard of the Internet, but none of them really understood its global nature and what was implied by this. For example, several users were surprised that many of their search results were from international sources. They expected that information returned would be local (even to the Suffolk region rather than to the UK as a whole) which meant that search results from places such as Minnesota were completely unexpected and unfathomable.
The Internet as an Unmanaged Resource
Users expected information to be returned to them in a clear and indexed fashion. This meant that the concept of iterative searching and matching information in terms of relevance was poorly grasped. In other words, users did not use the Internet with the premise of 'if this doesn't work, I'll try this'; instead, they assumed anything less than a perfect match was an irrevocable failure. Our participants often assumed that the Internet did not have relevant information when results returned to them did not appear to make sense, when in fact their queries were answerable.
The Scope of the Internet
Users could not begin to understand the range and nature of the material that the Internet offered. Many were surprised at its scope - for example, after searching for information about Bob Marley, one participant was convinced that the authors had deliberately preselected information that would interest him:
"How did you get all this stuff on there for me then? I mean, I've just walked in off the street, how did you know to have stuff in on Bob Marley and that?"
The Searching Concept
Most participants were happy with the concept of searching and finding when given a clear world example, such as finding some reading material for their holidays. However, they were confused by the concept of telling a computer an area of interest and getting some possible satisfier returned. For example, although users could understand the notion of going to a library and asking the librarian for information, these same users failed to understand the notion of using a search engine to find articles of interest on the Internet, even though the two processes are logically identical. One participant commented:
"What kind of brain capacity do you need to access the information? I wouldn't have the first clue how to set things out so I could find things. . . ."
Quality of Hits
Very few users were impressed with the quality of information the Internet contained. None of the participants said they found anything that would make them bother to use it, as they already had access to better quality alternatives. For example, a highly technically literate user (air force weapons technician) wanted to find information about Liverpool Football Club, and eventually found the Liverpool FC home page complete with match reports, interviews, pictures of stars, etc. Despite the fact that this site offers fans a vast amount of well-presented information about the club, the participant claimed that teletext and newspapers were quicker and easier, and offered better quality information. The speed and difficulty of searching and finding information to match requirements is clearly a critical factor influencing this (commonly held) perception. One user who required a lot of prompting to help him find what he wanted commented:
"Good grief, all this to get to where I want to"
Search Terms
Nearly all participants from both trials had difficulty formulating good searching keywords even when they had all the information they needed. In the real world, users go into a library or a shop and express their requirements in verbose or imprecise terms, or alternatively they browse through items on offer. They are not used to elaborating an artificial text string to match their requirements. Common errors included:
The central problem was that users did not seem to understand what were likely to be good quality differentiators. Thus one participant looking for Reebok trainers searched for 'sports shoes' rather than the more discriminatory 'Reebok'. Another entered 'competitive market share' when trying to find information about competitors for the Rover car company, not realising that he had to enter at least some contextual element.
Requisite World Knowledge
An unexpected finding was the degree to which participants lacked the necessary world knowledge that is required to make use of the Internet. Many users made requests that could easily have been satisfied if they had been in possession of crucial pieces of information. Some examples of this included:
The trials indicated that the process of finding information via search engines was simply too complicated for Internet-naive people to use without very high levels of support. It was suspected that none of the participants would have bothered to work out how to use the tools shown to them had they been left alone. Common remarks included:
"I'd need a couple of hours . . I'd press all the wrong keys if you left me alone."
The authors believe that the key Internet misconceptions as described in this paper are both widespread and enduring. More recent focus groups carried out by the authors indicated that some users who have had Internet access for over a year (and who claim to 'surf' on a regular basis) are still having problems using search engines:
". . .it came up with streams and streams of information and it just took ages to scroll through it all -- and it never came up with anything particularly useful. . .I then tried to enter words that were more specific to what we wanted, but in the end I just gave up because I couldn't find anything."
The following recommendations are made to help the
ease with which information can be found:
The results also have implications for issues which commercially-available Internet training packages should address, and how they should be positioned. There is a real risk that potential Internet users will be alienated unless they have a simple understanding of basic Internet concepts, however polished the interface design of search engines.
1. Jakob Nielsen's Alert Box columns on the Sun Microsystems Web site. <http://www.sun.com/columns/jakob>
2. Hix, D., & Hartson, R. (1993). Developing
user interfaces. New York: Wiley.
hdl:cnri.dlib/march97-pollock