Stories

D-Lib Magazine
April 1998

ISSN 1082-9873

Modeling Users' Successive Searches in Digital Environments

A National Science Foundation/British Library Funded Study

Amanda Spink
University of North Texas
Denton, Texas
spink@lis.admin.unt.edu

Tom Wilson, David Ellis & Nigel Ford
University of Sheffield
Sheffield, UK
t.d.wilson@sheffield.ac.uk, d.ellis@sheffield.ac.uk, n.ford@sheffield.ac.uk

Abstract

As digital libraries become a major source of information for many people, we need to know more about how people seek and retrieve information in digital environments. Quite commonly, users with a problem-at-hand and associated question-in-mind repeatedly search a literature for answers, and seek information in stages over extended periods from a variety of digital information resources. The process of repeatedly searching over time in relation to a specific, but possibly an evolving information problem (including changes or shifts in a variety of variables), is called the successive search phenomenon. The study outlined in this paper is currently investigating this new and little explored line of inquiry for information retrieval, Web searching, and digital libraries.

The purpose of the research project is to investigate the nature, manifestations, and behavior of successive searching by users in digital environments, and to derive criteria for use in the design of information retrieval interfaces and systems supporting successive searching behavior.

This study includes two related projects. The first project is based in the School of Library and Information Sciences at the University of North Texas and is funded by a National Science Foundation POWRE Grant <http://www.nsf.gov/cgi-bin/show?award=9753277> [Spi97]. The second project is based at the Department of Information Studies at the University of Sheffield (UK) and is funded by a grant from the British Library <http://www.shef. ac.uk/~is/research/imrg/uncerty.html> Research and Innovation Center [WEF97].

The broad objectives of each project are to examine the nature and extent of successive search episodes in digital environments by real users over time. The specific aim of the current project is twofold:

  • To characterize progressive changes and shifts that occur in: user situational context; user information problem; uncertainty reduction; user cognitive styles; cognitive and affective states of the user, and consequently in their queries; and
  • To characterize related changes over time in the type and use of information resources and search strategies particularly related to given capabilities of IR systems, and IR search engines, and examine changes in users' relevance judgments and criteria, and characterize their differences.

The study is an observational, longitudinal data collection in the U.S. and U.K. Three questionnaires are used to collect data: reference, client post search and searcher post search questionnaires. Each successive search episode with a search intermediary for textual materials on the DIALOG Information Service is audiotaped and search transaction logs are recorded. Quantitative analysis includes statistical analysis using Likert scale data from the questionnaires and log-linear analysis of sequential data. Qualitative methods include: content analysis, structuring taxonomies; and diagrams to describe shifts and transitions within and between each search episode. Outcomes of the study are the development of appropriate model(s) for IR interactions in successive search episodes and the derivation of a set of design criteria for interfaces and systems supporting successive searching.

1.0 Introduction

Recent research shows that users with a broader problem-at-hand often seek information in stages over extended periods and use a variety of information resources. As the time progresses, IR users search the same or different digital environments, such as IR systems -- online databases, CD-ROM databases, online public access catalogs (OPACs), the Web or digital libraries -- for answers to the same or evolving problem-at-hand. As users learn or progress in their work, or as they clarify a problem and/or question[s], or as their situation changes, users come back to IR systems of various kinds for more searches. For instance, while doing research, the great majority of scientists engage, at one time or another, in successive searching in relation to that research.

The process of repeated, successive searching of digital environments over time in relation to a given, possibly evolving, information problem (including changes or shifts in beliefs, and cognitive, affective, and situational states) is called a successive search phenomenon. The phenomenon is reflected in successive search episodes, which then become units for observation and analysis. The modeling of users in successive searches in digital environments is then successive user modeling. A key dimension is time, the key variable changes or shift in successive search episodes over time, and the key constant is the same or evolving information problem. The evolution, if any, of a problem and other cognitive, affective and situational variables can be mapped. The history of successive search episodes can be recorded and analyzed -- i.e., the phenomenon can be a subject of research.

However, the successive search phenomenon has not been investigated to any serious extent. One general reason is that modeling the information structure of texts is much easier than modeling user behavior. In addition, although IR interfaces might well support successive search, repeated uses and successive searches currently receive little, if any, support from present IR interfaces and procedures, or from Web search engines. IR systems, in general, are built following a single search paradigm, i.e., they are designed and operate on the assumption that every search episode is an end in itself. IR has been conceived in relation to highly structured information resources. However, IR has spread beyond the traditional IR databases and systems. It has become a major component (and a major headache and problem) in searching of widely distributed information resources on the World Wide Web. But neither the commercial systems (such as Dialog or Lexis/Nexis), nor the TREC (Text Retrieval Conference) experimental IR systems, nor the search engines on the Web, digital libraries, nor the "intelligent" agents in knowledge base applications, address or support successive searching. In other words, they do not support a verified searching behavior and need of users, as a rule. Research in this area is in its formative stage and is significant as the size and variety of information resources in IR systems and on the digital libraries and Web grow exponentially, and the problem of searching becomes critical. The research project discussed here is oriented toward deriving human dimensions and criteria for the design of IR interfaces and search engines.

2.0 Research Purposes and Objectives

This study examines the nature and extent of successive IR search episodes by a set of real users over time to:

This empirical research is based on observation of real-life, as opposed to laboratory, situations. Information needs are not imposed -- as in a laboratory study; they are user-initiated. The study has the strength of a basis in real user behavior as they interact with IR systems and search engines. A weakness of empirical studies is that the results should be treated as hypothesis for generalization and further confirmation.

3.0 Related Research

3.1 Information Retrieval Research

Research into the human or cognitive (user modeling) aspects of IR is also in its infancy with a growing body of research on users' interactivity and measures for observing user interactivity [SMSS91]. Major theoretical interactive IR models have recently emerged but have yet to be developed sufficiently to be empirically tested [BCST95][Ing96][Sar97]. IR researchers are also beginning to investigate the context of users' searches and evaluation [Ell97], and identify key elements in users' searching within the context of users' single search of an IR system.

IR research has overlooked any successive searches related to the same information problem. However, important elements within single searches have been identified, including feedback types and effective search term selection strategies [SKCT88][SMSS91][SS97]. The differences between end-user and intermediary searching behaviors have also been investigated [HY93]. Research has also shown that end-users perform different search sessions over time (each search related to a different information problem) including searches of successive databases or IR systems [SMSS91]. In addition, recent findings from general information behavior studies also have strong implications for the need to investigate users' successive searching behavior.

Research shows that humans seek information in stages over extended periods as their information problems change [Kuh93] and that they use different types of IR systems during information seeking processes (e.g., Web, CD-ROMs). However, recent studies also show that users also conduct successive searches over time when seeking information to solve information problems [RHB92][Spi96]. However, limited knowledge exists on the patterns of users' successive searching behavior.

3.2 Successive Searching Behavior

Data from several recent studies highlights the weakness of research based on the single search approach and the need for studies that classify and categorize users' successive searching behavior. Recent studies show users conduct successive IR searches when seeking information related to a particular information problem. In one recent study 18 (45%) of academic users were found to have a previous mediated online search on the same topic, frequently with the same search intermediary [SMSS91][Spi93]. M.H. Huang [Hua92] studied 44 end-users conducting online searches and found 19 end-users conducted successive searches. S.E. Robertson and M.M. Hancock-Beaulieu [RHB92] also identified successive searches by users of the OKAPI online catalog. They found a continuity of search topics and relevance judgments by the same users over successive searches as some users explored a topic over an extended period and interacted at intervals with the online catalog using identical or closely related search strategies. This research highlights the need for longitudinal research at a problem-level of analysis as opposed to a single search level of analysis of searching behavior.

Recent research [Spi96] shows that successive IR searches are a fundamental aspect of users' behavior when seeking information related to an information problem. Data from survey interviews with 200 academic users shows that for the same information problem:

  1. 113 (56.5%) users had conducted more than one IR search,
  2. 43 (21.5%) users had conducted five or more IR searches, and
  3. many users had conducted successive searches at different stages of their information seeking process.

A recent study showed that Web users also perform successive searches of the same Web search engine when seeking information on a particular topic over time [SBJ98].

3.3 Information Seeking Studies

Results from information seeking studies support the notion of the successive searches by showing that humans progress through a series of stages, adopt different strategies and exhibit different information behaviors at different stages of their information-seeking process [Ell89] [Kuh93]. Kuhlthau [Kuh93] found that the information search process of library patrons occurred in six clearly defined stages related to the cognitive, affective states and search activities of the users, including task initiation, topic selection, prefocus exploration, focus formulation, information collection, and search closure. Although Kuhlthau did not investigate the use of IR systems by library patrons, her findings suggest that IR system users continue to collect and seek information throughout their information-seeking process, using or requiring different types of information, conducting different types of searches, and using different search terms and strategies at different stages of an information seeking process [KSC92].

Much earlier, Wilson and Streatfield [WS80] had shown that social workers and their managers work tasks involved their seeking and exchanging information iteratively over time, suggesting that this behavior is characteristic of human communication in relation to specific issues of any kind, whether social or research-based. However, the nature of successive searches and related relevance judgments during a user's longitudinal information search process is in the early stages of development and presents important implications for the design of effective IR systems to support effective user searching.

3.4 Relevance Research

Recent relevance research also highlights the need to investigate how users' relevance judgments and relevance criteria change over successive searches. Relevance studies have largely focused on the nature of users' relevance judgments during a single search [Sch94]. Automatic relevance feedback techniques, based on users' judgments of highly relevant items, have been found to be more effective than traditional approaches to IR systems [SL96]. Recent research by Bateman [Bat98] is also beginning to explore how users' high relevance judgments and relevance criteria change over successive searches. Bateman's study found little change in users' criteria for documents judged "highly" relevant by a user over successive searches.

Recent research [SG97] [SGB in press] suggests that users' partial relevance judgments also play an important role in the search process and are linked to shifts in the user's information problem during single and successive searches. Findings from this study also suggest that partial or less highly relevant items may be related to shifts or changes in users' information problem or change stages during the early stages of their information search process. Further research is needed to explore the role of users' "high" and "partial" relevance judgments during an information search process including successive searches. The focus of the current project is to model users' successive searching behavior within the following conceptual and theoretical framework, including the relationship between relevance judgments and information seeking.

4.0 Conceptual and Theoretical Framework

While the existence and some basic statistics of successive searching were observed, as reviewed above, no conceptual framework or model has evolved specifically addressing this phenomenon in IR. Many user models exist in IR and in AI, as well, but none have addressed successive user modeling. Thus, in addressing the nature of successive searches, a major research effort has to be directed toward the development or adaptation of appropriate model or models of information seeking and interactive IR. We start with the models summarized below. The objectives of use of these models are to:

4.1 Information Seeking Models

First is the Kuhlthau Information Search Process (ISP) model [Kuh93]. The model incorporates a longitudinal view of the search process and identifies six evolving stages: Initiation, Selection, Exploration, Formulation, Collection, and Presentation. Second (particularly pursued by the Sheffield team) is Ellis's Behavioral Model [Ell89][ECH93] which defines the following characteristics of information seeking behavior without typifying these as stages: Starting, Chaining, Browsing, Differentiating, Monitoring, Extracting, Verifying, and Ending.

Wilson [Wil98] suggests that the Ellis and Kuhlthau models may be viewed as closely related, if a stage process is imposed on Ellis's characteristics. Under this revision, the activities of Chaining and Monitoring are seen as a deeper specification of Kuhlthau's "Collection" stage. Wilson outlines a model of information-seeking behavior as goal-determined problem solving. Four problem solving stages are delineated:

  1. Problem recognition,
  2. Problem definition,
  3. Problem resolution, and
  4. Solution statement.

Successive searching behavior is seen as related to the movement from one problem solving stage to the next, and in iterative loops within a stage as, for example, when one search in the Problem Definition phase leads to a further need to refine the definition and a successive search. In Wilson's model, the Kuhlthau and Ellis models describe the behavior within one loop in a problem-solving stage.

4.2 Uncertainty

The Sheffield team will also explore the concept of uncertainty (the formal title of the Sheffield project is "Uncertainty in Information Seeking") which has had little treatment in information science, except in defining information as that which reduces uncertainty. Under this definition, data that, for example, increases uncertainty is not information. Ingwersen [Ing92] has defined the relationship between uncertainty and information seeking as action undertaken to resolve doubts that cannot be resolved by thinking alone; this is broadly equivalent to Belkin's [KL96] suggestion for the need for further research on uncertainty "from the user's perspective to understand the full range of impact of uncertainty in all of its manifestations on the information-seeking behavior of human beings in the conduct of their daily lives." This project is intended to contribute to that research agenda.

4.3 Information Retrieval Model

This study also utilizes the Saracevic Stratified Model of IR interaction [Sar96a]. The model views the interaction as a dialogue between participants: user and computer (system) through an interface at a surface level; furthermore, each of the participants is depicted as having different levels or strata. Interaction is the interplay between various levels. On the user side elements involve at least these levels: cognitive, affective, and situational.

This general model has been extended to encompass specific processes or phenomena that play a crucial role in IR interaction: the notion of relevance, user modeling, selection of search terms, and feedback types [Sar96b] [Sar97][SSW97][SS97]. The model is adapted here to categorize classes of variables on the user and computer side and their interplays, which underlying the visible "things" on the surface level. In this project, they will help organize the type of observations to be made in and between successive search episodes. Queries, commands and responses are surface phenomena that can be observed and analyzed as to nature, content, and shifts on the surface level, as planned here. But a more significant and more difficult connection has to be made to the other levels on both the user and computer sides, and to the feedback loops among all levels, that affect shifts. These are being analyzed as well.

Third, is the Reasons-Interaction-Results (R-I-R) model used to describe use of information services by users [SK97]. This model and its facets, as well as questionnaires developed by [Sar89], serve as methodological tools for the study.

4.4 Relevance

An important variable examined in the proposed project is relevance. Relevance in IR is an attribute or criterion reflecting the effectiveness of interactive exchange of information between users and IR systems in a communicative contact. The interaction involves different levels or strata at which interaction are inferred producing an interdependent system of relevances. Thus, there is a distinction between system or algorithmic relevance; topical or subject relevance; cognitive relevance or pertinence; motivational or affective relevance; and situational relevance or utility. Together with the interaction model, this provides a framework and a set of categories along which the shifts and changes in relevance criteria and judgments will be examined.

4.5 Shifts

Finally, we have to deal with the difficult concept of shifts; a concept that was talked about a lot, but never elaborated specifically in IR. A shift signifies a change from one state to another of any variable involved, or from one phase or action to another, based on a reason and geared toward a result. It is clear from the above models which variables should be observed for detection of shifts. However, methodologically it is not clear how to record a given shift for a given variable, and how to categorize and describe given changes and their outcomes. Thus, an important part of the proposed research will be methodological in nature: establishing and testing various methods for categorizing and measuring shifts from the data obtained.

Similar methodological problems are encountered in analysis of phase transition in communication discourse, particularly in negotiations [Hol92]. There are significant similarities in negotiations and phase transitions in communication with type of activities that users are engaged in while 'negotiating' with IR systems for an effective search, thus the parallel. In general, we can think of a shift space within an episode and between episodes in which changes can be mapped. Shifts will be characterized, depending on the variable, as to changes in focus, content, direction, substitutions, trade-offs, and other characteristics. Shifts may also be viewed as an effort in managing change toward a result, and as such, as a trial and error procedure, as often mentioned in describing the IR process. Different type of diagrams will be explored to describe and model shifts. In particular, these will include Bayesian diagrams, semantic road maps, and feedback loops of mutual causality.

5. Research Design

5.1 Users

Unlike many prior studies, we are not collecting data under laboratory, controlled experimental, or other artificial conditions where study populations and questions are restricted. Data are being collected from a set of users engaged in tasks or problems that produce real information needs. Consequently, they search operational IR systems with or without assistance by a professional intermediary. Every effort will be made to preserve the reality of situations, observations, and recordings.

Over the institutions, a sample of over 100 users is currently participating in the project. A pilot phase has been conducted in Sheffield from October 1997 to February 1998, using respondents to a regular call for participants in the information searching teaching of students. The University of North Texas has been collecting data since January 1998. An informed consent form is provided to U.S. users. All users receive an explanation of research undertaken and details of their engagement.

Users generally are engaged in a longer activity (two months or more) in research, development, planning, or a similar project which creates information problems and needs information support from IR systems, including searching of networked information resources, such as those on the Web. A sample of some 100 users is large enough to provide qualitative and quantitative significant results, yet small enough to be manageable for intensive record collection and in depth analysis of what can be obtained.

5.2 Questionnaires

Three questionnaires are being used at UNT to record various aspects of context that are connected to context and not recordable in transactions: reference, client post search and searcher post search. The Sheffield site is also using a fourth questionnaire to collect information on changes in users' relevance judgments on identified items some two months after the initial search after the user has acquired the documents. These questionnaires capture answers to cognitive, affective, situational, relevance, and process variables suggested by the reviewed models. The questionnaires were tested in pilot applications.

5.3 Data Collection & Analysis

The data being collected includes: (1) transactions from logs, (2) numerical data and texts in response to given questionnaires, (3) texts retrieved and assessed relevance judgments, and (4) in Sheffield, responses to a standard test of cognitive styles. The project will last for eighteen months at UNT and two years at Sheffield. Quantitative and qualitative analysis methods will be employed. Quantitative analysis will concentrate on statistical analysis of the data from Likert scales. Standard statistical tools from the SPSS package will be employed. What happened in a previous search episode very much relates to what happens in the successive one. Thus, part of the quantitative analysis will be a search for and test of statistical models appropriate for this type of events, starting with models in log-linear analysis.

However, qualitative methods will predominate. The reason for this is that data involved, from search terms selected in queries, to answers to questions as to reasons, interactions, or results, is largely textual. Qualitative methods will be based on grounded theory [SC90]. The qualitative methods will include:

Quantitative analysis will concentrate on estimation of probabilities that can be used in various networks, such as Bayesian diagrams. So far, despite many suggestions to deploy such probability-based diagrams in IR, none have been implemented in practical situations, or even in laboratory test of any size. This is another methodological area for exploration.

6. Research Outcomes

6.1 Models

A major objective of the research is to develop models depicting successive search episodes in digital environments. Two kinds of models will be developed. First are derived successive search model (or models) -- derived from the collected data, thus these will be data-driven models. Second are integrative models -- integrating the derived models with other search models in the conceptual framework and changing those general models, as necessary, to reflect successive searches. These are much more conceptually than data based models. As much detail as possible will be provided in all the models, derived and integrative, to enable their application and testing in the future on other sets of data and in other contexts. One such model will concentrate on modeling the history of successive searches of a user and oriented toward the user, incorporating feedback loops, among others. The models will, according to objectives, characterize progressive changes and shifts that occur in the user and computer sides, in interactions, and in relevance criteria and assessments. The end result will be successive user modeling in IR.

6.2 Design Criteria

Another major objective of the project is to suggest criteria for use in design of interfaces that support successive searching in digital environments. Clearly, these will depend on the findings. But, we envision two sets of design criteria: one more applicable to the highly structured IR databases and systems, such as DIALOG, and the other to the much less structured and distributed information resources as found on digital libraries and the Web. In particular, the Web oriented criteria will be merged with results from present studies of the Excite search engine. Each of these two sets of criteria presents different problems and challenges. It is envisioned that at the completion of the project a partnership between human-oriented and system-oriented participants in the project may develop in order to implement and test some of the criteria. Thus, the end of this project is envisioned to lead to other collaborative projects.

7. Conclusion

This research project concentrates on a phenomenon that has been amply observed to exist when users search for information. A great many users engage repeatedly over time in accessing and searching information resources in digital environments related to the same or evolving problem-at-hand. Successive searching is widely practiced. Yet, users' successive searching has received little research attention, despite the obvious significance of a better understanding of the phenomenon. The research area is relatively new.

This should be considered as an exploratory study concentrating not only on the collection and analysis of data, but also on a number of methodological issues and problems. Methods for the investigation and depiction of shifts within and between successive search episodes is one of these problems. Obviously, there are a number of kinds of shifts and a number of levels. For instance, on the surface level there are semantic, syntactic, and logic shifts in selected search terms and statements; on the situational level there are shifts in problem definition; there are shifts in focus, and so on. It is not even clear how various shifts should be classified. In other words, by necessity, shifts and transitions will be a major focus of the proposed research.

Currently, IR systems and interfaces do not include assistance to users in successive search episodes. This research will not specifically design new or improve old interfaces to accommodate users' successive search episodes. However, design criteria derived here will be significant in the design of human-oriented systems. In fact, the design of effective human-oriented systems and interfaces is an impossibility without the research proposed. It is anticipated that this research will lead to further research on the phenomenon itself, but also to interdisciplinary cooperation between the Principal Investigators and the design community. Both sides can and will benefit.

8. Acknowledgment

This project is supported a POWRE grant from the National Science Foundation (ref. no.9753277) and a grant from the British Library Research and Innovation Center. The assistance of searchers and clients at the University of North Texas and the University of Sheffield is gratefully acknowledged. The authors also thank Tefko Saracevic of Rutgers University, IBM, and Microsoft for their valuable contribution to the development of this study.

9. References

[Bat98] J. Bateman. Modeling Changes in Relevance Criteria: An Information Seeking Study. Unpublished Dissertation. School of Library and Information Sciences, University of North Texas, 1998.

[BV85] N. J. Belkin, and B. C. Vickery. Interaction in Information Systems: A Review of Research from Document to Knowledge-Based Systems. London: British Library (Library and Research Report 35), 1985.

[BOB82]N.J. Belkin, R.N. Oddy, and H.M. Brooks. ASK for information retrieval, Parts 1 & 2. Journal of Documentation, 38: 61-73, 145-164, 1982.

[BCST95] N. J. Belkin, C. Cool, A. Stein, and S. Theil. Cases, Scripts, and Information-Seeking Strategies: On the Design of Interactive Information Retrieval Systems. Expert Systems With Applications, 9(3):379-395, 1995.

 [Ell89] D. Ellis. Behavioural Approach to Information Retrieval System Design. Journal of Documentation, 45:171-212, 1989.

[ECH93] D. Ellis, D. Cox, and K. Hall. A Comparison of the Information Seeking Patterns of Researchers in the Physical and Social Sciences. Journal of Documentation, 49:356-359, 1993.

[Ell97] D. Ellis. The Dilemma of Measurement in Information Retrieval. Journal of the American Society for Information Science, 47(1):23-36, 1997.

[For95] N. Ford. Levels and Types of Mediation in Instructional Systems: An Individual Differences Approach. International Journal of Human-Computer Studies, 43:241-259, 1995.

[FF93] N. Ford and R. Ford. Toward a Cognitive Theory of Information Accessing: An Empirical Study. Information Processing and Management, 29(5):569-585, 1993.

[Hol92] M. E. Holmes. Phase Structures in Negotiation. In L. L. Putnam and M. E. Roloff, eds. Communication and Negotiation. Newbury Park, CA: Sage, 1992.

[HY93] I. Hsieh-Yee. Effects of Search Experience and Subject Knowledge on the Search Tactics of Novice and Experienced Searchers. Journal of the American Society for Information Science, 44(3):161-174, 1993.

[Hua92] M. H. Huang. Pausing Behavior of End-Users in Online Searching. Unpublished Doctoral Dissertation. University of Maryland, 1992.

[Ing92] P. Ingwersen. Information Retrieval Interaction. London: Taylor Graham, 1992.

[Ing96] P. Ingwersen. Cognitive Perspectives of Information Retrieval Interaction: Elements of a Cognitive IR Theory. Journal of Documentation, 52(1):3-50, 1996.

[Kuh93] C. C. Kuhlthau. Seeking Meaning: A Process Approach to Library and Information Services. Norwood, NJ: Ablex Publishing, 1993.

[KL96] C. C. Kuhlthau and N. Ledet. The Relationship of Information and Uncertainty in Information Seeking. Proceedings of the 2nd International Conference on Library and Information Science: Integration in Perspectives, October 13-16, 1996. Edited by Peter Ingwersen and Neils Ole Pors: Royal School of Librarianship, (pp. 367-376), 1996.

[KSC92] C. C. Kuhlthau, A. Spink, and C. Cool. Exploration into Stages in the Information Search Process in Online Information Retrieval: Communication Between Users and Intermediaries. Proceedings of the 55th Annual Meeting of the American Society for Information Science, 29: 67-71, 1992.

[RHB92] S. E. Robertson and M. M. Hancock-Beaulieu. On the Evaluation of IR Systems. Information Processing and Management, 28(4): 457-466, 1992.

[Sar89] T. Saracevic. Modeling and Measuring the User-Intermediary-Computer Interaction in Online Searching: Design of a Study. Proceedings of the Annual Meeting of the American Society for Information Science, 26:75-80, 1989.

[Sar96a] T. Saracevic. Modeling Interaction in Information Retrieval (IR): A Review and Proposal. Proceedings of the Annual Meeting of the American Society for Information Science, 33:3-9, 1996.

[Sar96b] T. Saracevic. Relevance Reconsidered. Information Science: Integration in Perspectives. Proceedings of the Second Conference on Conceptions of Library and Information Science. Copenhagen, Denmark (pp. 201-218), 1996.

[Sar97] T. Saracevic. Extension and Application of the Stratified Model of Information Retrieval Interaction. Proceedings of the Annual Meeting of the American Society for Information Science, 34:313-327, 1997.

[SK97] T. Saracevic and P. Kantor. Studying the Value of Library and Information Services. I. Establishing a Theoretical Framework. II. Methodology and Taxonomy. Journal of the American Society for Information Science, 48(6):527-542, 543-563, 1997.

[SKCT88] T. Saracevic, P. Kantor, A. Y. Chamis, and D. Trivison. A Study of Information Seeking and Retrieving: I. Background and Methodology. Journal of the American Society for Information Science, 39(3):161-176, 1988.

[SMSS91] T. Saracevic, H. Mokros, L. Su and A. Spink. Interaction Between Users and Intermediaries During Online Searching. Proceedings of the 12th Annual National Online Meeting, 12:329-341, 1991.

[SSW97] T. Saracevic, A. Spink and M. M. Wu. Users and Intermediaries in Information Retrieval: What are They Talking About? Proceedings of the Sixth International User Modeling Conference, UM97, July, Sardinia, Italy. New York: Springer: 43-54, 1997.

[Sch94] L. Schamber. Human Behavior and Relevance. Annual Review of Information Science and Technology, 29:1-33, 1994.

[Spi93] A. Spink. The Effect of User Characteristics on Search Outcome in Mediated Online Searching. Online & CD-ROM Review, 17(5): 275-278, 1993.

[Spi96] A. Spink. A Multiple Search Session Model of End-User Behavior: An Exploratory Study. Journal of the American Society for Information Science, 46(8):603-609, 1996.

[Spi97] A. Spink. Interaction in Information Retrieval (IR): Successive Searches By Users Over Time. 1997. National Science Foundation 1997 POWRE Grant No. 9753277.

[SBJ98] A. Spink, J. Bateman, and B. J. Jansen. Searching Heterogeneous Collections on the Web: Behavior of EXCITE Users. Proceedings of the 1998 National Online Meeting, May, New York, 1998 (forthcoming).

[SG97] A. Spink and H. Greisdorf. Partial Relevance Judgments During Interactive Information Retrieval: An Exploratory Study. Sixtieth Annual Meeting of the American Society for Information Science, November 1-6, 1997, Washington D.C.:111-122, 1997.

[SGB in press] A. Spink, H. Greisdorf and J. Bateman. From Highly Relevant to Not Relevant: Examining Different Regions of Relevance. Information Processing and Management (in press).

[SL96] A. Spink and R. M. Losee. Feedback in Information Retrieval. Annual Review of Information Science and Technology, 31: 33-78.

[SS97] A. Spink and T. Saracevic. Interaction in Information Retrieval: Selection and Effectiveness of Search Terms. Journal of the American Society for Information Science, 48(5): 382-394.

[SC90] A. Strauss and J. Corbin. Basics of Qualitative Research: Grounded Theory Procedures and Techniques. Newbury Park, CA: Sage Publications, 1990.

[Wil98] T. D. Wilson. Models of Information Behavior Research. Unpublished paper. Department of Information Studies, University of Sheffield, 1998.

[WS80] T. D. Wilson and D. R. Streatfield. You Can Observe a Lot....: A Study of Information Use in Local Authority Social Services Departments. Sheffield: University of Sheffield, Postgraduate School of Librarianship and Information Science, 1980.

[WEF97] T. D. Wilson, D. Ellis and N. Ford. Uncertainty in Information Seeking. A Research Proposal to the British Library Research and Innovation Centre, 1997.

(Spaces in links that caused linking problems were removed 8/31/05.)

Copyright © 1998 Amanda Spink, Tom Wilson, David Ellis & Nigel Ford

Top | Magazine
Search | Author Index | Title Index | Monthly Issues
Previous Story | Next Story
Comments | E-mail the Editor

hdl:cnri.dlib/april98-spink