2. PAIRI
Agenda/Outline
Adaptivity in IR Interaction
Four Dimensions
Pecha-Kucha Presentation and
Timeline
Audience Interaction!
Wrapping Up
3. PAIRI
PRESENTERS
Birger Larsen, Royal School
of LIS, Copenhagen,
Denmark
Marianne Lykke, Aalborg
University, Denmark
Diane Kelly, University of
North Carolina at Chapel
Hill, USA
Peiling Wang, University of
Tennessee, Knoxville,
USA
ModeratorPeter Ingwersen, Royal School
of LIS,Copenhagen, Denmark
4. PAIRI
ADAPTIVITY in IR INTERACTION
… requires that IR systems adapt
to users’ situations, and
… the users adapt to the systems.
5. PAIRI
SYSTEM ADAPTION to USERS
… entails dynamic user modeling;
effective information architecture
founded in practice, and
enhanced search features, such as
search integration
relevance feedback
6. PAIRI
SEARCHER ADAPTION to
SYSTEMs through INTERACTION
… entails mental model building of
systems and modification; through
Learning – leading to
Knowledge change, such as
from an anomalous state of knowledge (ASK) towards
a coherent state of knowledge (CSK);
7. PAIRI
Adaption - Central DIMENSIONS:
through
integration of
information objects
of the Information
Retrieval system to
the current searcher
to context and
practice through
information
architecture
of searchers to the
Information
Retrieval systems
8. PAIRI
PRESENTATION – PECHA-KUCHA
Present Introduction – 7 minutes
First Dimension: 20 slides x 20 seconds (7 min)
Questions – interaction (10 minutes)
Second Dimension: 20 slides x 20 sec. (7 min)
Questions – interaction (10 minutes)
Third Dimension: 20 slides x 20 sec. (7 min)
Questions – interaction (10 minutes)
Fourth Dimension: 20 slides x 20 sec. (7 min)
Questions – interaction (10 minutes)
10 minutes vivid interaction on additional
audience questions
Wrapping up (5 min)
9. PAIRI
QUESTIONS ON PAPER SLIPS
During presentations you may write
questions to the panel (aside from oral
questions!!)
’Collectors’ will collect the slips and hand
over to Moderator
Moderator selects which questions to be
posed during last 10 minutes of interaction
Moderator knows which questions will be
dealt with by ensuing panellists.
14. PAIRI
How do we handle ‘verticals’ in libraries?
As separate silos!
User
Catalogue
.
Repositories Databases eJournals
JSTOR
Publisher 1
Publisher 2
eBooks
15. PAIRI
How to access the silos?
Information literacy!
= educate users how to use each system:
Content, fields, indexing, search operators,
interfaces…
But…
‘Why do we want to teach
our users to be librarians?’
(Dave Pattern, Library Systems Manager)
United Kingdom Serials Group
2009 Annual conference
16. PAIRI
Users don’t want silos!
‘Why is Google so easy
and the library so hard?’
(Claire Duddy - student)
United Kingdom Serials Group
2009 Annual conference
17. PAIRI
Challenge
Each silo has its own set of metadata
and standards developed for different
purposes
Very low common denominator…
19. PAIRI
Solution: Federated search?
“The jury is still out on federated search systems,
even though more libraries now have them.
There are murmurings that federated search
has lower-than-expected use and may not be
the magic search bullet we were led to believe”
(Tenopir, 2007)
20. PAIRI
Solution: Integrated search?
Harvest all of the relevant data sources,
normalize them into a single metadata
schema, and index all of them together in
one large union index
Add Google-like search box and ranking
(…a kind of federated search 2.0)
21. PAIRI
Solution: Integrated search?
Is being implemented in university and
research libraries as we speak
University of Huddersfield , UK
Queens University, Canada
State and Uni library, Denmark
…
Several commercial
products
Summon by Serial Solutions
EBSCO Discovery Service
WorldCat Local
ExLibris' Primo Central
III Encore Synergy
…
22. PAIRI
Solution: Integrated search?
But…
Development handed over to large
commercial vendors
or giant web search services
(Google; Amazon)
Where does that leave us as a field,
as practitioners and researchers?
23. PAIRI
Research opportunities
There is sooo much we don’t know:
How to optimise each type of information
object?
How to integrate information objects?
How to do relevance feedback in
integrated search?
How to best involve users in testing?
etc etc…
24. PAIRI
Research opportunities
So, we can study various elements
and details…
But how to generate real knowledge
of these colossal integrated
systems?
Larger, more complex than ever
How to evaluate these scientifically, and not
just rely on vendors?
25. PAIRI
Information Retrieval (IR) test
collection
Purpose: to facilitate studies of
integrating different document types
iSearch test collection
26. PAIRI
Three types of documents from
physics
18,841 book records
291,244 metadata records, incl. abstract
143,569 full text articles
iSearch test collection
27. PAIRI
65 thoroughly described Information tasks
1. Information need -What are you looking for?
2. Work task context -Why are you looking for this?
3. Knowledge state - Background knowledge of topic?
4. Ideal information -To solve your problem or task?
5. Search terms -Which search term would you use?
+ graded relevance assessments
iSearch test collection
28. PAIRI
What can we do with this?
Study these tasks
Optimise each document type
Optimise the integration
Simulate relevance feedback and interfaces?
What we can’t do
Study another domain
Study real user interaction
iSearch test collection
29. PAIRI
Adaption through integration
of information objects
Pertinent issues
How to design and evaluate systems that
successfully integrate genres, media and
document types?
Even with shallow data?
How to generate real knowledge and evaluate
these scientifically?
Without relying on what vendors offer?
35. PAIRI
Schultz, C. K. (1968). H.P. Luhn: Pioneer of Information Science
(p.32). London, UK: American Documentation Institute.
1950s: Luhn’s Selective
Dissemination of Information
36. PAIRI
1960s: Salton, Lesk, Rocchio,
Ide and Others
Ide, E. (1967). User interaction with an automated information retrieval system.
In G. Salton (Ed.) Information Storage and Retrieval: Scientific Report No. ISR-12.
37. PAIRI
Oddy, R. N. (1977). Information retrieval through man-machine dialogue. Journal of
Documentation, 33(1), 1-14.
1970s: Oddy’s Thomas
38. PAIRI
1980s: USER-MODEL-O-SAUR
Allen, R. B. (1990). User models:Theory,
method, and practice. InternationalJournal
of Man-Machine Studies, 32, 511-543.
Rich, E. (1983). Users are individuals:
Individualizing user models. International
Journal of Human-Computer Studies, 51,
323-338.
“While the term ‘user model’ emphasizes
the information about the person, it is
obvious that a great deal of situational,
task, or environmental information may
be encoded in the model.”
39. PAIRI
Croft,W. B., &Thompson, R. H. (1987). I3R: A new approach to the design of document
retrieval systems. Journal of the American Society for Information Science, 38, 389-404.
1980s: Intelligent IR
40. PAIRI
Maes, P. (1994). Agents that reduce work and information
overload. CACM, 37(7), 30-40.
1990s: Agents
42. PAIRI
Belkin, N. J., Cool,C., Kelly, D., Lin, S.-J., Park, S.Y., Perez-Carballo,J., & Sikora, C. (2001). Iterative
exploration, design and evaluation of support for query reformulation in interactive information retrieval.
Information Processing & Management 37(3), 404-434.
And more relevance feedback
43. PAIRI
What Caused Explicit-O-Saurus’
Demise?
Users are unwilling to put forth the effort
required to provide feedback
Users don’t have the additional cognitive
resources to engage in explicit feedback
45. PAIRI
Users are unwilling to put forth the effort required
to provide feedback.
http://www.pewinternet.org/
http://www.visualeconomics.com/how-the-world-spends-its-time-online_2010-06-16/
REALLY?
47. PAIRI
Users don’t have the additional cognitive resources
to engage in explicit feedback.
Well, maybe back then …
48. PAIRI
We need to figure out better ways of eliciting
feedback from users:
Better questions
Better measures
More creative
More engaging
More adaptive
Saving Explicit-O-Saurus
50. PAIRI
Saving Explicit-O-Saurus
Questions
How can we create better questions and measures?
How can we make the process creative, engaging
and adaptive?
Objection
Users are creatures of habit.
Let’s reintroduceTECHNIQUE
EXPLICIT-O-SAURUS
into the information ecology
Users will adapt …
51. PAIRI
Users’ Adaptability in IT
Environment
Adaptivity is a much needed system
functionality, yes but
System alone cannot solve all the
interaction problems
Humans are evolutionary learners
IT users must / can adapt to the system
53. PAIRI
Psychological View
Mooers’ Law?
Satisficing principle
Today, majority IT users
Do not really have a choice at one level
Do have choices at another level
55. PAIRI
Constructivist View
Problem-based learning
Social learning
• Observational learning specially relevant to
building IT skills
• Knowledge sharing
• Collective intelligence
56. PAIRI
Adaptability — What is it?
AdaptAbility is a key
metacompetency
which enables
individuals quickly,
effectively respond
to unexpected
environmental
changes
57. PAIRI
Adaptability in Contexts
In workplace: job adaptability
In multicultural society: cross-cultural
adaptability
In information environment: IT
adaptability
59. PAIRI
Job Adaptability
Creative problem solving (new …)
Dealing Uncertainty/unpredictable work
situations
Learning new tasks, tech, procedures …
Adapting culturally
Adapting interpersonally
Physical factors
60. PAIRI
Cross-Cultural Adaptability
Inventory (CCAI)
A survey of 50 questions to uncover individuals’
current strengths and weaknesses within four
critical skill areas proven necessary for
effective cross-cultural communication:
Emotional Resilience
Flexibility/Openness
Perceptual Acuity
Personal Autonomy
61. PAIRI
Adaptive Style Inventory
• Acting situation
• new
• on time
• Deciding Situation
• two alternatives
• an opportunity
• Thinking
• an idea
• analyzing something
• Valuing
• my feelings
• see the world as the another
person sees it
--Kolb Learning Style Inventory?
62. PAIRI
IT Adaptive Behaviour
We need systematic research to
understand factors underlying the IT
adaptive behavior
We need instruments to measure IT
adaptability (either adopt existing or
develop new relevant one)
63. PAIRI
Where do we start?
Adaptive situations
IT system being updated
Need a function never used
New IT system
“I have used it before, but how do I get to it
now”?
“The output doesn’t look right”!
“Where are my files I saw all of them just a
moment ago?! Or, no, I cannot lose them ...”
65. PAIRI
AdaptAbility vs. LearnAbility
User adaptability calls for the system’s
learnAbility
IT system learnability support user’s
adaptAbility
learnability is an aspect of usability –
rarely studied
67. PAIRI
Personalizing & Humanizing
Personalizing: Beyond user profiling
IT knowledge state
IT competency
Adaptive style
Humanizing : Beyond affordances
Demonstrate/encourage empathy
Affective responses
68. PAIRI
IT Affordance
Specifies the range of possible
actions about an object (physical
or digital)
Must be visible to the users to be
usable
Individual differences in perceived
affordances
When affordances fail users: an
action does not result in an
expected result --?
69. PAIRI
Building IT Adaptability as
Social Learning
Sharing learning experience
Transferring knowledge
Collecting user problems
Banking user strategies
70. PAIRI
Study of User Adaptability & IT
Design
How do we study users adaptive behavior
in adopting a new system, new
applications, new functions, new …?
How can personal adaptive behaviors help
system design?
71. PAIRI
Adaption to context and practice
Focus on adapting information
architecture (IA)
Focus on models and methods for
developing tailored information
architecture (IA)
72. PAIRI
Quick definition of IA (1)
ORGANISATION
SYSTEM
LABEL
SYSTEM
Structural design
Combination of
organisation, labelling,
search and navigation
74. PAIRI
Adapting IA
Organization system, e.g.
Categories
– tailored to user needs
Organization
– organized to user perspective
Labels, e.g.
Terminology
– jargon, language of youth, basic level
75. PAIRI
Adapting IA
Navigation system, e.g.
Tailored short cuts
Tailored links
Tailored recommending,
Search system, e.g.
Tailored filters
Tailored ranking
80. PAIRI
System development models
Plenty of models and approaches:
User-oriented
Domain-oriented
Work-task oriented
Interaction-oriented
Participation-oriented
…
All believe that we can grasp the context
81. PAIRI
System development models
Can we
grasp,
understand,
model the context and practice
– and adapt?
Are we
“heroes” between
“victims” and “tyrants”?
(Spinuzzi, 2004)
82. PAIRI
Borger.dk – an example
Development of organisation system for
Danish Government portal Borger.dk
83. PAIRI
Borger.dk – an example
Timeline
2008 Analysis, development, implementation
2009 Evaluation: usability tests, online survey
and search log analysis
2009 Redesign and implementation of
updated version and new features
84. PAIRI
Borger.dk – design methodology
Development phrase
Survey of domain of public digital
communication
Personas – user needs and behaviour
Bench marking – related portals and
organisations systems
Content analysis
Expert evaluation
Usability tests - 12 citizens completing 4 tasks
86. PAIRI
Borger.dk - redesign
Information needs
• 75% of needs met by categories
• Miss personal and factual information
Categories, organization, navigation
• Unclear labels
• Short cuts - to frequent categories and forms
• Cross references
• Prioritized ordering
• Search
• Synonym rings
• Best bets
Marianne Lykke & Brian
KirkegaardLunn
90. PAIRI
??? Adaption to context ???
Do “heroes” exist within system design
and adaption?
What models are feasible to adaption to
context and practice?
What tools should be developed?
92. PAIRI
WRAPPING UP the PANEL
Interesting discussions on:
Design and evaluation of integrated
systems?
How to avoid re-engineering of
vendor offerings?
Feasibility of explicit relevance
feedback for systems adaptivity?
93. PAIRI
WRAPPING UP the PANEL - 2
Interesting discussions on:
How does personal adaption behavior
help systems (design)?
Research methdology and tools
studying adaptive behavior to new ...?
How to adapt to context and work
practice?