Connaway, Lynn Silipigni, and Marie L. Radford. 2018. "Survey Research." Webinar presented by ASIS&T, January 23.

1. ASIS&T Webinar • 23 Jan. 2018
Survey Research
Lynn Silipigni Connaway, PhD
Senior Research Scientist &
Director of User Research, OCLC
connawal@oclc.org
@LynnConnaway
Marie L. Radford, PhD
Professor of Library & Information
Science, Rutgers University
mradford@rutgers.edu
@MarieLRadford

2. Senior Research Scientist & Director of
User Research, OCLC
Lynn Silipigni Connaway, PhD
Professor of Library & Information Science,
Rutgers University
Marie L. Radford, PhD

3. Agenda
• Popular LIS Research Methods
• Survey Research (Types,
Sampling, Advantages,
Disadvantages, Design)
• Data Analysis Tools & Methods
• Example: Seeking Synchronicity
• Mixed Methods Research
• Presentation of Findings

4. MOST WIDELY USED RESEARCH
METHODS IN LIS

5. Methodology 1950 1960 1965 1970 1975 Total* %*
Theoretical-analytical 11 17 11 36 52 127 14
Information system design 7 16 21 57 49 150 17
Surveys on the public 3 2 9 20 19 53 3
Survey or experiment on libraries, etc. 22 15 45 89 113 284 32
Bibliometric and similar studies 0 1 7 14 16 38 4
Content analysis 0 1 2 1 3 7 1
Secondary analysis 6 15 15 13 27 76 8
Historical methodologies 21 26 25 49 42 163 18
Descriptive bibliography 7 4 6 4 9 30 3
Comparative studies 0 2 6 4 7 19 2
Other and multiple 3 1 7 9 10 30 3
All papers* 76 96 139 274 315 900 100
Research Papers by Methodology & Year
(Powell, 1999)

6. Research Methods: JDoc 2001-2010
• N=367
• Theoretical approach, 38%
• Content analysis, 14%
• Questionnaire, 13.8%
• Experiment, 13.4%
• Interview, 13.4%
(Chu, 2015)

7. Research Methods:
JASIS&T 2001-2010
• N=554
• Experiment, 31%
• Bibliometrics, 23%
• Questionnaire, 14%
• Content analysis, 13%
• Theoretical approach, 12%
(Chu, 2015)

8. Research Methods:
LISR 2001-2010
• N=241
• Content analysis, 30%
• Questionnaire, 28%
• Interview, 20%
• Theoretical approach, 15%
• Experiment, 9%
(Chu, 2015)

9. Research Methods: JAL 2004-2013
Method Percentage (n=346)
Questionnaire 47.6
Test or Quiz 2.6
Diary 0.6
Content Analysis 27.2
Semi-structured Interviews 14.0
Analysis of existing statistics 6.6
Citation Analysis 6.3
Focus Group Interview 5.7
Observation 4.3
Log Analysis 3.4
Task Analysis 2.9
(Luo & McKinney, 2015)

10. SURVEY RESEARCH

11. Survey Research Advantages
• Explores many aspects of service
• Demographic information
• Controlled sampling
• High response rates possible
• Data reflect characteristics and opinions of
respondents
• Cost effective
• Can be self-administered
• Survey large numbers
• Online surveys (e.g., Survey Monkey) provide
statistical analysis

12. • Produces a snapshot of
situation
• May be time consuming to
analyze and interpret results
• Produces self-reported data
• Data lack depth of interviewing
• High return rate can be difficult
Survey Research Disadvantages

13. • Exploratory surveys
• Increase familiarity
• Clarify concepts
• Direct future research
• Descriptive surveys
• Includes analytical surveys
• Describe characteristics/population of
interest
• Estimate proportions in the population
• Make predictions
• Test associational relationships
• Explore causal relationships
Types of Surveys

14. • Nonprobability Sampling
• Accidental sample
• Quota sample
• Snowball sample
• Purposive sample
• Self-selected sample
• Incomplete sample
Sampling
• Probability Sampling
• Simple random
sample
• Systematic sample
• Stratified random
sample
• Cluster sample

15. Checklist for Designing Surveys
Good questions are:
• Related to problem at hand
• Multiple choice
• Open-ended
• Likert Scale
• Clear, unambiguous, precise
• Able to be answered by subjects
• Not double-barreled (“and”)
• Short
• Not negative
• Unbiased

16. • Paper or online (e.g., Survey Monkey)
• Consider order of questions
• Demographic q’s first
• Instructions
• Be specific
• Introduce sections
• Keep it simple
• Pretest!
Design Issues

17. DATA ANALYSIS TOOLS AND
METHODS

18. Ethnographic Analysis
• Use people’s own categories
• Avoid assuming what one will find
• Complementary to quantitative methods
• Retain ‘richness’/‘thick description’
• Numerical compatibility
(Asher 2017, 264)
(Connaway & Radford 2017, 282)

19. • Contain all data sources
• Create and apply codes
• Enable running queries
• Develop visualizations
• Deliver reports
(Connaway & Radford 2017)
Computer-Assisted Qualitative Data Analysis
Software (CAQDAS)

20. • Draw on data...in service of
developing new conceptual
categories
• Develop inductive abstract analytic
categories through systematic data
analysis
• Emphasize theory construction
rather than description or
application of current theories
(Connaway & Radford 2017)
Grounded Theory

21. “A major strategy for analysis of
qualitative data is the use of the
constant comparative method, which
embraces ‘constant comparisons’
defined as ‘the analytic process of
comparing different pieces of data
against each other for similarities
and differences.’”
(Connaway & Radford 2017, 298)

22. Quantitative Analysis
• Descriptive Statistics
• Survey Monkey
• Qualtrix
• Parametric and Non-parametric
Statistics
• SPSS
• SAS

23. Coding
Analyzing the gathered data involves coding the
responses (or placing each item in the appropriate
category), tabulating the data, and statistical
computations.
Categories for coding generally evolve from the data,
there are some common areas such as setting,
situation, perspective, process, activity, event,
relationship, role, practice, etc.

24. Interviews
• Conduct
• Transcribe
Codebook
• Themes
• Build
Codebook
NVivo
• Load
• Insert
Codebook
ICR
• Code portion
• Run ICR
Coding
• Code
Transcripts
• Upload to
NVivo
Analysis
• Export
Results
• Calculations

25. Interpreting Results
• Objectively analyze all data
• Interpret results with appropriate level of
precision
• Express proper degree of caution about
conclusions
• Use data as input in outcome measures
• Consider longitudinal study, compare
results over time
• Qualitative data requires special attention

26. EXAMPLE:
SEEKING SYNCHRONICITY

27. • Review the literature
• Focus group interviews with VRS librarians, users
and non-users
• Analyze VR transcripts
• VRS librarian, user, and non-user interviews and
online surveys
Formatting Objectives
Seeking Synchronicity was funded by IMLS, OCLC, and
Rutgers University. See more at http://www.oclc.org/research/
activities/synchronicity/default.htm.

28. Critical Incident Technique (CIT)
• Online surveys with CIT
• Flanagan (1954)
• Qualitative technique
• Focuses on most
memorable
event/experience
• Allows categories or
themes to emerge
rather than be imposed

29. CIT: VRS User Online Survey Questions
• Think about one experience using VRS in
which you felt achieved (or did not achieve) a
positive result
• Please describe the circumstances and
nature of your question.
• Describe why you felt the encounter was
successful (or unsuccessful).
• Did the chat format help your experience
to be successful (or unsuccessful)? If yes,
how?

30. • Think about one experience in which you felt you
achieved (or did not achieve) a positive result after
seeking library reference services in any format.
• Think about one experience in which you felt
you did (or did not) achieve a positive result
after seeking library reference services in any
format.
• Describe each interaction.
• Identify the factors that made these interactions
positive or negative.
CIT: VRS Non-User Online Survey Questions

31. CIT: VRS User Result
“The librarian threw in a cordial
sign-off and encouraged me to
pursue the reading. It was like
talking to a friendly librarian in
person.”
– VRS User (online survey)

32. MIXED METHODS RESEARCH

33. Mixed Methods Research
• Any combination of research
methods
• Qualitative
• Quantitative
• Participatory
• Action
• Design
• Equal attention to all stages of
research process
• Findings should be iterative and
informative
(Kazmer 2017, 232-233)

34. Triangulation
• Term coined by Webb et al. (1966)
• Multiple methods of data collection
(e.g., interviews – individual and group,
observation, literature, archives)
• Agree, or at least don’t contradict
(Miles & Huberman 1994, 266)
• Multiple investigators
• Multiple contexts/situations

35. Mixed Methods
“Perceived benefits of
mixing methods in ‘getting
research done’ came to be
seen as outweighing the
importance of the
philosophical difficulties in
their use.”
(Miles & Huberman, 1994)

36. Survey Research in Mixed Methods
• Survey development can follow and be
informed by initial data collection (e.g., it
can follow interviews or focus groups)
• Survey helps overcome limitations of
qualitative methods
• Can be administered to random sample
from large population
• Provides ability to generalize results
• Can confirm (triangulate) qualitative
findings

37. Justifications for Combining Mixed Methods
• Triangulation: convergence, corroboration,
correspondence, or results from different
methods.
• Complementarity: elaboration, enhancement,
illustration, clarification of results from one
method w/ results from another
• Development: uses results from one method to
help develop or inform another
• Initiation: discovery of paradox and contradiction,
new perspectives / frameworks, recasting of
questions or results from one method to another
• Expansion: extends breadth and range of enquiry
by using different methods for different inquiry
components

38. Key Questions in Implementing Mixed
Methods & Multi-Strategy Research
1. Are the methods employed simultaneously or
sequentially?
2. Which method, if any has priority? And why?
3. What is the function of the integration – for
example, triangulation, explanation, or exploration?
4. Are quantitative and qualitative data collected
simultaneously or sequentially?
5. How do the mixed methods impact on data analysis
– analysis by method & analyzed sequentially,
integrated analysis, cross-method analysis?

39. PRESENTATION OF FINDINGS

40. The Research Report
• Objectives:
• Acquaint readers with the problem
investigated
• Present study results
• Explain implications and importance
• Methods of delivery:
• Written
• Graphic
• Oral

41. Components of the Research Report
• Title
• Abstract
• Introduction and problem
• Literature review
• Conceptual framework
• Study design/methodology
• Results
• Summary and conclusions
• References/bibliography

42. Graphics
• May present information and
data more clearly and succinctly
• Keep it:
• Simple
• Clear
• Accurate
• Self-contained
• Reference in the text

43. Oral Presentations
• Plan and practice before giving your presentation
• Make an initial impact
• Establish rapport with your audience
• Stick to a few key points
• Make a strong conclusion
• Allow time for questions and discussion
• Be confident and enjoy yourself!

44. “The creative process is not like a situation
where you get struck by a single lightning
bolt. You have ongoing discoveries, and
there are ongoing creative revelations. Yes,
it's really helpful to be marching toward a
specific destination, but, along the way, you
must allow yourself room for your ideas to
blossom, take root, and grow.”
– Carlton Cuse

45. "When you take risks, you learn that there will be
times when you succeed and there will be times
when you fail, and both are equally important."
- Ellen DeGeneres

46. SM
Lynn Silipigni Connaway, PhD
connawal@oclc.org
@LynnConnaway
Marie L. Radford, PhD
mradford@rutgers.edu
@MarieLRadford
Questions & Discussion

47. ACKNOWLEDGEMENTS
The authors would like to thank Brittany Brannon for her
assistance in preparing this presentation.

48. References
Asher, A. (2017). “On ethnographic research: How do students find the information they need?”
In L. S. Connaway & M. L. Radford (Eds.), Research methods for library and information Science
(6th ed.). Westport, CT: Libraries Unlimited.
Chu, H. (2015). Research methods in library and information science: A content analysis.
Library & Information Science Research, 37(1), 36-41.
Connaway, L. S., & Radford, M. L. (2011). Seeking synchronicity: Revelations and
recommendations for virtual reference. Dublin, OH: OCLC Research. Retrieved from
http://www.oclc.org/reports/synchronicity/full.pdf
Connaway, L. S., & Radford, M. L. (2017). Research methods for library and information science
(6th ed.). Westport, CT: Libraries Unlimited.
DeGeneres, E. (2011). Seriously…I’m kidding. New York: Grand Central Publishing.
Flanagan, J. C. (1954). The critical incident technique. Psychological Bulletin, 51(4), 327-358.
Hernon P., & Altman, E. (1998) Assessing service quality: Satisfying the expectations of library
customers. Chicago: American Library Association.

49. References
Janes, J. (1999). Survey construction. Library Hi Tech, 17(3), 321-325.
Kazmer, M. (2017). “Mixed methods.” In L. S. Connaway & M. L. Radford, Research methods
for library and information science (6th ed.) (pp. 232-233). Westport, CT: Libraries Unlimited.
Luo, L., & McKinney, M. (2015). JAL in the past decade: A comprehensive analysis of
academic library research. The Journal of Academic Librarianship, 41(2),123-129.
Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis: A sourcebook. Beverly Hills:
Sage Publications.
Miles, M. B., Huberman, A. M., & Saldaña, J. (2014). Qualitative data analysis: A methods
sourcebook (3rd ed.). Thousand Oaks, CA: Sage.
Powell, R. (1999). Recent trends in research: A methodological essay. Library & Information
Science Research, 21(1), 91-119.
Webb, E. J., Campbell, D. T., Schwartz, R. D., & Sechrest, L. (1966). Unobtrusive measures:
nonreactive research in the social sciences (Vol. 111). Chicago: Rand McNally.

50. Image Attributions
Slide 3: https://www.flickr.com/photos/matso/8168817100/
Slide 6: https://flic.kr/p/brnnFP
Slide 7: https://www.flickr.com/photos/bull3t/3271752011
Slide 8: https://flic.kr/p/8HwEak
Slide 11: https://www.flickr.com/photos/evilmutent/3440911430
Slide 12: https://www.flickr.com/photos/joceykinghorn/12800643745
Slide 13: https://www.flickr.com/photos/pixel_addict/1842285901/
Slide 14: https://www.flickr.com/photos/mauroescritor/17137445937/
Slide 15: http://www.flickr.com/photos/didmyself/6530383391/
Slide 16: https://www.flickr.com/photos/deanhochman/14481958642/
Slide 18: https://www.flickr.com/photos/respres/6041698098/
Slide 19: https://www.flickr.com/photos/flakepardigm/4687752030/
Slide 20: https://www.flickr.com/photos/adambindslev/4727853016/
Slide 21: https://www.flickr.com/photos/carleycomartin/4030726428/
Slide 22: https://www.flickr.com/photos/larimdame/236343262/
Slide 23: https://www.flickr.com/photos/davebass5/14680844817/
Slide 25: https://www.flickr.com/photos/thomashawk/307750722/

51. Image Attributions
Slide 28: https://www.flickr.com/photos/carnesaurus/32160924864/
Slide 29: https://www.flickr.com/photos/nordforsk/32225539214/
Slide 30: https://www.flickr.com/photos/elliotmar/8913124849
Slide 31: https://www.flickr.com/photos/therichbrooks/2600080329/
Slide 33: https://www.flickr.com/photos/edgewoodchembiocenter/32447188374/
Slide 34: https://www.flickr.com/photos/wandrus/8434921527/
Slide 35: https://www.flickr.com/photos/ikhlasulamal/4538617347/
Slide 36: https://www.flickr.com/photos/dougww/2132598574
Slide 37: http://bit.ly/2kqpi8S
Slide 38: https://www.flickr.com/photos/dawncreations/2894171562
Slide 40: https://www.flickr.com/photos/gotcredit/33371980600/
Slide 41: https://www.flickr.com/photos/dannynic/7118094011/
Slide 42: https://www.flickr.com/photos/121021623@N02/15114540980/
Slide 43: https://www.flickr.com/photos/mukluk/307433536/
Slide 44: https://flic.kr/p/AuDyn
Slide 45: https://flic.kr/p/bq5zmW