Using Qualitative Data Analysis Software Michelle C. Bligh, Ph.D. Claremont Graduate University March 18, 2005

1. Using Qualitative Data
Analysis Software
Michelle C. Bligh, Ph.D.
Claremont Graduate University
March 18, 2005

2. Why Qualitative Assessment?
“Study the box.”

3. What is Qualitative Research?
"Qualitative inquiry is an umbrella term for
various philosophical orientations to interpretive
research.” - Glesne and Peshkin (1992)
"Qualitative research is a loosely defined
category of research designs or models, all of
which elicit verbal, visual, tactile, olfactory, and
gustatory data in the form of descriptive
narratives like field notes, recordings, or other
transcriptions from audio- and videotapes and
other written records and pictures or films.”
-Preissle

4. Advantages of Qualitative
Research
Greater depth and detail
Richness and holism
Flexibility/lack of constraints
Focus on naturally occurring, ordinary events in
their natural settings
Data are collected in close proximity to the
situation
Influences of context are not stripped away
Allow emphasis on processes, of how and why
rather than just what

5. Advantages of Qualitative
Research (continued)
Undeniability
Lead to new integrations/interpretations
Can avoid pre-judgments/halo effects
Consistency
Supplement, validate, explain, illuminate,
or reinterpret quantitative data

6. Disadvantages of Qualitative
Research
Extremely time-consuming/labor intensive
Data overload
Subjectivity/researcher bias
Reactivity
Dependent on researcher’s attributes/skills
Psychologically draining

7. Sources of Data
Open-ended questions
Logs, journals, or diaries
Observations
Stories
Case studies
Individual ‘interviews’/Oral exams
Discussion groups/Focus groups
Etc.

8. Your Approach Depends On…
1. The focus of your study and the themes
you want to address
2. The needs of those who will use the
information
3. Your resources (time, energy, money,
software available)

9. Qualitative Analysis (Miles & Huberman)
Data reduction
– Selecting, focusing, simplifying
Data display
– Creating organized, compressed
representations of information
Conclusion Drawing and Verification
– Deciding what things mean and testing them
for plausibility/validity

10. Coding
 Coding is analysis
 Codes are tags or labels for assigning units of
meaning to the descriptive or inferential
information compiled
 It is the meaning that matters
 Codes are used to retrieve and organize the
chunks of information, so you can quickly find,
pull out, and cluster the segments relating to a
particular topic

11. Types of Codes
Descriptive: attributing a class of
phenomena to a segment of text (e.g.,
spelling)
Interpretive: include a more complex,
underlying meaning (e.g., unsupported
argument)
Pattern: inferential and explanatory; group
codes into a smaller number of themes or
constructs; analogous to cluster and factor
analysis in statistics (e.g., thoroughness)

12. The process of coding
 Create a provisional “start list”
– Usually anywhere from 12 – 60
– Get them on a single page for reference
– Make sure they are organized/structured
 Create code definitions
 Revise coding scheme
– Filling in: adding, reconstructing preexisting codes
– Extension: recoding with a new theme or insight
– Bridging: seeing new relationships
– Surfacing: identifying new categories

13. The process of coding (cont.)
 Structure is key: codes should relate to one
another, they should be part of a governing
structure
 Structure includes larger, more conceptually
inclusive codes, and smaller, more differentiated
codes
 Pattern codes should represent a web of meaning
that is grounded in the data

14. Uses of Qualitative Software
Data reduction
– Retrieving text that has pre-determined
significance
Text exploration
– Helping researcher recognize underlying
themes of the text

15. Advantages of CAQDAS
Makes the sheer volume of data more
manageable
Helps to selectively retrieve information
– Can summarize results in structured lists and tables
Helps to evaluate the weight of supporting vs.
non-supporting data
– Can report results in comparative ways
Helps to provide linkages to other types of data
and perspectives
– Can integrate qualitative and quantitative data

16. Types of CAQDAS
Text retrieval
– Examples: the General Inquirer, CATA,
TEXTPACK, WordStat, Diction, ZyINDEX,
The Text Collector
Text analysis
– Examples:
• Atlas/TI,
• ETHNOGRAPH,
• NUDIST

17. How to Choose
What kind of computer user am I?
Am I choosing for one project or for many?
What kind of projects and databases will I be
working on?
What kinds of analyses am I planning to do?
How important is it to maintain close
proximity to the data?
What are your financial constraints/access to
programs?

18. Text Retrieval Programs
Designed to search for, retrieve, and/or count
words and phrases
Search programs
– Used in preliminary data analysis to determine
whether and where pre-specified words and phrases
appear and in what context
Content Analysis programs
– Take inventories (make frequency distributions) of all,
or pre-specified, words contained in text

19. Text Retrieval: Primary Questions
What words are addressed in a text?
Where are particular words used in a text?
How do documents differ in terms of
vocabulary usage?
What concepts are addressed in a text?
To what extent are concepts of interest
addressed in a text?

20. Typical Features of Text
Retrieval Programs
Generate text frequency distributions
Generate vocabulary comparisons among
different texts
Work with key-word lists
Generate key-word in context lists (KWIC)
Search for root words (innovat*)
Generate words category counts and statistics
Conduct proximity searches (w/i 5 words)
Conduct Boolean operator searches (innovation if
creativity not w/i 5 words)

21. Text Analysis Programs
Developed explicitly for the purposes of
description, interpretation, and theory building
Facilitate identifying and coding elements of
theoretical interest, establishing relationships and
building connections
A.k.a. Code-and-Retrieve Programs
(HyperQual2, Kwalitan, the Data
Collector)/Code-Based Theory Builders
(ATLAS/ti/NUDIST, Code-a-Text)

22. Primary Questions
How often do specific codes occur?
How often do specific code sequences
occur?
Are code sequences indicative of themes?
Are code linkages indicative of conceptual
relationships?

23. Primary Functions of Text
Analysis Programs
Attaching codes to segments of text
Searching for and assembling coded
segments of text
Searching for code sequences (look for
closely related or overlapping codes to
identify patterns and relationships)
Counting frequencies of codes, code
sequences, or counter-evidence

24. Practical Issues
Different types of programs can be used in
concert or sequentially
Text must be computer readable:
transcription, scanning, or importing
Special attention must be paid to formatting
issues
All CQDA programs still require
interpretation on the part of the researcher

25. Practical Issues (continued)
Reliability problems usually due to the ambiguity of
word meanings, category definitions, or coding rules
Construct validity: constructs should be correlated
with other measures of the same construct
Hypothesis validity: constructs should relate in
theoretical ways to other constructs
Face validity: constructs should appear to measure
what they do
Semantic validity: persons familiar with the
language of the texts should agree that the list of
words in a category have similar meanings

26. Advantages of CAQDAS
Stability of the coding scheme leads to increased
consistency
Explicit coding rules yielding comparable results
across multiple graders and over time
Saves time, freeing instructor to focus on
interpretation and explanation
Easy manipulation of text to create different
types of output and emphases
Ability to process large amounts of data in less
time and saves paper

27. Limitations of Text Retrieval
Programs
Lack of natural language processing capabilities
(ambiguous concepts, broader context is lost)
Insensitivity to negation, irony, tone
Inability of researcher to provide a completely
exhaustive listing of key words
Inability of software to resolve references back
and forth to words elsewhere in the text
Can result in “word crunching”: transforming
rich meanings into meaningless numbers

28. Limitations of Text Analysis
Programs
Initial time investment
Initial monetary investment
Output can be tricky for students
Can lead to a tendency to focus on details
rather than the big picture
They don’t do the analysis for you!