Mixed methods

1. MIXED
METHODS
9715 – Doctoral Seminar
Notes by M. Larsen
Nov. 28, 2014
‘The Third Methodological Movement’
‘The Third Path’
‘A research paradigm whose time has
come.

3. WHY MIXED METHODS
DESIGN?
To provide a more
complete picture of the
phenomenon under
study
To increase data validity
Enable researcher to
develop analysis and
build on original data
MM research – “can help
to clarify the formulation of
the research problem and
the most appropriate ways
in which problems or
aspects of problems may
be theorised and
studied….With multiple
methods the researcher
has to confront the
tensions between different
theoretical perspectives
while at the same time
considering the
relationship between the
data sets produced by the
different methods”
(Brannen, 1992, pp. 32-33)

4. CRITICS OF MIXED
METHODS
‘Incompatibility thesis’ - qualitative and
quantitative research paradigms cannot
and should not be mixed
(purists – paradigm wars)
Critics of MM - different research
paradigms embody incompatible
assumptions about the nature of the
world and what is important to know

5. CHALLENGING THE DIVIDE BETWEEN
QUALITATIVE AND QUANTITATIVE
RESEARCH
Hammersley (1992) “Deconstructing the
qualitative-quantitative divide”
- the distinction between these
methodological paradigms is limited
and dangerous
Presents 7 issues concerning the
quant/qual distinction and challenges
each of them.

6. 1) Qualitative vs quantitative
data
Assumptions:
Qual research - words
Quant – numbers
But… this distinction is problematic
- large proportion of research reports combine
both
- ethnographers use words like “Regularly”,
Frequently, Often, Sometimes, Generally,
Typically
If this is about precision, then precision doesn’t
necessarily mean the use of numbers

7. 1) Qualitative vs quantitative data
cont’d.
“We are not faced, then, with a stark
choice between words and numbers, or
even between precise or imprecise data.
Furthermore, our decisions about what
level of precision is appropriate in relation
to any particular claim should depend on
the nature of what we are trying to
describe, on the likely accuracy of our
descriptions, on our purposes, and on the
resources available to us; not on
ideological commitments to one
methodological paradigm or another.”
(p.43)

8. 2) The investigation of natural vs
artificial settings
Assumptions:
Quant – artificial setting – experimental
Qual – natural
But…this distinction is spurious
What happens in classroom is not
necessarily more natural than what goes
on in a psych lab
“To treat classrooms…as natural and
experiments as artificial is to forget that
social research is itself part of the social
world.” (p. 44)

9. 2) The investigation of natural vs
artificial settings
Reactivity – individuals alter their
performance or behavior due to the
awareness that they are being observed -
both quant and qual research can lead to
reactivity(Hawthorne effect)
“The terms ‘natural’ and ‘artificial’ have
misleading connotations. And while the
issue of ecological validity is important, it is
not the only important methodological issue.
Nor does research in ‘natural’ settings
guarantee ecological validity, any more than
research in ‘artificial’ settings automatically
debars us from it.” (p. 45)

10. 3) A focus on meanings rather
than behavior
Assumptions:
Qual research - interpretive (meaning)
Quant research – positivist (behaviours)
But… rare that qual research simply
documents point of view of participants
Researcher is involved in interpretation of
data
Much quant research concerned with
attitudes not just behavior

11. 3) A focus on meanings rather
than behavior
“As regards differences in the approach
that attitude researchers and
ethnographers employ in identifying
attitudes/perspectives, the contrast is
between the use of attitude scales and
more unstructured approaches…Here
again, we do not have a clear-cut
distinction between two contrasting
approaches.” (p. 46)

12. 4) Adoption or rejection of natural
science as a model
Assumptions:
Qual – reject natural science as model
Quant –natural science – exemplary
“Not even the most extreme positivist
would argue that the methods of physics
should be applied lock, stock and barrel to
the study of the social world. And there are
few supporters of qualitative research who
would insist that there is no aspect of
natural science method that is relevant to
social research. What is involved here is a
matter of degree.” (p. 47)

13. 5) An inductive vs deductive
approach
Assumption:
Quant = deductive or hypothetico-deductive
Qual = inductive
But… “Quant research does not always
test hypotheses: its goal is often
descriptive.” (Brannen, 1992, p. 8)
Some quant research is concerned with
theory generation
Some qual research is deductive
Lots of qual research is simply descriptive

14. 5) An inductive vs a deductive
approach
All research involves induction and deduction
to some degree– impossible that researchers
not be influenced by prior knowledge
“What is true is that one can distinguish
between studies that are primarily exploratory,
being concerned with generating theoretical
ideas, and those which are more concerned
with testing hypotheses. But these types of
research are not alternatives; we need both.
Nor need the former be quantitative and the
latter qualitative in other senses of those
terms.” (p. 48)

15. 6) The identification of cultural
patterns as against seeking scientific
laws Assumption:
Quant –committed to discovery of scientific laws
Qual – committed to identifying cultural patterns
Yet… much quant is concerned with description
Early qual researchers justified their practice by
claiming that it produced scientific laws and even today
they claim their goal is theory generation
“Thus the distinction between identifying patterns
and pursuing laws seems to provide little clear
basis for the division between quant and
qualitative methods.” (p. 50)

16. 7) Idealism vs realism
Assumption:
Quant – realist epistemology
Qual- idealist
“More important than the empirical question of
whether it is true that quant researchers are
realists and qual researchers idealists, though, is
the philosophical issue of whether there is any
necessary connection between qual method and
a particular epistemological position….history
suggest that there is little reason to believe that
there is such a connection. And we must
remember that there are many more than 2
epistemological positions available.” (p. 51)

17. METHODOLOGY OF COMBINING
APPROACHES – CONSIDERATIONS
1) Timing - What will the timing of qual
and quant methods be? What will the
order be? Will it be concurrent or
consecutive?
2) Weighting Dimension -What will be
the relative importance, weight or
priority, given to qualitative &
quantitative methods?
Equal
Emphasis on Qualitative or
Quantitative

18. METHODOLOGY OF COMBINING
APPROACHES – CONSIDERATIONS
cont’d.
3) Mixing Dimension - How will qual and
quant methods be mixed? How will the
2 data sets be mixed?
Merged
Embedded within one another
Connected in another way
Complementary
Integration

19. Fundamental Principle of Mixed
Methods Research
Combine the methods in a way that
achieves complementary strengths and
non-overlapping weaknesses (Johnson
& Onwuegbuzie, 2004)
1) Timing
2) Weighting
3) Mixing Dimension

20. MIXED METHODS DESIGNS
1) Triangulation/ Multiple Methods
Within-method – the same method
being used on different occasions
Between-method – using different
methods in relation to the same topic
Purpose of triangulation – to obtain
complementary quant and qual data on
the same topic

21. 2) Embedded Design
Different research questions require
different types of data to answer them
Complementary – qual and quant data
complement one another
1 or 2 phase

22. 3) Explanatory Design
1st phase – quantitative (e.g. surveys)
2nd phase – qualitative (e.g. classroom
observation)
Qualitative data needed to explain results
1st phase quant – may be used to guide the
selection of 2nd phase in-depth qual study

23. 4) Exploratory Design
1st phase – qualitative
2nd phase – quantitative
Need to develop a measurement
instrument
1st phase - get a deeper understanding
of the issue/phenomenon
2nd phase - survey to measure its
distribution and prevalence.

24. PRACTICAL ISSUES
1) Politics of research – evidence-based
research - AERA Scientifically Based Research
2) Costs
3) Researchers: skills, careers,
disciplines
4) Social Organization of the Research
Team