1. Quasi-Experimental
Research
FAREEHA TANVEER

2. Independent and Dependent Variables
• IV
• manipulated / controlled to test its impact on the dependent
variable
• IV can be treatment, program, or intervention
• IV for ANOVA also known as the breakdown variable - factor
• represented as levels in the analysis
• treatment group is Level 1, control group is Level 2
• Nonexperimental research (regression analysis) - predictor variable
2

3. • When IV cannot be manipulated - quasi- or alternate independent
variables
• demographic variables
• DV is simply the outcome variable
• DV’s variability is a function of IV and impact - treatment effect
3
Independent and Dependent Variables

4. Internal Validity
• Extent to which the outcome is based on IV as opposed to extraneous
or unaccounted-for variables
• Identification of anything responsible for an outcome (effect) outside
of the IV (cause) is considered to be a threat
• External validity is the extent to which the results can be generalized
to the relevant populations, settings, treatments etc
4

5. Threats to Internal validity
• History (any event occurring b/w pre- and post treatment)
• Maturation (growing, learning over time)
• Testing (practice and familiarity)
• Instrumentation (instruments change / revised over time)
• Statistical regression (tendency of extreme score to regress towards average)
• Selection bias
• largest threat to internal validity in quasi-experimental research as no random
assignment
• Attrition (drop-outs)
5

6. • Combination of selection and other treatments
• participants in one condition may have been exposed to a stressful event not
related to the experiment
• Diffusion (inadvertent application of the treatment to control group)
• Special treatment (attention to the control group)
• Sequencing effects
• Fatigue effects
• Practice effects
• Order effects
• Carry over effects
6
Cont…

7. Control
• Element to secure the validity of research designs
• Concept of holding some variables constant or systematically varying
the conditions of variables to minimize the influence of unwanted
variables
• Control can be applied actively within quantitative methods through
• Manipulation
• Elimination
• Inclusion
• Group assignment
• Statistical procedures
7

8. Design Notations
Treatment Design Notations Group Assignment Design Notations
8

9. Experimental Research
• Most powerful type of research in determining causation among
variables
• Cause and effect can be established through:
• Covariation (the change in the cause must be related to the effect)
• Temporal precedence (determining the strength of a cause and effect
relationship, the cause must precede the effect)
• No plausible alternative explanation (the cause must be the only explanation
for the effect)
• https://explorable.com/internal-validity
9

10. Essential Features
• Manipulation
• Elimination
• Inclusion
• Group or condition assignment
• Statistical procedures
• Random assignment of participants to conditions (not to be confused
with random selection)
• Random assignment of conditions to participants (counterbalancing)
10

11. Quasi-Experimental Research
• Field research
• Nonequivalent designs - participants are not randomly assigned to
each condition
• Major difference between experimental and quasi-experimental
research designs - level of control and assignment to conditions
• Actual designs are structurally the same, but the analyses of the data
vary
• Confounding variables influencing the outcome cannot be fully
attributed to the treatment
11

12. Salient Designs Part-I
• Between-subject approach
• Pre-test designs
• Post-test designs
• K-factor designs
• Regression discontinuity
• With-in subjects approach
• Repeated measure designs
• Switching replication designs
• Cross over and Latin square designs
12

13. Between-Subjects Approach
• Between-subjects approach (multiple group approach, parallel group)
• Effects of two or more groups on single or multiple dependent
variables can be observed
• With a minimum of two groups, the participants in each group will
only be exposed to one condition (one level of the independent variable)
• Advantage of having multiple groups is that it allows for the
• random assignment to different conditions
• comparison of different treatments
13

14. • If the design includes two or more dependent variables, it can be
referred to as a multivariate approach
• One dependent variable - univariate
• Actual designs are structurally the same, but the analyses of the data
vary
14
Between-Subjects Approach Cont…

15. Pre-test and Post-test Designs
• Basic pretest and posttest designs can be modified to compensate for
lack of group equivalency
• addition of multiple observations / inclusion of comparison groups
• A dashed line (---) between groups indicates the participants were
not randomly assigned to conditions
• Allows to evaluate the lack of group equivalency and selection bias
15

16. • Double Pretest Design for Quasi-Experimental Research
16
Pre-test and Post-test Designs Cont…

17. • Pretest and posttest between-subjects approach, also referred to as
an analysis of covariance design
• pretest measure is used as the covariate because the pretest should
highly correlate with the posttest
• 1-factor* pretest and posttest control group design - most common
between-subjects approach
• one factor representing one independent variable - single-factor
randomized-group design
17
Pre-test and Post-test Designs Cont…

18. • Advantage of including pretest measures allows for the researcher to
test for
• group equivalency (homogeneity between groups)
• for providing a baseline against which to compare the treatment
effects, which is the within-subject component of the design
• Pretest is designated as the covariate in order to assess the variance
[distance between each set of data points] between the pretest and posttest
measures
18
Pre-test and Post-test Designs Cont…

19. • Multiple posttest treatment measures can be taken by including a
time-series component
• Groups can be randomly / non-randomly assigned to conditions
• Non-random assignment – significant limitations due to threats to
internal validity
19
Pre-test and Post-test Designs Cont…

20. k-Factor Designs
• Between-subjects approach can include more than one treatment
(factor) and does not always have to include a control group
• alternative- or multiple-treatment design
• *k represents the number of factors [independent variables]
• Within-subjects k-factor design is referred to as the crossover design
20

21. • A between-subjects k-factor design should be used when a
researcher wants to examine the effectiveness of more than one type
of treatment and a true control is not feasible
• educational settings or Institutional Review Board considers withholding of
treatment from specific populations as unethical
• Opting k-factor design
• some researchers believe that using another treatment (intervention) as a
comparison group will yield more meaningful results, particularly when the
types of interventions being studied have a history of proven success
21
k-Factor Designs Cont…

22. • Most common threats to internal validity are related, but not limited,
to these designs:
• Experimental. Maturation, Testing, Attrition, History, and
Instrumentation
• Quasi-Experimental. Maturation, Testing, Attrition, History,
Instrumentation, and Selection Bias
22
k-Factor Designs Cont…

23. Post-Test Designs
• Between-subjects (or multiple-group) posttest design
• Two-group posttest control group design is one of the more common
approaches within this structure
• Aim is to ensure group equivalency and control for selection bias
• Examination of the differences between baseline (pretest) and
posttest measures following the treatment
• For internal validity, the two-group posttest control group design is
considered one of the strongest
23

24. • Posttest-only design is not recommended in education settings as
mostly conditions do not allow random assignment
• If random assignment is not used, then a cohort matching technique
should be used to assign participants to conditions
• homogeneous groups are assigned to conditions, such as participants from the
same class
• Researcher can match a group by grade level (cohort) and then assess
the effects of a treatment by contrasting the differences between
control and treatment group
24
Post-Test Designs Cont…

25. • Accessing scores on a standardized achievement test
• (Group 1: O1) of last year's seniors
• with (Group 2: O1) the current senior class's scores
• Group 2 received an educational specific intervention (X)
• Group 1 is designated as a historical cohort (homogeneous) control
group
25
Post-Test Designs Cont…

26. • Most common threats to internal validity are related, but not limited,
to these designs:
• Experimental. Generally, all threats to internal validity are
adequately controlled for
• Quasi-Experimental. History, Maturation, Statistical Regression, and
Selection Bias
26
Post-Test Designs Cont…

27. Regression Discontinuity Approach
• RD design
• Means of assigning participants to conditions within the design
structure by using a cutoff score (criterion) on a predetermined
quantitative measure (usually the dependent variable)
• For evaluating the effect of an intervention
• *Most basic design used in RD approaches is the two-group pretest
posttest control group design
• Researchers argue that the RD approach does not compromise internal
validity
27

28. 28

29. With-in Subjects Approach
• Challenges when conducting research are often related to
• access to participants
• inability to randomly assign the participants to conditions
• Although the pretest and posttest designs of between-subjects
approaches include a within-subject component, the objective is not
necessarily to test the within subject variances as intended with
within-subject approaches
• One group serves in each of the treatment conditions
• Referred to as repeated measures because participants are
repeatedly measured across each condition
29

30. • Advantage: can be used with smaller sample sizes
• Disadvantage: threats to internal validity
• primarily maturation and history
• sequencing effects (order and carryover effects) performance in one
treatment condition affects the performance in a second treatment
condition
• It is recommended to randomize the order of the treatments
(counterbalancing) to control for sequencing effects
30
With-in Subjects Approach Cont…

31. • Simplest within-subjects approach is the one-group with a single
pretest and posttest measure (quasi-experimental research 1-factor
design)
• This design can be extended to multiple pretest and posttest
measures and is designated as an interrupted time-series (ITS) design
• Also called the "time-series" approach
31
With-in Subjects Approach Cont…

32. • Designs within the repeated-measures approach are classified as
experimental as long as participants are randomly exposed to each
condition
• ITS design is an example of nonexperimental research and is referred
to as a longitudinal data structure because data is collected at varying
time points over days, months, or even years
32
Repeated Measures Approach

33. • Switching-replication design
• assessing the effects of treatment on the first group while
withholding the treatment to the second group
• second group is designated as a wait-list control group
• includes only one treatment or factor
• One of the most effective experimental designs at controlling for
threats to internal validity
• External validity should also be improved through the use of two
independent administrations of the same intervention
33
Switching Replication Design

34. 34
• Wait-list continuation design (for Random assignment)
• Switching replications design (for Non Random assignment)

35. • Crossover design (changeover design)
• which includes at a minimum two factors, can include more
• Repeated-measures version of the k-factor design
• Important to ensure a "washout period," or return to baseline,
between the adjacent treatment periods as a means to control for
sequencing effects (multiple-treatment interference)
• Ideal for eliminating issues associated with the between-subject
variations and when a limited number of test subjects are available
35
Crossover Design

36. Crossover Design
• If there are enough subjects to assign to groups to each condition,
then a between and within-subject analysis should be used
36

37. Latin Square Design
• Similar to the crossover design
• One-factor model with two procedural factors (one for the rows and
one for the columns)
• Each row and each column contain the treatment as a means to
counterbalance the order of effects
• Major assumption: no interaction (or very minimal) between rows and
columns
• Problem associated:
• small number of observations for each combination of factor levels
• carryover effects (the efficiency of this design is application and data
dependent)
37

38. • Most common threats to internal validity are related, but not limited,
to these designs:
• Experimental. History, Maturation, Testing, Instrumentation,
Attrition, and Sequencing Effects
• Quasi-Experimental. History, Maturation, Testing, Instrumentation,
Statistical Regression, Selection Bias, Attrition, and Sequencing
Effects
38
Latin Square Design Cont…

39. • If a researcher wishes to examine the effectiveness of three types of
emotive imagery techniques (strong, medium, weak) on professional
athletes' level of concentration (assuming a total of 75 athletes
divided between each condition is adequate with regard to power)
• Three different settings (office, home, field)
• Three types of formats (live, recorded, combination)
• This one-factor design includes emotive imagery (at three levels) and
then two procedural factors (format and setting), each at three levels,
hence the 3 x 3 framework
39
Latin Square Design Cont…

40. 40
Latin Square Design Cont…

41. Thank YOU
for patient listening
41