1. CHAPTER 13
EXPERIMENTAL
RESEARCH
By: David Cook, Anne D’Alonzo, Angela
McCoy, Carla Oden, Kim Ross, Brandi Young,

2. Summary of Presentation
Summary of Chapter 13: Experimental
1.
Research
Vocabulary Quiz/ Flash Cards
2.
Article 20: Summary and Discussion
3.
Class Experiment and Discussion
4.

3. Essential Characteristics
One of most powerful methodologies

Establish cause–and–effect among variables

Not always Easy to conduct

Powers and Problems in conducting experiments

ONLY type of research that directly attempts to

influence a particular variable
When applied properly – BEST type for testing

hypotheses about cause-and-effect relationships
Researchers MANIPULATE the independent variable

Consist of 2 Basic Conditions:

2 or more conditions or methods are compared

The independent variable is directly Manipulated


4. Comparison of 2 Groups of
Subjects
Experimental Group receives a treatment

Control Group receives no treatment

(medical/psych.)
Comparison Group receives a different

treatment (ed.)

5. Manipulation of the Independent
Variable
Researcher actively manipulates the

independent variables and decides which
group will get the form, when, where, and how.
Ex. Transparencies vs. no transparencies in a
statistics class.

6. Randomization (assignment &
selection similar, not identical)
Random Assignment: every individual has an equal chance to be

assigned to any of the experimental or control conditions. Each
member is given an arbitrary number and a table of random
numbers is used to select the members of the experimental and
control groups.
Takes place BEFORE the experiment begins

Process of assigning individuals to groups, not a result of the

distribution
The researcher formed groups that at the beginning of the study are

equivalent; they differ only in chance. This is intended to eliminate
the threats of EXTRANEOUS or additional variables – those which
researchers are BOTH aware and unaware that might affect the
outcome of the study.
No guarantee of equivalent groups unless both groups are

sufficiently large (most researchers want no fewer than 40 subjects
in each group).
Random Selection: every member of a population has an equal

chance of being selected to be a member of the sample.

7. Control of Extraneous Variables
Researchers must control any and all subject characteristics by

ensuring that the 2 groups are as equivalent as possible on all
variables except the one(s) being studied (the independent
variables).
Randomization

Holding certain variables constant. Ex., eliminate one gender –

drawback is reduced generalizability.
Building the variable into the design. Ex., include both genders and

analyze the effects.
Matching. Ex., match by age and assign 1 member of each pair to

each of the comparison groups.
Using subjects as their own controls. Ex., same students taught

algebra 1st by inquiry method than by lecture method.
Using analysis of covariance. Equate groups statistically on the

basis of a pretest or other variable. Posttest scores are adjusted
accordingly.

8. Group Designs in Experimental
Research
WEAK Experimental Designs:

Design can
The One-Shot Case Study. Remedy: a comparison

take a
could be made with another group. Seldom used.
variety of
Treatment … Observation

forms.
2. The One-Group Pretest-Posttest Design. A single

GOOD
group is measured after being exposed as well as
designs
before. Remedy: a comparison group to be added.
control many
Pretest … Treatment … Posttest

of the
threats to 3. The Static-Group Comparison Design or

Nonequivalent Control Group Design. Two already
internal
existing groups or static groups are used. NOT
validity.
randomly assigned.
4. The Static-Group Pretest-Posttest Design. Only

difference is that a pretest is given to both groups.
Pretest score is subtracted from posttest score.

9. TRUE Experimental Designs:
* Subjects are randomly assigned to treatment

Please Note this
groups. This controls the subject characteristics
distinction:
threat to internal validity.
Random
Selection and
1. The Randomized Posttest-Only Control

Random
Assignment differ
Group Design. Two groups by random
in purpose.
assignment. One group gets experimental
Random
Selection is to
treatment, other group does not. Both groups
provide a
representative
are posttested on the dependent variable.
sample. It may or
Note: the threat of subject
may not be
accompanied by
characteristics, maturation, and statistical
the random
assignment of
regression are well controlled. None of subjects
subjects to
are measured twice, so testing is not a threat.
groups. Random
Assignment is
BEST of all designs, provided there are at least
intended to
equate
40 subjects in each group. Mortality is a threat.
groups, and often
is not
Hawthorne (attitudinal) threat is possible.
accompanied by
random selection.

10. TRUE Experimental
Designs, cont.
2. The Randomized Pretest-Posttest Control

Group Design. This has a pretest. Two groups
of subjects are used and both groups are
measured twice. The use of a pretest raises
the possibility of a pretest treatment interaction
threat since it may “alert” members of the
experimental group. The pretest does check
whether the groups are really similar.

11. TRUE Experimental
Designs, cont.
3. The Randomized Solomon Four-Group

Design. Designed to eliminate the possible
effect of a pretest. It is random assignment of
subjects to 4 groups, with 2 of the groups
being pretested and 2 not. One pretested
group and one unpretested group is exposed
to the experimental treatment. All 4 groups are
posttested. This is the BEST control of threats
to internal validity. Requires a large sample
and much work.

12. TRUE Experimental
Designs, cont.
4. Random Assignment with Matching. Pairs

of individuals may be matched on certain
variables. The members of each matched pair
are then assigned to the experimental and
control groups at random. Matching may be
done mechanically or statistically – both
require a score for each subject on each
variable

13. Mechanical Matching:
Pairing 2 persons whose scores on a particular

variable are similar. After the matching is
completed for the entire sample, a check should
be made with a frequency polygon to ensure that
the 2 groups are equivalent on each matching
variable. TWO Problems: 1) Difficult to match on
more than 2 or 3 variables, making it necessary to
have a very large sample. 2) Some subjects will
need to be eliminated because there are no
matches for them, samples are no longer random.
1 member of each matched pair is randomly
assigned to experimental group, the other the
control group.

14. Statistical Matching:
Not perfect, however recommended over mechanical. Does

NOT lose subjects, nor does it limit the number of matching
variables. The sample is divided randomly at the beginning
and the statistical adjustments are made after the data have
been collected. Each subject is given a predicted score on
the dependent variable, based on the correlation between the
dependent variable and the variable(s) on which the subjects
are being matched. The difference between the predicted and
actual scores for each individual is used to compare
experimental and control groups. When pretest is matching
variable, the difference between the predicted and actual
score is called regressed gain score. This score is preferable
to the straightforward gain scores (posttest minus pretest
score for each individual) because it is more reliable.
*Need random assignment on all variables with above 2.


15. Quasi-Experimental Designs
Do not include

the use of random assignment

Other techniques

control threats to internal validity


16. The Matching-Only Design
Random assignment is not used

Matches subject in experimental and control groups

No assurance matches are equivalent

This design offers an alternative to random

assignment when 10 or more groups are available for
a method study
Groups can be randomly assigned to different

treatments
Individuals are matched with individuals receiving

other treatments
Matching is never a substitute for random assignment

The correlation between matching variables and the

dependent variable should be substantial

17. Counterbalanced Designs
In different orders all groups are exposed to all

treatments
This design Counterbalanced designs represent another technique for

controls threats
equating experimental and comparison groups
to internal
Many different treatments may be involved
validity 
A Three-Treatments Counterbalance Design involves

Internal
three groups
Validity:
Group 1: treatment 1+posttesting
observed 
differences on
treatment 2+ posttesting

the dependent
treatment 3+posttesting
variable are 
directly related
Group 2: treatment 2+posttesting

to the
treatment 3+posttesting
independent 
variable and
treatment 1+posttesesting

not due to
Group 3: treatment 3+postest
some other 
unintended
treatment 1+posttest

variable
treatment 2+postesting

Group treatments are in random order

By comparing the average scores for all groups

researchers determine the effectiveness of various

18. Time-Series Designs
Involves repeated measurements or

Involves observations over time

Both before and after treatments

Extensive amount of data collected

The Researcher has more confidence if the group

pretests and posttests cause improvement with
multiple tests.
Threats to internal validity endanger this design for

example history
Effectiveness is by analyzing pattern

Time-series design is a strong design, although it is

vulnerable to history

19. Factorial Designs
Extend the number of relationships that may

be examined in an experimental study
Researchers may study interactions of an

independent variable
Moderator variables may be either treatment

variables or subject characteristic
Factorial Designs are an efficient way to study

several relationships

20. Control of Threats to Internal
Validity: A Summary
Time series design suffer from instrument

decay and data collector bias
Unconscious bias on the part of data collectors

is not controlled by any of these designs
Implementers or data collectors can

unintentionally distort the results of a study
Regression is not likely to be a problem except

in the one-group pretest-posttest design

21. Evaluating the Likelihood of a
Threat to
Internal Validity
Consider the likelihood of threats in an

experimental study
A number of possible threats to internal validity

may exist
Researchers must question possible threats to a

study by using the following procedures:
Question factors related to the study affecting

dependent variables
Question comparison groups differing the same

factors
Evaluate the threats and plan to control them


22. Threats to Internal Validity:
Subject

Characteristics
Mortality

Location

Instrumentation

History

Maturation

Attitude of Subject

Regression

Implementation


23. Subject Characteristics
to affect critical thinking ability

1. Initial critical thinking ability

2. Gender


24. Mortality
Location and data could affect scores

Group numbers should be verified

Have an effect unless controlled: moderate

high

25. Location
Data collection and location differing for two

groups could affect posttreatment scores
Threats may differ for groups in different

locations
Likelihood of having an effect: moderate to

high

26. Instrumentation
Instrument decay
May affect any outcome

Could differ for groups

Unless controlled could have an effect

Data collector characteristics

Might affect critical thinking test scores

Use same data collectors to avoid decay

Likely to have an effect unless controlled

Data collector bias

Affects scores on critical thinking

Controlled by training implementers

Unless controlled likely to have an effect


27. History
Extraneous events

Affect both groups

Likely to have an effect unless controlled


28. Maturation
Affect outcome scores since critical thinking is

related to individual growth
No threat if instructors teach over same time

period
Likely to have an effect unless controlled


29. Attitude of Subject
Affect posttest scores

Perceived special attention could be a threat

Low moderate threat unless controlled


30. Regression
Will affect scores if subjects are selected on

the basis of extreme scores
Treatment unlikely to affect groups differently

The likelihood of having an effect is low


31. Implementation
Instructor characteristics are likely to affect

posttreatment scores
Different instructors teach different methods

control by monitoring instruction
The likelihood of having an effect is high


32. Identify threats to internal validity
Control of Experimental Treatments
Think of
different
 Improve internal validity of
variables
experimental study
- Decide if
 Researcher control well constructed
these things
would affect
experiment
things
differently
 Researcher controls the treatment
based on the
5’ws + how
evidence
 Researcher controls testing
- Threats
need to be
 Researchers seldom have total
minimized
control: problems must be faced……

33. An Example of Experimental
Research Study
Purpose/justification: explicit implications

Exemplify
Definitions: clearly defined context
typical 
methodology
Prior research: previous work connected

and permit
constructive Hypotheses: stated, implied, appropriate

criticism
Sample: target population indicated

Instrumentation: adequately described

Hold Procedures/internal validity: evident

students
threats
attention
Data analysis: are statistics correct

Results: clearly presented

Be reported
Discussion/interpretations: limitations

concisely
recognized with regard to population

34. The Effects of a Computer Simulation Activity Versus a
Hands-on Activity on Product Creativity in Technology
Education
Purpose/Justification: Study the effect of computer simulation

Definitions: The dependent variable, product creativity; is

confused by the discussion of creativity
Prior Research: The brief summaries of two studies on

outcomes of computer graphics and word processors are well
done, but we must assume they are the only ones
Hypotheses: Clearly stated

Sample: A convenience sample was used

Instrumentation: Product creativity was measured, validity is not

adequately addressed
Procedures/Internal Validity: Internal validity was

controlled, mortality did not occur
Data Analysis/Results: Lack of random selection

Discussion/Interpretation: Results do not support “the use of

computer simulation to enhance product creativity.”

35. Article Summary
Influence of Social Context on Reported

Attitudes of Nondisabled Students Towards
Students with Disabilities
 Article 20, page 149
Helpful research worksheet on iLearn


36. Research Question and
Hypothesis
Question: Would the mere presence of a

student with a disability affect the responses of
nondisabled students to a survey that assessed
their attitudes toward people with disabilities?
Hypothesis: Individuals would report a) a more

tolerant attitude toward persons with disabilities
and b) less discomfort in a social setting when
paired with a similar rather than dissimilar
individual.

37. Operational Definitions and Variables
Persons with disabilities: An individual who

uses a wheelchair to perform any daily living
activities
Independent variable:The presence of a

disabled person or non-disabled person.
Dependent variable: Amount of comfort and

attitude towards disabled person

38. Sampling
30 Louisiana State University students

 Ten people in the group have disabilities
 Don’t know how sample was selected or how
non-disabled participants were assigned to test
groups
Two confederates (one in wheelchair)


39. Instruments
Demographic questionnaire

Original Attitudes Toward Disabled People

Scale (ATDP-O)
Likert-type scale to measure general ease

(GME)

40. Procedure
Three experimental conditions

ND/ND: non-disabled (9 women/1 man; mean age = 22) and non-

disabled confederate
D/ND: non-disabled (9 women/1 man; mean age = 22.5) and

disabled confederate
D/D: disabled (4 women/6 men; mean age = 30.5) and disabled

confederate
One by one, groups escorted into testing room by the

same non-disabled experimenter
Experimenter explained data would be used in a large

study analyzing the way individuals viewed themselves
and others
Participants and confederate completed ATDP-

O, demographic questionnaire and GME
Experimenter remained in the room to answer questions


41. Instrument Validity and Reliability
We do not know if the instruments were

reviewed by an expert; only talks about
previous use of ATDP-O
Questions raised about scale accuracy

Researchers administered test only once


42. Internal Validity
Threats

 Subject characteristics
 Testing / reactivity
 Attitude of subjects
Controlled

 Instrumentation / data collector characteristics
 Location

43. Conclusions
Nondisabled individuals reported a more

favorable attitude toward persons with
disabilities when in the presence of such an
individual
Habituation between nondisabled and disabled

people may be a cost-effective way to begin
modifying negative attitudes
Interpretations speculative -- study has many

weaknesses

44. External Validity
Study can’t be generalized

Studies only people in wheelchairs and not

other disabilities
Sample consists of college students from one

Southern U.S. college
Results need replication


45. Classroom Experiment

47. Experiment: Tangram Assembly
Question: Does verbal communication decrease

the time it takes a group of four individuals
working in tandem to assemble of specific images
with tangrams?
Hypothesis: Since the only thing that will change

is whether or not people are allowed to talk our
hypothesis should state that verbal groups should
be able to complete the images more quickly.
(However, personally I think our class is full of
independent thinkers who may have difficulties
organizing verbally and the quiet teams may
actually do better.)

48. Variables
Independent: Ability to use verbal

communication (Qualitative)
Dependent: Time to assemble specific

images with tangrams (Quantitative)
Extraneous:

 Both groups are aware of the purpose of the
experiment
 Some individuals may be more visual learners
 Some individuals will have had more experience
using tangrams
 Not enough testing groups

49. DISCUSSION OF RESULTS

50. THANK YOU FOR PARTICIPATING IN OUR
EXPERIMENT