Machine Learning Model Validation (Aijun Zhang 2024).pdf
The quantitative process
1. The Quantitative ResearchThe Quantitative Research
Process:Process:
Techniques for ResearchTechniques for Research
Dr Fiona M Beals
2. Lecture AimsLecture Aims
Review key research methods brought to
quantitative research by experimental
designs
Outline the role of the quantitative
researcher
Introduce and look at methods of:
– Testing
– Surveys
– Observation and Interviewing
3. The SettingThe Setting
Experimental research vs
Quasi-Experimental Research
The need for empirical data
Sampling is key (stratified
random or purposive)
Key words are reliability and
validity (internal and external)
Significance is important
Eliminate bias
Remember variables
5. Why Test?Why Test?
Established tests tend to have measures of
reliability and validity
Testing before and after an intervention can
show evidence of change (and the direction
of change)
Tests for significance can occur (ANOVA,
Chi Square)
6. What to testWhat to test
Psychometric variables
Biological/Physiological
changes
Educational Changes (IQ etc)
7. How?How?
Don’t create your own
test instead find
established tests which
have measures of
reliability and validity
10. Traps in Questionnaire DesignTraps in Questionnaire Design
Ambiguity – unclear questions
Assumptions
– Multiple responses when really only one is wanted
– Memory stretching
– Knowledge demands
Double questions
Leading questions
Presuming questions
Hypothetical questions
Overlapping categories
11. Getting it rightGetting it right
Remember most people don’t want to write or
type
So quick ticks and clicks work
Follow the KISS principle
Use likerts for measuring variability in responses
Connect the question to the response
NEVER ask two questions in one!!!
Keep the survey to under seven minutes
PILOT, PILOT, PILOT
13. Observation and InterviewingObservation and Interviewing
Observation can have an important function
in quantitative designs but tends to focus on
descriptive elements
Interviewing needs
to be structured
Both observation
and interviewing should only be used for
triangulation of data and results
15. Know the basicsKnow the basics
Nominal
– =/ ≠
– Dichotomous (Gender)/Non-dichotomous (nationality)
– Mode
– Qualitative
Ordinal (rank order without degree of difference)
– =/ ≠, </>
– Dichotomous (truth, beauty, health)
– Non-dichotomous (opinion)
– Median (psychological tests do tend to break this rule)
– Qualitative
Interval (degrees of difference but without ratios)
– =/ ≠, </>, +/-
– Date, Latitude, Temperature
– Arithmetic mean (average using sum – what usually happens)
– Quantitative with an arbitrary point of origin (0)
Ratio
– =/ ≠, </>, +/-, ×/÷
– Age, mass, length, duration, energy etc.
– Geometric mean (average using product and the nth root)
– Quantitative with a unique and non-arbitrary zero
16. Know a little moreKnow a little more
Correct use of percentages
Data sets need to be over 30
Basic tests for significance
– Chi square
– T test
– ANOVA
Read research critically!!!
– Read for bias
– Read for incorrect use of statistics
– Read so you don’t make the same mistakes
Notas do Editor
Experimental most powerful in terms of examining causal linkages Quasi – next Most social resaerchers consider that two variables are causally related, that is one causes the other if: The cause preceeds the effect in time There is an empirical correlation between them The relationship is not found to be the result of some third variable effecting the two initally measured variables Reliability – the extent to which a measure produces consistent result – internal consistency, rides nicely with parallel test, shows in test and re-tests Can be improved by a careful selection of measures, use of objective criteria, multiple observations, large samples, pilot studies and triangulation Validity: the extent to which a meaus reflects what it is intended to measure Internal validity (various types – content is one) – improved by careful selection of measures, real-life situations, good experimental design and control of extraneous variables External validity (generalisability) improved by representative samples, replication Observational with a focus on descriping cause – weakest design