Assignment submission for SHU Year 2 - Portfolio of Research Skills

1. Quantitative Data Analysis
‘Attitudes Towards Research’
Lee James Cox
B0049872
MA in Technical Communication
Module: 14-7104-OOY-B-20123 – Portfolio of Research Skills (A-2012/3)
1

2. Introduction
This presentation aims to show:
How I would go about analysing
and presenting survey data
based on example of:
Excel spreadsheet of 50 summarised responses
for a research question of:
Attitudes Towards Research*
*Adapted by Sheffield Hallam University (SHU) based on Bjorkstrom and Hamrin [1] survey
2

3. Recognising limitations in the data set
1. Data set is missing key information to provide validity as to the source
–
–
–
–
Sampling or selection mechanism of how 50 subjects were chosen is not explained*
Key demographic information is missing (age and education are important variables for attitudes [2])
Data set may not representative. [3] Desired confidence level, population size and accuracy is
required. Instead, sample is likely to be opportunistic [4]*
Data set is only for a specific time period – potentially influenced by specific/recent events
2. Raw data is missing. Only aggregated data is available which limits analysis:
–
–
–
–
Did the Date or Time of day of the submission affect the responses?
No opportunity to identify and discard invalid responses, e.g. what if all answers were the same (all
low, all medium, or all high)
Interrelationships between an individuals answers is lost, e.g. if a person respond high to question 5,
are they also more likely to respond to question 4 as high?
Is there a correlation or cross-tabulation between a persons role and their attitudes?
3. Questions may be confusing
–
–
‘Professional Communication’ is an abstract idea and using concrete examples would remove
misunderstandings
The Likert [5] scale would have been more conclusive and less confusing than the Low  High rating.
Had subjects been able specify their level of agreement or disagreement, it would remove guess
work whether low agreement = disagreement, and whether moderate means neither ‘agree nor
disagree’.
3
*50 respondents likely to be SHU students from Nov 2011 MA in Technical Communications course

4. Collation of data
Even with the limitations of the data set, the data needs to be reshaped to assist with analysis:
1. Categorising: ‘Name your role’ question should be grouped to create an additional data point
2. Formatting: superfluous text and formatting in questions 2-10 should be reduced to make
data manipulation and graph production easier.
3. Labelling: headings should be renamed to include ‘Agreement’ to reduce misunderstandings
that headings may refere to ‘Priority’ or ‘Importance’
4. Significance: statistical inference [6] should be applied to assess whether any of the data or
results may help support the theories about this research topic.
4

5. Presenting the findings
1. Agree a suitable format for the audience. Report with technical data or presentation
with hand-outs?
2. Directly answer the research question as an executive summary highlight the most
important finding, e.g.:
‘4 out of 5 students believe external researchers are needed to develop standards.’
3. Provide appropriate conclusions for format of report, e.g. recommendation and next
steps
‘84% said relevant research results should be more widely spread to Professional Communicators
and therefore we recommend …’
4. Share the supporting data to establish validity of research
–
–
Raw data for other researchers to draw their own analysis and conclusions
Summary of data in tabular and graphical formats – highlighting key findings
5. Validate interpretations and propose alternative conclusions to consider
5

6. Validity & Reliability
Ethics
 ‘SurveyMonkey’ application protected user anonymity
 No personal questions to identify individuals
× No confirmation page or capturing of an email address to be rewarded with the results
Use of a Research Diary to record the research process is recommended
• Using a blog (Blogger, Wordpress) or offline in Microsoft Word
• To record what I was doing, feeling and thinking throughout the project
Excel would be the primary analysis tool because of small data set
• Easy to manipulate data – filtering, calculations, cross-tabulation
• Good range of table formats and graphs for the presentation
Testing is required
• Of conclusions - through peer review from tutors and students
• Over time - generated periodically (annually?) to trend attitude changes and issues
6

7. Weaknesses & Strengths of
Quantitative Data Analysis
Weaknesses
Strengths
Perception of having less ‘validity’ as questions
don’t offer opportunity to clarify or challenge the
questions and may not reflect subjects
understanding of topic.[7]
Perception of being more ‘reliable’ because
approach aims to control or eliminate extraneous
variables. Results are more independent of the
researcher.
Data not displaying significance may be neglected.
E.g. Q7 – 2 deviations [8] from the norm said they
highly agreed that doing research would
complicate their professional practice but this is
not explored further.
Best for testing and validating existing constructed
theories and hypotheses to identify areas to deep
dive with Qualitative analysis.
Large data sets may become unmanageable to
restrict the scope
Matters less about selection of samples – can be
more random using larger number of people.
Knowledge produced might be too abstract and
general for direct application to specific
situations or contexts relevant to the subject.
Can be easier to obtain precise, numerical data
more quickly.
Likely to have higher credibility.
7

8. Conclusions
Quantitative analysis relies completely on the data set available
for its validity. In this study, we show there were too many issues
that invalidated it.
Unfortunately the restricted set of data also presents little
opportunity to identify areas of interest for interpretation and
further analysis.
8

9. References
[1] Björkström M.E., Hamrin, E.K.F. (2001) Swedish nurses’ attitudes towards research and development
within nursing. Journal of Advanced Nursing. 34 (5): 706–14
[2] Carr, L. T. (1994). Referencing Bostrom, Hicks, Pearcey & Dyson in ‘The strengths and weaknesses of
quantitative and qualitative research: what method for nursing?’. Journal of Advanced Nursing. 20, 713.
[3] Krejcie, R., Morgan, D. (1970). Determining Sample Size for Research Activities. Educational and
Psychological Measurement. Online. Last accessed 1/12/13 at
http://opa.uprrp.edu/InvInsDocs/KrejcieandMorgan.pdf
[4] Duffy, M.E. (1985) Designing nursing research: the qualitative-quantitative debate. Journal of
Advanced Nursing. 10 (3), 225-232.
[5] Likert, R. (1932). A Technique for the Measurement of Attitudes. Archives of Psychology, 140, 1–55.
[6] Neyman, J. (1934). Statistical Inference in "On the two different aspects of the representative
method: The method of stratified sampling and the method of purposive selection", Journal of the
Royal Statistical Society, Vol. 97 (4), 557–625. Published by Wiley.
[7] Sandeowski, M. (1986) The problem of rigor in qualitative research. Advances in Nursing Science. 8
(2), 27-37.
[8] Cormack, D. (1991). Deviant cases. The Research Process in Nursing 2nd edition. Blackwell Scientific.
Oxford.
9