MEASURING LEARNING BELIEFS Publication -2005

MEASURING LEARNING BELIEFS
__________________
A
Dissertation
Presented to the
Faculty of the
Graduate School of Education
Alliant International University
__________________
In Partial Fulfillment
of the Requirements for the Degree of
Doctor of Education
__________________
by
Lawrence Smythe
San Diego, 2005

Abstract of Dissertation
MEASURING LEARNING BELIEFS
by
Lawrence Smythe, Ed.D.
Alliant International University
Committee Chairperson: Jerold Miller, Ed.D.
THE PROBLEM. Knowing how to use computing devices is important for business,
communications, education, and social interactions. It is therefore exceedingly important
to develop methods that facilitate learning to use electronic products. As a result,
learning experiences need to reflect a range of skill-expertise, prior knowledge,
perceptual styles, and epistemological beliefs. Measuring extent of relationship between
personal epistemologies and experience using electronic products quantifies a learning
diversity to create empowered learning experiences.
METHOD. Alliant International University and National University undergraduate
students, graduate students, and faculty completed identical printed and computer-based
versions of a questionnaire documenting technology education, electronic product
experience and the Schommer-Aikins Epistemology Questionnaire. Epistemologies are
personal beliefs about knowledge and learning with primary measurements of Certain
Knowledge (CK), Innate Ability (IA), Omniscient Authority (OA), Quick Learning (QL),
and Simple Knowledge (SK). Additional sections were Self-taught Technology
Education, Certificate-Apprenticeship Technology Education, Earned Degrees &

Certificates, and Experience Using Electronic Products. Half the participants completed
paper and half-completed Internet questionnaires.
RESULTS. This study measured the extent of relationship between experience
using electronic products and epistemological beliefs. It has confirmed the factor
structure of previous epistemological research and extended the field of knowledge to
include measurements of epistemological beliefs influenced by experiences using
electronic products. It is becoming increasingly important to accommodate the
organizational diversity of knowledge and learning beliefs to keep pace with rapid
technological change. To address these issues, a significant contribution of this study is
the presentation of epistemological beliefs as a knowledge organization guideline for
technical communications and consumer product development. Moreover, with the
advent of sophisticated products like automotive navigations and telematics, it has
become increasingly important to understand epistemological beliefs as a foundation for
structuring customer and corporate communications, user manuals and electronic product
engineering design.

© 2005
LAWRENCE SMYTHE
ALL RIGHTS RESERVED

DEDICATION
To Kristi, Adam, Guinevere, Mentors, Family, and Friends
iv

ACKNOWLEDGEMENTS
I would like to thank the faculty of Alliant International University for their
graciousness in accommodating my academic and professional interests. Being the only
automotive engineer and non K12 teacher in classes was rewarding because through these
experiences, I was able to combine Human Factors Engineering, and Experimental
Psychology with Education: Technology & Learning.
Substantive academic and professional enlightenment was derived from talented and
inspirational instructors. I would especially like to thank Dr. Maria Fernandez for her
foundational support throughout my academic studies, and Dr. Jill Saulque because it was
during her lectures that I connected Human Factors Engineering with Educational
Psychology. I would also like to thank Dr. Carolyn Salerno for her friendship, support and
introduction to practical applications of learning environments and knowledge management.
I also wish to thank my fellow-elder student and colleague Dr. Gary Parks for introducing me
to National University and thanks to Dr. Halyna Kornuta for her assistance and guidance
through my National University research. A special thanks to Dr. Joseph Marron for
becoming a committee member so late in the game. Finally, a very special thanks goes to
Dr. Jerold Miller for his guidance and friendship helping me through my dissertation
ideation, organization, research and finally composition.
I would like to credit my parents for the wonderful learning environment they
provided for me as a child and thank my children Adam and Guinevere who accepted the
amount of time I spent accomplishing this goal. Finally, I want to thank my wife, Kristine
for her connected understanding, patience, and loving-motivational support.
v

TABLE OF CONTENTS
Page
LIST OF TABLES............................................................................................................. x
LIST OF FIGURES ....................................................................................................... xiii
Chapter
1. INTRODUCTION.............................................................................................. 1
Background of the Problem ......................................................................... 3
Statement of the Problem............................................................................. 6
Purpose for Study......................................................................................... 9
Theoretical Framework................................................................................ 9
Importance of the Study............................................................................. 13
Research Questions.................................................................................... 14
Scope of the Study .................................................................................... 14
Definitions of Key Terms ........................................................................... 17
2. REVIEW OF THE LITERATURE................................................................... 20
Foundational Studies: Social Learning Theory ........................................ 21
Epistemological Definitions and Measures ............................................... 27
Education and Personal Epistemologies.................................................... 29
Information Processing: Skills-Based Competencies............................... 34
Development of Research Questions......................................................... 39
3. RESEARCH METHODOLOGY...................................................................... 40
Subjects...................................................................................................... 44
Instrumentation .......................................................................................... 45
vi

Chapter Page
General Procedures.................................................................................... 48
Data Analysis Process................................................................................ 50
Assumptions............................................................................................... 52
Limitations of the Study ............................................................................ 52
4. DATA ANALYSIS........................................................................................... 53
Findings ..................................................................................................... 53
Initial Analysis........................................................................................... 53
Patterns of Variable Associations.............................................................. 54
Factor Analysis .......................................................................................... 62
First research question: What is the extent of relationship
between paper verses Internet and questionnaire responses?................. 73
Second research question: What is the extent of relationship
between participant age and questionnaire responses?........................... 80
Third research question: What is the extent of relationship
between participant gender and questionnaire responses? ..................... 87
Fourth research question: What is the extent of relationship
between participant group and questionnaire responses?....................... 92
Fifth research question: What is the extent of relationship
between participant source and questionnaire responses?...................... 97
5. CONCLUSIONS DISCUSSION, AND SUGGESTIONS FOR FUTURE ............
RESEARCH.......................................................................................... 105
Conclusions.............................................................................................. 105
between paper verses Internet and questionnaire responses?............... 108
between participant age and questionnaire responses?......................... 108
vii

Chapter Page
between participant gender and questionnaire responses? ................... 109
between participant group and questionnaire responses?..................... 110
between participant source and questionnaire responses?.................... 112
Discussion................................................................................................ 114
Suggestions for Future Studies ................................................................ 126
REFERENCES CITED............................................................................................... 134
APPENDICES
A. EPISTEMOLOGICAL SUBSET FACTOR DEFINITIONS ........................ 154
Description of Questionnaire and Measurement Scales .......................... 152
B. LINEAR REGRESSION TABLES................................................................ 155
between paper verses Internet and questionnaire responses?............... 156
between participant age and questionnaire responses?......................... 157
between participant gender and questionnaire responses? ................... 161
between participant group and questionnaire responses?..................... 163
between participant source and questionnaire responses?.................... 165
C. INTRODUCTION AND INSTRUCTION STATEMENT FOR
ALLIANT INTERNATIONAL UNIVERSITY AND
NATIONAL UNIVERSITY.................................................................... 169
viii

Chapter Page
D. APPLICATION TO INSTITUTIONAL REVIEW BOARD FOR
FULL REVIEW AND APPROVAL OF RESEARCH WITH
HUMAN PARTICIPANTS AT NATIONAL UNIVERSITY ................ 172
INFORMED CONSENT FORM
INFORMATION ABOUT: MEASURING LEARNING BELIEFS ..... 175
APPROVAL TO CONDUCT RESEARCH AT
NATIONAL UNIVERSITY ................................................................... 177
E. APPROVAL TO CONDUCT RESEARCH WITH HUMAN SUBJECT
AT ALLIANT INTERNATIONAL UNIVERSITY................................ 178
AIU INSTITUTIONAL REVIEW BOARD PROPOSAL ...................... 181
SUBJECT BILL OF RIGHTS ................................................................. 184
F. CORRESPONDENCE WITH DR. M. SCHOMMER-AIKINS .................... 185
ix

LIST OF TABLES
Table Page
1. Instructions for the Schommer Aikins epistemology questionnaire.................... 15
2. Schommer Aikins: Epistemological factor definitions....................................... 27
3. G -power tests: Post-hoc test alpha and power determination............................ 41
4. Independent variable definitions:
P-I, AGE, GENDER, GROUP, and SOURCE ................................................ 42
5. Dependent variable definitions: STE, CTE, and ED .......................................... 43
6. Dependent variable definitions: Epistemology................................................... 44
7. Alliant International University and
National University subject database matrix .................................................. 45
8. Hardware and software combinations used to validate html questionnaire …… 48
9. Questionnaire distribution statistics..................................................................... 49
10. Outline of data analysis process......................................................................... 51
11. Test sample ages by group................................................................................. 54
12. One-sample Kolmogorov-Smirnov between Schommer Aikins and test data .. 54
13. Independent sample Pearson r: Schommer Aikins vs. test data ....................... 55
14. Paired Pearson r: Schommer Aikins vs. test data ............................................. 55
15. Paired sample t-tests: Schommer Aikins vs. test data ...................................... 56
16. Frequencies of earned degrees & certificates of respondents............................ 57
17. Comparison between STE4 and CTE4 .............................................................. 59
18. EEP questionnaire items presented as mean-score ranks .................................. 60
19. Paired-sample t-test between ste and cte to measure internal validity .............. 61
20. Factor analysis with independent variables with STE and CTE........................ 63
x

Table ..............................................................................................................................Page
21. Factor analysis with independent variables with STE, CTE, and EEP.............. 64
22. Factor analysis with variables of STE, CTE, EEP, and epistemology .............. 65
23. Factor analysis with STE, CTE, EEP, and epistemology .................................. 66
24. Factor analysis with EEP and epistemology...................................................... 67
25. Factor analysis with STE, CTE and epistemology ............................................ 68
26. Factor analysis with Epistemology and independent variables ......................... 69
27. Factor analysis with epistemology questions..................................................... 70
28. A comparison of significant epistemological factor hierarchies ....................... 71
29. Summary of epistemological factor hierarchies ................................................ 73
30. Descriptive statistics for paper and internet groups........................................... 73
31. Paired sample t-test between paper and internet................................................ 74
32. M and SD of significant ANOVA variables: P-I.............................................. 75
33. M and SD of significant linear regression variables: P-I.................................. 77
34. M and SD of significant ANOVA variables: AGE........................................... 81
35. M and SD of significant linear regression variables: AGE ................................ 83
36. Positive and negatively sloped data relationship: AGE.................................... 85
37. M and SD of significant ANOVA ..................................................................... 88
38. M and SD of significant linear regression variables: GENDER ...................... 89
39. M and SD of significant ANOVA variables: GROUP ..................................... 92
40. M and SD of significant regression variables: GROUP ................................... 93
41. Discriminant analysis canonical correlation significance: GROUP................. 95
xi

Table ..............................................................................................................................Page
42. Discriminant analysis: Wilks' Lambda and Chi-square: GROUP ................... 96
43. M and SD of significant ANOVA variables: SOURCE................................... 99
44. M and SD of significant regression variables: SOURCE............................... 100
45. Paired-sample Pearson r: Source .................................................................... 103
46. Paired-sample t-tests: Source.......................................................................... 104
A1. Epistemological abbreviations and definitions............................................... 153
A2. Epistemology questionnaire validity and reliability data .............................. 154
B1. Linear regression modeling: P-I..................................................................... 156
B2. ANOVA subsequent test of linear model significance: P-I ........................... 157
B3. One-way ANOVA: AGE ............................................................................... 158
B4. Results of linear regression analysis: AGE.................................................... 159
B5. ANOVA subsequent test of linear model significance: AGE........................ 160
B6. One-way ANOVA:GENDER ......................................................................... 161
B7. Linear regression modeling: GENDER.......................................................... 162
B8. ANOVA subsequent test of linear model significance: GENDER................ 163
B9. One-way ANOVA: GROUP .......................................................................... 164
B10. Linear regression modeling: GROUP .......................................................... 164
B11. ANOVA subsequent test of linear model significance: GROUP................. 165
B12. One-way ANOVA: SOURCE...................................................................... 165
B13. Linear regression modeling: SOURCE........................................................ 166
B14. ANOVA subsequent test of linear model significance: SOURCE .............. 167
B15. Descriptive statistics for questionnaire sections: AIU & NU ...................... 168
xii

LIST OF FIGURES
Figure Page
1. P-I: Regression standardized residuals plot of linear models ............................ 78
2. P-I: Observed verses expected cumulative residuals probabilities .................... 79
3. AGE: Regression standardized residuals plot of linear models ......................... 83
4. AGE: Observed verses expected cumulative residuals probabilities ................ 84
5. Defined linearity of SSA9 and AGE linear models ............................................ 85
6. Defused linearity of SSA9 and P-I linear models................................................ 86
7. GENDER: Regression standardized residuals plot of linear models ................. 90
8. GENDER: Observed verses expected cumulative residuals probabilities ......... 91
9. GROUP: Regression standardized residuals plot of linear models .................... 94
10. GROUP: Observed verses expected cumulative residuals probabilities .......... 95
11. GROUP: Discriminant analysis of questionnaire data ..................................... 96
12. SOURCE: Regression standardized residuals plot of linear models................ 101
13. SOURCE: Observed verses expected cumulative residuals probabilities...... 102
xiii

Chapter 1
INTRODUCTION
The purpose of this study is to measure personal epistemologies relative to novice
through expert electronic product experience to understand why some people have
difficultly learning to use new technologies. This study explores the relationship
between epistemological beliefs, education, and experience using electronic products.
Epistemology is the theory of science that investigates the origin, nature, methods, and
limits of knowledge (Newman, 2000). Personal epistemologies include belief in the
extent of knowledge integration, stability, source, speed of learning, and ability to learn
(Duell & Schommer-Aikins, 2001). The subject of this research addresses one topic in
this broad area by measuring personal epistemological profiles relative to experience
using electronic products. The five epistemological factors measured in this study are (a)
integration (simple knowledge), (b) stability (certain knowledge), (c) source (omniscient
authority), (d) speed of learning (quick learning), and (e) ability to learn (innate
learning). Personal epistemologies influence academic and skill-performance expertise;
therefore, measuring these relationships serves as a guide for development of user-
centered instruction for computing equipment, electronic products, and classroom
education.
Individual differences in age, computer experience, educational environments,
learning styles, and personal epistemologies influence learning outcomes. Schommer-
Aikins and Hunter (2002) used an epistemology questionnaire (Schommer, 1990) and
1

found that participants who believed in complex, tentative knowledge were more likely
to accept multiple viewpoints. They were also willing to modify their opinions, withhold
decisions until more information was available, and acknowledge that everyday issues
are complex and tentative. Lastly, higher levels of education appeared to correspond to
thinking beyond the classroom.
Development of mature beliefs coincides with development of expertise through
experience. Relationships therefore exist between novice skills, naïve beliefs, expert
skills, and mature beliefs. An example illustrating the relationship between experience
and expertise involves learning how to use electronic products. In a study measuring
learning performance using electronic products, Charness, Kelly, Bosman, and Mottram
(2001) used task performance measures to investigate how age and computer experience
influenced word processing training. Results showed age and computer-experience
influence training time and overall word processing performance. General knowledge of
word-processing transferred, so experienced adults were superior to novices on virtually
every measure, indicating that prior experience and knowledge are important variables
when learning technological skills.
Temple and Schmidt-Nielsen (1998) found that experience with word processing
might not always lead to mastery. While some users improve their skills, others are
content with familiar, yet inefficient methods. They proposed that inexperienced users
who prefer simple choices find the complexity of computers overwhelming and learn just
enough to accomplish tasks. However, inexperienced users who prefer complex choices
are more likely to go beyond minimum skills to explore increasingly complex computing.
In summary, experienced users have a learning advantage over novices, and willingness
to explore computer complexities involves preference of complex choices.
2

Sternberg (1999) proposed an additional view of how individual differences
influence learning. He proposed that different educational and work environments yield
differing results depending on individual learning style:
Many of the students we are consigning to the dust heaps of our classrooms have the
abilities to succeed. It is we, not they, who are failing. We are failing to recognize
the variety of thinking and learning styles they bring to the classroom, and teaching
them in ways that don’t fit them well. (p. 17)
It is Sternberg’s contention that “what happens to us in life depends not just on how well
we think, but also on how we think (p. 18).
Individual differences in experiences, preferences, and abilities influence domain-
specific competencies. The studies cited above show that individual differences in age,
computer experience, educational environments, learning styles, and personal
epistemologies influence learning outcomes. As a result, understanding the extent of
relationship between epistemological beliefs, education, and experience provides insight
to evaluate and/or develop electronic products, technical communications, user manuals,
and training programs.
Background of the Problem
Effective use of computers and computer related equipment is a major factor in
acquiring a quality education and maintaining competitive career skills. To be
competitive, successful career development requires effective use of constantly changing
computers, software, and electronic products. The continuous development of electronic
technologies and software applications has created a permanent educational demand that
parallels new product innovations. Bandura (2002) believes advances in electronic
technologies fundamentally changed the world and altered how a society communicates,
educates, works, and conducts daily activities. People must embrace specific
3

technological and sociostructural factors or face societal limitations based on extent and
kind of technological inabilities (Bandura 2001). People partially regulate their lives
according to expected outcomes in return for their efforts to produce. Moreover, these
expectancies influence how people relate to each other during social interactions. As a
result, inability to use electronic technology products becomes a societal concern because
these skills are required for social interactions, communications, education, and business
(Waetjen, 1993). In this way, technological competency is a means of identifying
preferred communication methods of social, educational, business, and potential
customer groups.
Electronic products have rapidly evolved due to advances in technological
capabilities, which yield increased product utility and complexity (Bellotti, Ducheneaut,
Howard, & Smith, 2003). However, product utility improvements oftentimes increase
the amount and complexity of available functions. These improvements involve
evolutionary changes by redesign of existing products to reflect new functions and
capabilities. New products can therefore appear excessively complex and overwhelming
without prior experience with similar devices or software. Not having prior experience
with similar products or finding products difficult to learn also restricts users from taking
full advantage of available functions and even discourages purchases (Quinn & Russell,
1986). In this way, prior experience using similar products influences new-product
perceptions of potential customers.
Previous experiences' using electronic products combined with effective user
interfaces builds a bridge (or scaffold) to learn new product technologies (Marcos, 1993).
Using different electronic products influences abilities to comprehend and integrate
technical information from known technologies, thereby facilitating learning unfamiliar
4

technologies. People who do not have these learning experiences and who do not
understand how to use basic electronic products find it increasingly difficult to use
newer, more complex products. This is because prior experience allows skills to transfer
when learning unfamiliar technologies used for e-commerce, education, entertainment,
and wireless communications (Mountford, Mitchell, O’Hara, Sparks, & Whitby, 1992).
Without these prior experiences, a means of accelerating learning is required to allow
rapid acquisition of basic through advanced skills.
Electronic technologies require continual development of technological skills. For
example, any document previously produced on a typewriter is now produced via
computer (Charness et al., 2001), and this requires a transition from paper to computer-
based processes. Similarly, measuring skills and knowledge has evolved towards greater
use of electronic technologies. It is therefore important to determine if currently used
paper and pencil tests provide equivalent results if provided as computer-based tests.
Clariana and Wallace (2002) measured content familiarity, computer familiarity,
competitiveness, and gender in relation to paper verses computer-based assessments.
The computer-based group outperformed the paper-based group and results were
unrelated to gender, competitiveness, or computer familiarity. However, content
familiarity was a factor, so higher-attaining students benefited most from computer-based
testing. Recommendations included anticipation of paper verses computer test-mode
effects for future studies.
Choi, Kim, and Boo (2003) measured the validity of paper-based and computer-
based language tests for teaching English as a foreign language (TEFL). Results support
comparability between paper- and computer-based subtests of listening, comprehension,
grammar, vocabulary, and reading comprehension. Similarly, Hunt, Hughes, and Rowe
5

(2002) developed a software-tool to assess students’ information technology skills.
Students preferred immediate automatic testing to traditional written tests, and
information technology skills improved using computer-based tests, but written tests did
not show this benefit. In like manner, Shulenberg and Yutrzenka (2001) measured the
equivalence of paper and pencil verses computer testing of the Beck Depression
Inventory-II. All participants received computer aversion and computer experience tests.
Two groups received conventional only, computerized only, conventional then
computerized, and computerized then conventional combinations of testing. Results
found computer and conventional testing provided equivalent measurement validity. In
summary, paper- and computer-based tests produce equivalent measures with minimum
test-mode effects like questionnaire formats. This is especially important when
compensating for participant characteristics such as familiarity with computers and
domain-knowledge expertise. This suggests that well-established paper-based tests are
candidates for computer-based administration. However, in each case, paper-based tests
require validation as computer-based tests before use.
Statement of the Problem
Membership in a technology-rich society requires comfortable and competent use of
computers and related computer equipment. Continuous career development is
imperative to remain competitive in a world of rapid technological development. These
technological developments include knowledge domains for both hardware and software
solutions (Bandura, 2002). An example is the Internet. Internet use is rapidly expanding
as a source for information for academic research and independent learning that supports
business, financial and e-commerce decisions. Technological competency is therefore a
6

means of identifying societal groups based on ability to use electronic products. To
expand the population of potential users, a means of accelerating learning is required to
promote rapid acquisition of basic through advanced technology skills. This study
addresses these needs by measuring how people organize their beliefs about knowledge
and leaning as a profile reflecting experience using electronic products.
Another problem addressed in this study includes understanding why some people
have a difficult time learning to use computers and new electronic technologies. The
inability to use rapidly evolving electronic products has become a societal concern
because electronic technologies are increasingly used for social interactions,
communications, education, and business (Waetjen, 1993). Technological competency is
therefore a means of identifying knowledge and skill-based social groups, educational
activities and in the business arena, defines potential customers. In this way, inability to
use electronic products defines requirements to develop a means of accelerating learning
for the rapid acquisition of basic through advanced technology skills. Continuous
development of technological skills therefore involves reassessment of educational
requirements. For example, since electronic documents have replaced an increasing
number of printed documents (Charness et al., 2001), individuals find themselves having
to transition from paper to computer-based reading, testing, and or searching for
reference documents. Print and electronic media are typically provided to learn and then
use computers and electronic products. For business, consumer, and educational
products, it is becoming increasingly important to provide easily understood user training
that simplifies learning new technologies (Bellotti, Ducheneaut, Howard, & Smith,
2003). To provide simplified technical communications it is paramount to determine if
paper and computer-based instructions are equivalent. One way to determined paper-
7

computer equivalencies is to administer a computer-based version of an established paper
questionnaire.
The epistemological questionnaire used in this study is by Schommer (1989, 1990,
1993, Schommer & Walker, 1995), who used print-based versions for numerous studies.
Other researchers (e.g., Clarebout, Elen, Luyten, & Bamps, 2001; Kardash & Scholes,
1996; Schommer-Aikins & Hunter, 2002) also used print-based versions of the same
Schommer Epistemology Questionnaire. However, extent of relationship between the
print-based and an equivalent computer-based version of the Schommer-Aikins
Epistemology Questionnaire is unreported (Schommer-Aikins, personal communication,
April 17, 2004).
Personal epistemological beliefs about knowledge and learning are associated with
problem solving and to some extent domain knowledge (Schommer & Walker, 1995), yet
beliefs remain more or less domain independent. In this respect, exposure to and
competence using electronic technologies is associated with the continuum of novice to
expert conceptual frameworks. Other factors such as age, education, and experience also
influence personal epistemologies and expertise such that sociobiological experiences
influence development of mature epistemological beliefs (Schommer, 1994) and
expertise (Charness et al., 2001). The development of expertise and mature
epistemologies is addressed by constructivist-learning theory. It is the combination of
cognitive, and skill-based activities that interactively create real changes based on
sociobiological predispositions (Wertsch & Tulviste, 1992). These experiences shape
personal beliefs about knowledge and learning and expertise. It is therefore the intent of
this study to measure the extent of relationship among epistemological factors: (a) Paper
verses Internet questionnaire administration, (b) age, (c) gender, (d) educational status
8

(undergraduate, graduate, faculty), (e) self-taught technology education, (f) certificate-
apprenticeship technology education, (g) earned degrees, and (h) experience using
electronic products.
Purpose for Study
The purpose of this study was to measure how epistemological beliefs influence
learning and using electronic products. Results provide an organizational map of
epistemological beliefs that reflects hierarchies of epistemological beliefs in relation to
experience using electronic products. Hierarchies of epistemological beliefs serve as
guides to create effective technical communications accommodating naïve through
mature beliefs for novice through expert electronic product users.
An additional purpose of this study was to measure the extent of relationship
between paper and computer-based questionnaire administrations. It is important to
determine equivalency of paper and computer-based tests to compare this study with
prior research measuring epistemological beliefs.
Theoretical Framework
The theoretical foundation of this study was social learning theory supported by the
work of Piaget (1955, 1957, 1968/1969), Vygotsky (1926), Perry (1968), Schommer
(1989, 1990), Brunswik (1937) and Hammond (2001). Piaget, Perry, and Schommer are
constructivists, while Brunswik and Hammond are functionalists. Constructivists and
functionalists focus on whole person issues, including origins of sociobiological basis of
behavior. Functionalists focus on sociobiological experiences as decision processes that
occur in response to environmental experiences and extend constructivist views by
9

measuring learning as human performance. Brunswik's measurement procedures were
named probabilistic functionalism (1935) and as ecological cue theory by Juslin (2001),
Juslin and Olsson (1997), Juslin, Winnman, and Olsson, (2000), and Vincent and Wang
(1998). Functionalists operationally define measures of sociobiological constructs
proposed by Piaget (Beilin, 1992) and Vygotsky (1926) by measuring experiences as
adaptive interactive between people and environments. Hammond (2001) comments on
Brunswik advance the notion that adaptation is an intentional, repeatable, and acceptable
outcome based on progressive domain independent and domain dependent decisions.
Adaptation is therefore an intentional act based on interactively organized purposeful
behaviors.
Brunswik’s (1935) measurement processes were similar to Vygotsky’s proximal
learning theory. Both proximal learning theory and probabilistic functionalism
emphasize the importance of understanding transitions between relatively stable levels of
understanding. In simplified terms, understanding experience involves measuring a
series of dynamic hypotheses, constantly updated with new information and
interpretations of experience. Learning is a systems-process of hypothesis testing, mental
model matching, and dynamic fits between expectations and experience (Shanks, 1995).
This process progressively refines interpretations of experience until results are judged
unbiased, stable, and predictable (Jones, Juslin, Olsson, & Winman, 2000). For example,
epistemological beliefs influence strategies that interpret and integrate information
(Schommer, 1989, 1990). This selection process identifies information as more or less
adaptive, indicating people exhibit individualistic styles or differences in their
approaches to learning Piaget (Stevens, 2000), Sternberg, (1997). As in Vygotsky’s
proximal learning theory and Brunswik’s probabilistic functionalism, epistemological
10

beliefs are learned responses to sociobiological experiences. These responses are
adaptive behaviors (solution-sets or schema) in response to problem-solving challenges
(Piaget, 1969 pp. 189 - 198).
Ecological Validity Theory (Juslin & Olsson, 1997) exemplifies the relationship
between epistemological beliefs and Brunswik’s theories of probabilistic mental models.
Decisions and cue validities are influenced by prior experiences and present choices
(Juslin, Winnman, & Olsson, 2000). These decisions utilize incorrect, correct, negative,
and positive outcomes relative to cue frequencies experienced within specific
environments. Within this context, specific-environmental experiences develop specific
beliefs about knowledge and learning (Vincent & Wang, 1998). As a result, cue
utilization serves as a generalized approach toward learning and problem solving. This
occurs as a process of selecting and then learning the most adaptive or acceptable
strategies from prior experience. However, since these approaches are best efforts
tempered by capabilities and intentions, they may not be the most adaptive—just
acceptable.
Dweck and Leggett (1988) documented and described major patterns of adaptive
(mastery-oriented) and maladaptive (helplessness) approaches to learning. These
learning approaches stem from implicit learning and beliefs. In the domain of intellectual
achievement, conceptualization of goals differentiates into performance and learning
categories. Deterioration of performance and avoidance of challenges is associated with
performance goals. Seeking challenging tasks and maintenance of striving under failure
are mastery-oriented leaning goals. Students who believed knowledge was a process of
adding learning experiences developed a sense of mastery, while students whose learning
approaches involved meeting specific performance goals experienced feelings of
11

helplessness when these goals were not achieved. Belief that learning is a process helps
students’ work-through failures, but belief that learning involves meeting performance
goals leads to expectations of all-or-none learning.
Problem solving processes are approaches influenced by individual differences in
personality, perception, and cognitive and/or thinking styles (Sternberg, 1999). Thinking
style is a preferred approach or response to experience, so activity and expertise are
functions of exposure, choice, and ability. What people are good at and what they like to
do tend to be the things they do best. When confronting novel or ill-structured
information, people exhibit naïve approaches to problem solving (Jehng, Johnson, &
Anderson, 1993). As experience increases, prevalence of mature problem solving
attitudes and techniques increase. In this way, positive prior experiences lead to
progressive increases in expertise that become expectations for future novel experiences.
This emphasizes the importance of viewing personality and personal epistemologies as
profiles of weighted attribute continuums shaped by learning.
Beliefs about knowledge and learning reflect thinking styles (Sternberg, 1999) and
epistemological factors of simple knowledge, certain knowledge, quick learning,
omniscient authority, and innate ability (Schommer, 1990). Thinking styles and personal
epistemologies reflect adaptive behaviors and individualized responses to socio-
biological experience. In the functionalist traditions of Brunswik (1943), beliefs
influenced human performance and expertise because learning was an active process-
exchange with the environment (Juslin & Olsson, 1997). To measure these interactions,
this study considered extent of relationship between learning and experience, using
electronic products as influenced by personal beliefs about knowledge and learning.
12

Importance of the Study
This study measured extent of relationship between experience using electronic
products and beliefs about knowledge and learning. If experience using electronic
products and beliefs about knowledge and learning are related, product instructions and
formal education may require different approaches for technically naïve through mature
students. Additionally, extent of relationship between paper- and computer-based
administrations of Schommer’s epistemology questionnaire is another import measure.
Knowing equivalence of paper and computer-based administrations allows educators to
choose between equally viable instruments for testing technologically naïve through
mature students. This also provides opportunities to obtain valid measures when
computers are not available or desired.
Prior research (e.g., Kardash & Scholes, 1996; Perry, 1968; Schommer, 1989;
Schommer-Aikins & Hunter, 2002) concentrated on non-technology issues while this
study focused on measuring profiles of epistemological beliefs as a function of self-
taught technology education, certificate-apprenticeship technology education, earned
degrees and certificates, and experience using electronic products.
With increased use of computers in our society, it is important to understand extent
of relationship between experience using electronic products and beliefs about
knowledge and learning. Knowing these relationships may indicate a need for different
instructions to meet the needs of people with epistemologically naïve to mature beliefs in
relation to novice through expert skills and behaviors. The challenge is to provide a
range of specific knowledge-domain instructions accommodating epistemological beliefs
and levels of expertise. Knowing how to use computing devices for communication,
education, and business is important. As a result, educational experiences need to reflect
13

a range of beliefs about learning, the range of prior knowledge, levels of skill expertise,
and epistemological beliefs. Measuring extent of relationship between personal
epistemologies and experience using electronic products is an effort to quantify learning
diversity to empower learning experiences.
Research Questions
This study explored the extent of relationship among personal epistemologies, self-
taught technology education, certificate-apprenticeship technology education, earned
degrees and certificates, and experience using electronic products. Identical paper- and
computer-based questionnaires administered to faculty, undergraduate, and graduate
school students at Alliant International University and National University addressed
these questions:
1. What is the extent of relationship between computer- and paper-based
questionnaires?
2. What is the extent of relationship between participant age, gender, educational
group, university attended, and (a) self-taught technology education, (b) certificate-
apprenticeship technology education, (c) experience using electronic products, and (d)
personal epistemologies?
Scope of the Study
Subject sampling was restricted to faculty, undergraduate, and graduate students
at Alliant International University (AIU) and National University (NU) without definition
of population demographics. Instead, classroom membership determined subject
selection and generally parallel AIU and NU demographics. Participants completed the
14

self-taught technology education, certificate-apprenticeship technology education, earned
degrees and certificates, and experience using electronic products questionnaires prior to
the Schommer-Aikins Epistemology Questionnaire. Presenting technology questions
before completing the epistemology questionnaire perceptually set respondents to focus
on experiences using or learning how to use electronic products. This framed the
knowledge domain so questionnaire responses reflected beliefs about knowledge and
learning related to personal experiences with electronic products. This technique
paralleled experimental methods used by Schommer (1990, 1993, Schommer & Walker,
1995), where subjects completed an epistemological questionnaire while keeping in-mind
mathematics or social science text. The Schommer-Aikins study asked respondents to
keep in-mind mathematics and social sciences when completing the epistemological
questionnaire. Rather, questionnaire instructions asked respondents to concentrate on
experiences with electronic products while answering questions. Table 1 presents the
instructions that request respondents to answer questions relative to experiences using
electronic products.
Table 1
Instructions for the Schommer-Aikins Epistemology Questionnaire
Please Keep in Mind Your Prior Experiences Using Electronic Products
While Answering Questions
For each statement rate the extent to which you Disagree or Agree
Strongly
Disagree
Strongly
Agree
There is no right or wrong answers for these questions.
We want to know what you really believe.
1 2 3 4 5
1 Epistemology questions
15

The demographics and experience using electronic products questionnaire used a
pattern of basic through advanced tasks as outlined in Microsoft's office-specialist
certification program for educators (Microsoft, 2004). This program presents task
complexity in a progressive manner to define mastery of common operations beginning
with physical activities to format, move, modify, or organize documents and information.
Higher skill levels are associated with customizing software preferences and advanced
features to evaluate, analyze, and communicate. The progression of skills includes (a)
Microsoft office 2003, (b) windows server 2003, (c) windows storage server 2003, (d)
visual studio.net, and (e) exchange server 2003. These programs describe a path
requiring competent use and then competent creation of software tools. As a result,
technical skills chosen as topics in this study parallel this organization and emphasized
continuums of competent physical use to competent creation of problem solving methods
and or processes. The intent was to assess extent of self-taught and certificate-
apprenticeship technical training as a continuum of primarily manual to primarily
cognitive tasks. It is important to note that self-taught and certificate-apprenticeship
sections are nearly identical. This is to identify differences between personal interests
and on-the-job training as opposed to formal training in similar subjects and tasks. This
is not a standardized test; therefore, results are specific to this research.
The experience using electronic products questionnaire also used a general format
presented by Microsoft’s office-specialist certification program for educators (Microsoft,
2004). The concept of starting out with elementary if not physical skills such as typing
and knowing how to physically maintain computers served as a model to identify
progressively more interactive and more purely cognitive tasks.
16

Topics for the experience using electronic products questionnaire were
constructed using results obtained from Internet searches using Copernic Agent
Professional (2003). Example search words are basic computer skills, internet, email,
and desktop publishing. In like manner, the self-taught and certificate-apprenticeship
technical training questionnaires were populated using search words like skilled trades, it,
computer technology, life sciences, technical schools, and university curriculum.
Statistical equivalence between this study’s sample population and published
results of the Schommer-Aikins Epistemology Questionnaire provides extent of
equivalence between this study and published data. Self-taught technology education,
certificate-apprenticeship technology education, earned degrees and certificates, and
experience using electronic products questionnaires are specific to this study and
represent an extension to this body of knowledge concerning the relationship among
personal epistemologies and domain knowledge competencies.
Definitions of Key Terms
Competencies: Expert use of electronic products.
Conceptual Frameworks: Domain knowledge organization.
Consciousness: Metacognitive awareness of mental functioning.
Constructivists: Broad group of social learning theorists advocating progressive
acquisition and organization of experiences through processes of sense making.
Correspondence: Extent of statistical or perceptual judgments of equivalence.
Electronic Products: Any number of software and computer hardware products for
office productivity, educational, entertainment, and employment activities.
Empirical: Comparing objects or events, picking out common or salient features,
17

and formulating general concepts.
Epistemology: Theory of the origin, nature, methods, and limits of knowledge.
Equilibration: For the process of bringing about homeostasis or balance
Expert Performance: Competence through ability to utilize richly structured
knowledge.
Explicit: Readily observable as in conscious recollection of facts.
Functionalist: Broad group of social learning theorists and human performance
professionals advocating objective measures of learning and cognitions.
Genetic Epistemology: Sociobiological basis of learning.
Immature (Naïve) Epistemology: Belief in simple, non-integrated, all-or-none
learning, unchanging learning ability, and duelist views of understanding and knowledge
creation.
Implicit: Implied, not directly expressed without conscious recollection of facts
Individuation: The process of experiential differentiation.
Knowing: Metacognitive experience of knowing information, facts, and knowledge.
Helplessness: Belief that fixed intelligence leads to deterioration of performance and
avoidance of challenges.
Mastery: Belief that incremental intelligence leads to maintenance of effort in spite
of failure and desire to seek new challenges.
Mature Epistemology: Belief in complex, relativistic, integrated, and incrementally
changing understanding of knowledge creation.
Mind-Body Dualism: Philosophical view that cognitive activity (mind) is
independent and separate from the physical body.
Naïve Epistemology: See Immature Epistemology.
18

Nature-Nurture: Philosophical argument that learning results from biological verses
learned behaviors.
Novice Performance: Competence level reflecting inability to utilize richly
structured knowledge.
Technology Education: Learning about and learning how to utilize various
manufactured products as knowledge creation and problem-solving tools.
Technologies: Any number of manufactured products used as problem solving and
knowledge creation tools.
19

Chapter 2
REVIEW OF LITERATURE
Epistemology is the theory of science that investigates the origin, nature, methods,
and limits of knowledge. Classical thought such as Descartes’ Epistemology was a
system of priorities that referred to matters that must first be confronted to acquire
knowledge (Newman, 2000). Current epistemological interest explores the development
of beliefs about knowledge and learning (Piaget, 1955; Schommer, 1989) in relation to
the logical and rational organization of knowledge. It is extremely important to note that
large bodies of work progressively create foundations to define and measure
epistemological constructs. Measurements of personal epistemologies and cognitive
development are parallel when viewed as combinations of sociobiological (Gardner,
1983), genetic epistemology (Piaget, 1968, 1969), and constructivist learning theories
(Vygotsky, 1997). These theories reflect individuation of capabilities, capacities, and
responses to learning and environmental adaptation. Each postulates the ability to
measure learning diversity as statistically predictable events with variability within and
between occurrences and people.
Allport (1954), Gardner (1983), Perry (1968), Piaget (1955), Sternberg (1997), and
Vygotsky (1997) hold various views of individuation through sociobiological learning
and differentiation. To these theorists, learning is defined as an individualistically
determined response to socio-biological and environmental adaptation through
purposeful problem solving. Personal epistemologies are characteristic yet unique
20

profiles reflecting a kind of personality of information processing and problem solving.
Accordingly, one portion of this study’s theoretical basis contends that learning is a
personal and purposeful response to experiences using electronic products. These
individualized experiences occur through intentional problem solving influenced by
personal beliefs in knowledge and learning. Measuring extent of relationship between
experience using electronic products and epistemological beliefs is significant because
results reflect both characteristic and individuated approaches towards problem solving.
The next sub-sections address one explanation of why epistemological beliefs are
multidimensional and develop somewhat independently.
Foundational Studies: Social Learning Theory
Piaget’s developmental theory employed several concepts established by prior
works: (a) measurable intelligence (Binet, 1905), (b) natural selection (Darwin, 1874),
(c) mind-body dualism (James, 1904), (d) medical models of consciousness by Janet
(Murchison, 1930), (e) Correlational statistics (Spearman, 1904), and (f) nature-nurture
equilibration (Wright, 1873). Piaget’s systems approach to the interactiveness of
biological and social learning is the foundation of genetic epistemology (van Geert,
1998).
Genetic Epistemology and sociobiological aspects of learning reflect logical
formulations as a means of transitioning between functional equilibrium levels.
Equilibration is the homeostatic process that strives for balance within logical
formulations through sociogenesis. In this way, Piaget (Lourenco & Machado, 1996)
viewed development as a series of small, progressive discontinuities that involve
qualitative and quantitative transformations. Discontinuities appear as stages because
21

they are points of unity at specific moments during transformations from one level of
equilibrium to another. Stages in this sense are successively attained multi-modal levels
that represent a constructed equilibrium reflecting biological, environmental, and social
learning. Since no two people share exact experiences or biological structures, learning
is individualized and characterized as shared (characteristic) yet unique. This follows
logical arguments by Vygotsky (1997) that learning is characteristically unique because
of natural cyclical fluctuations with patterns similar to and normally found in the
environment and shared by all living things. Therefore, stages represent plateaus
between biological and learning transitions that follow evolutionary patterns of natural
selection and exhibit patterns of fluctuations. Alternative views that leaning occurs as a
continuous process of intentional problem solving through progressive and additive
element utilizations (Tolman & Brunswik, 1935) is discussed in later sections.
Vygotsky’s concept of learning through social interactions is a key feature of
epistemological development (Wertsch & Tulviste, 1992). Vygotsky hypothesized that
an individual’s mental functioning can only be understood by examining experiences
derived from social and cultural processes. For Vygotsky, homeostasis (balance)
occurred as goal attainments achieved through successively accurate learning
estimations. This process continues until attaining true or veridical understanding.
Karpov and Haywood (1998) explored Vygotsky’s concepts of metacognitive and
cognitive mediation as the primary mechanisms of learning. Metacognitive mediation
refers to acquisition of semiotic tools of self-regulation, self-planning, self-monitoring,
self-checking, and self-evaluating. Children use these metacognitive or executive
processes to regulate and internalize learning. However, Vygotsky’s views are in conflict
with Piaget’s (1968) views of constructivist learning. Piaget also viewed learning as
22

constructed but not transmitted and Vygotsky believed discovery learning should occur
after presenting precise scientific knowledge. Vygotsky believed children should not
have to rediscover what humankind already knows. It is therefore essential for
instructors to moderate student acquisition of declarative knowledge and to guide
development of implicit knowledge through meaningful problem solving.
Vygotsky (1997) and Piaget (1968) shared sociobiological views of learning.
Vygotsky proposed a developmental learning theory employing the zone of proximal
development while Piaget developed the theory of assimilation, accommodation, and
equilibration. The zone of proximal development consists of successive solution
approximations that gradually result in increasingly correct answers. Similarly, Piaget’s
mechanism of assimilation is the process of fitting new experiences in relation to prior
learning. The process of adjusting to new perspectives of external reality and
equilibration is accommodation. It is the adjustment process to modify, optimize, and
balance learning as an interactive system (Hilgard & Bower, 1975, pp. 322-323). Piaget
and Vygotsky both used qualitative and quantitative methods to record observations of
behavioral data. Observational studies are time-event-based records of behavioral
occurrences producing sequential data. This process lends itself to linear explanations of
how knowledge evolves as accumulative patterns. In general, knowledge construction is
sequential (scaffolding), so expertise matures linearly, in stages, such that knowledge,
meaning, and understanding do not exist outside of meaningful, intentional activity
(Jonassen, Peck, & Wilson, 1999). In this way, students learn from thinking and learning
occurs from personal and socially influenced sense making (Vygotsky, 1997). Although,
linear, this process is by no means consistent within or between individuals.
23

Learning occurs in response to environmental influences and internal processes, so it
is adaptive and individually specific. In this way, learning has patterns reflecting
physical and cognitive states. Piaget (1955) referred to this process as logical
formalisations applied to equilibrated thought-structures. In certain cases, thought
development is a process of transformations from one level to another. Vygotsky (1997)
also referred to these processes but in slightly different terms. He viewed development
as a gradual structural progression through the accumulation of slight variances, so
growth (maturation and competencies) followed periods of rise, stagnation, and
abatement. These patterns also occur in nature as cyclical, developmental, and vary
within and between periods of occurrences. For Vygotsky, this constituted a fundamental
law of a child’s behaviors as patterns of change that occurred daily and over time.
Central to theories of epistemology is the proposition that learning is progressive,
has patterns, and reflects socio-biological experiences (Hofer & Pintrich, 1997). Piaget
(Lourenco & Machado, 1996), Vygotsky (1997), Perry (1968), and Schommer (1990)
hypothesized progressive systems of body (biological), mind (perception-consciousness),
nature (evolution and biological), and nurture (environmentally influenced learning)
relationships. The importance of this developmental system lay in the process itself.
Philosophical and scientific theory evolves with continuous acquisition of
knowledge that reflects the best information available. As information changes, theories,
and practices change, so accepted thought has a period of acceptance until proven
untenable or it requires modification (Brunswik, 1937). Brunswik’s views of adaptive
learning reflected his work to measure human behavior without resorting to subjective
sciences based on phenomenology or Gestalt psychology (Koffka, 1922). His work on
measuring perceptual constancies of sensory systems (Brunswik, 1937, 1943, 1955) led
24

to the supposition that people have the capacity to respond to different stimuli and
situations in the same way or the same stimuli and situations in different ways. This
culminated in proposing a process to measure uncertain or changing information. Simply
stated, people make their best guess at understanding and responding to experiential
changes. Each best guess is an instantaneous interpretation of dynamic changes. These
tachistoscopic captures are much like each frame in a continuous series of photographic
snapshots. The result is a string of interrelated points or periods of understanding. When
viewed as time-lines, these captures are like motion-picture films. This creates a
psychology that connects transitional states based on tentative and relatively fluid
information. The name coined for Brunswik’s theoretical view is probabilistic
functionalism because it dealt with probabilistic decisions, learning, and progressive
change in response to adaptive goal directed behavior. These views are instrumental in
explaining why epistemological beliefs are interrelated yet somewhat independent. If
learning occurs as a series of progressive best guess decisions (building blocks),
incomplete learning results in isolated views of unrelated components (concepts and
information). As learning progresses, new connections occur between existing building
blocks, so larger views, based on expanded knowledge interrelations, are developed. In
this way, personal epistemologies of simple knowledge, certain knowledge, innate
ability, omniscient authority, and quick learning (Schommer, 1990) all represent different
facets of knowledge construction. The process occurs as incremental creations of
learning elements that progress from naïve through mature epistemologies that parallel
novice through expert behaviors.
Connected and unconnected knowledge is a fundamental difference between
naïve and mature epistemologies (Dweck & Leggett, 1988; Schommer, Crouse and
25

Rhodes, 1992) and novice and expert behavior (Kalyuga, Chandler, & Sweller, 1998;
Woltz, Bell, Kyllonen, & Gardner 1996). Consequently, if probabilistic functionalism
(Brunswik, 1937, 1943, 1955) postulates are true, then personal epistemological profiles
reflect the manner in which sense is made of experience. As individuals gain experience,
they update their best guess understandings until those understandings are predictable
and consistent. Incomplete understanding consists of isolated or unconnected knowledge
with inconclusive or unresolved levels, and generally, is a knowledge system based on
incomplete information.
Kardash and Scholes (1996) measured people’s beliefs about knowledge, their
strength of beliefs in controversial issues, their tendency to enjoy effortful thinking, and
their interpretation of controversial issues. Results supported Schommer’s (1990)
research that epistemological beliefs are relatively independent and they influence critical
interpretation of knowledge. Findings also showed that general beliefs in certainty of
knowledge influence highly tentative and controversial information, so interpretations of
information conform to beliefs. In addition, strength of specific beliefs is as important as
general epistemological beliefs when constructing conclusions drawn from text.
Moreover, a person’s specific habitual approaches to complex and challenging tasks
influence beliefs and conclusion development of tentative and controversial information.
In summary, results showed how strengths of general and specific topic beliefs influence
the approach to and interpretation of information.
Schommer-Aikins, Brookhart, Hunter, and Mau (2000) measured middle school
children’s beliefs about knowledge and learning. The Schommer-Aikins Epistemology
Questionnaire (Schommer, 1990) measured beliefs that intelligence, learning, and
abilities are fixed or they improve over time. Results correlated to prior research
26

(Schommer and Dunnell, 1994) in that age was related to beliefs about stability of
knowledge and when students do not believe in quick-learning or fixed intellectual
ability, they have higher grade point averages. Similarly, Dweck and Leggett (1988)
found students displayed helpless behavior when they believed in fixed intelligence.
However, students who believed in incremental intelligence showed persistence, effort,
and used a variety of problem-solving strategies when confronting difficult tasks.
Epistemological Definitions and Measures
Central to the discussion of epistemological measures are debates whether beliefs
about knowledge and learning are uni-dimensional (Perry, 1968) or multidimensional
(Duell & Schommer-Aikins, 2001; Schommer, 1989). Schommer extended Perry’s work
by suggesting epistemological beliefs are more or less independent. Beliefs may not
develop at the same rate or at the same time because they are somewhat independent of
each other. Belief-profiles range between naïve and mature for epistemological factors of
simple knowledge (knowledge integration), certain knowledge (stability), innate abilities
(ability to learn), quick learning (speed of learning), and omniscient authority (source).
They are a system of interrelated yet somewhat independent sets of beliefs about
knowledge and learning and describe the extent of naïve to mature beliefs about learning.
They are mental representations of knowledge that influence integration and
interpretation of information. In summary, epistemological beliefs are more or less
independent from one another; they range from naïve to mature and develop somewhat
independently. Table 2 provides the definitions of belief-profile continuums.
27

Table 2
Schommer-Aikins: Epistemological factor definitions
Factor Description Knowledge Beliefs
Simple
Knowledge
Knowledge
Integration
Isolated through interrelated knowledge concepts
Certain
Knowledge
Stability of
Knowledge
Unchanging through constantly changing knowledge
Innate Abilities Ability to Learn
Fixed-at-birth through improves over time and with
experience
Quick Learning Speed of Learning
Quick or not at all through occurs as a continuous
process
Omniscient
Authority
Source of
Knowledge
Autocratic through developed by reasoning using
empirical evidence
The concept of stages proposed by Piaget (Beilin, 1992) and Vygotsky (1926)
suggests that while learning is a characteristic process that retains individuation among
all people, no two people experience exact patterns of learning even if they experience
similar sociobiological transformations during individuation. Learning and maturation
occur through social, biological, environmental, logico-mathematical, intuitive problem
solving, introspective thought, and other factors that develop at different rates and times
among people.
One view of individuation is uni-dimensional (Perry, 1968) and another is
multidimensional (Schommer, 1989). Uni-dimensional epistemological theory assumes
factor-dimensions develop together as interrelated systems. Multidimensional
epistemological theory accepts that epistemological factors develop collectively but adds
that beliefs may also develop somewhat independently. In either case, diversity and
28

individuation occur because of recombinant problem solving in response to
individualistic sociobiological experiences. The diversity of behaviors is characteristic
within expected ranges of epistemological beliefs, but individualized. In this way,
personal epistemologies reflect individualistic responses to environmental exposures.
Education and Personal Epistemologies
Epistemology is the philosophical theory of knowledge. Beliefs about knowledge
and experience of knowing include knowledge domains, intellectual skills, personal
learning, expectations (Schommer & Walker, 1995), declarative knowledge, and implicit
knowledge (Hofer, 1997; Hofer & Pintrich, 1997; Stevens 2000). Measurement of
perception, reasoning, and intellectual capabilities are of interest to learning theorists and
practitioners because people develop predispositions towards learning that influence
educational experiences (Bandura 1989).
Knowing that beliefs about learning are predispositions and/or expectations can
guide development of learning environments (Schommer, 1990). Learning theorists have
long speculated on developmental aspects of intellectual characteristics. One such
theorist, Piaget (1968, 1969), proposed the concept of genetic epistemology to explain his
views of intellectual development. Not surprisingly, when Piaget connected
psychological constructs of learning with philosophical beliefs about knowledge, it
garnered many years of lively debate.
Recently, Stevens (2000) revisited Piaget’s theories from a dynamic systems
perspective to examine how abilities become increasingly more robust and flexible over
time. In a similar manner, Schommer (1989, 1990, 1993, & 1994) proposed a systems
view to present a multidimensional theory of epistemology that identifies four relatively
29

independent belief components: (a) fixed ability, (b) simple knowledge, (c) quick
learning, and (d) certain knowledge, and the unsubstantiated factor of omniscient
authority. All components evolve as a function of experience, time, and knowledge
domain, but do not follow a set developmental pattern or sequence.
Prior views of epistemology (Hofer & Pintrich, 1997; Perry; 1968) also show that
beliefs about knowledge and knowing change as a function of experience and time.
However, Perry (1968) presented epistemological beliefs as a developmentally uni-
dimensional interpretation of his subjects’ pluralistic educational experiences.
Schommer (1989), on the other hand, agreed in spirit with the developmental nature of
beliefs as a function of time and experience. However, she found that beliefs did not
develop at the same time or necessarily in any sequence. Beliefs about knowledge are
measurable and individually specific but are also somewhat domain independent. In this
way, naïve views may be held for one knowledge-domain and expert views held for
another. Earlier views suggested changes in epistemological beliefs followed progressive
stages of development; current views emphasize relatively independent cognitive and
motivational constructs (Stevens, 2000). Nevertheless, generalized naïve or mature
epistemological beliefs are expected during novel experiences (Schommer & Walker,
1995).
Epistemological beliefs range between naive to mature for each of the five factors
of Fixed Ability, Simple Knowledge, Quick Learning, Certain Knowledge, and the
unsubstantiated factor of Omniscient Authority. Development of the Schommer-Aikins
Epistemological Questionnaire included two or more factor subsets to measure each of
the five primary factors. As an example, people can either oversimplify or complicate
information or compartmentalize one aspect. The questionnaire was written from a naive
30

perspective, so half the items yield mature agreement and half yield naive agreement
(Schommer, 1989). Each item in the questionnaire probes extent of agreement relative to
beliefs about knowledge and learning and not domain knowledge expertise. Schommer
(1990) measured epistemological beliefs and comprehension of social science and
physical science texts. Results found that as belief in quick learning increased beliefs
about certain knowledge increased. When this happened, students produced increases in
inappropriate or created absolute conclusions drawn from reading tentative text.
Schommer (1992) found that beliefs in simple knowledge were associated with poor
comprehension, poorly monitored comprehension of mathematical text, and simple study
strategies like memorization. Continuing in this research, Schommer and Walker (1995)
specifically measured and obtained supporting results that epistemological beliefs were
moderately domain-knowledge independent. Schommer-Aikins and Hunter (2002)
extended epistemological belief research into the way people think about everyday
controversial issues outside of the classroom.
Most of the research measuring epistemological beliefs referenced in previous
sections focuses on classroom and academic issues. Epistemological beliefs are most
evident in higher-order beliefs such as complex and tentative college level information.
When constructing epistemological beliefs about complex knowledge domains, early
information organizational activities build on framework issues. These framework issues
are building blocks used to create manageable schema, progressively populated to reflect
increasing complexity. However, there is at times a tendency to oversimplify
information resulting in reductive biases. This bias oversimplifies information and tends
to foster an over-reliance on single mental representations and rigid
compartmentalization of knowledge components.
31

Implicit knowledge is foundational to explicit-declarative knowledge (Eysenck,
1994 pp. 185 - 186). In like manner, epistemological beliefs involve explicit knowledge,
metacomprehension and are foundational to problem solving approaches (Schommer,
1993). Therefore, the question remains whether implicit knowledge and epistemological
beliefs are related. Beliefs about knowledge and learning comprise generalized problem-
solving approaches. These problem-solving approaches are well learned and not readily
available for overt measures, so measurement of epistemologies requires indirect
measures.
The extent of exposure to interactive electronic technologies provides distinctive
learning experience. Jonassen, Peck, and Wilson (1999) outlined these learning
experiences and emphasized the unique features of using computers as learning and
thinking tools. These unique features involve interactivity with a computer during
personal learning experiences. This includes utilizing interactive, problem solving with
devices requiring specific usability skills (Norton & Wiburg, 1998, pp. 29 - 33). An
example is the interactivity of person-to-person competition occurring in Internet
gaming. Internet gaming involves collaborative interactivity and is very similar to on-
line educational communities (White & Weight, 2000). Entertainment and educational
activities include interpersonal sharing of social experiences with someone in immediate
or remote locations. This occurs through media such as email, instant messaging, text, or
voice communications (Pena-Shaff, Martin, & Gay, 2001).
The instrument used in this present study was the Epistemology Questionnaire
created by Schommer who portrayed personal epistemologies as relatively independent
of domain-knowledge (Schommer & Walker, 1995). This conclusion was based on a
study that found epistemological beliefs relatively independent, for most students at a
32

consistent level of epistemological sophistication across knowledge-domain, such as
social science and mathematics. These results occur when questionnaires are answered
relative to specific knowledge-domains.
Research by Schommer (1989, 1990), and Schommer-Aiken, and Hunter (2002)
investigated why mental representations of knowledge are selected as a function of
beliefs about knowledge. Epistemological beliefs influence interpretation of information,
integration of knowledge, selection of study strategies, thinking about comprehension,
control of comprehension, and the mental representation of knowledge. The knowledge
and experiences individuals bring into learning and problem-solving situations influence
their beliefs about knowledge and learning. In other words, a person’s beliefs about
knowledge and learning influences his or her future experiences. Epistemological beliefs
are therefore characteristic preconceptions based on mind/body, nature/nurture learning.
Parallel concepts include schema or perceptual set, where frame of reference is a
readiness to respond in context dependent ways and provides judgment standards for
similar experiences (Brunswik, 1937).
Schommer and Walker (1995) required students to “take into account” specific
domain knowledge such as the sciences or social sciences when responding to
epistemology questionnaires. In a similar fashion, this present study requests participants
to answer questions about their education and experiences using electronic products prior
to completing the epistemology questionnaire. Directions request participants to
complete the epistemology questionnaire relative to experiences learning and using
electronic products. The intent is to measure extent of relationship between mature and
naïve epistemological beliefs relative to type of education and experience using
electronic products. Expectations are that mature epistemological beliefs coincide with
33

increased experience and mastery of electronic products. Conversely, naïve
epistemological beliefs coincide with less experience and novice mastery of electronic
products.
Kardash and Scholes (1996) used a portion of the paper-based epistemology
questionnaire (Schommer, 1990). Their research examined the influence of general
beliefs about certainty of knowledge, strength of a particular belief in a controversial
subject (AIDs), and tendency to engage and enjoy complex effortful thinking and
interpretation of text with available conflicting evidence. As expected, the less people
believed in the certainty of knowledge, the less extreme their initial beliefs about
controversial issues and the more they enjoyed cognitively challenging tasks and the
more likely they were also inclined to write accurate reflections of inconclusive, tentative
evidence. The results of the Kardash and Scholes’ study supported the generalizability of
epistemological factor structures (Schommer, 1990), namely that factors are relatively
independent and they influence the interpretation of knowledge. Their work also found
that strength of specific beliefs about controversial subjects is as important as general
epistemological beliefs in interpreting mixed and inconclusive topic evidence. This study
emphasizes the need to understand how strength of beliefs and belief factors interact to
influence interpretation of information. Furthermore, the way individuals habitually
approach complex and challenging tasks influences how they interpret inconclusive,
mixed evidence; therefore, they must be critical and openly explore new information to
reduce the chance of biased evaluations.
34

Information Processing: Skills-Based Competencies
Personal epistemologies change with age and educational experience (Schommer,
Calvert, Gariglietti, & Basjaj, 1997) and have a subtle but critical role in learning,
relative to expert flexibility within complex and changing contexts (Feltovich, Spiro, &
Coulson, 1989). However, expert performance does not hinge on innate abilities
(Ericsson & Charness, 1994, 1995, 1997). Oftentimes acquired skills and at times
physiological adaptations mediate expert performance. If age, educational experience,
and physiological adaptation influence expert performance and epistemological beliefs, is
this consistent across knowledge domains? Schommer and Walker (1995) investigated
part of this question by measuring personal epistemologies of college students attending
social science and mathematical programs, and Woltz, Bell, Kyllonen, and Gardner
(1996) investigated certain physical issues related to transfer of expert performance to
additional knowledge domains. Schommer and Walker showed that epistemological
beliefs are indeed similar across knowledge domains but are also moderately domain-
independent. Epistemologies are therefore predominantly domain-independent yet
flexible because they are profiles consisting of the five basic factors. As a result, belief
rebalancing occurs in response to familiar through unfamiliar contexts.
Expertise is a prototype (Sternberg, 1997) because individuals are experts in varying
degrees. This implies that people have a shared, albeit, broad view of what constitutes an
expert. Namely, expert behavior is a diverse collection of skills with specialization in
specific knowledge domains and varying degrees of expertise in others (Sternberg &
Frensch, 1992). For example, skills requiring proper sequences of actions or operations
are very important for general and specific expertise (Woltz et al., 1996). This research
measured data-specific sequencing skills that involved specific subjects, physical skills,
35

and mathematical skills. General skills also involved specific subjects, but included
specific application context, syntax, and language expressions to allow variability of
sequences within operations. Evidence from three studies suggests sequence-processing
skills generalized to many applications are more instrumental than specific sequencing
skills and discussed at length in Ericsson & Smith (1991).
Parallel to the naïve/mature continuum of epistemological factors is the
novice/expert continuum proposed by information-processing approaches. Ericsson and
Charness (1997) presented an argument that individuals improve their performance not
just through increased experience with activities, but by structured learning and effortful
adaptation. Historically, the study of expert performance was considered outside the
scope of general psychology because it was attributed to innate characteristics of
outstanding individuals. In direct contrast, expert performance can reflect extreme
adaptations to demands in well-defined domains. In this way, expert performance is
sociobiologically influenced (Gardner, 1983) and achieved through effortful problem
solving (Charness et al., 2001). In light of these studies, the transition from novice to
expert is influenced by differences in factors such as preferred level of activity,
temperament, and genetic components. In summary, expert performance contains
attributes of exposure to domain knowledge, differences in genetic, emotional, and
cognitive predispositions, learning experiences, and effortful adaptation to problem
solving experiences (Holyoake, 1991).
Learning influences transitions from naïve to mature epistemologies and from novice
to expert behaviors. Current discussions of novice to expert transitions include
knowledge acquisition that utilizes environmental opportunities (Hall, Chiarello, &
Edmondson, 1996; Stanovich & Cunningham, 1993), interactive knowledge
36

constructivism (Anderson, Reder, & Simon, 1998), relationships between
epistemological beliefs including conceptual changes (Qian & Alvermann, 1995, 2000),
and the development of domain knowledge expertise (Gardner, 1995). A common theme
in these studies is discussions acknowledging relationships between progressive learning,
knowledge, and expertise development. Practical classroom applications of progressive
learning, knowledge, and expertise development are found in Jonassen, Peck, and Wilson
(1999). They propose that students cannot learn solely from teachers or technologies.
Rather, students learn from thinking about what they are doing, what they believe, and
what they did. Moreover, students learn from what others accomplish and from the
beliefs of others. This is a constructivist approach because people interpret what they
experience in light of what they already know by reflecting on past and current
experiences through anchored instruction and problem-based learning (Norton & Wiburg,
1998, pp. 103 - 116). Briefly, constructivist’s view learning as knowledge constructed,
not transmitted.
Constructivist theory is compatible with sociobiological and information-processing
learning theories because they jointly acknowledge the existence of cognitive and skill-
based learning. Schommer (1990) typifies the cognitive approach to learning, while
Charness et al. (2001), Ericsson and Charness (1997), and Holyoake (1991) typify
information-processing views of learning, and Jonassen, Peck, and Wilson (1999)
exemplify constructivist views of learning. As a group, these researchers collectively
address cognitive and computer skill-based learning. For example, Jonassen, Peck, and
Wilson focused on learning with technologies that require engage thinking to facilitate
knowledge construction for representing ideas and beliefs and for producing organized
multimedia knowledge. Computers help explore information with unprecedented user
37

access to and evaluation of worldviews and perspectives. Educational technologies
provide experiential learning within an expanded public space that features exchange of
ideas, collaboration, consensus, and discourse within knowledge-building communities.
Lastly, electronic technologies are intellectual facilitators that encourage reflective
learning.
Computer technologies are integral parts of learning because they influence
knowledge acquisition and provide exposure to alternative views. These activities
involve sociobiological influences because computer learning is socially interactive,
introspective, and influenced by progressive levels of physical skill attainment and
biological maturity (O’Neill, 2000). Domain knowledge competence is therefore a
progressive refinement of information and concepts that allow experts to see patterns,
relationships, or discrepancies (Bransford, Brown, & Cocking, 2000. pp. 16-18).
Novices bring informal knowledge with them into classrooms, and education has the task
of moving students towards a more formal understanding by deepening and developing
conceptual frameworks.
Conceptual frameworks allow experts to utilize knowledge while addressing novel
experiences (Bransford et al., 2000). Expert conceptual frameworks are organized
around principles that support understanding. However, novice conceptual frameworks
can be built on informal and formal ideas that may yield inaccurate or incomplete
organization of principles that support constructed understanding. It is therefore
important to appreciate the extent of the formal and informal education that an individual
brings into learning experiences. In this way, educators can teach learning strategies,
broaden viewpoints, note comprehension failures, and guide development of accurate
conceptual frameworks.
38

Using computer technologies as learning and teaching tools influences the
development of conceptual frameworks. This occurs through knowledge construction
and domain knowledge competence that develops as a progressive refinement of
information and concepts to see patterns, relationships, or discrepancies (Bransford et al.,
2000, pp. 17-33). Expert knowledge is comprised of several key principles, and those
selected for this discussion were the following: (a) Experts notice features and
meaningful patterns of information; novices do not; (b) experts have organized content
knowledge reflecting deep subject understanding; (c) expert knowledge reflects
application contexts and conditions; and (d) experts flexibly retrieve important aspects of
knowledge. Therefore, knowledge domain expertise helps people develop awareness and
sensitivity to meaningful information as patterns unnoticed by novices.
Development of Research Questions
The purpose of this study was to measure extent of relationship between personal
epistemologies, learning, and experience, using electronic products. This study also
measured extent of relationship between paper and computer-based testing. Measuring
these relationships can guide development of instructions for electronic products and
educational materials. The foundation of these measures is the expected relationships
between naïve through mature epistemologies that correspond with novice through expert
electronic product experience. The research questions are the following:
1. What is the extent of relationship between computer and paper-based
questionnaires?
2. What is the extent of relationship between the dependent variables of self-
taught technology education, certificate-apprenticeship technology education, experience
39

using electronic products, epistemology questionnaire responses and the independent
variables of participant age, gender, educational group (undergraduate, graduate or
faculty), and university (Alliant International University or National University).
40

Chapter 3
RESEARCH METHOD AND PROCEDURES
This study measured extent of relationship between education, experience using
electronic products, and personal epistemologies using identical paper-based and
computer-based questionnaires. Thirty-six respondents from Alliant International
University (AIU) and National University (NU) participated in this study. Institutional
Review Boards of both universities approved this research. Data collection was
conducted with assistance through the graduate school of Education at Alliant
International University (see Appendix D) and the Office of Educational Effectiveness
and Assessment at National University (see Appendix E). Half of each subject group
completed printed questionnaires and half completed computer-based questionnaires.
Students from Alliant International University (AIU) attended programs at the Scripps
Ranch Campus in San Diego, California, and participants from National University (NU)
attended classes at the Technology Center in San Diego, California. AIU participants
attended Liberal Arts, and Online Education classes. NU students attended classes
emphasizing Information Technology because all students use a classroom computer.
The choice to measure students attending traditional Liberal Arts as opposed to
Information Technology classes was due to an expected relationship between experience
using electronic products and personal epistemologies.
Table 3 presents the a-priori statistical power of the test to estimate Alpha, Power
(1-beta) and effect size (f2) using the software G-Power 2.1.2 (1997) before data
41

collection. Power estimates are restricted to F-tests using multiple Correlation and
regression (Keppel & Zedeck, 1989 pp. 106 - 109). The test group is restricted to 36
participants chosen according to four criteria (age, gender, group, and source) and one
treatment level assignment (paper vs. Internet) for a total of five predictors. As a result,
statistical significance is set at a typical level (p < .05), but higher levels of significance
(p <.01) are desired.
Table 3
G -power tests: Post-hoc test, alpha, and power determination
F-test (MCR) Global: Sample Size = 36
Alpha = 0.050 Predictors = 5
Power (1-beta) = 0.9954 Critical Value = F(5,30) = 2.5336
Effect Size "f2" = 1.000 Lambda = 36.00
Accuracy and Effect Size Conventions
Small
f2 = .02
Medium
f2 = .15
Large
f2 = .35
Paper-based and computer-based surveys were distributed in-class. The
questionnaires consist of six sections presented as a single form to facilitate ease-of-use
and data collection. Instructors collected paper-based surveys and returned them
personally or by mail. Respondents emailed Internet surveys, and in the event of
software challenges, surveys were printed, then returned to instructors, returned by mail,
faxed, or saved as html documents and sent by email. Table 4 outlines the dependent and
independent variables used in this study. These variables provide the organizing
structure for data analysis using SPSS versions 12 and 13. This study is a repeated
42

measures design with two treatment levels, paper, and Internet questionnaire
administrations.
Table 4
Independent variable definitions: P-I, AGE, GENDER, GROUP, and SOURCE
Independent Variables (Numerical database value in parentheses)
P-I Paper (10) Internet (11)
Age 18-25 (1) 26–35 (2) 36–45 (3) 46–55 (4) 55+ (5)
Gender Female (1) Male (2)
Group Undergraduate (1) Graduate (2) Faculty (3)
Source National University (1) Alliant International University (2)
Table 5 presents the dependent variables to define technology education, formal
education, experiences using electronic products. The self-taught and certificate-
apprenticeship technology education sections present parallel questions to probe whether
differences exist between experience using electronic products, epistemology, and type of
technology education.
43

Table 5
Dependent variable definitions: STE, CTE, and ED
(STE) Self-Taught Technology Education and
(CTE) Certificate and Apprenticeship Technology Education
STE1 and CTE1 Mechanical Equipment Operation
STE2 and CTE2 Electro - Mechanical Equipment Operation
STE3 and CTE3 Computer - Information technology
STE4 and CTE4 Office Productivity Software
STE5 and CTE5 Programming
STE6 and CTE6 Other Technology Education
(ED)Earned Degrees and Certificates (Numerical database value in parentheses)
Certificates (C) None (1) Engineering (6)
Associates (A) Biological Sciences (2) Liberal & Fine Arts (7)
Bachelors (B) Business (3) Physical Sciences (8)
Masters (M) Computer Science (4) Social Sciences (9)
Doctorate (D) Education (5)
Table 6 presents the dependent variable organization for epistemology factors and
factor subset variables. Each epistemological factor has subset factors and these subset
factors have differing numbers of questions. Factor and subset factor abbreviations are
shown in parentheses. The table presents epistemological factors, the number of subset
questions and then subset factors.
44

Table 6
Dependent variable definitions: Epistemology
Epistemology Factors
No. of
Questions
Epistemology Subset Factors
5 (AA) Avoid AmbiguityCertain Knowledge
(CK) 6 (KC) Knowledge is Certain
4 (AL) Ability to Learn is Innate
5 (CLL) Can’t Learn How to Learn
Innate Ability
(IA)
4 (STW) Success is Unrelated to Hard Work
7 (DCA) Don’t Criticize AuthorityOmniscient Authority
(OA) 3 (DOA) Depend on Authority
5 (LQ) Learning is Quick
3 (LFT) Learn the First Time
Quick Learning
(QL)
2 (CWT) Concentrated Effort is a Waste of Time
11 (SSA) Seek Single) AnswersSimple Knowledge
(SK) 8 (AI) Avoid Integration
Subjects
A single 36 respondent test population from Alliant International University (AIU)
and National University (NU) created two groups of 18 respondents. Table 7 shows the
database organization and combined allocation of participants according to questionnaire
type and group membership.
45

Table 7
Alliant International University and National University subject database matrix
Questionnaire Group F M Group F M Group F M Total
Internet U-Grad 3 3 Grad 3 3 Faculty 3 3 18
Paper U-Grad 3 3 Grad 3 3 Faculty 3 3 18
Totals 6 6 6 6 6 6 36
Instrumentation
The questionnaire has six separate sections based on Tables 4, 5 and 6. Sections
contained between 4 and 20 questions, so respondents with smaller computer screens
could see all questions with response-box descriptors together. As a result, the Internet
version was as identical as possible to the print-based version because of common
formatting requirements. The first section requested age, gender, academic group, and
university affiliation. No personal information was required. All paper and email
responses were anonymous, but instructors received a summary of the completed study
for distribution to interested students.
The second section was self-taught technology education (STE), and the third
section was certification-apprenticeship technology education (CTE). These sections
asked parallel questions that progressed from mechanical equipment operation, electro-
mechanical equipment operation, computer and information technology, office
productivity software, programming, and “other” technology skills. The intent was to
measure each person’s experience as a range of physical (non-computer) through
cognitive (computer-based) activities.
46

The fourth section was earned degrees and certificates. Categories were certificates,
associates, bachelors, masters, and doctorate. Curricula included biological sciences,
business, computer science, education, engineering, liberal and fine arts, physical
sciences, social sciences, certificate, associates, and bachelors. Degrees received
multiple responses, so the first bachelor’s degree is B1 and a second bachelor’s degree is
B2.
The fifth section was electronic product experience (EEP). The format was similar
to self-taught technology education and certification-apprenticeship technology education
sections because the intent was to capture a range of primarily physical to primarily
cognitive skills. This section was slightly different from the STE and CTE sections.
Instead of responding with years’-experience, category responses were novice,
intermediate, advanced, expert. The intent was to capture internalized views of personal
experiences using electronic products without reference to external standards.
The sixth and final section was the Schommer-Akins Epistemology Questionnaire,
second revised edition for college students (personal communication with author April
17, 2004). The instructions to complete the questionnaire are “There are no right or
wrong answers for the following questions. We want to know what you really believe.
For each statement fill in the circle on the answer sheet for the degree to which you agree
or disagree”. Directions for both Internet and computer-based questionnaires were
similar to original directions but included the following statement: “Please keep in mind
your prior experiences using electronic products while answering questions” (see Table
1). This statement is in direct response to the Schommer and Walker (1995) investigation
to determine whether epistemological beliefs were similar across knowledge domains.
This instruction was on the questionnaire, subject consent, and subject instructions.
47

MEASURING LEARNING BELIEFS Publication -2005

MEASURING LEARNING BELIEFS Publication -2005

Recomendados

Recomendados

Mais conteúdo relacionado

Semelhante a MEASURING LEARNING BELIEFS Publication -2005

Semelhante a MEASURING LEARNING BELIEFS Publication -2005 (20)

MEASURING LEARNING BELIEFS Publication -2005