WCET Annual Meeting Presentation

1. AN ASSESSMENT-BASED
DEGREE
November 1, 2012
WCET Annual Conference

2. The Iron Triangle
2
Cost
Quality Access

3. Scalability of Content
3
 Open Educational Resources: accessible
to millions
 Independent Study
 Requires that learners be self-motivated

4. Types of OER
 Lecture notes Physics lecture
 Videos of classroom lectures
 Lessons designed for OER
 Courses
OpenStud
y  Discussion groups
Khan Academy
Open
University

5. What is OER’s value?
 Learning
 Certificates
But…
 OER by itself does not typically
award formal educational credit
Why not?

6. Academic credit
6
 What is credit?
 Assurance that someone knows something
 The something must be appropriate for the
particular academic program
 To provide that assurance, both the
someone and the something must be
verified

7. Decoupling assessment from the
7
classroom
“What you know is more important than where or
how you learned it.”
Credit should be based on knowledge, not
attendance
Knowledge gained through independent study
can be independently assessed

8. Aspects of validity for credit by
assessment
8
 Identity verification
 Assessment quality
 Appropriateness of knowledge
tested for a particular degree
program
 Scalability

9. Identity verification
9
Ryan Ruppe
Jeffery Turner
Steve Winton

10. Threats to validity--courses
10
 Did the person actually go through the course?
 Did the person do his/her own work?
 Is the person who took the course the same
person who is presenting the credential?

11. Threats to validity--
11
assessments
 Is the person taking the assessment the same
person who is claiming the knowledge?
 Is the person claiming the knowledge the same
person who is presenting the credential?

12. Assessment quality
12
A good assessment
 Measures knowledge of the subject
 Does not measure irrelevant characteristics
 Gives a person the same score regardless of
which form is taken
 Gives people of the same ability the same score

13. Threats to assessment quality
13
 Assessments may not measure quite the same
content as the material studied
 Not
all things in the world labeled “Sociology”
cover the same topics!
 Assessments may not cover material in
enough depth
A 10-question quiz is unlikely to cover the
equivalent of a college semester’s worth of
content

14. Threats to assessment quality
14
 The assessment might not be scored
consistently enough
 Different instructors have different standards
 Assessments might measure irrelevant
characteristics
 Unnecessarily complicated questions
 Questions with “giveaway” answers
 Trick questions

15. Appropriateness of content
15
 Match of assessment to OER
 How close are the assessment specifications to
the learning objectives of the OER material?
 Match of assessment to credit-granting body
 How close are the assessment
specifications/learning objectives to what is taught
at the institution where credit is sought?

16. Efficient assessment
 Machine-scored competency/proficiency
exams can handle large numbers of
examinees and be used for multiple OER
sources
 Machine-scored exams need not measure
simply mindless regurgitation of facts
 For some types of OER, more specific
assessments may be needed
 Portfolios for highly specialized content
Cost of a 3-credit multiple-choice
assessment:

17. Scalability - Generalizability
Less
generalizable
 Course final exams and homework
 Third-party assessments designed
for a specific course
 Competency/proficiency
assessments
More
generalizable

18. Scalability – Large groups
Less scalable
 Individual assessments
 Portfolios
 Research papers
 Oral examinations
 Human-scored group assessments
 Short answer questions
 Essay questions
 Machine-scored assessments
 Multiple-choice exams
 Machine-scored constructed response
More scalable exams

19. A model for low-cost
education
OER University

20. What is Charter Oak State College?
 Founded in 1973 in Connecticut
 Enrollment about 2,000
 Over 11,000 degrees awarded, with alumni from
all 50 states and a dozen countries
 Partnerships with military branches
 Associate and Baccalaureate degrees
 New England Association of Schools and Colleges
accreditation

21. What is Charter Oak State College?
 Aggregation model - with Cornerstone and
Capstone
 Outcome assessment courses
 Writing, research, program planning
 Transfer credit
 CBE, portfolio assessment, ACE and National
CCRS, CCAP, institutionally reviewed training
and certifications

22. Student Profile
CONCENTRATION PLAN OF STUDY (CPS) FORM
Business Administration
MINIMUM Number and Level of
MINIMUM SUBJECT
REQUIRED SOURCE: TEST OR INSTITUTION COURSE/TEST NO. & TITLE GRADE Credits Propose
REQUIREMENTS
CREDITS Lower Upper
Accounting: 3 credits CLEP Financial Accounting Prop (3)
Financial *
Managerial * 3 credits StraighterLine Managerial Acct. B 3
Finance: 3 credits DSST Prin. Finance Prop (3)
Principles * or
Managerial * or
Corporate
Management: 3 credits Excelsior Coll. Organizational Behav Prop 3
Organizational
Behavior
Management: 3 credits Northwood U. MGT 230 Prin. Mgt. A 2.7
Intro/Principles
Management*
Marketing: 3 credits Northwood U. MKT 208 Prin. Mark. C+ 2.7
Principles of
Marketing*
Strategic Processes 3 credits COSC MGT 499 Strategic Prop 3
Capstone Course Management

23. Program Planning
Hi Karen,
As per your request, I've included a list below of tests/ independent
studies courses I am planning on taking for my degree.
• Pending exams/courses:
• Cornerstone course (IDS 101)- in progress
• College Algebra - Straighterline, in progress
• English Composition II - Straighterline, in progress
• Mgmt Info Systems - DSST (area of concentration)
• Environment & Humanity – DSST
• *Business Ethics in Society - DSST (UL area of concentration)
• Organizational Behavior - Excelsior Exam(UL area of concentration)
• Principles of Public Speaking - DSST

24. Class of 2011-12 & CBE
2011-12 Charter Oak State
College Graduates:
Average of Test Credit
Graduates Average of Test Count Hours
No Testing 310 64%
Took 1 or more tests 176 5.23 17.64 36%
Grand Total 486 5.23 17.64

25. What Is Excelsior College?
 Independent institution serving adult students and other
non-traditional learners since 1971
 Enrollment 34,000; alumni 141,000
 Up to one-third of EC students are
 Members of historically under-represented groups
 Military or veterans
 Average age of graduates: late 30s
 Associate, Baccalaureate, and Masters degrees
 Middle States accreditation

26. What Is Excelsior College?
 Aggregation model
 EC online courses (available since 2004)
 Two national exam series developed in house: ECE and UExcel
 Transfer credit
 Consistently ranked #1 or #2 in acceptance of credit from
other accredited institutions
 Other CBE, portfolio assessment, ACE and National
CCRS, institutionally reviewed training and certifications
 Largest pre-licensure nursing program in the
country, entirely assessment based

27. Excelsior College Exam-Based Degree Paths
 New degree pathways emphasize Excelsior’s own CBE
products, along with a single required Capstone course
 Degree planning templates show suggested sequence
of exams; students with transfer credit have wider
choices
 Supported Independent Study model is based on Open
Courseware options reviewed and recommended for all
exams
 Initial degree templates for BS and BA in Liberal Studies
and AS and BS in General Business

28. Sample ASB Degree Template
 (ETC ASB example)

29. Infrastructure Needs
29
 Common definitions of assessment-based
learning among the Regional Accreditors
 Access to Federal Financial Aid
 Proposal:a demonstration program that lets a set
of low cost assessment approaches prove their
merits.
 National Database of existing credential and
training assessments

30. A New Narrative
30
 We need to add a chapter to the higher
education biography.
 The story begins with Socratic learning, moves
to libraries, adds faculty, morphs
universities, carves land grants from the
wilderness, upscales adult education into
Community Colleges, and invests in for-profits.
 Now the story must add competency-based
credentialing