Marking Modes in Moodle - configure for BTec/NCFE/APT for free! Also, Moodle dartMOOR attendance PlugOut.

Marking Modes
in Moodle
Configuring Gradebook for Awarding Body Best Practice
© 2014 // dBsMUSIC // Phineas de Thornley Head

// Abstract
● BTec / NCFE / APT best practice requires…
○ Written feedback specific to every learning outcome
○ Specific grading level descriptors for each of these
○ A Pass/Merit/Distinction grade for each Unit
○ A final numerical score for the qualification
○ An IV/Moderation commentary process
○ Grades returned to learners en masse
○ A tutorial system with SMART targets
● ‘Coercive Configuration’ can achieve all that in Moodle…
○ For free!
○ Without coding!
○ Without plugins!
● We also present a 'Plug-Out’: a Gamified Attendance system

// In short…
+ =

// dBs Music: Who we are
● dBs Music, founded 1999 as ‘Deep Blue Sound’
○ Directors, David Louis Puttick and Nigel Burt
● UK Partner College modal
○ Industry experienced tutors
○ Specialist equipment and software
○ Custom teaching spaces and studios
● Currently, 3 UK sites...
○ Plymouth, 1999
○ Cornwall, 2010
○ Bristol, 2012
● ...and 1 International Centre
○ Berlin, 2013 (independent)

// dBs Music: What we do
● Music production, engineering & composition
○ Further Education
￫ NCFE Level 1 Certificate
￫ BTec Level 2 Diploma in Music Technology
￫ BTec Level 3 Extended Diploma in Music Technology
○ Higher Education
￫ FdA in Electronic Music Production & Composition
￫ FdA in Audio Music Production
￫ FdA in Live Sound
￫ BA(Hons) top up
￫ BA(Hons) Fast Track
● Industry-lead learning
○ Tutors are all working musicians, producers, engineers, composers & performers
○ All Programs of Study include industry professional certification
￫ Official Ableton Live Training Centre
￫ Official ProTools Training Centre
￫ Official Apple Logic Pro Certification Centre

// dBs Music: Who we take
● “To create an environment that positively encourages everyone to maximise
their potential”
● Defining “Success”
○ Many of our students go on to successful careers in industry
￫ e.g. one of my ex-students is assistant engineer at Abbey Road and was closely involved in the creation of Underworld’s
score for London 2012 Olympic opening ceremony
○ However, jobs in industry are scarce, so we define “success” individually
￫ For whatever reason, school has failed many of our intake
￫ Success can just as well be the learner gaining the first complete, meaningful qualification of their lives
● Music Technology Sugars the Pill
○ Highly varied educational components clothed in the ‘sexy’ exterior of Rock’n’Roll
￫ English Functional Skills through writing about music
￫ Maths Functional Skills through acoustic and musical calculations
￫ ICT Functional Skills through computer use
￫ Music Appreciation & Composition
￫ Historical and Political Context
￫ Physics, Electronics, Digital Theory & Psychoacoustics
￫ Business, Life Skills and Health & Safety

// Modular Qualifications
● dBsMusic uses 3 main awarding bodies for FE
○ Edexcel BTec
○ NCFE
○ APT Access
● Definitions
○ Qualification: a subset of Units, drawn from the total number available
￫ e.g. BTec Extended Diploma = 18 Units
○ Unit: a topic-specific area of study
￫ e.g. “Recording Skills”
○ Learning Outcome: a specific goal to be evidenced by learners within a Unit
￫ e.g. “Set up a microphone correctly”
￫ in BTec, typically 4 per Unit
○ Grading (or Level) Descriptor: describes work of a given level of achievement in an outcome
￫ e.g. “Can competently set up a microphone without assistance”
￫ in BTec, these describe Refer, Pass, Merit and Distinction work, specific to each outcome

// What is ‘Best Practice’?
● Definitions
○ ‘Limiting Grades’ or ‘Blocking Criteria’
￫ Mandatory requirements of OFSTED, Edexcel etc. at External Inspection
￫ sufficient to prevent the inspection grade being higher if they are not met
○ ‘Best Practice’
￫ Recommendations made by OFSTED, Edexcel etc.
￫ greatly improve the chances of scoring top inspection grades where all Blocking Criteria have been met
● Best Practice
○ Assessment
￫ Specific grading level descriptors for each Learning Outcome should be clear in the feedback
￫ Learners should receive written feedback specific to each Learning Outcome of each Unit
￫ Indications for how to attain a higher grade for re-submission should be provided
￫ Marking should be released en masse, after IV’ing
○ Attendance
￫ Learners should have ownership of their attendance, monitored and targeted in tutorials
○ Tutorials
￫ Tutorials should contain individualised SMART targets, with progress assessed at subsequent tutorials

// Absent Functionality (?)
● We love Moodle!
○ dBs Music one of the first UK institutions to use Moodle, starting in 2004
￫ We are ‘paperless as possible’ and use Moodle for every aspect of our provision that it allows
● But there is always a wish list!
○ Some key components of success are missing from core Moodle
● Our wish list was…
○ Per-outcome Written Feedback’
￫ Outcomes have no feedback boxes
○ Per-outcome Grade Descriptors
￫ Scales are not Outcome-specific
○ Mass-release of IV’ed grading
￫ Nothing before Moodle 2.6 and even 2.6 has no provision for Moderation/IV commentary
○ Pastoral Tutorial and Target System
○ Gamified Attendance & Retention tracking

// Possible Approaches
● Solutions
○ Alter Core Code?
￫ Expensive
￫ Invasive
￫ Puts Moodle out of development path
○ Commission Plug-ins?
￫ Expensive
￫ Less invasive but still...
￫ Increases risk at upgrade and…
￫ Invites potential re-writing (and additional expense) with upgrade incompatibilities
○ Wait for Moodle?
￫ Free...
￫ Non-invasive...
￫ Maintains development path...
￫ ...but we needed things now!
● Use Moodle’s existing functionality to our advantage!

// Coercive Configuration: “Tricking” Moodle
● Coercive Configuration
○ Sound engineering is the art of elegant workarounds
￫ We take that idea and apply it to Moodle
○ Moodle is a highly diverse and rich system
￫ Careful consideration of its operation can suggest solutions
○ Configure added functionality by ‘tricking’ Moodle!
￫ Free!
￫ Non-invasive!
￫ Maintains development path!
● Half-Absent Functionality
○ Apparently absent functionality can be created or closely approximated by careful configuration
● Maintaining Elegance
○ Just because the solutions are ex post facto doesn’t mean they have to be kludges!
￫ We aim for elegant configuration wherever possible

// First Rule of Sound Engineering…

// Analyse the Problem
● What’s there already?
○ Advanced Grading Methods
￫ Rubrics offer individuated feedback boxes
○ Outcomes
￫ Grades display in Gradebook
○ Gradebook
￫ Flexible aggregation, calculation and display of grades
● Limitations?
○ Rubrics
￫ only return a standardised percentage score, not the 'lowest grade' required by BTec
○ Outcomes
￫ graded with drop-down Scales which lack individuated feedback and outcome-specific level descriptors
○ APT Access
￫ splits assessment and grading into distinct sections
￫ grade first against Assessment Criteria (as Pass/Fail)
￫ then, if passed, grade against distinct and independent Learning Outcomes (as Pass/Merit/Distinction)

// Outline the Solution
●
● Potential Existing Functionality adaptable to our needs
○ Use Outcomes to present grades in Gradebook
￫ Required upload and management of the 600 outcomes we use
○ Combine with Rubrics ‘tricked’ to return lowest grade
￫ Rubrics perfect for displaying per-outcome feedback
￫ Require mathsey cunning to return lowest grade
○ Use Letters to turn Rubric grade to “Pass/Merit/Distinction”
￫ Correctly set to ‘interpret’ the percentage output by the tricked Rubric
○ Use custom calculations to aggregate course totals
￫ Correctly scale these to numerical Unit scores for course total
Pass = 70, Merit = 80, Distinction = 90
○ Use Tracking to track grading turnarounds
￫ For Course Leader audits
○ Use ‘dummy’ assignments to control IV Process
￫ Split submission, feedback and moderation/IV into three assignments and control with ‘hiding’

// Moodle Definitions
● Speaking Moodlese is important!
○ Categorical Grade / Scale
￫ Grades awarded as one of a set of options in a non-continuous Scale, e.g. “Not Yet Achieved, Pass, Merit, Distinction”.
Moodle labels this internally 1-4 but Pass (2) is not “half a Distinction (4)” and no ‘2.5’ exists between Pass and Merit.
○ Numerical Grade / Rubric
￫ Grades awarded on a continuous numerical scale (e.g. a percentage). Rubrics output a numerical grade.
○ Letter
￫ A label applied to a numerical grade. e.g. a tutor awards a submission 75% and Moodle then displays 75% (B+).
○ Aggregation
￫ The method where a number of grades (both categorical and numerical, interchangeably) are combined to a single
representative grade, e.g. mean, median, lowest grade etc.
￫ Aggregation can convert Moodle’s internal representations of categorical grades into numerical grades.
￫ The mean of a Pass and a Merit could be set to display as “2.5” and used in later calculations as a numerical value.

● Moodle ‘Course’ = BTec ‘Qualification’

● Moodle ‘Topic’ (within a course) = BTec Unit

// Per-Outcome Feedback: Method Overview
● Recommendations
○ Setup requires considerable attention to detail
○ However, it is achievable by a Moodle Admin alone, with no coding
○ To make the solution scalable, involve course leaders as early as possible
● Assumptions
○ Moodle is 2.5 or later
○ Assignments address units on a 1:1 basis
￫ Where an assignment addresses > 1 unit, separate feedback areas for each should be created
● A Two-Part Process
○ Part I (for Moodle Admin) deals with global site set-up
￫ It must be completed and checked once, before the course starts
￫ Should only be attempted by someone with experience manipulating text-based data (e.g. Excel, csv)
￫ Requires a good eye for standardisation
○ Part II (for Course Leaders) deals with course-specific setup
￫ It does not require csv experience or Moodle Admin permissions

// Part I. 1. Add Scales
Go to [Administration > Site Administration > Grades > Scales] and Add a New Scale.
Enter a Standard Scale with the Name "BTec Standard", and the Scale as “Not Yet Achieved, Pass, Merit, Distinction”

// Part I. 2. Add Letters
Go to [Administration > Site Administration > Grades > Letters] and edit the default, setting all grades to 'Unused' except the top four,
A, A-, B+, and B. These can be made course specific but setting at site level means you don’t need to repeat this, so make this default
conform to the Awarding Body your college uses most, which for us is BTec. Set exactly as follows1:
'100%, Distinction' (replaces '93%, A')
'99%, Merit' (replaces '90%, A-')
'90%, Pass' (replaces '87%, B+')
'0%, Not Yet Achieved' (replaces '83%, B')

// Part I. 3: Site Completion Tracking
Go to [Administration > Site Administration > Advanced Features] and ensure ‘Enable Completion Tracking’ is checked

// Part I: 4. Course Completion Tracking
Go to [Administration > Site Administration > Courses > Add/Edit Courses] and for each course, edit settings (click the cog wheel) to enable
Completion Tracking

// Part I: 5. Install Progress Bar
Install the Progress Bar block (your Moodle provider may have to do this).

// Part I: 6. Enable Outcomes
Go to [Administration > Site Administration > Advanced Features] again and ensure that 'Enable Outcomes' setting is checked.

// Part I: 7. Add Site Outcomes
To add small numbers of outcomes, go to [Administration > Site Administration > Grades > Outcomes] and click 'Add a New
Outcome' (scroll to the bottom), entering details and ensuring that these are set as a Standard Outcome, and using the BTec Standard
scale created in 1).
For BTec, as for most Awarding Bodies, a Moodle outcome maps directly to a single learning outcome within a Unit, not the entire Unit.
However, occasionally, awarding bodies apply two or more grading criteria to the same outcome. In these cases, create dedicated
Moodle outcomes for each grading criterion (e.g. label L1a, L1b etc.)

Larger numbers of Outcomes should be uploaded in bulk. It is strongly recommended that a standardised naming convention be adopted.
Full name: Body LevX CourseName UnitY: LZ - "Description"
Short name: BODY_LEVEL_COURSE_UNIT_OUTCOME
BTec Lev3 MusTech Unit1: L1 - "Be able to use music technology equipment"
BTEC_LEV3_MT_U1_L1
Following careful IVing of the final list with all course leaders, create a csv file (strictly ssv) EXACTLY in the following format.
outcome_name;outcome_shortname;outcome_description;scale_name;scale_items;scale_description
"BTec Unit 1 L1 Be able to...";BTEC_U1_L1;;"BTec Standard";"Not Yet Achieved,Pass,Merit,Distinction";
"BTec Unit 1 L2 Know that...";BTEC_U1_L2;;"BTec Standard";"Not Yet Achieved,Pass,Merit,Distinction";
Note double-quotes, semicolons as field delimiters, double semicolons denoting (optional) empty description fields and reference to the
relevant scale (see 1). Note also that Moodle’s help files are incorrect: the format suggested at Moodle.org
(http://docs.moodle.org/22/en/Outcomes#Importing_outcomes) fails because it misses the quotes on the outcome_name.

Within (any) course, go to [Administration > Course Administration > Outcomes] and select 'Upload Outcomes' from top drop down
(an oversight in Moodle means this option is not available from Site Administration).

// Part I: 8. Import Course Outcomes
Within each course, go to [Administration > Course Administration > Outcomes] and add the Available Standard Outcomes (in the
right column) to the Outcomes User in Course (left column). Add all outcomes required in the course, but only the outcomes required in
the course. This last step may be optionally be completed by Course Leaders.

// Part I: Summary
1. Add Scale(s) to system
2. Edit Letter(s) at system or in course(s)
3. Enable Completion Tracking on system
4. Enable Completion Tracking in course(s)
5. Install Progress Bar Plugin
6. Enable Outcomes on system
7. Upload IV’d Outcomes csv to system
8. Import specific Outcomes to course(s)
Now over to course leaders!

// Part II: 9. Create Gradebook Folders
Go to [Administration > Course Administration > Grades] and then to…
[Administration > Grade Administration > Categories & Items > Simple View]

// Part II: 10. Create Aggregation Categories
Still in [Grade Administration > Categories & Items > Simple View]
we are now going to add sub-categories for each unit of the course.
(This can be done ad hoc too, but makes sense to create them all up
front.)
Note the ‘Lowest Grade’ aggregation needed for BTec.
Because Moodle knows the maximum and minimum numerical Scale
grades (here, 4 and 1 respectively), these are set automatically when
you select the Scale.
You do need to specify the number of the Pass grade though.
Recall from 1) that the BTec Scale is Not Yet Achieved, Pass,
Merit, Distinction. Moodle’s internal representation are the integers 1–4,
‘2’ standing for ‘Pass’, hence the Grade to Pass would here be 2.
ID Numbers are needed in step 15, below.

// Part II: 10. Create Aggregation Categories
Repeating for each Unit of the course, you should now see something like this:

// Part II: 11. Create Assignment
Return to the course, turn editing on and go to [Add a new activity > Add Assignment]
to create an assignment in the appropriate topic area.
Feedback Types must enable comments, and can
enable files, allowing annotated (e.g. spell-corrected)
transcripts to be uploaded.
Grading must be set as follows, with Grade ‘100’ and Method ‘Rubric
Grade Category should be the Unit the assignment addresses and
just places it in the correct folder created in 10, above.
Outcomes will have been enabled, IV’d and loaded into Moodle by your Admin,
plus imported into the course either by them or you (see 8). Scroll to ones
required and check all, but only, those required. Naming conventions adopted
in 7 will ensure these are grouped together but check!
Activity Completion should be set based on marking (from the tutor), not
submission (from the learner) because submission is already recorded
by the assignment activity type. Thus, we suggest tracking be set to
‘Show activity as complete when conditions are met’ with ‘Require
Grade’ checked, and setting the expected date to 2 weeks after
the deadline for submission.

// Part II: 12. Create Rubric
The page should forward to Advanced Grading Rubric Creation, but if not, go to
[Administration > Assignment Administration > Advanced Grading > Define Rubric]
A blank rubric is a grid of ‘criteria’ (= outcomes in rows) and ‘levels’ (columns)
You will need to ‘Add Level’ if the rubric is blank to make 5 columns. Set the levels of the first row EXACTLY as follows
Not Yet Achieved 0
Pass 100
Merit 110
Distinction 111

// Part II: 12. Create Rubric (the mathsey bit)
Rubrics calculate a continuous aggregate total score but BTec units require a categorical lowest score. These numerical scores mimic this behaviour.
This is a geometric recurrence relation based on Zeno’s 1st paradox, such that the score, S, of the ith passing outcome of n total outcomes is in the form
and the failing score is always = 0 (i.e. the limits on the sum progress from 1, not 0). That is, for a standard 4 outcome unit,
Total outcomes = n = 4
SNYA = S0 = 0
SPAS = S1 = 0 + 4-0 = 0 + 1/1 = 0 + 1 = 1
SMRT = S2 = 1 + 4-1 = 1+ ¼ = 1.25
SDST = S3 = 1.25 + 4-2 = 1.25 + 1/16 = 1.3125
This procedure requires Moodle 2.5.2 so Rubrics return values to 2 decimal places. Pre-2.5.2 releases only returned integers, breaking this functionality
but following my application for this to be permanently implemented in the standard build this was achieved in Moodle Tracker Ticket 33563
https://tracker.moodle.org/browse/MDL-33563
While most BTec Units have 4 outcomes (making n = 4 as in this example), we need to use n>=7 throughout, since the FMP is a 7 outcome unit. That
is, having n=7 works also with 4 outcome units, but having n=4 will not work with 7 outcome units, since NDDDDDD returns a score higher than
PPPPPPP.
Using ‘decades’ (i.e., 0,100,110,111) provides the most mathematically elegant solution since it returns integer values for raw scores and could
accommodate units with up to 10 learning outcomes. NPMD percentages are then 0%, 90.09%, 99.1%, 100%

Name each row of your Rubric (far left column) identically to the outcomes for your unit, e.g.

Repeat for all outcomes of the Unit to give the following. Note also that BTec like bit of feedback to specify the outcome number next to
each level descriptor (i.e. P1, M1, D1, then P2, M2, D2 etc.).

Copy/paste the level descriptors from the Unit document for each level (N/P/M/D) of each outcome (typically 1 through 4). This way, both
tutor and learner know exactly how their work relates to the levels described by the awarding body (pursuant to their codes of Best
Practice), and also neatly avoids tutors having to copy/paste these each time they grade.

With the rubric complete, click 'Save and make it ready'.

Checking this by clicking on the assignment should now show this:
Note that since the rubric and outcome marks cannot intercommunicate,
these must be carefully set by marking tutors to be identical, as here
This is the only slight drawback of the system,
but is unavoidable without changes to core code.

// Part II: 13. Organise Gradebook
Back in [Administration > Grade Administration > Categories and Items > Simple View] move the Rubric-based numerical grade
from its category in Outcomes, to Units.
There are now two totals for each project, one generated from the Outcomes (using the scales from 1, above, totaled within each
category using the 'lowest grade' criterion, from 10), and the other calculated by the rubric for each unit (using the letters from 2 and
maths from 12).

// Part II: 13. Organise Gradebook
Step 10 above ensured that Real grade (= scale) was displayed for individual Outcomes and Unit Category totals, but the Letter grade
must display for the project totals (from the rubrics).
We need to convert the numerical output of the rubric back into the corresponding
‘scale word’ so it looks the same as the outcome (scale based) grades.
To do this, for each rubric-generated project-total (with the cog-wheel icon) go to
[Administration > Grade Administration > Categories & Items > Simple View]
and click the edit icon (cog-wheel).
Expanding Grade Item to Show More, ensure that:
Grade to pass = 90
Grade Display Type = Letter
Multiplicator = 1
Offset = 0
(Recall the letters from Step 2:)
'100%, Distinction'
'99%, Merit'
'90%, Pass'
'0%, Not Yet Achieved'

// Part II: 13. Organise Gradebook (maths check)
Example:
Awarded in Rubric Rubric Marks Rubric Total Percentage Final Letter
P,P,P,P 100,100,100,100 400/444 90.09% Pass
M,M,M,M 110,110,110,110 440/444 99.09% Merit
D,D,D,D 111,111,111,111 444/444 100.00% Distinction
D,D,D,NYA 111,111,111,0 333/444 75.00% Not Yet Achieved
D,D,D,P 111,111,111,100 433/444 97.52% Pass
D,D,D,M 111,111,111,110 443/444 99.77% Merit
M,M,M,P 110,110,110,100 430/444 96.84% Pass
M,M,M,D 110,110,110,111 441/444 99.32% Merit
P,M,M,D 100,110,110,111 431/444 97.07% Pass

// Part II: 14. Configure Course Total
Rubrics are easy to grade and allow individuated feedback per outcome (which outcomes do not permit).
However, the gradebook does not display the marks breakdown of the individual learning outcomes within the rubric
It only shows the overall continuous percentage score, now converted to a letter (see 13).
Hence, categorical, scale-based outcome scores (which do show their individual outcome scores) are also required.
This is why we need gradebook to display two sets of totals, one from the outcomes and one from the rubrics.
That means we can't have both a correct raw sum total and percentage: one will be wrong.
However, this does not matter because we think of BTec in terms of raw scores, not percent.
Therefore, finally, we must calculate the raw sum course total.
(All rubric scores are percentages out of 100, and all outcome scale scores range 1-4, thus, for a BTec qualification of n units, the overall total is n * (100 + 4) = 104n
Grade Type Show Outcome Grades Breakdown Show Outcome Feedback
Rubric No Yes
Outcome Scale Yes No

We now encounter the reverse problem to step 12, where these categorical data must now be scaled back into continuous data.
BTec stipulate the following numerical scores are gained per unit
Not Yet Achieved: 0
Pass: 70
Merit: 80
Distinction: 90
First some checks;
In [Administration > Course Administration > Grades > Grade Item Settings] ensure
Grade Display Type: Real
Overall Decimal Points: 0
In [Administration > Grade Administration > Categories and Items > Full View] ensure that for the course (= qualification)
Aggregation method = 'Sum of Grades'
Aggregate only non-empty grades: unchecked
Include outcomes in aggregation: unchecked
Aggregate including subcategories: unchecked
Drop the lowest: none

// Part II: 14. Configure Course Total (the mathsey bit)
The following equation transforms Moodle’s internal number labels for NYA, P, M, D to 0, 70, 80, 90
...rounding the result to the nearest 10, where x is the categorical grade 1, 2, 3 or 4
Confusingly, Moodle displays these as % scores of the highest grade, so NYA=0%, P=33.33%, M=66.66%, D=99.99% but is still thinking in terms of 1-4
We therefore now need to convert our equation to Moodle syntax, where variables are encapsulated within double square brackets e.g. from 14 above, [[U1]]
would be Unit 1's categorical total.
However, while Moodle does permit some basic spreadsheet functionality it doesn't permit if/then statements (or this would be a *lot* easier!) AND its log(x)
function is broken, returning ln(x)
Therefore, to convert this to the scaled continuous result in Moodle, we require
=round(80*(ln([[U1]])^0.25),-1)
...where round(x,-1) is the 'round x to the nearest 10' function.
(Note the second argument of round(), after the comma: 2=2dp, 1=1dp, 0=nearest integer, -1=nearest 10).

In [Administration > Grade Administration > Categories and Items > Simple View] click the calculator next to Course Total
Check ID codes for each scale-based categorical Unit Total from step 10 above (e.g. U1, U2).
Our custom equation requires the ID numbers for the totals only, no need for individual learning outcomes, or Rubric totals.
Replace each occurrence of [[U1]], [[U2]] etc. with the actual unit ID
codes (from step 10c) encapsulated in double square brackets
so [[U1]] would be Unit 1's scale-based categorical total
Note that Moodle will not accept the formula with line breaks
With that, all totals will be correct, and provide a qualification total WHERE THE UNIT HAS A GRADE.
This is the only limitation with this system: if any units remain ungraded, the course total will report 104 x n where n = number of units.
This is because ‘Aggregate only non-empty grades’ will not work with custom grade calculations; as a result, Moodle returns ‘-’ (rather
than 0) for all non-graded projects, breaking the formula.

// Part II: Summary
9. Create ‘dummy’ Gradebook folders for Units, Outcomes and Quizes
10. Create categories in Outcomes per Unit, Scale display, ‘Lowest’ aggregation
11. Add each assignment using Rubric, Feedback, Outcomes, and Tracking
Feedback Comments: Yes
Grade: 100
Grading Method: Rubric
Grade Category: Unit addressed by assignment (from 10 above)
Outcomes: check ALL but ONLY outcomes addressed by project
Group Mode: Separate groups
Visible: Hide (if you require Batch Release of Marks)
Completion Tracking: ‘Show activity complete when conditions are met’
Require view is unchecked
Require grade is checked
‘Expect completed on’ set to assignment’s hand-in date + 2 weeks
12. Define Rubrics with Grade Descriptors, scoring 0,100,110,111 for NYA,P,M,D
13. Separate Gradebook rubric totals (as Real) from outcome totals (as Letters)
14. Configure Course Total with equation: = round(80*(ln([[U1]])^0.25),-1) + ...

// Example 1: Completed Feedback
● Feedback Example
○ Learning Outcome Identified
○ Grading Descriptor Identified
○ Outcome specific to each
○ Over Feedback used for general aspects,
professionalism, spelling/grammar etc.

// Results
● Gradebook Example with Two Assignments
○ Scale-based Outcome grades
￫ marked in scale dropdowns
○ Scale-based Outcome Total
￫ lowest mark aggregation of scales grades
○ Rubric/Letter-based grade
￫ percentage scale converted to letter
○ Numerical Overall Course total
￫ converted from outcome marks by custom equation
○ Another Rubric/Letter-based Unit

// Adaption to NCFE
● NCFE very similar to BTec but Pass/Fail only
● Setup
○ Scale
￫ Name “NCFE Pass/Fail”
￫ Scale ‘Not Yet Achieved, Pass’
○ Letters
￫ ‘100% Pass’, ‘0% Not Yet Achieved’
○ Outcomes
￫ Enter as for BTec
○ Outcome Aggregation
￫ Lowest grade, grade to pass = 2
○ Rubric
￫ as for BTec except grid is 3 x n (no Merit/Distinction, just Title, Not Yet Achieved, Pass)
○ Course Total
￫ scores 70 for each passed Unit

//
● Overview
○ A complex system to model because it uses has two interdependent sets of competencies
￫ One Pass/Fail, and, if passed, a further P/M/D grading
● Assessment Criteria
○ Like BTec, Units comprise Learning Outcomes.
￫ These in turn each break down into a number of subordinate Assessment Criteria (AC). For example;
Unit 1
Outcome 1: AC1, AC2
Outcome 2: AC1, AC2, AC3
Outcome 3: AC1
Outcome 4: AC1
￫ While Pass/Fail only, they may be passed at two NQF levels, Level 2 or Level 3 Pass, depending on the work’s attainment.
￫ All AC must be passed at Level 2, and the entire qualification requires at least 15 of the 20 Units to be at Level 3.
Adaption to APT Access

// Adaption to APT Access
● Grading Descriptors
○ If and only if all AC of the Unit have been passed at Level 3, the work ‘qualifies for grading’
￫ This is against 7 fixed Grading Descriptors (GDs) common to all Units.
￫ At least 2 of the 7 available must be applied to each Unit, and GD7 is always applied;
1. Understanding of the Subject
2. Application of Knowledge
3. Application of Skills
4. Use of Information
5. Communication and Presentation
6. Autonomy/Independence
7. Quality
￫ These are graded as Pass/Merit/Distinction only — there is no failure condition for GDs.
￫ The work has already passed because GDs cannot be considered unless it has passed all AC at Level 3.

● Other Observations
○ Scale and Rubric are independent (neither ‘knows’ the score of the other)
￫ We still need to include both a Refer and a Level 2 Pass Only option
￫ This accounts for the ‘non-qualifying’ AC conditions, thus rubrics grade AC and outcomes grade GDs.
○ APT do not aggregate their GD course totals
￫ Instead, learners are sent a transcript containing their grades for all units individually.
￫ The final course grade is purely in terms of an overall Pass/Fail, based on the AC
￫ (15/20 Units or 45/60 credits at Level 3 Pass).
● Solution
○ Use Rubrics for Assessment Criteria
￫ Having levels for Refer, Level 2 Pass Only (eligible for grading), Level 3 Pass Only (eligible for grading)
○ Use Scales/Outcomes for Grading Descriptors
￫ Having levels for Ungraded Refer, Ungraded Level 2 Pass Only, Level 3 Pass Only, Merit, Distinction

● Setup
○ Scale
￫ Name "APT Grading Descriptor R/P2/P3/M/D"
￫ Scale “Ungraded Refer, Ungraded Level 2 Pass Only, Level 3 Pass Only, Merit, Distinction”
○ Letters
￫ '100%, Level 3 Pass (eligible for Grading)', '90%, Level 2 Pass (not eligible for Grading)', '0%, Refer'
○ Outcomes
￫ Add just the 7 APT Grading Descriptors.
￫ Assessment Criteria are handled exclusively with the rubrics, so do not need to be added.
○ Outcome Aggregation
￫ Median grade, grade to pass = 3
○ Rubric
￫ as for BTec except grid is 4 x n (Title, Refer, Level 2 Pass (ineligible for grading), Level 3 Pass (eligible for grading))
○ Course Total
￫ Not needed for APT

// Internal Verification and Batch Release
● IV: a Limiting Grade
○ IV’ing of grades is not even Best Practice, it’s mandatory!
○ Moodle 2.5 has no simple was to release batches of grades en masse
○ Moodle 2.6 addresses this with grading Workflow and the ‘Released’ flag…
○ ...but still doesn’t accommodate moderation/IV commentary
● Solution: create 3 assignments for each project!
○ 1 contains the brief and submission upload but no grading at all
￫ always un-hidden
○ 2 contains all the complex grading setup above
￫ remains hidden until the IV process is complete and all work is marked
○ 3 is a direct copy of 2 for IV only
￫ always hidden
￫ contains initial sample of work for IV’ing
￫ marker and IV refer here until signed off
￫ work then marked in 2 and released when complete

// Tutorials
● Moodle still lacks a dedicated pastoral tutorial system
○ Create a ‘dummy’ course for Tutorials
￫ Enroll every learner to the ‘Tutorials’ cohort
○ Use Rubric-based assignments
￫ Create 6 identical assignments for their 6 half-termly tutorials received in a year
○ Address 4 measures:
￫ Attendance & Punctuality
are you attending everything you need to attend, on time?
￫ Behaviour & Communication
interpersonal 'soft' skills essential for getting on with people at work and in the wider world
￫ Employability & Professionalism
how are things shaping up for when the time comes to move on to the next stage in your career?
￫ Effort, Engagement & Achievement
how to make the most of the amazing potential inside you
○ At the next tutorial, the tutor grades progress towards targets set on these
￫ No Progress / Some Progress / Significant Progress / Targets Achieved or Maintained
○ Closely resembles professional appraisal systems, so good practice for employment

// Tutorials
○ Rubric
￫ Grades SMART targets set last time
￫ Sets new SMART targets for next time
○ General feedback
￫ Contains more general comments
￫ Copy/paste current Attendance stats

// Beyond Coercive Configuration
● Some functionality is beyond Coercive Configuration
○ “Tricking” requires there to be at least a starting point within existing functionality
○ When this is not present, we are faced again with the initial problem
￫ Alter Core Code? / Commission Plug-ins? / Wait for Moodle?
● Solution: write a dedicated application!
● Plug-outs
○ Plug-ins and other ‘standard’ solutions all present incompatibility risk
○ I write dedicated code that exists separately and independently from Moodle
○ Moodle then interacts with the plug-out, sending and receiving only what it needs
● Principle of Minimal Interaction
○ Interaction with Moodle is done solely on the basis of user id
￫ e.g. a simple Moodle block just calls the plug-out with the Moodle user_id in its GET call, and frames the page

// Our Attendance Plug-out: dartMOOR
● dynamic automated real-time Moodle Object Oriented Registration
○ Requirements too complex to use Coercive Configuration technique
○ Off-the-shelf solutions expensive and not optimised to our needs
○ I had already written a more basic system for dBs used from 2005-12
○ Developed over 2012/13 in PHP/mySQL
○ Runs across multiple sites from one code engine
○ Main features:
￫ In class real time registers with graphical data display
￫ Full analysis of all group, course, qualification etc. hierarchies
￫ Statistical operations including attendance time trend analysis & prediction
￫ Full student history, tracking group and course memberships
￫ Retention/achievement/success analyses
￫ Absence chasing features
￫ Gamified student-facing statistics in Moodle

// dartMOOR: Taking a register
○ Group
○ Calendar entry for slot
○ User pictures forwarded from Moodle
○ Register entries with roll-over tool
tips

// dartMOOR: Summary Data for a Group
○ Paginated Data Display
○ Automation options

// dartMOOR: Graphical Data Display
○ Register entry codes with roll-over
tips
○ Raw counts
○ Absolute counts
○ Comparative percentages

// dartMOOR: Timeline Analyses
○ Session counts
○ Cumulative % (statistically smoothed)
○ Diagnostic Times (2130 data points)

// dartMOOR: Trend Analyses
○ Based on comparing overall
attendance with that of last 6 weeks
○ Full regression analysis
○ Extrapolation to the course end date

// dartMOOR: Register View
○ Links to edit register
○ Register notes
○ Colour coded entries
○ Names here (data protection)

// dartMOOR: Retention Tracking
○ Based on official post-census stats
○ Based on internal pre-census stats
○ Retention 0% since course still
current (i.e. no Completers yet)

// dartMOOR: Gamified Attendance Gauges
○ Gamified Attendance Gauges
exist in ‘plug-out’ as an external
call to a dartMOOR page

// Gauge Effects
● Uptake
○ Huge buzz created when they went live
￫ Reports from tutors across all sites of students immediately working out what they needed to “beat my mate!”
● Statistics
○ True measures of change in attendance complex to calculate
○ Attendance can only ever decline from the perfect 100% you start the year with
￫ e.g. the measure of effect is a change in the rate of decline
○ With that metric, gauges improved attendance (=slowed decline) by +0.78% per week
￫ i.e. +78% relative to when they were not in place
● Further Work
○ Link to learner’s dartMOOR homepage from the gauges, showing more detailed information
○ Automated SMS attendance chasing
○ Integrated CRM and pre-enrolment marketing functionality
￫ Covering all aspects of the learner recruitment process into Moodle

// Summaries & Conclusions

// Summary: Coercive Configuration
● Half-Absent Functionality
○ Careful analysis of a problem
○ Careful examination of Moodle’s existing functionality
○ Apparently absent functionality can be created or closely approximated by careful configuration
● Pros
○ Free
○ Portable
○ Non-code-invasive
○ Requires no coding skills
○ (Comparatively) easy to implement
○ Can add significant functionality
● Cons
○ Can hit a brick wall of Moodle’s ‘base’ functionality

// Summary: Best Practice Marking Modes
● Marking Modes
○ Click-based grade descriptor grading
○ Individuated Learning Outcome feedback
○ Additional overall feedback box
○ Coloured flags for pass vs fail grades
○ Correctly displayed Learning Outcome grades
￫ (e.g. for BTec Not Yet Achieved / Pass / Merit / Distinction)
○ Correctly aggregated and displayed Unit grades
￫ (e.g. lowest grade for BTec)
○ Correctly aggregated and displayed numerical Course grades
￫ (e.g. for BTec NYA = 0, P=70, M=80, D=90)
○ Bulk released post-IV grades & feedback
○ Tracked reading of the Brief by the Learner
○ Tracked submission of the Assignment by the Learner
○ Tracker grading of the Assignment by the Tutor

// Summary: Plug-outs
● Plug-outs
○ Where required functionality exceeds the scope of Coercive Configuration, consider Plug-outs
● Pros
○ Allow functionality to be completely specified
○ Independent of Moodle code and development path (depending on requirements)
○ Do not require detailed knowledge of Moodle codebase
○ Minimal transaction of data between Moodle and plug-out minimises incompatibility risks
● Cons
○ Require some degree of data integration to tie systems together

// Conclusions
● Wins!
○ These systems support core components of learner success ‘synoptically’
○ Uptake has been extremely successful, with great feedback from tutors and learners
○ Additions are free, non-code, in development path and deployable out-of-the-box
○ Feedback systems conform to awarding body best practice and add new functionality to Moodle
○ Conforming to Best Practice has clear and valuable implications for Body EV assessment
○ OFSTED also view adherence to Awarding Body Best Practice as contributory to a Grade 1
￫ particularly with their current focus on individual action planning
￫ OFSTED also view positively the opportunity for learners to have ownership of their progress
￫ this system clearly contextualises learners’ work within the framework that it is assessed, to best maximise achievement
○ Student-facing attendance promotes ownership of their data
○ Gamification maximizes engagement and self-directed improvement
○ Rubric-based tutorials prompt tutors to set SMART attendance targets based on all of the above
○ Tutorials wrap together the assignment feedback and attendance strands

// Thank you for your kind attention

Marking Modes in Moodle - configure for BTec/NCFE/APT for free! Also, Moodle dartMOOR attendance PlugOut.

Recomendados

Recomendados

Mais conteúdo relacionado

Semelhante a Marking Modes in Moodle - configure for BTec/NCFE/APT for free! Also, Moodle dartMOOR attendance PlugOut.

Semelhante a Marking Modes in Moodle - configure for BTec/NCFE/APT for free! Also, Moodle dartMOOR attendance PlugOut. (20)

Último

Último (20)

Marking Modes in Moodle - configure for BTec/NCFE/APT for free! Also, Moodle dartMOOR attendance PlugOut.