This document summarizes a presentation about preparing for internship assessments in the Student Certification Assessment System (SCAS). It discusses the roles and responsibilities of coordinators, supervisors, teachers in facilitating internships and assessments. Requirements for students to receive a satisfactory rating are explained. The timing of activities before, during, and after internships is also covered. Support resources available for SCAS are outlined.
3. Agenda
• Getting To Know Each Other – Brooke
• SCAS Update – Karlene
• Internship Assessments – Brooke
• District Perspective – Lupe
• SCAS @ NAF NEXT – Brooke
• Questions - Brooke, Lupe, Karlene,
Getting Ready for the IAs
4. • What is your role?
• Are you in the same role this year as next?
• What internship supervisor orientation
activities do you conduct?
• Have you used SCAS before?
• Do you remember your log in?
Getting Ready for the IAs
7. Student Certification Assessment System –
SCAS
• A Rigorous Certification
• A Secure Assessment Program
• A Measurement of College and Career
Readiness
• Aligned with Industry Standards
• Designed with National Curriculum and
Assessments Experts
• Developed with NAF Network Input
What SCAS is…
8. • A Learning Management System
• A District Student Information
System (SIS)
• A Student Portfolio
• A Lesson Planning Tool
• A Formative Classroom
Assessment
What SCAS is NOT…
11. Number of Unique Students Starting an
Assessment in SCAS
SCAS By The Numbers…
12. Number of Academies that had students
complete one or more CPACs, EOCs, or
Internship Assessments (IAs): 9/12 - 2/14
• AOF 40 academies
• AOHT 26 Academies
• AOIT 20 Academies
• AOE 6 Academies
SCAS By The Numbers…
13. • Online System
• Policy and Practice
• Training and Outreach
• Value Add
Where NAF Is Going With SCAS
15. Policy and Practice
• Straightforward messaging
• Transparent requirements and
expectations
• Clear, defined processes
• Frequent and early updates
Where NAF Is Going With SCAS
16. Training and Outreach – Fall 2014
• Pilot Assessments
• District Champions and Power Users
• District Coaching
• Webinars
• Resources
• Technical Assistance and Support
Where NAF Is Going With SCAS
17. System Enhancements
• Class Roster With Raw Scores
• Internship Assessment Visibility
• Click/Dropdown To Enter Students
• Report Development
• PDFs
• Online, On Demand
Where NAF Is Going With SCAS
18. Adding Value and Benefits
• Post Secondary Partnerships
• Business Partnerships
• State Partnerships
• District Partnerships
Where NAF Is Going With SCAS
21. Issued upon graduation to a student who
has:
• Completed 4 NAF courses receive a
passing score on each of the 4
culminating projects and end-of-course
exams or end of course exams in 2 year-
long courses for AOE.
• Received a satisfactory rating on the
internship assessment.
• Met all local high school graduation
requirements.
SCAS Requirements…
22. To receive a satisfactory rating, students must:
• Participate in a 120-hour internship (2015)*
• Receive a 2, 3, 4 performance level within
each dimension; and
• Meet or exceed expectations on a majority
of the dimensions in the four categories;
and
• Be rated as “Meets Expectations/Usually”
(3) or “Exceeds Expectations/Always” (4) on
a set of dimension deemed “high value:”
SCAS Requirements…
23. • Majority of the dimensions
SCAS Requirements…
Part Category Dimensions
Part I Foundational Skills 7 listed
Applies Workplace Skills 7 listed
Self-Management Skills 9 listed
Part II Career Contextual
Knowledge Skills
4 listed
Part III Position-Specific Technical
Skills
Maximum 3 –
Supervisor entered
24. High Value Dimensions – Score 3 or 4
• Part IA3. Critical Thinking – Foundational Skills
• Part IB4. Teamwork/Collaboration – Applied
Workplace Skills
• Part IC9. Exhibits responsible and professional
behaviors – Self-Management and Personal
Responsibility,
• Part II4. Knows how to interact with supervisors,
clients, and teammates – Knowledge of the Field
and Organizational Context
•
SCAS Requirements…
26. Timing
• Before The Internship
• During The Internship
• After The Internship
Timing is Everything
27. Timing
• Before The Internship
• Examine Materials and Resources
• Share Assessment Tool
• Create and Manage Internship
• Calendar Dates
• Communication and Tracking
• Reviewing the Results
Timing is Everything
28. • A Continuum of WBL
• Aligned to Gold Standards
• Based on Skills and Dimensions
• Foundational Skills
• Applies Workplace Skills
• Self-Management Skills
• Career Contextual Knowledge Skills
• Position-Specific Technical Skills
Internship Assessment
42. Where To Find SCAS Information
Brooke Rice
National Academy Foundation
43. Support and Resources
• Guide
• FAQs
• Videos
• Webinars
• Coaching
• Professional development
• Pilot Participation
• Technical Support
Where To Find SCAS Information
44. • NAF NEXT
• Open Labs
• Today - July 9 @ 4:15 PM
• Thursday - July 10 @ 12:00 PM
• Friday - July 11 @ 1:30 PM
• Sessions
• SCAS Hands-On: Getting Ready for
the EOCs - Friday @ 1:15PM
• SCAS Panel Session: Unique
Solutions and Practical Ideas -
Friday @ 3:00PM
Where To Find SCAS Information
45. • To participate in SCAS or for
information about SCAS training,
contact a member of your NAF
Instructional Management Team.
• “Help” section with tips regarding
several key activities.
Where To Find SCAS Information
46. • Internet Testing Systems (ITS) support
team is available Monday through
Friday from 8 am to 5 pm EST online,
via email or by phone.
• An online request for assistance can
be submitted when logged into the
system. The ITS support email
is NAFsupport@testsys.com and
phone is 1-800-514-8494.
Where To Find SCAS Information
47. • If you have questions about
SCAS general practices or
continue to have difficulty, please
contact SCAS@naf.org.
Where To Find SCAS Information
Brooke
Online Versus Hands- On
If we can do the Hands On…then Lupe and Karlene will provide classroom support. If it is only demo…..then Lupe and Karlene can interject as appropriate.
Brooke
Role
District? SCAS Admin? Academy Director, Lead Teacher, Internship Coordinator, Principals, Teacher, Supervisor, Advisory Board Member?
If we have several from one group …we should tailor the session.
Karlene – Total Time 8 Minutes ****** Move quickly …..
There are three components of SCAS,
the End of Course Exam, the Culminating Project Assessment for Certification, and
the Internship Assessment.
The IA focuses on all but core academics ----
Growing interest on the part of Academies and Industry.
A Culminating Project equates to 3 assessments ---- Planning, implementation and final Product --- in the ITS system
EOC equate to 1 assessment in the ITS system
Therefore, each completion of a CPAC and EOC per course equates to 4 assessments within the ITS system.
A Culminating Project equates to 3 assessments ---- Planning, implementation and final Product --- in the ITS system
EOC equate to 1 assessment in the ITS system
Therefore, each completion of a CPAC and EOC per course equates to 4 assessments within the ITS system.
You are not alone……..
Had students complete one or more CPACs, EOCs, or Internship Assessments (IAs): 9/12 - 2/14
States – 18
Districts – 22
The SCAS working group (Kevin English, Laura Fidler, Brooke Rice, and Fred Press) have been focused on four key areas with the greatest challenges and the most opportunity to strengthen the SCAS project. Those areas are:
Online System
Policy and Practice
Training and Outreach
Value Add
Based on technical assistance calls and comments from the field, we have identified the following issues with the online system:
Manage Classes and Student
Requirement for teachers to type in the list of students’ names
online tool’s inability to discern hyphenated or two part names
System confusion with Students with the same last and first name
Reporting
Currently only NAF central system users are able to access user information or result information and only one student at a
Result information is provided on a daily basis on a flat file only accessible by a few people
Assessment completion information is not compiled or distinguished by assessment type
User accounts are linked to email accounts and therefore do not easily allow teacher to move from one academy or school to another
Neither teacher nor internship supervisors are able to see the results of the completed Internship Assessment
Certification Validation and Credentialing Process
Accommodations
Cut Scores
HAND OUT – Webinar Dates and Pilot Information
Outreach and Training
March – 2-Day Instructional Manager Training
NAF NEXT
EOC/CPAC Session
Internship Assessment Session
Panel Overview Session
Open Lab Session
Digital Materials and Media
FAQs – 12
Screen Shot Video Vignettes - 10 videos
PPTs
Requested Reports
Student SCAS Progress
Levels - National, Region, District, Academy
Number of students who have received a passing score as defined by SCAS business rules on 1, 2, 3, or 4 Culminating Projects (CPACs) and End-OF Course (EOCs) exams and an Internship Assessment
Internship Assessments
Levels - National, Region, District, Academy
Report the number of students and supervisors participating in the Internship assessments by companies or business of the supervisor.
This may require grouping based on key words - Verizon, JP Morgan, ATT, etc.
Transaction
Levels - National, Region, District, Academy
By theme the number of started and completed CPACs, EOCs and an Internship Assessment within an identified time frame (quarter, semester, school year)
Report should differentiate between assessments
Course and Learning Objective Analysis
Levels - National, Region, District, Academy
By learning objectives within the assessments, the number of correct versus not-correct answers. This would aggregate data…not by individual student.
Course and Score Analysis
Levels - National, Region, District, Academy
Report by dimensions and criteria the Average raw score and performance levels (0,1,2,3,4) for the Internship Assessment and the CPACs.
Add information released at NAF NEXT
Brooke
The IA assess dimensions …… hold up the Tool??? The IA content and format and process is copyrighted and proprietary ….. FYI.
NAF Student Certification is issued upon graduation to a student who has:
Completed four (4) NAF courses receive a passing score on each of the four (4) culminating projects and end of course exams or end of course exams in 2 year-long courses in the case of AOE
Received a satisfactory rating on the internship assessment.
Met all local high school graduation requirements at a level required for admission to its state’s public postsecondary institutions (four-year, two-year, and community colleges).
Participate in a 120-hour internship. – NOTE this will be a requirement for any internship that is a part of SCAS AFTER May 2015.
Receive a 2, 3, 4 performance level within each dimension - This means NO 1s or 0s
More than 50% of the items must be 3 or 4s within each dimension which equates to the the entire assessment
Foundational Skills
Applies Workplace Skills
Self-Management Skills
Career Contextual Knowledge Skills
Position-Specific Technical Skills
See next slide
More than 50% of the items (9 items) must be 3 or 4s within each dimension which equates to the the entire assessment
Foundational Skills
Applies Workplace Skills
Self-Management Skills
Career Contextual Knowledge Skills
Position-Specific Technical Skills
See next slide for High Value
MUST score a 3 or 4 on these items …. Should let supervisor know this….not to coerce but to inform
Brooke
Hand out the Role and Calendar – Karlene will revise and print
Brooke
Hand out the Role and Calendar – Karlene will revise and print
Brooke
Hand out the Role and Calendar – Karlene will revise and print
Examine Materials more on next slide
Where to get help and information on this …..open lab times
Brooke
Foundational skills assessed in the internship. Note that 3 of the seven skills assessed separately here are often represented together simply as communication skills (listening and observation, oral communication, and written communication). While that makes sense when trying to shorten a list of outcomes for purposes of communication, the input we received from industry representatives in our assessment development process was that these three distinctly different communication skills should be separately addressed in a performance assessment.
Applied Workplace skills assessed in the internship. Some of the skills grouped here are listed in our CCR definition document as foundational skills, and others form the core of what that document identifies as interpersonal skills. Again, these differences in grouping are largely a result of differences in the purposes of the lists: one to communicate to a broad audience, the other to assess performance in actual work situations.
Self-Management assessed in the internship. The list of self-management skills is much more detailed for purposes of assessment than it is in the CCR definition document. Every item in the assessment form is there because an argument was made during assessment development for keeping it separate. We suspect that with analysis of a sufficiently large number of results, we will be able to make an argument based on empirical evidence for reducing the number of items here by consolidating some of the more highly correlated ones.
Contextual knowledge of career requirements and the workplace are also assessed in the internship. Part II of the internship assessment includes only four items. The first two could be considered a check on the career content and contextual knowledge that is more specifically and thoroughly assessed through course-based project assessments and end of course exams. The third and fourth are important learnings that can really only be assessed in the context of a real work experience like the internship. In a shorter list, for communication purposes, the fourth item might be considered covered by interpersonal skills as assessed in the applied workplace skills category. But here the focus is on interaction between and within workplace defined roles.
Part III of the internship assessment is important for information gathering that will help NAF and the academies better understand what position-specific skills employers are expecting for program improvement purposes; but by definition, these are skills that are not necessarily needed across all career paths, so they are not counted in the internship assessment for CCR certification purposes.
Internship Coordinator clicks on Internships link in left navigation menu.
Confirm that students’ accounts exist on the SCAS first. The person with front line administrative responsibility for the Academy (e.g., Lead Teacher, Site Coordinator, Academy Director, etc.) creates student accounts as a part of the End-of Course (EOC) assessment (EOC exam with Culminating project). A student account has to be created only once for each student in the academy.
Student Internships that have already been entered will appear in a table labeled Active Student Internships.
To add new internship supervisors, select the Internship Supervisors Tab.
Click on the Add Location link to add new internship supervisors.
Supervisors only have to be entered in the SCAS once.
Entering the name and email address of the direct supervisor is preferred. If the direct supervisor name and email are entered, the direct supervisor will receive all email notifications directly.
An employer point of contact may be entered as the internship supervisor. If a single point of contact is entered and associated with several student internships, the point of contact will receive all email notifications and must then forward the email with the link to complete the assessment to the appropriate direct supervisor of the student.
Add the Supervisors to the SCAS.
Supervisors only have to be entered in the SCAS once.
The information required about the Supervisor includes:
First Name and Last Name
Email Address
Organization (Location)
The organization (company) name is sufficient if all internships are at one location, but if the organization has multiple locations (e.g. bank branch offices, retail outlets, medical offices, etc.) the information entered should include the location as well as the organization name.
This process must be repeated for each individual supervisor or point of contact.
After the Supervisor has been entered, internships with that Supervisor can be set up for multiple students at the same time and/or across time.
The next set up step is to match students with Supervisor.
Type in the Student Name. The system will search for a matching student name from student accounts that have already been created.
If the name of the student pops up on the matched list, select that student name. If the name of the student does not appear on the matched list, you cannot set up this internship until the student has been added to the system. An administrator must add the student. Once the student has been added to the system, the internship coordinator can proceed to set up the internship by selecting the internship supervisor and selecting the dates of the internship. Click Save when all required information has been entered.
Select Supervisor name from drop down list.
The final step is to enter the beginning and ending date of the internship.
The final step is to enter the beginning and ending date of the internship.
SCAS Automatically Sends Emails based on the calendar dates set up in the IA.
Once the Supervisor receives email notifications with information regarding the internship assessment, the Supervisor does not require an account on the SCAS and will never have to log in to the SCAS to complete the form.
There is a text box available to enter an example of something the student did. Supervisors are encouraged to include examples for any and all ratings when an example easily springs to mind. Providing examples is optional for ratings of 2 or 3.
After the supervisor has completed the assessment, the Internship Coordinator can view the completed assessment by clicking on the Complete link in the Assessment column.
This is a one student at a time process….. Not able to see aggregate data for IA results ……
Internship Coordinators may view results after an IA has been completed
Lupe
Lupe
Share two Lessons learned…something that Lupe would do again when implementing and something wthat she would not do again.
