Collecting and storing evidence is an important aspect of the continuous improvement process. In this session, leaders from two Distinguished Academies will share their evidence binders and strategies for documenting the academy story.
3. The Academy
Assessment
is a tool for
reviewing
your academy
program.
Evidence is used
to verify your
selections.
Welcome: Patti Smith
4. Presenters
Michele Halopoff
PAB Academy of Finance
Porterville High School
Porterville, California
Cindy Pierre
Academy of Information
Technology
Miami Central High School
Miami, Florida
5. Academy Development &
Structure:
1.2.d All academy students
receive career-themed guidance
on college exploration and
college choices based on their
individual interests.
Advisory Board:
2.3.a Advisory Board members
provide resources to ensure the
stability of the academy. In the
past year the academy has…
Curriculum and Instruction:
3.3.b There is an organized
approach to identifying
students at risk of failure,
accompanied by intervention
systems to address factors that
increase academic failure and
dropping out. Our academy
currently…
Work-Based Learning (WBL):
4.1.c A continuous evaluation
process is in place to assess the
effectiveness of the work-based
learning program and to plan
for future refinement and
implementation. The process to
assess and improve our
academy’s WBL program is…
What Evidence Would You Use to Document…
6. Category Conditions
Under Review • Scores below 15 on the Academy Assessment
• Verified by the regional team
• Academy is not meeting expectations and will need to make
improvements
Member
Academy
• Scores at least 15 on the Academy Assessment
• Verified by the regional team
• Academy in process of completing a 3- or 4-year program or
• Academy needs to improve to move to Certified status
Certified
Academy
• Overall score of 20 on the Academy Assessment
• Verified by the regional team
Model
Academy
Distinguished
Subset
• Overall score of 25 on the Academy Assessment
• Verified by regional team
• Score of at least 20 on each of the four elements
• Program completion with a graduated class
• Meets Thresholds
• Demonstrates high student performance
• Member in Good Standing
NAF Membership Status
7. • Document academy activities
• Provide proof that we are meeting criteria for
NAF Membership Status
School Accreditations - WASC
School District
Academy’s Advisory Board
Other Certifying Agencies
- NCAC
- ConnectED: Linked Learning
Other state and local requirements
• Academy team reflection tool
Why Collect Evidence
8. • CAPE Academy Annual Registration
Florida Legislature passed Career and Professional Education Act.
School districts with registered CAPE Academies are eligible for
additional funding in the Florida Education Finance Program
(FEFP).
• Industry Certifications
Create an evidence binder for intent forms
• Career and Technical Organizations (CTSO)
FBLA club documentation
• Administrative buy in
Provide new administrators information about academy
activities
• Academy sustainability
Other reasons to collect evidence
9. • Academy Assessment was our driving document
• Team met bi-monthly
Every agenda had specific assessment items to review
Open discussions of self-evaluation caused reflection of:
current practices
students’ needs
fidelity to NAF Academy Model
- especially in new teachers
goals for continuous improvement
- as an academy
- individually as teachers
• Interim meetings
Required to meet deadlines
Evidence Collection Process
10. • Collaborate with other NAF Academy Directors
Identify seasoned NAF lead teachers and collaborate
• NAF Academy District Support
Lead Teacher Meetings (Miami Florida Schools)
Support of NAF District liaison
• Share general information used by all NAF Academies
District Advisory Board Strategic Plan
Advisory Board sponsored activities and events
Evidence Collection Process
11. • What evidence was most representative of our academy
• Who had relevant evidence to provide
Teachers
Curriculum
Integrated projects
Academy Guidance Counselor
Enrollment data
Four Year Plan
Academy Administrator
Data
Advisory Board
Special meeting for self-evaluation & evidence
School District
Recruitment
Work-Based Learning Legal Contract & Handbook
Data & Administrative Support Documentation
Relevant Evidence
16. • Created a NAF Certification Folder
• Created sub-folders:
one for each team member
“NAF Resources”
“Not Used”
Academy Development & Structure
Advisory Board
Curriculum & Instruction
Work-Based Learning
• Created an Evidence Description document
logged standard and description
used to keep track of progress
able to search on key-words
allowed us to work individually
not dependent on others’ schedules
Managing Evidence & Staying Organized
17. • Create a list of instructions for your academy
to ensure consistency
• Combine documents that are related
scan into one PDF
• Videos must be posted on the Internet
• Document size constraints (5 MB)
convert large documents to PDF
save as reduced size PDF
• Save frequently
LiveBinders would freeze and data was lost
• Allow ample time for uploading process
distractions can interfere with progress
Tips & Tricks for Using LiveBinders
18. • Storage constraints for entire binder
Storage of 100MB
File size upload limit of 5MB
Benefit of Pro/Company license
• Public vs. Private online binders
Free version is only public
• Use collaboration feature
District supervisors can access and collaborate with
Academy Directors
• Access LiveBinder using Google Chrome
Tips & Tricks for Using LiveBinders
19. • Allow advisory board members and business
partners to review your evidence binder
Process will offer valuable opinion and information
Criticism and questions are important to the evidence
collecting process
• Share the whole story about your academy program
Challenges are an important part of the whole picture
Victories are the academy accomplishments on display
• Ask questions or for HELP!
Livebinder is a new process
⁻ Collaborating will simplify the process
• Follow the NAF Assessment Guide
Description and examples of the evidence to use
Only allowed to use evidence once
Tips & Tricks about the process
26. • Academy Assessment is the driving document
• Multiple Uses of Evidence:
ConnectED: Linked Learning
CPA
• Collaboration with academy team is crucial
Prompts buy-in and reflection
Promotes goals for continuous improvement
• Scrutinize evidence to maximize portrayal of
academy and culture
• Create a timeline – start with the end in mind
Allow ample time for uploading process
Distractions can interfere with progress
Summary
27. •How important collecting evidence was to the growth of our
program
Provide an opportunity to review all aspects of the academy
Evaluate hard questions and discuss what pieces were missing
• Receive feedback from all stakeholders
Include advisory board member/business partner
NAF local and district support
• Use pictures and graphic designs to show program at work
• Tell the whole story- the challenges and victories
The academy story: what we learn from evidence
28. Review issues raised in the ice breaker, did we cover all the
topics?
Closing, thank you- Patti
Closing
Notas do Editor
Intro to continuous improvement cycle and how Aa fits. Data center entries occur first, before you can open online AA. Gather a review and collect evidence throughout the year. The purpose of the Assess component is to gather various forms of information for analysis that can inform planning and actions for improvement
Today
2 distinguished academy leaders will share their process. Important to note they have different approaches. This is an informational session with a couple stops for clarification and questions.
10-15 minutes
Here is my idea for the icebreaker slide: four strategic actions displayed…
Take a look at these examples, they are four different standards, one from each section of the Academy Assessment
Think about the evidence that you would use to show that your academy does this…
In small groups – either by table or people can get up and meet others
Share and exchange ideas of evidence they would use for at least one of the standards…
In popcorn or whip around style, have people at each table share out ideas of evidence
Patti (volunteered) will place the reported issues on chart paper
Get stories about their issues concerning evidence
What has been the hardest part of collecting evidence to support your academy?
PAB’s evidence examples:
1.2.d – CareerZone worksheet, ASVAB signup list, field trip requests & itineraries (local daytrips and overnight field trips)
2.3.a – Agendas itemizing activities w/AB members, pictures of AB members interacting w/students, workshops/presentations/guest speakers, news paper articles
3.3.b – List of students at-risk, agendas/minutes of team meetings discussing student concerns & strategies, tutoring schedules, contracts and results
4.1.c – An old version of our WBL continuum compared to our new and improved plan
PAB’s obstacles for collecting evidence:
Consistency among teachers collecting evidence samples at time of activities - instead some wait until requested
Consistency throughout the academy taking pictures/video of events
Let’s try to address these issues raised during the session
Patti will create a handout for this slide…
May not use, but have just in case we want to discuss NAF membership status, which I mention in the next slide.
10 minutes each
Cindy – what does NCAC stand for???
Evidence Collection Process
The Academy Assessment is the driving document
During the fall we meet regularly as a team and every agenda had an item specific to reviewing the Academy Assessment document.
Open discussions were instrumental in deciding how to evaluate ourselves, who had relevant evidence, what evidence would be most representative of our academy, and who would provide that evidence
We also made a conscious effort to provide examples for as many “evidence binder suggestions” in the left-hand side of the Academy Assessment document
This process was very helpful because it caused us to reflect on our current practices, how well we were meeting the students’ needs and NAF guidelines, and what we could do to improve as an academy, and more specifically as teachers
Vigorous collaboration and reflection caused us to take more time than anticipated during some meetings. This resulted in interim team meetings to meet our timeline goals. Interim meetings were by definition meant only to complete the work of the prior meeting and not to discuss new agenda items.
Michelle I added the two bullets at the bottom of this slide as add on information specific to Miami Schools…
Cindy – I combined the bullets to make the font larger
As each standard in the Academy Assessment was reviewed, we scrutinized the evidence to determine the one(s) that would be most representative of our academy and its culture.
The advisory board section of the Academy Assessment was completed during a special advisory board meeting that took about an hour.
District support was instrumental in two areas:
Recruitment – our district has a total of ten pathways and one Pathways Office that oversees all the academies. In order to avoid duplication of effort, they plan and coordinate the recruitment for all academies at the district level
Work-Based Learning Contracts – this has also been standardized by the Pathways Office and has passed all the legal requirements of our school district
5-7 minutes each – 14 minutes total
First time users must sign up, otherwise just log in.
We started with a template that was provided by NAF,
we copied and renamed the template,
then we edited the LiveBinder
The LiveBinder contained five tabs:
Instructions
Academy Development & Enrollment
Advisory Board
Curriculum & Instruction
Work-Based Learning
Each tab contained a sub-tab for each standard
Can use Dropbox, using the same format
At first, managing the evidence and staying organized was overwhelming, but then we learned a few “tricks”:
Create a NAF Certification folder and make sure that every team member in the academy has access
Create subfolders for every team member so that they can start adding evidence in their respective folder. This allows team members to download evidence without cluttering other subfolders. Each evidence file should be named so that it can be easily identified.
Create another subfolder for “NAF resources” that you might want easily accessible but not in the way of other documents
Create another subfolder for “Not used” so that if you change your mind about a file or decide you want to use it as evidence for another standard, you can have it easily accessible
Create an additional four subfolders, one for each section of the Academy Assessment and tab in the LiveBinders:
Academy Development & Enrollment
Advisory Board
Curriculum & Instruction
Work-Based Learning
This may seem like too many folders, but that is how we were able to best manage the evidence
The actual process that we followed:
As team members identified evidence, they were encouraged to place a copy in their respective folder baring their name
As each standard in the Academy Assessment was reviewed, we scrutinized the evidence in team members’ folders to determine the one(s) that would be most representative of our academy
If used:
the document would be renamed with the standard appended to the front of the name - by doing so, the documents were automatically numbered and in the order of the LiveBinders sub-tabs
the document would also be moved (not copied) to its respective section folder
If not used:
the document was moved (not copied) to the “Not used” folder. This was important because it helped us stay organized, keep track of what evidence had already been considered, and reduced duplication of efforts. In addition, if we later on decided we wanted to use the document as evidence, it was readily accessible
We also created a MS Word document “Evidence Description”, and each time a document was deemed to be evidence, it was entered into this document with the description that was to be used in the LiveBinders:
This allowed us to work individually, and not depend on each other’s schedules
We also used this document to search for key-words and
keep track of our progress
To upload evidence into the LiveBinders, we would refer to the “Evidence Description” document for the description.
As the process continued, team members’ folders were emptied and empty folders were deleted
At first, we found LiveBinders to be tedious and time-consuming. Here are some tips:
Create a list of instructions for your academy (see our example) to ensure consistency across users that are uploading evidence
When documents are related, combine them. One way to achieve this is by scanning them into one PDF. This streamlines your evidence binder (by not having too many sub-tabs) and minimizes your data entry into LiveBinders
Videos can be great sources of evidence, but they must be posted on the Internet before you can use them. Some were already posted on YouTube and those that weren’t I posted on TeacherTube.
LiveBinders has a document size constraint of 5MBs. One way around this is to convert large documents into PDFs and then save them as a reduced size PDFs
Be sure to save frequently. Not sure why, but LiveBinders would freeze, and when we resumed, the work that was not saved was lost!
Give yourselves ample time to complete the upload process. The workplace has many distractions and emergencies can distract us from what is important
5 minutes each – 10 minutes total
Better to convert documents to PDF since it loads faster and you can See it in the window.
