Course Tech 2013, Rachelle Hall, Using Course-Level Assessment to Measure College-Wide Outcomes
•Transferir como PPTX, PDF•
0 gostou•360 visualizações
Assessing college-wide technology literacy can be easy with SAM. Glendale Community College has been using SAM to assess both course-level assessment and assess their students’ understanding of technology when they leave the college through college-wide assessment. Come and see how you can tie course level assessment into college-wide assessment outcomes through the use of one tool.
Recomendados
Mais conteúdo relacionado
Destaque
Destaque (9)
Semelhante a Course Tech 2013, Rachelle Hall, Using Course-Level Assessment to Measure College-Wide Outcomes
Semelhante a Course Tech 2013, Rachelle Hall, Using Course-Level Assessment to Measure College-Wide Outcomes (20)
Mais de Cengage Learning
Mais de Cengage Learning (20)
Course Tech 2013, Rachelle Hall, Using Course-Level Assessment to Measure College-Wide Outcomes
- 1. Using Course-Level Assessment to Measure College-Wide Outcomes Rachelle Hall ~ Glendale Community College
- 2. Session Objectives • Defining Technology Literacy • Establishing and Implementing the Plan • Challenges • Creating the Measurement • Tool Selection • Analyzing the Results • Q&A
- 3. Technology Literacy • How do you define technology literacy? Student demonstrates proficiency with technological tools
- 4. The Plan • Course selection – CIS105 (2,800 students) – BPC110 (750 students) • Parameters – Course completion – ‘C’ or better • Need C or better to transfer
- 5. Challenges
- 6. Measurement Level 1 Level 2 Level 3 • Word • File Management • Advanced Word • PowerPoint • Folder • Advanced • Excel Management PowerPoint • Access • Desktop • Advanced Excel Management • Advanced Access • Operating System
- 7. Tool Selection 2008-2009 2009-2010 SAM2010 2011-2012 myITLab SAM2010
- 8. Analyzing the Results • SAM Frequency Analysis • Export to Excel • Scrub the data • Import into Access • Query to fit the request
- 10. Technology Literacy Reports • College-Wide Assessment http://www.gccaz.edu/crs/Accreditation/SAA/SAA1.html
- 11. Q&A rachelle.hall@gccaz.edu 623-845-3231
Notas do Editor
- Thank you all for attending this session. I am please to have you in attendance. My name is Rachelle Hall and I am the Director of Assessment and Review at Glendale Community College located in Glendale, Arizona. I am also a residential Faculty Member and assistant Department Chair in the Business and Information Technology Department and have been with Glendale for 13.5 years.
- There are a few objectives I would like to cover in this session regarding how GCC developed their plan by covering these objectives, but I would like to make sure we have plenty of time for questions throughout. Please feel free to either ask questions during the topic matter or jot them down and ask during the Q &A session at the end. This session belongs to you so I want to make it as interactive as possible. Glendale or GCC had previously set goals as a college in two categories: primary learning outcomes which included writing, critical reading / critical thinking and critical reasoning or Math; and secondary learning outcomes which included oral communication, information technology and technology literacy. Primary learning outcomes are assessed annually, and secondary outcomes are assessed year 3 years. There is a college committee identified as College Wide Academic Achievement Committee or CWAAC which is responsible for assessing and reporting out to the community the results of these assessments. My focus for this session will be on Technology Literacy and how we defined, established and implemented the secondary outcome of technology literacy at our campus which I hope will prove to be helpful if this is a step you are looking at taking on your campus as well.
- How do you define technology literacy? Is it being able to work on a basic laptop or PC or using the latest tablet or mobile device? How about maintaining network servers or even computer repair?Sometimes just defining Technology Literacy at your institution is half the battle. GCC defines Technology Literacy simply as: student demonstrates proficiency with technological tools. At one time, Information Technology and Technology Literacy were combined which was quite confusing, at least for us. We define Information Technology as: student uses a variety of resources to locate, evaluate and use information to solve a problem or make a decision.
- Once we had an agreed-upon definition, the plan was our next hurdle. We knew we needed to assess our students on their proficiency in using technology tools, but exactly who do we assess and how? We can’t force students to take exams and we all know that asking them nicely doesn’t always reap the best results. I’m sure many colleges like GCC have a common computer course that transfers to universities that meets the articulation for an actual transfer or an elective and so on. At GCC this course is CIS105. This is the direct transfer course to all three (3) state universities in Arizona. This course is also required for approximately 92% of all certificates and degrees that are offered on our campus (including transfer degrees). We decided to include BPC110 which is the elective course for transfer. Once we had the course selected, we needed to define the parameters to define the cohort. We went broad and because students must achieve a C or higher to transfer the course or to receive a certificate or degree in their field of study, that is the only parameter we needed.
- I know what you are thinking. How are you going to assess the students in these courses if the parameters you set do not qualify the students until after the course is complete? Well, that was our first challenge. We were very set on the parameters so came to the conclusion, the students need to be assessed within the classroom after the final exam. All students will take the assessment, and those who do not meet the parameters, cannot be included in the cohort. That did not fair so well either. This was our second challenge.We were fortunate that CIS105 years prior had be developed and is delivered equally throughout the campus. There is a coordinator for the course and we use one LMS shell, one SAM shell and common assignments, quizzes and exams. However, because this initially was implemented as an additional assessment at the end of the course, students were not applying due diligence and the instructors were also not enforcing the necessity of this assessment and some were not administering and / or delivering the assessment to their students. This was a road block in our success, but only a minor one.We jumped back on the freeway and chose another route. We then decided that instead of having this assessment be an additional piece, we would collaborate with the faculty and we could make it the final exam in the course. This would remove the option for the faculty in that it must be administered, students must take a final in this course (that was a pre-defined requisite of the course) and we could implement the outcomes within the final exam. Success!!!
- The next step was what to measure. The CIS105 and BPC110 course competencies were very clear; however, these included areas that were beyond what we needed to include in this assessment. A taskforce was created and together we developed a final exam that not only assisted us in completing an efficient, effective and results oriented course-level assessment, it also would be able to deliver the data needed for the college-wide outcome for technology literacy as well. Level 1 assessed students on the introductory level of Word, PowerPoint, Excel and Access. This deals with document and file creation, formulas, presentation basics, etc. This level also included students from BPC110. Levels 2 and 3 only include CIS105. Level 2 added the file and folder management skills as well as desktop and operating system manipulation. Level 3 went into the more advanced levels of all 4 applications to include PivotTables, Indexing, Macros and VBA. Now many of you may ask, why are you teaching this level in an introductory course of CIS. Well, we ask ourselves that as well.We are located within the Maricopa District and there are 10 colleges within this district. Our courses are in a district bank and competencies cannot be changed within an individual campus without the competencies being changed at the district level; therefore changing the course for all 10 colleges. We have an IC (instructional council) with members from all of the campuses and mutually decide on the competencies and they also have to be approved by the state universities for them to accept this course as a transfer. The largest college, ASU, recently decided students needed to have advanced Access and Excel skills and in order to meet the articulation, we needed to change the course.
- Once we decided what we were going to measure, we needed to decide how we were going to measure it. At the college-wide outcome level, CWAAC decided they wanted to focus on the ability to perform the task. At the course-level, we wanted to make sure they could perform the task, critically think through a problem and find a resolution by choosing the correct technology and also be able to define and recognize computer concepts.When GCC first ran this assessment, it was in 2008. We were using SAM at the time and they had just rolled over to SAM 2010. I will admit, it was extremely rocky and the first assessment was a bit challenging to say the least. If anyone went for a ride with SAM when it transitioned from 2003 to 2010, it was pretty painful. Because we were gearing up for this assessment, we needed to hang tight. The plan was to use the Frequency Exams and pull the exams from BPC110 and CIS105. Because of the issues, the instructors of BPC110 quit using SAM altogether so they were not included in the cohort. In addition, there were only a small portion of the students who completed the exam so we had an extremely small cohort. Out of frustration in both the student and faculty groups, we transitioned to MyITLab. This tool did provide for a smoother environment for our students and faculty, however, not a friendly move for our IT staff. The necessity of having the large files loaded locally on the computers was not conducive in our environment. The reports were not easy to manipulate for me to pull data across the curriculum and this was an additional challenge. We moved back to SAM2010 and are now able to achieve all of our goals with one tool and both groups are satisfied with the transition.
- At the end of the fall semester, I run a frequency analysis report in SAM on the Final Exam. We use one common final exam and it is scheduled in all sections. The same procedure is practiced in BPC110. Because this exam also contains information that is used strictly for the Course-Level assessment, there is some scrubbing of the data to be done, but not much. I basically export it out of SAM as an Excel spreadsheet, scrub the data there and then import it into an Access Database so I can then query the data as I need. Results are broken down as follows:By sectionBy instructorTime of day the course is offeredYou name it and we can query and report. This does take a little time to set up initially, but once it is completed, it is fairly seamless.
- A sample of the frequency analysis report. This is combining all students who completed the final exam. I have also provided you with a copy of our 2008 Technology Literacy Report as well as the 2011. In our last HLC visit, (April 2012) we were commended for our improvements in college-wide and course-level assessment and our use of course-level assessment to measure college-wide goals. We also use course-level assessment to measure college-wide outcomes in our Oral Communications assessment.
- We have conducted two college-wide assessments of Technology Literacy to date: 2008 and 2011. We complete a full report and an executive summary for each. Here is the link for the reports for you to view at your leisure.
- Now, does anyone have any questions?Please feel free to contact me at rachelle.hall@gccaz.edu or call my office at 623-845-3231Thank you for your time. I know you have a choice of many sessions and I am honored you spent your time with me.