Unleash 2011
•Transferir como PPT, PDF•
0 gostou•673 visualizações
Presentatie gebruikt tijdens de Unleash 2011 conferentie in Madison, Wisconsin
Recomendados
Mais conteúdo relacionado
Mais procurados
Mais procurados (20)
Semelhante a Unleash 2011
Semelhante a Unleash 2011 (20)
Mais de Pierre Gorissen
Mais de Pierre Gorissen (20)
Último
Último (20)
Unleash 2011
- 1. Do You Know What Your Students Do with Your Lecture Recordings? May 17, 2011 Pierre Gorissen Fontys University of Applied Sciences Twitter: @PeterMcAllister
- 3. Quick show of hands…
- 4. How it all started for us..
- 5. Reports… Viewers per slide Viewers per slide Viewers per slide
- 16. How much of a lecture do you watch?
- 17. Would students really lie?
- 21. Mediasite
- 26. Server logs
- 27. Combine data
- 32. Now we are getting somewhere…
- 34. Heatmap Student 1 Student 188 T = 45 min T = 0 min
- 37. Final comment from one of the interviews: A recording of a bad lecture can actually improve that lecture... … the recording can be replayed at high speed!
Notas do Editor
- Hello and welcome to this session. Both online and here. My name is Pierre Gorissen from the Fontys University of Applied Sciences.
- Just to give you a quick overview of where I’m from: here is Madion and over there is Eindhoven where both the Fontys University of Applied Sciences and the Eindhoven University of Technology are located. Both universities participate in the research I’m going to talk to you about.
- Who is actually actively using the reports that you can create with Mediasite at the moment? For the online audience, I can’t get a show of hands from your but if you do use the reports, let me know through the comments option, I would love to hear about them. => Pick 2 from the audience.
- This is how it started for us. And I know that showing a Echo 360 slide during a Mediastie user conference might be dangerous and probably not good for my Mediasite Mojo or the chances of winning big prices, but that is the system we started using. And back then it had no reporting options. So we had to create something of our own.
- Because of the way the Echo360 system worked back then, it was relatively easy for us to create reports on the number of views of viewers per slide. Here you see three examples of reports that we created. Each for a different recording/presentation. In the first report you see a sharp decline in the number of viewers after the second slide. Which proved once more that just measuring “views” is kind of pointless. Views only tells you how many people start watching, not how quickly they stop watching. The second one is for a shorter presentation with less slides. And it does have a less sharp decline in viewers. The third one also is an interesting one, because it shows some sort of navigation within the recording. Sure, still the sharp decline after the first two slides, but in the middle and the end there are still some viewers left. We couldn’t explain this well enough, so one thing lead to another…
- And two years ago I was allowed to start a PhD Research called: Facilitating the use of recorded lectures with expert and social tagging The goal of all that is what you see as the third question. I’m going to talk to you about the first two questions.
- One of the things you have to know about a PhD research, I think it is the same everywhere, not just in the Netherlands. It is that you have to focus. My supervisor once said: you should have to be able to keep it small enough to write on a back of poststamp. You might want to try a postcard, but you get the idea. You can’t change the world.
- One thing that lead to is that we focused on recorded lectures only. So only recordings of live lectures. Not of students giving presentations, not of graduation ceremonies. Not small clips that staff recorded etc. That doesn’t mean those are not important, but given previous research, we expect students to use those differently.
- And we had more than enough to work with if we just focused on recorded lectures. The Eindhoven University has 5 Mediasite recording sets that record about 2,000 recordings per year each about 45 minutes long. They use student operators for the camera’s and though that is cheaper than regular staff, creating these 2,000 recordings per year isn’t cheap. Another important issue for them was that because they now record to the max of their capacity, they want to be sure that they record the right lectures.
- Bringing the research back to a single question, it looks something like this.
- We came up with a two step approach: let’s ask them and if that doesn’t work, see if we can measure their use. I’ll start with step 1 and come back to step 2 later.
- We did a survey amongs 1,122 students from both universitites. Why the differences in numbers: unlike a lot of surveys I’ve seen we didn’t just sent out the survey to any or all students that may or may not have viewed a recording. We selected 7 courses that had been recorded during the semester that just ended at the moment the survey was scheduled. Now because Fontys had only just started recording full lecture series at that point, we selected 6 courses from the Eindhoven University and only one from Fontys. We sent the students a personalized mail asking them specifically about that single course. I think it greatly improved our response rate. But it also makes it possible to compare their answer with results from step 2. We also did follow up interviews . Now we asked students at the last page of the online survey if we could contact them for follow up questions. A total of 120 checked YES there. When we then afterwards mailed them to ask if they still wanted to participate, only 14 said YES again.
- Probably not that surprising. Bandwidth is not a problem, travel distance isn’t either, so students watch from home, no need to do that at the university.
- If they miss the occasional lecture they watch (70%) and some use it as a replacement for live attendance all together. But not to many. Only 13% indicated that they prefer the recordings over the live lecture. And there are a lot of other reasons for students to miss a lecture.
- Now this also a reassuring result: students still prefer live lectures and if they don’t feel that they missed anything, they are way to busy with other things to watch recordings of lectures.
- But this question showed that we can’t take all their answers at face value.
- Would House be right? Do even students lie?
- The interviews do suggests it is true for some, but we don’t think it is likely that it is the case for such a large group.
- Good thing we had sort of counted on that and had a second option planned: try to measure the use.
- Remember this was the question. Let’s start with checking to see if Mediasite can provide us with the answers.
- So we checked out all the reporting options that Mediasite has.
- Server activity per date gives a nice overview of all views, but we needed info about one specific course.
- Total views per presentation gives all presentations for a course if filtered correctly, but we new just views wouldn’t be enough, we needed more detail.
- Another one: presentation summary with all views for a presentation over time. Still not quite what we needed.
- So, the question wasn’t that simple. We weren’t able to answer it with the existing reports. Now what? We had to find more info. One place to look for that info were the server logs created by the Microsoft Windows Mediaserver.
- The Microsoft Windows Media server is the streaming server that streams the video that is part of a recording in Mediasite. For each snippet of video that it sends to the viewer, it creates a logrow in a text file. The red part is one row.
- We created a setup where I received a backup of the Mediasite database from the Eindhoven University of Technology and a copy of all the logs. At the moment they have an archive of logs that consists of about 2 million rows. I imported both the database backup and the log files into a SQL Server database so I could run queries against them. For that I needed a decent workstation because there was a lot of data to crunch.
- And it wasn’t just the workstation that had a problem with the amount of data.
- Filter! If you start building reports, be it custom reports or reports in Mediasite, define what it is you are interested in. In our case. Your needs might be different, but make sure you don’t mix groups of users or recordings that you shouldn’t mix because of different viewing behaviours.
- Another thing we did was define something that we called a learner session. Mediasite is focused on recordings. But think about this scenario: a student is studying for an test for a course. For that course there are a number of recordings, could be as many as 30-40 per course. So that student could start by viewing a part of one recording, then maybe practice some assignments, read in one of the books for the course, then watch another part of a different recording and so on. We wanted to be able to identify that as one Learner Session. We also knew we weren’t interested in very short Learner Sessions. Because we expected that those we students that we just checking out recordings, instead of actually using them to study. And we had a lot of short learners session.
- So after you define what you’re interested in, it is time to clean up the data.
- The first, week 43 of 2009 (200943 - ①) coincides with the written test for C01P01, the first part of the course. The second, week 46 of 2009 (200946 - ②) is the week that the assignment for the first part of the course needed to be completed. The third peak in the graph is the biggest of them all during week 4 of 2010 (201004 - ③). This is the week leading up to the laptop test of the second part (C01P02) of the course. The last peak can be found in week 15 of 2010 (201015 - ④) which is before the second opportunity for both the written test for C01P01 and the laptop test for C01P02
- Skip…
- We don’t have the answers here yet. The log data doesn’t have the info we need to reconstruct the navigation. We are looking into extending the tracking in Mediasite to also include navigation and another possibility would be to combine our data with the log data available in e.g. Moodle. If a students views an assignment in Moodle and then views a recording, we have a better idea of why they started watching.