Slides from a presentation given at EuroMeduc (Second European congress on media literacy - Italy, Bellaria, 21-24 October 2009). This is the 'full version'; my actual presentation had to be trimmed down to 2/3 of the slides.

1. Evaluating media literacy as competences: what can we agree on? Pierre Fastrez Groupe de Recherche en Médiation des Savoirs Université catholique de Louvain, Belgium

2. Where do I stand? (a few oratoricalprecautions) Pierre Fastrez [GReMS/UCL] 2 What I intend to bring to the table a social/cognitive/communication scientist’s point of view a newcomer’s perspective EuroMeducBellaria – Oct. 22nd, 2009

3. A simple observation… Pierre Fastrez [GReMS/UCL] EuroMeduc Bellaria – Oct. 22nd, 2009 3 Media Education, a vivid field Media Literacy evaluation is still in its infancy still lacks systematic efforts to make it a coherent endeavour

4. Assessing ML : what priorities? Pierre Fastrez [GReMS/UCL] EuroMeduc Bellaria – Oct. 22nd, 2009 4 Agreement: Regarding theoretical definitions of Media Literacy Not on definitions and frameworks But rather on how these definitions and frameworks relate Regarding methodological principles for assessment

5. Outline Pierre Fastrez [GReMS/UCL] EuroMeduc Bellaria – Oct. 22nd, 2009 5 Assessing Media Literacy: what can we agree on? Theoretical issues Methodological issues A few proposals regarding Media Literacy competence assessment

6. Theoretical issues Pierre Fastrez [GReMS/UCL] EuroMeduc Bellaria – Oct. 22nd, 2009 6 Assessing ML requires an agreement on what it is Defining ML is dependent on Historical and cultural context Media landscape development Policies Epistemological views

7. Theoretical issues Pierre Fastrez [GReMS/UCL] EuroMeduc Bellaria – Oct. 22nd, 2009 7 Goal: identify common ground for Media Literacy definitions Independent of cultural contexts That can be specified and interpreted in these contexts Media Literacy lies in the individual’s competences not the media environment: media literacy is not equivalent to media education Outcome vs. process Evaluating the process is evaluating its efficiency with respect to the outcome Benchmarking outcomes is a sine qua non condition

8. Theoretical issues Pierre Fastrez [GReMS/UCL] 8 EuroMeduc Bellaria – Oct. 22nd, 2009

9. Theoretical issues Pierre Fastrez [GReMS/UCL] EuroMeduc Bellaria – Oct. 22nd, 2009 9 A fundamental hypothesis: Media Literacy is related to a number of generic skills E.g. media and communication technologies as mindtools These skills are generic in that they are not related to specific media (film, print, internet…) specific activity domains E.g. commercial information, product placement, minor protection, civic participation… A bi-lateral relationship (virtuous circle) These skills can be developed through specific media use and specific media education practices The more these skills are developed, the more they facilitate other media appropriation processes

10. Theoretical issues Pierre Fastrez [GReMS/UCL] EuroMeduc Bellaria – Oct. 22nd, 2009 10 Consequence of this hypothesis: we need to relate specific skills to their “common ground” E.g. “digital” skills are not of a different nature than “traditional media” skills E.g. Film studies vs. media education appreciating one’s media cultural heritage relies on the same skills as appreciating other cultures’ contribution to the media environment E.g. Understanding the semiotic specificities of a given media: Representing time in blogs and twitter (input flow and chronology) Representing time on the silver screen (editing and rhythm) E.g. Building typologies: reading the news and managing bookmarks E.g. Perceiving the producer’s intent: watching a documentary and moderating a forum

11. Methodological principles Pierre Fastrez [GReMS/UCL] EuroMeduc Bellaria – Oct. 22nd, 2009 11 Methodological issues regarding ML evaluation Definition of levels Competence and performance Assessment tools validation process

12. Levels of Media Literacy Pierre Fastrez [GReMS/UCL] 12 There are degrees of complexity in competences Operationalization: Levels (when it makes sense) Tools for researchers Not meant for unilateral ranking Which would assume one can weigh the relative importance of skills Rather, a multidimensional analysis tool EuroMeduc Bellaria – Oct. 22nd, 2009

13. Competence vs. performance Pierre Fastrez [GReMS/UCL] EuroMeduc Bellaria – Oct. 22nd, 2009 13 Competences are virtual, and can only be observed when actualized in practical and concrete tasks Evaluating competence through performance Consequences: The production vs. reception distinction should be set (for now) at the performance level (vs. as categories of skills) Performances are connected to different competences E. g. moderating a forum: social and cognitive competences

14. From concept to indicators Pierre Fastrez [GReMS/UCL] EuroMeduc Bellaria – Oct. 22nd, 2009 14

15. From concept to indicators Pierre Fastrez [GReMS/UCL] EuroMeduc Bellaria – Oct. 22nd, 2009 15

16. 2. An attempt at breaking down the Media Literacy concept Cognitive vs. social(vs. technical) skills Semio-cognitive vs. socio-cognitive skills Conceptual distinctions independent of specific activity fields Pierre Fastrez [GReMS/UCL] 16 EuroMeducBellaria – Oct. 22nd, 2009

17. Cognitive vs. social(vs. technical) skills Semio-cognitive vs. socio-cognitive skills Conceptual distinctions independent of specific activity fields Pierre Fastrez [GReMS/UCL] 17 EuroMeducBellaria – Oct. 22nd, 2009 2. An attempt at breaking down the Media Literacy concept

18. 2. An attempt at breaking down the Media Literacy concept Cognitive vs. social(vs. technical) skills Semio-cognitive vs. socio-cognitive skills Conceptual distinctions independent of specific activity fields Pierre Fastrez [GReMS/UCL] 18 EuroMeducBellaria – Oct. 22nd, 2009

19. Semio-cognitive skills Pierre Fastrez [GReMS/UCL] EuroMeducBellaria – Oct. 22nd, 2009 19 Dimensions (inductive grouping) : Reading (decoding / synthesizing / evaluating) Writing (coding, creating, producing) Navigating (searching, exploring, managing uncertainty) Organizing (keeping, sorting, annotating) Using responsibly (knowledge of use regulation, stakes and principles)

20. Socio-cognitive skills Pierre Fastrez [GReMS/UCL] EuroMeduc Bellaria – Oct. 22nd, 2009 20 Pragmatic socialization: The extent to which an individual poses communicational acts that establish socialrelationships in more or less extended social spaces. Informational socialization: The extent to which each individual extends and diversifies their relationships as a receiver, a relay, or a producerof information within social space.

21. Socio-cognitive skills Pierre Fastrez [GReMS/UCL] EuroMeduc Bellaria – Oct. 22nd, 2009 21 Social levels logic (criteria, not thresholds) From contact and participation, to networking and active presence, to cooperative networking Diversity (from tribe to society) Creativity and involvement (from follower to autonomous agent) Decentration (from egocentric to allocentric) Dimensions: Reception: media selection and exploration Expression: communicative acts, idioms and receivers Pragmatics: relational contexts and interactions Ethics: perceived rules and lines of conduct

22. Socio-cognitive skills Are informational socialization skills only socio-cognitive? Pierre Fastrez [GReMS/UCL] 22 EuroMeduc Bellaria – Oct. 22nd, 2009

23. Theoretical questions left open Pierre Fastrez [GReMS/UCL] EuroMeducBellaria – Oct. 22nd, 2009 23 How are Media Literacy skills correlated? From an analytical to a systemic point of view: connecting after having distinguished E.g. does critical thinking development facilitate creativity? Ethics as a socio-cognitive skill and responsible use as a semio-cognitive skill … Need to work on the structure of ML as a concept federating a set of competences Not just conceptually, but empirically

24. Back to methodology: validation Pierre Fastrez [GReMS/UCL] EuroMeducBellaria – Oct. 22nd, 2009 24 Building relevant ML assessment tools takes time and a lot of research ML dimensions (skills) can (and must) be translated into multiple indicators related to different media / activity domains Validation requires Multiple indicators for the same skill Internal consistency evaluation

25. Back to methodology: validation Pierre Fastrez [GReMS/UCL] EuroMeducBellaria – Oct. 22nd, 2009 25 Data / indicators based on (roughly in order of preference) Observation of specific tasks Experiments Participant observation Analysis of traces Personal environments / personal information spaces Including statistics Interviews and surveys Self-assessment tools Triangulation: a requirement for validation

26. Conclusions and implications Pierre Fastrez [GReMS/UCL] 26 Working on competence definitions and their operationalization is a requirement for ML assessment. Constructing ML assessment tools are a means for setting objectives for media educators bridging the gap between asymptotic goals (“be an active citizen and media user”) and practical know-hows (“know how to frame a close shot”) EuroMeduc Bellaria – Oct. 22nd, 2009

27. Thank you for your attention Questions? Comments? pierre.fastrez@uclouvain.be