Making the Argument for Learning Science in Informal Environments
Level: Intermediate
Demonstrating evidence of learning is becoming critical for museums in securing funding. Yet most evaluation measures do not reflect what goes on in informal environments. The National Academy’s recently summarized research on informal science learning, helping museum professionals make the case for programs. We will review that report (Martin, Arizona Science Center, was a co-author); findings from a national staff development project headed by the Phoenix Zoo (Hebert); and, findings from a visitor survey about conservation learning, conducted at the Arizona-Sonora Desert Museum (Colodner).
The report is important because it broadens the definition of science learning to include motivation, interest, and identity. The Phoenix Zoo project looked at how different institutions supported staff to integrate mathematics into interpretation and how differences in motivation for participating in training influenced staff learning. The Desert Museum conducted a survey on learning about conservation through interactions with docents, looking at changes in visitor knowledge and intentions. Their results allowed them to reflect on the types of interactions they are facilitating.
Discussion will highlight whether the ideas apply to learning in other kinds of museums. Naylor (Arizona Department of Education) represents the stakeholder voice, justifying field trips and other informal programs as valid educational experiences.
Chair: Mary Lou Naylor, Education Program Specialist, Arizona Department of Education
Panelists: Laura Martin, Director of Science Interpretation, Arizona Science Center; Gabrielle Hebert, Director of Visitor Experiences, The Phoenix Zoo; Debra Colodner, Director of Education, Arizona Sonora Desert Museum
Phoenix Zoo provided expertise and experience with conservation education in a zoo context TERC provided expertise and experience with mathematical activity development and teacher professional development TERC is a not-for-profit education research and development organization dedicated to improving mathematics, science, and technology teaching and learning. Project funded by Institute for Museum and Library Services 21 st Century Professionals Initiative
From the workshop description: The Phoenix Zoo project looked at how different institutions supported staff to integrate mathematics into interpretation and how differences in motivation for participating in training influenced staff learning.
Assumptions: Expand the use of living collections as teaching tools for concepts of conservation mathematical explorations could support visitor engagement with ideas of conservation zoos and aquariums can be a motivating context in which to highlight technical work and demonstrate how mathematics is applied in the real world mathematical reasoning about animal identification and behavior can support people’s understanding and caring about animals activities that promote curiosity and interest among members of the public and invite them to take part in simulated conservation work. involvement in these activities could lead to concern and empathy about animals and even, in turn, lead to further interest in the math and science of animals
Workshop Goals: Educate staff about the connections between math and conservation education that they could use themselves to enhance their institution’s programming The workshop structure should support participants first as math learners, then as math “facilitators.” This approach gives participants a deeper understanding of mathematical content, as well as an appreciation of the challenges involved in mathematical reasoning. In order to achieve participants’ abilities to use math and to see its relationship to the work of the institution, we would need to address their confidence and knowledge levels. In MiZA workshops, participants did each activity themselves before discussing how they might adapt it for their own context. Workshop activities should be presented in a way that reflects as much as possible the context in which they will be implemented. In MiZA workshops, data and measurement activities were introduced in the zoo and aquarium context, so that participants were thinking from the start how these math topics might be useful in their own institutions. Workshops should be designed to explicitly help participants implement in their own ways what they’ve experienced in the workshop. In MiZA workshops, we made clear at the beginning of the workshop that participants should be thinking all along about how these experiences could fit into their home institutions. We then provided supports throughout the workshop – discussion and planning time – so that participants could begin to anticipate the opportunities and challenges that awaited them.
Curator paper covers evolution of the workshop model. this relates the final product: The workshop began with an introduction of the project and its goals, a description of mathematical thinking, how it progresses as a child develops and why it is important, and an icebreaker data game. Most participants thought of math as addition and multiplication so began the workshop with a limited scope of the possibilities for using it in programming: math includes logical processing, critical thinking, working with data, measurement, etc. Description of mathematical thinking progression helped participants to start thinking about what to do with different audiences, and also provided more details for the broader definition of math Participants learned about how U.S. students rate poorly in international math testing and how improved math skills can improve success in every day life For each of the three math activities (described below) in the workshop, the following sequence of events occurred: Participants played the game/did the activity themselves, in order to have the experience of being a mathematics user. The whole group discussed their experiences doing the activity, the mathematical content and how they thought such an activity might fit into their institutional context. Discussion also emphasized the math, science, and conservation connections for each activity, providing a more concrete understanding for participants Small teams briefly considered how to implement the activity in their setting, including filling out a planning guide Final discussion included teams choosing one of the three activities to implement and considering in more detail how to follow these plans when they returned to their place of work. Each team filled out a final planning form and handed it in to the workshop leaders. Follow up by MiZA staff, designed to support participants as well as collect information about how they were using the ideas, was discussed.
Action plan requested a timeframe for which the participant wanted staff to follow –up. The goal was to follow up after they had begun their implementation so that we could talk through challenges and assistance they needed. Follow-up also served as a motivating factor to actually implement an activity.
Guess My Animal 20 questions format “ chooser” secretly chooses a card and others ask yes/no questions to determine which card it is Questions are based upon data on the cards sort and categorize data through numerical reasoning and logic example: using greater than and less than judgments formulating question based on data categories eliminating data that do not fit certain criteria Where’s the Math in “Guess My Animal?” This activity develops people’s skills in using logic, as well as in sequencing and classifying data. Working with both numerical and categorical data, people work on these skills: Putting numerical data in order. Understanding and accurately using concepts of “more than”, “less than”,”the same as” and “at least.” Formulating “yes/no” questions that are based on observable properties of the data set (rather than on information that is not provided). Formulating “yes/no” questions that will systematically eliminate a good portion of possibilities (versus formulating a question that will eliminate one or two possibilities.) Ultimately, a goal is to formulate questions that will eliminate a good portion of the possibilities each time. Keeping track of which cards have been eliminated and which questions have been asked. Science Connections for “Guess My Animal” In general this activity helps promote observation skills by asking the participants to look at details when formulating their questions, and it helps promote greater natural history knowledge by introducing the participants to specific facts. In addition, this activity can be designed to meet a number of science-related goals by changing the categories and types of animals. Here are some examples of goals that could be accomplished: Comparing animal adaptations by choosing categories such as locomotion type, diet, predator/prey, etc. Exploring biodiversity by choosing species that represent a diverse family (such as all the penguin species in the world, or all of the beetles found in the Sonoran Desert). Exploring an ecosystem by choosing a variety of animals from one ecosystem (such as the lower Colorado River basin). Conservation Connections for “Guess My Animal” The important links to conservation of this activity can be created through the conversations that happen while the game is taking place. With these cards you can: Develop emotional connections by using personal information about individual animals. Develop empathy for un-charismatic animals by introducing participants to exciting facts about the species. Discuss how observation skills enable scientists to use data to monitor the health of populations of ecosystems: for example, keeping track of the numbers and types of migratory birds moving through southern Arizona is important to monitor the quality of the habitat, also changes in average size of a population may indicate changes in overall health or quality of the habitat they live in. Discuss the diversity of a habitat and the reasons it supports the type of life it does.
Be An Animal Scientist observing animals and recorded the behavior paired with simulated experience in the classroom where students act like animals while other students record their behavior understand how to collect, record, and interpret observational data using columns and roas as well as use compiled data to create a bar graph. Where’s the Math in “Be An Animal Scientist?” This activity develops people’s skills in collecting, organizing, representing, describing, comparing, and telling the story of data sets. The mathematical skills they work on include the following: Gathering data in a systematic way, making use of charts and time sampling; Accurately categorizing data; Accurately counting data in different categories; Understanding how their own data relate to a cumulative data set, and that they can put more confidence in the cumulative data set. Creating and making use of data representations as a way to describe animal behavior. This includes determining which behaviors animals exhibit more and less frequently, and what is typical for the animal. Using two sets of data to compare animal behaviors at different points in time, in different spaces, or under different conditions. Science Connections for “Be An Animal Scientist” This activity develops various skills and processes that scientists use in order to conduct important research. The skills they work on include the following: Observing animal behavior both live and in photographs. Distinguishing and tracking individual animals. Understanding and distinguishing a set of behaviors. Understanding data collection procedures. Collecting behavioral data and using a data sheet. Representing and analyzing behavioral data. Forming a hypothesis related to a scientific question. Conservation Connections for “Be An Animal Scientist” Scientific research is an essential part of conservation; in order to protect a species you need to have information about it. Scientists use behavior information to understand how the animal uses its environment and what the animal’s basic needs are. This activity can be used to discuss the role of the scientist in the conservation process. In addition the activity can also: Build stronger connections to the observed animals by helping participants see them as unique individuals. Help participants see the links between a species and the environment it lives in by relating its behaviors to its environment.
Finder/Keepers Participants go fish (using paper fish with metal attached and a dowel rod with a magnet on the end of a string) measure fish record on data chart and make a decision about whether to keep or throw back Demonstrates how linear measurement is used in fish management For some species fish need to be a certain size in order to be kept – sometimes a minimum, sometimes a maximum, sometimes within a range Where’s the Math in “Measuring Fish”? This activity develops children’s skills in linear measurement in several ways: It gives them practice in determining what attributes to measure. They practice measurement techniques, including starting at the beginning, ending at the end, measuring in a straight line, leaving no gaps or overlaps, and keeping track of the number of units. They get practice measuring objects that are “in between” whole numbers, and begin understanding that the space between whole numbers represents an actual quantity. It helps them understand the idea of ‘reliability’, namely that measurements of the same lengths should be the same when they are measured by different people or when the measurements are repeated. They begin to understand that one needs to identify and record units (such as inches or centimeters) when measuring. Science Connections for “Measuring Fish” This activity helps participants: Understand how scientists use measurement in their work; Scientists need to know how to measure the body length of an animal to judge the age and health of an individual animal. Some animals can be identified by comparing the length of the individual to the average length for that species. Practice taking scientific measurements. Understand that scientists standardize their measurements across the field (for example, fish lengths are measured from the tip of the nose to the longest point on the rear fin). Conservation Connections for “Measuring Fish” Scientists use fish body measurements to categorize the animals by age. A fish can only be caught and removed from a lake when it fits into a certain category. Juveniles and fish of good breeding age can’t be taken so that the population can remain healthy. If you go fishing, you need to be able to measure the fish to determine which category it fits in.
Customizing the Activities Customization of the activities was a key success factor for the workshop. This slide focuses on how we allowed for ease of customization for the participants. pre-workshop planning allowed us to adjust the model we were using to the workshop site so the participants could see first-hand how customization worked. Discussion centered on how to modify for different age levels and audience types How to focus on different animals specific to participant collections planning guides and planning time allowed participants to think through how to customize the activity follow-up from MiZA staff helped them work through issues, questions, challenges, etc.
Evaluation Conducted by Garibay Group designed according to Logic Model as specified by IMLS Three main goals guided the project and evaluation; Confidence: Workshop participants will increase their confidence in their ability to incorporate math into their interpretive activities Implementation: Workshop participants will successfully adapt one of the workshop math activities to work with their own collections. Connections: Workshop participants will see connections among the math activities, science and conservation.
Methods: Mixed method approach (Green and Caracelli, 2002; Johnson & Turner, 2002) quantitative and qualitative data pre-workshop survey administered at the beginning of the workshop background information on respondent (job areas/departments, perceived confidence in integrating math in their work, and motivations for attending workshop) post-workshop survey administered at the end of the workshop (short-term outcomes) overall satisfaction and perceived learning and usefulness of the material in-depth phone interviews conducted post-workshop two rounds; 14 interviews total 5 organizations at the beginning to help shape adjustments to the project 9 organizations at the end of the project deeper understanding of participants’ perspectives and experience; gain insight into the varied organizational context n which activities were implemented and the ways in which such contexts influeneced outcome at a particular institution participants purposefully selected to ensure data from a wide range of organizations final online survey to all participants administered at the end of the project, after project staff follow-up (long-term) longer term outcomes 72% of organizations responded; 55% of individual participants debrief data from follow-up calls between MiZA staff and participants primary goal of phone calls to provide support and assistance track progress, identify barriers, document any unanticipated issues staff followed set protocol and debriefing process developed by the evaluation team workshop observations
Short Term results were measured immediately following the workshop using the paper survey: Math knowledge and confidence increased (average rating 3.34/6 to 5.28/6) (figure 2, page 8) Participants gained concrete, easy to incorporate math concepts and creative new ways to think about math After coding open-ended responses to the question of what participants learned from the workshop, the following clusters emerged: Gained ideas for incorporating math concepts/activities into a range of programming and for different potential audiences (N=111). Expanded concepts about what math is and how it can be applied (N=47). Found that math concepts are easy to incorporate (N=24). Gained ideas about math activities that are animal-specific (N=20).
60% of participants were able to implement one or more activities Which activities did they implement? Be an Animal Scientist – 45% Guess My Animal – 38% Finders/Keepers – 17% Most prominent reason for choosing one over the other was whether an activity fit well with and could be easily integrated into existing programming. Those who did implement activities identified the following challenges with the process: a) difficulty motivating other staff members; b) time involved in preparing/adapting activities; c) finding ways to integrate an activity into an already full program; d) uncertainty about how to adapt an activity for a specific age group.
for those who didn’t implement an activity they cited the following reasons a) not having enough time (70%), b) having a curriculum in place and being unable to add anything new to it (17%), c) that the activities did not fit with their organization’s goals (17%).
Professional Impact The overall project experience caused an increase in math confidence 22% said the most significant concept they took from MiZA was that the project changed their previous perceptions of what constituted math and provided them with new ways of approaching the issue of math implementation in programming math could be fun and subtle implemented in a creative way adapted to a variety of topics part of everything majority of respondents said that participating in MiZA increased their understanding of math, data, measurement, and logic. While not statistically significant, slightly more respondents believed their understanding of data, measurement, and logic increased more than of math in general
These results indicate participants agreeing to statements provided to them: 79% indicated that the project increased their understanding of how conservation and math related to each other 80% said implementing activities has a positive influence on engaging their audiences more deeply 82% indicated that implementing activities positively impacted their abilities to integrate math concepts and ideas into their own work. 80% stated that MiZA was professionally valuable while 18% indicated it was only “somewhat valuable” those who implemented activities rated the overall value higher than those who didn’t implement activities 87% would be likely to participate in future projects using math to help communicate conservation messages those who implemented activities were more likely to predict future participation 85% would recommend MiZA to their colleagues those who implemented the activities were more likely to recommend the workshop
Individual Factors that Influenced Outcomes Goal-oriented more likely to implement activities consistent with activity theory framework (Davydov, 1999) which predicts that someone’s mental goal will influence the meaning they make of a learning activity or event They had a specific reason for attending the workshop – looking for additional activities, direction from organization to implement math, knew which program they wanted to enhance Generalists were less successful in implementation – they just wanted to learn more about math or add to their repertoire in general speed and ease of customization meant more likely to implement We discussed how we set this up on an earlier slide Generalists experienced more barriers to implementation then those who were goal oriented Diagram on next slide
This chart shows the progression an individual goes through in order to implement an activity. Could be used to help shape future workshops. Goal-oriented individuals came to the workshop looking to find solutions for previously identified organizational or departmental goals. they may have a specific piece of curriculum or program that needs to be enhanced they may have been given an initiative to increase math activities for a particular audience Generalists came to the workshop with no specific goals, just additional knowledge to influence their work at some point in the future Both groups gained the same knowledge broadened conceptions of math concrete curriculum ideas understanding of incorporating math into programming During the workshop and afterwards, participants were asked to identify potential applications for what they learned goal-oriented could see a specific program or event that one of the activities could be integrated into generalists could see potential but didn’t have a specific program or event in mind Basic Barriers limited staff motivating/training staff time involved to adapt activity unclear how to adapt for specific age group fitting activity into fully developed program Generalists experienced additional barriers than those who were goal-oriented They struggled to identify a specific application for their knowledge and did not clearly see a fit between what they learned and their organizational goals As a result, generalists were much less likely to implement an activity and gain concrete, immediate value from the project Implications for the field may suggest that preparation should be done before beginning professional development workshops get institutional buy-in for sending the participant – how will this experience match goals guide participant to identify specific areas where tools will be needed most
From the interview data, features of the participating institutions themselves emerged as having an impact on the project outcomes (see Figure 2 ). Interestingly, by categorizing where the successful adopters worked, we found that small and large institutions afforded different advantages for adopting the new practices, with staff from both types being able to implement activities, but for different reasons.
The size of the institution affects the specialization of staff, numbers of programs, and communication among staff members In a small institution, staff need to be generalists and have the ability to impact programs and share new practices with the relatively few other staff members with whom they work. At larger institutions, staff tend to be more specialized, but often have more time to introduce new ideas and to design special programs to accommodate them. Literature about factors influencing change focuses more on formal educational institutions (e.g., Senge, Carlton-McCabe, Lucas, Smith, Dutton & Kleiner, 2000). The ways in which new practices are adopted in informal learning settings has not yet been studied in detail.
Implications for the Field Lessons from MiZA could shape success of future professional development for informal educators For example, several aspects of MiZA’s workshop structure contributed to its success. The most critical were: providing a set of activities that participants could take and readily adapt for their institution’s needs, putting structures in place to encourage and support participants in actually implementing activities. Helping participants identify and link activities to existing programming, addressing organizational structure issues, Future projects: including staff-driven evaluation of visitors’ experiences to help participants reflect on their “practices” can strengthen professional development initiatives at zoos and aquariums. Figuring out ongoing support for the incorporation of new educational practices is also something to consider in the future.
Laura Martin was at the Phoenix Zoo at the time but is now with the Arizona Science Center Andee, Tracey and Jan are all from TERC, IMLS – funder New England Aquarium and Henry Doorley provided content support for specific activities Garibay group - evaluator