This handout accompanies the slides on real time evaluation (http://bit.ly/1Py1ME4).

1. 1
Real Time Evaluation: Tips, Tricks, and Tools
of the Trade
Yuqi Wang
Wednesday,November11,2015
AEA 2015, Chicago,IL @yuqi_2w6
Survey
Creating real time surveys
 The short methodof surveyingin real time is to use software withthe abilitytosurveypeople
inreal time,andanalyze the results instantly.Oftentimes youcanembed the software in
webinarsand PowerPointpresentations. Some commonlyusedonline software are:
o www.GoToWebinar.com
o www.PollEverywhere.com
o www.Participoll.com
 The longmethod of surveyinginreal time isto use hard copiesof surveystocollectreal time
data, anduse Excel for data entryand analysis.Fora stepbystepprocessof how to do this,go
to thislink:http://bit.ly/1Mhs5Mg.
Tips and resources
 An Excel resource fordoingthe longmethod of surveyinginreal time iswww.excel-easy.com.
 Attentiontodetail andorganizationisimportant,especiallyforthe long method.Make sure all
your formulasfunctioncorrectlybefore dataentryday.
 Make sure the surveyyouare usinghasmostly,if notall,closed-endedquestions,like aLikert
Scale.Closed-endedquestionshave ashortanalysistime,andthe analysiswill be easyfor both
Excel andthe software todo.
 An AEA 2015 sessionon the topicyoucan go to is “GettingReal with Real Time Data
Collection” Thursday,November12, 3:00-4:30PM, Rooseveltroom, Ignite 31. Danielle
Toussaint,Ruthie Chang,andKristie Glatze of Hatchuel TabernikandAssociateswill reviewthe
evaluationof a2014 international techconference, andshow how the use of mixedmethods
helpedtoenhance andproduce memorable results.
H-form
Creating real time H-forms
Start withwritingthe questionortopicyouwantparticipantstoprovide theirfeedbackonat the topof
a blankpiece of paper.Draw a horizontal line inthe middle of the paper,andlabel itwiththe spectrum
of opinionsyouwantparticipantstoprovide feedbackon(e.g.stronglydisagree,disagree,neutral,
agree,andstronglydisagree).Leave enoughspace onthe leftandrightside of the paperto write “Why
do youdisagree?”and“Why doyou agree”,respectively.
Once everyone hasgottenachance to provide their feedback,the groupcanbegintodiscusswhatthey
shared,andstart to reach a group understandingaboutthe questionortopic.

2. 2
Real Time Evaluation: Tips, Tricks, and Tools
of the Trade
Yuqi Wang
Wednesday,November11,2015
AEA 2015, Chicago,IL @yuqi_2w6
Tips and resources
 Provide participantswithdotstickerstheycanplace anywhereonthe spectrumof opinions.You
can alsoprovide post-itnotes forparticipantsso theycanjotdowntheirreasoningbehindtheir
opinion.
 An AEA 2015 session onthe topicyoucan go to is“Supercharge Your Focus Groups!UsingAdult
FacilitationTechniquesto Capture rich Data and Ensuring Your Participants Have Fun inthe
Process!” Wednesday,November11,6:15-7:00PM, Grand Suite 2A/B. Lynne Franco of
EnCompassLLC will introduce atechnique andprocessthatinvolvesthe use of alarge stickywall
and index cards – toolsthat can draw out perspectivesandfostergroupprocessingdata.
Timeline
Creating real time timelines
Timelines canbe a good forparticipantstothinkabout whatworkedandwhat didn’t workfora
particulareventsotheyare able to applywhattheylearnedtofuture events.Butbefore creatinga
timeline, it’simportanttomake sure participantsare clearon whytheyare creatinga timeline,and
whatit isgoingto be usedfor.
Once you have a goodunderstandingof the timeline’s purposeandutility,one waytocreate a timeline
isto use blue painterstape,andlarge butcher-block paper.Use tape tocreate a single line onthe paper
– thiswill representthe pastand presentsectionsof the timeline.Use smallerpiecesof tape tocreate a
dottedline representing future.Have participantswriteonpost-itnotesmilestonesthathave occurred
withinthe spanof the timeline,andstickiton to the appropriate sections.Youcanalsohave
participantschoose afewmilestonestheylistedtoexplore the milestones’conditionsof successand
challenges.
Tips and resources
 You can begina timelinewhenaprojectstarts.Thisiscalleddoinga“pre-mortem”.Harvard
BusinessReviewhasagreat article on pre-mortems:http://bit.l y/1EMTn7a
 Angie Ficek wrote agreat AEA365 blog on creatingtimelineswithline charts:
http://bit.ly/1KGlUM8
 Color-codingpastandpresent eventsorgroupsof pastand present of events onyourtimeline
will be helpfulforyoutoorganize whathappenedinthe pastand present,and will be helpfulfor
youto spot anypatterns withinyourtimeline.
 An AEA 2015 sessiononthe topicyoucan go to is “PuttingData in Context:Timeliningfor
Evaluators” Friday, November13, 7:00-7:45AM, Burnham room. Kat Athanasiades,Smriti
Bajracharya,and Katherine Haughof InnovationNetworkwilldemonstrate how toincorporate a
timelineintoareport,use a timelinetotrack progressinternally,andutilize datavisualization
principlestocreate avisual timeline.