AoM 2013 proposal for a panel symposium on measuring the impact of entrepreneurship training - panelists from Twente, Chalmers, Aalto, Malaysia, Danish Foundation for Entrepreneurship and PACE plus top global trainer from EO. The first 3 programs are also among the very best in using students for technology commercialization.

1. Academy of Management Panel Symposium Proposal #15313
Academy of Management Panel Symposium Proposal
title:
Does Entrepreneurship Teaching/Training Actually Work?
A Symposium on Developing Metrics
Sponsors:
Lead: ENT Division
Also relevant: Teaching (TTC), Practice (PTC), TIM
Chair/organizer: Norris Krueger
Participating countries/programs: [I = invited]
Finland: Aalto University
Sweden: Chalmers University of Technology
Global: Entrepreneurs Organization (EO):
Netherlands: University of Twente
Denmark: Danish Foundation for Entrepreneurship
Denmark/EU: PACE/Aarhus University
Malaysia/USA: Stevens Institute of Technology/UKM
Discussant/Provocateur: Jon Potter, OECD Office of Entrepreneurship [I]
Discussant/Provocateur: Thom Ruhe, Ewing Marion Kauffman Foundation [I]
Discussant/Provocateur: Dianne HB Welsh, UNC-Greensboro, Technology Transfer
Society
Abstract
Does entrepreneurship education work? More important, how can we make it work? And how
can we measure the impact? This symposium focuses on measuring rigorously the impact of
entrepreneurial education in ways that we have rarely seen in single papers, let alone bringing
together some of the best programs in the world. These speakers are all from deeply experiential
programs and almost all from programs whose experiential activities center intensively around
technology commercialization. If scholars and educators want to know how to “move the needle”
1

2. Academy of Management Panel Symposium Proposal #15313
in turning ideas into reality, this panel will show both a set of programs doing enviable work and
some fascinating, robust tools for measuring the impact of experiential entrepreneurship
education.
Creating viable new businesses AND creating deep entrepreneurial thinkers? These
panelists will show you the state of the art of what we know about doing that and the immense
potential for future research on how do we grow the expert entrepreneurial mindset and measure
that growth.
2

3. Academy of Management Panel Symposium Proposal #15313
Overview
Research into entrepreneurial learning and entrepreneurial education has increasingly
recognized that to be effective requires transformative learning at a very deep level. The
literature on how we learn to become experts (e.g., Gladwell’s recent book, Outliers) reinforces
that one cannot learn to think like an expert without dramatic changes in how we organize what
we know. Knowing more is simply not enough; experts organize their knowledge far differently,
often in ways that are clearly not obvious to novices. Entrepreneurship education and training is
no different.
Why Assess Impact? In a world that increasingly recognizes the need for increases in
entrepreneurial thinking to support economic growth, it seems imperative that we identify what
methods are most effective at this transformational learning. It is equally imperative that we
identify what methods are less effective, even counterproductive. We propose here to integrate
existing theoretical and especially empirical field work to help us to (a) understand better why
and how our best pedagogies work, (b) why and how other pedagogies do not, and (c) create
mechanisms to allow us to map pedagogical methods to corresponding deep cognitive changes.
But we need good metrics.
The implications from great metrics? Imagine the ability to match pedagogical
exercises to learning needs –at a deep transformative level. If education research in general is
any predictor, we will eventually be able to not only identify which of our tools serve to make
which specific cognitive changes. For example, if trainees or students are weak at counterfactual
thinking (per Gaglio) what exercises are most effective at providing the needed cognitive scripts
and maps.
We all want to believe that entrepreneurship teaching and training matters. Matters
deeply. However, metrics have been in short supply, especially when we move past tests of
knowledge and skills. Several programs around the globe are tackling this with rigorous research
projects that simultaneously provide great practical value. We will be able to improve
entrepreneurial training but we first need to take a rigorous look at the cognitive impact of
entrepreneurship training methods. This PDW begins that effort in earnest with four relatively
new projects from the Netherlands (with tech entrepreneurs in an incubator), Denmark (youth
3

4. Academy of Management Panel Symposium Proposal #15313
entrepreneurs), Sweden (technology commercialization students) and globally (the renowned
Entrepreneurs Organization's peer learning groups.
We offer here short examples of the research projects attached to each initiative, however,
the PDW presentations will focus on these as opportunities:
Opportunities to identify tools that the audience can take home
Opportunities for the audience to offer advice to the participants
Opportunities to identify additional collaborations and possible extensions.
To this end, we have brought together several of the best initiatives existing today from well-
established programs like Chalmers to a brand-new project in Malaysia. We have invited expert
discussants from OECD and the Kauffman Foundation.
We fully intend for this to become a community of practice – we also want to learn what others
are doing as well as share the very best of what is out there.
4

5. Academy of Management Panel Symposium Proposal #15313
Sponsor Rationales
Why MED?
Does entrepreneurship education work? More important, how can we make it work? And how
can we measure the inpact? This symposium focuses on measuring rigorously the impact of
entrepreneurial education in ways that we have rarely seen in single papers, let alone bringing
together some of the best programs in the world. These speakers are all from deeply experiential
programs and almost all from programs whose experiential activities center intensively around
technology commercialization. If scholars and educators want to know how to “move the needle”
in turning ideas into reality, this panel will show both a set of programs doing enviable work and
some fascinating, robust tools for measuring the impact of experiential entrepreneurship
education.
Why ENT?
Does entrepreneurship education work? More important, how can we make it work? And how
can we measure the inpact? This symposium focuses on measuring rigorously the impact of
entrepreneurial education in ways that we have rarely seen in single papers, let alone bringing
together some of the best programs in the world. These speakers are all all from deeply
experiential programs and almost all from programs whose experiential activities center around
technology commercialization. If scholars and educators want to know how to “move the needle”
in turning ideas into reality, this panel will show both a set of programs doing enviable work and
some fascinating, robust tools for measuring the impact of experiential entrepreneurship
education.
Why TIM?
Technology commercialization-based entrepreneurial learning – we are looking deeply here.This
symposium focuses on measuring rigorously the impact of entrepreneurial education, all from
deeply experiential programs and almost all from programs whose experiential activities center
around technology commercialization. If scholars and educators want to know how to “move the
needle” in turning ideas into reality, this panel will show both a set of programs doing enviable
work and some fascinating, robust tools for measuring the impact of experiential
entrepreneurship education.
5

6. Academy of Management Panel Symposium Proposal #15313
Why PTC?
This symposium focuses on measuring rigorously the impact of entrepreneurial education in
ways that we have rarely seen in single papers, let alone bringing together some of the best
programs in the world. These speakers are all from deeply experiential programs and almost all
from programs whose experiential activities center around technology commercialization. If
scholars and educators want to know how to “move the needle” in turning ideas into reality, this
panel will show both a set of programs doing enviable work and some fascinating, robust tools
for measuring the impact of experiential entrepreneurship education.
Why Strategic Doing Initiative?
Creating viable new businesses AND creating deep entrepreneurial thinkers? This symposium
focuses on measuring rigorously the impact of entrepreneurial education, all from deeply
experiential programs and almost all from programs whose experiential activities center around
technology commercialization. If scholars and educators want to know how to “move the needle”
in turning ideas into reality, this panel will show both a set of programs doing enviable work and
some fascinating, robust tools for measuring the impact of experiential entrepreneurship
education.
6

7. Academy of Management Panel Symposium Proposal #15313
Key Presenters
University of Twente VentureLab -Aard Groen [presenter], Jeroen Kraajibrink, Gabi Kaffka
Measuring impact of entrepreneurship support in an integrated high-tech pre-incubator
Initiated and executed by the Nikos Institute of the University of Twente, VentureLab is a
pre-incubator facility at which entrepreneurs – even without business idea – are facilitated in
starting up their high-tech venture. It offers an intensive and integrated training and coaching
program, aimed at developing the necessary competencies to create a high-technology, high-
growth company. It also provides entrepreneurs office facilities and access to newly developed
technologies, venture capitalists and relevant networks of (international) companies and
scientists.
Measuring the impact of an integrated program such as VentureLab comes with at least
three major challenges. The first is that there is a distinction between the impact on the
entrepreneur and the impact on the venture. Hence, in measuring impact there are at least two
dependent variables. Second, the most important impact may only become visible after a delay of
a number of years – when we can see to what extent the entrepreneurs and ventures have grown
successfully. Third, because entrepreneurs do not operate in isolation and because they receive an
integrated offer, it is challenging to measure the individual impact of parts of the program.
To address these three challenges, VentureLab Twente is set up as a quasi-experiment
with extremely rich forms of data collection throughout and after the program. The following
data are collected:
Dependent variable: entrepreneur’s and venture performance:
1. Three four-monthly panel presentations in which an independent business panel assesses
the quality of the entrepreneurs and their venture.
2. Three four-monthly surveys in which entrepreneurs report about their individual progress
and the progress of their venture.
3. Yearly follow-up surveys and Chamber of Commerce data to assess venture growth and
performance.
Independent variables: attributing impact to elements of the program:
4. Attendance of trainings and coach meetings in order to assess the extent to which
entrepreneurs have actually participated.
7

8. Academy of Management Panel Symposium Proposal #15313
5. Training evaluations in which entrepreneurs give their opinion on a particular training in
terms of quality and usefulness.
6. Weekly diary in which entrepreneurs mention what they have learned in the past week,
and the results achieved in that week.
7. Exit interviews and focus groups in which entrepreneurs reflect on the quality and
usefulness of the program.
Control variables:
8. Intake interview and survey in which entrepreneurs report about their experience,
ambitions, capabilities and personality.
Since its inception in 2009, 200 entrepreneurs have entered the VentureLab program. This has
resulted in a voluminous and rich data set that forms the basis for various PhD projects. Research
is currently ongoing and the first results on impact are expected to be available shortly.
Danish Foundation for Entrepreneurship – Young Enterprise, Kare Moberg
Impact of Experiential Learning on Young Entrepreneurs – from ABC to PhD
Different educational levels have different learning goals. This must obviously be taken into
account when it comes to evaluation projects. Still, there is a need for comparability between
evaluation projects that target these different levels. The Danish Foundation for Entrepreneurship
– Young Enterprise has launched a major research project which measures the effects of
entrepreneurship education at basic, secondary as well as tertiary level.
At basic level the focus is on the learning process, in particular how entrepreneurship education
affects if the students internalize and take responsibility for their learning process, and how this
in turn affects their connectedness to school, classmates, educators and society. The survey is
based on two rounds of data collection. In all 1312 randomly selected 10th graders (574) and 9th
graders (738), are included in the analysis. The effects of two approaches to entrepreneurship
education are compared: the cognitive skill oriented “Entrepreneurship as a tradesmanship”
approach and the non-cognitive skill oriented “Entrepreneurship as a method” approach.
8

9. Academy of Management Panel Symposium Proposal #15313
The first has a positive effect of entrepreneurial intentions but its effects on school attachment
are negative; the opposite is true for the latter. At secondary and tertiary levels the evaluation
projects are mainly based on the effect that entrepreneurship education has on entrepreneurial
self-efficacy (ESE), and how this affects entrepreneurial behaviours. The evaluations are set up
as quasi experiments with tests before, during and after the education. At university level, 15
master-programmes are included in the sample, 7 in the control group and 8 with a strong focus
on entrepreneurship. Two rounds of data have been collected with a total of 1256 responses. This
allows us to analyze the effects that different educational designs have on different ESE
dimensions, in particular regarding their focus on the creation and discovery approaches.
In order to do this, however, we had to develop a new ESE scale which is not jargon biased
(possible to understand for students in the control group), and which covers five different skill-
sets: creativity, planning, marshaling of resources, financial literacy and managing ambiguity.
Chalmers University of Technology Martin Lackeus [presenter], co-authors: Karen Middleton
Williams, Mats Lundqvist
Measuring Entrepreneurship Education Performance
The School of Entrepreneurship at Chalmers University of Technology in Gothenburg,
Sweden has since 1997 championed an action-based approach to technology entrepreneurship.
Classes of about 30 selected student per year (approximately 50% from technology/science and
50% from business) work with real venture creation during their second year in teams of three,
trying to develop IP from technology transfer into a viable start-up.
Over the years, performance measurement of this and other educational venture creation
programs has focused on startup outcome or on individual level learning (through questionnaires
or more interactive evaluations). Although there arguably are significant effects coming from real
ventures being created – learning-wise and other-wise, you could also argue that outcome-based
measurement in combination with traditional individual-level assessments could and should be
complemented by other learning categories, such as team-level based and context-based (i.e. how
much students interacts with and draws from context). On the individual level, there is also
9

10. Academy of Management Panel Symposium Proposal #15313
reason to not only measure traditional know-how or know-what factors in venture creation
programs. Rather, know-why can arguably be the most important learning outcome for nascent
entrepreneurs to figure out – a concept related as much to emotionality and action as it is to
traditional cognitive learning.
This session draws upon 15 years of experience from and research around action-based
entrepreneurship education that among other things have produced a quarter of all the revenue
that university incubated technology ventures generated 2010 in Swedish. Measurements in four
categories will be addressed – the individual-based, group-based, context-based and outcome-
based. Focus will be on evaluating the development of entrepreneurial competence and
entrepreneurial ecosystems and on the appreciation of “know-why” in entrepreneurship.
At Chalmers, we investigate links between strong emotions and entrepreneurial learning
outcomes in a formal learning environment consisting of an action-based entrepreneurship
education program. Students’ own experiences were quantitatively and qualitatively assessed
during their participation in an entrepreneurship program where they were expected to start a real
venture as formal part of curriculum. This kind of learning environment has previously been
characterised as an emotional roller coaster with transformative learning experiences frequently
reported, due to the learning environment’s capacity to approximate the process of starting a real
venture quite accurately. It thus represents a rare opportunity to conduct laboratory studies on
nascent entrepreneurs. We continue to build on previous work on these so-called venture
creation programs (e.g., Babson paper in 2011). We now also ask: How are emotional amplitude
and entrepreneurial learning outcomes linked?
Entrepreneurs Organization Lesley Hayes, EO & University of Athabasca
Entrepreneur – Peer Learning Groups.
A number of entrepreneurial organizations offer international entrepreneurial support
systems – i.e. the Entrepreneur’s Organization with over 8,000 members globally running
businesses over $1MM USD in annual revenue. Their primary entrepreneurial learning structure
is a version of a peer-learning network popularized by Napoleon Hill - a ‘mastermind group’.
Underlying many current efforts to ‘educate’ entrepreneurs is the belief that
entrepreneurial cognition – or decisions - can be improved with training, skills & knowledge, and
these improvements lead to improvements in marketplace performance (Matlay, 2008).
10

11. Academy of Management Panel Symposium Proposal #15313
Knowledge is conceptualized as a transferrable, retainable and re-useable asset (Macpherson and
Holt, 2007). The emerging field of entrepreneurship research is littered with disagreements
regarding the source of entrepreneurial skills and successes as being primarily environmental,
political, biological, personality, skill or uniquely individual.
Traditional models of teaching are often a expert lecturer controlling a highly structured
learning environment; determining where and when learning occurs. However, research
indicates entrepreneurs learn best from experience, reflection, problem solving, and peer
interaction (Gibb, 1993; Deakins and Freel, 1998; Politis and Gabrielsson, 2007). EO & other
organizations offer programs that support entrepreneurs to learn from their own experiences.
This discussion will delve into the structures, language protocols, demographic and psychometric
variables which may be controlled for or influenced which can increase the potential for
generative (double-loop) entrepreneurial learning to occur.
There is minimal data collection from within these current groups, although one study by
Joakim Tell, “The emergent nature of learning networks’ (2008), did examine a learning network.
What are the best practices from the ‘60s and ‘70’s regarding optimum number of participants,
topics, structures, and other practices that influence these groups usefulness? Does group
dynamics, team cultures, social identity, social networks, or weak and strong ties, provide
insight, co-vary or predict into variables such as trust, cohesion, leadership and confidence.
What are the ways that we can support and measure Smilor’s statement “Effective entrepreneurs
are exceptional learners…they learn from other entrepreneurs, they learn from experience and
they learn by doing” (Smilor, 1997: 344)
Format:
As we are sharing here multiple overlapping but unique projects (at different stages of
development) we opt for the panel symposium format. We want to ensure that the audiences gets
enough time to:
(a) offer advice to one or more of the projects,
(b) identify ways to adapt one or more of these to their own interests and
c) share their own research opportunities around this theme.
It is clear that universities are eager to assess impact and the process of doing that has grown
immensely in quality… but it can only get better!
11