Calls for both practical and scholarly activities to be grounded more in actual evidence have become louder, especially in the last decade. Four domains in particular have embraced evidence-based thinking, resulting in the respective developments of evidence-based medicine, evidence-based management, evidence-based education and evidence-based policy. Despite the presumed benefits of drawing on different sources of evidence for decision-making in practice, whether in medicine, management, education or policy, this does not seem to prevail. Whilst one likely reason for this slow uptake could simply be down to practitioners not always having much time to consult the evidence-base in their day-to-day work, another reason might be that they are not aware of specific insights applicable to their domain of work or to practice in general.
This is where the workshop contributes:
Representatives from the four key domains engaged with evidence-based practice will share with the audience their latest insights and the consequences thereof for practice. Further, all speakers will discuss questions such as:
What do we have in common?
How can we learn from one another?
How can we combine insights from the four domains?
These will be discussed as part of a concluding panel.
Workshop organiser:
Dr Celine Rojon, University of Edinburgh, celine.rojon@ed.ac.uk
4. 4
Manager > ‘Manage’
introduced mid 16th century: from Italian
maneggiare, “to handle”, based on Latin
manus, “hand”.
Management?
(we’re all managers)
5. History of management/business schools
Late 1800s - started as trade schools
1950s - Ford and Carnegie reports criticised business
schools for not being sufficiently academic or scientific
1980s onwards – criticised for producing research
irrelevant to practice and students without relevant skills
or knowledge
5
6. On the one hand they fear . . . the scorn of other, more
traditional academic subjects. On the other hand, they
often stand accused of being less than relevant to
business. (Grey, 2001)
2000s onwards – criticized for producing ideas and
students that lead to financial crisis and poor ethical
behaviour (e.g., Enron, only one US president has MBA,
MBA students cheat more than other graduate students)
6
History of management/business schools
7. Management schools are big business
Nat. Center for Education Statistics (USA, 2011-’12)
20% of all Batchelor’s Degrees
25% of all Master’s Degrees
Higher Education Statistics Agency (UK, 2012-’13)
27% of all Higher Degrees awarded
18% of all Higher Education qualifications obtained
UK Council for International Student Affairs (2011-’12)
36% of all business school students are international
30% of all international students studying business
7
8. EBP in Management
Many similar ideas in past (e.g., Mode 1-Mode 2,
collaborative research, rigour and relevance)
Rousseau (2005) Presidential address
Peffer and Sutton (2006) Hard Facts, Dangerous Half-
Truths, and Total Nonsense: Profiting from Evidence-
based Management
8
9. EBP in Management
EBMgt Collaborative (2007-’08) Rousseau/Carnegie Mellon
Wiley Evidence-Based Management Insights (2007-’08)
(attempt to create Cochrane database for management)
Briner and Denyer
Presentation to AoM Board of Governors (2008) (attempt to
persuade to develop systematic reviews in management)
Center for Evidence-Based Management – more later
9
18. Teach managers how to
critically evaluate the validity,
and generalizability of the
evidence and help them find
‘the best available’ evidence
18
19. Evidence-based practice is about making decisions through
the conscientious, explicit and judicious use of
the best available evidence from multiple sources by
1. Asking: translating a practical issue or problem into an answerable question
2. Acquiring: systematically searching for and retrieving the evidence
3. Appraising: critically judging the trustworthiness and relevance of the evidence
4. Aggregating: weighing and pulling together the evidence
5. Applying: incorporating the evidence in the decision-making process
6. Assessing: evaluating the outcome of the decision taken
to increase the likelihood of a favorable outcome.
Definition
19
20. Evidence based decision
Professional
experience and
judgment
Organizational data,
facts and figures
Stakeholders’ values
and concerns
Scientific
research
findings
Ask
Acquire
Appraise
Aggregate
Apply
Assess
20
24. Differences?
Research: low internal validity
Lack of evidence summaries
Focus on multiple sources
Organizational (BIG) data
Stakeholders’ concerns
25. How evidence-based are we (managers)?
“I’ve never thought I need more
evidence before making a decision;
I know what needs to be done, we
get on with it and we get results.”
25
26. How evidence-based are we?
959 (US) + 626 (Dutch) HR professionals
35 statements, based on an extensive body of evidence
true / false / uncertain
HR Professionals' beliefs about effective human resource practices: correspondence
between research and practice, (Rynes et al, 2002, Sanders et al 2008)
26
27. 1. Incompetent people benefit more from feedback than
highly competent people.
2. Task conflict improves work group performance while
relational conflict harms it.
3. Encouraging employees to participate in decision
making is more effective for improving organizational
performance than setting performance goals.
True (likely) or false (not likely)?
27
36. Postgraduate Course
Some move towards systematic review methods
in management
Limited interest in teaching EBP in management
schools
Management schools still a bit trapped: Cash
cows for universities; focus on ‘top quality’
academic research; few incentives to be relevant
Future developments: academia
36
37. Postgraduate Course
Though issues around accountability, ethics,
corporate social responsibility could in part be
resolved by research and teaching EBP
Need for a professional doctorate?
Future developments: academia
39. In general
incentives to do non-EBP and punishment
(or no incentives) for doing EBP
39
Challenges
(same for practitioners and academics)
40. Incentives for managers
Not rewarded for doing what ‘works’ – few evaluations
Speed and action valued more highly than accuracy
and analysis
Managing and understanding power and politics to get
things done more valued than understanding and
using evidence to make decisions
It may be too late to change existing senior
managers…
40
41. Barriers from academic / buss school context
• In publishing and research
• In teaching content and teaching style
Barriers from organizational contexts
• Managers love fads and quick fixes
• Power and politics
41
Other challenges
43. Ambivalence about the value and applicability of
management research
Few incentives to get involved
Primary research (collecting new data) valued more highly
than secondary research (reviewing existing data)
EBMgt not academics‘responsibility – this is about practice
not research
Some concern that systematic reviews will expose the
limited nature of management research
Some academics are like ‘gurus’ and feel that EBMgt
might show their claims to be untrue
Why don’t academics like EBMgt?
44. Espoused and more implicit goals of
management school educators
ESPOUSED GOALS
To develop critical thinking
To help students understand the
full body of knowledge
To discuss gaps and limitations
of our knowledge
To challenge thinking and
assumptions
To educate
To teach students how to think
for themselves
To maintain quality standards
IMPLICIT GOALS
To help students feel
successful
To select those bits of that are
interesting or digestible or
‘cutting edge’ fads
To reassure students that what
we’re teaching them is solid
To make sure students are
satisfied
To entertain (edutainment)
Teach students what they need
to pass assessments
To give ever-higher grades
44
45. ESPOUSED GOALS
To advance scientific
understanding
Using the best research
techniques
Publishing all results and
replications – unbiased
Focus on what’s important
Being honest about existing
evidence
To disseminate all our
evidence and make publically
available
Collaboration & cooperation
IMPLICIT GOALS
To advance career
Use whatever techniques will
get you published
Publishing (mostly) only
positive results, no replications
Identifying ‘new’ or trendy
topics – creating empires
Exaggerating how much we
know
Locking up our evidence
behind publishers’ pay walls
Competition for resources,
slots in journals, between
universities 45
Espoused and more implicit goals of
management school researchers
46. Implications for research
Less focus on collecting new primary data
More focus on systematic reviews and
understanding what we know and don’t know
Focusing research efforts on what is
academically and/or practically important
without compromising ‘academic freedom’
Broader-based research training –
management researchers highly and
narrowly specialized
46
48. Undermines formal authority
They feel it constrains freedom to make
managerial decisions
Speed valued and rewarded more than accuracy
Feel they cannot use their own experience and
judgment (not true)
Managers not necessarily rewarded for doing
what works (organizations rarely evaluate)
THEY LOVE FADS & QUICK FIXES
Why don’t managers like EBMgt?
52. Postgraduate Course
How are fads a problem?
“The main problem…is their lack of any solid
intellectual foundation. Implicit in each fad is a
cause effect statement that is rarely made
explicit and never properly supported.”
52(Donaldson & Hilmer, 1998)
56. QUICK FIXES
Because quick fixes / fads
Can be career-enhancing for managers
Speed is often valued over accuracy
Do we crave quick and easy solutions?
So who needs or wants academic research?
So why do managers love quick fixes?
57. Implications for management practice
Focus on accountability
Focus on next generation
Management = profession
EBP > Professional standard >Accreditation
57
58. Think about this
How can a profession / domain be evidence-based
if it is not managed in an evidence-based way?
58
“No job is more vital to our society than that of a
manager. It is the manager who determines whether our
social institutions serve us well or whether they
squander our talents and resources.”
Henry Mintzberg