Call Girls In DLf Gurgaon ➥99902@11544 ( Best price)100% Genuine Escort In 24...
Multicriteria model to improve the use of performance evaluation instruments with focus on indicators
1. Proposal of use the Multicriteria Decision Aid for
Improve the use of Assessment Tools with Focus
on Performance Indicators
Msc. Edilson Giffhorn, PMP, IPMA - Speaker
PhD. Leonardo Ensslin
PhD. Sandra Rolim Ensslin
Msc. William Barbosa Vianna
24th European Conference on Operacional Research
11. – 14.7.2010, Lisboa - Portugal
2. Santa Catarina Federal University - Brazil
Production Engineering Department
Multicriteria Decision Aid Laboratory
2
3. Objective:
The aim of this paper is to present a process for improving
the way to use the instruments of performance assessment
on their way to identify, organize, measure and use the
Performance Indicators.
3
4. The use of indicators
as a management / 20th
Introduction control tool
century –
after 1950
20th
century – Evaluate processes
Instrument
until 1950 (Brandão, Guimarães, 2001)
of
Intervention 18th Scientific Management:
century Evaluation of merit (people)
(Guimarães et al. 1998)
Industrial Revolution:
Evaluation
Model Large corporations - production control
1494 (Leão, 1998)
Tratactus de Computis et Scripturis - Summa de
arithmetica, geometrica, proportioni et
proportionalita
Conclusions
(Brudan, 2009; Ensslin; Ensslin, 2009)
4
5. Performance Indicators
Introduction
After 1995
Instrument 3rd Generation Indicators
of
Intervention
1960 - 1995
2nd Generation Indicators
Evaluation
Model
Until
1960
1st Generation Indicators
Conclusions
Neely et al. (2003)
5
6. 1st Generation Indicators
Introduction
Instrument
of
Intervention
Emphasis Dimensions Orientation
Evaluation
Financial
Model
Control Efficiency
(cost, profit)
Productivity Accounting
Conclusions (quality) systems
Neely (1999); Ensslin, Ensslin (2009); Oliveira et al (2009)
6
8. Characteristics of New requirements
instruments of the context
Introduction
Generic context Specific context
Instrument Generic decision makers Specific decision makers
of
Intervention
Monitoring Continuous Improvement
Evaluation Ordinal and Cardinal
Model Ordinal scales
scales
Does not compare Comparison of
performance indicators performance by integration
Conclusions
Available quantitative Quali-quantitative
properties properties
Adaptation of Ensslin, Ensslin (2009)
8
9. 3rd Generation Indicators
Introduction
Instrument Emphasis Dimensions Orientation
of
Intervention
Customized to the Multicriteria (values
Improve specific
decision maker and preferences)
contexts in a
personalized way
Evaluation
Model
Conclusions
Neely et al. (2003), Paranjape (2006)
9
10. 3rd Generation Indicators
Introduction
Paradigm exchange
Instrument
of
Intervention Isolated measures Integrated measures
Quali-quantitative
Quantitative measures
Evaluation measures
Model
Specific measures for
Generic measures
each context
Conclusions
Measures to improve
Measures to control
performance
Adaptation of Ensslin, Ensslin (2009)
10
11. Even with the recognition of new requirements, many
applications of performance measurement continue to use the
Introduction 1st and 2nd generation.
Traditional historical
and financial
Instrument
of
orientation
Intervention Neely, Powell (2004)
Evaluation
Model
Focus on the measure,
Generic or pre-
misaligned with the
existing
Conclusions Denton (2005), Tangen (2003) goals and unbalanced
Leandri (2001)
11
12. Consequences
Introduction
Instrument Measures not aligned to strategy.
of
Intervention
Schneiderman (1999), Neely (2000), Bourne et al. (2002)
Evaluation
Model
Performance Evaluations poorly developed, misaligned with the
strategy and decision-makers do not understand the impacts of their
decisions and actions.
Conclusions Denton (2005)
12
13. Research Theme
Introduction
Instrument
of Explore ways of improving the identification
Intervention
/ construction of the Performance Indicators.
Evaluation
Model Provide greater chance of achievement the
strategic objectives based on the tools
selected.
Conclusions
13
15. MCDA-C selected due:
Introduction
Identify the important aspects in accordance with the decision
maker.
Instrument Expand the understanding of the context of the decision
of
Intervention
maker.
Allow to take into account the interests of the actors involved
with the monitoring of the decision maker.
Evaluation
Model Allow the decision maker to revise his views in the course of
the construction of their knowledge.
Recognize the ordinal and cardinal properties of the
indicators.
Conclusions
Ensure that the decision maker legitimize the process with
tools scientifically valid.
Ensslin et al. (2010)
15
16. Structuring Phase
Introduction
Identification of the Actors
Subsystem
Instrument
of
Intervention
Evaluation
Model
Conclusions
16
17. Structuring Phase
Introduction
Label:
Instrument
of Build a model to evaluate Performance Indicators as
Intervention
their contribution to the process of performance
evaluation.
Evaluation
Model
Conclusions
17
18. Structuring Phase
Introduction
Primary Assessment Elements (PAE):
Instrument
of
Intervention Goal clearly defined
Key objectives
Evaluation
Model
Mathematical operations
Precision
Practicality
Conclusions
Total = 63
18
19. Structuring Phase
Introduction
To Transform the PAEs in Concepts
PAE 61: mathematical operations
Objective underlying the PAE : Capactity to perform mathematical operations between
Instrument scales.
of
Intervention
Best possible Scales admit all statistical operations.
performance
Evaluation
Model
Worst possible
performance Scales admit only the statistical operations of mode,
counting and frequency.
Conclusions
Intensity: Very Strong
Concept 61: Ensure that mathematical operations can be performed
between the scales ... ignore this potentiality of the scales.
19
23. 51. Ensure the
operationalization of
the indicators ... the
set of measures would
Measurability Operationality
be implemented.
28. Ensure that the
Ensure the use of a resources demanded
management process for the use of the
for the set of measures indicator meets the
... use procedures not expectations of the
formalized. maker ... stop using
the indicators.
62. Ensure that 25. Ensure the use of
different groups of processes for
stakeholders consider reviewing and
there to ease the updating the
adoption of the indicators ...
indicators ... no indicators are static.
barriers to adoption.
73. Ensure that the Ensure that different Ensure that the 77. Decision maker
different actors actors have the same different actors to considers acceptable
understand the understanding of how recognize who is estimate of time to
importance of this to realize the extent ... responsible for make the measure ...
package ... there way to measure have performing the steps .. time required to
questioning the different there is a lack of perform the measure
necessity of the interpretations. commitment. exceeds the expected
measures. return for information.
24. General view of the Global Evaluation
final model. Build a model to evaluate Performance
72 Indicator 1
Indicators as their contribution to the
Introduction process of performance evaluation 68 Indicator 2
40 Indicator 3
X% Z% W%
Identification and Knowledge Mathematical
Construction Increase Foundations
Instrument
of A% B% C% D% E% F% G%
Intervention
PVF 4 – PVF 5 – PVF 6 – PVF 7 –
PVF 1 PVF 2 PVF 3 Value Substitution Ordinal Cardinal
Function Rates Scales Scales
Evaluation Excelence
Model
Good 100 100 100 100 100 100 100
Competitive
Neutral 0 0 0 0 0 0 0
Endanger
Conclusions
Legend: Impact profile of Indicator1
Impact profile of Indicator2
Impact profile of Indicator3
24
25. Conclusion
Introduction
The model will serve as a support instrument to make
Instrument
of Performance Evaluations, in order to have greater accuracy
Intervention
and alignment between the operational level, tactical and
strategic organization.
Evaluation
Model
The model will give ways for improving the identification / construction
of the Performance Indicators.
Conclusions
25
