Ethical stalking by Mark Williams. UpliftLive 2024
Effects of Quantity-Based Staff Performance Indicators on Performance Quality
1. Effects of Quantity-Based Staff
Performance Indicators on
Performance Quality
AN ACTION RESEARCH PROPOSAL
PRESENTED BY
SUZANNE SANNWALD
ASHFORD UNIVERSITY
FUNDAMENTALS OF EDUCATIONAL RESEARCH
EDU 671
DR. KATHY HOOVER
NOVEMBER 11, 2013
5. Variables: Performance Quantity
• Number of appointments
• Time spent on appointments
Quantity-Based
Performance
Indicators
• Staff performance evaluation
7. Research Questions
Evidence of student career success
Student satisfaction with appointment quality
Employee satisfaction with appointment quality
8. Locus of Control
Opportunity and responsibility to invite
conversations and raise questions
Data access already exists or may be gathered
Authority to implement change
9. Interventions
Analyze data & synthesize
findings to inform change
Improve communication
Adjust performance indicators
10. Action Research Group Membership
Department
Director
Career Coach Managers
Operational
Manager
15. Data Collection: Overview
Evidence of student career success
Resume approvals
Success stories
Numbers hired
Student satisfaction with appointment quality
Appointment number and length per student
Student satisfaction questionnaire
Interviews with students
Employee satisfaction with appointment quality
Employee satisfaction questionnaire
Interviews with employees
Observation by manager
16. Data Collection: Details
Research
Questions
1. How effective
are coaching
appointments
in terms of
helping
students
achieve career
goals?
2. Are students
satisfied with
the quality of
service they
receive in
coaching
appointments
with staff?
Data Collection Tool
(QT=Quantitative; QL=Qualitative)
1
2
3
Why this tool? Justify its use in your
study. How does it match to what you
are attempting to find and to measure?
Timeframe
How and when data will be
collected.
Number of approved resumes indicates how
many students are completing a full coaching
Number
students hired cycle of successfully revising a resume;
StudentNumber of
submitted
based on
success stories reflect whether or not students
approved
success
partner
are meeting self-identified goals such as
resumes (QT)
securing an interview or getting a job; and
stories (QL)
employer
reports (QT)
numbers hired represents employment
success.
Number of appointments and total
appointment length per student reflects
Average
Structured
engagement levels in terms of willingness to
number of
Appointment formal
participate in multiple and/or longer sessions;
appointments
satisfaction
interviews
questionnaires distributed to students after
and total
questionnaire with students appointments may anonymously measure
appointment
for students
after
satisfaction trends; and formal interviews may
length per
appointments capture more detailed feedback from students
(QT/QL)
student per
(QT/QL)
regarding staff interactions.
period (QT)
Number of approved resumes
will be tracked on a weekly
basis and cumulatively at the
end of the quarter; success
stories and number of hires will
be gathered as reported and
reviewed at the end of the
quarter.
Appointment numbers and
lengths will be tracked on a
weekly basis and cumulatively
for the quarter; appointment
questionnaires and interviews
will be conducted periodically
throughout the quarter with
results reviewed on a monthly
basis and cumulatively at the
end of the quarter
3. How satisfied
are employees
with the quality
of service they
Employee
are able to
questionnaire
provide to
(QT/QL)
students in
coaching
appointments?
Questionnaires may capture trends regarding
staff satisfaction with quality of service they
Privileged
are able to provide; ethnographic interviews
active
Informal
may allow room for exploring more openethnographic ended responses and generation of new
observation
interview with questions by staff; and observation by a
of
appointments employees
privileged, active observer may reveal
by manager
(QL)
information demonstrated by employees, but
(QL)
that is not self-reported.
Questionnaires and observation
will occur at least once per
employee within the first
month of the quarter.
Questionnaires and observation
will occur again at least once
per employee within the last
month of the quarter, along
with the addition of the
employee interview.
17. References
Aguinis, H. and Pierce, C. (2008). Enhancing the relevance of organizational behavior by
embracing performance management research. Journal of Organizational Behavior,
29(1), 139-45.
Ali, S. B., Mahdi, A., & Malihe, J. (2012). The effect of employees’ performance appraisal
procedure on their intrinsic motivation. International Journal of Academic Research in
Business and Social Sciences, 2(12), 161-168.
Decramer, A., Smolders, C., Vanderstraeten, A., Christiaens, J., & Desmidt, S. (2012).
External pressures affecting the adoption of employee performance management in
higher education institutions. Personnel Review, 41(6), 686-704.
doi:http://dx.doi.org/10.1108/00483481211263593
Garbus, J. (2012). Get hyper-focused on performance development. People & Strategy,
35(3), 6.
Mills, G. E. (2014). Action research: A guide for the teacher researcher (5th ed.).
Boston, MA: Pearson Education, Inc.
Noeverman, J., & Koene, B.A.S. (2012). Effects of managerial evaluative styles on
subordinate attitudes and performance: proposals for future research. International
Journal of Management, 29(3), 219-213.
Ojha, A. K., & Katsuri, A. (2005). “Successful” call centre employees: Understanding
employee attributes and performance evaluation processes. IIMB (Indian Institute of
Mangement Bangalore) Management Review, 17(2), 93-102
Notas do Editor
Intended Audience: Department leadership consisting of department director and career services managersToday I would like to share a proposal that I have prepared for your review. The purpose of today’s presentation is to provide you with some contextual background information, as well as a clear explanation of the action research plan that I recommend we consider adopting and implementing with our staff.After having shared the information, I invite a period of open dialogue regarding the proposal in order to address any questions you may have and to incorporate any feedback that you may have for its improvement.Once we have the opportunity to integrate feedback, I am seeking this team’s consensus in approving the action research plan for use with our staff.
As stated in the title, the purpose of this study is to describe the effects of quantity-based staff performance indicators on the performance quality of staff. This is a very dense statement, but do not worry, because I will be breaking all of this down to make more sense in the next few slides.All that you have to know from the start is that the research is ultimately aimed to uncover how our expectations as department leadership may be affecting the performance of our staff.
The reason why I am proposing the action research plan is because I think that there is a problem that exists, which is worthy of our attention.The problem that I would like us to address is in seeing whether or not the expectations that we are setting for staff are in fact creating the results that we desire.As we start to adopt formal Human Resource Management (HRM) systems for measuring employee performance, we have a responsibility to determine how well these systems are working.In accordance with HRM, employee performance management can be described as a “continuous process of identifying, measuring and developing the performance of individuals and teams and aligning performance with the strategic goals of the organisation” (Aguinis and Pierce, 2008, p. 139).The problem I am suggesting that we address is whether we—through the expectations that we are setting—are affecting staff performance in ways that we intend, as aligned with our goals.Or, are our expectations perhaps influencing staff performance in ways that we do not intend?What I hope we will be able to answer is what is the difference between our intended effects and the actual effects?Based on the differences that we may reveal, we may then ask ourselves if are there ways that we need to consider adjusting our expectations so that we may reap results that better align with our goals.
Let us dive in by defining the variables of the proposed action research.To begin with, when we are talking about staff, we are talking about our team of ten staff members who provide career coaching services to students. When we look at their staff performance, what we are really looking at is the interaction that occurs between our staff and students in the form of career coaching appointments.And, when measuring staff performance, we are really finding ways to apply measurement to the career coaching appointments that take place.While existing research provides general guidance about employee evaluation practices, it is necessary to approach employee evaluation with a custom lens tailored based on the organizational context and individual characteristics of employees and managers. Noeverman and Koene (2012) introduce the concept of “evaluative style[s] that are relevant and valid in the context studied, and accommodate changes in contemporary control system” (p. 229).Along these lines, it makes sense that while the proposed action research draws upon previous professional findings, it holds the potential to produce results unique to our specific organizational setting and participants.
In this proposal, the action research has been designed to guide us in evaluating the effectiveness of new HRM performance indicators that measure appointments based on quantity.As recentlyestablished, staff performance will now be measured based on the number of appointments that career coaches have with students, as well as the total amount of time spent on those appointments. These quantity-based measures will be applied formatively with managers reviewing staff performance during weekly one-on-one meetings. In this way, individual career coaches will be aware of their progress and may address any concerns that they have with managers on a regular basis.In a summative manner, the quantity-based measures will be utilized as part of formalstaff performance evaluations that occur on a quarterly basis, also contributing to an employee’s overall annual performance review.Since formal annual reviews are part of an staff member’s permanent employment record with the organization, and these scores are also used to inform annual consideration for pay increases and for identifying lower performers in the case of necessary reductions in force, we may assume that introduction of these new expectations may likely affect staff motivation and performance.We may, for instance, project that the new quantity-based expectations will result in an increase in staff performance in terms of quantity. This aligns with findings by Ali, Mahdi, and Malihe (2012) suggesting that continuous improvement within organizations “cannot be achieved unless the context of achieving will be provided by the performance evaluation process improvement” (p. 167). In other words, evaluation is necessary to provide employees with feedback in order to be motivated to improve. When we talk about improvement, though, will the improvement occur both in terms of quantity and quality? This is the question that I am proposing we try to answer.
In particular, the variable that I am proposing we research is whether the drive for increasing performance based on quantity will positively or negatively affect performance in terms of quality.And, when talking about performance quality, what we will really be looking at is the quality of appointments. How do our expectations that we are setting for staff affect the quality of appointments that staff have with students?After all, while we may hope to maximize staff productivity in terms of quantity, we want to ensure that we do not compromise the quality of services that we provide to our students.
In order to measure staff performance quality in terms of the quality of appointments, there are three research questions that I propose we seek to answer:How effective are coaching appointments in helping students achieve their career goals?Are students satisfied with the quality of service they receive in coaching appointments with staff?; and How satisfied are employees with the quality of service they are able to provide to students in coaching appointments?
You might be asking whether or not it is worth our time and efforts to answer these research questions.When selecting an area of focus for action research, Mills suggests that we start off by making sure that we are selecting an issue that is worthwhile. Included among criteria for consideration, he suggests that it be both “something you would like to change or improve” and “something within your locus of control” (Mills, 2014, p. 43). In speaking with all of you, I believe that we share a common goal of wanting to continuously improve the level of service that we provide to students.And, as members of the leadership of our department, I believe that addressing the research questions that I have just outlined falls clearly within our locus of control.We not only have the opportunity to invite these conversations and raise these questions—I believe we also have a responsibility to do so.We already have access to some of the data that will be necessary, as well as the ability to gather additional data.We hold the authority to implement change that is determined to be necessary—we have the power to initiate interventions that are determined to be appropriate.
Speaking of having the authority to implement change and the power to initiate interventions, let me add in a little more detail what interventions I foresee potentially coming from the action research.First of all, any interventions will be based on taking all of the data that has been collected, analyzing it, and synthesizing all findings to inform the recommendations and changes that result.If appointment quality is demonstrated to be adversely affected or is determined to be generally lower than desired, then leadership will need to consider adjusting practices to address the deficiencies. There may be a need to adjust performance indicators so that they better drive results that align with strategic goals.And/or, there may be an identified need to generally or more specifically improve communication. After all, research by those such as Ojha and Katsuri (2005) reveal that “extrinsic motivation does not have an influence on performance” (p. 101). In other words, supporting intrinsic motivations may be much more related to relationship building and communication rather than focusing attention on the use of incentives-related performance evaluation goals, such as the quantity-based standards.
The action research team will consist of the department director, two career coach managers, and myself as the department’s operational manager.All four of us will work together in developing, implementing, and eventually analyzing and applying the results of the action research. The director is ultimately responsible for all final decisions, as well as leading all department-wide communications. It is important that employees understand that the action research and related activity is all endorsed by the head of the department.The two career coach managers will be responsible for handling all smaller communications with their respective teams and individual direct reports. These managers will update staff on their progress on a weekly basis, providing feedback and support to staff. They will also contribute to data collection by observing staff and also helping develop and deliver surveys and interviews.Finally, the operational manager will provide general coordination for the overall action research process. This manager will be responsible for coordinating all data collection and analysis, and will also participate in making recommendations based on the evidence.
The largest potential obstacle may be resistance by career coaches themselves. There may be fear regarding what action research may reveal and how research findings may be interpreted and applied. To face this obstacle, the following negotiations may prove to be helpful:Communication: It is critical that from the beginning, career coaching staff are included in communications that establish transparency and build on previously established trust. As stated by Mills (2014), it is all about “how you approach this professional collaboration … [and] if you have been able to nurture your own and your colleagues’ understanding of the problems you have investigated and built a teamwide commitment to implementing action based on your findings” (p. 170). By communicating with our staff, we build the foundation for such professional collaboration and teamwork.Common goal: When communicating with staff, it may be useful to establish a shared purpose, which should be, and in this case is, to affect change that benefits students (Mills, 2014, p. 174). The intervention is ultimately about ensuring high quality career coaching appointments, and as such, resulting improvements hold the potential to benefit both employees and the students that they serve. If effects on quality are shown to be adverse, leadership will have the opportunity to adjust and more clearly communicate expectations for employees. As a result, employees may benefit from being provided with clearer and more well-directed expectations for their work. With intended improvements, students will also clearly benefit from being able to engage in higher quality appointments with staff.Ethics: Potential opposition by staff may also be alleviated if there is a clearly established and communicated ethical foundation for the research. Let us look at this in a little more detail on the next slide.
Regarding having an ethical foundation, the reason why I believe that the action research is appropriate for our employees is that, as explained by Garbus (2012), “employee perceptions of their development and career opportunities were the greatest statistical driver of employee engagement … [which] in turn was a driver of business results” (p. 7). By emphasizing that results are to be used as a genuine development tool applied equally across department staff, we may establish respect for employees and students, as well as overall beneficence and justice.All the same, we may aim to protect employees by ensuring opportunities for informed consent and guaranteed confidentiality.Potential for bias may be minimized through use of triangulated data collection procedures, which will be detailed later.And, we may clearly provide employees full assurance that findings will be strictly utilized to refine leadership practices and not for employee evaluation purposes. After all, if anything, the intent is that evidence uncovered through research may be presented as a call to action for leadership to better support employees and their opportunities for their growth and increased engagement.
The proposed intervention will occur within approximately a six-month period. There will be three main phases for implementation: 1) planning, 2) data gathering, and 3) analysis. The planning phase will commence about two months prior to the start of data gathering. Two months of preparation time will be necessary in order to accommodate the number of tasks that must be accomplished during this time frame. First, the initial plan for research must be established and clearly documented so that the proposal may be reviewed and adopted, and also so that information may be shared with employees when soliciting informed consent. Data gathering measures such as questionnaires and interviews must be developed, as well as plans for tracking statistics such as number of appointments and appointment lengths.Data gathering is intended to align with a fiscal year quarter, which is a three-month period beginning in January, April, July, or October. During the data gathering phase, tools developed during the planning phase will be applied. Anexplanation of the various data gathering activities will covered in a couple of slides, including a more detailed chronology for each of the data collection points.For instance, some data is tracked and reviewed on a weekly basis and also cumulatively at the end of the quarter. Meanwhile, other data is only gathered once at the beginning of the quarter and once at the end.Finally, the analysis phase will occur at the close of the data gathering period, and should last about one month in duration. The analysis phase occurs after all data has been gathered during the quarter under review. Analysis involves not only looking at the data in order to detect trends or meaningful lessons that may be extracted, but to also drive conversations that may inform changes that we as leadership may make in order to improve effectiveness of expectations provided and communicated to employees.
The main resource that is necessary for this action research proposal is time. While it is never easy to ask staff to make time for one more thing, “the goal is to evolve to the point where action and research become a part of [our] professional life” (Mills, 2014, p. 171). Rather than being seen as an additional and separate task, an action research approach is one that we should ideally adopt toward evaluating our practices so that we are constantly aware of what is working and what may be improved by change.Beyond the main resource requirement of time, there are only a couple of other resource requirements: reporting functionality and survey software for distributing web-based surveys to both students and career coaching staff. Since our department already has access to this technology, these inputs fortunately do not require additional financial investment for procurement.
The data collection tools will allow for the gathering of a mix of both quantitative and qualitative data in order to answer the three proposed research questions reviewed earlier. The data to be collected will include objective data that is gathered through standard reporting practices on criteria such as number of resume approvals, hiring rates by partner employers, and appointment numbers and lengths per student.Then, there is also qualitative data that will be gathered. Although subjective, questionnaires, interviews, and observations will likely reveal information that may not be captured through quantitative data alone.By utilizing a mixed approach, the strengths of each method may contribute valuable insight into forming a more complete understanding of effects.Similarly, the data is also being collected with triangulation in mind, as information will be collected to represent the perspectives of multiple sources including students, employees, and managers.By providing a mix of data sources, we may “address issues related to bias in the data collection” (Mills, 2014, p. 104). We will be able to have greater confidence in the results since they may be substantiated by multiple stakeholders.
This is a more detailed view of the data collection plan being proposed. Notice justification is provided for each of the data collection tools, as well as information about the timing of collection.I encourage you to each take a closer look at this plan in preparation for a more in-depth dialogue. I look forward to addressing any questions that you may have, as well as incorporating any new ideas that you may have to improve the proposed action research.It is my hope that we may approach this plan as a united leadership team, each feeling ownership of its implementation so that we may create positive change to support our staff and improve the quality of services provided to students.
In closing, here are the references cited throughout the action research proposal. I invite further discussion regarding these resources, as well as an open invitation to review additional sources of information that may help inform our discussion.Thank you for your time and attention.