1. Dr Peter B Sherren
Email- petersherren@gmail.com
Evaluation of a new ‘Limitations and withdrawal of care
framework’ order form – pilot study
Peter B Sherren*, Louise Turnbull, Selvarajah Yoganathan
Specialist registrar Anaesthesia and Intensive care, Sister Intensive Care,
Consultant Anaesthesia and Intensive care.
Department of Anaesthesia and Intensive care, Queen’s Hospital, Romford,
United Kingdom.
* Corresponding author
Keywords - ICU, end-of-life care, withdrawal of care, framework, palliative care
Evaluation of a new ‘Limitations and withdrawal of care framework’ order form on the intensive care unit, PBS
1

2. __________________________________________________________________________
Abstract
Introduction - The Intensive Care Unit (ICU) is a setting where death is common; it has
been suggested that 20% of patients die on the ICU. Given the majority of ICU deaths
involve the withholding or withdrawing of treatment, the importance of end-of-life care is
clear. Despite this frequency, studies suggest that the current quality of end-of-life care is
suboptimal. As a result, the authors developed a new framework to tackle this issue. The
framework produced was a five-page document, addressing the pertinent issues of end-of-life
care.
Methods – The new framework was introduced over a one-year period. An assessment
questionnaire was circulated to staff pre- and post- implementation to demonstrate any
improvements in end-of-life care.
Results - The framework was found to be helpful by 97% of respondents and was associated
with an improvement in communication and knowledge of end-of-life care. There was also an
increase in the number of staff who felt that patients, along with having their
analgesia/sedation needs met, were now experiencing care that was more conducive to a good
quality of dying.
Conclusions – The introduction of the new end-of-life care framework was associated with
improvements in: knowledge of the subject; clinician satisfaction with the process; and staff-
reported clinical care. Further work is required to assess the potential benefits of this work
beyond a single centre.
___________________________________________________________________________
Evaluation of a new ‘Limitations and withdrawal of care framework’ order form on the intensive care unit, PBS
2

3. ___________________________________________________________________________
Introduction
The Intensive Care Unit (ICU) is a setting where death is unfortunately common; it has
recently been suggested that 20% of patients die on the ICU1. End-of-life care is a vital skill
for all intensivists, given the majority of intensive care deaths (approaching 64.5% 2) involve
the withholding or withdrawing of treatment2-4. The UK National Health Service (NHS)
published guidelines in 1996 in an attempt to rationalise intensive care provision5. Amongst a
variety of principles involved in the decision-making process is the ideology that, once a
decision to limit support is made, the options available to the physician are: discharge;
making comfortable; or withdrawal of ventilatory support. The process of withdrawal is
undoubtedly more complex and a number of studies have suggested that the current quality of
end-of-life care is suboptimal on the ICU6,7. Given the concept that, when withdrawing care,
treatment never stops but simply the goals of care change8, it is clear that the medical
profession need to concentrate on education to deliver a consistently high standard in all
aspects of end-of-life care. Since contact with the dying ICU patient is not confined to one
profession or level of seniority, education should be multidisciplinary and inclusive in nature.
By doing so, it should be possible to avoid the previous frustration experienced by critical
care nurses with the end-of-life care provided by physicians9,10.
In 2004, the group behind the Liverpool Care Pathway for the Dying recognised the need for
continuity and high quality withdrawal on the ICU, when the comprehensive 22-page
framework Care of the Dying Pathway (LCP) Hospital Intensive Care Unit was produced11.
The concern is that health care workers may be discouraged to complete such a
comprehensive document given critically ill patients typically have a shorter dying phase,
with a mean of only four hours12. In fact, while implementing the LCP in a busy tertiary
hospital emergency department, Paterson et al demonstrated incomplete pathway
documentation in 49% of cases13. The level of documentation seemed to be directly related to
median survival time13. Because of this, there appears to be a need for a simplified
framework, which also addresses some of the technical details required for the withdrawal of
complex support.
Objective
The objective of the framework was to demonstrate improvements in staff’s knowledge and
experiences with end-of-life care.
Methods
The Framework – A new Limitations and withdrawal of care framework (Appendix 1) was
developed using guidance from a variety of sources including the General Medical Council’s
Good medical practice14 and the British Medical Association’s Withholding and Withdrawing
Life-prolonging Medical Treatment15. The aim was to develop a concise and easy to use
framework, which would improve communication and end-of-life care, while, at the same
time, not overly regimenting and standardising practice. The framework produced was a five-
page document containing limitation and withdrawal sections, as well as an appendix.
Following development, it was put forward at the critical care meeting to both senior nurses
Evaluation of a new ‘Limitations and withdrawal of care framework’ order form on the intensive care unit, PBS
3

4. and doctors, where feedback was used to modify the framework. It was ready for
implementation in August 2008.
Evaluation of Framework - The setting was a large district general hospital with a tertiary
neurosurgical centre in Romford (UK). The hospital has 939 beds with 16 general and 12
neuro critical care beds.
The framework was introduced to the general ICU in September 2008 and was used for a one
year period, at which time a post-framework analysis was undertaken. Prior to
implementation, there were teaching sessions arranged on use of the framework. These were
carried out for the nurses at their monthly meeting and for the doctors at their weekly
teaching. All consultants were versed in the use of the framework and one of the authors was
available by phone if any further questions arose regarding its use.
The framework was aimed at all members of staff who were closely involved in the
withdrawal process, such as the nurses and junior doctors. Although senior trainees and
consultants would be involved in completing the framework, the authors were keen to illicit
the opinions of the junior doctors and nurses that would be involved utilising the framework
to guide clinical practice.
Pre- and post-framework questionnaires (Appendix 2 and 3) would be analysed to
demonstrate whether there were any improvements in the following areas of end-of-life care:
• Staff background knowledge
• Quality of life for the dying patient, based on staff’s subjective opinion
• Clinician satisfaction
The staff’s background knowledge was assessed using 17 closed questions relating to end-of-
life care. The mean scores were compared following the introduction and use of the
framework for one year. Quality of life of the dying patient was assessed using staff’s
subjective experience of patient comfort during end-of-life care. The questions were ‘yes or
no’ answers, although, on this occasion, if they answered ‘yes’, they were asked to score their
confidence in their answer on a scale of 1-10. Scoring was used to evaluate whether or not the
confidence scores of staff answering ‘yes’ would improve post-framework introduction. Staff
satisfaction with the framework itself, and end-of-life care as a whole, was ascertained using
questions relating to the level of support, clarity of instructions and confidence in managing
withdrawal. Here again, ‘yes or no’ questions were asked, and ‘yes’ answers were confidence
scored 1-10.
The post-framework questionnaire was only to be completed by staff that had used the
framework.
The data was collected in an anonymous manner. The consultant body felt that written
consent and formal ethical approval was not required given it represented a quality
improvement process, and because no other framework was being utilised at that point in
time.
Evaluation of a new ‘Limitations and withdrawal of care framework’ order form on the intensive care unit, PBS
4

5. Statistics – The difference in the background knowledge mean (%) between the pre- and
post-framework questionnaire was analysed using the student’s t-test, given the parametric
distribution of the unequal groups.
To compare the two groups (pre- and post-framework) and the two outcomes (‘yes or no’
answers) within the ‘Quality of life for the dying patient’ and ‘Clinician satisfaction’
components of the questionnaire, two by two contingency tables were constructed and chi-
squared tests were performed.
‘Confidence scores (1-10)’ for each question are presented as medians. For further analysis,
the questions were categorised according to whether they related to ‘Quality of life for the
dying patient’ and ‘Clinician satisfaction’. To compare this categorical/ordinal data for the
two groups (pre- and post-framework) for each category, contingency tables and chi-squared
test were performed.
A p-value of less than 0.05 was considered statistically significant.
Results
On the general ICU at Queen’s Hospital there are approximately 51 nurses and up to 8 trainee
doctors rotating through the unit at any one time. A pre-framework questionnaire was
circulated prior to implementation, to assess knowledge and opinions regarding the end-of-
life care at that time. For the pre-framework questionnaire, 39 responses were collected, of
which 37 (63% of ICU staff) were complete. The framework was then introduced for a year
trial period, to allow enough time for a variety of people to gain experience of its use (Table
1).
Table 1. Professions and grades of the pre- and post-framework respondents.
Profession and grade Pre-framework Post-framework
n=37 n=33
Nurse
• Band 5 8 6
• Band 6 8 6
• Band 7 14 12
Doctor
• Foundation Year 3 4
Doctors 1-2
• Specialist Trainee, 2 2
Year 1
• Specialist Trainee 2 3
Year 2
During this period, 67 patients that had their care limited or withdrawn according to the
Intensive Care National Audit & Research Centre (ICNARC) database. The new framework
was completed for 54 of these patients during their end-of-life care. Between September and
October 2009, the post-framework data collection was undertaken. Any staff member that
said they had used the framework on one or more occasion, when directly questioned by the
Evaluation of a new ‘Limitations and withdrawal of care framework’ order form on the intensive care unit, PBS
5

6. authors, was asked to complete a questionnaire at that time. By simultaneously circulating
and then collecting the questionnaires face-to-face, we hoped to limit the potential response
bias of self-motivated individuals with positive experiences. Of the post-framework
questionnaires distributed, we collected 36 responses; of those questionnaires, 33 (56% of
ICU staff) were complete (Table 2).
Table 2. Pre- and post-framework questionnaire results, comparing general knowledge, clinician
satisfaction and quality of life of dying patients during end-of-life care.
*Denotes statistically significant result (p <0.05).
Pre Post p-
framework framework Value
n=37
N=33
Background knowledge on end 62.3 70.3 0.02*
of life care % (Standard (12.6)
deviation, SD) (14.1)
Clinical benefits and Staff Yes No Yes No p-
satisfaction Value
Is the current withdrawal practice 21 16 27 6 0.04*
meeting the sedation/analgesia
needs of the dying patients?
Is the current withdrawal practice 15 22 23 10 0.02*
conducive to a good quality of
life for the dying patient?
Do you feel supported during the
withdrawal process by
• Nursing staff?
28 9 27 6 0.57
• Junior Doctors? 20 17 20 12 0.63
• Consultants? 21 16 24 9 0.32
Do you receive clear/concise 16 21 29 4 0.0001
instructions on withdrawal, either *
verbal or written?
Do you feel confident in the 23 14 26 7 0.06
process of managing withdrawal
of care on the ICU?
Evaluation of a new ‘Limitations and withdrawal of care framework’ order form on the intensive care unit, PBS
6

7. Background knowledge - The average mean score for the background knowledge portion of
the questionnaire showed a statistically significant improvement from 62.3% to 70.3% (p-
0.02). Issues pertaining to the provision of sedation/analgesia during extubation and the dying
process showed the greatest improvement in scores post-framework. This was closely
followed by the group of questions relating to the technical aspects of withdrawing
ventilatory support.
Quality of life for the dying patient – There was a statistically significant increase in the
number of clinicians who felt that, in questions relating to the quality of life for the dying
patient, that there was an improvement in care provided post-framework (p- 0.001). Increase
in the number of staff that felt analgesia and sedation needs were being met was
demonstrated (21 to 27). The other positive outcome within this field was that a greater
proportion of the ICU staff felt that, post-introduction of the framework, the withdrawal
practice was conducive to a good quality of life for the dying patient.
Table 3. Questions relating to whether or not current practice is conducive to a good quality of life for
the dying patient. Comparison of confidence scores whether or not current practice is post
Table 3. Questions relating to of respondents who answered ‘yes’ pre- and
framework. conducive to a good quality of life for the dying patient
34
35
31 31
30
26
25
25
% of total responses
20 18
16 Pre-Framew ork
15 Post-Framew ork
10 8
6
5 3 3
0 0 0
0
1-4 5 6 7 8 9 10
Confidence score for yes answ ers (1-10)
Clinician satisfaction - This section of the questionnaire related to staff satisfaction with the
framework itself and end-of-life care has a whole. The feedback from the 33 respondents who
had experience with the framework was very positive, and 97% (32) found the framework
useful. Post-framework there was a statistically significant increase in the number of
clinicians who were satisfied with the current withdrawal process (p- <0.001). This was
measured by questions relating to: the levels of support experienced; the presence of verbal
and written communication of decisions; and overall confidence in managing withdrawal on
the ICU. In addition to the increase in the absolute number of clinicians that were satisfied,
there was also an increase in the confidence scores reported post-framework (p-value
<0.001).
Evaluation of a new ‘Limitations and withdrawal of care framework’ order form on the intensive care unit, PBS
7

8. Table 4. Questions relating to whether or not clinicians are satisfied with current withdrawal
practice. Comparison of4. Questions relating to whether or not ‘yes’ pre- andare
Table confidence scores of respondents who answered clinicians post-
framework satisfied with current withdrawal practice
35 33
31 31
30
26
% of total responses
25
20
20 Pre-Framew ork
14 15
15 13 Post-Framew ork
10
5 6
5 2 2
1 1
0
1-4 5 6 7 8 9 10
Confidence scores for yes answ ers (1-10)
Discussion
End-of-life care on the ICU is regrettably a topic that in the past has often been neglected6,7.
Despite the various guidelines and frameworks11 available for facilitating end-of-life care on
the ICU, our experience, and those of others, is that witnessed clinical practice is still
frequently suboptimal16. Nelson et al conducted a postal survey looking at end-of-life care
provision on 600 ICUs in the United States16. 80% of the respondents felt that the following
strategies were likely to improve end-of-life care: trainee role modelling by experienced
clinicians; clinician training in communication and symptom management; regular meetings
of senior clinicians with families; bereavement programs; and end-of-life care quality
monitoring16. However, few of these strategies were widely available on any of the
responding units16. The SUPPORT trial attempted to address this issue, with a multi-centre
randomised control trial17; however, it failed to demonstrate any benefit of providing a nurse
specially trained in end-of-life care on the ICU. The role of this nurse was to encourage
multiple contacts with the patient, family, physician and hospital staff to elicit preferences,
improve understanding of outcomes, encourage attention to pain control, and facilitate
advance care planning and communication. The reason for this failure was multi-factorial.
One of the principal limiting factors suggested was that the intervention did not actually
affect the system of care at the target institutions because of strong psychological and social
feelings that underpinned their current practices18. What can be learnt from this is that even
the best model will fail to achieve the desired outcome without all clinicians being in
agreement, and implementing the model in the manner it was intended.
The Limitation and withdrawal of care framework was introduced to provide a means for
quality improvement in the care provided to the dying patient on the ICU. Analysis of the
questionnaires demonstrated that the framework had been well received, with 97% of
respondents reporting it to be helpful. The key to this success was due to the fact that, as a
Evaluation of a new ‘Limitations and withdrawal of care framework’ order form on the intensive care unit, PBS
8

9. single centre study, there was scope to tailor the framework appropriately on the basis of
feedback received from the relevant staff. It was felt that, in some respects, this should be the
model for future research and the authors would suggest that trying to standardise practice
across multiple institutions will only result in non-compliance and further gaps in care.
The introduction of the framework, along with the associated and necessary education, did
instil some enthusiasm for end–of-life care and, despite a relatively crude assessment, there
was a statistically significant improvement in the staff’s level of knowledge of the topic.
Improving knowledge and communication during the withdrawal process were key outcomes.
As a result of this, it was hoped an improvement in the clinical care experienced by patients
would also be revealed. The post-framework analysis showed an increase in the number of
staff who now felt that patients were having their sedation and analgesia needs met. Further
to this, it was also felt that the care provided post-introduction of the framework was more
conducive to a good quality of life for the terminally ill.
Without direction, the improvements in knowledge would have been wasted, and one of the
most valuable achievements of the framework was to provide a medium for communicating
clear instructions for withdrawal. Although this was an understandable result when
introducing a framework, it was anecdotally one of the outcomes that nursing staff most
appreciated, and ensured a good continuity of care.
In questions relating to clinician satisfaction with the end-of-life care provided, there was a
statistically significant increase in both the number of clinicians satisfied with practice and
their confidence scores (p- <0.001).
There were a number of limitations with this work. The first sets of problems are those
inherently related to a pre- and post- questionnaire single centre design. As suggested earlier,
the benefit of adopting a single centre design is that the framework can be customised for the
relevant staff, however, this opened up the work to bias in the form of the temporal changes
associated with the surrounding interest and education. Along with this, performing a study in
a single institution where authors and respondents work together means that there may have
been an eagerness to assist in a positive outcome. Although attempts to limit response bias
were undertaken, there was still a risk of selection bias. Not all of the ICU staff would have
used the new framework within the year trial period; however, the overall response of 33 of
the 59 staff (56%), possibly represented a missed opportunity for feedback. A pre/post design
does not allow identification of the reasons for any potential improvements witnessed, all that
can be reported are the statistical differences demonstrated and the potential factors involved.
The best measures for patient quality of life during withdrawal are the self-reported
symptoms of the patient involved. However, this can be technically difficult when dealing
with the severe organ dysfunction and the inadequate cerebration often seen on the ICU. A
surrogate measure is the first-hand observations of the care givers in the patient’s final hours,
and it was these subjective opinions we relied upon in establishing potential improvements in
the quality of care.
The authors realise that the title of this framework ‘Limitations and withdrawal of care’ could
have negative connotations. Although the term ‘withdrawal of care’ is commonly used and is
universally understood terminology amongst intensivists; moving forward, we will endeavour
Evaluation of a new ‘Limitations and withdrawal of care framework’ order form on the intensive care unit, PBS
9

10. to encourage the use of the phrase ‘End-of-Life care framework’, in an attempt to shift the
focus away from the withdrawal of organ support and toward the vital process of providing
comprehensive and high quality care for the dying.
Conclusions
End-of-life care is, historically, an undervalued and sometimes poorly understood component
of care on the ICU. The new targeted framework was found to be helpful by the majority of
respondents and was associated with improvements in knowledge of the topic and clinician
satisfaction. There was also seen to be an increase in the number of staff who felt that
patients, along with having their analgesia/sedation needs met, were now experiencing care
that was more conducive to a good quality of life during their terminal illness. Follow-up
work and research should be undertaken to establish potential benefits of this framework
beyond a single centre.
___________________________________________________________________________
Authors' contributions - PBS and SY were involved in the design of the trial and
framework. PBS and LT were involved in data collection. PBS prepared the draft and all
authors were involved in revising the final manuscript. All authors have read and approved
the final manuscript.
Acknowledgments - This paper did not receive any grant or funding from any agency in the
public, commercial or not-for-profit sector.
Declaration of interests - All authors declare that there is no conflict of interest that could be
perceived as prejudicing the impartiality of the paper.
Presented Presented at the 30th International Symposium on Intensive Care and
Emergency Medicine, Brussels (March 2010)
___________________________________________________________________________
Evaluation of a new ‘Limitations and withdrawal of care framework’ order form on the intensive care unit, PBS
10

11. References
1. Angus DC, Barnato AE, Linde-Zwirble WT, et al. Use of intensive care at the end-of-
life in the United State: epidemiologic study. Crit Care Med 2004; 32: 638-643.
2. Wood GG, Martin E. Withholding and withdrawing life-sustaining therapy in a
Canadian intensive care unit. Can J Anaesth 1995; 42 (3): 186-191.
3. Prendergast TJ, Luce JM. Increasing incidence of withholding and withdrawing life
support from the critically ill. Am J Resp Crit Care 1997; 155: 15-20.
4. Treece PD, Engelberg RA, Crowley L, et al. Evaluation of a standardised order form
for the withdrawal of life support in the intensive care unit. Crit Care Med 2004; 32
(5): 1141-1147.
5. NHS Executive working party. Guidelines on admission to and discharge from the
intensive care and high dependency unit. London: Department of Health, March 1996.
6. Desbiens NA, Wu AW. Pain and suffering in the seriously hospitalised patients. J Am
Geriatr Soc 2000; 48: S183-186
7. Nelson JE, Meier D, Oei EJ, et al. Self reported symptom experience of critically ill
cancer patients receiving intensive care. Crit Care Med 2001: 29; 277-282.
8. Girbes ARJ. Dying at the end-of-life. Intensive Care Med 2002; 28: 1197-1199.
9. Asch DA, DeKay ML. Euthanasia among US critical care nurses: Practices, attitudes,
and social and professional correlates. Med Care 1997; 35: 890-900.
10. Asch DA, Shea JA, Jedrziewski MK, et al. The limits of suffering: Critical care
nurses’ views of hospital care at the end-of-life. Soc Sci Med 1997; 45:1661-1668.
11. Liverpool Care Pathway. Care of the Dying Pathway (LCP) Hospital Intensive Care
Unit. http://www.mcpcil.org.uk/liverpool-care-pathway/pdfs/LCP-ICU-version11.pdf
12. Hall JI, Rocker GM. End-of-life care in the ICU: Treatment provided when life
support is withdrawn. Chest 2000; 118: 1424-1430
13. Paterson BC, Duncan R, Conway R, et al. Introduction of the Liverpool Care Pathway
for the end-of-life care to emergency medicine. Emer Med J 2009; 26:777-779
14. General Medical Council. Withholding and withdrawing life-prolonging treatments:
Good practice in decision making. Good Medical Practice 2006. http://www.gmc-
uk.org/guidance/ethical_guidance/witholding_lifeprolonging_guidance.asp
15. BMA Medical Ethics Committee. Withholding and Withdrawing Life-prolonging
Medical Treatment. BMJ Books 1999; ISBN 0-7279-1456-1.
16. Nelson JE, Angus DC, Weissfeld LA, Puntillo KA, Danis M, Deal D, Levy MM,
Cook DJ. End-of-life care for the critically ill: A national intensive care unit survey.
Crit Care Med. 2006 Oct;34(10):2547-53
Evaluation of a new ‘Limitations and withdrawal of care framework’ order form on the intensive care unit, PBS
11

12. 17. The SUPPORT principle investigators. A controlled trial to improve care for seriously
ill hospitalized patients. The study to understand prognoses and preferences for
outcomes and risks of treatments (SUPPORT). JAMA 1995; 274:1591-8.
18. Lynn J, Arkes HR, Stevens M, et al. Rethinking fundamental assumptions:
SUPPORT's implications for future reform. Study to Understand Prognoses and
Preferences and Risks of Treatment. J Am Geriatr Soc. 2000; 48: S214-221.
Appendix Legend
Appendix 1. Limitation and withdrawal of care framework
Appendix 2. Pre-framework questionnaire
Appendix 3. Post-framework questionnaire
Evaluation of a new ‘Limitations and withdrawal of care framework’ order form on the intensive care unit, PBS
12