Journal of Public Health Advance Access originally published online on July 6, 2005
Journal of Public Health 2005 27(3):270-275; doi:10.1093/pubmed/fdi043
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
An investigation into GPs with high patient mortality rates: a retrospective study
Julie Billett
Julie Billett, Specialist Trainee in Public Health and Honorary Research Fellow, Department of Primary Care and Social Medicine, Imperial College London, Charing Cross Campus, St Dunstans Road, London W6 8RP
Nicholas Kendall
Nicholas Kendall, Assistant Director of Public Health, Adur, Arun and Worthing Primary Care Trust, 1 The Causeway, Goring-by-Sea, Worthing, West Sussex, BN12 6BT
Peter Old
Peter Old, Associate Director, National Clinical Assessment Authority, First Floor, Market Towers, 1 Nine Elms Lane, London SW8 5NQ
Address correspondence to Julie Billett E-mail: j.billett{at}imperial.ac.uk
Background As part of the work of the Shipman Inquiry, five general practitioners (GPs) in West Sussex were identified as having excessively high mortality rates. The aim of this study was to identify reasons for the higher-than-expected mortality rates of these five GPs.
Methods A retrospective analysis was made of routine mortality and patient registration data from primary-care practices in West Sussex, and the case notes of deceased patients were reviewed. Outcome measures included standardized mortality ratios (SMRs), proportion of deaths in nursing homes, and reviewers’ concordance with GP decisions to issue a death certificate.
Results The high death rates were not explained by the age and sex composition of each GP’s patient population. SMRs ranged from 145 to 239 (average for West Sussex = 100) and all differences from the West Sussex average were statistically significant (p <0.02). SMRs were highly correlated with the proportion of deaths occurring in nursing homes (Pearson’s correlation coefficient = 0.95, p = 0.015). Analysis of 153 deceased patients’ notes revealed no evidence of poor clinical practice. In 114 cases, at least one independent reviewer agreed with the decision to issue a death certificate. In the remaining 39 (25 per cent) cases, inadequate information in the patient’s record explained the reviewers’ uncertainty about issuing a certificate.
Conclusion A proportionately high registration of nursing home residents is the most likely explanation for the excessive mortality rates of these five GPs. This investigation was time-consuming and costly, and highlights the potential ramifications for primary-care organizations of introducing a national system for monitoring death rates in primary care.
Keywords: primary health care, mortality, population surveillance, retrospective studies
 |
Introduction |
|---|
 TOP Introduction MethodsResultsDiscussionAcknowledgementsReferences |
|---|
Following the case of Dr Harold Shipman, monitoring mortality data as an indicator of physician performance has been the subject of considerable discussion and debate.1–8 Various statistical methods have been proposed and applied to monitoring deaths associated with individual practitioners in primary and secondary care settings,1–3,6,9 yet questions remain regarding the sensitivity and specificity of these systems,5,7 and about their value as prospective monitoring tools capable of detecting aberrant performance sufficiently early.9,10 In December 2004, the independent Shipman Inquiry recommended the development of a national system for routinely monitoring GP patient mortality rates.11 Yet it remains unclear who should be responsible for investigating GPs flagged up as having unusually high death rates by such a system, or what methods should be used.
We conducted an investigation into five GPs in West Sussex whose registered patients were identified as having higher-than-expected death rates, to identify reasons for their outlying patient mortality. In this paper, we describe the methodology and findings of that investigation, and highlight some of the practical implications for primary-care organizations (PCOs) conducting such investigations at a local level.
|
Methods |
|---|
|
TOPIntroductionMethodsResultsDiscussionAcknowledgementsReferences |
|---|
The Shipman Inquiry commissioned researchers at Imperial College London to develop a statistical method for monitoring GP mortality rates and to ascertain whether such a method could be used to identify excess deaths amongst patients of a GP, such as Shipman. A cusum method was developed by the Imperial College researchers,12 and their findings from a pilot study using this method have been reported elsewhere.6 In this pilot, patient mortality data for 1009 GPs in five former Health Authorities were analyzed. When the alarm threshold was set to detect an increase in age standardized mortality of four standard deviations above the ‘in-control’ mean, the cusum plots flagged up 11 doctors, in addition to Shipman, as having higher-than-expected mortality rates.
The Solicitor to the Inquiry subsequently notified the relevant PCOs of the 11 GPs identified as having aberrant mortality rates, and requested further investigation. Five of these 11 GPs worked in West Sussex. Our investigation into these five GPs comprised two main components: first, a statistical analysis of death rates amongst their patients, and secondly, a review of clinical records of deceased patients. In each case, we restricted the focus of our investigation to the ‘marker’ year in which each GP’s cusum plot had crossed the alarm threshold.
Analysis of mortality rates
For the statistical analysis, we obtained data on each GP’s patient population from the Exeter patient registration database. To identify all deaths amongst each GP’s patient list, we obtained details of all patients who had been removed from the Exeter database in the relevant year where the reason for removal was ‘patient deceased’. For each deceased patient, information was then extracted from the Public Health Mortality File (PHMF) on: date of birth and death, place of death, certifying doctor, underlying cause of death and a communal establishment code (where recorded). From the PHMF, we also identified all deaths for which a medical certificate of cause of death (MCCD) had been issued by the five GPs in the year in question. These certified deaths included patients registered with the certifying GP, and also patients registered with other GPs.
For each GP, we calculated a crude death rate per 1000 registered patients, and a crude death certification rate. We also calculated SMRs with 95 per cent confidence intervals for each GP’s registered patient population. The expected number of deaths on each GP’s list was calculated by applying age- and sex-specific death rates from the West Sussex population in the relevant year. Regional variation in death rates is known to exist even after adjusting for age, sex and deprivation,12 and West Sussex references rates were therefore considered the most appropriate to use to derive expected mortality. Five-year age bands up to and including 85–89 years, and 90 years and over were used. This age and sex standardization was more refined than that used in the pilot study undertaken by Imperial College London researchers. Owing to data limitations in that pilot study, sex standardization had not been possible, and age standardization was performed using three broad age bands (0–64, 65–74, 75 and above). The 95 per cent confidence intervals for the SMRs were calculated using exact probabilities from the Poisson distribution for the lower and upper limits of the observed number of deaths. For each GP, we also examined the proportion of deaths occurring in nursing homes.
Review of clinical records
The review of clinical records was conducted by two experienced West Sussex GPs (‘the reviewers’). Using a structured form adapted from Professor Richard Baker’s audit of Shipman’s practice,13 each reviewer independently examined and extracted information from the GP case notes of patients for whom a MCCD had been issued by the five GPs. The reviewers were not blinded to the purpose of the investigation or to the identity of the five GPs, as it was not practical to remove all mention of each GP’s name from patient case notes. For each patient, the reviewers were asked to consider several specific issues, including the nature of the relationship between certified cause of death and the patient’s medical history, and whether they themselves would have issued a MCCD or referred the case to the coroner. Cohen’s Kappa statistic was calculated to assess agreement between the two reviewers.
|
Results |
|---|
|
TOPIntroductionMethodsResultsDiscussionAcknowledgementsReferences |
|---|
Statistical analysis of deaths
A total of 257 deaths occurred amongst patients registered with the five GPs during the marker years. In those same years, 159 death certificates were issued by the five GPs. There was considerable variation between the GPs with regard to the proportion of MCCDs that were issued for patients registered on their own list (Table 1). GP A, the only single-handed practitioner amongst the five GPs, had the highest proportion at 92% of all MCCDs issued.
|
The crude death rates for registered patients were substantially higher than the crude death rates for West Sussex as a whole in each marker year. For each GP, there was a statistically significant excess of deaths amongst their registered patient populations compared with the West Sussex average, after taking into account variations in the age and sex structure of their patient lists (Table 2).
|
Place of death
The place of death of deceased patients registered with each of the five GPs is shown in Table 3. In West Sussex in the period 1995–1999, 22 per cent of all deaths occurred in non-NHS hospitals and nursing homes. The proportion of deaths occurring in nursing homes and the SMRs for each of the five GPs’ patient populations were highly correlated (Pearson’s correlation coefficient 0.95, p = 0.015) (Fig. 1).
|
|
Review of clinical records
Of the 159 patients issued a death certificate by these five GPs in the relevant years, the primary-care records of 153 (96 per cent) patients were available for review.
One of the key questions posed to the reviewers was whether, based on the information available in the medical record, they would have issued a death certificate for the cause of death recorded on the patient’s death certificate. Reviewer 1 agreed with the GPs’ decision to issue a MCCD in 112 cases (73%), whilst reviewer 2 would have issued a MCCD in 92 cases (60%). On this issue, there was a good level of agreement between the two reviewers over and above that expected by chance (kappa 0.65, p <0.001) (Table 4).
|
In 39 cases, both reviewers were unanimous in their uncertainty about issuing a death certificate for the deceased patient. We took the view that these 39 deaths potentially provided more grounds for concern and warranted further investigation than the 114 deaths for which at least one (and in 90 cases, two) independent, experienced GP agreed with the certifying GP’s decision to issue a certificate.
In our subsequent analyses we therefore examined whether there were any systematic differences between these two particular groups of deceased patients (henceforward ‘group 1’ refers to the 39 deaths for which neither reviewer would have issued a MCCD, and ‘group 2’ refers to the 114 patient deaths for which at least one reviewer concurred with the certifying GP).
There were no differences between the two groups with respect to gender (72 per cent and 74 per cent of patients were female in groups 1 and 2, respectively), age (mean age 85.9 and 85.2 years, respectively), or place of death (89.7 per cent and 89.5 per cent died in a non-NHS communal establishment, respectively).
A greater proportion of deaths in group 1 had a cardiovascular underlying cause compared with group 2 (66.7 per cent versus 40.3 per cent, respectively; p = 0.04). Deaths from heart attack or stroke are more liable to be sudden than deaths from cancer and old age, and patients dying from these conditions may be less likely to have seen a physician in the weeks immediately before death. Therefore it might be expected that a higher proportion of deaths in group 1 were due to cardiac or cerebrovascular causes.
The proportion of deaths at which other persons were recorded as being present was smaller in group 1 than in group 2, although this difference was not statistically significant. There was no evidence of excessive controlled drug prescribing in group 1 deaths. Comparisons between the two groups with respect to other key variables assessed by the two independent reviewers are summarized in Table 5. Differences in proportions were examined using the 
2 test for association.
|
There were statistically significant differences between the two groups with respect to the reviewers’ assessment of the relationship between certified cause of death and the patient’s medical history. If such differences between the two groups were interpreted as genuine, then these findings would have provided further cause for concern and potentially heightened suspicion regarding this group of 39 deaths.
However, analysis of the reviewers’ assessment of patient record integrity (including open-ended comments made by each of the reviewers), in conjunction with detailed discussions between the investigation team (JB, NK and PO) and the two reviewers, revealed that these apparent differences actually reflected differences in the availability and quality of information recorded in the case notes of these two groups of patients.
On every dimension of record quality, the clinical notes relating to patients in group 1 were assessed as being of poorer quality than the notes of patients in group 2. In many instances, the reviewers highlighted a straightforward lack of information in the patient records, which prevented them from making a sound judgement regarding the appropriateness of death certification. In cases where the patient had been resident in a nursing home, the reviewers surmized that further details of patient care may have been recorded in separate notes held by the nursing home. The reviewers also noted a variation in practice between the five GPs with regard to printing out and inserting into the deceased patient’s paper records information held on practice computer systems prior to archiving. Many of the records, which contained only limited information, did not include a computerized printout, which suggested to the reviewers that additional information may have been available on practice computer systems.
|
Discussion |
|---|
|
TOPIntroductionMethodsResultsDiscussionAcknowledgementsReferences |
|---|
Our investigation confirmed that, after making appropriate adjustments for age and sex, death rates amongst patients registered with these five West Sussex GPs were higher than expected. More than one-third of their registered patient deaths occurred in nursing homes. This compares to a national figure of 10.5% of deaths occurring in non-NHS hospitals and nursing homes between 1998 and 2002.14
The number of care-home residents registered with each GP cannot be routinely ascertained from the Exeter database. Without this denominator information, it was not possible to calculate death rates for this particular patient population. However, the correlation between the proportion of deaths occurring in nursing homes and the SMR for each GP’s patient population suggested that high mortality rates were associated with nursing home deaths.
We concluded that the registration of large numbers of care-home residents, who are known to have high mortality,15–17 is the most likely explanation for these five GPs’ excessive mortality rates. This finding fits in with previous research in West Sussex, which demonstrated a significant correlation between nursing home deaths and mortality at electoral ward level.18 Our detailed review of deceased patient records provided no evidence of poor quality care or negligence. The review did, however, reveal some concerns regarding the quality of record keeping, and the completeness of patient records that are retained after death.
There are a number of limitations to our investigation. Perhaps most important of all is its restricted focus on deaths occurring in just one marker year for each GP. This cross-sectional ‘mortality snapshot’ tells us nothing about longitudinal trends over time. An expanded analysis of trends over time would have been more informative, but was prohibitively time and resource intensive. The process of identifying and assigning deaths to each GP’s patient list, which involved cross-referencing Exeter patient registration data against the PHMF, was particularly laborious and time-consuming. Analysis of mortality rates at GP or practice level would be greatly facilitated if details of a patient’s registered GP or practice were recorded as part of the death registration process.
Moreover, we did not compare the death certification rates of these five GPs to a group of control GPs, as was undertaken by Professor Baker in his audit of Dr Shipman.13 Our limited analysis highlights the considerable variation between GPs with respect to death certification rates. Choosing an appropriate control group of GPs for comparison would require detailed local understanding of those factors that influence death certification rates, namely patient case-mix and the organization of care within a practice. Relevant factors include whether GPs operate shared lists with their partners, whether they participate in out-of-hours deputizing services, or whether they provide care for particular institutions, such as hospices or care homes.
Also, lack of reviewer blinding may have introduced an element of observer bias. With additional time and resources, we could have attempted to mask the identity of the GPs from the reviewers and/or included sets of control records from deceased patients whose MCCDs were issued by other GPs.
Others have conducted investigations into GPs flagged up by the Shipman Inquiry as having excessively high mortality rates.11,19 Like us, these investigations concluded that high mortality rates were associated with a nursing-home effect. This conclusion raises two issues. First, it highlights the need for any national monitoring system to make adequate adjustments for patient case-mix associated with nursing-home populations, as well as other factors strongly associated with GP mortality rates, such as deprivation. Such adjustments are necessary to minimize the number of false-positive signals from such a monitoring system, and reduce the burden of unnecessary follow-up investigation. Accurate adjustments for nursing home populations would only be possible, however, if care-home residency was routinely captured, recorded and updated within the Exeter patient registration database. Secondly, this conclusion of a ‘nursing-home effect’ raises the question of what is an appropriate death rate amongst nursing-home residents. Quality of care can be poor in this environment.20 Hence, to conclude that high death rates are due to a nursing-home effect without any further understanding of the quality of care in nursing homes or of the number of deaths that would be ‘expected’ in a care home population is problematic.
Our investigation also highlighted the need to ensure that the quality and completeness of primary-care records, including those that are returned to PCOs when a patient dies, are regularly reviewed as part of local clinical governance processes. Moreover, in an era of computerized general practice, PCOs and individual practices need to be satisfied that adequate arrangements are in place for archiving deceased patients’ computer records, as well as paper records.
Whilst patient mortality is intuitively an appropriate performance indicator for surgeons undertaking procedures that carry a significant risk of death, the question of whether mortality is indeed a relevant and sufficiently sensitive indicator of GP performance is debatable. Other indicators of primary-care quality, especially when viewed in combination, such as evidence-based chronic disease management and prescribing indicators, or preventive intervention uptake rates, may be more useful in developing a rounded picture of the quality of care in this setting, and have the advantage of being more directly attributable to the practitioner or practice in question than mortality rates. If, however, mortality monitoring is to be specifically concerned with quality of care at or around the time of death, and the identification of GPs who are murdering their patients, then a focus on death certification rates and processes may be of more value. Unfortunately, surveillance of death certification rates poses a different set of methodological challenges, not least of which is establishing an appropriate control or reference rate for comparison.
Our investigation was a pragmatic attempt to identify reasons for high GP death rates using the expertise and resources available to a typical PCO. Local knowledge led us to investigate the possibility of an association between nursing home deaths and high mortality rates. The more qualitative investigative technique of case-notes review proved to be an essential component of the investigation. In particular the review enabled us to assess the quality of care provided by these five GPs, and increased our confidence in the final conclusion that there was no evidence of malpractice or negligence. Yet our investigation was resource-intensive, with direct and opportunity costs that ran into several thousands of pounds. The introduction of a national mortality monitoring system for GPs requires careful consideration of the consequences, not only for individual practitioners flagged up as having excessive mortality rates, but also for the PCOs that may be expected to conduct follow-up investigations, but in many case will lack the resources as well as the expertise to carry these out.
|
Acknowledgements |
|---|
|
TOPIntroductionMethodsResultsDiscussionAcknowledgementsReferences |
|---|
The work presented in this study was funded by Adur Arun and Worthing PCT, Crawley PCT, Mid-Sussex PCT and Western Sussex PCT.
|
References |
|---|
|
TOPIntroductionMethodsResultsDiscussionAcknowledgementsReferences |
|---|
- Mohammed MA, Cheng KK, Rouse A, Marshall T. Bristol, Shipman and clinical governance: Shewart’s forgotten lessons. Lancet 2001; 357: 463–467.[CrossRef][ISI][Medline]
- Spiegelhalter D, Grigg O, Kinsman R, Treasure T. Risk adjusted sequential probability ratio tests: applications to Bristol, Shipman and adult cardiac surgery. Int J Qual Health Care 2003; 15: 7–13.
[Abstract/Free Full Text] - Webb R, Esmail A. An analysis of practice-level mortality data to inform a health needs assessment. Br J Gen Prac 2002; 52: 296–299.[Medline]
- Lee E. Learning from death. Br J Gen Prac 2002; 52: 267–268.[ISI][Medline]
- Baker R, Jones D, Goldblatt P. Monitoring mortality rates in general practice after Shipman. Br Med J 2003; 326: 274–276.
[Free Full Text] - Aylin P, Best N, Bottle A, Marshall C. Following Shipman: a pilot system for monitoring mortality rates in primary care. Lancet 2003; 362: 485–491.[CrossRef][ISI][Medline]
- Frankel S, Sterne J, Davey-Smith G. Mortality variations as a measure of general practitioner performance: implications of the Shipman case. Br Med J 2000; 320: 489.
[Free Full Text] - Scanlon T. Mortality in general practice. Br J Gen Prac 2002; 52(479): 499.
- Poloniecki J, Sismanidis C, Bland M, Jones P. Retrospective cohort study of false alarm rates associated with a series of heart operations: the case for hospital mortality monitoring groups Br Med J 2004; 328: 375–379.
[Abstract/Free Full Text] - Fitzpatrick M. Auditing deaths. Lancet 2003; 362: 586.[Medline]
- The Shipman Inquiry. Fifth Report (Vol 1) Safeguarding patients: lessons from the past – proposals for the future. London: The Stationery Office, 2004.
- Aylin P, Best N, Bottle A, Marshall C. Monitoring of mortality rates in primary care. London: Imperial College of Science, Technology & Medicine, 2003. Available at http://www.the-shipman-inquiry.org.uk/mediainfo.asp?ID=77 (last accessed June 2004).
- Baker R. Harold Shipman’s Clinical Practice 1974–1998. London: Stationery Office, 2001.
- Office for National Statistics. Series DH1 (31–35): Mortality Statistics – General. London: The Stationery Office, 2000–2004.
- Bebbington A, Brown P, Darto R, Miles K, Netten A. Longitudinal study of elderly people admitted to residential and nursing homes: 30 months on. Canterbury, Kent: Personal Social Services Research Unit, University of Kent, 1999.
- Congdon P. Nursing home mortality: patterns and trends. In: Barking & Havering Health Authority. Public Health Research Report. London: Barking and Havering HA, 2002: 141. Available at http://www.bhha.org.uk/141.pdf (last accessed September 2004).
- Raines JE, Wight J. The mortality experience of people admitted to nursing homes. J Public Health Med 2002; 24: 184–189.
[Abstract/Free Full Text] - Williams E, Dinsdale H, Eayres D, Tahzib F. Impact of nursing home deaths on life expectancy calculations in small areas. J Epidmiol Community Health 2004; 58: 958–962.
[Abstract/Free Full Text] - Mohammed M, Rathbone A, Myers P, Patel D, Onions H, Stevens H. An investigation into general practitioners associated with high patient mortality flagged up though the Shipman inquiry – retrospective analysis. Br Med J 2004; 328: 1474–1477.
[Abstract/Free Full Text] - Fahey T, Montgomery AA, Barnes J, Protheroe J. Quality of care for elderly residents in nursing homes and elderly people living at home: controlled observational study. Br Med J 2003; 326: 580–585.
[Abstract/Free Full Text]
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  R. Dobson Inquiry into high mortality of GPs' patients finds no wrongdoing BMJ, July 23, 2005; 331(7510): 180 - 180. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||