Skip Navigation



Journal of Public Health Advance Access published online on December 20, 2007
Journal of Public Health, doi:10.1093/pubmed/fdm081
This Article
Right arrow Abstract Freely available
Right arrow FREE Full Text (PDF) Freely available
Right arrowOA All Versions of this Article:
30/1/69    most recent
fdm081v1
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Archive
Right arrow Download to citation manager
Right arrow Disclaimer
Google Scholar
Right arrow Articles by McCarthy, M.
Right arrow Articles by Rachet, B.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by McCarthy, M.
Right arrow Articles by Rachet, B.
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us  
What's this?

© The Author 2007, Published by Oxford University Press on behalf of Faculty of Public Health. All rights reserved
The online version of this article has been published under an open access model.

Is the performance of cancer services influenced more by hospital factors or by specialization?


Mark McCarthy1,
Preeti Datta1
Chris Sherlaw-Johnson1
Michel Coleman2
Bernard Rachet2

1 Department of Epidemiology and Public Health, University College London, 1–19 Torrington Place, London WC1E 6BT, UK
2 London School of Hygiene and Tropical Medicine, Keppel Street, London, WC1E 7HT, UK


Address correspondence to Mark McCarthy, E-mail: m.mccarthy{at}ucl.ac.uk

The Cancer Plan for England, introduced in 2000, has promoted cancer service specialization. We have investigated how far specialization and general hospital factors each contributed to service performance for four common cancers—breast, colorectal, lung and prostate—at the time of the Cancer Plan.

Performance measures of service standards, waiting time to treatment, satisfaction with care, in-hospital mortality and population-level survival were identified from secondary data sets for 167 acute hospitals and 34 cancer networks in England. We correlated rankings of networks and hospitals between the data sets using non-parametric statistics. At cancer network level, peer-review service standards were associated (P < 0.05) with 1-year survival for colorectal and lung cancers, and waiting times for lung cancer. At hospital level, standards were associated (P < 0.01) with waiting time to treatment for breast and colorectal cancers. However, there were stronger associations between specializations within hospitals: rankings of breast, colorectal and prostate cancers were highly associated (P < 0.001) for 5-year survival, patient satisfaction, standards and in-hospital mortality. Hospital-level differences appear to contribute more to variations in cancer performance than specialization differences within hospitals. The findings may be used for planning and commissioning better cancer services.

Keywords: cancer services, hospital, management, performance, planning, specialization


   Introduction
TOP
 Introduction
Methods
 Results
 Discussion
 Conclusion
 Acknowledgments
 References
 
In the UK National Health Service, patients usually first consult their general practitioner and may then be referred for hospital diagnosis, and treatment if necessary. Although cancer is diagnosed in primary care or general hospitals, it is treated in both general hospitals and tertiary care (e.g. radiotherapy and oncology services). Within a broader process of cancer services development, the Cancer Plan for England1 in 2000 proposed the development of specialized cancer services for different tumour types, linking general hospitals with tertiary care centres, increasing specialization in treatment and multi-disciplinary teams. Thirty-four cancer networks were created, confirming geographical and organizational links between hospitals for referral and treatment, and serving populations of between a half and three million. Advice on arrangements for clinical treatment has been set out in the Manual of Cancer Service Standards.2,3

Within clinical services, specialization may provide benefits to both doctors and patients. Specialized clinical teams have greater experience in their use of resources, and in managing variations in clinical condition. In prospective audit studies, specialist cancer services have been shown to achieve better clinical outcomes than generalists.4,5 Also, association has been shown between clinical outcomes and the volume of patients treated within the specialty.6,7

But can hospital-level factors contribute to cancer services performance as well? Studies of cancer outcomes in hospitals have generally used prospectively gathered clinical data, based on a single operation, disease or specialty. Hospital administrative data may be less detailed than clinical studies, but can be more complete8,9 and allow comparison across specialties. In Canada, Urbach and Baxter10 used administrative data to compare 30-day hospital mortality for five different operations across specialties. They found that mortality for a highly specialized operation, pancreatico-duodenectomy, was lower in regional hospitals than in rural low-volume hospitals, but was also lower in regional hospitals with a high-volume lung-resection compared to other high-volume pancreatico-duodenectomy hospitals. The authors suggested that ‘the lack of specificity of volume-outcome associations may indicate a more general relation between the overall volume of complex surgery done in a hospital and outcomes’.

In support of the Cancer Plan for England, the Department of Health commissioned research to investigate the use of existing national data sets for service comparisons.11 The research brief asked whether performance was related more to the hospital level, or to the level of the specialist services within the hospital. ‘Within a specific tumour type, do Trusts perform consistently well across a range of quality indicators? Do Trusts perform consistently well across a range of tumour types?’ We investigated this question using measures from five independent national data sets which recorded data for both by hospital and cancer specialization.


   Methods
TOP
 Introduction
 Methods
Results
 Discussion
 Conclusion
 Acknowledgments
 References
 
At the time of introducing the Cancer Plan, two non-routine surveys of cancer services were made. We drew on these, along three other sets of routinely collected data, to provide a multi-dimensional picture of cancer services. Data were available for four common cancers—breast, colorectal, lung and prostate (except no data on standards for prostate cancer)—for 167 NHS acute hospitals. (The term ‘hospital’ in this paper refers to the managerial grouping of affiliated local hospitals currently called within the NHS a ‘hospital trust’.) For cancer survival, deaths after age-sex adjusted relative survival analysis could only be compared statistically at the more aggregate level of cancer networks, and for comparisons by network we made averages of the hospital data. The data sets were assessed for their completeness and validity using DoCDat,12 a standardized inventory for clinical databases. All items had at least 80% completeness and most more than 95%.

Data sets
Cancer Service Standards by tumour-type were based on professional advice and specified in the Manual of Cancer Service Standards.2 In 2001, peer reviewers in each NHS region rated compliance with standards for every acute hospital in England. Comparable data were available for 152 hospitals (one region, Trent, covering four cancer networks and 15 hospitals did not collect the data in the standard way). We dichotomized the ratings absent/partially absent versus fully present. Between 36 and 39 standards were recorded about the organization of each tumour-specific service and we used an un-weighted sum of the scores. A higher total score indicates better compliance with the standards set.

Hospital Episode Statistics are a continuous data set of all admitted patients treated in NHS hospitals in England, and are held by the Department of Health.13 Each record contains a variety of administrative, clinical and patient information describing the care and treatment a patient received while in a hospital. We used a single dimension from this data set, in-hospital mortality by cancer type, which is drawn from the data recorded at discharge. We have inferred that lower mortality is better performance.

Cancer Waiting Times are collected by hospital trusts on patients referred by general practitioners with suspected cancer. (The data therefore include patients who turn out not to have cancer, and do not include patients diagnosed with cancer by another route.) The data are submitted quarterly by each hospital trust to the national Department of Health.14 The target level recorded for 2001–02 was the proportion of patients admitted in less than 15 days. A higher proportion is better performance.

The National Cancer Patient Survey was undertaken during 2001 to assess the experience of care of patients with common cancers discharged from acute hospitals in England in 1999–2000.15 The authors of the study had made a factor analysis16 which identified 10 leading dimensions with single questions to describe different aspects of the patient pathway of care covering the range of experience of before, during and after admission. We averaged the original responses (15 891 colorectal, 4011 lung, 25 772 breast, 10 992 prostate) to provide single survey scores by hospital across the four tumour types. A lower score (i.e. less dissatisfaction) indicates better performance.

Cancer Survival. 1- and 5-year relative survival for patients diagnosed in England between 1996 and 2001 (followed up to the end of 31 December 2002) were calculated from data provided by the national cancer registry.17 Survival data are estimated to include 90–97% of all cancer patients. For sample size reasons (in relation to the sub-groups needed for age and sex standardization) we only used survival estimates at cancer network level. A higher proportion surviving indicates better performance.

Statistics
All the measures showed significant variations across hospitals and networks. For comparisons, however, normal distributions could not be assumed, and rank correlations were tested, using Spearman's test where both variables were continuous, and Pearson's point-biserial test for cancer standards when one variable was a dichotomous variable and the other a continuous variable. Kendall's W-test was used to test agreement between each variable for the four tumour types together (Tables 1 and 2) and for each of the six performance measures (Tables 3 and 4). The study was approved by the South East Regional Ethics Committee, England.


View this table:
[in this window]
[in a new window]

  		Table 1 Association (rank correlation) between combinations of performance measures by tumour type at cancer network level

 


View this table:
[in this window]
[in a new window]

  		Table 2 Association (rank correlation) between combinations of performance measures by tumour type at hospital level

 


View this table:
[in this window]
[in a new window]

  		Table 3 Association (rank correlation) between tumour types for data set variables at network level

 


View this table:
[in this window]
[in a new window]

  		Table 4 Association (rank correlation) between tumour types for performance measures at hospital trust level

 

   Results
TOP
 Introduction
 Methods
 Results
Discussion
 Conclusion
 Acknowledgments
 References
 
There were relatively few statistically significant associations between the different data sets at cancer network and hospital levels. At cancer network level (Table 1), 1-year survival for colorectal cancer (r = 0.41, P = 0.03) and lung cancer (r = 0.43, P = 0.03) were positively associated with total standards score, i.e. there was higher short-term survival in networks with higher compliance to standards. There was also a non-significant association for colorectal cancer (r = 0.32, P = 0.10) for 5-year survival, but no association for breast cancer at either length of follow-up. Satisfaction, however, showed unexpected trends in the opposite direction: breast cancer 1-year survival (r = 0.34, P = 0.47) and lung cancer 5-year survival (r = 0.42, P = 0.014) were positively associated with total satisfaction score, i.e. higher short-term survival was associated with greater dissatisfaction. One-year relative survival showed no association with in-hospital mortality, while there was a significant inverse association (r = –0.39, P = 0.02) for lung cancer 5-year survival and in-hospital mortality. Waiting times to treatment were not associated with survival or satisfaction, although for lung cancer (r = 0.48, P = 0.01) there was a significant association between waiting times and cancer standards. No associations at all were found for prostate cancer. At hospital trust level (Table 2), the association between standards and waiting times was significant for breast and colorectal cancers (both r = 0.27, P = 0.003), but not significant for lung cancer (r = 0.17, P = 0.07). There were no significant associations between any measures and in-hospital mortality or satisfaction scores.

In contrast, there were strong associations between the different measures for tumour types within the same hospital. At cancer network level (Table 3), breast and colorectal cancers showed strong associations for all measures, with the range of values from waiting times just not significant (r = 0.33, P = 0.054) to satisfaction highly significant (r = 0.73, P <0.001). Breast and prostate cancer showed significant associations for all measures (without standards score), and colorectal and prostate cancers also showed four highly significant values (from r = 0.52, P = 0.001 to r = 0.46, P = 0.007). For the standards score, lung cancer was strongly associated with breast cancer (r = 0.62, P = 0.001) and colorectal cancer (r = 0.51, P = 0.006), and for other measures there were several significant, though less strong, associations. At hospital level (Table 4), nearly all the pairs of tumour types (survival excluded for this analysis) showed significant associations. However, again satisfaction score for lung cancer was not associated with satisfaction scores for breast and prostate cancers.


   Discussion
TOP
 Introduction
 Methods
 Results
 Discussion
Conclusion
 Acknowledgments
 References
 
Main finding
We have compared rankings of five independent measures of organizational performance for cancer hospitals and networks in the period of the start of the Cancer Plan for England. Performance measures for hospitals differed more between each other than between the cancer services within them. This suggests that the characteristics of a hospital itself may make an important contribution to cancer services performance.

What is already known
Evidence indicating that health-care system factors can affect clinical performance over and above individual practice has been reviewed for critical care services,18 and a literature review of organizational factors in palliative care has been published.19 The Improving Outcomes Guidance manuals for specific cancers20–22 in England, which draw evidence from the academic literature, are clinical in focus, while the organizational standards given in the Manual of Cancer Services in England, are based on opinion.3 Urbach and Baxter10 noted that most clinical studies have compared surgeon and hospital survival for individual tumour types, but few have looked for hospital-level effects in cancer treatment by comparing hospital performance across different cancers. Their study in Canada differed from ours in the types of cancer, the characteristics of hospitals and using only in-hospital mortality. However, we support their viewpoint of considering hospital-level effects as well as individual cancer service effects when evaluating performance of services.

What this study adds
The performance measures did vary to some extent within tumour types. Comparing cancer networks, there was a strong association between 1-year survival and compliance with standards for colorectal and lung cancer, although a lack of association for breast cancer was unexpected. Both breast and lung cancer showed significant associations between (longer) survival and (greater) dissatisfaction. An explanation for these associations through covariance is not clear. In the national survey of cancer patients,15 dissatisfaction was greater in women, younger people and ethnic minorities, but these are not related to better survival. Socio-economic position was not analysed in the survey, while a systematic review of 139 patient experience studies found association with age but not gender, ethnicity or socio-economic group.23 The association between waiting time satisfaction and achievement of cancer service standards for breast and colorectal cancers would be expected. The lack of association between higher satisfaction and higher proportion achieving the waiting times standard is less understandable. However, as has been noted, the satisfaction survey was drawn from all patients discharged with a cancer diagnosis, while waiting times data relate to those referred by a GP for treatment: as there are other pathways to a final cancer diagnosis, the two groups of patients would only partly overlap.

The performance measures we used were drawn from a range of sources, and we used only single dimensions. The waiting times for treatment are a sub-set of larger issues of access to services. We looked at cancer standards across specialties, but there are many aspects of hospitals more generally that could be investigated further, for example, in relation to staffing, information flows or research activities. The patient survey recorded responses across various aspects of care, which deserve investigation. Our measure of in-hospital mortality is limited because it depends on hospital discharge policies: hospitals will vary in the extent they are able, or wish, to discharge cancer patients home for terminal care. On the other hand, population-based survival drawn from cancer registries will include a proportion of terminal patients who may not have received in-hospital care.

Limitations of this study
Critical aspects of the study include the observational, cross-sectional design, the use of secondary data, the need for comparisons at aggregate rather than individual level and multiple statistical testing. Clinical studies based on prospective randomized design provide evidence of the efficacy of a particular treatment; but cannot explore the effects of different settings unless such data are deliberately collected, and observational designs are usually needed for this area of work.24 The data sets available for the study all related to the period of 2000–01, but a cross-sectional design is less strong than a prospective study. Clinical series can be flawed, because of incomplete data and patient selection, compared with hospital administrative data.8,9 Using secondary data also has the advantage of being able to compare multiple sites and use a range of performance measures. The limitation of aggregate data for our study, however, included the smaller number of units possible for comparisons (the cancer networks and hospitals) compared with individuals in a clinical study. Yet administrative factors operate at the aggregate level, and statistical strength must therefore come through making national rather than local comparisons. At the risk of bias from multiple statistical tests (which may be hidden in multi-variate analysis), we have presented our results as bi-variate correlations. Correlations between specialties (Tables 3 and 4) are substantially higher than correlations between different dimensions of hospital performance (Tables 1 and 2). We interpret this as hospital-level characteristics having greater impact on cancer service performance measures than (sub-) specialization.


   Conclusion
TOP
 Introduction
 Methods
 Results
 Discussion
 Conclusion
Acknowledgments
 References
 
The data in this study reflect the period at the beginning of the Cancer Plan for England. It would be appropriate to make analyses of routine data sets for the years of implementation of the Plan to investigate how variations have changed, and chart the comparisons between hospitals and their specialist services. There has been no repeat of the national patient survey, but a second survey of hospital cancer standards was completed in 2006. Research funding from the national charities and the government is coordinated, especially in cancer,25 and there is currently a review of the Cancer Plan by the Department of Health.26 In the light of our findings, beyond focusing on development of specialty teams, the Department of Health Cancer Team may wish to work with hospital managers and researchers in identifying what hospital level organizational factors create variations of cancer service performance, and how investment of NHS resources can best impact on service outcomes.


   Acknowledgments
TOP
 Introduction
 Methods
 Results
 Discussion
 Conclusion
 Acknowledgments
References
 
This study is based on research funded by the Service Delivery and Organisation Programme of the English Department of Health Research and Development Division, and all authors were members of that study team. Mark McCarthy conceived, oversaw the study and wrote the paper. Preeti Datta undertook statistical analyses. Chris Sherlaw-Johnson oversaw the statistical analysis. Michel Coleman and Bernard Rachet made the survival analyses. We are grateful to colleagues in our project team, including Dawn Wilkinson, Artak Khachatryan and Marina Thomas, and our advisory group including Celia Ingham-Clark, Jill Turner, Cheryl Cavanagh and Phil Hill.


   References
TOP
 Introduction
 Methods
 Results
 Discussion
 Conclusion
 Acknowledgments
 References
 

  1. Department of Health. The Cancer Plan for England (2000) London: Department of Health.
  2. Department of Health. Manual of Cancer Service Standards (2000) London: Department of Health.
  3. Department of Health. Manual of Cancer Standards (2004) 2nd edn. London: Department of Health.
  4. Smith JA, King PS, Lane RH, et al. Evidence of the effect of ‘specialization’ on the management, surgical outcome and survival from colorectal cancer in Wessex. Br J Surg (2003) 90(5):583–92.[CrossRef][ISI][Medline]
  5. Kingsmore D, Hole D, Gillis C. Why does specialist treatment of breast cancer improve survival? The role of surgical management. Br J Cancer (2004) 90:1920–5.[CrossRef][ISI][Medline]
  6. Harmon JW, Tang DG, Gordon TA, et al. Hospital volume can serve as a surrogate for surgeon volume for achieving excellent outcomes in colorectal resection. Ann Surg (1999) 230:404–11.[CrossRef][ISI][Medline]
  7. Mikeljevic JS, Haward RA, Johnston C, et al. Surgeon workload and survival from breast cancer. Br J Cancer (2003) 89(3):487–91.[CrossRef][ISI][Medline]
  8. Iezzoni L. Assessing quality using administrative data. Ann Intern Med (1997) 127(8 Pt 2):666–74.[Abstract/Free Full Text]
  9. Fine L, Keogh BG, Cretin S, et al. How to evaluate and improve the quality and credibility of an outcomes database: validation and feedback study on the UK Cardiac Surgery Experience. BMJ (2003) 326:25–8.[Abstract/Free Full Text]
  10. Urbach DR, Baxter NN. Does it matter what a hospital is "high volume" for? Specificity of hospital volume-outcome associations for surgical procedures: analysis of administrative data. BMJ (2004) 328:737–40.[Abstract/Free Full Text]
  11. Department of Health. Service Delivery and Organisation programme: research brief. Cancer services quality: CA1 – ‘Assessment of Interrelationships between different measures of the quality of cancer services’ (2003) London: Department of Health. http://www.sdo.lshtm.ac.uk/files/researchcall/65-brief.pdf.
  12. Black N, Payne M. Directory of clinical databases: improving and promoting their use. Qual Saf Health Care (2003) 12(5):348–52.[Abstract/Free Full Text]
  13. Department of Health. Hospital Episode Statistics. http://www.dh.gov.uk/en/Publicationsandstatistics/Statistics/HospitalEpisodeStatistics/index.htm.
  14. Department of Health. Cancer Waiting Times. http://www.performance.doh.gov.uk/cancerwaits/index.htm.
  15. Department of Health. 28173/National Surveys of NHS Patients – Cancer: National Overview (2002) London: Department of Health.
  16. Prescott A. National survey of NHS patients. Cancer: analysis by theme (2004) London: Department of Health. http://www.dh.gov.uk/en/Publicationsandstatistics/Publications/PublicationsStatistics/DH_4080519.
  17. Department of Health. Cancer registries. http://www.dh.gov.uk/en/Policyandguidance/Healthandsocialcaretopics/Cancer/DH_4068586.
  18. Angus DC, Black N. Improving care of the critically ill: institutional and health-care system approaches. Lancet (2004) 63:1314–20.
  19. Hearn J, Higginson IJ. Do specialist palliative care teams improve outcomes for cancer patients? A systematic literature review. Pall Med (1998) 12(16):317–32.[CrossRef]
  20. NICE. Improving outcomes in breast cancer (2002) London: National Institute for Clinical Excellence.
  21. NICE. Improving outcomes in colorectal cancers (2004) London: National Institute for Clinical Excellence.
  22. National Collaborating Centre for Acute Care 2005 The diagnosis and treatment of lung cancer. (2005) London: The Royal College of Surgeons of England.
  23. Crow R, Gage H, Hampson S, et al. The measurement of satisfaction with healthcare: implications for practice from a systematic review of the literature. Health Technol Assess (2002) 6(32):1–244.[Medline]
  24. Black N. Why we need observational studies to evaluate the effectiveness of health care. BMJ (1996) 312:1215–8.[Free Full Text]
  25. National Cancer Research Institute. Strategic Plan 2005-2008 (2005) London: National Cancer Research Institute.
  26. Department of Health. Cancer Reform Strategy board members announced (2007) 14:30. Press Release, Monday 12 February.

Add to CiteULike CiteULike   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us    What's this?



This Article
Right arrow Abstract Freely available
Right arrow FREE Full Text (PDF) Freely available
Right arrowOA All Versions of this Article:
30/1/69    most recent
fdm081v1
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Archive
Right arrow Download to citation manager
Right arrow Disclaimer
Google Scholar
Right arrow Articles by McCarthy, M.
Right arrow Articles by Rachet, B.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by McCarthy, M.
Right arrow Articles by Rachet, B.
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us  
What's this?