Journal of Public Health Advance Access originally published online on September 6, 2006
Journal of Public Health 2006 28(4):351-354; doi:10.1093/pubmed/fdl050
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Computer technology and the childhood immunization programme: an investigation of low uptake rates in Coventry Primary Care Trust
A. K. Lyon, Public Health Trainee1,*
M. Bardhan, Consultant in Public Health2
P. Barker, Consultant in Public Health2
P. Wilson, Former Information Services Manager3
1 Public Health Directorate, South Birmingham PCT, Birmingham, B13 8JL, UK
2 Public Health Directorate, Coventry PCT, Coventry, CV1 2GQ, UK
3 Information Services, Coventry PCT, Coventry, CV1 2GQ, UK
Address correspondence to A. K. Lyon, E-mail: akly0n{at}yahoo.co.uk
Background The childhood vaccination programme in Coventry Primary Care Trust (PCT) has resulted in consistently low uptake rates for a number of years. After an evaluation of operational processes, an examination of the data systems was performed.
Methods The under-reporting of vaccinations performed and methodology issues were investigated using data collected by the Child Health Information Department (CHID) and the Finance Department.
Results Data held by the Finance Department result in an uptake rate of between 0.79 and 1.92% higher than those held by the CHID. Locally, the Southwest Child Health (SWIFT) system, the programme used to calculate uptake rates by many of the PCTs across the country, excludes those children residing outside the boundary but registered within it. Coventry PCT’s Cover of Vaccination Evaluated Rapidly (COVER) statistics are based on a subgroup of the responsible population.
Conclusions The computer technology currently utilized by the NHS provides inaccurate statistics for the COVER programme. Systematic under-reporting to the CHID results in moderately lowered uptake rates. A programming anomaly in the SWIFT system has resulted in the collection of data based on a subgroup of the responsible population. This undermines the validity of the statistics collected and renders comparability of data between different PCTs, particularly those using different systems, difficult.
Keywords: child health information system, childhood immunisation programme, computer technology, uptake rate
 |
Introduction |
|---|
 TOP Introduction MethodResultsDiscussionConclusionAcknowledgementsReferences |
|---|
The Cover of Vaccination Evaluated Rapidly (COVER) programme monitors immunization coverage data for children who reach their first, second or fifth birthday during each evaluation quarter. The uptake figures for Coventry Primary Care Trust (PCT) have been consistently low for a number of years, and after an evaluation of operational processes, an examination of data systems was performed. Of the hypotheses tested, the use of computer technology was found to be crucial to two factors, the under-reporting of vaccinations actually performed and the method used to produce the COVER statistics themselves. The role of computer technology in relation to these factors has not previously been reported in the literature.
|
Method |
|---|
|
TOPIntroductionMethodResultsDiscussionConclusionAcknowledgementsReferences |
|---|
To test the hypothesis that vaccinations that have been performed were not being reported to the Child Health Information Department (CHID), we compared a list held by the CHID of children aged 2 years between January and March 2004 with an incomplete vaccination status with a list held by the Finance Department of completed vaccination courses claimed for by the general practitioner (GP) for the same period. This investigation was performed for one randomly selected general practice per locality, of which there are 13 within Coventry.
To test the hypothesis that uptake rates were being calculated in Coventry PCT using a different method to that of surrounding and comparator PCTs, we contacted the relevant personnel in Coventry, South Warwickshire, Solihull, North Warwickshire/Rugby, Walsall and Wolverhampton. Having identified an error with the methodology, we then ran further tests using the database of a neighbouring PCT.
|
Results |
|---|
|
TOPIntroductionMethodResultsDiscussionConclusionAcknowledgementsReferences |
|---|
Comparison of the lists held by the Coventry CHID and the Finance Department revealed that under-reporting to the CHID occurred across all vaccination courses, as summarized in Table 1. Including those children reported to the Finance Department but not to the CHID as having received the vaccinations resulted in a minimum increase in the uptake rates of 0.79% (diphtheria, pertussis, tetanus and polio) and a maximum increase of 1.92% [Measles, Mumps and Rubella (MMR)]. A range is given as not all children on the CHID list were identified on the list provided by the Finance Department. The lower limit includes only those children identified as having had the vaccination; the upper limit also includes those children for whom no information was found. Immunization for meningitis C is not reported to the Finance Department and so has not been assessed.
|
Consultation with the selected PCTs revealed that, locally, the majority of trusts utilize the Southwest Child Health System (SWIFT). Using this system, the PCT-responsible is identified population by the interaction of two factors—the base status and the primary factor. The base status is set nationally and is meant to identify those children of the relevant age for the quarter; the primary factor is set locally and identifies the PCT-responsible population. The base status function operates first, selecting the group from which the primary factor identifies those for whom the PCT is responsible. This investigation revealed that the method utilized by Coventry PCT to calculate the uptake rates does not differ substantially from that of the other PCTs. However, an anomaly in the base status code was identified. The base status code (set as ‘0139’) selects all those children born and living in the area (0), births that have been transferred into the area (1), children who have moved into the area (3) and those children who are resident in the area but are treated outside of it (9). From this group, the primary factor makes its selection, resulting in the identification of a subgroup of the responsible population (013). The COVER statistics exclude those children registered in the area but resident elsewhere; these children are identifiable by using the base status code 8.
To further investigate the effect of changing the base status code to correctly identify the PCT-responsible population, we reran the COVER statistics for a neighbouring PCT using the October–December 2005 quarter. Data from Coventry PCT were not used due to problems with the system. Using the 0138 coding, it was found that the number of children identified by the system increased by up to 17 and that the percentage of the population vaccination could be affected in either direction (see Table 2). However, the commonest effect was to increase the percentage vaccinated by a fraction of a per cent.
|
|
Discussion |
|---|
|
TOPIntroductionMethodResultsDiscussionConclusionAcknowledgementsReferences |
|---|
Main finding of this study
This study shows that the computer technology utilized by the NHS provides inaccurate statistics for the COVER programme. This study demonstrates how technology, in the case of under-reporting to the CHID, could be used to improve the accuracy of the statistics reported to the COVER programme; it also shows how programming factors can undermine the validity and comparability of the statistics.
What is known already
Although a number of published articles have examined the childhood immunization programme, to our knowledge none have investigated errors that may be exacerbated by, or arise due to, computer technology.1–7
What this study adds
This study shows that the validity of the COVER statistics can be both improved and undermined by computer technology. The impact of human factors, such as preferential reporting to the Finance Department, which has arisen because of separate reporting procedures and the obvious incentive for reporting to the Finance Department, could be reduced using computer technology. Improving the accuracy of the CHID’s records would have the effect of improving the uptake rates of individual PCTs.
The CHID’s records could be improved by streamlining the reporting systems so that GPs only have to report to one department [for example, the CHID which would then process and forward the information to the Finance Department for payment and quality and outcomes framework (QOF) purposes] or by enabling the different computer systems to exchange data directly. However, a better solution would be to have the immunization programme as a subset of the PCT register. The two systems are separate because of organizational history; this is not relevant now. Making the Child Health System a subset of the main PCT register, in a fashion similar to the Cervical Cytology System, would eliminate dual reporting and have the additional benefit of eliminating the many hours put in by information staff to ‘look up’ new changes to the main register such as new additions, moves, transfers between GPs, etc. Furthermore, having one computer system may benefit a future Human Papilloma Virus (HPV) immunization programme, in terms of observing HPV-related disease epidemiology.
Computer technology can also undermine the accurate production of statistics; this study has demonstrated that the nationally set base status for the SWIFT system results in calculation of the uptake rate for a subset of the PCT-responsible population. The Health Protection Agency defines the PCT-responsible population (on the website the term ‘PCT relevant’ is used) as ‘all children registered with a GP whose practice forms part of the PCT, regardless of where the child is resident, plus any children not registered with a GP, who are resident within the PCT’s statutory geographical boundary’.8 The current base status coding is probably a leftover from the previous era when the statistics were collected on a resident basis from health authorities.8 The base status code could incorporate the PCT-responsible population by replacing the 9 (resident in, treated out) with an 8 (resident out, treated in) resulting in a code of 0138. Children who are not registered with a GP will still be excluded from this system; the issue of identifying unregistered, resident children was not tackled by this study.
This study has shown that the uptake rates may go up or down following a change in the base status code. The impact of altering the base status code is likely to differ between PCTs and may benefit some trusts but not others. If some PCTs are utilizing the nationally set code (0139) whilst others are experimenting with 0138, then comparison of the performance of trusts is not taking place on an equal basis. This is also relevant to those trusts using programmes other than the SWIFT system. It is beyond the scope of this study to suggest how widespread the use of 0139 versus 0138 is nationally or to make a thorough evaluation of the significance of this error, but this is a matter that warrants further investigation.
Limitations of this study
Limitations of this analysis relate to the number of children on the CHID list who were not identified on the list provided by the Finance Department. This analysis was performed using the finance list relating to vaccination status at 1 April 2004. It is possible that details of the missing children were reported on lists relating to later quarters; examination of data from these lists may result in a better estimation of the size of the discrepancy between the two recording systems. In addition, the notes held by the GPs of the children recorded with different vaccination statuses in the two systems could be examined to confirm which system is more accurate. The conclusion from this analysis is based on the assumption that vaccinations claimed for have actually been performed. The generalizability of these results is dependent upon other areas having similar reporting systems and incentive schemes for GPs as those that operate in Coventry.
|
Conclusion |
|---|
|
TOPIntroductionMethodResultsDiscussionConclusionAcknowledgementsReferences |
|---|
The COVER programme forms part of the surveillance system for monitoring vaccine-preventable diseases. Accurate figures facilitate evaluation of the vaccination programmes and allow vaccine coverage to be estimated. The latter is important in assessing population susceptibility to disease that, in turn, enables the effective targeting of resources. This is particularly pertinent considering the ongoing problems with MMR. Systematic under-reporting and errors in identifying the PCT-responsible population are issues that should be addressed nationally to ensure that an accurate picture of vaccination uptake rates is obtained.
|
Acknowledgements |
|---|
|
TOPIntroductionMethodResultsDiscussionConclusionAcknowledgementsReferences |
|---|
We thank Brenda Gurnham of North Warwickshire PCT for her contributions to this study.
|
References |
|---|
|
TOPIntroductionMethodResultsDiscussionConclusionAcknowledgementsReferences |
|---|
- Sharland M, Atkinson P, Maguire H et al. Will London’s COVER meet the target? Commun Dis Rep CDR Rev 1995;5(4):R53.[Medline]
- White JM, Leon S. COVER (cover of vaccination evaluated rapidly): 34. Commun Dis Rep CDR Rev 1995;5(7):R105–6.[Medline]
- White JM, Rush M, Leon S et al. COVER/Körner 95-1 (April to June 1995). Vaccination coverage statistics for children up to 2 years old in the United Kingdom. Commun Dis Rep CDR Rev 1995;5(12):186–7.
- Sharland M, Atkinson P, Maguire H et al. Lone parent families are an independent risk factor for lower rates of childhood immunisation in London. Commun Dis Rep CDR Rev 1997;7(11):R169–72.[Medline]
- Anonymous COVER programme: October to December 1999 vaccination coverage statistics for children up to 5 years of age in the United Kingdom. Commun Dis Rep CDR Wkly 2000;10(12):109–10.[Medline]
- Harrington PM, Woodman C, Shannon WF. Low immunisation uptake: is the process the problem? J Epidemiol Community Health 2000;54(5):394–400.
[Abstract/Free Full Text] - Kendrick D, Hewitt M, Dewey M et al. The effect of home visiting programmes on uptake of childhood immunization: a systematic review and meta-analysis. J Public Health Med 2000;22(1):90–8.
[Abstract/Free Full Text] - Health Protection Agency. About the COVER Programme. http://www. hpa.org.uk/infections/topics_az/vaccination/cover_intro.htm (28 February 2005, date last accessed).
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||