It’s in your hands: the value of handsearching in conducting systematic reviews of public health interventions
Rebecca Armstrong
Rebecca Armstrong, Research and Training Officer, Cochrane Health Promotion and Public Health Field, Melbourne, Australia.
Nicki Jackson
Nicki Jackson, Lecturer, Auckland University of Technology, Auckland, New Zealand.
Jodie Doyle
Jodie Doyle, Field Coordinator, Cochrane Health Promotion and Public Health Field, Melbourne, Australia.
Elizabeth Waters
Elizabeth Waters, Professor, Chair of Public Health, Director, Cochrane Health Promotion and Public Health Field, Melbourne, Australia; School of Health and Social Development, Deakin University, Melbourne, Australia.
Faline Howes
Faline Howes, PhD, Cochrane Health Promotion and Public Health Field, Melbourne, Australia.
Address correspondence to Rebecca Armstrong. Email: rarmstrong{at}vichealth.vic.gov.au
Background While there is an emerging evidence base in public health, the evidence can often be difficult to find. Indexing of journals in MEDLINE has assisted those conducting systematic reviews to more easily identify published studies. However, information technology and the processes associated with indexing are not infallible. Studies may not be correctly marked by study design which may mean they are missed in the electronic searching process. Handsearching for evidence of intervention effectiveness has therefore become a recognized tool in the systematic review process.
Methods Resources to guide handsearching activity currently are clinically focused, and may not be sensitive to the characteristics of public health studies where study terminology may differ. In response to this issue, the Cochrane Health Promotion and Public Health Field (the Field) developed and implemented a small study to recruit and support handsearchers from around the world to identify health promotion and public health trials and systematic reviews. A strategic framework was developed to recruit and support handsearchers to search six public health-related journals.
Results In total, 131 trials and 21 systematic reviews were identified. The greatest value of handsearching was found to be in supplement editions and abstract sections of journals
Conclusions The study focused exclusively on indexed journals with the intention that tools and methods developed could be used to explore the potential for handsearching in non-indexed journals and for unpublished studies. The findings from this study will continue to support handsearching efforts and in doing so contribute to high quality systematic reviews of public health interventions.
Keywords: handsearching, systematic review, health promotion, public health
 |
Introduction |
|---|
 TOP Introduction MethodsFindingsDiscussionAcknowledgementsReferences |
|---|
The importance of retrieving all studies from a wide range of sources is an essential ingredient to sound systematic review methodology.1 Although doing so will yield studies that may very well be excluded from the review upon the inclusion criteria being applied, nevertheless it is important that all relevant studies are found and considered. The inclusion of an unbiased sample of studies is central to the validity of systematic reviews.2 This generally involves employing a range of strategies including database, internet and hand searching. Handsearching a journal ‘involves a manual page-by-page examination of the entire contents of a journal issue to identify all eligible reports of trials, whether they appear in articles, abstracts, news columns, editorials, letters or other text’.1
But is there enough value to be gained from searching indexed and non-indexed journals to justify the time and resources it takes to hand search journals for the studies they house? A Cochrane review has compared handsearching to MEDLINE searching to identify reports of randomized controlled trials (RCTs).3 This research was limited to 22 specialized health care journals published in United Kingdom. Thirty-five per cent of the trials identified by handsearching were not indexed in MEDLINE and most of these were meeting abstracts or studies published in supplements. The authors identified that the additional yield from handsearching (compared to MEDLINE) was apportioned to studies published before 1991 and for non-MEDLINE indexed parts of the journal.3 We were interested in exploring whether similar findings would be found in handsearching journals specific to health promotion and public health. Given the limited research in this area, a strategy was developed to focus specifically on indexed journals. As RCTs are not always appropriate for health promotion and public health interventions,4,5 CCTs were also included. It was envisaged that this strategy, if appropriate and effective, could be trialled on non-indexed journals and unpublished studies. This article is a follow-up to our previous article published in this journal which outlined sources of publication bias and described our plan to develop a handsearching strategy to locate RCTs, controlled clinical trials (CCTs) and effectiveness reviews.6
|
Methods |
|---|
|
TOPIntroductionMethodsFindingsDiscussionAcknowledgementsReferences |
|---|
This project aimed to develop, implement, trial and evaluate a strategic, sustainable framework to support the handsearching of indexed journals containing health promotion and public health trials and systematic reviews. A number of organizations involved in the collection of studies were contacted for advice, including the National Library of Medicine and the Core Public Health Journals Project (USA). As a result, a list of relevant non-indexed health promotion and public health journals was developed. A framework was also developed to guide the recruitment of handsearchers and provide them with the support they would need to identify health promotion and public health RCTs, CCTs and systematic reviews. The framework included four key steps: selecting journals for handsearching, recruitment of handsearchers, developing the capacity of handsearchers and fulfilling administrative requirements.
The six journals (a) with the highest yield of RCTs and CCTs in the Cochrane Central Register of Controlled Trials (CENTRAL) and (b) not currently being handsearched by the Cochrane Collaboration were prioritised as the six journals to be handsearched. A search strategy comprising health promotion and public health subject headings was performed in CENTRAL to determine the journals with the highest yield of trials. These journals were The American Journal of Health Promotion, Health Psychology, Health Education and Behavior, Southeast Asian Journal of Tropical Medicine and Public Health, Journal of School Health and the Journal of the American Dietetic Association.
The most feasible and efficient process of recruiting handsearchers for the Field was through recruitment of existing members on the Field’s contact database. Thirty-five members had previously indicated an interest in handsearching; only five (from the USA, Canada, Thailand, Australia and New Zealand) could participate in this project. Al handsearchers who volunteered were very proficient in identifying particular study designs. Many had previously taken part in handsearching activities for other Cochrane entities.
All journals’ issues (including supplements) published in 2003 and 2004 were handsearched. A total of 92 issues were handsearched. All handsearchers were required to code the studies they identified, as either: RCT, CCT, systematic review (SR) or NS (not sure). Handsearchers were required to document all issues searched, trials and systematic reviews identified and provide evidence from the journal which assisted them to code the study. A Field staff member double-checked that all codes assigned were correct and assessed all studies coded ‘NS’ (not sure). Ongoing support was provided to handsearchers through email for the duration of the project.
|
Findings |
|---|
|
TOPIntroductionMethodsFindingsDiscussionAcknowledgementsReferences |
|---|
The handsearching pilot identified 122 RCTs, 9 CCTs and 21 systematic reviews. The average time it took for one handsearcher to search one issue was 30 minutes, depending on the number of articles per issue.
Of the 131 RCTs and CCTs identified by handsearching, 125 would have been identified through a MEDLINE search using the populations, interventions, comparisons and outcomes framework (PICO). If the search included study design limits (PICOt), a further seven trials would have been missed.
The identification of trials through handsearching provided valuable information its value:
- Seven trials provided no information in the abstract, title or subject headings of the random allocation of interventions (randomization was mentioned in the methods section of the article).
- Ten trials provided information in the abstract, title or MESH headings regarding random allocation, but were not indexed as RCT or CCT (Publication Type).
- Two trials were abstracts only.
- Three studies were identified in supplement editions of journals which were not indexed in MEDLINE.
- One trial could not be found in MEDLINE (the entire issue appeared to be missing from MEDLINE).
Finding journals articles of trials which did not provide information regarding random allocation in the abstract has important implications given that database searching can only search the title, subject headings and abstract of an article. In these articles, information pertaining to randomization was found within the methods section of the full-text article. In addition, articles that indicate random allocation in the subject headings or abstract of the citation are not always appropriately marked as RCT (Publication Type). Articles of this nature would only be identified though handsearching.
|
Discussion |
|---|
|
TOPIntroductionMethodsFindingsDiscussionAcknowledgementsReferences |
|---|
The project findings indicate that the greatest benefit from handsearching indexed journals will result from searching supplement editions of journals and abstract sections. Current reviewers who search MEDLINE and apply a RCT filter (which includes random as a textword) should be able to locate the majority of studies which were identified through this handsearching study. Therefore publication type alone is insufficient to identify all trials. Further improvements in finding evidence would be made possible if authors of health promotion and public health studies are encouraged to report the use of random allocation in the abstract section of the article. Widespread adoption of the CONSORT statement7 by journals, which requires reporting of intervention allocation in the abstract, should ensure future RCTs and CCTs are easier to locate.
There are several barriers to handsearching with time and access to resources being the most prominent. Handsearching is a time-consuming task and is often completed by volunteers outside of work hours. Time will become more of an issue if study designs are broadened to include non-randomized studies, evaluation data and qualitative research. Recruiters for handsearching activities could encourage retention of handsearchers by simplifying the handsearching process (which may include purchasing online or paper journal subscriptions for handsearchers in developing countries). Furthermore, an effective handsearching strategy requires handsearchers to have basic knowledge of study design methods and terminology. Future developments for handsearching could also include online submission of handsearching results.
Role of the field
The Field are working closely with the EPPI-Centre to develop appropriate coding for studies in the CENTRAL database. Currently, all primary studies identified by the EPPI-Centre through database and handsearching are submitted to CENTRAL and are coded as SR-HEALTHP. However, further work is now required to retrospectively tag health promotion studies. The Field is also interested in coding the studies in greater detail to allow reviewers to find the evidence more easily. Reviewers of HPPH interventions should be encouraged to utilize the Field’s registers which are available through the EPPI-Centre at http://eppi.ioe.ac.uk and download the Field’s training resources on identifying both published and unpublished health promotion and public health literature (available at www.vichealth.vic.gov.au/cochrane/training/index.htm).
This article has outlined a strategy for handsearching that has been tested using MEDLINE-indexed journals. The lessons learnt during the pilot will only strengthen the strategy and provide an opportunity to explore the role of handsearching in identifying articles of trials relating to health promotion and public health interventions in non-indexed journals and unpublished studies.
Update of Cochrane protocols and reviews of relevance
Issue 3, 2005, of the Cochrane Library was released in July. Below are new health promotion and public health oriented protocols and reviews from issues 2 and 3, 2005.
Reviews
- Competitions and incentives for smoking cessation
- Environmental sanitary interventions for preventing active trachoma (bursary recipient)
- Interventions for promoting the initiation of breastfeeding
- Interventions implemented through sporting organizations for increasing participation in sport
- Policy interventions implemented through sporting organizations for promoting healthy behaviour change
- Psychological interventions for overweight or obesity
- Quit and Win contests for smoking cessation
- Red-light cameras for the prevention of road traffic crashes
- School-based prevention for illicit drugs use
- Universal neonatal hearing screening versus selective screening as part of the management of childhood deafness
- The ‘WHO Safe Communities’ model for the prevention of injury in whole populations
- Mass media interventions for promoting HIV testing
- Fluoridated milk for preventing dental caries.
Protocols
- Behavioural interventions for reducing HIV infection in employees in occupational settings
- Bicycle skills training for preventing bicycle-related injuries in children and young people
- Families and schools together (FAST) for improving outcomes of school-aged children and their families
- Interventions for reducing the use of baby walkers
- Interventions in the alcohol server setting for preventing injuries (bursary recipient)
- Interventions to increase influenza vaccination rates of those 60 years and older in the community and in institutions
- Interventions to promote the wearing of hearing protection
- Interventions to reduce harm from continued tobacco use
- Non-legislative interventions for the promotion of cycle helmet wearing by children
- Increased police patrols for preventing alcohol-impaired driving
- Interventions for drug-using offenders in the courts, secure establishments and the community
- Interventions for increasing the proportion of health professionals practising in under-served communities (bursary recipient)
- Interventions for preventing unintended pregnancies among adolescents
- Abstinence-based programs for HIV infection prevention in high-income countries
- Bicycle helmet legislation for the prevention of head injuries. Electronic mosquito repellents for preventing mosquito bites and malaria infection
- Individual and group-based parenting programmes for the prevention of child abuse and neglect
- Interventions for increasing fruit and vegetable consumption in pre-school children
- Interventions for the prevention of non-melanoma skin cancers in high-risk groups
- Physical activity and enhanced fitness to improve cognitive function in older people
- Traditional birth attendant training for improving health behaviours and pregnancy outcomes.
|
Acknowledgements |
|---|
|
TOPIntroductionMethodsFindingsDiscussionAcknowledgementsReferences |
|---|
The Cochrane Health Promotion and Public Health Field acknowledges the funding for this project received from the Commonwealth Government of Australia. The Field also acknowledges the generous support of the Victorian Health Promotion Foundation (VicHealth) and from the five handsearchers who volunteered their time to participate in the small handsearching study.
|
References |
|---|
|
TOPIntroductionMethodsFindingsDiscussionAcknowledgementsReferences |
|---|
- Higgins JPT and Green S, eds. Cochrane Handbook for Systematic Reviews of Interventions 4.2.5 [updated May 2005]. In: The Cochrane Library, vol. 3. Chichester, UK: John Wiley & Sons, 2005. 1–265.
- Egger M, Juni P, Bartlett C, Holenstein F, Sterne J. How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study. Health Technol Assess 2003; 7(1): 1–76.[Medline]
- Hopewel S, Clarke M, Lefebvre C, Scherer R. Handsearching versus electronic searching to identify reports of randomized trials. The Cochrane Database of Methodology Reviews 2002; 4: Art. No: MR 000001. DOI: 10.1002/14651858. MR 000001.
- McQueen DV . Strengthening the evidence base for health promotion. Health Promotion International 2001; 16: 261–268.
[Abstract/Free Full Text] - Victoria CG, Habicht J-P and Bryce J. Evidence-based public health: moving beyond randomized trials. Am J Publ Hlth Public Health Matters 2004; 94(3): 400–405.
- Howes F, Doyle J, Jackson N, Waters E. Evidence-based public health: the importance of finding "difficult to locate" public health and health promotion intervention studies for systematic reviews. J Publ Hlth Med 2004; 26(1): 101.
[Free Full Text] - Begg C, Cho M, Eastwood S et al. Improving the quality of reporting of randomized controlled trials. The CONSORT statement. JAMA 1996; 276(8): 637–639.
[Abstract/Free Full Text]
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||