Skip Navigation


Journal of Public Health Advance Access originally published online on February 27, 2007
Journal of Public Health 2007 29(2):157-164; doi:10.1093/pubmed/fdm001
This Article
Right arrow Abstract Freely available
Right arrow FREE Full Text (PDF) Freely available
Right arrow All Versions of this Article:
29/2/157    most recent
fdm001v1
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Archive
Right arrow Download to citation manager
Right arrowRequest Permissions
Right arrow Disclaimer
Google Scholar
Right arrow Articles by Elston, J.
Right arrow Articles by Stein, K.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Elston, J.
Right arrow Articles by Stein, K.
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us  
What's this?

© The Author 2007, Published by Oxford University Press on behalf of Faculty of Public Health. All rights reserved

A rapid needs assessment of the provision of Health Technology Assessment in the south-west peninsula



Julian Elston, Honorary Research Fellow

Ken Stein, Senior Lecturer in Public Health
Institute of Health and Social Care, Peninsula Medical School, Barrack Road, Exeter EX2 5DW, UK

Address correspondence to Julian Elston, E-mail: julian.elston{at}nhs.net

Background Key to delivering UK policies on clinical governance, evidence-based practice and value for money is Health Technology Assessment (HTA). Despite the provision of HTAs through the National Institute for Health and Clinical Excellence (NICE), local health organizations still undertake HTA and make decisions based on them. In some regions, capacity is provided by centralized arrangements, but in others provision is ad hoc. This rapid needs assessment evaluates the provision of HTA in the south-west peninsula, and its scope, content and quality.

Methods We used semi-structured interviews and documentary analysis to assess the need for HTA.

Results HTAs are most commonly used by drug and therapeutics committees and joint formulary committees. The scope of technologies assessed was predominantly drugs. The quality of literature review in HTAs was variable and virtually none considered value for money. Informants felt there was insufficient provision of local HTAs. Local focus and clinical engagement were seen as key to the implementation of appraisal decisions, but this was threatened by weak links with commissioning and processes to prioritize decisions across primary care trusts.

Conclusions The quality of some HTAs poses a risk to clinical and corporate governance.

Keywords: needs, assessment, rapid, Health Technology Assessment, decision-making, evidence


   Introduction
TOP
 Introduction
Methods
 Results
 Discussion
 Conclusions
 Competing interests
 Ethics approval
 Acknowledgements
 References
 
Clinical governance, evidence-based practice and value for money have been consistent themes in UK national health policy over the past decade.1–3 Central to this agenda is Health Technology Assessment (HTA). This is a systematic evaluation of the evidence on a health technology used to estimate clinical and cost effectiveness.4 HTA informs decisions on whether to use pharmaceuticals, medical devices, diagnostic techniques, surgical procedures or health promotion interventions. This decision-making process is sometimes called health technology appraisal,5 and takes into account expert and lay opinion and social and ethical considerations.

The use of HTA nationally has been bolstered by the establishment of the National Institute of Health and Clinical Excellence (NICE).5 However, the sheer volume of (new) technologies means that NICE cannot appraise them all. A small proportion of technologies are selected for appraisal by ministers using specific criteria.6,7 Moreover, appraisals have often been slower than anticipated, untimely and narrow in content.8,9 Thus, policy-makers at the local level still need to assess and make decisions on the use of new technologies. Indeed, guidance states that this is one of primary care trusts' responsibilities, even when NICE guidance is being developed.6

Until April 2002, local HTAs were mainly undertaken by public health departments in district health authorities, usually with input from public health practitioners skilled in epidemiology and health economics. The creation of primary care trusts dispersed much of this expertise, with some directors of public health reporting insufficient ‘critical mass’ to fulfil responsibilities.10 Primary care trusts' ability to jointly develop and fund mechanisms for undertaking HTAs appears to have been less than expected.11 Some health regions have funded specialist organizations, such as the aggressive research information facility,12 to provide HTA for commissioners or have funded integrated assessment-appraisal systems such as the midland therapeutics review and advisory committee, in which policy recommendations are developed by an independent committee of clinical experts.13 In the south-west peninsula there have been no such developments. Indeed, additional external support for HTA ceased prior to the 2002 reorganization.14

The re-organization of the NHS during 2006 required organizations to secure high quality, safe care and value for money.3,15 Commissioning was also given a much greater role. Therefore primary care trusts and general practice will need to continue reviewing the evidence on health technologies. However, questions have arisen about the capacity for providing HTA in the peninsula and their quality. We were therefore commissioned by the (now former) strategic health authority to carry out a rapid needs assessment for HTA in Devon and Cornwall.


   Methods
TOP
 Introduction
 Methods
Results
 Discussion
 Conclusions
 Competing interests
 Ethics approval
 Acknowledgements
 References
 
The needs assessment identified corporate need for local HTA.16 For the purposes of this study, local HTA included any reports used in decision-making in primary and secondary care that reviewed the clinical and cost effectiveness of health technologies. A rapid assessment was undertaken to inform the local NHS re-organization, in which the number of primary care trusts had been reduced from 11 to four. Data was collected between May and July 2006.

The study had three stages. First, we used the snowballing technique, starting with directors of public health to identify local committees producing or using local HTAs; we invited their Chairs to participate in the study, and mapped membership, governance, management and funding arrangements.17,18 Secondly, we carried out documentary analysis on (i) terms of reference and committee minutes and (ii) local HTAs from the past year (May 2005–06). The third stage explored the opinions of committee Chairs (or other nominated key members) using in-depth, semi-structured interviews.18 Views on local HTA capacity and quality were sought, as were opinions on what changes could strengthen existing arrangements following the re-organization.

Documentary analysis
We analysed documents using a five-stage process.19 Initially we developed a data extraction sheet to consider, systematically, the style, scope and content of relevant documents.20 This included a series of questions for each document. In the case of local HTAs these were developed from a normative framework used by the international network of agencies for HTA.21 This identifies key characteristics of a good quality HTA, such as: context for the assessment; description of the technology; literature review and critical appraisal; assessments of safety, cost impact and cost effectiveness/utility; discussion of implications; clearly stated recommendations. Meanwhile, the questions developed for analysing committees' terms of reference addressed the scope of committees' work, commissioning of local HTAs, membership, governance and financial arrangements. Analysis of committee minutes addressed the type and number of local HTAs appraised plus the discussion of assessment or appraisal processes. Extraction sheets were revised after having been piloted by two reviewers. Extracted data were entered into Microsoft Excel for analysis.

Interviews
We developed a semi-structured questionnaire on the role of the committee, the types of technology considered, the need for local HTA, the provision and sources of local HTA, their quality and measures to strengthen current arrangements. Questions were open-ended to enable interviewees to elaborate on answers.22,23 They were piloted on two interviewees and then refined. Interviews were conducted face-to-face or sometimes by telephone. Face-to-face interviews were recorded and transcribed while telephone interviews were written up from contemporaneous notes. Confidentiality and anonymity were assured to encourage frank and honest responses.

A summary of the documents received and analysed and interviews undertaken is presented in Table 1.


View this table:
[in this window]
[in a new window]

  		Table 1 Summary of documents received and analysed and interviews undertaken

 
Analysis
Issues identified from interviews and minutes were ‘triangulated’ to identify common themes. These were explored across locations or committee type to identify patterns. Discrepant views were also examined. Study findings were validated by direct feedback from committee members following presentation of the research.24


   Results
TOP
 Introduction
 Methods
 Results
Discussion
 Conclusions
 Competing interests
 Ethics approval
 Acknowledgements
 References
 
Mapping of structures using local HTA
Key informants identified 26 groups routinely using or commissioning local HTAs. These operated between primary and secondary care, and across different organizational boundaries and four geographical locales creating a complex, partially linked network of committees. It was not possible to categorise the function of committees into distinct groups as their remits all varied, but broadly they were called:

  • drug and therapeutics committees;
  • joint formulary committees;
  • prescribing advisory committees;
  • effective practice committees;
  • clinical effectiveness committees;
  • prescribing networks;
  • special case panels.
Drug and therapeutics committees and joint formulary committees centred on each of the five local acute trusts were the most common committees. In general, drug and therapeutics committees assessed new and existing (predominately) secondary care technologies and made recommendations to joint formulary committees about the inclusion or use of a technology in its formulary. Occasionally, joint formulary committees undertook their own reviews. Some of these committees deferred their decisions to prescribing advisory committees if the budgetary impact was estimated to be more than £10,000–50,000. The cancer network drug and therapeutics committee operated at a peninsula level, independently of any single joint formulary committee. However, most other clinical networks rarely commissioned HTAs. The peninsula prescribing network commissioned HTAs on behalf of drug and therapeutics committees and similar groups in the peninsula (with a specific budget to which all primary, secondary and mental health trusts contributed).

Several primary care trusts had clinical effectiveness committees which were focused on primary care prescribing. Most of these committees were connected to drug and therapeutics committees and the extent to which they commissioned or considered HTAs varied.

One effective practice committee worked across several acute and primary care trust boundaries to consider (almost exclusively) pharmaceutical technologies with significant implications for primary care prescribing. Few locales had such cross-trust arrangements.

All primary care trusts had special case panels. These considered whether to fund non-commissioned or low priority treatments. All reported reviewing the effectiveness of treatments to inform funding decisions, although a few commissioned other primary care trusts to undertake their reviews.

Committee membership primarily constituted pharmacists and medical consultants and to a lesser extent GPs, nurses and commissioners. Only three committees had mental health or patient representatives and only two university representation. None had a health economist.

Local HTA capacity
Documents were received from all 15 participating committees, representing a cross-section of committee types and the main geographical locales. Analysis of documents revealed that 62 local HTAs were considered by these committees over the previous year. Table 2 shows the percentages received by location and committee type. Nearly 60% (36/62) of local HTAs were provided by two pharmacists linked to drug and therapeutics or similar committees, one located in an acute trust in Devon and the other in Cornwall. The next most prolific user of local HTAs was the cancer network drug and therapeutics committee. Most of these HTAs were provided with joint pharmacist and clinician input. Eight reviews were commissioned by the peninsula prescribing network from a regional medicines information unit at Bristol Royal Infirmary. Only in one clinical effectiveness committee were reviews undertaken by an individual (training) in public health. Scrutiny of minutes revealed that 15% (8/54) of these HTAs reviewed the same technology. In addition, less than one in four local HTAs was shared (12/54) across locales, and only a third (4/12) of these were shared across more than two locations. Sharing of local HTAs was mentioned in only 20% (3/15) of groups' terms of reference. Besides technologies appraised by NICE, we identified 51 technologies from the minutes for which there were no local HTAs. This suggested that only half the potential need [51/(51 + 54)] for local HTAs was being met, although not all of these technologies would need an assessment, e.g. a joint formulary committee discussing a switch to a ‘generic’ drug. As six of these technologies were identified by more than one (unrelated) committee, this suggests scope for sharing up to 12% (6/51) of any extra capacity.


View this table:
[in this window]
[in a new window]

  		Table 2 Profile of local HTA provision in the peninsula by location and committee type

 
Respondents' views on existing capacity were mixed. For a few, existing capacity was sufficient, but most believed more capacity was needed. As one pharmacist noted:
I'm working flat out and [the Pharmacist] over the road is too.

Another respondent commented:
[The Pharmacist] hasn't got the capacity to do everything and to do it properly. There's a lot to do and you need to think how thorough does it need to be.

This suggested that capacity was closely linked to the quality or rigour of assessments.

Content of local HTA
The majority of local HTAs were on new or existing drugs, as indicated in Table 1. This bias towards pharmaceuticals was also reflected in committees' terms of reference (see Table 3) and corroborated by interviewees. Of the few non-pharmaceutical HTAs identified almost all were related to prescribing.


View this table:
[in this window]
[in a new window]

  		Table 3 Types of technology assessment undertaken identified from committees' terms of reference

 
Most local HTAs were of reasonable quality. However, there were potentially important weaknesses. Under half (10/24) clearly specified the question they were addressing, while a third (9/24) provided little or no background on the technology or condition. The quality of literature review was variable. Although 90% (22/24) of assessments provided details about the literature search, study design and population, in a quarter these details were scant. Similarly, there was little evidence of critical appraisal of studies in 25% (6/24) of assessments and in just under half (11/24) the critique was partial.

The most striking feature of the analysis of local HTAs was the near complete absence of economic evaluation. Even consideration of cost impact was limited. One in six local assessments (4/24) contained no estimate of cost impact and only a third of terms of reference (5/15) identified budgetary impact as a necessary consideration. More significantly, 60% (9/15) of terms of reference specified reviewing cost-effectiveness, but only one local HTA attempted to calculate cost effectiveness and, even then, with no consideration of costs beyond drug acquisition. This weakness was confirmed in interviews. In general, support for economic evaluation and public health input was reported to have effectively ceased since the 2002 NHS re-organization. As one interviewee commented:

It's difficult to look at cost-effectiveness unless the work has already been done and [there's] a paper that supports it.

There was also a view that pharmacists would generally be reluctant to undertake assessment on non-pharmaceutical technologies as they lack the skills and capacity to do so.

Quality of local HTAs
The variable quality of reports was an issue identified by many interviewees. Although those undertaken by the two most prolific pharmacists were generally well regarded, concerns were expressed about the quality of locally and externally commissioned assessments used by a few committees.

For some, quality was not only related to balance but also to readability and the presence of clear, decisive recommendations in assessments. Interviewees also expressed a degree of realism when discussing quality improvement. Investing more time and resources (i.e. in terms of scientific and economic rigour) might not make any difference to the decisions taken by committees. What was important was that quality was ‘fit for purpose’.

Commissioning of HTAs
Interviewees identified a number of issues relating to the process of commissioning local HTAs. For example, timeliness from technology selection to assessment was essential (i.e. two months was common), otherwise some could run the risk of being promptly superseded by NICE. To minimize this risk, committees only selected technologies that NICE was not proposing to review within the next 12–18 months. A flexible commissioning process, whereby the scope takes shape as the assessment progresses, was also considered important by some respondents, as was having authors who had knowledge of local issues facing primary and secondary care. This was more difficult to achieve when commissioning outside the peninsula.

Appraisal and implementation
Although not the main focus of this study, a major issue raised by interviewees (and also articulated in committee minutes) was the quality of local appraisal and implementation processes. Appraisals needed to ensure that assessments were discussed and debated robustly. For a few interviewees, having authors present their assessments was an important element in achieving this, allowing any misinterpretation to be challenged. Interviewees consistently identified a few committees that appeared subjective, and possibly biased, in their deliberations. As one interview noted:

The [committee] was thought to be self-serving, populated by people who wanted new drugs to be agreed ... and for a long time it was not objective.

A widespread concern was the weak links between appraisal and implementation. If this process was not working, interviewees noted, the quality of HTAs was irrelevant. A quarter of respondents suggested implementation is dependent on two things: (i) the degree of engagement and ownership by local clinicians and (ii) clear links with commissioning processes.

A local focus was important in developing and maintaining ownership by clinicians (consultants and GPs alike), especially in joint formulary committees and drug and therapeutics committees. Some respondents also noted that engagement was also partially dependent on delivery of committees' recommendations. Failure to implement decisions was creating much frustration and dissatisfaction. Implementation in acute trusts was easier to monitor when one formulary was involved but posed a great challenge where decisions involved several primary care trusts. Different, unclear and inefficient processes and different priorities and financial solvency were reportedly causing significant delays, breakdown of commissioner involvement and, as a consequence, uneven implementation across the peninsula. As one respondent commented:

[The committee] was making decisions that were sensible and agreed upon by everyone and [once the commissioner] realised that he was in the hot seat and expected to financially to underwrite it, he gave up coming.

Adapting to organizational change
In general, respondents did not offer specific suggestions for new HTA structures. Rather, comments supported retaining elements of the existing systems that were working well. In addition to an expressed need for greater sharing of skills and workload and more HTA capacity with broader scope, three main issues were identified: (i) not imposing peninsula or Devon-wide structures (particularly in joint formulary committees and drug and therapeutics committees) as enlargement is likely to disturb group dynamics and disrupt delivery; (ii) changes to arrangements should be delayed until after the re-organization has settled to minimise the impact of this process; and (iii) structures need a local focus and to engage clinicians effectively.


   Discussion
TOP
 Introduction
 Methods
 Results
 Discussion
Conclusions
 Competing interests
 Ethics approval
 Acknowledgements
 References
 
Main findings of this study
HTA is being used in a variety of loosely networked, local policy-making bodies in Devon and Cornwall. However, the scope and content of the 60 or more local HTAs undertaken each year was very limited. Nearly all were on drugs, and the few on non-pharmaceuticals were mainly related to prescribable technologies. There was also a near complete absence of economic evaluation. Only one report attempted to calculate the cost effectiveness of a technology, and only very crudely. The quality of reports was variable and reportedly weaker in clinical networks and in assessments that were externally commissioned.

Informants felt there was insufficient provision of local HTAs, despite the development of a prescribing network with a remit to commission HTAs. Evidence of duplication and lack of sharing of HTAs suggested some scope for increasing capacity through improved co-ordination. However, expansion of capacity may be restricted by lack of appropriate expertise, particularly in non-pharmaceutical technologies.

A local focus, clinical engagement and ownership were felt to be key to the implementation of local policy decisions, but this was being threatened by weak links with commissioning. The absence of a mechanism to co-ordinate priorities across primary care trusts was resulting in uneven implementation of HTA recommendations and ‘postcode prescribing’.

What is already known on this topic
We did not identify any other papers evaluating need for or provision of local HTA.

What this study adds
This study raises a number of issues relating to the scope, content and quality of local assessments and appraisal of HTA which are likely to be pertinent to other NHS regions where a network approach to HTA provision has evolved. Of particular note was the predominant focus on pharmaceuticals and near absence of value for money assessments. Furthermore, existing arrangements and resources appear to be configured to manage pharmaceuticals rather than appraise ‘low tech’ interventional procedures. With demand for local HTAs set to continue (despite NICE's efforts to increase its timeliness and output)6,25,26 the recent NHS re-organization provides an opportunity to remedy these weaknesses. Failure to do so will prevent trusts from fulfilling their responsibilities and leave decision-making open to challenge.

Lack of public health and health economic expertise to work on local HTAs, particularly on non-pharmaceutical technologies, appears to be contributing to the limitations outlined above. It is an issue likely to be relevant elsewhere. A network approach may provide some additional capacity. More efficient co-ordination could extend capacity through increased sharing and reduced duplication but this is unlikely to meet all the needs identified above. Investment in education and training and dedicated public health and health economic posts may also be required.

Collaboration with academic institutions could be an alternative. Some NHS regions (West Midlands) have already adopted this model (see Table 4). Finally, new, enlarged primary care trusts may have sufficient human and financial resources to set up a specialist unit to provide local HTAs. Whatever the approach, it will need to be evolutionary, building on existing, local relationships where possible—a significant challenge at a time of re-organization.


View this table:
[in this window]
[in a new window]

  		Table 4 Organizational structures used to provide local HTAs in England

 
In addition, the mechanisms linking appraisal to implementation will need to be addressed. Whether the assessment and appraisal processes should be separate or joined together will need to be decided, as this will have implications for the structures that are to deliver local policy recommendations. Either way, a local focus and clinical engagement appear important to the implementation of evidence-based policy recommendations.27 Primary care trusts will also need to harmonize prioritization processes if unequal access to technologies within regions is to be avoided. As no formal evaluations of these different approaches were identified in the literature it is difficult to recommend which structure, if any, may be most appropriate. Re-design requires a fine balance between investing in a system to provide local HTAs of sufficient quality and ensuring there are effective and transparent implementation mechanisms in place. Without the latter, efforts directed at the former will have limited impact.

Limitations of this study
Using rapid appraisal enabled us to paint a timely, current picture of HTA provision at a time of considerable change. Although we focused on a relatively small area, this allowed fairly comprehensive data collection within a short timescale. However, these strengths may limit how far the findings can be generalised to other areas. Although considerable efforts were made to engage individuals from all relevant health professions, organizations and locations, most key informants were pharmacists; no representatives from the mental health trust were available and engagement of professionals in Cornwall was weak. Furthermore, there were insufficient resources to carry out detailed analysis of all local HTAs received or to obtain a comprehensive picture of the scope and quality of assessments undertaken for special case panels. Nevertheless, the key findings were validated when presented back to committee members. Finally, HTA appraisal and implementation were not the main focus of this study despite this issue being strongly voiced by informants, and findings in this area warrant further study.


   Conclusions
TOP
 Introduction
 Methods
 Results
 Discussion
 Conclusions
Competing interests
 Ethics approval
 Acknowledgements
 References
 
Local provision of HTAs in the peninsula is at full stretch and quality is variable. Lack of consideration of value for money and variation in quality in HTAs present a potential risk in terms of clinical and corporate governance and may leave appraisal decisions open to challenge. Links between committees using HTA and with commissioning are weak, limiting capacity and undermining financial and health benefits. Furthermore, incongruent prioritization processes may be leading to unequal access to technologies.


   Competing interests
TOP
 Introduction
 Methods
 Results
 Discussion
 Conclusions
 Competing interests
Ethics approval
 Acknowledgements
 References
 
None.


   Ethics approval
TOP
 Introduction
 Methods
 Results
 Discussion
 Conclusions
 Competing interests
 Ethics approval
Acknowledgements
 References
 
Ethical approval was not required for this study.


   Acknowledgements
TOP
 Introduction
 Methods
 Results
 Discussion
 Conclusions
 Competing interests
 Ethics approval
 Acknowledgements
References
 
We thank those people who gave their time to be interviewed, supplied documents or assisted in other ways with this research, in particular Joanne Perry and Sara Gibbs.


   References
TOP
 Introduction
 Methods
 Results
 Discussion
 Conclusions
 Competing interests
 Ethics approval
 Acknowledgements
 References
 

  1. Department of Health. A First Class Service: Quality in the New NHS (1998) London: Department of Health.
  2. Department of Health. Standards for Better Health (2006) London: Department of Health.
  3. Nigel Crisp Sir. Commissioning a Patient-led NHS (2005) London: Department of Health.
  4. National Institute for Clinical Excellence. Guide to the Methods of Technology Appraisal (2004) London: National Institute for Clinical Excellence.
  5. National Institute for Clinical Excellence. Guide to Technology Appraisal (2004) London: National Institute for Clinical Excellence.
  6. DH NICE Liaison team. Good Practice Guidance on Managing the Introduction of New Healthcare Interventions and Links to NICE Technology Appraisal Guidance (2006) London: Department of Health.
  7. Department of Health. Selection Criteria for Referral of Topics to NICE (2006) July. London: Department of Health.
  8. Carlisle D. A furore too far: the widespread disquiet undermining NICE. Hlth Serv J (2006) 116(6011):14–15.
  9. Webster C. The National Health Service. A Political History (2002) 2nd edn. Oxford: Oxford University Press.
  10. Wanless D, Jones N, Anderson R, et al. Securing Good Health for the Whole Population. Final Report (2004) London: HM Treasury.
  11. Department of Health. Shifting the Balance of Power within the NHS. Securing Delivery (2001) London: HMSO.
  12. Department of Public Health and Epidemiology. Aggressive Research Intelligence Facility (2006) Birmingham: The University of Birmingham.
  13. Midland Therapeutics Review and Advisory Committee (MTRAC). Constitution (2004) Keele: Department of Medicines Management, Keele University.
  14. Development and Evaluation Committee. (2000) http://www.hta.nhsweb.nhs.uk/rapidhta/.
  15. Department of Health. Creating a Patient-led NHS. Delivering the NHS Improvement Plan (2005) London: Department of Health.
  16. Stevens A, Raftery J. Introduction. In: Health Care Needs Assessment. The Epidemiologically Based Needs Assessment Reviews—Stevens A, Raftery J, eds. (1997) 2nd series. Abingdon: Radcliffe Medical Press. xxxii–xliv.
  17. Keen J, Packwood T. Case study evaluation. BMJ (1995) 311:444–6.[Free Full Text]
  18. May T. Social research. Issues, Methods and Process (1997) 2nd edn. Buckingham: Open University Press.
  19. Elston J, Fulop N. Perceptions of partnership. A documentary analysis of Health Improvement Programmes. Public Health (2002) 116:207–13.[ISI][Medline]
  20. Abbott S, Shaw S, Elston J. Comparative analysis of health policy implementation. The use of documentary analysis. Policy Stud (2004) 25(4):259–66.[CrossRef]
  21. INAHTA. A Checklist for Health Technology Assessment Reports (2001) Stockholm, Sweden: International Network of Agencies for Health Technology Assessment Secretariat.
  22. Fontana A, Frey JH. Interviewing. The art of science. In: Collecting and Interpreting Qualitative Materials—Denzin NK, Lincoln YS, eds. (1998) Thousand Oaks, CA: Sage. 47–78.
  23. Jones S. Depth interviewing. In: Applied Qualitative Research—Walker R, ed. (1985) Aldershot: Gower. 45–55.
  24. Yin RK. Case Study Research. Design and methods (1994) 2nd edn. Thousand Oaks, CA: Sage.
  25. Cookson R, McDaid D, Maynard A. Wrong SIGN, NICE mess: is national guidance distorting allocation of resources? BMJ (2001) 323:743–5.[Free Full Text]
  26. National Institute for Health and Clinical Excellence. Guide to Single Technology Appraisal (STA) process (2006) London: National Institute for Health and Clinical Excellence.
  27. Coleman P, Nicholl J. Organisational context for quality: lessons from the fields of organisational development and change management. Quality in Health Care (2001) 10:229–37.[Abstract/Free Full Text]

Add to CiteULike CiteULike   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us    What's this?



This Article
Right arrow Abstract Freely available
Right arrow FREE Full Text (PDF) Freely available
Right arrow All Versions of this Article:
29/2/157    most recent
fdm001v1
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Archive
Right arrow Download to citation manager
Right arrowRequest Permissions
Right arrow Disclaimer
Google Scholar
Right arrow Articles by Elston, J.
Right arrow Articles by Stein, K.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Elston, J.
Right arrow Articles by Stein, K.
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us  
What's this?