Journal of Public Health Advance Access originally published online on November 14, 2007
Journal of Public Health 2008 30(1):23-29; doi:10.1093/pubmed/fdm073
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Fortification of selected foodstuffs with folic acid in the UK: consumer research carried out to inform policy recommendations
Alison Tedstone, Head of Nutrition Science1
Mark Browne, Senior Policy Officer1
Lynda Harrop, Nutritionist1
Claire Vernon, Researcher2
Victoria Page, Researcher2
Jill Swindells, Researcher2
Wendy Hayward, Researcher3
Jeannette Croft, Researcher3
Freda Harris, Researcher3
Lynn Stockley, Public Health Nutritionist4,
1 Food Standards Agency, Aviation House, 125 Kingsway, London WC2B 6NH, UK
2 Define Research and Insight, Colton House, Princes Avenue, London N3 2DB, UK
3 forum, 18 Lower Boston Road, London W7 2NR, UK
4 Independent Public Health Nutritionist, Timberland, Brockweir NP16 7NN, UK
Address correspondence to Lynn Stockley, E-mail: lynn.stockley{at}which.net
Background The UK Food Standards Agency Board identified four options to increase folate intake in women of reproductive age in order to reduce the risk of neural tube defect (NTD) affected pregnancies; these ranged from continuing with current policy, to mandatory fortification of bread or flour with folic acid. In order to appraise these options, the agency carried out a consultation, and also commissioned four pieces of research. This paper provides detailed information about two of the research studies, which used qualitative research approaches to gather consumer evidence.
Methods Study 1: This was carried out with people from a wide range of demographic backgrounds. A ‘reconvened group’ methodology was used, with five groups convened twice, in five geographical locations. In addition paired, in depth face-to-face interviews were conducted with female black and ethnic minority consumers. Study 2: This was carried out with young mothers living in deprived communities. The approach used for this study was in depth face-to-face interviews (n = 24). In addition, discussions were held in seven friendship groups.
Results Study 1: only a minority of participants knew about a link between spina bifida and folic acid, and these tended to be women with young families. After the provision of some information about the causes and impacts of NTDs, the majority were in favour of action to tackle the issue. Support for mandatory fortification increased considerably during the study, and at the final discussion, this option was most preferred. Study 2: In this group, there was a fatalistic approach to pregnancy and to health. The women were less likely to change established habits if this required effort, money or doing something unfamiliar. They tended to actively avoid thinking about risks, by rationalizing them. Mandatory fortification was preferred by the majority of respondents.
Conclusions In this research, mandatory fortification was the preferred option. There were outstanding concerns about risk, and the maintenance of consumer choice, which would need to be addressed in policy recommendations.
Keywords: nutrition, food, policy, consumer research
 |
Introduction |
|---|
 TOP Introduction MethodsResultsDiscussionFundingAcknowledgementsReferences |
|---|
Background
Research carried out in the 1980s highlighted the role of folic acid in reducing the risk of neural tube defect (NTD) formation.1,2 The publication of this research led to recommendations in the UK that to reduce the risk of occurrence of NTDs, all women of childbearing age should increase daily folic acid intake by 400 µg prior to conception, and during the first 12 weeks of pregnancy.3,4 It is difficult to obtain 400 µg per day folic acid from foods alone, and so the initial recommendations for the implementation of this advice, both in the UK and elsewhere, were through a combination of dietary change, encouraging periconceptual folic acid supplement use and voluntary fortification of specified foods by manufacturers.
There have been several campaigns5 to encourage the increased uptake of folic acid supplements, for example, in Canada, the USA, Australia, the Netherlands and the UK. The positive effects of these are often not sustained, and are more evident in women from higher socio-economic groups, and it is particularly difficult to reach those women who have unplanned pregnancies.5 In the UK, it is estimated that currently only about a quarter of women take folic acid supplements as recommended.6
The difficulty in encouraging supplement use, and the uncontrolled nature of voluntary fortification has led to the consideration of mandatory folic acid fortification. Across the world more than 40 countries have now introduced this,7 and others are considering it.7,8 In a recent assessment, Sweden rejected mandatory fortification, because of uncertainties about a possible link with cancers.9
The number of NTD-affected pregnancies in the UK does not appear to have changed since the 1990s.10 In contrast, NTD incidence has decreased in the USA, Canada and Chile, where there has been monitoring of the impact of mandatory fortification.6
UK developments
The Food Standards Agency (FSA) is an independent Government department established in 2000. Shortly after its inception, the FSA considered earlier advice to fortify flour with folic acid from the Department of Health's Committee on Medical Aspects of Food Policy,4 but agreed not to recommend this,11 because of concerns about delaying the diagnosis of vitamin B12 deficiency in the elderly, and limiting freedom to choose unfortified foods.
In 2006, the UK Government's Scientific Advisory Committee on Nutrition (SACN) published a new report.6 The Committee found no data to suggest that folic acid at doses up to 1 mg per day would delay the diagnosis of vitamin B12 deficiency, and additionally found that there had been no increase in reported cases of severe manifestation of vitamin B12 deficiency in the USA since mandatory fortification was introduced. It also considered concerns about a link with cancers, and concluded that the evidence for a link between folic acid and increased or reduced risk of cancers in humans is equivocal.
In the light of this, the FSA12 identified four possible options to increase the folate intake of women of reproductive age, thereby reducing the number of pregnancies affected by NTDs:
- Option 1—continue with the current policy of advice to women.
- Option 2—increase the effort to encourage young women to take folic acid supplements and make changes to diet to increase the consumption of folate rich foods.
- Option 3—encourage industry to fortify more foods with folic acid on a voluntary basis.
- Option 4—recommend the mandatory fortification of bread or flour with folic acid.
Research commissioned by the FSA to inform discussion of folic acid fortification policy in the UK
In order to further appraise these options, the FSA carried out a consultation on the options listed above, and also commissioned four pieces of research. The first two of these were an expert view on the ethical issues (Fuller-Deets and Dingwall, 2007, unpublished), and a review of the literature relevant to improving folic acid supplement uptake.5
However, the FSA also wanted to undertake work to gather consumer evidence, to complement the work of SACN, and the ethical and scientific literature reviews that had been commissioned. This was done through two qualitative studies to explore consumer attitudes to the options to increase the uptake of folic acid. Study 1 was carried out with participants from a wide range of demographic backgrounds. Study 2 was carried out with young mothers living in deprived communities.
|
Methods |
|---|
|
TOPIntroductionMethodsResultsDiscussionFundingAcknowledgementsReferences |
|---|
Study 1
The objectives of this study included the following.
- Encourage and facilitate informed deliberation among consumers on the possible fortification of flour with folic acid, and the attendant risks and benefits.
- Gauge existing levels of understanding and knowledge about current fortification practice.
- Assess consumers' current understanding and awareness of NTDs and the benefits of taking folic acid supplements.
- Gather consumers' views on the four options proposed by the FSA Board.
Recruitment was carried out by experienced recruiters, using the best practice methods recommended by the Association for Qualitative Research.15
One group was convened in each of five geographical locations: South England, North England, Wales, Scotland and Northern Ireland. Each group comprised 12 participants, equally split by demographic factors: gender, lifestage (young, family, older people, pensioners) and socio-economic group (ABC1 versus C2DE).
In addition, four paired, in depth, interviews were conducted with female black and ethnic minority (BME) consumers (African, Caribbean, Muslim, Hindu). The purpose of this was to indicate whether women from BME populations were likely to hold different views. Similarly to the groups, a two-stage approach was used for the interviews, with each interview lasting about one and a half hours.
Quota sampling was used. Potential subjects were excluded if they: had personal experience of NTDs or other genetic birth defects; were pregnant or had pregnant partners; were health professionals or anyone with a nutrition-related qualification; had a significant wheat allergy including coeliac disease.
Participants were asked to undertake a pre-task before the first group discussion, in order to sensitize them in very general terms to food-related issues. The task was to bring in examples of labels that highlighted added vitamins or minerals.
A topic guide was used as a framework for the sessions, and participants were asked to read material they were provided with between the sessions. Information throughout the sessions was presented in increasing detail and complexity, with two versions of the reflective material. This was to accommodate different levels of consumer needs, in terms of available time, interest in more detailed information and levels of literacy. As a minimal requirement, participants were asked to read the shorter version of the material. In practice, it was clear at the second session that respondents had complied with this minimum request; many had also looked at the longer version, and some had read it in detail.
Questionnaires were completed by respondents at the end of the first session, at home after completing their reflective reading, and at the end of the second session. These questionnaires tracked participants' views of the four options identified by the FSA.12 The group discussions and depth interviews were audio recorded and transcribed. A content analysis approach was adopted, identifying and exploring the themes and patterns in the responses, in order to correlate points of consistency and difference across the sample. Software was not used. The initial analysis was carried out by one of the researchers, and then discussed and scrutinized by two other researchers.
Participants were not told that the research was funded by the FSA until towards the end of the second session. They were paid a cash incentive of £115 spread across the two sessions to cover their time and encourage retention. The incentive was not mentioned until after recruitment.
Study 2
The objectives of this study included to explore:
- the health and lifestyle changes women made or intended to make prior to and/or during pregnancy.
- the barriers and facilitators to making any changes before and during pregnancy.
- levels of awareness and understanding about NTDs and folic acid.
- reactions to the four proposed options for increasing/encouraging folic acid intake.
Quota sampling was used. The inclusion criteria were: mothers with a youngest child <3 years; no education qualifications after the General Certificate of Secondary Education; living in one of the 30 most deprived Local Authorities in England and in Group G or F (‘Welfare Borderline’ or ‘Municipal Dependency’) of the MOSAIC neighbourhood classification system developed by Experian.16 In rural areas not covered by MOSAIC, large estates or known deprived communities were identified. The exclusion criteria were: women who did not consume wheat or had a restricted diet; had close experience of NTDs; were currently pregnant or had a history of drug problems.
Participants were selected so that there was an even spread of women who had made no/few lifestyle changes during pregnancy (e.g. eating more healthily, giving up smoking), those who had made some, and those who reported making many lifestyle changes. The sample aimed to include a minimum number of those who had planned to become pregnant and those who had not. The latter comprised two-thirds of the total number.
The presentation of information was tailored to fit the literacy levels and understanding in this group. In particular, the focus was on two main options: ‘encouraging food companies to add folic acid to more products’ and ‘making a law that folic acid should be added to flour products’. These were accompanied by more detailed information about what these two options would mean in practice, and what the advantages and disadvantages might be. The order in which these two options were presented to the participants was alternated.
The 20-minute follow-up telephone interviews were conducted 2–3 weeks after the initial interview/friendship group. Willing participants were identified, and from these 10 were selected to represent the wider sample.
The analytical methods were similar to those described for Study 1.
Participants were not told that the research was funded by the FSA prior to interview. Through the research process they were made aware of the involvement of a government agency or the FSA specifically. They were paid a cash incentive of £30, with an additional £10 for a follow-up interview.
|
Results |
|---|
|
TOPIntroductionMethodsResultsDiscussionFundingAcknowledgementsReferences |
|---|
Study 1
Of the 60 people recruited for group discussions, two dropped out between the first and second workshops.
Initial views
There was moderate awareness that some foods are fortified, although respondents were not clear about the distinction between voluntary and mandatory fortifications. Almost all respondents were surprised that the post-Word War II fortification of flour and margarine with selected vitamins and minerals is still practiced, but were not generally opposed to it in principle.
The term NTDs was not well recognized. Only a minority knew about a link between spina bifida and folic acid, and these tended to be women, and to have young families. After the provision of some basic information, the majority were in favour of action to tackle the issue of NTDs.
Key issues that influenced the discussions
There was considerable discussion about how to weigh up the balance of risks and benefits, with a sizeable number of participants concerned about further fortification without assurance that there would be no harm to anyone in the population.
The high rate of unplanned pregnancies in the UK was recognized as a key consideration, and the majority of respondents identified this as a reason to change the current approach to fortification. A minority advocated action to tackle the circumstances, which they saw as leading to unplanned pregnancy, for example, binge drinking.
Many participants wanted to retain consumer choice, and almost all wanted to see labelling applied to any further fortification.
Differences between population groups
The views of black and minority ethnic participants reflected those of the remainder of the sample. There were also few differences between men and women, or between socio-economic groups.
Younger people and those with young families were more in favour of fortification, with the latter most in favour of a mandatory approach.
Options for the way forward
Support for Option 1 (continuing with current policies) was relatively low, and decreased slightly during the process, finishing as the least supported option. The respondents who preferred it were concerned about risks, and wanted more research to be carried out.
Support for Option 2 (public information campaign) was initially high, but decreased noticeably during the process, although it was thought important that this type of action should take place as a complement to fortification. Those who finally supported this option were concerned about risks, but wanted to see some action. Those who moved away from it, doubted the effectiveness of a campaign in isolation from other actions. They also thought that to provide a high intensity and sustained campaign of the sort that might be effective would be expensive.
Support for Option 3 (voluntary fortification) increased slightly. Those who supported it did so largely because it maintained consumer purchasing choice. Some people moved away from it, because it seemed to them to be indiscriminate, and uncontrolled.
Support for Option 4 (mandatory fortification) increased considerably over the study, from four participants (total n = 68) to 30 (total n = 66) at the final discussion. This was because it appeared to be the most effective strategy, although many of its supporters nevertheless had continuing worries about risks, for example, in relation to diagnosing vitamin B12 deficiency.
Study 2
Five individuals dropped out after recruitment. Three of these were from the BME category, and were replaced; two were from a friendship group and were not replaced.
Characteristics of participants
Since two-thirds of the sample had unplanned pregnancies, they did not receive any advice until relatively late in pregnancy. For many of the women, even when they knew they were pregnant they were unlikely to consider making any lifestyle changes, and this was linked to being uncertain about whether to continue the pregnancy.
There was a tendency for people in this group not to be very interested in acquiring information. The younger participants and those from the lowest socio-economic groups in particular were wary about formal information sources, especially general practitioners. Verbal information and advice were most influential, particularly from mothers or sisters and midwives.
Most of those interviewed said that they made minimal lifestyle changes during pregnancy, but tended to continue with their established lifestyles. Changes to prevent something harmful happening to the baby were seen as more important than those that promoted health. Smoking and drinking were seen in the ‘preventing harm’ category, but increasing folic acid intake was not, with very little awareness of a link with NTDs.
The women were also less likely to change when this required effort, money or doing something unfamiliar. They tended to actively avoid thinking about risks, by rationalizing them. They also did not connect their own behaviour to levels of risk for their baby.
Participants believed that they were more likely to make changes if advice was clear and consistent, and actions were made as easy as possible (e.g. by direct provision of free supplements).
Options for the way forward
Option 1 (continuing with current policies) was unanimously rejected, with people believing that the government should take further action.
Option 2 (public information campaign) was welcomed, but there were doubts about its effectiveness in isolation.
Option 3 (voluntary fortification) elicited some positive reactions, but with the provisos that: high impact advertizing and labelling would be needed to identify fortified foods; they would need to be the same price as comparable foods; and taste the same. Many of the participants recognized that the impact of this option was likely to be minimal for them.
Option 4 (mandatory fortification) was preferred by the majority of participants, and seen as easy because it required no action on their part. There was some concern about potential risks, but it was thought that these could be addressed by actions such as ensuring that folic acid-free alternatives are available.
Follow-up
At the follow-up interviews, there was no change in views, except a hardening of the opinion that mandatory fortification was the best option.
|
Discussion |
|---|
|
TOPIntroductionMethodsResultsDiscussionFundingAcknowledgementsReferences |
|---|
Main findings
The key findings of these two studies in relation to informing the policy recommendations by the FSA's Board were:
- that there was a strong support for mandatory fortification from young women in deprived areas. These women are in a socio-economic group linked to a higher risk of NTD-affected pregnancies, as well as having higher rates of unplanned pregnancies.
- with information and time for deliberation, it was also the preferred option for people from a range of demographic backgrounds.
- that there were outstanding concerns about risk, and the maintenance of consumer choice, which would need to be addressed.
What is already known?
There have been several quantitative surveys carried out in the UK, which have assessed the levels of awareness of folic acid and NTDs, and knowledge about recommendations to increase uptake peri-conceptually.17–30 It can be difficult to compare results from these, because of the different definitions and methods used.5 Generally, the factors that are associated with lower awareness include: being a lone parent or unemployed or from a lower socio-economic group or being younger or from other racial/ethnic groups; lacking awareness/knowledge of the potential benefits, not being convinced of efficacy; and having a less healthy lifestyle. These support the qualitative findings on awareness and knowledge reported in this paper.
In the review commissioned as part of the FSA's research,5 only one qualitative study from the UK was included.31 This focused on influences on young women, women in low-income groups and those with poor educational backgrounds. The most relevant findings included a fatalistic view of becoming pregnant, and the importance of advice from mothers and friends compared with health professionals. The attitudes reported are consistent with the relevant aspects of Study 2, which also focused on young women from deprived areas.
What this work adds
In addition to assessing attitudes and knowledge, the studies reported here assessed consumer views of the options being considered by the FSA. The findings, together with the other FSA commissioned research5 (Fuller-Deets and Dingwall, 2007, unpublished), fed directly into the option appraisal carried out by the FSA Board in May 2007. Following this meeting, the Board agreed to recommend mandatory fortification of bread or flour to the UK Health Ministers.
The particular strengths of this research were that it enabled a consumer perspective to complement scientific discussions, and contribute in a unique way to the development of policy. One of the studies gathered spontaneous views of the options in a sample of women from the group most at risk of NTDs. The other used a different methodology, which enabled lay people to reach an informed and rational view of a complex issue. The results indicated that this had been achieved, with participants developing a more sophisticated understanding of the issues involved over the course of the research. Virtually all of those involved were able to reach a final decision that they were content with.
The results of these studies were considered a valuable input to the FSA Board's decision-making as it enabled the FSA to secure consumer evidence in an area where even organizations representing consumers were unsure of consumer views.
Limitations of this research
The scientific community can sometimes doubt the value of qualitative research. However, it provides an opportunity to provide participants with complicated information, and time for them to discuss this; flexibility for interviewers and facilitators to explore concerns; and to assess the wider environmental and personal context in which decisions are made.
Both of the qualitative studies described in this paper used methods that were appropriate to achieve their objectives. Study 1 adopted an approach that allowed time for reflection on complex issues. Study 2 used a method that is not only less demanding for participants but also elicits in depth information, which is relatively free from the influences of other participants (apart from the ‘friendship group’ element).
Of necessity, qualitative studies involve relatively small numbers of subjects, but they provide insights that cannot be provided by quantitative studies. Since the numbers are small, probability sampling is not possible. Instead, as in these studies, quota sampling is often used. This is a purposive approach, and the characteristics of the respondents reflect the variables chosen as quota controls, i.e. the sample may not be representative of the population group.
|
Funding |
|---|
|
TOPIntroductionMethodsResultsDiscussionFundingAcknowledgementsReferences |
|---|
Food Standards Agency.
|
Acknowledgements |
|---|
|
TOPIntroductionMethodsResultsDiscussionFundingAcknowledgementsReferences |
|---|
Studies 1 and 2 were commissioned by the Food Standards Agency, through the Central Office of Information, and carried out by forum and by Define Research and Insight respectively. The findings were used by the Food Standards Agency to inform policy development. The researchers would like to thank all of the respondents for their participation in this work.
|
References |
|---|
|
TOPIntroductionMethodsResultsDiscussionFundingAcknowledgementsReferences |
|---|
- MRC Vitamin Study Research Group. Prevention of neural tube defects: results of the MRC vitamin study. Lancet (1991) 338:131–7.[CrossRef][Web of Science][Medline]
- Czeizel AE, Dudas I. Prevention of the first occurrence of neural-tube defects by periconceptional vitamin supplementation. N Engl J Med (1992) 327(26):1832–5.[Abstract]
- Department of Health. Folic Acid and the Prevention of Neural Tube Defects: Report From an Expert Advisory Committee. (1992) London, UK: Department of Health.
- Department of Health. Report No 50: Folic Acid and the Prevention of Disease. (2000) London, UK: Department of Health.
- Stockley L, Lund V. Folic acid supplement uptake, particularly in low-income and young women: a series of systematic reviews to inform public health policy n the UK. Public Health Nutr. in press.
- Scientific Advisory Committee on Nutrition. Folate and Disease Prevention (2006) London: Scientific Advisory Committee on Nutrition.
- Food Safety Authority of Ireland. Report of the National Committee of Folic Acid Food Fortification (2006) Dublin: Food Safety Authority of Ireland.
- Food Standards Australia New Zealand. Initial Assessment Report: Consideration of Mandatory Fortification with Folic Acid (2004) Barton: Food Standards Australia New Zealand.
- Swedish Council on Technology Assessment in Health Care. Benefits of Fortifying Flour with Folic Acid with the Aim of Reducing the Risk of Neural Tube Defects: A Systematic Literature Survey (2007) Stockholm: Swedish Council on Technology Assessment in Health Care.
- Botto LD, Lisi A, Robert-Gnansia E, et al. International retrospective cohort study of neural tube defects in relation to folic acid recommendations: are the recommendations working? BMJ (2005) 330(7491):571.
[Abstract/Free Full Text] - Food Standards Agency. (2002) Board paper: FSA 02/05/02.
- Food Standards Agency. (2006) Board paper: FSA 06/04/05.
- Abelson J, Eyles J, McLeod CB, et al. Does deliberation make a difference? Results from a citizens panel study of health goals priority setting. Health Policy (2003) 66(1):95–106.[CrossRef][Web of Science][Medline]
- Abelson J, Forest PG, Eyles J, et al. Deliberations about deliberative methods: issues in the design and evaluation of public participation processes. Soc Sci Med (2003) 57(2):239–51.[CrossRef][Web of Science][Medline]
- Association for Qualitative Research. Qualitative Research Recruitment Best Practice—Rules and Guidelines (2002) St Neots, UK: Association for Qualitative Research.
- Experian. MOSAIC Classification System. http://www.ogro.org/experian.htm (July 2007, date last accessed).
- Blake M, Herrick K, Kelly Y. Health Survey for England 2002:Maternal and Infant Health (2003) London, UK: The Stationery Office.
- Glasgow Perinatal Effectiveness Committee. Audit/Re-audit of the Use of Folic Acid Supplements Preconception and in the First Trimester of Pregnancy (1998) Glasgow, UK: Glasgow Perinatal Effectiveness Committee.
- Haslam C, Lawrence W, Haefeli K. Intention to breastfeed and other important health-related behaviour and beliefs during pregnancy. Fam Pract (2003) 20(5):528–30.
[Abstract/Free Full Text] - Howell SR, Barnett AG, Underwood MR. The use of pre-conceptional folic acid as an indicator of uptake of a health message amongst white and Bangladeshi women in Tower Hamlets, east London. Fam Pract (2001) 18(3):300–3.
[Abstract/Free Full Text] - ICM. Research—1000 women aged 18+. Market Research (2004).
- Johnson K. A study of nutritional knowledge and supplement use in pregnant women. J Hum Nutr Diet (2000) 13(5):363.[Web of Science]
- Langley-Evans SC, Langley-Evans AJ. Use of folic acid supplements in the first trimester of pregnancy. J R Soc Health (2002) 122(3):181–6.[Web of Science][Medline]
- Mathews F, Yudkin P, Neil A. Folates in the periconceptional period: are women getting enough? Br J Obstet Gynaecol (1998) 105(9):954–9.[Web of Science][Medline]
- Neill AM, Laing RJ, Perez P, et al. The ‘Folic Acid Campaign’: has the message got through? A questionnaire study. J Obstet Gynaecol (1999) 19(1):22–5.[CrossRef][Medline]
- Relton CL, Hammal DM, Rankin J, et al. Folic acid supplementation and social deprivation. Public Health Nutr (2005) 8(3):338–40.[CrossRef][Web of Science][Medline]
- Rogers I, Emmett P. Diet during pregnancy in a population of pregnant women in South West England. ALSPAC Study Team. Avon longitudinal study of pregnancy and childhood. Eur J Clin Nutr (1998) 52(4):246–50.[CrossRef][Web of Science][Medline]
- Sen S, Manzoor A, Deviasumathy M, et al. Maternal knowledge, attitude and practice regarding folic acid intake during the periconceptional period. Public Health Nutr (2001) 4(4):909–12.[Web of Science][Medline]
- Sillender M. Continuing low uptake of periconceptional folate warrants increased food fortification. J Human Nutr Diet (2000) 13(6):425–31.[CrossRef]
- Sillender M, Pring DW. How effective was the Health Education Authority's folic acid campaign? J Obstet Gynaecol (2000) 20(3):271–6.[CrossRef][Medline]
- Health Education Authority. Folic Acid Research Report (1999) London, UK: Health Education Authority.
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||