Open Access Research

Prospective policy analysis: how an epistemic community informed policymaking on intentional self poisoning in Sri Lanka

Melissa Pearson12*, Anthony B Zwi2* and Nicholas A Buckley13

Author Affiliations

1 South Asia Clinical Toxicology Research Collaboration, Faculty of Medicine, University of Peradeniya, Sri Lanka

2 School of Public Health and Community Medicine, Faculty of Medicine, The University of New South Wales, Sydney Australia

3 Prince of Wales Hospital Clinical School, The University of New South Wales, NSW Australia

For all author emails, please log on.

Health Research Policy and Systems 2010, 8:19  doi:10.1186/1478-4505-8-19

The electronic version of this article is the complete one and can be found online at: http://www.health-policy-systems.com/content/8/1/19


Received:11 February 2010
Accepted:17 June 2010
Published:17 June 2010

© 2010 Pearson et al; licensee BioMed Central Ltd.

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Background

Policy analysis is often retrospective and not well suited to helping policy makers decide what to do; in contrast prospective policy analysis seeks to assist in formulating responses to challenging public policy questions. Suicide in Sri Lanka is a major public health problem, with ingestion of pesticides being the primary method. Previous policy interventions have been associated with reduced mortality through restricting access to the most toxic pesticides. Additional means of reducing access are still needed.

Methods

The prospective policy analysis comprised two stages. The first used a consensus activity within a well defined policy community to generate and frame policy options. The second broadened the analysis to include other stakeholders. We report the consensus activity with seven actors from agriculture, health, and academia. Policy options were identified through two rounds of discussion along with ratings by each participant on their degree of support for each option. Data were analysed quantitatively and discussions analysed with Nvivo 8 to code prominent and recurrent themes.

Results

The main finding was the strong support and consensus for two proposals: further regulation of pesticides and the novel idea of repackaging pesticides into non-lethal doses. Participants identified several factors that were supportive of future policy change including a strong legislative framework, good links between agriculture, health and academia, and a collaborative relationship with industry. Identified barriers and potential threats to policy change included political interference, difficulties of intersectoral collaboration, acceptability of options to the community, difficulty of implementation in rural communities and the challenge of reducing mortality.

Conclusions

The development and consideration of policy options within this epistemic community reflected an appreciation and understanding of many of the factors that can facilitate or thwart policy change. The understanding of context, evidence and ideas, implementation and impact influenced how the participants considered and rated the options. Use of epistemic community actors identified the level of support for each option, helped elaborate the particularities of context, as well as the power and influence of ideas. Further examination of the potential barriers and opportunities for these options will determine if broader consensus, involving a wider range of stakeholders, can be achieved and policy change promoted.

Background

Suicide in Sri Lanka

Suicide in Sri Lanka is a major public health problem. In 1995 the country had the highest rate of suicide worldwide (approximately 47 per 100,000) [1]. Recent analysis of the incidence of suicide has shown a substantial decline from the peak of 47 in 1995 (male 80 and female 28) to 24 per 100,000 in 2005 (male 37 and female 10) [2]. Young people are over-represented in admission rates following self-harm. In 2000, 60% of all self-harm admissions were aged 16-25 [3]. Of all the suicide deaths recorded, between 60 - 80% are caused by the ingestion of pesticides [4]. In many rural districts of Sri Lanka the most common cause of death from any cause is from the intentional ingestion of pesticides [5]. While deaths from suicide overall have been falling, there remains a considerable burden of intentional self poisoning [6-9].

Policy responses to the problem of suicide in Sri Lanka included establishing a Presidential Committee, legislative changes, and improved clinical management. The Presidential Committee (formed in 1996) developed a National Suicide Prevention Strategy in 1998 with the goals of:

• Reducing easy access to lethal methods.

• Promoting research on reducing the lethality of pesticides in use.

• Educating the public on less harmful use of pesticides

• Creating a culture which discourages suicides

• Ensuring survival after poisoning

• Removing legal barriers to the correct handling of those at risk [7].

Since the inception of the Control of Pesticides Act 1980 the Department of Agriculture has embarked on a concerted programme to regulate the most toxic pesticides. These restrictions have been associated with considerable success, as described above, in reducing the overall mortality from intentional self-poisoning [10,11]. In 2008 the Department of Agriculture announced a phased withdrawal of three pesticides (Paraquat, Dimethoate and Fenthion) based on strong evidence of the high death rates associated with their misuse in rural communities. Following these decisions there was concern from stakeholders about which interventions should, at that stage, be pursued and how best to continue to reduce mortality associated with the intentional ingestion of pesticides.

Health Policy Analysis

Health policy analysis is a reasonably developed field in industrialized nations but only a relatively small body of work has emerged from low and middle income countries [12,13]. Lessons to date have recently been synthesized [14,15]. Since the early 1990's there has been a growing interest in why policies do not achieve their anticipated impact and much research has moved away from analysis of the content of policy to focus more on the political dimensions of policymaking and implementation [16,17]. Most reported health policy analyses have been retrospective case studies looking at the success or failure of past policies [15], yet policymakers need prospective tools to help analyse strategies before they are implemented.

Health policy analysis draws on two prominent discourses, based on different disciplinary influences. While not mutually exclusive their traditions differ slightly with public policy analysis rooted in the social and political economy frameworks of public policy while evidence-informed policymaking is rooted in the scientific and health care domains and draws from, but refines, the evidence-based medicine traditions.

Within the health field a popular approach to examining policy is to utilise the 'policy triangle' - an elegant and simple tool which has been used extensively in low and middle income countries and identifies the content of policy, contextual factors, processes and actors as all deserving of attention [13]. Walt and Gilson suggest that these factors are intertwined and need to be systematically considered to achieve an holistic understanding [18]. Buse, an advocate of prospective policy analysis [19], has gone further to develop a policy engagement framework to prospectively analyse policy that incorporates strategies for change (see Figure 1). The policy engagement framework "offers a systematic approach to the ongoing collection, analysis and use of political information (e.g. concerning actors, their interests, institutions, ideas, and policy processes and context) that can alter the balance of power between those in support of and those resisting change by enabling pro-reformers to intervene more effectively in the policy process" [20]. This framework, like the political mapping approach of Reich [12], identifies the key domains of interest in analyzing policy prospectively.

thumbnailFigure 1. Policy Engagement Framework. Buse K, et al:2008 [39].

In addition to these domains developed through the policy sciences traditions, there is increasing focus on the pathways to promote evidence-informed policy-making [21]. Recent publication of the SUPPORT tools for evidence-informed policymaking seeks to guide policymakers and the policy community to access better knowledge to support policy making. The 18 papers in the SUPPORT suite address four broad areas related to policymaking: supporting evidence-informed policymaking, identifying needs, finding and assessing information and implementation [22]. They propose that well informed policy decisions are made based on using systematic reviews and knowledge of local conditions in order to assess benefits, harms and costs and make judgements about trade-offs, as seen in Figure 2. While these tools are specifically geared towards policymakers' uses of evidence to inform policy, they recognise that evidence is only one factor in a multi-faceted process and acknowledge that this framework could be used to complement other strategies.

thumbnailFigure 2. Well informed policy decisions. Oxman et al. 2009 [39].

Epistemic Communities

Lindblom argued that a good policy is one that commands consensus among policy makers and interest groups [23]. All the above frameworks and tools identify the importance of actors and stakeholders, reflecting the view of policy as the outcome of continuing interaction between diverse agendas, multiple stakeholders, and contested and ambiguous issues [24]. One important type of stakeholder network that is at times influential is an epistemic community - "a network of professionals with recognised expertise and competence in a particular domain and an authoritative claim to policy-relevant knowledge in that domain or issue area" [25]. This group is tightly integrated, limited in number, with high continuity and members having roughly equal status and power [26]. Thus, working with an epistemic community to generate a range of options for further analysis may itself yield important insights into policy, as well as into how the epistemic community operates, perceives the problem(s) and proposed solutions.

Methods

We adopted two approaches to this problem; the first part a consensus exercise with the epistemic community to develop a range of policy options, and the second applying analytic tools to specific policy proposals with a wider range of stakeholders. This article details the findings from the first approach.

Consensus methods were developed by the RAND Corporation in 1969 to bring scientific rigour into collective processes for decision-making. The underlying aim of consensus methods such as a Delphi method "is to determine the extent to which experts agree about a certain issue" [27]. The consensus exercise employed here sought to generate policy options to reduce deaths from intentional poisoning with pesticides and used a modified Delphi technique that included four phases. The first and second phase involved the generation of a range of options from the literature and discussion between panellists, followed by a first round of voting. The third phase included feedback of the results from the first round of voting, a discussion of where consensus and dissensus were present and finally a second round of voting and ranking of options. A face-to-face Delphi technique was used as group processes have been found to be superior to non-group processes in terms of the number, quality and generation of unique ideas [28]. This technique was further supported by the presence, willingness and feasibility of organising known experts in the field to participate in a group process. The exercise aimed to develop a range of options to address the problem of the use of pesticides in suicide and to assess the feasibility and desirability of each to better control this hazard.

The exercise was facilitated by a non-voting established topic expert (NB) who was well known to the majority of participants, in keeping with good practice in consensus exercises. The panel members were selected from the existing policy community and included two Department of Agriculture officers, two international toxicologists with significant experience in Sri Lanka, one international public health academic with experience in Sri Lanka, one local academic actively involved in the field for the past 20 years and one local senior clinician/researcher. Evidence has shown that while panel size is not particularly important, attention to panel composition is crucial [29,30]. The generation of suggested panel members was guided by the local study review team to ensure an appropriate range of informants. All panel members knew each other and had collaborated on numerous previous projects, but their 'base' was in different disciplines and institutions.

Data were analysed both quantitatively and qualitatively. Analysis of the consensus level and the direction of the consensus was based on the method developed by de Loe [28] for data analysis of Delphi policy exercises shown in table 1. This system describes the level of consensus between categories as well as where the support is located with a high level of consensus defined as 70% of ratings in one agreement category or 80% in two contiguous categories. This was found to be superior to median and inter-quartile range analysis [28]. Qualitative data were analysed thematically using Nvivo 8 software. Data were coded for emergent and divergent themes. Ethical approval for the study was obtained from the University of New South Wales and University of Ruhuna.

Table 1. Definition and levels of consensus

Results

Generation of Options

Eight options were generated in the first round of discussion and described in Table 2. All of these options were supported from the literature however the generation of a novel suggestion that incorporated several ideas together, repackaging into non-lethal doses, was an innovative outcome from the discussion.

Table 2. Description of options generated through consensus activity

Ratings of Options

The first round of voting generated a high level of support for six of the 8 options: regulation, safe storage, Integrated Pesticide Management (IPM), repackaging, advertising and using bio-pesticides. There were mixed levels of support and opposition to the two remaining policy options, taxation and dealer training programmes. In round one the level of consensus was high in seven of the eight options, with only price and tax incentives being contentious. The discussion between the panellists generated views about the strengths and weaknesses of the various options and has been summarised in Table 3. The second round of voting generated a greater level of differentiation between the options and reflected changes in response to the discussion regarding the desirability of each option. A high level of consensus was only achieved in five of the options. The policy options of regulation and repackaging were more strongly supported than others, with consensus growing for repackaging. This change observed between round 1 and 2 voting can be seen in Figure 3 where support was seen to drop away for IPM, bio-pesticides, price/tax, safe storage, dealer training and advertising. A final ranking of the options seen in Table 3 was initiated by the panellists and identified further regulation and repackaging as the preferred options, followed by IPM, safe storage, advertising, using bio-pesticides, training dealers and taxation.

Table 3. Final Ranking of Options

thumbnailFigure 3. Comparison of voting between the rounds demonstrating high levels of support for regulation and repackaging.

Policy Aspects

Through the discussion several themes about policy emerged: options were identified and framed in terms of their strengths and weaknesses, local conditions and contextual factors were highlighted and factors that could modify policy were identified. The panellists went through a subconscious but deliberate process of weighing up the various benefits, harms and costs of each option as seen in Table 4. This process was highlighted by the discussion surrounding price incentives and weighing the impact of political involvement.

Table 4. Strengths and Weakness of the identified options

"You probably could do this but you would probably end up with long protracted battles and perhaps a lot more political involvement than would bring you benefit." IT 2

Consideration by the panellists of the benefits, harms and costs through the rounds of discussion helped them to make final assessments of their levels of support or opposition to the various options.

Important contextual issues were highlighted by the panellists in relation to the policy process. The delegated authority through the Control of Pesticides Act 1980 to the Pesticide Technical and Advisory Committee (PeTAC) depoliticised many aspects of pesticide control and regulation. This allowed a technical focus for decision making, free of interference from local manufacturing commercial interests.

"In the Sri Lankan situation...there is no need for us to go further to get policy changes done. We have full freedom to make the decisions. There was never a necessity for us to convince policy makers or politicians." Ag 1

In addition to the decision making strength of the PeTAC committee the other perceived strengths of the legislation was its ability to have an impact at field level. PeTAC's decisions to withdraw registration for particular products were seen as important previous successes.

"Evidence for its impact is already in place" PH1

Strong links between decision making, research and technical advice assisted policymaking. In addition the relationships within agriculture between the public and private sector were strong reflecting close kinship ties generated from links made in the higher education system. These links remained important and the culture of the industry was seen as having been supportive of policy changes. Finally there was a strong belief that policymaking should be informed by evidence, and ongoing surveillance of poisoning was important to continuing the technical basis for decision making.

Several factors were identified as posing threats to some of the potential options. Risks of political interference resulting from pressure from commercial interests and political patronage structures within farming communities were identified. The lack of engagement in pesticide control of the farming communities has strengthened the ability of policymaking but highlighted a potential threat if the issue were to become politicised. Lack of evidence or insufficient evidence of impact was a key factor in the lower ratings accorded to some options, as illustrated by the following quote.

"We have no evidence of it working yet and until we have evidence of it working; it is hard to push it strongly as a policy" IT2

The multidimensional context of suicide and the intersectoral action required for solutions highlights the sensitivity of policy to the conflicting interests of each sector. It also made more complex how the problem was defined and what solutions were being proposed. At times this complexity was seen as an issue that those in the health and agriculture sectors could avoid:

"The social aspects of suicide ... are outside the borders of what pesticides are meant for" Ag 1.

Difficulties of implementation were often related to weak systems, weak linkages between the sectors and the likely costs. The acceptability of the policy options to the community and the need for behaviour change to accompany policy responses were highlighted as difficulties of implementation at the grass root level. In addition the commission and marketing system for pesticide dealers created market incentives that challenged safe use initiatives and programmes designed to reduce pesticide use.

Discussion

The themes emerging from the discussion within the epistemic community highlight the complex and multi-dimensional nature of policy. Much policy activity happens under the radar of the grand gestures of high profile speeches and policy documents, and a broader understanding of policy as a "continuing process of social action and interaction" [31] is warranted.

The consideration of policy options within this epistemic community, working on pesticide control to reduce suicide in Sri Lanka, is framed by how they conceived and understood policymaking. The idea of policy within this community is of a rational process of decision making, "a collective attempt to construct a policy in order to address some evident problem" [32]. This conceptualisation of policy draws from a 'stages' heuristic that sees policy as a progression through the stages of problem identification, discussion of options, rational choice and implementation. These frames for policy are also apparent in the evidence-based policymaking literature that applies a scientific rational model to decision making. The support and consensus generated for regulation and repackaging of dangerous pesticides was partly based on the delegated authority of the Pesticide Technical Advisory Committee and its ability to access scientific evidence. The support for these options can be understood as reinforcing their ideas and understanding of a linear and rational process through which policy is taken forward.

Despite the intuitive appeal of the rational decision-making frame, the discussion highlighted the limitations of this model in guiding action. Frameworks that incorporate political economy approaches such as the policy triangle and political mapping offer greater explanatory power. "Policy change is not simply a technocratic process based on rational analysis and that knowledge alone is not sufficient for policy change. Policy is profoundly political" [33]. A change in the voting patterns between the two rounds of policy option appraisal was heavily influenced by debates on the strength and weaknesses of each option and these were related not just to evidence of effectiveness, but also to considerations of context, the range of actors involved and their likely positions, feasibility of implementation and likely impact.

In the policy analysis triangle, actors are at the heart of the framework and this underlines the importance that individuals, organisations and networks play in influencing policy activity [18]. This exercise has identified a number of the key stakeholders and allows the mapping of the linkages and relationships as described from within this epistemic community. There are strong links and informal connections within this community (Figure 4) that are based on a shared understanding of the problem and long-term engagement in the sector. The relationships between members of the epistemic community and outside actors are mostly divided along sectoral lines (health and agriculture). The relationships between the actors outside this community need further exploration to understand their influence on policymaking and to frame the options beyond those generated by this epistemic community.

thumbnailFigure 4. Linkages and strength of relationships between epistemic community members and other policy actors.

The 'problem' was perceived in this community in three main ways: people swallowing pesticides (clearly not their intended use), lack of evidence for interventions that have an impact on mortality, and problems of implementation. These narratives of the problem point to a more complex interplay of factors that determine the relative suitability of options rather than simply their content. Within this local context the consensus and support for the options of regulation and repackaging were perceived to be more suitable because they reduced access to means, there was strong evidence that they could work and the legislation and authority existed for their implementation.

Knowledge and evidence are the key features of epistemic communities and therefore closer examination of the flow of knowledge, status of the evidence and its role within this community is useful. The majority of the evidence came from two sources, plant protection research and poisoning research. These two areas of research have direct linkages into the Pesticide Technical Advisory Committee and in turn this strengthened the ties between the epistemic community members and 'institutionalises their influence' [25]. The status of the evidence base was strong and much was known about the choice of poison from anecdotal sources as well as evidence from surveillance. This use of epidemiological data and clinical experience created strong links between the health actors and provided a solid foundation for advocacy within, and from, the epistemic community to the broader social policy domain.

Interaction between the epistemic community and policy makers is needed "to ensure knowledge generation is policy relevant"[34]. The linkages between the knowledge generators and the policymakers were strong and integral to this community and served a mutually reinforcing function. Knowledge and evidence was generated and interpreted within the community and placed on the agenda with decision makers. These linkages served to reinforce the boundary within the community and ensured that evidence informed policymaking was maintained [35]. However while this epistemic community is influential and well linked the solutions they generate are limited to ones that supported, or at least fitted comfortably within the status quo. The solutions they proposed offer the opportunity to change things from within agreed values, discourse and interests [36]. If more radical change is needed or desired then this community would need to engage in a wider process to generate support for such options.

This exercise exposed potential policy options that would garner support from this influential and powerful epistemic community and is akin to the deliberative process advocated for by Culyer and Lomas [37]. It produced outcomes that have been suggested as important features of deliberative processes including the contextualisation of proposed interventions, identified relevant clinical, social and political influences, conferred credibility, and identified impediments to implementation.

In addition to knowledge and evidence generated from within this community there is a need to consider other areas of influence on policymaking, as highlighted by Bowen and Zwi [21]. The SUPPORT tools also help to provide some guidance on additional areas that are necessary when framing options: technical feasibility fits with dominant values, acceptable in terms of budget and acceptability to stakeholders [38]. Further exploration is needed of the acceptability of the options to stakeholders, likely costs and cost-effectiveness in order to generate consensus beyond this community.

Limitations of the study

A limitation of this study was the number and range of panellists available to participate. However, a number of studies have shown that panel size is not always a good indicator of the success of a consensus exercise [29,30]. The consideration of panel size can only be understood in relation to the ability of the exercise to produce innovative ideas. In this case study, the process produced a new policy option that blended together, in an innovative way, several previously considered options. The advantages of having the face to face group process were that it enabled a robust discussion of the options and a thorough examination of the strengths and weaknesses of each. The limited range of experts meant that the panellists were familiar with the background literature and the main proposed solutions and so the discussion was able to be more intensive and focussed, with limited defensiveness on the part of the participants given the high levels of continuity and trust that operated within the community.

Conclusions

The conceptualisation of policy as multi-dimensional, which includes aspects of context, networks, knowledge, implementation and impact, influenced how the participants considered and rated the options. Development and consideration of policy options to reduce access to pesticides in Sri Lanka generated the strongest support and consensus for two options: further regulation of pesticides and repackaging of the most toxic formulations into safe doses. The consensus and support for these policy options reflects an understanding by the panellists of the factors that facilitate or thwart policy change within this specific problem and context. Further examination of the potential barriers and opportunities for the options identified will help ascertain the best strategy for implementation.

Engaging with the epistemic community provided a valuable insight into many important aspects of policymaking. The exercise provided an opportunity to identify the status of the evidence, debate the strengths and weaknesses of future policy options, and make judgements about trade-offs. The observed discussion revealed insights into how this community operates - the linkages and relationships within and outside of the community and the status and role of evidence and knowledge. Use of epistemic community actors revealed layers of contextual analysis that inform policymaking, even in ostensibly technical areas.

Competing interests

NB and MP work with the South Asian Clinical Toxicology Research Collaboration that aims to reduce deaths from pesticide poisoning through regional collaboration.

Authors' contributions

MP undertook the fieldwork, analysis and wrote the first draft of this paper. AZ provided guidance throughout in study design, conduct, analysis and writing of the paper. NB holds the grant under which this research has been funded, facilitated the exercise described, and contributed to write-up and analysis. All authors contributed to writing and agreed the final text.

Authors' information

MP is a research degree student supervised by AZ and co-supervised by NB and Dr Duncan McDuie-Ra.

Acknowledgements

We thank Dr Gamini Manuweera, Professor Ravindra Fernando, Dr Michael Eddleston, Professor Flemming Konradsen, Professor Andrew Dawson, Dr Indika Gawarammana and a former Director-General of Agriculture for agreeing to participate and offering their time and support. Additional thanks to Professor Dawson, Professor Fernando and Dr Gamini Manuweera for their guidance on participants, SACTRC Office and Faculty of Forensic Medicine, University of Colombo staff who facilitated the logistics and finally Dr A. Rouse who assisted in summarising the discussions. SACTRC provided the funding through a Wellcome Trust/NHMRC International Collaborative Research Grant GR071669MA.

References

  1. Ratnayake R: Capacity-building for emerging problems in developing countries: intentional self-poisoning and pesticides.

    Open Medicine 2008., 2(2) OpenURL

  2. Gunnell D, Fernando R, Hewagama M, Priyangika WDD, Konradsen F, Eddleston M: The impact of pesticide regulations on suicide in Sri Lanka.

    International Journal of Epidemiology 2007. PubMed Abstract OpenURL

  3. De Silva H, Kasturiaratchi N, Seneviratne S, Senaratne D, Molagoda A, Ellawala N: Suicide in Sri Lanka: points to ponder.

    Ceylon Medical Journal 2000, 45(1):17-24. PubMed Abstract OpenURL

  4. Social Conditions in Sri Lanka Department of Census and Statistics, Government of Sri Lanka; 2002.

  5. van der Hoek W: Analysis of 8000 hospital admissions for acute poisoning in a rural area of Sri Lanka.

    Clinical Toxicology 2006, 44(3):225-231. PubMed Abstract | Publisher Full Text OpenURL

  6. Abeyasinghe R, Gunnell D: Psychological autopsy study of suicide in three rural and semi-rural districts of Sri Lanka.

    Social Psychiatry and Psychiatric Epidemiology 2008, 43(4):280-285. PubMed Abstract | Publisher Full Text OpenURL

  7. MoH/WHO: National Report on the Prevention of Violence. Ministry of Health, Government of Sri Lanka; 2007.

  8. Samaraweera S, Sumathipala A, Siribaddana S, Sivayogan S, Bhugra D: Completed Suicide among Sinhalese in Sri Lanka: A Psychological Autopsy Study.

    Suicide and Life-Threatening Behavior 2008, 38(2):221-228. PubMed Abstract | Publisher Full Text OpenURL

  9. van der Hoek W, Konradsen F: Risk factors for acute pesticide poisoning in Sri Lanka.

    Tropical Medicine and International Health 2005, 10(6):589-596. PubMed Abstract | Publisher Full Text OpenURL

  10. Gunnell D, Fernando R, Hewagama M, Priyangika W, Konradsen F, Eddleston M: The impact of pesticide regulations on suicide in Sri Lanka.

    International Journal of Epidemiology 2007, 36(6):1235-42. PubMed Abstract | Publisher Full Text OpenURL

  11. Roberts D, Karunarathna A, Buckley N, Manuweera G, Sheriff M, Eddleston M: Influence of pesticide regulation on acute poisoning deaths in Sri Lanka.

    Bulletin of the World Health Organization 2003, 81:789-798. PubMed Abstract | PubMed Central Full Text OpenURL

  12. Reich M: Applied political analysis for health policy reform.

    Current issues in Public health 1996, 2:186-191. OpenURL

  13. Walt G, Gilson L: Reforming the health sector in developing countries: the central role of policy analysis.

    Health Policy and Planning 1994, 9(4):353. PubMed Abstract | Publisher Full Text OpenURL

  14. Gilson L, Raphaely N: The terrain of health policy analysis in low and middle income countries: a review of published literature 1994-2007.

    Health Policy and Planning 2008, 23(5):294. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  15. Walt G, Shiffman J, Schneider H, Murray S, Brugha R, Gilson L: 'Doing'health policy analysis: methodological and conceptual reflections and challenges.

    Health Policy and Planning 2008, 23(5):308. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  16. Buse K: How can the analysis of power and process in policy-making improve health outcomes?

    Volume ODI Briefing Paper 25, ODI 2007. OpenURL

  17. Reich M: Bangladesh pharmaceutical policy and politics.

    Health policy and planning 1994, 9(2):130. PubMed Abstract | Publisher Full Text OpenURL

  18. Buse K, Mays N, Walt G: Making health policy. Open University Press; 2005. OpenURL

  19. Buse K: Addressing the theoretical, practical and ethical challenges inherent in prospective health policy analysis.

    Health policy and planning 2008, 23(5):351. PubMed Abstract | Publisher Full Text OpenURL

  20. Buse K, Booth D, Murindwa G, Mwisongo A, Harmer A: Working Paper 7: Donors and the Political Dimensions of Health Sector Reform: The Cases of Tanzania and Uganda. In Good Governance, Aid Modalities and Poverty Reduction: Linkages to the Millennium Development Goals and Implications. London: Overseas Development Institute, London, UK; 2008. OpenURL

  21. Bowen S, Zwi A: Pathways to" evidence-informed" policy and practice: a framework for action.

    PLoS Medicine 2005, 2(7):600. Publisher Full Text OpenURL

  22. Lavis J, Oxman A, Lewin S, Fretheim A: SUPPORT Tools for evidence-informed health Policymaking (STP).

    Health Research Policy and Systems 2009, 7(Suppl 1):I1. PubMed Abstract | BioMed Central Full Text OpenURL

  23. Lindblom C: The science of" muddling through".

    Public administration review 1959, 19(2):79-88. Publisher Full Text OpenURL

  24. Colebatch H, Radin B: Mapping the work of policy.

    The work of policy: an international survey 2006, 217. OpenURL

  25. Haas P: Introduction: epistemic communities and international policy coordination.

    International organization 1992, 46(1):1-35. Publisher Full Text OpenURL

  26. Tantivess S, Walt G: The role of state and non-state actors in the policy process: the contribution of policy networks to the scale-up of antiretroviral therapy in Thailand.

    Health policy and planning 2008, 23(5):328. PubMed Abstract | Publisher Full Text OpenURL

  27. Jones J, Hunter D: Qualitative research: consensus methods for medical and health services research.

    British Medical Journal 1995, 311(7001):376. PubMed Abstract | PubMed Central Full Text OpenURL

  28. De Loe R: Exploring complex policy questions using the policy Delphi A multi-round, interactive survey method.

    Applied Geography 1995, 15(1):53-68. Publisher Full Text OpenURL

  29. Fink A, Kosecoff J, Chassin M, Brook R: Consensus methods: characteristics and guidelines for use.

    Am Public Health Assoc 1984, 74:979-983. Publisher Full Text OpenURL

  30. Rowe G, Wright G: The Delphi technique as a forecasting tool: issues and analysis.

    International Journal of Forecasting 1999, 15(4):353-375. Publisher Full Text OpenURL

  31. Colebatch H: Policy. second edition. Berkshire UK: Open University Press; 2002. OpenURL

  32. Colebatch H: What work makes policy?

    Policy Sciences 2006, 39(4):309-321. Publisher Full Text OpenURL

  33. Reich M: The politics of agenda setting in international health: child health versus adult health in developing countries.

    Journal of International Development 1995, 7(3):489-502. PubMed Abstract | Publisher Full Text OpenURL

  34. Jones N, Datta A, Jones H: Knowledge, policy and power: Six dimensions of the knowledge - development policy interface. London: Overseas Development Institute; 2009.

  35. Walt G: Health policy: an introduction to process and power. Witwatersrand University Press Publications; 1994. OpenURL

  36. Davies C: Understanding the policy process.

    The Critical Practitioner in Social Work and Health Care 2007, 203. OpenURL

  37. Culyer A, Lomas J: Deliberative processes and evidence-informed decision making in healthcare: do they work and how might we know?

    Evidence & Policy 2006, 2(3):357-371. OpenURL

  38. Lavis J, Wilson M, Oxman A, Grimshaw J, Lewin S, Fretheim A: SUPPORT Tools for evidence-informed health Policymaking (STP) 5: Using research evidence to frame options to address a problem.

    Health Research Policy and Systems 2009, 7(Suppl 1):S5. PubMed Abstract | BioMed Central Full Text OpenURL

  39. Oxman A, Lavis J, Lewin S, Fretheim A: SUPPORT Tools for evidence-informed health Policymaking (STP) 1: What is evidence-informed policymaking?

    Health Research Policy and Systems 2009, 7(Suppl 1):S1. PubMed Abstract | BioMed Central Full Text OpenURL