Maternal and perinatal guideline development in hospitals in South East Asia: the experience of the SEA-ORCHID project

Background Clinical practice guidelines (CPGs) are commonly used to support practitioners to improve practice. However many studies have raised concerns about guideline quality. The reasons why guidelines are not developed following the established development methods are not clear. The SEA-ORCHID project aims to increase the generation and use of locally relevant research and improve clinical practice in maternal and perinatal care in four countries in South East Asia. Baseline data highlighted that development of evidence-based CPGs according to recommended processes was very rare in the SEA-ORCHID hospitals. The project investigators suggested that there were aspects of the recommended development process that made it very difficult in the participating hospitals. We therefore aimed to explore the experience of guideline development and particularly the enablers of and barriers to developing evidence-based guidelines in the nine hospitals in South East Asia participating in the SEA-ORCHID project, so as to better understand how evidence-based guideline development could be facilitated in these settings. Methods Semi-structured, face-to-face interviews were undertaken with senior and junior healthcare providers (nurses, midwives, doctors) from the maternal and neonatal services at each of the nine participating hospitals. Interviews were audio-recorded, transcribed and a thematic analysis undertaken. Results Seventy-five individual, 25 pair and eleven group interviews were conducted. Participants clearly valued evidence-based guidelines. However they also identified several major barriers to guideline development including time, lack of awareness of process, difficulties searching for evidence and arranging guideline development group meetings, issues with achieving multi-disciplinarity and consumer involvement. They also highlighted the central importance of keeping guidelines up-to-date. Conclusion Healthcare providers in the SEA-ORCHID hospitals face a series of barriers to developing evidence-based guidelines. At present, in many hospitals, several of these barriers are insurmountable, and as a result, rigorous, evidence-based guidelines are not being developed. Given the acknowledged benefits of evidence-based guidelines, perhaps a new approach to supporting their development in these contexts is needed.


Background
Clinical practice guidelines (CPGs) are commonly used to support practitioners to improve practice. Guidelines are "systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances" [1]. Evidence-based guideline development emphasises the importance of linking guideline recommendations to the scientific evidence that supports them, identified through a rigorous systematic identification and appraisal of all relevant research. Evidence-based guidelines should be developed by a multidisciplinary group which includes both clinicians and consumers. This group considers the best available research evidence and uses this as the basis of their recommendations. Where there is no relevant research evidence, recommendations are based on the consensus of the group.
Despite the fact that development of evidence-based guidelines is increasing [2,3] and there is a standard process for guideline development [4], which has been described in several published guides such as those produced by the Australian National Health and Medical Research Council, the UK National Institute for Health and Clinical Excellence, the New Zealand Guidelines Group and the Scottish Intercollegiate Guideline Network [5-8], many studies have raised concerns about the quality of recently developed guidelines [9][10][11][12][13][14]. Methodological issues have also recently been highlighted in a review of use of evidence in World Health Organization guidelines [15].
The reasons why guidelines are not developed following the established evidence-based guideline development methods are not clear, though the recommended process is extremely time and resource-intensive [3,[16][17][18] and requires a specific set of skills in identifying, appraising and synthesising research.
The South East Asia Optimising Reproductive and Child Health in Developing Countries (SEA-ORCHID, http:// www.seaorchid.org) project is a five-year collaborative project between Thailand, Malaysia, the Philippines, Indonesia, and Australia. By establishing a network of researchers and teachers of evidence-based health care in nine hospitals, across four South East Asian countries, supported from Australia, SEA-ORCHID aims to increase the generation and use of locally relevant research and improve clinical practice in management of pregnancy and childbirth. SEA-ORCHID is jointly funded by the Wellcome Trust and the Australian National Health and Medical Research Council.
The SEA-ORCHID project consists of five stages; prestudy, pre-intervention data collection, intervention, postintervention data collection, and reporting and dissemination. Details of the project methods and pre-intervention audit of clinical practice have been published previously [19,20] see Figure 1.
The evaluation of the SEA-ORCHID project includes quantitative evaluation of change in clinical practice, health outcomes, research activity and guideline development or adaptation combined with qualitative investigation of the barriers to and enablers of these changes.
Quantitative data from the baseline assessment in the SEA-ORCHID project indicated that development of evidencebased guidelines according to recommended processes, or adaptation and implementation of existing evidence-based guidelines was very rare in the SEA-ORCHID hospitals. As is reported in more detail elsewhere [21], only a small number of CPGs based on systematic searches for evidence and multidisciplinary involvement had been developed at national or regional level and were locally implemented. There were a few evidence-based guidelines available at some of the hospitals and most hospitals had a wide range of non-evidence-based local protocols.
The SEA-ORCHID Investigators suggested that there were aspects of the recommended evidence-based guideline development or adaptation process that made it very difficult, or not feasible, in the participating hospitals. As a result of these barriers, the Investigators thought that it The stages of the SEA-ORCHID project Figure 1 The stages of the SEA-ORCHID project.
was unlikely that substantial progress would be made in guideline development within the SEA-ORCHID project.
We therefore aimed to explore the experience of guideline development and particularly the enablers of and barriers to developing and adapting evidence-based guidelines in the nine hospitals in South East Asia participating in the SEA-ORCHID project, so as to better understand how evidence-based guideline development and adaptation could be facilitated in these settings.

Methods
During the final nine months of the intervention phase of the SEA-ORCHID project (July 2007 -February 2008), semi-structured, face-to-face interviews were undertaken with healthcare providers (nurses, midwives, doctors) from the maternal and neonatal services at each of the nine participating hospitals.
The interviews were conducted by two Australian SEA-ORCHID team members, a psychologist (JS) and a health services researcher (TT). The interviewers did not have clinical experience in the research area, but both had a background in evidence-based practice and research methods. JS had experience in supporting behavioural change from a psychological perspective and TT had experience implementing evidencebased change in a clinical setting and substantial experience in developing evidence-based guidelines. Where possible the interviews were conducted with both interviewers present, however, where made necessary by time limitations, interviews were conducted by one or other of the interviewers.

Participants
Participants were healthcare providers working in the maternal and neonatal areas of the SEA-ORCHID hospitals. At each hospital we aimed to interview two junior and two senior doctors, and two junior and two senior nurses or midwives, as relevant to the hospital structure. Interviewees included all levels of clinicians, including those responsible for patient care and organisational policy setting. We had hoped to interview consumers however this was not covered by our ethics approval.
We also interviewed healthcare providers who had undertaken evidence-based practice training fellowships offered at the Australian SEA-ORCHID hospitals during the course of the project and the members of the SEA-ORCHID team at each of the hospitals.
Interviews were most often conducted with individuals or pairs of healthcare providers, however a small number of group interviews were conducted.

Identification and consent
SEA-ORCHID personnel were asked to identify potential participants at each of the sites. Selection was largely prag-matic, based on availability for interview. Potential participants received an explanatory statement and signed a consent form to indicate informed consent. In Malaysia and the Philippines, explanatory statement and consent forms were provided in English, and interviews were conducted in English. In Indonesia and Thailand, the explanatory statement and consent forms were translated into the local languages and interpreters were used for the interviews.

Ethics approval
Ethics approval was granted by the University of Sydney, Monash University and the relevant ethics committees in each of the South East Asian countries.

Description of questions
Interviews were loosely based on a pre-specified interview protocol, but the detailed questions asked varied according to the level of experience and interest, and previous responses of the interviewees. The questions were designed to explore the interviewees' awareness and experience of undertaking evidence-base practice, doing research and developing guidelines or protocols. In this paper we report the results relating to development of guidelines or protocols.

Analysis
Interviews were audio recorded, transcribed and de-identified, and the data analysed thematically using NVivo software. Data relevant to guideline development were analysed in emerging themes based around the value, product and process of guideline development.

Results
Seventy-five individual, 25 pair and eleven group interviews were conducted in the nine SEA-ORCHID hospitals. In a small number of hospitals we were unable to interview the planned number of staff as there were insufficient staff employed and/or available. In other hospitals more interviews were conducted because of wider interest.
Participants had a wide range of levels of experience with evidence-based guideline development. Very few nurses or midwives had personal experience of developing evidence-based practice guidelines. Familiarity with the process was more common among doctors, and particularly senior doctors. Several senior doctors had attempted, or were currently attempting, to write guidelines and a small number were very experienced, having produced evidence-based guidelines at a local, regional and national level. Many of those who were currently involved in evidence-based guideline development had been taught or supported to do this through the SEA-ORCHID Project. Most interviewees had some experience of developing guidelines in their units, following non-evidence-based methods.
Responses were grouped into emerging themes as described below.
While there was wide variation between hospitals in levels of resourcing, organisational structure, number of deliveries etc, the results of the interviews were remarkably consistent across hospitals and countries. Particular thematic differences between hospitals are noted in the text.
Themes emerging from analysis:

THE PRODUCT Use of evidence in guidelines
Almost all interviewees believed that it was important to base guidelines on evidence.

First a good guideline for me is one that is based on research and good research that has been verified and the bad ones would be those that are not based on valid research. Junior Nurse
It is important because basically if it is based on the evidence previously tested in studies then we know, ... we can have proof to the patient that we have done our best. Junior Doctor Many noted, however, that the guidelines they developed, or those developed by others, were not based on research evidence.
The guidelines are still, in most hospitals, especially in the local hospitals outside the centre of education, I think from the experience are based on textbooks, according to what they've got when they are residents. Senior Doctor

Format of guidelines
Interviewees thought that the format of guidelines was very important. However, two very different thoughts were expressed by interviewees about the ideal format of guidelines: that they must be simple and quick to read and that they should be detailed and thorough.
It should be short and easy for read. Junior Doctor I think a good guideline should be very specific, in detail, so that everybody can read it and everybody can understand it more.

Senior Doctor
Interviewees also suggested that a diagram or flowchart was a very useful format for a guideline.
Definitely the flowchart because it is easier to understand and faster. Junior Doctor

THE PROCESS Time consuming and difficult
One of the recurring barriers to development identified in the interviews was how difficult and time consuming the process of guideline development was when undertaken rigorously. Many interviewees mentioned guideline development methods that were designed to meet the AGREE (Appraisal of Guidelines Research and Evaluation) Instrument [22] criteria, the most widely accepted tool for appraising the quality of guidelines. Scope of guidelines Some interviewees noted that the scope or breadth of the guidelines had a substantial impact on the amount of work involved in developing the guideline.

So I think that why it takes a long time is because of the scope. Senior Doctor
Adapting existing guidelines Several of the interviewees noted that they adapted guidelines for use in their hospitals based on those produced in other settings.
So I think our guidelines are more adaptations from, more than guidelines, it's not like we start from scratch to develop them.

Senior Doctor
Several different concerns were expressed about the difficulty of adapting externally generated guidelines for use in SEA-ORCHID hospitals. These included relevance of the existing guideline to the new setting, methodological concerns about the process of adapting guidelines, and availability of guidelines to adapt.

Relevance
Relevance of guidelines was a central concern. Differences in patient population, in culture, economic status and availability of equipment and infrastructure were all reported to reduce the relevance of guidelines developed in other settings.

Methodological concerns
Among those clinicians who accepted the concept of adapting existing guidelines, and had found relevant guidelines, some had concerns about the validity of methods for adapting guidelines.

The process [of adapting a guideline] also I think maybe easier but we have to have a good methodology to do this. ... [Can]
you put the reference to this just only one this guideline or you have to look the reference from the good guideline already and then put it into your guideline? ... -He has done already but you copy on your own, is not good, I think. Senior Doctor Interviewees also noted that following established methods for appraising the quality of guidelines before adapting them was also very time consuming.

No, we have not used the AGREE tool to appraise them because I think even that takes a long time. Senior Doctor
Availability Two barriers were identified as limiting the availability of guidelines to adapt. The first was an access issue; in some SEA-ORCHID hospitals clinicians reported that they were very limited in being able to access recent guidelines. The second issue was the lack of existing evidence-based guidelines in areas of interest.

I believe that we can apply these guidelines to the situation here, but one of the problems that I encounter is it's very difficult for us to actually get the guidelines because of access problems and sometimes we get guidelines that's not updated. Junior Doctor
A: What we are using now are the American guidelines. But many of the points there aren't really applicable here. So adapting a new one. Q: Are the guidelines here that you're adapting, are they evidence-based guidelines?

A: That's it, they're not. Senior Doctor
Searching for evidence Interviewees noted that searching for evidence to underpin their guidelines was of fundamental importance to the development process, but also that it was very time consuming.

The search for literature is very important because we have to get the most updated, the highest level of evidence. That takes time. Senior Doctor
Some interviewees explained that they did not undertake a systematic review of the literature as recommended by accepted methods of guideline development, but took a more pragmatic approach.

A few papers, probably RCTs or guidelines from books, from journals ... five to six references for each chapter -that's all what we do. Not an extensive literature review on that topic. Senior Doctor
Other interviewees noted that they were limited in their ability to carry out searches for research evidence because of limitations on availability of computers, internet access and full text journal articles.
We have internet access here, but our computer is non-functional right now. ... It's, we have only one computer for that area, ... we're using only one for about 50 people actually. Junior Doctor The electronic resources is not very reliable. There is but sometimes it's running very slow ... and people don't want to sit there very long in front of the computer on the internet so that is also one of the challenges we have to face so not only the skills but also the resources itself is not very adequate. Senior Doctor Though not often linked directly to guideline development, participants, and particularly nurses, frequently noted that lack of skill in English and lack of skill in using computers and the internet limited their ability to search for evidence

I do this to search from website Cochrane Library but I have the problem with English, it's a little bit difficult for me to read. Junior Nurse
Well, the SEA-ORCHID project has already given us great facilities, like the computer for internet, for browsing. But unfortunately we have such lack of capacity in actually using the internet. Senior Nurse

Arranging guideline development group meetings
Interviewees highlighted that there were basic hurdles with the guideline development process that could not easily be overcome. One of the most frequently mentioned was the need to arrange meetings of those involved in guideline development.
So if you want to meet with the four of us, it's almost impossible to have everybody free at one time. We can make a minimum of three, but ... it's so difficult actually, sometimes they have something on, and then we have something on, and it gets postponed. And it dies off. Senior Doctor We're at a standstill now, we're still in review of literature, we haven't been able to move on, it's so hard to get everyone together. Senior Doctor

Multidisciplinarity
Having a multidisciplinary group responsible for guideline development, as is recommended by evidence-based guideline development methods, was not usual practice at most SEA-ORCHID hospitals. Most often the guidelines were largely developed by the senior medical staff. In some hospitals this was beginning to change.

I won't say that all guidelines, but most guidelines will actually involve the nurses because in the past there was only the doctors would decide. But recently the nurses contribution and this is slightly more. Senior Doctor
And in some hospitals it was usual practice to involve nurses and midwives in some aspects of the guideline development process.

Consumer involvement
None of the interviewees had experience in involving patients or their families or carers in the development of guidelines, as is recommended by evidence-based guideline development methods.

We don't usually involve the consumers.... During our stay in Australia, we had a visit with a group of consumers, and when I realise that we have to involve the consumers in developing guidelines, that unfortunately we don't easily involve them. Senior Doctor
Some interviewees were very uncomfortable with the idea of involving consumers in guidelines development. These interviewees were all less experienced with the guideline development process.

I don't think it's very practical, and it's not very safe. Because what patients know and what patients want do not always correspond to what is the best practice at that point in time. Junior Doctor
Other interviewees thought that there might be a role for consumers, but felt that there were substantial limitations on that role.

Q: What do you think about consumer involvement in a hospital setting?
A: Ideally that would be great.

Achieving consensus
The process of achieving consensus on the recommendations of the guideline was felt to be very important, especially in terms of enabling the later implementability of the guideline, but also a potential barrier to development if agreement could not be reached. One interviewee noted that what was need was a regular updating process, with dedicated clinicians to maintain the guidelines.
With these guidelines is that some of them are outdated already, so they have to be updated. So there has to be a committed set of people who will -like in Cochrane, that they have updates periodically. Senior Doctor

Discussion
Almost all the healthcare providers interviewed in these hospitals were positive about the potential for guidelines to improve and standardise practice -at least in theory. Interviewees were less confident that the guidelines were achieving their potential, largely as the result of two factors, barriers to the development of guidelines and barriers to the use of guidelines. These results echo those of other studies in similar settings. For example, a study of facilitators and barriers to the adoption of evidence-based perinatal care in Latin America found that guidelines were both valued and felt to be necessary but were not enough to change practice on their own (Belizan 2007).
Healthcare providers in our study were also clear that guidelines should be developed on the basis of the best available research evidence, but expressed strong concerns about the feasibility of the current methods to achieve that in their setting. The substantial time investment required to find evidence and lack of access to the internet and evidence sources, were repeatedly mentioned as barriers to this process. Limitations in skill in finding and appraising evidence for use in guidelines were mentioned only in passing, however this may reflect the fact that most interviewees had not undertaken the systematic reviews of research required by the established guideline development methods, and were unaware of the skills required. Lack of awareness of the development process or lack of acceptance of an evidence-based framework to clinical practice were also noted barriers.
Arranging times for the guideline development group to meet was a major barrier to guideline development. This is particularly interesting given that at most sites, these meetings were only of the senior medical staff. Involving nurses or midwives, junior staff and consumers would be likely to make arranging meetings only more difficult. Wider representation may also complicate the process of achieving consensus, which several clinicians reported was already very difficult.
One of the strongest messages from the interviews was the central importance of keeping guidelines up-to-date with advances in research. While acknowledging the work involved, interviewees felt very strongly that this should be a priority.
There are clearly substantial barriers to developing evidence-based guidelines in the SEA-ORCHID hospitals. The extensive time commitment required for systematic reviews, need for reliable access to research evidence and difficulties in arranging guideline development group meetings were fundamental problems that stopped interviewees from developing evidence-based guidelines. While adapting existing guidelines can be a useful approach, clinicians report that it is not always possible or appropriate. Issues like multidisciplinary representation and involvement of consumers were also reported as being difficult to address. If evidence-based guidelines are going to be developed or adapted and implemented in these resource-poor environments, perhaps a more pragmatic approach is needed. Such an approach might limit the process of searching for evidence to a small number of reliable sources of high quality evidence, require a less intensive multidisciplinary consultation process, or focus on a smaller number of priority clinical questions with the potential to more significantly impact to on clinical practice, rather than aiming to address an entire clinical area. It would be interesting to develop and test a simpler, more practical approach to guideline development and compare the results of this new approach against the established rigorous processes. Alternative strategies such as financial incentives or regulatory requirements for hospitals to have guidelines might also be beneficial in increasing the development of guidelines. Similar research in developed country settings would also provide useful information on to what extent the experience of guideline development is common across settings. This study has a number of limitations. Interviewees were selected on a largely pragmatic basis and their views may not be representative of all of their colleagues. However the healthcare providers interviewed provided a wide range of views on the issues discussed. Only a minority of the healthcare providers interviewed had in-depth experience with the evidence-based guideline development process, and those who were familiar with the process were more likely to be experienced and from a medical background, so these results may overemphasise the views of these groups of clinicians. However the fact that junior clinicians and nurses or midwives were less likely to have been involved in guideline development highlights one of the aspects of the recommended guideline development process which is not being followed at many of the SEA-ORCHID sites, a guideline development group that includes representation from all relevant professional disciplines, levels of seniority and consumers.

Conclusion
Healthcare providers in the SEA-ORCHID hospitals face a series of barriers to developing evidence-based guidelines. At present, in many hospitals, several of these barriers seem insurmountable, and as a result, rigorous, evidencebased guidelines are not being developed. Given the acknowledged benefits of evidence-based guidelines, perhaps a new approach to supporting their development in these contexts is needed.