PDF version

Research article

N.A. Qureshi,¹ Y. Neyaz,² T. Khoja,³ M.A. Magzoub,⁴ A. Haycox ⁵ and T. Walley⁶

فعَّالية ثلاثة من التدخلات استهدفت وصف الأطباء للأدوية في الرعاية الصحية الأولية في مدينة الرياض، في المملكة العربية السعودية

نسيم قرشي، يعقوب نياز، توفيق خوجة، محمد علي مقزوب، آلان هيكوكس، توم فيليي

الخلاصـة: أثبت عدد من الاستراتيجيات قدرتَه على تحسين جودة وصف الأطباء للأدوية. واستهدفَتْ هذه الدراسة تنفيذ وتقييم فعَّالية ثلاثة تدخلات تستهدف وصف الأطباء للأدوية واحتواء تكاليفها؛ وهذه التدخلات هي: تدريب الأطباء على الوصف الجيد للأدوية، والإجراءات التنظيمية والإدارية لتحسين الوصف الرشيد للأدوية، وأسلوب متعدد الجوانب يستفيد من الاستراتيجيتَ السابقتَ إلى جانب عناصر إضافية عليهما. واختار الباحثون عشوائياً ثلاثة مراكز تابعة للقطاع العام لتطبيق إحدى الاستراتيجيات في واحد منها، فتمّ تدريب واحد وستين طبيباً على وصف الأدوية، ثمّ طُلِبَ إليهم أن يستكملوا استبيانات قبل التدريب وبعده. وأجرى الباحثون مقارنات على مئة وصفة كُتِبَتْ في أعقاب بعد التدخل في كل مركزٍ. ووجد الباحثون أن التدخلات الثلاثة جميعها قد حسَّنت تحسيناً فعَّالاً جودةَ وصف الأدوية، كما حسَّنت تدوين الأطباء للمعلومات المتعلقة بالأدوية، مما أدَّى إلى الحصول على تقييمات إيجابية لدى المتدربين عن الدورة التدريبية. ومن أجل معرفة استمرار هذا التحسُّن في وصف الأطباء للأدوية، لابُدَّ من إجراء تقييمات متابعة تالية.

ABSTRACT: A number of strategies have been shown to improve the quality of drug prescriptions. The objective of this study was to implement and assess the effectiveness of 3 interventions on physicians’ prescribing and cost containment: training physicians about quality prescribing; regulatory and administrative measures to improve rational drug prescribing; and a multi-faceted approach using 2 these strategies plus additional elements. Three public health centres, 1 for each intervention, were randomly selected; 61 physicians were trained in drug prescribing and completed a pre- and post-training questionnaire; and 100 post-intervention prescriptions from each centre were compared. All 3 interventions effectively improved the quality of drug prescriptions and the notation of drug-related information and trainees returned positive evaluations of the training course. Whether or not physicians’ improvement in prescribing will be sustained is unclear and therefore subsequent follow-up evaluations are needed.

Efficacité de trois interventions sur le comportement prescripteur des médecins en soins de santé primaires à Riyadh (Arabie saoudite)

RÉSUMÉ: Plusieurs stratégies ont été exposées pour améliorer la qualité de prescription de médicaments. La présente étude visait la mise en oeuvre de trois interventions relatives au comportement prescripteur des médecins et à la maîtrise des coûts, puis l’évaluation de leur efficacité : la formation des médecins concernant une prescription de qualité ; des mesures réglementaires et administratives pour améliorer la prescription rationnelle de médicaments ; et une approche multidimensionnelle ayant recours à deux de ces stratégies mais aussi à d’autres éléments. Trois centres de santé publics, un pour chaque intervention, ont été sélectionnés de façon aléatoire ; au total, 61 médecins ont été formés à la prescription de médicaments et ont rempli un questionnaire avant et après la formation ; puis 100 prescriptions rédigées après l’intervention et issues de chaque centre ont été comparées. Les trois interventions ont toutes amélioré efficacement la qualité des prescriptions de médicaments et la rédaction des informations relatives aux médicaments. En outre, les participants ont renvoyé des évaluations positives sur la formation. Il reste à déterminer si l’amélioration du comportement prescripteur des médecins sera maintenue. Par conséquent, des évaluations de suivi supplémentaires sont nécessaires. 

---

¹General Administration for Mental Health and Social Services and Medical Research;
²General Directorate for Quality Development, Ministry of Health, Riyadh, Saudi Arabia (Correspondence to N.A. Qureshi: This e-mail address is being protected from spambots. You need JavaScript enabled to view it ). ⁴Department of Medical Education, College of Medicine, King Abdulaziz Medical City, Riyadh, Saudi Arabia.
³Cooperation Council for the Arab States of the Gulf, Riyadh, Saudi Arabia.
⁵School of Management;
⁶Department of Internal Medicine, University of Liverpool, Liverpool, United Kingdom.

EMHJ, 2011, 17(2): 172-179

Introduction

Previous studies have suggested that the prescribing behaviour of primary care physicians in Saudi Arabia was not optimal [1,2]. These prompted a review of methods by which the quality of prescribing could be improved. We describe here a series of pilot studies of interventions that were designed to contribute towards an ongoing programme of training and education to improve the prescribing behaviour of physicians in Saudi Arabia.

Education has been identified as one of the key measures for improving the prescribing behaviour of doctors. In the case of antibiotic prescribing a Cochrane review of 39 studies examined the effectiveness of professional interventions, alone or in combination, in improving the selection, dose and treatment duration of antibiotics prescribed by health care providers in the outpatient setting [3]. It found that interactive educational meetings were more effective than didactic lectures, while educational outreach visits and physician reminders produced mixed results, and the use of printed educational materials or audit and feedback alone generally resulted in very limited changes in prescribing. The conclusions were that interventions must be tailored to the prescribing behaviour, and address the barriers to change, in a particular community and that multi-faceted interventions were more effective than any one approach.

Another approach is regulatory changes to modify physicians’ prescribing behaviour. A wide range of organizational changes were implemented across European nations around 2003–04 with the aim of reducing drug budgets, reducing prescription errors, enhancing the quality and rationality of prescribing and improving patient safety and satisfaction . These included improved training, better pharmacovigilance, incentive schemes, indicative prescribing and fund-holding schemes and insurance and welfare reforms [4]. The effects of such regulations have been mixed, however. Simple changes which accord with professional views of what is best practice have been successful, while more dramatic and complex initiatives have required rigorous enforcement to achieve change and any success in achieving behavioural change has generally not been sustained [4–10].

Clearly a variety of regulatory, administrative and educational initiatives are required to achieve good quality, cost-effective prescribing, but the potential impact of any 1 intervention on physicians’ prescribing behaviour is currently unknown [3]. Cochrane reviews [11] and other studies [12,13] have provided important evidence on the impact of initiatives in the context of advanced health systems but, to date, there is no information on how to influence prescribing in the Gulf states. Certainly it is unclear to what extent models developed in industrialized countries can be applied in the context of the Saudi Arabian health care system.

The objective of the present study was to explore 3 types of intervention that have been shown to have some effectiveness in advanced health systems: physician training; regulatory and administrative measures; and multiple interventions. The aim was to evaluate the impact of different initiatives on the quality of public sector physicians’ prescribing in the setting of Saudi Arabia. In addition, a cost assessment was performed for the 3 interventions as cost analyses for interventions of this nature have not previously been evaluated in Saudi Arabia.

Methods

A range of studies was conducted at 3 government-funded primary health care (PHC) centres in Saudi Arabia in 2004. Three types of intervention were evaluated: training physicians in the elements of good quality prescribing; regulatory and administrative measures to improve rational drug prescribing; and a multi-faceted approach using these 2 strategies plus several additional elements. Although more than 1 health centre participated in the interventions, each health centre was the focus of 1 particular intervention. These centres were selected randomly from a pool of 30 large urban PHC centres (i.e. with 6 or more physicians).

Design of the interventions

The design for each intervention was based on a detailed review of the national and international literature on the comparative effectiveness of prescribing interventions. The exact structure of each intervention was agreed following an initial conference organized by the investigator (Y.N.) and attended by academics and health care policy-makers. This conference mainly focused on identifying and drafting the content for the training curriculum. The secretarial representatives from the Gulf Co-operation Council of Health Ministers organized the training course, while Ministry of Health (MOH) representatives took responsibility for the regulation and administration of the intervention sites. Consultants and advisors were financially remunerated.

Training intervention

A training course was developed and offered to all local PHC physicians. A total of 61 doctors attended, including the 12 study physicians: 6 from the PHC centre which received only the training intervention and 6 from the centre which received multiple interventions. In order to ensure that services at the selected health centres were unaffected, doctors from other nearby PHC centres provided extra consultations as necessary.

The course was designed by the first author (N.A.Q.) and 2 family physicians (T.K. & M.M.) and finalized in a consensus meeting of 6 consultants. The comprehensive WHO medication prescribing module provided the guiding principle in selecting the drug prescribing topics to be presented [14]. Six senior health professionals delivered the course: 2 public health experts, 2 pharmacists and 2 family physicians with a keen interest in drug prescribing. The training course lasted 4 days and addressed high-quality prescribing and other related issues: problems of irrational use of drugs; problems of irrational use of drugs in Saudi Arabia; investigating drug use problems; drug interactions; guidelines for good drug prescribing; framework for changing drug use problems; management and regulatory interventions; clinical guidelines for acute respiratory infection; rational use of antibiotics; and clinical guidelines for chronic diseases. The course was built around interactive lectures with open discussions and question-and-answer forums and workshops which included group work and exercises. This encouraged the doctors to share their experiences so that a consensus drug management plan could be developed based on local circumstances.

Managerial and regulatory intervention

A range of administrative interventions were applied to 2 of the intervention health centres: one of which was given this intervention only, while the other was given this intervention as a part of the multiple intervention approach.

The main administrative intervention was to enhance physicians’ familiarity with the available clinical guidelines developed by the Ministry of Health. These guidelines covered: treatment of acute respiratory infections as well as a range of chronic diseases; limiting the number of drugs per prescription for any patient to a maximum of 3; and using generic names rather than brand names when writing prescriptions.

The clinical guidelines were distributed to the PHC centres, together with a memorandum signed by higher health authorities advising PHC physicians to adhere strictly to these guidelines. It was the responsibility of the supervisor of each PHC centre to ensure that these guidelines were followed by physicians. Similarly a memorandum emphasizing a maximum of 3 drugs per prescription was also circulated at the PHC centres. Similar procedures were applied to encourage PHC physicians to use generic drug names on prescriptions. The principal investigator (Y.N.) made sure that such measures were rigorously and routinely applied at the 2 intervention sites.

Multiple interventions

The multiple interventions part of the study was a combination of the interventions utilized in the other centres with several additional elements. Interventions included: training physicians on achieving optimal prescribing; enhancing adherence to available clinical guidelines; promoting use of generic rather than brand name drugs; limiting items in each prescription to only 1 drug; providing relevant information to patients; ensuring that all necessary data were completed in the consultation; requesting pharmacists to ensure that all the required information was available and clear before any drug was dispensed; and ensuring that all appropriate regulatory measures were adhered to.

Evaluation of the interventions

Effect of all interventions on quality of physicians’ prescribing

To assess the impact of all 3 interventions on the quality of physicians’ prescribing behaviour, the second author (Y.N.) collected copies of all prescriptions completed by the physicians involved in each of the 3 interventions over a 1-day period. This was completed 1 week after implementing the measures at the intervention sites. If the desired sample was inadequate (i.e. less than 100) a further sample covering the same day of the week at a later date was also collected. From the included prescriptions, a sample of 100 prescriptions was identified for analysis using a simple random sampling technique.

The prescriptions were examined and 3 kinds of indicators of quality prescribing were assessed: information items (patient’s file number, name, age, sex and nationality; diagnosis; number of drugs per prescription; date; prescriber’s name and signature with stamp); drug-related items (number of items/prescription; type of medication prescribed; instructions to the pharmacist (legibility, strength and dose of medication; frequency and duration of use; instructions for use)); and types of prescription error: errors identified on prescriptions were classified using Neville et al.’s criteria [15]. Details of the prescribing analysis have been described in earlier studies [1,2].

Effect of training intervention on physicians’ self-reported prescribing behaviour

A separate evaluation of the training intervention was conducted through a pre- and post-training questionnaire to collect information about physicians’ self-reported prescribing behaviour and participants’ feedback about the training course (4-point Lickert scale). The questionnaire (available upon request from the authors) was designed by the principal investigator in collaboration with 6 health professionals. Prior to the start of the training course, the pre-training questionnaire was distributed to each participant from the 2 intervention PHC centres. Immediately at the end of the 4-day training programme, the second author (Y.N.) distributed the questionnaire for recording their post-training responses and also feedback responses. After collection of the questionnaires, a round table meeting was held in which each participant was invited to provide additional feedback concerning the value of the training programme.

Cost analysis

An analysis of the cost of training was made based on the daily salaries of consultants who delivered the training and of the PHC physicians who attended, multiplied by the number of training days, plus the cost of other overhead expenses related to audiovisual operators, food, transport and stationery,

Data analysis

The data were entered in the computer using SPSS software version 10. Frequencies were computed for all variables. Univariate analysis and chi-squared tests were conducted. These statistical tests were used for cross-tabulation. Absolute differences between pre- and post-intervention responses were calculated. The Z-test was also used to compare the physicians’ of pre- and post- intervention responses and 95% confidence intervals (CI) were also calculated. P-value ≤ 0.05 was considered significant.

Results

Effect of 3 interventions on quality indicators of physicians’ prescribing

More details of the pre-intervention analysis of information recorded by physicians in prescriptions (n = 1182) from 5 public PHC clinics are shown in earlier studies [1,2]. Analysis of the same prescribing indicators in prescriptions (n = 100) from each of the 3 intervention practices showed improvements in all the prescribing indicators after the interventions (Table 1). Patient’s age and the diagnosis were noted in 100% of prescriptions following the interventions. The recording of ICD-9 code, doctor’s name, doctor’s signatures and date were also improved, but not significantly so, in prescriptions at the 3 intervention sites compared with the pre-intervention sites. The greatest improvements after the multiple interventions were in the recording of patient’s file number (χ² = 18.8, df = 2, P < 0.001) and clinical findings (χ² = 13.0, df = 2, P < 0.001).

Table 1 Information recorded on prescriptions from public primary health care (PHC) centres: comparison of data from the pre-intervention sites (5 health centres) and post-intervention sites (3 health centres, each receiving a different intervention)

Drug-related indicators were recorded on the same prescriptions before and after the interventions (Table 2). All 3 types of intervention improved the quality of physicians’ prescriptions. After the interventions the number of prescriptions with multiple drugs was reduced and prescriptions for antibiotics declined. Legibility of prescriptions was better and all items concerned with instructions to the pharmacist improved.

Table 2 Drug-related information recorded on by prescriptions from public primary health care (PHC) centres: comparison of data from the pre-intervention sites (5 health centres) and post-intervention sites (3 health centres, each receiving a different intervention)

Effect of training intervention on physicians’ prescribing knowledge

All 12 doctors from the 3 target intervention sites completed the pre- and-post training questionnaires on prescribing knowledge after attending the training course on rational drug prescribing. All the knowledge items tested showed significantly higher scores after the training compared with before (Table 3).

 Table 3 Pre- and post-intervention responses of primary care physicians concerning knowledge about prescribing after receiving training on rational drug prescribing

Physicians’ feedback responses about the training course are shown in Table 4. A majority of participants felt that all aspects of the training course had been successful.

Table 4 Post-intervention feedback from primary care physicians after receiving training on rational drug prescribing

Cost analysis of the intervention strategies

Training intervention

The average monthly salary of a Saudi consultant in 2006 was SR 25 000 (without income tax). Besides developing the course curriculum, 6 consultants delivered lectures to the 61 GPs for 4 days at a cost of SR 19 992 (4 × 833 × 6). They also received SR 1000 per day (SR 500 for non-Saudis) for delivering the lectures. The total spent on consultants was therefore SR 25 992. The average monthly salary of a PHC physician in 2006 was SR 7000, which for 61 GPs attending the courses is SR 56 933 (4 × 233 × 61). Other overheads totalled around SR 5000. Thus, the net amount spent in training 61 PHC physicians was SR 87 925. If the amount spent on consultants was kept constant, the cost of training 100 physicians trained simultaneously by the same number of consultants would be SR 93 333, and the total amount would be SR 124 325, or SR 1243 per physician.

Managerial and regulatory interventions

Once the initial investment in developing guidelines has been incurred, the benefits of administrative interventions are spread around thousands of PHC physicians. The development of each practice guideline was costed at about SR 15 000, distribution of these guidelines to all 5200 health centres in Saudi Arabia was SR 26 000, stationery SR 4000 and overhead expenses SR 2000 was approximately SR 47 000, i.e. approximately SR 3–4 per physician.

Multiple interventions

The cost of the multiple interventions which included all the above costs plus regular follow-up visits by administrative staff was SR 4000 per physician in order to make sure that guidelines and various memoranda were followed by prescribing physicians. This is SR 4000 over and above the cost of training of PHC physicians and administrative and regulatory costs. The approximate total cost of all interventions per physician was SR 1270.

Discussion 

This study analysed 3 interventions concerned with improving the drug prescribing behaviour of PHC physicians: training in rational drug prescribing; administrative measures to control irrational prescribing; and a set of multiple interventions. In general, all the interventions appeared to be an effective means of improving the drug prescribing behaviour of physicians.

The prescribers’ self-perceived knowledge of drug interactions, rational prescribing and use of available guidelines all appeared to be greater in the training intervention site. The evidence to support this was obtained from comparison of the pre- and post-training intervention questionnaires. Each component of the training course was perceived as being beneficial by trainees.

The results obtained in this study can be compared with other evaluations of attempts to improve prescribing behaviours. For example, Francois et al. reported that a programme to improve the quality of prescription writing had only a moderate impact on prescribing in clinical practice and that continuous reinforcement was required if prescribing improvements were to be sustained. Nevertheless, they found that the hospital staff concerned had a positive opinion of the programme, which led to an increased awareness of prescribing issues [16]. Davis et al. found that a problem-based learning programme was received positively by physicians and was effective in improving physicians’ drug prescribing in an asthmatic population, in particular with regard to prescribing of inhaled corticosteroids and leukotriene antagonists [17]. Khoja and Al-Ansari in Saudi Arabia showed that training on national asthma guidelines led to better use of diagnostic tests and tools, better organization of care, and better management of the disorder through patient education and the use of self-management protocols [18]. But the focused nature of the interventions and outcome measured in the studies of Davis et al. and Khoja and Al-Ansari illustrate a key problem: how to demonstrate the sustainability of improvements in practice across a very wide range of disease areas. This would require follow-up studies to see whether such interventions have long-lasting effects in improving the prescribing behaviour of trained physicians.

A variety of intervention programmes have been reported to improve medication prescribing quality in specialized settings and primary care [4,19–26]. Herbert et al. assessed the impacts of individualized feedback on prescribing and interactive small group education aimed at encouraging evidence-based prescribing in PHC practice, in particular for patients with hypertension. They reported that evidence-based educational interventions coupled with personalized prescribing feedback with interactive group discussion could bring about modest but meaningful changes in physician’s prescribing with regard to the preference of thiazides [24]. In a US study, Belongia et al. demonstrated a decreased rate of prescriptions for antibiotics after a large-scale public and physician educational programme [22]. Our study supports these findings as prescriptions for antibiotic decreased substantially at all the intervention sites.

A study in the UK showed poor adherence to validated quality indicators by physicians treating patients with gout and asymptomatic hyperuricaemia [27]. Petrova also reported that national drug policies in the private and public pharmaceutical sectors were not sufficient to guarantee rational drug prescribing [28]. The results of these studies indicate that further research is needed to find more effective mechanisms that could change the scenario of drug prescribing that will benefit all stakeholders. It is also likely that continuing targeted training of PHC physicians in effective drug prescribing is needed because the prescribing scenario is constantly changing with the entry of new drugs into the market. Both of these approaches will result in enhanced patients’ safety, satisfaction, quality of life and good health outcomes [16,19,26,29].

In our study physicians’ skills relating to the notation of all items on prescriptions improved considerably in all the 3 intervention sites when compared with the pre-intervention sites, but only the multiple interventions strategy had a significant impact on recording of the file number notation and the clinical findings. A fully completed prescription is important to ensure the safe and effective dispensing of drugs by pharmacists. In the context of modifying physicians’ prescribing behaviour, health authorities should address the impact of barriers to good prescribing and to change the culture and behaviour of health organizations by minimizing resistance to change and incorporating institutional models, all of which are perceived to be important components of health system reform and quality improvement [30–32].

The cost analysis showed that if the estimated cost of training 100 physicians was generalized to the training of all doctors in prescribing in Saudi Arabia, the cost implications would be immense. For comparison purposes, it has been estimated that about 11% of the UK national health service total budget in 1995 was dedicated to staff training [33], a considerably higher proportion than is currently spent in Saudi Arabia. Our rough estimates were simply meant to be indicative and future cost analysis of such interventions undertaken by other investigators must use a broader and more comprehensive approach to the economic evaluation of such training packages.

To sum up, all 3 interventions tested in this pilot study improved the prescribing behaviour of physicians in Riyadh. Whether or not the improvement in physicians’ rational drug prescribing will be sustained is uncertain, and follow-up evaluations are needed to address this issue. Educationalists need to develop targeted courses in drug prescribing to assist in developing the prescribing skills of physicians. It is important to emphasize that the interventions outlined in this study aimed to improve the quality of prescribing of PHC physicians working in public health care sector in Saudi Arabia and may have limited applicability to physicians working in the private sector.

Acknowledgements

We owe sincere thanks to many physicians, MOH personnel and consultant trainers who collectively supported this project and helped tremendously in conducting medication prescribing training courses. All of the trained physicians deserve special thanks for their cooperation and perseverance in learning how to prescribe medications effectively and rationally to the health consumers. We also express thanks to Professor Nazeer Khan for his statistical expertise.

References

1. Neyaz Y et al. Medication prescribing pattern in primary care in Riyadh city, Saudi Arabia. Eastern Mediterranean Health Journal, 2011,17(2):149–155.
2. Khoja T et al. Medication errors in primary care in Riyadh city, Saudi Arabia. Eastern Mediterranean Health Journal, 2011,17(2):156–159.
3. Arnold SR, Straus SE. Interventions to improve antibiotic prescribing practices in ambulatory care. Cochrane Database of Systematic Reviews, 2005, 4:CD003539.
4. Walley T, Mossialos E. Financial incentives and prescribing. In: Mossialos E, Mrazek M, Walley T, eds. Regulating pharmaceuticals in Europe: striving for efficiency, equity and quality. Milton Keynes, UK, Open University Press, 2004.
5. Bligh J, Walley T. The UK indicative prescribing scheme: background and operation. PharmacoEconomics, 1992, 2:137–152.
6. Mitchell P. US urged to fight “Unacceptable” rate of medication errors. Lancet, 1999, 354:16.
7. Schubert K et al. Development of indicators for assessing the quality of prescribing of lipid-lowering drugs: data from the pharmacotherapeutic quality circles in Hesse, Germany. International Journal of Clinical Pharmacology and Therapeutics, 2001, 39:492–498.
8. Krobot KJ et al. The disparity in access to new medication by type of health insurance: lessons from Germany. Medical Care, 2004, 42:487–491.
9. Puig-Junoy J. Incentives and pharmaceutical reimbursement reforms in Spain. Health Policy, 2004, 67:149–165.
10. Cheng T. The impact of welfare reforms, health, and insurance status on welfare recipients’ health care access. Journal of Health Care for the Poor and Underserved, 2005, 16:588–599.
11. O’Brien MA et al. Continuing education meetings and workshops: effects on professional practice and health care outcomes. Cochrane Database of Systematic Reviews, Issue 3, 2006.
12. Harris CM et al. Prescribing—a case for prolonged treatment. Journal of the Royal College of General Practitioners, 1985, 35:284–287.
13. Chapman S, Durieux P, Walley T. Good prescribing practice. In: Mossialos E, Mrazek M, Walley T, eds. Regulating pharmaceuticals in Europe: striving for efficiency, equity and quality. Milton Keynes, UK, Open University Press, 2004.
14. De Vries T PGM et al. Guide to good prescribing: a practical manual. Geneva, World Health Organization, 1994 (WHO/DAP/94.11).
15. Neville RG et al. A classification of prescription errors. Journal of the Royal College of General Practitioners, 1989, 39:110–112.
16. François P et al. Evaluation of a program to improve the prescription-writing quality in hospital. International Journal of Health Care Quality Assurance Incorporating Leadership in Health Services, 2001, 14:268–274.
17. Davis RS et al. Changing physician prescribing patterns through problem-based learning: an interactive, teleconference case-based education program and review of problem-based learning. Annals of Allergy, Asthma and Immunology, 2004, 93:237–242.
18. Khoja TA, Al-Ansary LA. Asthma in Saudi Arabia: is the system appropriate for optimal primary care? Journal of Public Health Management and Practice, 1998, 4:64–72.
19. Dormuth CR et al. Effect of periodic letters on evidence-based drug therapy on prescribing behaviour: a randomized trial. Canadian Medical Association Journal, 2004, 171:1057–1061.
20. Madridejos-Mora R, Amado-Guirado E, Pérez-Rodríguez MT. Effectiveness of the combination of feedback and educational recommendations for improving drug prescription in general practice. Medical Care, 2004, 42:643–648.
21. Bantar C et al. Alternative indicators for monitoring the quality of a continuous intervention program on antibiotic prescribing during changing healthcare conditions. Journal of Chemotherapy (Florence, Italy), 2005, 17:277–282.
22. Belongia EA et al. Impact of statewide program to promote appropriate antimicrobial drug use. Emerging Infectious Diseases, 2005, 11:912–920.
23. Rausell Rausell VJ et al. Efectividad de una intervencion en la mejora de la calidad de prescripcion con receta medica en atencion especializada [Effectiveness of an intervention to improve medical prescription quality in specialized care]. Farmacia Hospitalaria, 2005, 29:86–94.
24. Herbert CP et al. Better Prescribing Project: a randomized controlled trial of the impact of case-based educational modules and personal prescribing feedback on prescribing for hypertension in primary care. Family Practice, 2004, 21:575–581.
25. Mihill C, Kirkness B. Hard rations: Getting the right treatment for the NHS. London, Association of the British Pharmaceutical Industry, 1999.
26. Walley T, Mrazek M, Mossialos E. Regulating pharmaceutical markets: improving efficiency and controlling costs in the UK. International Journal of Health Planning and Management, 2005, 20:375–398.
27. Mikuls TR et al. Suboptimal physician adherence to quality indicators for the management of gout and asymptomatic hyperuricaemia: results from the UK General Practice Research Database (GPRD). Rheumatology, 2005, 44:1038–1042.
28. Petrova GI. Monitoring of national drug policies–regional comparison between Bulgaria, Romania, Macedonia, Bosnia Herzegovina. Central European Journal of Public Health, 2001, 9:205–213.
29. Shikiar R, Rentz AM. Satisfaction with medication: an overview of conceptual, methodologic, and regulatory issues. Value in Health, 2004, 7:204–215.
30. Marinker M, Reilly P. Rational prescribing: How can it be judged? Controversies in health care policies: challenges to practice. London, British Medical Journal Publishing Group, 1994:89–110.
31. Øvretveit J. A comparison of approaches to health service quality in the UK, USA, and Sweden and of the use of organizational audit frameworks. European Journal of Public Health, 1994, 4:46–54.
32. Wagner C, DeBakker DH, Groenewegen PP. A measuring instrument for evaluation of quality systems. International Journal for Quality in Health Care, 1999, 11:119–130.
33. Azeem Majeed A, Evans N, Head P. What can PACT tell us about prescribing in general practice? British Medical Journal, 1997, 315:1515.