PDF version

A. Tanti,¹ A. Serracino-Inglott ^1,2 and J.J. Borg ^1,3

تصميم استمارة وطنية مشتركة للإبلاغ عن التفاعلات الدوائية الضارة وأخطاء المداواة

ايمي تانتي، أنتوني ساراسيتو إنجلوت، جون جوزيف بورغ

الخلاصة: تم تكليف هيئة الأدوية المالطية بإعداد استمارة إبلاغ تلتقط معلومات عالية الجودة عن حالات التفاعلات الدوائية الضارة وأخطاء المداواة لكي تفي الهيئة بالتزاماتها في مجال الصحة العامة التي حددتها تشريعات الاتحاد الأوروبي بخصوص التيقظ الدوائي. ويقدِّم هذا البحث وصفاً لعملية تقديم أول استمارة إبلاغ مشتركة عن التفاعلات الدوائية الضارة وأخطاء المداواة في الاتحاد الأوروبي للعاملين في مجال الرعاية الصحية، ولتحليل التقارير الناتجة عنها، ولتعزيز النظام الصحي. تم القيام بمراجعة الاستمارات الحالية الخاصة بالتفاعلات الدوائية الضارة وإدراج التوصيات الصادرة عن عمليات التدقيق التي قامت بها وكالة الأدوية الأوروبية ومنظمة الصحة العالمية. ومن ثمّ تم إعداد استمارة جديدة مشتركة للإبلاغ عن التفاعلات الدوائية الضارة وأخطاء المداواة، وتم اختبارها تجريبياً استناداً إلى دراسات حالات. وأعيد تصميم نظام جودة الهيئة (إيزو 9001 المعتمد)، ثم نظمت استراتيجية ترويجية. إن الطريقة التي استخدمت في مالطا يمكن أن تفيد البلدان التي تحتاج إلى إعداد نظم تتعلق بالإبلاغ عن التفاعلات الدوائية الضارة وأخطاء المداواة، وإلى تحسين جودة التقاط البيانات داخل نظمها.

ABSTRACT The Maltese Medicines Authority was tasked with developing a reporting form that captures high-quality case information on adverse drug reactions (ADRs) and medication errors in order to fulfil its public-health obligations set by the European Union (EU) legislation on pharmacovigilance. This paper describes the process of introducing the first combined ADR/medication error reporting form in the EU for health-care professionals, the analysis of reports generated by it and the promotion of the system. A review of existing ADR forms was carried out and recommendations from the European Medicines Agency and World Health Organization audits integrated. A new, combined ADR/medication error reporting form was developed and pilot tested based on case studies. The Authority’s quality system (ISO 9001 certified) was redesigned and a promotion strategy was deployed. The process used in Malta can be useful for countries that need to develop systems relative to ADR/medication error reporting and to improve the quality of data capture within their systems.

Élaboration d'un formulaire de notification national combinant les réactions indésirables aux médicaments et les erreurs de médication

RÉSUMÉ L'Autorité maltaise pour les médicaments (Maltese Medicines Authority) a été chargée d'élaborer un formulaire de notification visant à recueillir des informations de haute qualité sur les réactions indésirables aux médicaments et les erreurs de médication afin de satisfaire à ses obligations de santé publique définies par la législation de l'Union européenne sur la pharmacovigilance. Le présent article décrit le processus d'introduction du premier formulaire de notification combinant les réactions indésirables aux médicaments et les erreurs de médication pour les professionnels des soins de santé dans l'Union européenne, l'analyse des rapports issus de ce processus et la promotion de ce système. Les formulaires de notification des réactions indésirables aux médicaments existants ont été passés en revue et les recommandations de l'Agence européenne des médicaments ainsi que des audits de l'Organisation mondiale de la Santé ont été intégrés. Un nouveau formulaire de notification combinant les réactions indésirables aux médicaments et les erreurs de médication a été élaboré puis soumis à un essai pilote dans des études de cas. Le système qualité de l'Autorité maltaise (certifié ISO 9001) a été remanié et une stratégie de promotion a été déployée. Le processus appliqué à Malte peut être utile à d'autres pays qui ont besoin de mettre au point des systèmes de notification des réactions indésirables aux médicaments et des erreurs de médication et souhaitent améliorer la qualité du` recueil des données au sein de leurs systèmes.

¹Medicines Authority, Gzira, Malta (Correspondence to J. Borg: This e-mail address is being protected from spambots. You need JavaScript enabled to view it ). ²Department of Pharmacy, University of Malta, Msida, Malta. ³Department of Biology, School of Pharmacy, University of Tor Vergata, Rome, Italy.

Received: 03/06/14; accepted: 28/01/15

---

Introduction

In 2013 there were major changes to the regulation of human medicines in member states of the European Union (EU), particularly with implementation of Directive 2010/84/EU of the European Parliament and the Council of the European Union concerning pharmaco­vigilance (1,2). Member states were faced with the task of transposing this directive into national legislation. The challenges were substantial, especially for small member states with limited resources, such as Malta. The new obligation to report adverse drug reactions (ADRs) related to medication errors also required a new systematic approach to how we gathered data related to ADRs and medication errors in Malta.

Prior to 2012 the Malta Medicines Authority had adopted the World Health Organization (WHO) definition of an adverse drug reaction (ADR), since the Authority, as the national competent authority, reports to the WHO Global Monitoring System in Uppsala (WHO-UMC). The WHO’s definition for an ADR is “a response to a drug that is noxious and unintended and occurs at doses normally used in man for the prophylaxis, diagnosis or therapy of disease, or for modification of physiological function” (3). This definition focuses on harm incurred at doses of medicinal products normally used for treatment or prevention of disease, i.e. on ADRs related to the inherent properties of the drug or excipients at the licensed therapeutic dose for a specific indication in a specific population. Therefore, if a drug is taken at double the dose licensed for treatment of a disease and an ADR is experienced due to an overdose of this drug, i.e. a medication error is made, this scenario is not specifically covered by this definition. The new EU definition was wider and allowed for aspects of the use of a medicinal product to be taken into account. The new legislation defined an ADR as: “a response to a medicinal product which is noxious and unintended”. Therefore, other causes of ADR such as medication errors, abuse, misuse and occupational exposure are also covered by this definition (the term medication errors is henceforth used in this paper to cover the terms abuse, misuse and occupational exposure).

Member states and marketing authorization holders in the EU have mandatory obligations to report ADRs (and now also ADRs related to medication errors) in the pre- and post-authorization phase of drug development to the EudraVigilance Clinical Trial Module and EudraVigilance Post-Authorization Module. EurdaVigilance is a tool for the European Medicines Agency and national medicines authorities to use for monitoring the safety of medicines and in minimizing potential risks related to suspected ADRs. In order to facilitate ADR reporting, EU member states also have national databases to handle such reports. The new EU legislation required that ADRs related to medication errors should be transmitted to EudraVigilance but did not mandate collection of information on medication errors which do not lead to ADRs.

After internal consultation and a literature search, the Malta Medicines Authority took the decision also to allow for the collection of medication error reports that are not associated with an ADR. The decision was limited to data collection solely on medication errors related to medicinal products as this is the mandate conferred to the Malta Medicines Authority through the Malta Medicines Act (3,4). The aim was therefore to devise a single form that captured (i) ADRs, (ii) ADRs related to medication errors and (iii) medication errors related to medicinal products not associated with an ADR, to the highest quality of documentation where the basic elements of a case report are fulfilled. We wanted to provide a robust pharmacovigilance system for the process of developing a national ADR and medication error report form for Malta.

In this paper, we describe how a combined ADR and medication error reporting form was developed for Malta to enable the capture of the best quality information. The motivation for this was not only to respond to the new widened definition of ADRs as set out in the new EU pharmacovigilance legislation but also to improve ADR case data collection in Malta (1).

Methods

Background

At the time of the legislative update, national competent authorities who were collaborating with the WHO-UMC received a report which gave a detailed account of the quality of data each country was submitting to WHO-UMC. Specific parameters were used to measure quality and a score was given to participating countries as well as a number of recommendations on how the score could be improved. The European Medicines Agency also issued a report on the quality of country-specific data in EudraVigilance and its associated data warehouse.

The recommendations given by the WHO and the European Medicines Agency on how to improve the quality of data submitted to VigiBase™ (the WHO global ICSR database) for ADRs and EudraVigilance for ADRs/medication errors respectively were included in the new design of the combined form.

As part of the process to meet the important objective that the quality of data about ADRs was improved as well as to reduce misinterpretations, we also compiled an instruction sheet with step-by-step explanations on how to fill in each section of the new redesigned form. We adopted the National Coordinating Council for Medication Error Reporting and Prevention (NCC MERP) definition for medication errors since this definition is clear and understandable and also an internationally recognized standard already in use in many countries. The NCC MERP definition is “any preventable event that may cause or lead to inappropriate medication use or patient harm while the medication is in control of the health-care professional, patient or consumer”. A simplified definition for patients was then construed. This new reporting form is an improvement on the Maltese national ADR form which the Malta Medicines Authority issued in 2004. The final form and instruction sheet are available at http://www.medicinesauthority.gov.mt/adrportal.

We used a stepwise approach to design the ADR and medication error reporting form before testing the new form.

Step 1: Review of current ADR form and recommendations for improvement of quality reporting

Different ADR forms were received for review by the Malta Medicines Authority, including:

directly through EudraVigilance Post Authorization Module from marketing authorization holders reporting on individual case safety reports (ICSRs);

as paper copy forms from health-care professionals and patients in several formats (e.g. the largest teaching hospital has three different types, immunization clinics have their own vaccine form and the Council for International Organisations of Medical Sciences (CIOMs) form (http://cioms.ch/index.php/cioms-form-i) is used by some community health-care professionals and patients);

the United Kingdom’s Medicine and Healthcare products Regulatory Agency ADR “yellow” form as well as Ireland’s ADR form.

We checked the strengths and weaknesses of the paper-based ADR forms that local reporters were using and then compared these with the Malta Medicines Authority’s ADR form (form F-004v2, used between the years 2004 and 2013).

Step 2: Design of a combined form for reporting on ADRs and medication errors

Agreeing on data to be collected

Due to resource limitations we looked at what other countries were doing in their collection of data related to medication errors (5–8). We also searched the literature on medication error reporting systems to obtain information on what type of data should be collected to enable us to obtain a better quality report that could support regulatory action if required (9–12). The goal was to capture useful, practical data, not only regarding errors themselves but also regarding methods used to reduce the incidence of errors. At the same time, we kept all users in mind and tried to be clear and concise in our data questions.

Since we were combining medication error reporting with ADR reporting, the form for ADR and the form for medication error reporting were merged. The new additional fields that were found to be required for the medication error form are shown in Table 1.

Developing a system for analysing and recording medication errors

Coding: To ensure standardization of data entry we used a list of predetermined error categories. This list was adopted from previous work that our research group had carried out (11).

Classification of error-related harm: Since the medication error form is part of an ADR form, the reported degree of harm to patients can be derived from the seriousness of the reported ADRs (if any). The classification adopted is shown on Table 2.

Root cause analysis

It was decided that structured root cause analysis (RCA) using published methods (10) will be conducted for those incidents classified as moderate to severe harm errors. For low or no harm errors occurring frequently (more than 3 times) RCAs will also be conducted.

No-blame approach

To foster a “no-blame” culture we added a statement informing users that the submission of a report does not mean an admission of causality. Furthermore, the new ADR form has perforations on section 3 so that reporter details can be deleted following transmission into the European database of ADRs. It is important to note that we did not set out here to study health-care professionals’ perceptions of no-blame culture on ADR reporting. However, we are aware that this issue is very important; representatives of the Malta Medicines Authority have been called to present and clarify paper ADR forms received during litigations at the Malta Medicines Council. We consider that the measures in place to inform reporters that a filled ADR form does not mean an admission of causality and that reporter details can be removed are adequate to foster a no-blame culture.

Recording medication errors

A Microsoft® Access database was developed to capture the new data elements related to medication errors that did not lead to an ADR. In other cases where the error was associated with an ADR then the EudraVigilance/Vigibase™ databases acted as a repository.

Step 3: Piloting and validation of the new combined report form for ADRs and medication errors

We wanted to test the form and obtain feedback on any difficulties and complexities experienced by users, the time need to complete it and ways to improve the form. Four cases containing a mix of ADR and medication errors were compiled together with a feedback form containing 7 open-ended questions. A summary of the cases is as follows:

Case 1: medication error and ADR; confusion between Inderal® and Aderall® causing hypertension and headache in the patient.

Case 2: medication error and ADR; decrease in the absorption of aspirin due to co-administration of an antacid.

Case 3: ADR on a reaction suspected to be related to a new treatment.

Case 4: Several ADRs reported by an investigator of a clinical trial.

Cases 1 and 2 were written from a patient’s perspective, the other 2 cases studies were from a health-care professional angle. One of the 4 test cases with the feedback form and the report form were randomly distributed to 17 Malta Medicines Authority personnel including 5 participants who had no scientific background. The use of internal personnel as a first step in the process to design a new ADR form was considered an effective use of a readily available resource. However, a concern could be raised that this was a biased group as the personnel work for the Malta Medicines Authority. However, in an attempt to minimize bias, none of the personnel who took part in the study actually worked on any pharmacovigilance related activity or handled ADRs. The practice of using internal staff from other units is acceptable and not so uncommon as it is in line with the ISO 9001 quality management standard when carrying out internal audits. All cases and forms were collected within 3 weeks of initial distribution. Following comments received, the reporting form was amended and the form was recirculated to assess whether comments were resolved.

As a final stage in the process to design the new form, we submitted the form to the following stakeholders for feedback: the in-pharmacy department of Malta’s General hospital and Malta’s Primary Health Care Directorate. We received comments from the principal pharmacist and the head of the primary health care department that they found the form to be acceptable for use.

Step 4: Revision and update of SOP for management of ADR reporting

Due to the updates required in the ADR form, the standard operating procedure (SOP) for the management of ADR reporting called for a revision. The SOP was updated to come into line with the Malta Medicines Authority’s ISO 9001 certified quality system and all personnel were retrained.

Step 5: Strategy for introducing the combined reporting form for ADRs & medication errors

A strategy to promote the new form was adopted by the management of the Malta Medicines Authority. With an implementation plan of 1–2 years the main pillars to this strategy were as follows:

Updating the online ADR reporting form for health-care professionals with the new combined form for ADR and medication error.

Adding an additional and dedicated patient report form.

Changing the location of the new combined form to the front page of the website.

Updating online information on ADR reporting, including teaching resources on the new form, a slide presentation, and background articles.

Promoting the new combined form in direct-to-health-care professional education events, conferences and seminars.

Updating the Malta Medicines Authority guidelines on pharmacovigilance for health-care professionals and pharmaceutical industry professionals.

Results

Step 1: Review of current ADR form & recommendations for improvement of reporting

Through the audit recommendations of WHO-UMC and the European Medicines Agency, and the analysis of forms received at the Malta Medicines Authority, it was found that 6 out of the 12 data elements considered to be essential were not conducive to high-quality data collection. Areas for improvement identified were within the following fields: drug start date, ADR date of onset, frequency of dose, route of administration, outcome and seriousness. These fields needed to be improved to facilitate the conduct of causality assessment internally.

Step 2: Design of a combined form for reporting on ADRs and medication errors

The 6 fields identified in step 1 were amended as follows. The drug start date and ADR onset date were improved by creating fields to show that all 3 values (day, month and year) were essential in establishing temporality and in assessing causality between drug and reaction. We did this by creating 3 specific sections for data entry instead of an open box. Through visual depiction it is easy to perceive that the 3 parameters of the date are necessary for inclusion. The suspected medicine dose field was improved with the addition of frequency and route of administration.

The previous form did not enable the reporter to distinguish between ADRs that the patient had recovered from and ADRs which were unresolved. Therefore, for the section “Outcome from ADR” 2 additional rows were added so the reporter could express multiple ADR outcomes experienced in a patient. The same was applied for the “Seriousness” field, so a mix of serious and non-serious ADRs could be reported in any one case.

Step 3: Piloting and validation of the new combined report form for ADRs and medication errors

Of the 17 participants taking part in the pilot validation, 3 participants had secondary education, 14 had university education, 10 were pharmacists and 2 were physicians. Five did not have a scientific background.

Comments received/actions taken are described in Table 3. The average time to complete the form was 15 minutes. Two comments received (from respondents #10 and #12) were not addressed. Respondent #10 suggested that there should be a section specifically for over-the-counter medicine reports, while respondent #12 suggested that instructions should be divided into 2 parts, 1 part for the ADR form and 1 for medication error form, and placed before each section. Due to space and design limitations these comments were not taken on board.

Five comments were about problems with choosing the appropriate sections when completing reports. The major difficulty was that it was not easy for the reporter to know which sections had to be completed in case of a medical-error-only report, a medical-error-plus-ADR report or an ADR-only report. In order to address this we introduced a decision tree (see questions at the top of the new form), to clarify which sections should be filled in for different types of reports. A footnote on each page was also added to this effect.

Five comments were also received about the instructions page, where respondents did not realize that instructions were present since these were at the back of the form. This problem was resolved at the design stage due to the foldable nature of the form, whereby the back end became the front for the reporter.

Regarding the terminology and format used, 5 participants flagged the use of too many technical terms, that the form looked complicated and that the definition of an ADR should be substantiated with examples. We decided to add examples in section 1.3, and to simplify the terminology used as well as to include explanations/additional information in brackets as much as possible. The non-health-care professional subset (5 respondents) identified problems in assessing severity and reporting additional information requested in section 1.7. We addressed this by simplifying the terms used in these sections and arranging the format to include tick-boxes rather than a table format in previous versions.

The new Malta national ADR/medication error reporting form is presented in Figure 1 (also available at: http://www.medicinesauthority.gov.mt/reportingadversereactions?l=1).

Step 4: Revision and update of SOP for the management of ADR reporting

The new SOP was written to integrate 3 processes: receipt of report from health-care professional where an ADR is involved; receipt of an ICSR from a marketing authorization holder in EudraVigilance and; receipt of a medication error without an ADR.

Validation criteria for accepting medication error reports were decided upon. For a medication error report to be valid it must be related to a medicinal product and have a description of the event. Those reports which do not fulfil the criteria are channelled to other public service health-care entities (for medication errors not related to medicinal products) or further information is requested from reporter. Operators were trained and the process internally audited.

Step 5: Strategy for introducing the combined reporting form for ADRs and medication errors

Promoting spontaneous reporting of ADRs/medication errors was the main target of our implementation strategy, with multiple interventions implemented simultaneously, since this strategy would have more impact than single major interventions (13). A new website was released in March 2013, including a dedicated reporting portal, and specific pages were introduced on ADR reporting aimed at different users. A training presentation to improve ADR reporting is available online (14). The form was promoted at a seminar at the Maltese Ministry of Health in 2013. Attendees were primary health-care physicians who received an information pack containing the ADR form and cover letter. Teaching events on ADR reporting aimed at paediatricians were held in October 2013.

Over 1000 forms were distributed in a campaign targeting community pharmacists. All pharmacies received a package containing several forms and a letter encouraging ADR and medication error reporting. Leaflets for a “Know your medicine” campaign for patients were distributed. The leaflets included a section explaining the importance of ADR reporting and the reporting modalities available, with specific links to the website where the ADR form was available for download and printing. Approximately 55 000 leaflets were provided to pharmacists to distribute to patients.

National guidelines on pharmacovigilance have been updated to include information on the new form. To verify if the strategy adopted affects the ICSR reporting rate we will evaluate ICSRs reported over a 5-year period. In 2012, 255 reports were received, a per capita reporting rate of 615 reports per 1 million inhabitants per year. Increases in the ADR reporting over time will be taken as a measure of effectiveness of this promotion strategy.

Discussion

In the initial stages of this project, we weighed the positive and negative aspects of having a combined ADR/medication error form against having separate forms. Separate forms would have eliminated the problems we encountered in validation, whereby reporters found it hard to decide which sections to complete. On the other hand, having a combined form facilitated promotion of the strategy since a demand for the ADR form was already in existence. In addition, where both ADRs and medication errors have occurred, having a combined form facilitates reporting of both events. Furthermore, it is useful for regulators to have the link between a medication error and ADR done by the reporter for causality assessment purposes.

The development of a combined form has been successful. However, the reporting of ADRs and medication errors needs continuous promotion of a no-blame culture among health-care professionals. Research into ADR reporting has shown that those who undergo training are more likely to report ADRs and that continued educational initiatives are needed to sustain a successful ADR and medication error reporting programme (15–18). Information sessions to medical, pharmacy and nursing students are planned as part of our promotion strategy to achieve such targets.

Studies have shown that to improve the engagement of health-care professionals in the process of reporting, and for them to repeatedly submit ADR reports, reporters need to receive information and feedback that is relevant and useful to them (15–19). Some studies have reported that health-care professionals complain that ADR reporting systems are like a “black hole” in which information goes into a bureaucratic system and then disappears (15,20). When engineering our SOP we introduced a feedback and acknowledgment mechanism to give feedback to the reporter to minimize such concerns. Assessors therefore can provide feedback on causality of the ADR. Product information and risk management measures can also be provided to health-care professionals. Desai et al. reported that a lack of information on where/how to report and lack of access to reporting forms were reasons for not reporting ADRs at a tertiary care hospital (17). Furthermore, preferred methods for reporting are email and personal communications. In the Maltese national ADR reporting system reports can be made online by both health-care professionals and patients.

In conclusion, external audits carried out by the European Medicines Agency on the quality ICSRs reported by the Malta Medicines Authority to EudraVigilance (in 2010 and in 2014) show that the new form captures better quality data and is a confirmed improvement over the previous national ADR form. Furthermore, the new form captures information on medication errors, which was not possible with the previous form. The new ADR/medication error reporting form can therefore be judged to be a definite improvement over the old national ADR form as these data were simply not captured before. Furthermore, 2 internal audits have been carried on ADR reporting within the Malta Medicines Authority between 2012 and 2014, specifically focused on confirming that the ICSR reporting system fulfils European Commission implementing regulation no. 520/2012 on pharmacovigilance systems. Finally, in order to help facilitate direct patient reporting a simplified form is being designed for direct online reporting.

Acknowledgements

The authors wish to thank Dr Herbert Lenicker for his help in promoting the new form and Gavril Flores and Josette Sciberras for their helpful suggestions in the making of this form.

Disclaimer: The views expressed in this article are the personal views of the authors and may not be used or quoted as being made on behalf of, or reflecting the position of, any national competent authority or any University.

Funding: None.

Competing interests: None declared.

References

1. Directive 2001/83/EC of the European Parliament and of the Council of 6 November 2001 on the Community code relating to medicinal products for human use. Official Journal L. 2004; 311:67–128 (as amended by Directive 2010/84/EU, consolidated version 2012).
2. Borg JJ, Aislaitner G, Pirozynski M, Mifsud S. Strengthening and rationalizing pharmacovigilance in the EU: where is Europe heading to? A review of the new EU legislation on pharmacovigilance. Drug Saf. 2011 Mar 1;34(3):187-97. PMID: 21332243
3. Legal notice 61 of 2006 of the Malta Government; Pharmacovigilance Regulations. The Malta Government Gazette. 2006;231:2981.
4. Reporting a medication or vaccine error or hazard to ISMP [Internet]. Horsham (PA): Institute for Safe Medication Practices (https://www.ismp.org/orderforms/reporterrortoismp.asp, accessed 19 February 2015).
5. Report medication errors [Internet]. Rockville (MD): National Coordinating Council for Medication Error Reporting and Prevention (http://www.nccmerp.org/report-medication-errors, accessed 19 February 2015).
6. Medication error (ME) report form [Internet]. Kuala Lumpur: Pharmaceutical Services Divisions of the Ministry of Health Malaysia (http://www.pharmacy.gov.my/v2/en/documents/medication-error-me-report-form.html, accessed 19 February 2015).
7. Report a patient safety incident [Internet]. London: National Patient Safety Agency of the National Health Service (http://www.nrls.npsa.nhs.uk/report-a-patient-safety-incident/, accessed 19 February 2015).
8. U David. Medication error reporting systems: problems and solutions. New Medicine. 2001 November;1(2):61–5 (http://www.ismp-canada.org/download/Medication%20Error%20Reporting%20Systems%20-%20Problems%20and%20Solutions.pdf, accessed 19 February 2015).
9. Terzibanjana AR, Laaksonenb R, Weissb M, Airaksinena M, Wulijic T. Medication error reporting systems—lessons learnt. Executive summary of study of findings [Internet]. The Hague, The Netherlands: International Pharmaceutical Federation (http://www.fip.org/files/fip/Patient%20Safety/Medication%20Error%20Reporting%20-%20Lessons%20Learnt2008.pdf, accessed 19 February 2015).
10. Montesi G, Lechi A. Prevention of medication errors: detection and audit. Br J Clin Pharmacol. 2009 Jun;67(6):651–5. PMID:19594533
11. Tanti A, Camilleri M, Bonanno PV, Borg JJ Medication errors through a national pharmacovigilance database approach: a study for Malta. Int J Risk Saf Med. 2013;25(1):17–27. PMID:23442294
12. Anderson B, Fagerhaug T. Root cause analysis: simplified tools and techniques. 2nd ed. Milwaukee (WI): ASQ Quality Press; 2006.
13. Gonzalez-Gonzalez C, Lopez-Gonzalez E, Herdeiro MT, Figueiras A. Strategies to improve adverse drug reaction reporting: a critical and systematic review. Drug Saf. 2013 May;36(5):317–28. PMID:23640659
14. How to report adverse drug reactions (ADRs) [Internet]. Gzira, Malta: Medicines Authority (http://www.medicinesauthority.gov.mt/adrportal?l=1, accessed 19 February 2015).
15. Zolezzi M, Parsotam N. Adverse drug reaction reporting in New Zealand: implications for pharmacists. Ther Clin Risk Manag. 2005 Sep;1(3):181–8. PMID:18360558
16. Sweis D, Wong IC. A survey on factors that could affect adverse drug reaction reporting according to hospital pharmacists in Great Britain. Drug Saf. 2000 Aug;23(2):165–72. PMID:10945377
17. Desai CK, Iyer G, Panchal J, Shah S, Dikshit RK. An evaluation of knowledge, attitude, and practice of adverse drug reaction reporting among prescribers at a tertiary care hospital. Perspect Clin Res. 2011 Oct;2(4):129–36. PMID:22145123
18. Su C, Ji H, Su Y. Hospital pharmacists’ knowledge and opinions regarding adverse drug reaction reporting in Northern China. Pharmacoepidemiol Drug Saf. 2010 Mar;19(3):217–22. PMID:20033911
19. Biagi C, Montanaro N, Buccellato E, Roberto G, Vaccheri A, Motola D. Underreporting in pharmacovigilance: an intervention for Italian GPs (Emilia-Romagna region). Eur J Clin Pharmacol. 2013 Feb;69(2):237–44. PMID:22706618
20. Edwards IR. Spontaneous reporting–of what? Clinical concerns about drugs. Br J Clin Pharmacol. 1999 Aug;48(2):138–41. PMID:10417488