PDF version

Dalia Samhouri ^1,5, Kashef Ijaz ², Arash Rashidian ³, Stella Chungong ⁴, Antoine Flahault ⁵, Suzanne M. Babich ⁶ and Jaouad Mahjour ⁷

¹WHO Health Emergency Programme (WHE), WHO Regional Office for the Eastern Mediterranean, Cairo, Egypt. ²Centers for Disease Control and Prevention, Atlanta, United States of America. ³Division of Information, Evidence & Research, WHO Regional Office for the Eastern Mediterranean, Cairo, Egypt. 4WHO Health Emergency Programme (WHE), WHO headquarters, Geneva, Switzerland. ⁵Institute of Global Health, Faculty of Medicine, University of Geneva, Switzerland. ⁶Department of Health Policy and Management at Indiana University, Richard M. Fairbanks School of Public Health, Indianapolis, United States of America. ⁷Programme Management, WHO Regional Office for the Eastern Mediterranean, Cairo, Egypt (Correspondence to: Dalia Samhouri: This e-mail address is being protected from spambots. You need JavaScript enabled to view it ).

Abstract

Background: Joint External Evaluation (JEE) was developed as a new model of peer-to-peer expert external evaluations of IHR capacities using standardized approaches.

Aims: This study aimed to consolidate findings of these assessments in the Eastern Mediterranean Region and assess their significance.

Methods: Analysis of the data were conducted for 14 countries completing JEE in the Region. Mean JEE score for each of the 19 technical areas and for the overall technical areas were calculated. Bivariate and multivariate analyses were done to assess correlations with key health, socio-economic and health system indicators.

Results: Mean JEE scores varied substantially across technical areas. The cumulative mean JEE (mean of indicator scores related to that technical area) was 3 (range: 1–4). Antimicrobial resistance, Biosecurity and Biosafety indicators obtained the lowest scores. Medical countermeasures, personnel deployment and linking public health with security capacities had the highest cumulative mean score of 4 (range: 2–5). JEE scores correlated with most of the key indicators examined. Countries with better health financing system, health service coverage and health status generally had higher JEE scores. Adolescent fertility rate, neonatal mortality ratio and net primary school enrollment ratio were primary factors within a country's overall JEE score.

Conclusions: An integrated multisectoral approach, including well-planned cross-cutting health financing system and coverage, are critical to address the key gaps identified by JEEs in order to ensure regional and global health security.

Keywords: Eastern Mediterranean Region, joint external evaluation, communicable diseases, international health regulations, health finance.

Citation: Samhouri D; Ijaz K; Rashidian A; Chungong S; Flahault A; Babich SM; et al. Analysis of Joint External Evaluations in the WHO Eastern Mediterranean Region. East Mediterr Health J. 2018;24(5):477–487. https://doi.org/10.26719/2018.24.5.477

Received: 10/06/18; accepted: 03/07/18

Copyright © World Health Organization (WHO) 2018. Some rights reserved. This work is available under the CC BY-NC-SA 3.0 IGO license (https://creativecommons.org/licenses/by-nc-sa/3.0/igo).

---

Introduction

The revised International Health Regulations (2005) (IHR 2005) requires Member States to develop and maintain core public health capacities for surveillance and response at points of entry, including early detection, assessment, notification and reporting to WHO about events covered by their provisions, laid out in Annex 1 A and B of IHR (1). The Ebola epidemic in West Africa of 2014–2015 demonstrated that the world was ill-prepared to detect, prevent and respond to emerging infectious disease outbreaks (2–6). It also demonstrated that IHR (2005) mandated self-reporting by countries may not be truly reflective of the country’s public health capacity to prevent, detect and respond to major public health threats (7,8). The 2016 Zika virus outbreak once again put the IHR (2005) capacities under scrutiny, highlighting the importance of their implementation.

The IHR Review Committee on Second Extensions for Establishing National Public Health Capacities and on IHR Implementation convened in 2014. It recommended that the Director General consider a variety of approaches for the shorter- and longer-term assessment and development of IHR core capacities and the [WHO] Secretariat should develop options to move from exclusive self-evaluation to approaches that combine self-evaluation, peer review and voluntary external evaluations (9–11). To address this recommendation, WHO developed the IHR Monitoring and Evaluation Framework (IHRMEF) comprising of four components, notably the mandatory Annual Reporting, and the voluntary joint external evaluation (JEE), simulation exercises and After Action Reviews (AAR) (12–14).

The JEE was developed as a new model of peer-to-peer expert external evaluations of IHR capacities, carried out by a multidisciplinary external team of experts jointly with a multi-sectoral team of national experts, using a standardized score-based indicator data collection instrument (JEE Tool) (15–17).

As of July 2018, 78 countries, including 14 countries in the WHO Eastern Mediterranean Region (EMR), had carried out JEEs (18). The experience of in-country focal points during JEEs in these countries has been described (19). However, this paper provides a detailed descriptive analysis of outcomes of JEEs in the first 14 countries completing JEEs in the EMR. It also analyses groups of objectively selected demographic, socio-economic, mortality, morbidity, health financing, health workforce, service delivery, service provision and political stability indicators in these countries, and their correlation with JEE scores to assess for potential predictors. It provides suggested actions that countries, WHO and the international community could take to increase their effectiveness in increasing JEE scores to meet IHR obligations and ensuring global health security.

Methods

The study is based on analyses of data collected through the JEE processes for the first 14 EMR countries completing JEEs (Afghanistan, Bahrain, Jordan, Kuwait, Lebanon, Morocco, Oman, Pakistan, Qatar, Saudi Arabia, Somalia, Sudan, Tunisia and United Arab Emirates) between April 2016 and December 2017, as well as other key indicators, as described below.

JEE tool

The JEE tool consists of 19 technical areas organized by four main groups (i.e., prevent, detect, respond and points of entry (PoE) and IHR-related hazards). The 19 technical areas consist of 48 indicators1 that are measured by incremental 5-step definitive scoring criteria. The score for each indicator ranges on a Likert scale from 1 to 5 as follows: 1) no capacity; 2) limited capacity; 3) developed capacity; 4) demonstrated capacity; and 5) sustainable capacity. JEE scores of 3 and above were defined as high JEE scores and below 3 as low JEE scores. Only integer scores for indicators and technical areas are allowed (20).

A standardized JEE process was followed in the 14 EMR countries (18). The JEE tool was applied through in-country missions (external evaluation phase) to validate the information collected through the self-evaluation phase and background documents. Field visits to settings such as hospitals, primary health care centres, public health laboratories, veterinary laboratories, poison centres, emergency operating centres, airports, ports and ground crossings were also conducted when feasible and varied by country (21).

Selection of health system indicators

The EMR consists of 22 countries with an estimated population of 644 million (8.6% of global population in 2017) (22). The WHO Regional Office for the Eastern Mediterranean (WHO/EMRO) has developed a clear framework for health systems with 68 core indicators that focus on three main components: 1) monitoring health determinants and risks; 2) assessing health status, including morbidity and cause-specific mortality; and 3) assessing health system response were developed. The EMR Member States have been annually reporting on these indicators since 2014 (23,24).

Of these 68 core indicators, 32 indicators were selected to assess the correlation with the JEE indicators, as possible predictors of JEE scores. A descriptive overview of these 32 demographics, socioeconomic status, mortality, morbidity, health finance, health workforce, service delivery, service coverage and political stability indicators for the 14 EMR countries completing JEEs shows the wide range of countries in terms of population, resources and political challenges facing these Member States in the Region (Appendix 1). The remaining 36 indicators were excluded either due to incomplete data, (e.g. population with catastrophic expenditure) or were not directly related to any of the 19 technical areas of JEE tool (e.g. physical activity). Additionally, based on published data showing the impact of political stability on health systems strengthening (25–27), two political stability indicators were selected for inclusion from the WHO’s list of Global indicators and from the World Bank development indicators (28,29).

Statistical analysis

For each of the 19 technical areas, the mean JEE score was calculated for the 14 countries based on mean score of indicators related to that technical area. An overall JEE score was also calculated based on mean JEE scores across all 19 technical areas. We used the existing categorization of the countries of the Region into three groups (Group 1, Group 2 and Group 3)2 for comparative purposes (30). Only integer scores were allowed both for cumulative score and overall JEE score. To assess the distribution of scores across technical areas, measures of centrality (mean, median, and range) were calculated. Correlational analyses were conducted in pairwise comparisons to obtain Spearman rank correlation coefficient between JEE major groups, between JEE indicator on coordination and the response related indicators, and between overall JEE mean score and the selected health system core indicators. Logistic regression analysis was conducted to find potential associations between the overall JEE mean score and the key indicators. A stepwise selection process was followed using the 32 indicators with the overall JEE score until all remaining explanatory variables in the model showed statistically significant (P < 0.05) associations with the outcome variable.

Results

The overall mean JEE score across 19 technical areas in the 14 EMR countries was 3 (median 3, range: 1–4). The mean JEE score for the four main groups of technical areas was 3 (Median: 3, Range: 1–5). In reviewing the range, Prevent and Points of entry and other IHR related hazards had the lowest minimum score of 1; while respond and Points of entry and other IHR related hazards had the highest maximum score of 5. The overall mean JEE scores and mean JEE scores for the four main groups analysed by the three country groups showed that countries in Group 1 had an overall mean score of 4 (Median:, Range: 3–5). The countries in Group 2 had an overall mean score of 3 (Median:3, Range: 2–3), while the countries in Group 3 had an overall mean score of 2 (Median:2, Range: 1–3).

Examining each of 19 technical areas in the 14 EMR countries demonstrated that the cumulative mean score of IHR implementation varied across the 19 technical areas. The cumulative mean JEE score was 3 (median: 3; range: 2–4). Antimicrobial resistance (AMR) and biosecurity and biosafety had the lowest mean score of 2- limited capacity. Medical countermeasures and personnel deployment, and linking public health with security, had the highest mean scores of 4- demonstrated capacity (Table 1). The mean score for the majority of the indicators (31) was 3, while the mean score of 6 indicators was 2 and for 11 indicators was 4. Five of the 6 indicators with the lowest mean score of 2 related to AMR and biosafety biosecurity, while one related to workforce strategy specifically (Table 2).

Using Spearman rank correlation coefficients, countries’ JEE scores for prevent, detect and respond were correlated (P < 0.01). In addition, correlation was found between PoE and other IHR hazards and prevent, detect and respond. The JEE coordination indicator (defined Functional mechanism is established for the coordination and integration of relevant sectors in the implementation of IHR) and other response related indicators for zoonosis, food safety, chemical and radiation and emergency response operations were also found correlated (P < 0.01). The overall JEE score was correlated with the listed demographic and socioeconomic determinants and health risks set of variables except for total population, annual growth rate and adolescent fertility rate. Among health status indicators, the overall JEE score correlated with life expectancy rate and inversely correlated with all mortality indicators except with the communicable diseases mortality. No correlation was found with the morbidity indicators except with incidence rate of Hepatitis B. For health system response indicators, it also strongly correlated with all health finance indicators, except with general government expenditure on health as percentage of general government expenditure. The overall JEE score was also found inversely correlated with listed political instability indicators (Table 3).

Regression analysis showed that adolescent fertility rate, neonatal mortality ratio and net primary school enrollment ratio influenced the odds of a country scoring high on JEE. Countries with adolescent fertility rate of above 20; on average, received higher JEE scores. Holding all other variables constant, compared with countries with neonatal mortality ratio ≤ 8, countries with neonatal mortality ratio of > 8 were 4.65 times (OR 4.65, 95% CI: 1.42–15.19; P = 0.01) more likely to receive a lower JEE score. On average, net primary school enrollment above 80 increased the odds of countries receiving high JEE scores (OR 19.54, 95% CI: 5.24–72.82; P < 0.01) (Table 4).

Discussion

Countries in EMR seem to be doing well in technical areas such as immunizations, indicator based surveillance, diagnostics for priority pathogens, referral of laboratory samples, multisectoral response to public health emergencies and medical countermeasures. However, common gaps and recommendations identified by the countries during JEEs suggest that innovative ways and efforts need to be identified and enhanced to improve capacities such as antimicrobial resistance (AMR), biosafety and biosecurity, surveillance data analysis and interpretation, enhancement of laboratory quality management system, risk communication, and public health preparedness to all hazards, including at points of entry. Additionally, in the majority of countries, the human and animal sectors are not at par, which negatively affects the overall JEE scores for the aforementioned technical areas from a multisectoral standpoint. This suggests a critical need to improve capacities for the animal sector, such as targeted interventions in specific technical areas to help accelerate IHR (2005) implementation.

Most countries already have multiple governmental training programmes. However, the need to strengthen the number and distribution of sufficiently skilled human resources at all levels of the health system is critical. This requires developing a strategy for targeted health workforce development along with a career structure and a monitoring and evaluation component.

The analysis has shown that there is developed capacity of having a fully functioning Emergency Operating Centres (EOC) among the 14 countries (3). However, these EOCs are primarily managed by non-health sectors, such as defense for response to disasters and humanitarian emergencies. Inclusion of ministries of health as part of the management structure of these EOCs, or coordination among various EOCs in-country if managed by multiple sectors like defense and health, is critical to coordinate the effective public health response during outbreak or humanitarian emergencies.

A detailed review of national legislation is important to improve governance and facilitate the implementation of IHR (2005) capacities including cross border collaboration for surveillance and response to public health events (31). Many countries benefit from public health-related legislation that dates back a few decades and has not been updated with the requirements of the IHR and the development in public health systems in the specified countries. Additionally, mechanisms to enhance the public health management of foodborne diseases and food contamination, chemical, nuclear and radiological events appeared to be common gaps among the 14 countries. Therefore, mechanisms to enhance public health surveillance and response to chemical, nuclear and radiological events need further attention.

The strong correlations found between some indicators suggest that targeted interventions in specific technical areas may also accelerate the implementation of other technical areas under IHR (2005). For example, multisectoral coordination and regular information sharing between sectors may not only improve the development of IHR capacities, but also may improve notification of notifiable events under IHR as part of the overall enhancement of health information system. Such developments in the health information systems should aim for solutions that bring together vertical modalities of data collection under a systematic and comprehensive approach. Additionally, the strong correlation between the JEE scores and the burden of mortality indicators and health system related variables – and their determination by key developmental indicators – suggest that developing and implementing plans of action to meet the IHR (2005) capacities is critically needed as an integral part of the essential public health functions of national health systems. A recent systematic review of the

building blocks’ relevance to the Ebola outbreak underlines their importance in practice and as an evaluative framework (32).

Compared with Groups 2 and 3 countries in the Region, Group 1 countries tend to score higher on JEE. However, the overall lack of a significant correlation between JEE scores and government expenditure on health may be due to sample size. Another possible reason could be flaws in resources allocation and mobilization within the health system. For the latter, resources in the developing countries might be received but are targeting categorical vertical programmes such as maternal and child health and tuberculosis, but not for cross-cutting public health and multisectoral programmes. As such, development of IHR systems are more of a function of focused attentions to the requirements of such systems, while being affected by the general economic capabilities of the countries. Attention to health financing situation of the country is also important in order to develop feasible financing options to increase allocation of domestic resources to priority areas of health system development and response.

The paper has also shown that politically stable countries tend to score higher than less politically stable countries. However, a country can still develop its public health functions. Related plans of action need to be flexible enough to accommodate the changing situation and respond to the needs but manages to maintain public health capacities.

Limitations

Our analysis had limitations that include a small sample size, which could have resulted in identifying additional correlations that may have been significant and can provide additional information to improving JEE scores. Also, the sample included data analysis from 14 of the 22 countries in the Region, which may limit the generalizability. However, the results of the analysis do have face validity from a programmatic standpoint. There may also be other factors, untested here, that empirically play a predictive role in JEE scoring, including additional analysis and methods that could be utlized for future JEE-related programmatic research and decision-making.

Implementation of the JEE process is a work in progress in the WHO EMR. This paper helps to fill an important evidence gap in our understanding of JEEs and their relevance to countries. Throughout the process of conducting the JEE, countries have shown commitment of their national authorities to improve their IHR (2005) capacities. They have also found the JEEs to be valuable for multisectoral engagement and generating evidence for their policy-makers (19). This commitment may be further reflected in the coming months as these countries finalize the development of their national action plans for health security based on the JEE outcomes. Also, as JEE scores tend to differ between countries based on their category, it may be important to take into account the other components of the IHR monitoring and evaluation framework (IHRMEF), such as simulation exercises and after action reviews. This would provide a comprehensive view of the countries’ capacities and functionality, and consider prioritizing the focus of IHR implementation while developing National Action Plans for Health Security (NAPHS). This is not to imply that the other technical areas are less important, but given the challenges and reality check associated with implementation at country-level, it is important to prioritize and tailor implementation activities based on country needs, context and future plans for development.

Inclusive of universal health coverage needs to be considered so that the implementation of a plan covers the poorest and most vulnerable populations through public funds. Harmonizing the planning with the annual national budgeting processes, and increasing and appropriate allocation of the health sector portion in the national budget, is critical for implementation and sustainability of NAPHS. Support for countries could best be directed to improving measures of cooperation and organization in specific technical areas. In implementing the JEE and the other components of the IHR MEF, it is hoped that improved compliance in the application and implementation of the IHR (2005) will be achieved.

Conclusion

In conclusion, to effectively promote health and build capacity to prevent, detect and respond to diseases, a country needs to have in place a number of essential public health functions (33). The IHR (2005) core capacities, as represented in the JEE tool by the 19 technical areas, are a subset of such essential public health functions. Lessons from the JEE missions and these analyses show that compliance with the IHR (2005) appears to be within reach for most countries, thereby ensuring not only health security at the country level but globally.

Acknowledgements

The authors wish to acknowledge the leadership of its Member States in the Eastern Mediterranean Region for volunteering for JEEs and their commitment to implement and address gaps identified through their own resources as well as partners with international agencies and public and private donors. We also recognize the guidance and support received by the WHO JEE secretariat as well as from the US Department of Health and Human Services, the US Centers for Disease Control and Prevention, the US Department of Agriculture, the Food and Agricultural Organization of the United Nations, the Organization for Animal Health, and the Government of Finland; and express our appreciation to the National IHR focal points in WHO Regional Office for the Eastern Mediterranean (WHO/EMRO), as well as to the subject matter experts that participated and contributed to JEEs in WHO/EMRO.

Funding: None.

Competing interests: None declared.

Analyse des évaluations externes conjointes dans la Région OMS de la Méditerranée orientale

Résumé

Contexte : L’évaluation externe conjointe est un nouveau modèle d’examen externe par les pairs des capacités requises au titre du RSI utilisant des approches normalisées.

Objectifs : La présente étude avait pour objectif de consolider les résultats de ces évaluations dans la Région de la Méditerranée orientale et d’évaluer leur pertinence.

Méthodes : Une analyse des données a été menée dans 14 pays ayant conduit une évaluation externe conjointe dans la Région. Le score moyen des évaluations externes conjointes pour chacun des 19 domaines techniques, ainsi que celui des domaines techniques dans leur ensemble, ont été calculés. Des analyses bivariées et multivariées ont été menées afin d’évaluer les corrélations avec les indicateurs clés en matière de santé, de statut socio-économique et de systèmes de santé.

Résultats : Les scores moyens des évaluations externes conjointes variaient considérablement entre les domaines techniques. La moyenne cumulative des évaluations externes conjointes (moyenne des scores des indicateurs d’un domaine technique donné) était de 3 (fourchette comprise entre 1 et 4). Les indicateurs liés à la résistance aux antimicrobiens, à la sécurité et la sûreté biologiques affichaient les scores les plus bas. Les contre-mesures médicales, et les capacités liées au déploiement de personnel et au lien entre la santé publique et la sécurité obtenaient la moyenne cumulative la plus haute de 4 (fourchette comprise entre 2 et 5). Les scores des évaluations externes conjointes liés à la plupart des indicateurs clés ont été examinés. Les pays dotés d’un meilleur système de financement de la santé, ayant une meilleure couverture des services de santé, et dont la situation sanitaire était meilleure avaient généralement des scores d’évaluations externes conjointes plus élevés. Les taux de fécondité des adolescents, le ratio de mortalité néonatale et le ratio net d’inscription en école primaire constituaient des facteurs essentiels dans le score d’évaluation externe conjointe d’un pays.

Conclusion : Une approche multisectorielle intégrée, incluant un système de financement de la santé et une couverture transversaux et bien planifiés, est cruciale pour combler les lacunes principales identifiées par les évaluations externes conjointes et ainsi garantir la sécurité sanitaire régionale et mondiale.

تحليل التقييمات الخارجية المشتركة في إقليم منظمة الصحة العالمية لشرق المتوسط

داليا سمهوري، كاشف أيجاز، آرش رشيديان، ستيلا شونغونغ، أنتوان فلاهو، سوزان بابتش، جواد المحجور

الخلاصة

الخلفية: لقد طـُوِّرَ التقييم الخارجي المشترك كنموذج جديد للتقييمات الخارجية لمراجعة القرناء بشأن القدرات الخاصة باللوائح الصحية الدولية وذلك باستخدام نُهُج معيارية.

الهدف: هدفت هذه الدراسة إلى تجميع النتائج التي توصلت إليها هذه التقييمات في إقليم شرق المتوسط، وتقييم أهميتها.

طرق البحث: أُجـْرِيَ تحليل لبيانات التقييمات الخارجية المشتركة لعدد 14 بلداً الذين أكملوا هذه التقييمات في الإقليم. وحُسِبَ متوسط درجات التقييمات الخارجية المشتركة لكل مجال من 19 مجالاً تقنياً ولمجمل المجالات التقنية. وأُجريت تحليلات ثنائية المتغير ومتعددة المتغيرات لتقييم العلاقات المتبادلة مع المؤشرات الرئيسية الصحية، والاجتماعية-الاقتصادية، ومؤشرات النظام الصحي.

النتائج: تباين متوسط درجات التقييم الخارجي المشترك تبياناً كبيراً في المجالات التقنية. كان المتوسط التراكمي للتقييم الخارجي المشترك (متوسط درجات المؤشر المتصل بالمجال التقني) قد بلغ 3 درجات (النطاق: 1-4) وحصلت مؤشرات مقاومة مضادات الميكروبات، ومؤشرات السلامة البيولوجية والأمن البيولوجي على أدنى الدرجات. وقد حصلت التدابير الطبية، ونشر الموظفين، وربط الصحة العامة مع الأمن على أعلى درجة للمتوسط التراكمي وبلغت 4 درجات (النطاق: 2-5) ارتبطت درجات التقييم الخارجي المشترك مع أغلب المؤشرات الرئيسية التي فُحِصَت. وحصلت البلدان التي لديها مستوى أفضل من التمويل الصحي والتغطية بالخدمات الصحية والحالة الصحية عموماً على درجات أعلى في التقييم الخارجي المشترك. وارتبط معدل خصوبة المراهقين، ونسبة وفيات الولدان، ونسبة الالتحاق بالمدارس الابتدائية مع درجة التقييم الخارجي الإجمالي للبلد.

الاستنتاجات: إن اتباع نهج متكامل متعدِّد القطاعات، ويشمل نظاماً شاملاً ومخططاً له جيداً للتمويل الصحي والتغطية الصحية، أمر بالغ الأهمية لمعالجة الثغرات الرئيسية التي حددها التقييم الخارجي المشترك لضمان الأمن الصحي العالمي والإقليمي.

References

1. World Health Organization. World Health Assembly resolution WHA58.3. May, 2005. http://www.who.int/csr/ihr/WHA58-en.pdf, accessed May 2017.
2. Hoffman JJ. Making the international health regulations matter: promoting universal compliance through effective dispute resolution. In: Rushton S, Youde J, editors. Routledge handbook on global health security. Oxford: Routledge; 2014. pp. 239–51.
3. World Health Organization. Sixty-Eighth World Health Assembly. First report of the Ebola Interim Assessment Panel. Geneva. 2015, A68/25. Last seen World Health Organization: http://apps.who.int/gb/ebwha/pdf_files/WHA68/A68_25-en.pdf
4. Commission on a Global Health Risk Framework for the Future. The neglected dimension of global security: a framework to counter infectious disease crises. Geneva, 2016. Last seen 15 February, 2018. http://www.nap.edu/catalog/21891/the-neglected-dimension-of-global-security-a-framework-tocounter
5. World Health Organization. Report of the Ebola Interim Assessment Panel. July 2015. (http://www.who.int/csr/resources/publications/ebola/report-by-panel.pdf).
6. Moon S, Sridhar D, Pate MA, Jha AK, Clinton C, Delaunay S, et al. Will Ebola change the game? Ten essential reforms before the next pandemic. The report of the Harvard-LSHTM Independent Panel on the Global Response to Ebola. Lancet. 2015 Nov 28;386(10009):2204–21. https://doi.org/10.1016/S0140-6736(15)00946-0 PMID:26615326
7. World Health Organization. Sixty second Regional Committee for the Eastern Mediterranean. Assessment and monitoring of the implementation of the International Health Regulations (2005). Cairo, 2015; EM/RC62/R.3. (http://applications.emro.who.int/docs/RC62_Resolutions_2015_R3_16576_EN.pdf).
8. Gostin LO, Katz R. The International Health Regulations: The Governing Framework for Global Health Security. Milbank Q. 2016 06;94(2):264–313. https://doi.org/10.1111/1468-0009.12186 (http://scholarship.law.georgetown.edu/facpub/1770 PMID:27166578).
9. World Health Organization. Checklist and indicators for monitoring progress in the development of IHR core capacities in States Parties. Geneva, 2013. (http://apps.who.int/iris/bitstream/handle/10665/84933/WHO_HSE_GCR_2013.2_eng.pdf?sequence=1).
10. World Health Organization. Sixty-ninth World Health Assembly. Annual report on the Implementation of the International Health Regulations (2005). Annex to document A69.20; Geneva. 2016. Last seen (http://apps.who.int/gb/ebwha/pdf_files/WHA69/A69_20-en.pdf).
11. World Health Organization. Sixty-eighth World Health Assembly. Report of the Review Committee on Second Extensions for Establishing National Public Health Capacities and on IHR Implementation. Geneva, 2015; A68/22 Add.1: (http://apps.who.int/gb/ebwha/pdf_files/WHA68/A68_22Add1-en.pdf).
12. World Health Organization. Fifty-eight World Health Assembly. International Health Regulations (2005). Geneva. 2005; WHA58.3. last seen 15 February 2018 (http://apps.who.int/gb/ebwha/pdf_files/WHA58-REC1/english/A58_2005_REC1-en.pdf).
13. World Health Organization. Sixty second Regional Committee for the Eastern Mediterranean. Assessment and Monitoring of implementation of the International Health Regulations (2005): meeting the 2016 target. Cairo. 2015; EM/RC62/8. Last seen 10 January 208: (http://applications.emro.who.int/docs/RC_technical_papers_2015_8_16537_EN.pdf?ua=1).
14. Moon S, Leigh J, Woskie L, Checchi F, Dzau V, Fallah M, et al. Post-Ebola reforms: ample analysis, inadequate action. BMJ. 2017 01 23;356:j280. https://doi.org/10.1136/bmj.j280 PMID:28115316
15. Bell E. at Al. Joint External Evaluation—Development and Scale-Up of Global Multisectoral Health Capacity Evaluation Process, Emerging Infectious Disease, December 2017. Volume 23, Globalhealth security supplement. (http://wwwnc.cdc.gov/eid/not-found.html?aspxerrorpath=/eid/article/23/13/17-0949_article).
16. World Health Organization. Sixty-ninth World Health Assembly. Annual report on the Implementation of the International Health Regulations (2005). Annex to document A69.20; Geneva. 2016 (http://apps.who.int/gb/ebwha/pdf_files/WHA69/A69_20-en.pdf).
17. World Health Organization. Sixty-eighth World Health Assembly. Report of the Review Committee on Second Extensions for Establishing National Public Health Capacities and on IHR Implementation. Geneva, 2015; A68/22 Add.1: (http://apps.who.int/gb/ebwha/pdf_files/WHA68/A68_22Add1-en.pdf).
18. World Health Organization. Strategic Partnership Portal, building stronger health systems for sustainable health security. Geneva. 2016. Last seen 14, February, 2018: (https://extranet.who.int/sph/).
19. Samhouri D, Ijaz K, Thieren M, Flahault A, Babich SM, Jafari H, et al. World Health Organization Joint External Evaluations in the Eastern Mediterranean Region, 2016-17. Health Secur. 2018 Jan/Feb;16(1):69–76. https://doi.org/10.1089/hs.2017.0066 PMID:29406822
20. World Health Organization. Joint External Evaluation Tool. Last seen 14, February, 2018 http://apps.who.int/iris/bitstream/handle/10665/204368/9789241510172_eng.pdf?sequence=1).
21. Samhouri D, Ijaz K, Thieren M, Flahault A, Babich SM, Jafari H, et al. World Health Organization Joint External Evaluations in the Eastern Mediterranean Region, 2016-17. Health Secur. 2018 Jan/Feb;16(1):69–76. https://doi.org/10.1089/hs.2017.0066 PMID:29406822
22. World Health Organization. Eastern Mediterranean Region Framework for health information systems and core indicators for monitoring health situation and health system performance 2017. Cairo, 2017. Available at: http://applications.emro.who.int/docs/EMROPUB_2017_EN_16766.pdf?ua=1&ua=1. Accessed May, 2018
23. Alwan A, Ali M, Aly E, Badr A, Doctor H, Mandil A, et al. Strengthening national health information systems: challenges and response. East Mediterr Health J. 2016 02 1;22(11):840–49. https://doi.org/10.26719/2016.22.11.840 PMID:28177115
24. World Health organization. Eastern Mediterranean Region Framework for health information systems and core indicators for monitoring health situation and health system performance, Cairo, 201 (http://applications.emro.who.int/docs/EMROPUB_2017_EN_16766.pdf?ua=1).
25. Travis P, Bennett S, Haines A, Pang T, Bhutta Z, Hyder AA, et al. Overcoming health-systems constraints to achieve the Millennium Development Goals. Lancet. 2004 Sep 4-10;364(9437):900–6. https://doi.org/10.1016/S0140-6736(04)16987-0 (https://www.sciencedirect.com/science/article/pii/S0140673604169870) PMID:15351199
26. Chen L, Evans T, Anand S, Boufford JI, Brown H, Chowdhury M, et al. Human resources for health: overcoming the crisis. Lancet. 2004 Nov 27;364(9449):1984–90. https://doi.org/10.1016/S0140-6736(04)17482-5 (https://www.sciencedirect.com/science/article/pii/S0140673604174825) PMID:15567015
27. World Health Organization. Working together for health. The world health report, 2006. Policy Briefs. Geneva, 2006 (http://apps.who.int/iris/bitstream/10665/43471/1/9241594241_eng.pdf).
28. World Bank Group. World development indicators. Washington DC, 2017 (https://data.worldbank.org/indicator).
29. World Health Organization. Global Health Observatory (GHO) data, World Health Statistics 2017: Monitoring health for the SDGs. Geneva, 2017. (http://www.who.int/gho/publications/world_health_statistics/2017/en/).
30. World Health organization. Demographic, social and health indicators for countries of the eastern Mediterranean region, 2013 (http://applications.emro.who.int/dsaf/EMROPUB_2013_EN_1537.pdf).
31. World Health Organization. International Health Regulations. Review of National Legislation (http://www.who.int/ihr/legal_issues/legislation/en/)
32. Shoman H, Karafillakis E, Rawaf S. The link between the West African Ebola outbreak and health systems in Guinea, Liberia and Sierra Leone: a systematic review. Global Health. 2017 01 4;13(1):1. https://doi.org/10.1186/s12992-016-0224-2 PMID:28049495
33. Bettcher DW, Sapirie S, Goon EHT. Essential public health functions: results of the international Delphi study. World Health Stat Q. 1998;51(1):44–54. PMID:9675808