PDF version

Miguel Sánchez-Cárdenas,^1,2 Eduardo Garralda,^1,2 Danny van Steijn,^1,2 Nasim Pourghazian,³ Slim Slama,³ Marie-Charlotte Bouësseau,⁴ Carlos Centeno,^1,2 and Eastern Mediterranean Regional Expert Network on Palliative Care⁵

¹ATLANTES Global Palliative Care Observatory, Institute for Culture and Society, University of Navarra, Pamplona, Spain (Correspondence to: E. Garralda: This e-mail address is being protected from spambots. You need JavaScript enabled to view it ). ²Instituto de Investigación Sanitaria de Navarra, Pamplona, Spain. ³Noncommunicable Diseases Prevention Unit, Department of Universal Health Coverage/Noncommunicable Diseases, World Health Organization Regional Office for the Eastern Mediterranean, Cairo, Egypt. ⁴Integrated Health Services Department, World Health Organization Headquarters, Geneva, Switzerland 5Egypt (Samy Alsirafy), Islamic Republic of Iran (Maryam Rassouli), Iraq (Samaher A. Fadhil), Jordan (Omar Shamieh), Kuwait (Iman Al Diri), Lebanon (Hibah Osman, Huda Abu-Saad Huijer and Myrna A.A. Doumit), Morocco (Asmaa El Azhari), Oman (Bassim Al Bahrani), Pakistan (Muhammad Atif Waqar), Occupied Palestinian Territory (Hani S. Ayyash and Confidential), Qatar (Azza Adel Ibrahim Hassan), Saudi Arabia (Sami Ayed Alshammary).

Abstract

Background: The World Health Organization Office for the Eastern Mediterranean Region (WHO/EMRO), and the recently created palliative care experts network for the Eastern Mediterranean Region (EMR), decided to develop region-specific indicators for monitoring national palliative care development in the Region.

Aims: To identify relevant and feasible macro-indicators for palliative care development for the EMR.

Methods: Palliative care experts from the EMR were nominated and invited to complete a 2-round Delphi study to rate macro-indicators from previous studies and propose new ones based on the EMR regional characteristics. All indicators were assessed according to regional relevance (R) and feasibility (F). A content validity index (CVI) was calculated. Indicators with CVI ≥ 0.7/1, and scoring ≥ 7/9 for the R and F averages were selected.

Results: Twelve of the 22 countries in the Region were represented in the study. In the first round, 11 indicators were selected and 13 new ones proposed. In the second round, 15 indicators matched R, F and CVI criteria. Top-scored indicators were: existence of a current national palliative care strategy (R = 8, F = 8, CVI = 1); ratio of specialized services (for adults and children) in the country per population (R = 8, F = 7, CVI = 1); allocation of funds for palliative care in the national health budget by the Ministry of Health or equivalent government agency (R = 8, F = 6, CVI = 1); education for prequalification of doctors/nurses (R = 8, F = 8, CVI = 0.9); and availability of morphine and other strong opioids (R =8, F = 8, CVI= 0.9).

Conclusion: A baseline set of 15 region-specific indicators for measuring the development of palliative care were validated by experts in the EMR.

Keywords: palliative care, indicator, consensus, Delphi study, Eastern Mediterranean Region.

Citation: Sánchez-Cárdenas M; Garralda E; Van Steijn D; Pourghazian N; Slama S; Bouesseau M-C; et al. Region-specific macro-indicators for palliative care development in the Eastern Mediterranean Region: a Delphi study. East Mediterr Health J. 2022;28(8):560–568. https://doi.org/10.26719/emhj.22.036

Received: 20/09/21; accepted: 23/01/22

Copyright © World Health Organization (WHO) 2022. Open Access. Some rights reserved. This work is available under the CC BY-NC-SA 3.0 IGO license (https://creativecommons.org/licenses/by-nc-sa/3.0/igo)

---

Introduction

In 2015, 2 590 168 people died who needed palliative care across the 21 Member States of WHO/EMRO . Of those, 2 115 324 were adults, most of whom had cancer (26.3%), stroke (18.5%), dementia (11.9%), or AIDS, 14.3% died from external causes and 5% from other causes (1,2). An estimated 474 844 children were in need of palliative care (2). To date, estimations of access to palliative care services pose a major challenge in the EMR, where only an estimated 5% of adults who are in need of care actually receive it (3). The capacity of Member States to face this challenge remains little known and few studies have investigated the palliative care status across countries (4–6).

Through assessment of countries’ capacities, it is possible to estimate their health systems’ responses to suffering associated with advanced diseases and, therefore, to the needs of the patients, caregivers and families. Indicators predicting the development of palliative care in terms of existing policy, education, use of medicines, provision of services and professional activity related resources at different levels of care would be useful in this regard. Some recent studies have proposed indicators that are valid for any country in the world because of their high content validity and level of agreement among international experts (7,8). However, as demonstrated by the 2017 African study, indicators for evaluating palliative care development should be adjusted to regional contexts, and their implementation should be preceded by a discussion on the feasibility and relevance of each indicator (9,10).

The framework for action on cancer prevention and control in the EMR (11), updated in 2019, recommends actions to improve palliative care and the availability of essential drugs, for which is necessary to have metrics and indicators to evaluate progress in the region. Following an expert meeting in Beirut in September 2019, hosted by WHO/EMRO, a dialogue was initiated at the regional level regarding the development of a set of quality indicators for palliative care in the region.

WHO/EMRO recently established a regional network for palliative care; a community of practice dedicated to the development of national systems for quality evaluation, improvement and assurance of palliative care services (12). This was seen as an opportunity to bring together countries in the EMR to review previously published indicators on palliative care, and update the information published in the Atlas of Palliative Care in the Eastern Mediterranean Region 2017 (4). The aim of this study was to identify the most contextually relevant and feasible indicators for evaluating the development of palliative care in the EMR.

Methods

Delphi consensus process

A Delphi consensus process was used because it is particularly useful for gathering representative opinions from expert-based judgements and is often used to identify consensus (13). This study followed the standards proposed in the recommendations for conducting and reporting Delphi studies (CREDES) (14). The consensus process was designed through a 2-round online survey (Figure 1). The starting point was a set of 11 indicators (Table 1) from previous studies building on the WHO Public Health Strategy for the Integration of Palliative Care (15); the Mapping Levels of Palliative Care Development in 198 Countries: The Situation in 2017 (8); and the Brief Manual on Health Indicators Monitoring Global Palliative Care Development (7).

Experts panel

The criteria for participating in the Delphi study were: (1) being originally from the EMR; (2) knowledge of palliative care in the respective countries (as perceived by WHO/EMRO Member States); (3) previous participation in similar studies or scientific publications relevant to the topic of study; and (4) participation in palliative care networks or advocacy activities for at least 2 years.

Members of the pre-existing network of experts from the EMR met these criteria and were nominated as experts for this study. A sample of 13 experts, representative of the EMR, were considered appropriate because of their knowledge of the content validity matters in Delphi methodology (16). Previous studies show that the sample size can range from 7 to 15 (17). For 5 countries that had no representation in the network of experts, new experts were nominated based on the inclusion criteria.

First Delphi round

Round 1 began in April 2020. A questionnaire exploring 11 indicators was designed in SurveyMonkey, divided into 5 domains: policy, education, use of medicines, service provision, and professional activity. Each indicators had a definition and the questions with which it should be explored, and the metadata for each indicator are presented as supplementary material. The panel of experts rated the indicators on a Likert-type scale of 1–9 (1 being the least and 9 the most) for relevance and feasibility. Relevance was defined and explained in the survey as the degree to which the indicator was related to palliative care development at the national level, and feasibility as the degree to which an indicator could be easily obtained or collected from data sources. In Round 1, experts were allowed to propose new indicators, considering the specific characteristics of the EMR. Three researchers reviewed the suggested indicators, removing duplicate items already included in other indicators, and those not specific enough to be considered as standalone indicators. Participants who completed the previous round were invited to take part in successive rounds.

Second Delphi round

Round 2 was conducted between May and June 2020. Each expert was sent independently by email a table showing the values of their previous rating together with the group’s rating, and the comments collected in the first round. Experts were asked to rerate the indicators. Indicators suggested by the experts in the first round were listed to rate their relevance and feasibility, providing joint response categories and the number of experts that suggested their inclusion.

Data analysis

The results of the rating were presented in a matrix for analysis. Indicator selection was based on the average scores of relevance and feasibility: the global score. Values ≥ 7 were considered adequate. To verify coherence among experts’scores, the Content Validity Index (CVI) was calculated (number of experts calculating the highest scores [7–9 points] divided by the number of experts). A CVI of 1 indicated 100% unanimity. Consensus was obtained by indicators with global score > 7 and CVI ≥ 0.70. The results and analyses were presented and discussed with members of the EMR Palliative Care Expert Network.

Results

Participants

Thirteen palliative care experts representing 12 countries participated in the consensus process: Egypt, Iraq, Islamic Republic of Iran, Jordan, Kuwait, Lebanon, Morocco, Occupied Palestinian Territories, Oman, Pakistan, Qatar, and Saudi Arabia (Table 2), representing 59% of EMR countries. In the first round, 13 experts fully completed the survey and 11 did so in the second round (response rate 92%). Experts from Afghanistan, Tunisia, United Arab Emirates and Sudan did not participate in the consensus and no expert was identified in Bahrain, Djibouti, Syrian Arab Republic, Somalia and Yemen.

First round

Relevance of the indicators had a high score; they were all above 8-9 points except for the indicators related to a national law on palliative care and the existence of a specialization process in palliative medicine. The rating of relevance and feasibility for the indicators initially proposed to evaluate the development of palliative care in the EMR is presented in Figure 2A. Besides these indicators, experts proposed 20 new indicators in the first round; including existence of grants to finance palliative care research (n = 3); level of public awareness of palliative care in the country (n = 3); publications regarding national palliative care development (n = 3); and existence of systems of auditing, quality evaluation, improvement or assurance for palliative care services (n = 2). Other indicators reported by 1 expert only were: availability of centres of excellence for palliative clinical care, education and research; availability of palliative care for displaced persons or refugees; palliative care provision in long-term care facilities (nursing homes/residences); and general availability of palliative care services for nonmalignant diseases.

Second round

Thirteen indicators passed to the second round; 7 were excluded because they were already included or were not specific enough to be considered as standalone indicators (Supplementary Material 1). In the second round, 11 experts rerated the 11 initial indicators and scored the 13 new indicators (Figure 2B). Among 24 indicators presented in the second round, 15 matched the selection criteria: relevance (R) ≥ 7, feasibility (F) ≥ 7 and CVI ≥ 0.7 (Table 3).

The average value of indicators included in the analysis was 0.7, indicating that a majority of experts suggested using these indicators, thus reaching a consensus to include 15 indicators. The indicators related to specialized palliative care services; paediatric palliative care provision; budget line items for palliative care; and a current plan, programme, policy or national strategy for palliative care, reached a value of 1, which indicated a unanimous decision by experts to include these indicators in the analysis.

Nine indicators with an average CVI of 0.4 scored < 7.0, suggesting a consensus to exclude. Only 40% of the experts considered it necessary to use indicators related to palliative care professors in medical and nursing schools, quality assessment of palliative care services, volunteer programmes, national palliative care publications, participation in national and international palliative care conferences, specific services for noncancer patients, and palliative care in long-term care facilities.

Discussion

The experts agreed on 15 relevant and feasible indicators to monitor the development of palliative care in the EMR. Some indicators were scored unanimously selected because of their importance and feasibility , although they were slightly lower in terms of feasibility. These include a current national palliative care strategy (R = 8, F = 8, CVI=1); the number of specialized palliative care services (for adults and children) in the country per population (R = 8, F = 7, CVI =1); and the inclusion of a line item for palliative care in the national health budget for the Ministry of Health or equivalent government agency (R =8, F = 6, CVI = 1). Slightly worse scored indicators were: availability of morphine and other strong opioids (R = 8, F = 8, CVI = 0.9) and reported annual opioid consumption – excluding methadone – in morphine equivalence (ME) per capita (R = 8, F = 7, CVI= 0.8).

Most of these indicators have been considered critical for assessing palliative care development in the EMR and have been previously explored (4,5). However, compared to a previous international consensus on palliative care indicators, the importance of some indicators for the region seems different (18). Six indicators in this study scored above the mean obtained in the international consensus: availability of morphine and other strong opioids; specialized palliative care services per population (for children and adults); education for prequalified doctors and nurses; and inclusion of palliative care services in the basic package of health services (19). The availability of morphine may be explained by the historical difficulties in accessing pain medicines in the EMR, as reported by many international studies (3,5,19). Some have pointed out that oral morphine was available in < 20% of primary care facilities in the EMR (6), and that medicines continue not to be available and accessible in sufficient quantities (21). Some country-level studies have looked at the availability and accessibility of opioids and their importance in palliative care provision, with similar conclusions (22–26). The lack of specialized palliative care services for adults and children has also been reported elsewhere (4,8).

Several indicators were not part of the international consensus, but were specifically suitable to the EMR. These include: the level of public awareness of palliative care; inclusion of palliative care in health insurance plans; availability of centres of excellence for palliative clinical care, education and research; and availability of grants to finance palliative care research. This suggests that conducting region-specific discussions on the indicators did not only reiterate the importance of already-used indicators, but also added new ones that were relevant to the context. For instance, unlike other studies, the level of public awareness of palliative care in the EMR seems important. This may be due to the geographic and ethnic diversity across countries and the varying perceptions about end-of-life.

The involvement of EMR experts in determining the best indicators to measure palliative care development in the region allowed qualification of the indicators. According to several experts, the indicator of a national strategy for palliative care requires confirming whether it is actively being implemented, as existence of the strategy by itself does not guarantee the development of palliative care. Regarding the number of specialized palliative care services, experts pointed out that this indicator should evolve towards measuring capacity, quality, number of patients cared for, and distribution of palliative care resources to provide an accurate picture of specialized service provision. With regard to paediatric palliative care provision, the indicator should differentiate specifically dedicated programmes from broader programmes admitting adults and children. Inclusion of a line item for palliative care in the national health budget for the Ministry of Health or equivalent, although considered critical to ensure government support, could be more feasible by exploring the coverage of palliative care services by government insurance.

Despite providing a set of region-specific palliative care indicators, this study needs to acknowledge some limitations. Firstly, the number of countries participating may limit the representativeness of the study for the whole region, considering that participating countries are presumably the most developed. This could entail leaving out appropriate indicators for less-developed countries. Secondly, the list had up to 15 indicators, which meant a lot of data to gather, which was an onerous duty for countries’ health information systems. In this sense, alignment with existing country data systems would be useful for progress in monitoring palliative care. Lastly,

all variables except for reported annual opioid consumption – excluding methadone – in ME per capita, were not outcomes but rather structure or process indicators.

The immediate implication of these findings is the need to pilot these indicators in the EMR. One study has recently been submitted proving the ability of the indicators to depict the status of palliative care in the EMR. However, future research should focus on increasing the participation of experts from more countries (especially the highly populated countries). Apart from the network members, it is desirable to identify oncologists or other specialists from the current missing countries. These would not only validate the list of indicators by assessing whether they capture the capacity, distribution, quality and nature of specialized services, but also identify more outcome indicators, such as the number of people cared for by specialized services, or the provision of palliative care by primary healthcare resources.

Conclusions

Fifteen indicators were considered relevant and feasible for evaluation of the development of palliative care in the EMR, including health policy, palliative care services, access to medicines, education, and professional activity related indicators. The most contextually relevant, feasible, and unanimous agreed by experts were: a current national palliative care strategy; number of specialized palliative care services (for adults and children) in the country per population; and the inclusion of a line item for palliative care in the national health budget for the Ministry of Health or equivalent government agency.

Funding: None

Competing interests: None declared.

Macro-indicateurs spécifiques à la Région pour la mise en place de soins palliatifs dans la Région de la Méditerranée orientale : étude Delphi

Résumé

Contexte : Le Bureau régional de l'Organisation mondiale de la Santé (OMS) pour la Méditerranée orientale et le réseau d'experts des soins palliatifs récemment créé dans la Région ont décidé de mettre au point des indicateurs spécifiques à la Région pour suivre les progrès qui y sont réalisés en matière de soins palliatifs au niveau des pays.

Objectifs : Identifier des macro-indicateurs pertinents et réalisables pour la mise en place des soins palliatifs dans la Région de la Méditerranée orientale.

Méthodes : Des experts en soins palliatifs de la Région de la Méditerranée orientale ont été désignés et invités à participer à une étude Delphi à deux tours pour évaluer les macro-indicateurs des études précédentes et en proposer de nouveaux en fonction des caractéristiques régionales pour la Méditerranée orientale. Tous les indicateurs ont été évalués en fonction de leur pertinence (P) et de leur faisabilité (F) au niveau régional. Un indice de validité du contenu (IVC) a été calculé. Les indicateurs ayant un indice de validité de contenu supérieur ou égal à 0,7/1, et un score supérieur ou égal à 7/9 pour les moyennes P et F ont été sélectionnés.

Résultats : Douze des 22 pays de la Région étaient représentés dans l'étude. Lors du premier tour, 11 indicateurs ont été sélectionnés et 13 nouveaux indicateurs ont été proposés. Lors du second tour, 15 indicateurs correspondaient aux critères P, F et IVC. Les indicateurs les mieux notés étaient les suivants : existence d'une stratégie nationale au moment de l'étude pour les soins palliatifs (P = 8, F = 8, IVC = 1) ; ratio de services spécialisés (pour adultes et enfants) dans le pays par population (P = 8, F = 7, IVC = 1) ; allocation de fonds pour les soins palliatifs dans le budget national de la santé par le ministère de la Santé ou une institution gouvernementale équivalente (P = 8, F = 6, IVC = 1) ; formation pour la préqualification des médecins/infirmiers (P = 8, F = 8, IVC = 0,9) ; et disponibilité de morphine et d'autres opioïdes puissants (P = 8, F = 8, IVC = 0,9).

Conclusion : Un ensemble de référence de 15 indicateurs spécifiques à la Région pour mesurer la progression des soins palliatifs a été validé par des experts de la Région de la Méditerranée orientale.

مؤشرات كلية خاصة بإقليم شرق المتوسط لتطوير الرعاية الملطّفة في: دراسة بأسلوب دلفي

ميجيل سانشيز كارديناس، إدواردو جارالدا، داني فان ستيجن، نسيم بورغازيان، سليم سلامة، ماري شارلوت بوسو، كارلوس سينتينو وشبكة خبراء الرعاية الملطِّفة لاقليم شرق المتوسط

الخلاصة

الخلفية: إن مكتب منظمة الصحة العالمية لإقليم شرق المتوسط وشبكة خبراء الرعاية الملطِّفة لإقليم شرق المتوسط، التي أُنشئت مؤخرًا، قد قررا وضع مؤشرات خاصة بالإقليم لرصد تطوير الرعاية الملطِّفة في الإقليم.

الأهداف: هدفت هذه الدراسة الى تحديد المؤشرات الكلية المهمة والمجدية لتطوير الرعاية الملطِّفة في إقليم شرق المتوسط.

طرق البحث: جرى ترشيح خبراء في الرعاية الملطِّفة من إقليم شرق المتوسط، ودعوتهم إلى استكمال دراسة بأسلوب دلفي مكونة من جولتين، لتقييم المؤشرات الكلية من الدراسات السابقة، واقتراح مؤشرات جديدة، استنادًا إلى الخصائص الإقليمية لإقليم شرق المتوسط. وجري تقييم جميع المؤشرات حسب الأهمية الإقليمية (R) والجدوى (F). كذلك جرى حساب مؤشر صحة المحتوى (CVI). ووقع الاختيار على مؤشرات ذات صحة محتوى (CVI ) تبلغ ≥ 0.7/1، وتحقق ≥ 7/9 كمتوسط على مستوى الأهمية الإقليمية (R) والجدوى (F).

النتائج: شارك في هذه الدراسة اثنا عشر بلدًا من بلدان الإقليم البالغ عددها 22 بلدًا. وشهدت الجولة الأولى اختيار 11 مؤشرًا واقتراح 13 مؤشرًا جديدًا. أما في الجولة الثانية، فقد جرى الوقوف على 15 مؤشرًا مطابقًا لمعايير الأهمية الإقليمية (R) والجدوى (F) وصحة المحتوى (CVI). والمؤشرات التي حصلت على أعلى الدرجات هي ما يلي: وجود استراتيجية وطنية حالية للرعاية الملطِّفة (الأهمية الإقليمية (R) = 8 ، الجدوى (F) = 8، صحة المحتوى (CVI)=1)؛ ونسبة الخدمات المتخصصة (للبالغين والأطفال) في البلد لكل مجموعة سكانية (الأهمية الإقليمية (R) = 8 ، الجدوى (7) = 8، صحة المحتوى (CVI)=1)؛ وتخصيص أموال للرعاية الملطِّفة في ميزانية الصحة الوطنية التي تضعها وزارة الصحة أو وكالة حكومية مكافئة (الأهمية الإقليمية (R) = 8، الجدوى (F) = 6، صحة المحتوى (CVI)=1)؛ تثقيف الأطباء/ الممرضات بشأن الاختبار المسبق للصلاحية (الأهمية الإقليمية (R) = 8، الجدوى (F) = 8، صحة المحتوى (CVI)=0.9)؛ وتوفر المورفين والمواد الأفيونية القوية الأخرى (الأهمية الإقليمية (R) = ،8 الجدوى (F) = 8، صحة المحتوى (CVI)=0.9).

الاستنتاجات: تحقق الخبراء في إقليم شرق المتوسط من مجموعة أساسية تتألف من 15 مؤشرًا خاصًّا بالإقليم لتقدير تطوير الرعاية الملطِّفة.

References

1. Knaul FM, Farmer PE, Krakauer EL, De Lima L, Bhadelia A, Jiang Kwete X, et al. Alleviating the access abyss in palliative care and pain relief-an imperative of universal health coverage: the Lancet Commission report. Lancet. 2018 Apr 7;391(10128):1391–454. https://doi.org/10.1016/S0140-6736(17)32513-8 PMID:29032993

2. Connor SR, editor. Global atlas of palliative care (2nd edition). London: Worldwide Hospice Palliative Care Alliance; Geneva: World Health Organization; 2020.

3. Palliative care at regional level [website]. Cairo: World Health Organization Regional Office for the Eastern Mediterranean (http://www.emro.who.int/noncommunicable-diseases/palliative-care/palliative-care.html, accessed 26 March 2022).

4. Osman H, Rihan A, Garralda E, Rhee JY, Pons Izquierdo JJ, De Lima L, et al. Atlas of palliative care in the Eastern Mediterranean Region. Houston: IAHPC Press; 2017.

5. Fadhil I, Lyons G, Payne S. Barriers to, and opportunities for, palliative care development in the Eastern Mediterranean Region. Lancet Oncol. 2017 Mar;18(3):e176–84. https://doi.org/10.1016/S1470-2045(17)30101-8 PMID:28271872

6. Sharkey L, Loring B, Cowan M, Riley L, Krakauer EL. National palliative care capacities around the world: Results from the World Health Organization Noncommunicable Disease Country Capacity Survey. Palliat Med. 2018 Jan;32(1):106–13. https://doi.org/10.1177/0269216317716060 PMID:28677988

7. Arias-Casais N, Garralda E, López-Fidalgo J, Lima L De, Rhee JY, Centeno C. Brief Manual on Health Indicators Monitoring Global Palliative Care Development. Houston: IAHPC Press; 2019.

8. Clark D, Baur N, Clelland D, Garralda E, López-Fidalgo J, Connor S, Centeno C. Mapping levels of palliative care development in 198 countries: the situation in 2017. J Pain Symptom Manage. 2020 Apr;59(4):794–807.e4. https://doi/org/10.1016/j.jpainsymman.2019.11.009 PMID:31760142

9. Rhee JY, Luyirika E, Namisango E, Powell RA, Garralda E, Pons JJ, et al. APCA atlas of palliative care in Africa. Houston: IAHPC Press; 2017.

10. Rhee JY, Garralda E, Namisango E, Luyirika E, de Lima L, Powell RA, et al. Developing macroindicators of palliative care development in Africa: a process with in-country and international experts. J Palliat Med. 2018 Mar;21(3):342-353. https://doi/org/10.1089/jpm.2017.0207 PMID:29027836

11. Framework for action on cancer prevention and control in the WHO Eastern Mediterranean Region strategic interventions indicators [website]. World Health Organization; 2019 (http://www.who.int/nmh/ncd-tools/indicators-definition/en/ (accessed 8 November 2021).

12. Eastern Mediterranean Region Expert Network on Palliative Care [website]. Cairo: World Health Organization Regional Office for the Eastern Mediterranean (http://www.emro.who.int/noncommunicable-diseases/palliative-care/palliative-care.html, accessed 26 March 2022).

13. Chalmers J, Armour M. The Delphi technique. In: Liamputting P, editor. Handbook of research methods in health social science. Singapore: Springer Singapore; 2019:715–35.

14. Jünger S, Payne SA, Brine J, Radbruch L, Brearley SG. Guidance on Conducting and REporting DElphi Studies (CREDES) in palliative care: Recommendations based on a methodological systematic review. Palliat Med. 2017 Sep;31(8):684–706. https://doi/org/10.1177/0269216317690685 PMID:28190381

15. Stjernswärd J, Foley KM, Ferris FD. The public health strategy for palliative care. J Pain Symptom Manage. 2007 May;33(5):486–93. https://doi/org/10.1016/j.jpainsymman.2007.02.016 PMID:17482035

16. Bourrée F, Michel P, Salmi LR. [Consensus methods: review of original methods and their main alternatives used in public health]. Rev Epidemiol Sante Publique. 2008 Dec;56(6):415–23 (in French). https://doi/org/10.1016/j.respe.2008.09.006 PMID:19013039

17. Fitch K, Bernstein SJ, Aguilar MD, Burnand B, LaCalle JR, Lazaro P, et al. The RAND/UCLA appropriateness method user’s manual. Santa Monica: RAND; 2001.

18. Arias-Casais N, Garralda E, López-Fidalgo J, Rhee JY, Pons JJ, de Lima L, Centeno C. Consensus building on health indicators to assess PC global development with an international group of experts. J Pain Symptom Manage. 2019 Sep;58(3):445–53.e1. https://doi/org/10.1016/j.jpainsymman.2019.04.024 PMID:31163260

19. Arias N, Garralda E, De Lima L, Rhee JY, Centeno C. Global Palliative care and cross-national comparison: how is palliative care development assessed? J Palliat Med. 2019 May;22(5):580–90. https://doi/org/10.1089/jpm.2018.0510 PMID:30615544

20. Duthey B, Scholten W. Adequacy of opioid analgesic consumption at country, global, and regional levels in 2010, its relationship with development level, and changes compared with 2006. J Pain Symptom Manage. 2014 Feb;47(2):283–97. https://doi/org/10.1016/j.jpainsymman.2013.03.015 PMID:23870413

21. Silbermann M, Arnaout M, Daher M, Nestoros S, Pitsillides B, Charalambous H, et al. Palliative cancer care in Middle Eastern countries: accomplishments and challenges. Ann Oncol. 2012 Apr;23 Suppl 3(Suppl 3):15–28. https://doi/org/10.1093/annonc/mds084 PMID:22628412

22. El Azhari A. Cancer and palliative care in Morocco. In: Silbermann M, editor. Palliative care for chronic cancer patients in the community. Springer; 2021:243–8.

23. Alsirafy SA, Farag DE. A shortage of oral morphine in Egypt. Bull World Health Organ. 2016 Jan 1;94(1):3. https://doi/org/10.2471/BLT.15.156240 PMID:26769988

24. Barasteh S, Rassouli M, Parandeh A, Vahedian-Azimi A, Zaboli R, Khaghanizadeh M. Palliative care in the health system of Iran: a review of the present status and the future challenges. Asian Pac J Cancer Prev. 2020 Mar 1;21(3):845–51. https://doi/org/10.31557/APJCP.2020.21.3.845 PMID:32212816

25. Fadhil SA, Ghali HH. The current situation of palliative care services in Iraq. In: Silbermann M, editor. Palliative care for chronic cancer patients in the community. Springer;2021:341–9.

26. Soueidan S, Osman H, El-Jardali F. K2P policy brief. Integrating palliative care into the health system in Lebanon. Beirut: Knowledge to Policy (K2P) Center; 2018.