Select your language
Effectiveness of collaborative tele-mental health care for children with attention deficit hyperactivity disorder in primary care centres: randomized controlled trial in Dubai
Short research communication
Ammar Albanna,1 Karina Soubra,1 Deena Alhashmi,2 Zainab Alloub,1 Fatma AlOlama,3 Paul Hammerness,4 Jeyaseelan Lakshmanan,2 Lily Hechtman5 and Hesham M. Hamoda4
1 Al Jalila Children’s Specialty Hospital, Dubai, United Arab Emirates. 2 Mohamed Bin Rashid University of Medicine and Health Sciences, Dubai, United Arab Emirates. 3 Dubai Health Authority, Dubai, United Arab Emirates. 4 Department of Psychiatry, Boston Children’s Hospital and Harvard Medical School, Boston, United States of America. 5 McGill University, Montreal, Canada. (Correspondence to Ammar Albanna:
Abstract
Background: Attention deficit hyperactivity disorder is a common neurodevelopmental disorder. Accessing services for this disorder is a worldwide challenge and requires innovative interventions.
Aims: The Effectiveness of Collaborative Tele-Mental Health Services for ADHD in Primary Care trial is a registered randomized controlled trial that aims to investigate the effectiveness of tele-collaborative care for attention deficit hyperactivity disorder in primary health care centres in Dubai.
Methods: Six trained physicians started collaborative care clinics across Dubai. Eligible children with attention deficit hyperactivity disorder were randomized to receive tele-health collaborative care, or standard treatment. Waiting times and clinical and functional outcomes were measured in both groups and compared.
Results: Among the referred children (n = 112), 11 boys and 6 girls met the eligibility criteria (mean age 7.8 years). The drop-out rate at 6 months in the control group was 80% compared with 50% in the intervention group. The mean waiting time was significantly shorter in the intervention group (1.3 weeks) than the control group (7.1 weeks); P = 0.026. The mean difference in the Childhood Behavior Checklist total score over time was significantly higher in the intervention arm (P = 0.042), but the mean difference in the Vanderbilt scale was not significant.
Conclusion: Tele-collaborative care for children with attention deficit hyperactivity disorder within primary health care is feasible.
Keywords: attention deficit hyperactivity disorder, child psychiatry, mental health, primary care, Dubai.
Citation: Albanna A; Soubra K; Alhashmi D; Alloub Z; AlOlama F; Hammerness P, et al. Effectiveness of collaborative tele-mental health care for children with attention deficit hyperactivity disorder in primary care centres: randomized controlled trial in Dubai. East Mediterr Health J. https://doi.org/10.26719/emhj.23.076 Received 06/05/2022 ; accepted: 05/01/2023
Copyright © World Health Organization (WHO) 2023. Some rights reserved. This work is available under the CC BY-NC-SA 3.0 IGO license https://creativecommons.org/licenses/by-nc-sa/3.0/igo
Introduction
Attention deficit hyperactivity disorder (ADHD) is a pervasive neurodevelopmental disorder characterized by developmentally inappropriate levels of inattention and/or hyperactivity/impulsivity (1,2). ADHD is associated with long-term adverse outcomes and is a public health burden (3–5). Despite the availability of safe and effective treatments for ADHD (6), a large proportion of children with ADHD are unable to access services (7). This is because of limited availability of qualified professionals (8) and a lack of awareness of the condition among the public (9), among other reasons.
Dubai is the financial hub of the United Arab Emirates with a population of about 3.2 million (10). The prevalence of ADHD in school-aged students in the United Arab Emirates is about 4% (11). Specialized ADHD services are limited in Dubai, despite 2000 paediatric psychiatry assessments a year (12). Furthermore, the United Arab Emirates and the Eastern Mediterranean region in general have a considerable shortage of mental health resources for young people (13). Thus, new approaches are needed to improve health care access for children with ADHD.
Several methods have been implemented worldwide to improve access to paediatric mental health services in general (14) and for ADHD specifically, including tele-health services (15) and collaborative care models (16,17). This paper presents the results of a randomized controlled trial that examined the effectiveness of collaborative tele-mental health for children with ADHD in primary health care in Dubai: Effectiveness of Collaborative Tele-Mental Health Services for ADHD in Primary Care (ECTSAP) (ClinicalTrials.gov Identifier: NCT03559712) (18). The outcomes of the first phase of the ECTSAP trial are reported, which involved the development, implementation and evaluation of an intensive ADHD training programme tailored to the needs of primary health care physicians for the purpose of implementing a collaborative care model. The outcomes in a sample of children with ADHD randomized to receive care from the trained primary health care physicians (experimental arm) or standard care are examined.
Methods
Training of primary care physicians
A 35-hour ADHD course was designed and delivered. Pre- and post-course assessments were done, including multiple-choice questions and structured clinical stations to assess the participant’s knowledge and clinical interviewing skills. A confidence survey consisting of six items, five close-ended questions (Likert scale) and one open-ended question, was administered at baseline and at the end of the training programme.
Patients and eligibility
Children aged 6–12 years attending primary health care centres who met the DSM-5 criteria for ADHD (1), as determined by both clinical assessment and meeting the threshold on the Vanderbilt ADHD rating scale (19) in two or more settings, were enrolled in the study. Children with cardiac disorders, seizures, autism spectrum disorder, intellectual disability, and active primary psychiatric illness other than ADHD were not eligible for this study. Verbal consent was obtained from patients and signed consent was obtained from the parents. Randomization was done using a computer-generated randomization code at the primary health care centres at the time of the initial consultation to determine eligibility for the study. Individual allocation was sealed in sequentially numbered opaque envelopes which were opened after consent was obtained. The randomization sequence was created using Stata 14.2 software with a 1:1 allocation using random block sizes of two and four. The blocks of two were done for 60% of the participants and blocks of four for 40% of the participants. This mixing of block sizes was done to avoid the research team guessing the allocation.
Baseline assessments included: the Vanderbilt Behavioral Assessment Scale (19); the Columbia Impairment Scale (20); the Childhood Behavior Checklist (21); and the Strength and Difficulties Questionnaire (22). Parents also completed a questionnaire on sociodemographic, clinical and medical characteristics of their child. The as-usual treatment primarily consisted of referral to specialized mental health services. The participants in this control arm also completed the same baseline, 3- and 6-month assessments and scales. Research appointments to complete outcome scales were scheduled at 3 and 6 months after recruitment into the study and included administering the Vanderbilt Behavioral Assessment Scale and the Columbia Impairment Scale for both groups.
Statistical analysis
Continuous variables are presented as means and standard deviations (SD). Categorical variables such as sex are presented as numbers and percentages. Continuous outcome variables were compared using the Student t-test. As the outcome variables such as the Vanderbilt Behavioral Assessment Scale and Columbia Impairment Scale were measured at various times (baseline, 3 months and 6 months) repeated measures analyses for longitudinal data were done with an exchangeable correlation structure. Exact P values are presented.
Ethical approval
The project was approved by the Dubai Scientific Research Ethics Committee, under the Dubai Health Authority, and exempted by Harvard University given that the study was conducted in the United Arab Emirates and Harvard University had not clinical involvement. Administrative approval was obtained from Al Jalila Children’s Specialty Hospital management and primary health care centres in the Dubai Health Authority. The study is registered under ClinicalTrials.gov, identifier: NCT03559712 (18) and was funded by the Dubai-Harvard Center for Global Health Delivery in Dubai.
Results
Stage 1: training of primary health care physicians
All physicians working in childcare clinics in primary health care centres under the Dubai Health Authority were invited to be included in the study (n = 12). Six attended the 35-hour ADHD course, three declined due to time constraints and three did not respond. Physicians who participated in the training were similar to all the physicians in terms of sex distribution: 50% of physicians enrolled were males and 58% of the total physicians were males. After the training, participants reported improved confidence in the following areas: recognizing symptoms of ADHD (mean (SD): 1.66 (0.51), P = 0.001); using clinical tools (mean (SD): 1.83 (1.16), P = 0.012); developing a treatment plan (mean (SD): 2.83 (0.98), P = 0.001); prescribing and monitoring medications (mean (SD): 2.33 (1.36), P = 0.009); and answering parents questions about ADHD (mean (SD): 1.83 (1.16), P = 0.012). In terms of knowledge, the mean percentage pre-course score in the multiple-choice questions was 42%, while the mean percentage post-course score was 78% (P = 0.001). All candidates successfully passed the structured clinical stations.
Stage 2: Randomization phase, and ongoing training
In stage 2 of the training programme, primary care physicians attended monthly live ADHD online seminars and had weekly supervision meetings via videoconferencing with a senior child psychiatrist to discuss clinical cases. The six primary care physicians who we trained started ADHD collaborative care clinics that were distributed across different areas of Dubai. All primary care physicians under the Dubai Health Authority were informed about these research clinics. Once the referral by the primary health care doctor was made to an ADHD clinic, the child was screened by the research psychologist for eligibility criteria, consent was obtained, and the child was randomly assigned to one of the two study arms, either assessment and treatment by a trained primary care physician (intervention arm), or referral to mental health services (control arm).
Sample characteristics and findings
A total of 112 patients were referred to the collaborative care ADHD clinics, and 18% of the referred sample (n = 20) were eligible for inclusion in the study. The most common reason for non-eligibility was age younger than 6 years (44%), not meeting ADHD criteria (29%) and presence of autism comorbidity (12%). Of the 20 children enrolled, 10 were allocated to the intervention arm and 10 to the control arm. Figure 1 is a flowchart of the recruitment of patients into the study and Table 1 summarizes the baseline clinical characteristics of the sample and the Childhood Behavior Checklist scores.
The drop-out rate at 6 months in the control group was 80% compared with 50% in the intervention group.
The mean difference for the outcome in the Childhood Behavior Checklist internalizing scores was 12.3 (95% CI: –1.6 to 26.3) suggesting that the intervention arm had higher scores than the control arm (P = 0.078). Similarly, the difference in the Childhood Behavior Checklist externalizing score was 11.9 (95% CI: 0.9 to 22.9) which was significantly higher in the intervention arm (P = 0.036). The mean difference in the Childhood Behavior Checklist total score was 12.4 (95% CI: 0.5 to 24.4), again significantly higher in the intervention arm (P = 0.042).
The efficacy analyses, that is, the mean difference and 95% confidence intervals (CI) of the outcome variables (Vanderbilt scores and performance scores) between the two arms are presented in Table 2. The table also presents the findings of the repeated measures regression analyses for Vanderbilt outcomes as measured at baseline, 3 month and 6 months. In the outcome Vanderbilt scale, although the score in the intervention arm increased over time – mean difference 2.0 (95% CI: –6.5 to 10.5) – the difference was not statistically significant. Similarly, the outcome performance declined in the control arm at 6 months completely, as compared with the intervention arm. However, the mean difference over time was not statistically significant: –0.719 (95% CI: –2.2 to 0.81). The mean (SD) of waiting time was 1.3 (4.0) weeks in the experimental group which was significantly shorter compared with 7.1 (6.3) weeks in the control arm (P = 0.026).
Discussion
This paper describes ECTSAP, a clinical trial in Dubai to enhance access to mental health care in primary health care centres for children with ADHD using a collaborative tele-mental health approach. Primary health care physicians were successfully recruited and trained for this study, despite many obstacles, including busy schedules. As a result, ECTSAP trained physicians were able to assess and follow up children with ADHD through the collaborative care model. However, this result should be viewed with caution given the limited sample size and slow recruitment. The higher drop-out rate in the control arm may suggest that parents are either going elsewhere or giving up on having their children assessed and treated because of the very long wait times; this observation warrants further exploration. Despite the small size of the patient sample so far, its demographic and clinical characteristics (predominantly male, high oppositional defiant disorder and anxiety comorbidity) are similar to larger samples from other countries (23).
Our study has a number of limitations. First, the sample size of six trainees was small which raises questions about the generalizability of the findings. However, it is important to note that this number was 50% of the eligible physicians for our study and therefore represents good engagement and commitment. Furthermore, the patient sample was recruited through primary health care centres and recruiting from the community would improve the representativeness of the sample. One of the most surprising findings was the slow recruitment rate of patients so far. This may reflect existing barriers that require further investigation, such as public awareness and stigma.
An important finding is that the wait-time for assessment and treatment was significantly shorter in the intervention arm. This finding, together with the higher drop-out rate in the control arm, indicates that such innovative approaches can solve some challenges in the health care system, including long wait times for clinical services. Furthermore, the comparable outcomes in the two groups indicate that primary care physicians who are trained on ADHD and work collaboratively with experienced clinicians may be able to effectively diagnose and treat ADHD. However, the small sample size and limitations of our study must be borne in mind in relation to this finding.
Acknowledgement
We thank Dr Manal Taryam, CEO of Primary Healthcare in the Dubai Health Authority, and the management of Al Jalila Children’s Specialty Hospital for facilitating this clinical trial. We also thank Dr Haitham Mahmoud, Dr Idris Alhmid and Dr Aditya Garg from the Dubai Health Authority. Furthermore, we thank the Harvard Center for Global Health Delivery in Dubai for providing research and statistical support for this study, especially Dr Subhash Chandir for statistical support, Dr Jerome Galea for research methodology input, and Dr Nasreen Adamjee and Ms Rachele Cox for administrative support. Finally we thank primary health care Nurse Elize George from the Dubai Health Authority for coordination support.
Funding: This study was funded by a research grant from the Harvard Center for Global Health Delivery in Dubai (No. 027562-746845-0105).
Competing interests: None declared.
References
1. Diagnostic and statistical manual of mental disorders (DSM-5®). Arlington, VA: American Psychiatric Association; 2013.
2. Polanczyk G, de Lima MS, Horta BL, Biederman J, Rohde LA. The worldwide prevalence of ADHD: a systematic review and metaregression analysis. Am J Psychiatry. 2007;164:942–8. https://doi.org/10.1176/ajp.2007.164.6.942
3. Barbaresi WJ, Colligan RC, Weaver AL, Voigt RG, Killian JM, Katusic SK. Mortality, ADHD, and psychosocial adversity in adults with childhood ADHD: a prospective study. Pediatrics. 2013;131:637–44. https://doi.org/10.1542/peds.2012-2354
4. Biederman J, Petty CR, Woodworth KY, Lomedico A, Hyder LL, Faraone SV. Adult outcome of attention-deficit/hyperactivity disorder: a controlled 16-year follow-up study. J Clin Psychiatry. 2012;73:941–50. https://doi.org/10.4088/JCP.11m07529
5. Erskine HE, Ferrari AJ, Polanczyk GV et al. The global burden of conduct disorder and attention-deficit/hyperactivity disorder in 2010. J Child Psychol Psychiatry. 2014;55:328–336. https://doi.org/10.1111/jcpp.12186
6. MTA CG. National Institute of Mental Health Multimodal Treatment Study of ADHD follow-up: changes in effectiveness and growth after the end of treatment. Pediatrics. 2004;113:762–769. https://doi.org/10.1542/peds.113.4.754
7. Fridman M, Banaschewski T, Sikirica V, Quintero J, Chen KS. Access to diagnosis, treatment, and supportive services among pharmacotherapy-treated children/adolescents with ADHD in Europe: data from the Caregiver Perspective on Pediatric ADHD survey. Neuropsychiatr Dis Treat. 2017;13:947–58. https://doi.org/10.2147/NDT.S128752
8. Tettenborn M, Prasad S, Poole L, Steer C, Coghill D, Harpin V, et al. The provision and nature of ADHD services for children/adolescents in the UK: results from a nationwide survey. Clin Child Psychol Psychiatry. 2008;13:287–304. https://doi.org/10.1177/1359104507086347
9. Sikirica V, Flood E, Dietrich CN, Quintero J, Harpin V, Hodgkins P, et al. Unmet needs associated with attention-deficit/hyperactivity disorder in eight European countries as reported by caregivers and adolescents: results from qualitative research. Patient. 2015;8:269–81. https://doi.org/10.1007/s40271-014-0083-y
10. Population by gender and age groups – Emirate of Dubai. Dubai: Government of Dubai; 2019 (https://www.dsc.gov.ae/Report/DSC_SYB_2018_01%20_%2005.pdf).
11. Eapen V, Mabrouk AA, Zoubeidi T, Sabri1 S, Yousef S, Al-Ketbi J, et al. Epidemiological study of attention deficit hyperactivity disorder among school children in the United Arab Emirates. Hamdan Med J. 2009;2:119–27.
12. Dubai annual health statistical report – 2018. Dubai: Government of Dubai; 2019. https://www.dha.gov.ae/uploads/122021/ffd2f271-6e00-481e-9657-57bc5f05b736.pdf
13. Rahman A, Hamoda HM, Rahimi-Movaghar A, Murad M, Saeed K. Mental health services for youth in the Eastern Mediterranean Region: challenges and opportunities. East Mediterr Health J. 2019;25(2):80–1. https://doi.org/10.26719/2019.25.2.80
14. Kolko DJ, Campo J, Kilbourne AM, Hart J, Sakolsky D, Wisniewski S. Collaborative care outcomes for pediatric behavioral health problems: a cluster randomized trial. Pediatrics. 2014;133:e981–92. https://doi.org/10.1542/peds.2013-2516
15. Myers K, Vander Stoep A, Zhou C, McCarty CA, Katon W. Effectiveness of a telehealth service delivery model for treating attention-deficit/hyperactivity disorder: a community-based randomized controlled trial. J Am Acad Child Adolesc Psychiatry. 2015;54:263–74. https://doi.org/10.1016/j.jaac.2015.01.009
16. Silverstein M, Hironaka LK, Walter HJ, Feinberg E, Sandler J, Pellicer M, et al. Collaborative care for children with ADHD symptoms: a randomized comparative effectiveness trial. Pediatrics. 2015;135:e858–67. https://doi.org/10.1542/peds.2014-3221
17. Myers K, Stoep AV, Thompson K, Zhou C, Unützer J. Collaborative care for the treatment of Hispanic children diagnosed with attention-deficit hyperactivity disorder. Gen Hosp Psychiatry. 2010;32:612–4. https://doi.org/10.1016/j.genhosppsych.2010.08.004
18. Effectiveness of Collaborative Tele-Mental Health Services for ADHD in Primary Care (ECTSAP). ClinicalTrials.gov. Identifier: NCT03559712. ClinicalTrials; 2018 (https://clinicaltrials.gov/ct2/show/study/NCT03559712?show_desc=Y#desc).
19. Wolraich ML, Lambert W, Doffing MA, Bickman L, Simmons T, Worley K. Psychometric properties of the Vanderbilt ADHD diagnostic parent rating scale in a referred population. J Pediatr Psychol. 2003;28:559–67. https://doi.org/10.1093/jpepsy/jsg046
20. Singer JB, Eack SM, Greeno CM. The Columbia Impairment Scale: factor analysis using a community mental health sample. Res Soc Work Pract. 2011;21:458–68. https://doi.org/10.1177/1049731510394464
21. Bellina M, Brambilla P, Garzitto M, Negri GA, Molteni M, Nobile M. The ability of CBCL DSM-oriented scales to predict DSM-IV diagnoses in a referred sample of children and adolescents. Eur Child Adolesc Psychiatry. 2013;22:235–46. https://doi.org/10.1007/s00787-012-0343-0
22. Goodman R. Psychometric properties of the strengths and difficulties questionnaire. J Am Acad Child Adolesc Psychiatry. 2001;40:1337–45. https://doi.org/10.1097/00004583-200111000-00015
23. A 14-month randomized clinical trial of treatment strategies for attention-deficit/hyperactivity disorder. The MTA Cooperative Group. Multimodal Treatment Study of Children with ADHD. Arch Gen Psychiatry. 1999;56:1073–86. https://doi.org/10.1001/archpsyc.56.12.1073
Figure 1. Flowchart of recruitment into the clinical trial: Effectiveness of Collaborative Tele-Mental Health Services for ADHD in Primary Care
ADHD: attention-deficit hyperactivity disorder.
Table 1. Baseline characteristics of the sample
Table 2. Outcomes of Effectiveness of Collaborative Tele-Mental Health Services for ADHD in Primary Care trial
The evidence governance system in Oman: what is it, why is it needed, and the current situation
Sultana Al Sabahi1, Kamila Al-Alawi2, Alaa Hashish2, Jean Jabbour2
1Centre of Studies and Research, Ministry of Health, Muscat, Oman (Correspondence: S. Al Sabahi:
Abstract
Background: Policy-makers do not just consider the general question about what works; they also consider whether it will work in their context, and take into account social values such as affordability, acceptability, equity, equality, and human rights.
Aims: To explain the importance of having a system to govern evidence to inform policies, and reflect on the efforts of the Omani system to support evidence-informed policy-making. Methods: We reviewed the literature and analysed local documents.
Results: Because of the political nature of policy-making, good evidence for policy is judged by its relevance to the policy issue at stake.
Conclusion: Evidence-informed policy-making should focus on building a system for the good governance of evidence to ensure that rigorous, systematic, and technically valid evidence is used within policy-making processes.
Keywords: evidence-informed decision-making, evidence-informed policy-making, good governance, Oman
Citation: Al Sabahi S, Al-Alawi K, Hashish A, Jabbour J. The evidence governance system in Oman: what is it, why is it needed, and the current situation. East Mediterr Health J. 2023;29(8):xxx-xxx http://doi.org/10.26719/emhj.20.xxx Received: 28/09/22, Accepted: 22/12/22
Copyright: © Authors; licensee World Health Organization. EMHJ is an open access journal. All papers published in EMHJ are available under the Creative Commons Attribution Non-Commercial ShareAlike 3.0 IGO licence (CC BY-NC-SA 3.0 IGO; https://creativecommons.org/licenses/by-nc-sa/3.0/igo).
Introduction
It is recognized that for any action and decision, information is needed to determine whether the stated goals have been achieved or to guide the selection of potential options to achieve those goals. Evidence is what provides this information. Evidence-informed policy-making refers to using the best-available evidence to inform policies systematically and transparently (1). Despite the vast amount of evidence, which varies between personal experience, tacit knowledge, and the more systematic findings from organized professional research, advocates of evidence-informed policy-making have only promoted scientific evidence as the form best suited to guiding policy-making (2). However, the concept of evidence-informed policy-making has evolved to better address the political aspects associated with decision-making (e.g. interest groups, public opinion, economic situation, and role of institutions). This article aims to: (1) clarify the difference between evidence-based medicine and evidence-informed policy-making; (2) explain the importance of systems to govern evidence to inform policies; and (3) reflect on the efforts of the system in Oman to support evidence-informed policy-making. This is crucial to countries in the process of bringing evidence-informed policy-making into operation, and it is timely for Oman as the Government announced on 13 November 2022 establishment of a decision-making support unit under the General Secretary of the Council of Ministers (3).
Evidence-based medicine versus evidence-informed policy-making
Evidence-based medicine focuses on research to evaluate the effectiveness of interventions (4). There is a hierarchy of research evidence, with meta-analyses and randomized controlled trials considered to be the gold standard (5). The value placed on evidence-based medicine and its perceived success have provoked application of its basic principles to policy development (4), and there have been increased calls to use scientific evidence in policy-making (6).
Policy-makers need to consider whether their policies will work in their particular context (7). Various social factors may not be feasible for experimentation; for example, affordability, acceptability, equity, equality, and human rights (8). While good evidence in medicine is judged by its hierarchy, good evidence for policy-making is judged by its relevance to the issue at stake and then by its methodological quality (9). This is because evidence hierarchies were originally designed to judge the effectiveness of interventions rather than reflecting policy importance or relevance (8). Despite the difference in the approach to selecting what is considered good evidence for medicine and policy-making, evidence-based medicine and evidence-informed policy-making emphasize the importance of selecting, assessing, and synthesizing evidence systematically and transparently (10).
A system to govern the use of evidence in policy-making
To understand the need for a system to govern the use of evidence in policy-making, we need first to understand the political nature of policy-making. Politics is about who gets what, when, and how (11). Allocating scarce resources and setting priorities are the main concerns with policy-making (12). Hence, policy-making involves trade-offs between multiple, potentially competing social values, and it is considered to be a political process (12). This feature differentiates policy decisions from technical exercises, which have, in most circumstances, a single agreed outcome. In contrast, society has no consensus about which social outcomes are paramount or how to judge their particular value. The lack of agreement in the desired outcomes creates the potential for misuse or bias in using evidence to inform policies. The political bias in utilizing evidence might occur during: (1) evidence creation (e.g. manipulation of studies, or strategic design to achieve a desired outcome, even if the research is conducted in rigorous and valid ways, bias might occur in selecting which issues to research, how to study them, and what questions to ask within a study) (13); (2) evidence selection (e.g. ‘cherry-picking’ and strategic review of data to justify a predetermined position) (14); or (3) evidence interpretation (e.g. interpreting methodological rigour as an indication of policy relevance, or judging the quality of evidence based on the method alone) (15).
Evidence-informed policy-making should not focus on merely acquiring more evidence. Instead, it should focus on building a system to enhance the governance of evidence to inform the policy-making process in a systematic, rigorous, and technically valid way (8). These processes should be inclusive of, representative of, and accountable to the multiple social interests of the population served (8). Institutionalization of evidence-informed policy-making through a governance system is important to ensure system-wide and deeply rooted change in utilizing evidence. It should also avoid limitations associated with strategies that focus on individuals (e.g. connecting policy-makers and researchers, providing decision-makers with evidence, or providing training to individuals to broker knowledge) who might, over time, change roles or leave their positions (16).
In many countries, such a system has been created through building organizations that work to enhance the utilization of evidence in policy-making. These organizations can exist in different settings (e.g. as independent organizations, within universities, and as part of government departments) and have been called different names (think tanks, government-support organizations, and research centres) (15). It is important to note that institutions are no longer limited to simply being the administrative bodies or physical structures within a system. Currently, institutions are presented in the practices they embark upon, the rules by which they function, and the discursive narratives by which their work is comprehended (17). Therefore, creating a system to improve evidence utilization could be through construction or changing the existing structures that are concerned with evidence utilization, or by introducing changes to the principles by which those institutions function (8).
Components of evidence governance system in Oman
In Oman, the government has shown increasing interest in research and innovation in the last few decades. Several health centres in the health and non-health sectors have been established (e.g. Centre of Studies and Research at the Ministry of Health and the Medical Research Centre at Sultan Qaboos University). The Research Council (which is currently under the Ministry of Higher Education Research and Innovation) was established to be the funding body for research and innovation. The Government has reflected its interest in utilizing research findings through the Oman 2040 Vision, which emphasizes the importance of achieving their goals through sound and positive guidance and systematic evidence-based planning, and the importance of a knowledge-based economy (18). The Ministry of Higher Education Research and Innovation announced for 2022 a research funding programme that will target priority areas at the Ministry of Health and Ministry of Agriculture, Fisheries, Water, and Water Resources. This programme mandates the involvement of policy-makers in selecting priories, priority research to be funded, and developing a plan to implement the research findings. The Government of Oman has reinforced its interest in evidence-informed policy-making through its intention to establish a unit to support decision-making (announced 9 November 2021 and officially established on 13 November 2022) (3).
Oman has some institutional structures to support the utilization of evidence. For instance, the Ministry of Health has tasked its policy advice officers and technical advisors with planning, providing policy options, or commenting on proposed policies. These strategies may provide direct and well-integrated channels for expert advice, yet the appropriate capacity and capability remain challenging for such bodies. Another challenge is the lack of clarity in the mandate for utilizing evidence and limiting the scope of subjects for which such bodies can function. On other occasions, the Government convenes expert panels or technical working groups (e.g., Supreme COVID-19 Committee and Oman 2040 Vision Committee) to inform specific decisions, giving such groups different levels of autonomy.
There is evidence that policy-makers recognized the importance of evidence and used it to inform policies (19). However, it was not clear how that evidence was selected and synthesized, and in what way it informed policies. It is unclear which social values guided the policy decisions and whose voices were heard. These rules, norms, and procedures are important to achieve the core principle of evidence-informed policy-making (i.e. transparent and systematic utilization of evidence to serve the social values of the population). By institutionalization of the practice of evidence-informed policy-making in such approaches, we can envision how systems will mature and how evidence utilization will be improved.
Conclusion
Evidence-informed policy-making is the practice of informing policies with the best available evidence in a systematic and transparent manner. Policy-making is a struggle over different ideas and social values; therefore, scientific evidence and social values have both to be considered. Unlike evidence-based medicine, scientific evidence cannot be merely judged by its rigorous methods to inform the policy-making process. Instead, relevance is more important. As a result of the political nature of policy-making, bias in creating, selecting, or interpreting evidence can occur and affect achievement of the intended social goal. Therefore, efforts to support evidence-informed policy-making should be shifted from focusing on individuals to institutions. The institutions should consider the physical structure as well as the rules, norms, and regulations that mandate the good use of evidence in policy-making. Oman has shown interest in utilizing evidence to support policy-making. Different bodies have been working on different rules to support evidence-informed policy-making; however, the Government of Oman needs to create the mandates, rules, and regulations that ensure the institutionalization of good practice in utilizing evidence in policy-making. Besides establishing formal structures, the Government of Oman should focus on creating procedures, norms, rules, or even laws that reduce bias or improve reasonable scientific practice in policy-making. If the Government of Oman takes into consideration the points raised in this article when bringing into operation a decision-making support unit, it will help it take robust steps towards achieving its goal of informing public policies with the best-available evidence.
References
1. Annual Report 2017. EVIPNet Europe. Towards a world in which the best available research evidence informs policy-making (http://www.euro.who.int/__data/assets/pdf_file/0004/368833/Annual-report-2017-for-EVIPNet-Europe.pdf?ua=1, accessed 28 March 2023).
2. Iphofen R, O’Mathúna D. Ethical evidence and policymaking: interdisciplinary and international research. Bristol: Policy Press; 2022.
3. To amend the organizational structure of the General Secretariat of the Council of Ministers 2022. Oman: Ministry of Justice and Legal Affairs; 2023 (https://www.mjla.gov.om/legislation/gazettes/details.aspx?Id=626&type=G, accessed 20 March 2023).
4. Sheldrick RC, Hyde J, Leslie LK, Mackie T. The debate over rational decision making in evidence-based medicine: implications for evidence-informed policy. Evid Policy. 2021;17(1):147–59. https://doi.org/10.1332/174426419X15677739896923
5. Djulbegovic B, Guyatt GH. Progress in evidence-based medicine: a quarter century on. Lancet. 2017 Jul 22;390(10092):415–23. https://doi.org/10.1016/S0140-6736(16)31592-6 PMID:28215660
6. Godziewski C. Evidence and power in EU governance of health promotion: discursive obstacles to a “health in all policies” approach. J Common Market Stud. 2020 Sep;58(5):1307–24. https://doi.org/10.1111/jcms.13042
7. Cairney, P., & Oliver, K. (2017). Evidence-based policymaking is not like evidence-based medicine, so how far should you go to bridge the divide between evidence and policy?. Health research policy and systems, 15(1), 1-11.
8. Parkhurst J. The politics of evidence: from evidence-based policy to the good governance of evidence. Routledge; 2017.
9. Parkhurst JO, Abeysinghe S. What constitutes “good” evidence for public health and social policy-making? From hierarchies to appropriateness. Soc Epistemol. 2016;30(5–6):665–79. https://doi.org/10.1080/02691728.2016.1172365
10. Shaw L, Nunns M, Briscoe S, Anderson R, Thompson Coon J. A “Rapid Best‐Fit” model for framework synthesis: using research objectives to structure analysis within a rapid review of qualitative evidence. Res Synth Meth. 2020 May;12(3):368–83. https://doi.org/10.1002/jrsm.1462 PMID:33006277
11. Bhan T. Deserving poor in public sanitation: tracing the policymaking processes of who gets what, when, how, and why in Delhi. Environ Plan B Urban Analyt City Sci. 2022;49(8):2151–67. https://doi.org/10.1177/23998083221089325
12. Woods B, Rothery C, Revill P, Hallett TB, Phillips AN, Claxton K, editors. Setting research priorities in Global Health: Appraising the value of evidence generation activities to support decision-making in health care. York: Centre for Health Economics; 2018.
13. Lundh A, Lexchin J, Mintzes B, Schroll JB, Bero L. Industry sponsorship and research outcome. Cochrane Database Syst Rev. 2017 Feb 16;2(2):MR000033. https://doi.org/10.1002/14651858.MR000033.pub3 PMID:28207928
14. Schoofs EL, Fode M, Capogrosso P, Albersen M, Group ftEAoUYA. Current guideline recommendations and analysis of evidence quality on low-intensity shockwave therapy for erectile dysfunction. Int J Impot Res. 2019 May;31(3):209–17. https://doi.org/10.1038/s41443-019-0132-0 PMID:30911110
15. Al Sabahi S, Wilson MG, Lavis JN, El-Jardali F, Moat K. Examining and contextualizing approaches to establish policy support organizations – a mixed method study. International J Health Policy Manag. 2021 Aug 7;11(9):1788–800. https://doi.org/10.34172/ijhpm.2021.86 PMID:34380206
16. Al Sabahi S, Wilson MG, Lavis JN, El-Jardali F, Moat K, Vélez M. Examining and contextualizing approaches to establish policy support organizations – a critical interpretive synthesis. Int J Health Policy Manag. 2022 May 1;11(5):551–66. https://doi.org/10.34172/ijhpm.2020.181 PMID:33008260
17. Lowndes V, Roberts M. Why institutions matter. The new institutionalism in political science. Bloomsbury; 2013.
18. Oman Vision 2040. Moving forward with confidence. Oman: Ministry of Finance; 2020 (https://www.mof.gov.om/pdf/Vision_Documents_En.pdf, accessed 20 March 2023).
19. Al Sabahi S, Wilson MG, Lavis JN, El-Jardali F, Moat K. Insights from system leaders about operationalising a knowledge translation department in the Oman Ministry of Health. Evid Policy J Res Debate Pract. 2021;18(1):85–107. https://doi.org/10.1332/174426421X16123709152129
Forecasting daily confirmed COVID-19 cases in Algeria using ARIMA models
Messis Abdelaziz1,2, Adjebli Ahmed3, Ayeche Riad4, Ghidouche Abderrezak2; Ait-Ali Djida2
1Université de Bordj Bou Arréridj, El-Anasser, 34010, Bordj Bou Arréridj, Algérie; 2Laboratoire de Génie Biologique des Cancers, Université de Bejaia 06000, Bejaia, Algérie; 3Laboratoire d’Ecologie Microbienne, faculté des sciences de la nature et de la vie, université de Bejaia, 06000, Bejaia, Algérie; 4Laboratoire Caractérisation et Valorisation des Ressources Naturelles, Université de Bordj Bou Arreridj, 34010, El-Anasser, Bordj Bou Arréridj, Algérie. (Correspondence to:
ABSTRACT
Background: COVID-19 has become a worldwide threat affecting every country.
Aims: This study aimed to identify COVID-19 cases in Algeria using times series models for forecasting COVID-19.
Methods: Confirmed COVID-19 daily cases data were obtained from 21 March 2020 to 26 November 2020 from the Algerian Ministry of Health. Forecasting was done using the Autoregressive Integrated Moving Average (ARIMA) models (0,1,1) with Minitab 17 software.
Results: Observed cases during the forecast period were accurately predicted and placed within prediction intervals generated by ARIMA. Forecasted values of COVID-19 positives, recoveries and deaths showed an accurate trend, which corresponded to actual cases reported during 252, 253 and 254 days. Results were strengthened by variations of less than 5% between forecast and observed cases in 100% of forecasted data.
Conclusion: ARIMA models with optimally selected covariates are useful tools for predicting COVID-19 cases in Algeria.
Keywords: COVID-19, time series, double exponential smoothing, ARIMA, forecast, Algeria
Citation: Abdelaziz M, Ahmed A, Riad A, Abderrezak G2, Djida Ait-Ali. Forecasting daily confirmed COVID-19 cases in Algeria using ARIMA models. East Mediterr Health J. 2023;29(5):xxx–xxx. https://doi.org/10.26719/emhj.23.054
Received: 13/03/21; accepted: 08/12/22
Copyright: ©Authors; Licensee: World Health Organization. EMHJ is an open access journal. All papers published in EMHJ are available under the Creative Commons Attribution Non-Commercial ShareAlike 3.0 IGO licence (CC BY-NC-SA 3.0 IGO; https://creativecommons.org/licenses/by-nc-sa/3.0/igo).
Introduction
On 11 March 2020, the World Health Organization (WHO) declared COVID-19 as a pandemic. The pandemic had spread from mainland China to other countries and territories, disrupting socioeconomic activities. As of 26 November 2020, COVID-19 had infected more than 60 776 978 people globally, killed more than 1 428 228, and resulted in a lockdown that forced people to stay in their homes (1).
Algeria reported its first COVID-19 case on 25 February 2020. By 26 November, it had reported 79 110 confirmed cases, 51 334 recoveries and 2352 deaths (2).
SARS-CoV-2, the COVID-19 virus is very infectious, and many people were not following the non-pharmaceutical public health prevention measures recommended by the Algerian government and other governments to control the pandemic, thus increasing the risks of transmission (2,3).
Accurate forecasting of COVID-19 case trends was essential for preparedness by health authorities to manage the pandemic and resource planning. Time series models such as ARIMA have been widely used to statistically model and forecast infectious disease trends (4). ARIMA models are preferred in this context because they are suitable for investigating short-term effects of acute infectious diseases and are flexible and appropriate for several trajectories (4,5). ARIMA models have been used in several studies to forecast COVID-19 outbreak trends (6-9).
In this study, we developed ARIMA models using daily COVID-19 confirmed and active cases in Algeria to identify the best fitting model of COVID-19 cases from 21 March 2020 to 26 November 2020.
Materials and methods
Data source
Data for this study included confirmed COVID-19 daily cases data obtained from 21 March 2020 to 26 November 2020 from the Algerian Ministry of Health (10).
Methods
The following equation highlights the exponential smoothing method and the ARIMA processes (11):
ARIMA model for time series sata ARIMA model is stated as follows:
Where:
ɸ(B) is an autoregressive operator
Ө(B) is a moving average operator
(1 − B)d is a differencing operator. It is the expression of dth consecutive differencing so as to make the series stationary
Zt is a Gaussian white noise series with mean zero and variance (σ2z).
ARIMA forecast is based on previous values and portrayed by 3 terms – p, d, q. Where p is the order for the auto regressive expression (AR), q is the order for the moving average expression (MA) and d is the number of differencing required making the time arrangement fixed.
The experiment was carried out using Minitab 17 programming software (12). In general, the equation can be approached using a regression model:
εt = errors from the accompanying conditions.
Results
Using the time-series model approach, the pattern of COVID-19 data distribution in Algeria showed an exponential distribution pattern, where the addition of positive cases increased significantly everyday of the pandemic. The distribution pattern was the same for the number of people who recovered and died (Figure 1). For the positive COVID-19 cases, the mean absolute percentage error (MAPE) value was smaller than the error rate at 10% (Table 1). The increase in the number of people who were positive for COVID-19 directly affected the prediction model for patients who recovered and died (Figure 1). For recovered cases, MAPE value was smaller than the error value set at 10% error rate (Table 1). The recovery rate for COVID-19 patients increased simultaneously with the number of positive cases because of the non-pharmaceutical public health measures taken by the government from 21 March 2020. For deaths due to COVID- 19, the MAPE value was greater than the error value set at 10% (Table 1). The increase in mortality was possibly due to the extent of infection and the medical history of the patients.
In the time series model with 5% error probability (α), the graph followed the ARIMA process (0,1,1), with the P value MA 1 (0.0%) smaller than α.
Estimated results of parameters model for COVID-19 positive data using ARIMA models:
Referring to equation (4), mathematically, the ARIMA model (0,1,1) can be stated using the following coefficients:
γt = 317.65 − 0.879et-1
Same as COVID-19 positive data, in the time series model with 5% error probability (α), the graph followed the ARIMA process (0,1,1) with the P value MA 1 (0,0%) smaller than α.
Estimated of parameters for COVID-19 Recovery data results using ARIMA model:
Referring to equation (4), mathematically the ARIMA model (0,1,1) can be stated as follows: γt = 205.53 − 0.30et-1
After the positive and recovery data were analysed in time series model with the 5% error probability (α), the graph followed the ARIMA process (0,1,1) with the P value MA 1 (0.00%) smaller than α.
All estimated parameter results of the ARIMA model:
Referring to equation (4), mathematically, the ARIMA model (0,1,1) can be stated.
The results of predictions of COVID-19 cases in Algeria (positive, recovered and deaths) showed a gap in the resulting distribution patterns, where the increase in the number of positive cases was not offset by an increase in the number of patients who recovered and a decrease in the number of patients who died. This indicates that public behaviour did not comply with the rules set by the government (physical distancing, large-scale social restrictions, washing of hands, and mask use).
Discussion
From the time WHO declared COVID-19 a pandemic on 11 March 2020, several countries experienced an exponential increase in COVID-19 cases (3), which put a lot of pressure on most healthcare systems worldwide. In response, health authorities attempted to forecast the trend of the pandemic, but this proved difficult because COVID-19 is a novel disease with limited data and knowledge about its trends and dynamics (2). Our forecast showed an accurate trend, which corresponded to the number of positive cases observed and reported by the Ministry of Health in Algeria during three days (252, 253 and 254). The same situation was observed for forecasted recoveries and deaths.
This finding was strengthened by variations of less than 5% between the forecast and observed cases in 100% of the forecasted data points (Table 2). Similar studies conducted in South Korea, Iran and Italy predicted similar case trends using ARIMA models (6-8).
The strengths of this study include: firstly, this is the first paper to report the use of ARIMA models to forecast COVID-19 cases and trends in Algeria. Secondly, this was the first attempt to use smoothen case data to improve accuracy as compared to similar studies on ARIMA models for COVID-19 conducted in other countries (6-7). Thirdly, we used several independent covariates, which provided more accurate signals to develop short-term model predictions for immediate outbreak response. Finally, we optimized the model training and validation period to provide the highest number of data points to generate the best fit model.
Conclusion
This study demonstrates the effectiveness of ARIMA models as an early warning strategy that can provide accurate COVID-19 forecasts on larger data points (251 days). Forecasted values of COVID-19 positives, recoveries and deaths showed an accurate trend which corresponded to the actual cases observed and reported by the Ministry of Health in Algeria during three days (252, 253 and 254). We are confident that the ARIMA model can be used to generate accurate and reliable forecasts of daily COVID-19 cases until the end of COVID-19, with the addition of new data points and independent covariates.
Declaration of competing interest
I have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgement
The authors would like to thank the Ministry of Health, Population and Hospital Reform of Algeria and the Johns Hopkins University for publicly releasing the updated COVID-19 datasets.
References
- World Health Organization. Coronavirus disease (COVID-19) outbreak situation. Geneva: World Health Organization. https://www.who.int/emergencies/diseases/novel-coronavirus-2019.
- Johns Hopkins University. Novel coronavirus (COVID-19) cases. Baltimore: Johns Hopkins University. https://coronavirus.jhu.edu/region/algeria.
- Lounis M. Epdemiology of coronavirus disease 2020 (COVID-19) in Algeria. New Microbes and New Infections 2021;39:100822. DOI: https://doi.org/10.1016/j.nmni.2020.100822.
- Allard R. Use of time-series analysis in infectious disease surveillance. Bull World Health Organ 1998;76:327–333.
- Imai C, Hashizume M. A systematic review of methodology: Time series regression analysis for environmental factors and infectious diseases. Trop Med Health 2015;43:1–9. DOI: https://doi.org/10.2149/tmh.2014-21.
- Benvenuto D, Giovanetti M, Vassallo L, Angeletti S, Ciccozzi M. Application of the ARIMA model on the COVID-2019 epidemic dataset. Data in Brief 2020;29:1-4. DOI: https://doi.org/10.1016/j.dib.2020.105340.
- Chintalapudi N, Battineni G, Amenta F. COVID-19 virus outbreak forecasting of registered and recovered cases after sixty-day lockdown in Italy: A data driven model approach. J Microbiol Immunol Infect 2020;53:396-403. DOI: https://doi.org/10.1016/j.jmii.2020.04.004.
- Singh S, Sundram BM, Rajendran K, Law KB, Aris T, Ibrahim H, Dass SC, Gill BS. Forecasting daily confirmed COVID-19 cases in Malaysia using ARIMA models. J Infect Dev Ctries 2020;14. DOI: https://doi.org/10.3855/jidc.13116.
- Singh RK, Rani M, Bhagavathula AS, Sah R, Rodriguez-Morales AJ, Kalita H, Nanda C, Sharma S, Sharma YD, Rabaan AA, Rahmani J, Kumar P. Prediction of the COVID-19 pandemic for the top 15 affected countries: Advanced Autoregressive Integrated Moving Average (ARIMA) model. JMIR Public Health Surveill 2020;6. DOI: https://doi.org/10.2196/19115.
- Algerian health and hospital reform minister: carte épidémiologique. https://www.covid19.gov.dz/carte/).
- Konarasinghe KMUB. Modeling COVID-19 epidemic of USA, UK, and Russia. JNFHBS 2020;1:1-14.
- Minitab 17 Statistical Software. Minitab, Inc.: State College, PA, USA, 2010. www.minitab.com.
