PDF version

A. Youssef¹

استخدام التذكير بالرسائل القصيرة لتحسين الحضور إلى عيادة خارجية للطب الباطني في المملكة العربية السعودية: دراسة معشاة ذات شواهد

عادل يوسف

الخلاصة: يعتبر التغيب في العيادات الخارجية من المشكلات الجسيمة. وتهدف هده الدراسة إلى تقييم كغاءةإرسال تذكيرات بالرسائل القصيرة إلى الهواتف الجوالة للمرضى SMS في خفض معدلات التغيب عن المواعيد المقررة. وقد أجرى الباحث دراسة سريرية معشاة في عام 2011، في عيادة خارجية للطب الباطني في مستشفى في المنطقة الشرقية في المملكة العربية السعودية. ويد بلغ عذذ المرضى المؤهلين للاندراج ضمن الدراسة 502 مريضا، وقد تم تعيينهم عشوائياً لتلقي رسالة تذكرية قصيرة بموعدهم (مجموعة التدخل) مقابل مجموعة لم تلق أية تذكيرات (مجموعة الشواهد). واتضح أن معدل التغيب أخفض بمقدار يعتد به لدى مجموعة التذكير (26.3%) مما هو عليه لدى مجموعة عدم التذكير (39.8%). ولدى استخدام التحوف اللوجسني المتعدد المتغيرات، وجد الباحث أن العمر والجنسية لم تؤثر عل معدل الحضور، إلا أن الاتصال الأول المباشر مع المستشفى (معدل الأرجحية 7.40 = OR)، وعدم تلقي رسالة تذكرية (معدل الأرجحية 0.56 = OR) هما عاملان مهمان في التغيب. وقد سئل جميع المرضى عن آرائهم حول التذكير بالرسائل القعيرة(وعددهم 76) فأجابوا بأنهم يرغبون في مواصلة تلقيهم للتذكيرات بها في المستقبل.

ABSTRACT Non-attendance at outpatient appointments is a major problem. This study aimed to evaluate the efficacy of sending short message service (SMS) reminders to patients' mobile phones in reducing non-attendance at scheduled appointments. A randomized clinical trial was conducted in 2011 in an internal medicine outpatient clinic at a hospital in Eastern Province, Saudi Arabia. Eligible patients (n = 502) were randomly allocated to receive an SMS reminder of their appointment (intervention group) or no reminder (control group). The non-attendance rate was significantly lower in the reminder group (26.3%) than the non-reminder group (39.8%). In multivariate logistic regression, age, sex and nationality did not affect attendance rates but having their first contact with the hospital (OR = 7.40) and not receiving an SMS reminder (OR = 0.56) were significant factors in non-attendance. All patients surveyed about their perceptions of the SMS reminder (n = 76) reported they would like to continue to receive SMS reminders in the future.

Utilisation de rappels par minimessages afin d'améliorer l'assiduité des patients dans un service de consultations externes en médecine interne en Arabie saoudite : essai contrôlé randomisé

RÉSUMÉ La non-présentation des patients aux rendez-vous de consultations externes constitue un problème majeur. La présente étude visait à évaluer l'efficacité de l'envoi de rappels par minimessages (SMS) sur les téléphones portables des patients afin de réduire le nombre de rendez-vous non honorés. Un essai clinique randomisé a été mené en 2011 dans un service de consultations externes de médecine interne au sein d'un hôpital de la province orientale d'Arabie saoudite. Les patients éligibles (n=502) ont été aléatoirement répartis dans le groupe recevant un rappel de leur rendez-vous par SMS (groupe bénéficiant de l'intervention) ou dans le groupe ne recevant pas de rappel (groupe témoin). Le taux de non-présentation aux rendez-vous était significativement inférieur dans le groupe ayant bénéficié de l'intervention (26,3 %) par rapport au groupe n'ayant pas reçu de rappel (39,8 %). À l'analyse de régression logistique multivariée, l'âge, le sexe et la nationalité n'influaient pas sur les taux de non-présentation. En revanche, prendre rendez-vous pour la première fois avec l'hôpital (OR = 7,40) et le fait de ne pas recevoir de rappel par SMS (OR = 0,56) étaient des facteurs importants favorisant une non-présentation. Tous les patients interrogés pour l'enquête sur leur perception du rappel par SMS (n=76) ont indiqué qu'ils souhaitaient continuer à recevoir des rappels de leur rendez-vous par SMS à l'avenir.

¹Department of Health Information Management and Technology, College of Applied Medical Sciences, University of Dammam, Dammam, Saudi Arabia (Correspondence to: This e-mail address is being protected from spambots. You need JavaScript enabled to view it ).
Received: 28/02/13; accepted: 28/10/13
EMHJ, 2014, 20(5): 317-323 

---

Introduction

Non-attendance at outpatient clinic appointments is a worldwide problem that affects the efficiency of health-care delivery systems and results in poorer outcomes and loss of continuity of care for patients [1], in addition to creating a cycle of ever-lengthening waiting lists [2-6]. While non-attendance may be due to transport difficulties, conflict with other appointments or family responsibilities [7-10], patients cite forgetful-ness as the most important reason for non-attendance [8], as demonstrated by the success of reminders in improving patient attendance [11]. Many methods to remind patients about their appointments have been studied, including personal phone calls, automated phone calls and mailed letters [11-14]. But, despite their effectiveness, these methods have their limitations. For example, telephone reminders are costly and repeated calls may be needed to reach the patient [15,16].

Mobile phones using the short message service (SMS) are now commonplace in both developed and developing countries. Saudi Arabia is one of the fastest growing markets for mobile phone use in the Middle East, with a penetration rate standing at 198% [17]. Mobile phone messages have the advantage of being convenient, inexpensive and able to reach to the intended person immediately [18]. These characteristics allow the technology to be widely used for health information transfer and efficient and effective delivery of health-care services [19]. Mobile phone messages have been used to improve patient compliance with medications (e.g. improve adherence to asthma treatment [20]), improve follow-up ofpsy-chiatric patients (e.g. schizophrenia patients benefiting from text messages [21]) and in health promotion (e.g. to improve sunscreen use [22] and smoking cessation [23]). Reminders have also been shown to be effective in many settings, including emergency departments, primary care and pulmonary clinics [11,24,25].

Over the past few years there have been several studies (observational and interventional) that showed consistent success of SMS reminders in reducing appointment non-attendance. However, most of these studies were conducted in countries other than Saudi Arabia. A recent observational study (historical cohort) conducted in Saudi Arabia was the first to report the effectiveness of text messaging in reducing the non-attendance rate in hospital outpatient clinics [26]. However, similar to other observational studies the results may have been biased by other non-controlled factors. In a previous study we demonstrated that the effectiveness of text messaging reminders varies according to the type of outpatient clinic[27]. In this article we present the results of a randomized controlled trial (RCT) to evaluate whether SMS reminders can reduce the non-attendance rate in a hospital internal medicine outpatient clinic. Secondary aims were to assess patient perceptions about receiving reminder messages and to explore factors related to non-attendance.

Methods

Study design and setting

This was a RCT conducted at the internal medicine outpatient clinic at a major teaching hospital in Eastern Province of Saudi Arabia during the period 5 April to 4 July 2011. The internal medicine clinic was selected because it was known to have the highest patient volume with the longest waiting times for outpatient appointments. The services provided in outpatient clinics in the hospital are free of charge.

This study was approved by the institutional board review of the hospital. Patient approval was not sought because patients provide their phone number at registration with the understanding that the hospital may use the number as a contact source for the patient.

Sample

For a 2-arm RCT, a sample of 151 per group was required to be able to detect a 15% reduction due to an SMS reminder with a power of 80% and a = 5%. For a power of90%, 198 per group were needed. However, we were able to include a larger number of patients in the study.

All patients whose mobile number were documented in the hospital electronic database and were due to attend an appointment at the internal medicine outpatient clinic during the following week were eligible for the study. Because the SMS reminder messages were sent in Arabic, non-Arabic speaking patients were excluded from the study (about 10% ofthe study population). Patients younger than 18 years old were also excluded. Each week a list of patients with appointments for the following week was retrieved from the hospital electronic database. A random sample of 20 patients was selected from the list on each day of the week. Using computer-generated random numbers, these 20 patient appointments were then randomly allocated to receive a reminder text message or not to receive such a reminder (10 appointments per group). This process continued over the study period until the required sample size was collected.

Intervention

A standard text message reminder was sent manually to the intervention group using one of the mobile phone service providers in Saudi Arabia 48 hours before the scheduled appointment. It started with a greeting at the beginning of the message followed by a reminder ofthe date, clinic name and the hospital name. The text message did not include the patient's name.

Data collection

Other data collected from the electronic database included: patient's age, sex, nationality, type of visit (first contact or previous contact with the hospital) and whether the patient attended the scheduled appointment (show or no-show). The sequence of the randomization was concealed from the individual who provided the appointment list. The statistician was also blinded to the patient group.

Within 1 week after the appointment, we conducted a telephone survey ofa group ofparticipants in the intervention group regarding their perceptions toward the SMS reminder. A random sample of 100 patients was selected from the intervention group and contacted for phone interview using the phone number used during the study to send the SMS reminder. A research assistant was trained and provided with the questionnaire to conduct the phone interview.

Statistical analysis

The statistical analysis was performed using Stata, version 11. The primary outcome for the study was non-attendance rate. Patients were considered non-attenders if they were registered in the electronic system as no-show for their scheduled appointment. The non-attendance status was calculated based on intention-to-treat analysis. The non-attendance rate in the SMS-reminder group was compared with that in the non-reminder group using the chi-squared test. To control for confounding factors, multivariate logistic regression was conducted with attendance status as the dependent variable (yes/no) and SMS reminder (yes/ no) as the main exposure factor. Other factors controlled for in the model included patient's age, nationality (Saudi/Arab non-Saudi), type of visit (first contact/had previous contact). For the odds ratio (OR) calculation, the non-reminder group was used as the reference group. The OR and 95% confidence intervals (CI) are presented. P-values < 0.05 were considered to be statistically significant. The multivariate logistic regression would also allow for hypothesis generation regarding types of patients more likely to not attend their appointment.

Results

Baseline characteristics

All included patients completed the study and the final analysis was performed on 502 patients, 251 per group. The baseline characteristics of the 2 groups are presented in Table 1. Comparison between groups shows adequate randomization, with equal distribution of group characteristics at baseline. The mean age was 52.5 (SD 14.5) years and a majority of the study population were Saudi nationals (90.8%).

Non-attendance rate (primary outcome)

The non-attendance rate for the control group (39.8%) was significantly higher than for the SMS-reminder group (26.3%) based on a 2-sample test of proportion (P = 0.001). Multivariate logistic regression analysis showed that patients who received SMS reminders were 44% less likely to miss their appointment compared with those who did not receive SMS reminder (OR = 0.56, 95% CI: 0.38-0.82, P = 0.001) (Table 2).

Using number needed to treat calculation, for every 7 SMS reminders sent, 1 additional patient showed up for the appointment due to the reminder message. There was no evidence of interaction between receiving an SMS reminder and patient characteristics.

Factors associated with non-attendance

The non-attendance rate was much higher among patients who had their first visit compared with those who had had prior contact with the hospital (OR = 7.4; 95% CI: 3.20-17.1, P = 0.001). Nevertheless, the SMS reminder was similarly effective in reducing the non-attendance rate in these 2 groups of patients. Patients' age, sex and nationality were not associated with an increased risk ofnon-attendance (Table 2).

Patients' perceptions of SMS reminders

Out of the 251 patients who were sent an SMS reminder, we attempted to contact 100 patients. Of these 100 patients, 76 were successfully reached and were surveyed over the phone. Table 3 presents the response to the main questions; 75 out of 76 patients (99%) had read the message and 74 (97%) found the message was clear. More than three-quarters (59, 77%) agreed that the SMS was useful to remind them about the appointment. All respondents (100%) reported they would like to continue to receive SMS reminders in the future and recommended providing the SMS reminders to others.

Discussion

This study demonstrated that SMS reminders to patients' mobile phone were effective in reducing the non-attendance rate among patients with internal medicine outpatient clinic appointments compared with those who were not send a reminder. This SMS reminder effect was consistent irrespective of patients' sex, nationality or type of visit, as evidenced by the multivariate regression and lack of a significant interaction between having an SMS reminder and these factors on attendance rate.

The results of our study were in line with other RCTs that examined the effect of SMS reminders on non-attendance in outpatient clinics. In a meta-analysis of 8 RCT studies, the summary OR was 1.48 (95% CI: 1.331.72) compared with 1.78 in our study (95% CI: 1.23-2.63) [28]. The effect in these combined studies was similar in both primary health care and hospital outpatient clinics [2].

In a pre- and post-observational study, Altuwaijri et al. showed that integrating the SMS reminders with the electronic medical records reduced non-attendance rates in 16 outpatient clinics in the Saudi National Guard hospital by 4.1% [26]. Their reduction in non-attendance attributed to SMS reminders was much lower than the reduction in our study. This variation could be because of the difference between the sociodemographic characteristics of the 2 study populations. Another possibility is that their study included clinics with different specialties including obstetrics and gynaecology. These clinics have patients who may behave differently towards their appointments and in their response to SMS reminders compared with patients with appointments at internal medicine clinics.

Other methods of reminding patients about their appointments were also shown to be effective. A meta-analysis of 23 interventional studies demonstrated that mailed and telephone reminders were consistently effective in reducing non-attendance rates [29]. Using computer-generated postcard reminders, Quattlebaum et al. achieved about 48% reduction in non-attendance [30]. Most ofthe studies that compared both SMS and phone reminders (automated and personal) found that there was no significant difference in the effectiveness between these 2 methods, with SMS reminders being more cost-effective. SMS messaging is cheaper than telephone calls with regard to staff costs and telephone company fees. Furthermore, the information technology (IT) infrastructure for SMS messaging already exists in most hospital IT systems, including our study hospital, where the software for sending automated text messages can be integrated with existing electronic health records and administrative hospital systems. The system could also be automated to collect patients' mobile phone numbers provided at the time the appointment is booked. Furthermore, the system could be automated to send the reminder message at a pre-specified time before the appointment with almost no labour cost. Once the system is set up, the cost of running it grows little as the number of messages increases. It is also possible to negotiate with the telecommunication companies to reduce the unit cost with the increase in the volume of SMS messages sent.

Multivariate analysis of factors associated with non-attendance in this study found that age was not associated with rates of non-attendance. This disagrees with the results of other studies showing that older people were significantly less likely to be non-attenders [31,32]. Although older individuals have more health problems and are more likely to be aware oftheir own health needs [32], other factors may preclude patients from attending their appointments, even for older people who may have greater needs. In a study by Lacy et al. 3 reasons were cited by patients as to why they do not show up for their appointments: emotional barriers; perceived disrespect for the patient's beliefs and time by the health care system; and lack of understanding of the scheduling system [10]. Our study did not address whether any of these factors may have contributed to the similar attendance rates among older and younger patients. Our study also found that patients whose appointment was the first contact with the hospital had the highest risk of not attending their appointment, a finding which was in line with some studies [32], but in disagreement with others [31,33]. In our study follow-up patients were probably more comfortable with the system and more connected with the provider and therefore more likely to attend their appointment to maintain continuity of care [32].

The positive perceptions of patients in our study regarding SMS reminders were consistent with other studies [34-38]. One study reported that 98% of patients with mobile phones were willing to receive routine mobile phone text message reminders of their outpatient appointment [37]. In the study by O'Connor et al. SMS messaging was found to be the most preferred method of appointment reminder by 47% of the patients and telephone calls by 26% [38]. They also found that more than 94% were agreeable to receiving their reminder on any day or time of day, including holidays.

Our study had some limitations. First, this was a single hospital study, which limits the generalizability of our results. Similarly, our results may be applicable only to patients attending internal medicine outpatient clinics; other specialties may have different degree of effectiveness of reminders on patient attendance rate. In a study by Downer et al. the reduction in non-attendance rates with the use of SMS reminders ranged from 3% to 27%, with reminders having the lowest effectiveness in surgical departments [39]. Secondly, we did not confirm if all patients in the intervention group had received the SMS reminder. However, assuming that all these patients failed to attend their appointment, our results would be an underestimate of the effectiveness of the SMS reminders. Thirdly, this study did not examine the effectiveness of SMS reminders on patients with a history of repeated failure to attend their appointments. In a RCT that focused on patients with history of repeated non-attendance, the authors did not detect a significant reduction in non-attendance rate as a result of texting appointment reminders to this group of patients [40] and it is possible that this group of patients needs a different method than SMS reminders. Fourthly, we did not ascertain the reason for the clinic visit and it is important to note that clinical presentation could affect the priority placed by patients on the need for keeping an appointment. Fifthly, this study included only Arabic-speaking patients and these patients represent a small percentage of patients using the clinic. However, it is expected that non-Arabic speaking patients would show similar results, and converting the messages to other languages can be easily done. Several studies conducted in other countries consistently indicated the effectiveness of SMS reminders [41-43]. Finally, our study did not examine the effect of timing of the reminder before the appointment on the attendance rate. In a systematic review of 15 studies, none of the studies found any strong effect on non-attendance rate with reminders timed to between 1 day and 1 week before the appointment [44].

Future research needs to look at the effectiveness of the SMS reminders in different populations and different environments. Effectiveness in older patients is particularly needed given their greater need for health-care appointments and their less familiarity with SMS messages. Effectiveness of direct phone calls or other reminder methods for this group should be examined. The economic analysis of SMS reminders is also an important area of research. RCTs on the effectiveness of SMS reminders also need to be extended to other types of outpatient clinics and other types of providers.

Conclusion

This study provided evidence that SMS reminders were effective in reducing the non-attendance rate at internal medicine outpatient clinic appointments in a major Saudi hospital. The success of the SMS reminders was effective irrespective of patients' age, sex or type of visit. Patients who received SMS reminders were satisfied with the reminder and recommended official implementation of this service.

Acknowledgements

Funding: None.

Competing interests: None declared.

References

1. Karter AJ et al. Missed appointments and poor glycemic control: an opportunity to identify high-risk diabetic patients. Medical Care, 2004, 42:110-115.
2. Griffin SJ. Lost to follow-up: the problem of defaulters from diabetes clinics. Diabetic Medicine, 1998, 15(Suppl. 3):S14-S24.
3. Car J et al. SMS text message healthcare appointment reminders in England. Journal of Ambulatory Care Management, 2008, 31:216-219.
4. Bundy DG et al. Open access in primary care: results of a North Carolina pilot project. Pediatrics, 2005, 116:82-87.
5. O'Connor ME, Matthews BS, Gao D. Effect of open access scheduling on missed appointments, immunizations, and continuity of care for infant well-child care visits. Archives of Pediatrics and Adolescent Medicine, 2006, 160:889-893.
6. Sola-Vera J et al. Factors associated with non-attendance at outpatient endoscopy. Scandinavian Journal of Gastroenterology, 2008, 43:202-206.
7. Ross JD et al. Why do patients default from follow-up at a genitourinary clinic?: a multivariate analysis. Genitourinary Medicine, 1995, 71:393-395.
8. Neal RD et al. Reasons for and consequences of missed appointments in general practice in the UK: questionnaire survey and prospective review of medical records. BMC Family Practice, 2005, 6:47.
9. Mitchell AJ, Selmes T. Why don't patients attend their appointments? Maintaining engagement with psychiatric services. Advances in Psychiatric Treatment, 2007, 13:423-434.
10. Lacy NL et al. Why we don't come: patient perceptions on no-shows. Annals of Family Medicine, 2004, 2:541-545.
11. Sawyer SM, Zalan A, Bond LM. Telephone reminders improve adolescent clinic attendance: a randomized controlled trial. Journal of Paediatrics and Child Health, 2002, 38:79-83.
12. Lieu TA et al. Effectiveness and cost-effectiveness of letters, automated telephone messages, or both for underimmunized children in a health maintenance organization. Pediatrics, 1998, 101:e3.
13. Reti S. Improving outpatient department efficiency: a randomized controlled trial comparing hospital and general-practice telephone reminders. New Zealand Medical Journal, 2003, 116:U458.
14. Reekie D, Devlin H. Preventing failed appointments in general dental practice: a comparison of reminder methods. British Dental Journal, 1998, 185:472-474.
15. Chen ZW et al. Comparison of an SMS text messaging and phone reminder to improve attendance at a health promotion center: a randomized controlled trial. Journal of Zhejiang University. Science. B., 2008, 9:34-38.
16. Liew SM et al. Text messaging reminders to reduce non-attendance in chronic disease follow-up: a clinical trial. British Journal of General Practice, 2009, 59:916-920.
17. Mobile phones in Saudi Arabia. Country report. Chicago, Illinois, Euromonitor International, 2013 (http://www.euromonitor. com/mobile-phones-in-saudi-arabia/report, accessed 26 January 2014).
18. Milne RG, Horne M, Torsney B. SMS reminders in the UK national health service: an evaluation of its impact on "no-shows" at hospital out-patient clinics. Health Care Management Review, 2006, 31:130-136.
19. Krishna S, Boren SA, Balas EA. Healthcare via cell phones: a systematic review. Telemedicine Journal and e-Health, 2009, 15:231-240.
20. Strandbygaard U, Thomsen SF, Backer V. A daily SMS reminder increases adherence to asthma treatment: a three-month follow-up study. Respiratory Medicine, 2010, 104:166-171.
21. Montes JM et al. A short message service (SMS)-based strategy for enhancing adherence to antipsychotic medication in schizophrenia. Psychiatry Research, 2012, 200:89-95.
22. Armstrong AW et al. Text-message reminders to improve sunscreen use: a randomized, controlled trial using electronic monitoring. Archives of Dermatology, 2009, 145:1230-1236.
23. Free C et al. Smoking cessation support delivered via mobile phone text messaging (txt2stop): a single-blind, randomised trial. Lancet, 2011, 378:49-55.
24. Haynes JM, Sweeney EL. The effect of telephone appointment-reminder calls on outpatient absenteeism in a pulmonary function laboratory. Respiratory Care, 2006, 51:36-39.
25. O'Brien G, Lazebnik R. Telephone call reminders and attendance in an adolescent clinic. Pediatrics, 1998, 101:e6.
26. Altuwaijri MM et al. The effect of integrating short messaging services' reminders with electronic medical records on non-attendance rates. Saudi Medical Journal, 2012, 33:193-196.
27. Youssef A et al. Effectiveness of text message reminders on nonattendance of outpatient clinic appointments in three different specialties: a randomized controlled trial in a Saudi Hospital. Journal of Taibah University Medical Sciences, 2014, 9(1):23-29.
28. Guy R et al. How effective are short message service reminders at increasing clinic attendance? A meta-analysis and systematic review. Health Services Research, 2012, 47:614-632.
29. Macharia WM et al. An overview of interventions to improve compliance with appointment keeping for medical services. Journal of the American Medical Association, 1992, 267:18131817.
30. Quattlebaum TG, Darden PM, Sperry JB. Effectiveness of computer-generated appointment reminders. Pediatrics, 1991, 88:801-805.
31. Lehmann TN et al. Missed appointments at a Swiss university outpatient clinic. Public Health, 2007, 121:790-799.
32. Parikh A et al. The effectiveness of outpatient appointment reminder systems in reducing no-show rates. American Journal of Medicine, 2010, 123:542-548
33. Wick A, Koller MT. Views of patients and physicians on follow-up visits: results from a cross-sectional study in Swiss primary care. Swiss Medical Weekly, 2005, 135:139-144.
34. Vervloet M et al. SMS reminders improve adherence to oral medication in type 2 diabetes patients who are real time electronically monitored. International Journal of Medical Informatics, 2012, 81:594-604.
35. Christensen A et al. Survey of patient and physician assessment of a compliance reminder device in the treatment of hypertension. Blood Pressure, 2009, 18:280-285.
36. Sahm L et al. Electronic reminders to improve medication adherence-are they acceptable to the patient? Pharmacy World and Science, 2009, 31:627-629.
37. Koury E, Faris C. Mobile phones and clinic appointments:the start of a beautiful new friendship? British Journal of Healthcare Computing and Information Management, 2005, 22:18-20.
38. O'Connor MB, Bond U, Saunders JA et al. The preferences for outpatient clinic appointment reminders among rheumatology patients. Journal of Clinical Rheumatology, 2008, 15(5):258-259.
39. Downer SR, Meara JG, Da Costa AC. Use of SMS text messaging to improve outpatient attendance. Medical Journal of Australia, 2005, 183:366-368.
40. Fairhurst K, Sheikh A. Texting appointment reminders to repeated non-attenders in primary care: randomised controlled study. Quality and Safety in Health Care, 20 08, 17:373-376.
41. Prasad S, Anand R. Use of mobile telephone short message service as a reminder: the effect on patient attendance. International Dental Journal, 2012, 62:21-26.
42. Foley J, O'Neill M. Use of mobile telephone short message service (SMS) as a reminder: the effect on patient attendance. European Archives of Paediatric Dentistry; Official Journal of the European Academy of Paediatric Dentistry, 2009, 10:15-18.
43. Guy R et al. How effective are short message service reminders at increasing clinic attendance? A meta-analysis and systematic review. Health Services Research, 2012, 47:614-632.
44. Hasvold PE, Wootton R. Use of telephone and SMS reminders to improve attendance at hospital appointments: a systematic review. Journal ofTelemedicine and Telecare, 2011, 17:358-364.