Research article 

PDF version

H. Almoallim¹

تحديد الكفاءات وتقييم أولويَّاتها في المنهج التعليمي الجامعي للطب الباطني في المملكة العربية السعودية

هاني المعلم

الخلاصـة: هدف الباحث إلى تحديد كفاءات المعارف والمهارات في الطب الباطني في المنهج التعليمي الجامعي للطب الباطني في المملكة العربية السعودية، وذلك استناداً إلى عمل جماعي وباستخدام الكتب المرجعية الشائعة. واستخدمت طريقة دلفي Delphi باعتبارها طريقة متَّفقاً عليها لتحديد الكفاءات في الطب الباطني وتحديد أولوياتها. وقد قامت مجموعة من عشرين طبيباً بتبويب الكفاءات التي جرى تحديدها بدرجات تتراوح بين الصفر وبين ثلاث درجات (0: تعني أن لا حاجة لمعرفتها، 1: تعني أن من المثير للانتباه التعرف عليها، 2: تعني أنه ينبغي معرفتها، 3: تعني لابد من معرفتها). وبعد صياغة النتائج، أجريت جولة أخرى بطريقة دلفي شارك فيها خمسة خبراء في الطب الباطني. وقد جرى تحديد ما مُجْمَلُه 1513 كفاءة معارف و189 كفاءة مهارات كما حُدِّدَتْ أولوياتها. وتقابل هذه الكفاءات اثنتي عشرة منظومة في الطب الباطني. ثم تم فَرْزُ جميع الكفاءات التي بلغت درجتها 2.2-3.0 واعتُبِرَتْ كفاءات محورية للمنهج التعليمي الجامعي للطب الباطني. ولا يخفى أن مثل هذا التحديد للكفاءات وتعيين أولوياتها ينبغي أن يكون له أثره على عملية إصلاح المنهج التعليمي.

ABSTRACT: To determine knowledge and skills competencies in internal medicine for the undergraduate curriculum in Saudi Arabia, competencies were identified based on group work utilizing common textbooks. The Delphi Technique was used as a consensus method to determine and prioritize competencies in internal medicine. A group of 20 clinicians rated the identified competencies from 0–3 (0: no need to know, 1: interesting to know, 2: should know and 3: must know). After formulating the results, a second Delphi round was conducted with 5 experts in internal medicine. A total of 1513 knowledge competencies and 189 skills competencies were determined and prioritized. The competencies corresponded to the 12 systems in internal medicine. All competencies rated 2.2–3.0 were produced separately and considered core competencies for the undergraduate internal medicine curriculum. Determining and prioritizing competencies should influence the curriculum reform process.

Détermination et classement par ordre de priorité des compétences dans le programme de médecine interne de premier cycle en Arabie Saoudite

RÉSUMÉ: Pour déterminer les compétences théoriques et pratiques en médecine interne dans le programme de premier cycle en Arabie saoudite, un travail de groupe a été réalisé sur des manuels communs. La méthode Delphi a été utilisée en tant que méthode d’obtention d’un consensus pour déterminer les compétences utiles en médecine interne et les classer par ordre de priorité. Un groupe de 20 cliniciens a attribué une note allant de 0 à 3 aux compétences identifiées (0 : connaissance inutile, 1 : connaissance intéressante, 2 : connaissance utile, 3 : connaissance indispensable). Après la formulation des résultats, une deuxième vague de consultation selon la méthode Delphi a été menée auprès de cinq experts en médecine interne. Au total, 1513 compétences théoriques et 189 compétences pratiques ont été identifiées puis classées par ordre de priorité. Les compétences correspondaient aux douze systèmes de médecine interne. Toutes les compétences notées de 2,2 à 3,0 ont été présentées séparément et considérées comme des compétences clés pour le programme de médecine interne de premier cycle. La détermination des compétences et leur classement par ordre de priorité devraient influer sur le processus de réforme du programme.

EMHJ, 2011, 17(8): 656-662

---

¹Department of Medicine, Medical College, Umm Alqura University, Mecca, Saudi Arabia (Correspondence to H. Almoallim: This e-mail address is being protected from spambots. You need JavaScript enabled to view it ).
Received: 15/01/09; accepted: 25/02/10

Introduction

One of the current trends in curriculum planning is to base it on outcomes [1]. These outcomes should be clearly defined in the form of competencies. Curriculum designers should know precisely what they want from medical students in terms of knowledge, attitudes and skills competencies. Otherwise, teaching may miss certain competencies important for the practising physician; on the other hand, it may focus on unnecessary or less important competencies. The epidemiology of diseases differs between countries and common diseases locally in Saudi Arabia such as dengue fever and tuberculosis may not be well addressed in standard textbooks. Therefore, curriculum designers are required to consider such differences.

The Delphi Technique a research approach is employed to develop consensus amongst a heterogeneous population. It is “a method for systematic collection and aggregation of informal judgment from a group of experts on specific questions and issues” [2]. The Delphi Technique utilizes iterative multistage processes; 2 or more rounds are conducted using a questionnaire survey. Each round uses information gathered during previous rounds to converge toward a consensus of the group’s opinions. It is valued for its probability in avoiding drawbacks often associated with group dynamics in structured and unstructured direct interactions. The technique has many advantages over other group decision-making methods because it allows participants’ anonymity while preventing the possible bias of dominating the consensus development by a powerful individual in face-to-face meetings. It facilitates ownership and increases acceptance of the generated consensus. In addition, the study’s statistical group response allows qualitative data to be transformed into quantitative data; and it overcomes the geographical limitation of the participation of a wide range of experts and stakeholders [3].

The primary objective of this study was to determine and prioritize knowledge and skills competencies in undergraduate internal medicine curriculum in Saudi Arabia.

Methods

Delphi

The Delphi Technique is considered one of the most useful techniques in identifying competencies in medical education [3,4] A rigorous stepwise approach for use of the technique was followed in this research in conjunction with several guidelines and recommendations (Figure 1) [5].

Prior to application of the Delphi Technique, a list of all internal medicine competencies was developed. This was done using a team of 29 volunteers, final-year medical students and interns from Umm Alqura University Medical College, Mecca. Team members were divided into 12 groups corresponding to the 12 systems in internal medicine. Each group was asked to write up all internal medicine competencies corresponding to their assigned system. Knowledge and skills competencies were identified based on standard medical textbooks [6,7]. All groups received detailed instructions on “how to write competencies” and “how to choose the observable verbs that describe precisely the cognitive function to be achieved by each objective” (Figure 1). To unify the work and style, the author reviewed all the competencies for each system and modifications were introduced where needed, e.g. any knowledge competency felt to be a skill was moved to the skills competencies section of that system and vice versa. Written competencies were then reviewed by a panel of 12 experts and any missing competencies added. The second stage was the application of iteration 2 of the Delphi Technique. This was aimed at developing a consensus on the core competencies essential for undergraduate medical students.


Figure 1 Steps followed in identifying and rating the competencies based on the Delphi Technique  

In the first Delphi round, a booklet containing all competencies of all systems was sent with an instruction letter to a total of 30 participants. Participants were asked to rate each competency as 0, 1, 2, and 3 corresponding to: no need to know the subject, good and interesting to know, should know and must know the subject respectively. If the judgment of the rater was between 2 levels, the following numbers were assigned 0.5, 1.5 and 2.5 respectively. A period of 2 months was allowed to return the booklet. After collecting all data, the weighted response for each competency was calculated following a standard method [8].

The second round of Delphi was conducted with a list of all competencies with the weighted responses. This was sent to 6 experts in internal medicine. The same instructions and system of rating was followed, and the booklets were returned after 2 months. The final weighted responses were then calculated.

Rating and analysis of data

The following technique was used to analyse the ratings of each competency and the combined responses of all participants. First, the 4 levels of ratings were assigned; to obtain a weighted response for each competency, the number of responses in each level was multiplied by the assigned number, and then the products were added together and divided by the total number of responses, which gives a mean score for each competency (0.0–3.0). This technique has been described previously [8]. Using this method, all knowledge and skills competencies in internal medicine were ranked and the relative importance for each competency in the curriculum was determined.

Participants

In the first stage of developing the list of all competencies, 29 medical students and interns who graduated from the medical college of Umm Alqura University volunteered to help with this project.

Written competencies were reviewed by a group of 12 consultants in internal medicine. All were trained in North America and have been involved in teaching undergraduate medical student and residents in Umm Alqura University.

In the first Delphi round, a group of 30 clinicians who had different levels of experience and different specializations were invited to participate; they included faculty staff, internship directors, practising general internists, senior medical residents and interns (Table 1). Participants were selected to represent all medical colleges in Saudi Arabia. The practising general internists were from 5 major educational hospitals in 5 different cities. All of the non-internists selected (an obstetrician, a surgeon, a family physician and a paediatrician) were full professors in their institutions, with an academic qualification in medical education.

Table 1 Composition of groups in the two Delphi rounds

In the second Delphi round, a total of 6 experts in internal medicine were involved (Table 1). These experts represent 5 different medical colleges in Saudi Arabia and all had wide clinical and educational experience in undergraduate medical education.

Results

For the first Delphi survey, 20 out of the 30 participants completed the survey booklet, a response rate of 66.6%. Only 1 expert general practitioner out of 3 in our region responded to this round. For the second survey, 5 out of 6 participants completed the booklet, a response rate of 83.3%.

Internal medicine systems were organized alphabetically: allergic and immunologic disorders, cardiovascular disorders, dermatologic disorders, endocrine diseases, gastrointestinal diseases, haematologic diseases, infectious diseases, neurologic disorders, oncologic disorders, pulmonary diseases, renal diseases and rheumatologic diseases. The competencies for each system were organized under 2 major categories knowledge competencies and skills competencies, with disease subheadings as outlined in the textbook (Table 2). A total of 1703 competencies were identified for the 12 systems, of which 1514 were knowledge competencies and 189 skills competencies (All of these competencies were produced in a booklet format at the end of the study; this is available by direct request from the author).

Table 2 Number of knowledge and skills competencies for all systems.

All competencies in knowledge and skills with weighted response of ≥ 2.2 were considered core competencies. Table 3 shows an example of the core skills competencies for cardiovascular disorders.

Table 3 Core skills competencies (rated ≥ 2.2–3.0) for cardiovascular disorders

Table 4 shows the weighted responses of the 12 knowledge competencies determined for congestive heart failure (CHF) based on experts’ ratings in the second Delphi survey. Prioritization is apparent in this example as the results clearly reflect the level of importance of this disease, 11 out 12 knowledge competencies were rated > 2 (should know the subject) and 7 out of these 12 were actually rated > 2.5, close to 3 (must know the subject).

Table 4 Knowledge competencies (rated 0.0–3.0) for congestive heart failure (CHF)

The mean weighted responses for all disease subheadings for all systems were calculated. This is to give further prioritization and relative importance of diseases within each system. The mean weighted responses for knowledge competencies for all cardiovascular disorders are shown in Table 5. The number of determined competencies for each disease has also been counted (full package for all mean weighted responses of all disease subheadings for all systems is available through direct request from the author.

The correlation coefficient for all results comparing clinicians’ group ratings in the first Delphi to experts’ ratings in the second Delphi was 0.90 for knowledge competencies and 0.70 for skills competencies.

Table 5 Mean weighted response of all disease subheadings of cardiovascular diseases

Discussion 

Determining competencies is an essential component to the success of any educational programme. A strict methodology was followed in this comprehensive study to determine and prioritize knowledge and skills competencies for the undergraduate curriculum in internal medicine in Saudi Arabia. It is hoped that this effort will direct medical educators in developing competency-based curricula with more focus on essential and must-know competencies. There is an extensive effort in medical education literature addressing reforming and constructing new curricula, not only for internal medicine core clerkship [9,10], but also other specialties [11]. The type of methodology followed in this study is unique in medical education literature [12].

One of the advantages of developing a core list of should know and must know competencies is to help in designing a national guide for internal medicine teaching in medical colleges in Saudi Arabia. It would also help in designing a clerkship curriculum in internal medicine based on the specified competencies that augment and serve as a continuum of what had been taught in the undergraduate curriculum. It was reported in a different health system than Saudi Arabia that the use of a national guide is associated with improved ability to meet clerkship accreditation criteria, improved performance of students on the clerkship exam and increased clerkship time devoted to ambulatory care [12]. There is no published national guideline for minimal acceptable competencies of a medical graduate in Saudi Arabia [13]. This effort of developing a comprehensive guide and follow its implementation on the national level should result in the rapid adoption by medical colleges, with subsequent improvement in the quality of medical education [11].

One of the applications of the result of this study is to properly design assessment methods. Competency-based assessment measures what doctors can do in controlled representations of professional practice [14]. Competencies (knowledge or skills) drawn from this research can be used as the competencies that should be fulfilled and/or demonstrated by candidates during examinations, including objective-structured oral examination. Rethans et al. proposed that all assessments under examination-like settings should be referred to as competence-based assessments, whereas assessments in actual practice should be referred to as performance-based assessments [14].

The mean weighted responses for cardiovascular diseases may help planners in organizing curriculum content based on this rating and in blue-printing cardiovascular-related assessments. The high ratings of competencies for CHF (knowledge: out of 12, 11 rated > 2 and 7 rated > 2.5) may direct curriculum planners to use these competencies as outcomes while designing and implementing courses and educational interventions. It also means that CHF must be considered while planning curriculum content and while designing assessment methods in internal medicine courses. The same rationale can be used when addressing the weighted responses for other competencies for all diseases in all systems outlined in this study.

There are significant sets of competencies in each system addressing issues taught in basic sciences in the classical curricular designs as in our college. The findings of this study should assist the redesigning of curriculum content for many basic sciences in our college. In outcome-based education, clear identification of competencies in all disciplines will prevent the flow of unnecessary information that has no relevance to the graduating doctors. This will also minimize the redundancy and overlap of information across disciplines. However, this should be conducted through a careful curriculum planning process.

Expert internists with subspecialty qualification from 4 different cities in Saudi Arabia with extensive academic experience were included in the first and second Delphi rounds. It has been suggested that a minimum figure of 20 or so would be appropriate for a medical discipline [4]. To provide representative information, some studies have employed over 60 participants [15] while others have involved as few as 15 participants [16]. Obviously the larger the sample size, the greater the generation of data, which in turn influences the amount of data analysis to be undertaken [5].

The aim was to include around 30 participants in the first round of Delphi in this study, however, only 20 clinicians participated. They included clinicians from different specialties who all were experts in their fields with qualifications in medical education. Interns who finished their internal medicine rotations were included as well: they are newly encountering the professional life of a doctor where they can judge the applicability of what they have been taught and what they have experienced in the rotation. Internal medicine residents with different levels of training were also included as they are considering internal medicine as a career. Their views would likely be influenced by the relative relevancy of diseases, based on their experience. General internists with different subspecialties were included to enrich the outcome of this round with their generalist and subspecialist views.

There were some limitations to this study. The results cannot be generalized to different societies with different disease epidemiology. Any application of the findings should consider the local epidemiology of diseases. The findings are not time sensitive as they should go for revision after a specified period of time (determined by the curriculum planners). Issues that were not addressed in this research work were principles of curriculum planning, teaching methods that should be employed to achieve these competencies, time for inpatient versus outpatient training for medical students, communication skills, ethics and attitudes.

Competencies in knowledge and skills for internal medicine have been determined and prioritized. This study would greatly influence the curriculum reform process that has been adopted by many medical colleges in the region. It should direct curriculum planners to organize curriculum contents, to base teaching on important competencies and to design valid assessment methods. It should help designing a national guide for internal medicine teaching in medical colleges in Saudi Arabia.

Acknowledgements

Experts: Riyadh Alsolymani, Mohammad Alqasimi, Mansour Aljenadi, Emad Koshak, Abdulsalam Noorwali. Lina Bissar, Samar Badreddine, Saed Alghamdi, Albager Mohammad, Abdulhaleem Gasim, Atef Gabbani, Atef Alzahrani, Asim Alsaedi, Amal Abdulwahab, Ayman Abdo, Nashwa Bannani, Abdulwahab Altelmisani, Taymor Khattab, Kamran Hamed, Samer Sabban, Emad Khojah, Hussam Alem, Fahad Musally, Emad Azhari, Moafaq Kalantan, Riyadh Alleheibi, Ammar Saati, Ismail Alghamdi, Adeeb Albulkhi, Khalid Alghamdi, Turki Bafaraj, Sultan Alwajeeh, Waleed Hafiz, Mohammad Aloofi, Norah Alshehri, Norah Alzahrani, Ghadah Noh, Hanan Alharthi, Rabab Buoghdadi, Leena Alwafi, Heba Alqurashi, Eman Alqahtani, NorAlhuda, Nashwa Bannani, Nuha Hemesh, Safaa Alsanoosi, Marwa Ameen, Nuha Filfilan and Shuaa Basloom

Special thanks to: Salwa Aldahlawi for revising the manuscript and Taymor Khattab and Rania Zaini for advice given during research.

References

1. Shumway JM, Harden RM. AMEE Guide No. 25: the assessment of learning outcomes for the competent and reflective physician. Medical Teacher, 2003, 25(6):569–584.
2. Reid N. The Delphi Technique: it’s contribution to the evaluation of professional practice. In: Ellis R, ed. Professional competence and quality assurance in the caring professions. New York, Chapman Hall, 1988.
3. De Villiers MR, de Villiers PJ, Kent AP. The Delphi technique in health sciences education research. Medical Teacher, 2005, 27(7):639–643.
4. Dunn WR, Hamilton DD, Harden RM. Techniques of identifying competencies needed of doctors. Medical Teacher, 1985, 7(1):15–25.
5. Hasson F, Keeney S, McKenna H. Research guidelines for the Delphi survey technique. Journal of Advanced Nursing, 2000, 32(4):1008–1015.
6. Myers AR. NMS Medicine. Philadelphia, Lippincott Williams & Wilkins, 2005.
7. Talley NJ, O’Connor S. Clinical examination: a systematic guide to physical diagnosis. Oxford, Wiley–Blackwell, 2005.
8. Lawrence PF et al. Determining the content of a surgical curriculum. Surgery, 1983, 94(2):309–317.
9. Goroll AH et al. Reforming the core clerkship in internal medicine: the SGIM/CDIM project. Society of General Internal Medicine/Clerkship Directors in Internal Medicine. Annals of Internal Medicine, 2001, 134(1):30–37.
10. Bass EB et al. National survey of Clerkship Directors in Internal Medicine on the competencies that should be addressed in the medicine core clerkship. American Journal of Medicine, 1997, 102(6):564–571.
11. Olson AL et al. A national general pediatric clerkship curriculum: the process of development and implementation. Pediatrics, 2000, 106(1 Pt 2):216–222.
12. Jablonover R.S. et al. Evaluation of a national curriculum reform effort for the medicine core clerkship. Journal of General Internal Medicine, 2000, 15(7):484–491.
13. Bajammal S et al. The need for national medical licensing examination in Saudi Arabia. BMC Medical Education, 2008. 8(1):53.
14. Rethans JJ et al. The relationship between competence and performance: implications for assessing practice performance. Medical Education, 2002. 36(10):901–909.
15. Alexander J, Kroposki M. Outcomes for community health nursing practice. Journal of Nursing Administration, 1999. 29(5):49–56.
16. Fiander M, Burns T. Essential components of schizophrenia care: a Delphi approach. Acta Psychiatrica Scandinavica, 1998, 98(5):400–405.