PDF version

Fatma Birgili,¹ Media Baybuga,¹ Hatice Ozkoc,² Oktay Kuru,³ Thea van de Mortel⁴ and Adile Tümer¹

¹Division of Nursing; ³Division of Physical Therapy and Rehabilitation, Faculty of Health Sciences; ²Department of Statistics, Faculty of Science, Muğla Sıtkı Koçman University, Muğla, Turkey. ⁴School of Nursng and Midwifery, Griffith University, Southport, Australia. (Correspondence to: Fatma Birgili: This e-mail address is being protected from spambots. You need JavaScript enabled to view it ).

Abstract

Background: Currently, there is no validated Turkish language instrument to assess the Hand Hygiene Questionnaire.

Aims: This study determined the validity, structure and reliability of a Turkish translation of the Hand Hygiene Questionnaire scale in order to assess health care students’ belief and practices regarding hand hygiene (HH) and inform strategies to improve HH compliance.

Methods: The content validity index, confirmatory factor analysis and reliability statistics were employed. We recruited 595 nursing and physiotherapy students to participate in the study.

Results: The content validity index ranged from 0.77 to 0.86. Cronbach’s alpha ranged from 0.74 to 0.95. The fit measures of the model using confirmatory factor analysis were χ2 = 1276.18 (degrees of freedom (df) = 461); root mean squared error of approximation was 0.064. Normed fit index was 0.95 and comparative fit index was 0.97.

Conclusion: This translation offers a reliable and valid means of assessing the beliefs and practices of Turkish health care students regarding hand hygiene.

Keywords: hand hygiene, health care students, beliefs, reliability and validity, Turkey

Citation: Birgili F; Baybuga MS; Ozkoc H; Kuru O; van de Mortel T; Tümer A. Validation of a Turkish translation of The Hand Hygiene Questionnaire. East Mediterr Health J. 2019;25(5):299-305 https://doi.org/10.26719/emhj.18.039

Received: 10/04/17; accepted: 08/10/17

Copyright © World Health Organization (WHO) 2019. Some rights reserved. This work is available under the CC BY-NC-SA 3.0 IGO license (https://creativecommons.org/licenses/by-nc-sa/3.0/igo).

---

Introduction

Background

Health care-associated infection increases patient morbidity, mortality and health care costs. For example, a matched case–control study of the outcomes of bloodstream-associated Staphylococcus aureus infection demonstrated an excess length of stay of 32 days, a 7-fold increased risk of dying and health care costs around 3 times higher than for noninfected patients (1). One of the steps in the transmission of pathogens from health care workers’ hands to patients is inadequate hand antisepsis (2). Thus, hand hygiene (HH) is considered an important cost–effective and practical measure to reduce the incidence of health care-associated infection (3), and various studies have demonstrated that increased HH compliance by health care workers reduces health care-associated infection rates (2,4,5). Given the importance of HH, evidence-based guidelines have been published for specific HH measures related to health care (6). There is also a large body of literature focusing on health care providers’ HH adherence and factors that influence this (7,8).

As health care students go on to become the health care professionals of the future, information about factors that influence their HH beliefs, knowledge and practices is also important. However, studies examining these factors among Turkish health care students are limited. For example, Celik and Koçaşli (9) examined the hand washing habits of Turkish nursing students via a questionnaire and found that 80.2% of students reported washing their hands before and after clinical interventions. However, the survey instrument did not undergo psychometric testing so the reliability of the results is unknown. Ergin et al. used a questionnaire to examine knowledge, practices and skills regarding hand washing among Turkish medical and education students, however, the study focused on hand washing in non-health care situations rather than health care-related hand washing and the reliability and validity of the survey instrument were not reported (10). Both studies also failed to examine HH using waterless, alcohol-based hand rubs, which is considered best practice by the World Health Organization (WHO) in a 2009 report (11).

Conceptual framework

One instrument that investigates health care students’ HH knowledge, beliefs and practices is the Hand Hygiene Questionnaire (HHQ), which was originally validated with Australian nursing students (12). The HHQ includes 3 scales (36 items): a hand hygiene beliefs scale (HBS) (19 items), a hand hygiene importance scale (HIS) (3 items) and a hand hygiene practices inventory (HHPI) (14 items). The HHPI and HBS include some items previously adapted from a handwashing practices inventory that was validated with general university students and subsequently applied to registered nurses (12). Responses are scored on a 5-point Likert scale (1 = strongly disagree, 5 = strongly agree) where high numbers indicate agreement with the item (12).

The questionnaire was developed using social cognitive theory as a conceptual framework. Social cognitive theory explains how behaviour is learned, and is a comprehensive ecological theory that looks at the impacts of social, environmental and individual factors on behaviour (13). It explains the acquisition and maintenance of behaviours, but also provides a framework to develop interventional strategies to change those behaviours (13). It has also been suggested that various factors influence how behaviour is learned, including the types of role models encountered, beliefs about the outcomes of behaviour and the value attached to these outcomes, the degree of self-efficacy and capacity to self-regulate, and the types of reinforcing factors that are encountered (14).

The HHQ has been translated into Greek, Italian and Swedish, and validated with nursing and medical students in those populations (15–17). However, there is no available instrument for the evaluation of similar parameters for health care students in Turkey. A reliable and valid tool is needed to measure Turkish health care students’ HH beliefs and practices in order to inform educational strategies to improve HH compliance among health care students. Accordingly, the aim of this study was to translate the HBS, HHPI, and HIS into Turkish and to assess their validity and reliability with this population.

Methods

Study design

A cross-sectional design was used to conduct psychometric testing of a Turkish translation of the HHQ. The content validity and internal reliability parameters of the HBS, HHPI and HIS were analysed using the content validity index, Cronbach’s alpha, test–retest coefficients, item-to-total correlations, and confirmatory factor analysis.

Setting and subjects

The study was conducted with nursing and physiotherapy students in the School of Health Sciences at Muğla Sıtkı Koçman University, Muğla, Turkey, during their free time in a classroom environment. In order to reduce the probability of making an error of inference when conducting factor analysis, Nunnally (18) suggested a ratio of 10 cases per scale item, while Comfrey and Lee (19) suggested that an overall sample size of 500 is very good, thus a convenience sample exceeding 500 students was sought.

Students who had completed at least 1 semester of clinical practice were included in this study. All participation was voluntary and students’ responses were anonymous.

Ethics consent

Ethics approval was obtained from the Muğla Sıtkı Koçman University Ethics Committee (20121227). To maintain confidentiality, a pseudonym was used for each participant. Participants were provided with a cover letter that addressed the purpose of the study. Volunteers completed a written consent form.

Instrument translation and data collection

The translation and evaluation process followed the WHO Process of translation and adaptation of instruments (20). The HHQ scales were independently translated into Turkish by 2 authors (MSB, FB). The translations were carried out by these translators and 2 other bilingual health professionals. Back translation was completed by 3 other professionals who were blind to the original version and who were native speakers of both Turkish and English. The latter application helps to ensure that the conceptual meanings of the original and translated versions are equivalent (21,22).

The authors, along with an experienced translator, and several health professionals, formed a bilingual expert committee that reviewed both versions and afterwards reached consensus on any discrepancies. This process was followed by a cognitive debriefing assessing the comprehensibility level and cognitive equivalence of the translation, which was carried out by nursing students and academic staff, a total of 15 respondents.

Face validity is audience-specific and refers to how well an instrument is understood and accepted by the audience. Turkish language experts were consulted regarding the intelligibility and simplicity of the questionnaire to determine face validity. The content validity index 4-point rating scale was used to rate the relevance, clarity and simplicity of the items which these respondents indicated independently using the following values: 1 = not relevant, 2 = somewhat relevant, 3 = quite relevant and 4 = very relevant. The content validity index represented the proportion of total items rated as either quite relevant or very relevant, and a content validity index rating of 0.80 was considered valid (23). The expert committee additionally reviewed the debriefing results.

After translation, a 2-part questionnaire was distributed that included demographic questions (age, sex, class, department, HH education and source of education) and the scales. The questionnaire was distributed to participants present at the beginning of their courses and took ~20 minutes to complete. The completed questionnaires were collected by the researchers.

Data analysis

The analyses were conducted using SPSS, version 20.0, and the LISREL 8.54 program pack. Content validity was calculated using the Lawshe technique (24). Experts’ views were obtained on scale items. Following correction of items that lacked coherence, experts’ views were obtained again. Finally, content validity indices were calculated for both the total HHQ and subdimensions as described earlier.

Convergent validity was achieved by looking at the significance of the correlation coefficients provided in Table 1 in terms of same dimensions. To determine discriminant concurrent criterion validity, respondents’ scores were sorted from small to large, 27% of the first and last scores were compared, and an analysis was conducted to determine whether there was a significant difference.

The homogeneity of the scales was assessed using Cronbach’s alpha and item-to-total score correlations. Cronbach’s alpha coefficients of ≥ 0.7 and item-to-total correlations > 0.25 indicate acceptable internal consistency, while a Cronbach’s alpha coefficient of ≥ 0.8 is considered good (25,26). Since there is no Turkish study on validity and reliability in relation to the HHQ, in order to examine the stability of the scale in the study, test–retest methods were used instead of parallel form reliability (27). To assess test–retest reliability, volunteers were asked to complete the questionnaire again 2 weeks later, as done during its original application (12). Pearson’s correlation was used to define the test–retest coefficient (28).

Confirmatory factor analysis was used in this study to determine construct validity as it examines the relationships between each item and how they belong to the subdimensions (23). Floor–ceiling effects were examined respectively for the dimensions in the study. Floor–ceiling effects occur when > 15% of students respond with a lowest or highest score, making it difficult to assess the students’ level of ability (29).

Results

There were 595 participants, two-thirds of whom were female. All (100%) of students given the survey responded; 86.6% of the participants were in the nursing department, 28% were in 4th year and 27.7% were in 3rd third year. The power of the sample was 98%. The age range of the participants was 18–34 [mean 21.4, standard deviation (SD) 2.02] years, It was determined that 97.8% of the students had been educated on HH, and 97.6% had received education related to HH during their university education.

The scales were determined to have face validity as the experts agreed that the number of questions was acceptable, the sequence of the questions seemed reasonable, and the survey was both understandable and simple. The content validity index of the HHQ was 0.8, and ranged from 0.77 to 0.86 for the subdimensions (Table 1). Convergent validity was examined in terms of the significance of the correlation coefficients. Each of the 3 dimensions as a pair of the scale obtained correlation coefficients among themselves in positive ways. This suggests that any change in a subdimension will affect the whole of the scale. This will increase the overall score of the HHQ scale. The correlation coefficients obtained between each scale of the 3 pairs were positive. This result is also an improvement given that the others occurring on any scale effect will be positive. This will enable the general score of the HHQ scale to increase. The analysis for discriminant concurrent criterion validity found a significant difference (t = –44.349, P < 0.001), demonstrating that the constructs that theoretically should not be related to one another were in actual fact not related.

The Cronbach’s alpha coefficients of the scales ranged from 0.74 to 0.95, the mean item-to-total correlations ranged from 0.27 to 0.73, and the 2-week test–retest values ranged from 0.51 to 0.61 (Table 1). The item correlation was statistically significant (< 0.05). The scale means (± standard error of the mean) were: HBS 3.87 (± 0.39), HHPI 4.71 (± 0.39), and HIS 4.60 (± 0.57) (Table 1). Item means for each scale are shown in Tables 2, 3 and 4.

Construct validity was examined using confirmatory factor analysis. The fit measures of the model were χ2 = 1276.18 [degrees of freedom (df) = 461], root mean squared error of approximation = 0.064, normed fit index = 0.95 and comparative fit index = 0.97. Root mean squared error of approximation values of 0.9 and normed fit index values of ≥ 0.95 (30). Thus, the model was considered to be appropriate.

The standardized coefficients of the model are given in the loading factor columns in Tables 2, 3 and 4. According to these results, both the subdimensions and the total scale were valid. Analysis of our students’ score distributions demonstrated that there was no floor–ceiling effect for the HBS (both 0.16%). A ceiling effect was encountered in both dimensions (41% and 53% respectively) while no floor effect was observed for the HHPI and HIS (both 0.16%). However, no problematic questions were encountered in these dimensions. Interpretation should be made considering the ceiling effect in these dimensions for this reason. However, there is no floor or ceiling effect for any of the scales.

Discussion

This study determined the reliability and validity of the Turkish translation of the HHQ with nursing and physiotherapy students. The internal consistency and validity of the final scales were satisfactory–excellent in this study with the exception of the test–retest stability values. The Cronbach’s alpha coefficients (0.74–0.95) compared well with those of the original pilot version (0.74–0.80), and the handwashing practices inventory (0.76) (12), and all coefficients were at levels considered adequate–excellent.

The item-to-total correlations were 0.27, 0.63 and 0.73 for the HBS, HHPI and HIS compared with those reported for the original scales, which were 0.37, 0.33 and 0.61 respectively (12); while the first is lower than the original, the other 2 are higher; all values are, however, within the acceptable range (2,30).

The mean scores we calculated [HBS: 3.87 (standard error of the mean ± 0.39), HHPI: 4.71 (± 0.39) and HIS: 4.60 (± 0.57)] are similar to those reported in the original study 3.88 (± 0.06), 4.76 (±0.03), and 4.29 (± 0.10) (12). The mean HHPI score is also similar to that obtained on the handwashing practices inventory (4.45) (12). Interestingly, the mean score of the respondents on the original handwashing practices inventory was considerably lower (3.58). However, that sample was obtained from general university students rather than health care students, which may indicate the impact of health education on students’ perceptions of HH as mentioned in the study by van de Mortel (12). Based on the results of the content validity index, reliability tests, and the confirmatory factor analysis, the scales were deemed acceptable.

The study had several limitations. Firstly, only nursing and physical therapy students were included. As the medical faculty was newly established at the time of the study, medical students had not yet attended clinical placement and therefore were not included as participants. Secondly, according to Yaghmaie there is no completely objective method to determine content validity (31). Calculation of a content validity index relies on the professional subjective judgement of a panel of experts, which again is influenced by the interpretation of individuals. However, having a number of experts on the panel helps to overcome the subjective nature of the decision-making process. Thirdly, the test–retest results for the HBS, HHPI and HIS (0.51–0.6) were lower than the original values (0.79–0.89) reported by van de Mortel (12). However, Carmines, Zeller and DeVellis suggested that a low test–retest correlation may not indicate that the reliability of the test is low but may indicate a true change has occurred in attitudes (32,33).

Multiple factors may influence test–retest stability. For example, the participant may learn from their first experience to modify the way they complete the items on the retest, or intervening experience or knowledge acquired by completing the test items or during the test–retest interval can cause them to change their attitudes, as can changes in environmental conditions that influence performance at time 1 versus time 2 (32,34). Given some of the participants had not previously received HH education, it is possible that completing the scale items stimulated learning or attitude changes that were reflected in their responses on the second attempt. Finally, Trinic et al. indicate that while factor analysis can identify patterns of covariation, it cannot provide an explanation for why responses covary (35). The interpretation relies on the researcher, which introduces the potential for subjective decisions.

Conclusion

The Turkish version of the HHQ demonstrated acceptable validity and reliability, and thus provides a means of better understanding the HH practices and beliefs of Turkish nursing and physiotherapy students. Use of the validated instrument to examine these practices and beliefs will provide data to guide interventional strategies to improve the compliance of health care students with published HH guidelines. Further testing in a wider range of health care disciplines is needed to confirm these results.

Funding: None.

CoValidation d’une traduction du Questionnaire sur l’hygiène des mains en turc

Résumé

Contexte : Actuellement, il n’existe pas d’instrument validé en turc pour évaluer le questionnaire sur l’hygiène des mains.

Objectifs : La présente étude a déterminé la validité, la structure et la fiabilité d’une traduction turque de l’échelle du questionnaire sur l’hygiène des mains afin d’évaluer les croyances et les pratiques des étudiants en soins de santé concernant l’hygiène des mains et orienter les stratégies visant à améliorer le respect de l’hygiène des mains.

Méthodes : L’indice de validité du contenu, l’analyse factorielle confirmatoire et les statistiques de fiabilité ont été utilisés. Nous avons recruté 595 étudiants en sciences infirmières et en physiothérapie pour participer à l’étude.

Résultats : L’indice de validité du contenu était compris entre 0,77 et 0,86. L’alpha de Cronbach variait de 0,74 à 0,95. Les mesures d’ajustement du modèle à l’aide de l’analyse factorielle confirmatoire étaient χ2 = 1276,18 (degrés de liberté = 461), l’erreur quadratique moyenne de l’approximation étant de 0,064. L’indice d’ajustement normalisé était de 0,95 et l’indice d’ajustement comparatif de 0,97.

Conclusions : Cette traduction offre un moyen fiable et valable d’évaluer les croyances et les pratiques des étudiants en soins de santé turcs en ce qui concerne l’hygiène des mains.

mpeting interests: None declared.

التحقق من الترجمة التركية لاستبيان نظافة الأيدي

فاطمة بيرجلي، ميديا سوباسي بايبوجا، خديجة أوزكوتش، أوكتاي كورو، ثيا فان دي مورتيل، عديلة تومر

الخلاصة

الخلفية: لا توجد أداة لغوية متحقق منها باللغة التركية في الوقت الحالي لتقييم استبيان نظافة الأيدي.

الأهداف: حددت هذه الدراسة صحة الترجمة التركية لمقياس استبيان نظافة الأيدي وهيكله وموثوقيته، بهدف تقييم آراء طلاب الرعاية الصحية وممارساتهم فيما يتعلق بنظافة الأيدي، والاسترشاد بها في وضع استراتيجيات لتحسين الامتثال لنظافة الأيدي.

طرق البحث: تم استخدام مؤشر صحة المحتوى، وتحليل العوامل التأكيدية، وإحصاءات الموثوقية. وقمنا باختيار 595 ممرضة وطالب علاج طبيعي للمشاركة في الدراسة.

النتائج: تراوح مؤشر صحة المحتوى بين 0.77 و0.86. وتراوح معامل ألفا كرونباخ بين 0.74 و0.95. وبلغت قياسات الملائمة للنموذج باستخدام تحليل العوامل التأكيدية (χ2 = 1276.18; df = 461)؛ وكان متوسط الجذر التربيعي للخطأ بالتقريب 0.064. وبلغ مؤشر الملائمة المُعيَّر 0.95، ومؤشر الملائمة المقارن 0.97.

الاستنتاجات: تقدم هذه الترجمة وسيلة موثوقة وصالحة لتقييم آراء وممارسات طلاب الرعاية الصحية الأتراك فيما يتعلق بنظافة الأيدي.

References

1. Primo MG, Guilarde AO, Martelli CM, Batista LJ, Turchi MD. Healthcare-associated Staphylococcus aureus bloodstream infection: length of stay, attributable mortality, and additional direct costs. Braz J Infect Dis. 2012 Nov–Dec;16(6):503–9. http://dx.doi.org/10.1016/j.bjid.2012.10.001
2. Pittet D, Simon A, Hugonnet S, Pessoa-Silva CL, Sauvan V, Perneger TV. Hand hygiene among physicians: performance, beliefs, and perceptions. Ann Intern Med. 2004 Jul 6;141(1):1–8. http://dx.doi.org/10.7326/0003-4819-141-1-200407060-00008
3. Kelcíkova S, Skodova Z, Straka S. Effectiveness of hand hygiene education in a basic nursing school curricula. Public Health Nurs. 2012 Mar–Apr;29(2):152–9. http://dx.doi.org/10.1111/j.1525-1446.2011.00985.x
4. Harrington G, Watson K, Bailey M, Land G, Borrell S, Houston L, et al. Reduction in hospital-wide incidence of infection or colonization with methicillin-resistant Staphylococcus aureus with use of antimicrobial hand-hygiene gel and statistical process. Infect Control Hosp Epidemiol. 2007 Jul;28(07):837–44. http://dx.doi.org/10.1086/518844
5. Grayson ML, Melvani S, Druce J, Barr IG, Ballard SA, Johnson PDR, et al. Efficacy of soap and water and alcohol-based hand-rub preparations against live H1N1 Influenza virus on the hands of human volunteers. Clin Infect Dis. 2009 Feb 1;48(3):285–91. http://dx.doi.org/10.1086/595845
6. Kleinpell RM, Munro CL, Giuliano KK. Targeting health care–associated infections: evidence-based strategies. In: RG Hughes, ed. Patient safety and quality: an evidence-based handbook for nurses: Vol. 2 (pp. 577-600). Rockville, Maryland: Agency for Healthcare Research and Quality; 2008.
7. Boyce JM, Pittet D. Guideline for hand hygiene in health-care settings: recommendations of the Healthcare Infection Control Practices Advisory Committee and the HICPAC/SHEA/APIC/IDSA Hand Hygiene Task Force. Infect Control Hosp Epidemiol. 2002;23 S12:S3–40. http://dx.doi.org/10.1086/503164
8. Gould DJ, Moralejo D, Drey N, Chudleigh JH. Interventions to improve hand hygiene compliance in patient care. Cochrane Database Syst Rev. 2010 Sep 8;(9):CD005186.
9. Celik S, Koçaşli S. Hygienic hand washing among nursing students in Turkey. Appl Nurs Res. 2008 Nov;21(4):207–11. http://dx.doi.org/10.1016/j.apnr.2006.12.001
10. Ergin A, Bostancı M, Önal Ö, Bozkurt AI, Ergin N. Evaluation of students’ social hand washing knowledge, practices, and skills in a university setting. Cent Eur J Public Health. 2011 Dec;19(4):222–7. http://dx.doi.org/10.21101/cejph.a3664
11. WHO guidelines for hand hygiene in healthcare. First global patient safety challenge: clean care is safer care. Geneva: World Health Organization; 2009 (http://apps.who.int/iris/bitstream/handle/10665/44102/9789241597906_eng.pdf;jsessionid=61FA606713B67D1FC98C7278713E1A1B?sequence=1, accessed 21 January 2019).
12. van de Mortel TF. Development of a questionnaire to assess health care students’ hand hygiene knowledge, beliefs and practices. Aust J Adv Nurs. 2009;26(3):9–16.
13. Baranowski J, Perry CL, Parcel GS. How individuals, environments, and health behavior interact: social cognitive theory. In: Glanz K, Rimer BK, Lewis FM, eds. Health behavior and health education: theory, research and practice, 3rd ed. San Francisco: Jossey-Bass; 2002:165–84.
14. Bandura A. Health promotion by social cognitive means. Halta Educ Behav. 2004 Apr;31(2):143–64. http://dx.doi.org/10.1177/1090198104263660
15. van de Mortel TF, Apostolopoulou E, Petrikkos G. A comparison of the hand hygiene knowledge, beliefs, and practices of Greek nursing and medical students. Am J Infect Control. 2010;38(1):75–7. http://dx.doi.org/10.1016/j.ajic.2009.05.006
16. van de Mortel TF, Kermode S, Progano T, Sansoni J. A comparison of the hand hygiene knowledge, beliefs and practices of Italian nursing and medical students. J Adv Nurs. 2012 Mar;68(3):569–79. http://dx.doi.org/10.1111/j.1365-2648.2011.05758.x
17. van de Mortel TF, Apostolopoulou E, Petrikkos G, Hedberg E, Sansoni J, Edlund B, et al. The hand hygiene knowledge, beliefs, practices and education of healthcare students. Can J Infect Control. 2012b;27(3):161–7.
18. Nunnally JC. Psychometric theory. 2nd ed. New York: McGraw Hill; 1978.
19. Comfrey AL, Lee HB. A first course in factor analysis. Hillsdale, New Jersey: Lawrence Erlbaum Associates; 1992.
20. Process of translation and adaptation of instruments. Geneva: World Health Organization; 2017 (http://www.who.int/substance_abuse/research_tools/translation/en/, accessed 26 Januaary 2019).
21. Chapman DW, Carter JF. Translation procedures for the cross-cultural use of measured instruments. Educ Eval Policy Anal,1. 1979;1(3):71–6. https://doi.org/10.3102/01623737001003071
22. Chen HY, Boore JRP. Translation and back-translation in qualitative nursing research: methodological review. J Clin Nurs. 2010 Jan;19(1–2):234–9. https://doi.org/10.1111/j.1365-2702.2009.02896.x
23. Atkinson TM, Rosenfeld BD, Sit L, Mendoza TR, Fruscione M, Lavene D, et al. Using confirmatory factor analysis to evaluate construct validity of the Brief Pain Inventory (BPI). J Pain Symptom Manage. 2011 Mar;41(3):558–65. http://dx.doi.org/10.1016/j.jpainsymman.2010.05.008
24. Lawshe CH. A quantitative approach to content validity. Person Psychol. 1975 Dec;28(4):563–75. http://dx.doi.org/10.1111/j.1744-6570.1975.tb01393.x
25. Andresen EM. Criteria for assessing the tools of disability outcomes research. Arch Phys Med Rehabil. 2000 Dec;81(12 Suppl. 2):S15–20. http://dx.doi.org/10.1053/apmr.2000.20619
26. Chen YW, Coxson HO, Reid WD. Reliability and validity of the Brief Fatigue Inventory and Dyspnea Inventory in people with chronic obstructive pulmonary disease. J Pain Symptom Manage. 2016 Aug;52(2):298–304. http://dx.doi.org/10.1016/j.jpainsymman.2016.02.018
27. Erdoğan S, Nahcivan N, Esin N. Hemşirelikte araştirma. Istanbul: Nobel Tıp Kitabevleri; 2014 [in Turkish].
28. Wang CY, Sheu CF, Protas EJ. Test-retest reliability and measurement errors of six mobility tests in the community-dwelling elderly. Asian J Gerontol Geriatr. 2009;4:8–13. https://doi.org/10.1053/apmr.2002.32743
29. Salehi M, Niroumand S, Erfanian MR, Sajjadi RB, Dadgarmoghaddam M. Validation of Persian version of WHOQOL-HIV BREF questionnaire in Islamic Republic of Iran. East Mediterr Health J. 2016;22(9):647–53. http://dx.doi.org/10.26719/2016.22.9.647
30. Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct Equ Modeling. 1999 Nov;6(1):1–55. http://dx.doi.org/10.1080/10705519909540118
31. Yaghmaie F. Content validity and its estimation. J Med Educ. 2003;3(1):25–7. 10.22037/jme.v3i1.870
32. Carmines EG, Zeller RA. Quantitative applications in the social sciences. Thousand Oaks, California: Sage Publications; 1979.
33. DeVellis RF. Scale development: theory and applications, vol. 26. Thousand Oaks, California: Sage Publications; 2016.
34. Freitas JDCD, Da Hora HRM, Vianna DS, Costa HG. RIOT-a tool for estimating the reliability of surveys. Int J Inf Commun Technol. 2015;7(4-5):481–94. https://doi.org/10.1504/IJICT.2015.070324
35. Trninić V, Jelaska I, Štalec J. Appropriateness and limitations of factor analysis methods utilised in psychology. Fizička Kultura. 2013;67(1):5–17.