Psychometric Properties of Research Competency Scales: A Systematic Review
Propiedades psicométricas de las escalas de competencias investigativas: una revisión sistemática
##plugins.themes.bootstrap3.article.sidebar##
PDF (Español (España))
FLIP
PDF
XML
PDF (Español (España))
FLIP
XML (Español (España))
XML
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Este obra está bajo una licencia de Creative Commons Reconocimiento-NoComercial-SinObraDerivada 4.0 Internacional.
##plugins.themes.bootstrap3.article.main##
The study aimed to explore the psychometric properties of research competency scales published in the scientific literature in seven databases: Scopus, Web Of Science, ERIC, Embase, EBSCO, PubMed and SciELO between 2014 and June 2023. Thirteen papers were systematized, where 11 scales were found that measured content validity using indexes such as the S-CVI, Aiken's V and item-test correlations. Similarly, the results of the exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) presented adequate fit indices such as CFI, GFI, TLI, SRMR and RMSEA, which validated the factor structure. Reliability was mostly evaluated through Cronbach's Alpha coefficient, yielding high and consistent values. In addition, coefficients such as McDonald's Omega, ordinal Omega, Guttman and Spearman Brown have been used, all reflecting a value above .70 and internal consistency in the measurements. In synthesis, instruments such as the RCS, SRCS, RPCS, SPRDS, EAHIF, AHABI, ERL, self-perception questionnaires and rubrics have proven to be effective tools in the assessment and development of research skills. In that sense, these scales can improve the assessment and development of research skills in university students, and it is expected that future research will use them to evaluate pedagogical approaches and international contexts.
##plugins.themes.bootstrap3.article.details##
Alavi, M., Hunt, G. E., Thapa, D. K., & Cleary, M. (2022). Conducting Systematic Reviews of the Quality and Psychometric Properties of Health-Related Measurement Instruments: Finding the Right Tool for the Job. Issues in Mental Health Nursing, 43(4), 317–322. https://doi.org/10.1080/01612840.2021.1978599
Aliaga-Pacora, A. A., Juárez-Hernández, L. G., & Herrera-Meza, R. (2021). Diseño y validez de contenido de una rúbrica analítica socioformativa para evaluar competencias investigativas en posgrado. Apuntes Universitarios, 11(2), 62–82. https://doi.org/10.17162/au.v11i2.632
Almanasreh, E., Moles, R., & Chen, T. F. (2019). Evaluation of methods used for estimating content validity. Research in Social and Administrative Pharmacy, 15(2), 214–221. https://doi.org/10.1016/j.sapharm.2018.03.066
Andrade, C. (2021). The Inconvenient Truth About Convenience and Purposive Samples. Indian Journal of Psychological Medicine, 43(1), 86–88. https://doi.org/10.1177/0253717620977000
Barrios, E., & Ulises, D. (2020). Diseño y validación del cuestionario “Actitud hacia la investigación en estudiantes universitarios”. Revista Innova Educación, 2(2), 280–302. https://doi.org/10.35622/j.rie.2020.02.004
Böttcher-Oschmann, F., Groß, J., & Thiel, F. (2019). Validierung eines Fragenbogens zur Erfassung studentischer Forschungskompetenzen über Selbsteinschätzungen – Ein Instrument zur Evaluation forschungsorientierter Lehr-Lernarrangements. Unterrichtswissenschaft, 47(4), 495–521. https://doi.org/10.1007/s42010-019-00053-8
Bracho, M. S., Fernández, M., y Díaz, J. (2021). Técnicas e instrumentos de recolección de información: Análisis y procesamiento realizado por el investigador cualitativo. Revista Científica UISRAEL, 8(1), 107–121. https://doi.org/10.35290/rcui.v8n1.2021.400
Buntins, K., Kerres, M., & Heinemann, A. (2021). A scoping review of research instruments for measuring student engagement: In need for convergence. International Journal of Educational Research Open, 2, 100099. https://doi.org/10.1016/j.ijedro.2021.100099
Castellanos, Y. A., & Rios-González, C. M. (2017). The importance of scientific research in higher education. Medicina Universitaria, 19(74), 19–20. https://doi.org/10.1016/j.rmu.2016.11.002
Castro-Rodríguez, Y. (2021). Systematic review of the instruments for measuring research skills in higher medical education. Revista Habanera de Ciencias Médicas, 20(2), 1–14. https://www.medigraphic.com/cgi-bin/new/resumenI.cgi?IDARTICULO=108609
Castro-Rodríguez, Y. A. (2023). Marco de referencia de las competencias investigativas para la Educación Médica. Revista Cubana de Información en Ciencias de la Salud, 34(0). https://acimed.sld.cu/index.php/acimed/article/view/2190
Cater, M., Ferstel, S. D., & O’Neil, C. E. (2016). Psychometric Properties of the Inventory of Student Experiences in Undergraduate Research. The Journal of General Education, 65(3–4), 283–302. https://doi.org/10.5325/jgeneeduc.65.3-4.0283
Charumbira, M. Y., Berner, K., & Louw, Q. A. (2021). Research competencies for undergraduate rehabilitation students: A scoping review. African Journal of Health Professions Education, 13(1), 52–58. https://doi.org/10.7196/AJHPE.2021.v13i1.1229
Chen, Q., Huang, C., Castro, A. R., & Tang, S. (2021). Instruments for measuring nursing research competence: A protocol for a scoping review. BMJ Open, 11(2), e042325. https://doi.org/10.1136/bmjopen-2020-042325
Cobos, F., Peñaherrera, M., & Ortiz, A. M. (2016). Validation of a questionnaire on research-based learning with engineering students. Journal of Technology and Science Education, 6(3), 219–233. https://doi.org/10.3926/jotse.227
Cota, L. V., & Beltran-Sanchez, J. A. (2021). Propiedades métricas de cuatro subescalas para medir la competencia investigativa de docentes universitarios mexicanos. Innovación Educativa, 21(85), 1. https://dialnet.unirioja.es/servlet/articulo?codigo=8868985
da Silva, F. C., Gonçalves, E., Arancibia, B. A. V., Bento, G. G., Castro, T. L. da S., Hernandez, S. S. S., & da Silva, R. (2015). Estimators of internal consistency in health research: The use of the alpha coefficient. Revista Peruana De Medicina Experimental Y Salud Publica, 32(1), 129–138. http://www.scielo.org.pe/pdf/rins/v32n1/a19v32n1.pdf
da Silva, R. C., Ribeiro, M. D., Pereira, W., & Mollo, W. R. (2023). Mentoring in research: Development of competencies for health professionals. BMC Nursing, 22(1). https://doi.org/10.1186/s12912-023-01411-9
Duru, P., & Örsal, Ö. (2021). Development of the Scientific Research Competency Scale for nurses. Journal of Research in Nursing, 26(7), 684–700. https://doi.org/10.1177/17449871211020061
Echevarría-Guanilo, M. E., Gonçalves, N., & Romanoski, P. J. (2019). Psychometric Properties of Measurement Instruments: Conceptual Basis and Evaluation Methods - Part Ii. Texto & Contexto - Enfermagem, 28. https://www.redalyc.org/journal/714/71465278102/html/
Escobedo, M. T., Hernández, J. A., Estebané, V., & Martínez, G. (2016). Modelos de ecuaciones estructurales: Características, fases, construcción, aplicación y resultados. Ciencia & trabajo, 18(55), 16–22. https://doi.org/10.4067/S0718-24492016000100004
García-González, J. R., & Sánchez-Sánchez, P. A. (2020). Diseño teórico de la investigación: Instrucciones metodológicas para el desarrollo de propuestas y proyectos de investigación científica. Información tecnológica, 31(6), 159–170. https://doi.org/10.4067/S0718-07642020000600159
García-Peñalvo, F. J. (2022). Desarrollo de estados de la cuestión robustos: Revisiones Sistemáticas de Literatura. Education in the Knowledge Society (EKS), 23, e28600–e28600. https://doi.org/10.14201/eks.28600
Gess, C., Geiger, C., & Ziegler, M. (2019). Social-Scientific Research Competency. European Journal of Psychological Assessment, 35(5), 737–750. https://doi.org/10.1027/1015-5759/a000451
Gros, S., Canet-Vélez, O., Contreras-Higuera, W., Garcia-Expósito, J., Torralbas-Ortega, J., & Roca, J. (2022). Translation, Adaptation, and Psychometric Validation of the Spanish Version of the Attitudes towards Research and Development within Nursing Questionnaire. International Journal of Environmental Research and Public Health, 19(8), 4623. https://doi.org/10.3390/ijerph19084623
Groß, J., Wolf, R., Schladitz, S., & Wirtz, M. (2017). Assessment of educational research literacy in higher education: Construct validation of the factorial structure of an assessment instrument comparing different treatments of omitted responses. Journal for Educational Research Online, 9(2), 37–68. http://nbn-resolving.de/urn:nbn:de:0111-pedocs-148962
Guerrero-Narbajo, Y., Rosario-Quiroz, F., & Santos-Morocho, J. (2023). Diseño y validación de la Escala de Autoevaluación de Habilidades para la Investigación Formativa (EAHIF) en estudiantes universitarios. Revista Cubana de Educación Superior, 42(2), 139–155. https://revistas.uh.cu/rces/article/view/6920
Hernández, C. A., Gamboa, A. A., & Avendaño, W. R. (2021). Validación de una escala para evaluar competencias investigativas en docente de básica y media. Revista Boletín Redipe, 10(6), 393–406. https://doi.org/10.36260/rbr.v10i6.1335
Hernández, R. M., Saavedra-López, M. A., Calle-Ramirez, X. M., & Rodríguez-Fuentes, A. (2021). Index of Undergraduate Students’ Attitude towards Scientific Research: A Study in Peru and Spain. Jurnal Pendidikan IPA Indonesia, 10(3), 416–427. https://doi.org/10.15294/jpii.v10i3.30480
Howard, A., & Michael, P. G. (2019). Psychometric Properties and Factor Structure of the Attitudes Toward Research Scale in a Graduate Student Sample. Psychology Learning & Teaching, 18(3), 259–274. https://doi.org/10.1177/1475725719842695
Ianni, P. A., Samuels, E. M., Eakin, B. L., Perorazio, T. E., & Ellingrod, V. L. (2021). Assessments of Research Competencies for Clinical Investigators: A Systematic Review. Evaluation & the Health Professions, 44(3), 268–278. https://doi.org/10.1177/0163278719896392
Ipanaqué-Zapata, M., Figueroa-Quiñones, J., Bazalar-Palacios, J., Arhuis-Inca, W., Quiñones-Negrete, M., & Villarreal-Zegarra, D. (2023). Research skills for university students’ thesis in E-learning: Scale development and validation in Peru. Heliyon, 9(3), e13770. https://doi.org/10.1016/j.heliyon.2023.e13770
Jager, J., Putnick, D. L., & Bornstein, M. H. (2017). More than Just Convenient: The Scientific Merits of Homogeneous Convenience Samples. Monographs of the Society for Research in Child Development, 82(2), 13–30. https://doi.org/10.1111/mono.12296
Jeréz, I. E. H., Laza, O. U., Martorell, L. de la C. M., & Hernández, L. L. (2022). Habilidades investigativas de los licenciados en enfermería en el Instituto de Hematología e Inmunología. Revista Cubana de Hematología, Inmunología y Hemoterapia, 38(4). https://revhematologia.sld.cu/index.php/hih/article/view/1706
Kaur, R., Hakim, J., Jeremy, R., Coorey, G., Kalman, E., Jenkin, R., Bowen, D. G., & Hart, J. (2023). Students’ perceived research skills development and satisfaction after completion of a mandatory research project: Results from five cohorts of the Sydney medical program. BMC Medical Education, 23(1). https://doi.org/10.1186/s12909-023-04475-y
Madadizadeh, F., & Bahariniya, S. (2023). Tutorial on how to calculating content validity of scales in medical research. Perioperative Care and Operating Room Management, 31, 100315. https://doi.org/10.1016/j.pcorm.2023.100315
Mallidou, A. A., Borycki, E., Frisch, N., & Young, L. (2018). Research Competencies Assessment Instrument for Nurses: Preliminary Psychometric Properties. Journal of Nursing Measurement, 26(3), E159–E182. https://doi.org/10.1891/1061-3749.26.3.E159
Maric, D., Fore, G. A., Nyarko, S. C., & Varma-Nelson, P. (2023). Measurement in STEM education research: A systematic literature review of trends in the psychometric evidence of scales. International Journal of STEM Education, 10(1), 39. https://doi.org/10.1186/s40594-023-00430-x
McKechnie, D., & Fisher, M. J. (2022). Considerations when examining the psychometric properties of measurement instruments used in health. AJAN - The Australian Journal of Advanced Nursing, 39(2). https://doi.org/10.37464/2020.392.481
Merino-Soto, C., Fernández-Arata, M., Fuentes-Balderrama, J., Chans, G. M., & Toledano-Toledano, F. (2022). Research Perceived Competency Scale: A New Psychometric Adaptation for University Students’ Research Learning. Sustainability, 14(19), 12036. https://doi.org/10.3390/su141912036
Mokkink, L. B., Terwee, C. B., Knol, D. L., Stratford, P. W., Alonso, J., Patrick, D. L., Bouter, L. M., & de Vet, H. C. (2010). The COSMIN checklist for evaluating the methodological quality of studies on measurement properties: A clarification of its content. BMC Medical Research Methodology, 10(1), 22. https://doi.org/10.1186/1471-2288-10-22
Newman, I., Lim, J., & Pineda, F. (2013). Content Validity Using a Mixed Methods Approach: Its Application and Development Through the Use of a Table of Specifications Methodology. Journal of Mixed Methods Research, 7(3), 243–260. https://doi.org/10.1177/1558689813476922
Nolazco-Labajos, F. A., Guerrero, M. A., Carhuancho-Mendoza, I. M., & Saravia, G. del P. (2022). Competencia investigativa estudiantil durante la pandemia. Revista de Ciencias Sociales, XXVIII(6), 228–243. https://produccioncientificaluz.org/index.php/rcs/article/view/38834
Oviedo, H. C., & Campo-Arias, A. (2005). Aproximación al uso del coeficiente alfa de Cronbach. Revista Colombiana de Psiquiatría, 34(4), 572–580. http://www.scielo.org.co/scielo.php?script=sci_abstract&pid=S0034-74502005000400009&lng=en&nrm=iso&tlng=es
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S., … Alonso-Fernández, S. (2021). Declaración PRISMA 2020: Una guía actualizada para la publicación de revisiones sistemáticas. Revista Española de Cardiología, 74(9), 790–799. https://doi.org/10.1016/j.recesp.2021.06.016
Qiu, C., Feng, X., Reinhardt, J. D., & Li, J. (2019). Development and psychometric testing of the Research Competency Scale for Nursing Students: An instrument design study. Nurse Education Today, 79, 198–203. https://doi.org/10.1016/j.nedt.2019.05.039
Ranganathan, P., & Aggarwal, R. (2020). Study designs: Part 7 – Systematic reviews. Perspectives in Clinical Research, 11(2), 97–100. https://doi.org/10.4103/picr.PICR_84_20
Roberts, L. D., & Povee, K. (2014). A brief measure of attitudes toward mixed methods research in psychology. Frontiers in Psychology, 5. https://www.frontiersin.org/articles/10.3389/fpsyg.2014.01312
Rockinson-Szapkiw, A. (2018). The development and validation of the scholar–practitioner research development scale for students enrolled in professional doctoral programs. Journal of Applied Research in Higher Education, 10(4), 478–492. https://doi.org/10.1108/JARHE-01-2018-0011
Rodríguez, A., Caurcel, M. J., Gallardo, C. del P., & García, A. (2023). Reconocimiento y actitud hacia la investigación educativa en la Universidad. Revista Interuniversitaria de Formación del Profesorado. Continuación de la antigua Revista de Escuelas Normales, 98(37.1). https://doi.org/10.47553/rifop.v98i37.1.97824
Smith, S. E., Newsome, A. S., Hawkins, W. A., Bland, C. M., & Branan, T. N. (2020). Teaching research skills to student pharmacists: A multi-campus, multi-semester applied critical care research elective. Currents in Pharmacy Teaching and Learning, 12(6), 735–740. https://doi.org/10.1016/j.cptl.2020.01.020
Swank, J. M., & Lambie, G. W. (2016). Development of the Research Competencies Scale. Measurement and Evaluation in Counseling and Development, 49(2), 91–108. https://doi.org/10.1177/0748175615625749
Toro, R., Peña-Sarmiento, M., Avendaño-Prieto, B. L., Mejía-Vélez, S., & Bernal-Torres, A. (2022). Análisis Empírico del Coeficiente Alfa de Cronbach según Opciones de Respuesta, Muestra y Observaciones Atípicas. Revista Iberoamericana de Diagnóstico y Evaluación – e Avaliação Psicológica, 63(2), 17. https://doi.org/10.21865/RIDEP63.2.02
Torres, S. A., & Manchego, J. L. (2023). Research competencies in Ibero-American university students: A systematic review. LATAM Revista Latinoamericana de Ciencias Sociales y Humanidades, 4(1), 2784–2802. https://doi.org/10.56712/latam.v4i1.454
Tuononen, T., & Parpala, A. (2021). The role of academic competences and learning processes in predicting Bachelor’s and Master’s thesis grades. Studies in Educational Evaluation, 70, 101001. https://doi.org/10.1016/j.stueduc.2021.101001
Valderrama, M. W., Pérez, C. L., Llaque, G., & Matute, J. C. (2022). Investigative skills in university students. A systematic review. LACCEI International Multi-Conference for Engineering, Education, and Technology, 1–9. https://doi.org/10.18687/LACCEI2022.1.1.127
Ventura-León, J., & Peña-Calero, B. N. (2020). El mundo no debería girar alrededor del alfa de Cronbach ≥ ,70. Adicciones, 33(4), 369–372. https://doi.org/10.20882/adicciones.1576
Vieno, K., Rogers, K. A., & Campbell, N. (2022). Broadening the Definition of ‘Research Skills’ to Enhance Students’ Competence across Undergraduate and Master’s Programs. Education Sciences, 12(10), 642. https://doi.org/10.3390/educsci12100642
Xiao, L., & Hau, K.T. (2022). Performance of Coefficient Alpha and Its Alternatives: Effects of Different Types of Non-Normality. Educational and Psychological Measurement, 83(1), 5–27. https://doi.org/10.1177/00131644221088240
Zakariya, Y. F. (2022). Cronbach’s alpha in mathematics education research: Its appropriateness, overuse, and alternatives in estimating scale reliability. Frontiers in Psychology, 13. https://doi.org/10.3389/fpsyg.2022.1074430
Zangaro, G. A. (2019). Importance of Reporting Psychometric Properties of Instruments Used in Nursing Research. Western Journal of Nursing Research, 41(11), 1548–1550. https://doi.org/10.1177/0193945919866827
