Common Errors of Reliability and Validity Testing in Guidance and Counselling Research
DOI:
https://doi.org/10.51278/bse.v3i3.868Keywords:
Common Errors, Reliability and Validity, Guidance and CounselingAbstract
This research aims to identify common problems that occur in reliability and validity testing in research in the field of guidance and counselling. This research uses the Systematic Literature Review (SLR) method, which examines and reviews several articles related to common errors in reliability and validity testing. This research searched for several articles from various sources such as Google Scholar, Semantic Scholar, Springer, Routledge Taylor & Francis Group, Research Gate, and frontiersin.org within the past 5 years (2013-2023). The articles that have been searched are then analysed for common errors that appear related to reliability and validity testing, the results of this analysis are presented in tabular form and explained descriptively. The results of this study indicate that several studies in the field of counselling have 9 common errors that are often made in conducting quantitative research. These errors are: not including reliability and validity, not iterating and revising unreliable items, not conducting content validity analysis, not conducting construct validity analysis, mismatching the instrument with the characteristics of the item, the absence of judgment validity, not describing favourable and unfavourable items, limited samples on the instrument, and doubtful construct validity and the need for reliability testing on a wide scale. His research can be a study and reference material in conducting quantitative research, especially in the field of guidance and counselling.
Keywords: Common Errors, Reliability and Validity, Guidance and Counseling
References
Ahmaddien, I., & Syarkani, Y. (2019). Statistika Terapan dengan Sistem SPSS. Bandung: ITB Press.
Astuti, B. (2019). Profil Kemandirian Belajar Mahasiswa Bimbingan dan Konseling. Jurnal Penelitian Ilmu Pendidikan, 12(1), 63-74. https://doi.org/10.21831/jpipfip.v12i1.24327.
Atitsogbe, K. A., Bernaud, J. L. (2022). Vocational Values Scale: Initial Development and Testing of the Student form (VVS–S). Int J Educ Vocat Guidance. https://doi.org/10.1007/s10775-022-09561-z
Awaluddin. T & Wirda, H. (2023). Development of Strengthening Good Character Education Scale as a Need Assessment Instrumen for Guidance and Counseling Teachers. JISAE: Journal of Indonesian Student Assessment and Evaluation, 9(1), 68-78. https://doi.or/10.21009/JISAE
Blau, G., & DiMino, J. (2019). Prepared for Counseling: Introducing a Short Scale and Correlates. Measurement and Evaluation in Counseling and Development, 1–11. http://dx.doi.org/10.1080/07481756.2019.1595816
Boerchi, D., & Tagliabue, S. (2018). Assessing Students’ Perception of Parental Career-related Support: Development of a New Scale and A New Taxonomy. International Journal for Educational and Vocational Guidance, 18(2), 181–201. http://dx.doi.org/10.1007/s10775-017-9354-1
Caldwell, S., Wusik, K., He, H. et al. Development and Validation of the Genetic Counseling Self-Efficacy Scale (GCSES). J Genet Counsel, 27(2), 1248–1257. https://doi.org/10.1007/s10897-018-0249-1
Chan, S. W., Ismail, Z., & Sumintono, B. (2016). Assessing Statistical Reasoning in Descriptive Statistics: A Qualitative Meta-analysis. Jurnal Teknologi (Sciences & Engineering), 72(2), 1-6. http://dx.doi.org/10.11113/jt.v78.8995.
Choi, S. I., & Gerstein, L. H. (2019). Beliefs and Evaluations About Counseling Services (BEACS): Scale Development. Measurement and Evaluation in Counseling and Development, 1–18. http://dx.doi.org/10.1080/07481756.2019.1691458
Cobb, G. W., & Moore, D. S. (1997). Mathematics, Statistics, and Teaching. The American Mathematical Monthly, 104(9), 801–823. https://doi.org/10.1080/00029890.1997.11990723.
Danial, E., dan Wasriah, N. (2009). Metode Penulisan Karya Ilmiah. Bandung: Laboratorium Pendidikan Kewarganegaraan UPI.
Darmawan, E. & Suryahadikusumah, A. R. (2022). Learning Styles Profile and Their Impact on Learning Behavior of Islamic Guidance and Counseling Students During Pandemic. Psikis: Jurnal Psikologi Islami, 8(2), 203-208.
Eliza, R., Ahmad, R., Asnah, M.I., & Afdal (2023). Pengembangan Modul Bimbingan Karir Untuk Meningkatkan Kematangan Karir Siswa Sekolah Menengah. G-Couns: Jurnal Bimbingan dan Konseling, 7(3), 671-682
Fitriyah, F. F. (2019). Pengaruh Permainan Tradisional Gobak Sodor dalam Bimbingan Kelompok terhadap Peningkatan Interaksi Sosial Anak Autis. Education and Human Development Journal, 5(1), 13-20. http://dx.doi.org/10.33086/ehdj.v5i1.1293
Ghozali, I. (2021). Desain Penelitian Kuantitatif dan Kualitatif untuk Akuntansi, Bisnis, dan Ilmu Sosial Lainnya. Yoga Pratama.
Gonzalez, E., Sperandio, K. R., Mullen, P. R., & Tuazon, V. E. (2020). Development and Initial Testing of the Multidimensional Cultural Humility Scale. Measurement and Evaluation in Counseling and Development, 1–16. https://doi.org/10.1080/07481756.2020.1745648
Hamlet, D. (1992). Are We Testing for True Reliability?. IEEE Software, 21-27.
Heale, R. & Twycross, A. (2015). Validity and Reliability in Quantitative Studies. Evid Based Nurs, 18(3), 66-67.
Jared, M. L., Kok-Mun, N., & David, B. V. (2019). The Counseling Training Environment Scale: Initial Development and Validity of a Self-Report Measure to Assess the Counseling Training Environment. Measurement and Evaluation in Counseling and Development, http://dx.doi.org/10.1080/07481756.2019.1595813
Keum, B. T., Brady, J. L., Sharma, R., Lu, Y., Kim, Y. H., & Thai, C. J. (2018). Gendered Racial Microaggressions Scale for Asian American Women: Development and initial validation. Journal of Counseling Psychology, 65(5), 571–585. https://doi.org/10.1037/cou0000305
Kurniasih, N. (2020). Peran Statistik dalam Penelitian. Al-Muqkidz: Jurnal Kajian Keislaman, 8(2), 273-279.
Mahaly, S. (2021). Efektivitas Pelaksanaan Layanan Bimbingan PribadiOleh Guru Bimbingan Konseling. Al-Ittizaan: Jurnal Bimbingan Konseling Islam, 4(1), 1-6. http://dx.doi.org/10.24014/ittizaan.v4i1.13238
Martias, L. D. (2021). Statistika Deskriptif Kumpulan Informasi. FIHRIS: Jurnal Ilmu Perpustakaan dan Informasi, 16(1), 40-59.
Matondang, Z. (2009). Validitas dan Reliabilitas suatu Instrumen Penelitian. Jurnal Tabularasa PPs Unimed, 6(1), 87-97.
Nasution, H. F. (2016). Instrumen Penelitian dan Urgensinya dalam Penelitian Kuantitatif. Al-Masharif: Jurnal Ekonomi dan Keislaman, 4(1), 59-75. https://doi.org/10.24952/masharif.v4i1.721
Nawantara, R. D., & Sancaya, S. A. (2021). Skala Emotional Well Being (EWB) bagi Mahasiswa di Masa Pandemi COVID 19. Jurnal Bimbingan Dan Konseling Borneo, 3(2). https://doi.org/10.35334/jbkb.v3i2.2346
Rahman, I. K., Rosidah, N.S., & Tamam, A.M. (2023). Development of a Scale for Measuring the Competencies of Islamic Counselors. Islamic Guidance and Counseling Journal, 6(1), 28-44, https://doi.org/10.25217/igcj.v6i1.3133.
Rahman, T. G., & Ilyas, A. (2019). Perilaku Narsistik Pengguna Media Sosial di Kalangan Mahasiswa dan Implikasi dalam Layanan Bimbingan Dan Konseling. E-Jurnal Inovasi Pembelajaran SD, 7(4), 1-8. http://dx.doi.org/10.24036/e-jipsd.v8i2.7905
Rangkuti, A. A. (2017). Statistika Inferensial untuk Psikologi dan Pendidikan. Jakarta: Kencana.
Ridni Eliza, Riska Ahmad, Mega Iswari Biran Asnah, & Afdal. (2023). Pengembangan Modul Bimbingan Karir Untuk Meningkatkan Kematangan Karir Siswa Sekolah Menengah. G-Couns: Jurnal Bimbingan Dan Konseling, 7(03), 671–682. https://doi.org/10.31316/gcouns.v7i03.4947
Riyanto, J., Lestari, L. P. S., & Suranata, K. (2023). Pengembangan Panduan Bimbingan Karir Berbasis Pendidikan Kecakapan Hidup (Life Skill) dengan Pendekatan Teori Karir Super untuk Meningkatkan Perencanaan Karir Siswa di SMK Negeri 2 Singaraja. Jurnal EDUCATIO: Jurnal Pendidikan Indonesia, 9(1), 479. https://doi.org/10.29210/1202323065
Saputra, K. F., Suarni, N. K., & Dantes, N. (2020). Development of a Self-Efficacy Scale for Vocational School. Bisma The Journal of Counseling, 4(2), 154-160, https://doi.org/10.15345/IOJES.2016.02.007
Shaheen, L., Hamid, H. S. A., & Yan-Li, S. (2022). Validation of Self-Efficacy for Self-Regulation of Academic Writing Scale: An EFA and CFA with Measurement Invariance. International Journal of Education, Psychology and Counseling, 7(46), 18-33. https://doi.org/10.35631/IJEPC.746002. DOI:10.35631/IJEPC.746002.
Stalikas, A., Kyriazos, T., Yotsidi, V., & Prassa, K. (2018). Using Bifactor EFA, Bifactor CFA and Exploratory Structural Equation Modeling to Validate Factor Structure of the Meaning in Life Questionnaire, Greek Version. Psychology, 09, 348-371.
Sumanto. (2014). Statistika Deskriptif. jakarta: PT Buku Seru.
Sutton, G. W. (2020). Applied Statistics Concepts for Counselors Second Edition. USA: Sunflower Press
Triandini, E., Jayanatha, S., Indrawan, A., Werla Putra, G., & Iswara, B. (2019). Metode Systematic Literature Review untuk Identifikasi Platform dan Metode Pengembangan Sistem Informasi di Indonesia. Indonesian Journal of Information Systems, 1(2), 63–77. https://doi.org/10.24002/ijis.v1i2.1916.
Turner, M., Kitchenham, B., Brereton, P., Charters, S., & Budgen, D. (2010). Does the technology acceptance model predict actual use? A systematic literature review. Information and Software Technology, 52(5), 463–479. https://doi.org/10.1016/j.infsof.2009.11.005
Wong, Y. J., Shea, M., Wang, S.-Y., & Cheng, J. (2019). The Encouragement Character Strength Scale: Scale development and psychometric properties. Journal of Counseling Psychology, 66(3), 362–374.
Astuti, B. (2019). Profil Kemandirian Belajar Mahasiswa Bimbingan dan Konseling. Jurnal Penelitian Ilmu Pendidikan, 12(1), 63-74. https://doi.org/10.21831/jpipfip.v12i1.24327.
Atitsogbe, K. A., Bernaud, J. L. (2022). Vocational Values Scale: Initial Development and Testing of the Student form (VVS–S). Int J Educ Vocat Guidance. https://doi.org/10.1007/s10775-022-09561-z
Awaluddin. T & Wirda, H. (2023). Development of Strengthening Good Character Education Scale as a Need Assessment Instrumen for Guidance and Counseling Teachers. JISAE: Journal of Indonesian Student Assessment and Evaluation, 9(1), 68-78. https://doi.or/10.21009/JISAE
Blau, G., & DiMino, J. (2019). Prepared for Counseling: Introducing a Short Scale and Correlates. Measurement and Evaluation in Counseling and Development, 1–11. http://dx.doi.org/10.1080/07481756.2019.1595816
Boerchi, D., & Tagliabue, S. (2018). Assessing Students’ Perception of Parental Career-related Support: Development of a New Scale and A New Taxonomy. International Journal for Educational and Vocational Guidance, 18(2), 181–201. http://dx.doi.org/10.1007/s10775-017-9354-1
Caldwell, S., Wusik, K., He, H. et al. Development and Validation of the Genetic Counseling Self-Efficacy Scale (GCSES). J Genet Counsel, 27(2), 1248–1257. https://doi.org/10.1007/s10897-018-0249-1
Chan, S. W., Ismail, Z., & Sumintono, B. (2016). Assessing Statistical Reasoning in Descriptive Statistics: A Qualitative Meta-analysis. Jurnal Teknologi (Sciences & Engineering), 72(2), 1-6. http://dx.doi.org/10.11113/jt.v78.8995.
Choi, S. I., & Gerstein, L. H. (2019). Beliefs and Evaluations About Counseling Services (BEACS): Scale Development. Measurement and Evaluation in Counseling and Development, 1–18. http://dx.doi.org/10.1080/07481756.2019.1691458
Cobb, G. W., & Moore, D. S. (1997). Mathematics, Statistics, and Teaching. The American Mathematical Monthly, 104(9), 801–823. https://doi.org/10.1080/00029890.1997.11990723.
Danial, E., dan Wasriah, N. (2009). Metode Penulisan Karya Ilmiah. Bandung: Laboratorium Pendidikan Kewarganegaraan UPI.
Darmawan, E. & Suryahadikusumah, A. R. (2022). Learning Styles Profile and Their Impact on Learning Behavior of Islamic Guidance and Counseling Students During Pandemic. Psikis: Jurnal Psikologi Islami, 8(2), 203-208.
Eliza, R., Ahmad, R., Asnah, M.I., & Afdal (2023). Pengembangan Modul Bimbingan Karir Untuk Meningkatkan Kematangan Karir Siswa Sekolah Menengah. G-Couns: Jurnal Bimbingan dan Konseling, 7(3), 671-682
Fitriyah, F. F. (2019). Pengaruh Permainan Tradisional Gobak Sodor dalam Bimbingan Kelompok terhadap Peningkatan Interaksi Sosial Anak Autis. Education and Human Development Journal, 5(1), 13-20. http://dx.doi.org/10.33086/ehdj.v5i1.1293
Ghozali, I. (2021). Desain Penelitian Kuantitatif dan Kualitatif untuk Akuntansi, Bisnis, dan Ilmu Sosial Lainnya. Yoga Pratama.
Gonzalez, E., Sperandio, K. R., Mullen, P. R., & Tuazon, V. E. (2020). Development and Initial Testing of the Multidimensional Cultural Humility Scale. Measurement and Evaluation in Counseling and Development, 1–16. https://doi.org/10.1080/07481756.2020.1745648
Hamlet, D. (1992). Are We Testing for True Reliability?. IEEE Software, 21-27.
Heale, R. & Twycross, A. (2015). Validity and Reliability in Quantitative Studies. Evid Based Nurs, 18(3), 66-67.
Jared, M. L., Kok-Mun, N., & David, B. V. (2019). The Counseling Training Environment Scale: Initial Development and Validity of a Self-Report Measure to Assess the Counseling Training Environment. Measurement and Evaluation in Counseling and Development, http://dx.doi.org/10.1080/07481756.2019.1595813
Keum, B. T., Brady, J. L., Sharma, R., Lu, Y., Kim, Y. H., & Thai, C. J. (2018). Gendered Racial Microaggressions Scale for Asian American Women: Development and initial validation. Journal of Counseling Psychology, 65(5), 571–585. https://doi.org/10.1037/cou0000305
Kurniasih, N. (2020). Peran Statistik dalam Penelitian. Al-Muqkidz: Jurnal Kajian Keislaman, 8(2), 273-279.
Mahaly, S. (2021). Efektivitas Pelaksanaan Layanan Bimbingan PribadiOleh Guru Bimbingan Konseling. Al-Ittizaan: Jurnal Bimbingan Konseling Islam, 4(1), 1-6. http://dx.doi.org/10.24014/ittizaan.v4i1.13238
Martias, L. D. (2021). Statistika Deskriptif Kumpulan Informasi. FIHRIS: Jurnal Ilmu Perpustakaan dan Informasi, 16(1), 40-59.
Matondang, Z. (2009). Validitas dan Reliabilitas suatu Instrumen Penelitian. Jurnal Tabularasa PPs Unimed, 6(1), 87-97.
Nasution, H. F. (2016). Instrumen Penelitian dan Urgensinya dalam Penelitian Kuantitatif. Al-Masharif: Jurnal Ekonomi dan Keislaman, 4(1), 59-75. https://doi.org/10.24952/masharif.v4i1.721
Nawantara, R. D., & Sancaya, S. A. (2021). Skala Emotional Well Being (EWB) bagi Mahasiswa di Masa Pandemi COVID 19. Jurnal Bimbingan Dan Konseling Borneo, 3(2). https://doi.org/10.35334/jbkb.v3i2.2346
Rahman, I. K., Rosidah, N.S., & Tamam, A.M. (2023). Development of a Scale for Measuring the Competencies of Islamic Counselors. Islamic Guidance and Counseling Journal, 6(1), 28-44, https://doi.org/10.25217/igcj.v6i1.3133.
Rahman, T. G., & Ilyas, A. (2019). Perilaku Narsistik Pengguna Media Sosial di Kalangan Mahasiswa dan Implikasi dalam Layanan Bimbingan Dan Konseling. E-Jurnal Inovasi Pembelajaran SD, 7(4), 1-8. http://dx.doi.org/10.24036/e-jipsd.v8i2.7905
Rangkuti, A. A. (2017). Statistika Inferensial untuk Psikologi dan Pendidikan. Jakarta: Kencana.
Ridni Eliza, Riska Ahmad, Mega Iswari Biran Asnah, & Afdal. (2023). Pengembangan Modul Bimbingan Karir Untuk Meningkatkan Kematangan Karir Siswa Sekolah Menengah. G-Couns: Jurnal Bimbingan Dan Konseling, 7(03), 671–682. https://doi.org/10.31316/gcouns.v7i03.4947
Riyanto, J., Lestari, L. P. S., & Suranata, K. (2023). Pengembangan Panduan Bimbingan Karir Berbasis Pendidikan Kecakapan Hidup (Life Skill) dengan Pendekatan Teori Karir Super untuk Meningkatkan Perencanaan Karir Siswa di SMK Negeri 2 Singaraja. Jurnal EDUCATIO: Jurnal Pendidikan Indonesia, 9(1), 479. https://doi.org/10.29210/1202323065
Saputra, K. F., Suarni, N. K., & Dantes, N. (2020). Development of a Self-Efficacy Scale for Vocational School. Bisma The Journal of Counseling, 4(2), 154-160, https://doi.org/10.15345/IOJES.2016.02.007
Shaheen, L., Hamid, H. S. A., & Yan-Li, S. (2022). Validation of Self-Efficacy for Self-Regulation of Academic Writing Scale: An EFA and CFA with Measurement Invariance. International Journal of Education, Psychology and Counseling, 7(46), 18-33. https://doi.org/10.35631/IJEPC.746002. DOI:10.35631/IJEPC.746002.
Stalikas, A., Kyriazos, T., Yotsidi, V., & Prassa, K. (2018). Using Bifactor EFA, Bifactor CFA and Exploratory Structural Equation Modeling to Validate Factor Structure of the Meaning in Life Questionnaire, Greek Version. Psychology, 09, 348-371.
Sumanto. (2014). Statistika Deskriptif. jakarta: PT Buku Seru.
Sutton, G. W. (2020). Applied Statistics Concepts for Counselors Second Edition. USA: Sunflower Press
Triandini, E., Jayanatha, S., Indrawan, A., Werla Putra, G., & Iswara, B. (2019). Metode Systematic Literature Review untuk Identifikasi Platform dan Metode Pengembangan Sistem Informasi di Indonesia. Indonesian Journal of Information Systems, 1(2), 63–77. https://doi.org/10.24002/ijis.v1i2.1916.
Turner, M., Kitchenham, B., Brereton, P., Charters, S., & Budgen, D. (2010). Does the technology acceptance model predict actual use? A systematic literature review. Information and Software Technology, 52(5), 463–479. https://doi.org/10.1016/j.infsof.2009.11.005
Wong, Y. J., Shea, M., Wang, S.-Y., & Cheng, J. (2019). The Encouragement Character Strength Scale: Scale development and psychometric properties. Journal of Counseling Psychology, 66(3), 362–374.
Downloads
Published
2023-12-22
How to Cite
Azzahrah, H., Ahman, A., Fauzi, I., Hafiyani, S., & Agustina, S. (2023). Common Errors of Reliability and Validity Testing in Guidance and Counselling Research. Bulletin of Science Education, 3(3), 289–298. https://doi.org/10.51278/bse.v3i3.868
Issue
Section
Articles
