Content Validation of a Critical Thinking Skills Test on Temperature and Heat (CTS-THT)
DOI:
https://doi.org/10.61142/esj.v4i3.405Keywords:
Critical thinking, Science education, Secondary student, Temperature and heat, Instrument ValidationAbstract
This study aims to develop and validate a Critical Thinking Skills Test on Temperature and Heat (CTS-THT) for junior high school students based on Ennis’s critical thinking framework. The research employs an instrument development design, beginning with a literature review, followed by a test blueprint operationalizing five aspects of critical thinking: basic clarification, basic support, inference, advanced clarification, and strategies and tactics into essay items. The CTS-THT assesses students’ ability to analyze arguments, interpret experimental data, define key physics terms, and justify decisions related to temperature and heat phenomena. Content validity is examined through expert judgment by five validators: two experts in physics education, two in educational assessment, and one in science content. The validators rate each item on five dimensions using a 0–1 scale, and content validity is quantified using Aiken’s V index. The results show that all 42 items achieve Aiken’s V coefficients of at least 0.60, indicating satisfactory content validity. Most items are valid without revision, while only a few require minor or substantive improvements. These findings indicate that the CTS-THT is a content-valid instrument for assessing junior high school students’ critical thinking skills on temperature and heat. Theoretically, this study contributes a domain-specific, Ennis-based instrument addressing gaps in existing critical thinking and physics tests. Practically, the CTS-THT provides a curriculum-aligned instrument to help science teachers assess and foster students’ critical thinking through more targeted instruction and assessment.
References
Aiken, L. R. (1985). Three coefficients for analyzing the reliability and validity of ratings. Educational and Psychological Measurement, 45(1), 131–142. https://doi.org/10.1177/0013164485451012
Bernard, R. M., Zhang, D., Abrami, P. C., Sicoly, F., Borokhovski, E., & Surkes, M. A. (2008). Exploring the structure of the Watson-Glaser Critical Thinking Appraisal: One scale or many subscales? Thinking Skills and Creativity, 3(1), 15–22. https://doi.org/10.1016/j.tsc.2007.11.001
Butler, H. A. (2012). Halpern critical thinking assessment predicts real-world outcomes of critical thinking. Applied Cognitive Psychology, 26(5), 721–729. https://doi.org/10.1002/acp.2851
Dewi, S. E., Setyosari, P., & Hanafi, Y. (2025). Effect of inquiry-based learning on students' critical thinking: A meta-analysis. Journal of Institutional Research South East Asia, 23(3), 18–39.
Ennis, R. H. (1985). A logical basis for measuring critical thinking skills. Educational Leadership, 43, 44–48.
Ennis, R. H. (1993). Critical thinking assessment. Theory into Practice, 32(3), 179–186. https://doi.org/10.1080/00405849309543594
Gavronskaya, Y., Larchenkova, L., Berestova, A., Latysheva, V., & Smirnov, S. (2022). The development of critical thinking skills in mobile learning: Fact-checking and getting rid of cognitive distortions. International Journal of Cognitive Research in Science, Engineering and Education, 10(2), 51–68. https://doi.org/10.23947/2334-8496-2022-10-2-51-68
Ghanizadeh, A., Bajestani, G. S., Hosseinpour, F., Hosseini, A., Makhloughi, F., & Toosi, M. B. (2024). Mindfulness-enhancing instruction (MEI): Contributions to electroencephalogram (EEG) dynamics, higher order thinking skills (HOTS), and effective learning. Thinking Skills and Creativity, 53, 101611. https://doi.org/10.1016/j.tsc.2024.101611
Illene, S., Feranie, S., & Siahaan, P. (2023). Create multiple-choice tests based on experimental activities to assess students' 21st century skills in the heat and heat transfer topic. Journal of Education and Learning (EduLearn), 17(1), 44–57. https://doi.org/10.11591/edulearn.v17i1.20540
Joseph, V., Shaharudin, S. M., Hamid, N. A., & Rajak, N. A. (2025). Development and evaluation of the BVEE kit for teaching circle topics: A validity and usability study in Form 2 mathematics. International Journal of Information and Education Technology, 15(2). https://doi.org/10.18178/ijiet.2025.15.2.2236
Masek, A., & Yamin, M. (2012). The impact of instructional methods on critical thinking: A comparison of problem-based learning and conventional approach in engineering education. International Scholarly Research Network, 2012, Article 759241. https://doi.org/10.1155/2012/759241
Megananda, A., Muzayyanah, E., Darmayanti, H. P., & Priana, Z. I. (2021). Development of digital distance measurement instrument based on Arduino Uno for physics practicum. IMPULSE: Journal of Research and Innovation in Physics Education, 1(2), 80–88. https://doi.org/10.29303/impulse.v1i2.85
Mu’aziyah, S. E. S., Hidayat, T., Sriyati, S., & Lutianasari, L. (2023). Implementation of the Merdeka Curriculum using Citizen Science Project Weather-it to improve critical thinking skills of junior high school students. Jurnal Penelitian Pendidikan IPA, 9(3), 1470–1479. https://doi.org/10.29303/jppipa.v9i3.2277
Nurhayati, N., Wahyudi, W., Suhandi, A., Muslim, M., Kaniawati, I., & Saputri, D. F. (2026). Enhancing prospective teachers' skills in designing physics learning: Developing electronic learning materials with interactive reading, scaffolding and modeling (IRTaMS) strategies. IJIS Edu: Indonesian Journal of Integrated Science Education, 8(1), 27–45.
Olivares-Rodríguez, C., Guenaga, M., & Garaizar, P. (2017). Automatic assessment of creativity in heuristic problem-solving based on query diversity. Dyna, 92(4), 449–455. https://doi.org/10.6036/8243
Parkkinen, V.-P., Wallmann, C., Wilde, M., Clarke, B., Illari, P., Kelly, M. P., Norell, C., Russo, F., Shaw, B., & Williamson, J. (2018). Evaluating evidence of mechanisms. In Evaluating evidence of mechanisms in medicine (pp. 77–90). Springer. https://doi.org/10.1007/978-3-319-94610-8_6
Pina, J., Ellis, G. W., Rudnitsky, A., Mazur, R., McGinnis-Cavanaugh, B., & Huff, I. (2021, July). Developing a measure to capture middle school students' interpretive understanding of engineering design. ASEE Virtual Annual Conference Content Access. https://doi.org/10.18260/1-2--36934
Popescu, A., & Morgan, M. (2007). Teaching information evaluation and critical thinking skills in physics classes. The Physics Teacher, 45(8), 507–510. https://doi.org/10.1119/1.2798365
Pramasdyahsari, A. S., Setyawati, R. D., Aini, S. N., Nusuki, U., Arum, J. P., Astutik, L. D., Widodo, W., Zuliah, N., & Salmah, U. (2023). Fostering students' mathematical critical thinking skills on number patterns through digital book STEM PjBL. Eurasia Journal of Mathematics, Science and Technology Education, 19(7). https://doi.org/10.29333/ejmste/13342
Prasojo, L. D., Wijayanti, W., Yuliana, L., Agus, N., Habibi, A., & Yaakob, M. F. M. (2020). Instruments' validation of access to motivation, skills, and use of digital technology: EFL context in Indonesia. Studies in English Language and Education, 7(2), 308–322. https://doi.org/10.24815/siele.v7i2.16788
Rahayu, S., & Alsulami, N. M. (2024). Assessing higher order thinking skills of the 21st century learners using socio-scientific issues as a context. AIP Conference Proceedings, 3106(1), 070009. https://doi.org/10.1063/5.0214793
Saputri, M., Saminan, S., & Nisa, Z. (2026). Differences in students’ critical thinking and collaboration skills between problem-based learning and discovery learning in physics. Equator Science Journal, 4(2), 210–222. https://doi.org/10.61142/esj.v4i2.372
Sartono, B., Sunarno, W., Prayitno, B. A., & Indriyanti, N. Y. (2025). Development and validation of a critical thinking assessment on temperature and heat for secondary physics education. Educational Process: International Journal, 16, 264. https://doi.org/10.22521/edupij.2025.16.264
Seibert, S. A. (2021). Problem-based learning: A strategy to foster Generation Z's critical thinking and perseverance. Teaching and Learning in Nursing, 16(1), 85–88. https://doi.org/10.1016/j.teln.2020.09.002
Shafie, H., Majid, F. A., & Ismail, I. S. (2022). Developing a 21st century technological pedagogical content knowledge (TPACK) instrument: Content validity and reliability. International Journal of Education, 14(3), 100–115. https://doi.org/10.5296/ije.v14i3.19980
Susiyawati, E., Sudibyo, E., & Sari, D. A. P. (2021). Development and validation of an instrument for assessing middle school students' critical thinking skills. International Journal of Assessment and Evaluation, 28(2), 1–13. https://doi.org/10.18848/2327-7920/CGP/V28I02/1-13
Suyasa, P. W. A., & Divayana, D. G. H. (2022). Instrumen evaluasi model discrepancy-CSE-UCLA dalam rangka menunjang evaluasi efektivitas pelaksanaan pembelajaran synchronous. Jurnal Pedagogi dan Pembelajaran, 5(2), 197–207. https://doi.org/10.23887/jp2.v5i2.48447
Suyasa, P. W. A., Kurniawan, P. S., Ariawan, I. P. W., Sugandini, W., Adnyawati, N. D. M. S., Budhyani, I. D. A. M., & Divayana, D. G. H. (2019). Empowerment of CSE-UCLA model based on Glickman quadrant aided by visual application to evaluate the blended learning program on SMA Negeri 1 Ubud. Journal of Theoretical and Applied Information Technology, 96(18), 6203–6219. https://doi.org/10.33193/JATIT.2020.96.18.6203
Velmovská, K., & Bartošovič, L. (2016). Developing critical thinking skills in physics classes. In Critical thinking: Theories, methods and challenges (pp. 1–43).
Wei, J. H., Chuang, H. H., & Smith, T. J. (2022). The relationship between a school culture's openness to creative solutions and inquiry-based teaching practices. Journal of Creative Behavior, 56(3), 382–395. https://doi.org/10.1002/jocb.535
Wesnawa, I. G. A., Kartowagiran, B., Jaedun, A., Hamdi, S., Hadi, S., Sunendar, D., Abdul, L., Laliyo, R., Christiawan, P. I., & Gede, D. (2022). Content validation of digital instrument for measurement of pedagogic competence for social science teacher candidates in the Industrial Revolution 4.0 era in Indonesia. International Journal of Information and Education Technology, 12(12), 1424–1430. https://doi.org/10.18178/ijiet.2022.12.12.1767
Widyaningsih, S. W., Yusuf, I., Prasetyo, Z. K., & Istiyono, E. (2021). The development of the HOTS test of physics based on modern test theory: Question modeling through e-learning of Moodle LMS. International Journal of Instruction, 14(4), 51–68. https://doi.org/10.29333/iji.2021.1444a
Yahfizham, I., Yusti, I., & Hamzah, M. L. (2021). The test of construct validity for one-factor model. Journal of Educational Research and Evaluation, 5(3), 429–435. https://doi.org/10.15294/jered.v5i3.4093
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Sunaryo Romli, Andi Suhandi, Muslim Muslim, Ida Kaniawati, Mohd Zaidi Bin Amiruddin
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
