Development and Rasch Validation of an Energy Literacy and Sustainable Behavior Instrument for High School Students

Neng Sriatun; Didit Ardianto; Indarini Dwi Pursitasari

doi:10.61142/esj.v4i3.502

Authors

Neng Sriatun Universitas Pakuan https://orcid.org/0009-0009-8266-097X
Didit Ardianto Universitas Pakuan https://orcid.org/0000-0002-7019-1231
Indarini Dwi Pursitasari Universitas Pakuan https://orcid.org/0000-0003-1988-8241

DOI:

https://doi.org/10.61142/esj.v4i3.502

Keywords:

Education for sustainable development, Energy literacy, Instrument validation, Rasch measurement, Sustainable behavior

Abstract

This study aimed to develop an instrument and obtain preliminary validity and psychometric evidence for measuring energy literacy and sustainable behavior among senior high school students. The instrument was developed using the ADDIE model and comprised three dimensions: energy knowledge, energy attitudes, and sustainable behavior. The initial instrument contained 64 items: 28 dichotomous knowledge items, 16 four-point Likert attitude items, and 20 four-point frequency behavior items. Four experts assessed content validity using the Content Validity Index (CVI), and Rasch analysis examined responses from 94 students (37 Grade X and 57 Grade XI). The I-CVI for all items and the S-CVI/Ave were both 1.000. Rasch analysis identified three misfitting items—Items 2 and 10 in energy knowledge and Item 9 in sustainable behavior. Removing these items resulted in a 61-item instrument. Person reliability ranged from 0.79 to 0.84, while item reliability ranged from 0.84 to 0.93. Wright maps indicated reasonable central item coverage but weaker targeting at higher levels of energy attitudes and sustainable behavior. Principal Component Analysis of Rasch Residuals indicated that the residual structure requires further investigation; strict unidimensionality was not conclusively established. Exploratory Welch’s t-tests showed higher person measures among Grade XI students in energy knowledge (p < .001), energy attitudes (p = .004), and sustainable behavior (p = .044). Overall, the findings provide preliminary evidence rather than definitive validation. Larger and more diverse samples are needed to further examine the instrument’s internal structure and generalizability.

References

Agbedahin, A. V. (2019). Sustainable development, education for sustainable development, and the 2030 Agenda for Sustainable Development: Emergence, efficacy, eminence, and future. Sustainable Development, 27(4), 669–680. https://doi.org/10.1002/sd.1931

Ajzen, I. (1991). The theory of planned behavior. Organizational Behavior and Human Decision Processes, 50(2), 179–211. https://doi.org/10.1016/0749-5978(91)90020-T

Almanasreh, E., Moles, R., & Chen, T. F. (2019). Evaluation of methods used for estimating content validity. Research in Social and Administrative Pharmacy, 15(2), 214–221. https://doi.org/10.1016/j.sapharm.2018.03.066

Azli Syam, N. A., Isa, N. K. M., Yunos, M. Y. M., Isa, N. J. M., & Sormin, D. (2026). Content validity of a sustainable food management assessment instrument for island hotels using I-CVI and mean analysis. Discover Applied Sciences, 8(3), Article 331. https://doi.org/10.1007/s42452-026-08340-7

Blackwood, J., Carpentier, S., Deng, W., & Van de Winckel, A. (2023). Preliminary Rasch analysis of the multidimensional assessment of interoceptive awareness in adults with stroke. PLOS ONE, 18(6), Article e0286657. https://doi.org/10.1371/journal.pone.0286657

Boateng, G. O., Neilands, T. B., Frongillo, E. A., Melgar-Quiñonez, H. R., & Young, S. L. (2018). Best practices for developing and validating scales for health, social, and behavioral research: A primer. Frontiers in Public Health, 6, Article 149. https://doi.org/10.3389/fpubh.2018.00149

Boone, W. J. (2016). Rasch analysis for instrument development: Why, when, and how? CBE—Life Sciences Education, 15(4), Article rm4. https://doi.org/10.1187/cbe.16-04-0148

Chodkowska-Miszczuk, J., Kola-Bezka, M., Lewandowska, A., & Martinát, S. (2021). Local communities' energy literacy as a way to rural resilience—An insight from inner peripheries. Energies, 14(9), Article 2575. https://doi.org/10.3390/en14092575

Conner, M., & Norman, P. (2022). Understanding the intention-behavior gap: The role of intention strength. Frontiers in Psychology, 13, Article 923464. https://doi.org/10.3389/fpsyg.2022.923464

Dangkua, T., Mooduto, Y., & Tilome, A. (2022). Energy literacy education characteristics in Gorontalo City, Indonesia: Cognitive scale. Journal La Lifesci, 3(2), 82–91. https://doi.org/10.37899/journallalifesci.v3i2.608

Das, R. R., & Richman, R. (2022). The development and application of a public energy literacy instrument. Canadian Journal of Science, Mathematics and Technology Education, 22(1), 42–67. https://doi.org/10.1007/s42330-022-00196-4

DeWaters, J. E., & Powers, S. E. (2011). Energy literacy of secondary students in New York State (USA): A measure of knowledge, affect, and behavior. Energy Policy, 39(3), 1699–1710. https://doi.org/10.1016/j.enpol.2010.12.049

DeWaters, J., & Powers, S. (2013). Establishing measurement criteria for an energy literacy questionnaire. The Journal of Environmental Education, 44(1), 38–55. https://doi.org/10.1080/00958964.2012.711378

Dick, W., Carey, L., & Carey, J. O. (2001). The systematic design of instruction (5th ed.). Longman.

Gao, Y., Chen, X., Zhang, S., Yang, J., Liu, R., Moyle, W., Pu, L., & Shen, W. (2025). Confirmatory factor analyses of the Mandarin Chinese version of the perceived stressors in intensive care units among healthcare professionals. Frontiers in Public Health, 13, Article 1434440. https://doi.org/10.3389/fpubh.2025.1434440

Hakam, L. I., Hakam, D. F., Wiryono, S. K., & Rahadi, R. A. (2025). The impact of sustainability literacy, social support, and attitudes towards innovation on sustainable energy consumption among Indonesian youth. Frontiers in Sustainable Cities, 7, Article 1606031. https://doi.org/10.3389/frsc.2025.1606031

Ismail, I., Riandi, R., Kaniawati, I., Sopandi, W., Supriyadi, S., Suhendar, S., & Hidayat, F. A. (2024). Gender roles in understanding and implementing green energy technology in Indonesian schools: Rasch analysis. Qubahan Academic Journal, 4(3), 298–314. https://doi.org/10.48161/qaj.v4n3a752

Isnawati, I., Sriyati, S., Agustin, R. R., Supriyadi, S., Kasi, Y. F., & Ismail, I. (2024). Analysis of question difficulty levels based on science process skills indicators using the Rasch model. Tadris: Jurnal Keguruan dan Ilmu Tarbiyah, 9(1), 31–41. https://doi.org/10.24042/tadris.v9i1.22410

Janmaimool, P., & Khajohnmanee, S. (2019). Roles of environmental system knowledge in promoting university students' environmental attitudes and pro-environmental behaviors. Sustainability, 11(16), Article 4270. https://doi.org/10.3390/su11164270

Jekauc, D., Voelkle, M. C., Giurgiu, M., & Nigg, C. R. (2025). Unveiling the multidimensional nature of the intention–behavior gap: Dynamics of intentions and behaviors. European Journal of Health Psychology, 32(1), 34–50. https://doi.org/10.1027/2512-8442/a000162

King, D., & Henderson, S. (2018). Context-based learning in the middle years: Achieving resonance between the real-world field and environmental science concepts. International Journal of Science Education, 40(10), 1221–1238. https://doi.org/10.1080/09500693.2018.1470352

Li, C., Lin, Y., Guo, H. Y., Zhao, H., Xiong, J., Xiao, X. Y., & Zhang, L. (2025). Psychometric evaluation of the Online Education Student Satisfaction Scale: A translation and validation study among Chinese nursing students. BMC Nursing, 24(1), Article 1193. https://doi.org/10.1186/s12912-025-03864-6

Oe, H., Yamaoka, Y., & Ochiai, H. (2022). A qualitative assessment of community learning initiatives for environmental awareness and behaviour change: Applying UNESCO Education for Sustainable Development (ESD) framework. International Journal of Environmental Research and Public Health, 19(6), Article 3528. https://doi.org/10.3390/ijerph19063528

Polit, D. F., & Beck, C. T. (2006). The content validity index: Are you sure you know what's being reported? Critique and recommendations. Research in Nursing & Health, 29(5), 489–497. https://doi.org/10.1002/nur.20147

Polit, D. F., Beck, C. T., & Owen, S. V. (2007). Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Research in Nursing & Health, 30(4), 459–467. https://doi.org/10.1002/nur.20199

Raghu, S. J., & Rodrigues, L. L. R. (2022). Solid waste management behavior among the student community: Integrating environmental knowledge and situational factors into the theories of planned behavior and value belief norm. Journal of Environmental Planning and Management, 65(10), 1842–1874. https://doi.org/10.1080/09640568.2021.1949969

Ramezani, S. G., & Mostafavi, Z. S. (2025). Developing and validating a comprehensive scale for accreditation standards and quality assurance in e-learning institutions. Education and Information Technologies, 30(15), 21139–21187. https://doi.org/10.1007/s10639-025-13587-5

Saefi, M., Fauzi, A., Kristiana, E., Adi, W. C., Muchson, M., Setiawan, M. E., Islami, N. N., Ningrum, D. E. A. F., Ikhsan, M. A., & Ramadhani, M. (2020). Validating of knowledge, attitudes, and practices questionnaire for prevention of COVID-19 infections among undergraduate students: A Rasch and factor analysis. Eurasia Journal of Mathematics, Science and Technology Education, 16(12), Article em1926. https://doi.org/10.29333/ejmste/9352

Safitri, D., Ibrahim, N., Sujarwo, S., Marini, A., Pratama, I., Dewiyani, L., Julia, V., Saputro, R. H., Muharrani, N. P., Sudrajat, A., & Marfu, A. (2026). Understanding pro environmental behavior through local wisdom and AR based environmental education using PLS SEM and MGA approach. Discover Sustainability, 7(1), Article 633. https://doi.org/10.1007/s43621-026-02931-5

Soeharto, S. (2021). Development of a diagnostic assessment test to evaluate science misconceptions in terms of school grades: A Rasch measurement approach. Journal of Turkish Science Education, 18(3), 351–370. https://doi.org/10.36681/tused.2021.78

Soeharto, S., & Csapó, B. (2022). Assessing Indonesian student inductive reasoning: Rasch analysis. Thinking Skills and Creativity, 46, Article 101132. https://doi.org/10.1016/j.tsc.2022.101132

Soeharto, S., Martono, M., Hairida, H., Akhmetova, A., Arifiyanti, F., Csapó, B., & Charalambous, C. (2024). The metacognitive awareness of reading strategy among pre-service primary teachers and the possibility of rating improvement using Rasch analysis. Studies in Educational Evaluation, 80, Article 101319. https://doi.org/10.1016/j.stueduc.2023.101319

Sriatun, N., Ardianto, D., Permana, I., & Afriana, J. (2025). Strategies for promoting energy literacy in physics education: Insights from a systematic literature review. Journal of Innovative Science Education, 14(1), 54–63. https://doi.org/10.15294/jise.v14i1.20856

Sudarmin, S., Ariyatun, A., Permanasari, A., Hutagalung, F. D., Pujiastuti, R. S. E., Sari, D. S., & Asmaningrum, H. P. (2026). Evaluating the validity and effectiveness of the ethno-SSI project–integrated virtual reality in improving sustainable environmental literacy. European Journal of STEM Education, 11(1), Article 33. https://doi.org/10.20897/ejsteme/18731

Sukarelawan, M. I., Jumadi, Kuswanto, H., Soeharto, & Hikmah, F. N. (2021). Rasch analysis to evaluate the psychometric properties of junior metacognitive awareness inventory in the Indonesian context. Jurnal Pendidikan IPA Indonesia, 10(4), 486–495. https://doi.org/10.15294/jpii.v10i4.27114

Suryadi, N. R. S., Nurmegawati, L., Mu’aziyah, S. E. S., & Perdani, A. S. (2025). Rasch model for analysis of scientific attitude instruments in the context of secondary school science education. Equator Science Journal, 3(2), 98–106. https://doi.org/10.61142/esj.v3i2.234

Tesio, L., Caronni, A., Simone, A., Kumbhare, D., & Scarano, S. (2024). Interpreting results from Rasch analysis 2. Advanced model applications and the data-model fit assessment. Disability and Rehabilitation, 46(3), 604–617. https://doi.org/10.1080/09638288.2023.2169772

Usman, M., Suyanta, Pujianto, & Huda, K. (2021). Energy literacy of junior high school students in Indonesia: A preliminary study. In Proceedings of the 6th International Seminar on Science Education (ISSE 2020) (pp. 609–614). Atlantis Press. https://doi.org/10.2991/assehr.k.210326.088

Wahira, W., & Tolla, I. (2023). Analysis of the needs for developing the competence of elementary school supervisors through analysis design development implementation evaluation (ADDIE) model. Kasetsart Journal of Social Sciences, 44(4), 1303–1312. https://doi.org/10.34044/j.kjss.2023.44.4.34

Wang, H., Li, J., Mangmeechai, A., & Su, J. (2021). Linking perceived policy effectiveness and proenvironmental behavior: The influence of attitude, implementation intention, and knowledge. International Journal of Environmental Research and Public Health, 18(6), Article 2910. https://doi.org/10.3390/ijerph18062910

Wang, X., & Chen, S. (2024). Psychometric evaluation of the extended inclusion of nature in self scale in Chinese adults through the Rasch analysis. People and Nature, 6(6), 2431–2448. https://doi.org/10.1002/pan3.10726

Zidny, R., Sjöström, J., & Eilks, I. (2020). A multi-perspective reflection on how indigenous knowledge and related ideas can improve science education for sustainability. Science & Education, 29(1), 145–185. https://doi.org/10.1007/s11191-019-00100-x