Evaluating Indonesian upper secondary school students’ information literacy in science learning: A Rasch model approach
Keywords:
Evaluation, Information literacy, rasch modelAbstract
This study evaluates Indonesian upper secondary school students' information literacy across the cognitive, psychomotor and affective domains in science learning. The primary objectives were to validate an information literacy assessment covering all three domains and to classify both the difficulty level of test items and students’ proficiency. A total of 727 students in the 10th, 11th, and 12th grades in Yogyakarta, Indonesia, participated in the study. Each domain was assessed through distinct paper-based tasks. The results indicated that the test demonstrated acceptable validity and reliability across all domains based on Rasch model parameters. Differential Item Functioning (DIF) analysis revealed that only one item in the cognitive domain showed slight to moderate DIF. No DIF was found in the affective domain, while several items in the psychomotor domain showed substantial DIF. Overall, students’ information literacy skills were categorised as high, regardless of gender or grade level. Although female students have higher scores than male students, the difference was not statistically significant. Students performed best in the affective domain compared to the cognitive and psychomotor domains. This study also identified the relative difficulty of test items in each domain, offering a comprehensive analysis of students' information literacy skills.
Downloads
References
Adarkwa, A. B. (2024). Information literacy among students in higher learning institutions in Ghana: The case of Kwame Nkrumah University of Science and Technology. The International Information & Library Review, 56(2), 167–183. https://doi.org/10.1080/10572317.2024.2335607
Ahmad, A., Kamin, Y., & Md Nasir, A. N. (2018). Applying psychomotor domain for competency-based teaching in vocational education. Journal of Physics: Conference Series, 1049(1), Article 012049. https://doi.org/10.1088/1742-6596/1049/1/012049
Al-Qerem, W., Fadhil, O. N., Jarab, A., Hammad, A., Al-Asmari, F., Zidan, R., & Eberhardt, J. (2025). Validation of the Arabic eHealth literacy questionnaire: A factor and Rasch analysis study. Frontiers in Public Health, 13, Article 1542477. https://doi.org/10.3389/fpubh.2025.1542477
Al-Qerem, W., Jarab, A., Khdour, M., Eberhardt, J., Alasmari, F., Hammad, A., Zumot, R., & Khalil, S. (2024). Assessing mental health literacy in Jordan: A factor analysis and Rasch analysis study. Frontiers in Public Health, 12, Article 1396255. https://doi.org/10.3389/fpubh.2024.1396255
Anderson, L. W., Krathwohl, D. R., & Bloom, B. S. (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives. Allyn & Bacon.
Andrich, D. (2017). Advances in social measurement: A Rasch measurement theory. In Perceived health and adaptation in chronic disease. https://doi.org/10.1201/9781315155074-7
Avinç, E., & Doğan, F. (2024). Digital literacy scale: Validity and reliability study with the Rasch model. Education and Information Technologies, 29(17), 22895–22941. https://doi.org/10.1007/s10639-024-12662-7
Boh Podgornik, B., Dolničar, D., Šorgo, A., & Bartol, T. (2016). Development, testing, and validation of an information literacy test (ILT) for higher education. Journal of the Association for Information Science and Technology, 67(10), 2420–2436. https://doi.org/10.1002/asi.23586
Boone, W. J., Staver, J. R., & Yale, M. S. (2014). Rasch analysis in the human sciences. Springer. https://doi.org/10.1007/978-94-007-6857-4
Bowers, C. (2022). Physical literacy and information literacy as mixed terrain: Possibilities for academic libraries. Georgia International Conference on Information Literacy.
Burns, E., Gross, M., & Latham, D. (2019). The information literacy continuum: Mapping the ACRL framework to the AASL school library standards. School Libraries Worldwide, 25(1).
Buschek, D., Zürn, M., & Eiband, M. (2021). The impact of multiple parallel phrase suggestions on email input and composition behaviour of native and non-native English writers. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Article 458, pp. 1–13). Association for Computing Machinery. https://doi.org/10.1145/3411764.3445372.
Chan, S. W., Looi, C. K., & Sumintono, B. (2021). Assessing computational thinking abilities among Singapore secondary students: A Rasch model measurement analysis. Journal of Computers in Education, 8(2). https://doi.org/10.1007/s40692-020-00177-2
Chase, M. (2024). Teaching intersectionality in instructional librarianship: Asynchronous information literacy instruction in the health sciences. Issues in Science and Technology Librarianship, 105. https://doi.org/10.29173/istl2794
Darman, D. R., Suhandi, A., Kaniawati, I., Samsudin, A., & Wibowo, F. C. (2024). Development and validation of Scientific Inquiry Literacy Instrument (SILI) using Rasch measurement model. Education Sciences, 14(3), Article 322. https://doi.org/10.3390/educsci14030322
Delmond, A. R., Weber, E. M., & Busch, H. S. (2024). An interdisciplinary assessment of information literacy instruction. The Journal of Academic Librarianship, 50(5), Article 102944. https://doi.org/10.1016/j.acalib.2024.102944
Ferrés, J., & Piscitelli, A. (2012). Media competence: Articulated proposal of dimensions and indicators. Comunicar, 19(38). https://doi.org/10.3916/C38-2012-02-08
Hernández-Marín, J.-L., Castro-Montoya, M.-D., & Figueroa-Rodríguez, S. (2024). Alfabetización mediática, informacional y digital: Análisis de instrumentos de evaluación. Investigación Bibliotecológica: Archivonomía, Bibliotecología e Información, 38(99), 55–73. https://doi.org/10.22201/iibi.24488321xe.2024.99.58865
Hossain, M. A., & Sormunen, E. (2025). Empowering library and information science students as information literacy educators in librarianship professions: A study from Bangladesh. Information and Learning Sciences. Advance online publication. https://doi.org/10.1108/ILS-07-2024-0077
Hussain, B. A., Li, S., & Alsanad, A. (2022). Assessment of information literacy abilities: A case study of Pakistan. Sustainability, 14(7), Article 3807. https://doi.org/10.3390/su14073807
Hwang, G.-J., Zou, D., & Wu, Y.-X. (2023). Learning by storytelling and critiquing: A peer assessment-enhanced digital storytelling approach to promoting young students’ information literacy, self-efficacy, and critical thinking awareness. Educational Technology Research and Development, 71(3), 1079–1103. https://doi.org/10.1007/s11423-022-10184-y
Ibrahim, S. E., Ismail, M. R., & Subramaniam, T. T. (2024). Information literacy assessment. International Journal of Media and Information Literacy, 9(2). https://doi.org/10.13187/ijmil.2024.2.339
Katz, H. L. (2024). Putting the cart before the horse: A study of introductory political science students and the evolution of an assignment on information literacy. Journal of Political Science Education, 20(2), 198–217. https://doi.org/10.1080/15512169.2023.2249148
Kodrat, D. S., Tambunan, D. B., Hartono, W., Tengb, P. K., & Ling, C. P. (2024). Comparative study of the impact of information literacy, digital literacy and media literacy on employability between Indonesia and Malaysia. Decision Science Letters, 13(4), 853–866. https://doi.org/10.5267/j.dsl.2024.8.006
Liu, L., Wang, S., Chen, J., Li, H., Gui, J., & Zhang, R. (2024). Information literacy of higher vocational college students in digital age. Journal of Librarianship and Information Science. Advance online publication. https://doi.org/10.1177/09610006231224445
Locoro, A., Fisher, W. P., & Mari, L. (2021). Visual information literacy: Definition, construct modeling, and assessment. IEEE Access, 9, 71053–71071. https://doi.org/10.1109/ACCESS.2021.3078429
Manhique, I. L. E., & Casarin, H. de C. S. (2025). Inserção da competência em informação na formação de bibliotecários em Moçambique. RDBCI: Revista Digital de Biblioteconomia e Ciência da Informação, 23, Article e025009. https://doi.org/10.20396/rdbci.v23i00.8677089
Maričić, M., Anđić, B., Soeharto, S., Mumcu, F., Cvjetićanin, S., & Lavicza, Z. (2025). The exploration of continuous teaching intention in emerging-technology environments through perceived cognitive load, usability, and teacher’s attitudes. Education and Information Technologies, 30(7), 9341–9370. https://doi.org/10.1007/s10639-024-13141-9
Ndosi, M., Almeida, C., Dawson, J., Dures, E., Greenwood, R., Bromhead, A., Guly, C., Stern, S., Hill, C., MacKie, S., & Robson, J. C. (2024). Validation of a patient-reported outcome measure for giant cell arteritis. Rheumatology, 63(1). https://doi.org/10.1093/rheumatology/kead201
Nuraini, N., Mahanal, S., Susilo, H., Sulisetijono, & Zubaidah, S. (2025). Advancing of biology students’ information literacy: An investigation of three different learning models. Journal of Turkish Science Education, 22(1), 33–47. https://doi.org/10.36681/tused.2025.003
Olson, D., Bates, S. L., Yarbrough, S., Westall, S., Denis, M. K. C., & Barnett, M. (2023). Information literacy curriculum mapping in the health sciences. Journal of Information Literacy, 17(1). https://doi.org/10.11645/17.1.3319
Pinto, M. (2010). Design of the IL-HUMASS survey on information literacy in higher education: A self-assessment approach. Journal of Information Science, 36(1). https://doi.org/10.1177/0165551509351198
Planinic, M., Boone, W. J., Susac, A., & Ivanjek, L. (2019). Rasch analysis in physics education research: Why measurement matters. Physical Review Physics Education Research, 15(2). https://doi.org/10.1103/PhysRevPhysEducRes.15.020111
Prabowo, T. T., Sitthiworachart, J., & Lo, C. K. (2024). Demographic analysis of information literacy self-efficacy among library and information science students in Indonesia. Library & Information Science Research, 46(1), Article 101281. https://doi.org/10.1016/j.lisr.2024.101281
Punter, R. A., Meelissen, M. R. M., & Glas, C. A. W. (2017). Gender differences in computer and information literacy: An exploration of the performances of girls and boys in ICILS 2013. European Educational Research Journal, 16(6). https://doi.org/10.1177/1474904116672468
Rahmah, E., Mohamed Shuhidan, S., & Wan Yahaya, W. A. (2024). Status of local knowledge and information literacy among the Minangkabau community in Indonesia. Record and Library Journal, 10(1), 1–11. https://doi.org/10.20473/rlj.V10-I1.2024.1-11
Ramadhani, R., Prahmana, R. C. I., Soeharto, S., & Saleh, A. (2024a). Integrating traditional food and technology in statistical learning: A learning trajectory. Journal on Mathematics Education, 15(4), 1277–1310. https://doi.org/10.22342/jme.v15i4.pp1277-1310
Ramadhani, R., Syahputra, E., & Simamora, E. (2024b). The construct validity of self-regulated learning questionnaire for senior high school students: A Rasch model analysis. International Journal of Evaluation and Research in Education, 13(3). https://doi.org/10.11591/ijere.v13i3.26816
Riwayani, R., Istiyono, E., Supahar, S., Perdana, R., & Soeharto, S. (2024). Analyzing students’ statistical literacy skills based on gender, grade, and educational field. International Journal of Evaluation and Research in Education, 13(2), 842–851.
Siddiq, F. (2017). Learning in digital networks – ICT literacy: A novel assessment of students’ 21st-century skills. Computers & Education, 109, 11–37. https://doi.org/10.1016/j.compedu.2017.01.014
Singh, R., & Kumar, S. (2019). Information literacy competency level of social science researchers with respect to information use ethics: A study. DESIDOC Journal of Library & Information Technology, 39(2), 101–108. https://doi.org/10.14429/djlit.39.2.13507
Smith, D., & Sanger, S. (2023). Scaffolded, embedded required: Information literacy education in undergraduate health sciences. Journal of the Canadian Health Libraries Association / Journal de l’Association des bibliothèques de la santé du Canada, 44(2), 27–35. https://doi.org/10.29173/jchla29666
Soeharto, S., & Csapó, B. (2022). Assessing Indonesian student inductive reasoning: Rasch analysis. Thinking Skills and Creativity, 46. https://doi.org/10.1016/j.tsc.2022.101132
Soeharto, S., Martono, M., Hairida, H., Akhmetova, A., Arifiyanti, F., Benő, C., & Charalambos, C. (2024). The metacognitive awareness of reading strategy among pre-service primary teachers and the possibility of rating improvement using Rasch analysis. Studies in Educational Evaluation, 80, Article 101319. https://doi.org/10.1016/j.stueduc.2023.101319
Sumintono, B., & Widhiarso, W. (2015). Aplikasi pemodelan Rasch pada assessment pendidikan. Penerbit Trim Komunikata.
Taber, K. S. (2018). The use of Cronbach’s alpha when developing and reporting research instruments in science education. Research in Science Education, 48(6). https://doi.org/10.1007/s11165-016-9602-2
Tasu’ah, N., Diana, D., Sugiariyanti, S., & Martika, T. (2023). Teacher information literacy for inclusive early childhood education (ECE) to provide literacy and numeracy for special needs children in Central Java-Indonesia. International Journal of Learning, Teaching and Educational Research, 22(12), 262–278. https://doi.org/10.26803/ijlter.22.12.13
Turpo-Gebera, O., Zea-Urviola, M., Huamaní-Portilla, F., Girón-Pizarro, M., Pérez-Zea, A., & Aguaded-Gómez, I. (2023). Media and information literacy in secondary students: Diagnosis and assessment. Journal of Technology and Science Education, 13(2), 514. https://doi.org/10.3926/jotse.1746
Vongkulluksn, V. W., Shortt, M. B., Akinkuolie, B., & Xie, K. (2024). Information literacy instruction in naturalistic high school science classrooms: Instructional strategies and associations with students’ prior knowledge. Teaching and Teacher Education, 144, Article 104572. https://doi.org/10.1016/j.tate.2024.104572
Wackerle-Hollman, A., Hojnoski, R., Missall, K., Abuela, M. A. A., & Running, K. (2024). Using empirical information to prioritize early literacy assessment and instruction in preschool and kindergarten. Assessment for Effective Intervention, 49(4), 190–201. https://doi.org/10.1177/15345084241247059
Wallin, G., Chen, Y., & Moustaki, I. (2024). DIF analysis with unknown groups and anchor items. Psychometrika, 89(1). https://doi.org/10.1007/s11336-024-09948-7
Wu, T. Y., Lien, B. Y. H., Lequerica, A. H., Lu, W. S., & Hsieh, C. L. (2019). Development and validation of the occupational therapy engagement scale for patients with stroke. Occupational Therapy International, 2019, Article 3164254. https://doi.org/10.1155/2019/3164254
Zhu, S., Yang, H. H., MacLeod, J., Yu, L., & Wu, D. (2019). Investigating teenage students’ information literacy in China: A social cognitive theory perspective. The Asia-Pacific Education Researcher, 28(3), 251–263. https://doi.org/10.1007/s40299-019-00433-9
Zwick, R., Thayer, D. T., & Lewis, C. (1999). An empirical Bayes approach to Mantel-Haenszel DIF analysis. Journal of Educational Measurement, 36(1). https://doi.org/10.1111/j.1745-3984.1999.tb00543.x
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Journal of Turkish Science Education
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
This license enables reusers to copy and distribute the material in any medium or format in unadapted form only, for noncommercial purposes only, and only so long as attribution is given to the creator. CC BY-NC-ND includes the following elements: Credit must be given to the creator; only noncommercial uses of the work are permitted; no derivatives or adaptations of the work are permitted.
