The Development and Validation of the Metacognitive Awareness Scale for Middle School Students
Abstract
This study aimed to develop a psychometrically sound instrument to assess the metacognitive awareness of middle school students. A 5-point Likert-type instrument was developed to assess middle school students' metacognitive knowledge and regulatory processes. The study was conducted with a total of 1009 students. Exploratory factor analysis (EFA) was conducted with data from 475 students. Preliminary diagnostics supported factorability (KMO = .952; Bartlett’s χ² = 8377.999, p < .01). Based on statistical and theoretical criteria, 17 items were removed, yielding a unidimensional structure consisting of 29 items that explained 38.392% of the total variance. Factor loadings ranged between .469 and .644. Confirmatory factor analysis (CFA) was subsequently performed with an independent sample of 534 students. Model fit indices (χ²/df = 2.770, CFI = .904, TLI = .899, IFI = .904, NFI = .870, RFI = .860, RMSEA = .048) indicated an acceptable fit for the proposed one-factor model. Internal consistency was high (Cronbach’s α = .942), and item–total correlations ranged from .467 to .651. Test–retest reliability, assessed with 126 students from a public middle school over a three-week interval, demonstrated excellent temporal stability (ICC(3,1) = .997, 95% CI [.996, .998]). Overall, the scale demonstrated strong validity and reliability and provides a developmentally appropriate instrument for assessing metacognitive awareness in middle school students.
Downloads
References
Abacı, R., Çetin, B., & Akın, A. (2006). Biliş ötesi farkındalık envanteri: Geçerlik ve güvenirlik çalışması. 14. Ulusal Psikoloji Kongresi Özet Kitapçığı, Hacettepe Üniversitesi, Ankara.
Akın, A., Abacı, R., & Çetin, B. (2007). The validity and reliability study of the Turkish version of the Metacognitive Awareness Inventory. Educational Sciences: Theory & Practice, 7(2), 655–680.
Aktamış, H., & Uça, S. (2010). Motivasyonel, bilişsel ve bilişüstü yeterlilikler ölçeği'nin Türkçeye uyarlanması. İlköğretim Online, 9(3), 980–989.
Annevirta, T., Laakkonen, E., Kinnunen, R., & Vauras, M. (2007). Developmental dynamics of metacognitive knowledge and text comprehension skill in the first primary school years. Metacognition and Learning, 2, 21–39. https://doi.org/10.1007/s11409-007-9002-8
Baker, L., & Brown, A. L. (1980). Metacognitive skills and reading. In P. D. Pearson (Ed.), Handbook of reading research (pp. 1–74). Longman.
Barnett, M., & Kaiser, D. (1978). The relationship between intellectual-achievement responsibility attributions and performance. Child Study Journal, 8, 209–215.
Bonett, D. G. (2002). Sample size requirements for testing and estimating coefficient alpha. Journal of Educational and Behavioral Statistics, 27(4), 335–340. https://doi.org/10.3102/10769986027004335
Bonett, D. G., & Wright, T. A. (2015). Cronbach's alpha reliability: Interval estimation, hypothesis testing, and sample size planning. Journal of Organizational Behavior, 36(1), 3–15. https://doi.org/10.1002/job.1960
Büyüköztürk, Ş. (2002). Faktör analizi: Temel kavramlar ve ölçek geliştirmede kullanımı. Kuram ve Uygulamada Eğitim Yönetimi, 32, 470–483.
Büyüköztürk, Ş., Kılıç-Çakmak, E., Akgün, Ö. E., Karadeniz, Ş., & Demirel, F. (2009). Bilimsel araştırma yöntemleri (3. baskı). Pegem Akademi.
Cheng, S. T., & Hamid, P. N. (1997). Dispositional optimism in Chinese people: What does the life orientation test measure? International Journal of Psychology, 32(4), 221–230.
Cole, D. A. (1987). Utility of confirmatory factor analysis in test validation research. Journal of Consulting and Clinical Psychology, 55(4), 584–594. https://doi.org/10.1037/0022-006X.55.4.584
Conway, J. M., & Huffcutt, A. I. (2003). A review and evaluation of exploratory factor analysis practices in organizational research. Organizational Research Methods, 6(2), 147–168. https://doi.org/10.1177/1094428103251621
Cortina, J. M. (1993). What is coefficient alpha? An examination of theory and applications. Journal of Applied Psychology, 78(1), 98–104. https://doi.org/10.1037/0021-9010.78.1.98
Costello, A. B., & Osborne, J. W. (2005). Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assessment, Research, and Evaluation, 10(1), 7. https://doi.org/10.7275/jyj1-4868
Coutinho, S. A., & Neuman, G. (2008). A model of metacognition, achievement goal orientation, learning style and self-efficacy. Learning Environments Research, 11(2), 131–151. https://doi.org/10.1007/s10984-008-9043-8
Cross, D. R., & Paris, S. G. (1988). Developmental and instructional analyses of children's metacognition and reading comprehension. Journal of educational psychology, 80(2), 131-142.
Çakıroğlu, A. (2007). Üstbiliş. Türkiye Sosyal Araştırmalar Dergisi, 11(2), 21–27.
Çetinkaya, P., & Erktin, E. (2002). Assessment of metacognition and its relationship with reading comprehension, achievement, and aptitude. Boğaziçi Eğitim Fakültesi Dergisi, 19(1), 1–11.
Çögenli, A., & Güven, M. (2014). Bilişüstü öğrenme stratejileri belirleme ölçeğinin geçerlik ve güvenirlik çalışması. Dicle Üniversitesi Ziya Gökalp Eğitim Fakültesi Dergisi, 22, 283–297.
Demir, Ö. (2013). A validation and reliability study of the metacognition scale in Turkey. Global Journal of Human Social Science, 13(10), 26–35.
Desoete, A. (2008). Multi-method assessment of metacognitive skills in elementary school children: How you test is what you get. Metacognition and Learning, 3(3), 189–206. https://doi.org/10.1007/s11409-008-9026-0
DeVellis, R. F. (2017). Scale development: Theory and applications (4th ed.). SAGE Publications.
Field, A. (2024). Discovering statistics using IBM SPSS statistics. Sage publications limited.
Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive-developmental inquiry. American Psychologist, 34(10), 906–911. https://doi.org/10.1037/0003-066X.34.10.906
Flavell, J. H. (1985). Cognitive development. Prentice-Hall.
Ford, J. K., MacCallum, R. C., & Tait, M. (1986). The application of exploratory factor analysis in applied psychology: A critical review and analysis. Personnel Psychology, 39(2), 291–314. https://doi.org/10.1111/j.1744-6570.1986.tb00583.x
George, D., & Mallery, M. (2010). SPSS for Windows step by step: A simple guide and reference (10th ed.). Pearson.
Güner, P., & Erbay, H. N. (2021). Metacognitive skills and problem-solving. International Journal of Research in Education and Science, 7(3), 715–734. https://doi.org/10.46328/ijres.1594
Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2019). Multivariate data analysis.
Howard, B. C., McGee, S., Shia, R., & Hong, N. (2000, April). Metacognitive self-regulation and problem-solving: Expanding the theory base through factor analysis. [Paper presentation] American Educational Research Association (AERA) Annual Meeting, New Orleans, LA, United states.
Hox, J. J., & Bechger, T. M. (1998). An introduction to structural equation modeling. Family Science Review, 11(4), 354–373.
Jacobs, J. E., & Paris, S. G. (1987). Children’s metacognition about reading: Issues in definition, measurement and instruction. Educational Psychologist, 22, 255–278. https://doi.org/10.1207/s15326985ep2203_4
Jahangard, Z., Soltani, A., & Alinejad, M. (2016). Exploring the relationship between metacognition and attitudes towards science of senior secondary students through a structural equation modeling analysis. Journal of Baltic Science Education, 15(3), 340–349. https://doi.org/10.33225/jbse/16.15.340
Karakelle, S., & Saraç, S. (2007). Çocuklar için Üst Bilişsel Farkındalık Ölçeği (ÜBFÖ-Ç) A ve B formları: Geçerlik ve güvenirlik çalışması. Türk Psikoloji Yazıları, 10(20), 87–103.
Karakelle, S., & Saraç, S. (2010). Üst biliş hakkında bir gözden geçirme: Üstbiliş çalışmaları mı yoksa üstbilişsel yaklaşım mı? Türk Psikoloji Yazıları, 13(26), 45–60.
Kline, R. B. (1998). Principles and practice of structural equation modeling. Guilford Press.
Koo, T. K., & Li, M. Y. (2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15(2), 155–163. https://doi.org/10.1016/j.jcm.2016.02.012
Kuhn, D. (2000). Metacognitive development. Current Directions in Psychological Science, 9(5), 178–181. https://doi.org/10.1111/1467-8721.00088
Lecce, S., Zocchi, S., Pagnin, A., Palladino, P., & Taumoepeau, M. (2010). Reading minds: The relation between children’s mental state knowledge and their metaknowledge about reading. Child Development, 81, 1876–1893. https://doi.org/10.1111/j.1467-8624.2010.01517.x
Mokhtari, K., & Reichard, C. A. (2002). Assessing students’ metacognitive awareness of reading strategies. Journal of Educational Psychology, 94, 249–259. https://doi.org/10.1037/0022-0663.94.2.249
Namlu, A. G. (2004). Bilişötesi öğrenme stratejileri ölçme aracının geliştirilmesi: Geçerlilik ve güvenirlik çalışması. Anadolu Üniversitesi Sosyal Bilimler Dergisi, 2, 123–136.
Owo, W. J., & Ikwut, E. F. (2015). Relationship between metacognition, attitude and academic achievement of secondary school chemistry students in Port Harcourt, Rivers State. Journal of Research & Method in Education, 5(6), 6–12.
Özsoy, G. (2011). An investigation of the relationship between metacognition and mathematics achievement. Asia Pacific Education Review, 12, 227–235. https://doi.org/10.1007/s12564-010-9129-6
Öztürk, E. (2012). Okuma stratejileri üstbilişsel farkındalık envanterinin Türkçe formunun geçerlik ve güvenirlik çalışması. İlköğretim Online, 11(2), 292–305.
Pereira-Laird, J. A., & Deane, F. P. (1997). Development and validation of a self-report measure of reading strategy use. Reading Psychology: An International Quarterly, 18, 185–235. https://doi.org/10.1080/0270271970180204
Pintrich, P. R. (2002). The role of metacognitive knowledge in learning, teaching, and assessing. Theory into Practice, 41(4), 219–225. https://doi.org/10.1207/s15430421tip4104_3
Rayner, A. (2005, September). Reflections on context based science teaching: A case study of physics students for physiotherapy [Poster presentation]. Annual UniServe Science Blended Learning Symposium Proceedings, Sydney.
Rattray, J., & Jones, M. C. (2007). Essential elements of questionnaire design and development. Journal of Clinical Nursing, 16(2), 234–243. https://doi.org/10.1111/j.1365-2702.2006.01573.x
Rosseel, Y. (2020). Small sample solutions for structural equation modeling. In Small sample size solutions (pp. 226–238). Routledge. https://doi.org/10.4324/9780429273872-19
Safran, M. (Ed.) (2019). Dünya eğitimine yön verenler (Cilt 1-2). Millî Eğitim Bakanlığı Yayınları.
Schermelleh-Engel, K., Moosbrugger, H., & Müller, H. (2003). Evaluating the fit of structural equation models: Tests of significance and descriptive goodness-of-fit measures. Methods of Psychological Research Online, 8(2), 23–74.
Schraw, G. (1998). Promoting general metacognitive awareness. Instructional Science, 26(1–2), 113–125. https://doi.org/10.1023/A:1003092417751
Schraw, G., & Dennison, R. S. (1994). Assessing metacognitive awareness. Contemporary Educational Psychology, 19, 460–475. https://doi.org/10.1006/ceps.1994.1033
Schraw, G., & Moshman, D. (1995). Metacognitive theories. Educational Psychology Review, 7(4), 351–371. https://doi.org/10.1007/BF02212307
Seçer, İ. (2018). Psikolojik test geliştirme ve uyarlama süreci: SPSS ve LISREL uygulamaları (2. baskı). Anı Yayıncılık.
Sperling, R. A., Howard, B. C., Miller, L. A., & Murphy, C. (2002). Measures of children’s knowledge and regulation of cognition. Contemporary Educational Psychology, 27, 51–79. https://doi.org/10.1006/ceps.2001.1091
Streiner, D. L., Norman, G. R., & Cairney, J. (2015). Health measurement scales: A practical guide to their development and use (5th ed.). Oxford University Press. https://doi.org/10.1093/med/9780199685219.001.0001
Sun, J. (2005). Assessing goodness of fit in confirmatory factor analysis. Measurement and Evaluation in Counseling and Development, 37(4), 240–256. https://doi.org/10.1080/07481756.2005.11909764
Sümer, N. (2000). Yapısal eşitlik modelleri: Temel kavramlar ve örnek uygulamalar. Türk Psikoloji Yazıları, 3(6), 49–74.
Tabachnick, B. G., & Fidell, L. S. (2007). Using multivariate statistics (5th ed.). Pearson Education.
Tabachnick, B. G., & Fidell, L. S. (2013). Using multivariate statistics (6. Baskı). MA: Pearson.
Taber, K. S. (2018). The use of Cronbach’s alpha when developing and reporting research instruments in science education. Research in Science Education, 48(6), 1273–1296. https://doi.org/10.1007/s11165-016-9602-2
Tavakol, M., & Dennick, R. (2011). Making sense of Cronbach’s alpha. International Journal of Medical Education, 2, 53–55. https://doi.org/10.5116/ijme.4dfb.8dfd
Tavşancıl, E. (2019). Tutumların ölçülmesi ve SPSS ile veri analizi (6. baskı). Nobel Yayıncılık.
Turgut, T., & Erden, N. S. (2013). Olumsuz test ifadelerinin iç tutarlılığa ve faktör yapısına etkileri. Istanbul University Journal of the School of Business Administration, 42(2).
Vaske, J. J., Beaman, J., & Sponarski, C. C. (2017). Rethinking internal consistency in Cronbach’s alpha. Leisure Sciences, 39(2), 163–173. https://doi.org/10.1080/01490400.2015.1127189
Veenman, M. V. J., Van Hout-Wolters, B. H. A. M., & Afflerbach, P. (2006). Metacognition and learning: Conceptual and methodological considerations. Metacognition and Learning, 1, 3–14. https://doi.org/10.1007/s11409-006-6893-0
Vrugt, A., & Oort, F. J. (2008). Metacognition, achievement goals, study strategies and academic achievement: Pathways to achievement. Metacognition and Learning, 3, 123–146. https://doi.org/10.1007/s11409-008-9022-4
Williams, B., Onsman, A., & Brown, T. (2010). Exploratory factor analysis: A five-step guide for novices. Australasian Journal of Paramedicine, 8(3), 1–13. https://doi.org/10.33151/AJP.8.3.93
Wong, N. Y., Rindfleisch, A., & Burroughs, J. E. (2003). Do reverse-worded items confound measures in cross-cultural consumer research? The case of material values scale. Journal of Consumer Research, 30(1), 72–91. https://doi.org/10.1086/374697
Yıldız, E., Akpınar, E., Tatar, N., & Ergin, Ö. (2009). İlköğretim öğrencileri için geliştirilen biliş üstü ölçeği’nin açımlayıcı ve doğrulayıcı faktör analizi. Kuram ve Uygulamada Eğitim Bilimleri, 9(3), 1573–1604.
Young, A., & Fry, J. D. (2008). Metacognitive awareness and academic achievement in college students. Journal of the Scholarship of Teaching and Learning, 8(2), 1–10.
