A Randomized Control Trial Study: Evaluating Pedagogical Practices in Science Classrooms
- Rafael Lara-Alecio
- Beverly J. Irby
- Fuhui Tong
- Kara L. Sutton-Jones
- Cindy L. Guerrero
- Shifang Tang
- Huiwen Pang
Abstract
In this randomized control study, we evaluated science teachers’ pedagogical practices via classroom observations following ongoing, intensive, and structured instructional support sessions. These sessions included virtual professional development (VPD) and virtual mentoring and coaching (VMC) that accompanied a literacy-infused science curriculum. Using a low-inference observational instrument, we explored the direct impact of VPD and VMC on fifth-grade science teachers’ observed time allocation in a randomized controlled trial (RCT) validation study. The observations were collected three times during science instruction from 121 teachers in 68 schools from 35 public school districts in the U.S. state of Texas, during the 2017-2018 school year. Preliminary findings revealed pedagogical differences in time allocation among teachers between treatment and control classrooms. We identified improved instructional practices within treatment classrooms, which suggests the intervention had a positive effect by enhancing the quality of pedagogy as well as the content-area instruction in science.
References
- Author. (1994).
- Author. (2009).
- Author. (2013).
- Author. (2015a).
- Author. (2015b)
- Author. (2016).
- Author. (2018a).
- Author. (2018b).
- Author. (2018c).
- Author. (2019a).
- Author. (2019b).
- Author. (2020).
- Author. (2021a).
- Author. (2021b).
- Author. (2021c).
- Author. (2022a).
- Author. (2022b).
- Bang, E. (2013). Hybrid-mentoring programs for beginning elementary science teachers. International Journal of Education in Mathematics, 1(1), 1-15. https://ijemst.net/index.php/ijemst/article/view/31
- Bang, E., & Luft, J. A. (2014). Exploring the written dialogues of two first-year secondary science teachers in an online mentoring program. Journal of Science Teacher Education, 25, 25-51. https://doi.org/10.1007/s10972-013-9362-z
- Binmohsen, S. A., & Abrahams, I. (2022). Science teachers continuing professional development: Online vs. face-to-face. Research in Science & Technological Education, 40(3), 291-319. https://doi.org/10.1080/02635143.2020.1785857
- Bragg, L. A., Walsh, C., & Heyeres, M. (2021). Successful design and delivery of online professional development for teachers: A systematic review of the literature. Computers & Education, 166, 104158. https://doi.org/10.1016/j.compedu.2021.104158
- Brophy, J. E., & Good, T. L. (1974). Teacher-student relationships: Causes and consequences. Holt, Rinehart & Winston.
- Buyers, A., Koba, S., Sherman, G., Scheppke, J., & Bolus, R. (2011). Developing a web-based mechanism for assessing teacher science knowledge. Journal of Science Teacher Education, 22, 273-289. https://doi.org/10.1007/s10972-011-9227-2
- Bybee, R., & Landes, N. M. (1990). Science for life and living: An elementary school science program from Biological Sciences Improvement Study (BSCS). The American Biology Teacher, 52(2), 92-98. https://doi.org/10.2307/4449042
- Cady, J. A., Aydeniz, M., & Rearden, K. T. (2011). E-learning environments for math and science teachers. Journal of Curriculum and Instruction, 5(1), 17-33. https://doi.org/10.3776/joci.2011.v5n1p17-33
- Calderón, M., Slavin, R., & Sánchez, M. (2011). Effective instruction for English learners. The Future of Children, 21(1), 103-127. https://psycnet.apa.org/doi/10.1353/foc.2011.0007
- Cardichon, J., Darling-Hammond, L., Yang, M., Scott, C., Shields, P. M., & Burns, D. (2020). Inequitable opportunity to learn: Student access to certified and experienced teachers. Learning Policy Institute. https://learningpolicyinstitute.org/media/392/download?inline&file=CRDC_Teacher_Access_REPORT.pdf
- Carson, C. D., Callard, C., Gillespie, R., Choppin, J., & Amador, J. M. (2019). Bridging the distance: One-on-one coaching supports rural teachers. The Learning Professional, 40(6), 66-70. https://learningforward.org/journal/coaching/30872-2/
- Carson, C., & Choppin, J. (2021). Coaching from a distance: Exploring video-based online coaching. Online Learning, 25(4), 104-124. https://doi.org/10.24059/olj.v25i4.2881
- Cavanaugh, C., & Dawson, K. (2010). Design of online professional development in science content and pedagogy. A pilot study. Journal of Science Education and Technology, 19, 438-446. https://doi.org/10.1007/s10956-010-9210-2
- Coley, J. D., & Hoffman, D. M. (1990). Overcoming learned helplessness in at risk readers. Journal of Reading, 33, 497-502. http://www.jstor.org/stable/40032010
- Community for Advancing Discovery Research in Education (CADRE). (2017). Emerging design principles for online and blended teacher professional development in K-12 STEM Education. Education Development Center, Inc. http://cadrek12.org/resources/emerging-design-principles-online-and-blended- teacher-professional-development-k-12-stem
- Cummins, J. (1986). Empowering minority students: A framework for intervention. Harvard Educational Review, 56, 18-36. https://psycnet.apa.org/doi/10.17763/haer.56.1.b327234461607787
- Darling-Hammond, L., Hyler, M. E., & Gardner, M. (2017). Effective teacher professional development. Learning Policy Institute. https://learningpolicyinstitute.org/sites/default/files/product-files/Effective_Teacher_Professional_Development_REPORT.pdf
- Dede, C., Eisenkraft, A., Frumin, K., & Hartley, A. (2016). Teacher learning in the digital age: Online professional development in STEM education. Harvard Education Press.
- Dede, C., Ketelhut, D. J., Whitehouse, P., Breit, L., & McCloskey, E. M. (2009). A research agenda for online teacher professional development. Journal of Teacher Education, 60(1), 8-19. https://doi.org/10.11772F0022487108327554
- Diaz, S., Moll, L. C., & Mehan, H. (1970). Sociocultural resources in instruction: A context-specific approach. In Beyond language: Social and cultural factors in schooling language minority students (pp.187-230). Bilingual Education Office, California State Department of Education.
- Fairman, J. C., Mills, T., & Lech, P. L. (2019). Challenges with teacher retention and staffing and shortages in Maine school districts. https://digitalcommons.usm.maine.edu/cgi/viewcontent.cgi?article=1041&context=cepare_improvement#:~:text=Maine%20schools%20experiencing%20the%20greatest,significantly%20lower%20than%20in%20other
- Fischer, C., Fishman, B., Dede, C., Eisenkraft, A., Frumin, K., Foster, B., Lawrenz, F., Jurist Levy, A., & McCoy, A. (2018). Investigating relationships between school context, teacher professional development, teaching practices, and student achievement in response to a nationwide science reform. Teaching and Teacher Education, 72, 107-121. https://doi.org/10.1016/j.tate.2018.02.011
- Fishman, B., Konstantopoulos, S., Kubitskey, B. W., Vath, R., Park, G., Johnson, H., & Edelson, D. C. (2013). Comparing the impact of online and face-to-face professional development in the context of curriculum implementation. Journal of Teacher Education, 64(5), 426-438. https://doi.org/10.1177/0022487113494413
- Fishman, B., Marx, R., Best, S., & Tal, R. (2003). Linking teacher and student learning to improve professional development in systemic reform. Teaching and Teacher Education, 19(6), 643-658. https://doi.org/10.1016/S0742-051X(03)00059-3
- Friedrichsen, P., Chval, K. B., & Teuscher, D. (2007). Strategies and sources of support for beginning teachers of science and mathematics. School Science and Mathematics, 107(5), 169-181. https://doi.org/10.1111/j.1949-8594.2007.tb17781.x
- Gamez, R., & Parker, C. A. (2018). Becoming science learners: A study of newcomers’ identity work in elementary school science. Science Education, 102(2), 377-413. https://doi.org/10.1002/sce.21323
- Gaudin, C., & Chaliès, S. (2015). Video viewing in teacher education and professional development: A literature review. Educational Research Review, 16, 41-67. https://doi.org/10.1016/j.edurev.2015.06.001
- Germuth, A. A. (2018). Professional development that changes teaching and improves learning. Journal of Interdisciplinary Teacher Leadership, 2(1), 77-90. https://doi.org/10.46767/kfp.2016-0025
- Gosselin, D. C., Thomas, J., Redmond, A., Larson-Miller, C., Yendra, S., Bonnstetter, R. J., & Slater, T. (2010). Laboratory Earth: A model of online K-12 teacher coursework. Journal of Geoscience Education, 58(4), 203-213. https://doi.org/10.5408/1.3534859
- Gupta, A., & Lee, G. (2020). The effects of a site-based teacher professional development program on student learning. International Electronic Journal of Elementary Education, 12(5), 417-428. 10.26822/iejee.2020562132
- Gwet, K. L. (2008). Computing inter‐rater reliability and its variance in the presence of high agreement. British Journal of Mathematical and Statistical Psychology, 61(1), 29-48. https://doi.org/10.1348/000711006x126600
- Hathaway, D., & Norton, P. (2012). An exploratory study comparing two modes of preparation for online teaching. Journal of Digital Learning in Teacher Education, 28(4), 146-152. https://doi.org/10.1080/21532974.2012.10784695
- Jaber, L. Z., Dini, V., Hammer, D., & Danahy, E. (2018). Targeting disciplinary practices in an online learning environment. Science Education, 102(4), 668–692. https://doi.org/10.1002/sce.21340
- Jackson, J. K., Huerta, M., Garza, T., & Narvaez, R. (2019). Examining the effects of a professional development initiative on English learning and economically disadvantaged adolescents’ scores on a high-stakes science test. Journal of Science Teacher Education, 30(2), 122-143. https://doi.org/10.1080/1046560X.2018.1538299
- Joyce, B., & Calhoun, E. (2010). Models of professional development: A celebration of educators. Corwin Press.
- Kim, S., Raza, M., & Seidman, E. (2019). Improving 21st-century teaching skills: The key to effective 21st-century learners. Research in Comparative and International Education, 14(1), 99-117. https://doi.org/10.1177/1745499919829214
- Krashen, S. (1981). Second language acquisition and second language learning. Pergamon Press.
- Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 159-174. https://doi.org/10.2307/2529310
- Learning Forward. (2011). Standards for professional learning: Quick reference guide. Journal of Staff Development, 32(4), 41-44.
- Lee, O., Maerten-Rivera, J., Penfield, R. D., LeRoy, K., & Secada, W. G. (2008). Science achievement of English language learners in urban elementary schools: Results of a first-year professional development intervention. Journal of Research in Science Teaching: The Official Journal of the National Association for Research in Science Teaching, 45(1), 31-52. https://doi.org/10.1002/tea.20209
- Lee, O., Grapin, S., & Haas, A. (2023). Teacher professional development programs integrating science and language with multilingual learners: A conceptual framework. Science Education, 107, 1302–1323. https://doi-org.srv-proxy2.library.tamu.edu/10.1002/sce.21807
- Lee, S. C., Nugent, G., Kunz, G. M., Houston, J., & DeChenne-Peters, S. (2018). Case study: Value-Added benefit of distance-based instructional coaching on science teachers’ inquiry instruction in rural schools. Journal of Science Teacher Education, 29(3), 179–199. https://doi.org/10.1080/1046560x.2018.1432226
- Llosa, L., Lee, O., Jiang, F., Haas, A., O’Connor, C., Van Booven, C. D., & Kieffer, M. J. (2016). Impact of a large-scale science intervention focused on English language learners. American Educational Research Journal, 53(2), 395-424. https://doi.org/10.3102/0002831216637348
- Luz, M. R. M. P., Rolando, L. G. R., Salvador, D. F., & Souza, A. H. S. (2018). Characterization of the reasons why Brazilian science teachers drop out of online professional development courses. International Review of Research in Open and Distributed Learning, 19(5), 145-164. https://www.irrodl.org/index.php/irrodl/article/view/3642
- McChesney, K., & Aldridge, J. M. (2021). What gets in the way? A new conceptual model for the trajectory from teacher professional development to impact. Professional Development in Education, 47(5), 834-852. https://doi.org/10.1080/19415257.2019.1667412
- Melton, J., Miller, M., & Brobst, J. (2019). Mentoring the mentors: Hybridizing professional development to support cooperating teachers’ mentoring practice in science. Contemporary Issues in Technology and Teacher Education, 19(1), 23-44. https://citejournal.org/volume-19/issue-1-19/science/mentoring-the-mentors-hybridizing-professional-development-to-support-cooperating-teachers-mentoring-practice-in-science/
- Meskill, C., & Oliveira, A. W. (2019). Meeting the challenges of English learners by pairing science and language educators. Research in Science Education, 49, 1025-1040. https://doi.org/10.1007/s11165-019-9837-9
- Monk, D. H. (2007). Recruiting and retaining high-quality teachers in rural areas. The Future of Children, 155-174. http://doi.org/10.1353/foc.2007.0009
- Moon, J., Passmore, C., Reiser, B. J., & Michaels, S. (2014). Beyond comparisons of online versus face-to-face PD: Commentary in response to Fishman et al., “Comparing the impact of online and face-to-face professional development in the context of curriculum implementation.” Journal of Teacher Education, 65(2), 172-176. https://doi.org/10.1177/0022487113511497
- National Research Council. (2010). Preparing teachers: Building evidence for sound policy. https://doi.org/10.17226/12882
- Nugent, G., Kunz, G., Houston, J., Kalutskaya, I., Wu, C., Pedersen, J., Lee, S., DeChenne, S., Luo, L., & Berry, B. (2016). The effectiveness of technology-delivered science instructional coaching in middle and high school. Working paper. National Center for Research on Rural Education. https://r2ed.unl.edu/workingpapers/2016/R2Ed-WorkingPaper-2016-1.pdf
- O’Leary, M., Cui, V., Kiem, M. T., Dang, D. T., Nguyen, G. T. H., & Hoang, K. H. T. (2023). The role of classroom observation in the development and assessment of schoolteachers in Vietnam: A review of national policy and research. Asia-Pacific Journal of Teacher Education, 51(4), 395-407. https://doi.org/10.1080/1359866X.2023.2191307
- Ossola, A. (2014). Why kids won’t quit technology. The Atlantic. http://www.ikzadvisors.com/wp-content/uploads/Why-Kids-Wont-Quit-Tech_-Atlantic.pdf
- Padrón, Y. N., & Waxman, H. C. (1993). Teaching and learning risks associated with limited cognitive mastery in science and mathematics for limited-English proficient students. In Office of Bilingual Education and Minority Language Affairs (Eds.), Proceedings of the Third National Research Symposium on Limited English Proficient Students. Focus on middle and high school issues (Vol. 2, pp. 511-547). National Clearinghouse for Bilingual Education.
- Quinn, F., Charteris, J., Adlington, R., Rizk, N., Fletcher, P., & Parkes, M. (2022). The potential of online technologies in meeting PLD needs of rural teachers. Asia-Pacific Journal of Teacher Education, 50(1), 69-83. https://doi.org/10.1080/1359866X.2020.1849538
- Rodgers, W. J., Kennedy, M. J., VanUitert, V. J., & Myers, A. M. (2019). Delivering performance feedback to teachers using technology-based observation and coaching tools. Intervention in School and Clinic, 55(2), 103–112. https://doi.org/10.1177/1053451219837640
- Sancar, R., Atal, D., & Deryakulu, D. (2021). A new framework for teachers’ professional development. Teaching and Teacher Education, 101, 103305. https://doi.org/10.1016/j.tate.2021.103305
- Schaefer, J. M., & Ottley, J. R. (2018). Evaluating immediate feedback via bug-in-ear as an evidence-based practice for professional development. Journal of Special Education Technology, 33(4), 247–258. https://doi.org/10.1177/0162643418766870
- Sexton, D. M. (2018). Becoming a math teacher: Recruitment, preparation, and practice. (ERIC No. ED587885) [Doctoral dissertation, University of California, Santa Cruz]. ERIC.
- Simonsen, L., Luebeck, J., & Bice, L. (2009). The effectiveness of online paired mentoring for beginning science and mathematics teachers. Journal of Distance Education, 23(2), 51-68. http://www.ijede.ca/index.php/jde/article/view/592/893
- Sims, S., & Fletcher-Wood, H. (2021). Identifying the characteristics of effective teacher professional development: a critical review. School Effectiveness and School Improvement, 32(1), 47-63. https://doi.org/10.1080/09243453.2020.1772841
- Sutcher, L., Darling-Hammond, L., & Carver-Thomas, D. (2016). A coming crisis in teaching? Teacher supply, demand, and shortages in the US. Learning Policy Institute. https://learningpolicyinstitute.org/media/179/download?inline&file=A_Coming_Crisis_in_Teaching_REPORT.pdf
- Sweigart, C., Collins, L., Evanovich, L., & Cook, S. (2016). An evaluation of the evidence base for performance feedback to improve teacher praise using CEC’s quality indicators. Education and Treatment of Children, 39, 419-444. https://doi.org/10.1080/1046560X.2020.1727622
- Tang, A., Li, W., & Liu, D. (2022). The impact of teachers’ professional development in science pedagogy on students’ achievement: Evidence from TIMSS 2019. Journal of Baltic Science Education, 21(2), 258-274.
- Tudge, J. (1992). Vygotski, the zone of proximal development, and peer collaboration: Implications for classroom practice. In L.C. Moll (Ed.), Vygotski and education: Instructional implications and applications of sociohistorical psychology (pp.155-172). Cambridge University Press.
- Unver, A. O., Okulu, H. Z., Muslu, N., Yilmaz, Y. O., Senler, B., Arabacioglu, S., & Bektaş, O. (2023). The readiness of stakeholders in the scientific inquiry-supported mentoring project. International Journal of Education in Mathematics, Science and Technology, 11(1), 37-55. https://doi.org/10.46328/ijemst.2651
- Valiandes, S., & Neophytou, L. (2018). Teachers’ professional development for differentiated instruction in mixed-ability classrooms: investigating the impact of a development program on teachers’ professional learning and on students’ achievement. Teacher Development, 22(1), 123-138. https://doi.org/10.1080/13664530.2017.1338196
- Vera, E. M., Heineke, A., Israel, M., Hill, M., Goldberger, N., Hook, K., & Anderson, B. (2022). Learning about English learners: teachers’ and leaders’ perceptions of effective professional development. International Multilingual Research Journal, 16(2), 93-112. https://doi.org/10.1080/19313152.2021.1971474
- Vygotsky, L. S. (1978). Mind and society: The development of higher psychological processes. Harvard University Press.
Keywords
United States, classroom observation, evaluation, randomized control trial, science, pedagogical practices, virtual professional development, virtual mentoring and coaching
Affiliations
Downloads
How to Cite
- [ ACM ]
- [ ACS ]
- [ APA ]
- [ ABNT ]
- [ Chicago ]
- [ Harvard ]
- [ IEEE ]
- [ MLA ]
- [ Turabian ]
- [ Vancouver ]