AERA 2025

Authors

Erin Furtak, University of Colorado – Boulder; Clarissa Deverel-Rico, BSCS Science Learning

Abstract

Culturally responsive approaches to science teaching encourage us to consider how classroom assessment – both in its design and enactment – can create space for learners to make connections with their prior experiences as they make sense of phenomena that are interesting to them and relevant to their lives. In this chapter, we examine the enactments of two phenomenon-based, high school biology formative assessment tasks to understand the ways in which variations in teachers’ responsiveness to student ideas through their classroom practices can enable students’ learning. We highlight how even when tasks are the same, students’ opportunities to learn through formative assessment can be expanded by the ways teachers structure learning environments around the tasks. We use coded classroom data to illustrate how teachers encouraged students to share their experiences, use everyday language, and their own ways of cultural expression to make sense of scientific phenomena. We use the lens of culturally responsive teaching to inform our discussion of implications for practice and to identify future directions for science formative assessment.

Time and Location

8–9:30 a.m. MDT, The Colorado Convention Center, Floor: Meeting Room Level, Room 710

Authors

Jeffrey Snowden, BSCS Science Learning; Brian M. Donovan, University of Colorado – Boulder; Lindsey Mohan, BSCS Science Learning; Emily Harris, BSCS Science Learning

Abstract

This study evaluated the impact of localized climate change curriculum on high school students’ environmental science agency (ESA). Using a cohort-controlled quasi-experiment, we compared outcomes between students taught with a business-as-usual approach and those experiencing a localized curriculum. We assessed changes in climate change knowledge, science identity, roles/expertise in science, and foundations for change. Results showed significant positive effects of the localized curriculum on knowledge and foundation for change (p < .001 for both). Treatment group students scored 0.257 logits higher on knowledge and 0.246 logits higher on foundation for change. Effects on roles/expertise and science identity were positive but not statistically significant. Findings suggest localized climate education can bridge the gap between knowledge and action.

Time and Location

11:40 a.m.–1:10 p.m. MDT, The Colorado Convention Center, Floor: Terrace Level, Bluebird Ballroom Room 3B

Authors

William R. Penuel, University of Colorado – Boulder; Carol Pazera, University of Texas at Austin; Zoë Buck Bracey, BSCS Science Learning; Daniel C. Edelson, BSCS Science Learning; Kate Henson, University of Colorado – Boulder; Brian J. Reiser, Northwestern University; Tamara Sumner, University of Colorado – Boulder; Douglas A. Watkins, Denver Public Schools

Abstract

Purpose
The purpose of this poster is to use our study of improving science education across the United States to represent Design-Based Implementation Research (DBIR; Penuel et al., 2011) as a formal approach to improvement research in education.

Perspective
DBIR has roots in learning sciences and policy research (Fishman et al., 2013). It engages educators and leaders in partnerships for systems-level change toward educational equity (Penuel et al., 2022). Co-design is a key practice, focusing on the design of instructional guidance infrastructures like curriculum (Penuel, 2019). Mapping the improvement space is a collaborative process shaped by sociopolitical contexts (Hopkins et al., 2022; Penuel et al., 2013). Guided by theories of learning and implementation, interventions are tested using mixed methods approaches (Russell et al., 2013). Sustaining improvement focuses on leveraging and transforming existing infrastructures.

Modes of Inquiry
In this poster, we showcase our efforts to use DBIR as a Developers Consortium in collaboration with our partners in ten states to improve science education through enhanced instructional materials and linked professional development.

Data Sources
We drew on the following data sources: student exit tickets and surveys, teacher surveys, student assessments, and teacher interviews.

Findings
Building the foundational conditions: A nonprofit organization contracted with a Developers Consortium of four research organizations and a partner district to develop materials based on collaboratively developed, evidence-based design specifications. Structures included regular input from a steering committee of ten state education agency leaders as part of 4-month development and testing cycles.

Mapping the improvement space: Activities included a National Academies workshop, a market research study, and monthly state steering committee meetings.

Identifying a theory of (and ideas for) improvement: The materials were guided by an instructional approach called storylines (Reiser et al., 2021), emphasizing coherent learning sequences from the student’s perspective. Storylines are a form of problem-based learning with strong evidence (Hmelo-Silver, 2004). Change relies on sustained, high-quality professional learning (Gonzalez et al., 2022).

Iterating and measuring: Initial unit versions were field-tested in 300 classrooms. Data on student experience, teacher practices, and teacher professional learning were collected. These data, along with direct feedback and co-revision workshops with teachers, were used to improve units. All units were formatively and externally evaluated using a rubric developed by WestEd and EdReports, and effects on student learning were measured.

Spreading and sustaining improvement: Spread is supported by an independent nonprofit that provides free materials and contracts with publishers and kit providers, helping to fund future revisions. Over 30 certified professional development providers also support spread.

Significance
This study shows that a design and development study can be more than a small-scale effort to test a single innovation but can be conducted at scale with robust partnerships.

Time and Location

11:40 a.m.– 1:10 p.m. MDT, The Colorado Convention Center, Floor: Terrace Level, Bluebird Ballroom Room 2A

Authors

Jeffrey Snowden, BSCS Science Learning; Brian M. Donovan, University of Colorado – Boulder; Lindsey Mohan, BSCS Science Learning; Emily Harris, BSCS Science Learning

Abstract

Objectives
To achieve a vision of science learning where learners’ interests and identities are integrated into sensemaking around real-world phenomena (NRC, 2012; NASEM, 2024), we must better understand how to design and work within equitable, coherent assessment systems (Shepard et al., 2018). We explore one approach to designing science assessments that (re)center learners’ interests and identities.

Theoretical Framework
We draw on culturally sustaining approaches (Alim & Paris, 2017) that emphasize responding to and sustaining the knowledge and practices of youth as essential educational goals, particularly of youth who have historically been underrepresented or marginalized from school or science (Randall, 2021). Assessments can help learners make meaningful connections to their interests and science-linked identities (Penuel et al., 2019) and include metrics of experience (NASEM, 2024) that help surface the voices of learners.

Methods
We draw on student experience data from 520 middle schoolers from across the U.S. who each completed two of eight 6th grade, Earth science tasks in the pilot phase of a larger project to validate a pool of standards-aligned middle school assessments for use with high-quality curricula. Students provided demographic information, completed two phenomenon-based assessments, and responded to ‘experience’ items for each task. We focus on the ‘Relevance’ and ‘Liked’ items described in Table 1, and the tasks in Table 2. We particularly elevate the voices of historically underrepresented and marginalized populations, and will report on the full data set in the paper.

Results
Our preliminary analyses show that the majority of students found the tasks relevant, selecting self, family, friends, and/or community 56% (n=520) across the eight tasks. We also notice differences between individual tasks (Figure 1), and are beginning to explore these differences with the open-response items. Preliminary grounded coding of the ‘Liked’ item has surfaced the following themes: Multimodal representations (e.g. images), multiple ways for demonstrating understanding, opportunities to think and share ideas, use of prior knowledge/skills, and interesting topic. To illustrate interesting topic, one Hispanic or Latine student who selected ‘self’ for the Rubber Ducks task (with lowest relevance ratings) said, “It was fun because I never knew that 30,000 ducks were all over the ocean,” while another who selected all relevance categories, wrote, “Que analizaba y aprendí de cómo funcionan las corrientes del océano pacífico,” demonstrating, opportunities to think and use of prior skills.

Our initial exploration reveals what students may find relevant and what they may like in taking the assessments and it surfaces tensions with assessment developers’ designs for students. Though we have more to explore with these data, as white assessment researchers, we seek to diversify whose voices are represented and inform our forthcoming revisions. Further, while we are committed to elevating the voices of underrepresented students, we do not wish to essentialize, amplifying the need for further and thoughtful analyses.

Significance
This work highlights a possibility for centering learners’ voices in decisions around science classroom assessment by including metrics of experience to inform assessment (re)design.

Time and Location

1:30–3 p.m MDT, The Colorado Convention Center, Floor: Meeting Room Level, Room 107

Session Type: Business Meeting

Abstract

Come join the AERA Classroom Assessment SIG community for a brief business meeting followed by networking and refreshments. We look forward to meeting you for this once-a-year opportunity dedicated to all who are interested in classroom assessment research and practice. This session is well attended by emerging, established, and legendary scholars in classroom assessment as well as people from a variety of other roles (e.g., educators, publishers, scholars from adjacent fields). All are welcome!

Time and Location

7:00 PM – 8:30 PM (MDT), The Colorado Convention Center, Floor: Terrace Level, Bluebird Ballroom Room 2D

Authors

Joseph Taylor, University of Colorado – Colorado Springs/American Institutes for Research; Christopher D. Wilson, BSCS Science Learning; Charlotte Chen, American Institutes for Research; So Jung Park, American Institutes for Research; Bo Zhu, American Institutes for Research; Molly A.M. Stuhlsatz, BSCS Science Learning; Melia E. Repko-Erwin, American Institutes for Research; Dong Hoon Lee, American Institutes for Research; Jesse D. Levin, American Institutes for Research; Amy E. Trauth, American Institutes for Research

Abstract

This study examines the scalability and impact of the Science Teachers Learning from Lesson Analysis (STeLLA®) program, which aims to enhance science teaching aligned with the Next Generation Science Standards. Through a one-year cycle of professional learning communities, STeLLA focuses on student thinking and instructional coherence. The study involved fourth and fifth grade teachers across multiple cohorts in Tennessee and Kentucky, using a randomized trial design. Key findings indicate that STeLLA significantly improves teacher content knowledge, though its impact on student achievement was positive but not statistically significant. The study highlights the potential for scaling STeLLA with local leaders and hybrid delivery models, despite reduced effect sizes compared to expert-led implementations, suggesting a trade-off between cost and effectiveness at scale.

Time and Location

9:50–11:20 a.m. MDT, The Colorado Convention Center, Floor: Meeting Room Level, Room 106

Authors

Clarissa Deverel-Rico, BSCS Science Learning; Patricia Olson, BSCS Science Learning; Cari F. Herrmann-Abell, BSCS Science Learning; Christopher D. Wilson, BSCS Science Learning

Abstract

Current science education reform efforts envision learners drawing on their cultural and linguistic backgrounds while engaging with disciplinary practices and ideas to make sense of compelling, relevant phenomena. This vision requires coherence between curriculum, instruction, and assessment. We seek to understand if currently available classroom assessment tasks support an equitable vision of science education. Preliminary findings from characterizing 104 tasks reveal that most tasks target current national science standards and include a phenomenon. However, there are challenges in framing phenomena that attend to learners’ interests and identities and engage learners in authentic sensemaking. Further, while some task materials embody the idea of assessments FOR learning and asset-based views of learners, we see room for improvement in aligning with these commitments.

Time and Location

11:40 a.m.–1:10 p.m. MDT, The Colorado Convention Center, Floor: Ballroom Level, Four Seasons Ballroom 1