What works best for teachers and students in science education interventions? Statistics can provide some insight—but only if interpreted in context. For instance, the way a study is conducted may impact the outcome, independent of the actual effectiveness of the intervention.
To help researchers understand study results in context, BSCS Science Learning reviewed hundreds of studies in science education while taking into account the various factors contributing to the outcomes. This work resulted in published findings for student outcomes (AERA Open Journal) and teacher outcomes (The Journal of Research on Educational Effectiveness) and online tools for researchers to use when planning or evaluating studies of science education interventions.
The online tools, POWER Calculator for Student Outcomes, and POWER Calculator for Teacher Outcomes use data from the studies BSCS reviewed to estimate the likely effect size for a new study based on its characteristics, such as the nature of the study, the scientific discipline, characteristics of teachers or students, and other key variables.
When planning a study, researchers can use the POWER calculators to determine how many participants will be required to obtain a statistically significant result, giving researchers and funders increased confidence that they will obtain such a result without spending money and time unnecessarily on participants that are not needed. Once a study is completed, the tool enables users to interpret the size of their study’s effect in the context of similar studies.
In addition to the research findings and POWER calculators, BSCS has published the two data sets related to student and teacher outcomes. Researchers who want to do their own meta-analyses of the studies can explore the data from different angles, while efficiently using BSCS’s coding system. Click here for data set 1 (student outcomes) and click here for data set 2 (teacher outcomes).
Results from these meta-analyses are now published in AERA Open and the Journal of Research on Educational Effectiveness.
Plants are all around us. They are essential to our existence. Yet despite their ubiquity, how many science teachers truly understand plant biology and know how to effectively teach it to their students?
Digging Deeper, a BSCS Science Learning research and development project, has been investigating this question for the past four years. Building on the success of PlantingScience, Digging Deeper adds a robust professional learning model that engages teachers and scientists as collaborators in teaching and facilitating real-world science experiences for students.
Until now, the impact of bringing teachers and scientists together to learn, reflect on, and implement meaningful science experiences for students has been unexplored. Results of Digging Deeper will help validate methods and protocols that not only serve to eradicate “plant blindness” and prepare future scientists for plant biology challenges facing the planet but are also relevant to developing teacher and student expertise in inquiry-based science in any STEM discipline.
The Next Generation Science Standards (NGSS) identifies three equally important dimensions to learning science: science and engineering practices, crosscutting concepts, and disciplinary core ideas. This focus on 3D learning has created a critical need for assessments that can measure students’ ability to use these three dimensions together to make sense of real-world phenomena.
BSCS Science Learning is currently continuing a line of research started at the American Association for the Advancement of Science (AAAS)* to directly address this need. In our ASPECt-3D project, we will develop and validate scenario-based assessment tasks to measure students’ ability to use the three dimensions to make sense of energy-related phenomena. The assessment tasks will consist of multiple-choice and constructed-response items aligned to elementary, middle, and high school NGSS standards. Our assessment study will include a diverse range of teachers and students in grades 4-12 from urban, rural, and suburban locations across the United States.
As a result of this project, we will produce sets of NGSS-aligned assessments for measuring students’ 3D understanding of energy. We will also develop supporting materials for teachers, including the unpacking of the performance expectations, summaries of student misconceptions and difficulties, scoring rubrics, psychometric properties of the assessments, and guidelines for the use of the assessments and interpretation of results. Additionally, we will use our findings to create a professional learning workshop to help support assessment developers and classroom teachers in creating their own three-dimensional assessments. All products will be made available online for elementary, middle, and high school grade levels.
*ASPECt-3D builds on a previously funded IES project (R305A120138) focused mainly on assessing students’ understanding of science content. The researchers developed three vertically equated, multiple-choice instruments to assess students’ progress on the energy concept.
What exactly does a scientist do? How does she collect information and make sense of it all?
Data Nuggets, a four-year research project between Michigan State University and BSCS, is studying the effects of bringing real-world data into middle and high school classrooms. Students are given the opportunity to dive into an actual scientist’s research and practice looking for patterns in the data, develop explanations about natural phenomena, and identify hypotheses and predictions.
From topics such as “Won’t you be my urchin” to “Sticky situations: big and small animals with sticky feet,” teachers get to pick and choose from numerous options that get their students to think like a scientist, while the scientists who write the curriculum have an opportunity to share their research findings with a new audience in science.
Current research suggests that scientific models can help teachers transform their science instruction and enhance student learning. This premise grounds the Model-Based Educational Resource (MBER)—developed by Dr. Cindy Passmore and colleagues at UC Davis—which engages high school biology students in constructing models to make sense of science. Now researchers are wondering: How effectively can this approach to biology education support next generation science learning?
BSCS Science Learning has been awarded a grant to study the impact of the MBER program through a cluster-randomized trial (CRT) and expand the promise of efficacy and feasibility established in previous work. Throughout this project, we will revise the MBER program, develop associated assessment, and conduct an experimental study with 32 teachers in diverse California schools.
Our General Research Questions
- What is the impact of MBER on high school students’ science achievement?
- What factors influence that impact?
This study will also address a significant gap in the research on next generation curriculum materials. As we seek to advance the field’s knowledge about the impact of innovative materials on student learning, we will examine the following exploratory research questions:
- How does using MBER develop teachers’ vision of the Next Generation Science Standards (NGSS)?
- How is student learning mediated by the fidelity of implementation of the materials?
- How do teachers interact with materials designed to be modified for their classroom context?
- To what extent do MBER materials provide equitable opportunities to learn and close achievement gaps?
In addition to generating important research findings, the materials revised and studied in this project will be open source and freely available to teachers and schools, thereby maximizing the broader impacts of this work.
Middle school science teachers are always searching for professional learning (PL) opportunities and classroom curricula that are NGSS aligned. But time is limited, and high quality NGSS-aligned materials are scarce. That’s why BSCS Science Learning’s Three-Dimensional Teaching and Learning project, or 3D Middle School Science, is valuable.
Since 2015, 3D Middle School Science has been developing and testing digital curriculum materials and associated curriculum-based PL. For teachers, this project provides PL focused on how to implement (1) an NGSS-aligned unit and (2) high-leverage science teaching strategies through video-based lesson analysis. These STeLLA strategies help teachers reveal, support, and challenge student thinking while maintaining a coherent science content storyline. In conjunction with the PL element, teachers are supported in an interactive online environment and through online synchronous discussions with a facilitator and colleagues.
Students in 3D Middle School Science classrooms are immersed in an online environment that aids their understanding of complex concepts. A body systems unit challenges them to explore and ultimately solve a medical mystery: “What’s Wrong with M’Kenna?” Over the course of several lessons, students investigate how and why M’Kenna is constantly sick, unable to keep her food down, and losing weight. They use scientific reasoning skills and argumentation to identify the digestive system as the problematic organ system—and then engage with a series of interactive experiences, simulations, and animations to observe and analyze the differences between M’Kenna’s digestive system and a healthy person’s digestive system.
Ultimately, students solve the mystery and learn important lessons about how the features of specialized cells enable body systems to function, and they use that understanding to explain all of M’Kenna’s symptoms based on how body systems interact. More importantly, they learn to use the inquiry-based practices of scientists to construct their own understanding of complex phenomena.
3DMSS is available in Canvas or in a free, stand-alone website here.
Constructed-response assessments, in which students use their own language to demonstrate knowledge, are widely viewed as providing greater insight into student thinking than multiple-choice assessments. In the past, constructed-response assessments were expensive and time consuming to score. But recent advances in technology and measurement research are making them a feasible option for education settings. Lexical analysis and machine-learning technologies allow researchers to use computers to score student and teacher writing. The goal is to develop computer models that score written responses with the same levels of accuracy and reliability as human expert scorers.
BSCS Science Learning is leveraging these technologies in two research projects: PCK*lex and ArguLex.
The first project, PCKlex, explores measurement of teachers’ pedagogical content knowledge (PCK)—the type of teacher knowledge that bridges content knowledge and how to effectively teach the content in classrooms. It builds on several STeLLA studies that have measured PCK as an outcome of professional learning, as well as on the work of the BSCS PCK Summit in 2012, which brought together researchers from around the world to develop a consensus model of PCK. The product of the PCKlex project will be a computer scoring instrument that measures teachers’ PCK. The instrument will analyze teachers’ written descriptions of instructional practices they are observing through a video analysis task. In this task, teachers are exposed to carefully selected video clips from science lessons in which content-specific pedagogical moves are strategically illustrated. The computer scoring instrument will accurately reflect the time-consuming process of human scoring and will be available online. It will provide rapid PCK scores for research and evaluation purposes as well as formative feedback for teacher educators, professional learning providers, and teachers themselves. This project is a collaboration with the AACR group at Michigan State University.
Following in the footsteps of PCK*lex is ArguLex, a project that applies similar technologies to the measurement of students’ abilities to engage in scientific argumentation. Explanation and argument are essential practices in the Next Generation Science Standards (NGSS). However, these new standards will only have a meaningful impact if they are accompanied by high quality assessments that are closely aligned with a three-dimensional vision for teaching and learning science. Such assessments require a shift away from reliance on the efficiency and affordability of multiple-choice items and towards the use of more subjective, written tasks, aligned to NGSS performance expectations. The goal of the ArguLex project is to use automated analysis and machine learning techniques to develop an efficient, valid, and reliable measure of students’ placement on a learning progression for argumentation. Additionally, we are interested in the degree to which the computer scoring models are more or less biased against English language learners than humans scoring the same data (relative linguistic bias) and the capacity for automated scoring to differentiate between linguistic fluency and argumentation ability.
Racism is a serious problem in the United States. Research has shown that the biology curriculum can affect how students think about race. It can lead students to believe more strongly in three misconceptions:
- People of the same racial group are genetically uniform.
- People of disparate races are categorically different.
- Biologically-influenced abilities cannot change.
Individuals often justify racism with these misconceptions by arguing that it is pointless to try and reduce social inequality, because race biologically determines ability.
How can such beliefs be (un)learned through biology education?
What We’ve Learned
Teaching about human difference is not socially neutral.
Insights from our research have begun to illustrate how biology education affects the development of racism, for better or worse. We’ve learned:
- When biology education causes youth to perceive too much genetic variation between racial groups, it can increase prejudice.
- Conversely, the way we teach biology can reduce racial prejudice by helping students understand that there is more genetic variation within racial groups than there is between them.
In sum, the humane genetics research project is beginning to suggest that genetics education can create humane or inhumane outcomes depending on how it addresses human difference. If this hypothesis is correct, then learning about the social and quantitative complexities of human genetic variation research could prepare students to become informed participants in a society where human genetics is invoked as a rationale in sociopolitical debates concerning racial inequality.
At present, genetics education does very little to address how information about human genetic difference is distorted by racialist ideologues (The New York Times). Instead, our scholarship suggests that genetics curricula could actually contribute to harmful racial ideologies (The Atlantic). The kind of genetics education that we envision would promote human welfare by exposing the scientific flaws in biological justifications of racism and sexism. Our research and development explores how to bring that kind of education into existence.
What are the important ideas to teach students?
To learn about the content of a humane genetics education click here.
It is difficult to predict the potential impact of implementing a more humane genetics education in all school settings because we have not yet studied our intervention using a nationally representative sample of schools. Nevertheless, we can make some predictions about the clinical significance of implementing a more humane genetics education using statistics from our recent field experiments in 8th-12th grade biology classrooms, such as the one below:
Donovan, B. M., Semmens, R., Keck, P., Brimhall, E., Busch, K. C., Weindling, M., Salazar, B. (2019). Towards a More Humane Genetics Education: Learning about the social and quantitative complexities of human genetic variation research could reduce racial bias in adolescent and adult populations. Science Education, 1–32. https://doi.org/DOI: 10.1002/sce.21506
For example, using data from this most recent publication we can calculate the number needed to treat, which tells us how many people need to receive an intervention in order to prevent one additional case of a disease. The disease we are trying to prevent through our research is racism. Studies have found that racism is a public health problem because it is significantly associated with mortality in African Americans (e.g. read this study). The racially biased beliefs we have attempted to prevent through our intervention research are the following:
- the belief that there is more genetic variation between races than there is within them.
- the belief that racial groups differ cognitively and behaviorally simply because of genetic differences between races.
Changing these beliefs through genetics education is important because previous studies have found that people use these beliefs to justify racially prejudiced policies.
To calculate the number needed to treat one merely takes the inverse of the absolute risk reduction (or 1/ARR). We found that our humane genetics intervention reduced the risk that students developed a racially biased perception of genetic variation by 16.2%, and this risk reduction was statistically significant (p < 0.05). Likewise, we found that our humane genetics intervention resulted in a 6.6%, statistically-significant, reduction in the risk of students believing that racial groups differ cognitively and behaviorally simply because of their genes.
Our results therefore suggest that for every six students who learn from our intervention, we can prevent one additional student from developing the misperception that there is more genetic variation between races than there is within them. Furthermore, for every 15 students who learn from our intervention, our results suggest that we can prevent one additional student from agreeing that racial groups differ cognitively and behaviorally because of genetic differences between races. Altogether, if a biology classroom has 30 students, then our results suggest that implementing a more humane genetics education could prevent five students from developing the misperception that there is more genetic variation between races than within them and two of these students may also be prevented from believing that racial groups differ cognitively and behaviorally because of genes.
For a deeper dive into our line of research, review our research statement and published papers below. Click here to watch a video of the presentation, Towards a More Humane Genetics Education, or here to watch a video of the presentation, Genomics Literacy Matters.
Watch the American Association for the Advancement of Science (AAAS) 2019 briefing, Better Biology Instruction for a More Equitable Society, here (the presentations begin at 9 minutes and 18 seconds).
What We’re Currently Exploring
- Genetics education could affect the development of racial bias among adolescents. This work is supported by the National Science Foundation under Award No. 1660985.
- Genetics education could affect the development of gender bias among adolescents. This work is supported by the National Science Foundation under Award No. 1956152.
- Genetics education could affect the development of undergraduates’ motivation to pursue STEM. This work is supported by the National Science Foundation under Award No. 1914843.
- Donovan, B. M., Semmens, R., Keck, P., Brimhall, E., Busch, K. C., Weindling, M., Duncan, A., Stuhlsatz, M., Buck Bracey, Z., Bloom, M., Kowalski, S., Salazar, B. (2019) Towards a More Humane Genetics Education: Learning about the social and quantitative complexities of human genetic variation research could reduce racial bias in adolescent and adult populations . Science Education.
- Donovan, B.M., Stuhlsatz, M., Edelson, D.C., Buck Bracey, Z.B. (2019) Gendered Genetics: How reading about the genetic basis of sex differences in biology textbooks could affect beliefs associated with science gender disparities . Science Education.
- Donovan, B.M. (2018). Looking backwards to move biology education toward its humanitarian potential: A review of Darwinism, Democracy, and Race . Science Education.
- Donovan, B. M. (2017) Learned inequality: Racial labels in the biology curriculum can affect the development of racial prejudice . Journal of Research in Science Teaching. 54(3), 379-411.
- Donovan, B. M. (2016). Framing the genetics curriculum to support social justice: An experimental exploration of how the biology curriculum influences students’ beliefs about the racial achievement gap . Science Education. 100(3), 586-616.
- Donovan, B. M. (2015a). Putting humanity back into the teaching of human biology . Studies in History and Philosophy of Science Part C: Studies in History and Philosophy of Biological and Biomedical Sciences. 52, 65-75.
- Donovan, B. M. (2015b). Reclaiming race as a topic of the United States biology curriculum . Science Education. 99(6) 1092-1117.
- Donovan, B. M. (2014). Playing with fire? The impact of the hidden curriculum in school genetics on essentialist conceptions of race . Journal of Research in Science Teaching, 51(4), 462–496.
- Donovan, B. M., Moreno Mateos, D., Osborne, J. F., & Bisaccio, D. J. (2014). Revising the Economic Imperative for US STEM Education . PLoS Biology, 12(1), e1001760.
Media Mentions –
BSCS Science Learning has developed a nationally-recognized program for teacher learning called STeLLA®, Science Teachers Learning from Lesson Analysis. K-12 science teachers who want to implement research-based curriculum, improve their teaching, or navigate next generation science all have something to gain from this proven program. And so do their students.STeLLA is based on a 17-year line of research and development at BSCS. It has demonstrated impacts on both teacher and student learning above and beyond any impacts from a traditional science teacher professional learning program.
STeLLA’s impact is significant across contexts. It works in preservice and inservice settings for elementary, middle, and high school teachers.
So what’s next? We’re translating research into practice by creating broadly accessible versions of the program.
STeLLA is now available online and will soon be available in hybrid format. We’re currently working to expand STeLLA to different grade levels and science disciplines.
How STeLLA Works
STeLLA helps teachers motivate students to learn science. Specifically, it supports teachers in learning to use effective teaching strategies through a powerful video-based lesson analysis approach. Strategies include engaging student thinking and organizing instruction in a way that connects science ideas. Teachers learn to use these strategies by analyzing classroom videos, and sharing their thinking in facilitated sessions with other teachers. The STeLLA program takes place in-person, online, or in a hybrid format over the course of one school year (typically 90 hours), during which teachers apply what they’re learning in their own classrooms.
A Program Based on Research
Over the last 17 years, STeLLA has demonstrated impacts on both teacher and student learning above and beyond any impacts from a traditional science teacher professional learning program.
See our growing line of research at-a-glance: Development of STeLLA
STeLLA Programs Now Available
BSCS now offers broadly accessible versions of STeLLA. Middle school teachers can register for STeLLA for A Medical Mystery, a fully online program that supports teachers in the enactment of an NGSS-aligned curriculum unit. Elementary teachers can register for STeLLA Online, a fully online program that supports teachers in the enactment of Water Cycle and Earth’s Changing Surface units.
STeLLA Scale Up and Sustainability
BSCS is partnering with leaders and science educators throughout Tennessee and Kentucky to help address a national need for high quality science teacher professional learning. With the largest research grant in the organization’s history, BSCS will test, refine, and scale up its STeLLA model with the goal of making a new hybrid (online and in-person) version broadly available. During this five-year project, BSCS will work with 4th and 5th grade teachers primarily from high needs, rural districts and schools. Regional leaders including PIMSER (the Eastern Kentucky University Partnership Institute for Math and Science Education Reform), the Tennessee Aquarium, and Instruction Partners, will help ensure the new STeLLA model is aligned to state science standards and the impact is sustainable long-term. Meanwhile, BSCS is exploring strategies to provide equitable access to STeLLA across the country.
STeLLA for A Medical Mystery
BSCS developed a fully-online STeLLA course to support middle school teachers in the enactment of a 3D body systems curriculum unit: A Medical Mystery. This NGSS-aligned unit, developed in partnership with Oregon Public Broadcasting, immerses students in an online environment that challenges them to use scientific reasoning skills and argumentation. For eight weeks, students investigate and ultimately solve, "What's Wrong with M'Kenna?" Field-test teachers participated in 11 weeks of STeLLA professional learning to enhance their enactment of the unit. The ultimate goal is to make STeLLA for A Medical Mystery broadly accessible to middle school teachers across the country.
A decade ago, BSCS introduced STeLLA to 144 teachers and 2,800 students across Colorado’s Front Range. BSCS conducted a rigorous experimental study of this entirely in-person professional learning program. As a result, both teacher and student learning improved significantly. Since then BSCS has wondered: would an entirely online version of the STeLLA CO program be similarly impactful? That’s what BSCS is exploring and evaluating today during its STeLLA Online program. The ultimate goal is to support more teachers than ever before by offering STeLLA in an online format that is convenient, accessible, and effective.
BSCS is partnering with University of Colorado, Boulder; University of Colorado, Colorado Springs; and University of Northern Colorado, Greeley. This is the first STeLLA project designed to collaborate with university faculty and cooperating teachers educating preservice middle and high school science teachers. The program has the potential to impact one-third of all new secondary science educators certified in Colorado annually. Research is being conducted to develop a new understanding of the benefits and challenges associated with bringing STeLLA to this new context. And ultimately, research will explore STeLLA’s effectiveness on first-year practice outcomes for preservice secondary science teachers. Learn more here.
STeLLA High School
BSCS partnered with Jefferson County Public Schools, Kentucky to deliver a version of the STeLLA program to one-third of the district’s high school biology teachers. Louisville is the first region in the United States to benefit from research on STeLLA’s effectiveness at the high school biology level. The PD program took place throughout the 2017–2018 school year. Following a successful intervention, BSCS is spearheading leadership development work to deliver STeLLA at a broader and sustainable, district-wide level.
BSCS partnered with Minneapolis Public Schools, St. Paul Public Schools, and the University of Minnesota to bring STeLLA to a broad audience across the state of Minnesota accessed through online resources. This was the first opportunity for BSCS to expand STeLLA beyond elementary school and to incorporate engineering design in alignment with Minnesota’s new academic standards in science. The resulting STeLLA program covered STEM teaching and learning across elementary, middle, and high schools in Minnesota. BSCS has expanded the impact of this state-wide work by building leadership capacity among STeLLA-prepared teachers.
BSCS partnered with California State Polytechnic University, Pomona and a high-needs, urban district in California to test a dissemination model of the intensive STeLLA program. The ultimate goal was for STeLLA to successfully reach and impact every elementary school teacher and their students in the high-needs, urban district, where more than 65% of students are English language learners. BSCS created model units and professional learning materials covering grades K-6 and conducted leadership development programs to prepare the university science faculty and PUSD elementary school teachers to lead STeLLA programs. Research is being conducted to identify the effectiveness of a STeLLA program delivered by trained school district and teacher leaders on both elementary teacher professional learning and student outcomes. Preliminary results are promising. An initial impact study revealed statistically significant improvements in student outcomes that are comparable to results in BSCS-led STeLLA programs.
BSCS partnered with the University of New Mexico and the University of Houston-Victoria on an intensive STeLLA project designed to prepare preservice elementary school science teachers. A much broader and more interactive version of the original ViSTA, this program proved to be a resounding success, demonstrating improvements in teachers’ science content understanding, scientific reasoning skills, and most significantly, an extraordinary advancement in outcomes among these teachers’ students. In their first year of teaching, ViSTA Plus participants performed two standard deviations higher in effectively improving science learning outcomes for students than their university peers who did not participate in the ViSTA Plus program. These findings suggest that the STeLLA approach is effective at preparing new teachers to have immediate and positive impact on student learning.
BSCS developed a free online course to help high school science teachers frame complex energy concepts in a relevant and compelling way for students. The STeLLA course covers six units—Coal, Nuclear, Wind, Geothermal, Biofuels, and Solar Energy—and includes 34 engaging classroom videos, 30 content animations, and 20 interactive learning experiences. While originally designed for teachers, EMAT is also a beneficial resource for teacher educators and district PD leaders. Research findings show EMAT to be effective at enhancing teachers’ content knowledge and their ability to reveal, support, and challenge student thinking.
BSCS partnered with school districts along Colorado’s Front Range in a randomized-controlled experiment involving 144 teachers and more than 2,800 students. BSCS compared outcomes for 4th and 5th grade teachers in the STeLLA program with outcomes for teachers who participated in a more traditional science teacher PD program focused only on content deepening. The students of teachers in both groups learned as a result of their teachers' participation in PD. However, there was a substantial difference in the learning of students whose teachers were in the STeLLA group compared with those in the comparison group. The difference in scores is equivalent to 23 percentile points. Test results also showed STeLLA students were able to answer questions involving more-complex scientific reasoning.
BSCS explored the value of a STeLLA-inspired program for preservice elementary science teachers by developing and studying the impact of online, videocase-based modules designed as tools to support teacher education courses. As a result, 30 participating university instructors and their students, the preservice teachers, significantly increased their science content knowledge and ability to analyze video-recorded classroom lessons for powerful instruction practices. Though a small-scale study, ViSTA’s results were promising and inspired future investigation on the effectiveness of science teacher preparation.
Researcher Kathy Roth laid the foundation for the signature line of STeLLA research that continues at BSCS today. Roth’s team at LessonLab Research Institute partnered with California State Polytechnic University, Pomona in a study involving 32 upper elementary school teachers in California. Researchers compared outcomes for 4th, 5th, and 6th grade teachers in the STeLLA program with outcomes for teachers who participated in a more traditional science teacher professional learning (PL) program focused only on content deepening. Despite a relatively small sample size, the STeLLA study provided strong evidence that elementary teachers can improve their science instruction and deepen their science content knowledge in ways that directly impact students’ learning by participating in a videocase-based, analysis-of-practice program.
High quality science education is more important than ever. Teachers must prepare students to succeed in a 21st century society, where scientific reasoning and critical thinking skills are essential. To prepare teachers to achieve this goal, BSCS is working to bring the STeLLA approach to teachers nationwide through partnerships with schools, districts, teacher educators, and funders. Learn more.