ManyClasses is a collaborative research project investigating the generalizability of educational interventions in real classrooms.
For decades, researchers from psychology, cognitive science, and education have attempted to identify practices that improve student learning. When studied in authentic educational contexts, research on these practices are often limited to just one class, a convenience sample, which limits the generalizability of inferences that can be drawn about the practice’s benefits. To provide legitimate estimates of the benefits of recommended practices for student learning, research needs to extend beyond the bounds of a single classroom.
Toward that goal, ManyClasses is a research model for evaluating the efficacy of educational practices across a variety of authentic learning contexts. As with similar efforts in psychology (Many Labs, Many Babies, ManyPrimates), the core feature of ManyClasses is that researchers measure an experimental effect across many independent samples – in this case, across many classes. Rather than conducting an embedded learning study in just one educational context, a ManyClasses study will examine the same research question in dozens of contexts, spanning a range of courses, institutions, formats, and student populations. By drawing the same experimental contrast across a diversity of educational implementations, and then analyzing pooled results, we can assess the degree to which an experimental effect generalizes across multiple classes, and systematically investigate how a manipulation might be more or less effective for different students in different contexts.
Who is running this project?
The ManyClasses team is:
- Paulo Carvalho (Carnegie Mellon University)
- Josh de Leeuw (Vassar College)
- Emily Fyfe (Indiana University)
- Rob Goldstone (Indiana University)
- Ben Motz (Indiana University)
Click to see Ben Motz and Emily Fyfe describe the ManyClasses project
How does ManyClasses work?
A ManyClasses project is an experiment that compares two or more classroom practices, randomly assigned to different students at different times, across many different classes. A central goal of the ManyClasses model is to maintain the rigor of a randomized experiment while also allowing teachers the flexibility to prepare materials that are authentic to their institutional and disciplinary norms. Researchers recruit participating teachers, and outline the desired differences between different versions of the assignment according to the research question (e.g., assignments with immediate vs. delayed feedback, assignments with worked examples vs. problems to solve). Given the research goals, participating teachers create instructional material for their classes and then present the material to their students as part of their normal classroom routine (e.g., as online homework). ManyClasses researchers provision different versions of the assignment to different students according to randomly assigned experimental conditions. Finally, teachers report relevant learning outcomes corresponding to the different assignments, and researchers analyze anonymized pooled results for the different classroom samples.
A controlled educational experiment that brings together dozens of teachers in different disciplines at different institutions presents no shortage of challenges, but also represents an ideal model for drawing inferences about the generalizability of learning principles across educational contexts.
See the Projects page for details on ManyClasses studies.