Context
The most recent study from McKinsey estimates that Black students may fall behind by 10.3 months and Hispanic students by 9.2 months due to school closures. These effects can follow children for a lifetime, eclipsing their ability to ever fully realize their potential. McKinsey estimates that Black and Brown students may lose 15-18% of their lifetime earning potential as a direct result of COVID-19.
As soon as schools closed last spring, Springboard Collaborative set out to help parents and teachers make distance learning a fruitful experience. Goal-setting is an essential ingredient in Springboard’s recipe for impact, the FELA method. Typically, we leverage schools’ existing assessment infrastructure to set measurable goals with families and track progress. When classrooms closed, most schools stopped assessing students altogether. Even NAEP testing will be deferred by at least a year, which risks keeping the sector in the dark about the extent of COVID learning regression.
In order to maintain an actionable data set through which teachers and families could set and achieve goals, Springboard developed a literacy assessment that parents could administer at home. This tool will play a pivotal role in the fulfillment of Springboard’s mission. Giving families visibility into their children’s reading development will enable them to be more savvy and demanding consumers of their kids’ education. This, in turn, can make schools more equitable places that value parents as experts, invest in them as partners, and are accountable to families regardless of race or income. In its first version, however, Springboard’s parent-facing assessment does not quantify children’s reading levels. (V1 is a questionnaire that qualitatively measures mastery of reading strategies).
Methodology
In order to quantitatively track student progress—and evaluate the efficacy of Springboard’s low-cost virtual programming—we collaborated with schools in New York to structure this study. The analysis compares i-Ready Lexile scores from before the shutdown (Feb/Mar 2020) and i-Ready Lexile scores from the fall (Sept/Oct 2020). As in Springboard’s recent external evaluation, we leveraged school-based assessment data—as opposed to internal program data—in order to compare a treatment and control group over a 6-month period.
Literacy scores were procured from a charter network whose overall performance mirrors that of district schools across the country. As opposed to higher-performing charters, this network is representative of Springboard’s typical school partners.
We captured data for 175 Rising 2nd through 5th graders that participated in Springboard Learning Accelerator (SLA) last summer. SLA is the light-touch, low-cost program that Springboard recently launched as a scalable solution that can be implemented remotely (and in person, as reopenings permit). The control group included 439 students at the same schools that did not participate in programming.
Results
Springboard participants averaged a 256-point Lexile gain, relative to a control group that grew just 41 points amidst school closures.
Rising 2nd graders who participated in the SLA program experienced an average of 344 Lexile points of growth between the pre and post assessments, compared to an average 113 points of growth from the pre to the post assessment for non-Springboard participants.
-Springboard students pre-shutdown: n=51
-Springboard students post-shutdown: n=55
-Non-Springboard students pre-shutdown: n=139
-Non-Springboard students post-shutdown: n=130
Rising 3rd graders who participated in the SLA program experienced an average of 247 points of growth between the pre and post assessments, compared to an average 41 points of growth from the pre to the post assessment for non-Springboard participants.
-Springboard students pre-shutdown: n=53
-Springboard students post-shutdown: n=53
-Non-Springboard students pre-shutdown: n=117
-Non-Springboard students post-shutdown: n=124
Rising 4th graders who participated in the SLA program experienced an average of 188 points of growth between the pre and post assessments, compared to an average 11 points of growth from the pre to the post assessment for non-Springboard participants.
-Springboard students pre-shutdown: n=42
-Springboard students post-shutdown: n=39
-Non-Springboard students pre-shutdown: n=82
-Non-Springboard students post-shutdown: n=101
-Rising 5th graders who participated in the SLA program experienced an average of 178 points of growth between the pre and post assessments, compared to an average 51-point regression from the pre to the post assessment for non-Springboard participants.
-Springboard students pre-shutdown: n=25
-Springboard students post-shutdown: n=22
-Non-Springboard students pre-shutdown: n=63
-Non-Springboard students post-shutdown: n=75
Key takeaways
When parents and teachers work together, they can turn the COVID slide into a springboard! The comparison group in this study had the typical experience of distance learning: a combination of synchronous and asynchronous instruction with neither goal-setting nor meaningful collaboration with families. The treatment group was comprised of similar students and families with the very same teachers in the very same schools. The fact that the treatment group made more than six times the progress of the control group is a testament to the power of the FELA method, characterized by goal-setting and effective parent-teacher collaboration. This analysis affirms what Springboard already knows to be true: family engagement is the force multiplier that enables schools to dramatically improve their outcomes with their existing people and assets.
These results are also significant in that it is the first data set for Springboard Learning Accelerator, our light-touch, low-cost program. SLA costs ~80% less than Flagship programming, but the early results are comparable. Such a dramatic reduction in cost without a commensurate reduction in outcomes is only possible because of the investments Springboard has made in understanding our impact drivers and codifying our core methodology: the FELA method. The fulfillment of Springboard’s mission at absolute scale—10 million children—hinges on our ability to work through others. SLA is an important step toward ‘unleashing’ our method such that schools can successfully implement FELAs with ever greater independence.
Finally, this study corroborated an important finding from Springboard’s recent external evaluation of Flagship programming: the students furthest behind grade-level expectations are the ones making the most progress.
Limitations
In this study, we could not control for other variables that might have influenced a student’s reading progress. Unlike Springboard’s prior external evaluation, this study did not do propensity matching.
In some cases, pre- and post-assessment sample sizes vary slightly, due to student transiency and missing assessment data. Sample sizes are noted above.
The advantage of using internal program data is that Springboard has rigorous vetting processes to ensure accuracy. The advantage of using external school-based data is that, in theory, this is a more objective data set that measures growth over a sustained period. However, schools tend to have more variable assessment practices that are difficult control for. Of course, data analysis can only be as reliable as the underlying data set. These are the tradeoffs Springboard weighs when learning from internal vs. external evaluation.