Testing Teaching
What Do Changes in State Test Scores Imply for Later Life Outcomes?
Elena Doty et al.
NBER Working Paper, December 2022
Abstract:
In the three decades before the pandemic, mean achievement of U.S. 8th graders in math rose by more than half a standard deviation on the National Assessment of Educational Progress (NAEP). Between 2019 and 2022, U.S. students had forfeited 40 percent of that rise. To anticipate the consequences of the recent decline, we investigate the past relationship between NAEP scores and students' later life outcomes by year and state of birth. We find that a standard deviation improvement in a birth cohort's 8th grade math achievement was associated with an 8 percent rise in income, as well as improved educational attainment and declines in teen motherhood, incarceration and arrest rates. If allowed to become permanent, our findings imply that the recent losses would represent a 1.6 percent decline in present value of lifetime earnings for the average K-12 student (or $19,400), totaling $900 billion for the 48 million students enrolled in public schools during the 2020-21 school year.
Shadow Education, Pandemic Style: Social Class, Race, and Supplemental Education during Covid-19
Jennifer Lee, Natasha Quadlin & Denise Ambriz
Research in Social Stratification and Mobility, forthcoming
Abstract:
Research on shadow education -- i.e., one-on-one or group learning intended to supplement children's experiences in school -- has documented persistent social class and racial/ethnic inequalities. Yet, as with many things during the Covid-19 pandemic, the nature of shadow education changed dramatically. Much supplemental education shifted online, potentially increasing accessibility; many universities became test-optional, potentially reducing the demand for the shadow education industry; and a new form of shadow education -- learning pods -- emerged to take pandemic schooling from a more individual to a more collective experience. In this article, we use data from a sample of U.S. parents of K-12 students stratified by race/ethnicity (N = 1,911) to assess social class and racial/ethnic inequalities in shadow education in 2020-21, the first full academic year of the Covid-19 pandemic. We are also the first scholars, to our knowledge, to assess high-quality data on the use of learning pods. We find that during the pandemic, African American and South Asian students were more likely than White students to use test preparation services and online supplemental education, and that African Americans, East Asians and Latinx were more likely to utilize private tutoring. We find few disparities by family income, however, thus supporting the idea that some forms of shadow education have become more accessible than they once were. Regarding learning pods, we find that pods were most common among African American families and families with parents who were less educated and worked fulltime. Thus, most learning pods were not a means of "opportunity hoarding," as some scholars originally feared, but instead provided sorely needed childcare and support during a time of social turbulence.
U.S. School Finance: Resources and Outcomes
Danielle Handel & Eric Hanushek
NBER Working Paper, December 2022
Abstract:
The impact of school resources on student outcomes was first raised in the 1960s and has been controversial since then. This issue enters into the decision making on school finance in both legislatures and the courts. The historical research found little consistent or systematic relationship of spending and achievement, but this research frequently suffers from significant concerns about the underlying estimation strategies. More recent work has re-opened the fundamental resource-achievement relationship with more compelling analyses that offer stronger identification of resource impacts. A thorough review of existing studies, however, leads to similar conclusions as the historical work: how resources are used is key to the outcomes. At the same time, the research has not been successful at identifying mechanisms underlying successful use of resources or for ascertaining when added school investments are likely to be well-used. Direct investigations of alternative input policies (capital spending, reducing class size, or salary incentives for teachers) do not provide clear support for such specific policy initiatives.
Investigating the Causal Effects of Arts Education
Daniel Bowen & Brian Kisida
Journal of Policy Analysis and Management, forthcoming
Abstract:
There is a paucity of research on the causal relationship between arts learning and educational outcomes. Investigating these relationships has become imperative as policymakers increasingly prioritize empirical evidence of educational impacts, which often leads to curriculum narrowing that favors traditionally-tested subjects. Employing a randomized controlled trial with 42 elementary and middle schools in Houston, Texas, we find that randomly assigning arts educational opportunities reduces disciplinary infractions, improves writing achievement, and increases students' emotional empathy. Students in elementary schools, which were the primary focus of the program, also experience increases in school engagement, college aspirations, and cognitive empathy. As the first large-scale randomized control trial of arts learning in an authentic school setting, these findings provide strong evidence that the arts can produce meaningful impacts on students' academic outcomes and social-emotional development. Education policymakers should consider these benefits when assessing the role of the arts in schools.
Preferences, Inequities, and Incentives in the Substitute Teacher Labor Market
Matthew Kraft, Megan Lane Conklin & Grace Falken
NBER Working Paper, December 2022
Abstract:
We examine the labor supply decisions of substitute teachers - a large, on-demand market with broad shortages and inequitable supply. In 2018, Chicago Public Schools implemented a targeted bonus program designed to reduce unfilled teacher absences in largely segregated Black schools with historically low substitute coverage rates. Using a regression discontinuity design, we find that incentive pay substantially improved coverage equity and raised student achievement. Changes in labor supply were concentrated among Black and Hispanic substitutes from nearby neighborhoods with experience in incentive schools. Wage elasticity estimates suggest incentives would need to be 50% of daily wages to close fill-rate gaps.
Mixed Signals? Economically (Dis)advantaged Students' College Attendance Under Mandatory College and Career Readiness Assessments
Christian Michael Smith & Noah Hirschl
Educational Researcher, forthcoming
Abstract:
In 2015, Wisconsin began mandating the ACT college entrance exam and the WorkKeys career readiness assessment. With population-level data and several quasi-experimental designs, we assess how this policy affected college attendance. We estimate a positive policy effect for middle/high-income students, no effect for low-income students, and greater effects at high schools that had lower ACT participation before the policy. We further find little evidence that being deemed college-ready by one's ACT scores or career-ready by one's WorkKeys scores affects college attendance probabilities. Pragmatically, the findings highlight the policy's excellence and equity consequences, which are complex given that the policy has principally helped advantaged students. Theoretically, the findings shed light on students' (dis)inclinations to update educational beliefs in light of new signals.
In-Person Schooling and Youth Suicide: Evidence from School Calendars and Pandemic School Closures
Benjamin Hansen, Joseph Sabia & Jessamyn Schaller
NBER Working Paper, December 2022
Abstract:
This study explores the effect of in-person schooling on youth suicide. We document three key findings. First, using data from the National Vital Statistics System from 1990-2019, we document the historical association between teen suicides and the school calendar. We show that suicides among 12-to-18-year-olds are highest during months of the school year and lowest during summer months (June through August) and also establish that areas with schools starting in early August experience increases in teen suicides in August, while areas with schools starting in September don't see youth suicides rise until September. Second, we show that this seasonal pattern dramatically changed in 2020. Teen suicides plummeted in March 2020, when the COVID-19 pandemic began in the U.S. and remained low throughout the summer before rising in Fall 2020 when many K-12 schools returned to in-person instruction. Third, using county-level variation in school reopenings in Fall 2020 and Spring 2021 - proxied by anonymized SafeGraph smartphone data on elementary and secondary school foot traffic - we find that returning from online to in-person schooling was associated with a 12-to-18 percent increase teen suicides. This result is robust to controls for seasonal effects and general lockdown effects (proxied by restaurant and bar foot traffic), and survives falsification tests using suicides among young adults ages 19-to-25. Auxiliary analyses using Google Trends queries and the Youth Risk Behavior Survey suggests that bullying victimization may be an important mechanism.
Labor market returns to MBAs from less-selective universities: Evidence from a field experiment during COVID-19
Christopher Bennett
Journal of Policy Analysis and Management, forthcoming
Abstract:
Master's degree enrollment and debt have increased substantially in recent years, raising important questions about the labor market value of these credentials. Using a field experiment featuring 9,480 job applications submitted during the early months of the COVID-19 pandemic, I examine employers' responses to job candidates with a Master of Business Administration (MBA), which represents one-quarter of all master's degrees in the United States. I focus on MBAs from three types of less-selective institutions that collectively enroll the vast majority of master's students: for-profit, online, and regional universities. Despite the substantial time and expense required for these degrees, job candidates with MBAs from all three types of institutions received positive responses from employers at the same rate as candidates who only had a bachelor's degree-even for positions that listed a preference for a master's degree. Additionally, applicants with names suggesting they were Black men received 30 percent fewer positive responses than otherwise equivalent applicants whose names suggested they were White men or women, providing further evidence of racial discrimination in hiring practices.
The Impact of School Spending on Civic Engagement: Evidence from School Finance Reforms
Erdal Asker, Eric Brunner & Stephen Ross
NBER Working Paper, December 2022
Abstract:
A primary rationale for public provision of K-12 education and state financing of school spending is that education fosters civic engagement and the development of social capital. However, limited evidence exists on whether and how school spending affects civic engagement. Virtually all studies focus on the impact of educational attainment (as opposed to school spending) on political activity. We provide the first causal evidence on how school spending affects volunteerism as well as voting. The court-ordered and legislative school finance reforms that occurred throughout the United States over recent decades led to large and plausibly exogenous shocks to K-12 school spending. We estimate difference-in-difference-in-differences (DDD) models to isolate the causal impact of school spending on civic engagement. Using data from the National Education Longitudinal Study of 1988 (NELS), the Education Longitudinal Study of 2002 (ELS), and the High School Longitudinal Study of 2009 (HSLS), we find that exogenous increases in school spending led to increases in the probability that young adults volunteer and the amount of time they spend volunteering. In contrast, we find little evidence that school spending impacts voting. Consistent with prior studies, we find that increases in school spending increase high school graduation and college attendance.
Performance or Progress? The Physical and Rhetorical Removal of Indigenous Peoples in Settler Land Acknowledgments at Land-Grab Universities
Theresa Ambo & Theresa Rocha
American Educational Research Journal, forthcoming
Abstract:
Land acknowledgments are an evolving practice to recognize local Indigenous Peoples as traditional stewards of their homelands. Using a content and discourse analysis, we conduct the first empirical study of U.S. land acknowledgment statements focusing on the 47 land-grab universities created under the 1862 Morrill Act. We find that LGUs tend to adopt statements in urban areas, where federally recognized tribes are present, and at institutions with over 100 enrolled Native American students. Land acknowledgment statements also commonly name local Indigenous Peoples yet often fail to articulate their responsibilities to them, include superficial gestures, and center multicultural language. We offer "rhetorical removal" to describe the tendency of land-grab universities to deploy language that selectively erases Indigenous Peoples and, thus, argue that statements must directly address settler colonial legacies of violence and redistribute material support for Indigenous students and partnerships with Native nations.
Professional Development at Scale: The Causal Effect of Obtaining an SEI Endorsement Under Massachusetts's RETELL Initiative
Jesse Bruhn et al.
Educational Evaluation and Policy Analysis, forthcoming
Abstract:
We apply a difference-in-difference design to measure the causal effect of a teacher obtaining an endorsement in Sheltered English Immersion under Massachusetts's Rethinking Equity in the Teaching of English Language Learners initiative on student's learning outcomes. More than 35,000 in-service public school teachers completed the semester-long course. We find no effect on English learners' (ELs) average test scores, but modest positive spillovers for students with disabilities and other non-EL students. Training benefited teachers recently hired by their district but had no effect on longer serving teachers.