Tested well
Reducing socioeconomic disparities in the STEM pipeline through student emotion regulation
Christopher Rozek et al.
Proceedings of the National Academy of Sciences, forthcoming
Abstract:
Educational attainment is one lever that can increase opportunity for economically disadvantaged families — especially in Science, Technology, Engineering, and Math (STEM). Unfortunately, students from lower-income backgrounds often perform poorly and fail high school STEM courses, which are a necessary step in pursuing fast-growing and lucrative STEM careers, graduating high school, and matriculating to college. We reasoned that, because high school STEM courses often use high-stakes tests to gauge performance, and such tests can be especially stressful for lower-income students, interventions that help students regulate their negative emotions during tests should reduce the achievement gap between higher- and lower-income students. In a large-scale (n = 1,175) field experiment conducted in ninth grade science classrooms, students were asked to complete a control exercise, or they were given the opportunity to complete an exercise to help them regulate their worries and reinterpret their anxious arousal before their tests. We found significant benefits of emotion regulation activities for lower-income students in terms of their science examination scores, science course passing rate, and students’ attitudes toward examination stress, suggesting that students’ emotions are one factor that impacts performance. For example, 39% of lower-income students failed the course in the control group compared with only 18% of students failing the course if they participated in the emotion regulation interventions — a reduction in course failure rate by half. Our work underscores the crucial importance of targeting students’ emotions during impactful points in their academic trajectories for improving STEM preparedness and enhancing overall academic success.
Measuring opportunity in U.S. higher education
Caroline Hoxby & Sarah Turner
Stanford Working Paper, January 2019
Abstract:
In identifying whether universities provide opportunities for low-income students, there is a measurement challenge: different institutions face students with different incomes and preparation. We show how a hypothetical university's "relevant pool" — the students from whom it could plausibly draw — affects popular measures: the Pell share, Bottom Quintile share, and Intergenerational Mobility. Using a proof by contradiction, we demonstrate that universities ranked highly on the popular measures can actually serve disproportionately few low-income students. We also show the reverse: universities slated for penalties on the popular measures can actually serve disproportionately many low-income students. Furthermore, the Intergenerational Mobility measure penalizes universities that face relatively equal income distributions, which are probably good for low-income students, and rewards universities that face very unequal income distributions. In short, by confounding differences in university effort with differences in circumstances, the popular measures could distort university decision making and produce unintended consequences. We demonstrate that, with well-thought-out data analysis, it is possible to create benchmarks that actually measure what they are intended to measure. In particular, we present a measure that overcomes the deficiencies of the popular measures and is informative about all, not just low-income, students.
Market power and price discrimination in the US market for higher education
Dennis Epple et al.
RAND Journal of Economics, forthcoming
Abstract:
We estimate an equilibrium model of private and state college competition that generates realistic pricing patterns for private colleges using a large national data set from the National Postsecondary Student Aid Study (NPSAS). Our analysis distinguishes between tuition variation that reflects efficient pricing to students who generate beneficial peer externalities and variation that reflects arguably inefficient exercise of market power. Our findings indicate substantial exercise of market power and, importantly, sizable variation in this power along the college quality hierarchy and among students with different characteristics. Finally, we conduct policy analysis to examine the consequences of increased availability of quality public colleges in a state.
Is the Pen Mightier Than the Keyboard? The Effect of Online Testing on Measured Student Achievement
Ben Backes & James Cowan
Economics of Education Review, February 2019, Pages 89-103
Abstract:
Nearly two dozen states now administer online exams to deliver testing to K-12 students. These tests have real consequences: their results feed into accountability systems, which have been used for more than a decade to hold schools and districts accountable for their students’ learning. We examine the rollout of computer-based testing in Massachusetts over 2 years to investigate test mode effects. Crucial to the study design is the state administering the same exam (PARCC) in online and offline formats each year during the transitional period. We find an online test penalty of about 0.10 standard deviations in math and 0.25 standard deviations in English language arts (ELA), which partially but not fully fades out in the second year of online testing.
Cyber Charter Schools and Growing Resource Inequality among Public Districts: Geospatial Patterns and Consequences of a Statewide Choice Policy in Pennsylvania, 2002–2014
Bryan Mann & David Baker
American Journal of Education, February 2019, Pages 147-171
Abstract:
An analysis from 2002 to 2014, aligning media reporting of the effectiveness of the fully online K–12 cyber charter school model with data on enrollment flows to cyber charter schools and expenditure and demographic indicators across all 500 residential public school districts in Pennsylvania, finds a three-part geospatial-social process. Initial high-tech cachet surrounding the option stimulated statewide spread in enrollments, but over time growth in student flows became more pronounced among disadvantaged, lower tax-base public school districts. As mass media coverage shifted to a research-substantiated narrative of the model’s academic ineffectiveness, cyber charter enrollments declined first in districts with higher parent educational attainment and then intensified. With the large movement of students, the mean amount of public funds transferred from residential districts in 2014 was about $800,000 (standard deviation about $3,100,000). With dubious academic benefits, districts with the lowest tax base lost significant revenue to cyber charter providers.
The Impact of Replacing Principals on Student Achievement in DC Public Schools
Elias Walsh & Dallas Dotter
Education Finance and Policy, forthcoming
Abstract:
The 2007 Public Education Reform Amendment Act (PERAA), led to 39 percent of the principals in DC Public Schools (DCPS) being dismissed before the start of the 2008–2009 school year, and additional principal exits over the next few years. We measure the impact of replacing these principals on school-wide student achievement by measuring the changes in achievement that occurred when principals were replaced, and comparing these changes to achievement in comparison schools within DCPS that kept the same principal. We find that after a new principal's third year in a school, average school-wide achievement increased by 4 percentile points (0.09 standard deviations) compared to how students in the school would have achieved had DCPS not replaced the previous principal. For students in grades 6 to 8, the gains were larger and statistically significant in both math and reading.
Which New Yorkers vote with their wallets? The impact of teacher quality data on household sorting, and residential and school demographics
Elizabeth Rivera Rodas
Economics of Education Review, February 2019, Pages 104-121
Abstract:
This paper examines the relationship between the February 2012 New York City teacher quality data release, the racial and ethnic composition of elementary and middle public schools and their neighborhoods, and housing prices. A unique dataset that links teacher quality to school characteristics, housing prices and characteristics, and Census data is used to estimate a difference-in-difference model to estimate the impact of the teacher quality release on housing prices. These results are then used to estimate the impact of the data release on residential and school demographics. The results in this study provide the first evidence of the effects of teacher quality measures on the housing market in New York City and the demographic shifts in residential and school mobility patterns as a result. The data suggests that the housing market responds significantly to the new information that was provided by the release of the teacher quality information, even when taking into consideration the school grades and other variables that may influence teacher quality measures. The magnitude of how much housing prices increase is greatly impacted by certain neighborhood demographics and the results show that Hispanic neighborhoods with a high proportion of free and reduced price lunch students are among the neighborhoods that had the highest increase in housing prices due to the teacher quality release. Not only that, but the areas that had the highest increase in housing prices due to the teacher quality release have experienced increases in the proportion of white students and the neighborhoods are becoming more racially diverse.
Selective Retention Bonuses for Highly Effective Teachers in High Poverty Schools: Evidence from Tennessee
Walker Swain, Luis Rodriguez & Matthew Springer
Economics of Education Review, February 2019, Pages 148-160
Abstract:
Research has well established that racially isolated schools with high concentrations of low-income students disproportionately struggle to recruit and retain highly effective teachers, limiting disadvantaged students’ opportunities to be exposed to high-quality instruction and driving institutional and community instability. This study estimates the effect of selective retention bonuses (SRB) for highly effective teachers on low-performing, high poverty schools’ ability to elevate student performance by increasing access to effective instruction. The theory of action behind the bonus program is simple: SRBs result in greater numbers of highly effective teachers at participating schools, who subsequently drive larger student gains than the teachers who would otherwise fill their positions. To examine whether students in high poverty schools benefit from retention of highly effective teachers, we use differences in eligibility for schools to offer bonuses and the discrete timing of the program in a matched sample, difference-in-differences framework. Results indicate that schools who offered SRBs saw greater test score gains in subsequent years, especially on state reading exams.
The Effects of Closing Urban Schools on Students’ Academic and Behavioral Outcomes: Evidence from Philadelphia
Matthew Steinberg & John MacDonald
Economics of Education Review, forthcoming
Abstract:
Urban districts throughout the country are increasingly closing schools in response to declining enrollment and academic underperformance. We estimate the impact of public school closings in Philadelphia on student achievement and behavioral outcomes. While school closures had no effect on the average achievement of displaced students, achievement increased among displaced students attending higher-performing schools following closure. The achievement of students attending receiving-schools, however, was negatively affected by the receipt of displaced students. School absences increased significantly for displaced students following closure. We also find that the achievement of displaced and receiving-school students declined as the fraction of displaced students attending a receiving-school increased, and displaced students missed more days of school and received more suspension days the farther they traveled to their new school following closure. These findings suggest that the academic and behavioral consequences of closing urban schools depend on the school settings displaced and receiving-school students experience in the wake of closures.
Testing the Effectiveness of “Managing for Results”: Evidence from an Education Policy Innovation in New York City
Weijie Wang & Ryan Yeung
Journal of Public Administration Research and Theory, January 2019, Pages 84–100
Abstract:
“Managing for results” (MFR) is a performance management system that decentralizes authority to managers in exchange for greater accountability in performance. Although MFR makes much theoretical sense, the evidence of the effectiveness of MFR has not been as conclusive. In this study, we use panel data methods to examine the impact of a particular MFR reform in New York City, the Empowerment Zone (EZ), which focused on providing city public school principals greater autonomy to improve school outcomes. In addition, we use objective measures of both performance management and organizational performance. Our differences-in-differences estimates suggest that the EZ had a significant and positive effect on school performance as measured by proficiency rates in standardized mathematics exams, overall performance, and Regents diploma graduation rates, though the effects were not immediately apparent.
The signal quality of grades across academic fields
James Thomas
Journal of Applied Econometrics, forthcoming
Abstract:
I use transcript data from Duke University and a correlated learning model to measure the signal quality of grades across academic fields. I find science, engineering, and economics grades are significantly more informative than humanities and social science grades. The correlated learning structure allows grades in one field to signal abilities in all fields. This sometimes generates information spillovers so powerful that science, engineering, and economics grades inform humanities and social science beliefs more than humanities and social science grades. I show grade compression reduces signal quality but cannot explain the differences in signal quality across academic fields.
Curriculum requirements and subsequent civic engagement: Is there a difference between ‘forced’ and ‘free’ community service?
Ailsa Henderson, Steven Brown & Mark Pancer
British Journal of Sociology, forthcoming
Abstract:
Despite figures showing the growth of mandatory community service programmes, there is mixed empirical evidence of their effectiveness. This paper addresses the relationship of mandated community service to one of its purported aims: subsequent volunteerism. It compares current volunteerism among four university student cohorts: those doing no service in secondary school, those volunteering with no requirement, those volunteering both before and after the introduction of a requirement, and those introduced to service through a requirement. The analysis indicates that (1) students who were introduced to service through a mandated programme exhibit current levels of engagement no greater than non‐volunteers; (2) this relationship stems largely from the different service experiences of our four cohorts and relates to the fact that service satisfaction and duration, as well as background variables account for current levels of civic engagement. The findings suggest that mandatory service programmes might well be failing the very population they seek to target, particularly in weaker, less structured programmes.
An Extra Year to Learn English? Early Grade Retention and the Human Capital Development of English Learners
David Figlio & Umut Özek
NBER Working Paper, January 2019
Abstract:
In this study, we use microdata from 12 Florida county-level school districts and a regression discontinuity design to examine the effects of early grade retention on the short-, medium-, and long-term outcomes of English learners. We find that retention in the third-grade substantially improves the English skills of these students, reducing the time to proficiency by half and decreasing the likelihood of taking a remedial English course in middle school by one-third. Grade retention also roughly doubles the likelihood of taking an advanced course in math and science in middle school, and more than triples the likelihood of taking college credit-bearing courses in high school for English learners. We also find that these benefits are larger for foreign born students, students with higher latent human capital in third grade as proxied by their math scores, students whose first language is Spanish, and students in lower-poverty elementary schools.
The Four-day School Week and Parental Labor Supply
Jason Ward
University of Illinois Working Paper, October 2018
Abstract:
In this study I estimate the effect of adoption of the four-day school week, a permanent reduction in annual days of schooling, on parental employment. Using a difference-in-differences empirical model, I estimate causal effects of the four-day school week on parental employment and earnings across four states — CO, ID, OK, OR — with large increases in the use of the policy in the last decade. Estimates indicate that, among mothers with children all between ages 5 and 13, increasing four-day week enrollment from zero to 25 percent of an area’s students causes an 11% decrease in employment, a 12% decrease in annual hours worked, and evidence of an associated decrease in maternal earnings. In contrast to these estimates, I find that the policy led to an 18% increase in annual weeks worked by single mothers. The labor supply of married fathers was not significantly affected by adoption of the four-day school week.
Comparing the Effects of Analysis-of-Practice and Content-Based Professional Development on Teacher and Student Outcomes in Science
Kathleen Roth et al.
American Educational Research Journal, forthcoming
Abstract:
This study tests the influence of a video-based, analysis-of-practice professional development (PD) program on upper-elementary teachers’ science content knowledge, pedagogical content knowledge, and teaching practice and on their students’ achievement. Using a cluster-randomized experimental design, the study compares the outcomes for teachers in an analysis-of-practice program with those of teachers in a content-deepening program. Mediational analyses explore the relationship between teacher outcomes and student learning. In comparison with the content-deepening PD program, the analysis-of-practice PD program significantly impacted teachers’ knowledge and practice. Mediation analyses revealed a strong relationship between teaching practice and student learning. The study advances the field beyond the currently accepted consensus model of effective PD toward an empirically tested model.