Degrees of freedom
Daily Online Testing in Large Classes: Boosting College Performance while Reducing Achievement Gaps
James Pennebaker, Samuel Gosling & Jason Ferrell
PLoS ONE, November 2013
Abstract:
An in-class computer-based system, that included daily online testing, was introduced to two large university classes. We examined subsequent improvements in academic performance and reductions in the achievement gaps between lower- and upper-middle class students in academic performance. Students (N = 901) brought laptop computers to classes and took daily quizzes that provided immediate and personalized feedback. Student performance was compared with the same data for traditional classes taught previously by the same instructors (N = 935). Exam performance was approximately half a letter grade above previous semesters, based on comparisons of identical questions asked from earlier years. Students in the experimental classes performed better in other classes, both in the semester they took the course and in subsequent semester classes. The new system resulted in a 50% reduction in the achievement gap as measured by grades among students of different social classes. These findings suggest that frequent consequential quizzing should be used routinely in large lecture courses to improve performance in class and in other concurrent and subsequent courses.
----------------------
Evaluating Education Programs That Have Lotteried Admission and Selective Attrition
John Engberg et al.
Journal of Labor Economics, January 2014, Pages 27-63
Abstract:
We study the effectiveness of magnet programs in an urban district that ration excess demand by admission lotteries. Differential attrition arises since students who lose the lottery are more likely to pursue options outside the school district than students who win the lottery. When students leave the district, important outcome variables are often not observed. The treatment effects are not point-identified. We exploit known quantiles of the outcome distribution to construct informative bounds on treatment effects. We find that magnet programs improve behavioral outcomes but have no significant effect on achievement.
----------------------
Katherine Michelmore
Cornell Working Paper, November 2013
Abstract:
As of the early 2000s, the gap in college enrollment between children growing up in the highest income quartile and the lowest income quartile was over 50 percentage points (Bailey and Dynarski 2011). There is much debate in the literature about what role household income plays in producing this gap. A major impediment in studying this question is the lack of plausibly exogenous variation in income. The Earned Income Tax Credit (EITC) is one potential source of exogenous variation in household income that may increase educational attainment among low-income youth. Using variation in state EITC benefit generosity, I use a difference-in-difference framework to analyze how an increase in household income affects the educational attainment of children from low socioeconomic status households. Conservative estimates suggest that following an increase in the maximum EITC by $1,000, 18-23 year olds growing up in likely EITC- eligible households are 1 percentage point more likely to have ever enrolled in college and 0.3 percentage points more likely to complete a bachelor’s degree. These results are concentrated among individuals who were younger than 12 at the time of state EITC implementation, suggesting that the EITC increases educational attainment primarily by providing extra income to households with young children. I find no effect of EITC expansions on older children, for whom the EITC acts as a form of financial aid.
----------------------
Does it pay to pay teachers more? Evidence from Texas
Matthew Hendricks
Journal of Public Economics, January 2014, Pages 50–63
Abstract:
This study presents robust evidence on the relationship between teacher pay and turnover using detailed panel data from Texas. While controlling for changes in district and local labor market characteristics, I estimate an overall turnover elasticity of -1.4 and show that the effect is largest for inexperienced teachers, declines with experience, and disappears around 19 years of experience. Combining these results with what we know about the relationship between teacher value-added and experience, I show that paying teachers more improves student achievement through higher retention rates. The results also suggest that adopting a flat salary schedule may be a cheap way to improve student performance. I find no evidence that pay effects vary by the teacher’s gender or subject taught.
----------------------
Do College-Preparatory Programs Improve Long-Term Outcomes?
Kirabo Jackson
Economic Inquiry, January 2014, Pages 72–99
Abstract:
This paper presents an analysis of the longer-run effects of a college-preparatory program implemented in inner-city schools that provided teacher training in addition to payments to 11th- and 12th-grade students and their teachers for passing scores on Advanced Placement (AP) exams. Affected students passed more AP exams, were more likely to remain in college beyond their first and second years, and earned higher wages. Effects are particularly pronounced for Hispanic students who experienced a 2.5-percentage-point increase in college degree attainment and an 11% increase in earnings. While the study is based on nonexperimental variation, the results are robust across a variety of specifications, and most plausible sources of bias are ruled out. The results provide credible evidence that implementing high-quality college-preparatory programs in existing urban schools can improve the long-run educational and labor-market outcomes of disadvantaged youth.
----------------------
Joanne Carlisle, Ben Kelcey & Dan Berebitsky
American Educational Research Journal, December 2013, Pages 1360-1391
Abstract:
The purpose of this study was to examine third-grade teachers’ support for students’ vocabulary learning in high poverty schools characterized by underachievement in reading. We examined the prevalence and nature of discourse actions teachers used to support vocabulary learning in different literacy lessons (e.g., phonics); these actions varied in the cognitive demands placed on the students. Results showed that teachers rarely engaged students in cognitively challenging work on word meanings. Various lesson features and student and teacher characteristics were associated with teachers’ support for students’ vocabulary learning (e.g., teachers’ knowledge about reading). A major finding was that the extent of teachers’ support of their students’ vocabulary learning was significantly related to gains in reading comprehension across the year.
----------------------
Learning from the Test: Raising Selective College Enrollment by Providing Information
Sarena Goodman
Federal Reserve Working Paper, September 2013
Abstract:
In the last decade, five U.S. states adopted mandates requiring high school juniors to take a college entrance exam. In the two earliest-adopting states, nearly half of all students were induced into testing, and 40-45% of them earned scores high enough to qualify for selective schools. Selective college enrollment rose by 20% following implementation of the mandates, with no effect on overall attendance. I conclude that a large number of high-ability students appear to dramatically underestimate their candidacy for selective colleges. Policies aimed at reducing this information shortage are likely to increase human capital investment for a substantial number of students.
----------------------
The Potential of Urban Boarding Schools for the Poor: Evidence from SEED
Vilsa Curto & Roland Fryer
Journal of Labor Economics, January 2014, Pages 65-93
Abstract:
The SEED schools, which combine a “No Excuses” charter model with a 5-day-a-week boarding program, are America’s only urban public boarding schools for the poor. We provide the first causal estimate of the impact of attending SEED schools on academic achievement, with the goal of understanding whether changing a student’s environment is an effective strategy to increase achievement among the poor. Using admission lotteries, we show that attending a SEED school increases achievement by 0.211 standard deviation in reading and 0.229 standard deviation in math per year. However, subgroup analyses show that the effects may be driven by female students.
----------------------
Michael Woolley et al.
American Educational Research Journal, December 2013, Pages 1309-1335
Abstract:
Research and theory suggest that students learn more effectively when they perceive course content as relevant to their futures. The current research assessed the impact of CareerStart, a middle grades instructional strategy designed to advance the occupational relevance of what students are being taught in the core subjects — math, science, language arts, and social studies. CareerStart was introduced randomly in 7 of 14 middle schools in a diverse district with 3,295 students followed for 3 years. The analyses examined impact on end-of-grade test scores on math and reading exams. Findings confirm a significant treatment effect for math performance but no effect for reading performance.
----------------------
From Birth to School: Early Childhood Initiatives and Third-Grade Outcomes in North Carolina
Helen Ladd, Clara Muschkin & Kenneth Dodge
Journal of Policy Analysis and Management, Winter 2014, Pages 162–187
Abstract:
This study examines the community-wide effects of two statewide early childhood policy initiatives in North Carolina. One initiative provides funding to improve the quality of child care services at the county level for all children between the ages of 0 to 5, and the other provides funding for preschool slots for disadvantaged four-year-olds. Differences across counties in the timing of the rollout and in the magnitude of the state financial investments per child provide the variation in programs needed to estimate their effects on schooling outcomes in third grade. We find robust positive effects of each program on third-grade test scores in both reading and math. These effects can best be explained by a combination of direct benefits for participants and spillover benefits for others. Our preferred models suggest that the combined average effects on test scores of investments in both programs at 2009 funding levels are equivalent to two to four months of instruction in grade 3.
----------------------
Tom Ahn
Economics of Education Review, forthcoming
Abstract:
In 2006, North Carolina put in place high school exit standards requiring students to pass a series of high-stakes exams across several years. I use a regression discontinuity (RD) approach to analyze whether passing or failing one of these exams (Algebra I) impacts a student's decision between choosing a more rigorous college-preparatory math curriculum and an easier ‘career’ track math curriculum. I find a 5 percentage point gap in the probability of selecting the rigorous curriculum between 9th grade students who just passed and those who just failed the exam. RD results across two years (one year in which the graduation standards were not in place) suggests that the discontinuity arose due to fewer students opting into the college track as a result of the exam results.
----------------------
The Labor-Market Returns to Community College Degrees, Diplomas, and Certificates
Christopher Jepsen, Kenneth Troske & Paul Coomes
Journal of Labor Economics, January 2014, Pages 95-121
Abstract:
This article provides one of the first rigorous estimations of the labor-market returns to community college certificates and diplomas, as well as estimations of the returns to the more commonly studied associate’s degrees. Using administrative data from Kentucky, we estimate panel-data models that control for differences among students in precollege earnings and educational aspirations. Associate’s degrees and diplomas have quarterly earnings returns of nearly $2,400 for women and $1,500 for men, compared with much smaller returns for certificates. There is substantial heterogeneity in returns across fields of study. Degrees, diplomas, and — for women — certificates correspond with higher levels of employment.
----------------------
Jan Hughes, Oi-Man Kwok & Myung Hee Im
American Educational Research Journal, December 2013, Pages 1336-1359
Abstract:
The effect of retention in first grade (Year 1) on parents’ educational expectations was tested in a sample of 530 ethnically diverse and academically at risk children. Participants attended one of three school districts in Texas. Of the 530 children, 118 were retained in first grade. Retention had a negative effect on parent expectations in Year 2, which was maintained in Year 3. Year 2 parent expectations partially mediated the effect of retention in first grade on Year 3 reading and math achievement and child academic self-efficacy. All effects controlled for Year 1 measures of the outcome. Results were similar across gender, economic adversity, and ethnicity. Implications for minimizing the negative effect of retention on parents’ expectations are suggested.
----------------------
Paul Manna
Policy Studies Journal, November 2013, Pages 683–706
Abstract:
Are states with more centralized approaches to education governance more likely to have higher student achievement and lower achievement gaps between poor and nonpoor students? This article addresses that question by theorizing about the effects of political, administrative, and fiscal centralization on student outcomes. It tests competing hypotheses about the degree to which centralization across these three dimensions is associated with the promotion of academic excellence (higher achievement) and equity (narrower achievement gaps). The results demonstrate the virtue of studying academic performance through the lens of governance and more distal system-level variables rather than, as has been common in the literature, more narrow policy-oriented measures. The findings show that strong relationships exist between student outcomes and the degree of political centralization and administrative centralization in a state, yet there are no apparent associations with fiscal centralization. The results also illustrate that governing arrangements are not consistently related to the advancement of excellence and equity. In terms of administrative centralization, specifically, apparent trade-offs may exist.
----------------------
The effect of abolishing university tuition costs: Evidence from Ireland
Kevin Denny
Labour Economics, forthcoming
Abstract:
University tuition fees for undergraduates were abolished in Ireland in 1996. This paper examines the effect of this reform on the socio-economic gradient to determine whether the reform was successful in achieving its objective of promoting educational equality that is improving the chances of low socio-economic status (SES) students progressing to university. It finds that the reform clearly did not have that effect. The results are consistent with recent findings for the UK which show that the socio-economic gradient in second level attainment largely explain the socio-economic gradient in higher education participation.
----------------------
Better Luck Next Time: Learning Through Retaking
Verónica Frisancho et al.
NBER Working Paper, November 2013
Abstract:
In this paper we provide some evidence that repeat taking of competitive exams may reduce the impact of background disadvantages on educational outcomes. Using administrative data on the university entrance exam in Turkey we estimate cumulative learning between the first and the nth attempt while controlling for selection into retaking in terms of observed and unobserved characteristics. We find large learning gains measured in terms of improvements in the exam scores, especially among less advantaged students.
----------------------
Rajeev Darolia
Economics of Education Review, February 2014, Pages 38–50
Abstract:
A growing number of students are working while in college and to a greater extent. Using nationally representative data from the 1997 National Longitudinal Survey of Youth, I analyze the effect of working on grades and credit completion for undergraduate students in the United States. Strategies to identify the causal relationship between working and academic performance include student-level fixed effects to control for permanent, unobserved characteristics that may affect both work and study intensity, and system GMM models to account for potentially endogenous relationships between working and academic performance that vary over time. I examine the consequences of working for heterogeneous subgroups, with a particular focus on differences between full-time and part-time students. I find no evidence that students’ grades are harmed by marginal work hours, but that full-time students complete fewer credits per term when increasing work.
----------------------
Wayne Grove & Andrew Hussey
Labour Economics, forthcoming
Abstract:
A large literature has focused on estimating the returns to schooling and has typically done so by incorporating institutional heterogeneity in quality along merely one dimension (such as average SAT scores). Using longitudinal survey data of registrants for the GMAT exam and school level information from other sources, we create, in the context of graduate management education, multiple indices of school quality, and estimate the effect of these quality measures on multiple indicators of career success. In particular, we create quality measures of MBA programs based on: (1) institutional and curricular factors, (2) characteristics of the student body, and (3) characteristics of the faculty. We create aggregate quality indices by combining individual proxies using factor analysis. We also extend the literature by considering the effects of quality on both earnings and non-monetary outcomes, namely attainment of managerial goals relative to initial individual expectations, self-assessed skill gains, and various measures of job satisfaction. We include several unique individual control variables, and further control for unobserved heterogeneity through the use of instrumental variables and individual fixed effects. Results indicate that the quality of peers matters most for earnings, especially for OLS estimates. When individual fixed effects are included, estimates of quality premiums diminish somewhat, emphasizing the importance of controlling for selection into programs of varying quality. School quality is also an important predictor of several non-pecuniary outcomes.
----------------------
Tom Ahn
Journal of Human Capital, Fall 2013, Pages 230-273
Abstract:
Teacher effort, a critical component of education production, has been understudied in the literature because of measurement difficulties. I use a principal-agent model, North Carolina data, and the state’s accountability system that awards cash for school-level academic growth to distill effort from teacher absence and capture its effect. I find low effort at low and high probabilities of bonus receipt, high effort when the bonus outcome is in doubt, and free-ridership. Teachers respond to incentives, and effort affects achievement. Policy simulations with individual-level incentives eliminate free-rider effects but reduce effort by pushing teachers into the tails of the probability of bonus receipt distribution.
----------------------
Dominik Becker
Rationality and Society, November 2013, Pages 422-469
Abstract:
The objective of this paper is to integrate the idea of Pygmalion or self-fulfilling prophecy research into the subjective expected utility framework of inequality in educational opportunities. The theoretical section develops a formal model about the impact of teachers’ expectations on students’ educational transitions in sense of a self-fulfilling prophecy. In the empirical section, I test this model to predict both students’ educational success (in terms of high school graduations) and their university transitions. Analyses control for selection bias and unobserved heterogeneity by means of a bivariate probit model. I find that even net of both students’ performance and motivation, teachers’ expectations show significant effects on students’ educational success (Abitur), but not on their university transitions. This finding is stable against several robustness checks.
----------------------
Value-added models and the measurement of teacher productivity
Tim Sass, Anastasia Semykina & Douglas Harris
Economics of Education Review, February 2014, Pages 9–23
Abstract:
Research on teacher productivity, as well as recently developed accountability systems for teachers, relies on “value-added” models to estimate the impact of teachers on student performance. We consider six value-added models that encompass most commonly estimated specifications. We test many of the central assumptions required to derive each of the value-added models from an underlying structural cumulative achievement model and reject nearly all of them. While some of the six popular models produce similar estimates, other specifications yield estimates of teacher productivity and other key parameters that are considerably different.
----------------------
The Co-Twin Methodology and Returns to Schooling - Testing a Critical Assumption
Örjan Sandewall, David Cesarini & Magnus Johannesson
Labour Economics, January 2014, Pages 1–10
Abstract:
Twins-based estimates of the return to schooling have featured prominently in the economics of education literature. Their unbiasedness hinges critically on the assumption that within-pair variation in schooling is explained by factors unrelated to wage earning ability. This paper develops a framework for testing this assumption and shows, in a large sample of monozygotic twins, that the twins-based estimated return to schooling falls if adolescent IQ test scores are included in the wage equation. Using birth weight as an alternative proxy for ability yields qualitatively similar results. Our results thus cast doubt on the validity of twins-based estimates.
----------------------
Strategic Involuntary Teacher Transfers and Teacher Performance: Examining Equity and Efficiency
Jason Grissom, Susanna Loeb & Nathaniel Nakashima
Journal of Policy Analysis and Management, Winter 2014, Pages 112–140
Abstract:
Despite claims that school districts need flexibility in teacher assignment to allocate teachers more equitably across schools and improve district performance, the power to involuntarily transfer teachers across schools remains hotly contested. Little research has examined involuntary teacher transfer policies or their effects on schools, teachers, or students. This article uses administrative data from Miami-Dade County Public Schools to investigate the implementation and effects of the district's involuntary transfer policy, including which schools transferred and received teachers, which teachers were transferred, what kinds of teachers replaced them in their former schools, and how their performance — as measured by their work absences and value-added in math and reading — compared before and after the transfer. We find that, under the policy, principals in the lowest performing schools identified relatively low-performing teachers for transfer who, based on observable characteristics, would have been unlikely to leave on their own. Consistent with an equity improvement, we find that involuntarily transferred teachers were systematically moved to higher performing schools and generally were outperformed by the teachers who replaced them. Efficiency impacts are mixed; although transferred teachers had nearly two fewer absences per year in their new positions, transferred teachers continued to have low value-added in their new schools.
----------------------
Retained Students and Classmates’ Absences in Urban Schools
Michael Gottfried
American Educational Research Journal, December 2013, Pages 1392-1423
Abstract:
Research in grade retention has predominantly focused on the effect of this practice on the retained student. This study contributes to the limited body of research examining the effect of retained classmates on the outcomes of other students in the same classroom. Using a longitudinal data set of all elementary school students in a large urban school district, this study evaluates how the percentage of retained classmates affects other students’ absence patterns, both unexcused and excused. Focusing on absences as an outcome is key, as they signal educational disengagement and highly correlate with schooling and lifelong success. Based on quasi-experimental methods, the results indicate that a greater percentage of retained classmates increases other students’ absences. The effect is only present on unexcused absences, not excused absences, hence signaling an increase in disengagement in other students. Individual- and classroom-level moderating effects are evaluated, and policy implications for classroom assignment are discussed.
http://onlinelibrary.wiley.com/doi/10.1111/ecin.12040/abstract