Category Archives: Education

ADDITIONAL RESOURCES VERSUS ORGANIZATIONAL CHANGES IN EDUCATION

ADDITIONAL RESOURCES VERSUS ORGANIZATIONAL CHANGES IN EDUCATION: EXPERIMENTAL EVIDENCE FROM KENYA

E. Duflo, P. Dupas & M. Kremer (2009)

Principal Research Question and Key Result Is it enough to simply add more resources to education in order to increase measurable outcomes, or is there a need to also change the incentives that face education providers? Using a field experiment the authors can directly compare the outcomes of two such policies that occur in the same context. They find that there are significant effects on test scores for children from assigning them to a class with a short-term contract teacher as opposed to a regular civil service teacher. There are also significant effects when both types of teacher are subject to supervision by a trained committee of parents. Overall the results only persist for kids in contract teacher classes with parental supervision.

 

Theory Throwing more resources (teachers) at educational facilities may not actually increase test scores. This could be for a number of reasons for example insufficient learning materials, poor incentive structures, other variables (such as health) that prevent learning, or simply that class sizes even when significantly reduced are still too large for the extra resources to have any meaningful impact. It is important to assess the practical relevance of such factors as they will have implications for policy. The factor being examined here is incentives. They want to examine in effect whether reforming school systems might be more effective than simply increasing resources.

 

Motivation Whilst access to education has increased hugely in the developing world, it has become clear that this has not necessarily translated into increased competency in basic skills. There is evidence that increasing participation without changing methodology or environment (deworming, school meals etc.) may have little effect.  Some countries prefer to hire less experienced, short contract teachers on lower pay than government teachers as they are thought to be easier to motivate. However, field experiments have not been designed to evaluate both the effect of increasing resources, and changing incentives. Until now that is…
Data
  • The data are from an experimental program in Western Kenya. 140 out of 210 schools were randomly selected. Of these 70 became part of the Extra Teacher Program (ETP) and 70 were control schools. The ETP schools were given funding to hire an extra contract teacher. Whilst contract teachers are already used in Kenya, as they are self-funded by parent contributions there is a chance that the presence of a contract teacher is correlated with variables of how important education is in the community. The use therefore of experimental data ensures that the treatment is distributed independently of other characteristics that may affect exam results. This teacher would teach a randomly assigned 50% of year 1 students for a whole year, and then follow them into the second year. The regular civil service teacher had the remainder of the class. For a further subset of 35 schools (of the ETP schools) a school committee was trained in how to monitor evaluation of the contract teacher and were encouraged to hold a 1 year review to see if the contract would be renewed.
  • There were very few requests to move classes, so randomization should have been largely preserved.
  • The outcome variable was scores in standard tests

 

Strategy
  • OLS regression with dummies for whether students were in ETP schools, interacted with dummies for civil/contract teacher further interacted with dummies for whether there was a committee overseeing.
  • As well as test scores as dependent variable, they also use attendance measures to evaluate teacher incentives in terms of effort.

 

Results Teacher Effort

  • Civil service teachers in ETP schools 15% less likely to be in school relative to comparison schools. They may have taken advantage of extra teacher to work less. Contract teachers in ETP schools with no committee 30% more likely to be in school than civil service counterparts in same ETP schools indicating strong incentives to perform due to contract. The presence of a committee did not appear to change the attendance of the contract teachers. The civil service teacher with committee was 9% more likely to be in class perhaps because the incentive facing the contract teacher meant he was not incentivized to do the work of the civil service teacher.

Scores

  • Students in the reduced class sizes showed no statistically significant improvement in scores.
  • Students in contract ETP schools score 0.21 SDs higher than colleagues in civil service classes and 0.24 than non-ETP schools. This is robust to controls for teacher demographics. This could be either because the committees are better at picking teachers, or because they face better incentives.
  • Students in committee schools in the contract class did not do better than non-committee contract kids. However students in civil committee class did do better than the comparison schools, so the committee seems to have had an effect on the civil service teachers.
  • The effects all disappear in post program evaluation except for the committee schools.

 

Robustness  
Problems
  • There was a roughly 20% attrition rate in tests which is worrying for the results, if those students were the less able then the results will be upward biased. It is not clear why a subsample of only 60 children were given tests and not every child.
  • It is not totally clear that incentives are driving the different results. The committees could be better at picking teachers. Additionally the contract teacher remained with the class for two years, whereas the civil service teachers chop and change; thus it could be the continued presence of one teacher that is making the difference.
  • Whilst the findings on class size are interesting they can say nothing about the effect of reducing class size from say 80 to 10.
  • It is not clear that the presence of committees improves outcomes through monitoring rather than general awareness about education in the community.
  • External validity is hard to assert, as the Kenyan system is very specific in terms of the process of hiring teachers etc. Additionally, we cannot assume that making all teachers contract workers would have beneficial effects as the positive increase in effort by contract teachers could be being driven by the fact that they want to become civil service teachers.

 

Implications
  • The fact that smaller class sizes did not improve performance indicates that simply reducing class size is not efficient (perhaps because reducing from 80 kids to 40 is not enough of a difference, or alternately because reduction in class size also reduced teacher effort).
  • The fact that only the committee schools saw persistent benefits suggest that whatever it is about having contract teachers is only made permanent by the presence of committees.

 

 

Advertisements

SCHOOLING AND LABOR MARKET CONSEQUENCES OF SCHOOL CONSTRUCTION IN INDONESIA

SCHOOLING AND LABOR MARKET CONSEQUENCES OF SCHOOL CONSTRUCTION IN INDONESIA: EVIDENCE FROM AN UNUSUAL POLICY EXPERIMENT

E. Duflo

American Economic Review, Sept 2002

Principal Research Question and Key Result Can investments in infrastructure increase educational attainment and does this then have labour market implications? The estimates suggest that in the context of Indonesia, each new school per 1000 children was associated with an increase of 0.12 to 0.19 in years of education resulting in 15. To 2.7% increase in earnings for those fully exposed to the program.

 

Theory If education increases the productive capacity of an individual, then having more education will lead to increased earnings all else constant as the wage rate equals the marginal product of capital.  There are important assumptions underlying this theory that may not always be satisfied in practice. For example, firms are assumed to have no wage setting power and to be able to directly observe levels of human capital. Additionally there can be no externalities, and in particular there can be no offsetting equilibrium effects if we are to find statistically significant associations.

 

Motivation There is a large body of evidence that suggest that there are significant returns to education, and that these returns tend to be higher in developing countries where the marginal effect of further education is higher. However cross country regressions of wage on education are difficult due to the incompatibility of educational quality and hence data, as well as problems of controlling for important unobserved characteristics such as community, and ability. This paper uses a natural experiment to look for exogenous changes in the availability of schooling in regions of Indonesia, and how this changed the amount of education attained, and consequently how that impacted the wages of those affected.

 

Data
  • In 1973 Oil revenues in Indonesia were mobilized for the INEPRES project to provide increased educational facilities with a focus on provision in areas where enrollment rates were historically low. c.62,000 schools were constructed. At the same time the government recruited and trained a suitable number of teachers.
  • Data is a cross section from the 1995 census in Indonesia, of men born between 1950 and 1972. This data was matched with regional census data to match where these men were schooled, and how many INEPRES schools were constructed in that area. The date of birth and the region of birth jointly determine how much an individual was exposed to the program. E.g. children over 12 in 1974 did not benefit as they had already left primary education by the time the schools came into being. For younger children, exposure to the programme is a decreasing function of their age i.e. the older they are, the less exposed they were. Region of birth also denotes intensity of exposure as some regions received more schools than others.
  • The schools measure is schools constructed per 1000 children.

 

Strategy Effect on Education

  • First, basic DID using means is calculated in the form of summary stats.
  • This is then done in a regression analysis including an interaction term between the intensity in region of birth, and whether the individual is old enough to have befitted (i.e. falls in the treatment group). The control group is therefore those who are not young enough to have benefitted from the policy. She controls for birth year fixed effects, region specific fixed effects and a vector of region specific observables.

Wages

  • Same idea as above basically

IV Strategy

  • If program had no effect on wages other than through education then the program can be used as an instrument. Indeed she shows  using different subsamples that there is no wage effect in regions that did not see an education effect which indicates that the increase in wages in the applicable regions is being driven by the increase in years of education. This makes the exogeneity restriction very plausible.
  • Thus she uses the interactions between the age in 1974 and the program intensity in region of birth as instruments for changes in educational achievement.

 

Results Education

  • Summary stats reveal that those kids in areas of high program intensity had lower years of education and wages which reflects the fact that schools were targeted at educationally poor regions primarily. The difference between the high/low areas differenced by the differences in received/did-not-receive treatment (by age) is the casual effect. That is, one extra school per 1000 children increases education by 0.13 years.
  • The DID regression gives results of 0.12 years for the whole sample, and 0.2 years for the subsample of those employed in 1995.

Wages

  • The DID specification indicates that increasing one school per thousand increases earnings of those fully exposed to the program by 1.6 to 4%.

IV

  • The 2SLS estimates increase upon the OLS estimates although they are statistically equal, and they are robust to the inclusion of the controls as outlined above. They are higher in sparsely populated regions which indicates that a local average treatment effect (LATE) is being estimated (see problems below).  The wage earners are found to earn c.10% more if there was 1 extra school per 1000 in their area.

 

Robustness
  • A placebo difference in difference using older age groups from 1974 returns results very close to 0 which lends support to the assumptions underlying the D-I-D.
  • Controls for other INEPRES programs in the region as well as initial enrollment rates which increases the estimates somewhat indicating the main results are not being downward biased by omitted variables.
  • A further control [placebo] experiment in the DID regression is undertaken that returns very tiny results.
  • Interacts a dummy indicating being of a certain age with the schools constructed per region to see effect on all age ranges. The coefficients only start increasing at age 12 and all coefficients are significant from age 0 to 8. This indicates that those who were not old enough to benefit from INEPRES in fact did not which lends support the identification strategy.
  • Presents results for subsamples of regions that show that the program had no effect in densely populated areas. In sparsely populated areas each new school significantly reduced the distance to education.

 

Problems
  • Whilst it is suggested that the results are conservative as the program was targeted at areas that had low enrolment rates the data suggest otherwise. When log(1-enrolment rate) is regressed on log(INEPRES Schools) the coefficient is 0.12. If the program had been targeted as stated this coefficient should be close to 1. This means that there could be bias in the results if the program was actually targeted at areas where the returns to education were already likely to be higher based on wealth, or some other unobserved characteristics.
  • If employment opportunities in different regions were changing such that people’s attitudes to education were changing (but to different degrees) then we have time varying and region specific variation, that could lead people to exploit the program in different ways. This correlation would confound the DID results which rely on the fact that there is no region specific time varying variable that is correlated with the program.
  • Micro level data cannot properly account for the externalities generated by such a program. For example there were almost certainly spillover benefits on fertility, mortality etc. which are not accounted for, and therefore it is possible that the wage estimates are somewhat underestimated.
  • There is likely to be bias in the wage results as data are only collected for those employed, not including the self or unemployed. If family pressures to be employed meant that children of such families were likely to receive more education then we have an omitted variable problem that could upward bias the results. This should be partially solved by the IV strategy which is admittedly strong.
  • The IV restrictions are plausibly upheld. The instruments a strong in the first stage (relevance) and plausibly exogenous. However, the resulting estimate is only a LATE for those capable of being affected by the IV. In this case that will be those for whom distance to school is a factor that colours their decision as to whether to attend or not.
  • As with any such study there are external validity concerns. In particular the huge drive for education in Indonesia at that time made the program ripe for success, a success which is by no means guarantees in other contexts. There could also be long run general equilibrium effects which would confound any short run benefits. Additionally since the IV indicates that the program affects those capable of being affected by the IV (i.e. those for whom distance to school is an important factor when deciding whether to attend or not), this indicates that such a program will not work in all settings (e.g. urban settings). The program was particularly well designed, and care was taken to ensure that teacher quality was not affected. In other settings it is not clear that teacher quality could be maintained such that the benefits of expansion of schooling are not offset by reductions in quality.

 

Implications
  • The IV results indicate that estimations of returns to education in developing countries are not biased upwards due to unobserved community effects as some have argued.
  • Increasing access to education can have benefits, but generally only in regions where distance to nearest educational facility is an issue. In the densely populated regions in this study where the program would only have affected class size, there were no observable effects.  For such urban regions other tactics will need to be used in order to increase participation rates in education.