Category Archives: Incentives

TEACHER PERFORMANCE PAY

TEACHER PERFORMANCE PAY: EXPERIMENTAL EVIDENCE FROM INDIA

K. Muralidharan & V. Sundararaman

NBER Working Paper No. 15323 (2009) 

Principal Research Question and Key Result Does performance based pay for teachers improve student performance? In an experiment in India, students who had teachers subject to performance incentives performed between 0.28 and 0.16 standard deviations better than those in comparison schools.

 

Theory It is not clear that monetary incentives will always align the preferences of the principal and the agent. In some cases they may crowd out intrinsic motivation leading to inferior outcomes. Psychological literature indicates that if incentives are perceived by workers as a means of exercising control they will tend to reduce motivation, whereas if they are seen as reinforcing the norms of professional behaviour then this can enhance intrinsic motivation.

Additionally whether incentives are at a class or school level will be of importance. This is because in the school results model (how schools perform on aggregate) there will be incentives to free ride. This is not the case if incentives operate at the individual teacher level. The problem may be reduced in small schools where teachers are better able to monitor each other’s efforts at a relatively low cost.

 

Motivation There are generally two lines of thought regarding how to improve school quality. The first argues that increase inputs are needed. This might include text books, extra teachers, better facilities etc. The other option is to implement incentive based policies to improve existing infrastructure, and perhaps improve individual selection into the teaching sector.

 

Experiment/Data The experiment took place in Andhra Pradesh which has been part of the Education for All campaign in India, but sees absence rates of around 25% and low student level outcomes. There were 100 control schools, 100 group bonus schools (all teachers received same bonus based on average performance of the school), and 100 individual bonus schools (incentive based on performance of students of a particular teacher). Focussing on average scores ensures that teachers do not just focus on getting those kids near the threshold up, thus excluding less able children. No student is likely to be wholly excluded given the focus on averages. Additionally, there was no incentive to cheat, as children that took the baseline test, but not the end of year test were assigned a grade of 0 which would reduce the average of the class.

A test was administered at the start of the programme/school year which covered material from the previous school year. Then at the end of the programme a similar test was given, with similar content, and then a further test which examined the material from the current school year (that they have just completed). The same procedure was done at the end of the second year. Having overlap in the exams means that day specific measurement error is reduced. The tests included mechanical and conceptual questions.

 

Specification

Tijkm(Yn) = α + β[Tijkn(Y0)] + γ(Incentives) + δ(Zm) +εk + εjk+ εijk 

T is the test score, where i j k m indicate student, grade, school, and mandal (region) respectively. Y0 indicates baseline tests, and Yn indicates the end of year tests. The baseline results are included to improve efficiency by controlling for autocorrelation between the test scores across multiple years. Zm is a vector of mandal dummies (fixed effects) and standard errors are clustered at the school level.  Delta is the coefficient of interest.

 

Results Students in incentive schools scored 0.15 standard deviations higher than the comparison schools at the end of the first year and 0.22 at the end of the second. This averages across maths and languages (although disaggregated the effect for maths was higher). NB Whilst the year 1 to year 0 comparison is valid, and the year 2 to year 0 is valid as well, technically the comparison of year 2 to year 1 (column three of table II) is not experimental estimation as year 1 results are already post experimental outcomes.

They examine for heterogeneous treatment effects by including relevant variables and interacting them with the INCENTIVE dummy., and find that none of them (no. students/school proximity/school infrastructure/parental literacy/caste/sex/) see differential effects from the programme indicating that the benefits are widely based and not conditional on a set of predetermined characteristics. The only interaction for which there is a mall effect, is household affluence. These then are broad-based gains. As the variance of test scores in individual school went up, this might indicate that teachers responded differently, as it seems there were no barriers for all types of children and schools to benefit from the programme (no heterogeneous effects).

When they include teacher characteristics such as education and training, the see no significant effect, but when they interact these measures with the INCENTIVES dummy they are positive and significant, indicating that high quality teachers alone may not sufficient if they are not incentivized to use their skills to maximum effect.

Teachers that were paid more responded less, presumably as they are more experienced (less conducive to change) and the bonus represented a smaller fraction of their total income.

Happily the results were similar for both the conceptual and mechanical questions, indicating that real learning is taking place, rather than just rote reproduction. Additionally students in incentive schools performed better in non-incentive subjects like science. NB it is possible that teachers diverted energy from teaching non-incentive subjects to teaching incentive subjects for obvious reasons. This result does not disprove that, but it says that in the context studied improvement in teaching in certain subjects can have spillovers into other subjects.

Both group and individual incentives were effective. However, schools size was typically between 3 and 5 teachers, so probably too small to separate effects. Group incentives may not work in larger schools.

Interestingly there was no increase in teacher attendance. In interviews after the experiment teachers said they gave extra classes, and were more likely to have set and graded homework.

Robustness
  • They tested the equality of observable characteristics across the control/treatment groups and could not reject the null that they were equal indicating that randomization was successful. Additionally, all schools (including control) were given the same information and monitoring, to ensure that differences in the treatment were not merely due to the Hawthorne effect.
  • There was no significant difference in attrition, and the average teacher turnover was the same across schools indicating that there was no sorting of teachers into the incentive schools.
  • They control for school and household characteristics which does not change the estimated value of delta, thus confirming the randomization.
  • A parallel study provided schools with money to purchase extra inputs, and the incentive levels were set such that they came to a similar amount of funds as the input schools. The input schools did see a positive effect, but to a much lesser degree. Additionally, the incentive programme actually ended up costing much less.

 

Interpretation Programme design is extremely important. In particular how the teachers feel about incentives may affect performance, and the size of schools may mean that benefits from group incentives are not seen due to the ability of teachers to freeride on the back of their colleagues.

Given that the study was compared with an input study in the same region and found improved results, it would seem that funding should be allocated to incentive schemes rather than input schemes. In addition, rather than raiding pay by 3% each year, that 3% could be allocated to the bonus scheme, and thus it would actually cost virtually nothing to run (other than the administering of the tests etc.). However, a mix of policies is probably a good idea, especially since the incentive scheme did not improve absence rates. As other literature has shown improving infrastructure etc. can lead teachers to be present more, so this could be one option for the input schemes.

 

Advertisements

INCENTIVES – Notes

NOTES ON INCENTIVES 

Market failures associated with public goods mean that education/health etc. is generally provided by the state. Yet public sector workers need to be incentivized to do good work. There are lingering questions about how to prevent absenteeism, inspire effort etc. The danger is that without correctly aligning incentives investment in infrastructure may be useless. Essentially it is a principal-agent problem; the effort of government workers is only imperfectly observed by proxy. Incentives therefore need to be based upon what the agent cares about in order to align with the principal (i.e. getting money, or avoiding censure).

                                                                                                            

Missing in Action: Teacher and Health Worker Absence in Developing Countries N. Chaudhury, J. Hammer, M. Kremer, K. Muralidharan & F.H. Rogers (Journal of Economic Perspectives 2006)

 

In a Nutshell

This paper formulates the problem nicely. In a cross country survey they find 19% of teachers and 35% of health workers are absent, and these rates tend to be higher in poorer areas. Higher ranking workers are more likely to be absent. Absence is not strongly affected by wages, but is affected by physical infrastructure which indicates that they are unlikely to be fired for absence, but their decisions as to attend are affected by the physical conditions under which they work.  Additionally, the survey reported that absence rates are not driven by the same individuals always being absent indicating that this is not a case of bas apples, but a system wide problem.

There are certain structural issues with service provision in developing countries. Firstly the system is often highly centralized which does not allow for much local monitoring. Salaries are determined by seniority which leaves little scope for performance based pay. Wages are not typically responsive to local labour market conditions, and are compressed relative to the private sector. Disciplinary actions for absence are often missing. Additionally a variety of informal service providers have arisen and they are often operated by the same government officials i.e. teachers offering tutoring, and health workers having private practice.

Correlation analysis across countries gives some indication of what is driving absence. In particular, status and poor infrastructure seem to be correlated with higher absence. The literacy rate of parents is associated with lower absence (perhaps through better monitoring, demand etc.). Having been inspected recently leads to lower absence.  Higher salaries decrease absence.

This is suggestive of the following policy priorities: increase local control, improve civil service sector, upgrade facilities, performance related pay. Not all of these are politically viable as the civil service tends to be a well-organized interest group. Additionally, as the poor and those receiving services are a disparate group they may suffer collective action problems in achieving better provision.

                                                                                                              

 

Addressing Absence A. Banerjee & E. Duflo

In a Nutshell

This paper looks at evidence from randomized control trials that seek to provide incentives to service providers with a variety of mechanisms:

  1. External Control: external control is when someone who has no stake in the performance of the service being delivered has the job of monitoring performance and basing reward/punishment incentives on the monitored performance. This could be a direct measure (such as recording presence/absence) or a more indirect measure such as test scores. In Dulfo & Hanna treatment schools were given cameras and teachers had to take photos at beginning end of the day on which the date was imprinted. They were rewarded for being present more than 21 days in a month, and penalized if not. This resulted in increased teacher attendance – absence dropped from 36% to 18% in the treatment schools. This did not necessarily indicate that the teachers were actually teaching. The benefit of this programme is that monitoring was impersonal; there was no scope for head teachers etc. to cheat the system. In the long run however, in a non-experimental setting, even with impersonal monitoring such as this, head teachers have to be willing to apply the reward/punishment structure, which is not a given.
  2. Rewards for performance rather than presence: Prizes were awarded for good exam results. The treatment group saw an increase in results, but no effect on absenteeism. Rather teachers held more preparation sessions. This indicates that such programmes will not be effective to increase attendance, although they may be useful in conjunction with other measures as there was an effect on the outcome of interest.
  3. Beneficiary control over Service Providers: give greater control to the beneficiaries. This is based on the view that recipients should be at the centre of service provision. There needs to be a demand for the service, and a mechanism by which beneficiaries can really affect performance – this is rarely the case as they do not generally have the power to hire/fire nor set salaries. Experiments in this area have yielded disappointing results. An experiment that asked a local to monitor presence of a health worker did not improve attendance, and a school committees experiment had similarly lackluster results. It is suggested that in many settings beneficiaries are not actually upset about the state of service provision – they have low expectations and as a result have little desire to invest time and energy into making better services. [See paper below for rebuttal]. This indicates that increasing demand for quality service may be a key way to get better outcomes.
  4. Demand side interventions: An incentives to learn initiative in Africa whereby the best performing students were given a scholarship for the following two years increased presence of both teachers and pupils in the treatment group. Why there was an effect on teacher is not clear, it may have been that they were inspired by the increased attention of the students, they had higher status when one of their students got the scholarship, parents may have started to be more serious about education when there was a financial incentive to do so etc. Interestingly the effect was present also for boys who were not eligible to win the scholarships.

All of this suggests that some combination of programmes may be effective. Raising demand can be a good way to increase outcomes, and also to generate an environment in which local monitoring will be effective – a virtuous circle. Also, incentivizing teachers’ presence and performance might be a good way to increase attendance, and effort exerted in delivering the service.

                                                                                                                  

Power to the People: Evidence from a Randomized Control Field Experiment on Community-Based Monitoring in Uganda M. Bjorkman and J. Svensson

 

In a Nutshell

Under the right conditions community based monitoring can be effective. Community based monitoring groups were set up to monitor local health providers. NGOs assisted in forming the groups and facilitating a discussion about what the people wanted from their health service, and drawing up a plan to improve them. This was then discussed with the health provider and a sort of contractual plan was drawn up. Under these circumstances they find a significant relationship between the degree of community monitoring and health utilization and health outcomes. The reason they theorize they found results where others have failed (see above) is that there is a lack of relevant information that prevents benefits from general community monitoring. Thus, as the community group was given access to a large amount of information, including local health data outcomes, and information about what might be expected from health providers, they were better able to come to an agreement on what services should look like, and hence more able to effectively monitor. In sum, a lack of information and failure to agree on expectations of what it is reasonable to demand from a provider was holding back individual and group based enforcement.

Treatment times fell, child mortality fell nearly 50%, and people were more likely to use the health facilities.

This paper effectively increases demand for better service provision, and also provides a mechanism for a community to achieve that level of service.

INCENTIVES TO TEACH BADLY

INCENTIVES TO TEACH BADLY? AFTER-SCHOOL TUTORING IN DEVLOPING COUNTIRES

S. Jayachandran (2008)

Principal Research Question and Key Result Does the ability of teachers to offer paid tuition outside of the school alter their incentives to deliver the in school service? The results of the analysis indicate that tutoring has a negative effect on test scores which suggests that being able to offer tutoring gives perverse incentives for teachers during the school day.  
Theory There are two theoretical links from tutoring to achievement. Student achievement is a function s(m, t) i.e. a function of material taught in school (m) and tutoring):

  1. Tutoring and School are substitutes then δ2s/(δm/δt) = smt < 0: This just states that the value of tutoring increases when less material is taught during the school day. This implies that a teacher can raise demand for tutoring by decreasing the amount of teaching during the school day.
  2. Tutoring and School are compliments then smt>0: This would  hold if there was some threshold level of achievement that students were trying to reach, and for students just shy of the threshold they could benefit from tutoring. This would incentivize the teacher to teach more material, such that there were more students who were able to get close to the threshold level.

The utility to the teacher depends both on his profit, and on the costs of raising/lowering the amount taught. This implies that a tradeoff between the costs of changing m, and the benefits of higher profits induced by changing m.

Given that the results indicate that tutoring and schooling are substitutes, this implies that policies to restrict the provision of t, may increase the amount of m, and also that increasing the costs of lowering m (for example through stricter supervision) could also be welfare creating.

Another possibility is to increase the number of third party tutors. If such tutors offer a higher valued service (for example through smaller tutoring groups), then the teacher will be incentivized to teach  more during the school day, as if less is taught then some students will be diverted to the higher quality third party tutor. The increased competition will reduce the cost of tutors, and more people will take up tutoring, and everyone enjoys the benefits of being taught more in the school day. The reason this holds is that there is less incentive to manipulate m when only some of the students induced to then purchase tutoring will do so from them.

 

Motivation In the developing world many students attend outside tutoring sessions and it is common for the student’s own teacher to also serve as the tutor. This is not common in the developed world. This could be because there is a lower opportunity cost of time due to income effects in the developing world. It could be that there is smaller supply of educated non-teachers who can serve as tutors. Also, less effective means of monitoring teachers by supervisors and parents may increase the ability to rent seek by teachers thus increasing their interest in providing tutoring – this might incentivize teachers to avoid teaching the curriculum in schools in order to generate demand for their fee-generating tutoring classes. If this is the case then all students are made worse off (by less formal education), but those who are hit the most are those who are unable to afford (or otherwise do not demand) tutoring. As such, rather than making the education sector more efficient (by improving access to education for weaker students/those who demand more education) it may actually create inefficiencies. In this case banning teachers from tutoring, or reducing the barriers of entry for third party tutors could be welfare increasing for all students even for those who do not take up outside tutoring. 
Experiment/ Data Data are from a large nationwide survey of students, schools, teachers and families conducted in Nepal. There are 3850 public schools and 890 private schools in Nepal. Students who have completed year 10 and taken the national exam for which the results are recorded are the focus. A random sample of schools is chosen. There are demographic details of the schools, as well as data on whether the student took tutoring, and subjective measures of the quality of school teaching. 
Strategy

ExamScoreijk = βOffersjk + θTakesijk + λi + ρk + εijk                (1) 

This is a fixed effects  model. i is individual, j is school and k is subject. Offers is a dummy that equals 1 if the school offers tutoring in that subject. Thus it is identified by comparing subjects within a school. Making the estimation within school reduces endogeneity concerns such as schools with more resources/brighter students providing more/less tutoring.

To test the notion that there is negative selection into Offers, a regression with Offers as the dependent variable is regressed on σ(PriorExamScore)ijk . If sigma is negative this implies that selection into offering tutoring is negative, as passing the exam is associated with a reduction in offering of tutoring.                                               (2).

A DID estimation is estimated

ExamScoreijk = βOffersjk + θTakesijk + τ(Public * Offers) + λi + ρk + εijk                  (3) 

Where tau is the differential effect that offering tutoring has in a public school (as opposed to a private school). The assumption is that the unobservable elements that encourage selection into OFFERS are the same across private/public schools. An interaction between Public*Takes is also included.

 

Results The results from (1) are negative for Offers, but not really significant. Takes is negative and significant. This indicates that worse students may be selecting into tutoring classes. Whilst some endogeneity is removed by looking within school, it is still possible that whether the school offers tutoring in a specific subject is driven by individual student/teacher ability in that particular subject. Thus, the negative coefficient on Offers could just be reflecting the negative spillovers from the school having to offer tutoring in the first place (due to low quality students).This is partially rebuffed by the results of the Offers regression (2) which shows no relationship between offering tutoring, and past achievement, although this is analyzed on a non-random subsample of the data.

The results of (3) are that tau is negative and significant indicating that when tutoring is offered students in public schools are differentially more likely to fail the exam (presumably as private school teachers are less able to vary the amount of material that is taught in the school day due to better monitoring/financial incentive). The Takes*Public interaction is positive, indicating that selection is negative (I don’t get this bit).  The effect is larger when the sample is restricted to small towns as the school more likely to behave like a monopolist with control over both schooling and tutoring. In urban areas there is likely to be more competition.

Using whether the teacher completed the in school curriculum as the dependent variable, it is shown that the coefficient on OFFERS is negative, indicating that offering tutoring may be incentivizing teachers to teach less.

Robustness
  • Uses different samples.
  • Test alternative hypotheses: could be mechanical fatigue, but the relationship between teacher effort and offers is only marginally significant and negative.

 

Problems
  • If preventing teachers from tutoring decreases wages in the educational sector sufficiently, this may have the effect of dissuading talented teachers from entering the profession, and in the long run this could damage the education sector and be welfare reducing for all students.
  • The subjective measures were based on post exam reflections which could indicate recall bias/be affected by personal feelings toward the teacher.
  • The DID estimation may estimate the differential effect of tutoring in public schools but it cannot speak to the direction of causation. It is still possible that results are driven by negative spillovers from tutoring provision i.e. negative selection.

 

Implications One reason for poor educational outcomes in developing countries could be that teachers lack strong performance incentives. This could indicated that a partial ban on teacher’s tutoring or encouraging third party entrants could be welfare improving, although this will depend on how people sort into those professions. Additionally there may be political constraints that prevent this course of action, as civil service teachers will tend to be well unionized, and a politically visible component of society. That private schools perform better could be an indication that performance pay, or increased monitoring by parents due to a financial stake in the education provided could be useful for increasing test scores.The results could have implications for other sectors. In particular, health workers with a private practice on the side may be facing very similar incentive structures. In actual fact, the incentives may be even stronger, as only one patient observes the outcome of their effort, whereas in a school, potentially many student/parents etc. observe the outcome. This could mean that the costs of varying m for health workers is much lower (as detection is harder) and hence they are more likely to do so in order to increase extractable rents.