Technical STATA assignment - Experts only

EC338 assignment 2017-18 – Frequently Asked Questions

Question 6

 Should I be able to replicate the estimates in Table 6 of Eyles and Machin (2015)?

No. In contrast to clark.dta, which contains the EXACT data used by Clark (2009), academies.dta does

NOT contain the data used by Eyles and Machin (2015). The most important difference is that

academies.dta is at school-level (i.e. each observation contains an average of the variable in question

across all pupils in the same cohort in the same school), while the data used by Eyles and Machin

(2015) is individual-level data. (Note the vast difference in sample sizes.)

 How close should my estimates be to those in Eyles and Machin (2015)?

They should be of the same order of magnitude (i.e. in the same ball park). So, for example, if the

estimate in the paper suggests that becoming an academy increases attainment by 10-15% of a SD and

your estimates suggest that it increases attainment by 50% of a SD, then your estimates are not close

enough. (Note: this does not necessarily imply that anything less than 50% of a SD is close enough. You

need to use your judgement. If you have used the same specification as Eyles and Machin (2015), then

you should get something much closer than that.)

 Should I include year fixed effects in my model?

You should follow the specifications used by Eyles and Machin (2015) as far as possible EXCEPT where

otherwise noted in the assignment question.

 Are the Key Stage 2 outcomes for the cohort entering the school in the year specified in the data or

for the cohort taking their Key Stage 4 outcomes in the year specified in the data?

The latter. The school-level data in academies.dta essentially allows you to estimate school value-

added by taking the difference between the average outcomes of the same group of pupils at the start

and end of their secondary school careers, while the individual-level data used by Eyles and Machin

(2015) estimates school value-added by averaging the difference in outcomes between the start and

end of secondary school for each individual in the cohort.

 When I try to replicate Column 7 of Table 6, my model only produces seven treatment effect

estimates (i.e. one is omitted). Is that OK?

Unfortunately not. You should be able to estimate treatment effects for all eight periods.

 When you say “restrict attention to years up to and including 2009”, do you mean focus on

conversions that occurred up to and including 2009 or outcomes observed up to and including 2009?

I mean the latter, i.e. you should restrict year rather than yrbecomeacademy.

 Can we just include the six Key Stage 2 variables separately or do we need to amalgamate them into

a single summary variable, as Eyles and Machin seem to do in their Table 6?

You can just include the six variables as separate controls in your model.

Question 9

 Does the question ask us how the impact on converted academies has changed since they became

academies, or how the impact of becoming an academy changes across years (i.e. was it better to

become an academy in 2006 than in 2010)?

Neither. Question 9 does not explicitly ask whether or how the impact of academy status changes over

time (except to the extent that Column 7 in Table 6 of Eyles and Machin does). The question asks you to

use a different identification strategy to that used in Question 6, and it is the comparison between

these two identification strategies that I am asking about here.

 Should I be able to estimate all eight treatment effects here, as was the case in Question 6?

No – and it should hopefully be clear why that is the case if you think about the identification strategies

used to estimate the effects here vs. Question 6.

This version: 8am on 14th December 2017