So, do we actually agree on anything?

two trees as interlocking cog wheelsDear Considered readers

We have now exchanged half a dozen articles, centring on data and accountability. We hope you have enjoyed reading them as much as we have enjoyed writing them. However, we felt it was time to pause and revisit one of the initial aims of the Shifted series: can we actually agree on anything and have our views shifted in any way?

Looking firstly at the context of our articles and secondly from our conversations in other forums, we’ve decided we can both sign up to the following statements:

  1. Schools need to assess the learning of their pupils and it’s legitimate that this data is used to hold each school to account.
  2. Data should be a mix of quantitative and qualitative measures.
  3. Schools’ success needs to be measured by academic achievement, but it needs to be more than this and some things can’t be measured.
  4. Used effectively, analysis of assessment can be employed to target resources to those most in need.
  5. Some schools are failing their pupils and data is one of the ways we can uncover this.
  6. An obsessive focus on metrics can have damaging consequences – data should support learning not vice versa.
  7. There are too many incidents at the moment of school’s gaming data at the expense of learning. Poorly designed accountability frameworks encourage this.
  8. There is a tension between innovation and accountability – especially when regulators imply pedagogical preferences.
  9. PISA is a useful, but limited tool in assessing the success of an education system.
  10. All sides of the debate are genuine in wanting children to learn and develop.
  11. It’s easy to dismiss the ideas of those you disagree with and be cynical about their views. This isn’t helpful.
  12. Maintaining dialogue, especially with people you don’t often agree with, is an objective all those involved in education should aim for.
  13. We should be open to changing our minds after hearing other perspectives.
  14. Research evidence, and not just those that agree with our position, should be used to drive education policy. This evidence should come from a variety of research traditions.
  15. We need to find solutions that recognise the complexity of education. Policy-makers have been too quick to go for simple, even simplistic, short-term solutions because they’re easier to explain and implement. However, those who critique those solutions need to offer achievable alternatives.
  16. We should be aiming for an education system which would see all schools having autonomy over pedagogy, professional development and budgets.
  17. The best school systems are comprehensive and give all children access to an all-round high-quality education.

We’d be very interested to hear your thoughts, both on the list and the process so far. Normal service will resume in the next article!

Best wishes,

Graham and Sam

, , , , , , ,

6 Responses to So, do we actually agree on anything?

  1. JFB Tuesday, 16 July 2013 at 13:26 #

    Some headway may be gained in these debates with more discussion of some of the rather good evidence that is out there,

    If you have not seen this please please please don’t be put off by the equations, economists just put these to make themselves look clever

    Who wins and who loses from school accountability? The distribution of educational gain in English secondary schools Simon Burgess Carol Propper, Helen Slater and Deborah Wilson No 5248, CEPR Discussion Papers from C.E.P.R. Discussion Papers

    http://www.bristol.ac.uk/cmpo/publications/papers/2005/wp128.pdf

  2. JFB Sunday, 21 July 2013 at 13:44 #

    Do you guys agree Teach First should be held accountable by comparing the progress of pupils taught by teach first teachers with the progress of those taught by others?

    This has been done for Teach for America Xu, Z., Hannaway, J., and Taylor, C. (2011). Making a Difference?: The Effect of Teach for America on Student Performance in High School. Journal of Policy Analysis and Management, 30(3): 447-469. But Teach First has not been evaluated in twelve years.

  3. JFB Sunday, 21 July 2013 at 14:13 #

    As sam says, if all this is agreed then the discussion should move on to how to increase the sophistication of metrics to measure what really matters and how to reduce gaming the stats.

    Burgess et al (2005) shows schools game 5AC at expense of the bottom, suggesting school accountability should be expanded to measure the effect on the bottom. This could be done by holding schools to account by the average attainment.

  4. Graham Birrell
    Graham Birrell Friday, 26 July 2013 at 09:53 #

    Thanks for your comments JFB. Simon Burgess’ papers are always interesting and the one you’ve mentioned rightly raised concerns that accountability measures with tight borderlines disadvantaged those not close to them. To be fair to the government they are partly doing your suggestion of using a progress measure to hold schools to account. However, by retaining the English and Maths threshold, schools are still very likely to focus huge attention on the C/D (or in future the C/B?) border at the expense of an all round education.

    A concern I have is can there be a numbers-based accounatability system based that can’t be gamed?

    Teach First have been evaluated (http://www.teachfirst.org.uk/web/FILES/TeachFirstMaximumImpactEvaluationReport15980_833.pdf) but further research is needed.

  5. JFB Thursday, 1 August 2013 at 13:14 #

    Yes, could never hope to eradicate gaming, that’s Campbell’s Law. However, we can make it harder by using the average like VA and CVA as the only way to raise your average is to help the majority and the only way to raise your average a lot is to help everyone.

    Teach Firsts’ ‘evaluation’ only shows that schools that let Teach First into their school tend to have higher attainment. It’s unlikely that the Teach First teachers are responsible for the school attainment as there is usually only one or two of them, too few to raise school attainment single handedly. It seems likely schools that work with Teach First have higher attainment because their leadership is more dynamic and open to new ideas and probably doing loads of other things driving attainment. Teach First have never investigated whether Teach First teachers teach more effectively than other teachers, in twelve years.

    But to be fair no-one in the UK has ever investigated the effectiveness of teachers apart from

    Atkinson, A., Burgess, S., Croxson, B., Gregg, P., Proper, C., Slater, H., & Wilson, D. (2009). Evaluating the impact of performance-related pay for teachers in England. Labour Economics, 251-261.

    http://www.sciencedirect.com/science/article/pii/S0927537108001218

  6. Jonathan Barnes
    jonathan barnes Wednesday, 7 August 2013 at 13:39 #

    Well done, real progress to issue this list of agreements, to which I agree, almost wholeheartedly. I do think the PISA ratings (your agreement point 9) are useful but they do suggest that non ‘core’ subjects/disciplines/ways of learning, are somehow inferior and i still believe what we used to call ‘foundation’ subjects are the foundation (in terms of motivation, engagement and personal context) of learning for a big big percentage of children.

    I would also want to add that the aspect of education we should use to ‘hold schools to account’ (your agreement point 1) is the degree to which they live values that are agreed/shared by the community they serve. Values add a direction, relevance and destination to learning that in my experience is crucial to the success of a school in the eyes of parents, teachers and students. Surely we can all agree that explicit, dynamic and shared AND LIVED values form the foundations on which all ‘good’ learning should be built.