Thursday, August 17, 2006

Polly maths

WARNING: this post is lengthy and detailed, and does not relate to a recent column.

Polly Toynbee appeared on Radio 4's Any Questions on the 14th July, 2006 (transcript). I would have blogged about it at the time, but I couldn't find the transcript immediately and I didn't want to rely on my memory to criticise. I did, however, stumble across the transcript on the web the other day, and discovered that she distorted as badly as I remembered. Take, for example, this exchange:

LAWSON ...The facts are these: That certainly there's been a great deal - there's been a great increase in carbon emissions over the past centenary as a result of mankind and carbon emissions - carbon - CO2, carbon dioxide, is not a pollutant, it is what plants need to grow on, it is actually a life force like oxygen. The - but the nevertheless there has been a big increase. What has happened to the world's temperature so far - over the past 100 years - and this is not in dispute, this is accepted on all sides - the temperature of - the average temperature of the world has increased by two thirds of one degree centigrade... Nevertheless, there's always a risk that carbon dioxide emissions may contribute more, they've contributed something to this two thirds of a degree and therefore it is sensible to take out an insurance policy and may be nuclear power is a sensible insurance policy to take out.

CLARKE Polly Toynbee.

TOYNBEE Well I find that very interesting I think Lord Lawson is one of the last of a very rare breed that perhaps we ought to be protecting. He is a climate change denier. And there are almost none of them left. I used to get loads of e-mails and loads of correspondence on this from various scientists all over the world and they have all shut up and gone away, except for Lord Lawson. There is virtually nobody and certainly no reputable scientist left and even George Bush has had to face up to the unpalatable truth that global warming is certainly happening and we are to blame.

So, Lawson says that there has been a rise in CO2 emissions, says that there has been a rise in temperature and that the CO2 emissions have "contributed something to this two thirds of a degree", and he is branded a "climate change denier"? What? It almost feels like a prepared rant rather than a reply to what Lawson actually said.

-o0o-

But that's not what I wanted to write about. On Any Questions, Polly made the following claim:

[T]he figures show that if you take a very clever very poor child at the age of 22 months and you compare them with a very dim but well off child at 22 months, there they are at opposite ends of the spectrum, by the time they're six years old they will have crossed over and their trajectories will go in opposite directions.

It was a claim which intrigued me at the time, and I was reminded of it when I re-read the transcript. So I did a little bit of digging. Polly wrote this on 22 January, 2003:

Take babies tested for attainment at the age of 22 months: at one end of the scale is a very bright child from a poor home and at the other end is a dim but rich baby. At just under two years old, the bright child scores 85 points on the scale while the dim one scores only 10. But the two children are already on a steep trajectory in the opposite directions, the poor/bright one travelling fast downwards, the rich/dim one moving up, as their social backgrounds counteract their inborn abilities.

By the time they hit nursery school, at the age of three, they have nearly converged - (poor/bright scores only 55 now, while dim/rich has risen up to 45). At the age of six the children's lines cross, and then diverge for ever more as they head off into opposite futures. Anything that happens by the time they reach school is only remedial, seeking to pull up the poor child's scores to where it began.

On 3 June, 2003, she wrote this (and some of this may seem familiar):

Take babies tested for attainment at the age of 22 months: at one end of the scale is a very bright child from a poor home and at the other end is a dim but rich baby. At just under two years old, the bright child scores 85 points on the scale while the dim one scores only 10. But the two children are already on a steep trajectory in opposite directions: the poor/bright one travelling fast downwards; the rich/dim one moving up, as their social backgrounds counteract their inborn abilities. By the time they hit nursery school aged three, they have nearly converged - poor/bright scores only 55 now, while dim/rich has risen to 45. At the age of six the children's lines cross, and then diverge for ever more as they head off into opposite futures.

So anything that happens by the time they reach school is only remedial, seeking to pull up the poor child's scores to where it began.

On 3 September, 2003, worried that her readers may still not have tired of the wonders of the copy-and-paste function of her word processor, she writes:

He [David Bell, chief inspector of schools] quotes influential research from Dr Leon Feinstein of the LSE, whose findings electrified the education ministers. Testing babies for attainment at the age of 22 months, their progress was followed according to social class. It found very bright children from poor homes and dim but rich babies at the other end of the scale were already on a steep trajectory in the opposite directions, the poor/bright travelling fast downwards, the rich/dim moving up. By nursery school at three, they have nearly converged. At the age of six, the children's lines cross and then diverge for evermore as they head off into opposite futures.

So the rest of school is just remedial to repair early damage already done.

We're given a bit of a respite until 2 April, 2004 when, the meme is shortened down to:

A baby's fate is virtually fixed at 22 months: school is too late.

These have all been from the Guardian, which at least has the merit of a voluntarily paying readership who can stop buying the paper if they tire of buying endlessly recycled prose. That is not true in the case of a pamphlet she wrote for the Mayor of London (pdf link here), where she wrote:

Take a very bright poor child at the top end of the ability scale and a very dim well-off child at the bottom end of the scale at the age of 22 months and test them again as they both reach nursery school at the age of three: the poor/bright child will have slipped far down the ability scale while the rich/dim kid will have risen up it. They are both on a steep trajectory in opposite directions, the bright/poor downwards, the dim/rich upwards. At the nursery school gates their scores converge. By the time they reach the age of six in primary school the children’s lines cross, with the rich/dim heading on upwards and the poor/bright one falling back as they head off into opposite futures for ever more. Any schooling thereafter will only offer remedial help for the damage done to the poor child in the earliest years.

According to Ken Livingstone, this article cost about £7,000 -- and the tendering was not competitive (source). The next time Ken decides to spend tax money I contributed towards to copy and paste bits of old Guardian articles together, I will undercut £7,000 as a fee. But then again, I don't write things like:

This week Ken Livingstone has shown how real bravery in the face of near-universal attack and predictions of disaster is winning through on London's congestion charge.

in a national newspaper, so maybe my back doesn't need scratching quite as much as hers.

-o0o-

But what of the actual research? Well, it is inaccurate précis of an article by Leon Feinstein published in the Feburary 2003 issue of Economica, entitled: "Inequality in the Early Cognitive Development of British Children in the 1970 Cohort". I haven't found a free copy online, but it can be purchased through the Social Science Research Network (www.ssrn.com). [UPDATE: See comments for a link to a free version of the pdf. Thanks, Chris.]

Here is figure 2 from the article, which is what Polly is trying to summarise:

Where "High SES" means that the child's parents had high socio-economic status (i.e. "father was professional/managerial and mother was similar or registered housewife") at the time of birth, "Low SES" means the parents had low socio-economic status (i.e. "father in semi-skilled or unskilled manual occupation and mother similar or housewife") and "Q at 22m" refers to the child's quartile in tests carried out at 22 months. "High Q", then, means that the child was in the top quarter of children tested at 22 months.

On the face of it, then, you can see why Toynbee characterises it the way she does -- it does appear to suggest that "rich/dim" kids do overtake "poor/bright" ones at between six and seven years (assuming the trend lines are indeed linear).

However, here are four big things that Polly does not tell you about the resarch.

ONE It is carried out on children born in 1970. That's right -- it is measuring what happened over thirty years ago.

TWO When she talks about dim and bright kids at 22 months, she doesn't tell you what is being measured. I will. Here is the questionnaire (pdf link, sorry) that was used in the research. Here are some of the things that were measured:

  • Can he balance on one foot for one second?
  • Can he jump in one place?
  • Can he put on his pants?
  • Can he draw a vertical line?

Now, I'm not for one second suggesting that these are not broadly reliable indicators of general development but to call a kid who cannot balance on one foot for one second at the age of 22 months "dim" is pretty hypocritical of someone who complains that failing the 11+ made her feel that:

I was judged on that day and I was found stupid. And I was very lucky, I was middle class for a start which meant that I was likely to get a leg up in life, unlike lots of children who were dammed forever by that exam, quite unfairly and quite wrongly. [source]

THREE Polly shortens the comparison of socio-economic status to "rich" versus "poor". In fact, it is not clear that it is the socio-economic status which is making the difference. In fact, Table 3 in the Feinstein article shows that the background variables at birth (parents' socio-economic status, parents' schooling, number of siblings, sex, and mother's age) account for only 25% of a child's ranking in tests carried out at 10 years of age.

Furthermore, socio-economic status is quite a small part of that. If you were born in 1970 and your father's socio-economic status was low (3, 4 or 5), this cost you -7.2 percentiles in your test score ranking at age ten. Having an older sibling would have cost you -8.9 percentiles and two older siblings would have cost you -13.9 percentiles.

In other words, having two older siblings damages your test score results at age ten almost twice as much as having a father of lower socio-economic status.

Oh, and the single biggest thing that makes a difference is having a mother with a degree. This boosts your test score results at the age of ten by +25.2 percentiles.

Why is this important? Well, let's not forget that Polly argues for specific policies and uses the "facts" that she cites to support these. In the her article of 2 April, 2004, for example, she uses it to argue for an "increase in the top rate of tax", because poverty causes poor school results. That this might have been true thirty years ago, and that even then the explanatory power of parent's socio-economic status was relatively low is actually quite helpful context to allow us to judge the quality of her policy prescriptions.

FOUR Polly wrote that:

A baby's fate is virtually fixed at 22 months: school is too late.

No. Table 4 of Feinstein's article shows that of the children who were in the bottom quartile of test results at age 22 months, 32.3% went on to get A-levels or higher qualifications, compared to 43.3% from the top quartile. Doing badly in a test of jumping in one place aged 22 months is not tantamount to failure.

7 comments:

Mr Eugenides said...

Art.

Anonymous said...

Here is a pdf version of the Feinstein paper you refer to:
http://www.rlab.lse.ac.uk/opening/papers/feinstein.pdf

Tristan said...

Re: Polly's love of Ken (probably second only to her viking warrior Gordon)

I was first alerted to Polly when I read her calling for Ken to have more powers in a free Guardian Lite thing.
I'd never read such nonsense, and you can bet if it wasn't a social democrat/socialist mayor she'd not have said that, but would have been calling for Labour councils to hav more power...

It did have two good affects however. I was alerted to her and it disuaded me from buying the Grauniad...

Carps said...

Wonderful.

Anonymous said...

Simply beautiful.

Anonymous said...

Excellent fact-checking. I wonder if you have any disclosures to make.

GS

FactcheckingPollyanna said...

GS -- what do you want to know?