Roland ‘Prezbo’ Pryzbylewski: I don’t get it. All this so we score higher on the state tests? If we’re teaching the kids the test questions, what is it assessing in them?
Grace Sampson: Nothing. It assesses us. The test scores go up, they can say the schools are improving. The scores stay down, they can’t.
Roland ‘Prezbo’ Pryzbylewski: Juking the stats.
Grace Sampson: Excuse me?
Roland ‘Prezbo’ Pryzbylewski: Making robberies into larcenies. Making rapes disappear. You juke the stats, and majors become colonels. I’ve been here before.
Grace Sampson: Wherever you go, there you are.
The Wire, Season Four
Education experts have slammed the Turnbull Government for using “incredibly flimsy evidence” to justify not proceeding with agreed Gonski funding increases to public schools and have demanded the public release of the underlying NAPLAN data to test its claim. From The Guardian:
Over the weekend his education minister, Simon Birmingham, released an “analysis” of 375 schools that had improved in their Naplan results, which the minister said showed some of those schools had also seen their funding decline and therefore there was no correlation between funding and school results. Birmingham reportedly said the analysis of the results from these schools – representing about 4% of all schools – proved that the Gonski reforms had been a failure…
The schools education program director at the Grattan Institute, Peter Goss, said “the analysis the minister has released is incredibly flimsy given the importance of the underlying question” and described the link between student performance and funding as “vexed and complex”…
Goss called on the government to release the three to five years of funding and Naplan data for the 375 schools, because it took that long for improvements in teaching practice to show up.
Jim McMorrow, an honorary associate professor of education at the University of Sydney, said it was “not possible to judge the integrity or accuracy of the minister’s claims without his making available publicly the full set of data underpinning the analysis” and added that “simplistic analyses and the misuse of statistics, whether or not by design, can do real damage to the most needy students”…
And a former principal and fellow of the Centre for Policy Development, Chris Bonnor, said he didn’t think “a clear pattern of anything” could be determined in just one year.
All fair enough criticisms. Using only 4% of a dataset to justify a political point is highly questionable. It is not possible to draw any statistical or practical conclusions from such a small sample, especially given that it has most likely been cherry-picked rather than random.
There is another potential ‘fly-in-the-ointment’ too that could skew the NAPLAN results. Maybe the schools have taught their students how to score better in the NAPLAN test, thus giving the illusion of improved school performance?
A few years back, lecturers Arathi Sriprakash and Tony Loughland from the University of Sydney penned a harsh rebuke of NAPLAN in The Conversation, claiming that the test “severely narrows the school curriculum, compounds disadvantage and creates undue anxieties for young students”, as well as a pre-occuption with scoring well in the test above all else:
When schooling is focused so heavily on preparing for high-stakes tests, it is not surprising that teachers have to resort to “defensive pedagogies” – teaching strategies that ensure good examination results. This is made worse by relentless preoccupations with “teacher effectiveness”, where performance is measured by student outcomes. Teachers do more than teach to the test, but the scope for this is diminished under high-stakes scrutiny.
Just like in The Wire, I wonder whether schools are “juking the stats” – teaching the kids the test so that the school/teachers look better – rather than actually improving educational outcomes?
Making NAPLAN results a key determinant of school funding could certainly give rise to such perverse incentives.
