How can we find out how well our students are learning? Or how well a teacher or a school is teaching? I’m sure most teachers will tell you to look at the results: the students’ writing, discussions, lab reports, equation resolving, translation, and so on. Go to the concert. Watch the play. Attend the games. You’ll learn about the quality of the teaching and the school leadership.
But this isn’t how governments, education scholarship, or the news media assesses education. They prefer to rely on simple statistics from standardized tests. The Organization for Economic Cooperation and Development runs the PISA tests. It’s member governments set store by PISA results, as do even the most intellectually serious news media organizations[1]. In the United States, many school and districts use the Northwest Education Association’s MAP test, as well as state government tests. These are only a few examples.
The problem is that PISA, MAP, and other standardized tests are not as good at measuring student learning as they are at measuring how much students care about the tests. And, unless something is riding on their results, like secondary school or college admissions, why should students care? Another seemingly meaningless test in a calendar already popping with them? Would you care if you were a kid?
Moreover, the skills that standardized tests measure are myopically narrow. The PISA reading test, like the language component of most standardized tests, measures progress in basic reading comprehension only. It tells us nothing about how deeply students can analyze what they read, let alone whether they can write about it (save for a small number of questions that might elicit one or two sentence responses at best).
The MAP reading test is even narrower. It doesn’t require any written responses at all.
This kind of test encourages teachers to focus on short non-fiction passages and on surface-level comprehension. It treats education as mere skill-acquisition, not as a passport to culture and the life of the mind.
The thinking behind this way of assessing education is what I call the quantitative tyranny. It holds that we have to assess by way of numbers; that we can’t do it through observation of teaching, samples of students’ writing and other work, samples of teachers’ feedback on students’ work, or other evidence of learning and learning culture. Because that’s wishy-washy, right?
Whereas numbers are focused, impartial representations of the truth, unsullied by sociological factors or by whether students do or should give a damn about standardized tests. Right? And numbers are capable of representing everything. Right? Even something as intangible as a child’s intellectual growth is not beyond the omniscient reach of figures in tables and graphs. Of course not!
Actually, the only real problem with a holistic, observation and investigation-based approach to assessing education is that it’s expensive. And what politician would dream of spending money on improving education when we could have tax cuts and market-distorting subsidies instead?
There are early signs that AI might be part of an inexpensive solution. Like many teachers, I’ve played around with Char GPT and other tools like School AI and Khanmigo to see how they might help me. Chat GPT can actually deliver a solid critique of an eighth-grade student’s essay. In future, might AI be able to score writing samples or even recorded discussions to give insight into the efficacy of a teacher, a teaching method, a school, or a school or district-wide approach? I don’t consider myself a full-fledge techno-optimist, but I’m holding out some hope.
I’d like to hold out hope for proper investment in education and the holistic evaluation of education. But I’m no politico-optimist.
[1] “Special Report: Must Try Harder.” The Economist, 13 July 2024. https://www.economist.com/special-report/2024-07-13.
Ignition for Education 
Leave a comment