New York – Schools around the world are trying to figure out what education in the time of COVID-19 will look like — and specifically how, where personal contact isn’t possible, to monitor and assess students’ progress. The experience of Hadrien Pellous, a high school senior in London, offers a caution: Don’t leave it up to an algorithm.
Pellous attended one of the more than 3,000 schools — most in the United States — that offer a standardized curriculum managed by a Geneva-based organization called the International Baccalaureate. The program includes high-stakes final exams, which typically account for 80 percent of the final grade in a subject and can thus decide a student’s college prospects. When the coronavirus pandemic made in-person testing too risky, the organization cancelled the exams and replaced them with something very different: a statistical model designed to predict how students would have done.
In opting for a model, the International Baccalaureate was following a global trend. Statistical models are increasingly being used in high-stakes areas, such as assessing teachers and deciding who gets parole, on the grounds that they offer objectivity. All too often, though, they employ variables that are biased or out of the subject’s control — such as flawed baseline test scores or the zip code where a person grew up. Also, they’re often opaque, leaving people with little sense of how exactly they operate and even less recourse to correct errors.
The organization hasn’t disclosed the exact workings of its model, which a spokesman insisted isn’t an "algorithm” (even though it fits the definition perfectly). Judging from what is known, it doesn’t look good. One input is graded coursework, which is flawed because some students hadn’t completed assignments when the grading plan was announced, and hence had a chance to put in more effort. Another is "expected grades” produced by teachers — who might be biased in favor of certain types of students, such as boys in math. A third is the school’s historical performance, which could hobble academic stars or be statistically meaningless for smaller schools.
For Pellous, the result was disastrous. The model gave him a total of 28 points for the six subjects he studied — far short of his expected 37 points and low enough to cancel the conditional offers he had received to study petroleum engineering at Aberdeen University and mining engineering at Exeter University. No explanation was forthcoming, except that the model was reviewed for fairness and statistical robustness (I requested but did not receive a definition of fairness from the organization). Now Pellous and his parents are reduced to making desperate pleas for a review.
Pellous is not alone. Many students and teachers have expressed shock at the outcome. As of July 15, more than 20,000 people had signed an online petition demanding more transparency and review, and complaining about shoddy and inexplicable results. In Pellous’ case, it’s hard to see how the historical data of a school with a graduating class of about 20 should be relevant. It’s clearly not reasonable to suggest that if students at a school did poorly in the past, then all students from that school should be downgraded now, no matter how hard they worked.
As things stand, all the people who feel wronged by the grading algorithm must pay for a review — which, for six subjects, would cost a total of about $540. That seems like a lot to ask, given how unthoughtful and crude the system has already proven to be.
As schools find their budgets pinched by falling enrollment and added costs related to COVID-19, more are likely to consider some kind of automated grading. One can only hope they will learn from the International Baccalaureate’s example. The education system is supposed to expand opportunity, not sort people based on a flawed concept of their potential.
Cathy O’Neil is a Bloomberg Opinion columnist. She is a mathematician who has worked as a professor, hedge-fund analyst and data scientist. She founded ORCAA, an algorithmic auditing company, and is the author of "Weapons of Math Destruction.”
In a time of both misinformation and too much information, quality journalism is more crucial than ever.
By subscribing, you can help us get the story right.
With your current subscription plan you can comment on stories. However, before writing your first comment, please create a display name in the Profile section of your subscriber account page.