LOS ANGELES – The education ministry’s draft plans for a new privately managed exam in fiscal 2020 or partially maintaining the current government-backed exam until fiscal 2023 for university admissions will likely provoke a heated debate. But in the final analysis, it will make little difference because of the limitations of all standardized tests.
The purpose of administering either standardized exam is to be able to help universities rank students. If test designers loaded up their instruments with questions measuring the most important content effectively taught in high schools, scores would in most cases be clumped closely together. That would make comparisons hard to achieve.
To avoid that possibility, test designers aim to engineer score spread. There’s nothing sinister about this practice. It’s how all standardized tests are designed by necessity. The SAT, which is as close to a university admissions test as those used in Japan, has long done this. The SAT achieves this goal by including many questions that reflect the socio-economic backgrounds of students, since doing so has long produced the desired outcome.
Supporters of either exam now under consideration in Japan will say “a test is a test.” But this reflects a fundamental misunderstanding of the difference between an aptitude test and an achievement test. The former predicts how well a student is likely to perform in a future setting. The latter predicts what a student has already learned.
Although both an aptitude test and an achievement test overlap, they are not the same. This confusion is reflected in the changes in the name of the SAT over the decades. In 1926, when the test was conceived by Carl Brigham as an instrument of meritocracy, it was called the Scholastic Aptitude Test on the assumption that it measured innate ability.
But in 1994, the name was changed to the Scholastic Assessment Test in response to concerns the original name was associated too much with eugenics. In 1997, the name was changed once again to just the SAT, which quite properly is ambiguous.
Before Japan chooses one standardized test over another, it would be well advised to consider the experience of that in the United States. More and more colleges and universities have either entirely scrapped standardized tests as a basis for admission in favor of an applicant’s grade point average, or have made them optional.
Bates College was the first to provide long-term evidence of this approach. It made the SAT optional in 1984. In fall 2004, the college announced that it had found virtually no differences in the four-year academic performance and on-time graduation rates of 7,000 submitters and non-submitters.
According to a paper in the Journal of Public Economics, a later high school grade point average is approximately five times more predictive of whether a student drops out of college within two years, and two times more predictive of eventual labor market earnings.
Standardized exams are not new. They were used in imperial China in the seventh century for those applying for government jobs. Applicants had to write essays about Confucian philosophy and compose poetry. The invention of the printing press accelerated the growth of such tests in Europe.
But that was then and this is now. Japan’s high schools are known for the rigor of their curriculum and the dedication of their students. It’s hard to understand why standardized tests continue to be given such heavy weight in determining university admissions.
Walt Gardner writes the Reality Check blog for Education Week.