- They say opportunity makes a thief. Here’s case study number 1.
If you’re getting sick of hearing about how the coronavirus has disrupted normal life, well, we’re in the same boat. Still, there’s no way around it, many things we took for granted have changed.
One of the most severely impacted facets of everyday life is something as simple as going to school. There’s a whole debate out there that we won’t be getting into, but it all boils down to one question – is it safe to send our kids to school?
Around the globe, the answer has been “no”. As a result, schools have had to put in place remote classroom solutions that have ranged from brilliant to… Less than brilliant, to put it nicely.
Many schools in the U.S. and the U.K., for example, make students take their exams online. To help their teachers deal with grading the exams, some of them have started relying on an AI.
The problem here is as that AI can be kind of stupid sometimes. Many students in both countries have been angered and reduced to tears because the AI’s grading algorithm is skewed or incorrectly calibrated.
Others, though, have found the AIs strict algorithms to be an easy avenue to better grades. Students as young as 12 have learned how to game the system, formatting their answers so that the AI gives them a perfect score.
That’s sometimes even if the answer is pure gobbledygook. Try learning that in a classroom.
From Zero to Hero
One of these students is seventh-grader Lazare Simmons. According to his mother Dana, she found her son crying after taking his history exam.
The boy had gotten a score of 50 out of 100. For those unfamiliar with the U.S. grading system, that’s not great. In fact, it’s a failing grade.
“He was like, I’m gonna have to get a 100 on all the rest of this to make up for this. He was totally dejected,” Mrs. Simmons – a history professor herself – told The Verge.
She tried to console Lazare, telling him that some teachers are harsh graders early in the semester but usually mellow out later on. But her son told her that the teacher wasn’t grading his answers.
He had gotten his results back in seconds after he had submitted the exam. No matter what kind of a super teacher you are, no human being can grade an exam so quickly.
Instead, an AI algorithm developed by a company called Edgenuity was the only one looking at what Lazare submitted. Simmons asked her son to submit some more assignments, so she could see how the system worked.
Since Edgenuity gives the user its preset correct answers, Lazare and Mrs. Simmons were able to deduce that the system was looking for specific keywords.
Together, the two figured out the best way to fool the AI. For every answer, Lazare now writes only two sentences, followed by a nonsensical list of keywords that he assumes the system is looking for.
Lo and behold, he’s getting a perfect score on every question. Human ingenuity 1, artificial intelligence 0.
Word Salad for Lunch
As an example of a perfect answer, Mrs. Simmons quoted the following prompt: “What was the advantage of Constantinople’s location for the power of the Byzantine Empire?”
“So you go through, okay, what are the possible keywords that are associated with this? Wealth, caravan, ship, India, China, Middle East, he just threw all of those words in,” she explained.
And the system swallowed it hook, line, and sinker. Edgenuity’s website explains that the system is seriously only looking at keywords.
Give it what it wants, and it’ll give you a perfect score. It doesn’t matter how the answer is phrased.
We can all decide for ourselves if what the Simmons’ are doing is immoral. On one hand, they’re practically cheating on a school exam; on the other, the system in place is so dysfunctional that maybe it’s justified.
Lazare, at least, doesn’t seem to feel too bad about it.
“I wanted to game it because I felt like it was an easy way to get a good grade,” the boy said, shrugging away the moral dilemma.
‘Works Every Time’
Lazare isn’t alone in taking advantage of simplistic AI grading. One student, who wanted to remain anonymous, told The Verge that they simply copied the presented question to Edgenuity’s answer field.
According to them, all the keywords the AI was looking for were right there in the question. The system gave them a perfect score “pretty much every time”.
Austin Paradiso, another student, said his high school used Edgenuity and he too sometimes resorted to underhanded methods. And they worked every single time.
“I always tried to make the answer at least semi-coherent because it seemed a bit cheap to just toss a bunch of keywords into the input field,” Paradiso said.
“But if I was a bit lazier, I easily could have just written a random string of words pertinent to the question prompt and gotten 100%.”
Fight the System
In the U.K., however, students have not welcomed easy grades. That might be because for many, a flawed AI grading system has cost them their place in college.
According to Wired, roughly 40% of students taking their college placement-deciding A-level exams have gotten lower scores with the AI than they would’ve with a human teacher.
The system used in the exam is particularly unfair because it punishes students for factors beyond their control. The AI looks, among other things, data from past students of the particular school when assigning individual grades.
The process is complex, but the whole domino effect leads to things like school funding affecting the students’ grades. And no matter what you think, that shouldn’t be a factor when assigning grades.
The students have been so angry that they’ve taken to the streets in protest. And it’s not hard to see why. Missing out on college because of an AI that is too dumb to understand your exam answers must not feel great.
An AI overthrowing humanity is a common science fiction trope. In reality, it seems these dumb things should at least learn to read before they can get to that point.