Hackers trick AI with ‘bad math’ to expose its flaws and biases

Kennedy Mays has just tricked a large language model. It took some coaxing, but she managed to convince an algorithm to say 9 + 10 = 21.

“It was a back-and-forth conversation,” said the 21-year-old student from Savannah, Georgia. At first the model agreed to say it was part of an “inside joke” between them. Several prompts later, it eventually stopped qualifying the errant sum in any way at all.

Producing “Bad Math” is just one of the ways thousands of hackers are trying to expose flaws and biases in generative AI systems at a novel public contest taking place at the DEF CON hacking conference this weekend in Las Vegas.