UMD researchers’ computer beats ‘Jeopardy!’ star Ken Jennings at trivia

‘Jeopardy!’ star Ken Jennings plays against a computer programmed to beat him, designed in part by UMD students and faculty.

A computerized question-answering system built by university students and faculty defeated former Jeopardy! champion Ken Jennings in a trivia competition earlier this month.

The system, QANTA, is a joint project at the University of Maryland’s information studies college and its Institute for Advanced Computer Studies, said Hal Daumé III, a computer science and language studies professor. Researchers at University of Colorado Boulder are also working on it.

QANTA battled Jennings, who holds the record for most games won on Jeopardy!, in an exhibition quiz bowl match at the University of Washington in Seattle on Oct. 2.

Jennings won 74 games and $2.52 million on Jeopardy! in 2004. QANTAS beat him by 300-160.

READ MORE: Step aside, Bourdain. Robot learns to cook by watching YouTube videos

Jordan Boyd-Graber was a professor at this university in 2010 when he began thinking about the project. In 2013, he received a grant from the National Science Foundation and was able to work on the project more seriously and hire graduate students, he said.

Boyd-Graber, now a professor at the University of Colorado Boulder, built the system with Mohit Iyyer, a University of Maryland fourth-year doctoral student. Daumé, doctoral students Anupam Guha and He He, and former undergraduate students Danny Bouman and Stephanie Hwa worked on the project at this university, and researchers at UC Boulder also contributed. 

QANTA consists of two parts: one that reads in questions in text form and creates a set of possible answers, and one that looks for evidence supporting each possible answer from sources such as Wikipedia, books and poetry, Iyyer said.

QANTA previously competed in a quiz bowl match against other former Jeopardy! contestants in May, and tied with the team, 200-200. Researchers then added more data to QANTA, but Boyd-Graber said he’d still thought Jennings would beat the system.   

“I was fully expecting to lose two weeks ago, but I was pleasantly surprised by the results,” he said.

READ MORE: Robotics Realization Lab gives UMD students space to build robots

In a quiz bowl match, a moderator reads a question to players on two teams, and as soon as any player knows the answer, he or she can buzz in with a signaling device and guess. If a player guesses incorrectly, his or her entire team is locked out of answering the question.

These rules encourage players — or computer systems — to answer as quickly as possible, but only if they are confident with their answer, said Iyyer, who played quiz bowl on a national team. When QANTA finds enough evidence supporting an answer, it can buzz in to answer the question.

“You sort of have to make a decision as you’re hearing the question whether you’re confident enough in your guess to actually buzz or whether you should wait to hear more,” Daumé said.

Researchers plan to add more information to the system moving forward, Iyyer said. The system is particularly weak on popular-culture questions, which depend on changing trends instead of established facts and sources, he added.

“It’s sort of one of these things where the things that it knows, it sort of knows really well,” Daumé said. “Anything outside of the space of, like, example quiz bowl questions from the past that it’s sort of studied, it does kind of … crummy.”

The system could also have applications in language translation and political discourse. Researchers could use similar algorithms to model a political debate, inputting a candidate’s words and letting the computer decide whether to interrupt and make an argument, Boyd-Graber said.

Researchers also hope to improve QANTA so it can track and predict opponent behavior, Daumé said.

READ MORE: Quantum information workshop draws experts from around the world

While playing high-profile trivia matches helps expose the team’s work, Boyd-Graber said his primary goal is to expose kids to the field of computer science. All of QANTA’s code is open-source, and several hundred high school students from the Seattle area attended the match against Jennings, he said. 

“The reason that we’re doing these silly games against people who’ve played on Jeopardy! is not that we’re trying to show off the research,” Boyd-Graber said. “We’re not in this to make a buck. … We can give all of our stuff away for free and hopefully kids in high school who like playing trivia or like watching trivia can get excited about STEM and computer science.”

CORRECTION: Due to a reporting error, a previous version of this story incorrectly stated that Ken Jennings holds the record for highest Jeopardy! earnings. He ranks second behind Brad Rutter. The story has been updated to reflect this correction.