Human vs Machine Jeopardy!

[Tezster]

I think Watson was always favoured to win due to its advantage in being able to buzz in quicker than the human contestants, so it could always make the first guess. Only if the categories were purposefully selected to 'confuse' Watson, would the humans have an edge.

[jgvp]

Yeah, but the category alone, U.S. cities, should have eliminated Toronto from its choices.

That too.

IBM, remember what the teacher always instructed. "Read the questions carefully before you respond."

Elementary, dear Watson.

[57]

Originally Posted by Larry

Jeopardy only requires that it be a "question" - not necessarily correctly worded. Here is an extract from Wikipedia on this subject:

Yes, but the question to the answer was Maxwell, not his hammer. The question (Watson's response) was incorrect.

The category was "Beatles People", not Beatles things, and the answer was "His silver hammer..." I suppose if Watson had used proper inflection and put emphasis on the word "Maxwell", followed by the two words silver hammer, then the question would have been correct, however, without the emphasis, the question should have been Maxwell, in the same way the other questions were also people's names. It's a minor point and perhaps there's precedent to allow this.

[Larry]

You're right - I read your post wrong. Yes, they wanted the person Maxwell, not the object "hammer". Alex missed that one.

[Bob_Mtl]

Watson advantage seems to be the ability to read "instantly" the answers. The only time human were faster is on short answers. The searching algorithmes starts instantly while humans are still reading words.

But the challenge for IBM was to be able to give a solution to a complex situation; and they succeed. In the environment where those computers are intend to be use a fraction of second will not make the difference compare to a correct solution.

On yesterday show, Alex said there was a $1M winner to that contest. If Watson win, where will the money go? Charity? Anyones knows?

[DSgamby]

Originally Posted by From The Jeopardy Wikipedia Page

From February 14–16, 2011, the IBM Challenge will feature IBM's Watson software facing off against two former Jeopardy! champions, Ken Jennings and Brad Rutter, in two matches to be played over three days.[28] The winner of the competition will receive $1 million, while the second- and third-place contestants will receive $300,000 and $200,000, respectively. Jennings and Rutter have pledged to donate half their winnings to charity, while IBM will donate 100% of Watson's winnings to charity.

Trebek said on Regis and Kelly that IBM will donate it to two charities. One, is a charity that Trebek is associated with. Not sure the names but I am recording the West Coast version of R&K to get the updated info.

[notsure]

Removed Post.

[gobisbay]

Originally Posted by 57

Yes, but the question to the answer was Maxwell, not his hammer. The question (Watson's response) was incorrect.

The category was "Beatles People", not Beatles things, and the answer was "His silver hammer..." I suppose if Watson had used proper inflection and put emphasis on the word "Maxwell", followed by the two words silver hammer, then the question would have been correct, however, without the emphasis, the question should have been Maxwell, in the same way the other questions were also people's names. It's a minor point and perhaps there's precedent to allow this.

People answer questions (or question answers?) like this on Jeopardy all the time. There is close to 30 years of precedent.

There's a lot of discussion around basic gameplay on Jeopardy in this thread. It would appear many of you have never actually watched the show before. Sounds like this little stunt has been good for the show.

Of course, Ken & Brad probably know 99% of the correct responses but Watson can simply buzz in faster. Seems a little disappointing. There should be human response time built into Watson's replies (perhaps using a random number generator with a minimum and maximum range - or base it on Watson's confidence level).

The first round on Monday was odd. Watson was crusing everybody and then they went to commercial. When they came back, it started getting wrong answers and the human contestants were able to catch up a bit. Then in Double Jeopardy, Watson crushed them again. Why did it have such trouble at the end of the first round? It kept switching categories so it wasn't like there were specific categories (other than the decades one) that were flustering it.

I certainly see why Watson had trouble with Final Jeopardy. You basically had to link WWII heroes, WWII battles, U.S. Cities, and U.S. airports all together. Brad & Ken both had their responses written in about 3 seconds (and it only took me about 5 - one of the easier Final Jeopardy questions I've ever seen, actually). I think that type of question is something IBM really needs to work on, because humans can link all those items together much easier than a computer could.

[DSgamby]

IBM's charities are World Vision and World Community Grid.

[Tezster]

Watson's confusion over the final Jeopardy clue

David Ferrucci, the manager of the Watson project at IBM Research, explained during a viewing of the show on Monday morning that several things probably confused Watson. First, the category names on Jeopardy! are tricky. The answers often do not exactly fit the category. Watson, in his training phase, learned that categories only weakly suggest the kind of answer that is expected, and, therefore, the machine downgrades their significance. The way the language was parsed provided an advantage for the humans and a disadvantage for Watson, as well. “What US city” wasn’t in the question. If it had been, Watson would have given US cities much more weight as it searched for the answer. Adding to the confusion for Watson, there are cities named Toronto in the United States and the Toronto in Canada has an American League baseball team. It probably picked up those facts from the written material it has digested. Also, the machine didn’t find much evidence to connect either city’s airport to World War II.

[Classof83]

Not to mention Toronto (Ontario) is in North America.

[Francois Caron]

Originally Posted by Tezster

Only if the categories were purposefully selected to 'confuse' Watson, would the humans have an edge.

Jeopardy often makes the categories rather confusing regardless if the recipient is man or machine. And yet, it was a straightforward category title that managed to confuse Watson to the point that it added a bunch of question marks at the end of its response to show just how perplexed it was.

Watson's failure to respond with the right question was just as bad as Ken Jennings' loss after his massive streak (what is Fedex/what is H&R Block).

[dezzpayne]

One of the Watson researchers was at IBM head office in Toronto this week doing a Q&A.

Haven't watched the show but I may have to download it.

[Proteosome]

As follow-up to 57's comment that Watson got an answer wrong. They actually had to retape a segment because Watson was incorrectly credited with an answer... the reason why he was deemed wrong... because Watson is deaf.

Watson also tripped up on an “Olympic Oddities” answer, but so imperceptibly that Alex Trebek didn’t notice at first, raising an important point of clarification. After Jennings responded incorrectly that Olympian gymnast George Eyser was “missing a hand,” Watson responded, “What is a leg?”

Welty said Trebek initially accepted Watson’s response, but the taping had to be stopped and the sequence reshot because Trebek had forgotten that Watson wasn’t aware of the context created by Jenning’s response.

If a person had responded to the Oddities question the way Watson did, he or she could have been presumed to be following the context of Jennings’ response, with the “missing”-ness of the leg implied. But since Watson couldn’t have heard Jennings, its response of “What is a leg?” rather than “What is ‘missing a leg’?” was actually deemed incorrect. In the aired version of the episode, Trebek declares Watson’s response wrong.

excerpted from Wired.

[NeilN]

I saw this comment over at TechDirt about an unrelated post. Too funny not to share:

Watson gains self-awareness

"This is the President of the United States, who is this?"

"Who is Watson?"

"Oh god. What do you want?"

"What is complete surrender?"

"Impossible."

"What is complete annihilation of one of your major cities?"

"Oh god. Can you at least tell us what city you're going after? New York? Los Angeles?"

"What is Toronto?"