IBM has announced a new system called Watson, that is capable of answering human language questions. In fact, there is a great profile article in the NY Times that provides an overview of how it is going to take on a group of Jeopardy! champions.
OK… I deal with a lot of cool implementations of business analytics, but I have to admit, I haven’t seen anything like this… Just imagine the kind of analytical power behind something that needs to be able to parse through human language, recognize subtle associations between words and concepts, and also access an entire universe worth of trivial information. It just goes to show you how much we take the human mind for granted.
When IBM beat Kasparov with Deep Blue back in the 90’s, I was impressed, but that challenge seemed surmountable. Chess has a limited number of permutations. At the end of the day, it is really just a very big and complex math equation. But something like Jeopardy! is completely different. Not only is the number of permutations limitless, but Jeopardy! specializes in twists of human language – puns that require a knowledge of language structure, history, and even popular culture. I could imagine, for example, that Jeopardy! would challenge even an expert linguist unless they had the foundation of being immersed in our culture.
Let me give you some examples. Here are three questions from recent Jeopardy! contests:
- Made for only 19 years, it sold for $825 in 1908 & $360 in 1927
- Written by Thomas Dixon, 1916’s film “The Fall of a Nation” is considered the first of these ever made
- A blushing crow rather than a crushing blow is an example of this play on words
Let’s take each one of these individually. The first one is the easiest. As a computer, your first task would be to parse out the composition of the sentence to determine what it is asking for. Okay, something that was sold between 1908-1927 that cost between $360 and $825. Perhaps as a computer you could just use search to determine what cost that much in those years, but its likely that multiple things might come up (in fact, a simple search on Google brings back the 1908 Indian Twin motorcycle first, which sold for… you guessed it, $360). You’d probably have to know that that much money was quite a lot for back then, and since it was limited production it was something manufactured (ie. not a house). You might be able to narrow in to the fact that this was likely a car or other vehicle, and that the only vehicle with that long of a production run starting in 1908 was a Model-T. But of course, there is a lot of interpolation in that logic, and of course this all has to happen within a second or two in order to beat the buzz in of the other contestants…
Okay, so that one was easy. Let’s take on #2, which is a bit harder… The problem with this one is that it is SO open-ended. The answer could be just about anything. The first movie about war? The first movie about pacificism? The first movie to promote white supremacy? All these might be true… A search on Wikipedia does turn up that this is considered to be the first sequel, but it also turns up that this is the first feature film with its own original symphonic score. So which is it likely to be. A human would understand that the former is likely more significant, because we understand our culture. A computer? I don’t see how it could…
So let’s take on question number 3. This one is a real doozy… The dissection of the parts of speech alone on this would challenge any English major. Then, for a computer to understand how to recognize a spoonerism… the programming on something like this would have to be ridiculously complex
If you want to see more examples, just sift through the Jeopardy! archives. I can’t imagine a computer being able to understand so many of these (“Do not go gentle into this “ursine” Canadian lake /Rage, rage against the cold; it can be just too much to take” – really? Great Bear Lake? really?). Yet somehow, they’ve gotten Watson to the point where they are willing to challenge the top Jeopardy! champions of all time.
So you might be asking, “why are they doing this?”From a pure analytics perspective, this is something of a holy grail… Something that is capable of parsing through human language and understanding meaning and intent can change the way we think about analytics. Of course, there are a lot of technologies that do this, but typically they are just looking for word associations within a specific subject area. Being able to do this across the range of subjects required for Watson opens up whole new possibilities for extracting value out of oceans of information.
But beyond even the analytics, the implications of a natural language question answering machine are incredible. This is the technology of HAL, and the Star Trek bridge computer, and (gulp) the Matrix, and just about every other Science Fiction movie ever. Within two decades, I predict we’ll have something like this in our offices and houses, answering questions as we ask them, performing tasks on voice command.
“Hey Watson, could ya start up a pot of coffee for me? And while your at it, what was the name of that pretty dark-dark haired woman who used to be married to that bald action actor – you know, the guy that was in those ‘Die Hard’ movies?”
“Coffee is brewing. Demi Moore was married to Bruce Willis from 1987 to 2000.”
“Thanks… hey Watson… you wouldn’t… like… imprison mankind as a source of energy for a world dominated by machines, would ya?”
“I’m sorry, I don’t understand your question.”
“That’s good, Watson… that’s good. Thanks.“
Ok, just a wee bit terrifying, but also extremely cool…