For the second year in a row, Google Assistant has taken home the crown as the smartest of the virtual assistants in a battle against Amazon Alexa, Apple’s Siri, and Microsoft’s Cortana. During Loup Ventures’ second annual “Smart Speaker IQ Test,” the four virtual assistants were asked 800 questions. Google Assistant understood 100 percent of the questions, and answered nearly 88 percent of the questions correctly, according to 9to5Google.
The research-driven venture capital firm’s test was conducted using four leading smart speakers: Apple HomePod, Amazon Echo, Google Home, and Harman Kardon Invoke (powered by Microsoft’s Cortana).
The speakers were graded on both understanding the questions and correctly answering them. The 800 questions were split into five categories: Local (i.e. Where is the nearest coffee shop?), Commerce (Can you order me more paper towels?), Navigation (How do I get to uptown on the bus?), Information (Who do the Twins play tonight?), and Command (Remind me to call Steve at 2 p.m. today).
Loup Ventures explained that they conduct frequent tests in order to “better understand the practical use cases of AI and the emergence of voice as a computing input,” according to the company’s official report.
Google Assistant led the way in answering questions from four out of the five categories, falling behind in commands, where Siri answered 88 percent of questions correctly. Loup Ventures noted that this may be due to the large amount of music-related questions that the command set included, which Apple’s HomePod specializes in.
The #GoogleAssistant knows a little something about mastering conversation. Here’s how you can too this holiday season (we’ve all been at family dinners) → https://t.co/JzH0zUEx7p pic.twitter.com/kwqQYjsQRn— Google (@Google) December 14, 2018
Still, Google Assistant dominated overall, followed by Siri, who understood 99.6 percent of questions and answered 74.6 percent correctly. In a very close third place came Alexa, who understood 99.0 percent of questions and answered 72.5 percent correctly.
Finally, Cortana trailed behind in fourth place. While Cortana did rank higher than Alexa in understanding with 99.4 percent, it only answered 63.4 percent of questions correctly, TechSpot reported.
When compared to last year’s test at this time, it is clear that the speakers saw dramatic improvements. In 2017, Google Assistant had only 81 percent correctly. The competition was also much closer this year than last, as Alexa came in second place in 2017 with 64 percent answered correctly – 17 percentage points away – while this year was less than one.
As Apple Insider pointed out, Siri saw great improvements in the year since the previous test. Siri improved by 22 percentage points, while Google Assistant, Alexa, and Cortana only improved between 7 and 9 percent.
“With scores nearing 80-90 percent, it begs the question, will these assistants eventually be able to answer everything you ask? The answer is probably not, but continued improvement will come from allowing more and more functions to be controlled by your voice,” Loup Ventures concluded.
“This often means more inter-device connectivity (e.g., controlling your TV or smart home devices) along with more versatile control of functions like email, messaging, or calendars.”