x PhoneArena is hiring! News writer

New test shows that Google Assistant is the smartest virtual personal assistant

Siri messes up

Siri messes up

Virtual personal assistants were tested in a new study that contained 4,952 questions that were asked to all four assistants. Each response was divided into one of five categories. Did the assistant answer verbally? Was the answer sourced from a database like the Knowledge Graph? Did the answer come from a third party source like Wikipedia? Did the assistant fail to understand the question? Did the assistant understand the question, but fail to answer correctly?

The four virtual personal assistants tested were Alexa, Microsoft's Cortana on the Invoke smart speaker, Google Assistant on the Google Home, Google Assistant on a smartphone, and Siri. While all four can handle other requests such as setting an alarm, ordering flowers and getting dinner reservations, this testing was done strictly to measure which assistant is the most knowledgeable.

The overall winner was Google Assistant on a smartphone. The latter attempted to answer the most questions (in other words, it said things like "Sorry, I don’t know that" fewer times than the competition) and had the highest percentage of "fully and correctly answered"questions. Cortana on the the Invoke was second in the percentage of correct answers, followed by Google Assistant on Google Home, Alexa and then Siri. Back in February, Apple's virtual assistant finished last in a study that was limited to assistants from smart speakers only.

Comparing the 2018 test to last year's results, only Cortana on the Invoke showed an improvement in the percentage of correct answers (86% to 92.1%). Google Assistant on a smartphone was not part of the study last year. While Alexa's percentage declined, it did increase the number of questions it attempted to answer by 2.7 times over last year. Cortana had the lowest percentage of flat out incorrect answers, followed by Google Assistant on both a smartphone and on a Home smart speaker. All of those assistants were wrong well under 1% of the time. On the other hand, the percentage of wrong answers spoken by Alexa rose sharply from just under 1% in 2017 to over 3% this year. Siri answered incorrectly about 3% of the time last year, and a bit over 3% this year.

Google Assistant on a smartphone had the largest number of answers that were attributed to a third party source. This happened on over half of its responses. Alexa had the fewest number of answers sourced from a third-party, while the Invoke's Cortana showed the only year-to-year decline in this category. While Siri relied on a third party source only a bit over 20% of the time, it had the largest year-to-year increase. Alexa had the largest percentage of questions that it answered with a joke, followed by Siri, which had a big decline in this category.

This test seems to confirm what has become obvious; Siri is losing ground quickly. The test also tells us that perhaps Alexa has become a bit overrated and Cortana underrated. It also reveals that Google Assistant is actually the smartest virtual personal assistant at this point in time.


source: StoneTemple

FEATURED VIDEO

28 Comments

haikallp

Posts: 319; Member since: Feb 10, 2012

Talking about voice assistant. I hate Bixby's voice recognition. Google Assistance is way ahead in terms of recognizing my voice. Granted, Bixby doesn't havw UK English support.

umaru-chan

Posts: 298; Member since: Apr 27, 2017

What happened to legendary voice assistant bixby? Why it is not included in this comparison. I sure bixby with the backup from Samsung would have easily win this comparison.

miketer

Posts: 460; Member since: Apr 02, 2015

Thanks for the good laugh. That's a wonderful start for my weekend and I owe it to you!

iushnt

Posts: 2998; Member since: Feb 06, 2013

They would have been ahead of Siri anyways.

NateDiaz

Posts: 1067; Member since: Mar 03, 2018

Let Samsung release a stable version first.

darkkjedii

Posts: 30341; Member since: Feb 05, 2011

That’s actually true.

darkkjedii

Posts: 30341; Member since: Feb 05, 2011

Where Bixby excels, is command execution. No assistant currently out, is as deeply integrated in command execution as Bixby is. It has its strong point as well as low, but it works well. We know you wanna troll, but as an avid iOS user as well as android, I can honestly say, I’d rather Google AI, and Bixby over Siri. My Note 8 has the first 2, and my iPad Pro has Siri...I speak from experience...you just troll.

ZEUS.the.thunder.god

Posts: 1066; Member since: Oct 05, 2011

This.

Sammy_DEVIL737

Posts: 1352; Member since: Nov 28, 2016

Does this changes the fact that Siri is just a JOKE compared to other assistants?

worldpeace

Posts: 2968; Member since: Apr 15, 2016

GA in first place and Siri in last place? Wow, what a surprise /s

Skizzo

Posts: 396; Member since: Jul 14, 2013

This was a surprise to exactly no one, haha. Tests like these have been done countless times, by countless people, so Google Assistant taking first place was no surprise, however, I am presently surprised by Cortana I must say.

Scott93274

Posts: 6018; Member since: Aug 06, 2013

Cortana had the misfortune of being associated with a platform that was on its way out. I've personally never used it, but there were aspects of it that I thought Microsoft did better than others. For me though, I still dig Google Assistant. Though maybe I might give it a shot on my desktop just to see how it compares.

Cat97

Posts: 1532; Member since: Mar 02, 2017

The smartest assistant is the one that has the most data about you. Of course Google wins. But is this a good thing ?

bucky

Posts: 3759; Member since: Sep 30, 2009

Was gonna say the same thing. No feeding skynet. That said, Siri still sucks.

ZEUS.the.thunder.god

Posts: 1066; Member since: Oct 05, 2011

Do you know any other way a smart assistant can be smart without your data?

Linn_Drum

Posts: 79; Member since: Mar 21, 2018

Lmao @ Siri coming last. Surprised? Of course not.

Skizzo

Posts: 396; Member since: Jul 14, 2013

Supposedly Apple is also aware of this finally and is planning on investing a ton of resources into making Siri competitive again, now that it got caught resting and was overtaken by nearly everyone. We'll have to wait and see what happens, but if this is true, that only means that Google and Amazon aren't just gonna sit on their ass and let Apple overtake them either. Should be good for the future of smart assistants in general. Competition is great for all of us!

Joosty

Posts: 457; Member since: Mar 14, 2013

This was always true. Siri blows.

antroid

Posts: 339; Member since: Jan 24, 2018

I was shocked when Cortana came second just after Google Assistant.

therealestmc

Posts: 678; Member since: Jul 23, 2012

Cortana is actually pretty good. Google better watch for Microsoft in the AI department. They're not too far behind Google. Apple is lollygaging with all the privacy nonsense.

Rager722

Posts: 562; Member since: Jan 30, 2013

Lmao Alexa compared Tiger woods to an ACTUAL cheetah!

buccob

Posts: 2942; Member since: Jun 19, 2012

Supposedly fail for Alexa, but that was a huge win in my opinion! I laughed out loud

therealestmc

Posts: 678; Member since: Jul 23, 2012

Why are you telling us stuff we're already knew about?

Scott93274

Posts: 6018; Member since: Aug 06, 2013

I've had a very poor opinion of Virtual Assistants in the past, and my frequent comments in the past have reflected that opinion. But my opinion on that has changed significantly as I've slowly adapted to using them more and more frequently. I've also purchased my first house which will be completed in late July/early August and all one of the things I'm most excited about is all the home automation I can put into the home. the house comes standard with loads of smart home features like the Ring Doorbell, Honeywell Thermostat, Skydrop sprinkler system, And even the garage door opener is Wi-Fi connected... though I'm not sure why that last one is necessary. But yeah, I'm looking forward to putting my virtual assistant to full use in the very near future.

Sammy_DEVIL737

Posts: 1352; Member since: Nov 28, 2016

Congratulations Scott93274 for your very first house.

Scott93274

Posts: 6018; Member since: Aug 06, 2013

Thanks! Yeah, I'm damned tired of renting.

darkkjedii

Posts: 30341; Member since: Feb 05, 2011

Congrats on the home Scott.

Scott93274

Posts: 6018; Member since: Aug 06, 2013

Thanks Dark!

Latest Stories