Siri gets pummeled by Google Assistant in new digital assistant 'IQ test'

Siri gets pummeled by Google Assistant in new digital assistant 'IQ test'
AI and AR seem to be emerging as the key battlefields for today’s tech industry leaders aiming to become tomorrow’s innovators, with complex deep learning, artificial neural networking, natural language processing, and real-world augmentation methods likely to separate the true trailblazers from the doomed trend followers.

Of course, when it comes to a hot area of research and development like AI-powered virtual assistants, being first is not everything. Apple’s Siri made its iOS debut way back in 2011, failing however to progress at an anticipated pace as late market entrants Alexa and Google Assistant rapidly turned up the heat. 

Numerous recent surveys and in-depth research papers have proven at least Google’s voice agent is superior to Siri in many crucial ways, either displaying a better understanding to queries in general or providing more useful answers to specific questions.

The latest such report focuses specifically on Siri, Google Assistant, Alexa and Cortana’s comparative capabilities on smartphones rather than smart speakers or other smart home-controlling devices, declaring a clear overall winner.

Incredibly enough, a Pixel XL-tested Google Assistant was able to understand all 800 questions posed by a team of LoupVentures researchers and analysts. In contrast, Siri on OS 11.4 misunderstood 11 queries, which still results in an impressive 99 percent success rate, while Alexa and Cortana stood at “only” 98 percent, with 13 and 19 misinterpreted questions respectively.


It’s obviously important to point out Alexa and Cortana are not baked into a mobile operating system right now, supporting iPhones via third-party apps that are understandably not as optimized as their pre-installed rivals.

In other words, the Amazon and Microsoft-developed digital assistants have a perfectly acceptable excuse for only scoring 61.4 and 52.4 percent accuracy rates in terms of correctly answering questions. We can’t say the same about Siri, which lags behind Google Assistant, with accuracy scores of 78.5 and 85.5 percent respectively. 

That’s right, 85.5 percent of those 800 questions, tackling various “local”, “commerce”, “navigation”, “information”, and “command” topics, were not only comprehended, but correctly answered by the Pixel XL’s pre-loaded voice assistant. That’s simply mind-boggling, and we can’t wait to see how much better Google can get in areas like natural language processing.

source: LoupVentures

FEATURED VIDEO

13 Comments

1. iPhoneFanboy

Posts: 286; Member since: Apr 21, 2018

For a company that doesn't collect droves and droves of personal data on you, that close of a gap between Google and Apple is insanely impressive, especially when Google dumps everything they have into their assistant while Apple is just chillen, lmao. I'm more worried about the Google and Amazon gap, as Amazon's Echo devices are the most used and popular digital home assistants right now. Gotta bait and click though to draw in those drooling, foaming at the mouth, Android bots to the comment section.

7. GreenMan

Posts: 2697; Member since: Nov 09, 2015

The data generated from search engine queries is crucial for training an AI; and I, for one, can't fault Google for this. And for that purpose; Microsoft has Bing, Google has Google (Duh!) while Apple and Samsung have nothing. Google collects 3.5 billion searches per day alone and 1.2 trillion searches per year worldwide and Apple simply doesn't have access to such vast amount of raw data; and you better believe Apple wish for it every single day to be competitive in The AI department! No surprise to see Siri stooping to the level of Bixby; nothingness in other words. Apple either has to invest heavily in R&D; which was the case back in Mr. Jobs days or just... Collect revenues; make investors super rich without worrying about technological advancements. The best thing Apple did with the iPhone lineup in the past 5 years is 3D Touch and even that technology is on the verge of extinction. Go figure or go fish, hombre. G'Day!

8. apple-rulz

Posts: 2112; Member since: Dec 27, 2016

“No surprise to see Siri stooping to the level of Bixby; nothingness in other words.” Siri came in a pretty close second to Google, while Cortana and Alexa rounded out the other contenders. Bixby wasn’t even tested. You have failed in your attack on Apple.

13. sgtdisturbed47

Posts: 966; Member since: Feb 02, 2012

Not sure why the article title says they got "pummeled" considering how poorly Cortana did. Very misleading.

12. Phullofphil

Posts: 1785; Member since: Feb 10, 2009

You are obviously biased toward google and you will fit everything into your argument. You are wrong. Not 100% but mostly wrong.

11. lyndon420

Posts: 6785; Member since: Jul 11, 2012

Apple is gunning for your data, so don't be fooling yourself.

2. umaru-chan

Posts: 358; Member since: Apr 27, 2017

What happened to the legendary voice assistant Bitchby I mean Bixby. Why it's not included in this test? Or is it not even worth considering to test? LMAO.

3. iPhoneFanboy

Posts: 286; Member since: Apr 21, 2018

Bixby got transferred to another department, refrigerators LOL

5. apple-rulz

Posts: 2112; Member since: Dec 27, 2016

Ha ha good one, +100 for you! :)

4. Whitedot

Posts: 809; Member since: Sep 26, 2017

In fact there is no video recording of actual test, results should be taken with a pinch of salt. I am not worried Google is the best at the moment but Siri is one drunk AI assistant right now.

6. Tipus

Posts: 868; Member since: Sep 30, 2016

iZombies got so butthurt. :))))

9. talon95

Posts: 998; Member since: Jul 31, 2012

Click bait title? Apple didn't do too bad for the lack of real effort they put into Siri. Bixby is just a process automation tool, it lets you verbalize phone actions for the physically impaired, it's not much of a search assistant. The vision tool is interesting, but I still don't want any of it.

10. Reybanz88

Posts: 97; Member since: Jul 28, 2016

Bixby actually works well! I wish they would have tested it against the others.

Latest Stories

This copy is for your personal, non-commercial use only. You can order presentation-ready copies for distribution to your colleagues, clients or customers at https://www.parsintl.com/phonearena or use the Reprints & Permissions tool that appears at the bottom of each web page. Visit https://www.parsintl.com/ for samples and additional information.