Could Samsung's Bixby be getting super-smart real soon? Samsung wins top scores in two AI competitions

Could Samsung's Bixby be getting super-smart real soon? Samsung wins top scores in two AI competitions
With smartphone hardware becoming quite powerful and reaching a bit of a plateau in recent years, manufacturers have been turning to all sorts of different features and solutions in order to keep evolving their products, make them more useful each consecutive year, and keeping the industry alive. Software is a big deal and AI is getting a ton of attention. All phones get virtual assistants and everyone is striving to make theirs smarter, faster, and more useful to the consumer.

Samsung's Bixby launched to a lukewarm reception with the Galaxy S8. Users were not pleased with the assistant's basic features, but the company remains committed to developing the assistant continuously and pushing it on users with an actual hardware button on all of its top-tier smartphones.

Early in 2018, Samsung said that — now that Bixby is out in enough hands — the company will focus on evolving it and making it an exceptional AI assistant. And, with Samsung's ConZNet (Context Zoom-in Network ) — a new software spawned by Samsung AI Center — winning two top global machine reading competitions, it may just be keeping its word.

Microsoft's MARCO (MAchine Reading Comprehension) and the University of Washington's TriviaQA are machine reading comprehension competitions, which task AI algorithms with processing actual user queries taken from various Q&A samples, then providing an answer. Additionally, both contests will have participants' AI write research-based documents, such as news articles or blog posts.

In MS MARCO, the AI is tasked with answering a random user query, plucked from Microsoft's Bing search engine. The program is presented with ten web documents and then is asked to compose an answer, which is relevant to the user query, but also reads and sounds as if it had been written by a human.

TriviaQA is a reading comprehension dataset with over 650 000 question-answer-evidence groups. It has complex, compositional questions, with variable lexical qualities, and requires cross-sentence reasoning to find answers. The AI is required to understand the question asked, present a short and to-the-point answer, and then give us the evidence (sources used) for said answer. TriviaQA has been tested with two baseline algorithms — a basic AI and a state-of-the-art neural network. The two of them had a performance rating of 23% and 40%, respectively, while a human's performance in said test is still 80%. So, AI still has some growing to do.

Samsung's ConZNet has ranked first in both competitions. However, it's worth noting that it held first place in MARCO from the 14th of June to the 4th of July, when it was beat by a VNET Baidu NLP algorithm. As for TriviaQA, the exact score the AI got is

So, what does that mean for the end user? Jihie Kim, Head of Samsung Research's Language Understanding Lab says that her department has already held open events, where they discussed the use of AI with engineers from both the home appliance and smartphone departments. And, since Samsung is pretty big on developing its IoT ecosystem with Bixby at the heart of it, it's easy to imagine that was a big point of conversation.

Additionally, customer service departments were also very interested in how ConZNet can be used to enhance chatbot usability and helpfulness.

At this point, however, we can't know how long it will be before we see the upgrade make it down to consumer products.

sources: Samsung (1, 2); MS MARCO



17. Nonames

Posts: 2; Member since: Jul 09, 2018

It struggles when i tell bixby to call a restaurant which i say by name... meaning instead of saying what is the phone number for restaurant pizza hut or whatever i say call the nearest pizza hut. Actually i just tested and neither example works. It also doesn't work when i say navigate to pizza hut. But sometimes it does open google maps for some queries. The inconsistency is the problem

7. Jason2k13

Posts: 1491; Member since: Mar 28, 2013

As of now, Google and siri is definitely better than Bixby... Samsung is still playing catchup.

8. sissy246

Posts: 7129; Member since: Mar 04, 2015

Agree with the Google assistant being the best out, but Bixby is way better the Siri.

13. NateDiaz

Posts: 1094; Member since: Mar 03, 2018

Bixby sucks.

14. VasiliS7

Posts: 205; Member since: Jan 10, 2017

How ever try to make an AI algorithm to think better than you... I am sure you make it like a Boss.

19. NateDiaz

Posts: 1094; Member since: Mar 03, 2018

Oh stfu it can't even show scorecards, that's basic stuff.

6. TechSceptic

Posts: 1156; Member since: Feb 05, 2018

This is pretty impressive stuff from Samsung's Bixby. I honestly didn't think it was that developed yet, but it's clearly looking like Samsung is making quick progress, as a result of their commitment to it. It will pretty damn impossible to de-throne Google Assistant tho, but it could possibly challenge Alexa's second place, as Samsung produces a lot of products that could benefit and expand the Bixby circle of influence.

5. cmdacos

Posts: 4429; Member since: Nov 01, 2016

Don't really care about results for random queries as that's what google assistant is for. On device management is where I use Bixby regularly and it is untouchable in that regard. Looking forward to it getting even better.

2. apple-rulz

Posts: 2198; Member since: Dec 27, 2016

In its current form Bixby sucks, however it does seem Samsung is determined to make Bixby a contender, therefore it’s entirely plausible Bixby 2.0 will catch up to and then surpass Siri, Cortana and Alexa. Google though-not happening when it comes to sheer information/navigation and getting it quickly, but if Samsung can continue to leverage making Bixby useful in device control like they currently do, that I could see as a real selling point.

3. NateDiaz

Posts: 1094; Member since: Mar 03, 2018

Man bixby is undoubtedly the worst assistant ever created. Try asking "FIFA scorecard" and it opens Samsung Pay like wtf Last time when I asked Spain vs Portugal, showed me results about best western European country to move in TechSceptic as usual didn't believe I had to show him screenshots, well he didn't reply.

9. Papa_Ji

Posts: 930; Member since: Jun 27, 2016

Baby... why were you trying to use Bixby on your iPhone? iphone's are not made for smart things like Bixby. Stick with your crappy Siri on your CrapX. If you really want feel smart then... TRY BIXBY ON GALAXY (S or Note).

11. NateDiaz

Posts: 1094; Member since: Mar 03, 2018

I own everything, including you.

10. Macready

Posts: 1834; Member since: Dec 08, 2014

Just did and it worked perfectly fine (tested your example). In fact, since the last update (last week), the annoying 3 second delay is gone too and results are now instant. I still like that I can fully change every setting in my phone with it. That's where it had a head start.

12. NateDiaz

Posts: 1094; Member since: Mar 03, 2018

I won't lie for some s**t like that. Check out both pictures here

15. Macready

Posts: 1834; Member since: Dec 08, 2014

I didn't say you lied, I said it works fine here. Which is a testament to the fact that pronunciation matters and some AI's work better with certain pronunciations than others and I bet vice versa. A screenshot of a fail for 1 person tells us exactly nada, nothing. Conversely, the fact that it works for me, tell you nothing either, apparently. But it does show that there are examples of both (failure and success) available.

16. Nonames

Posts: 2; Member since: Jul 09, 2018

It worked just great for me. Literally pulled up the webpage in samsung browser with the latest results. Using note 8 with oreo. Not sure why it isn't working for you

18. uberzero

Posts: 53; Member since: Jul 07, 2018

Interesting. A question, did you teach Bixby to be aware of your pronunciation? Recently Bixby gained support of TheScore app featuring all needed information, did you recently try it? You claimed Bixby is the worst assistant ever yet you knew it is just a year old and was designed to assist device like smartphones in this case.

4. TechSceptic

Posts: 1156; Member since: Feb 05, 2018

I didn't think i would say this, but i pretty much agree with you, apple-rulz. Google assistant is the AI champ for now, and then with a second place to Alexa because of the fact that it is present in as many products as Google Assistant (possibly more?), but it just isn't as good as GA. Bixby will have some trouble getting into as many products as those two, but since it's Samsung that's backing it, and seems committed, it will be possibly to change that, as Samsung produces so many products. It's a strength that they have over the other two. It will take time tho, quite a bit of it.

20. uberzero

Posts: 53; Member since: Jul 07, 2018

When it comes to assist a device, Bixby proved to be the best by far especially with the ability to teach multiple commands. None other AI Assistants come close in that department. From day one, Samsung made clear the goal is not to compete with Google Assistant when it comes to online search considering its age. Both complements each other.

1. JMartin22

Posts: 2429; Member since: Apr 30, 2013

They kind of slandered their marketing efforts with Bixby in the minds of some tech enthusiasts by trying to superimpose it on their users with the whole dedicated hardware key and such. But it’s turning out to be a fantastic software piece; even if it’s late to the table

Latest Stories

This copy is for your personal, non-commercial use only. You can order presentation-ready copies for distribution to your colleagues, clients or customers at or use the Reprints & Permissions tool that appears at the bottom of each web page. Visit for samples and additional information.
FCC OKs Cingular's purchase of AT&T Wireless