How Google Now was built (and a bit on Siri too)

This article may contain personal views and opinion from the author.
How Google Now was built (and a bit on Siri too)
You'll probably notice pretty quickly, but Google Now is a product/platform that really excites us. We can't say for sure that it will live up to the potential, but if it can, it could be something pretty special in the new field of intelligent push. Of course, Google didn't build this product alone, there was a long, strange journey to get here. Where Siri was a product that had an identity when Apple bought it and reimagined it for iOS, Google Now has been something of a Frankenstein monster pieced together from a number of products - some built in-house, some acquired, and some built by personnel hired specifically for the job.

There is a bit of an argument about this whole process, with some claiming that Google Now is better than Siri because it was built "from the ground up" by Google (which isn't wholly true), whereas Apple just bought Siri (which also isn't quite correct). It's all a silly argument based on geek pride, but it did get us looking into the process by which these products came to be what they are. 

Of course, what these products are is something of a trouble spot as well, because people insist on pulling both under the heading of "personal assistant", and trying to compare them as similar. Such comparisons fall flat for us because "personal assistant" is too general a heading. A bike and a jet are both "transportation devices", but that doesn't mean it's a good idea to compare them. In this case, Siri is a tool for finding information and organizing your life, which is based on active participation. Google Now is a tool not only for finding info and organizing your life, but more importantly it is a platform which will passively learn your behavior and interests in order to have relevant information ready for you, without the need for actively searching. Sure, there is overlap in what these two products do, but that doesn't mean comparisons are necessary, or even useful.  

We can't really get behind the argument that one way is better than the other, because it quickly becomes a slippery slope. The problem is that at some point, an acquired product can't be attributed to the original creators anymore, even when the original creator comes along in the acquisition. The prime example for us is Android itself. Android was created by the Android team headed by Andy Rubin, then acquired as a fully functioning product by Google. Andy and the team still work for Google, but Android has become a wholly Google product (ignoring forks and NGAs, of course). Similarly, Siri was bought by Apple as a fully functioning product, but to say that was the same app that now anthropomorphizes the iPhone just isn't correct.

The bit about Siri

As a standalone app back in 2010, Siri could make restaurant reservations, book movie tickets, get a taxi, find info or weather. Some of the functionality had to be removed from the app when Apple bought it, because Apple wasn't going to use services like Bing, Yahoo, or Google for search results, but in place of that functionality, Siri got deep integration into iOS, meaning options for scheduling, reminders, calling/messaging contacts, playing music, and dictation. As Siri matures, it is gaining back some of the partnerships that Apple ditched, but gaining more valuable partnerships along the way. Although key partnerships with services like OpenTable and MovieTickets are still missing, Yelp and Wikipedia are in the iOS version of Siri, and the biggest addition is in the partnership with Wolfram Alpha as a big knowledge base.

The point is that for better or worse, Siri is nowhere near the same app that it was when Apple purchased it. Sure, the backbone of voice recognition powered by Nuance is still at the center of Siri, but what the app can do has changed dramatically. What we see in iOS is no longer the product of the Siri developers simply purchased by Apple and stuck into iOS, this is an Apple product with features and uses dictated (no pun intended) by Apple.

Now... Google Now

On the other hand, Google didn't buy one singular product and transform it for Google Now. As we said, this product/platform is more of a Frankenstein monster made up of a number of acquisitions, hires, and in-house development that can be traced back at least 5 years. In many ways, Google Now is an initiative similar to Google+, which is intended to be a platform to unify a number of existing Google products and services, including search, places, travel, suggestions, and of course speech recognition.

Where Apple has partnered with Nuance and its Dragon software for the speech recognition behind Siri, Google has taken the last 5 years to build its own speech database. That project began with the hiring of Mike Cohen in 2007. Cohen was actually a co-founder of Nuance. He spearheaded the Google initiative to build a speech recognition database, which began collecting data through GOOG-411 in 2007 - the free place information service Google once ran - before getting a ton of data through dictation and voice actions on Android over the past 2 years or so, not to mention contributions from Google Voice and its terrible voicemail transcriptions (which have of course gotten better as Google's voice recognition database has grown). Cohen left Google earlier this year, but the speech recognition work will go on.

On the other side of the voice coin, Google also needed a nice voice to respond to users just like Siri does. Luckily, Google had purchased Phonetic Arts back in December of 2010. That purchase was originally made to make robo-voices sound better in Google Translate, as well as the accessibility text-to-speech option found in Android. Phonetic Arts did a lot of work making robo-voices sound better, and that technology has come in very handy with Google Now, because it does sound like the assistant voice is smoother than Siri. As the voice database was growing to a sufficient level, Google also needed to beef up search results.

If you hadn't noticed, Google has been slowly transitioning from a search engine into more of a knowledge provider. More and more when you search, you'll not only get the search results, but an actual answer to your query, such as word definitions, sports scores, movie showtimes, and flight information. While some of that knowledge has come through partnerships and searchable content, big pieces of the Google Now puzzle came through acquisitions

Flight information is gathered via searchable results from, but eventually, Google could move to its own information repo which it purchased in the form of ITA Software (acquired July 2010). ITA Software now powers Google's Flight Search for tickets, and that could easily become part of Google Now's results. Place results, which are a huge part of any mobile search product (because more and more "mobile" really just means "local") have also gotten a boost from a couple choice acquisitions. Google hired the entire team behind (May 2010), an online travel guide, in an effort to give local results a boost. Then, more recently, moves to purchase Clever Sense (December 2011), makers of local recommendation app Alfred, and restaurant guide Zagat (September 2011). Google notably had a falling out with Yelp, and the purchase of Zagat was the answer to that loss of local data.

Possibly the biggest acquisition of all was Metaweb, which Google purchased in mid-2010. Apple of course has to partner with information services like Wikipedia, Wolfram Alpha, and Yelp, but it's Google's business to provide organized data and knowledge to users, and Metaweb is the future of that endeavor. Metaweb just recently made its debut within Google in the form of Google's Knowledge Graph and Semantic Search. Now, rather than searching for keywords, we're searching semantic objects, which means there should be better differentiation between homographs (so Google now knows whether you're searching the meteorological "thunder" or the NBA Thunder.) These results have shown up in the Knowledge Graph block to the right of standard results.

These blocks of information have all made the pretty easy transitions into being "cards" in the Google Now UI. And, on that topic, we have to mention the hire of Matias Duarte in May 2010, who has been one of the strongest forces behind getting the traditionally engineer-oriented Google to make well-designed products. Matias has always loved the "card" metaphor, and so we feel pretty safe in assuming the cards in Google Now were his idea. For those of you that ever used webOS, you know that cards were a central metaphor of that platform (designed by Matias), then we saw the multitasking menu of Android change to cards when Matias arrived and now we get the cards of information as part of Google Now. 

The information side of Google Now is pretty strange to follow, but the learning side of the platform is actually pretty easy, because it uses two things that Google has been cultivating for a long time now: web history and location services. Web history encompasses both your search history, and any visited pages from those search results, where location services is obviously looking at where you spend your time and where you travel. If you have an Android phone with location services enabled, Google probably already knows where you live and where you work, and all of that data can easily be found on your Latitude dashboard (note: this is all private data that only you can see, Google doesn't share it.) When you combine that location data with your personal calendar, and Google's ever expanding navigation options for driving, public transport, walking, and biking, you get a pretty powerful data set that only needs a bit of a nudge to predict what information you may need at a given time and place. 


We can't say that Google Now was completely built in-house by Google any more than Apple built Siri in-house. Both took very different development paths because of the nature of the product, the nature of the respective companies, and the resources available to each company. Google is a software company that has been working hard for years to learn who you are (as an abstract entity, not as an individual), while Apple doesn't have that information to leverage and had to build a product that can essentially start to harvest that data. As we said before, we fully expect Siri to continue its transformation, and eventually become an intelligent push service like Google Now, but whatever the end point, the journey of Siri becoming a fully realized product is much cleaner than Google Now. Apple has always tended to work from a centralized format, creating one singular product designed for a purpose within the larger whole of the Apple ecosystem, while Google is a far more messy affair.

Google captures various services and products for a number of different reasons, and may not always have an overarching plan to bring everything together. Since Larry Page has stepped into the CEO office, that has begun to change, but it can still be a pretty long and windy road leading from products, acquisitions, and hires into a cohesive platform like Google Now. 



43. ispeakevo

Posts: 2; Member since: Jun 08, 2012

What's also interesting is how the author refers to Google road as being "messy" while saying Apple's is "cleaner"... Its this type of media baiting that helps perpetuate the Idea the idea and belief, apple is better, ahead, smoother, and more inventive. It doesn't matter what the voice sounded like, but, in the 80's IBM computers was talking back to us. And regardless of the amount of mistakes, in the late 90's my Motorola phone was recognizing my voice. And before an actual device labeled as an Iphone came to market, Google understood what I was saying. I'm just so tired of the blatant crap of bias misleading information that continues to flow from this venue of tech news.

38. dreign91

Posts: 15; Member since: May 17, 2012

Michael H I must say I feel a tad bit smarter every time I read your articles. I'm very excited for Google Now as well. Yes in the beginning there will be a learning curve, but with all the sources of info your phone can pull from (Gmail, currents, play store, search, places, etc) this has the potential to be legend....(wait for it).... dary! What will be truly awesome is when Google now is put together with the Google Glass project

45. alekhos

Posts: 2; Member since: May 17, 2012

Yeah...that would definitively shape a sheepish world, given the nature of Google Now, as explained by Michael. Think, already, with the proper training (this is kind of a A. I. software), Google Now can predict what info you need, but, as it was pointed out here, it would soon be telling you what to do and you would be like "Oh, ok I´ve forgotten that, thanks". Think that comes from your phone, which may be in your pocket, but maybe at some place not instantly reachable or even left over around (you may want to shower or have s** or something) and you might not get the notification instantly. Now think you´re wearing your glasses. How could you avoid being told most of the time what you should do or were you should be and how to get there fast? Of course you could choose NOT to pay attention...but the question is...when the time comes...will we "look to another side"?...wait, you can´t even do that! The thing is right before your eyes! And probably more sooner than later it will be sounding in your ears also...natural voice and all (perhaps a voice of your choice). Now get the whole picture...

32. satydesh

Posts: 71; Member since: Nov 22, 2011

android fans are really aggressive thats Power given by Open source to demand more!!!

24. good2great

Posts: 1042; Member since: Feb 22, 2012

what would be funny is if Google released an iOS version of Google you guys would be pissed huh? lol looking at how its broken down its not impossible to happen... lol

25. Sackboy117

Posts: 178; Member since: Oct 17, 2011

it would probably become #1 on the App Store and would most likely reach out to previous iPhone models.

33. SBrown

Posts: 9; Member since: Jul 02, 2010

I usually don't participate on these comment threads, but I must admit, that was an excellent civil comeback. +1

26. protozeloz

Posts: 5396; Member since: Sep 16, 2010

is quite posible at least from goggles side of things

34. MartyK

Posts: 1043; Member since: Apr 11, 2012

It wouldn't surprise anyone if Google did place this on IOS or Window phones, in fact most Android Users wouldn't be upset at all. Until, IOS Market dept and some crazy Ifan, start screaming out, "Google copy" and then Apple trieds to patent Google now.

37. remixfa

Posts: 14605; Member since: Dec 19, 2008

there are not enough hooks in the "google" apps on iOS to make it work the same. With Android, it comes with all the apps built in to the system. On iOS, not only would it take major app upgrades (like Maps for example.. huge difference between goog maps on android and iOS), it would also require that all those different Google apps be downloaded or preinstalled in iOS with additional hooks built into the OS itself (like Apple would allow that). And you think the jokes about iOS becoming more "android" with each update is bad now? Just imagine what kind of jokes would be slinging if iOS had a mandatory "google apps" suite on it. lol. those jokes would never end. :)

5. maxican16

Posts: 364; Member since: Sep 29, 2011

The important thing is what works best for each user. For me there is no question, it's Google Now. I don't want to have a conversation with my phone, nor do I want to ask the same question time and time again (just have it ready for me already!), or be given incorrect answers (which happens a third of the time on siri). Google Now is already incredibly functional and it will only improve. And when voice search doesn't have an answer, it gives me a list of Google results that only improve the more I use my phone. Take note Apple... THIS is true innovation.

4. sid07desai

Posts: 290; Member since: May 03, 2012

Google is more credible for Android than Apple for Siri. When Google bought Android, it was just in the beginning stages. And then they came up with improvements like Ice Cream Sandwich on their own. And then they created Google Now. Their own creation. Apple did little to make certain changes to Siri but overall it cannot be credited to Apple. As far as efforts are concerned, I think Google has done a better job. But as Michael mentioned in the above comment, the comparison is silly. We users can only wait and see how these evolve on their own.

31. jroc74

Posts: 6023; Member since: Dec 30, 2010

Agree 100%. At its core...Siri was already developed before Apple bought it. Google Now wasnt. Thats one of the big things about Apple that Apple, iPhone fanboys dont wanna admit. If Apple is innovative for buying companies, technology, apps and adding their twist to can anyone say Google isnt? It boggles the mind.

3. tedkord

Posts: 17519; Member since: Jun 17, 2009

But...I thought Google didn't even think of an intelligent talking assistant until Siri, then they just copied.

7. smartphonemad

Posts: 42; Member since: Jul 10, 2012

That's what Apple fans want people to believe :L I mean, with all the people that create ideas for google do you not think someone would have thought of it?

14. protozeloz

Posts: 5396; Member since: Sep 16, 2010

are sarcasms so hard to get these days?

16. MichaelHeller

Posts: 2734; Member since: May 26, 2011

This is why we need to standardize punctuation for sarcasm on the web. So many people miss it in the written word. Easiest is just bracketing punctuation to convey sarcasm, since it wouldn't need new keystrokes to be learned. ie Fanboys are so rational[!]

19. protozeloz

Posts: 5396; Member since: Sep 16, 2010

sarcasm standard would be kinda hard since there are a few keys used for sarcasm but my favorite is "/s" ie phonearena is full of rational fanboys /s

41. ardent1

Posts: 2000; Member since: Apr 16, 2011

>ie Fanboys are so rational[!] That would be British sarcasm and would be lost on most Americans. "In BrE, I think we generally describe remarks as ‘sarcastic’ when we’re saying the opposite of what we mean e.g. ‘Wow, that’s a surprise’ when something was very predictable, or ‘Nice weather, eh?’ when it’s pouring with rain, or ‘Punctual, as always.’ when someone who always comes late finally arrives. So sarcasm can be either nice (a funny joke) or nasty (an unkind remark) but some element of ‘saying the opposite of what you mean’ needs to be involved for something to be labeled sarcastic." (source:

29. jroc74

Posts: 6023; Member since: Dec 30, 2010

tedkord shoulda added 2 more "but"

40. ardent1

Posts: 2000; Member since: Apr 16, 2011

Unfortunately American don't employ sarcasm as well as the British. What people think is sarcasm really isn't sarcasm.

46. smartphonemad

Posts: 42; Member since: Jul 10, 2012

It is when you cant hear a persons tone

15. Sniggly

Posts: 7305; Member since: Dec 05, 2009

my apologies, man. I meant to thumb this comment up. It was a good snarky little comment. :)

27. tedkord

Posts: 17519; Member since: Jun 17, 2009

Yeah, it was snarkier than I intended. I've been working 13 days straight, 12 hours per day, 1 day off and start over, for several weeks now. I am exhausted, and maybe a little cranky.


Posts: 456; Member since: Nov 19, 2011

Siri isn't even the first virtual assistant. your point is pointless. Anyways, Apple bought Siri in 2010 as a response to Google voice dictation and voice actions that debuted in 2009, no apple voice competitor until late 2011, years later.. it took Apple two years to launch Siri... Google Now is an extension on every thing search and they didn't just whip that pure awesomness in 6 months... If they did? that is a company I like to back!!! Apple let Google run voice wild for years without a competing product and even longer without Navigation. I like my companies to react and innovate quickly not take years to offer MMS and copy n paste!

30. jroc74

Posts: 6023; Member since: Dec 30, 2010

lol! +1 to ya... I REALLY, REALLY like this article. Apple and Google took different paths for each product. Whoever says Google is copying Apple/Siri....needs to just play in traffic....sorry. Ok...I only 1/2 mean it...kinda. Articles like this ....Apple, iPhone fanboys need to really pay attention to. Apple isnt the only company that innovates. How hard is that to understand?

1. hateftotti

Posts: 234; Member since: May 03, 2011

Google Now>>>>>>>>>>>>>>>>>siri Just wait to see how apple want to mislead its fans again.

2. MichaelHeller

Posts: 2734; Member since: May 26, 2011

That's like saying TV is much better than radio. They are two different things that have different purposes, but have a small set of overlapping features. The comparison is silly.

6. Droid_X_Doug

Posts: 5993; Member since: Dec 22, 2010

I wonder if the evolutionary path doesn't reflect some of the philosophy of the O/S environments in which the products function? Siri is running in an environment that if there is going to be deep integration, requires the development work to be done by the iOS team. Conversely, given the more open environment of Android, more things can be done by 'outsiders'.

8. MichaelHeller

Posts: 2734; Member since: May 26, 2011

That's probably part of it. Google has more leeway to fold in external products as needed, and has APIs available for the rest. Surprisingly, there's no real Google Now API, although the other APIs like Directions, Places, and Calendar all feed into Google Now.

Latest Stories

This copy is for your personal, non-commercial use only. You can order presentation-ready copies for distribution to your colleagues, clients or customers at or use the Reprints & Permissions tool that appears at the bottom of each web page. Visit for samples and additional information.
FCC OKs Cingular's purchase of AT&T Wireless