How Google Now was built (and a bit on Siri too)

This article may contain personal views and opinion from the author.
How Google Now was built (and a bit on Siri too)
You'll probably notice pretty quickly, but Google Now is a product/platform that really excites us. We can't say for sure that it will live up to the potential, but if it can, it could be something pretty special in the new field of intelligent push. Of course, Google didn't build this product alone, there was a long, strange journey to get here. Where Siri was a product that had an identity when Apple bought it and reimagined it for iOS, Google Now has been something of a Frankenstein monster pieced together from a number of products - some built in-house, some acquired, and some built by personnel hired specifically for the job.

There is a bit of an argument about this whole process, with some claiming that Google Now is better than Siri because it was built "from the ground up" by Google (which isn't wholly true), whereas Apple just bought Siri (which also isn't quite correct). It's all a silly argument based on geek pride, but it did get us looking into the process by which these products came to be what they are. 

Of course, what these products are is something of a trouble spot as well, because people insist on pulling both under the heading of "personal assistant", and trying to compare them as similar. Such comparisons fall flat for us because "personal assistant" is too general a heading. A bike and a jet are both "transportation devices", but that doesn't mean it's a good idea to compare them. In this case, Siri is a tool for finding information and organizing your life, which is based on active participation. Google Now is a tool not only for finding info and organizing your life, but more importantly it is a platform which will passively learn your behavior and interests in order to have relevant information ready for you, without the need for actively searching. Sure, there is overlap in what these two products do, but that doesn't mean comparisons are necessary, or even useful.  

We can't really get behind the argument that one way is better than the other, because it quickly becomes a slippery slope. The problem is that at some point, an acquired product can't be attributed to the original creators anymore, even when the original creator comes along in the acquisition. The prime example for us is Android itself. Android was created by the Android team headed by Andy Rubin, then acquired as a fully functioning product by Google. Andy and the team still work for Google, but Android has become a wholly Google product (ignoring forks and NGAs, of course). Similarly, Siri was bought by Apple as a fully functioning product, but to say that was the same app that now anthropomorphizes the iPhone just isn't correct.

The bit about Siri

As a standalone app back in 2010, Siri could make restaurant reservations, book movie tickets, get a taxi, find info or weather. Some of the functionality had to be removed from the app when Apple bought it, because Apple wasn't going to use services like Bing, Yahoo, or Google for search results, but in place of that functionality, Siri got deep integration into iOS, meaning options for scheduling, reminders, calling/messaging contacts, playing music, and dictation. As Siri matures, it is gaining back some of the partnerships that Apple ditched, but gaining more valuable partnerships along the way. Although key partnerships with services like OpenTable and MovieTickets are still missing, Yelp and Wikipedia are in the iOS version of Siri, and the biggest addition is in the partnership with Wolfram Alpha as a big knowledge base.

The point is that for better or worse, Siri is nowhere near the same app that it was when Apple purchased it. Sure, the backbone of voice recognition powered by Nuance is still at the center of Siri, but what the app can do has changed dramatically. What we see in iOS is no longer the product of the Siri developers simply purchased by Apple and stuck into iOS, this is an Apple product with features and uses dictated (no pun intended) by Apple.

Now... Google Now

On the other hand, Google didn't buy one singular product and transform it for Google Now. As we said, this product/platform is more of a Frankenstein monster made up of a number of acquisitions, hires, and in-house development that can be traced back at least 5 years. In many ways, Google Now is an initiative similar to Google+, which is intended to be a platform to unify a number of existing Google products and services, including search, places, travel, suggestions, and of course speech recognition.

Where Apple has partnered with Nuance and its Dragon software for the speech recognition behind Siri, Google has taken the last 5 years to build its own speech database. That project began with the hiring of Mike Cohen in 2007. Cohen was actually a co-founder of Nuance. He spearheaded the Google initiative to build a speech recognition database, which began collecting data through GOOG-411 in 2007 - the free place information service Google once ran - before getting a ton of data through dictation and voice actions on Android over the past 2 years or so, not to mention contributions from Google Voice and its terrible voicemail transcriptions (which have of course gotten better as Google's voice recognition database has grown). Cohen left Google earlier this year, but the speech recognition work will go on.

On the other side of the voice coin, Google also needed a nice voice to respond to users just like Siri does. Luckily, Google had purchased Phonetic Arts back in December of 2010. That purchase was originally made to make robo-voices sound better in Google Translate, as well as the accessibility text-to-speech option found in Android. Phonetic Arts did a lot of work making robo-voices sound better, and that technology has come in very handy with Google Now, because it does sound like the assistant voice is smoother than Siri. As the voice database was growing to a sufficient level, Google also needed to beef up search results.

If you hadn't noticed, Google has been slowly transitioning from a search engine into more of a knowledge provider. More and more when you search, you'll not only get the search results, but an actual answer to your query, such as word definitions, sports scores, movie showtimes, and flight information. While some of that knowledge has come through partnerships and searchable content, big pieces of the Google Now puzzle came through acquisitions

Flight information is gathered via searchable results from, but eventually, Google could move to its own information repo which it purchased in the form of ITA Software (acquired July 2010). ITA Software now powers Google's Flight Search for tickets, and that could easily become part of Google Now's results. Place results, which are a huge part of any mobile search product (because more and more "mobile" really just means "local") have also gotten a boost from a couple choice acquisitions. Google hired the entire team behind (May 2010), an online travel guide, in an effort to give local results a boost. Then, more recently, moves to purchase Clever Sense (December 2011), makers of local recommendation app Alfred, and restaurant guide Zagat (September 2011). Google notably had a falling out with Yelp, and the purchase of Zagat was the answer to that loss of local data.

Possibly the biggest acquisition of all was Metaweb, which Google purchased in mid-2010. Apple of course has to partner with information services like Wikipedia, Wolfram Alpha, and Yelp, but it's Google's business to provide organized data and knowledge to users, and Metaweb is the future of that endeavor. Metaweb just recently made its debut within Google in the form of Google's Knowledge Graph and Semantic Search. Now, rather than searching for keywords, we're searching semantic objects, which means there should be better differentiation between homographs (so Google now knows whether you're searching the meteorological "thunder" or the NBA Thunder.) These results have shown up in the Knowledge Graph block to the right of standard results.

These blocks of information have all made the pretty easy transitions into being "cards" in the Google Now UI. And, on that topic, we have to mention the hire of Matias Duarte in May 2010, who has been one of the strongest forces behind getting the traditionally engineer-oriented Google to make well-designed products. Matias has always loved the "card" metaphor, and so we feel pretty safe in assuming the cards in Google Now were his idea. For those of you that ever used webOS, you know that cards were a central metaphor of that platform (designed by Matias), then we saw the multitasking menu of Android change to cards when Matias arrived and now we get the cards of information as part of Google Now. 

The information side of Google Now is pretty strange to follow, but the learning side of the platform is actually pretty easy, because it uses two things that Google has been cultivating for a long time now: web history and location services. Web history encompasses both your search history, and any visited pages from those search results, where location services is obviously looking at where you spend your time and where you travel. If you have an Android phone with location services enabled, Google probably already knows where you live and where you work, and all of that data can easily be found on your Latitude dashboard (note: this is all private data that only you can see, Google doesn't share it.) When you combine that location data with your personal calendar, and Google's ever expanding navigation options for driving, public transport, walking, and biking, you get a pretty powerful data set that only needs a bit of a nudge to predict what information you may need at a given time and place. 


We can't say that Google Now was completely built in-house by Google any more than Apple built Siri in-house. Both took very different development paths because of the nature of the product, the nature of the respective companies, and the resources available to each company. Google is a software company that has been working hard for years to learn who you are (as an abstract entity, not as an individual), while Apple doesn't have that information to leverage and had to build a product that can essentially start to harvest that data. As we said before, we fully expect Siri to continue its transformation, and eventually become an intelligent push service like Google Now, but whatever the end point, the journey of Siri becoming a fully realized product is much cleaner than Google Now. Apple has always tended to work from a centralized format, creating one singular product designed for a purpose within the larger whole of the Apple ecosystem, while Google is a far more messy affair.

Google captures various services and products for a number of different reasons, and may not always have an overarching plan to bring everything together. Since Larry Page has stepped into the CEO office, that has begun to change, but it can still be a pretty long and windy road leading from products, acquisitions, and hires into a cohesive platform like Google Now. 



1. hateftotti

Posts: 232; Member since: May 03, 2011

Google Now>>>>>>>>>>>>>>>>>siri Just wait to see how apple want to mislead its fans again.

2. MichaelHeller

Posts: 2734; Member since: May 26, 2011

That's like saying TV is much better than radio. They are two different things that have different purposes, but have a small set of overlapping features. The comparison is silly.

6. Droid_X_Doug

Posts: 5993; Member since: Dec 22, 2010

I wonder if the evolutionary path doesn't reflect some of the philosophy of the O/S environments in which the products function? Siri is running in an environment that if there is going to be deep integration, requires the development work to be done by the iOS team. Conversely, given the more open environment of Android, more things can be done by 'outsiders'.

8. MichaelHeller

Posts: 2734; Member since: May 26, 2011

That's probably part of it. Google has more leeway to fold in external products as needed, and has APIs available for the rest. Surprisingly, there's no real Google Now API, although the other APIs like Directions, Places, and Calendar all feed into Google Now.

9. Aeires unregistered

Both are personal assistants, to remind you of events and pull information you require. How is that different? The only big difference I see is one is far more intuitive than the other, but at the end of the day, they're both personal assistants.

11. MichaelHeller

Posts: 2734; Member since: May 26, 2011

There is huge variation in the space of "personal assistant". A to-do notebook can be considered a "personal assistant", so can an actual human helper. The term "personal assistant" is so general as to be useless. To go back to my original analogy, that's like saying TV, radio, MP3 players, etc are the same because they're all "media devices". A bike and a jet are both "transportation devices", but that doesn't make it a good idea to compare them.

17. Aeires unregistered

Except both are apps on a cell phone. Comparing a TV to a radio is silly, but comparing one app on a cell phone to another app on a cell phone that essentially does the same thing isn't silly.

18. MichaelHeller

Posts: 2734; Member since: May 26, 2011

GTAIII and Angry Birds are both cell phone apps that "essentially do the same thing" (entertain). Shall we compare them? How about Hookt and Skype (communication)? How good will those comparisons be? It all depends on how specific your comparison criteria are. If you want to compare "personal assistants", you won't get very good comparison points because of the huge variation in methods and features across the thousands of apps that fall in that category. For example, you could compare WriteRoom and Office, because they are both "productivity tools" with an overlap on word processing, but the comparison is going to be pretty useless.

28. Aeires unregistered

How about just the two that the article is about. I'll be blunt with this so there's no misunderstanding the point. I don't think Apple put near as much work into Siri as you make out. Apple purchased a finished product and tweaked it to fit their needs, hardly deserving the credit of making Siri. As for Google, a Frankenstein approach isn't that too far off to imagine. Google is well known for trying crazy things and sometimes they nail it, like with Google Now. If you look at everything that went into Now, it's remarkable, and that shouldn't be downplayed because they purchased some portions and hired the talent needed to pull it off. All the resources still had to be put together, which was no small feet. But at the end of the day, they're both personal assistants. The graphics in this article says "Hello, I'm Siri, your new personal assistant."

10. itiswhatitis

Posts: 423; Member since: Jan 23, 2012

If comparison is silly then i guess 99% of the World's population is silly!

12. MichaelHeller

Posts: 2734; Member since: May 26, 2011

This is why my job is difficult...

13. itiswhatitis

Posts: 423; Member since: Jan 23, 2012

I guess its like you said from one of you articles you must be throwing 'millions of pebbles' whatever it is good luck with that,its worth the try....

22. remixfa

Posts: 14605; Member since: Dec 19, 2008

ok.. how about this for a quick metaphore.. if you think of Google's dozens of services as "cities" unto themselves, they are much like the US on a map.. cities here and there and everywhere. What Google Now does is basically build information highways between all these "cities" so they can now work together. Google Now takes all the information from all these cities (Maps, Search, ect ect ect) and uses it to figure out what you want... and eventually it might know what you want before you do. It watches you while it watches the cities and learns over time. Its pretty much free to grab any information from nearly anywhere from the way its designed. Apple is the opposite. They have one massive city where everything is done. Siri was build IN to the city to work inside it. It does not go outside the city walls as designed. As Apple's city grows, Siri will grow with it. Siri takes all the information within Apple's city and keeps it ready for you when you need it. It however will never go beyond the scope of Apple's city. Some of the base features of both may be the same, but they operate completely different and they are built to do different things from each other. We have joked that Siri was the beginnings of Skynet. In all honesty, Google Now is way closer to Skynet that Siri will be.. maybe ever. Its always watching and learning what you do. Compared to that, Siri is a rockem-sockem robot.

23. good2great

Posts: 1042; Member since: Feb 22, 2012

beautiful explanation remixfa!!! this lets me know that you are a very unbiased person and at least understand how both OS's work... this definitely doesn't take away from Michael H. article you pretty much explained it to educate the trolls in good article Michael H. there are a few readers in here that actually enjoy the content of this site than coming here to crack jokes and bash/harass each other about which OS is better...

36. remixfa

Posts: 14605; Member since: Dec 19, 2008

Thanks :) Yea, I never meant to take away from michael. only help reduce his headache from trying to make people understand. If it helps someone, then it was worth typing it up. I dont know that true trolls can be educated though. Isnt that against the trolling Alma mater? lol

42. thinking

Posts: 130; Member since: Jan 19, 2012

This is not about which OS is better. Siri/Now are not OSs. They are personal assistants that try to answer your questions. So, I very firmly feel that they intend to do the same thing. They may be doing it differently. They're still trying to do the same thing. So, the comparison is bound to be there.

20. MartyK

Posts: 1043; Member since: Apr 11, 2012

I disagree with ALL; the difference is this: Ifan feels everything Apple market dept says/ puts out is pure-first -time-ever-magic (from Touch screen to Siri). Whereas, the rest of the world KNOWS that every thing is ( in Mike words) peice together, heck the whole COMPUTER language is peice together so there is really nothing new. Except when it comes to Ifan and Apple, then it's brand new, SOMEBODY have to call them out on it.

3. tedkord

Posts: 17415; Member since: Jun 17, 2009

But...I thought Google didn't even think of an intelligent talking assistant until Siri, then they just copied.

7. smartphonemad

Posts: 42; Member since: Jul 10, 2012

That's what Apple fans want people to believe :L I mean, with all the people that create ideas for google do you not think someone would have thought of it?

14. protozeloz

Posts: 5396; Member since: Sep 16, 2010

are sarcasms so hard to get these days?

16. MichaelHeller

Posts: 2734; Member since: May 26, 2011

This is why we need to standardize punctuation for sarcasm on the web. So many people miss it in the written word. Easiest is just bracketing punctuation to convey sarcasm, since it wouldn't need new keystrokes to be learned. ie Fanboys are so rational[!]

19. protozeloz

Posts: 5396; Member since: Sep 16, 2010

sarcasm standard would be kinda hard since there are a few keys used for sarcasm but my favorite is "/s" ie phonearena is full of rational fanboys /s

41. ardent1

Posts: 2000; Member since: Apr 16, 2011

>ie Fanboys are so rational[!] That would be British sarcasm and would be lost on most Americans. "In BrE, I think we generally describe remarks as ‘sarcastic’ when we’re saying the opposite of what we mean e.g. ‘Wow, that’s a surprise’ when something was very predictable, or ‘Nice weather, eh?’ when it’s pouring with rain, or ‘Punctual, as always.’ when someone who always comes late finally arrives. So sarcasm can be either nice (a funny joke) or nasty (an unkind remark) but some element of ‘saying the opposite of what you mean’ needs to be involved for something to be labeled sarcastic." (source:

29. jroc74

Posts: 6023; Member since: Dec 30, 2010

tedkord shoulda added 2 more "but"

40. ardent1

Posts: 2000; Member since: Apr 16, 2011

Unfortunately American don't employ sarcasm as well as the British. What people think is sarcasm really isn't sarcasm.

46. smartphonemad

Posts: 42; Member since: Jul 10, 2012

It is when you cant hear a persons tone

15. Sniggly

Posts: 7305; Member since: Dec 05, 2009

my apologies, man. I meant to thumb this comment up. It was a good snarky little comment. :)

27. tedkord

Posts: 17415; Member since: Jun 17, 2009

Yeah, it was snarkier than I intended. I've been working 13 days straight, 12 hours per day, 1 day off and start over, for several weeks now. I am exhausted, and maybe a little cranky.


Posts: 456; Member since: Nov 19, 2011

Siri isn't even the first virtual assistant. your point is pointless. Anyways, Apple bought Siri in 2010 as a response to Google voice dictation and voice actions that debuted in 2009, no apple voice competitor until late 2011, years later.. it took Apple two years to launch Siri... Google Now is an extension on every thing search and they didn't just whip that pure awesomness in 6 months... If they did? that is a company I like to back!!! Apple let Google run voice wild for years without a competing product and even longer without Navigation. I like my companies to react and innovate quickly not take years to offer MMS and copy n paste!

30. jroc74

Posts: 6023; Member since: Dec 30, 2010

lol! +1 to ya... I REALLY, REALLY like this article. Apple and Google took different paths for each product. Whoever says Google is copying Apple/Siri....needs to just play in traffic....sorry. Ok...I only 1/2 mean it...kinda. Articles like this ....Apple, iPhone fanboys need to really pay attention to. Apple isnt the only company that innovates. How hard is that to understand?

Latest Stories

This copy is for your personal, non-commercial use only. You can order presentation-ready copies for distribution to your colleagues, clients or customers at or use the Reprints & Permissions tool that appears at the bottom of each web page. Visit for samples and additional information.