How Google Now was built (and a bit on Siri too)

You'll probably notice pretty quickly, but Google Now is a product/platform that really excites us. We can't say for sure that it will live up to the potential, but if it can, it could be something pretty special in the new field of intelligent push. Of course, Google didn't build this product alone, there was a long, strange journey to get here. Where Siri was a product that had an identity when Apple bought it and reimagined it for iOS, Google Now has been something of a Frankenstein monster pieced together from a number of products - some built in-house, some acquired, and some built by personnel hired specifically for the job.

There is a bit of an argument about this whole process, with some claiming that Google Now is better than Siri because it was built "from the ground up" by Google (which isn't wholly true), whereas Apple just bought Siri (which also isn't quite correct). It's all a silly argument based on geek pride, but it did get us looking into the process by which these products came to be what they are. 

Of course, what these products are is something of a trouble spot as well, because people insist on pulling both under the heading of "personal assistant", and trying to compare them as similar. Such comparisons fall flat for us because "personal assistant" is too general a heading. A bike and a jet are both "transportation devices", but that doesn't mean it's a good idea to compare them. In this case, Siri is a tool for finding information and organizing your life, which is based on active participation. Google Now is a tool not only for finding info and organizing your life, but more importantly it is a platform which will passively learn your behavior and interests in order to have relevant information ready for you, without the need for actively searching. Sure, there is overlap in what these two products do, but that doesn't mean comparisons are necessary, or even useful.  

We can't really get behind the argument that one way is better than the other, because it quickly becomes a slippery slope. The problem is that at some point, an acquired product can't be attributed to the original creators anymore, even when the original creator comes along in the acquisition. The prime example for us is Android itself. Android was created by the Android team headed by Andy Rubin, then acquired as a fully functioning product by Google. Andy and the team still work for Google, but Android has become a wholly Google product (ignoring forks and NGAs, of course). Similarly, Siri was bought by Apple as a fully functioning product, but to say that was the same app that now anthropomorphizes the iPhone just isn't correct.

The bit about Siri

As a standalone app back in 2010, Siri could make restaurant reservations, book movie tickets, get a taxi, find info or weather. Some of the functionality had to be removed from the app when Apple bought it, because Apple wasn't going to use services like Bing, Yahoo, or Google for search results, but in place of that functionality, Siri got deep integration into iOS, meaning options for scheduling, reminders, calling/messaging contacts, playing music, and dictation. As Siri matures, it is gaining back some of the partnerships that Apple ditched, but gaining more valuable partnerships along the way. Although key partnerships with services like OpenTable and MovieTickets are still missing, Yelp and Wikipedia are in the iOS version of Siri, and the biggest addition is in the partnership with Wolfram Alpha as a big knowledge base.

The point is that for better or worse, Siri is nowhere near the same app that it was when Apple purchased it. Sure, the backbone of voice recognition powered by Nuance is still at the center of Siri, but what the app can do has changed dramatically. What we see in iOS is no longer the product of the Siri developers simply purchased by Apple and stuck into iOS, this is an Apple product with features and uses dictated (no pun intended) by Apple.

Now... Google Now

On the other hand, Google didn't buy one singular product and transform it for Google Now. As we said, this product/platform is more of a Frankenstein monster made up of a number of acquisitions, hires, and in-house development that can be traced back at least 5 years. In many ways, Google Now is an initiative similar to Google+, which is intended to be a platform to unify a number of existing Google products and services, including search, places, travel, suggestions, and of course speech recognition.

Where Apple has partnered with Nuance and its Dragon software for the speech recognition behind Siri, Google has taken the last 5 years to build its own speech database. That project began with the hiring of Mike Cohen in 2007. Cohen was actually a co-founder of Nuance. He spearheaded the Google initiative to build a speech recognition database, which began collecting data through GOOG-411 in 2007 - the free place information service Google once ran - before getting a ton of data through dictation and voice actions on Android over the past 2 years or so, not to mention contributions from Google Voice and its terrible voicemail transcriptions (which have of course gotten better as Google's voice recognition database has grown). Cohen left Google earlier this year, but the speech recognition work will go on.

On the other side of the voice coin, Google also needed a nice voice to respond to users just like Siri does. Luckily, Google had purchased Phonetic Arts back in December of 2010. That purchase was originally made to make robo-voices sound better in Google Translate, as well as the accessibility text-to-speech option found in Android. Phonetic Arts did a lot of work making robo-voices sound better, and that technology has come in very handy with Google Now, because it does sound like the assistant voice is smoother than Siri. As the voice database was growing to a sufficient level, Google also needed to beef up search results.

If you hadn't noticed, Google has been slowly transitioning from a search engine into more of a knowledge provider. More and more when you search, you'll not only get the search results, but an actual answer to your query, such as word definitions, sports scores, movie showtimes, and flight information. While some of that knowledge has come through partnerships and searchable content, big pieces of the Google Now puzzle came through acquisitions

Flight information is gathered via searchable results from flightstats.com, but eventually, Google could move to its own information repo which it purchased in the form of ITA Software (acquired July 2010). ITA Software now powers Google's Flight Search for tickets, and that could easily become part of Google Now's results. Place results, which are a huge part of any mobile search product (because more and more "mobile" really just means "local") have also gotten a boost from a couple choice acquisitions. Google hired the entire team behind Ruba.com (May 2010), an online travel guide, in an effort to give local results a boost. Then, more recently, moves to purchase Clever Sense (December 2011), makers of local recommendation app Alfred, and restaurant guide Zagat (September 2011). Google notably had a falling out with Yelp, and the purchase of Zagat was the answer to that loss of local data.

Possibly the biggest acquisition of all was Metaweb, which Google purchased in mid-2010. Apple of course has to partner with information services like Wikipedia, Wolfram Alpha, and Yelp, but it's Google's business to provide organized data and knowledge to users, and Metaweb is the future of that endeavor. Metaweb just recently made its debut within Google in the form of Google's Knowledge Graph and Semantic Search. Now, rather than searching for keywords, we're searching semantic objects, which means there should be better differentiation between homographs (so Google now knows whether you're searching the meteorological "thunder" or the NBA Thunder.) These results have shown up in the Knowledge Graph block to the right of standard results.

These blocks of information have all made the pretty easy transitions into being "cards" in the Google Now UI. And, on that topic, we have to mention the hire of Matias Duarte in May 2010, who has been one of the strongest forces behind getting the traditionally engineer-oriented Google to make well-designed products. Matias has always loved the "card" metaphor, and so we feel pretty safe in assuming the cards in Google Now were his idea. For those of you that ever used webOS, you know that cards were a central metaphor of that platform (designed by Matias), then we saw the multitasking menu of Android change to cards when Matias arrived and now we get the cards of information as part of Google Now. 

The information side of Google Now is pretty strange to follow, but the learning side of the platform is actually pretty easy, because it uses two things that Google has been cultivating for a long time now: web history and location services. Web history encompasses both your search history, and any visited pages from those search results, where location services is obviously looking at where you spend your time and where you travel. If you have an Android phone with location services enabled, Google probably already knows where you live and where you work, and all of that data can easily be found on your Latitude dashboard (note: this is all private data that only you can see, Google doesn't share it.) When you combine that location data with your personal calendar, and Google's ever expanding navigation options for driving, public transport, walking, and biking, you get a pretty powerful data set that only needs a bit of a nudge to predict what information you may need at a given time and place. 

We can't say that Google Now was completely built in-house by Google any more than Apple built Siri in-house. Both took very different development paths because of the nature of the product, the nature of the respective companies, and the resources available to each company. Google is a software company that has been working hard for years to learn who you are (as an abstract entity, not as an individual), while Apple doesn't have that information to leverage and had to build a product that can essentially start to harvest that data. As we said before, we fully expect Siri to continue its transformation, and eventually become an intelligent push service like Google Now, but whatever the end point, the journey of Siri becoming a fully realized product is much cleaner than Google Now. Apple has always tended to work from a centralized format, creating one singular product designed for a purpose within the larger whole of the Apple ecosystem, while Google is a far more messy affair.

Google captures various services and products for a number of different reasons, and may not always have an overarching plan to bring everything together. Since Larry Page has stepped into the CEO office, that has begun to change, but it can still be a pretty long and windy road leading from products, acquisitions, and hires into a cohesive platform like Google Now.