Googlebot now crawls the web as Android not an iPhone

Googlebot now crawls the web as Android not an iPhone
Google is making a big change to the way the so-called "Googlebot" crawls the web. The Googlebot is the webcrawler that searches everything on the Internet in order to update Google's enormous database and make sure you get the freshest links possible when you do a search. For the first time ever, the default mobile crawler ID will be Android and not an iPhone.

The Googlebot's smartphone user-agent has been an iPhone, and the most recent version of the user-agent has been pretending to be an iPhone running iOS 8.3. However, on April 18th that is going to change and the Googlebot will begin have a default smartphone user-agent pretending to be a Nexus 5X running Android 6.0.1.

The official explanation for this change is so Google's "renderer can better understand pages that use newer web technologies." Presumably this means web tech like HTML5, which Google has been pushing for a long time. Google also claims that the Googlebot renderer has "evolved" to become more like Chrome than Safari, which does kind of make sense given that the latest stats from NetMarketShare say mobile Chrome makes up almost 43% of the market while mobile Safari is only 33%. 

Google claims that 99% of sites should see no effect from the change, and those that do will only see issues if it is looking for a specific Googlebot user-agent. 

FEATURED VIDEO

41 Comments

1. Derekjeter

Posts: 1497; Member since: Oct 27, 2011

PA I would like to take this time to say your website sucks now. The ads are making it horrible for me to browse not only on my iPhone 6s plus but my new S7 edge and my ipad air 2. Just thought i would let you know how my experience has been the last 2 weeks.

2. Podrick

Posts: 1285; Member since: Aug 19, 2015

Why don't you use an adblocker? Both Safari and Samsung browser supports adblockers. You've been here for so long, you obviously know that.

9. quakan

Posts: 1418; Member since: Mar 02, 2011

I have yet to find a great ad blocker for iOS. Most ads still come through. On mobile, it seems like I'm scrolling forever to get through a story and make it to the comment section on this site.

14. sgodsell

Posts: 7369; Member since: Mar 16, 2013

The real reason to NOT use an iPhone for crawling the web is because of Safari. Which happens to be the least compatible HTML5 browser currently on the market. It use to be Microsoft's Internet explorer was the one that held that title, but now Microsoft's Edge browser is the ahead of Safari. If you don't believe me check it out for yourself. HTTP://html5test.com Shame on you Apple. Most compatible HTML5 browser goes to Chrome.

20. AlikMalix unregistered

And how is that relevant to any user? If you look the items hit and miss - the ones that don't have a check on it is stuff you never find needed on a webpage: I mean gamepad support? ... for a webpage? Yey - next time I need to use a gamepad to play a game on Facebook - I'll install chrome. Bet I would not.

31. Ordinary

Posts: 2454; Member since: Apr 23, 2015

WebP, WebRTC 1.0, Data channel, Clipboard API, Security, HTML imports and so on. It may look like nothing to you but it makes surfing a much better experience.

33. AlikMalix unregistered

Hey are all those things for? But to me on safari on iOS or chrome on android is the same result, same outcome, same speed, same experience. But like I said, if you must have it, install chrome and be on your way.

35. Mxyzptlk unregistered

Which has absolutely nothing to do with anything pertaining to this article. Just more blind hate from sgodsell.

19. AlikMalix unregistered

Bull s**t!!!! I'm using an add blocker on my iPad 3 and my iPhone 6s and my iPhone 5s and NOT EFFING ONE AD IS SEEN!!! I even used a free one before and that worked too. Any blocker should work. Mine rhymes with "curify" - it's the first one I tried and it's the first one that works perfectly. I cannot tell you the name of the app because last time I did I was banned.

3. Mxyzptlk unregistered

Dude I agree, this site is more unstable on mobile than Lollipop. I had to resort to using Firefox.

4. BradyCrack

Posts: 835; Member since: Dec 29, 2015

You can't say one thing without bashing android can you?

5. Podrick

Posts: 1285; Member since: Aug 19, 2015

He can't help being a troll. He needs attention from everyone.

12. Mxyzptlk unregistered

Should I post links or are you ready to admit you're wrong?

18. Podrick

Posts: 1285; Member since: Aug 19, 2015

Ok, then I'll post links on how horrible iOS 8 was, worse than Lollipop. You're just in denial that you're a butthurt troll and bring up Android in non related articles.

21. AlikMalix unregistered

Podricko, yes iOS 8 in the beginning had issues. But you are eluding the elephant in the room - iOS users got ther patches within weeks and after a couple months we had a stable glitch free operating system. 3rd of all android users are stuck with lollipop and another third is about to get lollipop, and get stuck in that too.

24. joey_sfb

Posts: 6794; Member since: Mar 29, 2012

fanboys will forever be in denial, you claim iOS 8 problem was patch within weeks, so let examine the number of weeks it take. iOS 8 was release on 17th Sep 2014. 22th Apr 2015 Linkhttp://www.forbes.com/sites/gordonkelly/2015/04/22/apple-ios-8-no-ios-zone/#51002e426c4b 28th July 2015 Linkhttp://www.digitaltrends.com/mobile/ios-8-problems/ Mxyzptik and AlikMalix, you guys are not even Android users. You don't use it on a day to day basis. All you know are what Apple propaganda machines lies pretty much like what North Korean regime does. Yet, you will come to PA to do your fanboy routines. What value does that give you? To easy your sense of insecurity of having an overprice gadget that you are not sure whether its worth the money? Let me tell you its worth it. OK. Be at ease, apple has your back. Android user don't like to stick with one device, that our main problem. We like changes and tinker with everything Android.

25. AlikMalix unregistered

You missed the point. So a few users had problems for longer than I did. But I'll meet you there, some users had problems for the first half a year - that's not the point. With iPhones ios8 is now history, with android lollipop is either current problems for half of users or the future problem for the other half. Anything that had a security issue (like the first link you posted) has been patched in weeks - I even remember that article. Either way, if you got a bad android version - tough shnit - you're stuck, even if Google made an update. . If you got a glitch on iOS, it's usually updated in weeks - and nearly everyone can get it (obviously unless you're still carrying a 5 year old iPhone).

26. joey_sfb

Posts: 6794; Member since: Mar 29, 2012

I think I really deserve some prize or award since I am the only one that can use Android Lollipop successfully on a day to day basis with zero issue. LOL! AlikMalix thinks for a moment if Android Lollipop is that problematic why does it hold a 36.1% user base? Another thing to think about, Android fan base are primarily geek, we would not hide the flaw for Google if Lollipop is s**t. So far, I have Sony Bravia TV, Note 12.2, Tab S 8.4, Note 3 all these device runs Lollipop and I am cool with it. I personally do prefer Marshmallow on my Nexus 6 and Nvidia Shield TV. But lollipop is fine too! Backtrack a bit, I want to say iOS device are great device too! Just that I personally prefer to buy Android because Android is more affordable and got more things for me to play. I aim to get iPad Pro to replace my iPad 3 but the hefty price tag put me off. I don't think that gadget deserve that much of my money.

28. Adreno

Posts: 755; Member since: Mar 12, 2016

@joey_sfb That's one damn good point and post. You indeed deserve some award. Heck, lollipop 5.0.1 runs great on my G Pro 2. No issues!

29. Adreno

Posts: 755; Member since: Mar 12, 2016

@joey_sfb That's one damn good point and post. You indeed deserve some award. Heck, lollipop 5.0.1 runs great on my G Pro 2. No issues!

34. AlikMalix unregistered

Joey, and I had iOS 8 and all its early versions - it was a non issue for me either. We blow this crap out of proportion that affects maybe a few people - I'm just replying to you "tit for tat". The only thing I want to disagree is that android fan base are primarily geek.- have you met apple fan base - it's the same. You can't be a fan of anything software or hardware unless you're a geek. Everyone I know that uses android or iOS - 90% can't tell you anything about their phone other than who makes it. But even today I see people that call their android phone an iPhone. And I already explained why 3rd of all android is in lollipop (especially when M has been out for how long now) - they're simply stuck on Lolipop, and the other 3rd are stuck in older versions. Account that for outdated security flaws, glitches, and hardware optimization problems. I don't disagree with your comment, I've read posts related to iOS issues (and android for that matter) and most of the user base around me is iOS and a few have android devices and as a geek I always ask if they have this issue or that and guess what nearly 99% the answer is "no". I use iOS extensively - Siri, iCloud, other features that most dont know about... I customize all my apps and features to best serve my needs and not be annoyingly "there" heavy apps, heavy games. And so on - and I'll tel you iOS 8 was only a bit of an issue on my old iPad 3 - it toile couple of months and the issues were fixed except for one that was fixed in May I think - it was loading a wallpaper took too long and kinda hanged for about 20 seconds - but still did it. On my 5s and my wife's 5 - it had no issues. Just saying that if you're gonna talk how bad and horrible iOS 8 was - I just pointed out that android versions have issues that are bigger (according to articles) but where it cuts the cake is that iOS users got their updates - android users didn't.

41. joey_sfb

Posts: 6794; Member since: Mar 29, 2012

I don't like to talk s**t about iOS or Apple. I do use Apple products for both entertainment and productivity. Such as an old iPad 1 and iPad 3. I do also have 2 Macbook Pro, one older model at home and one other in the Office. When I bitch I point out the specify problem i might have with the company. Example, I am not a fan of Apple removing standard ports from their Macbook as such USB, SD card slot or Apple planned obsolescence business practises. Mxyzptik is illogical therefore i just troll him for amusement. AlikMalix more reasonable so can still reason.

36. Mxyzptlk unregistered

iOS 8 has been fixed and patched up while some people just now getting to lollipop will be stuck on lollipop while others are slowly waiting for 5.1 or 6.0 and you want to talk about butthurt and denial?

39. joey_sfb

Posts: 6794; Member since: Mar 29, 2012

Mxyzptik, you are really very nice! Such concern for us Android users. LOL! Don't worry we are fine. I suggest you save your energy on fixing you own gadgets problems.

40. Mxyzptlk unregistered

Wallow in self-pity and denial.

43. joey_sfb

Posts: 6794; Member since: Mar 29, 2012

Mxyzptik... having only Apple toy doesn't mean you need to wallow in self-pity and denial. Its pretty decent toy, good for at least the first 12 months after release.

11. Mxyzptlk unregistered

For the record, that was a dig at LOLLIPOP, not Android itself. Please learn to read next time.

16. Podrick

Posts: 1285; Member since: Aug 19, 2015

And this article wasn't about lollipop smartass.

30. Adreno

Posts: 755; Member since: Mar 12, 2016

@Podrick Nice shutting that Mxyzptlk guy's mouth. I too have noticed him riding on Android's ass bashing lately.

38. Mxyzptlk unregistered

Nice butt kissing there. You should take a mint after that smooching.

Latest Stories

This copy is for your personal, non-commercial use only. You can order presentation-ready copies for distribution to your colleagues, clients or customers at https://www.parsintl.com/phonearena or use the Reprints & Permissions tool that appears at the bottom of each web page. Visit https://www.parsintl.com/ for samples and additional information.