In 2019, Microsoft Bing announced new Bingbot user-agent names that fit better with its evergreen Bingbot crawling and rendering service. Microsoft’s Fabrice Canel has now said that by the Fall of 2022 the old user-agent will stopped being used and the search company will transition to the new user-agents fully.
Old user-agent. Microsoft said it will stop using its historical user-agent by Fall 2022. That user-agent looks like this:
Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)
New user-agent. Bing will use a user-agent that identifies the specific version of Microsoft Edge is crawling your site. Here is the format for both desktop and mobile:
Desktop – Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) Chrome/W.X.Y.Z Safari/537.36
Mobile – Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Mobile Safari/537.36 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)
“W.X.Y.Z” will be substituted with the latest Microsoft Edge version Bing is using, for eg. “100.0.4896.127″.
Do we need to worry? Most sites do not need to worry. Microsoft said “For most web sites, there is nothing to worry as we will carefully test the sites to dynamically render fine before switching them to Microsoft Edge and our new user-agent.” But if you have hardcoded any user agents into your scripts, you will need to revise those scripts to ensure BingBot can continue to crawl your site.
How to test. Bing previously said you can test it by installing the new Microsoft Edge browser “to check if your site looks fine with it.” Bing said “if it does then you will not be affected by the change.” “You can also register your site on Bing Webmaster Tools to get insights about your site, to be notified if we detect issues and to investigate your site using our upcoming tools based on our new rendering engine,” Bing added.
Bing added “we will carefully test websites before switching them to our new user-agent Bing Webmaster Tools URL Inspection has already started using the new desktop user-agent for the Live URL Test to help you investigate potential issues.”
Google also. Google is also migrating to the new GoogleBot this month. Google is currently testing the new user agents, so you may be able to see them in your log files. I do not believe it is fully rolled out yet for Google.
Why we care. You probably should have been prepared for this change, since it was announced back in 2019. But in any event, this change can impact your site if you had any user agent detection methods for BingBot. Make sure to test your site to see if it supports the new user-agent. Most sites probably do not need to worry about this but you have done any advanced bot detection, you may need to take steps to update those scripts.
New on Search Engine Land
About The Author
Barry Schwartz a Contributing Editor to Search Engine Land and a member of the programming team for SMX events. He owns RustyBrick, a NY based web consulting firm. He also runs Search Engine Roundtable, a popular search blog on very advanced SEM topics. Barry can be followed on Twitter here.
Leave a Comment