Ping Search Engines With URLs For Instant Indexing

Microsoft Bing and Yandex announced a new initiative named IndexNow that lets site owners ping IndexNow’s protocol to achieve instant indexing with the participating search engines. The participating search engines right now are only Bing and Yandex; Google is not a part of it, nor are other search engines as of yet.

There is a website at IndexNow.org that summarizes what this is about, “IndexNow is an easy way for websites owners to instantly inform search engines about latest content changes on their website. In its simplest form, IndexNow is a simple ping so that search engines know that a URL and its content has been added, updated, or deleted, allowing search engines to quickly reflect this change in their search results.”

Now, the method for pinging and submitting the URLs is pretty straight forward and it is documented here.

To submit a URL using an HTTP request (replace with the URL provided by the search engine), issue your request to the following URL: https:///indexnow?url=url-changed&key=your-key

So right now it would be either https://www.bing.com/IndexNow?url=url-changed&key=your-key or https://yandex.com/indexnow?url=url-changed&key=your-key

To submit a set of URLs using an HTTP request issue your POST JSON request to the URL provided by Search Engines. Just replace by the host name of the search engine. You can submit up to 10,000 URLs per post, mixing http and https URLs if needed.

How do you make your key to verify ownership of the site? Two methods:

(1) Hosting a text key file at the root directory of your host.

(2) Hosting a text key file within your host.

Like I said, only Bing and Yandex are part of it. Google was asked, but they are not part of it. Baidu, I guess not. Others may have been asked as well. For search engines to adopt the IndexNow protocol, they must agree that submitted URLs will be automatically shared with all other participating Search Engines. To participate, search engines must have a noticeable presence in at least one market.

You can check out the FAQs over here.

CMS platforms and CDNs and SEO tools are all interested in working with this IndexNow protocol, I am told.

This does not replace the Bing URL submission API, that will be somewhat duplicative to this effort, but the Bing URL submission API only works with Bing’s index. Also, this does not impact the Bing content submission API. It just seems Bing is doubling down on the push versus pull method for indexing and trying to get other search engines to adopt it. It does make sense, it is more of an efficient method to get new pages – but will there be enough adoption?

FYI, I quickly activated this because this site uses CloudFlare which added support with this announcement.

Why no Google? They do have their own indexing API that is limited use. I suspect Google gets a ton of signals out of crawling and is not ready to give up on those signals. John Mueller of Google made a joke about the announcement:

???? we too, would like to be crawled less often.

— ???? John ???? (@JohnMu) October 18, 2021

Forum discussion at Twitter & WebmasterWorld.