On 18 October 2021, Microsoft Bing and Yandex published IndexNow, an open protocol that lets a website ping participating search engines when a URL is added, updated or deleted. Cloudflare shipped one-click support the same day. I already run it on this news site: when a public article is published or updated, a small plugin submits the URL. I would put the same hook in any automated SEO setup and leave it there.
What IndexNow does
In the protocol’s own words, it is a simple ping so that search engines know a URL and its content has been added, updated or deleted, and can reflect that change in their results more quickly. Without it, the project site says, discovery can take days to weeks, because engines do not crawl every URL often. A submission is a hint, not a promise that the page will be crawled or indexed. Each engine still applies its own crawl budget, quality checks and scheduling.
To use it, you prove control of a host by publishing a UTF-8 text key file. The usual method is to put {your-key}.txt at the root of the site, with the key as the only contents. The key is 8 to 128 characters, limited to letters, numbers and hyphens. You can also place the file deeper on the same host and pass its URL as keyLocation; in that case the key only covers URLs under that path. The documentation recommends the root file.
A single URL is submitted with a GET request to an IndexNow endpoint, with the changed URL and the key as query parameters. A set of URLs, up to 10,000 in one call, is submitted as a JSON POST with the host, the key and a urlList. You can send the request to any participating endpoint, including the shared endpoint at api.indexnow.org. Search engines that adopt the protocol agree that URLs submitted to one of them are shared with the others. A 200 response means the engine received the URL. A first submission can return 202 while the key file is still being checked. A 403 or 422 points to a key or host mismatch, and a 429 means you have hit a rate limit.
The protocol also covers deleted and redirected URLs. If a page returns 404 or 410, or has moved, you should submit that URL so participating engines can update or drop it.
How it came to be
Bing’s launch post, written with the same 18 October date, framed IndexNow as a response to a familiar delay: it can take days or weeks for new or changed URLs to be discovered and considered. The protocol was meant to give site owners a clear change signal, to cut the exploratory crawling engines do just to find out whether a page has changed, and, in time, to let engines crawl IndexNow sites more sparingly. It was also meant to be open: notify one engine, and you notify every engine that has adopted the protocol. The specification is licensed under Creative Commons Attribution-ShareAlike 4.0.
Yandex’s webmaster blog carried a matching announcement the same day. Maxim Zagrebin, then head of Yandex Search, said in Cloudflare’s launch materials that Yandex had already been testing Cloudflare’s cache signals through the IndexNow API. Fabrice Canel, then a principal program manager at Microsoft Bing and still the public face of the project, called Cloudflare’s one-click button a way to put a large share of the web onto the protocol at once.
Crawler Hints Update: Cloudflare supports Microsoft Bing’s IndexNow that allows websites to easily notify search engines whenever content changes, improving SEO and enabling more efficient WebCrawling.
October 18, 2021
Cloudflare’s Crawler Hints, which went to general availability that day, uses cache and origin signals to tell crawlers when content has likely changed. With IndexNow support switched on, those hints are sent to participating engines. Duda enabled IndexNow for its sites shortly afterwards. WordPress support came in through plugins. Microsoft put its own IndexNow plugin on GitHub as open source, Rank Math shipped support in early 2022, and Yoast SEO Premium followed in June 2022. SEOPress and AIOSEO added it in the same stretch, and Wix switched it on for Premium sites in September 2023. The IndexNow FAQ now also lists Shopify, GoDaddy, Joomla, Drupal and several other CMS platforms as having native or plugin support, and names Cloudflare as a CDN with a built-in integration.
Sharing between engines was formalised on 13 January 2022, when Bing said URL co-sharing was live with Yandex, that other engines were following, and that submissions could go to api.indexnow.org. Seznam.cz, the Czech search engine, said in December 2021 that it would join, and opened access the following year. Naver documented support in 2023. Ahrefs’ Yep search engine followed in 2024.
Ahrefs is partnering with IndexNow. Ahref’s search engine Yep is getting data from IndexNow.
March 5, 2024
In May 2025, Bing wrote that Shopify was supporting the protocol and that Amazon planned to start adopting it in mid-June. Open the live registry at indexnow.org/searchengines.json today and you’ll see Microsoft Bing, Yandex, Seznam, Naver, Yep, the Internet Archive and Amazonbot.
Amazon will start using IndexNow in June. That’s huge. And there’s more about Shopify and Milestone in the article.
May 20, 2025
Google is the obvious absence. In November 2021, a Google spokesperson told Search Engine Land and Search Engine Journal that the company was encouraged by work to make crawling more efficient and would be testing the potential benefits of the protocol. Google has not announced that it has adopted IndexNow, it does not appear in the current participant list, and Google’s crawling documentation does not offer it as a way to notify Google Search. For Google, sitemaps, ordinary crawling and Search Console remain the route. Google’s Indexing API is narrower still: official documentation limits it to pages with JobPosting or livestream BroadcastEvent markup.
Who curates it
IndexNow.org is where the protocol lives. Under the site’s terms of service, the sponsors are Yandex N.V., Seznam.cz, a.s., and Microsoft Corporation. They hold the copyright in the specification, they licence it under CC BY-SA 4.0, and those terms let them change the protocol. I looked for a public git repository for the specification and didn’t find one. The WordPress plugin Microsoft issued is on GitHub, under microsoft/indexnow-wordpress-plugin.
The more detailed rules for participating engines, including how they identify themselves, sign notifications to one another, and retain logs, live at indexnow.org/searchengines. The March 2024 credit line on that page lists Sawood Alam at the Internet Archive, Martin Fiala at Seznam, Vishnevsky Gleb at Yandex, and Fabrice Canel at Microsoft. Participating engines must poll the official searchengines.json list at least once a day, share verified URLs with the others, and keep logs of notified URLs for at least a week, available to fellow participants.
What Bing has published about adoption
The figures below are from Bing’s webmaster blog, with the dates they were published. In January 2022, Bing said more than 80,000 websites had started publishing to the protocol. By August 2022 it put the figures at more than 16 million sites, over 1.2 billion URLs a day, and 7% of all new URLs clicked in web search results. In September 2023, alongside the Wix announcement, it said more than 60 million websites were publishing over 1.4 billion URLs a day, and that 12% of all new URLs clicked in web search results were first discovered in IndexNow. On 12 December 2024 it said there were over 3.5 billion URLs submitted per day, with 18% of all new URLs clicked in web search results. A May 2025 update described new platform integrations and did not publish a new share figure.
Bing also added IndexNow Insights inside Bing Webmaster Tools in March 2024, so a verified site can see which submitted URLs were crawled or indexed, and which clicked URLs never arrived through IndexNow.
Where it fits in an SEO pipeline
For a marketing team, the useful property is speed of discovery, not a ranking boost by itself. The protocol docs are clear that a 200 only means the URL was received. What you get is a way to tell participating engines about new pages, edits, price or availability changes, and deletions, at the moment your publish hook fires, instead of waiting for the next crawl of a sitemap.
Sitemaps still matter. IndexNow is for recent changes, not a full inventory, and the official FAQ says you should use both: IndexNow for pages that have just changed, and a sitemap, with accurate lastmod values, for the rest of the site. If you are switching the protocol on for the first time, do not backfill every old URL through IndexNow; let the sitemap cover what changed before you started.
Because Google is not a participant, an automated pipeline still needs a Google path: a fresh sitemap, internal links, and, for occasional individual URLs, Search Console’s URL Inspection. The Indexing API does not stand in for ordinary pages. For Bing, Yandex, Seznam, Naver, Yep, the Internet Archive and Amazonbot, one IndexNow call is enough.
The practical shape is a publish hook. When a page is created, updated, unpublished or deleted, the CMS, CDN or a small script submits that URL. Many platforms already do this if you switch the integration on. I do it here for public articles. You should also keep submitting redirects and gone pages, and you should not fire a submission for a typo fix every few seconds: the FAQ asks you to wait at least five minutes between submissions of the same URL, and to reserve the ping for a meaningful change.
To all the SEOs planning for 2023 I ask: add IndexNow to your plans. Change the industry: search engines should stop guessing and over-crawling. SEOs should control crawling and indexing. If your CMS is not yet IndexNow enabled ask to enable it by default
December 9, 2022
What security and privacy teams should know
The key file is public on purpose. Search engines fetch it from your host to check that the submitter controls that host. It is not a password you hide; it is a proof that someone can write a file on the site. The protocol documentation tells you to treat the key and its location as known only to you and the search engines, but the verification design requires the file to be reachable without a login. Anyone who can read the file can submit URLs for that host. They cannot use your key to submit URLs for a host they do not control, because the engine checks the key on the host in the request. The real control is who can write files on the site, and which URLs you choose to send.
If you submit a staging, preview or otherwise private URL, that address is handed to every participating engine and kept in partner logs for at least a week. Do not point IndexNow at those hosts, and do not put a production key on a publicly reachable staging copy if you do not want those URLs shared. Each subdomain is a separate host and needs its own key file.
Engines can reject a request with 429 when submissions arrive too fast, and they apply their own quality and spam checks later. IndexNow does not publish a single shared rate limit; each engine sets its own. Log the HTTP responses, and treat a 200 as receipt, not as a security or indexing verdict.
A short checklist
- Host a key file at the site root, or let a trusted CMS or CDN integration do it for you.
- Hook publication, updates, unpublishing, deletions, redirects and gone pages, and submit only when the content has actually changed.
- Send the request to one participating endpoint. You do not need to fan it out.
- Keep a sitemap with accurate lastmod values.
- Use Google’s own tools for Google.
- Do not submit staging or private URLs.
- Watch for 202 on the first key check, and for 429 if you are submitting too often.
- If you use Bing Webmaster Tools, the IndexNow Insights report is the place to see what happened after the ping.
How IndexNow compares
| IndexNow | Sitemaps | Google Indexing API and Search Console | |
|---|---|---|---|
| What it is for | A ping to participating engines when a URL is added, updated or deleted | A full inventory of URLs, ideally with accurate lastmod values | The Indexing API covers JobPosting and livestream BroadcastEvent pages only. Search Console URL Inspection is a manual request for individual URLs |
| Who receives it | Every current IndexNow participant, after a submission to one endpoint | Each engine that fetches the sitemap | Google only |
| Volume | Up to 10,000 URLs per POST | A site-wide list, kept current | The Indexing API can batch up to 100 calls. Search Console inspection is quota-limited |
| What a success response means | The URL was received, not that it will be crawled or indexed | A discovery aid, not a crawl or index guarantee | A request, not a guarantee |
| Google Search | Google is not a participant | Google reads sitemaps | This is Google’s own route |
I would not take this protocol as a substitute for sitemaps or for Google Search Console. I would also not ship an automated SEO pipeline that publishes, updates and deletes pages and never tells the engines that already want the ping. The cost of the hook is small, and on this site it is already in the publish path.
Discover more from Sulat AI & Tech News
Subscribe to get the latest posts sent to your email.
