Search engines discover new content in two ways:
Crawling: Bots visit your site periodically, follow links, and index what they find. High-authority sites with frequent updates get crawled more often. Smaller sites might wait days between visits.
Sitemaps: Your sitemap.xml tells search engines what pages exist on your site. They check it periodically, but "periodically" could mean once a day or once a week depending on your site's perceived importance. Ghost updates sitemaps automatically, so there's nothing you need to do manually. Your sitemap is located at example.com/sitemap.xml.
Both methods are passive. You publish content and hope search engines notice.
Enter IndexNow
IndexNow flips this model. Instead of waiting for search engines to visit, you tell them directly: "Hey, I just published something new. Come have a look."
It's a simple protocol developed by Microsoft in 2021. When content changes on your site, you send a POST request to the IndexNow API with the URLs that changed. The API distributes this to all participating search engines, and they prioritize crawling those specific URLs.
IndexNow's FAQs have an up to date list of search engines that participate:
Google has been publicly testing IndexNow but hasn't joined the consortium yet. They already have their own instant indexing API for specific use cases, so their position on IndexNow remains a bit unclear, unfortunately.
Why This Matters for Publishers
If you're running a news site, speed matters. Being first in search results for a breaking story can mean thousands of additional readers. Waiting hours for Bing to discover your coverage means missing that window.
Even for blogs and business sites, faster indexing has benefits:
- Time-sensitive content reaches search results while it's still relevant
- Corrections and updates propagate faster, reducing outdated information in search
- Deleted content disappears from search results more quickly
- New sites can build search presence faster than relying on crawl discovery alone
How Magic Pages Handles This
Today I woke up to an email from a customer, asking whether I have any experience with implementing IndexNow in a Ghost site. Frankly, I had none. I wanted to send over some resources on how this can be put together with Ghost's webhooks − but I realised that this followed the same pattern I already use for Bunny.net cache invalidation and search updates for the Typesense-based search: when a post changes, do something.
So, the infrastructure was in place.
I've integrated IndexNow directly into Magic Pages's cache invalidation system. When you publish, update, or delete content in Ghost, a few things happen simultaneously now:
- Your CDN cache refreshes so visitors see the latest version
- If you have the Advanced Search activated, your content gets indexed freshly
- IndexNow notifications fire to search engines
If you want to know more about the technical background on how I do this, have a look here:
Jannis Fedoruk-Betschki
The whole thing happens automatically for every Magic Pages site in the background. Each site has a unique IndexNow key stored in the backend database database, and we serve the verification file dynamically when search engines check ownership.
I've also built in smart deduplication. If you're iterating on a post (e.g. fixing typos, tweaking headlines) we don't want to flood search engines with notifications for every save. Each URL only gets submitted once per 24-hour period.
What About Google?
Google still relies on traditional discovery methods for most sites. They'll find your content through:
- Your
sitemap.xml(Ghost generates this automatically) - Links from other sites
- Regular crawling based on your site's authority
For high-priority indexing on Google, you can manually submit URLs through Google Search Console. But for most publishers, their standard crawling is fast enough once your site has established some authority.
The IndexNow integration primarily accelerates discovery on Bing and the other participating search engines.
If you're hosting your Ghost site on Magic Pages, IndexNow is already working for your site. Every existing site received a unique key automatically, and new sites get one at signup.
There's nothing to configure, no settings to toggle, and no code to add. Publish your content like you always do, and search engines will know about it within seconds.