Perplexity, Copilot & ChatGPT Search Optimization

Why Perplexity, Copilot and ChatGPT don't cite you – a four-layer diagnosis

Missing citations almost always come down to one of four layers: the crawler can't fetch the page, the engine doesn't have it in its index, the page has no passage that matches the question, or other sources look more credible. I check them in that order, because each layer only matters once the one below it works.

The order matters for a practical reason. Rewriting content is the most expensive part of the job, and it achieves nothing if the firewall answers PerplexityBot with a 403 or Bing only knows your homepage. So I always spend the first week on the technical layers and only then choose which pages to rewrite.

AI crawler access – how I check robots.txt, the firewall and server logs

A correct robots.txt is only the first of three checkpoints. A crawler can be allowed in the file and still be blocked by a WAF rule, a CDN bot-protection setting or a rate limit. That's why I confirm access in the server logs, not in the configuration.

I covered the split between training and search crawlers, and how to configure robots.txt, in the GEO guide and in the post on AI crawlers. Here I focus on verifying it in practice.

Verify by IP address, not by name

Anyone can send a user-agent string, so a rule like "let in everything that calls itself PerplexityBot" also opens the door to impostors. OpenAI and Perplexity publish the IP ranges of their crawlers as JSON files, and Microsoft documents how to verify Bingbot with a reverse DNS lookup. In firewall rules I therefore pair the crawler name with its official address pool.

What I look for in the logs

1. I filter visits from the crawlers above over the last 30 days and group them by status code.
2. I look for 403, 429 and 503 responses, which usually mean a WAF block, a rate limit or bot protection.
3. I compare the URLs the crawler fetches with the pages I want cited. If key pages never show up in the logs, the problem is internal linking or the sitemap.
4. I check that the response contains the full content without running JavaScript, because AI search crawlers generally don't render pages like a browser. Details are in the post on JavaScript SEO and AI crawlers.
5. After every rule change I go back to the logs a few days later. OpenAI's documentation says a robots.txt change for ChatGPT search takes effect after roughly 24 hours.

A case worth knowing about

In August 2025 Cloudflare described tests on its blog in which content from domains blocking Perplexity's crawlers still appeared in Perplexity answers. According to Cloudflare, some of the traffic came from undeclared IP addresses and posed as a regular Chrome browser. Perplexity disputed the findings. The practical lesson for a site owner: robots.txt is a request, not a lock. If content truly must stay out of AI, it needs access control such as a login.

Bing Webmaster Tools and IndexNow – the setup I do in week one

Copilot runs on Bing's index, and tests from 2024 suggested ChatGPT leans on it heavily too, so verifying the site in Bing Webmaster Tools and implementing IndexNow is the cheapest lever in the whole service. The step-by-step setup and the protocol's background are in the post on Bing, Copilot and IndexNow; here I cover the technical details IT teams ask about most.

IndexNow – how a submission works

• Key. I generate a key and host it in a text file `{key}.txt` at the domain root. According to the IndexNow documentation the key is 8 to 128 characters long.
• Submission. A single URL can be sent with a GET request, and many URLs with one POST request carrying a JSON list. The documentation allows up to 10,000 URLs per submission.
• Sharing. Participating search engines share submissions with each other, so one ping reaches all of them.
• When I submit. On publishing, on a meaningful update and on deletion. I don't resubmit unchanged URLs, because it wastes quota and can end in a 429.

On Next.js sites I hook the submission into the CMS publish event or the deployment pipeline. On WordPress a plugin is usually enough, but I always confirm in Bing Webmaster Tools that submissions actually arrive.

The AI Performance report – how I read it

The AI Performance report in Bing Webmaster Tools shows citations in Copilot and Bing's AI summaries, along with the queries that led to them. Those queries are the most valuable part for me: they show how the system rephrases users' questions before it goes looking for sources. I add them to the question set I measure visibility on and to the list of topics the content should cover.

ChatGPT Search – what you can realistically do to get ChatGPT to cite your site

In ChatGPT Search you control three things: OAI-SearchBot's access, your presence in the indexes OpenAI uses, and the quality of the passages on your pages. You don't control whether the user turns search on or which model answers.

• Blocking OAI-SearchBot, according to OpenAI's documentation, removes the site from search answers, although it may still appear as a navigational link. It's a common, accidental cause of missing citations.
• Bing and Google. OpenAI doesn't publish a full list of its search providers, so I make sure the site is indexed in both engines, not just Google.
• Referrals in GA4. ChatGPT adds utm_source=chatgpt.com to some links. Without a utm_medium, GA4 can drop that traffic into the "Unassigned" channel, so I create a separate channel group for AI.
• Stores. OpenAI publishes a product feed spec for merchants who want their products shown in ChatGPT shopping. That's an e-commerce topic I cover in the post on agentic commerce.

Brand and recommendation questions ("which company should I choose for…") belong to ChatGPT citation strategy, because there the answer depends largely on mentions in third-party sources rather than on your own site.

Perplexity – its own index, Perplexity-User and the Comet browser

Perplexity runs its own index and two crawlers: PerplexityBot builds the index and Perplexity-User fetches pages at question time. According to Perplexity's documentation, to be cited you need to allow PerplexityBot in robots.txt and accept traffic from its published IP ranges; behind a WAF you may need to add an exception manually.

Perplexity shows sources with every answer, and in the setups I've analysed, visits from perplexity.ai were usually recognised in GA4 as referrals from that domain. That makes it a good place to check quickly whether reworked pages are starting to bring in visits.

Publishers can also earn from appearing in Perplexity. In August 2025 the company announced the Comet Plus subscription and said 80% of its revenue would go to publishers, based on traffic, citations and agent actions on their content (Axios, Press Gazette). For most service businesses that's a curiosity, but content-heavy sites should check the current terms.

How I rework content for citations in Copilot, Perplexity and ChatGPT

I rework content so every section opens with a sentence that answers the question on its own, and the rest supplies the evidence: figures with sources, steps, comparisons. In October 2025 Microsoft published guidance on writing content its AI systems select for answers, and it lines up with what works in Perplexity and ChatGPT.

A paragraph before and after

Before: "Our cutting-edge solutions help clients save time and money through an innovative approach." That paragraph answers no question, so it has no chance of being cited.

After: "Rolling out system X in a company of up to 50 people usually takes 3–4 weeks and covers data import, permission setup and team training. Pricing starts at Y." That sentence can be lifted out of context and dropped into an answer, so AI has a reason to pick it. The values X and Y always come from the client; I never make them up.

More patterns like this are in the post on how to write content AI cites, and trust signals such as authorship and sources are covered in E-E-A-T in 2026.

Comparison pages, rankings and pricing – the formats AI search engines reach for

For questions like "X or Y", "how much does it cost" or "best tool for…", AI search engines look for pages that compare options against clear criteria. If you don't have that page, AI will cite a portal, a forum or a competitor that does.

I build these pages to a few rules that separate an honest comparison from an ad dressed up as a ranking:

• The comparison criteria are listed at the top and are the same for every option.
• Competitors are described fairly, including the situations where they're the better choice.
• Prices and specs carry a checked-on date and a link to the source.
• The page has an author with real experience in the topic and an update date.
• The verdict isn't decided in the first sentence. The recommendation depends on the reader's situation.

How I test visibility in Perplexity, Copilot and ChatGPT before and after changes

I measure visibility on a fixed question set, asked repeatedly in each of the three engines, recording the full answer and the cited URLs. A single test is worthless, because AI answers change from one query to the next. The measurement method and metrics are covered in more depth in the GEO guide; here are the settings specific to these platforms.

1. I run the baseline before any change, otherwise there's no way to show an effect.
2. I ask questions in the language and location of the target market, and separately for each market if the company sells abroad.
3. In ChatGPT I make sure the answer actually uses search, because without it there are no citations to measure.
4. I record not just whether the brand appears, but which URLs are cited and which domains win instead.
5. I add queries from the AI Performance report in Bing Webmaster Tools to the question set, because they show the real phrasing Copilot uses.
6. In GA4 I create a separate AI channel with a regex covering chatgpt.com, perplexity.ai and copilot.microsoft.com, placed above the "Referral" channel. The setup is described in the post on measuring traffic from ChatGPT and Perplexity.

For repeatable measurement I use my own tool, described in the AnswerLyzer case study. If you'd rather use an off-the-shelf tool, the options are compared in the post on AI visibility monitoring tools.

A Perplexity and Copilot optimization timeline – the first 90 days

The service usually runs for about three months: the first weeks cover technical work and the baseline, the middle is content rework, and the end is monitoring and fixes. Pricing starts from €1,200 and depends mainly on how many pages need reworking.

First citations on reworked pages usually appear once crawlers fetch them again, so IndexNow shortens that wait in Bing and Copilot. If the diagnosis shows the problem is how models see the brand rather than the site itself, I start with an AI visibility audit. When the problem is that the company isn't recognised as an entity, knowledge graph engineering helps.