𝗛𝗼𝘄 𝗠𝘂𝗰𝗵 𝗼𝗳 𝗬𝗼𝘂𝗿 𝗕𝗹𝗼𝗴 𝗗𝗼𝗲𝘀 𝗔𝗜 𝗦𝗲𝗮𝗿𝗰𝗵 𝗔𝗰𝘁𝘂𝗮𝗹𝗹𝘆 𝗚𝗿𝗮𝗯?
When an AI performs a web search, how much of your content does it see? Does it read your whole article or just the title?
I tested Anthropic's tools to find out. Here is how it works.
AI search happens in two distinct stages:
Stage 1: WebSearch The AI uses a search engine to find a list of results. At this stage, the AI sees almost nothing from your page. It only sees:
- The URL
- The page title
- The last updated date
If your title is vague, the AI might skip your site entirely.
Stage 2: WebFetch If the AI finds a relevant title, it sends a request to grab the page body. This is where the content comes in.
However, there is a hidden layer. Tools like Claude Code use a middle model (Haiku) to summarize the page first. The main AI model often never sees your original text. It only sees the summary created by the middle model.
This means your writing is filtered before the user ever sees it.
How to write for AI search:
- Optimize your titles: Do not use "Implementation Notes." Use "Complete Implementation Notes: Content Negotiation for HTML-to-Markdown." Give the AI enough data to click.
- Front-load your value: The summary model reads from the top down. Put your main conclusions in the first few sentences of every section.
- Design for citations: AI quotes are very short (about 125 to 150 characters). Write standalone sentences that make sense without any surrounding context.
- Use Markdown: Use H2 tags, bullet points, and tables. These act as signposts for the AI to understand your structure.
One final tip: Check your robots.txt. You can allow search bots to cite your site while blocking training bots from using your data to train their models.
Stop writing just for humans. Start writing for the agents that find you.
Optional learning community: https://t.me/GyaanSetuAi