Tag: perplexity

  • How to measure AI search visibility without fooling yourself

    How to measure AI search visibility without fooling yourself

    Somebody on your team types your company name into ChatGPT, sees a competitor in the answer, and takes a screenshot. By lunch it’s in a Slack channel with three worried emoji. By Friday there’s a meeting about your “AI visibility problem.”

    I get why that happens. But one screenshot is one answer, to one prompt, on one day, for one person. It isn’t a trend. It’s an anecdote with a nice crop.

    The good news is that you can measure AI search a lot better now than you could a year ago. Google, Microsoft, OpenAI, and Perplexity all publish something you can actually use. None of it is complete, and some of it is easy to misread.

    In this guide, I’ll show you how I set up AI search measurement for a site: which numbers are real, where to find them, how to log the stuff no tool reports, and the mistakes that make people think they’re winning (or losing) when they aren’t.

    Key takeaways

    • Google now has a Generative AI performance report in Search Console that shows impressions from AI Overviews and AI Mode by page, country, device, and date.
    • Bing Webmaster Tools has an AI Performance report that counts citations across Microsoft Copilot, Bing’s AI summaries, and some partner experiences.
    • Clicks from ChatGPT, Perplexity, Gemini, and friends show up in your analytics as referral traffic. You can group them into their own channel in about ten minutes.
    • Your server logs tell you whether AI crawlers can reach your pages at all. That’s a readiness check, not a result.
    • Everything else, like whether an assistant mentions you by name, has to be logged by hand. Do it on a schedule, with the same prompts, or don’t bother.

    Why AI visibility is so easy to mismeasure

    Classic SEO had a pretty clean loop. You ranked, people saw you, some clicked, and Search Console counted all three.

    AI answers break that loop in a few places.

    First, a lot of people get what they need without clicking. Pew Research Center looked at the browsing of 900 U.S. adults and found that Google users clicked a traditional result in 8% of visits when an AI summary appeared, versus 15% when one didn’t. So traffic alone undercounts how often you’re seen.

    Second, the answers move around. Ask the same question twice and you can get two different sets of sources. Change the wording, the location, or whether you’re logged in, and it shifts again.

    Third, every platform reports something different. Google counts impressions. Bing counts citations. ChatGPT and Perplexity don’t give you a dashboard at all, just the clicks they send.

    So here’s my rule. Never add these numbers together, and never compare one platform’s metric to another’s. A Google impression and a Bing citation aren’t the same unit. Treat them like temperatures in Celsius and Fahrenheit. Both useful. Not addable.

    The four layers of AI search measurement

    I think about this in four layers, from “can they even reach us?” to “did it make money?”

    LayerQuestion it answersWhere the data livesHow solid is it?
    AccessCan AI crawlers fetch my pages?Server or CDN logsVery solid
    VisibilityDo my pages show up in AI answers?Search Console, Bing Webmaster ToolsSolid, but partial
    MentionsDoes the assistant name or recommend us?Your own prompt logDirectional only
    OutcomesDo those visits turn into leads or sales?Analytics, CRMSolid for what it can see

    Most of the bad reporting I see skips straight to layer three, because it’s the fun one. Start at the bottom instead.

    Layer 1: Make sure the crawlers can actually get in

    If a bot can’t fetch your page, nothing else in this post matters.

    Each of the big players publishes its crawler names. OpenAI’s crawler documentation lists OAI-SearchBot for ChatGPT’s search features, GPTBot for model training, and ChatGPT-User for fetches a person triggers. OpenAI says sites that block OAI-SearchBot won’t be shown in ChatGPT search answers, though they can still appear as plain navigational links.

    Perplexity’s crawler page lists PerplexityBot, which surfaces sites in its results, and Perplexity-User, which fetches pages when someone asks a question.

    For Google, it’s simpler than people think. Google’s guide to AI features and your website says AI is built into Search, and your robots.txt rules for Googlebot are the control. Google-Extended is a separate token for some of Google’s other AI systems. Blocking it isn’t how you opt out of AI Overviews.

    How to check access in your logs

    1. Pull 30 days of raw access logs from your host or CDN.
    2. Filter by user agent. Something like grep -E "OAI-SearchBot|ChatGPT-User|PerplexityBot|Perplexity-User|GPTBot" access.log gets you started.
    3. Look at the status codes. You want 200s. A wall of 403s usually means a firewall or bot-protection setting is turning them away.
    4. Check which URLs they hit. If they only ever fetch your homepage, your internal linking may be hiding everything else.
    5. Verify that the traffic is real. Anyone can fake a user agent, so check the IP against the ranges each company publishes on those same pages.

    A quick aside: most JavaScript analytics never see bots, because crawlers usually don’t run your tracking tag. If your GA4 shows zero AI crawlers, that tells you nothing. Logs are the only honest source here.

    And remember what this layer proves. Crawler visits mean access works. They don’t mean you were cited.

    Layer 2: Read the platform reports that exist

    This is the part that changed the most in 2026, and it’s the part I’d set up first.

    Google Search Console: the Generative AI performance report

    In June 2026, Google announced Search Generative AI performance reports in Search Console. As of August 31, 2026, the help page says they’ve rolled out to all websites.

    Here’s what the report gives you:

    • Impressions from AI Overviews and AI Mode combined
    • Pages, grouped by the final URL that was linked
    • Countries, devices, and dates
    • A filter for text searches versus multimodal ones, like Google Lens and Circle to Search

    And here’s what it doesn’t give you, at least as I write this: clicks, click-through rate, position, or the queries that triggered the impressions. It also doesn’t split AI Overviews from AI Mode.

    That’s fine. Use it for what it’s good at. It tells you which of your pages Google is actually putting in front of people inside AI answers.

    Two things to keep in mind. AI feature data still counts in your normal Performance report too, inside the Web search type. So don’t add the two reports together. And Google’s impressions documentation explains that every link in an AI Overview gets the same position, while a follow-up question in AI Mode counts as a brand new query.

    One more setting worth a look. Search Console now has a Search generative AI control under Settings. If someone excluded the site from AI features, the report will be empty and you’ll have your answer.

    Bing Webmaster Tools: AI Performance

    Microsoft got here first. It introduced AI Performance in Bing Webmaster Tools as a public preview in February 2026.

    The AI Performance report counts how often your content is cited as a source across Microsoft Copilot, AI-generated summaries in Bing, and select partner integrations. You get total citations, cited pages, page-level citation counts, and “grounding queries,” which are grouped phrases tied to the content that got cited.

    In June 2026, Bing added Intents, Topics, Citation Share, and Compare in preview. Citation Share is the interesting one. It shows what percentage of the citations for a grounding query went to your site.

    Microsoft is refreshingly blunt about the limits. Its own help page says the report doesn’t measure rankings, authority, or importance. It just shows which content was cited. I wish more vendor dashboards said that out loud.

    If you haven’t verified your site in Bing Webmaster Tools, do it this week. It’s free, and it’s the only first-party citation count you’ll get right now.

    Layer 3: Track AI referral traffic in GA4

    Clicks from assistants arrive as normal visits. By default, GA4 dumps most of them into Referral, where they sit between a forum link and a newsletter you forgot about.

    Pull them out into their own channel.

    Set up an “AI assistants” channel

    Google’s own custom channel groups help page walks through this exact example. Here’s the short version:

    1. In GA4 Admin, under Data display, open Channel groups. You need Editor access.
    2. Click Create new channel group. This starts as a copy of the default group.
    3. Add a channel called “AI assistants.”
    4. Set the condition to Source matches regex.
    5. Paste in a pattern for the assistants you care about.
    6. Click Reorder and move “AI assistants” above Referral. GA4 puts each session in the first channel it matches, so if it sits below Referral, it’ll never fill.
    7. Save, then open Reports, Acquisition, Traffic acquisition, and switch the dimension to your new group.

    Google’s help page includes a sample regex. I’d tighten it. Theirs opens with a pattern that matches any source containing the letters “ai,” so something like mail.google.com could end up in your AI channel. Not ideal. I prefer listing hostnames:

    chatgpt\.com|perplexity\.ai|copilot\.microsoft\.com|gemini\.google\.com|claude\.ai|grok\.com

    Before you trust it, open your Referral report and check the exact hostnames you’re getting. Add any you see, and remove any you don’t care about.

    Know where this number is weak

    ChatGPT has been tagging its search links with utm_source=chatgpt.com, which OpenAI documented in its publisher guidance. That helps. But plenty of AI-driven visits still show up as Direct, because apps and copied links often don’t pass a referrer.

    So treat AI referral traffic as a floor, not the full count. It’s the visits you can prove, not the visits that happened.

    The good part is that once these sessions have their own channel, you can see what they do. Do they read more than one page? Do they fill out your contact form? That’s the layer that matters to a business owner.

    Layer 4: Build a prompt log you can trust

    There’s no report that tells you whether ChatGPT recommends you when someone asks “best SEO agency in Miami.” You have to check by hand.

    That’s fine, as long as you do it like a measurement and not like a mood.

    How to set up the log

    1. Pick 20 to 30 prompts. Use the questions your buyers actually ask, not your brand name. Your intent-first keyword research is a great source for these.
    2. Freeze the wording. Same prompts, every time. If you change a word, start a new row.
    3. Pick your platforms. For most businesses, I’d check ChatGPT, Perplexity, Gemini, Copilot, Google AI Mode, and Grok.
    4. Control the setup. Use a logged-out or clean session where you can, note the location, and note the mode (search on or off).
    5. Run it on a schedule. Monthly is plenty for most sites. Weekly if you’re in the middle of a big content push.
    6. Log the same fields every time. See the table below.
    7. Run each prompt more than once. If you’re cited in one of three runs, write that down. That’s the honest answer.

    Here’s the sheet I use:

    FieldExample entry
    Date2026-11-21
    Platform and modePerplexity, default search
    PromptHow do I check if my site is in AI Overviews?
    Runs3
    Mentioned by name?1 of 3
    Our URL cited?Yes, the audit guide
    Other sources citedGoogle docs, two publishers
    NotesAnswer pulled a step list from our H2s

    After three months, you’ll have something worth reading: a rough sense of which topics you show up for and which you don’t. After one week, you’ll have noise.

    A word on paid tracking tools

    There are now plenty of tools that run prompts at scale and chart your “share of voice” in AI answers. Some are useful, especially if you need hundreds of prompts.

    Just ask every vendor the same questions. Which prompts? Which locations? Logged in or out? How many runs per prompt? If they can’t tell you, the chart is decoration.

    Turn the four layers into a monthly report

    You don’t need a fancy dashboard. You need the same handful of numbers, in the same order, every month.

    Here’s the checklist I’d use:

    • Access: Any new 403s or 5xx errors for AI crawlers? Did a firewall change?
    • Google: Generative AI impressions this month versus last, and the top ten pages by impressions.
    • Bing: Total citations, top cited pages, and any grounding queries that surprised you.
    • Referrals: AI assistants sessions, engagement, and key events like form fills.
    • Prompt log: How many prompts mention you at least once, out of the total.
    • Changes: What you published, updated, or fixed this month, so you can line it up against the numbers.

    That last one is the most overlooked. Without a change log, you’ll stare at a bump in March and have no idea whether it came from your new guide, a Google update, or pure luck.

    How to use the data to decide what to fix

    Measurement is only worth it if it changes what you do next. Here’s how I read the patterns.

    Lots of crawler hits, almost no impressions or citations

    The bots can reach you, but your pages aren’t being picked. That usually points to the content itself. Is the page specific? Does it answer the question directly? Is it clear who wrote it and why they know? My post on E-E-A-T in practice covers what that looks like on a page.

    Impressions in Search Console, but your best pages are missing

    Look at which pages Google does show. Then compare them to the ones you wish it showed. Often the “winners” are the pages with direct answers and clean structure, and the “losers” are long brochure pages. That’s a rewrite brief.

    Citations on Bing, nothing in your prompt log

    Totally possible. Bing’s report covers its own surfaces and some partners, while your prompt log might focus on ChatGPT. Different systems, different sources. Don’t force them to agree.

    Referral visits that bounce

    People are arriving from AI answers and leaving fast. Either the answer already covered what they needed, or the page doesn’t match what the assistant promised. Check the landing page against the prompt that likely sent them.

    Nothing anywhere

    Go back to basics. Run a proper SEO site health audit first. Google is clear that a page has to be indexed and eligible for a snippet before it can be a supporting link in AI Overviews or AI Mode. A lot of “AI visibility problems” are plain old indexing problems wearing a new hat.

    What not to do

    A few habits I’d drop right now:

    • Don’t report screenshots as trends. One answer is one answer.
    • Don’t add Google impressions, Bing citations, and referral sessions into one “AI score.” They measure different things.
    • Don’t change your prompts every month. You’ll be comparing apples to last month’s oranges.
    • Don’t block AI crawlers by accident. Check your firewall and bot settings after every hosting or CDN change.
    • Don’t promise anyone a citation. No one controls these answers. Not me, not your agency, not the tool vendor with the very confident sales deck.
    • Don’t let the AI numbers replace the business numbers. Leads and sales still decide whether this is working.

    Where this fits with the rest of your SEO

    Measurement tells you where you stand. It doesn’t move you.

    The work that actually moves these numbers is the same work I wrote about in how to rank better in LLMs and AI search: clear entities, pages worth citing, honest structured data, crawlable sites, and real mentions around the web. Measurement just keeps you honest about whether it’s paying off.

    And be patient. Meaningful SEO trust usually takes six to eight months of steady work. AI visibility rides on that same foundation, so don’t judge it after three weeks.

    FAQ

    Can I see exactly which prompts mention my brand in ChatGPT?

    No. OpenAI doesn’t offer a publisher dashboard for that. You can see the clicks ChatGPT sends you in analytics, and you can log prompts by hand. That’s it for now.

    Does Search Console separate AI Overviews from AI Mode?

    Not in the Generative AI performance report as it stands. It combines both into one impressions number. You can break it down by page, country, device, and date.

    Is Bing’s AI Performance report worth setting up if most of my traffic is from Google?

    Yes. It’s free, it takes a few minutes, and it’s the only first-party report that counts citations. Even if Bing is a small slice of your traffic, the citation data tells you which of your pages AI systems find useful as sources.

    Why does my GA4 show no AI crawler traffic?

    Because most crawlers don’t run JavaScript tracking tags. Check your server or CDN logs instead.

    How often should I check my prompt log?

    Monthly for most businesses. Weekly only if you’re actively publishing or fixing a lot and want faster feedback. More often than that mostly measures randomness.

    What’s a good number of AI citations?

    There isn’t a universal benchmark I’d trust. Compare yourself to your own last quarter, page by page and topic by topic. That’s the trend that matters.

    Want help measuring this the right way?

    If you’d like someone to set up the reports, build the prompt log, and connect it all to the work that actually improves visibility, that’s part of our SEO and AI search services. We’ll start with access and the data you already have, then build from there.

    Not sure where to start? Get in touch and tell me what you’re seeing. I’ll tell you honestly what’s worth tracking and what’s noise.