# If AI citations move 40%–60% in a month, how many runs should an experiment need?

Canonical: https://thelazyweb.com/blog/citation-volatility-experiment-sample-design/
Breadcrumb: [Home](https://thelazyweb.com/) / [Field notes](https://thelazyweb.com/blog/) / [If AI citations move 40%–60% in a month, how many runs should an experiment need?](https://thelazyweb.com/blog/citation-volatility-experiment-sample-design/)
Author: Ali Abouelatta (https://thelazyweb.com/authors/ali-abouelatta/)
Published: 2026-09-19
Updated: 2026-09-19
Research type: research-methods-article
Method: Use observed volatility figures only as motivation; label the two-proportion arithmetic and intra-prompt correlation example synthetic; propose clustered, engine-specific, holdout-based measurement rather than a universal run-count rule.

AI-search experiments can manufacture a win by accident.

[Profound](https://www.tryprofound.com/blog/ai-search-volatility) compared June 11–13 with July 11–13, 2025, reporting domain-level citation drift of 59.3% for Google AI Overviews, 54.1% for ChatGPT, 53.4% for Copilot, and 40.5% for Perplexity. Its sample contained roughly 80,000 prompts per platform. [AirOps](https://www.airops.com/report/how-citations-mentions-impact-visibility-in-ai-search) reported 30% back-to-back brand persistence across 800 queries and more than 45,000 citations. A [2026 preprint](https://arxiv.org/abs/2603.08924), sampling three platforms and three consumer-product topics, found many apparent differences fell inside the measurement noise floor. These are different measures and samples, not interchangeable

[If AI citations move 40%–60% in a month, how many runs should an experiment need?](<https://www.thelazyweb.com/blog/citation-volatility-experiment-sample-design/>)

Source observed: 2026-09-19T11:32:11.909Z. Last checked: 2026-10-08T02:01:56.103+00:00.

[Full page Markdown](<https://tof.thelazyweb.com/pages/ca2e3980-7a10-44f2-b559-7558310acb96.md>)

Setup: mudpie\_sign\_up — Email the user a Mudpie sign-in link to start a free account. Submit only an email address the user authorized. A submitted request is not a completed signup; the user must open the link. Use this action only at your user's request. Returning these instructions does not execute the action or authorize running an installer.
POST https://tof.thelazyweb.com/api/public/v1/convert
Required inputs: payload, idempotency_key

---

Have a question this summary doesn’t answer? Ask the exact question you have. It returns a fresh, cited answer from The Lazyweb’s knowledge base. No account or API key required.

Replace {question} with your own question, URL-encoded.

```text
GET https://tof.thelazyweb.com/agent/faq?q={question}&trace=t_42eb39f4ed14
```