Method
Effective sample size
Effective sample size is the honest count of how much information a weighted sample actually carries. Because prompts and assistants carry unequal importance weights, a batch of several hundred answers can carry the statistical strength of considerably fewer, and every confidence range has to be built from the smaller number.
Why the row count is the wrong number
Weighting concentrates influence. When a handful of prompts carry most of the weight, the result rests mostly on those few, and quoting the total number of answers collected overstates what the figure is standing on. Using the raw count would make every confidence range look narrower than it really is.
Where it shows up
In the width of every reported range. A slice with few prompts, or with very uneven weights, gets a wider range — which is the mechanism that stops narrow slices from looking more certain than they are.
Fewer answers than the row count suggests
A month's batch might collect several hundred answers, but if a handful of prompts carry most of the importance weight, the result rests mostly on those few. The honest sample size is materially smaller than the number of rows collected — often something closer to half — and every confidence range is built from the smaller figure.
Related terms
- Confidence range — Built from this number, not from the raw count of answers.
- Importance weight — Unequal weights are what make the two numbers differ.
- Noise threshold — Also derived from it, which is why narrow slices need bigger moves.
Last reviewed . Definitions are reviewed quarterly, and whenever the underlying measurement changes.
Don't let AI decide your brand's future without you.
See exactly where you stand vs. competitors—and what to do about it.