How we measure

What we ask, of which systems, and what the results cannot show.

Systems and sample

Each report names the systems measured, the question sample and the measurement date, so you can see exactly what the result describes.

API and interface

Consumer-interface differences are recorded. A web-enabled API and a non-web-enabled model are not compared as if identical.

Questions and repeats

You confirm the exact wording and locale. Wording, locale and system context are frozen for comparisons. Each question is asked more than once, because the same question can return different answers.

What is recorded

  • Mention
  • Recommendation
  • Negative context
  • Ambiguous result
  • Cited source
  • Failed request

A provider error is not proof that a brand was absent.

Limits

  • A change of model, source access, locale or question set limits a direct before-and-after comparison.
  • A sample frequency is not consumer market share.
  • The software cannot prove that a particular fix caused a change.