Zasobyaninaa guide to ratings, polls and statistics

An independent guide to ratings, polls and statistics. Not affiliated with the former website on this domain, with politicians, parties or government authorities; this is not campaign material.

Practical articles

Practical articles

The first article in this section explains how to check a number from the news or an advert. The result: you find the original source, understand what exactly was measured and among whom, estimate the margin of error, tell percentages from percentage points and the mean from the median, notice truncated chart axes and stop confusing coincidence with cause. All numbers in the examples are hypothetical and made up purely for illustration.

Checking a number is not about hunting for a trick. It is five to ten minutes in which you find out what the number is really telling you.

— Editor's note

Before you start

  • The number itself and the exact phrase it appears in: "rose by", "on average", "one in two"
  • A link to the source, if there is one, or the name of the organisation being cited
  • A couple of minutes to open the original source, not just the retelling
  • Being prepared for the answer to be "there isn't enough data" — that is a result too

  1. Find the original source

    Retellings often lose important details: who carried out the study, when and among whom. Look for the original publication — an organisation's report, an official statistics table, a methodology page. If the original source is not given and cannot be found, treat the number with caution.

    Hypothetical example: an article says "according to a study, 70% of shoppers prefer X". If it never says whose study it is, this is not yet a fact but a claim.

  2. Find out who commissioned and who conducted it

    A sound poll discloses who commissioned it and who carried it out. Being commissioned does not in itself make a result false, but it helps you understand why these particular questions were asked and why these particular numbers were published.

    Hypothetical example: a survey on the benefits of a product was commissioned by its manufacturer. That is a reason to read the question wording more carefully, not to reject the result automatically.

  3. Work out what exactly was measured

    Look at the exact wording of the question or the definition of the indicator. "Satisfied" and "fairly satisfied" are different answers; "have used at least once" and "use regularly" are different indicators.

    Hypothetical example: "80% have tried the service" does not mean that 80% use it. You can try something once and never come back.

  4. Check the sample and the margin of error

    Find out the size and composition of the sample, the survey method and the dates. For a simple random sample at a 95% confidence level and a proportion of around 50%, the margin of error is roughly ±0.98/√n: about ±4.9 points with 400 respondents, about ±3.1 with 1,000 and about ±2.2 with 2,000. This margin does not account for poor question wording, non-response or an unrepresentative sample.

    Hypothetical example: in a poll of 1,000 people, 52% choose option A and 48% option B. A 4-point gap is comparable to the margin of error, so you cannot confidently claim that one is ahead.

  5. Tell percentages from percentage points

    If a share rises from 10% to 15%, that is an increase of 5 percentage points but of 50 percent. Both statements are correct, but they make a different impression. Also check whether the absolute size is given: relative growth from a small base looks more impressive than it is.

    Hypothetical example: "sales doubled" could mean a rise from two units to four. Without absolute numbers, no conclusion can be drawn.

  6. Check whether it is a mean or a median

    The arithmetic mean is sensitive to extreme values; the median is not. That is why incomes, house prices and similar indicators are often reported as medians. If a text simply says "on average", check what exactly was calculated.

    Hypothetical example: nine people earn 50,000 each and one earns 1 million. The mean is 145,000 and the median is 50,000. Here the median describes the typical person better.

  7. Look at the chart axes

    A truncated vertical axis turns a small difference into a chasm, and uneven intervals on the horizontal axis distort the speed of change. Check what value the axis starts at, the spacing of the tick marks and the period the data covers.

    Hypothetical example: the axis starts not at zero but at 95. Bars with values of 96 and 98 then look as if the second is three times taller than the first.

  8. Don't confuse coincidence with cause

    Correlation does not imply causation: two quantities can move together because of a third factor or by chance. What is more, when groups are combined, a trend can disappear or reverse — this is Simpson's paradox. Before concluding that "A causes B", ask what else might have affected both.

    Hypothetical example: on hot days, both ice cream sales and the number of swimmers go up. Ice cream does not make people swim — both depend on the weather.

Common mistakes

  • Trusting a retelling without opening the original source
  • Assuming a large sample is automatically a good one
  • Treating a difference smaller than the margin of error as a real difference
  • Confusing percentages with percentage points
  • Not noticing that a chart's vertical axis does not start at zero
  • Drawing a conclusion about cause from a single correlation

Alternatives

  • If the original source cannot be found, look for the same topic in official statistics publications
  • If the survey methodology is not disclosed, compare the result with other studies on the same topic
  • If you have no time, hold off on a conclusion and do not pass the number on until you have checked it

Final checklist

  • The original source of the number has been found
  • The commissioner and the provider of the research are known
  • The exact wording of the question or the definition of the indicator is clear
  • The sample size, survey method and dates are known
  • Percentages have not been confused with percentage points
  • It is clear whether this is a mean or a median
  • The chart axes have been checked
  • No conclusion about cause has been drawn from a single correlation

Article catalogue for English-speaking readers

This section will feature standalone articles for English-speaking readers in the UK, the US and elsewhere. They are written with an eye to where readers usually find their own national statistics and how rankings, reviews and polls are typically published in English-language media, and they are not translations of materials from other language versions of the site.

Topics the editors are planning

  • How to read school league tables and university rankings: what lies behind the scores
  • Star ratings online: how an average score works and how many reviews you really need
  • Finding the right table in national statistics releases without getting lost
  • Inflation and prices: why your own shopping experience can differ from the official figure

Editorial rules

  • Articles in this catalogue are written separately for each language version and are not translations.
  • Some articles link to third-party websites, including gaming services; we do not control their content, prices or terms.
  • We do not publish our own rankings, unverifiable polls or reviews.
  • Questions and answers

    Short answers about polls, ratings, reviews and basic statistical concepts.

  • Reference

    A glossary: sampling and polls, descriptive statistics, charts and data.

  • Reading lists

    Reading paths: getting started, before you buy, numbers in the news.

Sources

  • Rosstat (Federal State Statistics Service)
  • AAPOR — Transparency Initiative
  • ESOMAR — International Code
  • Textbooks on statistics and survey methodology
  • Encyclopaedic sources

Information is current as of the preparation date. Research methods, official statistics releases and the standards of professional associations change over time, so always check the latest data in the original sources.