How many times did you wash or sanitise your hands yesterday? How many people did you stay near within 2 meters for more than 2 minutes yesterday?
Surveys asking citizens these two questions, or very similar ones, have informed several governments policy-decisions as to whether people are adhering to official COVID-19 restrictions. In Denmark prominent political researchers, conducting these surveys, have even made headlines announcing that “CITIZENS ARE SLACKING ON THEIR BEHAVIOUR!” and have weighed in on crucial government decisions based on the answers.
The only problem is that survey questions pertaining to such peripheral routine behaviours are not valid methods for measuring actual behaviours – according to our most recent experiment published in the leading behavioural science journal Behavioural Public Policy.
How we reduced hand hygiene compliance by almost 50%
In the paper Reporting on One’s Behaviour: a survey experiment on the non-validity of self-reported COVID-19 hygiene relevant routine behaviours we replicated two of the core questions of the 27.2 million DKR HOPE-project from Aarhus University which has been a main academic source informing policy-decisions as to the behaviour of Danes during the pandemic.
Fact Box: The HOPE Project
The HOPE project is an interdisciplinary research project with the goal of investigating how democracies react and cope with the COVID-19 pandemic and what the effects of these reactions and coping strategies are.
The HOPE project group, in their own words, seeks to investigate how the trajectory of the pandemic, decisions of governments, international media and social media landscapes as well as citizens’ behavior and well-being all interrelate and affect each other.
The only twist was that we added an experimental design testing the validity of such questions by anchoring the answers of a national representative sample. The results showed, as expected, that people cannot report on such peripheral routine behaviours as they are not registered in episodic memory. We reduced reported hand hygiene compliance by almost 50% merely by anchoring the questions!
Still, such surveys have been widely used to inform crucial policy-decisions relative to the COVID-19 pandemic.
Two lessons learned
Two lessons may be learned from the paper. The first: if you want to know about people’s behaviours, sending out surveys is not a valid approach in most cases – and identifying those cases may be done by use of methodologies from Behavioural Insights. The second: in studying routine behaviours, you have to focus on behavioural patterns as they play out in their contexts, rather than people and their self-reports concerning such behaviours.
To read more about the experiment and how to go about measuring the frequencies of COVID-19 relevant behaviours go read the paper here.