A worrying amount of data in the soft sciences depends on the results of questionnaires and surveys – and plenty more money is spent by business and government on polling individuals to find out what consumers/the nation thinks, but how much of what we read in survey results is true?

This isn’t a scientific piece of research, but I offer two significant pieces of evidence.

The first is about a survey from a respectable organization that asked about green companies – the sort of survey beloved of journalists who need a bit of copy. The team in question, according to their press release, polled ‘1,500 opinion formers.’ Sounds impressive, but I was a bit suspicious that it didn’t say how many responses they got. I asked, and to give them a tick for honesty, they told me that they had had 90 respondents. Doesn’t sound quite so impressive as 1,500, does it?

My second piece of evidence indicates that the responses made in surveys might be misleading. One of my (many) failings is quite enjoying filling in the occasional questionnaire. I realize this is a failing (and is probably why the novel I once wrote with a hero whose hobby was answering those Readers Digest style ‘you, yes you MR FRED BLOGGS of ACACIA AVENUE have won…’ competitions never got published), but there it is – especially as quite a few survey companies now offer jolly prizes and (rather measly) financial incentives for filling in their questionnaires.

I am always – well almost always – honest when I do this, but I often find I take a particular slant, because it’s obvious that this isn’t what the survey writers want to discover. And I suspect (though I haven’t taken a survey) that there are many others out there like me who skew the data through being too knowing or just downright perverse.

You may wonder how I know that the writers of the surveys have a particular outcome in mind. Apart from it being a touch naive to think otherwise, I recently saw a survey with this, verbatim content as the label for one of the check boxes (I have replaced the brand name to protect the semi-innocent):

Ethical presents are a fad (for example, saving an acre of rainforest) could we take this out as it conflicts with the XXXX brand

... someone had obviously flagged up the survey for an edit, but the comment had been missed. Hmm.

So next time you read ‘95% of the country don’t believe in global warming’ or ‘6 out 10 voters don’t know what a molecule is’ or whatever, add a big pinch of statistical salt.