I was really looking forward to last night’s Horizon on BBC2. The BBC’s flagship science programme was coming over all consumer, and I’ve been a sucker for such television ever since I appeared on Watchdog (as an expert, not a con man).
The idea was to take a rampant professor round a supermarket and she would highlight what made scientific sense, and what didn’t, from the claims for the various products. Ace.
The professor herself was a bit of a let-down, because she was an professor of obstetrics. I’ve nothing against the medical profession, and we’ve the wonderous Ben Goldacre who manages to be both a doctor and a great spotter of scientific no-nos, but I suppose I was expecting a ‘real’ scientific professor. But that’s just a physics degree talking.
It started well with the Prof. laying into statements based on too small a trial and such like sins. Then, however, the programme did its own test, on superfruits. This involved five (yes, five!) sets of twins, one of which ate extra portions of ordinary fruits, the other extra portions of superfruits. They compared changes in the vitamin levels in their blood afterwards.
Now I’ve got at least three problems with this. The ridiculous sample size. The total lack of even single blind conditions. And the test didn’t seem even to be looking for what superfruit enthusiasts claim is their main benefits – rises in anti-oxidant levels. Did the Prof. mention any of these shortcomings? No.
Later on, checking out the merits of organics there were two strange happenings. First, the Prof said there were no large scale trials showing nutritional benefits for organics, instead briefly dismissing a couple of trials, small by implication, that had contradictory results.
From a comment made during an interview, it seems likely the programme was made last June/July, which would mean at least some segments were filmed before the outcome of the major EU/Newcastle University trial published in October. Yet it seems remiss of the programme makers not to mention this, instead leaving the public with the opinion there were still no large scale trials.
A little later, the Prof. compared welfare standards at Eastbrook Farm (just down the road from me), a big organic farm, and a conventional factory-style farm. This comparison seemed wrong. She should surely have compared welfare on a non—organic but still free range farm (there’s one practically next door to Eastbrook), to see if the organic part made a difference, but chose instead to include extra variables. This was supposed to be testing whether organics’ welfare systems are any better, but ended up comparing two different things.
All in all, I wasn’t happy. Am I too fussy? I don’t think so. The BBC’s flagship science programme, specifically trying to be scientific about these product claims, could have done a lot better.