Health stories: Reading between the lines
We are all rightly concerned about our health, and recognise that what we eat has a profound effect on our physical and mental well-being. It is also very appropriate that the media - whether newspapers, magazines, TV programmes or even Internet Web sites - should recognise these concerns and provide us with sensible guidelines based on current and emerging scientific research. Few people have time to read the learned journals on diet and nutrition, and even fewer have the academic training to understand them fully. So we rely on journalists and popular writers to make the material more accessible.
The problem, however, is that the increased reporting of health and food issues has led not to clarity and confidence, but to confusion and anxiety. Is a fat-free diet really healthy? Does red meat increase the risk of cancer? How much broccoli do I need to eat, and what exactly are antioxidants? Should I be worried about genetically modified foods?
The result in extreme cases is a new kind of phobia - the fear of food itself, nourished by a diet of often sensationalist and misleading articles which purport to have the status of scientific fact. Even those of us more resistant to such neuroses can often suffer from health 'advice' indigestion - the uncomfortable feeling that we don't really know what is good and bad for us.
Away from the magazines and newspapers, we still cannot escape from the apparent complexities of a balanced diet. The humble task of shopping in a supermarket now requires lengthy analysis of product labels and at least a working knowledge of the language of 'nutrition information' - kj/kcal and gs of carbohydrate, fat, sugar sodium etc. per 100g - and Organic Chemistry 101 to
Of course we need this information. People have a right to know what they are eating, as much as they have a right to balanced information and advice about food and nutrition. But in the same way that we need new skills to decode the wrapper on a packet of biscuits, we need novel ways of discerning food fact from food fiction - sensible, scientifically based advice from cranky alarms.
We at SIRC are not alone in these concerns. Earlier this year the Harvard School of Public Health, in conjunction with the International Food Information Council Foundation, set up an advisory group to establish guidelines on the reporting of food and nutrition information. They recognised the damage that was being done to public confidence by stories and articles in the press which ranged from the somewhat misleading to simply untrue nonsense.
The group's recommendations, however, were targeted at journalists and journal editors. There was little concern for providing readers with the means of evaluating the stories for themselves. It is fine to encourage journalists to ask a few pertinent questions, rather than simply regurgitating a press release from a journal or organisation whose credibility is not always guaranteed. It is also fine to ask journal editors themselves to be a bit more responsible before attempting to boost their publication's circulation with a tasty, but not necessarily completely sound, small-scale study. But this does little to encourage the level of critical public appraisal that is required in order to limit the growth of unnecessary health panics and sheer food faddism.
By way of small remedy, SIRC has compiled its own guidelines to assessing food and nutrition media stories. These are by no means foolproof, but they will help to distinguish the more rational and soundly-based reports from those about which we should express some doubt.
What is the source?
Any report or advisory article should clearly indicate the sources of its information. Phrases such as "it is well established that" or "scientists have concluded" are indications that the story is light on evidence.
In addition, the article should make clear whether it is referring to a single study, perhaps recently reported in a scientific journal, of a series of consistent studies, all of which come to substantially similar conclusions.
A lot of media articles are based on just one 'newsworthy' journal article. Reuters, CNN and many other agencies put these out daily on their wire and Internet services. In the scientific community, however, a single study is rarely given credence unless it has been corroborated (or 'replicated') by further studies. Only then will responsible scientists be happy about circulating the findings to the general public.
Some media reports concern work which has been paid for by commercial interests. The food industry, for example, invests large sums of money each year on research concerning the nutritional and safety aspects of its products. Similarly, much reported research is funded by charitable, 'independent' or government organisations. It is often tempting to dismiss what comes from the former as mere commercial propaganda and trustingly accept the conclusions of the latter because of its 'well-meaning' intent. But that would be a mistake. While industry-funded research may be seen as having vested interests, that which is funded by a range of single-issues organisations often has similarly clear agendas. The same, critical appraisal should be applied to all.
Can we trust the journals?
Major academic journals serve to report research findings to other professionals in the science and health communities. They are the primary source of knowledge in any field. The Journal of the American Medical Association and The New England Journal of Medicine are among the best known in the United States. In the UK The British Medical Journal and The Lancet have similar 'heavy-weight' status. The independence of some of the journals has, however, recently been called into question. In the US in particular, the pursuit of advertising revenues (mainly from pharmaceutical companies) has led to a drive to extend readership and public interest in the journals.
There is rarely unanimity of opinion within the scientific and health community. While there may be areas of consensus, such as that concerning the beneficial effects of fresh fruit and vegetable intake and variation within diet, there is no magical formula which has stood the test of time or continuing research. Just about everything we eat or drink has, at some stage been shown to either be good for us, or very bad. We should, therefore, be suspicious of articles which offer no alternative views to that which is being reported.
Some journalists seem to think that basic information such as the type and size of the sample used in a reported study is just 'technical stuff' which will not interest readers. This can lead to some very misleading articles. In a lot of nutrition studies, for example, the subjects are not people but rats, mice or other species of animal. The fact that a mouse may drop dead from drinking the human equivalent of several hundred gallons of a particular fizzy drink would probably be seen by most people as a poor indication of that beverage's real dangers to real people.
In human studies we also need to know the size and composition of the sample - something which surprisingly is reported in only a minority of articles. Small-scale studies - sometimes involving only a handful of subjects - may produce quite striking results, but few scientists would be willing to draw general conclusions from them. Similarly, large-scale studies which involve, say, only middle-aged medical practitioners may not produce results which are relevant to younger and less professional people.
Pies, Damned Pies and Statistics
It is commonly assumed that you can do anything with statistics. That, in fact, is not true. Statistics should serve to prevent the wrong conclusions being drawn. They should also rule out reliance on results which might be purely due to chance - random variations which occur all the time. The problem arises only when statistics are used in a highly selective manner or where the basic rules of analysis are broken - not always easy to determine from a journal article, let alone a news report. A good journalist, however, should at least tell us if the results appear to be statistically significant or not. Be very suspicious about the use of words such as 'trend'. This usually indicates that the results were not significant and that no confidence should be placed in them.
Even if the results are genuinely significant, in statistical terms, the conclusions drawn may still be misleading. Suppose, for example, we read that the regular consumption of a particular food stuff (F) significantly increases our chances of developing a particular type of disease (D). Often the results will be presented in terms of 'relative risk'. If the relative risk is, for example, 2.1 then the study shows that people who eat F are more than twice as likely to suffer from D. This sounds quite alarming. So should we refuse to eat F? Not necessarily.
What we need to know, and what we are very rarely told, is the absolute risk of D - in other words, how common is D in the population as a whole. If D is quite common - perhaps one person in 50 suffers from it - then doubling the risk of D is quite serious. The odds have changed from one in 50 to 2 in 50. But if D is a very rare disease - only one person in a million will suffer from it - then the implications of the study are much less dramatic. Even if you eat F you are still very unlikely to suffer from D.
The lack of this kind of informed reasoning in media health stories is a major cause of the spread of disinformation, and it is often cynically exploited by those who seek to scare people into following particular dietetic ideologies. Nowhere has this been more evident than in the banning of beef on the bone in the UK because of the risk of new variant CJD. All of the data pointed to the fact that although there might be a minimal, but unproved, increase in relative risk from eating such beef, the real risk of contracting CJD in this way was, and still is, so small as to be virtually zero. In this case, however, people were not allowed to make up their own minds about whether they should or should not eat rib of beef.
With very common causes of mortality, such as cancer, stroke and heart disease, the picture is even more confusing. A food substance, such as salt may be shown to increase blood pressure in many people, and increased blood pressure is associated with strokes. So, you might think, avoiding salt might be a way of preventing death from a stroke. This, however, would be a reasonable conclusion to draw only if the data showed that people who eat a lot of salt die of strokes at a significantly younger age than people who eat less salt.
Stroke, like heart disease and many cancers, is a disease which afflicts most people in the later stages of their lives. The sad truth is that we will all die of something. Only where a substance or an aspect of lifestyle can be shown to shorten life expectancy significantly, rather than just being associated with one reason for dying rather than another, can it be said to be truly harmful.
Rarely do major diseases have a single cause. They are usually the result of a complex interaction between many factors, including genetic, environmental and lifestyle components. Many media reports, however, can lead us to believe that if we gave up something which we might otherwise enjoy, we could completely escape that particular affliction.
Clearly, this is not the case. Non-smokers die of lung cancer. Vegetarians die of cancer of the colon. Tee-totallers die of liver complaints including cirrhosis. People who never go out in the sun contract skin melanomas. Always, there are other factors at work than the single element being examined in a scientific study.
Quite often the alleged benefits of a particular nutritional element are the result not so much of the element itself but of the lifestyle and general diet of the people who consume it most. People who eat lots of 'healthy' foods, including fibre, carrots, broccoli etc. also tend to drink alcohol less, take more exercise, avoid too many fatty foods and smoke less. Only when a study can rule out all of these other factors - and often we do not know what these factors might be - can we say that there is a causal link between two things. That is not to say that things like fibre and broccoli have no beneficial effects at all. But there is no point in being a boozy, chain-smoking couch potato and expecting these foods to make you healthy.
There is, however, one thing which increases the likelihood of dying relatively young, even when all of the other factors have been taken into account. It is one of the biggest killers even among those who lead lifestyles which, by any criteria, are clearly healthy ones. This single, incontrovertible risk-factor is that of being poor.
Read past the headline.
Headlines above articles in newspapers are normally written not by the journalists themselves but by sub-editors. Their job is to make even routine reports sound interesting. In doing so, however, they can be very misleading. A recent story, for example, announced "Health Scare Over Mineral Water". A control failure at a bottling plant had allowed small amounts of benzene to be retained in the mineral water. On reading the story, however, it was clear that health inspectors and independent scientists all agreed that this had posed absolutely no health risk at all. The bottles were withdrawn purely for quality control reasons. Those who did not bother to read the story properly, however, were left with the association of 'health scare' and 'mineral water' - something which was quite unfounded.
So how can I believe anything?
The point of identifying pitfalls and biases in media reporting of health and food issues is not to make everybody switch off. A total resistance to health messages is as dangerous as the uncritical acceptance of every scare story. Rather, it is to encourage a bit of critical examination - just like buying a used car.
Following the guidelines above will help. But so too will checking out the sources of the article and other views which may exist concerning the conclusions drawn in it. There are thousands of web sites which will help you to do this, and some of the more sensible ones are listed in our links pages. You can even access professional databases such as MedLine to see what other research has been done in a particular field.
The media play a vital role in alerting us to real hazards and dangers associated with food and nutrition. They can warn us, for example, of the possible side effects of quack remedies or miracle diets. They can also point us towards balanced diets which not only keep us healthy but which are actually enjoyable to eat. On the other hand, the sheer volume of health news and nutrition 'advice' can leave us punch-drunk and totally confused. We need to be able to identify what is sound and useful and to reject what is clearly nonsense. SIRC's regular features on stories around the world are designed for this purpose. But don't take them on trust either - check them out just like any other reports.