Nutrition as a science is extraordinarily broad and the relationship between our diets and our health is complex. Whilst there is new research in this area being published every day all research has limitations. It’s a much harder and time consuming task than you may think for scientists to come up with clear, clinically proven answers to the question, what should I be eating for optimal health?
To illustrate some difficulties with researching diets I’m going to discuss research in terms of the Paleolithic diet (I’m not arguing for or against the Paleo diet, I’m trying to illustrate some of the challenges of researching how diets affect our health in general).
I performed a quick search of my university database for existing academic literature on the Paleo diet. I refined the search to articles published in a scholarly or peer reviewed journal (to eliminate conference proceedings, ebooks, book reviews and so on). That pulled up just over 2,100 articles. I further refined that to actual research articles, there were only 103. Results will vary depending on which databases you are searching. However my point I’m trying to make here is whilst there are thousands of blogs, books, magazine and newspaper articles on the subject the academic peer reviewed research is still relatively scant.
Let’s take a closer look at the first article from my refined search. It was a 2010 study by researchers at the University of Lund, titled, “A Paleolithic diet is more satiating per calorie than a Mediterranean-like diet in individuals with ischemic heart disease.” The research studied 29 male IHD patients who had either impaired glucose tolerance or diabetes type 2 with a waist circumference greater than 94cm. They were randomized to eat either the Paleo or Mediteranean diet and the patients kept a weighed food record and subjective rating of satiety. The results showed that the Paleolithic group were as satiated as the Mediterranean group but consumed less energy. The Paleo group (but not the Mediterranean) also showed relative changes in leptin, weight and waist circumference. The researchers concluded that the Paleolithic diet is more satiating per calorie than a Mediterranean-like diet.
The research method used a randomized controlled trial (or RCT), the “golden standard” of nutritional research, but even RCTs have limitations. Here, the sample size is small (29 patients), they were all men and with a pre existing condition. It’s quite likely that these results can be replicated in other groups of people, but they weren’t testing premenopausal women or African Americans or children, for example. The article also doesn’t state what other factors were controlled, such as exercise, stress or sleep. These are all important things because something we do know in this area is that genetics, environment and age all play a part in how our bodies interact with food.
Other limitations include: part of the data was based on the patients’ subjective reports, which is a common measure, but it is subjective data and we need to keep that in mind. Furthermore, the study duration was only 12 weeks so no long term effects can be observed (longitudinal studies for diet are really expensive and challenging).
I’m not ripping apart their methodology. The research design was fine and quite typical of this kind of study. The researchers themselves understand the limitations of their research. As readers we just need to remember that there are limitations. Sometimes researchers can over interpret their data (leap to a conclusion that isn’t there), although quite often, it’s the readers and distributors of that information that do this too. We can’t say from this research that the Paleo diet is the all round best or healthiest diet and we certainly can’t conclude from this study that the Paleo diet can cure diabetes.
Building banks of knowledge is a slow process because research needs to be focused in order to isolate results. This research used the independent variable of diet (that is, the bit that you are changing to see the effects) to focus on only a few key dependent variables (what you are observing) of satiety, weight, leptin and waist circumference. Too many variables and researchers can’t come to any meaningful conclusions. It means we learn just pieces of the puzzle at a time. Sometimes we think the puzzle pieces are coming together in one way, but in fact, down the track, they end up coming together in a different way.
Furthermore, often we can observe results but are not sure why we are seeing what we are seeing. For example, this research observes that the Paleo diet was more satiating than the Mediterranean diet. But why is that so? Was the Paleo diet more satiating because of the lower carbs? Or lack of sugar? Or lack of dairy? Or was it the increased fat? Or protein? We can make good guesses. We can combine a number of studies to help build the picture (there are numerous studies on low carb, high protein diets that we could help with our reasoning here) but the truth is, there is still so much that we really don’t know for sure. Different views on how the pieces of the puzzle fit together can result in quite different views on the bigger picture.
So when doctors claim that the Paleo diet can reverse insulin resistance, cure autoimmune diseases, get rid of your acne, irritable bowl and leaky gut are they lying? No, I wouldn’t say that. There is a huge amount of information that is derived from observations and experiences reported by patients and / or people experimenting on themselves. These subjective self reporting results can be very compelling and shouldn’t be ignored (and note some of the data used in the academic research article was collected in a similar fashion). The difference being self reporting, particularly to a doctor you have a relationship with and one that you have chosen, assumably because you like their approach, can be limited by selective memory, exaggeration and bias. It’s a great base for further research analysis. However, it is not scientific proof. One doctor with a particular approach can have excellent results with his patients whilst another doctor with a completely different approach can also have excellent results with her patients. Hence, another way we end up with two conflicting views to the “best” approach.
Researcher bias happens when data collection and analysis is skewed by the researcher’s prior perceptions. This is a real problem. But I’m also seeing a lot of blogger and blogee bias out there too. Bloggers, I find, are very passionate people and extremely defensive about their choice in how they eat. I think it’s great that bloggers are getting nutrition information out there, but many bloggers will proclaim (without even batting an eyelid or a reference for that matter) that a certain diet can turn you into a super model over night and find you a perfect partner to boot. Blogees can be just as bad. It’s common to cherry pick what we read to support our preconceived positions. We feel good finding other people that agree with our ideas. If we like a particular way of eating we are likely to look toward “more of that” than we are to search for the opposite view. There’s something to think about.
I think another way we’ve ended up with not just different views on what to eat but such extremes of the spectrum is (and perhaps I’m sounding a little skeptical here) sensationalism sells. Diet books that can divulge a new secret I imagine sell better than ones that are too cautious in their recommendations.
The good news is there are lots and lots of sound research projects being carried out on the links between diet and health and we are learning more about this complex and interesting field on a daily basis. Furthermore, whilst there are many extreme views on nutrition out there, there is also quite a lot “they” can agree on and I’m going to talk about that in a future post.
For now my message is, before we go blindly following every piece of research we read let’s just be a little open minded to the limitations.
The article I refer to in this post is:
Tommy Jönsson, Yvonne Granfeldt, Charlotte Erlanson-Albertsson, Bo Ahrén and Staffan Lindeberg. A Paleolithic diet is more satiating per calorie than a Mediterranean-like diet in individuals with ischemic heart disease. Nutrition and Metabolism 2010, 7:85
And please go here to like The Leaping Zucchini on Facebook.