"The Latest Research"
by Dr. Mike Israetel, Co-founder and Chief Sport Scientist |
Mar 01, 2015
For lifters that want the best results, a continual education in science is a must. Continual, because science is always being updated as new research is done and new discoveries are made. In the search for new data on training and diet, a couple of strategies to employ and pitfalls to avoid may be helpful.
1.) Science does not change overnight (usually)
If a new study comes out with a piece of data that's completely unprecedented, this does not mean that it cancels out all other studies before it. For example, a study recently (2014) came out describing the possibility of gut microbe alterations caused by artificial sweeteners, and a tentative link between artificial sweetener consumption and insulin dynamics was drawn. Does this mean that all of the hundreds of studies on artificial sweeteners that found no connection to insulin dynamics were wrong and can be swept under the rug? Absolutely not. This study merely suggests that a possible relationship MIGHT exist... a relationship that will have to be investigated further in multiple future studies so that a meaningful relationship (or lack thereof) may be understood. No matter how new a study or how old the previous studies, novelty does not directly add to truth value. In the same vein, when comparing two studies, the newer study does not get any bonus points for being new... only the study design can be considered a plus or minus in the comparison. Many of the best studies on carbohydrate metabolism were in the books by the late 70's, and so far they've stood the test of time perfectly well. Much better than many other newer studies, in fact.
2.) Using reviews in the proper timeframes
Not only does a new study not get any credit just for being new, we also have to remember that a single study proves very little, if anything at all. The best path to the likely truth is a collection of all the studies on a topic. Such a collection is called a comprehensive review of the literature, and it's essentially the Golden Fleece of science. A new study may suggest a new interesting area of research, but it takes dozens of studies to show a time-tested, conclusive relationship. If you look at ALL of the studies on a subject at once, you can tell which way the general consensus points. Because reviews are a serious and highly demanding piece of work, and because they require lots of studies for inclusion (if a question has only 5 studies done on it, a lit review is pointless as no strong causal relationship can only be drawn from that few studies), they are not done very often. A good review follows the previews good review by between 5 and 10 years in most cases. That's enough time for multiple new studies to be done on the subject, and potentially either add data to support the old conclusions or even to suggest new ones. Doing a review every 2 years might be too often, as not much in the way of new research can be included, and the review is essentially replicated from the old one, with so much work offering no novel perspective.
Thus when citing reviews, it's TOTALLY OK if the latest review is between 5 and 10 years old at the time of citation. Because so few studies are done in the interim (especially if it's a very focused topic and the timeframe is closer to 5 years), even if they were included, the main conclusions of the review would likely not change much, if at all. The BALANCE of the evidence would still likely be the same, or very similar. Thus when you use a review that's 8 years old to refine your diet or training and someone critiques it for its antiquity, don't worry about it! In lively and well-researched fields, you wont really have to meet that situation for long, as new reviews are in fact published every 5-10 years anyway! New reviews are best, but even older reviews are almost always better than any one new study.
3.) If multiple reviews are available, check them ALL
In mature fields, multiple comprehensive reviews of the literature are often available. In fact, some fields are so complex, that no one review can truly be completely comprehensive, and it takes multiple reviews to form the best tentative understanding. For example, the relationship between saturated fat intake and cardiovascular disease has been studied for decades. To date, 14 attempts at comprehensive reviews have been published. These reviews have many of their studies in common, but not all. Different reviews have slightly different inclusion criteria. Some only limit their studies to those on healthy people. Some on diseased populations. Some on longitudinal studies, some on cross-group comparisons, and some even make an attempt to include most all kinds of data. What were are left with is 14 reviews over twice as many years, each review covering a big segment, though not all, of the studies done to date on saturated fat's relationship to cardiovascular disease. So which review is the right one? Is it the newest?
Nope, the answer is that ALL reviews must be considered. Just like all studies taken together are better than just one study, all reviews taken together are likely better than any one review. Where do you get information pooled together on ALL the reviews? Well, a textbook is the best place to go for that. In fact, one main function of textbooks is the aggregation of all reviews in a field, in order to communicate the most general and widely accepted conclusion to the reader. When we look at the saturated fat and disease controversy, we can assess that the large majority of the reviews point a positive association (saturated fat in the diet likely contributes to higher CVD risk). Several other reviews are less conclusive, and one review proposes that saturated fats carry no additional disease risk. Because the latter review is also one of the latest, people were citing it left and right to justify the "bacon craze" of 2012. What they were missing were the 10 or so other reviews that suggested a much more unfortunate relationship. When it comes to reviews, there usually aren't many more than 10 in a single branch of exercise and sport science (14 is way off the norm), so it's very realistic to be able to gather them all and make a sound conclusion, especially if you have textbook access!
In your quest for updated knowledge, use new studies as suggestions (rather than definite truths), look to the comprehensive reviews, and if needed, check out ALL of the reviews so far done. YES, that's a lot of work (and yes, a textbook or trade book like the RP Diet might come in super handy), but having done this work you can be MUCH more sure that you're doing the right thing or at least not doing the wrong one. On the other hand, you can be one of those people that blows new studies out of proportion, changes their training, and then embarrassingly changes the training back when it's made clear the study was not representative of reality. Your call. :)
Don't forget to check out Dr. Mike's latest product, the RP Automated Diet Template at the link below: