Letters on Evidence
It is often helpful to see what others are actually doing in the process of evidence review.
We will periodically share some of our experiences in hopes that they will share the lessons of evidence work.
Dear Colleagues in Evidence-Based Care,
When does gold become tarnished? A case of the literature on Donor Breast Milk for Preterm Infants—a teachable moment on systematic reviews
Systematic reviews, especially with meta-analysis, are often considered scholarly works at the top of the pyramid (See the Oxford Centre for Evidence-Based Medicine for example, at http://www.cebm.net/?o=1025.) This is because they typically combine randomized controlled trials (RCTs) that may be limited by small sample sizes, enabling stronger conclusions to be better derived. Yet, this seemingly golden offering of scholarly literature may have its limitations.
A systematic review by Boyd, Quigley, and Brocklehurst (2007) on donor breast milk for preterm infants compared to formula is a frequently quoted reference on the subject. The seven studies examined included five randomized controlled trials. The section of the review that receives the most ongoing attention in the literature is the combined effects of three of the studies by meta-analysis on the variable of confirmed necrotizing enterocolitis (NEC), a complication of high concern in premature infants. This analysis combined two RCTs and one observational study. Separately, the sample sizes for these studies ranged from 39 to 162; the combined sample size became 268. Individual study results did not meet the minimal level of statistical significance of p=0.05. Combined evidence from these studies created a relative risk (RR) of 0.21, 95% CI of 0.06-0.76, p=0.017. The conclusion was that donor milk reduces NEC by about 79% compared to formula. At face value, this is an enticing result. Why the worry?
The concerns with this analysis are partially acknowledged by the authors. The articles used for this study are dated no later than the early 1980’s with data from the 1970’s and the beginning of the 1980’s. Babies included in the studies were 30-33 weeks of age and 1310-1954g, much larger than the premature infant surviving today in our neonatal intensive care units. These studies involved non-fortified milk and exclusive feeding of the control and treatment groups. This too is contrary to today’s practices as fortification is much more the standard practice now. One of the authors (Quigley) went on to perform an updated review. This review (Quigley, Henderson, Anthony, McGuire, 2007) is published in the Cochrane Database of Systematic Reviews, a much revered source of scholarly literature. Here five studies were combined in meta-analysis with the addition of a more recent study (Schanler, 2005) of sizeable impact. Unlike several of the other comparisons in this document, the heterogeneity is assessed to be low (I2 of 0.00%) and results favored donor breast milk, with a RR of 2.46, 95% CI 1.19-5.08, p=0.015. This confidence interval was much better than some of the singular studies that reported such wide variances as 0.11 to 60.38. Yet, more limitations exist in this review. Growth restricted preterm infants that are already at high risk for NEC were noted here as excluded. Again, many of the studies came from the pool described in the article by Boyd and colleagues and infant size and age along with the fortification issue remain. Preparation of the donor milk may also have differed in the early studies compared to today. In summary, the golden scholarly product is tarnished.
A final note is that evidence reviews cannot end with the statistical analysis. Donor milk costs $3.50 per ounce or more via standard milk banks. Cost effectiveness needs to be evaluated in order to make this costly recommendation. Given the limitations of the literature and the cost involved, a local team of experts decided against widespread adoption of donor breast milk for premature infants.
Boyd, C. A., Quigley, M. A., & Brocklehurst, P. (2007). Donor breast milk versus infant formula for preterm infants: Systematic review and meta-analysis. Archives of Diseases in Childhood, Fetal and Neonatal Edition, 92, F169-F175. doi: 10.1136/adc.2005.089490 Quigley, M., Henerson, G., Anthony, M. Y., McGuire, W. (2008). Formula milk versus donor breast milk for feeding preterm or low birth weight infants. Cochrane Database of Systematic Review 2007, Issue 4. Art. No.: CD002971. doi: 10.1002/14651858.CD002971, pub2
Dear Colleagues in evidence-based care,
Preventing delirium in elderly hip surgery patients can seem like an elusive goal. Much has been written on it but little remains of value after careful analysis. Most sources revert to the now classic work by Marcantonio and colleagues (2001). In this study, a geriatric consult service was the experimental focus. Ten categories composed the “bundle” of care to be initiated. Among these were generic expectations such as ensuring fluid and electrolyte balance and adequate nutrition, but also geriatric concept-based interventions such as eliminating excessive medications and those medications especially problematic for elders such as benzodiazepines, and promoting early mobilization. An average of 9.5 interventions per patient were instituted. The incidence of delirium overall was high but was lower in the intervention group (32 vs. 50%, p=0.04). The intensity of delirium was assessed to be less in the intervention group as well, with severe delirium manifested only 12% of the time vs. 29% in the usual care group.
The most disappointing findings were that the length of days of delirium and the overall length of stay were not significantly different. Delirium at discharge was similar between groups as well. The study may have been underpowered with a total sample size of only 126 patients. Its strengths were that it was a randomized controlled trial with reasonably good adherence to protocol. On a practical level, we are left with a large “bundle” and we don’t know which interventions are the most important to emphasize in prioritizing care. Little different findings were found in examining the greater literature base on the topic.
A recent topic offering some hope for improvement in the area is the work on melatonin. Two studies have indicated that delirium is positively impacted by use of melatonin (Al-Aama, Brymer, Gutmanis, Woolmore-Goodwin, Esbaugh, & Dasgupta, 2010; Sultan, 2010). The former study in Canada demonstrated that melatonin was associated with a 12% delirium rate compared to a 31% rate when compared to placebo. DeJonge and colleagues (2011) expect to have additional data available on this subject in 2013. While two studies may not be sufficient to change practice, melatonin comes with few risks to patients, as it is a naturally occurring substance. The results of this third trial are a hopeful development in the face of little new to assist clinicians in delirium prevention and management.
Al-Aama, T., Brymer, C., Gutmanis, I., Woomore-Goodwin, S. M., Esbaugh, J., & Dasgupta, M. (2011). Melatonin decreasese delirium in elderly patients: A randomized, placebo-controlled trial. International Journal of Geriatric Psychiatry, 26, 687-694.
deJonghe, A, et al. (2011). The effects of melatonin versus placebo on delirium in hip fracture patients: study protocol of a randomized, placebo-controlled, double blind trial. BMC Geriatrics, 11, 34.
Marcantonio, E. R. , Flacker, J. M., Wright, R.J., & Resnick, N.M. (2001). Reducing delirium after hip fracture: A randomized trial. Journal of the American Geriatrics Society, 29, 516-522.
Sultan, S. S. (2010). Assessment of role of perioperative melatonin in prevention and treatment of postoperative delirium after hip arthroplasty under spinal anesthesia in the elderly. Saudi Journal of Anesthesia, 4(3), 169-173.
Dear Colleagues in evidence-based care,
Are meal replacement plans better than low calorie diets?
Meal replacement plans such as Slimfast and Medifast have been around a long time now. Is there any evidence on their effectiveness? The hesitant answer is yes. Why hesitant? Many of these studies are funded by these vendors. Certain populations are excluded that are very important in today’s society.
In a 2010 publication, the VP of Scientific and Clinical Affairs at Medifast was the lead author on a study of adults aged 18-65. Both diet plans were restricted to 1000 kilocalories a day, a seemingly good recipe for success to begin with. Early weight loss favored the meal replacement plan (12.3% vs. 6.9%). At week 40 there were no significant differences in the BMI reduction (7.8 vs. 5.9%) nor the biochemical markers studied.
A 2004 study was funded by Slimfast and also focused on adults aged 20-65. Here, again, both groups lost weight. The authors noted that the meal replacement plan subjects found it easier to understand and follow the food amounts.
A 2007 study funded by Slimfast found mean weight lost was not difference between groups. The low calorie group lost 8.4% of their original weight and the meal replacement group lost 6.2%. Both groups reduced calorie and fat consumption and increased protein intake over one year. Women 50 years and under were the subjects.
Ahrens was another Slimfast funded study from 2003. Again, both groups lost over 5% of their base body weight, the marker of significant body weight reduction set by the CDC. Further, there were no differences in biomarkers, including blood pressure, cholesterol, triglycerides or lipids. This study spanned 22 weeks.
A 2003 meta-analysis was provided by a member of the Slimfast nutritional institute. This meta-analysis combined six studies but two of these included diabetics that may have been quite different. Again the age was limited to those under 65. 88% of the population were women. Attrition was a problem in many studies and sometimes approached 50%. All six individual studies showed significant weight lost in both groups. Pooling the data had interesting results. More participants lost over 5% of their baseline body weight in the meal replacement group p<0.0001) at one year. The low calorie diet group lost between 2.61 and 4.35 Kg and the meal replacement plan lost 6.97 to 7.31 Kg and this difference was statistically significant. Was this influenced by including the diabetic patients? We do not know.
An interesting study came from Australia (Truby, 2008). Again the population was adults 65 years and under. Four diet plans were assessed including the Atkins, Weight Watchers, Slim Fast and a local product. Although they only followed patients for two months, they found all diet groups lost significantly more weight than the control group. Differences were found in the levels of minerals and vitamins in patients. Slimfast showed a decrease in niacin and an increase in zinc in this older study.
An obvious conclusion is that bias may well have influenced this body of literature. The elderly are conspicuously absent from these studies. All studies involved calorie reductions for both groups. Weight loss was evident in almost all cases. The conundrum is the meta-analysis. We leave that to your assessment. We have validated our literature search with the assistance of a medical librarian. It is possible that we did not locate all relevant studies . Reported here is a summary of some of the best found for adults.
Ahrens, R. A., Hower, M., & Best, A. M. (2003). Effects of Weight Reduction Interventions by Community Pharmacists. Journal of the American Pharmacists Association, 43(5), 583-589.
Ashley, J. M. et al. (2007). Nutrient adequacy during weight loss interventions: A randomized study in women comparing the dietary intake tin a meal replacement group with a traditional food group. Nutrition Journal, 6, 12. doi: 10.1186/1475-2891-6-12
Davis, Coleman, et al. (2010)Efficacy of a meal replacement diet plan compared to a food-based diet plan after a period of weight loss and weight maintenance: a randomized controlled trial. Nutrition Journal 9:11.
Heymsfield (2003). Weight management using a meal replacement (PMR) strategy: meta and pooling analysis from six studies. International Journal of Obesity.
Noakes, Foster, et al.(2004).Meal replacements are as effective as structured weight-loss diets for treating obesity adults with features of metabolic syndrome.
Journal of Nutrition. 134: 1894-1899. (AUSTRALIA)
Truby, Hiscutt, et al.(2008). Commercial weight loss diets meet nutrient requirements in free living adults over 8 weeks: a randomized controlled weight loss trial.
Nutrition Journal, 7:25. (AUSTRALIA)