Dear All,
I've enjoyed following the discussion.
A letter would seem to be in order.
Could I suggest that (some of) the members of this list write it, and reference this discussion? It would seem to be a good way of "extending" what will be a short letter, and of drawing more people's attention to the list.
BW,
Matt
Sent from my BlackBerry® wireless device
-----Original Message-----
From: EVIDENCE-BASED-HEALTH automatic digest system <[log in to unmask]>
Sender: "Evidence based health (EBH)" <[log in to unmask]>
Date: Thu, 16 May 2013 00:00:10
To: <[log in to unmask]>
Reply-To: "Evidence based health (EBH)" <[log in to unmask]>
Subject: EVIDENCE-BASED-HEALTH Digest - 14 May 2013 to 15 May 2013 (#2013-124)
There are 5 messages totaling 3174 lines in this issue.
Topics of the day:
1. why did CMAJ publish this study? (5)
----------------------------------------------------------------------
Date: Tue, 14 May 2013 23:33:36 +0000
From: "McCormack, James" <[log in to unmask]>
Subject: Re: why did CMAJ publish this study?
Hello Everyone:
I always find it interesting the discussion that goes on around studies that look at these sorts of interventions. We have 1000s of studies that have looked at changing surrogate markers - this is just another one. Good for them getting it published in the CMAJ.
I applaud all researchers who try to answer medical questions and these authors did a good job of looking at what happens to risk factors when subjects were randomized to the type of naturopathic care described. It is an interesting question - hardly earth shattering but nonetheless, in my mind reasonable research.
The problem I see with this study really has nothing to do with the design but rather some of the wording in the paper
1) The title of the article is a best misleading
"Naturopathic medicine for the prevention of cardiovascular disease: a randomized clinical trial"
It wasn't that at all - the title should have been "Visits for naturopathic care for the reduction of cardiovascular risk factors - a randomized clinical trial"
2) The first line of the interpretation was "We found a significant reduction in the risk of cardiovascular disease following counselling about nutritional and physical activity provided by naturopathic doctors." They found nothing of the sort - they found a reduction in the risk of cardiovascular risk factors NOT cardiovascular disease.
In my opinion, the main critique of this study should be the following - and by critique I mean more contextualizing
1) We have no idea if this intervention would lead to reduced cardiovascular disease. 15 large RCTs in a row (I haven't seen any showing a positive outcome but please let me know if I have missed any) over the last 3-5 years have shown little if any benefit and even some harm from changing surrogate markers
LIPIDS
AIM-HIGH, HPS2-THRIVE (niacin)
ACCORD (fibrates)
dalOUTCOMES (dalcetrapib)
BLOOD PRESSURE
ALTITUDE (aliskiren)
VALISH, AASK, ACCORD (aggressive BP lowering)
DIABETES
ACCORD, ADVANCE, VADT (aggressive A1c lowering)
ROADMAP (olmesartan)
ORIGIN (insulin)
GENERAL
ACTIVE (irbesartan/afib)
CRESCENDO (rimonabant)
2) The only dietary intervention that has really been evaluated for cardiovascular outcomes is the Mediterranean diet and and the most recent trial showed that over 5 years it reduced the number of cardiovascular events by around 1% compared to a low fat diet - at least the dietary recommendations in this study were for this sort of nutrition intervention. The increased activity seems a reasonable suggestion as well.
3) Fish oil supplements which were recommended in this study have been a huge bust as far as I recall.
4) I have not seen any studies that have shown that using the concept of metabolic syndrome improves the ability to risk stratify over and above the individual risk factors and we really don't have a treatment for the "metabolic syndrome" other than what we should advise for most people - eat in moderation, be more active, and maybe for some metformin/BP control and maybe a low dose statin.
Those are some of my thoughts.
James McCormack, BSc(Pharm), Pharm D
Professor
Faculty of Pharmaceutical Sciences
UBC, Vancouver, Canada
therapeuticseducation.org<http://therapeuticseducation.org>
604-603-7898
On 2013-05-14, at 9:22 AM, Paul Elias <[log in to unmask]<mailto:[log in to unmask]>> wrote:
I think if there is any issue with the publication, then a response to author should be written and allow the author to reply. Gordon Guyatt is not the principle author (but realize his name is offered given his seniority in the EBM field) but I am sure that if someone is moved to reply to author, and he is asked a question (s) for clarification, he will address concerns and questions.
Best,
Paul E. Alexander
--- On Tue, 5/14/13, Susan Fowler <[log in to unmask]<mailto:[log in to unmask]>> wrote:
From: Susan Fowler <[log in to unmask]<mailto:[log in to unmask]>>
Subject: Re: why did CMAJ publish this study?
To: [log in to unmask]<mailto:[log in to unmask]>
Received: Tuesday, May 14, 2013, 11:30 AM
Have you written Gordon Guyatt and asked him?
--
Susan Fowler, MLIS
Medical Librarian
Evidence at Becker:
http://beckerguides.wustl.edu/ebm
Systematic Reviews Guide:
http://beckerguides.wustl.edu/SystematicReviews
Becker Medical Library, Washington University in St. Louis
314-362-8092
[log in to unmask]<x-msg:[log in to unmask]>
On Mon, May 13, 2013 at 5:28 PM, write words <[log in to unmask]<x-msg:[log in to unmask]>> wrote:
The CMAJ just published a study on naturopathy. The press release I received about the paper said, “Treatment by naturopathic doctors shows reduction in cardiovascular risk factors. Randomized controlled trial.”
The trial showed nothing of the sort. I blogged about this last week:
http://www.lastwordonnothing.com/2013/05/09/crappystudy/
Here's a link to the study itself (Gordon Gyatt is listed as senior author): http://www.cmaj.ca/content/early/2013/04/29/cmaj.120567
The CMAJ knew that the study was useless. From an editorial that accompanied the study:
"We can learn nothing new from this trial about supplements or any other individual component of care, because the trial was not designed to allow their evaluation." http://www.cmaj.ca/content/early/2013/04/29/cmaj.130614.full.pdf
Is there something I'm missing here? Does a study that's not designed to measure anything have some use I'm overlooking?
Best,
Christie
Christie Aschwanden
Email: [log in to unmask]<x-msg:[log in to unmask]>
Website: www.christieaschwanden.com<http://www.christieaschwanden.com/>
Blog: www.lastwordonnothing.com/category/christie/<http://www.lastwordonnothing.com/category/christie/>
Twitter: @cragcrest
------------------------------
Date: Wed, 15 May 2013 05:57:19 +0300
From: "Dr.Rajaa Al-Raddadi" <[log in to unmask]>
Subject: Re: why did CMAJ publish this study?
I agree with Paul that response to author should be written and allow the author to reply.
Rajaa Raddadi
MBBS,ABCM,RICR
Consultant Community Medicine &Research Methodology
Best Wishes
Sent from my iPhone
On 15 May 2013, at 02:33 AM, "McCormack, James" <[log in to unmask]> wrote:
> Hello Everyone:
>
> I always find it interesting the discussion that goes on around studies that look at these sorts of interventions. We have 1000s of studies that have looked at changing surrogate markers - this is just another one. Good for them getting it published in the CMAJ.
>
> I applaud all researchers who try to answer medical questions and these authors did a good job of looking at what happens to risk factors when subjects were randomized to the type of naturopathic care described. It is an interesting question - hardly earth shattering but nonetheless, in my mind reasonable research.
>
> The problem I see with this study really has nothing to do with the design but rather some of the wording in the paper
>
> 1) The title of the article is a best misleading
> "Naturopathic medicine for the prevention of cardiovascular disease: a randomized clinical trial"
>
> It wasn't that at all - the title should have been "Visits for naturopathic care for the reduction of cardiovascular risk factors - a randomized clinical trial"
>
> 2) The first line of the interpretation was "We found a significant reduction in the risk of cardiovascular disease following counselling about nutritional and physical activity provided by naturopathic doctors." They found nothing of the sort - they found a reduction in the risk of cardiovascular risk factors NOT cardiovascular disease.
>
> In my opinion, the main critique of this study should be the following - and by critique I mean more contextualizing
>
> 1) We have no idea if this intervention would lead to reduced cardiovascular disease. 15 large RCTs in a row (I haven't seen any showing a positive outcome but please let me know if I have missed any) over the last 3-5 years have shown little if any benefit and even some harm from changing surrogate markers
>
> LIPIDS
> AIM-HIGH, HPS2-THRIVE (niacin)
> ACCORD (fibrates)
> dalOUTCOMES (dalcetrapib)
>
> BLOOD PRESSURE
> ALTITUDE (aliskiren)
> VALISH, AASK, ACCORD (aggressive BP lowering)
>
> DIABETES
> ACCORD, ADVANCE, VADT (aggressive A1c lowering)
> ROADMAP (olmesartan)
> ORIGIN (insulin)
>
> GENERAL
> ACTIVE (irbesartan/afib)
> CRESCENDO (rimonabant)
>
> 2) The only dietary intervention that has really been evaluated for cardiovascular outcomes is the Mediterranean diet and and the most recent trial showed that over 5 years it reduced the number of cardiovascular events by around 1% compared to a low fat diet - at least the dietary recommendations in this study were for this sort of nutrition intervention. The increased activity seems a reasonable suggestion as well.
>
> 3) Fish oil supplements which were recommended in this study have been a huge bust as far as I recall.
>
> 4) I have not seen any studies that have shown that using the concept of metabolic syndrome improves the ability to risk stratify over and above the individual risk factors and we really don't have a treatment for the "metabolic syndrome" other than what we should advise for most people - eat in moderation, be more active, and maybe for some metformin/BP control and maybe a low dose statin.
>
> Those are some of my thoughts.
>
> James McCormack, BSc(Pharm), Pharm D
> Professor
> Faculty of Pharmaceutical Sciences
> UBC, Vancouver, Canada
>
> therapeuticseducation.org
>
> 604-603-7898
>
>
>
>
> On 2013-05-14, at 9:22 AM, Paul Elias <[log in to unmask]> wrote:
>
>>
>> I think if there is any issue with the publication, then a response to author should be written and allow the author to reply. Gordon Guyatt is not the principle author (but realize his name is offered given his seniority in the EBM field) but I am sure that if someone is moved to reply to author, and he is asked a question (s) for clarification, he will address concerns and questions.
>>
>>
>>
>>
>>
>>
>>
>>
>> Best,
>> Paul E. Alexander
>>
>>
>>
>>
>>
>> --- On Tue, 5/14/13, Susan Fowler <[log in to unmask]> wrote:
>>
>> From: Susan Fowler <[log in to unmask]>
>> Subject: Re: why did CMAJ publish this study?
>> To: [log in to unmask]
>> Received: Tuesday, May 14, 2013, 11:30 AM
>>
>> Have you written Gordon Guyatt and asked him?
>>
>> --
>> Susan Fowler, MLIS
>> Medical Librarian
>>
>> Evidence at Becker:
>> http://beckerguides.wustl.edu/ebm
>>
>> Systematic Reviews Guide:
>> http://beckerguides.wustl.edu/SystematicReviews
>>
>> Becker Medical Library, Washington University in St. Louis
>> 314-362-8092
>> [log in to unmask]
>>
>> On Mon, May 13, 2013 at 5:28 PM, write words <[log in to unmask]> wrote:
>> The CMAJ just published a study on naturopathy. The press release I received about the paper said, “Treatment by naturopathic doctors shows reduction in cardiovascular risk factors. Randomized controlled trial.”
>>
>> The trial showed nothing of the sort. I blogged about this last week:
>>
>> http://www.lastwordonnothing.com/2013/05/09/crappystudy/
>>
>> Here's a link to the study itself (Gordon Gyatt is listed as senior author): http://www.cmaj.ca/content/early/2013/04/29/cmaj.120567
>>
>> The CMAJ knew that the study was useless. From an editorial that accompanied the study:
>>
>> "We can learn nothing new from this trial about supplements or any other individual component of care, because the trial was not designed to allow their evaluation." http://www.cmaj.ca/content/early/2013/04/29/cmaj.130614.full.pdf
>>
>> Is there something I'm missing here? Does a study that's not designed to measure anything have some use I'm overlooking?
>>
>> Best,
>> Christie
>>
>> Christie Aschwanden
>> Email: [log in to unmask]
>> Website: www.christieaschwanden.com
>> Blog: www.lastwordonnothing.com/category/christie/
>> Twitter: @cragcrest
>
------------------------------
Date: Wed, 15 May 2013 07:15:12 +0100
From: Michael Power <[log in to unmask]>
Subject: Re: why did CMAJ publish this study?
Good points James
Just two refinements of your suggestions
1) The title should have been along these lines "Social grooming in the
form of naturopathic medicine for the reduction of cardiovascular risk
factors: an unblinded randomized controlled trial"
2) As Richard Lehmann says, fish oil is best taken, not in capsules,
but as turbot gently fried in butter and seasoned with lemon, salt, and
pepper.
Michael
From: Evidence based health (EBH)
[mailto:[log in to unmask]] On Behalf Of McCormack, James
Sent: 15 May 2013 00:34
To: [log in to unmask]
Subject: Re: why did CMAJ publish this study?
Hello Everyone:
I always find it interesting the discussion that goes on around studies that
look at these sorts of interventions. We have 1000s of studies that have
looked at changing surrogate markers - this is just another one. Good for
them getting it published in the CMAJ.
I applaud all researchers who try to answer medical questions and these
authors did a good job of looking at what happens to risk factors when
subjects were randomized to the type of naturopathic care described. It is
an interesting question - hardly earth shattering but nonetheless, in my
mind reasonable research.
The problem I see with this study really has nothing to do with the design
but rather some of the wording in the paper
1) The title of the article is a best misleading
"Naturopathic medicine for the prevention of cardiovascular disease: a
randomized clinical trial"
It wasn't that at all - the title should have been "Visits for naturopathic
care for the reduction of cardiovascular risk factors - a randomized
clinical trial"
2) The first line of the interpretation was "We found a significant
reduction in the risk of cardiovascular disease following counselling about
nutritional and physical activity provided by naturopathic doctors." They
found nothing of the sort - they found a reduction in the risk of
cardiovascular risk factors NOT cardiovascular disease.
In my opinion, the main critique of this study should be the following - and
by critique I mean more contextualizing
1) We have no idea if this intervention would lead to reduced cardiovascular
disease. 15 large RCTs in a row (I haven't seen any showing a positive
outcome but please let me know if I have missed any) over the last 3-5 years
have shown little if any benefit and even some harm from changing surrogate
markers
LIPIDS
AIM-HIGH, HPS2-THRIVE (niacin)
ACCORD (fibrates)
dalOUTCOMES (dalcetrapib)
BLOOD PRESSURE
ALTITUDE (aliskiren)
VALISH, AASK, ACCORD (aggressive BP lowering)
DIABETES
ACCORD, ADVANCE, VADT (aggressive A1c lowering)
ROADMAP (olmesartan)
ORIGIN (insulin)
GENERAL
ACTIVE (irbesartan/afib)
CRESCENDO (rimonabant)
2) The only dietary intervention that has really been evaluated for
cardiovascular outcomes is the Mediterranean diet and and the most recent
trial showed that over 5 years it reduced the number of cardiovascular
events by around 1% compared to a low fat diet - at least the dietary
recommendations in this study were for this sort of nutrition intervention.
The increased activity seems a reasonable suggestion as well.
3) Fish oil supplements which were recommended in this study have been a
huge bust as far as I recall.
4) I have not seen any studies that have shown that using the concept of
metabolic syndrome improves the ability to risk stratify over and above the
individual risk factors and we really don't have a treatment for the
"metabolic syndrome" other than what we should advise for most people - eat
in moderation, be more active, and maybe for some metformin/BP control and
maybe a low dose statin.
Those are some of my thoughts.
James McCormack, BSc(Pharm), Pharm D
Professor
Faculty of Pharmaceutical Sciences
UBC, Vancouver, Canada
therapeuticseducation.org
604-603-7898
On 2013-05-14, at 9:22 AM, Paul Elias <[log in to unmask]> wrote:
I think if there is any issue with the publication, then a response to
author should be written and allow the author to reply. Gordon Guyatt is not
the principle author (but realize his name is offered given his seniority in
the EBM field) but I am sure that if someone is moved to reply to author,
and he is asked a question (s) for clarification, he will address concerns
and questions.
Best,
Paul E. Alexander
--- On Tue, 5/14/13, Susan Fowler <[log in to unmask]> wrote:
From: Susan Fowler <[log in to unmask]>
Subject: Re: why did CMAJ publish this study?
To: [log in to unmask]
Received: Tuesday, May 14, 2013, 11:30 AM
Have you written Gordon Guyatt and asked him?
--
Susan Fowler, MLIS
Medical Librarian
Evidence at Becker:
http://beckerguides.wustl.edu/ebm
Systematic Reviews Guide:
<http://beckerguides.wustl.edu/SystematicReviews>
http://beckerguides.wustl.edu/SystematicReviews
Becker Medical Library, Washington University in St. Louis
314-362-8092
[log in to unmask]
<x-msg:[log in to unmask]>
On Mon, May 13, 2013 at 5:28 PM, write words <[log in to unmask]
<x-msg:[log in to unmask]> > wrote:
The CMAJ just published a study on naturopathy. The press release I received
about the paper said, "Treatment by naturopathic doctors shows reduction in
cardiovascular risk factors. Randomized controlled trial."
The trial showed nothing of the sort. I blogged about this last week:
http://www.lastwordonnothing.com/2013/05/09/crappystudy/
Here's a link to the study itself (Gordon Gyatt is listed as senior author):
http://www.cmaj.ca/content/early/2013/04/29/cmaj.120567
The CMAJ knew that the study was useless. From an editorial that accompanied
the study:
"We can learn nothing new from this trial about supplements or any other
individual component of care, because the trial was not designed to allow
their evaluation."
http://www.cmaj.ca/content/early/2013/04/29/cmaj.130614.full.pdf
Is there something I'm missing here? Does a study that's not designed to
measure anything have some use I'm overlooking?
Best,
Christie
Christie Aschwanden
Email: [log in to unmask] <x-msg:[log in to unmask]>
Website: www.christieaschwanden.com <http://www.christieaschwanden.com/>
Blog: www.lastwordonnothing.com/category/christie/
Twitter: @cragcrest
------------------------------
Date: Wed, 15 May 2013 06:21:11 +0000
From: "McCormack, James" <[log in to unmask]>
Subject: Re: why did CMAJ publish this study?
Agree - you had me at the word "butter" :)
On 2013-05-14, at 11:15 PM, Michael Power <[log in to unmask]<mailto:[log in to unmask]>> wrote:
Good points James
Just two refinements of your suggestions
1) The title should have been along these lines “Social grooming in the form of naturopathic medicine for the reduction of cardiovascular risk factors: an unblinded randomized controlled trial”
2) As Richard Lehmann says, fish oil is best taken, not in capsules, but as turbot gently fried in butter and seasoned with lemon, salt, and pepper.
Michael
From: Evidence based health (EBH) [mailto:[log in to unmask]<mailto:[log in to unmask]>] On Behalf OfMcCormack, James
Sent: 15 May 2013 00:34
To: [log in to unmask]<mailto:[log in to unmask]>
Subject: Re: why did CMAJ publish this study?
Hello Everyone:
I always find it interesting the discussion that goes on around studies that look at these sorts of interventions. We have 1000s of studies that have looked at changing surrogate markers - this is just another one. Good for them getting it published in the CMAJ.
I applaud all researchers who try to answer medical questions and these authors did a good job of looking at what happens to risk factors when subjects were randomized to the type of naturopathic care described. It is an interesting question - hardly earth shattering but nonetheless, in my mind reasonable research.
The problem I see with this study really has nothing to do with the design but rather some of the wording in the paper
1) The title of the article is a best misleading
"Naturopathic medicine for the prevention of cardiovascular disease: a randomized clinical trial"
It wasn't that at all - the title should have been "Visits for naturopathic care for the reduction of cardiovascular risk factors - a randomized clinical trial"
2) The first line of the interpretation was "We found a significant reduction in the risk of cardiovascular disease following counselling about nutritional and physical activity provided by naturopathic doctors." They found nothing of the sort - they found a reduction in the risk of cardiovascular risk factors NOT cardiovascular disease.
In my opinion, the main critique of this study should be the following - and by critique I mean more contextualizing
1) We have no idea if this intervention would lead to reduced cardiovascular disease. 15 large RCTs in a row (I haven't seen any showing a positive outcome but please let me know if I have missed any) over the last 3-5 years have shown little if any benefit and even some harm from changing surrogate markers
LIPIDS
AIM-HIGH, HPS2-THRIVE (niacin)
ACCORD (fibrates)
dalOUTCOMES (dalcetrapib)
BLOOD PRESSURE
ALTITUDE (aliskiren)
VALISH, AASK, ACCORD (aggressive BP lowering)
DIABETES
ACCORD, ADVANCE, VADT (aggressive A1c lowering)
ROADMAP (olmesartan)
ORIGIN (insulin)
GENERAL
ACTIVE (irbesartan/afib)
CRESCENDO (rimonabant)
2) The only dietary intervention that has really been evaluated for cardiovascular outcomes is the Mediterranean diet and and the most recent trial showed that over 5 years it reduced the number of cardiovascular events by around 1% compared to a low fat diet - at least the dietary recommendations in this study were for this sort of nutrition intervention. The increased activity seems a reasonable suggestion as well.
3) Fish oil supplements which were recommended in this study have been a huge bust as far as I recall.
4) I have not seen any studies that have shown that using the concept of metabolic syndrome improves the ability to risk stratify over and above the individual risk factors and we really don't have a treatment for the "metabolic syndrome" other than what we should advise for most people - eat in moderation, be more active, and maybe for some metformin/BP control and maybe a low dose statin.
Those are some of my thoughts.
James McCormack, BSc(Pharm), Pharm D
Professor
Faculty of Pharmaceutical Sciences
UBC, Vancouver, Canada
therapeuticseducation.org<http://therapeuticseducation.org>
604-603-7898
On 2013-05-14, at 9:22 AM, Paul Elias <[log in to unmask]<mailto:[log in to unmask]>> wrote:
I think if there is any issue with the publication, then a response to author should be written and allow the author to reply. Gordon Guyatt is not the principle author (but realize his name is offered given his seniority in the EBM field) but I am sure that if someone is moved to reply to author, and he is asked a question (s) for clarification, he will address concerns and questions.
Best,
Paul E. Alexander
--- On Tue, 5/14/13, Susan Fowler <[log in to unmask]<mailto:[log in to unmask]>> wrote:
From: Susan Fowler <[log in to unmask]<mailto:[log in to unmask]>>
Subject: Re: why did CMAJ publish this study?
To: [log in to unmask]<mailto:[log in to unmask]>
Received: Tuesday, May 14, 2013, 11:30 AM
Have you written Gordon Guyatt and asked him?
--
Susan Fowler, MLIS
Medical Librarian
Evidence at Becker:
http://beckerguides.wustl.edu/ebm
Systematic Reviews Guide:
http://beckerguides.wustl.edu/SystematicReviews
Becker Medical Library, Washington University in St. Louis
314-362-8092
[log in to unmask]<x-msg:[log in to unmask]>
On Mon, May 13, 2013 at 5:28 PM, write words <[log in to unmask]<x-msg:[log in to unmask]>> wrote:
The CMAJ just published a study on naturopathy. The press release I received about the paper said, “Treatment by naturopathic doctors shows reduction in cardiovascular risk factors. Randomized controlled trial.”
The trial showed nothing of the sort. I blogged about this last week:
http://www.lastwordonnothing.com/2013/05/09/crappystudy/
Here's a link to the study itself (Gordon Gyatt is listed as senior author): http://www.cmaj.ca/content/early/2013/04/29/cmaj.120567
The CMAJ knew that the study was useless. From an editorial that accompanied the study:
"We can learn nothing new from this trial about supplements or any other individual component of care, because the trial was not designed to allow their evaluation." http://www.cmaj.ca/content/early/2013/04/29/cmaj.130614.full.pdf
Is there something I'm missing here? Does a study that's not designed to measure anything have some use I'm overlooking?
Best,
Christie
Christie Aschwanden
Email: [log in to unmask]<x-msg:[log in to unmask]>
Website: www.christieaschwanden.com<http://www.christieaschwanden.com/>
Blog: www.lastwordonnothing.com/category/christie/<http://www.lastwordonnothing.com/category/christie/>
Twitter: @cragcrest
------------------------------
Date: Wed, 15 May 2013 07:31:45 +0100
From: Michael Power <[log in to unmask]>
Subject: Re: why did CMAJ publish this study?
Marilyn, you are right, regression to the mean would affect both groups equally – provided no bias was introduced by selection or analysis – however, regression to the mean adds a little more (unnecessary) noise) to the signal – this is only a small red flag
More questions raise themselves each time I look at the paper
“At each site, we invited 120–140 participants with the highest risk of cardiovascular disease (based on the ratio of total cholesterol to HDL; mean: 5.18; range: 1.8–14.8).”
Table 1 shows “Baseline characteristics of participants at high risk of cardiovascular disease”
But: a ratio of total cholesterol to HDL cholesterol of 1.8 (see the range above) would be quite low.
Also there were 124 participants in the naturopathic group and 122 in the control, but only 105 and 112 of these contributed to the average total-C to HDL-C ratio and fewer still contributed to the averages of the individual components of the lipid profile. Doesn’t make sense and is incompatible with the description of the selection procedure.
Michael
From: Marilyn Mann [mailto:[log in to unmask]]
Sent: 14 May 2013 13:24
To: Michael Power; [log in to unmask]
Subject: Re: why did CMAJ publish this study?
Dear Michael and Christie
I share your skepticism with respect to this study.
Michael, can you explain why regression to the mean would produce a bias in favor of the naturopathy group? Why wouldn't it affect both groups equally?
I am particularly skeptical about any intervention involving the use of plant sterol supplements or functional foods containing plant sterols because, while plant sterols lower LDL, they have never been shown to prevent cardiovascular events (i.e., there have been no RCTs testing whether plant sterols lower the risk of clinical endpoints). Moreover, there is uncertainty as to whether plant sterols are safe given that the use of plant sterol supplements increases serum plant sterols. We know that very high plant sterol levels, as in sitosterolemia, cause xanthomas and premature cardiovascular disease. Whether more moderate elevations in plant sterols cause harm is not known.
Regards
Marilyn Mann
Twitter: @MarilynMann
Blog: http://marilynmann.wordpress.com
Sent from my Verizon Wireless Phone
-----Original message-----
From: Michael Power <[log in to unmask]>
To: [log in to unmask]
Sent: Tue, May 14, 2013 08:50:35 GMT+00:00
Subject: Re: why did CMAJ publish this study?
Christie
I shall resist the temptation to speculate why the CMAJ published the trial, or with what expertise the peer reviewers reviewed the paper. But, I do wonder if they would have published the trial if the results had been "negative".
I have not seen the press release, but it would have been irresponsible of it not to have commented on the size of the effect, the precision of the results, the risks of bias and error, the applicability of the results and implications for practice, and how the results fit with similar studies.
If I had been asked, I would have recommended against funding the study, because the prior probability of it producing a useful result was very low if you are a skeptic and very high if you are a believer, and thus the study would have been unlikely to change anything. I wonder if this is the reason no sample size calculation was reported?
There are a number of significant risks to bias, including:
Randomization was conducted by the Canadian College of Naturopathic Medicine
Participants were selected on the basis of higher ratios of total cholesterol to HDL cholesterol - because cholesterol measurements are quite variable, and selection seems to have been done on only one measurement, this would have introduced a risk of bias from "regression to the mean".
"The naturopathic doctors collected all biometric and validated questionnaire measures", and were not blinded to study group.
Tables 1 (baseline) and 2 (results) are not comparable. First, table 1 shows data plus/minus standard deviations, while table 2 shows data plus/minus standard errors of the means. This is more a problem of spin than substance, as using the SE makes the data in table 2 look more precise than the data with SD in table 1. Secondly, and most problematically, table 1 shows real data, i.e. data with real numerators and denominators, while table 2 shows imagined, or at least engineered data - the real numbers have been adjusted for baseline measure of outcome variables, and missing data have been created by "a multiple imputation". This data engineering is particularly problematic given the opacity of the imputation process and the 30% drop out rate. With a 30% drop out rate the missing data rate would be > 30%. I wonder why a table 3 with real outcome data was not published? If it had been, we would be able to see whethere or not the data engineering had created statistically significant results.
This seems to be the first trial of its kind - and we know that first publications often turn out to have results that are more extreme than subsequent trials.
These issues are a forest of flapping red flags for bias.
The response to these issues, especially the data engineering, should have been to adjust the confidence intervals (i.e. widen them). ButI can only speculate that the reason this was not done is that all significant differences would have evaporated faster than n-butyl glycol (i.e.163 times faster than ether http://www.siegwerk.com/fileadmin/user_upload/cc/Data_Sheets/TM/Verdunstungsgeschwindigkeit_e.pdf).
Even if the results are real, the implications for practice would be to do nothing different - the intervention is so vaguely specified that it can not be repeated, and the outcomes are patient-unimportant outcomes, and thus should not be the basis for offering treatment.
The funders were poor stewards of the money that was entrusted to them to manage.
Michael
------------------------------
End of EVIDENCE-BASED-HEALTH Digest - 14 May 2013 to 15 May 2013 (#2013-124)
****************************************************************************
|
