Sponsorship [of pharmaceutical research and drug trials] by manufacturers has been found to be associated with a reduced likelihood of the reporting of adverse results. Likewise, a significant link has been found between industry funding and the likelihood that results of a randomized trial will support a new therapy. … One proposed solution to this problem is to increase public funding for the conduct of research on therapeutic effectiveness. Ironically, that may well aggravate the problem. In July 2007,
Shouldn't only asking the questions you expect to give a positive answer be considered a kind of bias? Not one that merits some sort of punishment, but a bias nonetheless.
RE the meta-finding that sponsored drug studies more frequently have positive answers regarding the treatments of the sponsors - this not necessarily indicative of twisting the truth and is better explained as a form of editorial control over research questions. That is, if you spend money developing a drug, you're going to try to develop uses of it that you have a pretty good feeling in advance WILL WORK, and you'll ask limited questions appropriate to that. You can have a better than 50% sense of whether something will work before you investigate it with rigor to get a MUCH better than 50% answer that it works. So I would argue the effect we're seeing is more the result of conservative investigations than bias, and this is not inappropriate. But it's worth thinking of ways to distinguish the two possibilities.
This guy believes that it was industry lobbyists who wanted the chilling effect:
The Finance Committee bill also includes language requested by industry lobbyists (pages 1138–1139) that threatens to withdraw federal funding for 5 years from any investigator who publishes a report on research funded by the proposed institute that is not “within the bounds of and entirely consistent with the evidence.” Determinations regarding such consistency would be made by the newly created research entity, which would have industry involvement both in its governance and in study design. To allow scientists — and their institutions, which receive the support for the conduct of research — to be punished for the publication of work that is not approved by this entity is essentially to cede authority over the dissemination of government-funded research to a body that is at least partially controlled by persons with a potential commercial interest in its outcome. This move would be a major retrograde step that would both inhibit the conduct of CER and call its integrity into question. In addition, because researchers and their institutions will seek to avoid such punishment, this provision is likely to result in prolonged arguments, taking place out of public view, regarding which data are acceptable to publish, thereby impeding and delaying publication. The American public, which would be paying for this research, deserves better.
The medical industry doesn't want reports denigrating their products. 3 industry seats out of 15 isn't a big deal though.
I'm not a scientist so your answer sounds very reasonable.
The breadth of the 5 year minimum moratorium on government contracts applies to the 'study-conducting entity' but whether this would encompass an entire university department (say) I don't know.
In spite of the concern over this portion of the bill, the Institute still appears to be very politically isolated. Fears of the PCORI galavanting around 'muzzling' scientists based on a political agenda seem unlikely.
Considering your opinion on peer review the need for such a clause is not immediately apparent, but the organisation of the PCORI does not seem to lend itself to political manipulation, and in this sense it may still be better than the pressures of corporate funding.
Why throw good money after bad if the research is trash?
The established mechanism for doing this is peer review. When it works well, the panel reviewing the grant renewal will identify the prior results as trash, and score the application poorly. I've seen all sorts of criticisms of peer review, but inadequate identification of trash is seldom one.
Besides, defunding the entire institutions because Dr. Incompetent in Lab 723 published crap seems a little extreme.
The quoted phrasing makes it sound like a company pushing a product is more likely than others (perhaps other companies, perhaps government, perhaps non-profits) to publish research that supports said product (it is logically possible that it is the others who are more biased against it, though not likely). It also suggests that researchers complain more about restrictions placed on them by government compared to industry. Those are two distinct issues which are not incompatible.
The 2007 quote is no doubt reflective of a broader culture in the Bush administration of an over-zealous control of information, and often down-right secretiveness.
However, the threat of politicization seems to be pretty slim to me, based off this one clause.
If you read the rest of the details of the Patient Centered Outcomes Institute it's a not-for-profit institute, that appears quite isolated from the whims of the executive branch.
The Board of the Institute is made up of the NIH Director, the director of the AHRQ and 17 others, selected with 6 year terms by the Comptroller General (which, if anyone doesn't know, is the head of the Government Accountability [formerly Audit] Office, an office of the legislative branch).
The institute, as far as i can see, has to demand some accountability from the researcher? Why throw good money after bad if the research is trash?
Despite that, if there is any 'muzzling' going on it must, needless to say, be at the behest of the legislative as well as executive branches, and the politically isolated Board of the institute.
I'm a little confused. We start off by saying that corporate-sponsored research is 'more likely' to be biased than non-so-sponsored research of the same thing, and then we finish by pointing out that government-sponsored research is more likely to be biased? What's being compared to what here? I thought if a study wasn't corporate-sponsored it was usually government-sponsored.
So researchers should be able to defraud the government at will? Yes, there are dangers from the politicization of science, but there are also dangers from unaccountability. The former can't be prevented through law but only through administration.
“Any research published under clause(ii)(IV) shall be within the bounds of and entirely consistentwith the evidence and findings produced underthe contract with the Institute under this subparagraph."
means?
Seriously. Isn't "the evidence and findings produced" identical with "research"? How can this clause be violated? I don't get it.
It's in the final version of the bill. Section 6301 of the Patient Protection Act (Public Law No: 111-148) adds the following section to the Social Security Act:
"Any research published under clause(ii)(IV) shall be within the bounds of and entirely consistentwith the evidence and findings produced underthe contract with the Institute under this subparagraph.If the Institute determines that those requirementsare not met, the Institute shall not enter intoanother contract with the agency, instrumentality, orentity which managed or conducted such research fora period determined appropriate by the Institute (butnot less than 5 years)."
This act was subsequently modified by reconciliation, but looking at the table of contents reveals that Section 6301 was not among the modified sections.
Shouldn't only asking the questions you expect to give a positive answer be considered a kind of bias? Not one that merits some sort of punishment, but a bias nonetheless.
RE the meta-finding that sponsored drug studies more frequently have positive answers regarding the treatments of the sponsors - this not necessarily indicative of twisting the truth and is better explained as a form of editorial control over research questions. That is, if you spend money developing a drug, you're going to try to develop uses of it that you have a pretty good feeling in advance WILL WORK, and you'll ask limited questions appropriate to that. You can have a better than 50% sense of whether something will work before you investigate it with rigor to get a MUCH better than 50% answer that it works. So I would argue the effect we're seeing is more the result of conservative investigations than bias, and this is not inappropriate. But it's worth thinking of ways to distinguish the two possibilities.
This guy believes that it was industry lobbyists who wanted the chilling effect:
The Finance Committee bill also includes language requested by industry lobbyists (pages 1138–1139) that threatens to withdraw federal funding for 5 years from any investigator who publishes a report on research funded by the proposed institute that is not “within the bounds of and entirely consistent with the evidence.” Determinations regarding such consistency would be made by the newly created research entity, which would have industry involvement both in its governance and in study design. To allow scientists — and their institutions, which receive the support for the conduct of research — to be punished for the publication of work that is not approved by this entity is essentially to cede authority over the dissemination of government-funded research to a body that is at least partially controlled by persons with a potential commercial interest in its outcome. This move would be a major retrograde step that would both inhibit the conduct of CER and call its integrity into question. In addition, because researchers and their institutions will seek to avoid such punishment, this provision is likely to result in prolonged arguments, taking place out of public view, regarding which data are acceptable to publish, thereby impeding and delaying publication. The American public, which would be paying for this research, deserves better.
The medical industry doesn't want reports denigrating their products. 3 industry seats out of 15 isn't a big deal though.
Chris, there a great many possible biases that this could entrench besides "the whims of the executive branch."
I'm not a scientist so your answer sounds very reasonable.
The breadth of the 5 year minimum moratorium on government contracts applies to the 'study-conducting entity' but whether this would encompass an entire university department (say) I don't know.
In spite of the concern over this portion of the bill, the Institute still appears to be very politically isolated. Fears of the PCORI galavanting around 'muzzling' scientists based on a political agenda seem unlikely.
Considering your opinion on peer review the need for such a clause is not immediately apparent, but the organisation of the PCORI does not seem to lend itself to political manipulation, and in this sense it may still be better than the pressures of corporate funding.
The
Why throw good money after bad if the research is trash?
The established mechanism for doing this is peer review. When it works well, the panel reviewing the grant renewal will identify the prior results as trash, and score the application poorly. I've seen all sorts of criticisms of peer review, but inadequate identification of trash is seldom one.
Besides, defunding the entire institutions because Dr. Incompetent in Lab 723 published crap seems a little extreme.
The quoted phrasing makes it sound like a company pushing a product is more likely than others (perhaps other companies, perhaps government, perhaps non-profits) to publish research that supports said product (it is logically possible that it is the others who are more biased against it, though not likely). It also suggests that researchers complain more about restrictions placed on them by government compared to industry. Those are two distinct issues which are not incompatible.
The 2007 quote is no doubt reflective of a broader culture in the Bush administration of an over-zealous control of information, and often down-right secretiveness.
However, the threat of politicization seems to be pretty slim to me, based off this one clause.
If you read the rest of the details of the Patient Centered Outcomes Institute it's a not-for-profit institute, that appears quite isolated from the whims of the executive branch.
The Board of the Institute is made up of the NIH Director, the director of the AHRQ and 17 others, selected with 6 year terms by the Comptroller General (which, if anyone doesn't know, is the head of the Government Accountability [formerly Audit] Office, an office of the legislative branch).
The institute, as far as i can see, has to demand some accountability from the researcher? Why throw good money after bad if the research is trash?
Despite that, if there is any 'muzzling' going on it must, needless to say, be at the behest of the legislative as well as executive branches, and the politically isolated Board of the institute.
I'm a little confused. We start off by saying that corporate-sponsored research is 'more likely' to be biased than non-so-sponsored research of the same thing, and then we finish by pointing out that government-sponsored research is more likely to be biased? What's being compared to what here? I thought if a study wasn't corporate-sponsored it was usually government-sponsored.
So researchers should be able to defraud the government at will? Yes, there are dangers from the politicization of science, but there are also dangers from unaccountability. The former can't be prevented through law but only through administration.
What is the purported legitimate purpose of this clause? Specifically, is there an example of research in the past at which this clause is targeted?
Can anyone explain what
“Any research published under clause(ii)(IV) shall be within the bounds of and entirely consistentwith the evidence and findings produced underthe contract with the Institute under this subparagraph."
means?
Seriously. Isn't "the evidence and findings produced" identical with "research"? How can this clause be violated? I don't get it.
Thanks! (I'm the author of the piece that Robin quotes).
Another wonderful example of respect for "free speech."
Thanks Steven!
It's in the final version of the bill. Section 6301 of the Patient Protection Act (Public Law No: 111-148) adds the following section to the Social Security Act:
"Any research published under clause(ii)(IV) shall be within the bounds of and entirely consistentwith the evidence and findings produced underthe contract with the Institute under this subparagraph.If the Institute determines that those requirementsare not met, the Institute shall not enter intoanother contract with the agency, instrumentality, orentity which managed or conducted such research fora period determined appropriate by the Institute (butnot less than 5 years)."
This act was subsequently modified by reconciliation, but looking at the table of contents reveals that Section 6301 was not among the modified sections.