Tamiflu and transparency - Some general thoughts on data access and decision making Mike Clarke

Tamiflu and transparency
Some general thoughts on data
 access and decision making

         January 28 2010

           Mike Clarke
         UK Cochrane Centre
    School of Nursing and Midwifery,
        Trinity College Dublin

“Oseltamivir 150 mg daily is effective in
preventing lower respiratory tract complications
in influenza cases (OR 0.32, 95% CI 0.18 to
0.57), especially bronchitis (OR 0.40, 95% CI
0.21 to 0.76) and pneumonia (OR 0.15, 95% CI
0.03 to 0.69), but not in ILI cases (OR 0.21,
95% CI 0.02 to 2.04). Both NIs are effective in
preventing complications of all types in the
intention-to-treat (ITT) population (OR 0.49,
95% CI 0.38 to 0.62), although these
observations are based on single studies
(Kaiser 2003; Makela 2000) the combined
denominator is fairly substantial (2991).”

Oseltamivir did not reduce influenza lower
respiratory tract complications (risk ratio 0.55,
95% CI 0.22 to 1.35),

“Data on the effectiveness of oseltamivir
against the complications of influenza are
confusing…. The remaining published
evidence is insufficient to answer the
question about the effectiveness of either
neuraminidase inhibitor on reducing the
complications of lower respiratory tract
infection, antibiotic use, or admissions to
hospital.”

The Background

April 2009
 Reports of Mexican or swine flu (H1N1)

June 2009
 WHO declares pandemic

July/August 2009
 NIHR commission a series of research studies, including
 the updating of relevant Cochrane reviews

 Billions of Euros spent on Tamiflu

Tamiflu and influenza symptoms

Neuraminidase inhibitors have modest
effectiveness.

They shorten the duration of symptoms by one
to day days, when taken within 48 hours of
onset.

Tamiflu and complications: certainty
•   Roche [roche.com] (2005): "Tamiflu delivers ... [a] 67% reduction in
    secondary complications such as bronchitis, pneumonia and sinusitis in
    otherwise healthy individuals"
•   Kaiser et al (2003): "Our analysis found that early treatment of influenza
    illness with the neuraminidase inhibitor oseltamivir significantly reduced
    influenza-related LRTCs, associated antibiotic use, and the risk of
    hospitalization. This effect was observed in both at-risk subjects and
    otherwise healthy individuals."
•   EU EMEA (2009): "The proportion of subjects who developed specified
    lower respiratory tract complications (mainly bronchitis) treated with
    antibiotics was reduced from 12.7% (135/1063) in the placebo group to
    8.6% (116/1350) in the oseltamivir treated population (p = 0.0012)."
•   US CDC (2008): "In a study that combined data from 10 clinical trials, the
    risk for pneumonia among those participants with laboratory-confirmed
    influenza receiving oseltamivir was approximately 50% lower than among
    those persons receiving a placebo and 34% lower among patients at risk for
    complications (p

Tamiflu and complications: uncertainty
 •   Roche [tamiflu.com] (2009): "Treatment with TAMIFLU has not been proven
     to have a positive impact on [asthma, emphysema, other chronic lower
     respiratory diseases, pneumonia, other respiratory diseases, pneumonitis,
     and influenza-related death].”
 •   US FDA (2008): "Serious bacterial infections may begin with influenza-like
     symptoms or may coexist with or occur as complications during the course
     of influenza. TAMIFLU has not been shown to prevent such complications."
 •   Burch et al (2009): "Overall, little information was available on the effects of
     either drug on the incidence of complications, and there were very few
     events, in both the healthy adult and at-risk populations. Furthermore,
     weaknesses in the available evidence limit the reliability and the ability to
     generalise any results relating to the effect of these drugs on the rates of
     complications."
 •   Canadian Coordinating Office for Health Technology Assessment (2002):
     "There is insufficient evidence to show that oseltamivir reduces
     complications, hospitalizations or death when used to treat: normally
     healthy people suspected of having influenza, or; those who are at risk for
     developing complications."

What would
 you do?

Updating the Cochrane review
Feedback from Keiji Hayashi (July 14 2009)
 The complications data in the Cochrane review were
 from a single peer-reviewed study by Kaiser et al, which
 had meta-analysed ten manufacturer funded trials from
 the late 1990s. Two of these were published in peer
 reviewed journals. Eight were apparently either
 unpublished or published only in abstract form.

Plan by the Cochrane authors
 Try to verify the data on all ten trials and to do their own
 meta-analysis.

What would
 you do?

Timeline
August 2009
 Lead author, Tom Jefferson, wrote to the
 authors of Kaiser et al, but was told that they no
 longer had the files and to contact Roche. He
 also wrote to authors of the two published trials.
 One responded, directing him to the
 manufacturer.

September 2009
 Tom asks Roche for data.

Timeline
October 2 2009
 Roche indicated a willingness to share data, but sent Tom a
 "confidentiality agreement," containing a clause saying that he
 agrees "not to disclose ... the existence and terms of this
 Agreement“.

October 3 2009
 Tom doesn’t sign but asks for clarification.

October 7 2009
 Roche ask Tom to restate which data he was seeking. He does.
 Roche replies that it is unable to provide data because it had already
 provided it for a similar meta-analysis being started by an
 independent expert influenza group. They say that the Cochrane
 request might conflict with that review.

Timeline
October 13 2009
 Tom asked Roche to outline its concerns and explain why sending
 data to multiple groups of independent researchers should pose a
 problem. Roche did not answer these questions.

October 21 2009
 Roche emailed Tom excerpts of company reports from all clinical
 trials used by Kaiser et al. Jefferson wrote to Roche explaining that
 the files were insufficient to verify the effects on complications
 claims in Kaiser et al and the methods used in the trials.

October 28 2009
 Roche said it would send more information the following week. Tom
 told them that the deadline was now past, but that the Cochrane
 reviewers would accept any additional information for future
 updates.

What would
 you do?

Fig 6 Effect of oseltamivir compared with placebo on complications (including pneumonia,
bronchitis, or "other lower respiratory tract infections") requiring antibiotics in laboratory
confirmed influenza, based on study by Kaiser et al and three other studies (complications
  included pneumonia, bronchitis, otitis media, and sinusitis). Unpublished studies were
                                          excluded

                      Jefferson, T. et al. BMJ 2009;339:b5106

Copyright ©2009 BMJ Publishing Group Ltd.

What would
 you do?

Implications
Independent trials
  The only trials were done or sponsored by the
  manufacturer.

Unpublished trials
  Most of the trials had not been published in full

Access to raw data
  Some of the authors had not seen the data. None of the
  lead authors seemed able to provide it, or new analyses.

Trust
• Much of the evidence on drug safety and effectiveness is
  taken on trust.
• Governments and international bodies have relied
  heavily on Kaiser et al and on observational studies to
  justify the stockpiling and widespread use of oseltamivir.
• They probably assumed that others had looked critically
  and comprehensively at the complete dataset. Those
  others might include drug regulators and health
  technology assessors such as NICE, as well as journal
  editors and Cochrane reviewers.
• Unfortunately, that might not be the case.

Conclusion
The current system isn’t working. Worse than that, it
gives a false sense of security. The system’s failures
have left a legacy of drug evaluations for which, in the
absence of better information, we must assume the
same levels of confusion and uncertainty as for
oseltamivir. The drug industry directly or indirectly
undertakes the majority of all drug evaluations, so most
of the evidence used to support drug policy and
treatment remains shrouded in secrecy. In only a
minority of cases will the data have been subject to full
independent analysis and interpretation. In many if not
most cases, the only people who have seen the entire
dataset are company employees.

What needs to be done?
•   We need more publicly funded trials and the drug industry should
    fund independent trials for licensing their drugs.
•   Governments should enact relevant laws.
•   Researchers who said in their protocols that they would do certain
    analyses should be required to do them or explain why they have
    not done so, and to make the results available.
•   Authors should state who saw the full primary dataset and who saw
    only summary data, who decided what analyses to do and report,
    and who is the "custodian" for the data. The custodian should be
    able and willing to show the full dataset to systematic reviewers and
    other researchers.
•   Journals should consider adopting a policy of independent statistical
    review and should push for independent, individual patient data
    meta-analyses wherever possible, especially for drugs to be used in
    common conditions and purchased in large quantities to protect the
    public’s health.

What would
 you do?