Negative Headlines, Scientific Vulnerability and the Future of Genetic Tests
July 19th, 2011
Posted by Shane Climie, Ph.D.
Though the life sciences industry is making great progress with genetic tests and targeted therapies, a recent article in the New York Times (“How Bright Promise in Cancer Testing Fell Apart”) exposes disturbing and cautionary insights into the application of this technology. The article revealed that research on the application of complex genetic tests to match cancer patients to the best available therapy may not be as promising as it seems.
The article describes a situation that emerged just over a year ago that raised serious doubts about a test developed by researchers at Duke University. The test used gene expression signatures to characterize tumors and to identify those most likely to respond to different drugs.
Duke researchers published several papers that described the discovery and validation of the test, and clinical trials were initiated. A company was set up to provide the test and clinicians at Duke were using the test to guide drug treatment in cancer patients. However, at least one patient died within months after receiving cancer treatment based on what is now known to be a worthless test. Clinical trials were halted and scientific papers in prestigious journals were retracted. Pressure to halt clinical trials came from independent investigators who could not replicate the findings and who found serious deficiencies in the original data on which the test was based.
The Times article highlights several difficult issues. Foremost was the suggestion that data related to genetic research is too complex to be managed properly under standard research conditions. As stated by Dr Lajos Pusztai (researcher at the M.D. Anderson Cancer Center at the University of Texas) in the Times article, it takes a fully diversified staff to effectively manage and analyze the complexity of data in genetic studies. He went on to say, in what was clearly a warning to the community, that unless such a team is in place, researchers “…oversee a machine they don’t understand.”
Genomic technologies hold great promise for the future of targeted therapies and for associated diagnostics. We have seen the successful application of tests based on similar technology and we are starting to see many more tests based on next-generation DNA sequencing, epigenetic testing, SNP analysis, metabolomics, protein expression profiles and other multivariate approaches.
However, based on the incident at Duke (and others including Ovasure and Ovacheck), it seems some researchers (and investors and patients) may still be relying too much on limited validation studies that fail to integrate the complexity of the disease and recognize the level of risk in rapidly evolving scientific strategies. As mainstream media cover these industry setbacks, doubt about the underlying technology itself is planted in the minds of the general public.
So what can those of us within the industry do to address these warnings and quell public concerns?
First, we appreciate that many tests are developed using complex methods that can be difficult to apply. The data is so complex that only a computer can do the appropriate analysis – and the clinical utility is only as good as the underlying quality of the data (which relies on patient selection as well as assay performance and clinical correlation) and associated analysis methods. We need to stress these underlying principles in our dialogue with all stakeholders.
Second, we must remain vigilant about the importance of both clinical validation and analytical validation prior to the adoption of such tests in any setting. Many research groups can become enamored of preliminary data, but hard slogging is needed to ensure that a test is reliable and reproducible and that it can be used to address clinically important questions.
How can we enable better or more streamlined validation? Do you think more regulatory oversight is needed in this area? What is the role of consortia and biobanks? What might be the impact of the issues on the entry of sequencing-based diagnostics into clinical use? We welcome your thoughts on this important topic.
Like this content? Don’t miss an update.
Leave a Reply