The document discusses criteria for evaluating genomic tests, including whether the test is clinically meaningful, how it was validated, reliability, practicality, and cost-effectiveness. It emphasizes that tests must provide value beyond traditional measures and require multiple studies as evidence. Genomic tests are developed through a process similar to drug development, starting with initial studies to develop relevant profiles which are then validated in additional studies with prospective design. Sources of variability must be rigorously controlled and clinical context is important.

1. Criteria to evaluate genomic tests Christer Svedman, MD, PhD Director, Medical Affairs, Europe Genomic Health, Inc.

2. Agenda Background How genomic tests are developed Main criteria to evaluate genomic tests

3. Background

4. Molecular Individualization:The Promise of Cancer Genomics Cancer genomics is the study of the structure, transcription into mRNA, and translation into protein of the human genome, including genes in the cancer cell and in the host that contribute to the development of cancer and its progression from localized disease to one that grows uncontrolled and metastasizes 4 National Cancer Institute. Available at: http://www.cancer.gov/cancertopics/understandingcancer/cancergenomics.

5. Genomic Assays May Be Useful Across the Continuum of Cancer Diagnosis and Treatment Course of Disease Management Disease Risk Assessment Screening Differential Diagnosis Prognosis & Staging Treatment Selection Treatment Monitoring Surveillance 5

6. The Expanding Role of Genomics in Oncology It is well known from clinical practice that not all patients and not all tumors are the same Genomic-based diagnostics enable us to treat cancer based on the individual biology, by providing insight into: Prognosis: What is the likelihood a patient will have disease recurrence? Also may impact estimate of absolute treatment benefit since absolute benefit is typically greater with greater baseline risk (e.g., nodal status) Prediction of treatment benefit: What is the likelihood that a patient will have a greater or lesser relative benefit from treatment? It is likely that within the next decade a molecular diagnostic will play a role in diagnosis of almost all cancers 6 Simon R. J Clin Oncol. 2005;23:7332-7341; Gen Engineering and Biotechnoloy News (October 3, 2008)

7. 7 Number of Genes Associated with Disease Year Source: Online Mendelian Inheritance in Man Cumulative Pace of Disease Gene Discovery 1981-2005

8. Moore’s Law of DNA Sequencing Technology TwoMonths 15 Minutes Time 13 Years Cost $3,000,000,000 $350,000 $1,000 Free? 2002 2002 2005 2006 2001 2010 2011 2014 2015 2003 2004 2007 2008 2009 2012 2013 2016 First DraftofHumanGenomeCompleted FinalDraft of HumanGenome Completed IlluminaandABISequencingTechnologyCommercialized Pacific Biosciences and VisigenCommercializationExpected 454 Sequencing TechnologyCommercialized HelicosCommercializationExpected

9. Applications of High Throughput Sequencing mRNA Profiling miRNA Profiling Chromosomal Analysis Mutation Analysis Personal Genomic Sequencing ncRNA Profiling Methylation Analysis

10. Biomarker Field – A Poor Track Record Many thousands of papers have been published about biomarkers in cancer Very few biomarkers have been proven to have clinical utility and are incorporated in clinical practice

11. Common Problems With Biomarker Studies Clinical/Biology problems Lack of focus on clinical context and evidence needed to change treatment decisions Heterogeneity of cancer Lack of sufficient biologic rationale Study problems Small single institution studies Type I statistical error issues (multiple comparisons) Lack of pre-specification of methods, eligibility criteria, endpoints, and analysis plan Assay problems Lack of standardization/reproducibility Lack of controls

12. How genomic tests are developed

13. Biomarker Field – Principles for Success Delivering what patients, physicians, and payors need Tests must be “Fit for Purpose” and provide value beyond traditional measures Multiple studies are required to provide the evidence Technical innovation brought to standardized implementation Requires collaboration and significant resources to do it right

14. The development of a clinical useful genomic test requires a development program similar to that of a drug candidate Initial studies are required where a relevant genomic profile is developed The selected profile must then be validated in one or preferably several otherstudies Must be prospectively designed but can be performed on archived samples (so called prospective-retrospective trials) 14 Simon RM, et al. J Natl Cancer Inst. 2009;101:1446-1452

15. Revised Criteria for Level of Evidence in Tumor Marker Studies: Proper Study Design Determines the Strength of Results *Level of evidence IV and V studies will never be satisfactory for determination of medical utility 15 Simon RM, et al. J Natl Cancer Inst. 2009;101:1446-1452.

16. Rigorous efforts needed to reduce all sources of variability Preanalytical errors Analytical errors Post analytical errors 16

17. Sources of Variability Scoring system Time to slicing and fixation Reportingelements Method of tissue processing Interpretationcriteria Time of fixation Post-analytical Pre-analytical Use of imageanalysis Type offixation IHC testing variation Assayvalidation Assayconditions Equipment calibration Analytical Control materials Laboratoryprocedures Test reagents Staff competence Type of antigen retrieval Wolff et al 2007

18. KRAS Mutation Predicts Absence of Benefit with Monoclonal Anti-EGFR Therapy in mCRC 18 Analysis of tumor specimens from CO.17 Phase III randomized clinical trial (n=394) Karapetis et al. N Engl J Med 2008;359:1757-65.

19. KRAS in Colorectal Cancer Exploratory retrospective analyses generated hypothesis that patients with KRAS-mutant colorectal cancer (CRC) do not respond to EGFR-targeted therapies (cetuximab, panitumumab) Hypothesis was successfully validated in prospectively-defined analyses of archival specimens from randomized trials of EGFR-targeted therapies in advanced CRC (e.g. CO.17, OPUS, CRYSTAL) Results from retrospective validation studies of KRAS led to rapid change in clinical practice: Major phase III clinical trials involving EGFR-targeted therapies were halted to permit re-design incorporating KRAS status (e.g. NCCTG N0147 and PETACC-8) KRAS testing included in treatment guidelines to guide use of EGFR-targeted agents; now widely adopted in practice 19

21. likelihood of benefit with addition of chemotherapy to tamoxifen? 0.6 Proportion without Distant Recurrence 0.5 0.4 0.3 N Events 424 33 227 31 0.2 All Patients Tam + Chemo P = 0.02 0.1 Tam 0.0 2 4 6 8 10 12 0 Years Paik et al. J Clin Oncol. 2006.

22. Development of the Oncotype DX® Breast Cancer Assay 2001 Breast Cancer Technical Feasibility 2002 Model Building Studies Based on 250 genes 2002 Selection of Final 21-Gene List & Algorithm 2003 Standardization and Validation of Analytical Methods 2003 Validation Studies in NSABP B-14 and Kaiser Permanente

24. Reference normalization compensates for these differences in sample processing and sample ageHollandes Formalin

26. Some tumors contain significant amounts of non-tumor elements (e.g., biopsy cavities, skin, smooth muscle) which require manual microdissection

27. Thus, if < 50% invasive carcinoma, manual microdissection is always performed14 ER 12 r = 0.73, p = 0.001 10 8 ER Whole Section 6 4 2 0 0 2 4 6 8 10 12 14 ER Enriched Tumor ER = estrogen receptor Differences in non-tumor tissue may impact single gene assessment

29. Among block expression: standard deviation < 1.0 normalized expression unitsPoster presented at: United States-Canadian Academy of Pathology 93rd Annual Meeting; March, 2004; Vancouver, British Columbia.

31. Greater magnitude of TAM benefit

33. Lower magnitude of TAM benefit

34. Clear chemotherapy benefit1) Paik et al NEJM 2004, 2) Habel et al Breast Cancer Research 2006 3) Paik et al JCO 2006, 4) Gianni et al JCO 2005

35. Oncotype DX® clinical validation:NSABP B-14, Distant Recurrence Distant recurrence over time 100% 10-Year rate of recurrence = 6.8%*95% CI: 4.0%, 9.6% 90% 80% 10-Year rate of recurrence = 14.3%95% CI: 8.3%, 20.3% 70% 10-Year rate of recurrence = 30.5%*95% CI: 23.6%, 37.4% 60% Proportion without distant recurrence 50% All Patients, n = 668 40% RS < 18, n = 338 30% 20% RS 18-30, n = 149 10% P < 0.001 RS ≥ 31, n = 181 0% 0 2 4 6 8 10 12 14 16 Years RS, Recurrence Score® result *10-Year distant recurrence comparison between low- and high-risk groups: P < 0.001 26 26 Paik S, et al. N Engl J Med. 2004;351:2817-2826.

36. The Oncotype DX® assay can predict benefit from chemotherapy and tamoxifen Recurrence Score® Result < 18 18-30 ≥ 31 Tamoxifen Chemotherapy benefit Tamoxifen + CHEMO 0 5 10 15 20 25 30 35 40 10-Year absolute risk BC death (%) (95% CI) 27 Adapted from Paik S, et al. J Clin Oncol. 2006;24:3726.

37. Main criteria to evaluate genomic tests

39. How was the test validated?

40. How reliable/reproducible is the test?

41. How practical is the test?

43. 31 How was the test validated Number and size of validation studies? Other statistical considerations Prospective or prospective-retrospective studies? In what patient population? Homogenous or heterogeneous.

44. 32 How reliable is the test? What are the potential pre-analytical, analytical and post analytical sources of variability. Are controls used? What is the analytical performance, accuracy and precision of the test? Laboratories performing the test should be appropriately regulated

45. 33 How practical is it to use? Fresh frozen samples or paraffin embedded? Laboratory provides appropriate service to facilitate use? Are results obtained within acceptable timeframe before decision-making? Reimbursement?

46. Is the test cost-effective Increasing focus on health-economical aspects of all interventions in medicine New medicines as well as diagnostics must be cost-effective in order to receive reimbursement.

47. Biomarker Field – Principles for Success Delivering what patients, physicians, and payors need Tests must be “Fit for Purpose” and provide value beyond traditional measures Multiple studies are required to provide the evidence Technical innovation brought to standardized implementation Requires collaboration and significant resources to do it right

48. DetailsResearch design and statistical issues Abstracted from: Simon R. A checklist for evaluating reports of expression profiling for treatment selection. Clin Adv Hem Onc 2006; 219-24. 36

49. DetailsEconomic implications and validity Adapted from Weinstein M, O’Brien B, Hornberger J. et al. Principles of good practice for decision analytic modeling in health-care evaluation: report of the ISPOR Task Force on Good Research Practices--Modeling Studies. Value Health 2003; 6:9-17. 37