Guidance for Industry
`
`E9 Statistical Principles for Clinical
`Trials
`
`U.S. Department of Health and Human Services
`Food and Drug Administration
`Center for Drug Evaluation and Research (CDER)
`Center for Biologics Evaluation and Research (CBER)
`September 1998
`ICH
`
`Sandoz Inc. IPR2016-00318
`Sandoz v. Eli Lilly, Exhibit 1118-0001
`
`

`
`Guidance for Industry
`E9 Statistical Principles for Clinical
`Trials
`
`Additional copies are available from:
`
`Office of Training and Communications
`Division of Drug Information (HFD-240)
`Center for Drug Evaluation and Research (CDER),
`5600 Fishers Lane, Rockville, MD 20857 (Tel) 301-827-4573
`http : //www.fda. gov/cde r/gui dance/index.htm
`
`or
`
`Office of Communication, Training, and Manufacturers Assistance (HFM-40)
`Center for Biologics Evaluation and Research (CBER)
`1401 Rockville Pike, Rockville, MD 20852-1448
`http ://www.fda.gov/cber/guidelines.htm ; (Fax) 888-CBERFAX or 301-827-3844
`(Voice Information) 800-835-4709 or 301-827-1800
`
`U.S. Department of Health and Human Services
`Food and Drug Administration
`Center for Drug Evaluation and Research (CDER)
`Center for Biologics Evaluation and Research (CBER)
`September 1998
`ICH
`
`Sandoz Inc. IPR2016-00318
`Sandoz v. Eli Lilly, Exhibit 1118-0002
`
`

`
`TABLE OF CONTENTS
`
`I.
`
`INTRODUCTION ....................................................................................................................................................... 1
`
`A. BACKGROUND AND PURPOSE (1.1) ........................................................................................................................... 1
`B. SCOPE AND DIRECTION (1.2) ..................................................................................................................................... 2
`
`II. CONSIDERATIONS FOR OVERALL CLINICAL DEVELOPMENT ................................................................. 4
`
`A.
`I.
`
`C.
`
`TmAL CONTEXT (2.1) ................................................................................................................................................ 4
`SCOPE OF TmALS (2.2) ............................................................................................................................................. 6
`DESIGN TECHNIQUES TO AVOID BIAS (2.3) ............................................................................................................. 10
`
`III. TRIAL DESIGN CONSIDERATIONS ................................................................................................................... 14
`
`A.
`B.
`
`C.
`
`D.
`E.
`F.
`
`DESIGN CONFIGURATION (3.1) ................................................................................................................................ 14
`MULTICENTER TRO, LS (3.2) .................................................................................................................................... 16
`TYPE OF COMPARISON (3.3) .................................................................................................................................... 18
`GROUP SEQUENTIAL DESIGNS (3.4) ......................................................................................................................... 21
`SAMPLE SIZE (3.5) ................................................................................................................................................... 21
`DATA CAr’ru~ AND P~OCESSING (3.6) ................................................................................................................... 23
`
`IV. TRIAL CONDUCT CONSIDERATIONS .............................................................................................................. 23
`
`A.
`B.
`C.
`D.
`E.
`F.
`
`TRIAL MONITORING AND INTERIM ANALYSIS (4.1) ................................................................................................. 23
`CHANGES IN INCLUSION AND EXCLUSION CRITERIA (4.2) ........................................................................................ 24
`ACCRUAL RATES (4.3) ............................................................................................................................................ 24
`SAMPLE SIZE ADJUSTMENT (4.4) ............................................................................................................................. 24
`INTERIM ANALYSIS AND EAP&Y STOPPING (4.5) ...................................................................................................... 24
`ROLE OF INDEPENDENT DATA MONITORING COM~TTEE (IDMC) (4.6) ............................................................... 26
`
`V.
`
`DATA ANALYSIS CONSIDERATIONS ................................................................................................................ 27
`
`A.
`B.
`
`PRESPECIFICATION OF THE ANALYSIS (5.1) ............................................................................................................. 27
`ANALYSIS SETS (5.2) .............................................................................................................................................. 27
`
`C. MISSING VALUES AND OUTLIERS (5.3) .................................................................................................................... 31
`D.
`DATA TRANSFORMATION (5.4) ................................................................................................................................ 31
`E.
`ESTIMATION, CONFIDENCE INTERVALS, AND HYPOTHESIS TESTING (5.5) .............................................................. 32
`F.
`ADJUSTMENT OF SIGNIFICANCE AND CONFIDENCE LEVELS (5.6) ............................................................................. 33
`G.
`SUBGROUPS, INTERACTIONS, AND COVAPaATES (5.7) ............................................................................................. 33
`H.
`INTEGRITY OF DATA AND COMPUTER SOFTWARE VALIDITY (5.8) ............................................................................. 34
`
`EVALUATION OF SAFETY AND TOLERABILITY ....................................................................................... 34
`
`A.
`B.
`C.
`D.
`E.
`
`SCOPE OF EVALUATION (6.1) ................................................................................................................................... 34
`CHOICE OF VARIABLES AND DATA COLLECTION (6.2) ............................................................................................. 34
`SET OF SUBJECTS TO BE EVALUATED AND P~ESENTATION OF DATA (6.3) ............................................................... 35
`STATISTICAL EVALUATION (6.4) .............................................................................................................................. 36
`INTEGRATED SUMMARY (6.5) ................................................................................................................................... 37
`
`REPORTING ....................................................................................................................................................... 37
`
`A. EVALUATION AND REPORTING (7.1) ........................................................................................................................ 37
`B. SUMMARIZING THE CLINICAL DATABASE (7.2) ......................................................................................................... 39
`
`GLOSSARY (ANNEX 1) ................................................................................................................................................... 41
`
`Sandoz Inc. IPR2016-00318
`Sandoz v. Eli Lilly, Exhibit 1118-0003
`
`

`
`GUIDANCE FOR INDUSTRY1
`
`E9 Statistical Principles for Clinical Trials
`
`This guidance represents the Food and Drag Administration’s (FDA’s) current thinking on this topic. It
`does not create or confer any rights for or on any person and does not operate to bind FDA or the public.
`An alternative approach may be used if such approach satisfies the requirements of the applicable
`statutes and regulations.
`
`I.
`
`INTRODUCTION
`
`A.
`
`Background and Purpose (1.1) 2
`
`The efficacy and safety of medicinal products should be demonstrated by clinical trials that
`follow the guidance in E6 Good Clinical Practice: Consolidated Guidance adopted by the
`ICH, May 1, 1996. The role of statistics in clinical trial design and analysis is
`acknowledged as essential in that ICH guidance. The proliferation of statistical research in
`the area of clinical trials coupled with the critical role of clinical research in the drug
`approval process and health care in general necessitate a succinct document on statistical
`issues related to clinical trials. This guidance is written primarily to attempt to harmonize
`the principles of statistical methodology applied to clinical trials for marketing
`applications submitted in Europe, Japan and the United States.
`
`As a starting point, this guidance utilized the CPMP (Committee for Proprietary Medicinal
`Products) Note for Guidance entitled Biostatistical Methodology in Clinical Trials in
`Applications for Marketing Authorizations for Medicinal Products (December 1994). It
`was also influenced by Guidelines on the Statistical Analysis of Clinical Studies (March
`1992) from the Japanese Ministry of Health and Welfare and the U.S. Food and Drug
`Administration document entitled Guideline for the Format and Content of the Clinical
`
`1 This guidance was developed witNn flae Expert Working Group (Efficacy) of flae Intemafional Conference on
`
`Harmonisafion of Technical Requirements for Regislxafion of Pharmaceuticals for Human Use (ICH) and has been subject to
`consultation by the regulatory paxties, in accordance wifla flae ICH process. This document has been endorsed by flae ICH
`Steering Committee at Step 4 of flae ICH process, February 1998. At Step 4 of flae process, flae final draft is recommended for
`adoption to flae regulatory bodies of flae European Union, Japan, and flae United States. This guidance was published in
`FederalRegister on September 16, 1998 (63 FR 49583), and is applicable to drug and biological products.
`
`2 Arabic numbers reflect flae orgaxtizational breakdown in flae document endorsed by flae ICH Steering Committee at
`
`Step 4 offlae ICH process, February 1998.
`
`Sandoz Inc. IPR2016-00318
`Sandoz v. Eli Lilly, Exhibit 1118-0004
`
`

`
`and Statistical Sections of a New Drug Appfcation (July 1988). Some topics related to
`statistical principles and methodology are also embedded within other ICH guidances,
`particularly those listed below. The specific guidance that contains related text will be
`identified in various sections of this document.
`
`E1A
`E2A
`
`E2B
`
`E2C
`
`E3
`E4
`E5
`E6
`E7
`E8
`EIO
`M1
`
`M3
`
`The Extent of Population Exposure to Assess Clinical Safety (March 1995)
`CBnical Safety Data Management: Definitions and Standards for Expedited
`Reporting (March 1995)
`CBnical Safety Data Management: Data Elements for Transmission of Individual
`Case Safety Reports (January 1998)
`CBnical Safety Data Management: Periodic Safety Update Reports for Marketed
`Drugs (November 1996)
`Structure and Content of Clinical Study Reports (July 1996)
`Dose-Response Information to Support Drug Registration (November 1994)
`Ethnic Factors in the AcceptabiBty of Foreign CBnical Data (June 1998)
`Good CBnical Practice: ConsoBdated GuideBne (April 1996)
`Studies in Support of Special Populations: Geriatrics (August 1994)
`General Considerations for CBnical Trials (December 1997
`Choice of Control Group in CBnical Trials (September 1999)
`Standardization of Medical Terminology for Regulatory Purposes (November
`1999)
`NoncBnical Safety Studies for the Conduct of Human CBnical Trials for
`Pharmaceuticals (July 1997)
`
`This guidance is intended to give direction to sponsors in the design, conduct, analysis, and
`evaluation of clinical trials of an investigational product in the context of its overall
`clinical development. The document will also assist scientific experts charged with
`preparing application summaries or assessing evidence of efficacy and safety, principally
`from clinical trials in later phases of development.
`
`B.
`
`Scope and Direction (1.2)
`
`The focus of this guidance is on statistical principles. It does not address the use of
`specific statistical procedures or methods. Specific procedural steps to ensure that
`principles are implemented properly are the responsibility of the sponsor. Integration of
`data across clinical trials is discussed, but is not a primary focus of this guidance.
`Selected principles and procedures related to data management or clinical trial monitoring
`activities are covered in other ICH guidances and are not addressed here.
`
`This guidance should be of interest to individuals from a broad range of scientific
`disciplines. However, it is assumed that the actual responsibility for all statistical work
`associated with clinical trials will lie with an appropriately qualified and experienced
`statistician, as indicated in ICH E6. The role and responsibility of the trial statistician (see
`Glossary), in collaboration with other clinical trial professionals, is to ensure that
`statistical principles are applied appropriately in clinical trials supporting drug
`
`Sandoz Inc. IPR2016-00318
`Sandoz v. Eli Lilly, Exhibit 1118-0005
`
`

`
`development. Thus, the trial statistician should have a combination of education/training
`and experience sufficient to implement the principles articulated in this guidance.
`
`For each clinical trial contributing to a marketing application, all important details of its
`design and conduct and the principal features of its proposed statistical analysis should be
`clearly specified in a protocol written before the trial begins. The extent to which the
`procedures in the protocol are followed and the primary analysis is planned a priori will
`contribute to the degree of confidence in the final results and conclusions of the trial. The
`protocol and subsequent amendments should be approved by the responsible personnel,
`including the trial statistician. The trial statistician should ensure that the protocol and any
`amendments cover all relevant statistical issues clearly and accurately, using technical
`terminology as appropriate.
`
`The principles outlined in this guidance are primarily relevant to clinical trials conducted
`in the later phases of development, many of which are confirmatory trials of efficacy. In
`addition to efficacy, confirmatory trials may have as their primary variable a safety
`variable (e.g., an adverse event, a clinical laboratory variable, or an electrocardiographic
`measure) or a pharmacodynamic or pharmacokinetic variable (as in a confirmatory
`bioequivalence trial). Furthermore, some confirmatory findings may be derived from data
`integrated across trials, and selected principles in this guidance are applicable in this
`situation. Finally, although the early phases of drug development consist mainly of clinical
`trials that are exploratory in nature, statistical principles are also relevant to these clinical
`trials. Hence, the substance of this document should be applied as far as possible to all
`phases of clinical development.
`
`Many of the principles delineated in this guidance deal with minimizing bias (see
`Glossary) and maximizing precision. As used in this guidance, the term bias describes the
`systematic tendency of any factors associated with the design, conduct, analysis, and
`interpretation of the results of clinical trials to make the estimate of a treatment effect (see
`Glossary) deviate from its true value. It is important to identify potential sources of bias as
`completely as possible so that attempts to limit such bias may be made. The presence of
`bias may seriously compromise the ability to draw valid conclusions from clinical trials.
`
`Some sources of bias arise from the design of the trial, for example an assignment of
`treatments such that subj ects at lower risk are systematically assigned to one treatment.
`Other sources of bias arise during the conduct and analysis of a clinical trial. For example,
`protocol violations and exclusion of subjects from analysis based upon knowledge of
`subj ect outcomes are possible sources of bias that may affect the accurate assessment of the
`treatment effect. Because bias can occur in subtle or unknown ways and its effect is not
`measurable directly, it is important to evaluate the robustness of the results and primary
`conclusions of the trial. Robustness is a concept that refers to the sensitivity of the overall
`conclusions to various limitations of the data, assumptions, and analytic approaches to data
`analysis. Robustness implies that the treatment effect and primary conclusions of the trial
`are not substantially affected when analyses are carried out based on alternative
`assumptions or analytic approaches. The interpretation of statistical measures of
`
`Sandoz Inc. IPR2016-00318
`Sandoz v. Eli Lilly, Exhibit 1118-0006
`
`

`
`uncertainty of the treatment effect and treatment comparisons should involve consideration
`of the potential contribution of bias to the p-value, confidence interval, or inference.
`
`Because the predominant approaches to the design and analysis of clinical trials have been
`based on frequentist statistical methods, the guidance largely refers to the use of frequentist
`methods (see Glossary) when discussing hypothesis testing and/or confidence intervals.
`This should not be taken to imply that other approaches are not appropriate; the use of
`Bayesian (see Glossary) and other approaches may be considered when the reasons for
`their use are clear and when the resulting conclusions are sufficiently robust.
`
`CONSIDERATIONS FOR OVERALL CLINICAL DEVELOPMENT
`
`A.
`
`Trial Context (2.1)
`
`1.
`
`Development Plan (2.1.1)
`
`The broad aim of the process of clinical development of a new drug is to find out
`whether there is a dose range and schedule at which the drug can be shown to be
`simultaneously safe and effective, to the extent that the risk-benefit relationship is
`acceptable. The particular subjects who may benefit from the drug, and the specific
`indications for its use, also need to be defined.
`
`Satisfying these broad aims usually requires an ordered program of clinical trials,
`each with its own specific objectives (see ICH E8). This should be specified in a
`clinical plan, or a series of plans, with appropriate decision points and flexibility
`to allow modification as knowledge accumulates. A marketing application should
`clearly describe the main content of such plans, and the contribution made by each
`trial. Interpretation and assessment of the evidence from the total program of trials
`involves synthesis of the evidence from the individual trials (see section VII.B).
`This is facilitated by ensuring that common standards are adopted for a number of
`features of the trials, such as dictionaries of medical terms, definition and timing of
`the main measurements, handling of protocol deviations, and so on. A statistical
`summary, overview, or meta-analysis (see Glossary) may be informative when
`medical questions are addressed in more than one trial. Where possible, this
`should be envisaged in the plan so that the relevant trials are clearly identified and
`any necessary common features of their designs are specified in advance. Other
`major statistical issues (if any) that are expected to affect a number of trials in a
`common plan should be addressed in that plan.
`
`2.
`
`Confirmatory Trial (2.1.2)
`
`A confirmatory trial is an adequately controlled trial in which the hypotheses are
`stated in advance and evaluated. As a rule, confirmatory trials are necessary to
`provide firm evidence of efficacy or safety. In such trials the key hypothesis of
`interest follows directly from the trial’s primary obj ective, is always predefined,
`
`Sandoz Inc. IPR2016-00318
`Sandoz v. Eli Lilly, Exhibit 1118-0007
`
`

`
`and is the hypothesis that is subsequently tested when the trial is complete. In a
`confirmatory trial, it is equally important to estimate with due precision the size of
`the effects attributable to the treatment of interest and to relate these effects to their
`clinical significance.
`
`Confirmatory trials are intended to provide firm evidence in support of claims;
`hence adherence to protocols and standard operating procedures is particularly
`important. Unavoidable changes should be explained and documented, and their
`effect examined. A justification of the design of each such trial and of other
`important statistical aspects, such as the principal features of the planned analysis,
`should be set out in the protocol. Each trial should address only a limited number
`of questions.
`
`Firm evidence in support of claims requires that the results of the confirmatory
`trials demonstrate that the investigational product under test has clinical benefits.
`The confirmatory trials should therefore be sufficient to answer each key clinical
`question relevant to the efficacy or safety claim clearly and definitively. In
`addition, it is important that the basis for generalization (see Glossary) to the
`intended patient population is understood and explained; this may also influence the
`number and type (e.g., specialist or general practitioner) of centers and/or trials
`needed. The results of the confirmatory trial(s) should be robust. In some
`circumstances, the weight of evidence from a single confirmatory trial may be
`sufficient.
`
`3.
`
`Exploratory Trial (2.1.3)
`
`The rationale and design of confirmatory trials nearly always rests on earlier
`clinical work carried out in a series of exploratory studies. Like all clinical trials,
`these exploratory studies should have clear and precise objectives. However, in
`contrast to confirmatory trials, their obj ectives may not always lead to simple tests
`of predefined hypotheses. In addition, exploratory trials may sometimes require a
`more flexible approach to design so that changes can be made in response to
`accumulating results. Their analysis may entail data exploration. Tests of
`hypothesis may be carried out, but the choice of hypothesis may be data dependent.
`Such trials cannot be the basis of the formal proof of efficacy, although they may
`contribute to the total body of relevant evidence.
`
`Any individual trial may have both confirmatory and exploratory aspects. For
`example, in most confirmatory trials the data are also subj ected to exploratory
`analyses which serve as a basis for explaining or supporting their findings and for
`suggesting further hypotheses for later research. The protocol should make a clear
`distinction between the aspects of a trial which will be used for confirmatory proof
`and the aspects which will provide data for exploratory analysis.
`
`Sandoz Inc. IPR2016-00318
`Sandoz v. Eli Lilly, Exhibit 1118-0008
`
`

`
`B.
`
`Scope of Trials (2.2)
`
`1.
`
`Population (2.2.1)
`
`In the earlier phases of drug development, the choice of subjects for a clinical trial
`may be heavily influenced by the wish to maximize the chance of observing specific
`clinical effects of interest. Hence they may come from a very narrow subgroup of
`the total patient population for which the drug may eventually be indicated.
`However, by the time the confirmatory trials are undertaken, the subjects in the
`trials should more closely mirror the target population. In these trials, it is
`generally helpful to relax the inclusion and exclusion criteria as much as possible
`within the target population while maintaining sufficient homogeneity to permit
`precise estimation of treatment effects. No individual clinical trial can be expected
`to be totally representative of future users because of the possible influences of
`geographical location, the time when it is conducted, the medical practices of the
`particular investigator(s) and clinics, and so on. However, the influence of such
`factors should be reduced wherever possible and subsequently discussed during the
`interpretation of the trial results.
`
`2.
`
`Primary and Secondary Variables (2.2.2)
`
`The primary variable (target variable, primary endpoint) should be the variable
`capable of providing the most clinically relevant and convincing evidence directly
`related to the primary objective of the trial. There should generally be only one
`primary variable. This will usually be an efficacy variable, because the primary
`objective of most confirmatory trials is to provide strong scientific evidence
`regarding efficacy. Safety/tolerability may sometimes be the primary variable, and
`will always be an important consideration. Measurements relating to quality of life
`and health economics are further potential primary variables. The selection of the
`primary variable should reflect the accepted norms and standards in the relevant
`field of research. The use of a reliable and validated variable with which
`experience has been gained either in earlier studies or in published literature is
`recommended. There should be sufficient evidence that the primary variable can
`provide a valid and reliable measure of some clinically relevant and important
`treatment benefit in the patient population described by the inclusion and exclusion
`criteria. The primary variable should generally be the one used when estimating
`the sample size (see section Ill.E).
`
`In many cases, the approach to assessing subject outcome may not be
`straightforward and should be carefully defined. For example, it is inadequate to
`specify mortality as a primary variable without further clarification; mortality may
`be assessed by comparing proportions alive at fixed points in time or by comparing
`overall distributions of survival times over a specified interval. Another common
`example is a recurring event; the measure of treatment effect may again be a simple
`dichotomous variable (any occurrence during a specified interval), time to first
`
`Sandoz Inc. IPR2016-00318
`Sandoz v. Eli Lilly, Exhibit 1118-0009
`
`

`
`occurrence, rate of occurrence (events per time units of observation), and so on.
`The assessment of functional status over time in studying treatment for chronic
`disease presents other challenges in selection of the primary variable. There are
`many possible approaches, such as comparisons of the assessments done at the
`beginning and end of the interval of observation, comparisons of slopes calculated
`from all assessments throughout the interval, comparisons of the proportions of
`subjects exceeding or declining beyond a specified threshold, or comparisons
`based on methods for repeated measures data. To avoid multiplicity concerns
`arising from post hoc definitions, it is critical to specify in the protocol the precise
`definition of the primary variable as it will be used in the statistical analysis. In
`addition, the clinical relevance of the specific primary variable selected and the
`validity of the associated measurement procedures will generally need to be
`addressed and justified in the protocol.
`
`The primary variable should be specified in the protocol, along with the rationale
`for its selection. Redefinition of the primary variable after unblinding will almost
`always be unacceptable, since the biases this introduces are difficult to assess.
`When the clinical effect defined by the primary objective is to be measured in more
`than one way, the protocol should identify one of the measurements as the primary
`variable on the basis of clinical relevance, importance, objectivity, and/or other
`relevant characteristics, whenever such selection is feasible.
`
`Secondary variables are either supportive measurements related to the primary
`obj ective or measurements of effects related to the secondary objectives. Their
`predefinition in the protocol is also important, as well as an explanation of their
`relative importance and roles in interpretation of trial results. The number of
`secondary variables should be limited and should be related to the limited number
`of questions to be answered in the trial.
`
`3.
`
`Composite Variables (2.2.3)
`
`If a single primary variable cannot be selected from multiple measurements
`associated with the primary objective, another useful strategy is to integrate or
`combine the multiple measurements into a single or composite variable, using a
`predefined algorithm. Indeed, the primary variable sometimes arises as a
`combination of multiple clinical measurements (e.g., the rating scales used in
`arthritis, psychiatric disorders, and elsewhere). This approach addresses the
`multiplicity problem without requiring adjustment to the Type I error. The method
`of combining the multiple measurements should be specified in the protocol, and an
`interpretation of the resulting scale should be provided in terms of the size of a
`clinically relevant benefit. When a composite variable is used as a primary
`variable, the components of this variable may sometimes be analyzed separately,
`where clinically meaningful and validated. When a rating scale is used as a
`primary variable, it is especially important to address factors such as content
`
`Sandoz Inc. IPR2016-00318
`Sandoz v. Eli Lilly, Exhibit 1118-0010
`
`

`
`validity (see Glossary), inter- and intrarater reliability (see Glossary), and
`responsiveness for detecting changes in the severity of disease.
`
`4.
`
`Global Assessment Variables (2.2.4)
`
`In some cases, global assessment variables (see Glossary) are developed to
`measure the overall safety, overall efficacy, and/or overall usefulness of a
`treatment. This type of variable integrates obj ective variables and the
`investigator’s overall impression about the state or change in the state of the
`subject, and is usually a scale of ordered categorical ratings. Global assessments
`of overall efficacy are well established in some therapeutic areas, such as
`neurology and psychiatry.
`
`Global assessment variables generally have a subj ective component. When a
`global assessment variable is used as a primary or secondary variable, fuller
`details of the scale should be included in the protocol with respect to:
`
`The relevance of the scale to the primary obj ective of the trial;
`
`The basis for the validity and reliability of the scale;
`
`How to utilize the data collected on an individual subj ect to assign him/her
`to a unique category of the scale;
`
`How to assign subjects with missing data to a unique category of the scale,
`or otherwise evaluate them.
`
`If objective variables are considered by the investigator when making a global
`assessment, then those obj ective variables should be considered as additional
`primary or, at least, important secondary variables.
`
`Global assessment of usefulness integrates components of both benefit and risk and
`reflects the decisionmaking process of the treating physician, who must weigh
`benefit and risk in making product use decisions. A problem with global usefulness
`variables is that their use could in some cases lead to the result of two products
`being declared equivalent despite having very different profiles of beneficial and
`adverse effects. For example, judging the global usefulness of a treatment as
`equivalent or superior to an alternative may mask the fact that it has little or no
`efficacy but fewer adverse effects. Therefore, it is not advisable to use a global
`usefulness variable as a primary variable. If global usefulness is specified as
`primary, it is important to consider specific efficacy and safety outcomes separately
`as additional primary variables.
`
`Sandoz Inc. IPR2016-00318
`Sandoz v. Eli Lilly, Exhibit 1118-0011
`
`

`
`5.
`
`Multiple Primary Variables (2.2.5)
`
`It may sometimes be desirable to use more than one primary variable, each of
`which (or a subset of which) could be sufficient to cover the range of effects of the
`therapies. The planned manner of interpretation of this type of evidence should be
`carefully spelled out. It should be clear whether an impact on any of the variables,
`some minimum number of them, or all of them, would be considered necessary to
`achieve the trial obj ectives. The primary hypothesis or hypotheses and parameters
`of interest (e.g., mean, percentage, distribution) should be clearly stated with
`respect to the primary variables identified, and the approach to statistical inference
`described. The effect on the Type I error should be explained because of the
`potential for multiplicity problems (see section V.F); the method of controlling
`Type I error should be given in the protocol. The extent of intercorrelation among
`the proposed primary variables may be considered in evaluating the impact on
`Type I error. If the purpose of the trial is to demonstrate effects on all of t