OPEN 0 ACCESS Freely available online
`
`;~:. PLoS one
`
`Results and Outcome Reporting
`What Makes it Happen?
`
` nI
`
`ClinicalTrials.gov,
`
`Olga Kirillova*
`
`IICOLL LLC, Portland, Oregon, United States of America
`
`Abstract
`
`Background:At the end of the past century there were multiple concerns regarding lack of transparency in the conduct of
`clinical trials as well as some ethical and scientific issues affecting the trials’ design and reporting. In 2000 ClinicalTrials.gov
`data repository was developed and deployed to serve public and scientific communities with valid data on clinical trials.
`Later in order to increase deposited data completeness and transparency of medical research a set of restrains had been
`imposed making the results deposition compulsory for multiple cases.
`
`Methods:We investigated efficiency of the results deposition and outcome reporting as well as what factors make positive
`impact on providing information of interest and what makes it more difficult, whether efficiency depends on what kind of
`institution was a trial sponsor. Data from the ClinicalTrials.gov repository has been classified based on what kind of
`institution a trial sponsor was. The odds ratio was calculated for results and outcome reporting by different sponsors’ class.
`
`Results:As of 01/01/2012 118,602 clinical trials data deposits were made to the depository. They came from 9068 different
`sources. 35344 (29.8%) of them are assigned as FDA regulated and 25151 (21.2%) as Section 801 controlled substances.
`Despite multiple regulatory requirements, only about 35% of trials had clinical study results deposited, the maximum
`55.56% of trials with the results, was observed for trials completed in 2008.
`
`Conclusions:The most positive impact on depositing results, the imposed restrains made for hospitals and clinics. Health
`care companies showed much higher efficiency than other investigated classes both in higher fraction of trials with results
`and in providing at least one outcome for their trials. They also more often than others deposit results when it is not strictly
`required, particularly, in the case of non-interventional studies.
`
`Citation: Kirillova O (2012) Results and Outcome Reporting n ClinicalTrials.gov, What Makes it Happen? PLoS ONE 7(6): e37847. doi:10.1371/I
`
`
`journal.pone.0037847
`
`Editor: Laxmaiah Manchikanti, University of Louisville, United States of America
`
`Received February 20, 2012; Accepted April 25, 2012; Published June 13, 2012
`Copyright: ß 2012 Olga Kirillova. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits
`unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
`
`Funding: The author has no support or funding to report.
`
`Competing Interests: The author is a founder of IICOLL Limited Liability Company. This does not alter the author’s adherence to all the PLoS ONE policies on
`sharing data and materials.
`
`* E-mail: olga@iicoll.com
`
`Introduction
`
`Clinical studies are important and one of the biggest part of
`modern health care research in US. Besides they are ones of the
`most expensive and, dealing with human subject and people
`health, required to be done with a special care. At the end of the
`past century there were multiple concerns regarding lack of
`transparency in the conduct of clinical trials as well as some ethical
`and scientific issues affecting the trials’ design and reporting [1,2].
`In response on request
`to increase transparency of medical
`research and novel drugs development,
`the Food and Drug
`Administration issued a Modernization Act, Section 113 of which
`required the development of a data registry [3]. So, in February
`2000 ClinicalTrials.gov data repository was developed and
`deployed (Zarin, 2010 Everything You Ever Wanted to Know About
`ClinicalTrials.gov, on-line presentation). At that time it was designed to
`help potential participants find trials, and was primarily focused on
`people with serious or life-threatening conditions. Since then
`through careful review process it was substantially improved to
`become more complete and accurate. In September 2007 Food
`and Drug Administration Amendments Act (FDAAA) was enacted
`
`with a legal requirement of trials registration for a broader group
`of trials than had previously been required under FDAMA [4]. In
`2008, a database for reporting summary results was added to the
`registry [5]. Today technological advancement in large scale data
`processing,
`internet
`speed and cheap and getting cheaper
`electronic storage devices gives us an opportunity to deal with
`large scale data obtained from multiple sources and get a bigger
`picture of a clinical study.
`In recent years there were several papers related to clinical
`trials: general reviews of clinical data repository ClinicalTrials.gov
`progress and development [5–7], investigation on how likely and
`soon a trial registered with ClinicalTrials.gov will result in a peer
`reviewed publication [8,9], concerns related to completeness of an
`outcome in the trials reporting [10], and rigorous study of
`comparative effectiveness and its relationship to funding sources
`[11].
`Characteristic feature of the previous research is that one or
`other kind of selection has been performed rather than meta-
`analysis of all data available. Another point with lack of attention,
`in our opinion,
`is classification of
`institutions
`sponsoring/
`conducting a trial.
`
`PLoS ONE | www.plosone.org
`
`1
`
`June 2012 | Volume 7 |
`
`Issue 6 | e37847
`
`MPI EXHIBIT 1063 PAGE 1
`
`MPI EXHIBIT 1063 PAGE 1
`
`Apotex v. Novo - IPR2024-00631
`Petitioner Apotex Exhibit 1063-0001
`
`

`

`In this study we performed overall meta-analysis of the clinical
`trials deposited into ClinicalTrials.gov repository as of January 1,
`2012; developed advanced classification of trials sponsors and
`compare the results for different classes in two most important
`aspects of
`the deposited information: outcome reporting and
`deposition of clinical results data. Also we tried to decipher what
`factors make the results and outcome reporting more plausible or
`more difficult and whether it depends on the sponsor.
`
`Methods
`
`Data
`Now significant number of clinical study records got public and
`everybody can download them from the site in a well structured
`format that makes the data processing easier and allows to keep
`the original structure and reduce potential errors usually occurring
`when plain text data need to be processed. We took the
`opportunity downloaded, processed and analyzed the data trying
`to decipher interesting regularities and to gain insight into the state
`of clinical research.
`Data has been obtained from ClinicalTrials. gov repository. The
`last update has been done on 01/01/2012 and should contain all
`the clinical trials records as of the pointed date. The data were
`downloaded and imported into an in-house database. They were
`obtained in XML format, so all preexisting formatting has been
`saved. Parsing has been done by in-house developed perl script
`utilizing XML::Simple library for ease of XML parsing.
`
`Enhancement and Information Retrieval
`While different kind of institutions take part in clinical research,
`they can be one of two types: for- or non-profit. Moreover, non-
`profit institutes are far non homogeneous among themself, they
`can have fairly different goals, primary duties, and follow different
`kind of regulations. So, in relation to a clinical trial the difference
`between a national
`institute and a hospital may be as big as
`between a university and a pharmaceutical company. Therefore,
`in the presented study non-profits have been further subdivided
`into four classes: Research/Educational Institutions (edu) consist-
`ing of universities, colleges, academia, and other alike institutes
`primarily focused on research and education; Hospitals & clinics
`(hos) - organizations with primary focus on providing health care
`service for people with health issues; collaborations including
`associations, networks and other non-government institutions able
`to include in itself different kind of participants (col) and national
`and government organizations (gov). For-profit sponsors were put
`into one class (com), including itself pharmaceutical and other
`commercial companies of health care sector conducted and
`deposited trials’ data. Classification schema is shown in Fig. 1.
`One has to note that the original data had sponsors classification.
`Namely, original classification had four classes: ‘Industry’, ‘NIH’,
`‘Other’, and ‘U.S. Fed.’ We enhanced and slightly altered it in the
`way that ‘NIH’ and ‘U.S. Fed’ classes were joined into one class
`(gov). This class was extended to include other non US national
`(com) class is quite
`and governments sponsored institutions.
`consistent with ‘Industry’ in the original classification. And ‘Other’
`has been distributed primarily into col, hos and edu classes.
`Classification has been performed by in house text-mining
`classificator designed as:
`
`1. define keywords for a given class (like ‘University’,’College’,
`‘Universita`’, etc. for edu class; ‘Hospital’, ‘Clinics’, ‘Hoˆpitaux’,
`‘Klinik’, etc. for hos class; ‘Company’, ‘Inc.’, ‘Corp.’, etc. for
`companies);
`
`2. make dictionaries for each class;
`
`1
`r- non-~rofit 7
`
`I
`edu
`
`I
`gov
`
`I
`col
`
`Results and Outcome Reporting
`
`I
`
`for-profit
`I
`com
`
`sponsor
`
`hos
`
`Figure 1. Schema of the classification.
`doi:10.1371/journal.pone.0037847.g001
`
`like ‘Hospital’ has higher priority than
`3. define priorities,
`‘University’ or ‘College’ in other words ‘University Hospital’
`will be classified as hos rather than edu.
`
`We passed all records through the classificator, with supple-
`mentary classification of records, which did not passed through,
`using agency class information from original classification of the
`sponsors. We used a leading sponsor of
`the trial
`in the
`classification. Then partial manual
`inspection and corrections
`were made.
`So, we got trials distribution into classes as shown in Table 1.
`Overall correspondence between the depository classification
`and one described in this paper is shown in Table 2.
`One has to note, that it is very tricky to make a precise
`classification for over 118,000 trials coming from over 9,000
`different sources, especially taking into account that deposits have
`been made from different countries and therefore, the sponsors are
`pointed in different languages. Besides, as it often happens, the
`texts may have multiple typographic errors. So, eventually our
`classification may have some errors but we do believe that it is not
`significant taking into account the set size. After the automatic
`classification manual refinement of the results has been made.
`
`Statistical Analysis
`Since 1951 medical statisticians use the odds ratio (OR) as
`a measure of effect size, to describe the strength of association
`or non-independence between two binary data characteristics
`[12]. It is used as a descriptive statistic, where results are rather
`qualitative than quantitative or an answer on a question is
`either
`‘yes’ or
`‘no’. That perfectly suites our
`research of
`reporting clinical trials results and outcomes (for each trial one
`either has been reported or not). Additional beneficial feature of
`the odds ratio for our study is that it can be estimated using
`some types of non-random samples. The trials in the depository
`are definitely non-random taking into account that one sponsor
`usually deposits more than one trial.
`So, we performed the odds ratio calculation as
`
`OR~
`
`p11p00
`p10p01
`
`where pyx comes from the joint distribution of two binary random
`variables X and Y
`
`Y~1 Y~0
`p11
`p10
`p01
`p00
`
`X~1
`X~0
`
`in our case:
`X = 1 if results were deposited (outcome reported), 0 otherwise,
`Y = 1 if the trial has been classified as belonging to a given class
`(edu, com, gov, hos), 0 otherwise.
`
`·ffiii.
`-~-·
`
`PLoS ONE | www.plosone.org
`
`2
`
`June 2012 | Volume 7 |
`
`Issue 6 | e37847
`
`MPI EXHIBIT 1063 PAGE 2
`
`MPI EXHIBIT 1063 PAGE 2
`
`Apotex v. Novo - IPR2024-00631
`Petitioner Apotex Exhibit 1063-0002
`
`

`

`Results and Outcome Reporting
`
`Table 1. Classification of trials’ sponsors.
`
`Research/Educational Institutions (edu)
`
`Universities, colleges, academia, research institutes
`
`32295 trials (27.2%)
`
`Companies (com)
`
`National and Government Organizations (gov)
`
`Hospitals & Clinics (hos)
`
`Collaborations (col)
`
`pharmaceutical and other for-profit businesses of
`health care sector
`
`federal, municipal, and other government kind of
`sponsored non-profit organizations
`
`hospitals & clinics sponsoring clinical trials
`
`organizations involving different institutions
`
`38018 trials (32.1%)
`
`19414 trials (16.4%)
`
`17198 trials (14.5%)
`
`10011 trials (8.4%)
`
`Brief description and absolute and relative number of trials deposited into ClinicalTrials.gov 01/01/2012.
`doi:10.1371/journal.pone.0037847.t001
`
`We made conference interval estimate utilizing R software
`package (www.r-project.org), using t-test distribution and 95%
`confidence level.
`
`Results and Discussion
`
`As of 01/01/2012 118,602 clinical trials data deposits were
`made to the depository. They came from 9068 different sources.
`35344 (29.8%) of them are assigned as FDA regulated and 25151
`(21.2%) as Section 801 controlled substances. 70929 (60%) trials
`had a treatment purpose.
`To get a bigger picture, we calculated how number of started
`and completed trials progresses year over year from the lunch of
`the depository. 2011 was the only year through the decade of the
`repository existence when the number of
`trials completed
`exceeded the number of trials started (Fig. 2). In 2009 number
`of trials started came to some kind of saturation. Interestingly, it
`happened after the last recession (12/2007–6/2009) and the
`recession itself did not made a notable impact on clinical trials
`research (US Business Cycle Expansions and Contractions, http://www.
`nber.org/cycles.html).
`Another interesting feature we have observed, came from the
`distribution of trials among phases (1–4) for investigated classes
`(Fig. 3). For companies the number of trials per phase increases to
`phase 3, then it drops, gov and col classes have maximum at
`phase 2, while educational/research institutions have more trials
`for phase 4 than for phase 3. Currently we do not have an
`
`Table 2. Correspondence between classification described in
`this paper and one present in the ClinicalTrials.gov repository.
`
`class (current)
`
`class (original)
`
`number of trials
`
`com
`
`edu
`
`gov
`
`col
`
`hos
`
`unclassified
`
`Industry
`
`Other
`
`Other
`
`Industry
`
`U.S. Fed
`
`NIH
`
`Industry
`
`Other
`
`Other
`
`Industry
`
`Other
`
`Other
`
`doi:10.1371/journal.pone.0037847.t002
`
`37076
`
`942
`
`32118
`
`177
`
`1974
`
`9197
`
`776
`
`7467
`
`9851
`
`160
`
`17198
`
`1666
`
`explanation for this phenomenon but would like to present it for
`community discussion.
`
`The Results and Outcome Reporting
`In order
`to better understand drug safety and efficacy,
`biomedical community has to have clinical trials results not just
`a brief description. They also very important for establishing
`effectiveness measures ‘‘doing the right trials’’ [13]. So, availability
`of clinical results to public became one of the biggest concerns in
`clinical research [1,5]. Besides, recently investigators have found
`that reporting, even among registered trials, was done selectively
`[14]. In response to these concerns, since 2007 FDAAA regulation
`requires to deposit the study results in case ‘‘all of the drugs,
`biologics, or devices used in that study have been approved by the
`FDA for at least one use’’ [4]. At the same time, the use of such
`registries as ClinicalTrials.gov has been demanded by the
`International Committee of Medical Journal Editors (ICMJE).
`As of 2005 the ICMJE has required trial registration before
`participant enrollment as a prerequisite for publication in any of its
`member journals [15].
`Taking into account described above concerns as well as
`multiple efforts taken in recent years to achieve research trans-
`parency,
`spread from the FDA requirements
`to scientific
`publications in peer reviewed journals [16], we investigated how
`many trials have the results uploaded into the result database and
`what factors or regulations were more stimulating than others.
`Summary statistics for the deposits year-by-year, obeying different
`imposed requirements is given in Tables 3,4.
`Overall, only 4927 (4%) of the deposits had reported clinical
`results and 6.82% of completed trials (having completion date as of
`12/31/2011 or earlier). Certainly cumulative effect of taking into
`account all the imposed requirements as:
`
`N a trial has to be completed as assigned in its overall status;
`N FDA and specifically Section 801 regulations;
`N availability of references to a peer reviewed journal (particu-
`N explicit notice of the phase (from 2 to 4);
`N description of the study type as ‘interventional’
`
`larly ICMJE members);
`
`gives better chance for scientific community and general public
`to see the results but it still does not seems to be enough. Overall
`the cumulative requirements returned only about 35% of trials
`with the deposited results with the maximum 55.56% for trials
`completed in 2008. That means 3 years ago from the dates of the
`current analysis, while according to the FDA regulations the
`results have to be reported within 12 months of the completion
`date as it is specified in the filings. Section 801 of FDAAA
`requiring mandatory disclosure of specific clinical trial information
`
`.·ifii. .
`. ~. ~
`
`PLoS ONE | www.plosone.org
`
`3
`
`June 2012 | Volume 7 |
`
`Issue 6 | e37847
`
`MPI EXHIBIT 1063 PAGE 3
`
`MPI EXHIBIT 1063 PAGE 3
`
`Apotex v. Novo - IPR2024-00631
`Petitioner Apotex Exhibit 1063-0003
`
`

`

`16000
`
`• started
`14000 +completed
`
`Results and Outcome Reporting
`
`12000
`
`10000
`
`8000
`
`6000
`
`4000
`
`en ca
`..., -0
`"-Q)
`~ ::,
`
`·;::
`
`C:
`
`0 , - -
`2000
`
`- ,
`2002
`
`2004
`
`2006
`
`2008
`
`2010
`
`year
`
`Figure 2. Number of trials started and completed each year since launching ClinicalTrials.gov repository.
`doi:10.1371/journal.pone.0037847.g002
`
`on ClinicalTrials.gov, containing provisions for proof of compli-
`ance and authorizing penalties for noncompliance [4], alone has
`the highest impact on the results depositing. At the same time we
`note that 4701 trials do not obey any of
`the investigated
`requirements, set for the results deposition (or eventually it is not
`pointed explicitly in the filings) but trials’ conductors/sponsors
`deposited the results anyway.
`The next point of our research was to check whether the trials
`data are different for different responsible institutions (sponsors).
`We look for how deposition of the results varies among different
`classes of sponsoring the trials institutions, taking into account all
`the applied regulations. It appears, government backed organiza-
`tions less than others comply with the policy to deposit results of
`clinical trials. Industrial companies demonstrated the best perfor-
`mance in this aspect. And that would be expected taking into
`account that they have higher fraction of new drug applications
`
`and, therefore, more trials obeying restrictions imposed by the
`FDA regulations. Detailed statistics is present in Table 5.
`Also clinical
`trials design and reporting policy requires
`investigators to disclosure outcomes of the conducted trials. This
`has well grounded reasons, at first, trial participants have the right
`to know abut known (from the previous study) risk by participating
`in trials. Secondly, public availability of this information will
`benefit next generation of clinical researchers and provides more
`rational use of healthcare resources. Eventually, outcome report-
`ing may be biased, moreover, some researchers state that the bias
`occurs regardless of the funding source [17,18], others claim that
`pharmaceutical industry companies are more prone to the bias
`[8,19,20]. Namely,
`the previous research showed that
`trials’
`conductors are more enthusiastic for positive outcome reporting in
`literature [8]. Two aspects make this very likely: firstly, a paper
`with no results to show or describing something that did not went
`
`~
`
`-~ ... .., ....
`0 ...
`
`Q)
`.0
`E
`::;
`i::
`
`12000
`
`10000
`
`8000
`
`6000
`
`4000
`
`2000
`
`0
`
`[I) col
`~ com
`~ gov
`Iii hos
`1111 edu
`
`phase 1
`
`phase 2
`
`phase 3
`
`phase 4
`
`Figure 3. Number of trials assigned to different phases.
`doi:10.1371/journal.pone.0037847.g003
`
`PLoS ONE | www.plosone.org
`
`4
`
`June 2012 | Volume 7 |
`
`Issue 6 | e37847
`
`MPI EXHIBIT 1063 PAGE 4
`
`MPI EXHIBIT 1063 PAGE 4
`
`Apotex v. Novo - IPR2024-00631
`Petitioner Apotex Exhibit 1063-0004
`
`

`

`Results and Outcome Reporting
`
`Table 3. Number of completed trials obeying imposed requirements with results and total, deposited into ClinicalTrials.gov.
`
`completion year
`
`Overall
`
`with results
`
`total
`
`2011
`
`2010
`
`2009
`
`2008
`
`2007
`
`2006
`
`169
`
`894
`
`1270
`
`1328
`
`385
`
`135
`
`81
`
`13945
`
`11732
`
`10588
`
`8869
`
`6515
`
`4714
`
`3632
`
`FDA regulated
`
`with results
`
`total
`
`114
`
`593
`
`899
`
`959
`
`253
`
`99
`
`61
`
`4475
`
`3899
`
`3795
`
`3084
`
`1464
`
`848
`
`657
`
`%
`
`1.21
`
`7.62
`
`11.99
`
`14.97
`
`5.91
`
`2.86
`
`2.23
`
`Section 801
`
`with results
`
`total
`
`93
`
`491
`
`750
`
`814
`
`190
`
`56
`
`32
`
`3134
`
`2649
`
`2643
`
`2244
`
`990
`
`523
`
`408
`
`%
`
`2.55
`
`15.21
`
`23.69
`
`31.1
`
`17.28
`
`11.67
`
`9.28
`
`%
`
`2.97
`
`18.54
`
`28.38
`
`36.27
`
`19.19
`
`10.71
`
`7.84
`
`2005
`
`2004
`
`2003
`
`2002
`
`2001
`
`2000 and before
`
`103
`
`55
`
`40
`
`16
`
`20
`
`2076
`
`1337
`
`840
`
`547
`
`1142
`
`4.96
`
`4.11
`
`4.76
`
`2.93
`
`1.75
`
`90
`
`52
`
`39
`
`16
`
`18
`
`530
`
`389
`
`179
`
`84
`
`149
`
`16.98
`
`13.37
`
`21.79
`
`19.05
`
`12.08
`
`31
`
`16
`
`6
`
`9
`
`17
`
`333
`
`248
`
`94
`
`47
`
`82
`
`total
`
`4496
`
`65937
`
`6.82
`
`3193
`
`19553
`
`16.33
`
`2505
`
`13395
`
`9.31
`
`6.45
`
`6.38
`
`19.15
`
`20.73
`
`18.7
`
`doi:10.1371/journal.pone.0037847.t003
`
`as expected, may be rejected in the review process, secondly, for
`companies there is no point to publish a negative outcome, since
`there is no peer reviewed publications in FDA requirements and
`a publication for them has rather an advertisement purpose. But
`depositing results and describing outcome in the repository gives
`community better chances to see how the trial has been conducted
`in detail and definitely is not so time and efforts consuming as
`writing a full paper. How different investigated classes use this
`opportunity?
`4 of 5 assigned classes have very similar outcome reporting
`statistics close to 3/4 of deposits, while government class provides
`outcome description significantly more seldom than others.
`Educational/research class provides more comprehensive out-
`come description reporting more often not only the primary one
`but the secondary as well. Overall statistics for outcome reporting
`
`is considerably more optimistic than one for the results data being
`submitted into the repository. See Table 6 for details.
`
`Odds Ratio
`Switching from the data already known to an estimate of a future
`efficiency in the results and outcome reporting we utilized the odds
`ratio. Conceptually the odds of a successful event are defined as
`the ratio of the probability of success over the probability of
`failure. In our case OR allows us to estimate reporting efficiency as
`the ratio of cases where the results or outcome have been
`submitted into the depository (success) over cases where this has
`not been done and compare classes of the suggested classification
`to see whether the behavior is different depending on what kind of
`institution is responsible for a conducted trial. Since here we focus
`
`Table 4. Number of completed trials obeying imposed requirements with results and total, deposited into ClinicalTrials.gov.
`
`completion year phases 2–4
`
`with publications
`
`interventional
`
`all requirements together
`
`with results total
`
`%
`
`with results total %
`
`with results
`
`total
`
`%
`
`with results
`
`total %
`
`156
`
`11194
`
`1.39
`
`6
`
`61
`
`9.84
`
`2011
`
`2010
`
`2009
`
`2008
`
`2007
`
`2006
`
`113
`
`638
`
`973
`
`1079
`
`306
`
`94
`
`47
`
`6200
`
`5445
`
`5316
`
`4733
`
`3815
`
`2795
`
`2268
`
`1.82
`
`11.72
`
`18.3
`
`22.8
`
`8.02
`
`3.36
`
`2.07
`
`16
`
`71
`
`96
`
`138
`
`57
`
`27
`
`12
`
`495
`
`602
`
`659
`
`710
`
`637
`
`454
`
`396
`
`3.23
`
`11.79
`
`14.57
`
`19.44
`
`8.95
`
`5.95
`
`3.03
`
`785
`
`1188
`
`1262
`
`373
`
`131
`
`76
`
`9440
`
`8811
`
`7396
`
`5610
`
`4062
`
`3181
`
`8.32
`
`13.48
`
`17.06
`
`6.65
`
`3.23
`
`2.39
`
`24
`
`47
`
`75
`
`26
`
`16
`
`9
`
`84
`
`111
`
`135
`
`85
`
`45
`
`46
`
`28.57
`
`42.34
`
`55.56
`
`30.59
`
`35.56
`
`19.57
`
`2005
`
`2004
`
`2003
`
`2002
`
`2001
`
`46
`
`21
`
`6
`
`8
`
`2000 and before
`
`18
`
`1323
`
`824
`
`434
`
`291
`
`485
`
`3.48
`
`2.55
`
`1.38
`
`2.75
`
`3.71
`
`15
`
`5
`
`3
`
`1
`
`6
`
`255
`
`138
`
`95
`
`67
`
`167
`
`5.88
`
`3.62
`
`3.16
`
`1.49
`
`3.59
`
`103
`
`53
`
`40
`
`16
`
`20
`
`1858
`
`1176
`
`689
`
`429
`
`698
`
`5.54
`
`4.51
`
`5.81
`
`3.73
`
`2.87
`
`7
`
`2
`
`1
`
`1
`
`6
`
`27
`
`17
`
`5
`
`4
`
`14
`
`25.93
`
`11.76
`
`20
`
`25
`
`42.86
`
`total
`
`3349
`
`33929
`
`9.87
`
`447
`
`4675 9.56
`
`4203
`
`54544
`
`7.71
`
`220
`
`634
`
`34.7
`
`doi:10.1371/journal.pone.0037847.t004
`
`.·ifii. .
`. ~. ~
`
`PLoS ONE | www.plosone.org
`
`5
`
`June 2012 | Volume 7 |
`
`Issue 6 | e37847
`
`MPI EXHIBIT 1063 PAGE 5
`
`MPI EXHIBIT 1063 PAGE 5
`
`Apotex v. Novo - IPR2024-00631
`Petitioner Apotex Exhibit 1063-0005
`
`

`

`Results and Outcome Reporting
`
`Table 5. Number of trials with results and overall obeying all
`the imposed restrains as of 01/01/2012 for each assigned
`class.
`
`Table 6. Outcome reporting by different classes.
`
`class
`
`trials with results
`
`total
`
`hos
`
`edu
`
`col
`
`com
`
`gov
`
`21
`
`19
`
`8
`
`168
`
`10
`
`65
`
`194
`
`86
`
`428
`
`156
`
`%
`
`32.31
`
`9.79
`
`9.30
`
`39.25
`
`6.41
`
`number of
`trials with
`at least one
`outcome
`
`7288
`
`29433
`
`7342
`
`13197
`
`24613
`
`%
`
`72.8
`
`77.42
`
`37.82
`
`76.74
`
`76.21
`
`number of
`trials with
`more than
`one outcome %
`
`2397
`
`10375
`
`2182
`
`4763
`
`9758
`
`23.94
`
`27.29
`
`11.24
`
`27.7
`
`30.22
`
`class
`
`col
`
`com
`
`gov
`
`hos
`
`edu
`
`doi:10.1371/journal.pone.0037847.t005
`
`doi:10.1371/journal.pone.0037847.t006
`
`on the interclass difference, we omitted col class because of it
`intersects with others.
`At first, we performed OR calculation for the entire set of trials.
`Here one can see substantial difference between com class and
`others in comparison of
`the results’ presence in the deposits
`(Table 7). Also one has to note that for the government sponsored
`class the OR is almost an order less than for others in outcome
`reporting. While, others are fairly close to each other. In other
`words, generally if a clinical trial has been conducted by a for-
`profit company, we have a higher chance to get the study results
`and outcome reported while the non-profit sector still needs
`substantial improvement especially regarding results of its trials. In
`this aspect our analysis does not support the previous research
`where the researchers concluded that for trials funded by industry,
`results reporting is less likely [21]. edu and hos classes are fairly
`close in both outcome and results reporting.
`Then we look for how the numbers change if we take into
`account all mentioned above requirements, enforcing clinical
`results data deposition. In this case the investigated pool shrank to
`584 trials. Calculating OR for this reduced set one can see some
`changes as for results as for the outcome reporting (Table 8).
`Actually, the most positive impact on outcome reporting the
`imposed restrains made for hospitals and clinics. For companies
`both ratios got less than in no restrain case. Also now one can see
`considerable difference in effectiveness of results and outcome
`reporting for edu and hos classes. The restrains being developed
`for the results deposition, somehow made positive impact on
`outcome reporting for edu and gov classes. So, imposing restrains
`lead to results reporting efficiency decrease for com,
`increase
`substantially for hos, not significantly for edu and even less for
`gov classes.
`
`Interventions
`Another characteristics impacting the reporting efficiency is
`what kind of intervention (if any) had been performed in the trial.
`Overall, top 3 intervention kinds are: drug, procedure and device.
`While all investigated classes have higher interest in new drug
`development. Companies are especially focused on drugs trials
`(73% of interventional trials) and pay surprisingly little attention to
`procedure development. For procedures the biggest contribution
`comes from hospitals (Table 9). One of possible explanation,
`coming from the data analysis, ‘procedure’ trials are often more
`time consuming than other. Namely, average duration of a ‘drug’
`trial deposited into the repository is about 984 days, while for
`a ‘procedure’ trial it is 1302 days and a ‘device’ trial in average
`lasts for 1048 days. We compared efficiency for different classes
`and intervention types. Here efficiency is defined as percentage of
`number of
`trials with results for given conditions (class,
`in-
`tervention type) to the total number of trials for these conditions.
`
`‘Procedure’ trials for all except hos classes have lower efficiency
`in results reporting. For col and com ‘device’ trials have the
`highest efficiency. For hos, gov and edu classes the highest
`efficiency was observed for ‘drug’ trials.
`
`Enrollment
`Patient enrollment is one of the most important and time-
`consuming aspects in clinical
`trials conduct. The depository
`requires to provide information on how many arms has been in the
`study and how many participants has been or anticipated to be
`enrolled in the trial.
`Looking through the decade of the data collection for how many
`participants have been enrolled in a trial and how many arms
`a trial had. Appear, the number of arms pretty much consistent
`and in average is about 261 for all investigated classes. Data
`regarding enrollment, seem more interesting. While Clinical-
`Trials.gov general policy requires
`‘‘Upon study completion,
`(Clinical-
`change Type to Actual and update the enrollment’’
`Trials.gov Protocol Data Element Definitions http://prsinfo.clinicaltrials.
`gov/definitions.html), number of participants enrolled in the trials
`varies very widely from 0 to 99999999. 255 completed trials have
`0 enrollment, 205 (80.4%) of them are interventional studies.
`Neither of them had the results deposited but 66 (25.9%) of them
`reported outcome of the study. 3 completed trials had 99999999
`enrollment. All of
`them were classified as observational and
`neither of
`them had results deposited or outcome reported.
`Considering only completed trials with the results, minimum
`enrollment became 1 and maximum enrollment became 2323608.
`So, the results deposition substantially reduces the enrollment
`range and adds confidence to the data. Providing the results allows
`other researchers to get an idea of how to accomplish higher
`enrollment into a trial. Particularly, in the trial NCT01236053
`with the highest enrollment assigned (2323608 participants) it is
`stated: ‘‘Patients were not recruited for nor enrolled in this study.
`
`Table 7. Odds ratios and confidence intervals for four
`investigated classes.
`
`results
`
`outcome
`
`OR
`
`CI
`
`OR
`
`CI
`
`com
`
`gov
`
`hos
`
`edu
`
`7.7789
`
`0.1320
`
`0.3983
`
`0.3240
`
`(7.7779, 7.7799)
`
`(0.1318, 0.1322)
`
`(0.3980, 0.3986)
`
`(0.3237, 0.3244)
`
`1.8245
`
`0.1995
`
`1.5609
`
`1.6123
`
`(1.8220, 1.8269)
`
`(0.1982, 0.2009)
`
`(1.5591, 1.5627)
`
`(1.6100, 1.6146)
`
`All trials were taken into account.
`doi:10.1371/journal.pone.0037847.t007
`
`.·ifii. .
`. ~. ~
`
`PLoS ONE | www.plosone.org
`
`6
`
`June 2012 | Volume 7 |
`
`Issue 6 | e37847
`
`MPI EXHIBIT 1063 PAGE 6
`
`MPI EXHIBIT 1063 PAGE 6
`
`Apotex v. Novo - IPR2024-00631
`Petitioner Apotex Exhibit 1063-0006
`
`

`

`Table 8. Odds ratios and confidence intervals for four
`investigated classes.
`
`Table 10. Clinical trials enrollment for different classes.
`
`Results and Outcome Reporting
`
`results
`
`outcome
`
`OR
`
`3.66
`
`0.16
`
`1.59
`
`0.43
`
`CI
`
`(3.62, 3.7)
`
`(0.15, 0.17)
`
`(1.58, 1.61)
`
`(0.42, 0.45)
`
`com
`
`gov
`
`hos
`
`edu
`
`OR
`
`1.4
`
`0.45
`
`1.28
`
`1.76
`
`CI
`
`(1.36, 1.44)
`
`(0.43, 0.48)
`
`(1.26, 1.3)
`
`(1.74, 1.79)
`
`For trials possessing the results deposition restrains.
`doi:10.1371/journal.pone.0037847.t008
`
`class
`
`trials
`
`max
`
`average
`
`participants total
`
`col
`
`com
`
`gov
`
`hos
`
`edu
`
`4495
`
`2120000
`
`1308.19
`
`5880327
`
`25873
`
`4300000
`
`1055.88
`
`27318851
`
`9550
`
`7410
`
`99999999
`
`67128927
`
`33298.38
`
`317999544
`
`9493.47
`
`70346597
`
`15111
`
`10050956
`
`1812.4
`
`27387147
`
`with reported outcome
`
`col
`
`com
`
`gov
`
`hos
`
`edu
`
`3295
`
`2120000
`
`1348.35
`
`4442817
`
`20634
`
`4300000
`
`3633
`
`5726
`
`200000
`
`120000
`
`801.89
`
`616.54
`
`287.34
`
`16546258
`
`2239879
`
`1645327
`
`11583
`
`2100000
`
`1148.83
`
`13306867
`
`with clinical results deposited
`
`col
`
`com
`
`105
`
`3482
`
`59510
`
`2323608
`
`866.72
`
`1553.2
`
`91006
`
`5408229
`
`This study is a retrospective observational study. Data from
`medical records or insurance claims databases are anonymized
`and used to develop a patient cohort. All diagnoses and treatments
`are recorded in the course of routine medical practice’’.
`The biggest overall variation was observed for government
`sponsored sector. Hospitals, according to the presented data, have
`an order higher enrollment
`than companies. That would be
`expected taking into account hospitals’ primary mission. At the
`same time, companies enrollment
`twice as big as one of
`educational/research class (Table 10).
`As we mentioned above the dispersion in the participants
`enrollment will be significantly decreased if we will consider only
`trials with reported outco