“Basic research is like shooting
`an arrow in the air and. where it lands.
`
`painting a target."
`
`Homer Adkins. I984
`Nature 312, 212.
`
`Food for Thought
`Look Back in Anger - What Clinical Studies
`Tell Us About Preclinical Work
`
`Thomas Hartung
`Johns Hopkins University, Bloomberg School of Public Health. CAAT, Baltimore, USA and University of Konstanz,
`CAAT-Europe, Gennany
`
`Summary
`Misled by animal studies and basic research? Whenever we take a closer look at the outcome ofclinical
`trials in afield such as, most recently, stroke or septic shock, we see how limited the value of our preclinical
`models was. For all indications, 95% of drugs that enter clinical trials do not make it to the market. despite
`all promise of the (animal) models used to develop them. Drug development has started already to decrease
`its reliance on animal models: In Europe, for example, despite intreasing R&D expenditure, animal use
`by pharmaceutical companies dropped by more than 25 % from 2005 to 2008. In vitro studies are likewise
`limited: questionable cell authenticity, over-passoging, mycoplasma infections, an.d lack ofdi}f)"erentiation
`as well as tron-homeostatic and flan-plt)’.'iir)l()giC culture conditions endanger the relevance of these models.
`The standards of statistics and reporting often are poor, further impairing reliabilit_v. Alarming studies from
`industry show miserable reproducibility of landmark studies. This paper discussesfactors contributing to
`the lack ofrepmducibility and relevanc'e ofpre-clinical research. The conclusion: Publish less but of better
`quality and do not rely on theface value of animal studies.
`
`Keywords: preclinical studies, animal studies, in vitro studies, toxicology, safety pharmacology
`
`Introduction
`
`Toxicology (Fig.1).
`
`The prime goal of biornedicine is to understand, treat, and pre-
`vent diseases. Drug development represents a key goal of re—
`Search and the pharmaceutical industry. A devastating attrition
`rate of more than 90% for substances entering clinical trials
`has received increasing attention. Obviously, we often are not
`putting our money on the right horses... Side effects not pre-
`dicted in time from toxicology and safety pharmacology con-
`tribute 20-40% to these failures, indicating limitations of the
`toolbox, which is considerably larger than what is applied to en-
`vironmental chemicals. with the exception of pesticides. Here,
`the question is raised whether quality problems of the disease
`models and basic (especially academic) research also contribute
`to this. In a simplistic view. clinical trials are based on the pillars
`of basic research/pre-clinical drug development, and toxicology
`
`What does this tell Us for areas where we have few or no clini-
`
`The temple of
`biomedica| science
`
`'
`
`,
`
`Clinical
`Studies
`
`'
`
`__ "
`I
`
`.
`
`Basic research
`
`cal trials to correct false conclusions? Toxicology is a prime ex-
`ample, where regulatory decisions for products traded at $ 10
`
`Fi9- 1: Clinical trials are based On the Piiiafs Of basic
`|'9593|‘¢h/ P|'e'Cii"iCfli Ci"-'9 d3V9i°Pm9"is and l°Xi¢°l°9V
`
`ALTEX 30, 3113
`
`Z75
`
`Patent Owner, UCB Pharma GmbH — Exhibit 2056 - 0001
`
`

`
`HARTUNG
`
`Jgj
`
`trillion per year are taken only on the basis of such testing (Bot-
`tini and I-lartung, 2009. 2010). Are we sorting out the wrong
`candidate substances? Aspirin likely would fail the preclinical
`stage today (Hartung, 2009c). Rats and mice predict each other
`for complex endpoints with only 60% accuracy and. predicted
`together, only 43% of clinical toxicities of candidate drugs ob-
`served later (Olson et al., 2000). New approaches that rely on
`molecular pathways of human toxicity currently are emerging
`under the name “Toxicology for the 2 I 5‘ Century".
`Doubt as to animal models also is increasing: A number of in-
`creasingly systematic reviews summarized here more and more
`show the limitations. A National Academy of Sciences panel
`recently analyzed the suitability of animal models to assess the
`human efficacy of countermeasures to bioterrorism:
`It could
`neither identify suitable models nor did it recommend their de-
`velopment; it did, however, call for the establishment of other
`human-relevant tools. In line with this, about $ 200 million have
`been made available by NIH, FDA, and DoD agencies over the
`last year to start developing a human-on-a-chip approach (Har-
`tung and Zurlo, 2012).
`Academic research represents a major stimulus for drug de-
`velopment. Obviously, basic research also is carried out in phar-
`maceutical industry, but quality standards are different and the
`lesser degree of publication makes them less accessible for anal-
`ysis. Obviously, academic research comes in many flavors, and
`when pinpointing some critical notions here,each and every one
`might be unfair and not hold for a given laboratory. Similarly.
`the author and his generations of students are not free from the
`alleged (mis)behaviors. It is the far too frequent, retrospective
`view, imprinted from experiences from quality assurance and
`validation that will be shared here.
`
`Consideration 1:
`
`The crisis of drug development
`
`The situation is clear: Companies spend more and more money
`on drug development, with an average of $ 4 and up to $ 11 bil-
`lion quoted by Forbes for a successful launch to the market‘.
`The number of substances making it to market launch is drop-
`ping, and their success does not necessarily compensate for the
`increased investment. The blockbuster model of drug industry
`seems largely busted.
`The situation was characterized earlier (Hartung and Zurlo,
`2012), and more recent figures do not suggest any turn for the
`better: Failure rates in the clinical phase of development now
`reach 95% (Arrowsmjth. 2012). Analysis by the Centre for
`Medicines Research (CMR) of projects from a group of 16 com-
`panies (representing approximately 60% of global R&D spend-
`ing) in the CMR lntemational Global R&D database reveals that
`the Phase ll success rates for new development projects have
`fallen from 28% (2006-2007) to |8% (2008-2009) (Arrowsmith.
`
`2011a). 51% were due to insufficient efficacy. 29% were due
`to strategic reasons. and 19% were due to clinical or preclini-
`cal safety reasons. The average for the combined success rate
`at Phase III and submission has fallen to --50% in recent years
`(Arrowsmith. 20l lb). Taken together. clinical phases II & Ill
`now eliminate 95% of drug candidates.
`This appeared to correspond to dropping numbers of new
`drugs, as observed between I997 and 2006, as we have occa-
`sionally referenced (Bottini and Hartung. 2009. 20l0), though
`this has been shown to be possibly largely an artifact (Ward et
`al., 2013). We also have to consider that attrition does not end
`with the market launch of drugs: Unexpected side effects lead to
`withdrawals — Wikipedia, who knows it all, lists 47 drugs with-
`drawn from the market since 19903, which represents roughly
`the number of new drug entities entering the market in two years.
`This does not even include the drugs for which indications had
`to be limited because of problems. There also are examples of
`drugs that made it through the trials to the market but, in ret-
`rospect, did not work (see for examples the AP press coverage
`in October 2009 following the US Government Accountability
`Office report analyzing I44 studies, and showing that the FDA
`has never pulled a drug off the market clue to a lack of required
`follow-up about its actual benefits3).
`At the same time, combining the results of 0.32% fatal adverse
`drug reactions (ADR) (Lazarou et al., 1998) (total 6.7% ADR)
`of all hospitalized patients in the US in 1998, with a 23-fold in-
`crease of fatal ADR from 1998-2005 (Moore et al., 2007). leads
`to about 1% of hospitalized patients in the US dying from ADR.
`This suggests that drugs are not very safe, even after all the
`precautionary tests, and corresponds to the relatively frequent
`market withdrawals.
`
`The result of this disastrous situation is that pharma compa-
`nies are eating each other up, often in the hope of acquiring a
`promising drug pipeline, only to find out that this was wishful
`thinking or losing so much time in the merger that the delay of
`development compromises the launch of the pipeline drugs.
`
`Consideration 2:
`
`Clinical research, perverted by conflict
`of interest or role model?
`
`A popular criticism of clinical drug development (as, e.g..
`prominently stressed in Ben Goldacre’s recent book “Bad
`Pharma”, 2012) is the bias from the pressure to get drugs to
`the market. In fact, there is also a publication bias, i.e., the
`more successful a clinical study. the more likely it will be pub-
`lished. It has been shown that studies sponsored by industry are
`seven times more likely to have positive outcomes than those
`that are investigator-driven (Bekelman et al., 2003: Lexchin,
`2003). However, this does not take into account how much
`more development efforts go into industrial preclinical drug
`
`1 http:.f.'www.lorbas.com.'sites.fmat1hewherper.'201 2."02l1 Oithe-IruIy-staggering-cost-of-inventing—new-drugs!
`2 htlpzf/en.wikipedia.org.’wikin’List_ol_withdrawn_drugs (Accessed June 21, 2013)
`3 http:l‘.'usatoday30.usatoday.com.'news.l‘hea|lhl2009-1D-26-fda-drugs_N.htm ?csp=34
`
`276
`
`ALTEX 30, 3113
`
`Patent Owner, UCB Pharma GmbH — Exhibit 2056 - 0002
`
`

`
`=23:
`
`HARTUNG
`
`development compared to what academic researchers have at
`their disposal.
`Actually, clinical studies have extremely high quality stand-
`ards: They are mostly randomized, double-blind, and placebo-
`controlled, as well as usually multi-centric. They require ethical
`review, follow Good Clinical Practice, and are carried out by
`skilled professionals. In recent years. the urge to publish and
`register has increased strongly. Clinical medicine also brought
`about Evidence-based Medicine (EBM). which we have several
`times praised as an objective, transparent, and conscientious
`way to condense information for a given controversial question
`(Hoffmann and l-Iartung. 2006; Hartung. 2009a, 2010). All to-
`gether, these attributes are difficult to match in other fields.
`So we might say that clinical research is pretty good even in
`acknowledging its biases, if at all. of overestimating success. ln
`a simple view, the clinical pipeline, despite enormous financial
`pressures, has very sophisticated tools to promote good science.
`If this is true, we put our money on the wrong horses in clinical
`research to begin with. We have to analyze the weaknesses of
`the preclinical phase to understand why we are not improving
`attrition rates.
`
`Consideration 3:
`
`Bushing onimol toxicology ogoin?
`
`Sure, to some extent. It is one purpose of this series of articles
`to collect arguments for transitioning to new tools. The quoted
`data from Arrowsmith would suggest that toxic side-effects
`contribute to 20% of attrition each in phase II and III. Probably,
`we need to add some percent for side-effects noted in phase I,
`i.e., first in humans, and post-market adverse reactions. Thus an
`overall figure of 30-40% seems realistic.
`However, we first have to distinguish two matters: One is the
`observed effects in humans, which were not sufficiently antici-
`pated. Another is the findings in animal toxicity studies done in
`parallel to the clinical studies. It is a common misunderstand-
`ing among lay audiences that clinical studies commence after
`toxicology has been completed. For reasons of timing. however,
`this is not possible, and the long-lasting studies are done at least
`in parallel to phase 11. Currently, when first acquiring data on
`humans. animal toxicology is incomplete. The two types of tox-
`icological data also are very different: The toxicological effects
`observed in human trials of necessarily short duration and little
`or no follow-up observation are necessarily different from the
`chronic systemic animal studies at higher doses. Fortunately,
`typical side-effects in clinical trials are mild, the most common
`one (about halfof the cases) is drug-induced liver injury (DILI),
`observed as a painless and normally easily reversible increase
`in liver enzymes in blood work though possibly extending to
`the more severe and life-threatening liver failure. The Innova-
`tive Medicine Initiative has tackled this problem in a project
`
`4 http:ttwww.imi.europaeutcontenttmip-dili
`
`based on an initiative we started with industry at ECVAM:
`“Many medicines are hurntful to the liver. and drug-incluced liv-
`er injury (DIL1) now ranks as the leading cause of liver failure
`and transplantation in western countries. However, predicting
`which drugs will prove trtxtc to the liver is extremely dtfficttlt,
`and often problems are not detected until at drug is already on
`the market.”4 The hallmark paper by Olson et al. (2000) gives us
`some idea of this and the retrospective value of animal models
`in identifying such problems: “Liver toxicity was only the fourth
`most frequent HT [human t0xictty}..., yet it led to the second
`highest terttiimttion rate. There was also less concordance be-
`tween atzimal and human toxicity with regard to liver function,
`despite liver toxicity being common in such studies. There was
`no relation between liver H73‘ and therapeutic clc.r.s'.\'."
`A completely different question is: What animal findings ob-
`tained parallel to clinical trials lead to abandoning substances?
`Probably not that many. Cancer studies are notoriously false
`positive (Basketter et al., 2012), even for almost half of the
`tested drugs on the market: furthermore. genotoxicants usu-
`ally have been excluded earlier. Reproductive toxicity will lead
`mainly to a warning against using the substance in pregnancy,
`which is a default for any new drug, as nobody dares to test
`on pregnant women. The acute and topical toxicities have been
`evaluated before being applied to humans. The same holds true
`for safety pharmacology, i.e., the assessment of cardiovascular,
`respiratory, and neurobehavioral effects, as well as excess target
`pharmacology. This leaves us with organ toxicities in chronic
`studies. In fact, if not sorted out by “investigative" toxicology,
`this can impede or delay drug development. “Fortunately,” dif-
`ferent animal species often do not agree as to the organ of toxic-
`ity manifestation, leaving open a lot of room for discussion as to
`translation to humans.
`
`Compared to clinical studies, toxicology has some advan-
`tages and some disadvantages as to quality: First, there are
`internationally harmonized protocols (especially ICI-I and
`OECD) and Good Laboratory Practice to quality-assure their
`execution. However, we use outdated methods, mainly intro-
`duced before 1970. which were systematically rendered pre-
`cautionary/oversensitive. e.g., by using extremely high doses.
`The mechanistic thinking of a modern toxicology comes as
`“mustard after the meal," mainly to argue why the findings are
`not relevant to humans. What is most evident when compar-
`ing approaches: clinical studies have one endpoint. good sta-
`tistics, and hundreds to thousands of treated individuals with
`relevant exposures. Toxicology does just the opposite: Group
`sizes of identical twins (inbred strains) are minimal, and we
`study a large array of endpoints at often “maximum tolerated
`doses" without proper statistics. The only reason is feasibility,
`but these compromises combine in the end to determine the
`relevance of the prediction made. We have made these points
`in more detail earlier (Hartung, 2008a, 200%). For a some-
`what different presentation. please see Table I which combines
`
`ALTEX 30, 30.1‘ I 3
`
`277
`
`Patent Owner, UCB Pharma GmbH — Exhibit 2056 - 0003
`
`

`
`HARTUNG
`
`D
`
`Tab. 1: Differences between and methodological problems of animal and human studies critical to prediction
`of substance effects
`
`Subjects
`0 Small groups of (often inbred, homogenous genetic background) animals vs. large groups of individuals with heterogeneous
`genetic background
`0 Young adult animals vs. all ages in human trials
`- Animals typically only of one gender
`- Disparate animal species and strains, with a variety of metabolic pathways and drug metabolites, leading to variation in
`efficacy and toxicity
`Disease models
`
`0 Artificial diseases, i.e., different models for inducing illness in healthy animals or injury with varying similarity to the human
`condition of sick people
`0 Acute animal models for chronic phenomena
`- Monofactorial disease models vs. multifactorial ones in humans
`
`- Especially in l<nock—out mouse models the adaptive responses in animals are underestimated compensating for the knock-out
`Doses
`
`0 Variations in drug dosing schedules (therapeutic to toxic) and regimens (usually once daily) that are of uncertain relevance
`to the human condition (therapeutic optimum)
`- Pharmaco- and toxicokinetics of substances differ between animals and humans
`
`circumstances
`
`- Uniform, optimal housing and nutrition vs. variable human situations
`- Animals are stressed
`
`0 Never concomitant therapy vs. frequent ones in humans
`
`Diagnostic procedures
`- No vs. intense verbal contact
`
`I Limited vs. extensive physical exam in humans
`- Limited standardized vs. individualized clinical laboratory examination in humans
`- Predetermined timing vs. individualized one in humans
`- Extensive histopathology vs. exceptional one in humans
`0 Length of follow up before determination of disease outcome varies and may not correspond to disease latency in humans
`- Especially in toxicological studies the prevalence of health effects is rarely considered when interpreting data
`
`Study design
`- Variability in the way animals are selected for study, methods of randomization, choice of comparison therapy (none, placebo, vehicle),
`and reporting of loss to follow up
`- Small experimental groups with inadequate power, simplistic statistical analysis that does not account for potential confounding,
`and failure to follow intention to treat principles
`0 Nuances in laboratory technique that may influence results may be neither recognized nor reported. e.g.. methods for blinding
`investigators
`- Selection of a variety of outcome measures, which may be disease surrogates or precursors and which are of uncertain relevance
`to the human clinical condition
`
`- Traditional designs, especially of guideline studies, offering standardization but prohibiting progress
`
`The table combines arguments from (Olson et al., 2000; Pound et al., 2004; and Hartung, 2008a).
`
`arguments from different sources (Pound et al., 2004; Olson
`et al., 2000; Hartung, 200821) showing reasons for differences
`between animal studies and human trials.
`
`Consideration 4:
`Sorting out substances with precautionary
`toxicology before clinical studies?
`The case of genofoxicily assays
`
`Perhaps the even more important question with regard to attri-
`tion is, which substances never make it to clinical trials, that
`
`would have succeeded but whose progress was hindered by
`wrong or precautionary toxicology‘?Again we have to ask, what
`findings lead to the abandonment of a substance. This is more
`complicated than it seems, because it depends on when in the
`development process such findings are obtained and what the
`indication of the drug is. To put it simply, a new chemotherapy
`will not be affected very much by any toxicological finding.
`In early screening, we tend to be generous in excluding sub-
`stances that appear to have liabilities. An interesting case here
`is gcnotoxicity — due to the fear of contributing to cancer and
`the difficulty of identifying human carcinogens at all, this often
`is a brick wall. in addition, the relatively easy and cheap as-
`
`278
`
`ALTEX 30,3113
`
`Patent Owner, UCB Pharma GmbH — Exhibit 2056 - 0004
`
`

`
`—t:>>
`
`HARTUNG
`
`sessment of genotoxicity with a few in vitro tests allows front-
`loading of such tests. Typically, substances will be sorted out if
`found positive. The 2005 publication of Kirkland et al. gave the
`stunning result that while the combination of three genotoxicity
`tests achieves a reasonable sensitivity of 90+% for rat carcino-
`gens, also more than 90% of non-carcinogens are false positive,
`i.e., a miserable specificity. Among the false positives are com-
`mon table salt and sugar (Pottenger et al., 2007). With such a
`high false positive rate, we would eliminate an incredibly large
`part of the chemical universe at this stage.
`This view has been largely adapted, leading to an ECVAM
`workshop (Kirkland et al., 2007) and follow-up work (Lorge et
`al., 2008; Fellows et al., 2008; Pfuhler et al., 2009, 2010; Kirk-
`land, 20l0a,b; Fowler et al., 2012a,b) financed by Cosmetics
`Europe and ECVAM, and finally changes in the International
`Conference on Harmonization (ICH) guidance, though not yet
`at the OECD, which did not go along with the suggested 10-fold
`reduction in test dose for the mammalian assays.
`However, the “false positive” genotoxicity issue (Mouse Lym-
`phoma assay and Chromosomal Aberration assay) has been chal-
`lenged more recently. Goilapucli et al. from Dow presented an
`analysis of the Mouse Lymphoma Assay at SOT 2012. “Since
`the MLA has undergone significant procedural enhancements in
`recent years, a project was undertaken to reevaluate the NTP
`data according to the current standards (IWGT) to assess the
`assay performance capabilities. Data from more than 1900 ex-
`periments representing 342 chemicals were examined against
`acceptance criteria for background mutant frequency, cloning
`efiiciency, positive control values, and appropriate dose selec-
`tion. In this reanalysis, only 17% of the experiments and 40% of
`the “positive” calls met the current acceptance standards. Ap-
`proximately 20% of the test chemicals required >1000 ug /mL
`to Satisfy the criteria for the selection of the top concentration.
`When the concentration is expressed in molarity, approximately
`58, 32, and 10% of the chemicals required 51 mM, >1 to 510
`mM, and >10 mM, respectively, to meet the criteria for the top
`concentration. More than 60% of the chemicals were judged as
`having insufiicient data to classify them as positive, negative, or
`equivocal. Of the 265 chemicalsfrom this list evaluated by Kirk-
`land et al. (2005, Mutat Res., 584, I), there was agreement be-
`tween Kirkland calls and our calls for 32% of the chemicals.”
`Astra-Zeneca (Fellows et al., 2011) published their most re-
`cent assessment of 355 drugs and found 5% unexplained posi-
`tives in the Mouse Lymphoma Assay: “Of the 355 compounds
`tested, only 52 (15%) gave positive results so, even if it is as-
`sumed that all of these are non-carcinogens, the incidence of
`‘false positive’ predictions of carcinogenicity is much lower
`than the 61% apparentfrom analysis of the literature. Further-
`more. only I 9 compounds (5%) were positive by a mechanism
`that could not be associated with the compounds primary phar-
`macological activity or positive responses in other genotoxicity
`assays.”
`Snyder and Green (2001) earlier found less dramatic false
`positive rates for marketed drugs. FDA CDER did a survey on
`
`5 http:lt'ec.europa.eulheaIth.'ph_risklcommitteesI04_socpIdocslsocp_s_08.pdf
`3 http'.I.Iwww.ebtox.com
`
`the most recent «-750 drugs and found that positive mammalian
`genotoxicity results (CA or MLA) did not affect drug approval
`substantially (Dr Rosalie Elesprue, personal communication).
`Only 1% was put on hold for this cause. However, this obvious-
`ly addresses a much later stage of drug development, at which
`most genotoxic substances already have been excluded.
`In contrast, an analysis by Dr Peter Kasper of nearly 600
`pharmaceuticals submitted to the Gennan medicines authority
`(BfArM) between 1995 and 2005, gave 25-36% positive results
`in one or more mammalian cell tests, and yet few were carcino-
`genic (Blakey et al., 2008). It is worth noting that an evaluation
`by the Scientific Committee on Consumer Products (SCCP) of
`genotoxicity/mutagenicity testing of cosmetic ingredients with-
`out animal experiments5 showed that 24 hair dyes tested posi-
`tive in vitro were all then found negative in vivo. This would be
`very much in line with the Kirkland et al. analysis. However, we
`argued earlier (Hartung, 2008b}: “The question might, however,
`be raised whether mutagenicity in human cells should be ruled
`out at all by an animal test. A genotoxic eflect in vitro shows
`that the substance has a property, which could be hazardous.
`Differences in the in vivo test can be either species-specific (rat
`versus human) or due to kinetics (does not reach the tissue at
`sufiiciently high concentrations). These do not necessarily rule
`out a hazard toward humans, especially in chronic situations or
`hypersensitive individuals. This means that the animal experi-
`ment may possibly hide a hazardfor humans.”
`In conclusion, flaws in the current genotoxicity test battery
`are obvious. There is promise of new methods, most obviously
`of the micronucleus test, which was formally validated and led
`to an OECD test guideline. There is some validation for the
`COMET assay (Ersson et al., 2013), which compared 27 sam-
`ples in 14 laboratories using their own protocols; the variance
`observed was mainly between laboratories/protocols, i.e., 79%.
`Thus standardization of the COMET assay is essential, and we
`are desperately awaiting the results of the Japanese validation
`study for the CONIET assay in vivo and in vitro. New assays
`based, e.g., on DNA repair measurement promise better accu-
`racy (e.g., Walmsley, 2008; Moreno-Villanueva et al., 2009,
`201 1). Whether the current data justify eliminating the standard
`in vitro tests and adopting the in vivo comet assay as specified in
`the new ICI-I S2 guidance before validation can be debated. This
`guidance in fact decreases in vitro testing and increases in viva
`testing (in its option 2 as it replaces in vitro mammalian tests
`entirely with two in vivo tests). It is claimed that they can be
`done within ongoing sub-chronic testing, but this still needs to
`be shown because the animal genotoxicity tests require a short
`term (2-3 day) high dose, while the sub-chronic testing neces-
`sitates lower doses.
`
`What to do‘? We need an objective assessment of the evidence
`concerning the reality of “false positives.” This could be a very
`promising topic for an evidence—based toxicology collaboration
`(EBTC") working group. Better still, we should try to find a bet-
`ter way to assess human cancer risk without animal testing. The
`animal tests are not sufficiently informative.
`
`ALTEX 30. 30/13
`
`279
`
`Patent Owner, UCB Pharma GmbH — Exhibit 2056 - 0005
`
`

`
`HARTUNG
`
`lgj
`
`What does this mean in the context of the discussion here?
`
`lt shows that even the most advanced use of in vitro assays to
`guide drug development is not really satisfactory. Though the
`extent of false positives, i.e., innocent substances not likely to
`be developed further to become drugs. is under debate. it ap-
`pears that no definitive tool for such decisions is available. The
`respective animal experiment does not offer a solution to the
`problem, as it appears to lack sensitivity. Thus, the question
`remains whether genotoxicity as currently applied guides our
`drug development well enough.
`
`Consideration 5:
`
`If animals were fortune tellers of drug efiicacy,
`they would not make a lot of money...
`
`A large part of biomedical research relies on animals. John Io-
`annidis recently showed that almost a quarter of the articles in
`PubMed show up with the search term “animal .” even a little
`more than with “patient" (loannidis, 2012). While there is in-
`creasing acknowledgement that animal tests have severe limi-
`tations for toxicity assessments, we do not see the same level
`of awareness for disease models. The hype about genetically
`modified animal models has fueled this naive appreciation of
`the value of animal models.
`
`The author had the privilege to serve on the National Academy
`of Science panel on animal models for countenrieasures to bio-
`terrorism. We have discussed this recently (Hartung and Zurlo,
`2012): The problem for developing and stockpiling drugs for the
`event of biologicallchemical terrorism or warfare is that (fortu-
`nately) there are no patients to test on. So, the question to the
`panel was how to substitute in line with the animal rule of FDA
`with suitable animal models. in a nutshell, our answer is: There
`are no such things as sufficiently predictive animal models to sub-
`stitute for clinical trials (NRC. 2011). Any drug company would
`long to have such models for drug development, as the bulk of
`development costs is incurred in the clinical phase; for counter-
`measures we have the even more difficult situation of unknown
`
`pathophysiology, limitations to experiment in biosafety facilities,
`disease agents potentially designed to resist interventions, and
`mostly peracute diseases to start with. So an important part of the
`committee’s discussions dealt with the attrition (failure) rate of
`drugs entering clinical trials (see above). which does not encour-
`age using animal models to substitute for clinical trials at all.
`In line with this, a recent paper by Seok et al. (2013) showed
`the lack of correspondence of mouse and human responses in
`sepsis, probably the clinical condition closest to biological war-
`fare and terrorism. We discussed this earlier (Leist and Hartung,
`2013) and here only one point shall be repeated, i.e., though not
`necessarily as prominent and extensive. several assessments of
`animal models led to disappointing results, as referenced in the
`comment for stroke research.
`
`In toxicology. we have seen that different laboratory species
`exposed to the same high doses predict each other no better than
`
`7 htlprllwww.nc3rs.org.ukldown|oaddoc.asp?icl=695
`3 http:llwww.wc8.ccac.calfiIest'WC6_DecIaration_ol_Montrea|_FlNAL.pdl
`
`60% -— and there is no reason to assume that any of them predict
`humans better at low doses. We lack such analysis for drug ef-
`ficacy models systematically comparing outcomes in different
`strains or species of laboratory animals. It is unlikely that results
`are much better.
`
`In this series ( Hartung , 20083) We have addressed the shortcom-
`ings of animal tests in general terms. Since then, the weaknesses
`in quality and reporting of animal studies, especially. have been
`demonstrated (MacCallum, 2010; Macleod and van der Worp.
`2010; Kilkenny et al., 2010; van der Worp and Macleod, 2011),
`further undermining their value. Randomization and blinding
`rarely are reported, which can have important implications. as
`it has been shown that animal experiments carried out without
`either are five times more likely to report a positive treatment
`effect (Bebarta et al., 2003). Baker et al. (2012) recently gave an
`illustration of poor reporting on animal experiments, stating that
`in "180 papers on multiple sclerosis listed on PubMed in the past
`6 months, we found that only 40% used appropriate .s'tatistic'.\‘ to
`compare the efifects of gene-knockozit or treatment. Appropriate
`statistics were applied in only 4% of neuroimrnttnological stud-
`ies published in the past two years in Nature Pirlalishing Group
`journals, Science and Cell” (Baker et al., 2012).
`Some more systematic reviews of the predictive value of
`animal models have been little favorable, see Table 2 (Roberts,
`2002: Pound et al., 2004; Hackam and Redelmeier, 2006: Perel
`
`et al., 2007: Hackam, 2007: van der Worp et al., 2010). Hack-
`man and Rede1meier(Hackam and Redelmeier, 2006). for exam-
`ple, found that of 76 highly cited animal studies, 28 (37%: 95%
`confidence interval [CI]. 26%-48%) were replicated in human
`randomized trials. 14 (18%) were contradicted by randomized
`trials,and 34 (45%) remain untested. This is actually not too bad,
`but the bias to highly cited studies (range 639 to 2233) already
`indicates that these studies survived later repetitions and transla-
`tion to humans. There are now even more or less "systematic"
`reviews of the systematic reviews (Pound et al.. 2004: Mignini
`and Khan, 2006: Knight, 2007: Briel et al., 2013). showing that
`there is room for improvement. They definitely do not have the
`standard of evidence-based medicine. In the context of evidence-
`
`based med icine, “A systematic review involves the application of
`scientific strategies, in ways that limit bias, to the assembly, crit-
`ical appraisal, and .s'yn.thesis of all relevrmt sturiies that (address
`a specific clinical question” (Cook et al.. 1997). But the concept
`is maturing. See. for example. the NC3R whitepaper “S_vstem.atic'
`reviews of animal research"7 or the “Montréal Declaration on
`Systematic Reviews of Animal Stuclies."8 The ARRIVE guide-
`line (Kilkenny et al., 2010) and the Gold Standard Publication
`Checklist (GSPC) to improve the quality of animal studies (Hoo-
`ijmans et al., 2010) facilitate the evaluation and standardization
`of publications on animal studies.
`No wonder that in vitro studies are increasingly considered:
`“According to a new market report by Transparency Market
`Research. the global in vitro toxicity tes