8309936
`
`Dungan, Christopher Wright
`
`A MODEL OF AN AUDIT JUDGMENT IN THE FORM OF AN EXPERT
`SYSTEM
`
`University of Illinois at Urbana-Champaign
`
`PH.D. 1983
`
`University
`Microfilms
`International 300 N. Zeeb Road, Ann Arbor, MI 48106
`
`Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
`
`eBay Inc. et al. Exhibit 1002 Page 1
`
`

`
`"-
`
`Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
`
`eBay Inc. et al. Exhibit 1002 Page 2
`
`

`
`PLEASE NOTE:
`
`In all cases this material has been filmed in the best possible way from the available copy.
`Problems encountered with this document have been identified here with a check mark_..j_.
`
`1.
`
`2.
`
`3,
`
`4.
`
`5.
`
`6.
`
`7.
`
`8.
`
`9.
`
`10.
`
`11.
`
`12.
`
`13.
`
`14.
`15.
`
`Glossy photographs or pages __
`
`Colored illustrations, paper or print __
`
`Photographs with dark background __
`
`Illustrations are poor copy __
`
`Pages with black marks, not original copy __
`
`Print shows through as there is text on both sides of page __
`
`Indistinct, broken or small print on several pages Y
`
`Print exceeds margin requirements __
`
`Tightly bound copy with print lost in spine __
`
`Computer printout pages with indistinct print __
`
`Page(s)
`author.
`
`Page(s)
`
`lacking when material received, and not available from school or
`
`seem to be missing in numbering only as text follows.
`
`Two pages numbered
`
`__ . Text follows.
`
`Curling and wrinkled pages __
`Other ____________________________________ __
`
`University
`Microfilms
`International
`
`Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
`
`eBay Inc. et al. Exhibit 1002 Page 3
`
`

`
`j
`
`j
`
`j
`
`j
`
`j
`
`j
`
`j
`
`j
`
`j
`
`j
`
`j
`
`j
`
`j
`
`j
`
`j
`
`j
`
`Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
`
`eBay Inc. et al. Exhibit 1002 Page 4
`
`

`
`A MODEL OF AN AUDIT JUDGMENT
`IN THE FORM OF AN EXPERT SYSTEM
`
`BY
`
`CHRISTOPHER WRIGHT DUNGAN
`
`B.Sc., Ohio University, 1957
`M.B.A.~ University o~ AlabaMa, 1962
`J.D.~ University o~ Louisville, 1976
`
`THESIS
`
`SubMitted in partial ~ul~i11Ment o~ the requireMents
`~or the degree o~ Doctor o~ Philosophy in Accountancy
`in the Graduate College o~ the
`University o~ Illinois at Urbana-ChaMpaign, 1993
`
`Urbana, Illinois
`
`Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
`
`eBay Inc. et al. Exhibit 1002 Page 5
`
`

`
`.< _,.~ ~.r~.· .... ,:""_ -c, -''--: ..... ....:_~..... .. _.".
`- ..
`- ._-
`-
`
`UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN
`
`THE GRADUATE COLLEGE
`
`DECEMBER 1982
`
`WE HEREBY RECOMMEND THAT THE THESIS BY
`
`CHRISTOPHER WRIGHT DUNGAN
`
`ENTITLED A MODEL OF AN AUDIT JUDGMENT IN THE FORM OF AN EXPERT
`
`SYSTEM
`
`BE ACCEPTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR
`
`THE DEGREE OF __ D_O_CT_O_R_O_F_P_H_II_.O_S __ O_P_H_Y~_-.-;-___________ _
`
`__ {~1Utk~
`M~~ Di""" of Th"i, R,,,.,,h
`J~r~tC""""'~
`
`Head of Department
`
`Committee on Final Examinationt
`'7.
`
`Chairman
`
`t Required for doctor's degree but not for master's.
`
`0-;; J i
`
`Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
`
`eBay Inc. et al. Exhibit 1002 Page 6
`
`

`
`TABLE OF CONTENTS
`
`iii
`
`CHAPTER ONE: I NTRODUCT I ON ••••••••••••••••••••••••••••••••••••••• 1
`
`• 1
`,"-,UST I rICA T I ON. • • • • .. II •
`A NEW METHODOLOGY: EXPERT SYSTEMS TECHNOLOGY ••••••••••••••••• 5
`SUITABILITY OF AN EXPERT SYSTEM AS A MODEL
`OF AUD I T .JUDGMENT ........................................... 7
`PREVIEW OF THE AUDITOR SYSTEM •••••••••••••••••••••••••••••••• 9
`
`•
`
`•
`
`•
`
`•
`
`•
`
`•
`
`•
`
`•
`
`•
`
`•
`
`•
`
`..
`
`•
`
`•
`
`•
`
`•
`
`•
`
`•
`
`•
`
`•
`
`.. ..
`
`•
`
`.. •
`
`•
`
`.. •
`
`•
`
`•
`
`•
`
`•
`
`•
`
`•
`
`•
`
`•
`
`•
`
`•
`
`..
`
`CHAPTER TWO: THE NATURE OF MODELS OF AUDIT JUDGMENT •••••••••••• 11
`
`TRADITIONAL MODELS AND THEIR USES ••••••••••••••••••••••••••• 13
`Brunswik's Lens Model ....•.••..•.•.•.....••....•••..•...• 13
`ANOVA ......................................................... 14
`PRIOR RESEARCH USING TRADITIONAL MODELS ••••••••••••••••••••• 16
`Studies o~ Source Reliability •••••••••••••••••••••••••••• 16
`Studies Relating to Internal Control and Audit Planning •• 19
`Studies Concerning Materiality ••••••••••••••••••••••••••• Zl
`LIMITATIONS OF TRADITIONAL METHODS •••••••••••••••••••••••••• ZZ
`NONTRADITIONAL MODELS ••••••••••••••••••••••••••••••••••••••• 24
`Verbal Protocol Analysis ••••••••••••••••••••••••••••••••• 24
`Expert SysteMs ...................................................... 26
`
`CHAPTER THREE: HOW AN EXPERT SYSTEM WORKS •••••••••••••••••••••• 35
`
`INTRODUCTION TO AUDITOR AND TO AL/X ••••••••••••••••••••••••• 35
`AUDITOR FROM A USER1S POINT OF VIEW ••••••••••••••••••••••••• 36
`The User Responds to Guestions ••••••••••••••••••••••••••• 36
`The System Reports to the User ••••••••••••••••••••••••••• 38
`AUDITORIS KNOWLEDGE BASE ••••• ~ •••••••••••••••••••••••••••••• 43
`Rules as Knowledge ............................................. 43
`Introduction to Positive Weight (PW) and Negative
`Weight (NW) ................................................. 44
`THE ROLE OF AL/X IN OPERATION OF AN EXPERT SYSTEM ••••••••••• 49
`Revision Involving IF:THEN Spaces •••••••••••••••••••••••. 49
`Revisions Involving AND and NOT Spaces ••••••••••••••.•••• 60
`Se 1 ect i on ......•.......•..........•...•.........•........ 62
`Other Capabilities o~ AL/X ••••••••••••••••••••••••••••••• 66
`
`CHAPTER FOUR: SELECTION OF A JUDGMENT FOR MODELING ••••••••••..• 68
`
`CRITERIA AND MEANS OF SELECTION ••••••••••••••••••••••••••••• 68
`APPLICATION OF THE CRITERIA ••••••••••••••••••••••••••••••.•• 74
`EXERCISE OF JUDGMENT IN THE VALUATION OF RECEIVABLES •••••••• 77
`Overy i ew •••••••••••••••••••••••••••••••••••••••••••••.••• 78
`"Worst-Case" Estimation ••••••••••••••••. , ••••••••••••••••• 78
`Conc:lusion ••••••.••••..•••.•.•.••••.••••.•••••••.•••••..• 82
`
`Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
`
`eBay Inc. et al. Exhibit 1002 Page 7
`
`

`
`TABLE OF CONTENTS (continued)
`
`iv
`
`CHAPTER FIVE: BUILDING THE SYSTEM •••••••••••••••••••••••••••••• 84
`
`I NTRODUCT I ON. • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • 84
`AN HYPOTHESIS IS CHOSEN ••••••••••••••••••••••••••••••••••••• S4
`RULES ARE FORMULA TEO •••••••••••••••••••••••••••••••••••••••• 86
`Cues are Provided by Experts ••••••••••••••••••••••••••••• 86
`Rules are Developed ~roM the Cues •••••••••••••••••••••••• 90
`PARAMETERS ARE SPECIFIED •••••••••••••••••••••••••••••••••••• 93
`Experts are Polled to Learn ths Relative Strength o~
`Each Rule •••••••••••••••.••••••.•• '" •••••••••.•••••••• • 93
`Positive Weights are Developed ~roM COMposite Relative
`Strengths ............................................. 96
`Negative Weights are Assigned by the Researcher ••••••••• 100
`Prior Degrees o~ Belie~ are Assigned to each Rule ••••••• 101
`DESCRIPTION OF THE SYSTEM PRIOR TO REFINEMENT •••••••••••••• 102
`REr I NEMENT ••••••••••••••••••••••••••••••••••••••••••••••••• 103
`Introduction ............................................ 103
`Changes were Made in ParaMeters ••••••••••••••••••••••••• 105
`New Ru 1 es were Added •••••••••••••••••••••••••••••••••••• 107
`Interactions were Added •••••••••••• & • • • • • • • • • • • • • ~ • • • • • • 107
`EXAMPLES OF OPERATION OF THE SYSTEM •••••••••••••••••••••••• lll
`
`First Fact Situation •••••••••••••••••••••••• ...... • ••••• 111
`...... • ••••• 112
`Second Fact Situation •••••••••••••••••••• ......... • ••••• 115
`
`First ConSUltative Session with AUDITOR ••
`
`Second Consultative Session with AUDITOR • • • • • • • • • • • • • • • • 115
`
`CHAPTER SIX: VALIDATION ••• . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . • ••••• 118
`
`INTRODUCTION ••••••••••••••••••••.•••••• _ •••••••••. _ ••••••••• 118
`II OPEN-BOOK II VALIDATION ••••••••••••••••••••••••••••••••••••• 118
`User Interacts with AUDITOR and COMpares with Work
`Papers ••••••••••••••••••••••••••••••••••••••••••••••• 118
`Resu 1 ts ••••••••••••••••••••••••••••••••••••••••••••••••• 120
`'·SLIND" VALIDATION ••••••••••••••••••••••••••••••••••••••••. 123
`Introduction .................................. _ ......... 123
`User Interacts with AUDITOR ••••••••••••• ~ •• ~.N • • • • • • • • • • 1Z4
`Validator COMpares the Experts •••••••••••••••••••••••••• 1Z7
`Results •••••.•.•••.••.••••...••••••••.•.•••.••••••.•..•• 127
`
`CHAPTER SEVEN: TAXONOMY AND ANALYSIS •••••••••••••••••••••••••• 130
`
`TAXONOMY ••••••••••••••••••••••••••••••••••••••••••••••••••• 130
`ANALYSIS OF EVIDENCE SOURCES USED IN THE SYSTEM •••••••••••• 134
`COMparison with Previous Studies •••••••••••••••••••••••• 134
`Rel~tive IMportance o~ the Rules.......
`• •••••••••• 136
`Interactions AMong the. Rules...........
`•••••••.•••.
`.141
`COMparison with Pro~essional Standards.
`••••••••••••
`.145
`FURTHER ANALySIS ••••••••••••••••••••••••••
`.149
`
`Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
`
`eBay Inc. et al. Exhibit 1002 Page 8
`
`

`
`TABLE OF CONTENTS (continued)
`
`v
`
`CHAPTER EIGHT: SUMMARY ••••••••••••••••.••••••••••••••••••.•••• 156
`
`SIGNIFICANCE OF THIS PROwECT ••••••••••••••••••••••••••••••• 156
`OVERVIEW OF THE AUDITOR SYSTEM~=a •••••••••••••••••••••••••• 157
`LI M ITAT IONS ••••••••••••••••••••••••••••••.••••••••••••••••• 158
`SUGGESTIONS FOR IMPROVEMENTS ••••••••••••••••••••••••••••••• 160
`FUTURE USES OF EXPERT SYSTEMS IN AUDITING •••••••••••••••••• 162
`CONCLUSION ••••••••••••••••••••••••••••••••••••••••••••••••• 164
`
`APPENDIX A: FEATURES AND OPERATIONS OF AL/X ••••••••••••••••••• 165
`
`APPENDIX B: FACTORS INVOLVED IN THE EXERCISE OF SELECTED
`AUDIT JUDGMENTS •.•••••••.•••.•.••••••••••..•••..•.•.•••.•.• 176
`
`APPENDIX C: WORK PAPERS FROM VALIDATION PROCEDURES •••••••••••• 183
`
`APPENDIX D: NETWORK DESCRIPTION FILE •••••••••••••••••••••••••• 184
`
`LIST OF REFERENCES •••••••••••••••••••••••••••••••••••••••••••• 190
`
`VI TA •••••••••••••••••••••••••••••••••••••••••••••••••••••••••• 201
`
`Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
`
`eBay Inc. et al. Exhibit 1002 Page 9
`
`

`
`LIST OF" TABLES
`
`vi
`
`TABLE 1: TABLE OF" USER'S CV. PROBABILITIES. AND DEGREES •••••••• 54
`
`TABLE 2: INCREMENTAL WEIGHTS: SPACE INOTPAY" ••••••••••••••••••• 56
`
`TABLE 3: INCREMENTAL WEIGHTS: SPACE
`
`'OACTIVE" ••••••••••••••••••• 58
`
`TABLE 4: RELATIVE STRENGTHS OF" THE RULES •••••••••••••••••••• ~ •• 95
`
`TABLE 5: WEIGHTS PRIOR TO REF"INEMENT ••••••••••••••••••••••••••• 99
`
`TABLE 6: COMPARISON OF" WEIGHTS BEFORE AND AFTER
`REF' I NEMENT • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • 106
`
`TABLE 7: SUMMARY RESULTS OF FIRST VALIDATION •••••••••••••••••• 121
`
`TABLE 8: CONVERSION OF DEGREE AND PROBABILITY INTO VERBAL
`..JUDGMENTS. • • • • • • • • • • • • • • • • • • • • • • • • • • • I'If
`
`•
`
`•
`
`•
`
`•
`
`•
`
`•
`
`•
`
`•
`
`•
`
`•
`
`•
`
`• 1 25
`
`TABLE 9: SUMMARY RESULTS BLIND VALIDATION ••••••••••••••••••••• 128
`
`TABLE 10: RULES AND THEIR USES DURING VALIDATION •••••••••••••• 137
`
`TABLE 11: RULES CLASSIFIED BY PRESUMPTION ••••••••••••••••••••• 147
`
`TABLE 12: COMPARISON OF AUDITOR'S DEGREE OF" BELIEF WITH
`THE CONCLUSION REACHED BY AN AUDITOR •••••••••••••• 152
`
`Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
`
`eBay Inc. et al. Exhibit 1002 Page 10
`
`

`
`LIST OF FIGURES
`
`vii
`
`FIGURE 1: GRAPH OF PROBABILITY AND DEGREE •••••••••••••••••••••• 42
`
`FIGURE 2: OVERVIEW OF AN EXPERT SYSTEM ••••••••••••••••••••••••• 48
`
`FIGURE 3: OVERVIEW OF THE PROCESS OF REVISION •••••••••••••••••• 51
`
`FIGURE 4: GRAPH OF CV AND INCREMENTAL WEIGHT FOR "NOTPAY" •••••• 57
`
`FIGURE 5: GRAPH OF CV AND INCREMENTAL WEIGHT FOR
`
`.. ACTIVE ........ 59
`
`FIGURE 6: OVERVIEW OF THE METHOD FOR SELECTING THE
`NEXT GUEST I ON •••••••••••••••••••••••••••••••••••••••••••••• 64
`
`FIGURE 7: INFERENCE NETWORK FOR A HYPOTHETICAL SYSTEM •••••••••• 6S
`FIGURE a= UNEDITED LIST OF CUES •••••••••••••••••••••••••••••••• 88
`
`FIGURE 9: PRELIMINARY LIST OF RULES •••••••••••••••••••••••••••• 92
`
`F'IGURE 10: COMPLETE LIST OF RULES INCLUDING ORIGINAL AND NEW
`RULES, INTERACTIONS, AND NEGATIVE FORMS OF RULES •••••••••• 109
`
`Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
`
`eBay Inc. et al. Exhibit 1002 Page 11
`
`

`
`CHAPTER ONE
`
`1
`
`INTRODUCTION
`
`lhe objective 0+ this work was to create. in the +orM 0+ a
`
`cOMputerized expert systeM called AUDITOR1 an operacing Model 0+
`
`one 0+ the judgMent processes carried out by pro+essional
`
`auditors. A+ter considering nUMerous alternatives. the particular
`
`judgMent process chosen +or ~he project was an aspect 0+ the
`
`auditors' evaluation 0+ the client's allowance +or bad debts +or
`
`purposes 0+ judging its adequacy. AUDITOR presents expert advice
`
`in the +or~ 0+ an estimate 0+ the probability that a given accounc
`
`balance will prove to be uncollectible.
`
`This document describes the project in a series 0+ eight
`
`chapters. A SUMMary 0+ their contents is as +ollows:
`
`ONE: An introduction to expert systeMS and a brie+ preview
`0+ the results 0+ this study.
`
`TWO: A review 0+ other types 0+ Models 0+ auditors' judgMent.
`
`THREE: An explanation 0+ the Methods by which an expert
`systeM siMulates an expert's judgMent.
`
`rOUR: Descripti~n 0+ the criteria and Means by which
`evaluation 0+ the adequacy 0+ the allowance +or bad debts
`was chosen +or study.
`
`FIVE: Details 0+ the Means by which expert auditors were
`utilized to provide the expertise +or the AUDITOR systeM.
`
`SIX: (lscription 0+ validation 0+ the systeM to test its
`expertise in COMparison with hUMan experts.
`
`SEVEN: Description and analysiS 0+ the systefi',.
`
`EIGHT: SUMMary.
`
`.JUSTIFICATION
`The exercise 0+ pro+essional judgMent is but one variety 0+
`
`Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
`
`eBay Inc. et al. Exhibit 1002 Page 12
`
`

`
`2
`
`hUMan probleM solving~ which is itsel~ an aspect o~ cognitive
`
`behavior. The study o~ these Matters proceeds under the belie~
`
`that the processes ~i~ su~~iciently understood, can be iMporoved
`
`in accuracy, speed, and consistency.
`
`The way in which auditors Make pro~essional judgMents has
`
`aroused considerable interest in recent years. Particul~rly since
`
`1~.?0, stUdies o~ auditoro;' judgMent processes hav~ been undertaken
`
`in volUMe by researchers ~roM the ranks o~ auditors, acadeMic
`
`accountants, and cognitive psychologists. In order to ~aMiliariz~
`
`the reader with the nature o~ this past work and also to provide a
`
`basis ~or contrast with this present study, the Most iMportant o~
`
`this previous work is reviewed later in this papE?r. However~ a
`
`brie~ stateMent o~ SOMe o~ the questions which prior researcher~
`
`have ~oun~ to be o~ interest will be valuable at this point:
`
`Do auditors display consensus in their evaluations o~
`internal control?
`(Ashton, 1974; Reckers and Taylor, 1979).
`
`Do auditors display consensus in their budgeting o~ audit
`tiMe?
`(woyce, 1976; GauMnitz, et al., 1980).
`
`Are auditors Bayesian revisioner<;? (Biddle and woyce, 1979).
`Should they be? (Ward~ 1975; Dacey and Ward, 1980).
`Can they be? (woyce and Biddle~ 1979; Felix 1976; Chesley~
`1977).
`
`Most o~ these stUdies were carried out in a classical
`
`hypothesis-testing Methodology. However, a di~~erent approach hao;
`
`sur~aced More recently which, along wjth the work reportE?d in this
`
`present study. ~ollows a clinical judgMent paradigitl borrowed ~ro.·1'1
`
`cognitive psychology. The basic paradigM involves the
`
`investigation o~ clinical (pro~essional) judgMent as an instanc~
`
`o~ Multi-diMensional judgMent in the ~ace o~ uncertainty. Three
`
`researchable and interesting question are said to be at the heart
`
`Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
`
`eBay Inc. et al. Exhibit 1002 Page 13
`
`

`
`0": such work:
`
`What are the diMensions that in":luence the judgMents?
`
`How much weight does the judge give to each?
`
`How are these e":":ects combined in the judgment process?
`
`3
`
`,",ohnson7 1972
`
`Researchers into these matters are essentially model
`
`buildersr seeking to represent the judgment processes 0": auditors.
`
`Mock and Watkins (1980) 7 Mock and Turner (1981), and Biggs ~nd
`
`Mock (1980) are among those who have attempted to construct models
`
`0": auditor judgment.
`
`'Theil" imMediate precursor seems to be Biggs
`
`(1978) in his study 0": the judgment 0": ":inancial analysts. Later 7
`
`Biggs and Mock (1980) studied ":our auditors to see i": their
`
`judgment processes were siMilar7 a project expanded by Mock and
`
`Turner (1981). Mock and Watkins (1980) explored the e":":ect on
`
`audi torsi decisions 0''': the use 0": two di":":erent types 0":
`
`dOCuMentation 0": evidence.
`
`Although these studies ":all short 0": providing a theoretical
`
`":oundation ":01" auditing7 and may appeal" "so.,:t" wMen compared with
`
`More tightly controlled ex~eriMental research, they are justi~ied
`
`in light o~ the very liMited prescriptive and descriptive
`
`knowledge concerning auditor behavior.
`
`"What do auditors do, and
`
`why do they do it?"
`
`is still a very real question.
`
`Much o~ this recent research has used techniques 0": verbal
`
`protocol analysis (VPA), Most o~ten identi":ied as deriving ":roM
`
`the work 0": Newell and SiMon (1972). VPA is a techn ique o-l!
`
`"process tracing". Researchers attempt to iMply the nature o~ the
`
`underlying cognitive processes ~rom the think-aloud verbal
`
`Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
`
`eBay Inc. et al. Exhibit 1002 Page 14
`
`

`
`4
`
`productions o~ the subjects as they concurrently solve probleMs.
`
`These concurrent verbal productions are b~lieved by the
`
`researchers to be More accurate in re~lecting the subjects'
`
`cognitive processes than would be retrospective accounts o~ the
`
`same events. (There are doubts that pre-decisional processes ar~
`
`consciously accessible ~rom MeMory.) The Models created ~rom these
`
`verbal productions are in the ~orM o~ !ei2Qg~_sQ§~~s~~§' that is,
`
`detailed sequential listings o~ the utterances o~ the subjects,
`
`grouped in what the researchers believe to be Meaning~ul episode~
`
`(Biggs, 1978).
`
`Expert systems are also models o~ experts' judgment
`
`processes, constructed with the aid o~ experts who are challenge~
`
`to justi~y their decisions. The activ~ participation o~ the
`
`experts themselves is an essential ingredient in the building o~
`
`such systems. The research described in this present paper has as
`
`its objective the creation o~ an expert system ~Qg§l o~ certain
`
`judgment processes o~ auditors. The judgment process chosen ~or
`
`study is that o~ the auditors' valuation o~ trad~ accounts
`
`receivable o~ large, commercial clients. Numerous practiCing
`
`auditors were the experts who cooperated in this project. Th3
`
`involvement o~ several experts was ~elt to broaden the base o~ the
`
`system and enhance its credibility.
`
`In the knowledge domain called auditing, development o~
`
`expert systems such as AUDITOR, together with study o~ their
`
`characteristics, can be expected to cast light upon the nature ~~
`
`expert judgment in this ~ield. Additionally, and particularly
`
`since the pre-decisional judgMent processes o~ auditors are so
`
`poorly understood, a ~unctioning expert system can provide by
`
`Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
`
`eBay Inc. et al. Exhibit 1002 Page 15
`
`

`
`de~ault a starting point ~or later developMent o~ a theory o~ the
`
`processes involved. That is, a systeM which MiMics experts'
`
`judgMents is at least aY££iSi~Q~ a~ a Model o~ their behavior
`
`(Newell and SiMon, 1972; Young, 1979).
`
`5
`
`A NEW METHODOLOGY: EXPERT SYSTEMS TECHNOLOGY
`
`It is hardly to be questioned that a variety o~ research
`
`approaches is desirable in the ~orMative years o~ study o~ a
`
`discipline. Einhorn et ala (1979) and Payne et ala (1978) argu~
`
`~or ~ultiMethod approaches in the study o~ hUMan in~orMation
`
`processing, comparisons aMong which will then result either in
`
`enhancing credibility o~ the di~~erent Methods or in highlighting
`
`areas crying out ~or More research.
`
`In an accounting and auditing
`
`context, such variety o~ Method is explicitly endorsed by Ashton
`
`(~orthcoMing), and by Abdel-Khalik and Ajinkya (1979) who echo
`
`Denzin (1978) in advocating "triangulation," that is, Multiplicity
`
`o~ Methods.
`
`The approach utilized in this prese~t study can be called
`
`that o~ synthetic, a posterior, descriptive, Model building (MOCK
`
`and Watkins, 1980). It accepts the ~enet that analytiC Models are
`
`~eeble in the ~ace o~ the cOMplexity o~ the audit task.
`
`This Model-building technique is inductive, beginning with
`
`data supplied by expert auditors and proceeding through the
`
`construction, re~ineMent, and validation o~ an 2e~rs~ing
`
`~2g~1--th~t is, a a~at~m. Such an aCCOMplishMent lays the
`
`groundwork ~or ~uture ~orMation o~ theory--a process re·~erred to
`
`as "direct research" (Mintzberg, 1979), "grounded theory
`
`developMent" (Glaser and Strauss, 1967), and "naturalistic
`
`Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
`
`eBay Inc. et al. Exhibit 1002 Page 16
`
`

`
`6
`
`research" (Oenzin, 1978).
`
`It ~alls within the pre-theoretical
`
`school not only o~ the recent work utilizing verbal protocols but
`
`also that o~ a longer tradition re~lected in Argyris (1954) Lewin
`
`and Schi~~ (1970i~ and Saker (1977).
`
`An expert systeM--an outgrowth o~ the "arti~icial
`
`intelligence ll MoveMent-- is a COMputer prograM which can give
`
`advice on a high level o~ COMpetence in a specialized dOMain o~
`
`hUMan expertise. A test o~ the expertise o~ the systeM is that
`
`hUMan, expert practitioners within the specialized dOMain can be
`
`expected to acknowledge the COMpetence o~ the advice given.
`
`That an expert systeM May be used as a research tool is
`
`recognized in the ~ields o~ both cognitive and computer science.
`
`Newell and Simon (1976), ~or exaMple, asserted that lIeach new
`
`program is an experiment ll
`
`• Additionally, researchers in cognitive
`
`science cite these advantages when advocating computer Modeling as
`
`a tool ~or explaining and predicting COMplex cognitive behavior:
`
`1. Existence o~ a running program ensures the Model has been
`stated precisely.
`
`2. Adequacy o~ the model is attested to by the ~act that it
`can solve problems.
`
`Hunt and Pollock, 1974
`
`As an operational Model o~ an expert's jUdgment, an expert
`
`systeM is susceptible to the saMe type o~ controlled manipulation
`
`by which the ~amiliar technique o~ siMulation can be used to
`
`analyze changes in the re~erent systeffi. However, an expert systeM
`
`has characteristics o~ sY~Q2~!~~ and e~2s~issQ!1!~~ not typically
`
`encountered in simulations. That is, expert systems are not only
`
`believed capable o~ doing so~ •• they are now actively providing
`
`Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
`
`eBay Inc. et al. Exhibit 1002 Page 17
`
`

`
`consultative advice to their clients in real world settings
`
`(Michie, 1980). Thus, an expert syste~ incorporating aspects o~
`
`auditors' judgMent processes not only casts light on the nature o~
`
`those processes, but also deMonstrates its own validity through
`
`its expert per~orMance.
`
`7
`
`SUITABILITY OF AN EXPERT SYSTEM AS A MODEL OF AUDIT ~UDGMENT
`
`It May be possible to describe auditing as a Mainly
`
`cereMonial ~unction whose so-called "standards" are, in reality,
`
`Myths (Boland, 1981). However, in the More conventional and
`
`accepted view, auditing is a process o~ gathering, weighing, and
`COMbining evidence prior to the expression of a judgMent in the
`
`+orM 0+ an opinion on a set 0+ +inancial stateMents. The auditor
`
`assesses the strength and diagnosticity o~ the evidence in
`
`deterMining the nature 0+ the opinion which he subsequently
`
`renders (De+liese et a1., 1975; CarMichael, 1972). Also, auditors
`
`evaluate alternative sources 0+ evidence on a cost-bene+it basis
`
`with due regard to the necessity +or reducing uncertainty prior to
`
`the expression 0+ an opinion in order to reduce the auditor's risk
`
`(Shakun, 1979)
`
`With those descrip~ions 0+ auditing in Mind, consider this
`
`description 0+ a
`
`judgMent process in a discipline which, +or the
`
`MOMent, will be unnaMed:
`
`The decision Maker selects in+orMation sources +roM a range
`0+ alternatives on the basis 0+ SOMe criterion 0+ expected
`value. The +indings +roM the various sources seleted fI',ust
`be COMbined in Making a +inal decision while allowing +or
`any unreliability or uncertainty in the data.
`fox, 1980
`
`Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
`
`eBay Inc. et al. Exhibit 1002 Page 18
`
`

`
`8
`
`While this description' seemingly could +it as well 2ygi~2~§:
`
`decision tasks it was intended by its author (Fox) to describe
`
`clinical diagnosis 0+ diseasep which is the knowledge dOMain in
`
`which builders 0+ e~pert systeMs have taken the MOSt interest.
`
`Addtional descriptions 0+ the characteristics which ~ake the
`
`probleM-solving approaches 0+ a selected discipline aMenable to
`
`Modeling in an expert syst~m include these:
`
`The dOMain is one in which diverse +actors must be
`identi+ied and synthesized to +orm judgmentsp evaluat~
`alternativesp and Make decisions. Years 0+ experience are
`brought to the problem at at hand; experience and subjectiv~
`judgment playa Major role. The domain is not easily
`amenable to precise scienti+ic +or~ulation.
`(Ouda et al.p 1979)
`
`The domain lacks a strong matheMatical str~cturep is
`incorrigibly non-nuMerical, and is too complex +or adequat~
`(Michaelsen, 1982)
`analytical speci+ication.
`
`The knowledge which the expert brings to the task is
`largely heuristic knowledge, experimental, uncertain-(cid:173)
`--mostly good guesses and good practice in lieu 0+
`+acts and rigor--Much 0+ this private to the expert.
`How else explain internships 0+ guild-like apprenticeship
`to a presumed master 0+ the cra+t? What the
`Master really knows is not written in the textbooks.
`(FeigenbauMp 1979)
`
`Speci+ic analogies can be drawn between auditing and
`
`knowledge domains which already have been mapped partially with an
`
`expert system. Discovery sampling to detect the presence 0+ a
`
`critical accounting error or irregularity can be likened to tissu8
`
`culturing to reveal the prasence 0+ pathogenic bacteria. Both
`
`diagnostic tests lead perhaps to +urther eVidence-gathering +or
`
`the prupose 0+ con+irmation 0+ hypotheses regarding the health and
`
`well-being 0+ the client. Further evidence-gathering--in th~ +orM
`
`0+ a total body scan in the one discipline, or 0+ review 0+
`
`Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
`
`eBay Inc. et al. Exhibit 1002 Page 19
`
`

`
`9
`
`subsequent ~ash receipts in the other--~~y lead to the Most
`
`probable diagnosis and to the pre+erred treatMent. For an auditor
`
`the relevant diagnosis Might be "allowance +or bad debts is
`
`understated," and the prescription, "increase the allowance by
`
`$100,000".
`
`In SUM, ~or purposes o~ this paper, auditing can be
`
`viewed as a process 0+ choosing which types 0+ evidence to
`
`evaluate, having due regard +or cost and risk, in order to select
`
`between cOMpeting hypotheses which Make use+ul assertions about
`
`signi+icant aspects 0+ the client's +inancial stateMents. Thesa
`
`choices aMong sources 0+ evidence, the weights to be attached to
`
`the evidence, and the selection aMong cOMpeting hypotheses which
`
`relate to an aspect 0+ the valuation 0+ a client's allowance +or
`
`bad debts will be expressed in the ~orM o~ an expert cOMputerized
`
`systeM called AU01TOR.
`
`PREVIEW OF THE AUDITOR SYSTEM
`
`Four auditors contributed their expertise in the +orM o~ th~
`
`cues to which they attend in order to judge the adequacy 0+ the
`
`client's allowance +or bad debts. For large COMMercial clients
`
`this judgMent is ~acilitated by a process 0+ scrutiny 0+
`
`individually large, delinquent accounts receivable. These cues
`
`were converted into rules. When inserted into the systeM these
`
`rules +orMed an in+erence network 0+ the +orM IF (evidence
`
`required by the rule): THEN (hypothesis). The hypothesis expressed
`
`the judgMent expected +roM the systeM. A+ter a process 0+
`
`re+ineMent 0+ this rule base, during which several auditors
`
`operated and critiqued the systeM, new rules and interactions
`
`Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
`
`eBay Inc. et al. Exhibit 1002 Page 20
`
`

`
`10
`
`between these rules were entered into the systeM to iMprove its
`
`perJorMance. (The building oJ the sy~teM and its reJineMent are
`
`described in Chapter 5.> The result consists oJ twenty-Jive
`
`separate rules. two negative ~orMs oJ these rules7 and thi~teen -
`
`interactions which join rules together with other rules. These
`
`rules are applied by an expert systeM soJtware p~ckage called AL/X
`
`to create an operating systeM called AUOITOR7 which deMonstrates
`
`expertise in a liMited area oJ auditors' judgMent.
`
`(The details
`
`oJ the Junctioning oJ AL/X are described in Chapter 3). Since
`
`AUDITOR. in e~Ject. siMulates the judgMent oJ hUMan expert
`
`auditors 7 it can be said to constitute a mgg~l o~ their judgMent
`
`processes in one narrow area oJ practice. (The reasons Jor
`
`selection o~ this area are detailed in Chapter 47 and More
`
`traditional Models oJ auditors' judgMent are described in Chapter
`
`2). Validation7 carried out to evaluate the systeM's perJorMance.
`
`indicates AUDITOR produces conclusions siMilar to that o~ an
`
`expert hUMan auditor in nineteen o~ twenty-one cases (Chapter 6).
`
`In Chapter 7 the rules incorporated in AUDITOR are analyzed ~or
`
`their reJlection o~ the auditors' belie~ in the reliability o~
`
`alternative sources o~ evidence. Also7 their utilization during
`
`the two validation procedures is studied ~or the light it casts on
`
`audit practice. and the rule base is exaMined to see how well it
`
`re~lects pro~essional standards. Finally. the work is sUMMarizedr
`
`suggestions are Made ~or iMproveMent in AUDITOR. and the possible
`
`uses o~ such systeMs are explored (Chapter 8). As ~or nOW7 let us
`
`look at studies oJ audit judgMent which have been carried out by
`
`More ~aMiliar techniques (Chapter Two).
`
`Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.
`
`eBay Inc. et al. Exhibit 1002 Page 21
`
`

`
`CHAPTER TWO
`
`11
`
`THE NATURE O~ MODELS O~ AUDIT JUDGMENT
`eygit iyggm~nt' as used here, means a judgment Made by an
`experienced auditor in the per~ormance o~ his pro~ession. This
`
`de~inition is intended to re~lect the common usage.
`
`In the literature o~ psychology, a iyggm~nt is a type o~
`
`problem-solving which typically involves delay, uncertainty, and
`
`variability. The judge, in e~~ect, ponders the evidence,
`
`considers various probabilities, and Makes his judgment; other
`
`judges, or the same judge at a later date, may on occasion Mak:~
`
`slightly di~~ering judgments ~roM the same evidence. A judgment
`
`is the assignment o~ an object to a SMall nUMber o~ speci+ied
`
`categories or, in More ~orMal terms a process:
`
`• • • which begins with unordered objects, events,
`or persons, assigns theM to speci~ied response c~t~gories
`so as to MaxiMize the correspondence between the responses
`and the critical dimension o~ the stimulus objects, and thus
`ends with a more order 1 y s ,i tuat i on •
`
`(Johnson, 1972)
`The types o~ judMents made by auditors seem to be analogou~
`
`to clinical judgments or clinical in~erences o~ other +ields in
`
`which an evaluation o~ evidence results in the assignment o~ a
`
`situation or event to a small number 0+ categories on the basis o~
`
`multidimensional cues.
`
`ror example, the auditor assigns trade
`
`accounts receivable (TAR) to the category (1) needs adjustment or
`
`(2) needs no adj:Jstment, on the basis 0+ his evaluation 0+ the
`
`evidence. SiMilarly, he decides (judges) whether or not to r~ly
`
`strongly on internal control as a result o~ his evaluation o~ cue~
`
`related to such matters as division o~ duties, competence of
`
`personnel, etc.
`
`Reproduced with permission of the copyright owner. Further reproduction prohibited