`
`
`
`
`
`
`
`A Distributed Simulation System for Team Decisionmaking
`
`
`
`
`
`
`
`
`
`
`
`
`
`Alan A. Song and David L. Kleinman
`
`
`
`
`Department of Electrical and Systems Engineering
`
`
`
`The University of Connecticut
`Storrs, CT 06269-3157
`
`
`
`
`Abstract
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`This paper gives an overview of a unigue dis-
`
`
`
`
`
`
`tributed, real-time simulation system for studying team
`
`
`
`
`decisionmaking and coordination -
`the DDD (Dis-
`
`
`
`
`
`
`
`tributed, Dynamic, Dectsionmaking) paradigm. The
`
`
`
`
`
`DDD paradigm captures the essential elements in real-
`
`
`
`
`
`world decisionmaking problems and integrates them
`
`
`
`
`
`
`
`
`inte a controlled, computer-mediated,
`laboratory set-
`
`
`
`
`
`
`ting. The DDD simulation system is implemented ona
`
`
`
`
`
`network of UNIX workstations with real-time control,
`
`
`
`
`
`on-line data acquisition,
`interactive graphical display,
`and a simulated inter-human communication network,
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`With a highly reconfigurable user interface and a flex-
`
`
`
`
`
`ible scenario generaior, DDD has been used in many
`
`
`
`
`
`
`team decistanmaking experiments with different prab-
`
`
`
`
`
`lem context,
`including military command and control,
`job scheduling, and medical diagnosis.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`1
`
`Introduction
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`In large scale systems that involve humans, ma-
`
`
`
`
`
`
`
`chines, computers, etc., the problem scope and com-
`
`
`
`
`
`
`
`
`plexity often requires that the decisionmaking func-
`
`
`
`
`
`
`
`
`tion be distributed over several humans. Quite of-
`
`
`
`
`
`
`
`
`ten such systems have a team of human decisionmak-
`
`
`
`
`
`
`
`ers who are geographically separated, but who must
`
`
`
`
`
`
`
`
`
`coordinate to share their information, resources and
`
`
`
`
`
`
`
`
`activities in order to attain common goals in what
`
`
`
`
`
`
`
`is generally a dynamic and uncertain task environ-
`
`
`
`
`
`
`
`ment. Although problem contexts can be different
`
`
`
`
`
`
`
`among various systems (e.g., military command and
`
`
`
`
`
`
`
`control, electric power distribution, air traffic control),
`
`
`
`
`
`
`
`
`
`
`the essential elements of decisionmaking remain the
`
`
`
`
`
`
`
`
`
`
`same. In order to study problemssuch as those above
`
`
`
`
`
`
`on a scientific basis, we have developed a unique dis-
`
`
`
`
`
`tributed simulation system, termed DDD(Distributed
`
`
`
`
`
`
`
`
`
`Dynamic Decisionmaking) paradigm, that abstracts
`and simulates the essential elements of real world de-
`
`
`cisionmaking problems.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Unlike existing large scale simulation systems such
`
`
`
`
`
`
`
`
`as SIMNET[18] which stresses high fidelity, special-
`
`
`
`
`
`
`
`
`
`ized, full scale simulation, the DDD paradigm stresses
`
`
`
`
`
`
`
`
`the small team (typically with less than ten team
`
`
`
`
`
`
`
`
`members) with an abstracted, low fidelity task envi-
`
`
`
`
`
`
`
`
`ronment, and emphasizes the basic aspects of interac-
`
`
`
`
`
`
`
`
`
`tion and coordination that are central to “teamness”.
`
`
`
`
`
`
`
`
`
`It simulates the real-world problems in such a manner
`
`
`
`
`
`
`
`
`as to be amenable to study in a controlled laboratory
`
`
`
`
`
`
`
`
`setting. The task environment in DDD is reconfig-
`
`
`
`
`
`
`
`
`urable for different problem context. For example, in
`
`
`
`
`
`
`
`
`our previous research, the system has been configured
`
`
`
`
`
`
`
`as naval command and control, military situation as-
`
`
`
`
`
`
`
`
`
`sessment, medical diagnosis, job scheduling in manu-
`
`
`
`
`
`
`
`facturing systems, etc. The DDD system can be used
`
`
`
`
`as a versatile tool for studying/training small teams
`in military or industry.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`The DDD paradigm is built upon the body of
`
`
`
`
`
`
`knowledge we have accumulated during the last ten
`
`
`
`
`
`
`
`
`
`
`years in performing model-driven, basic experimental
`
`
`
`
`
`
`
`research [1] [2] [3]. As the backbone of our normative-
`
`
`
`
`
`
`
`
`
`descriptive research for team decisionmaking and co-
`
`
`
`
`
`
`ordination, the DDD paradigm has been used for more
`
`
`
`
`
`
`
`
`
`than fifteen team-in-the-loop experiments, and proved
`
`
`
`
`
`
`
`
`to be a very powerful empirical research and training
`
`
`
`
`
`
`
`
`tool (6]-[17]. The DDD paradigm is implemented on
`
`
`
`
`
`
`
`a network of UNIX workstations, with facilities pro-
`
`
`
`
`
`
`viding real-time control and on-line data acquisition,
`
`
`
`
`
`an interactive display/interface media, and a comput-
`
`
`
`
`
`
`
`
`erized inter-human communications and information
`
`
`
`
`
`
`
`
`network within which delay and occasional failure can
`
`
`
`
`
`
`
`
`be manipulated. The simulation system can run on
`
`
`
`
`
`
`
`workstations connected by a local area network, or
`
`
`
`
`
`
`
`on remote workstations connected by Internet.
`Its
`
`
`
`
`
`
`
`
`X11/Motif based graphical interface is highly recon-
`
`
`
`
`
`
`
`
`
`
`figurable (viz., for different problem context, the look
`
`
`
`
`
`
`
`
`and feel can be very different). Currently, it can sup-
`
`
`
`
`
`
`
`port up to seven-person hierarchical or parallel team
`
`
`
`
`
`
`
`
`
`
`(expandable if desired). The DDD simulation system
`
`
`
`
`
`
`has the flexibility to examine a variety of ways in which
`information processing and resource allocation prob-
`
`
`
`
`
`0-8186-6440-1/94 $04.00 © 1994 IEEE
`
`
`
`
`
`129
`
`Petitioner Riot Games,Inc. - Ex. 1034, p. 129
`
`Petitioner Riot Games, Inc. - Ex. 1034, p. 129
`
`

`

`
`
`
`Shared Database
`
`
`
`
`World” Events
`
`
`
`
`DMO(Leader)
`
`
`
`
`
`
`
`
`
`
`Figure 1: The Distributed Dynamic Decisionmaking
`
`Environment
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`lems can be solved by a team of decisionmakers (DMs)
`
`
`
`
`
`under different organizational architectures and infor-
`
`
`mation structures.
`
`
`
`
`
`
`
`
`
`This paper gives an overview of the DDD simula-
`
`
`
`
`
`
`
`
`tion system with an emphasis on newly developed fea-
`
`
`
`
`
`
`
`
`
`
`
`tures that are not included in ourearly report [3]. The
`
`
`
`
`
`
`
`
`
`remainder of this paper is organized into two sections
`
`
`
`
`
`
`
`
`
`and a conclusion. In section 2, the team decisionmak-
`
`
`
`
`
`
`
`
`ing environment is described, and the basic elements
`
`
`
`
`
`
`
`
`of the DDD paradigm including resources, tasks,
`in-
`
`
`
`
`
`
`formation, and responsibility are discussed. Section
`
`
`
`
`
`
`
`
`
`3 reviews the main features of the simulation system
`
`
`
`
`
`
`including system architecture, user interface, scenario
`
`
`
`
`
`generator, experimental variables, built-in distributed
`
`
`
`
`
`
`
`database, and training support tools. Finally, section
`
`
`
`
`4 offers concluding remarks.
`
`
`
`
`
`
`
`
`2 Basic Elements of DDD paradigm
`
`
`
`
`
`
`The DDD paradigm is implemented as a computer-
`
`
`
`
`
`
`driven interactive game among several decisionmakers
`
`
`
`
`
`
`
`
`(DMs) who maybe geographically separated (see Fig-
`
`
`
`
`
`
`
`
`
`
`ure 1). In a real time simulation session, each DM sits
`
`
`
`
`
`
`
`
`
`at a workstation which is capable of displaying the
`
`
`
`
`
`tactical situation and sending/receiving information
`
`
`
`
`
`
`
`to/from the other players. Team decisionmaking is
`
`
`
`
`
`
`formulated as a process of allocating limited resources
`
`
`
`
`
`
`
`
`
`
`
`to a variety of tasks in a dynamic and uncertain en-
`
`
`
`
`
`
`
`
`
`vironment. Thus, the essential elements of teamdeci-
`
`
`
`
`
`
`
`sionmaking are abstracted as:
`i) resources (e.g., ma-
`
`
`
`
`
`
`
`
`chine tools, man powers, sensors, weapons, etc.},
`ii)
`
`
`
`
`
`
`
`
`tasks (jobs to process, e.g., parts, unidentified targets,
`
`
`
`
`
`
`
`enemy airplanes, etc.),
`iii) information (e.g., sensor
`
`
`
`
`
`
`
`
`measures, intelligent sources, reports, etc.), and iv) re-
`
`
`
`
`
`
`
`
`
`sponsibility (i.e., who should do what, at what time).
`
`
`
`
`
`
`
`
`
`To achieve the team goal, co-acting DMs must pro-
`
`
`
`
`
`
`cess distributed information to:
`i) estimate/identify
`
`
`
`
`
`
`
`
`various task attributes, and ii) determine and sched-
`
`
`
`
`
`
`
`
`
`ule their resources to process specific tasks. The DMs
`
`
`
`
`
`
`
`
`are thus required to coordinate their information, ac-
`
`
`
`
`
`
`
`
`
`tions, and resources in a timely and accurate manner.
`
`
`
`
`
`
`
`
`
`Below we describe in more detail the salient elements
`
`
`
`
`of the DDD paradigm.
`
`
`2.1 Resources
`
`
`
`
`
`
`
`
`
`The resources are divided into several classes de-
`
`
`
`
`
`
`
`
`pending on the design parameters of the experiment.
`
`
`
`
`
`
`
`
`
`
`
`All resources of a given class will have the samefea-
`
`
`
`
`
`
`
`
`tures with respect,
`to capacities (i.c., sensor range,
`
`
`
`
`
`
`
`
`weaponstrength, etc.). The only difference amongre-
`
`
`
`
`
`
`
`
`
`
`sources in a given class is the number of sub-resources
`
`
`each carries.
`
`
`
`
`
`
`
`‘The sub-resources are located on board their par-
`
`
`
`
`
`
`
`
`ent resource. A sub-resource does not become an in-
`
`
`
`
`
`
`
`
`dependent resource until it is launched from the par-
`
`
`
`
`
`
`
`
`
`ent resource. The DM can launch one or more sub-
`
`
`
`
`
`
`
`resources that will become available after a certain
`
`
`
`
`
`
`
`launch time delay. The sub-resources can only stay
`
`
`
`
`
`
`
`
`
`away from their parent for a limited time period. An
`
`
`
`
`
`industrial example of resource/sub-resource could be
`
`
`
`
`
`
`the manager (resource) that hires temporary employ-
`
`
`ees (sub-resources).
`
`
`
`
`
`
`
`
`The strength of a resource can be described as a
`
`
`
`
`
`
`
`
`generalized vector which stands for strength in differ-
`
`
`
`
`
`
`
`
`
`ent aspects. Each resource has its effective range. For
`
`
`
`
`
`
`
`
`
`example, a sensor resource can have three ranges: a
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`detection zone, a measurement zone, andaclassifica-
`Resources are basic elements of the system. A re-
`
`
`
`
`
`
`
`
`
`tion zone.
`source can carry other resources called sub-resource,
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`for example, a destroyer can carry somehelicopters,
`Each resource is controlled by the DMs who own
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`and the helicopters can carry some sonobuoys, etc. In
`it(a resource can be owned by multiple DMs), and the
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`this way the resources can be nested down to anyde-
`control of any resource may be transferred during the
`
`
`
`
`
`
`
`
`
`
`
`
`sired level of detail.
`simulation from one DM to another with an attendant
`
`
`
`
`
`
`
`
`
`
`
`130
`
`
`
`Petitioner Riot Games,Inc. - Ex. 1034, p. 130
`
`Petitioner Riot Games, Inc. - Ex. 1034, p. 130
`
`

`

`
`
`
`
`transfer time delay.
`
`
`
`
`2.2 Tasks
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`A team is presented with multiple tasks having dif-
`
`
`
`
`
`
`
`ferent deadlines, processing times, attributes, and pri-
`
`
`
`
`
`orities. During the real time experiment, tasks appear,
`
`
`
`
`
`
`
`
`
`move/maneuver and disappear according to a scenario
`that is under the control of the experiment designer.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`The tasks are also divided into classes. For exam-
`
`
`
`
`
`
`
`
`
`ple, we can have AA, AB, AN which may correspond
`
`
`
`
`
`
`
`
`
`
`to different air targets such as a backfire bomber, a
`
`
`
`
`
`
`
`
`
`
`bird, or a civilian airplane. The hostility of each task
`
`
`
`
`
`class, i.c., whether they are threats or neutrals, can be
`defined by the experiment designer.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Bach task has an attribute vector a with elements
`
`
`
`
`
`that characterize it quantitatively. For example, the
`
`
`
`
`
`
`
`
`attributes can include strength, evasiveness, vulnera-
`
`
`
`
`
`
`
`bility, ete. These attributes are random from task to
`
`
`
`
`
`
`
`
`task, but have a probability distribution (mean and
`
`
`
`
`
`
`
`
`standard deviation) that is unique to task class. The
`
`
`
`
`
`
`
`
`
`resources r required to successfully process a task is a
`
`
`
`
`
`mapping of the attributes of that task and will gener-
`ally depend on task class.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Tasks can be processed in one or more operations,
`
`
`
`
`
`
`
`
`each operation can be assigned to different DMs. Two
`
`
`
`
`
`
`
`
`types of processing are possible: sequential and par-
`
`
`
`
`
`
`
`
`
`allel. The sequential processing requires two or more
`
`
`
`
`
`
`
`
`
`
`DMsto process in sequence, the next operation cannot
`
`
`
`
`
`
`
`
`
`be started before the current oneis finished, for exam-
`
`
`
`
`
`
`
`ple, a part in a manufacturing line may need molding,
`
`
`
`
`
`
`
`
`
`
`painting, and assembling. The parallel processing re-
`
`
`
`
`
`
`
`
`
`quires two or more DMs(or resources) to process at
`
`
`
`
`
`
`
`
`the same time, all required operations must be syn-
`
`
`
`
`
`
`
`
`
`chronized to complete the processing, for example, to
`
`
`
`
`
`
`
`
`
`
`diagnose a disease, all blood test, X-ray, and urine test
`must be finished before thefinal decision can be made.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`The DDDalso includes complex tasks such as ac-
`
`
`
`
`
`
`
`
`
`
`tive tasks and dynamically attributed tasks. An ac-
`
`
`
`
`
`
`
`
`tive task can change its trajectory according to the
`
`
`
`
`
`
`
`current situation and the treatment it received. A
`
`
`
`
`
`
`
`
`
`
`dynamically attributed task can change its attributes
`
`
`
`
`
`
`
`
`
`as a function of time and/or location of the task (for
`
`
`
`
`
`
`
`
`a simple task, the trajectory and true attributes are
`
`
`
`
`
`
`
`set by the scenario generator and remain unchanged
`
`
`
`
`
`
`during the real-time session). These complex tasks
`
`
`
`
`
`
`
`
`provide facilities to investigate team decisionmaking
`
`
`and cocerdination issues in more complex and reactive
`task environment.
`
`2.4
`
`
`
`
`
`
`
`2.3 Responsibility Structure
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`The overlap in task processing responsibilities of
`
`
`
`
`
`
`the team members can be adjusted based on the exper-
`
`
`
`
`
`
`
`
`
`
`
`
`imental condition. Responsibility can be preassigned
`
`
`
`
`
`
`
`
`in a variety of ways, e.g., by task class or by geo-
`
`
`
`
`
`
`
`
`
`graphical location. Under the conditions of no overlap
`
`
`
`
`
`
`
`
`we have a digjoint team requiring no coordination in
`
`
`
`
`
`
`
`
`task processing. As overlap is increased, conflicts in
`
`
`
`
`
`
`
`
`the overlapping areas of joint responsibility will occur
`
`
`
`
`
`
`
`
`
`
`
`
`which will need to be resolved through coordination.
`
`
`
`
`
`
`
`A new feature of the DDD isthe ability to modify on-
`
`
`
`
`
`
`
`line task reaponsibility on a task-by-task basis. Thus,
`
`
`
`
`
`
`
`the responsibility for individual task prosecution can
`be (re)assigned dynamically by the team leader.
`
`
`
`Information and Communication
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Information and communication are two major as-
`
`
`
`
`
`pects in team decisionmaking and coordination. Dif-
`
`
`
`
`
`
`
`ferent structures of information/communication and
`
`
`
`
`
`
`
`their impacts on decisionmaking are important re-
`
`
`
`
`
`
`
`search issues. The DDD paradigm provides a variety
`
`of mechanisms to manipulate information and com-
`munication.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`The information structure of the team can be ma-
`
`
`
`
`
`
`
`nipulated easily via the DDD paradigm. This is imple-
`
`
`
`
`
`
`
`
`
`mented by establishing an information network within
`
`
`
`
`
`
`
`
`the simulation system. Every DMs can be assigned a
`
`
`
`
`
`
`
`
`
`
`level of “tie-in” to the information network depending
`
`
`
`
`
`
`
`
`
`on different task. A high level of “tie-in” means that
`
`
`
`
`
`
`
`
`
`
`
`the DM can get almost all measurements obtained by
`
`
`
`
`
`
`
`
`
`
`other DMs, and a low level of “tie-in” means that the
`
`
`
`
`
`
`
`DM can only rely on his own resources. Therefore, a
`
`
`
`
`
`
`
`centralized, a partially centralized or a decentralized
`
`
`
`
`
`
`
`
`
`information structure can be accomplished by setting
`
`
`
`
`
`
`
`different network “tie-in” levels by task type for differ-
`
`
`
`
`
`
`
`
`ent DMs. Furthermore,different roles in a hierarchical
`
`
`
`
`
`
`
`
`team mayhavedifferent levels of information aggrega-
`
`
`
`
`
`
`
`
`tion. For example, a team leader may have informa-
`
`
`
`
`
`
`
`
`tion on overall situation without details, in contrast,
`
`
`
`
`a subordinate may have information on detailed local
`
`
`
`
`
`
`
`
`situation within his responsibility.
`
`
`
`
`
`
`
`
`
`Communication among DMs is the major way in
`which the team members can share their local infor-
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`mation, and coordinate actions on resource transfer
`
`
`
`
`
`
`
`
`
`and task processing. In DDD paradigm, communica-
`
`
`
`
`
`
`
`tion between different DMs is mainly carried out by
`
`
`
`
`
`
`electronic messages (Verbal exchanges based on multi-
`
`
`
`
`
`
`
`
`person communication/recording system can also be
`
`
`
`
`
`
`
`
`
`incorporated to the DDD paradigm, see (16]). In order
`to simplify the data analysis, all electronic message are
`
`
`
`131
`
`Petitioner Riot Games,Inc. - Ex. 1034, p. 131
`
`Petitioner Riot Games, Inc. - Ex. 1034, p. 131
`
`

`

`
`
`—
`
`Global
`DB
`
`WS/Globai
`process
`8
`Network
`ywcn, RATATAT
`
`Scenario
`Generator
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`preformatted. Our underlying model for the commu-
`
`
`
`
`
`
`
`nication channel contains a variety of features that are
`
`
`
`
`
`
`
`
`important to human decisionmaking. To simulate the
`
`
`
`
`
`
`
`
`communication and data processing delay in real situ-
`
`
`
`
`
`
`
`
`ations, a (random and/or fixed) time delay in message
`
`
`
`
`
`
`
`transfer was introduced. To simulate the limitation on
`
`
`
`
`
`
`
`
`communication capacity (or channel access), the num-
`
`
`
`
`
`
`
`
`ber of communications (N) in a fixed time window (T}
`
`
`
`
`
`
`
`
`
`
`can be specified. Message loss and information scram-
`
`
`
`
`
`
`
`ble due to the network failure can also be simulated
`
`
`
`
`
`
`
`within the DDD. Finally, the communication network
`
`
`
`
`
`
`structure can be defined by a communication matrix,
`i.e., Who can communicate with whom.
`
`
`
`
`
`
`
`
`
`
`
`
`Figure 2: The Architecture of the DDD Paradigm
`
`
`
`
`
`
`
`3 The Features of DDD Paradigm
`
`
`
`
`
`
`Interactive Display
`
`
`
`3.2
`
`
`
`3.1 System Architecture
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`The general architecture of the DDD environmentis
`
`
`
`
`
`
`
`shown in Figure 2. The DDD paradigm runson a net-
`
`
`
`
`
`
`
`
`work of UNIX workstations. In a real-time simulation
`
`
`
`
`
`
`
`
`
`session, eight (or more) processes run concurrently on
`
`
`
`
`
`
`different workstations, with all of the control and com-
`
`
`
`
`
`
`
`
`
`
`munication information traffic carried over network.
`
`
`
`
`
`
`
`
`In the figure, Global is a process that works as the
`
`
`
`
`
`
`
`“control center” for the environment by controlling the
`
`
`
`
`
`
`
`
`clock and timing, synchronizing the other processes,
`
`
`
`
`
`
`
`and sending out various control messages according to
`
`
`
`
`
`
`
`
`the experimental scenario. Each Local Process con-
`
`
`
`
`
`
`
`
`
`trols execution within a workstation (WS) and inter-
`
`
`
`
`
`
`
`
`acts with the User Interface and the Global Process.
`
`
`
`
`
`
`
`
`Each User Interface receives commands from a DM
`
`
`
`
`
`
`
`
`and displays the dynamictactical situation. The Sce-
`
`
`
`
`
`
`
`nario Generator is used for assisting the experimental
`
`
`
`designer in developing various system parameters for
`a given experiment.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`In the DDD environment,
`the global and local
`
`
`
`
`
`
`
`
`
`
`processes are implemented via 4 message-passing ap-
`
`
`
`
`
`
`
`
`
`proach. Each action of the DM is composed ofcertain
`
`
`
`
`
`
`
`
`
`events transferred in the form of messages. For exam-
`
`
`
`
`
`
`
`ple, when a display object receives a “process” com-
`
`
`
`
`
`
`
`
`
`mand issued by DM through a mouse/keyboard event,
`
`
`
`
`
`
`
`it sends a message “PROCESS EVENT”to a local
`
`
`
`
`
`
`
`
`
`
`database object that triggers the method “process”
`
`
`
`
`
`
`
`
`
`which in turn sends a message to the global process
`
`
`
`
`
`
`
`and then other local processes to update the state of
`
`
`
`
`
`
`
`
`all relevant objects. Thus, synchronization is achieved
`via the LIFO queueing and processing of messages.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`The user interface in DDD is very flexible. While
`
`
`
`
`
`
`
`
`all the facilities for decisionmaking are basically the
`
`
`
`
`
`
`
`
`same, the look-and-feel can be different. according to
`
`
`
`
`
`
`
`
`
`
`different problem contexts of the scenario. Some ex-
`
`
`
`
`
`
`
`
`
`
`
`
`
`amples of display at an individual node are shown in
`
`
`
`
`
`
`
`
`
`
`figure 3 and figure 4. Figure 3 is the screen of our ba-
`
`
`
`
`
`
`
`
`
`sic paradigm [2] [3], three types of targets are shown
`
`
`
`
`
`
`
`
`on the screen: air, surface, and submarine;
`the re
`
`
`
`
`
`
`
`
`sources are ships and airplanes, and sub-resources are
`
`
`
`
`
`
`
`helicopters. Figure 4 is a screen form our REST (Re-
`
`
`
`
`
`
`
`ward Structure) experiment [11], where triangles and
`
`
`
`
`
`
`
`
`circles represent targets assigned to different decision-
`
`
`
`
`
`
`
`
`
`makers (combined triangle and circle means two DMs
`
`
`
`
`
`
`
`
`
`
`are responsible for the target).
`In these figures, the
`
`
`
`
`
`
`
`
`screen is divided into four major parts: the main dis-
`
`
`
`
`
`
`
`
`
`play, the status panel, the communication panel and
`
`
`
`
`
`
`
`the prompt panel. The main display displays the ob-
`
`
`
`
`
`
`
`jects that represent resources and targets. Different
`
`
`
`
`
`
`
`
`targets arrive and move according to a experimenter-
`
`
`
`
`
`
`
`
`defined scenario; targets must be processed within a
`
`
`
`
`
`
`
`
`limited time window. All commands related to the
`
`
`
`
`
`
`
`
`objects can be issued by pull-down menus, pop-up
`
`
`
`
`
`
`
`
`windows or double- click associated with the objects.
`
`
`
`
`
`
`
`
`
`The communication panel is composed of an incoming
`
`
`
`
`
`
`
`
`
`window which is used to display the messages from
`
`
`
`
`
`
`
`
`other players, and an outgoing window which is used
`
`
`
`
`
`
`
`
`
`
`
`to display the feedback information when a message
`
`
`
`
`
`
`
`
`is sent out. The status panel is used to display the
`
`
`
`
`
`
`
`
`current time, strength and the dynamics of resource
`
`
`
`
`
`
`
`
`
`transfer/utilization. The prompt panel is used to dis-
`
`
`
`
`
`
`
`
`
`play prompts or error messages. All the display icons,
`
`
`
`
`
`
`menus, windows and messages can be modified or tai-
`lored according to different experimental designs.
`
`
`
`132
`
`Petitioner Riot Games,Inc. - Ex. 1034, p. 132
`
`Petitioner Riot Games, Inc. - Ex. 1034, p. 132
`
`

`

`
`
`Gian
`(Goo Bet)
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`se
`Gleraterrg Le praeee
`Bitererng te igre
`
`
`
`
`
`
`
`
`
`
`
`
`
`Figure 4: The Screen of the REST Experiment
`
`
`
`
`
`3.3 Scenario Generator
`
`
`
`
`
`
`
`
`
`
`A scenario generator has been developed to assist.
`
`
`
`
`
`
`
`
`
`
`the user in setting up the experiment.
`It is used to
`
`
`
`
`
`
`
`
`
`configure the resources, to define the tasks, and to de-
`
`
`
`
`
`
`
`sign the movements of tasks. The scenario generator
`
`
`
`
`
`
`
`is capable of representing a stochastic and imperfectly
`
`
`
`
`
`
`known environment. For example, unexpected or low
`
`
`
`
`
`
`
`probability events can be introduced, and false infor-
`
`
`
`
`
`
`
`
`mation and/or false threats can be employed to per-
`
`
`
`
`
`
`
`
`
`turb the system. The intention is to represent a world
`
`
`
`
`
`
`
`
`
`that is difficult. to predict,
`in which a hostile adver-
`
`
`
`
`
`
`sary introduces uncertainties into one’s estimation of
`
`
`
`
`
`
`
`
`the current state of the system, thus makinginferences
`
`
`
`
`
`
`
`about future states rather unpredictable. All task ar-
`
`
`
`
`
`
`
`rival times, task arrival positions, and task movements
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`133
`
`
`
`
`
`
`
`
`
`
`can be either automatically generated according to
`
`
`
`
`
`
`
`
`
`a certain random function, or a certain pattern, or
`
`
`
`
`
`
`
`specifically designed on a task by task basis.
`
`
`
`
`
`
`
`
`Two interfaces are provided for the scenario gener-
`
`
`
`
`
`
`
`
`ator. The first oneis a flexible experiment description
`
`
`
`
`
`
`
`
`
`language, XS language, which can be used to define
`
`
`
`
`
`
`
`
`
`the “rules” of the DDD game, describe the resource
`
`
`
`
`
`
`
`
`
`
`and the task environment. Three types of items can be
`
`
`
`
`
`
`
`
`
`described via XS language, they are: 1) general items;
`
`
`
`
`
`
`
`
`2) resource information; 3) task information. In gen-
`
`
`
`
`
`
`
`
`
`
`
`eral items one can set the overall features of the exper-
`
`
`
`
`
`
`
`
`
`iment, such as the numbers of DMs, simulation time
`
`
`
`
`
`
`
`and communication delay etc. In resource information
`
`
`
`
`
`
`
`
`one can describe the characteristics of the resources
`
`
`
`
`
`
`
`
`such as maximal velocity, strength, and ranges, etc.
`
`
`
`
`
`
`
`
`
`In task information one can define task attributes, the
`
`
`
`
`
`
`
`
`resource required to prosecute the task, the decision-
`
`
`
`
`
`
`
`
`
`
`
`makers who are able to see or process the task, etc.;
`
`
`
`
`
`
`
`
`
`
`one can also describe the task arrival times, initial po-
`
`
`
`
`
`
`
`
`sitions, velocities, and the maneuvers of the tasks.
`
`
`
`
`
`
`
`
`A graphical active database modeling tool for sce-
`
`
`
`
`
`
`
`
`nario generation has also been developed [5]. This
`
`
`
`
`
`
`
`
`tool has utilized data modeling techniques to correctly
`
`
`
`
`
`
`
`and precisely specify large amount diverse, intricate,
`
`
`
`
`
`
`and interdependent information including the struc-
`
`
`
`
`
`
`
`
`
`
`
`ture of the decision team, the sharing of data, the in-
`
`
`
`
`
`
`
`
`
`teraction and exchange of data among DMs, and the
`
`
`
`
`
`
`
`
`data required by the different DMs. Furthermore, the
`
`
`
`
`
`
`
`structura] information can be graphically specified by
`
`
`
`
`
`
`
`the experimenter, and changes in structural informa-
`
`
`
`
`
`
`
`tion automatically cascades to investigate changes of
`
`
`
`
`
`
`related information throughout the experimental sce-
`
`
`
`
`
`
`
`
`nario, resulting in time saving and consistent design.
`
`
`
`3.4 Experimental Variables
`
`
`
`
`
`
`
`
`The DDD paradigmis powerful enough to manip-
`
`
`
`
`
`
`
`
`ulate a variety of independent. variables(IVs) that al-
`
`
`
`
`
`
`
`
`
`low for the study and evaluation of different command
`
`
`
`
`
`
`
`
`and control configurations. Some of the major IVs
`
`
`
`
`
`
`
`i) internal variables (team structure, responsi-
`are:
`
`
`
`
`
`
`bility structure, information structure, and communi-
`
`
`
`
`
`
`
`cation structure), and ii) external variables (tempo,
`
`
`
`
`
`uncertainty, resource quantity, information quality).
`
`
`
`
`
`
`
`The number and type of dependent variables (DVs)
`
`
`
`
`
`
`
`
`this paradigm can handle is quite flexible. To date,
`
`
`
`
`
`over 100 performance, strategy, coordination, and
`
`
`
`
`
`
`workload measures have been collected and analyzed
`
`
`
`in various experiments.
`
`
`
`
`
`
`
`All essential operations taken by the DMs are
`
`
`
`
`
`
`
`
`
`
`
`recorded in a log file. This file can be used to generate
`
`
`
`
`
`various dependant variables and statistics. Another
`
`
`
`
`
`
`
`
`
`
`
`important function of this file is that it can be used in
`
`
`
`
`
`
`
`
`
`
`
`Petitioner Riot Games,Inc. - Ex. 1034, p. 133
`
`Petitioner Riot Games, Inc. - Ex. 1034, p. 133
`
`

`

`play back modeto automatically replay the game for
`review,
`
`3.5 Distributed Database
`
`The DDD paradigm includes different resources,
`tasks, coordination tools, decision tools, display tools,
`and an on-line data acquisition tool. All of these as-
`pects involve different kinds of data that are too com-
`plex to manage without using proper database tech-
`niques. The requirements for the distributed database
`can beillustrated by the following DDD features:
`
`e Response Time Requirement: Dynamic decision-
`making actions usually have stringent response
`time requirements. The conflict between time
`limit and the system’s response becomes moresig-
`nificant due to the network communication delays
`and the time necessary to update many graphical
`displays and data items.
`
`Several DMs may fre-
`@ Concurrency Control:
`quently read/write some related database re-
`sources concurrently. Thus, concurrency control
`under time pressure is critical to the system.
`
`Low Computational Overhead: The DDD envi-
`ronment is both compute-intensive and data- in-
`tensive. The main body of the environmentis
`devoted to simulation. Thus, the computational
`overhead of a database management mechanism
`must be kept low so that the system can properly
`handle the real-time decisionmaking actions.
`
`the information and transitioning from the conceptu-
`alization to an implementation. The database man-
`agement mechanisms we have used has proved to be
`very effective in supporting our real-time distributed
`dynamic decisionmaking experiments [4] (5).
`
`3.6 Training Support Features
`
`A global control panel was developed so that the
`experimenter can control the pace of an experiment.
`Using the control panel, the experimenter can pause
`or continue the scenario, speed up or slow down the
`game clock. A play back mode with fast play and
`slow motion capability was also built in. These fea-
`tures provide useful tools for instructors. According to
`different training requirement, the feedback informa-
`tion can be designed based on the dependent variables
`that collected and computed on-line. The information
`can be chosen to be displayed on a task by task basis,
`on a per gamebasis, or by time period, with graphical
`and/or numerical form. Because of the large amount
`of dependent variables collected, we can choose the
`optimal feedback information among them according
`to different instructional needs. These features have
`been proved to be very effective for subject training
`in our past experiments.
`
`4 Conclusion
`
`Complex Data Types: The DDD environment
`requires a database that handles complex data
`types, captures the structure of the data, and
`considers the operational semantics of the data
`objects.
`
`The DDD paradigm we have developedis a generic
`paradigm that characterizes team decision processes
`in which limited, shareable resources must be allo-
`cated to identify and process tasks in a dynamic and
`uncertain environment. It is a research/training tool
`amenable to systematic and scientific study while re-
`taining the essential features of real-world tactical de-
`cisionmaking. The DDD paradigm has been used ex-
`The built-in database in DDD has considered all
`tensively in our research. In over fifteen experiments,
`above aspects. To meet the real-time requirements,
`DDDhas been provedto beaflexible, easy to use, yet
`the database in DDD was designed asapartially repli-
`versatile too] for studying distributed decisionmaking
`cated distributed database with a priority based trans-
`in small team configurations. It has enabled us to em-
`pirically study numerousissues and test model-driven
`action management mechanism incorporated. To han-
`dle the shared access requirements, we used a hybrid
`hypotheses in distributed decisionmaking and coordi-
`method that combines a priority based concurrency
`nation. The DDD paradigm is currently used as the
`control policy with a certain distributed locking mech-
`main test-bed for our research that studies the adap-
`anism, so that the system can process transactions
`tation of organizational structure to task environment
`within soft deadlines while guaranteeing that the data
`which is an important step in reaching longer term
`consistency is not violated. To handle complex data
`goals such as the development of a computational the-
`types, we used the object-oriented data model] which
`ory of organization design, and the understanding of
`has a clear advantage over the classical data models,
`how to design organizations ofintelligent agents for
`particularly from the perspective of conceptualizing
`high performance.
`
`Petitioner Riot Games,Inc. - Ex. 1034, p. 134
`
`

`

`
`
`Acknowledgements
`
`
`
`
`
`
`
`
`
`The work reported here was supported in part by
`
`
`
`
`
`
`
`
`the Office of Naval Research under contracts NO0014-
`
`
`
`90-J-1753 and N00014-93-I-0793.
`
`References
`
`
`
`[2]
`
`
`
`
`
`
`
`
`
`
`
`
`{i} D. L. Kleinman, D. Serfaty, and P. B. Luh, “A Re-
`
`
`
`
`
`
`search Paradigm for Multi-Human Decision Mak-
`
`
`
`
`
`
`ing,” Proc. 1984 American Control Conference ,
`
`
`pp. 6-11.
`
`
`
`
`
`
`
`
`A. Song, D. L. Kleinman and D. Serfaty, “A Re-
`
`
`
`
`
`
`search Paradigm for Studying Naval Team Deci-
`
`
`
`
`
`
`sionmaking,” Proc. 7th Annual Workshop on Com-
`
`
`
`
`
`
`
`mand and Control Decision Aiding, April 1990.
`
`
`
`
`
`
`
`[3] D. L. Kleinman and A. Song,
`“A Research
`
`
`
`
`
`Paradigm