......H.
`
`my....
`
`0
`
`
`
`
`._....w........--.o..mYn..a.T.C.
`Wrmu.tmoer0H.cMWD.1So1H229.nee.1_..eod.dH..
`
`..H
`
`
`
`
`
`FEE.....F....IE...,.u..n.I..ri.H..LH....H.-f.........rpr...»SE...
`
`
`
`
`
`
`
`.1n.1n3H5M.M3.5,M.9mL.M..n_W.._........4....B..1..E.m.JM..1..M.M_H.3.H.
`..«..u..nu__.._..,..5J.3._..1
`
`
`
`_
`
`
`
`
`
`
`
`
`
` .......____.M_X_._...-H...L...H._W__._....._kn.I._x(1:!!!4.ArM..4.u_..(J.-.l._.|..1_‘.4.ll...-u.r‘n
`
`,.._R_..___H..T_H.
`A.._Cn,.mlr.EM._F..._._FV.__._
`J__1T“U”2".r.TW1_W..I.u.D.u___..n.uj,TL......i...
`._\..-uiz-.I._._1_....../.__j..n.In|.II|
`
`....M_u_..W...m_n9.u_.._il.7...._-......_¢_mw.....r...Gn)..L._m..__.MN......
` ......__..._."__..._Wm.M.
`
`_..,_AU.._I.._._..._W_...-..I1.1.
`
`IE“-...!.l.|I3:“.«IT...1r..n..m.
`T...__.8_--.1.?L_!i2L_:-._.t__fl".4H_._.
`
`1T____D___1_P_.E.._......___-_W_3LT.q2._.H-__-3_.MIr.«I__mu_.we._.3R3_NW_.91.
`.LTJ_2H.
`IT_.nM.
`
`-slum....A.5.4:
`5...Llt.'T..__..._.,....E5
`
`.1|lJIl.xl|s:I1|.l1Il.|.....~1lulI.l|el!|IIl..I1I|.I+u.41.lX|%1E-1..
`I...I:Ii..I.u.1.InBN.......
`
`n.1:!2-!-i-!-|:1-ax.
`
`rm.9B1:301
`
`emN1
`
`BOEING
`Ex. 1031, p. 126
`
`
`
`

`
`E E
`
`I_._. I3.
`
`If‘ ‘S
`
`I I —.
`
`:11
`
`m
`
`-4I:
`
`
`
`__-__ _ __,‘.-.
`
`._..__
`
`Figure 10. Mode 2 East
`
`.~,—-—
`
`c
`
`«v
`
`,.I .5.
`
`Velocity History _m__--1
`
`_.............._..._._......_..
`
`___ ,_._- .._.__._ ._.._
`
`I
`
`I. I u
`A;
`
`II
`
`I
`
`I...
`
`Figure 11. Mode 2 North
`Acceleration History
`
`Figure 12. Mode 3 Scenario
`
`‘-‘UFITH 5i
`__ -.
`.
`._ _.
`.
`
`_. ___
`
`
`
`I
`
`1
`
`-'i_
`'.’I,
`P-rrf
`I
`
`L__
`1
`
`III
`
`._._-.- ___“;
`
`_ ____________'
`I
`
`I
`
`r¥I
`
`I
`
`I .___...L_?.
`
`~
`
`7-
`I
`I.
`
`-I
`
`.,,_,
`
`I:I'1
`
`-r1'1‘1_—II‘1
`
`-.m -I
`
`1:2.4»__.
`
`L
`H
`E
`”
`E
`I;
`
`E 1| '2'
`
`cums
`
`L E :-
`
`V
`
`can _
`
`.
`
`I31
`
`__
`
`Li
`
`-,
`
`I 1 I I I i I I VI | I
`
`—
`--»
`
`rnarrr
`IHTE'CEPTHR
`
`U
`
`'
`
`,
`..
`
`____
`
`_
`
`_
`
`__
`
`___
`
`'
`
`.4
`
`=
`
`1”!“ ~"’_‘_‘ F
`-
`
`
`n
`
`.,_ _.
`I
`In
`
`_...4
`I
`I‘
`
`Ir '__m.‘V£.bT
`
`
`
`-r-.I_-3|; _pN[.g
`
`BOHNG
`
`Ex.1031,p.127
`
`BOEING
`Ex. 1031, p. 127
`
`

`
`:-Hfi&Jv'?V'5/ "VI),-‘\,\.,‘.
`._ -_l_ _
`n»
`|
`
` T....I51.o...r.|l::.mu\|1....I.l..l.II1!-1.1.171:F
`
`h
`
`.I.MMV;y'1..W.t.C
`.1C1MawW...,a
`ttwrs0oaDYNYEYrrr303o3ottteSeses.1did.1dOHOHWVH
`.3.1_4......5:1
`
`drm
`
`
`
`_.r_.L_Lr.P.:_mn..»_rr.IFor...\_.»v|_L._—.1n.r_(.R_...F:.l..
`
`
`
`
`
`
`
`Ll
`
`_|§!T,---2-gE-_
`
`I.«LH2J.4»H.._4:.1...H_.._4_____
`
`--I.-1+1Iii».-..1.wxFu
`
`
`
`
`
`
`
`
`
`:.______ll,
`I
`
`NEIFZT!-4.
`
`’-!F.‘L
`
`1'-J-3’-"OF?
`
`it__HW,.‘MAwW____ax_P__4---1.1+.1+;_-5fl---_!,.1E__“fly.7__1W1W1MH1fl_1km____7;-ii-:-.--.-......r..M-I---i.--E--4:4.mM_w_W__..um..__um.H
`J!._1..14:_J§._,,.....
`
`BOEING
`
`Ex. 1031, p. 128
`
`!I
`
`I_|
`
`_.7
`
`_.
`
`_
`
`
`
`
`
`..Tu..vE..ETW___¢mnu._.__._..
`
`BOEING
`Ex. 1031, p. 128
`
`
`
`
`

`
`
`
`"I
`
`BOEING
`Ex. 1031, p. 129
`
`BOEING
`Ex. 1031, p. 129
`
`

`
`TSSION 4
`
`CREW SYSTEMS -
`
`HUMAN FACTORS AND
`ARTIFICIAL INTELLIGENCE 1
`
`Chairmen:
`
`Remus N. Bretoi
`NASA Ames Research Center
`
`Major Dean Cole
`Air Force Aerospace Medical
`Research Lab
`
`including displays, controls and
`This session focuses on (he interface between crew and vehicle,
`automation; and the impact of new technologies such as computer graphics and artificial intelligence on
`crew performance.
`
`'
`
`BOEING
`
`Ex. 1031, p. 130
`
`BOEING
`Ex. 1031, p. 130
`
`

`
`84-2616
`
`
`
`THE EVALUATION OF DISPLAY SYMBOLOGY: A CHRONOMETRIC STUDY OF VISUAL SEARCH
`
`Roger Remington
`
`NASA - Ames Research Center
`Aero-Space Human Factors Research Division
`MS 239-3
`Moffett Field, CA 94035
`
`Douglas Williams
`
`Psycho-Linguistic Research Associates
`2055 Sterling Avenue
`Menlo Park. CA 94025
`
`Abstract
`
`Three single-target visual search tasks were used to
`evaluate a set of CRT symbols for a helicopter traffic
`display. The search tasks were representative of the kinds of
`information extraction required in practice, and reaction
`time was used to measure the efficiency with which symbols
`could be located and identified. The results show that fami-
`liar numeric symbols were responded to more quickly than
`qraphic symbols. The addition of modifier symbols such as a
`nearby flashing dot or surrounding square had a greater dis-
`ruptive
`effect
`on
`the
`graphic
`symbols
`than
`the
`alphanumeric characters. The results suggest that a sym-
`bol set is like a list that must be learned. Factors that
`affect the time to respond to items in a list. such as fami-
`liarity and visual discriminability, and the division of list
`items into categories, also affect the time to identify sym-
`bols.
`
`
`Introduction
`
`The selection and evaluation of symbology for cathode
`ray tube (CRT) displays is an important part of display
`design, yet there are no generally agreed upon procedures
`for selecting symbologies or evaluating candidate sets. It is
`not difficult to design tests to compare a small number of
`alternate sets. The focus. however. is often not on the rela-
`tive performance of different sets. but whether a given set is
`adequate. As a practical design issue it is not possible to
`generate sets of alternate symbols and check all possible
`combinations. The approach has been to provide guidelines
`for the use of certain symbol attributes (3,7) or the develop-
`ment of performance—based criteria
`While these guide-
`lines are useful. general principles are often overidden by
`situationally specific factors.
`It seems more fruitful
`to
`search for display principles that pertain to a restricted
`class of displays to be used under similar circumstances.
`We were confronted with these problems in responding
`to a request to certify that a set of symbols selected for use
`on a 3-inch circular CRT helicopter traffic situation display
`were adequate for quick recognition and and low confusabil-
`ity. The symbol set is shown in Table 1. The rows of Table 1
`are the symbols identifying a category of other aircraft; the
`columns are "marker" conditions giving additional status
`information. Each cell entry shows a given symbol in a par-
`ticular marker condition. the first column being no marker.
`The numbers in the margins are the numbers used to refer
`to a given symbol or marker. Each symbol in Table 1 is
`defined in terms of the distribution of "on" cells in a pixel
`matrix. The set includes both numeric and graphic sym-
`bols.
`The numbers were chosen to designate certain
`characteristics of the crafts they represented; the graphic
`characters, however, were an entirely abstract representa-
`tion. The goal of the experiments was to determine whether
`the selected symbols were.
`in all marker configurations,
`sufficiently distinguishable from one another so that each
`
`This paper is declared a work of the U.S.
`Government and therefore is in the public domain.
`
`84
`
`could be identified quickly and accurately.
`This instance is typical of the design process. The sym-
`bols in Table 1 were chosen by committee, and the require-
`ment for a semantic or graphical relationship between the
`symbols and their associated threats refiects some intuition
`common to the committee members that such a relation-
`ship would improve performance by providing a mnemonic
`aid for
`identification. This assumes both that
`symbol
`identification is the crucial time—consuming mental process,
`and
`that
`the
`graphic
`representation will
`facilitate
`identification. While such mnemonics may be useful with
`very large symbols sets.
`these relationships become less
`important when the set size is small and overlearned.
`In
`these cases. visual discriminability is an important deter-
`miner of display access time.
`It might be more important
`to choose symbols that are not visually confusable
`Identification involves memory access; hence. the use’
`of graphic symbols "will facilitate identification only when
`they facilitate memory access. Studies of lexical access
`have shown that the time to identify a letter string as a word
`is inversely related to its frequency of occurrence. High fre-
`quency words require less presentation time and are less
`subject to masking than low frequency items
`The same
`relationship holds for picture naming latency. Subjects are
`faster to name or recognize ob‘ects whose names have a
`higher frequency of occurrence 2.5.6.8). Word frequency is
`often associated with frequency of exposure to an object,
`but there is no direct evidence showing that the frequency
`of a particular graphic symbol determines identification
`time. It would be important
`to know for display design
`whether unfamiliar,
`idiosynchratic symbols required more
`processing time than highly familiar digits or letters. Christ
`and Corso (3) have found that digits are responded to more
`quickly than letters or graphic symbols. but only onsome
`tasks. We sought to clarify the role of symbol familiarity by
`focusing on a particular display type and confining our con-
`clusions to similar displays.
`A visual search paradigm was used to evaluate the
`discriminability of the symbols and assess the effects of
`familiarity. Measuring the time required to find a target
`symbol among a set of distractors has advantages over tech-
`niques that measure discriminability by ratings or by the
`confusability of items in noise.
`In practice, pilots search the
`display to find and identify other aircraft. or con.fi.rm a
`reported siting. and determine the appropriate action.
`Visual search then is an important component of display
`use. The time to react to potential collisions could deter-
`mine the pilot's survival. Since the experimental display
`hardware is identical
`to the one intended for use.
`lb}?
`arrangement of symbols on the display and symbol discriml‘
`nability can be evaluated to determine their contribution t0__
`actual search time.
`
`BOEING
`
`Ex. 1031. p. 131
`
`
`
`BOEING
`Ex. 1031, p. 131
`
`

`
`the screen and remained on for one second. Half a second
`after the target was extinguished a display of four symbols
`appeared and subjects were to press one of two keys as
`quickly as possible to indicate whether the target was
`present (positive trial) or absent (negative trial). Half the
`subjects pressed the left key to indicate that the symbol
`was in the set. and pressed the right key when they failed to
`find the target symbol. For the other half of the subjects
`this key assignment was reversed.
`In actual use the display
`can have up to eight symbols. but the use of four symbols is
`representative of moderately severe threat situations. and
`it is unlikely that the pilot would have the opportunity to
`make reasoned tactical maneuvers with more than two or
`three threats. Subjects were instructed to respond as
`quickly as they could. Since both speed and accuracy were
`of interest. subjects were encouraged to respond as soon as
`they thought they knew whether the target was in the set or
`not. and not wait until they were sure. At the end of every
`54 trials subjects were given a short break. Each 4232 trial
`session lasted approximately 90 minutes.
`Half of the trials were‘ positive. half negative. Each
`symbol served as target an equal number of times in each of
`the four quadrant conditions. For positive trials. each tar-
`get frame was presented equally often. On negative trials
`the non-target frame was chosen quasi-randomly from the
`set of frames not containing the target. with the constraint
`that each of the frames be uses about equally often.
`Sub'ects.
`Sixteen non-pilot volunteers served as sub-
`ject ts were student and staff volunteers as well as
`paid volunteers from the NASA - ARC subject pool who
`received $9.00 each for their participation. All subjects had
`normal or corrected normal vision.
`Results
`
`A preliminary analysis of variance showed no main
`effects of key assignment nor any interactions of key assign-
`ment with other variables. so the results were pooled over
`key assignment. An analysis of variance on mean reation
`times in each condition showed main effects of
`targets
`(F[8.15] = 5.521; p < .001). quadrant (F 3,15] = 10.612; p <
`.001). and positive/negative trial type (F 1.15] = 45.839; p <
`.001). There were also significant interactions of target and
`uadrant (F[24.15] = 3.453; p < .001). target and trial type
`?F[B.15] = 13.298; p < .001), quadrant and trial type (F'[3.15]
`= 5.424; p < .01). and a three-way interaction of target, qua-
`drant. and trial type (F[24.15] = 5.11; p < .001). An analysis
`of variance on the arcsin transforms of
`the proportion
`correct for each subject in each condition (Myers. 1971)
`showed similar effects. expect that there was no significant
`main effect of targets nor an interaction of quadrant and
`condition.
`‘
`-
`The top of Figure 1 shows the mean reaction time. and
`reaction times for positive and negative trial types plotted
`separately for each target. Reaction times on positive trials
`represent the time taken to find the indicated target in the
`set. while reaction times to negative trials reflect the time
`to correctly indicate that the target was not in the set. A
`Duncan's range test showed that the main effect of targets
`resulted from significantly faster reaction times to target 4
`than all other targets except
`target 2. Target 2 was
`significantly faster than all remaining targets except targets
`6 and 7. No other difierences were significant. The symbols
`arrange themselves into three groups with symbol 4 being
`the fastest. symbols 2, 6. and 7 being about 34 ms slower.
`and the remaining symbols being an additional 24 ms.
`slower.
`
`The bottom of Figure 1 shows proportion correct for
`each target on both positive and negative trials. There were
`more misses than false alarms. The average percent correct
`for positive trials was 86.6, for negative trials 93.7. There is
`no indication of a speed-accuracy trade-off. The significant
`interaction of trial type and targets refiects the fact that
`there were no differences in proportion correct for different
`targets on negative trials. but a Duncan's range test shows
`that
`for positive trials targets
`1 and 2 were missed
`significantly more often than all others. while target 3 was
`missed less often.
`
`85
`
`BOEING
`
`Ex. 1031. p. 132
`
`Experiment 1
`
`:I
`
`.
`
`
`
`_
`_
`
`it
`.
`
`
`
`: I.
`od
`-'7APParatus. The subject sat in a sound attenuated booth
`.
`ah‘ adjustable helicopter seat and viewed a 5 inch Sony
`-.5000 color monitor through the double- paned glass win-
`'’Q:. *.of the booth at a distance of 25 inches. The rest of the
`was shrouded with black material
`to eliminate
`(actions, and all but a 3 inch diameter portion of the CRT
`.'. gen was masked. On a shelf in front of the subject was an
`' I; ma», board covered in black cloth on which two micro-
`tches were mounted, one labeled "yes" the other "no".
`..
`' An Appie 11+ computer was used to control the experi-
`"
`L,-randomize the presentation of stimuli. and record
`_'. fionses and response times. The stimuli were presented
`‘ ."a ‘SOL-20 computer using a Cromemco TV dazzler color
`-"H (j set programmed to give a 128 X 128 pixel display of
`.
`;
`' colored symbols on a black background. The Apple
`'5‘; SOL communicated over an RS232 serial line. On each
`1'the Apple sent an ASCII string to the SOL indicating
`' "
`‘h items to display, and a signal that blanked the display
`Ur.
`“é appropriate times. Since the Applebegan timing
`'~..'a it sent
`the first symbol.
`the reaction times are
`wu
`.
`‘red by a constant amount that refiects the time to
`,'|. hsmjt the information and the time for the SOL software
`'-
`isplay the indicated items. This delay was approxi-
`' X.
`ely 700 ms. and was a constant aflecting reaction times
`1;: conditions equally.
`_
`timuli. Each symbol and marker from Table 1 was
`._r'
`'-r-i—as*a_—character in a 16 X 16 pixel matrix using a
`- e'mco TV dazzler color board and associated software.
`ere 36 pixels per i.nch. making the 16 X 16 matrix a
`r'é of 0.44 inches per side. At a viewing distance of 25
`5 each matrix subtended approximately 1 degree of
`uai angle horizontally and vertically. Not all stimuli were
`n equal horizontal or vertical extent. and each used
`erent numbers of cells in the matrix.
`In the first experi-
`'.Eient only the first column of symbols, those without mark-
`5, were used. Thus not all of the matrix is needed to
`lcpresent each symbol.
`_
`- The stimuli were presented in frames. Each frame-con-
`J’
`sled of four symbols with one of the symbols designated as
`"' _ target symbol for that frame and the three distractor
`imuli chosen quasi-randomly with the constraint that each
`I
`* + .
`' rget appear at least once with every other symbol and
`--_
`I‘
`I it all symbols occur about equally often. The experimen-
`L‘
`'
`.3: 1 displays were modeled closely on the actual display.
`l--’ ‘Ln_bols could occur in one of two concentric circular rings.
`"-‘ e inner ring was on the circumference of a circle with a
`ter at the center of the display and a radius of approxi-
`te_ly 21 pixels (1.34 deg visual angle). The radius of the
`cle for the outer ring was approximately 120 pixels (3.8
`-_yis_ual angle). Within a frame symbols could occur in
`- inner and outer rings, and at any of the 12 clock posi-
`TI.
`_in each ring. The separation of symbols within a par-
`_54 __r[a_r frame was manipulated by restricting the location of
`x-. fnbqls in that frame to a specified number of quadrants.
`I §'_iI.r,the highest density frames required all stimuli to fall
`one randomly selected quadrant.
`In the least dense
`_ es "each symbol occupied a separate quadrant. This
`I figure does not insure that the distance. or dispersion
`_ bols in the 4-quadrant condition.
`for example.
`is
`Egreater than in the 3—quad.rant condition. It was. how-
`-QSY to implement, produced the required separations
`_
`iwgrage. and the values were adjusted as the frames
`.P_F_0 uced to avoid any obvious discrepancies such as
`*‘-a—dJa°e“t figures on a quadrant boundary. There were
`I_ griglnevs in all. Each symbol appeared equally often as
`I
`-_.
`. W1 h 12 frames for each target. 3 frames for each
`. e ‘in each quadrant.
`EEHEFL3_ Experiment 1 was intended to
`'8-_efi_eC1tSCrirIunability of the symbols without markers
`-33 trials Eff symbol separation on search times. There
`f _1_O8 S in each experimental session. divided into 4
`£0‘ me trials each. Each trial began with the presen-
`arget Symbol which appeared in the center of
`
`.
`
`_-1
`
`"'
`
`rm.
`ire-
`the
`ion
`0ri~
`‘n-1°
`bol
`355-
`ale.
`nth
`es?
`1“
`.61“
`t
`an
`
`use
`'ien'
`ess
`frd
`
`are
`
`b)’
`
`'.l'l6
`of
`get
`:h-
`
`on.
`lay
`er-
`
`ni-
`
`BOEING
`Ex. 1031, p. 132
`
`

`
`Discussion
`
`revealed potential problems with th
`1
`Experiment
`symbol set.
`If only mean reaction times to targets are cor
`sidered the symbol set looks reasonably uniform. The rang
`of the means is only about 80 ms. An analysis of positiv
`and negative trials, however, reveals a potential probler.
`with symbol 3, the triangle. Positive trials reflect the tim
`to find the designated target; correct responses on negativ
`trials reflect the time to reject all of the distractor symbols
`Fast reaction times for positive trials coupled with lonj
`reaction times for negative trials could reveal a bias towar.
`seeing a particular figure. This is the pattern shown by sym
`bol 3, the triangle. On positive trials it is one of the fastest
`second only to symbol 4. On negative trials, however. it i
`by far the slowest. When looking for the triangle. then. sub
`jects had great difficulty in rejecting the distractor symbols
`The accuracy data support this. Subjects made more falsu
`alarms on the triangle than other symbols, but had a highe
`proportion of hits than any other symbol. Operators WOUll
`tend to mistake the threat designated by the triangle fo'
`the one actually displayed, especially under time pressure
`It is important to note that similarity ratings or confusioi
`matrices might have shown symbol 3 more confusable thai
`others, but that wouldn't have translated easily into a realis
`tic performance score. By using a visual search task the
`captures salient aspects of the display's use the informatioi
`can be directly related to aspects of performance.
`Many of the effects of Experiment 1 conform to result:
`from the psychological literature for search and matching
`tasks. Responses to negative trials took longer as would b:
`expected since on the average more symbols must be exam
`ined than in positive trials. The familiar numeric characters
`were responded to more quickly than the less familiai
`graphic characters. This speed advantage does not resull
`from subjects trading speed for accuracy since, with th:
`possible exception of target 2,
`there is no difference ir
`accuracy between the numeric and graphic characters.
`The effects of separation determined by the quadranl
`manipulation showed an unanticipated outcome.
`It was
`expected that the time to find a target would increase as
`the separation increased. This was true for quadrant condi-
`tions one through three, but the four quadrant conditior
`was unexpectedly fast.
`In this condition. symbols in eact
`frame were arranged one in each quadrant. This would
`result in a more symmetric arrangement than other qua-
`drant conditions. which could have facilitated search.
`
`
`Experiment 2
`Experiment 1 tested only the symbols themselves. Tc
`provide a more complete and representative test Experi-
`ment 2 exarriined subjects’ ability to locate target symbols
`in the presence of the markers shown in Table 1. Since
`these markers will appear in conjunction with the targets ir
`the actual threat situations it
`is important
`to know hovi
`much they will interfere with target detection. The additior.
`of markers was expected to act as visual noise, making
`visual comparisons more difiicult.
`The
`target
`always
`occurred in conjunction with a marker, but was presented in
`isolation when given as the standard at the beginning of
`each trial.
`‘Thus,
`the similarity of target
`to standard is
`reduced. These conditions favor those symbols with robust
`internal representations. Numerals are very farniliar and
`seen in under many different font and visual conditions. The
`graphicsymbols, on the other hand, are distinguishable, but
`rarely. if ever. encountered. The exception to this, symbol
`number three (triangle), is very similar to symbol five and
`whateverfainiliarity advantages it may have may be offset
`by this similarity. There were indications in Experiment 1
`that subjects had problems with these two symbols. Hence.
`the addition of markers should have less detrimental effect
`on the numeric symbols than on the graphic symbols.
`
`Method
`
`Stimuli rLn_d Apparatus. The same stimulus frames from
`Experiment
`1 were used. The frames were modified to
`include the designated marker for each target symbol and
`any other markers required in that frame. The assignment
`
`BOEING
`
`Ex. 1031, p. 133
`
`I500
`
`FIGURE 1
`
`5 I‘-100
`u-I
`
`ZP
`
`
`
` v\
`Ca.
`
`I300
`
`I100
`
`I100
`Nuugu
`5miioLs
`
`TAICET5
`
`-z0
`
`I-V(
`Lu
`of
`
`/‘
`
`C.as..a
`
`‘rflfine-rs
`
`plots reaction time for each
`The top panel of Figure
`quadrant separately for positive and negative trial
`types
`alongwith the mean. There was no efiect of quadrant for
`negative trials. For positive trials there was a steady
`increase in reaction time as the separation of the figures is
`increased, until the 4-quadrant condition where reaction
`time was unexpectedly fast. The bottom of Figure 2 shows
`proportion correct for each quadrant separately for each
`trial .type._ The advantage for the 4-quadrant condition on
`positive trials was not due to a speed-accuracy trade—off,
`since subjects were better as well as
`faster
`in
`that
`condition.
`
`FIGURE '2
`
`
`
`_,‘||-goo
`
`i J E)
`
`-
`
`I300
`
`3iuI0
`
`.1
`Elloo
`
`10
`
`NUMBER op QUADRANT5.
`
`"No' -mag
`
`\J
`
`/°
`
`/~
`
`52
`
`'24o
`
`ticO-
`
`NUMBER or QLJADRANT5
`
`BOEING
`Ex. 1031, p. 133
`
`

`
`-gtractor Symbol t° marker was rand°m within each
`Figure 4 plots reaction time for each mark separately
`_
`.'
`-
`for positive and negative trial types. The bottom panel of
`U1‘?-
`cedure. The design of Experiment2was
`Figure 4 shows proportion correct. Though there is a
`Desi n and pm
`terbalanced with respect to all the vari-
`significant interaction in the reaction ti.me data between
`.‘.’.—’.1“{31:t:ryi§.:l:ii’ 1. and reflects the primary concern of mark and trial type it is clear that both follow the same pat-
`iv
`-
`. es:-in tie effects of target-marker combinations. Each
`tern with respect to the different mark conditions. When
`‘
`d
`-
`'* Ssmg
`ually often with each marker. Every
`reaction time for each mark is plotted for each target, as in
`1:‘ .2; ;{::yI;acl)r(::‘Ur:;ld with one of the four markers. so that
`Figure 5, the complex effects that lead to the interaction
`.5‘
`-
`t had to identify the target on positive trials
`between target and mark can be seen.
`In general.
`the
`3'
`- Ils‘u't3J§‘c
`one of the four markers. Since each target
`square and tail markers are more disruptive than the oth-
`- sregaérd ltnhgree times in each quadrant condition it was not
`ers; the ghosted targets being somewhat more easily recog-
`.._’._- to completely map the four markers into each
`nized. The square and tail affect all targets about equally.
`.
`,1, uadrant condition. The assignment of markers ‘to The magnitude of the effects of the ghost and dot, however,
`fie. q
`mbols was done randomly with the constraint
`depend more on the specific symbol.
`'
`k
`actor Sy
`r serve as distractor about equally often,
`T tfigirihrgraefb: an equal number of frames with 0,1, and 2
`_ E.
`“'5 kérs on the distractor items. .
`_
`'1 'ub'ects. Sixteen new non-pilot subjects were selected
`I.
`-
`.
`'._ . Jfffe NASA - ARC subject pool and student and stafi
`L F eers
`.
`
`ii
`.
`
`I
`
`'
`
`I
`in Experiment 1 there were no effects of keyt‘assigAr;
`“Q, the data were collapsed over that c_ond1 ion.
`of variance on mean correct reaction times showed
`VTH
`I
`.1n‘efi'e'ctS of targets (F[8.120] '-‘- 17-Oi: P < -0001 . mark‘
`413.45] = 58; p < _.ooo1). ‘and trial type (
`1.15] =
`3;“ 'p < .0001). All interactions were significant
`<
`I
`I 6I')"‘with the interaction of markers and trial type being
`eakest (p < .02).
`‘.
`igure 3 plots the mean reaction time to each target
`‘with reaction times to positive and negative trials for
`in}
`_arget. The bottom of Figure 3 shows the proportion
`." e‘c't'-for each target as a function of
`trial
`type. As
`éctegl, targets 3 and 5 were very difficult for subjects.
`p_'§:"_;';iall;.' on negative trials. The pattern seen in Experi-
`-fr nt_-1 is amplified here. Accuracy data shows the same
`ects as the reaction time data.
`
`-
`
`mun 2 3
`
`F|6URE 4
`
`,.,m,
`
`V\
`2
`
`“limo
`5
`Ir-_
`
`g,5.,..
`;
`‘L12
`:2
`
`Woo
`
`’3"°
`
`[-0
`
`¢3H
`%.‘1D
`34(‘-
`
`.go
`
`MARK
`
`ExPERmeu'r 1
`
`MARK BY TRML 1-ypg
`
`Discussion
`The addition of visual noise. in the form of the marker
`conditions,
`increased overall mean reaction time and
`differentially affected the times to find or reject different
`targets. The general result was that
`the difficult cases
`noted in Experiment 1 were affected more than the easier
`conditions. This is especially clear in the case of symbols 3
`and 5. These were troublesome for subjects in the first
`experiment,
`and the same performance patterns are
`amplified in Experiment 2. That the 4 markers affected
`individual symbols to difierent degrees doesn't change this
`conclusion. With the exception of the good performance of
`symbol 8 in the ghost condition and the poor performance
`of symbol 4 in the clot condition, symbols 3.5.7.8. and 9 are
`consistently poorer than symbols 1,2,4. and 6. The former is
`the set of graphic symbols used; the latter is the set of
`number symbols and the asterisk. a common lexical charac-
`‘er-
`
`BOEING
`
`Ex. 1031, p. 134
`
`.
`
`'
`
`(D
`7
`
`[1.UO."‘f'D('D.'3('D('D
`
`i—
`
`e
`r
`r
`d
`3
`'
`
`n n
`
`;—
`it
`1'1
`
`-
`
`1L
`is
`
`as
`
`.
`
`
`
`1
`
`3
`
`5
`+
`TARGET
`
`i.
`
`1
`
`e
`
`1
`
`87
`
`
`
`BOEING
`Ex. 1031, p. 134
`
`

`
`l
`
`'1 cm
`
`lbw
`
`|$ch
`
`I'1oe
`
`-’\
`5
`w.
`EF
`
`2 E
`
`5'U
`C.H)
`u
`
`
`
`I3°°
`Murqaen
`Svmaoi
`
`1
`X
`
`2
`B
`
`
`
`correlated with mean target reaction times from Experi
`ment 2 (r = .83). and only moderately correlated with thu
`time from Experiment 1 (r = .53).
`Discussion
`
`Experiments 2 and 3 require diflerent require subject
`to extract different information from the same display.
`1)
`Experiment 2 the markers were not informative and had t
`be ignored in deciding whether a symbol was the target
`Experiment 3 requires a similar identification, but havin.
`found the target subjects must then disregard the targe
`and respond to the markers. The longer reaction times i"
`Experiment 3 reflect not only the increased difficulty c
`response selection. but the more complicated decision prc
`cess as welt Yet. this increased difliculty affected all tax
`gets equally. The pattern of reaction times across targets i
`due to the targets and the respective frames. not to an
`response or additional decisional complexities. The reactio
`time on each trial can thus be considered to have at leas
`two independent components: search time and respons
`selection and execution.
`
`3
`A
`
`4-
`4
`TARGET
`
`5
`K
`
`"1
`L
`L A *4
`
`U
`
`FIGURE é
`
`Ieoo on
`
`Eu
`
`fZ ;
`
`_l1oa
`
`70 5 g
`
`llzau
`#
`
`ISM:
`Monaco.
`SYMBOL
`
`1
`X.
`
`2
`B
`
`3
`A
`
`5
`4-
`A’
`4-
`TARGETS
`
`(.
`Q
`
`1
`/\
`
`8
`D4
`
`'1
`Ll
`
`%_' _ /4%‘.
`
`___(
`
`‘
`
`7-.2
`-n
`
`1'.’K0.2\.v0-
`
`TA Rc.e'rs
`
`E*P(-.RlN\(-fwr
`
`q
`
`Ghosting tends to be more difficult to detect in EXPE
`ment 3.
`though this is not statistically significant.
`T
`trend is interesting since the tendency in Experiment 2 V
`for the ghost to be more easily ignored. The symmetry ht
`is appealing. and it reveals something about both the efie_-
`of the markers and subjects’ strategies. Ghosting consl
`of deleting selected figure elements while preserving 1
`shape of the symbol. There is less chance for distortion
`masking than with the square or tail, and less chance
`diverting attention than with the blinking dot. Ghost‘
`would leave the target more recognizable than the DU
`markers, but would, itself, be more difficult to detect. V1
`then was reaction time to ghosted symbols i.n Experimen
`
`88
`
`BOEING
`
`Ex. 1031, p. 135
`
`ExP:RIMgm- 1
`MARK KY TfiR(‘E_-r
`
`Experiment 3
`The marker conditions provide important status infor-
`mation about other aircraft. An additional test was required
`to determine whether the subjects could quickly and reli-
`ably identify the marker associated with a target symbol.
`This procedure would also provide information about how
`performance would change when a source of difliculty other
`than visual interference was manipulated. Experiment 2
`made the discrimination of visual symbols more difficult,
`Experiment 23 makes response selection more difficult, since
`the number of markers is
`four
`rathe_r
`than the two-
`alternative yes/no response of the previous experiments.
`Visual discriminability plays only a part in the contribution
`to total response to a threat display. The selection of an
`appropriate action also contributes.
`If the response selec-
`tion componcnt
`is
`independent of visual aspects of
`the
`display then the response times for each symbol in Experi-
`ment 2 should be increased by a constant factor in Experi-
`ment 3.
`If
`the time to select and execute a response
`depends on the discriminability of the display items, then
`increasing the difficulty of response selection will change
`the relative reaction times for each symbol in Experiment 2.
`
`
`Design and Procedure
`In Experiment 3 the target symbol appeared on every
`trial. and the subjects‘ task was to identify which of the four
`markers was paired with the symbol. The same frames used
`in the previous studies were used. Subjects no longer sat in
`a sound isolated chamber, but were in a darkened room with
`the monitor at the same distance as before. The procedure
`was identical to the earlier experiments except that sub-
`jects were to press one of four micro- switches to indicate
`which of the four markers was associated with a given tar-
`get. The assignment of the four keys to the four markers
`was determined by latin square arrangement. so that each
`target was associated with each key equally often.
`In all
`other respects Experiment 3 was identical to the previous
`experiments. Twenty volunteers served as subjects.
`Results
`
`The data were collapsed over the response mapping
`condition and an analysis of variance was done with targets
`and markers as within subjects variables.
`The only
`significant main effect was target (l7‘[8.14-4] = 31.1, p < .001).
`There was no main effect of markers nor any interaction
`between markers and targets. Figure 6 shows mean reac-
`tion time for each target. These mean values are highly
`
`BOEING
`Ex. 1031, p. 135
`
`

`
`the reaction time to non-marked targets in Experi-
`
`hfrlo Since targets always had markers in Experiment 2.
`{S could profit by searching for markers. The number
`)7
`‘lee kers in a frame had a strong effect on the reaction
`‘_ar'1'hus_ we hypothesize that the greater difficulty in
`'n1z-mg the ghost as a marker offset its advantage in
`Cfgrving the shape of the symbol.
`that could lead to
`‘£22685 -In display design procedures.
`Summary
`Three experiments examined search performance with
`6 Small Situation display on which digits and graphic sym-
`#15 designated other aircraft. The time required to locate
`d identify the individual symbols showed that the graphic
`.mbo1s were in all cases inferior to the numeric stimuli.
`)crea51ng the difficulty of
`the visual discrimination
`a nifies the differences between graphic and numeric
`rsbols, whi