PROCEEDINGS
`SPIE-The International Society for Optical Engineering
`,;:f ~GoF;7;;7::,-.,
`( ~ f.'.;\'f 1 fi n?7
`)
`'<~,,~·!zile_y}~ ;:~ ~~;~~m::sr~_,✓,l
`-...
`
`I
`
`25th A/PR Workshop
`Emerging Applications
`of Computer Vision
`
`David Schaefer
`Elmer F. Williams
`Chairs/Editors
`
`16-18 October 1996
`Washington, D.C.
`
`Sponsored by
`SPIE-The International Society for Optical Engineering
`AIPR Executive Committee
`
`Published by
`SPIE-The International Society for Optical Engineering
`
`r!1
`
`Volume 2962
`
`I
`l
`,
`! 1
`!
`
`SPIE is an international technical society dedicated to advancing engineering and scientific
`applications of optical, photonic, imaging, electronic, and optoelectronic technologies.
`
`TX 4-468-318
`\IIIIHIHWIIH
`
`•TX0004468318•
`
`

`

`-JlJ1&3f'
`'//is/
`1996¼
`C6f'/ !l-
`
`tE
`
`The papers appearing in this book comprise the proceedings of the meeting mentioned on the
`cover and title page. They reflect the authors' opinions and are published as presented and
`without change, in the interests of timely dissemination. Their inclusion in this publication does
`not necessarily constitute endorsement by the editors or by SPIE.
`
`Please use the following format to cite material from this book:
`Author(s), "Title of paper," in Emerging Applications of Computer Vision, David Schaefer,
`Elmer F. Williams, Editors, Proc. SPIE 2962, page numbers (1997).
`
`ISSN 0277-786X
`ISBN 0-8194-2366-1
`
`Published by
`SPIE-The International Society for Optical Engineering
`P.O. Box 10, Bellingham, Washington 98227-0010 USA
`Telephone 360/676-3290 (Pacific Time) • Fax 360/647-1445
`
`Copyright 0 1997, The Society of Photo-Optical Instrumentation Engineers.
`
`Copying of material in this book for internal or personal use, or for the internal or personal use
`of specific clients, beyond the fair use provisions granted by the U.S. Copyright Law is
`authorized by SPIE subject to payment of copying fees. The Transactional Reporting Service
`base fee for this volume is $10.00 per article (or portion thereof), which should be paid directly
`to the Copyright Clearance Center (CCQ, 222 Rosewood Drive, Danvers, MA 01923. Payment
`may also be made electronically through CCC Online at http://www.directory.net/copyright/.
`Other copying for republication, resale, advertising or promotion, or any form of systematic or
`multiple reproduction of any material in this book is prohibited except with permission in
`writing from the publisher. The CCC fee code is 0277-786X/97/$10.00.
`
`Printed in the United States of America.
`
`...
`
`

`

`-=•·t
`
`Contents
`
`vii
`ix
`
`A/PR Executive Committee
`Introduction
`
`SESSION 1
`
`2
`
`14
`
`26
`
`MEDICAL APPLICATIONS
`Local force model for cardiac dynamics analysis based on CT volumetric image sequences
`[2962-02]
`J. G. Tamez-Pena, Univ. of Rochester; C. W. Chen, Univ. of Missouri/Columbia; K. J. Parker,
`Univ. of Rochester
`
`Small object detection using morphological filtering and multiresolution analysis with
`application to microcalcification detection in mammograms [2962-03]
`L. Chen, C. W. Chen, K. J. Parker, Univ. of Rochester
`Active contour based on the elliptical Fourier series applied to matrix-array ultrasound of the
`heart [2962-04]
`R. Drezek, G. D. Stetten, T. Ota, C. Fleishman, E. Lily, C. Lewis, C. J. Ohazama, T. Ryan,
`D. Glower, J. Kisslo, 0. T. von Ramm, Duke Univ.
`
`SESSION 2
`
`36
`
`IMAGE MINING
`Wavelet index of texture for artificial neural network classification of Landsat images
`[2962-05]
`H. H. Szu, Naval Surface Warfare Ctr.; J. Le Moigne, NASA Goddard Space Flight Ctr.;
`N. Netanyahu, Univ. of Maryland/College Park; C. C. Hsu, Trident Systems, Inc.; M. Francis,
`Univ. of Southwestern Louisiana
`
`45
`
`55
`
`63
`
`78
`
`Registration of satellite imagery utilizing the low-low components of the wavelet transform
`(2962-06]
`E. Kaymaz, B.-T. Lerner, KT-Tech, Inc.; W. J. Campbell, J. le Moigne, NASA Goddard Space
`Flight Ctr.; J. F. Pierce, U.S. Naval Academy
`
`Spectral unmixing of remotely sensed imagery using maximum entropy [2962-07]
`S. R. Chettri, NASA Goddard Space Flight Ctr.; N. Netanyahu, Univ. of Maryland/College Park
`and NASA Goddard Space Flight Ctr.
`Spectral imaging applications: remote sensing, environmental monitoring, medicine, military
`operations, factory automation, and manufacturing (2962-08]
`N. Gat, S. Subramanian, Opto-Knowledge Systems, Inc.; J. Barhen, Oak Ridge National Lab.;
`N. Toomarian, Jet Propulsion Lab.
`Extracting an image similarity index using meta-data content for image mining applications
`[2962-09]
`S. Raghavan, LNK Corp.; R. F. Cromp, NASA Goddard Space Flight Ctr.; 5. Srinivasan,
`R. Poovendran, LNK Corp.; W. J. Campbell, NASA Goddard Space Flight Ctr.; L. Kanai,
`LN K Corp. Inc.
`
`iii
`
`

`

`SESSION 3
`
`94
`
`97
`
`SOCIAL IMPACT
`Milestones on the road to independence for the blind [2962-10)
`K. Reed, NASA Goddard Space Flight Ctr.
`
`Social impact of computer vision (2962-11]
`H. Baetjer, Loyola College
`
`SESSION 4
`
`AUTO AND FLIGHT SAFETY AIDS
`
`104
`
`111
`
`122
`
`Multiple vehicle detection and tracking [2962-13)
`M. Betke, E. Haritaoglu, L. S. Davis, Univ. of Maryland/College Park
`
`Image processing using acousto-optical tunable filtering [2962-14)
`.
`L. J. Denes, B. Kaminsky, M. S. Gottlieb, P. Metes, Carnegie Mellon Research Institute;
`S. Simizu, R. T. Obermyer, C. J. Thong, M. J. Uschak, S. G. Sankar, Advanced Materials Corp.
`
`Real-time visual processing in support of autonomous driving [2962-16)
`M. Nashman, National Institute of Standards and Technology; H. Schneiderman, Carnegie
`Mellon Univ.
`
`133
`
`Real-time landmark-based optical vehicle self-location [2962-17)
`M. D. Squires, M. P. Whalen, G. Moody, C. J. Jacobus, Cybernet Systems Corp.
`
`SESSION 5
`
`REAL-TIME EVENT UNDERSTANDING
`
`144
`
`Autonomous video surveillance [2962-20)
`B. E. Flinchbaugh, T. J. Olson, Texas Instruments Corporate Research Labs.
`
`SESSION 6
`
`MILITARY APPLICATIONS
`
`154
`
`162
`
`171
`
`180
`
`192
`
`RADIUS testbed system (2962-22]
`D. J. Gerson, S. E. Wood, Jr., CIA Office of Research and Development
`
`IU for military and intelligence applications: how automatic will it geH (2962-23)
`J. L. Mundy, GE Corporate Research and Development Ctr.
`
`User interface representations for image understanding (2962-24)
`M.A. J. Puscar, A. J. Hoogs, Lockheed Martin Corp.
`
`Region of interest identification in unmanned aerial vehicle imagery (2962-25]
`J. L. Solka, D. J. Marchette, G. W. Rogers, E. C. Durling, J. E. Green, D. Talsma, Naval Surface
`Warfare Ctr.
`
`RADIUS testbed database: temporal queries and optimization [2962-26]
`R. Cardenas, A. J. Hoogs, Lockheed Martin Corp.
`
`SESSION 7
`
`INFORMATION FROM TWO-DIMENSIONAL IMAGES
`
`202
`
`Integrated system for automated financial document processing (2962-27]
`K. Hassanein, S. Wesolkowski, R. Higgins, NCR (Canada); R. Crabtree, A. Peng, NCR
`
`iv
`
`

`

`213
`
`226
`
`236
`
`Inspection of surface-mount device images using wavelet processing [2962-28]
`G. Carillo, S. D. Cabrera, A. A. Portillo, Univ. ofTexas/EI Paso
`
`Automated building extraction using dense elevation matrices [2962-29]
`A. A. Bendett, U. A. Rauhala, J. J. Pearson, GOE Systems, Inc.
`
`Fusing mainstream and media processors to solve embedded imaging applications affordably
`(2962-30]
`R. Rinn, C. Fleischer, Parsytec Inc.
`
`SESSION 8
`
`FACIAL AND GESTURE RECOGNITION
`
`244
`
`253
`
`Computing 3D head orientation from a monocular image sequence [2962-31]
`T. Horprasert, Y. Yacoob, L. S. Davis, Univ. of Maryland/College Park
`
`FERET (Face Recognition Technology) program [2962-32]
`P. J. Rauss, P. J. Phillips, M. K. Hamilton, U.S. Army Research Lab.; A. T. DePersia, National
`Institute of Justice
`
`264
`
`Face recognition using hybrid systems [2962-34]
`S. Gutta, J. Huang, H. Wechsler, George Mason Univ.; B. Takacs, Physics Optics Corp.
`
`SESSION 9
`
`LAW ENFORCEMENT APPLICATIONS
`
`276
`
`287
`
`Infrared facial recognition technology being pushed toward emerging applications [2962-35]
`D. C. Evans, Technology Recognition Systems, Inc.
`
`Hyperspeed data acquisition for 3D computer vision metrology as applied to law enforcement
`(2962-36]
`B. R. Altschuler, Walter Reed Army Medical Ctr.
`
`295
`
`Commercialization of the weapons team engagement trainer: update of process [2962-37]
`J. W. Healy, J. Horey, R. T. McCormack, R. S. Wolff, E. E. Purvis 111, Naval Air Warfare Center
`
`302
`
`Author Index
`
`/
`
`V
`
`

`

`+
`!
`
`I
`4
`
`i
`
`\
`
`Autonomous Video Surveillance
`
`Bruce E. Flinchbaugh and Thomas J. Olson
`
`Texas Instruments Corporate Research Laboratories
`P.O. Box 655303, MIS 8374, Dallas, TX 75265
`
`ABSTRACT
`
`fl,
`\
`I
`I
`,tr!
`
`..
`
`i
`
`•
`
`This presentation highlights needs for autonomous video surveillance in the context of physical secu(cid:173)
`rity for office buildings and surrounding areas. Physical security is described from an operational
`perspective, defining the principal responsibilities and concerns of a physical security system. Capabilities
`and limitations of current video surveillance technology are described, followed by examples of how com(cid:173)
`puter vision techniques are being used and advanced for autonomous video surveillance systems.
`
`Keywords: video surveillance, scene monitoring, computer vision, security
`
`1. Physical Security Systems and Operations
`
`Major activities of physical security for office buildings today are: access control, intrusion detection,
`guard patrols, CCTV surveillance, alarm monitoring, response dispatching, and investigations. Autono(cid:173)
`mous video surveillance technology will ultimately improve productivity and effectiveness in all of these
`activities. The primary responsibilities of physical security are described below.
`
`1.1 Access Control
`
`The most common approach to physical security is to control who may enter a building or an area at
`the perimeter. By restricting access to trusted individuals (e.g., employees), many opportunities for securi(cid:173)
`ty breaches are eliminated. Door locks and keys provide much of this security, and many buildings deploy
`guards at building entrances to control who enters. The primary automatic access control technology in use
`today is provided by electronic badge reader systems. In this approach, an electronically readable badge is
`issued to each person with access privileges, and the person may use the badge like a key to open doors
`where guards are not stationed. A drawback of keys and electronic badges is that they may be used by un(cid:173)
`authorized individuals to gain access. For tighter security in access control, a variety of biometric access
`control technologies are available to measure physical characteristics of people: voice verifcation, retina
`scanners, fingerprint scanners, hand scanners, face recognition, and body weight measurements. The pri(cid:173)
`mary limitation of access control technology is that it does not defeat security breaches by insiders.
`
`1.2 Intrusion Detection
`
`A second line of defense against physical security breaches is to detect situations where people gain
`unauthorized access to a facility. This may occur at regular access control points (e.g., doors) or at other
`places (e.g., windows and fences). For example, an unauthorized person might enter by "piggy backing"
`on access by authorized person (i.e., by slipping through a door before it closes), or by any number of
`physical "breaking and entering" approaches. Various technologies are available to detect intrusions. The
`most commonly used devices are door switches and infrared motion detectors, like those found in many
`home security systems. Infrared motion detectors operate by detecting rapid heat changes in an area that
`
`144
`
`SPIE Vol. 2962 • 0277-786X/97/$10.00
`
`

`

`+
`!
`
`I
`I .,.
`
`I
`
`T
`
`~
`;
`I
`I
`4:";
`
`... :·
`
`...
`i .. I
`!
`
`l
`
`4
`I
`
`~
`i
`
`I
`~
`
`~
`
`I
`i
`
`l
`
`I -~
`
`e.~~
`i
`I
`
`are caused when a person enters the area. Other intrusion detection techniques include sound detectors,
`g_lass-break detectors, light beams, and various other electromagnetic change detectors. Although an intru(cid:173)
`sion detector may be used to signal alarms, it does not provide a description of the specific situation that
`causes an alarm, thereby requiring a subsequent follow-up action to respond to the alarm or investigate the
`incident later.
`1.3 Guard Patrols
`In addition to guards who monitor entrances for access control, a common security practice is for
`guards to periodically patrol sites to visually confirm that the premises are secure and to identify and report
`adverse conditions (e.g., missing property and damage from vandalism). Nearly all of the technology that
`supports these visual surveillance activities in practice today involves mechanical and electronic devices
`that the guards use to prove that they visited specific areas and to report observations. However, some ad(cid:173)
`vanced robotic security systems attempt to accomplish various guard patrol responsibilities. For example,
`mobile robots equipped with sonic sensors for navigation and detection have been installed for experimen(cid:173)
`tal security applications to patrol buildings, but this approach faces many problems to overcome before it
`can compete effectively with guards and more-reliable video surveillance approaches.
`1.4 CCTV Surveillance
`Closed circuit television (CCTV) camera networks that supply video data to security system centers
`are often used to support physical security operations in buildings and surrounding areas. In a small sys(cid:173)
`tem, a few cameras may be cabled to TV monitors for remote viewing by guards or other security personel.
`Another common practice is the use of time-lapse video tape recorders to record video data from one or
`more cameras. Some time-lapse recorders provide a multiplexed recording capability so that several cam(cid:173)
`eras may be recorded on a single tape. In large CCTV security systems, many cameras are cabled to an
`array of TV monitors and video tape recorders, to support live observations as well as after-the-fact inves(cid:173)
`tigations using recorded video data.
`CCTV cameras may be mounted with a fixed field of view, or mounted on a pan and tilt mechanism
`that can be remotely controlled by an observer to view a wider area. Video cameras may also be mounted
`on a small platform that moves along a track, enabling a single camera to scan a much wider area ( e.g., by
`moving along a long wall in a large parking garage).
`The primary autonomous video surveillance systems available today are known as "video motion de(cid:173)
`tectors" or "VMDs". In principle, VMDs can be programmed for various tasks such as intrusion detection
`and to signal alarms for fairly complex situations. For example, some VMDs can be programmed to signal
`an alann when something moves across the field of view from left to right, while not signaling an alarm
`when something moves from right to left. However, in practice available VMDs produce too many false
`alarms in typical environments [6], and they require substantial operator training and experience to pro(cid:173)
`gram the system.
`1.5 Alarm Monitoring and Response Dispatching
`Large physical security systems generally have a centralized control center where security staff moni(cid:173)
`tor alarms, dispatch guards to respond to incidents, and maintain a record of the incidents and their
`resolution. In some cases, control center operators actively monitor remote cameras to detect incidents and
`enoaoe in wide-ranoino visual surveillance tasks. Although people are very good at these tasks, long peri(cid:173)
`0 r inactivity in a scene make this monitoring job tedious. It is also impractical for
`od; ;f routine activity
`one person to reliably and simultaneously monitor hundreds of cameras. Thus control ce_nter operators typ(cid:173)
`ically spend most of their time monitoring and processing alarms from autonomous devices.
`
`0
`
`; l
`
`I
`
`145
`
`

`

`1.6 Investigations
`Although the primary goal of physical security is to prevent security breaches, or to detect and re(cid:173)
`spond in time to minimize the impact, a rapid and accurate capability to investigate incidents after the fact
`is also a key activity of physical security. Unlike retail store security (where a frequent threat is shoplifting
`by customers) office building security often faces the challenge of determining which insider has stolen
`something after it is determined that a theft has occurred. In this matter, differences between physical secu(cid:173)
`rity and information security begin to blur. For example, evidence for theft of trade secrets might be
`visually observable (as when an employee opens a file drawer and reads or copies a document) even
`though the document may be physically replaced in the drawer within a few minutes. For cases where a
`large material property item has been stolen, time-lapse video recordings of building entrances and exits
`provide a useful record of events for investigators to search for suspects. However, the limited video sur(cid:173)
`veillance tool for investigators in this regard remains a video tape player with a fast-forward button, and
`automatic video surveillance systems to assist investigators in detecting theft of intellectual property are
`beyond the state of the art.
`
`2. Computer Vision in Autonomous Video Surveillance Systems
`
`Texas Instruments has demonstrated a variety of autonomous video surveillance capabilities involving
`computer vision. In this section we summarize several TI capabilities for surveillance in office building
`environments.
`
`Our approach to video surveillance has been to exploit data that is readily computed from incoming
`video streams by using domain-specific constraints and contextual information to interpret the data. The
`video processing is primarily a matter of detecting areas of dynamic change via thresholded image differ(cid:173)
`ences and forming connected components for the subsequent analysis. In addition to devising new methods
`for providing more accurate and reliable information about moving entities and their surroundings, we are
`focusing on computer vision for interpreting scene events involving complex spatio-temporal conditions
`and interactions.
`
`2.1 People Tracking and Position Mapping
`
`The most basic function of a surveillance system is to provide situational awareness, i.e., to inform se(cid:173)
`curity personnel of what is going on in the monitored area. TI has demonstrated an end-to-end real-time
`video surveillance system that uses a visual memory paradigm to integrate information from multiple cam(cid:173)
`eras and to provide situational awareness. The visual memory is an object-oriented database that supports a
`variety of spatio-temporal queries [1, 3, 4]. The system detects and tracks people at ten frames per second,
`applies a ground plane constraint to estimate their 3-D positions, and records their positions and other in(cid:173)
`formation in the visual memory. A graphical interface allows users to construct security-related queries
`and view the results.
`
`Figure 1 shows the system being used to monitor activity in a hallway. To detect people, the system
`differences live video frames with a background reference image to estimate regions of change. These re(cid:173)
`gions are grouped into collections that are consistent with the size and shape of a person walking in the
`field of view. This process is repeated until all regions are either interpreted as part of a person, or dis(cid:173)
`missed if sizes and shapes of neighboring regions are inconsistent with such an interpretation. The user
`interface presents video surveillance observations from the visual memory in an interactive map display.
`Users interact with a map of the monitored region, and can pan, scroll and zoom to focus on a region of in(cid:173)
`terest. Queries enable the display of both current positions of objects of interest and historical positions
`(i.e., paths) of particular objects. Users can also specify alarms to be generated when someone enters a par(cid:173)
`ticular region.
`
`146
`
`

`

`♦
`
`•·
`
`FIGC~RE 1. TI people tracking and position mapping system. The system tracks moving objects in video from
`secunty cameras, as at left Square icons in the map-based interface at right represent location histories of humans at
`left. Pop-up window gives a detailed view of one observation record. Shaded rectangles in map are alarm regions.
`
`2.2 Detection of People Carrying Boxes
`After a theft has been reported in an area monitored by time-lapse video recorders, security staff typi(cid:173)
`cally review video tapes for the period during which the theft occurred. Tapes captured at building access
`points often reveal how the stolen material left the facility, or yield a short list of people who may have re(cid:173)
`moved the material. However, the process of reviewing the tapes can be extremely tedious, especially when
`the period of interest spans many days.
`In these experiments, TI demonstrated a system for screening security video tapes to identify seg(cid:173)
`ments that may contain images of people carrying boxes. This task was suggested by experts in physical
`security operations, who often look for boxes as part of theft investigation. The algorithm uses perceptual
`grouping techniques to identify collections of lines that may be projections of rectangular solids. If it finds
`a high-quality grouping of an appropriate size, it flags that video frame for inspection by a human and pro(cid:173)
`ceeds to the next frame. This relieves the investigator of the need to examine and interpret every frame.
`The box detection algorithm was tested on 500 frames taken from time-lapse video sequences show(cid:173)
`ing humans passing through a revolving door. Figure 2 shows a typical frame. The system exhibited a 93%
`detection rate at a false alarm rate of 13%. Details of the experiments and algorithm are given in a forth(cid:173)
`coming paper [5].
`2.3 Event Description for Video Indexing Using Motion Graphs
`Future autonomous video surveillance systems will need to be able to classify motions and interac(cid:173)
`tions of objects into events that are meaningful and important to security staff. TI has developed an
`Automatic Video Indexing (AVI) system ([2], figure 3), which performs event recognition on surveillance
`video tapes. As in the case of the box detection work described above, the immediate goal of this project
`was to assist human investigators in finding relevant segments of security video recordings; however, the
`event recogntion algorithm is general and can be applied to many video understanding tasks.
`In the AVI system, object motions and interactions are described by a directed acyclic graph called a
`motion graph. Each node of the graph is an observation of an object, which is simply a region of change
`detected by image differencing. In the motion graph, each object is tracked and is linked to its predecessor
`and successor in time. Forks and joins in the graph represent complex interac~ions. For example, if a per(cid:173)
`son enters a scene, puts down an objects and leaves, the graph will contain a chain of nodes representing
`the person, with a fork node whose successors are the continuation of the person track and a chain of ob-
`servations of a stationary object.
`
`147
`
`

`

`f
`
`t
`
`t
`
`,
`
`,
`
`r
`
`FIGURE 2. Box Detection in a security video image. Lines overlaid on the figure were identified by the detection
`algorithm as providing evidence for a box. In testing on 500 frames, the algorithm achieved a 93% detection rate at a
`false alarm rate of 13%.
`
`FIGURE 3. The Automatic Video Indexing system detects significant events in security videotapes. Above, the
`system identifies an instance of removal of an object from the scene, defined as disappearance of a stationary object
`(the briefcase) while in contact with a moving object (the human). An overlaid box highlights the objects involved in
`the removal event. Other removals from other points in the videotape are shown on the clipboard at right.
`
`Figure 4 provides an example of how the motion graph is constructed and interpreted. Vertical lines
`rep~esent frames, here compressed to ID to simplify the diagram. Each node of the graph consists of an ob(cid:173)
`servation of an object at a particular place in a particular frame. Links between objects constitute
`hypotheses about identity, constructed by tracking objects over time. Nodes with two or more successors
`correspond to places where the tracking algorithm determined that an object split into multiple objects.
`Similarly, nodes with multiple predecessors result from the merging of multiple objects. Events are defined
`
`148
`
`

`

`t
`
`t
`
`7
`
`,
`
`"' f
`
`1
`
`r
`
`Entrance
`
`·\1
`! I -
`I
`
`Remoll,a/ J f
`
`Entrance
`
`FIGURE 4. Automatic Video Indexing motion graph example. The motion graph captures the motions and
`interactions of objects in the scene. Predicates on the graph correspond to event classes such as Entrance, Exit,
`Removal, et cetera.
`by charactenst1c structures m the graph, as shown m the figure. For example, a cham of observations be(cid:173)
`ginning near the edge of the image constitutes an Entrance event, a moving object that splits into a moving
`object and a stationary one constitutes a Deposit event, and so on.
`The AVI video indexing capability allows extremely rapid access to significant events in long time(cid:173)
`lapse security videos. The algorithms were demonstrated live at the 1996 Image Understanding Workshop
`and have been used with both visible and infrared video data.
`
`3. Advanced Proof-of-Concept Demonstrations
`In autonomous video surveillance research, TI is integrating the capabilities described in the preced(cid:173)
`ing section to produce a series of proof-of-concept demonstrations. Together these demonstrations
`illustrate how physical security operations of the future will use autonomous video surveillance systems.
`Key challenges are computing event graphs on-line at ten frames per second, and devising event graph
`analysis methods to exploit contextual information in useful office surveillance scenarios.
`This research is focussed on three overall surveillance scenarios, for monitoring hallway, office, and
`building perimeter areas. In each area, a camera provides live video data of scenes in the field of view,
`while the AVS system monitors the video to analyze events and signal alarms.
`3.1 Hallway Surveillance
`In this scenario, suggested previously in Figure 1, the autonomous video surveillance system detects
`and tracks people as they walk in office building hallways. Alarms are interactively defined for conditions
`such as when someone loiters in a specified area or enters a particular office. This visual assessment pro(cid:173)
`vides information to augment other security system data, such as biometric access control information at
`building entrance points.
`
`149
`
`

`

`, lj
`
`FIGURE 5. Office Surveillance Scenario. Video event recognition can be used to detect unauthorized access to
`documents, file drawers, computers, et cetera.
`
`t
`
`;
`
`.,
`'
`
`t
`
`FIGURE 6. Perimeter Surveillance scenario. The AVI event recognition algorithms can be used with low-cost
`infrared cameras to detect security threats in tptal darkness.
`
`3.2 Office Surveillance
`
`Inside individual offices, the autonomous video surveillance system will maintain a situational aware(cid:173)
`ness record of events and signal alarms for a variety of specified conditions. For example, an alarm may be
`specified for events in which someone enters the office and places a briefcase on the desk, but not if the
`person leaves a document on the desk. Using contextual information such as time of day and access control
`identification, the system will report other alarm conditions that are functions of the number of people in
`the room and what they do. A representative office surveillance scene is shown in Figure 5.
`3.3 Perimeter Surveillance
`
`For perimeter monitoring scenarios, a TI NightSight infrared camera will provide video data for the
`surveillance system to monitor areas outside the building at night. For example, the system could monitor a
`building entrance and signal an alarm if someone walks by and leaves a box outside the door (e.g., as illus(cid:173)
`trated in Figure 6), but not if someone loiters without leaving a box.
`
`150
`
`

`

`•
`
`t
`
`;
`
`.,
`'
`
`t
`
`The ability to reliably discriminate significant and insignificant events is important to reduce false
`alarms for physical security applications, and poses many challenges for computer vision research to ad(cid:173)
`vance the state of the art.
`
`4. Acknowledgements
`
`The autonomous video surveillance capabilities described in this paper represent the research of many
`members of the technical staff at TI over the past five years. Among them the primary contributors are Tom
`Bannon, Frank Brill, Jon Courtney, Chris Kellogg, and Kashi Rao.
`
`5. Bibliography
`
`[1] Bannon, T., "Visual Memory Prototype Demonstration", TI Internal Technical Report CSL-ITR-93-01-
`25, Jan. 1993.
`·
`[2] Courtney, J. "Automatic video indexing via object motion analysis", TI Internal Technical Report, to
`appear in Pattern Recognition.
`[3] Flinchbaugh, B., and T. Bannon, "Autonomous Scene Monitoring System", Proc. 10th Annual Joint
`Government-Industry Security Technology Symposium, American Defense Preparedness Association,
`June 1994.
`[4] Kellogg, C., "Visual Memory", TI Internal Technical Report, also MIT Media Lab MS thesis, 1993.
`
`[5] Rao, K., and P. Sarwal, "A computer vision system to detect 3-D rectangular objects", TI Internal Tech(cid:173)
`nical Report. Accepted for IEEE Workshop on Applications of Computer Vision (WACV 96).
`
`[6] Vigil, J., "An evaluation of exterior video motion detection systems. Volume 1: Intrusion detection
`tests. Volume 2: Nuisance alarm results", Sandia Reports SAND92-0108/l, SAND92-0108/2, Sandia
`National Laboratories, 1992 .
`
`151
`
`