111111
`
`1111111111111111111111111111111111111111111111111111111111111
`US009106789Bl
`
`c12) United States Patent
`Shipman, Jr. et al.
`
`(10) Patent No.:
`(45) Date of Patent:
`
`US 9,106,789 Bl
`Aug. 11, 2015
`
`(54) VIDEOCONFERENCE AND VIDEO
`VISITATION SECURITY
`
`(75)
`
`Inventors: Bobby L. Shipman, Jr., Jonesboro, AR
`(US); Laura Shipman, Jonesboro, AR
`(US)
`
`(73) Assignee: Tech Friends, Inc., Jonesboro, AR (US)
`
`( *) Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 611 days.
`
`(21) Appl. No.: 13/355,211
`
`(22) Filed:
`
`Jan.20,2012
`
`(51)
`
`(52)
`
`(58)
`
`(2006.01)
`(2006.01)
`(2011.01)
`
`Int. Cl.
`H04N7114
`H04N7115
`H04N 21144
`U.S. Cl.
`CPC ................. H04N 7115 (2013.01); H04N 71147
`(2013.01); H04N 71155 (2013.01); H04N
`21144008 (2013.01)
`
`Field of Classification Search
`CPC .......... H04M 2201/40; H04M 3/4936; H04M
`3/5191; H04N 5/272; H04N 7/14; H04N
`7/141; H04N 7/142; H04N 7/144; H04N
`7/147; H04N 7/148; H04N 7/15; H04N 7/152;
`H04N 7/155; H04N 7/157
`USPC ............ 348/14.03, 14.08, 14.1, 14.11, 14.01,
`348/14.02, 14.04, 14.05, 14.06, 14.07,
`348/14.09, 14.12, 14.13, 14.14, 15.15,
`348/14.16; 379/265.03; 455/414.1, 566;
`704/270
`See application file for complete search history.
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`5,274,453 A * 12/1993 Maeda .......................... 348/584
`8,571,192 B2 * 10/2013 Etter ........................ 379/202.01
`8,648,894 B2 * 2/2014 Laney eta!. ............... 348/14.03
`
`100
`
`2005/0128283 A1 * 6/2005
`2006/0087554 A1 * 4/2006
`2006/0087555 A1 * 4/2006
`2007/0188596 A1 * 8/2007
`2007/0285504 A1 * 12/2007
`2008/0201158 A1 * 8/2008
`2010/0245532 A1 * 9/2010
`201110096139 A1 * 4/2011
`2012/0127284 A1 * 5/2012
`2012/0262271 A1 * 10/2012
`2012/0274724 A1 * 1112012
`2012/0281058 A1 * 1112012
`2013/0182061 A1 * 7/2013
`2013/0263227 A1 * 10/2013
`2013/0278631 A1 * 10/2013
`
`Bulriss et a!. ................ 348/14.1
`Boyd eta!. ................. 348/14.09
`Boyd eta!. ................. 348/14.09
`Kenoyer .................... 348/14.08
`Hesse ........................ 348/14.08
`Johnson et al. ................... 705/1
`Kurtz eta!. ................ 348/14.03
`Rudolf ....................... 348/14.11
`Bar-Zeev et al.
`............... 348/53
`Torgersrud eta!. .......... 340/5.53
`Cross eta!. .................. 348/14.1
`Laney eta!. ............... 348/14.03
`Stedman et a!.
`........... 348/14.03
`Gongaware et a!. . ............. 726/4
`Border eta!. ................. 345/633
`
`FOREIGN PATENT DOCUMENTS
`
`W02012057655 A1 * 5/2012
`wo
`* cited by examiner
`
`........ G06F 17/30793
`
`Primary Examiner- Khai N Nguyen
`(74) Attorney, Agent, or Firm- Fogarty, L.L.C.
`
`(57)
`
`ABSTRACT
`
`Systems and methods for providing video visitation security
`to a controlled-environment facility are disclosed. In some
`embodiments, a method may include detecting an image of a
`face in a video stream transmitted between a resident (e.g., an
`inmate) of a controlled-environment facility (e.g., a jail or
`prison) and a non-resident of the controlled environment
`facility during a video visitation session. For example, the
`first image may occupy a given area on a display upon ren(cid:173)
`dering of the video stream. As such, the method may include
`defacing (e.g., blurring, clouding, darkening, distorting,
`masking, shading, etc.) other areas on the display outside of
`the given area. Moreover, in various implementations, a
`method may also include detecting additional faces, deter(cid:173)
`mining and/or validating any number of visitors (e.g., via
`facial recognition), identifying certain attempts to circum(cid:173)
`vent security procedures, as well as taking pre-established
`corrective measures.
`
`28 Claims, 6 Drawing Sheets
`
`GTL 1004
`
`

`
`U.S. Patent
`
`Aug. 11, 2015
`
`Sheet 1 of 6
`
`US 9,106,789 Bl
`
`100
`
`130
`
`110
`
`120
`
`D oo
`
`135A
`~~
`
`FIG. 1
`
`0
`
`0
`
`300
`i
`
`340
`
`310
`
`335
`
`USER
`CONNECTION
`RULE
`DATABASE .....-----~ MANAGER ~----~ DATABASE
`
`SECURITY
`ENGINE
`
`320
`
`RAW
`VIDEO
`
`SECURE
`VIDEO
`
`INMATE VIDEO
`
`SECURE FAMILY
`VIDEO
`
`MEDIA
`SERVER
`
`FAMILY VIDEO
`
`SECURE INMATE
`VIDEO
`
`315
`
`FIG. 3
`
`INMATE
`VIDEO
`VISITATION
`CLIENT
`
`325
`
`FAMILY
`CLIENT
`
`330
`
`

`
`U.S. Patent
`
`Aug. 11, 2015
`
`Sheet 2 of6
`
`US 9,106,789 Bl
`
`a
`~~
`
`0
`
`

`
`U.S. Patent
`
`Aug. 11, 2015
`
`Sheet 3 of 6
`
`US 9,106,789 Bl
`
`VIDEO VISITATION SYSTEM
`
`410N
`400
`;
`-
`0 0 0 I PROCESSOR
`t
`4108
`
`410A
`
`430
`
`PROCESSOR I I PROCESSOR
`t
`
`I/O INTERFACE
`
`MEMORY 420
`-
`DATA
`STORAGE
`
`PROGRAM
`INSTRUCTIONS
`
`4{5
`
`4{5
`
`NETWORK
`INTERFACE
`
`440
`-
`
`INPUT/OUTPUT
`DEVICE(S)
`
`450
`-
`
`WIRED AND/OR WIRELESS _
`NETWORK CONNECTION
`
`FIG. 4
`
`!
`VIDEO
`DEVICE(S)
`\
`460
`
`AUDIO
`DEVICE(S)
`\
`470
`
`!
`DISPLAY(S)
`
`\
`480
`
`

`
`U.S. Patent
`
`Aug. 11, 2015
`
`Sheet 4 of 6
`
`US 9,106,789 Bl
`
`520A
`
`5058
`
`5258
`5308
`
`5108 FIG. 5B
`
`FIG. 5A
`
`520C
`I
`
`500C
`I
`
`FIG. 5C
`
`

`
`U.S. Patent
`
`Aug. 11, 2015
`
`Sheet 5 of 6
`
`US 9,106,789 Bl
`
`FIG. 6A
`
`FIG. 6B
`
`705
`
`VERIFY FIRST USER'S
`LOGIN INFORMATION
`
`715
`
`VERIFY SECOND USER'S
`LOGIN INFORMATION
`
`NO
`
`NO
`
`725
`
`INITIATE VIDEO
`VISITATION, RECORDING,
`AND/OR MONITORING
`FIG. 7
`
`730
`
`BLOCK VISITATION
`AND/OR ALERT
`AUTHORITIES
`
`

`
`U.S. Patent
`
`Aug. 11, 2015
`
`Sheet 6 of 6
`
`US 9,106,789 Bl
`
`805
`
`PERFORM FACE
`DETECTION TO
`DETECT ALL
`FACES IN
`NON-RESIDENT'S
`END
`
`825
`
`APPLY IMAGE
`PROCESSING
`(e.g., BLUR) TO
`NON-FACIAL AREA
`
`835
`
`FIG. 8
`
`NO
`
`NO
`
`840
`
`FINISH
`VISITATION
`AND/OR
`ALERT
`AUTHORITIES
`
`NO
`
`905
`
`CONTINUOUSLY OR
`PERIODICALLY
`DETECT FACE(S)
`
`FINISH VISITATION
`AND/OR
`ALERT AUTHORITIES
`
`925
`
`FIG. 9
`
`

`
`US 9,106,789 Bl
`
`1
`VIDEOCONFERENCE AND VIDEO
`VISITATION SECURITY
`
`TECHNICAL FIELD
`
`This specification relates generally to videoconferencing
`and video visitation, and, in one embodiment, to video visi(cid:173)
`tation security for controlled-environment facilities.
`
`BACKGROUND
`
`In the United States, after a person is arrested, detained, or
`otherwise committed to a correctional facility, that person is
`typically afforded certain visitation rights. Particularly, while
`residing within a correctional facility, an inmate may desire or
`need to communicate with non-residents of that facility, such
`as, for example, family members, friends, lawyers, etc.
`Depending upon the facility's internal procedures (e.g., pre(cid:173)
`determined days and times when visitation is allowed, etc.),
`the inmate may request that the visitation be arranged. Con(cid:173)
`versely, a non-resident desiring to have the visitation may
`contact the facility to initiate the request.
`In many situations, visitations are performed in person(cid:173)
`i.e., the non-resident physically travels to the correctional
`facility, complies with a security clearance protocol (e.g., ID
`verification, metal detector, body scan, etc.), and meets with
`the incarcerated resident in a secured location, usually under
`supervision of the facility's staff (e.g., a police officer or
`deputy, U.S. Marshal, etc.). Because in-person visitation can
`consnme a significant amount of resources, and because 30
`many correctional facilities are typically operating at or
`above capacity, visitation requests may have to be booked a
`significant amount of time in advance.
`As the inventors here of have recognized, however, recent
`developments in telecommunication systems and computer 35
`networks now allow remote, video visitation to take place.
`Generally speaking, by implementing video visitation sys(cid:173)
`tems, a correctional facility can potentially utilize its
`resources more efficiently, while providing better and more
`responsive services to its residents.
`
`40
`
`2
`recognition may include retrieving a video visitation record
`including a facial characteristic of the authorized visitor.
`Additionally or alternatively, the method may include
`detecting a second image of a second face in the video stream
`during the video visitation session, the second image occu(cid:173)
`pying a second area on the display upon rendering of the
`video stream. In these cases, defacing may include defacing
`areas on the display outside the first and second areas com(cid:173)
`bined. Furthermore, the method may include retrieving a
`10 video visitation record corresponding to the video visitation
`session, where the video visitation record includes a nnmber
`of authorized visitors, and determining that the number of
`authorized visitors is greater than 1. Conversely, ifthennmber
`of authorized visitors is equal to 1, the method may include
`15 taking corrective action.
`In some implementations, detecting and defacing may
`occur continuously during the video visitation session, and
`the size and position of the first image may change throughout
`the video visitation session. As such, the method may include
`20 detecting a change in the first image, determining a magni(cid:173)
`tude of the change, and taking corrective action in response to
`the magnitude of the change meeting a magnitude threshold.
`Additionally or alternatively, the method may include deter(cid:173)
`mining a duration of the change and taking the corrective
`25 action in response to the duration of the change meeting a
`duration threshold.
`In some cases, the method may include detecting an alpha-
`betic, nnmeric, or symbolic character, anon-biometric object,
`a bodily movement unrelated to speech, a hand gesture or sign
`language, etc., and taking appropriate corrective action (e.g.,
`defacing the offending content or material, etc.). For
`example, the method may include determining a correlation
`between a mouth movement and speech in the video stream.
`If the correlation drops below a threshold level, the method
`may mute the audio portion of the video stream, blank the
`video portion of the stream, etc. Additionally or alternatively,
`the method may identifY an affiliation of the resident or non(cid:173)
`resident with a given gang or group based, at least in part,
`upon the offending content.
`In another illustrative, non-limiting embodiment, a method
`may include identifYing a video visitation session between a
`resident of a correctional facility and a non-resident of the
`correctional facility, detecting one or more images of one or
`more faces present in a video stream transmitted from the
`45 non-resident to the resident during the video visitation ses(cid:173)
`sion, where the one or more images of the one or more faces
`occupy one or more areas upon rendering of the video stream
`on a screen, and degrading images on the screen outside of the
`one or more areas.
`For example, the method may include retrieving a video
`visitation record, the record indicating a nnmber of visitors
`expected during the video visitation session, and taking cor(cid:173)
`rective action in response to the number of expected visitors
`being smaller than the number of one or more images of the
`one or more faces. Conversely, in response to the number of
`expected visitors being greater than the nnmber of one or
`more images of the one or more faces, the method may
`include recording an absence of an expected visitor in the
`video visitation record.
`Additionally or alternatively, the method may include
`retrieving a video visitation record, the record including a
`facial characteristic of visitors expected during the video
`visitation session, performing facial recognition upon the one
`or more images of one or more faces and, in response to the
`facial recognition determining that at least one of the one or
`more faces do not match facial characteristic, taking correc-
`tive action. The method may further include detecting a
`
`SUMMARY
`
`This specification relates to systems and methods for pro(cid:173)
`viding video visitation security to controlled-environment
`facilities. In an illustrative, non-limiting embodiment, a
`method may include detecting a first image of a first face in a
`video stream transmitted between a resident of a controlled-
`environment facility and a non-resident of the controlled
`environment facility during a video visitation session. The 50
`first image may occupy a first area on a display upon render(cid:173)
`ing of the video stream. The method may then include defac(cid:173)
`ing other areas on the display outside of the first area.
`For example, the controlled-environment facility may be a
`jail or prison, and the resident may be an inmate. Moreover, 55
`defacing may include blurring, clouding, darkening, distort(cid:173)
`ing, masking, shading, or the like. In some cases, the video
`stream may be a live, a pre-recorded, or a delayed transmis(cid:173)
`sion video message.
`In some implementations, the method may also include 60
`performing facial recognition on the first image and, in
`response to the facial recognition determining that the first
`face does not match a face of an authorized visitor, causing
`that some form of corrective action be taken (e.g., termination
`of the video visitation session, recording of the video visita- 65
`tion session, flagging of the video visitation session, notifi(cid:173)
`cation of a third-party, etc.). Moreover, performing the facial
`
`

`
`US 9,106,789 Bl
`
`DETAILED DESCRIPTION
`
`4
`3
`change in the one or more images, determining a magnitude
`intended to limit the specification to the particular form dis(cid:173)
`closed, but, on the contrary, the intention is to cover all modi(cid:173)
`of the change, determining a duration of the change in
`fications, equivalents and alternatives falling within the scope
`response to the magnitude of the change meeting a magnitude
`threshold, and taking the corrective action in response to the
`of the claims. Also, any headings used herein are for organi(cid:173)
`duration of the change meeting a duration threshold.
`zational purposes only and are not intended to limit the scope
`of the description. As used herein, the word "may" is meant to
`In yet another illustrative, non-limiting embodiment a
`method may include identifYing a video visitation session
`convey a permissive sense (i.e., meaning "having the poten(cid:173)
`between a resident of a controlled-environment facility and a
`tial to"), rather than a mandatory sense (i.e., meaning
`non-resident of the controlled-environment facility and
`"must"). Similarly, the words "include," "including," and
`detecting one or more images of one or more faces present in 10 "includes" mean "including, but not limited to."
`a video stream transmitted by the non-resident and received
`by the resident, the one or more images of the one or more
`faces occupying one or more facial areas upon rendering of
`the video stream on a screen accessible to the resident during
`the video visitation session. The method may also include, in 15
`response to a number of the one or more faces matching an
`expected number of faces prescribed by a video visitation
`record, performing facial recognition upon the one or more
`images of the one or more faces. The method may further
`include, in response to the facial recognition determining that 20
`the one or more faces match one or more faces of expected
`visitors and prescribed by the video visitation record, allow(cid:173)
`ing the video visitation session to take place while disfiguring
`areas outside of the one or more facial areas on the screen.
`In various embodiments, one or more of the techniques
`described herein may be performed by one or more computer
`systems. In other various embodiments, a tangible computer(cid:173)
`readable storage medium may have program instructions
`stored thereon that, upon execution by one or more computer
`systems, cause the one or more computer systems to execute 30
`one or more operations disclosed herein. In yet other various
`embodiments, a system may include at least one processor
`and a memory coupled to the at least one processor, the
`memory configured to store program instructions executable
`by the at least one processor to cause the system to execute 35
`one or more operations disclosed herein.
`
`This specification discloses systems and methods for pro(cid:173)
`viding video visitation security to controlled-environment
`facilities. Various types of controlled-environment facilities
`are present in today's society, and persons may be voluntary
`or involuntary residents of such facilities, whether tempo(cid:173)
`rarily or permanently, full-time or part-time. Examples of
`controlled-environment facilities may include correctional
`institutions (e.g., municipal jails, county jails, state prisons,
`federal prisons, military stockades, juvenile facilities, deten(cid:173)
`tion camps, home incarceration environments, etc.), health(cid:173)
`care facilities (e.g., hospitals, nursing homes, mental health
`25 facilities, rehabilitation facilities, such as drug and alcohol
`rehabilitation facilities, etc.), restricted living quarters (e.g.,
`hotels, resorts, camps, dormitories, barracks, etc.), and the
`like.
`the ones
`Controlled-environment facilities, such as
`described above, may sometimes be thought of as a small
`community or city, perhaps walled or otherwise access
`restricted, wherein various activities occur within the com(cid:173)
`munity and between the community and those outside the
`community in the daily operation thereof. Such a community
`includes a number of individuals and enterprises directly
`associated therewith, including management, staff, and
`inmates, residents, patients, or guests, and a number of indi(cid:173)
`viduals and enterprises indirectly associated therewith,
`including friends and family of residents, vendors, govern-
`40 ment agencies, providers of services to residents, and indi(cid:173)
`viduals with a connection to the facility or its residents. Infor(cid:173)
`mation is often exchanged and transactions are often
`conducted by, between, among, and on behalf of the afore(cid:173)
`mentioned individuals and enterprises in performing their
`45 day-to-day activities. For example, residents of a controlled(cid:173)
`environment facility may receive visitation, including video
`visitation, from non-residents of that facility.
`For convenience of explanation, various examples dis(cid:173)
`cussed herein are presented in the context of correctional
`50 facilities. For instance, in some of the embodiments discussed
`below, a controlled-environment facility may be referred to as
`a jail or prison, and its residents may be referred to as arrest(cid:173)
`ees, detainees, or inmates. It should be understood, however,
`that the systems and methods described herein may be simi-
`55 larly applicable to other types of controlled-environment
`facilities and their respective residents (e.g., a hospital and its
`patients, a school dormitory and its students, etc.).
`Turning now to FIG. 1, a block diagram of an illustrative
`environment where various techniques described herein may
`60 be implemented is depicted according to some embodiments.
`As shown, prison 100 may host a plurality of inmates. In a
`typical scenario, immediately upon arrival at prison 100, each
`inmate may be subject to an "intake" or "booking" process,
`whereby certain information may be collected by prison staff
`65 via prison administration terminal or system 105. For
`example, the inmate's name, gender, address, date of birth,
`etc. may be received by system 105 and stored in a prison
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`Reference will now be made to the accompanying draw(cid:173)
`ings, wherein:
`FIG. 1 is a block diagram of an illustrative environment
`where various systems and methods described herein may be
`implemented according to some embodiments.
`FIG. 2 is a block diagram of a video visitation area within
`a controlled-environment facility according to some embodi(cid:173)
`ments.
`FIG. 3 is a block diagram of a video visitation security
`system according to some embodiments.
`FIG. 4 is a block diagram of a computer system, device, or
`station configured to implement various techniques disclosed
`herein according to some embodiments.
`FIGS. SA-C are diagrams illustrating various video visita(cid:173)
`tion security techniques according to some embodiments.
`FIGS. 6A-E are diagrams illustrating additional video visi(cid:173)
`tation security techniques according to some embodiments.
`FIG. 7 is a flowchart of a video visitation authentication
`process according to some embodiments.
`FIG. 8 is a flowchart of a video visitation security method
`according to some embodiments.
`FIG. 9 is a flowchart of another video visitation security
`method according to some embodiments.
`While this specification provides several embodiments and
`illustrative drawings, a person of ordinary skill in the art will
`recognize that the present specification is not limited only to
`the embodiments or drawings described. It should be under(cid:173)
`stood that the drawings and detailed description are not
`
`

`
`US 9,106,789 Bl
`
`5
`administration database. According to some embodiments, in
`addition to identification information, system 105 may also
`be configured to receive biometric samples (e.g., fingerprints,
`voice prints, iris scans, facial images or signatures etc.) and
`store one or more such samples as biometric signatures that
`may later be used, for example, to validate the identifY of an
`inmate over the course of a video visitation session.
`In some embodiments, prison 100 may allow an inmate to
`make or receive phone calls using device 115 via communi(cid:173)
`cation system 110. To that end, communication system 110
`may include one or more analog switches, IP gateways, PBX
`systems, etc. that provide communication systems to prison
`100. For example, an inmate within prison 100 may use
`device 115 to make a telephone call to (or to receive a tele(cid:173)
`phone call from) a non-resident of prison 100 operating
`device 125 through public switched telephone network
`(PSTN) 120.
`Although shown as telephones in FIG. 1, devices 115 and
`125 may include any communication device such as, for
`example, an analog telephone, a Voice over IP (VoiP) phone, 20
`a video visitation system, an Internet terminal, etc. For
`example, in some embodiments, non-resident device 115
`may be operable to communicate with device 125 through
`network 140. Network 140 may include, for example, the
`Internet. Additionally or alternatively, network 140 may 25
`include wireless cellular networks or the like.
`During their commitment to prison 100, inmates may be
`allowed to visit non-residents at a designated location within
`prison 100, such as, for example, visitation area 130. In some
`cases, visitation area 130 may include one or more rooms 30
`where an inmate may interact with a visiting non-resident in
`person. Additionally or alternatively, visitation area 130 may
`allow an inmate to communicate with a remotely located
`non-resident via a video visitation station orterminal200A-N
`(shown in FIG. 2). For example, a non-resident may use 35
`computer or communication devices 135A-N to communi(cid:173)
`cate or otherwise interact with a resident operating one of
`stations 200A-N over network 140.
`In some embodiments, prison administration system 105
`may store records that identifY visitation requests made by
`inmates and/or non-residents. These records may be used by
`system 150 and/or prison staff, for example, to help schedule
`future visitation sessions, for investigative purposes, etc. As
`such, each record may include, for instance, a type of visita(cid:173)
`tion (e.g., in person or by video), a date and time when the 45
`visitation is requested to take place, an identification of the
`inmate involved in the visitation, and an identification of the
`non-resident(s) involved in the visitation. In the case of a
`video visitation, the visitation record may also include facial
`images and/or signatures of the parties expected to be 50
`involved or otherwise authorized to participate in the visita(cid:173)
`tion. In some cases, a pre-registration or enrollment proce(cid:173)
`dure may be put in place requiring persons wishing to par(cid:173)
`ticipate in a video visitation session to first appear in person or
`submit pictures to prison 100 so that system 105 may collect 55
`information sufficient to generate corresponding facial
`images and/or signatures.
`FIG. 2 is a block diagram of a video visitation area within
`a controlled-environment facility according to some embodi(cid:173)
`ments. At the scheduled day and time of a video visitation, 60
`prison staff may direct the involved inmate to leave his or her
`cell and travel to one of video visitation stations 200A-N
`within visitation area 130. Alternatively, video visitation sta(cid:173)
`tions 200A-N may be implemented in a mobile or portable
`device (e.g., a tablet computer or the like), and therefore
`prison staff may simply take the video visitation to the inmate
`in his or her cell. In some implementations, video visitation
`
`6
`stations 200A-N may be separated by dividers 235 and moni(cid:173)
`tored by surveillance system 240 (e.g., closed-circuit televi(cid:173)
`sion or CCTV).
`As illustrated in FIG. 2, a given one of video visitation
`stations 200A-N (e.g., 200A) may include enclosure or base
`205, display or screen 210, camera and/or microphone 215,
`and speaker or audio output 220. Camera 215 may include a
`digital video camera or the like, and screen 210 may include
`a liquid crystal display (LCD), plasma, or cathode ray tube
`10 (CRT) display. In some cases, screen 210 may include a
`touchscreen configured to accept data input from a user. Addi(cid:173)
`tionally or alternatively, video visitation station 200A may
`include handset or headset 225 configured to perform audio
`15 input and/or output operations. Moreover, in various embodi(cid:173)
`ments, video visitation station 200A may also include a bio(cid:173)
`metric sensor (e.g., thumbprint scanner, iris scanner, etc.), a
`radio frequency identification (RFID), a barcode scanner, or
`the like.
`Upon arrival at video visitation 130, an inmate may be
`directed to an available one of one of video visitation stations
`200A-N (e.g., 200A) and may take seating 230. Station 200A
`may then take authentication information from the inmate
`such as, for example, ID, username, biometric information,
`bar code, RFID bracelet, facial recognition, etc. Once the
`inmate's identification is verified (e.g., matched against a
`video visitation record), the inmate may then have a video
`feed or stream established between station 200 and a non(cid:173)
`resident. In some cases, the non-resident may also be authen(cid:173)
`ticated prior to the initiation of the video visitation session.
`In some implementations, communication system 110 may
`establish and maintain communication between video visita(cid:173)
`tion stations 200A-N and non-resident's devices 135A-N dur(cid:173)
`ing a session. As illustrated in FIG. 1, communication system
`110 may be located inside of prison 100. In other embodi(cid:173)
`ments, however, system 110 and/or components thereof may
`be disposed outside prison 100. Furthermore, FIG. 1 shows a
`single prison 100 served by a single communication system
`110 for ease of explanation only. In other embodiments, two
`40 or more prisons may be served by the same communication
`system 110 and/or two or more of each such system may serve
`the same prison. Also, it should be noted that communication
`system 110 may be distributed across different geographical
`areas.
`FIG. 3 is a block diagram of video visitation security sys(cid:173)
`tem 300. In some embodiments, system 300 may be stored
`and/or executed, at least in part, by any given one of video
`visitation stations 200A-N. In other embodiments, system
`300 may be implemented by a dedicated visitation computer
`and/or by communication system 110. As illustrated, system
`300 may include connection manager 310 operable to estab-
`lish and maintain a connection between two endpoints, such
`as, for example, inmate video visitation client or resident
`client 325 (e.g., station 200A) and family client 330 (e.g.,
`device 135A). As such, system 300 may implement a set of
`telecommunication technologies that allow the endpoints to
`communicate via two-way video and audio transmissions
`concurrently or simultaneously. In some cases, the video
`communication may include a live, pre-recorded, or delayed
`transm1sswn video message. Furthermore,
`although
`described in an example herein as a "family client," it should
`be understood that client 330 may be used in any other video
`visitation or teleconference setting such as, for example, in an
`inmate's appearance at a video courtroom trial, bail/status
`65 hearings, attorney meeting, etc. It will be understood that the
`systems and methods used herein are not limited to a con(cid:173)
`trolled-environment facility, but may be used in any other
`
`

`
`US 9,106,789 Bl
`
`7
`video visitation or teleconference setting not associated with
`an inmate, family, and/or prison.
`Connection manager 310 may be configured to retrieve a
`video visitation record from rules database 340 and/or prison
`administration system 105, for example, and to initiate a
`connection to at least one of the endpoints identified in the
`video visitation record. As such, connection manager 310
`may be configured to receive a connection request, authenti(cid:173)
`cate one or more users, and ensure that the requested connec(cid:173)
`tion is allowed, for example, based on rules database 340. 10
`Examples of rules stored in database 340 may include authen(cid:173)
`ticationrules (e.g., is the person allowed to communicate with
`the target recipient; does a resident have unused visitation
`time for the relevant time period; is the resident under any
`restrictions or conditions-e.g., lockdown, behavior, illness, 15
`etc.-that would cause denial of a visitation request; is the
`controlled-environment facility currently allowing visita(cid:173)
`tions; do the biometrics for the individuals match those stored
`in a database of pre-enrolled or registered individuals; etc.).
`In some implementations, connection manager 310 may 20
`also be configured to assign a stream name and password for
`sending raw audio/video stream, assign a stream name and
`password for receiving secure audio/video stream, save
`stream information in a connection table, and return the
`stream names and passwords to the client requesting a con- 25
`nection. Connection manager 310 may also be configured to
`call and/or initiate security engine 320 to process incoming
`streams.
`In various embodiments, security engine 320 may execute
`user authentication, facial detection, object detection, Optical
`Character Recognition (OCR), Intelligent Character Recog(cid:173)
`nition (ICR), and/or facial recognition operations. Generally
`speaking, various image and video processing operations are
`known to persons of ordinary skill in the art. For example,
`particularly with respect to face detection, security engine 35
`320 may implement an algorithm similar to "Facial Feature
`Detection Using Haar Classifiers," Wilson and Fernandez,
`Journal of Circuits, Systems, and Computers (JCSC) 21, 4
`(April 2006), which is hereby incorporated by reference
`herein in its entirety. Security engine 320 may also be con- 40
`figured to evaluate various data during a video conferencing
`session and to implement appropriate corrective action in
`response to actual or potential security breaches. One or more
`of corrective actions may be prescribed, for example, by rules
`database 340. Additionally or alternatively, security engine 45
`320 may be programmed with a set of such rules.
`In operation, security engine 320 may connect to media
`server 315's incoming stream using unsecured stream name
`and password for reading. Security engine 320 may also
`connect to media server 315's outgoing stream using secured 50
`stream name and password for writing. Then, security engine
`320 may read the unsecured video stream and, while video is
`received, it may, for example, (a) create a full frame image;
`(b) perform image detection algorithms on image (facial
`detection, facial recognition, biometric template matching, 55
`object detection, OCR, ICR, etc.); (c) if no face is detected,
`blank the full image; (d) if illegal or unauthorized text or
`objects are detected within the image, blank the full image;
`(e) blank non-facial areas in the full image; (f) encode the
`modified image for video transmission (e.g., H.264); (g) write 60
`the video stream to the media server secured stream; etc.
`These, and other operations, are described in more detail in
`connection with FIGS. 5-9 below.
`In some embodiments, a resident may interact with soft(cid:173)
`ware client 325 interfa