UNITED STATES PATENT AND TRADEMARK OFFICE
`_____________________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`_____________________
`
`
`
`GLOBAL TEL*LINK CORPORATION
`Petitioner
`
`v.
`
`SECURUS TECHNOLOGIES, INC.
`Patent Owner
`____________________
`
`Patent No. 9,083,850 B1
`_____________________
`
`
`
`PETITION FOR INTER PARTES REVIEW OF U.S. PATENT NO. 9,083,850
`
`
`
`
`
`
`
`
`
`Mail Stop “PATENT BOARD”
`Patent Trial and Appeal Board
`U.S. Patent and Trademark Office
`P.O. Box 1450
`Alexandria, VA 22313-1450
`
`

`
`U.S. Patent No. 9,083,850 B1
`
`TABLE OF CONTENTS
`
`Introduction. .................................................................................................... 1 
`I. 
`II.  Mandatory notices. ......................................................................................... 2 
`A. 
`Real parties-in-interest. ......................................................................... 2 
`B. 
`Related matters. ..................................................................................... 2 
`C. 
`Lead and back-up counsel. .................................................................... 2 
`III.  Grounds for standing (37 C.F.R. § 42.104(a)) ............................................... 2 
`IV.  Statutory Grounds for the challenge. .............................................................. 3 
`V.  Overview of the ’850 patent. .......................................................................... 5 
`VI.  Claim construction. ......................................................................................... 8 
`VII.  Level of ordinary skill. ................................................................................... 8 
`VIII.  Ground 1: Claims 1, 5, 8, 9, and 14 are unpatentable under 35 U.S.C. § 103
`as obvious over Shipman in view of Garrison. .............................................. 9 
`A. 
`Shipman discloses a video visitation system using blurring techniques
`to obscure the background of the video. ............................................... 9 
`Garrison utilizes depth of field manipulation to blur an image. ......... 16 
`Rationale for combining Shipman and Garrison. ................................ 17 
`The combination of Shipman and Garrison renders independent claims
`1, 8, and 14 obvious. ........................................................................... 18 
`1. 
`The combination of Shipman and Garrison teaches or suggests
`the preamble of independent claims 1, 8, and 14. ................... 20 
`The combination of Shipman and Garrison teaches or suggest
`the “receiving limitation” of independent claims 1, 8, and 14. 23 
`The combination of Shipman and Garrison teaches or suggests
`the storing limitation of claims 8 and 14. ................................ 27 
`The combination of Shipman and Garrison teaches or suggests
`the adjusting limitation of claims 1, 8, and 14. ........................ 28 
`The combination of Shipman and Garrison teaches or suggests
`the providing limitations of claims 1, 8, and 14. ..................... 35 
`The combination of Shipman and Garrison renders dependent claims 5
`and 9 obvious. ...................................................................................... 36 
`
`B. 
`C. 
`D. 
`
`E. 
`
`2. 
`
`3. 
`
`4. 
`
`5. 
`
`
`
`- ii -
`
`
`

`
`U.S. Patent No. 9,083,850 B1
`
`C. 
`
`D. 
`
`E. 
`
`IX.  Ground 2: Claims 2-4, 15-18 are unpatentable under 35 U.S.C. § 103 as
`obvious over Shipman in view of Garrison and Mayhew. ........................... 38 
`A. 
`Rationale for combining Mayhew with Shipman and Garrison. ........ 41 
`B. 
`The combination of Shipman, Garrison, and Mayhew renders
`dependent claims 2 and 15 obvious. ................................................... 43 
`The combination of Shipman, Garrison, and Mayhew renders
`dependent claims 3 and 17 obvious. ................................................... 45 
`The combination of Shipman, Garrison, and Mayhew renders
`dependent claims 4 and 18 obvious. ................................................... 47 
`The combination of Shipman, Garrison, and Mayhew renders
`dependent claim 16 obvious. ............................................................... 48 
`X.  Ground 3: Claims 6, 7, 10, 11, and 19 are unpatentable under 35 U.S.C.
`§ 103 as obvious over Shipman in view of Garrison and Gotsopoulos. ...... 50 
`A. 
`Rationale for combining Gotsopoulos with Shipman and Garrison. .. 54 
`B. 
`The combination of Shipman, Garrison, and Gotsopoulos renders
`dependent claims 6 and 10 obvious. ................................................... 56 
`The combination of Shipman, Garrison, and Gotsopoulos renders
`dependent claims 7 and 11 obvious. ................................................... 58 
`XI.  Ground 4: Claims 12 and 20 are unpatentable under 35 U.S.C. § 103 as
`obvious over Shipman in view of Garrison, Gotsopoulos, and Johnson. .... 59 
`A. 
`Rationale for combining Johnson with Shipman, Garrison, and
`Gotsopoulos. ........................................................................................ 61 
`The combination of Shipman, Garrison, Gotsopoulos, and Johnson
`renders dependent claims 12 and 20 obvious. ..................................... 62 
`XII.  Ground 5: Claims 13 and 21 are unpatentable under 35 U.S.C. § 103 as
`obvious over Shipman in view of Garrison and Johnson. ............................ 64 
`A. 
`The combination of Shipman, Garrison, and Johnson renders
`dependent claims 13 and 21 obvious. ................................................. 64 
`XIII.  Conclusion. ................................................................................................... 66 
`
`
`
`C. 
`
`B. 
`
`
`
`- iii -
`
`
`

`
`U.S. Patent No. 9,083,850 B1
`
`Exhibit List
`
`1005
`
`1006
`
`1007
`
`1008
`
`Exhibit No. Description
`1001
`U.S. Patent No. 9,083,850 B1 to Higgs
`1002
`Expert Declaration of Dr. Iain Richardson
`1003
`Expert CV of Dr. Iain Richardson
`1004
`U.S. Patent No. 9,106,789 to Shipman, Jr. et al. (“Shipman”), titled
`“Videoconference and Video Visitation Security”
`U.S. Patent No. 7,911,513 to Garrison et al. (“Garrison”), titled
`“Simulating Short Depth of Field to Maximize Privacy in
`Videotelophony”
`U.S. Patent No. 6,734,900 to Mayhew. (“Mayhew”), titled “Real
`Time Camera and Lens Control System for Image Depth of Field
`Manipulation”
`U.S. Patent Publication No. 2008/0201158 A1 to Johnson et al.
`(“Johnson”), titled “Real Time Camera and Lens Control System for
`Image Depth of Field Manipulation”
`“Remote Controlled DSP Based Image Capturing and Processing
`System Featuring Two-Axis Motion,” by Gotsopoulos et al.
`(“Gotsopoulos”)
`American Heritage Dictionary
`
`
`1009
`
`
`
`- iv -
`
`
`

`
`U.S. Patent No. 9,083,850 B1
`
`I.
`
`Introduction.
`
`Global Tel*Link Corporation petitions for inter partes review of claims 1–21
`
`of U.S. Patent No. 9,083,850 to Higgs, titled “Video Blurring in a Secure
`
`Environment” (hereinafter “the ’850 patent”)1. Petitioner Global Tel*Link
`
`Corporation will demonstrate that a reasonable likelihood exists that all 21 claims of
`
`the ’850 patent are unpatentable.
`
`The purported novelty of the ’850 patent is keeping an individual (or face of
`
`an individual) in focus while simultaneously blurring the background of a video by
`
`manipulating the depth of field of the camera image. But, such blurring techniques
`
`have existed for decades, and the concepts were well known to any person with even
`
`an ordinary knowledge of camera systems. (Richardson Decl., ¶ 72.) Indeed, these
`
`techniques disclosed in the ’850 patent for adjusting the depth of field in a camera
`
`system were disclosed in a U.S. Patent to Christopher Mayhew almost 15 years prior
`
`to the filing date of the ’850 patent.
`
`The ’850 patent merely claims the application of these known blurring
`
`techniques to the specific scenario of a video call involving a resident of a secure
`
`environment. But, such a combination would have been obvious to a person of
`
`ordinary skill in the art at the time of the purported invention. Indeed, applying
`
`
`1 The ’850 patent is provided as GTL 1001.
`
`- 1 -
`
`

`
`U.S. Patent No. 9,083,850 B1
`
`blurring techniques to an inmate video call was disclosed in a U.S. Patent to Bobby
`
`and Laura Shipman. Accordingly, Petitioner requests that the Board institute trial on
`
`the grounds set forth herein.
`
`II. Mandatory notices.
`
`A. Real parties-in-interest.
`
`The real party-in-interest of Petitioner is Global Tel*Link Corporation.
`
`B. Related matters.
`
`To Petitioner’s knowledge, the ’850 patent is not involved in any related
`
`matters that might affect, or be affected by, a decision in this proceeding.
`
`C. Lead and back-up counsel.
`
`Petitioners appoint Lori A. Gordon (Reg. No. 50,633) as its lead counsel,
`
`Steven M. Pappas (Reg. No. 73,904) as back up counsel, and Michael B. Ray
`
`(Reg. No. 33,997) and Michael D. Specht (Reg. No. 54,463) as additional back-up
`
`counsel, all at the address: STERNE, KESSLER, GOLDSTEIN & FOX, 1100 New York
`
`Avenue, N.W., Washington, D.C., 20005, phone no. (202) 371-2600, facsimile
`
`(202) 371-2540. Petitioners consent to service by email at: lgordon-ptab@skgf.com,
`
`spappas@skgf.com, mspecht-PTAB@skgf.com, mray-PTAB@skgf.com, and
`
`PTAB@skgf.com.
`
`III. Grounds for standing (37 C.F.R. § 42.104(a))
`
`Pursuant to 37 C.F.R. § 42.104(a), the undersigned and GTL certify that the
`
`‘850 patent is available for inter partes review and that GTL is not barred or
`
`
`
`- 2 -
`
`

`
`U.S. Patent No. 9,083,850 B1
`
`estopped from requesting this inter partes review on the grounds identified herein.
`
`IV. Statutory Grounds for the challenge.
`
`The ’850 patent was filed on June 29, 2013. The patent makes no priority
`
`claim, and therefore the earliest possible effective filing date of the ’850 patent is
`
`June 29, 2013.
`
`The grounds of unpatentability set forth herein apply the following prior art
`
`references:
`
`U.S. Patent No. 9,106,789 to Shipman, Jr. et al. (“Shipman”), titled
`
`“Videoconference and Video Visitation Security,” is at least prior art under 35
`
`U.S.C. § 102(a)(2) because it was filed on January 20, 2012, prior to the earliest
`
`priority date of the ’850 patent. Shipman is provided as Exhibit 1004.
`
`U.S. Patent No. 7,911,513 to Garrison et al. (“Garrison”), titled “Simulating
`
`Short Depth of Field to Maximize Privacy in Videotelophony,” is at least prior art
`
`under 35 U.S.C. § 102(a)(1) because it issued on March 22, 2011, prior to the
`
`earliest priority date of the ’850 patent. Garrison is provided as Exhibit 1005.
`
`U.S. Patent No. 6,734,900 to Mayhew. (“Mayhew”), titled “Real Time
`
`Camera and Lens Control System for Image Depth of Field Manipulation,” is at
`
`least prior art under 35 U.S.C. § 102(a)(1) because it issued on May 11, 2004, prior
`
`to the earliest priority date of the ’850 patent. Mayhew is provided as Exhibit 1006.
`
`U.S. Patent Publication No. 2008/0201158 A1 to Johnson et al.
`
`
`
`- 3 -
`
`

`
`U.S. Patent No. 9,083,850 B1
`
`(“Johnson”), titled “Real Time Camera and Lens Control System for Image Depth
`
`of Field Manipulation,” is at least prior art under 35 U.S.C. § 102(a)(1) because it
`
`published on August 21, 2008, prior to the earliest priority date of the ’850 patent.
`
`Johnson is provided as Exhibit 1007.
`
`“Remote Controlled DSP Based Image Capturing and Processing System
`
`Featuring Two-Axis Motion,” by Gotsopoulos et al. (“Gotsopoulos”) is at least
`
`prior art under 35 U.S.C. 102(a)(1) because it published at least of December 1,
`
`2010, prior to the earliest priority date of the ’850 patent. Gotsopoulos is provided as
`
`Exhibit 1008.
`
`As detailed below, GTL requests inter partes review of claims 1-21 on five
`
`grounds:
`
`GROUND 1: Claims 1, 5, 8, 9, and 14 are unpatentable under 35 U.S.C.
`
`§ 103 as obvious over Shipman in view of Garrison.
`
`GROUND 2: Claims 2-4 and 15-18 are unpatentable under 35 U.S.C. § 103
`
`as obvious over Shipman in view of Garrison and Mayhew.
`
`GROUND 3: Claims 6, 7, 10, 11, and 19 are unpatentable under 35 U.S.C.
`
`§ 103 as obvious over Shipman in view of Garrison and Gotsopoulos.
`
`GROUND 4: Claims 12 and 20 are unpatentable under 35 U.S.C. § 103 as
`
`obvious over Shipman in view of Garrison, Gotsopoulos, and Johnson.
`
`
`
`
`
`- 4 -
`
`

`
`U.S. Patent No. 9,083,850 B1
`
`GROUND 5: Claims 13 and 21 are unpatentable under 35 U.S.C. § 103 as
`
`obvious over Shipman in view of Garrison and Johnson.
`
`V. Overview of the ’850 patent.
`
`Correctional institutions implemented video visitation techniques prior to the
`
`filing of the ’850 patent. However, according to the ’850 patent, “[t]he introduction
`
`of video services into inmate housing and common area locations for the purposes
`
`of video visitation and/or video calling has introduced an unanticipated issue related
`
`to what details are conveyed through the video to whoever is viewing it at the other
`
`end.” (’850 patent, 1:7-11.) For example, individuals in the background of the video
`
`acted up or caused disruptions. (’850 patent, 1:16-18.) In addition, the video may
`
`also capture inappropriate or private background details such as showers,
`
`bathrooms, or interiors of cells. (’850 patent, 1:14-16.)
`
`The ’850 patent sought to address these concerns through “methods and
`
`systems for video blurring in a secure environment.” (’850 patent, 2:51–55.) To blur
`
`the video, the ’850 patent “adjust[s] a depth of field parameter” for received video
`
`“such that an image of a first object at a first distance from the video visitation
`
`device is in focus and an image of a second object at a second distance from the
`
`video visitation device is blurred.” (’850 patent, 1:53-60.) In various embodiments,
`
`adjusting the depth of field parameter may involve “adjusting the f-stop setting of
`
`[the] camera associated with [a] video visitation device.” (’850 patent, 1:61-63.)
`
`
`
`- 5 -
`
`

`
`U.S. Patent No. 9,083,850 B1
`
`And, “adjusting the f-stop settings may further include adjusting a focal length of a
`
`lens coupled to the video visitation device” or “adjusting an aperture setting of the
`
`camera associated with the video visitation device.” (’850 patent, 1:61-67.)
`
`Alternatively, the ’850 patent teaches that the depth of field may be manipulated
`
`“through digital processing techniques.” (’850 patent, 7:44-46.) The ’850 patent
`
`does not indicate or limit the specific digital processing techniques employed to
`
`achieve blurring when using digital processing techniques. (Richardson Decl., ¶ 32.)
`
`The blurring techniques described in the ’850 patent may be used in
`
`conjunction with video visitation devices “to enable inmates to participate in video
`
`visitation sessions with non-residents of the correctional facility via video call.”
`
`(’850 patent, 4:59-65.) An example of a video visitation device 103 is illustrated in
`
`Figure 2 of the ’850 patent (reproduced below).
`
`
`
`- 6 -
`
`

`
`U.S. Patent No. 9,083,850 B1
`
`
`
`The video visitation device of the ’850 patent “includes display 204, camera
`
`205, and handset 201 coupled to device 103 via wire 209.” (’850 patent, 6:14-15.)
`
`The ”camera 205 may be any suitable imaging device such as, for instance, a video
`
`camera or webcam equipped with Charge-Coupled Devices (CCDs),
`
`Complementary Metal-Oxide-Semiconductor (CMOS) active pixel sensors, etc.”
`
`(’850 patent, 6:14-22.)
`
`Figure 9 of the ’850 patent (reproduced below) illustrates “a blurred video
`
`frame 900.” In the frame, the first object 304 may be a face 901. The second object
`
`305 may be anything located at a predefined distance from face 901, for example in
`
`the background 902 which is blurred.” (’850 patent, 11:65-12:1.) The ’850 patent
`
`explains that “a second inmate in the background 902 may have blurred features to
`
`
`
`- 7 -
`
`

`
`
`
`maintainn the privaacy of the ssecond inmmate 903.”
`
`
`
`
`
`
`
`
`
`VI. CClaim consstruction.
`
`
`
`
`
`U.SS. Patent NNo. 9,083,8550 B1
`
`
`
`
`
`
`
`(’850 pateent, 12:14-
`
`16.)
`
`
`
`
`
`PPetitioner hhas accordeed terms thheir plain aand ordinarry meaningg as undersstood
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`by a perrson of orddinary skilll in the art
`
`
`
`
`
`
`
`and consisstent with tthe disclossure. No exxplicit
`
`
`
`
`
`
`
`
`
`construcctions are nnecessary tto determinne the pateentability oof the claimms of the ’8850
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`patent.
`
`TThe claim tterms of th
`
`
`
`
`
`
`
`
`
`e ’850 pateent are to bbe given thheir broadeest reasonabble
`
`
`
`
`
`
`
`
`
`interpreetation, as uunderstoodd by a person of ordinnary skill iin the art.
`
`
`
`
`
`
`
`
`
`
`
`
`
`VII. LLevel of orrdinary skkill.
`
`
`
`
`
`
`
`OOne of ordiinary skill in the art wwould havee a Bacheloor’s (B.S.)
`
`
`
`
`
`
`
`
`
`
`
` degree in
`
`
`
`Computter Sciencee, Electronnic Engineeering or ann equivalennt field, toggether withh at
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`least twwo years of f academic or industryy experien
`
`
`
`
`
`
`
`
`
`ce in a releevant fieldd, or a Mastter’s
`
`
`
`
`
`
`
`(M.S.) ddegree in CComputer SScience, Electronic EEngineeringg or an equuivalent fieeld,
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`- 8 -
`
`

`
`U.S. Patent No. 9,083,850 B1
`
`together with at least one year of academic or industry experience in a relevant field.
`
`A relevant field could include data communications, image or video processing or
`
`communications, surveillance system design or similar. (Richard Decl., ¶ 24.)
`
`VIII. Ground 1: Claims 1, 5, 8, 9, and 14 are unpatentable under 35 U.S.C.
`§ 103 as obvious over Shipman in view of Garrison.
`
`A.
`
`Shipman discloses a video visitation system using blurring
`techniques to obscure the background of the video.
`
`Similar to the ’850 patent, Shipman is directed to “providing video visitation
`
`security to a controlled-environment facility.” (Shipman, Abstract.) Shipman
`
`specifically discloses techniques for detecting an image of a face in a video
`
`transmitted between an inmate in a secure environment and a non-resident of the
`
`secure environment. (Shipman, 1:43-52.) Other areas of the video outside of the
`
`detected face may then be obscured using techniques such as blurring, clouding,
`
`darkening, distorting, masking, or shading. (Shipman, 1:52-57.) Figure A of the
`
`Richardson Declaration (reproduced below) annotates Figure 1 of Shipman to
`
`highlight the components involved in a video visitation session. (Shipman, 4:58-60.)
`
`
`
`- 9 -
`
`

`
`
`
`
`
`
`
`U.SS. Patent NNo. 9,083,8550 B1
`
`
`
`
`
`Ricchardson Annotatedd Figure AA
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Inn Shipmann, inmates ccan make oor receive vvoice callss via commmunication
`
`
`
`
`
`
`
`system 110 using traditionall telephonee device 1115. (Shipmman, 5:8-100.) In this
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`scenarioo, calls cann be complleted over ppublic swittched telepphone netwwork (PSTNN)
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`120 to aa non-residdent outside of the priison. (Shippman, 5:133-17.) Howwever, in
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Shipmaan, inmatess also have the abilityy to commuunicate witth non-resiidents via
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`:63-67.) Foor examplee, an inmatte may usee a video
`
`
`
`
`
`video. ((Shipman, 5:32-38, 5
`
`
`
`
`
`
`
`
`
`visitatioon station tto communnicate withh a non-resiident operaating a devvice havingg
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`video caapabilities such as coommunicattion devicees 135A-NN. (Id.) The
`
`
`
`
`
`
`
`
`
`
`
`video
`
`
`
`
`
`
`
`
`
`
`
`communnication seession betwween the innmate and nnon-resideent also occcurs via
`
`
`
`
`
`
`
`communnication syystem 110 but in this case over
`
`
`
`
`
`
`
`
`
`packet-swwitched netwwork 140.
`
`
`
`son Decl.,
`(Shipmaan, 5:35-400; Richard
`
`
`
`
`
`
`
`
`¶ 36.) Thee non-residdent can vieew the inmmate’s
`
`
`
`
`
`
`
`video feeed via his or her devvice 135. (RRichardsonn Decl., ¶ 336.)
`
`
`
`
`
`
`
`
`
`
`
`
`
`- 10 -
`
`

`
`
`
`
`
`
`
`U.SS. Patent NNo. 9,083,8550 B1
`
`
`
`AAn inmate ccan use a vvideo visitaation devicce in a videeo visitatioon area witthin
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`the corrrectional faacility. An example laayout of viideo visitattion area 1
`
`
`
`
`
`
`
`
`
`
`
`
`
`in Figurre 2 of Shippman (reprroduced beelow). (Shiipman, 5:558-60.)
`
`
`
`
`
`
`
`
`
`
`
`
`
`30 is depiccted
`
`
`
`
`
`
`
`
`
`TThe video vvisitation aarea has onne or more
`
`
`
`
`
`Each video visitattion stationn 200A-N iincludes a
`
`
`
`
`
`
`
`
`
`and/or mmicrophonne 215, andd a speaker or audio ooutput 220
`
`
`
`
`
`
`
`
`
`. (Shipmann, 6:4-8.) WWhen
`
`
`
`separate v
`
`
`
`ideo visitaation stationns.
`
`
`
`display or
`
`
`
`screen 2100, a camer
`
`a
`
`
`
`
`
`
`
`
`
`
`
`
`
`an inmaate arrives in video viisitation arrea 130, thee inmate ooccupies annd is
`
`
`
`
`
`
`
`authentiicated at onne of the vvideo visitaation statioons 200A-NN. (Shipmaan, 6:20-300.)
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`The commmunicatioon system 110, descrribed abovee, then estaablishes a
`
`
`
`
`
`
`
`
`
`
`
`
`
`betweenn the inmatte’s video visitation sstation 2000 and a nonn-resident’
`
`
`
`
`
`video sess
`
`ion
`
`s device.
`
`
`
`son Declarration
`
`
`
`
`
`
`
`
`
`(Shipmaan, 6:31-344; Richardson Decl., ¶ 38.) Figuure B of thhe Richard
`
`
`
`
`
`
`
`
`
`
`
`
`
`(reproduuced beloww) depicts vvideo visittation statioon 200A inntegrated wwith the viddeo
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`- 11 -
`
`

`
`
`
`visitatioon securityy system illlustrated inn Figure 1 oof Shipmaan.
`
`
`
`
`
`
`
`
`
`
`
`
`
`U.SS. Patent NNo. 9,083,8550 B1
`
`
`
`
`
`
`
`Ricchardson Annotatedd Figure BB
`
`
`
`
`
`
`
`
`
`TThe video vvisitation system is d
`
`
`
`
`
`epicted in
`
`
`
`further dettail in Figuure 3 of
`
`
`
`
`
`
`
`Shipmaan (reproduuced beloww). The systtem may innclude a coonnection
`
`
`
`
`
`
`
`
`
`
`
`
`
`manager 3310, a
`
`
`
`media sserver 315, a securityy engine 3220, an inmaate client 3325 and a ffamily cliennt
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`330. Shhipman describes thatt inmate video visitattion client
`
`
`
`
`
`(or residennt client) 3325 is
`
`
`
`
`
`
`
`
`
`a compoonent of viideo visitattion stationn 200A. (RRichardsonn Decl., ¶ 4
`
`
`
`2.) The fammily
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`client reesides withhin the viewwing devicce of the noon-residentt. The connnection
`
`
`
`
`
`
`
`manageer 310 withhin the secuure facilityy acts to maaintain a coonnection
`
`
`
`
`
`
`
`
`
`
`
`
`
`between twwo
`
`
`
`endpoinnts such as “inmate video visitaation clientt resident cclient 325 ((e.g., statioon
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`200A) aand family client 3300 (e.g., dev
`
`
`
`
`
`
`
`ice 135A)..” (Shipmaan, 6:50-555; Richardsson
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Decl., ¶¶ 40.) Shipmman descriibes that thhe media seerver 315,
`
`
`
`
`
`security enngine 325,, and
`
`
`
`
`
`connecttion managger 310, as well as ru
`
`
`
`
`
`
`
`les and useer databasees, may be
`
`
`
`
`
` implemennted
`
`
`
`- 12 -
`
`

`
`
`
`
`
`
`
`
`
`in commmunicationns system 1110. (Shipmman, 6:48--50.)
`
`
`
`
`
`
`
`
`
`U.SS. Patent NNo. 9,083,8550 B1
`
`
`
`FFigure C off the Richaardson Declaration (reeproducedd below) in
`
`
`
`
`
`
`
`
`
`
`
`
`
`video viisitation syystem of Fiigure 3 intoo these commponents ((depicted iin Figures
`
`
`
`
`
`
`
`
`
`
`
`tegrates thhe
`
`1 and
`
`
`
`
`
`
`
`
`
`2) as dirrected by SShipman .
`
`
`
`
`
`- 13 -
`
`

`
`U.S. Patent No. 9,083,850 B1
`
`Communication System 110
`
`Camera
`215
`
`INMATE VIDEO
`
`FAMILY VIDEO
`
`SECURE FAMILY VIDEO
`
`SECURE INMATE VIDEO
`
`Video visitation station
`200A
`
`Network
`140
`
`FAMILY
`VIDEO
`
`SECURE
`INMATE
`VIDEO
`
`Non-resident device
`135
`
`
`
`Richardson Annotated Figure C
`
`During a video visitation session, resident client 325 in the video visitation
`
`device provides non-secure “inmate video” to media server 315 which then forwards
`
`the received non-secure video to security engine 320. (Shipman, 6:48-50.) Security
`
`engine 320 processes the non-secure video. For example, the ’850 patent, Shipman
`
`recognizes that inappropriate activities or private areas may be captured during a
`
`video session. (See, e.g., Shipman, 11:18-20 and Figures 6A-6E.) Therefore, for
`
`security and/or privacy reasons, the security module of Shipman executes a facial
`
`detection operation and blurs areas of the video feed outside of the detected face.
`
`(Shipman, 11:9-16; Richardson Decl., ¶ 41.) Shipman refers to this modified video
`
`
`
`- 14 -
`
`

`
`
`
`
`
`
`
`
`
`U.SS. Patent NNo. 9,083,8550 B1
`
`
`
`
`
`
`
`secure viddeo to
`
`
`
`as the “secure video.” The seecurity enggine 320 reeturns the pprocessed
`
`
`
`
`
`
`
`the meddia server wwhich in tuurn transmmits this “seecure inmaate video” tto the famiily
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`client. AA non-secuure video frfrom a non--resident iss processedd in the samme mannerr.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`(Shipmaan, 8:35-400.)
`
`
`
`
`
`
`
`
`
`
`
`FFigures 5A and 6A off Shipman (reproduceed below),
`
`
`
` depict an
`
`
`
`exemplaryy
`
`
`
`“secure video.” (SShipman, FFigures 5A
`
`
`
`
`
`
`
`and 6A; RRichardsonn Decl., ¶ 444–45.) In tthis
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`video, thhe face of a non-residdent is detected. (Shiipman, 11::8-11.) Thee “video
`
`
`
`
`
`
`
`securityy module 320 may bluur, cloud, ddarken, disstort, maskk, shade or
`
`
`
`
`
`
`
`otherwise
`
`
`
`
`
`
`
`
`
`
`
`
`
`deface oother areass 520A.” (SShipman, 111:8–15; RRichardson
`
`
`
`
`
`Decl., ¶ 4
`
`5.)
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`gnals ia hand sigRRecognizinng that indivviduals maay try to coonvey inforrmation vi
`
`
`
`
`
`
`
`or the inntroductionn of objectts, Shipmann also incluudes the abbility to deeface or bluur
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`other seelected regiions in the video. Th
`
`
`
`
`
`
`
`is is depictted in Figuure 6A (repproduced
`
`
`
`
`
`
`
`below) that defacees the backkground off the video
`
`
`
`
`
`
`
`as well as
`
`
`
`the objectt 615A
`
`
`
`introducced in the fframe whicch overlaps face 605AA. (Shipmman, 11:62--67.) Shipmman is
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`thus ablle to detectt images wwithin a viddeo feed thaat are permmitted to bee transmittted
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`- 15 -
`
`

`
`U.S. Patent No. 9,083,850 B1
`
`between residents and non-residents of a secure environment, while blurring or
`
`defacing areas outside of the permitted images. (Richardson Decl., ¶ 45.)
`
`
`
`Although Shipman discloses blurring portions of a video feed, Shipman does
`
`not explicitly describe how the blurring is achieved. Garrison provides these details.
`
`B. Garrison utilizes depth of field manipulation to blur an image.
`
`Garrison, like the ’850 patent and Shipman, recognizes that a video session
`
`may capture inappropriate or private scenes. Therefore to address security and
`
`privacy concerns, Garrison segregates an object in the foreground (e.g., a user) of an
`
`image and blurs the background to render it indistinct. In this way, the “displayed
`
`video of a user in the foreground is kept in focus while the background appears to be
`
`out of focus.” (Garrison, Abstract.) Garrison achieves blurring using depth of field
`
`manipulation.
`
`Specifically, Garrison employs techniques for “simulating depth of field
`
`effects in a video image.” (Garrison, 10:20-21.) The videophone of Garrison
`
`includes a camera 514 having a lens oriented towards the videophone user.
`
`(Garrison, 4:26-28.) The camera first captures a video image “having a long or
`
`substantially infinite depth of field.” (Garrison, 10:23-25.) The captured video
`
`image is then “spatially segregated into a target portion for which focus is
`
`maintained and a remaining portion for which blurring is applied” using image
`
`processing techniques. (Garrison, 10:27-30; see also id. at 5:34-48.) The blurred
`
`
`
`- 16 -
`
`

`
`U.S. Patent No. 9,083,850 B1
`
`remaining portion is combined with the unblurred target portion to produce a
`
`composite image in which only the target portion of the image appears in focus.
`
`(Garrison, 10:34-41; Richardson Decl., ¶ 48.) To determine the target portion for an
`
`image, object detection techniques are employed in which a specific feature, such as
`
`a user’s face, head, and shoulders, are dynamically detected and tracked as the user
`
`moves. (Garrison, 5:39-43; Richardson Decl., ¶ 48.)
`
`C. Rationale for combining Shipman and Garrison.
`
`A person of ordinary skill would be motivated to combine Shipman with
`
`Garrison for several reasons. First, a person or ordinary skill would have looked to
`
`Garrison’s blurring technique that simulates short depth of field blurring function of
`
`Shipman. (Richardson Decl., ¶ 75.) Second, a person of ordinary skill would have
`
`looked to Garrison to improve the facial recognition image processing techniques
`
`used in Shipman. (Id.) For example, the object detection techniques disclosed in
`
`Garrison enable any object detected in an image to act as a “target portion” of the
`
`image, rather than limiting the target portion to a detected face as in Shipman.
`
`(Richardson Decl., ¶ 75; Garrison, 5:34-43.) Areas outside of the target portion may
`
`then be blurred as described in both Shipman and Garrison to address privacy
`
`concerns. (Richardson Decl., ¶ 75.) Finally, both Shipman and Garrison are in the
`
`same field––video processing to enhance security and privacy.
`
` Furthermore, it would have been obvious to one of ordinary skill in the art to
`
`
`
`- 17 -
`
`

`
`U.S. Patent No. 9,083,850 B1
`
`incorporate the processing techniques of Garrison that identify a target area of the
`
`user’s video feed and “blur … other areas … in the video feed” into the security
`
`module of Shipman because Shipman’s security module already performs
`
`identification and blurring. (Richardson Decl., ¶ 76; Shipman, 11:12-16.)
`
`Petitioner’s expert, Dr. Richardson, explains that a core benefit of using Garrison’s
`
`technique is that the component architecture described in Shipman need not be
`
`modified to incorporate the image processing techniques of Garrison. (Richardson
`
`Decl., ¶ 77.) Therefore, the combination of Shipman and Garrison would have been
`
`nothing more than combining prior art elements according to known methods to
`
`yield the predictable and desirable result of providing a blurring technique in video
`
`communication between residents and non-residents of a secure environment.
`
`(Richardson Decl., ¶ 77.)
`
`D. The combination of Shipman and Garrison renders independent
`claims 1, 8, and 14 obvious.
`
`Independent claims 1, 8, and 14 are directed to a method, tangible computer
`
`readable medium, and system, respectively. Each of these claims includes
`
`substantially overlapping limitations as highlighted in the following table. For
`
`example, each claim includes a receiving limitation, the same adjusting limitation,
`
`and the same providing limitation. In addition, both claims 8 and 14 including the
`
`same storing limitation.
`
`
`
`
`
`- 18 -
`
`

`
`U.S. Patent No. 9,083,850 B1
`
`Claim 14
`A system, comprising:
`
`Claim 8
`Claim 1
`
`[P] A method, comprising: A tangible computer
`readable medium
`comprising computer
`executable code that,
`when executed by a
`processing device,
`causes the processing
`device to perform
`operation comprising:
`
`
`[A]
`
`
`
`a data communication
`device configured to
`receiv[ing/e] video from a video visitation device
`in a secure environment
`[receiving from limitation]
`
`
`
`a data storage device
`coupled to the data
`communication device
`and configured to
`stor[ing/e] the video received from the video
`visitation device in a data storage device;
`[storing limitation]
`
`
`receiving video at a
`video visitation device
`in a secure
`environment
`[receiving at
`limitation]
`
`
`[S]
`
`[B]
`
`
`
`
`
`
`
`data processor
`configured to
`adjust[ing] a depth of field parameter for the video, such that an image of a
`first object at a first distance from the video visitation device is in focus and
`an image of a second object at a second distance from the video visitation
`device is blurred; and
`[adjusting limitation]
`
`
`- 19 -
`
`

`
`U.S. Patent No. 9,083,850 B1
`
`
`
`
`
`wherein the data
`communication device
`is configured to provide
`provid[ing/e] the [adjusted] video to a viewing device located outside of the
`secure environment.
`[providing limitation]
`
`[C]
`
`
`
`1.
`
`The combination of Shipman and Garrison teaches or
`suggests the preamble of independent claims 1, 8, and 14.
`
`The combination of Shipman and Garrison teaches or suggests a “method”2
`
`[1.P] and “system” [14.P] performing the steps recited in claim 1 or configured to
`
`perform the operations set forth in claim 14. For example, Shipman discloses
`
`“systems and methods for providing video visitation security to controlled-
`
`environment facilities.” (Shipman, 4:14-16.) Shipman’s “video visitation security
`
`method” applies video processing techniques (e.g., blurring) to select areas of a
`
`video image to enhance security and privacy of the video communications.
`
`(Shipman, 13:66-14:40; Richardson Decl., ¶ 79.) Like Shipman, Garrison describes
`
`a video communication “arrangement 400 [the recited “system”] in which two
`
`videophone users are engaged in a video telephony session.” (Garrison, 4:7-9.)
`
`Specifically, “videophone 408 in home 413” which is “coupled over a network 418
`
`to videophone 426 used by user 430 in home 435.” (Garrison, 4:9-11.) Garrison also
`
`
`2 For ease of discussion, claim language is indicated by italics.
`
`
`
`- 20 -
`
`

`
`U.S. Patent No. 9,083,850 B1
`
`enhances security and privacy of the video “simulating depth of field effects in a
`
`video image” in order to blur portions of the video and describes that its method
`
`“may be utilized by either videophone 408 or 426 in FIG. 4.” (Garrison, 10:20-23.)
`
`The combination of Shipman and Garrison also teaches or suggests a
`
`“tangible computer rea