U.S. Patent No. 9,083,850
`
`
`
`
`
`
`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`_____________________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`_____________________
`
`GLOBAL TEL*LINK CORPORATION
`Petitioner
`
`v.
`
`SECURUS TECHNOLOGIES, INC.
`Patent Owner
`____________________
`
`Patent No. 9,083,850 B1
`_____________________
`
`
`
`DECLARATION OF DR. IAIN RICHARDSON
`IN SUPPORT OF PETITION FOR
`INTER PARTES REVIEW OF
`U.S. PATENT NO. 9,083,850
`
`
`
`
`
`GTL 1002
`
`
`
`
`
`
`
`
`
`UNITED STATES PATENT AND TRADEMARK OFFICE
`_____________________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`_____________________
`
`GLOBAL TEL*LINK CORPORATION
`Petitioner
`
`v.
`
`SECURUS TECHNOLOGIES, INC.
`Patent Owner
`____________________
`
`Patent No. 9,083,850 B1
`_____________________
`
`
`
`DECLARATION OF DR. IAIN RICHARDSON
`IN SUPPORT OF PETITION FOR
`INTER PARTES REVIEW OF
`U.S. PATENT NO. 9,083,850
`
`
`
`
`
`GTL 1002
`
`
`
`
`
`
`I, Iain Richardson, PhD., declare as follows:
`
`- 2 -
`
`U.S. Patent No. 9,083,850
`
`1.
`
`I have been retained on behalf of Global Tel*Link Corporation
`
`(“GTL”) for the above-captioned inter partes review proceeding. I understand that
`
`this proceeding involves U.S. Patent No. 9,083,850 (“’850 patent”), titled “Video
`
`Blurring In A Secure Environment.”
`
`2.
`
`I have reviewed and am familiar with the specification of the ‘850
`
`patent issued on July 14, 2015. I will cite to the specification using the following
`
`format: (the ‘850 patent, 1:1–10). This example citation points to the ‘850 patent
`
`specification at column 1, lines 1–10.
`
`3.
`
`I have also reviewed and am familiar with the following prior art:
`
` U.S. Patent No. 9,106,789 to Shipman, Jr. et al. (“Shipman”). I understand
`
`that Shipman has been provided as Exhibit 1004.
`
` U.S. Patent No. 7,911,513 to Garrison et al. (“Garrison”). I understand that
`
`Garrison has been provided as Exhibit 1005.
`
` U.S. Patent No. 6,734,900 to Mayhew (“Mayhew”). I understand that
`
`Mayhew has been provided as Exhibit 1006.
`
` “Remote Controlled DSP Based Image Capturing and Processing System
`
`Featuring Two-Axis Motion,” by Gotsopoulos et al., (“Gotsopoulos”). I
`
`understand that Gotsopoulos has been provided as exhibit 1008.
`
`
`
`
`
`
`
`
`
`- 3 -
`
`U.S. Patent No. 9,083,850
`
` U.S. Patent Publication No. 2008/0201158 A1 to Johnson et al. (“Johnson”).
`
`I understand that Johnson has been provided as Exhibit 1007.
`
`4.
`
`I have also reviewed and am familiar with the other materials cited or
`
`referred to in this declaration.
`
`5.
`
` I have relied on these materials to varying degrees. Citations to these
`
`materials that appear below are meant to be exemplary but not exhaustive.
`
`6.
`
`I am familiar with the technology at issue as of the June 29, 2013
`
`filing date of the ‘850 patent, which I also understand is the earliest possible
`
`priority date.
`
`7.
`
`I have been asked to provide my technical review, analysis, insights
`
`and opinions regarding the above-noted references that form the basis for the
`
`grounds of rejection set forth in the Petition for Inter Partes Review of U.S. Patent
`
`No. 9,083,850.
`
`I.
`
`Qualifications.
`
`8.
`
`I am an expert in video and image compression, decompression and
`
`transmission. I am the author of four books and over 70 journal and conference
`
`papers on image and video coding and communications, including two widely
`
`cited books on the H.264 / MPEG-4 industry standards for video compression. I
`
`
`
`
`
`
`
`
`have specific expertise in the area of video telephony and conferencing systems,
`
`- 4 -
`
`U.S. Patent No. 9,083,850
`
`including the use of filtering or blurring in such systems.
`
`9.
`
`I received an M.Eng. in Electronic and Electrical Engineering from
`
`Heriot-Watt University in Edinburgh in 1990 and a Ph.D. in Video Compression
`
`from Robert Gordon University in 1999. I worked as a Digital Signal Processing
`
`(“DSP’) Hardware Designer with GEC Avionics Ltd. from 1990 to 1993. In 1993,
`
`I took up a post as a Lecturer, then Reader, and eventually a Full Professor in the
`
`field of image and video compression in the School of Engineering at The Robert
`
`Gordon University and in 2009 was honored as an Honorary Professor with the
`
`Robert Gordon University, a position I maintain to this day.
`
`10. At Robert Gordon University, I founded and ran an image
`
`communication technology research laboratory. I carried out original research in
`
`the field of data, image, and video compression, initiated and managed research
`
`projects, and supervised research students.
`
`11. A particular focus of my research at the Robert Gordon University has
`
`been the efficient communication of visual information via videoconference and
`
`videotelephone systems. I have written a number of scientific papers on this topic.
`
`My research team and I investigated visual communications for deaf people and
`
`developed videoconferencing technology that improved sign language
`
`
`
`
`
`
`
`
`communication by selectively blurring the background in a video scene and
`
`- 5 -
`
`U.S. Patent No. 9,083,850
`
`providing the video to a remotely located viewer (Muir and Richardson,
`
`“Perception of Sign Language and its Application to Visual Communications for
`
`Deaf People”, Journal of Deaf Studies and Deaf Education, September 2005). With
`
`my colleague Ying Zhong, I investigated the effect of blurring or increasing the
`
`compression of background and foreground regions in video images and the effect
`
`on compressed video transmission (Zhong, Richardson et al, “Influence of Task
`
`and Scene Content on Subjective Video Quality”, Lecture Notes in Computer
`
`Science, 2004). I investigated the application of videoconferencing and remote
`
`video monitoring in healthcare facilities and restricted working environments such
`
`as offshore installations as part of a major EU research project (Richardson et al,
`
`“Telemedicine and Teleconferencing: the SAVIOUR project”, IEE Computing and
`
`Control Journal, January 1996).
`
`12.
`
`In 1995, during my employment at the Robert Gordon University, I
`
`co-founded and became Technical Director, equivalent of Chief Technology
`
`Officer, at 4i2i Communications Limited. I held this position until 2000. In my role
`
`at 4i2i, I was responsible for and engaged in the design of software and hardware
`
`modules for compression and communication systems, including the design of
`
`
`
`
`
`
`
`
`video processing algorithms in software and hardware for implementation on
`
`- 6 -
`
`U.S. Patent No. 9,083,850
`
`processing devices.
`
`13.
`
`In 2007, I founded Vcodex Limited and currently serve as the CEO,
`
`wherein I provide technical and strategic advice to technology companies
`
`particularly relating to video, image, and data compression for streaming.
`
`14.
`
`In 2009, I founded Onecodec Limited and served as CEO, leading the
`
`company’s development of innovative video, image, and data compression and
`
`storage software and systems. Onecodec’s technology and business activities were
`
`merged with Vcodex Limited in 2015.
`
`15. My Curriculum Vitae is attached as Exhibit 1003, which contains
`
`further details on my education, experience, publications and other qualifications
`
`to render an expert opinion. My work on this case is being billed at a rate of $600
`
`per hour, with reimbursement for actual expenses. My compensation is not
`
`contingent upon the outcome of this inter partes review.
`
`II. My understanding of claim construction.
`
`16.
`
`I understand that, during an inter partes review, claims are to be given
`
`their broadest reasonable construction in light of the specification as would be read
`
`by a person of ordinary skill in the relevant art at the time of the invention. For the
`
`
`
`
`
`
`
`
`purpose of this proceeding, I have used June 2013 as the approximate time of the
`
`- 7 -
`
`U.S. Patent No. 9,083,850
`
`invention.
`
`III. My understanding of obviousness.
`
`17.
`
`I have been advised certain legal standards are to be applied by
`
`technical experts in forming opinions regarding meaning and validity of patent
`
`claims.
`
`18.
`
`I understand that a patent claim is invalid if the claimed invention
`
`would have been obvious to a person of ordinary skill in the field at the time of the
`
`purported invention, which is often considered the time the application was filed.
`
`This means that even if all of the requirements of the claim cannot be found in a
`
`single prior art reference that would anticipate the claim, the claim can still be
`
`invalid.
`
`19. As part of this inquiry, I have been asked to consider the level of
`
`ordinary skill in the field that someone would have had at the time the claimed
`
`invention was made. In deciding the level of ordinary skill, I considered the
`
`following:
`
` the levels of education and experience of persons working in the field;
`
` the types of problems encountered in the field; and
`
` the sophistication of the technology.
`
`
`
`
`
`
`
`
`
`20. To obtain a patent, a claimed invention must have, as of the priority
`
`- 8 -
`
`U.S. Patent No. 9,083,850
`
`date, been nonobvious in view of the prior art in the field. I understand that an
`
`invention is obvious when the differences between the subject matter sought to be
`
`patented and the prior art are such that the subject matter as a whole would have
`
`been obvious at the time the invention was made to a person having ordinary skill
`
`in the art.
`
`21.
`
`I understand that to prove that prior art or a combination of prior art
`
`renders a patent obvious, it is necessary to: (1) identify the particular references
`
`that, singly or in combination, make the patent obvious; (2) specifically identify
`
`which elements of the patent claim appear in each of the asserted references; and
`
`(3) explain how the prior art references could have been combined in order to
`
`create the inventions claimed in the asserted claim.
`
`22.
`
`I also understand that when considering the obviousness of a patent
`
`claim, one should consider whether a teaching, suggestion, or motivation to
`
`combine the references exist so as to avoid impermissibly applying hindsight when
`
`considering the prior art. I understand this test should not be rigidly applied, but
`
`that the test can be important to avoid such hindsight.
`
`23.
`
`I understand that certain objective indicia can be important evidence
`
`regarding whether a patent is obvious or nonobvious. Such indicia include:
`
`
`
`
`
`
`
`
`commercial success of products covered by the patent claims; a long-felt need for
`
`- 9 -
`
`U.S. Patent No. 9,083,850
`
`the invention; failed attempts by others to make the invention; copying of the
`
`invention by others in the field; unexpected results achieved by the invention as
`
`compared to the closest prior art; praise of the invention by the infringer or others
`
`in the field; the taking of licenses under the patent by others; expressions of
`
`surprise by experts and those skilled in the art at the making of the invention; and
`
`the patentee proceeded contrary to the accepted wisdom of the prior art. I am not
`
`aware of any objective indicia that would impact my analysis set forth herein.
`
`IV. Level of ordinary skill in the art.
`
`24. Based on the technologies disclosed in the ‘850 patent, one of
`
`ordinary skill in the art would have a Bachelor’s (B.S.) degree in Computer
`
`Science, Electronic Engineering or an equivalent field, together with at least two
`
`years of academic or industry experience in a relevant field, or a Master’s (M.S.)
`
`degree in Computer Science, Electronic Engineering or an equivalent field,
`
`together with at least one year of academic or industry experience in a relevant
`
`field. A relevant field could include data communications, image or video
`
`processing or communications, surveillance system design or similar.
`
`This description is approximate, and a higher level of education or skill might
`
`make up for less experience, and vice versa.
`
`
`
`
`
`
`
`
`V. The ‘850 patent claims.
`
`- 10 -
`
`U.S. Patent No. 9,083,850
`
`25.
`
`I understand that the ‘850 patent has not previously been the subject
`
`of an inter partes review proceeding, post grant review proceeding, or a covered
`
`business method review proceeding before the U.S. Patent and Trademark Office’s
`
`Patent Trial and Appeal Board (the “Board”).
`
`26.
`
`I understand that the ‘850 patent was originally filed with 21 claims,
`
`none of which were amended prior to issuance of the ’850 patent. The claims of
`
`the ’850 patent use terms that are known in the field such as focal length, aperture,
`
`f-stop, and depth of field. Thus, I applied the plain and ordinary meaning of these
`
`terms under the broadest reasonable interpretation standard. In my opinion, no
`
`terms require specific construction to assess the patentability of the claims.
`
`VI. Summary of the claimed subject matter in the ‘850 patent.
`
`27. The ‘850 patent is directed to providing “systems and methods for
`
`video blurring in a secure environment.” (‘850 patent, Abstract.) Figure 1 of the
`
`’850 patent, which I have reproduced below, illustrates an environment in which
`
`these described systems and methods for video blurring may occur.
`
`
`
`
`
`
`
`
`
`- 11 -
`
`U.S. Patent No. 9,083,850
`
`(’850 patent, Figure 1.)
`
`28. Communication processing system 101 “may provide telephone
`
`services, videoconferencing, online chat, and other communication services to a
`
`controlled-environment facility.’ (Id. at 3:63–66.) The ’850 patent defines a
`
`controlled-environment facility broadly as including correctional institutions or
`
`facilities (e.g., municipal jails, county jails, state prisons, federal prisons, military
`
`stockades, juvenile facilities, detention camps, home incarceration environments,
`
`etc.), healthcare facilities (e.g., hospitals, nursing homes, mental health facilities,
`
`rehabilitation clinics, such as drug and alcohol rehabilitation facilities, etc.),
`
`restricted living quarters (e.g., hotels, resorts, camps, dormitories, barracks, etc.),
`
`
`
`
`
`
`
`
`
`
`and the like.” (Id. at 3:41–51.) Visitation system 130 within the controlled-facility
`
`- 12 -
`
`U.S. Patent No. 9,083,850
`
`may “provide, schedule, and manage visitation services to residents and non-
`
`residents of a controlled-environment facility” via a publicly switched telephone
`
`network (PSTN) 107 or a packet-switched network 110. (’850 patent, 5:53–59.)
`
`29. The blurring techniques described in the ’850 patent may be used in
`
`conjunction with “[v]ideo visitation devices 103a-n (each collectively referred to
`
`as ‘video visitation device 103’) [that] may have conferencing capabilities to
`
`enable inmates to participate in video visitation sessions with non-residents of the
`
`correctional facility via video call.” (Id. at 4:59–63.) An example of video
`
`visitation device 103 is illustrated in Figure 2 of the ’850 patent, which I have
`
`reproduced below.
`
`
`
`
`
`
`
`
`
`- 13 -
`
`U.S. Patent No. 9,083,850
`
`
`
`(’850 patent, Figure 2.)
`
`30. The ’850 patent describes that “device 103 includes display 204,
`
`camera 205, and handset 201 coupled to device 103 via wire 209 “the camera 205
`
`may be any suitable imaging device such as, for instance, a video camera or
`
`webcam equipped with Charge-Coupled Devices (CCDs), Complementary Metal-
`
`Oxide-Semiconductor (CMOS) active pixel sensors, etc.” (’850 patent, 6:14–22.)
`
`31. The ’850 patent describes one technique for video blurring: “adjusting
`
`a depth of field parameter” for received video, “such that an image of a first object
`
`at a first distance from the video visitation device is in focus and an image of a
`
`second object at a second distance from the video visitation device is blurred.”
`
`
`
`
`
`
`
`
`This technique was well-known prior to the ’850 patent. Garrison describes the use
`
`- 14 -
`
`U.S. Patent No. 9,083,850
`
`of “Bokeh” techniques that manipulate the “depth of field” to keep primary subject
`
`in focus while remaining portion is out of focus. (Garrison, 3:13–21.)
`
`32.
`
`In embodiments described in the ’850 patent, adjusting the depth of
`
`field parameter involves “adjusting an f-stop setting of a camera associated with
`
`the video visitation device” which “may further include adjusting a focal length of
`
`a lens coupled to the video visitation device” or “adjusting an aperture setting of
`
`the camera associated with the video visitation device.” (’850 patent, 1:61–57.)
`
`Alternatively, the ’850 patent describes that the depth of field may be manipulated
`
`“through digital processing techniques.” (’850 patent, 7:44–46.) Although the ’850
`
`patent discloses the blurring result achieved by adjusting the depth of field using
`
`digital processing techniques (e.g., an image of a first object at a first distance in
`
`focus while an image of a second object at a second distance is blurred), the patent
`
`does not indicate or limit the specific digital processing techniques employed to
`
`achieve blurring.
`
`33. Figure 4 of the ’850 patent, which I have reproduced below, illustrates
`
`two components that can be used to adjust the depth of field of an image: image
`
`processor 402 and camera controller 404. The ’850 patent explains that “[t]he
`
`image processor 402 may be configured to change certain parameters of the video
`
`
`
`
`
`
`
`
`received from the video visitation device 103 such that portions of the video
`
`- 15 -
`
`U.S. Patent No. 9,083,850
`
`frames are blurred.” (Id. at 8:19–22.) Alternatively or additionally, camera
`
`controller 404 “may also be used to blur the video. In such an embodiment, the
`
`camera controller 404 may communicate commands to a lens
`
`mechanism 303a,b and/or to an aperture mechanism 302 for controlling the depth
`
`of field of the camera 205, and thereby blurring portions of images captured by the
`
`camera 205.” (Id. at 8:50–56.)
`
`
`
`(’850 patent, Figure 4.)
`
`34. Figure 9 of the ’850 patent, which I have also reproduced below,
`
`shows an example blurred frame as produced by the blurring techniques of the
`
`’850 patent: “As illustrated, the first object 304 may be a face 901. The second
`
`object 305 may be anything located at a predefined distance from the face 901, for
`
`
`
`
`
`
`
`
`example in the background 902 which is blurred.” (Id. at 11:65–12:1.) The ’850
`
`- 16 -
`
`U.S. Patent No. 9,083,850
`
`patent explains, “as shown in FIG. 9, a second inmate in the background 902 may
`
`have blurred features to maintain the privacy of the second inmate 903.” (Id. at
`
`12:14–16.)
`
`
`
`(’850 patent, Figure 9.)
`
`VII. Summary of the prior art.
`
`A.
`
`Shipman––A video visitation security system in a controlled-
`environment.
`35. Shipman is generally directed to “providing video visitation security
`
`to a controlled-environment facility.” (Shipman, Abstract.) Shipman describes
`
`techniques for detecting an image of a face in a video stream transmitted between
`
`an inmate in a secure environment and a non-resident of the secure environment,
`
`and obscuring other areas of the video outside of the detected face by blurring,
`
`
`
`
`
`
`
`
`clouding, darkening, distorting, masking, or shading. (Id. at 1:43–57.) Below, I
`
`- 17 -
`
`U.S. Patent No. 9,083,850
`
`have annotated Figure 1 of Shipman for my Declaration. Annotated Figure A
`
`highlights components involved in a video visitation session. (Id. at 4:58–60.)
`
`Richardson Annotated Figure A
`
`
`
`36.
`
`In Shipman, Prison 100 may host a plurality of inmates in a secure
`
`environment. (Shipman, 4:61.) Inmates have the ability to make or receive voice
`
`calls via communication system 110 using a traditional telephone device 115. (Id.
`
`at 5:8–10.) Such voice calls can be completed over public-switched telephone
`
`network 120 to a non-resident operating device 125 outside of the prison. (Id. at
`
`5:13–17.) Inmates may also be allowed to communicate with non-residents via
`
`video. (Id. at 5:32–38, 5:63–67.) For example, an inmate may use a video visitation
`
`station within visitation area 130 or elsewhere within prison 100 to communicate
`
`
`
`
`
`
`
`
`with a non-resident operating one of computer of communication devices 135A-N.
`
`- 18 -
`
`U.S. Patent No. 9,083,850
`
`(Id.) Communication between the inmate’s video visitation station and the non-
`
`resident’s communication device occurs via communication system 110, but in this
`
`case over packet-switched network 140. (Shipman, 5:35–40.) The non-resident can
`
`view the inmate’s video feed outside of prison 100 via one of devices 135A-N.
`
`37. Figure 2 of Shipman, which I have reproduced below, illustrates an
`
`example layout of video visitation area 130, as described above, within a secure
`
`environment. (Shipman, 5:58–60.) As illustrated in Figure 2, video visitation area
`
`130 includes one or more video visitation stations 200A-N. When a video
`
`visitation is scheduled, “prison staff may direct the involved inmate to leave his or
`
`her cell and travel to one of video visitation stations 200A-N within visitation area
`
`130.” (Id. at 5:60–63.)
`
`
`
`
`
`
`
`
`
`- 19 -
`
`U.S. Patent No. 9,083,850
`
`
`
`38. Video visitation stations 200A-N each include a display or screen 210,
`
`a camera and/or microphone 215, and a speaker or audio output 220. (Id. at 6:4–8.)
`
`Camera 215 may include a digital video camera. (Id.) When an inmate arrives in
`
`video visitation area 130, the inmate may be authenticated at one of video
`
`visitation stations 200A-N. (Shipman, 6:20–30.) Shipman describes that
`
`communication system 110 may then establish communication between the
`
`inmate’s video visitation station and a non-resident’s device. (Id. at 6:31–34.)
`
`39. For clarity, I have edited Figure 1 of Shipman to illustrate where a
`
`video visitation station 200A would reside within Shipman’s environment. I have
`
`produced this edited figure below.
`
`
`
`
`
`
`
`
`
`- 20 -
`
`U.S. Patent No. 9,083,850
`
`Richardson Annotated Figure B
`
`
`
`As illustrated, video visitation station 200A resides within the controlled
`
`environment 100 (e.g., the prison), and is coupled to communication system 110.
`
`(Shipman 6:31–33.) Communication system 110 may be located inside controlled
`
`environment 100 or may be partly or wholly disposed outside controlled
`
`environment 100. (Shipman 6:34–37.)
`
`40. Figure 3 of Shipman (reproduced below) illustrates a video visitation
`
`security system that includes a connection manager 310, a media server 315, a
`
`security engine 320, an inmate client 325 and a family client 330. The connection
`
`manager 310 acts to maintain a connection between two endpoints such as “inmate
`
`video visitation client resident client 325 (e.g., station 200A) and family client 330
`
`(e.g., device 135A).” (Shipman, 6:50–55.)
`
`
`
`
`
`
`
`
`
`- 21 -
`
`U.S. Patent No. 9,083,850
`
`
`
`41. During a video visitation session, resident client 325 provides non-
`
`secure “inmate video” to media server 315. (Id. at 8:27–34.) Media server 315 then
`
`forwards the received video feed to security engine 320. (Id.) Security engine 320
`
`processes the video feed and returns a processed “secure video” to media server
`
`315, which then transmits the secure video to non-resident client 330. (Shipman,
`
`8:27–34.) To produce the “secure video,” security module may execute a facial
`
`detection operation and blur areas of the video feed outside of the detected face
`
`area. (Id. at 11:9–16.) Shipman describes that a non-secure video received from the
`
`family client is processed in the same manner before delivery to the resident client.
`
`
`
`
`
`
`
`
`
`42. To illustrate the relationship between components described in
`
`- 22 -
`
`U.S. Patent No. 9,083,850
`
`Shipman’s Figure 1 and 2 with the components of Shipman’s Figure 3, I have
`
`produced a new figure below (Richardson Figure C). Specifically, Richardson
`
`Figure C integrates inmate video visitation client or resident client 325 into video
`
`visitation station 200A and family client 330 into non-resident device 135 as
`
`described by Shipman at 6:54–55. A camera that is part of video visitation station
`
`200A is interfaced to the inmate video visitation client 325 “for recording resident
`
`audio and video.” (Shipman, 7:64–66.) Shipman also describes that the media
`
`server 315, security engine 325, and connection manager 310, as well as rules
`
`database 340 and user database 335, may be implemented in communications
`
`system 110. (Id. at 6:48–50.) Communication between the inmate video visitation
`
`client and family client occurs via communication system 110 over network 140.
`
`(Id. At 5:35–40.)
`
`
`
`
`
`
`
`
`
`- 23 -
`
`U.S. Patent No. 9,083,850
`
`Communication System 110
`
`Camera
`215
`
`INMATE VIDEO
`
`FAMILY VIDEO
`
`SECURE FAMILY VIDEO
`
`SECURE INMATE VIDEO
`
`Video visitation station
`200A
`
`Network
`140
`
`FAMILY
`VIDEO
`
`SECURE
`INMATE
`VIDEO
`
`Non-resident device
`135
`
`
`
`Richardson Annotated Figure C
`
`43. As in the ’850 patent, Shipman recognizes that inappropriate activities
`
`or private areas may be captured during a video session. (See, e.g., Shipman,
`
`11:18–20, Figures 5A–5C and 6A–6E.) Shipman discloses therefore, for security
`
`and privacy reasons, that the security module of Shipman executes a facial
`
`detection operation and blur areas of the video feed outside of the detected face
`
`area. (Shipman, 11:9–16.)
`
`44. Shipman provides an example “secure video” feed in Figures 5A and
`
`6A, which I have reproduced below.
`
`
`
`
`
`
`
`
`
`- 24 -
`
`U.S. Patent No. 9,083,850
`
`
`
`(Shipman, Figures 5A and 6A.)
`
`45.
`
`In Figure 5A, the security module 320 detects a face and then “may
`
`blur, cloud, darken, distort, mask, shade or otherwise deface other areas 520A in
`
`the video feed.” (Shipman, 11:8–15.) Figure 6A depicts an example where an
`
`individual in a video attempts to convey information by holding up an object or
`
`making gestures, for example. Therefore, as an added security feature, an object
`
`615A overlapping face 605A may also be defaced so that it is excluded from the
`
`transmitted video feed. (Id. at 11:62–67.) This enables the techniques employed by
`
`Shipman to detect images within a video feed that are permitted to be transmitted
`
`between residents and non-residents of a secure environment, while blurring or
`
`otherwise defacing areas outside of the permitted images.
`
`
`
`
`
`
`
`
`
`- 25 -
`
`U.S. Patent No. 9,083,850
`
`B. Garrison––Short depth of field image processing techniques for
`use in a video telephony session.
`46. Garrison describes segregating an object in the foreground of an
`
`image and blurring the background to render it indistinct, and thus “displayed
`
`video of a user in the foreground is kept in focus while the background appears to
`
`be out of focus.” (Garrison, Abstract.) Garrison achieves blurring through a
`
`technique that manipulates the depth of field.
`
`
`
`
`
`(Garrison, Figure 5.)
`
`47. Figure 5 of Garrison, which I have reproduced above, illustrates a
`
`videophone used in a video telephony session between two users. (Id. at 4:7–9,
`
`
`
`
`
`
`
`
`4:20–22.) The videophone of Garrison includes a display 502 attached to a base
`
`- 26 -
`
`U.S. Patent No. 9,083,850
`
`505 with mounting arm 512. (Id. at 4:22–24.) The videophone also includes a
`
`camera 514 having a lens oriented towards the videophone user, and a microphone
`
`to capture voices and other sounds for a videophone call. (Id. at 4:26–30.)
`
`48. Garrison describes techniques to blur an image or portion of an image
`
`by “simulating depth of field effects in a video image.” (Id. at 10:20–21.) For
`
`example, the camera 514 of the video phone first captures a video image “having a
`
`long or substantially infinite depth of field.” (Garrison, 10:23–25.) The captured
`
`video image is then “spatially segregated into a target portion for which focus is
`
`maintained and a remaining portion for which blurring is applied” using image
`
`processing techniques. (Id. at 10:27–30; see also id. at 5:34–48.). The blurred
`
`remaining portion is combined with the unblurred target portion to produce a
`
`composite image in which only the target portion of the image appears in focus.
`
`(Id. at 10:34–41.) To determine the target portion for an image, object detection
`
`techniques may be employed in which a specific feature, such as a user’s face,
`
`head, and shoulders, are dynamically detected and tracked as the user moves,
`
`essentially allowing for any object or objects at a particular distance to remain in
`
`focus. (Garrison, 5:39–43.) Alternatively, fixed templates may be applied to a
`
`
`
`
`
`
`
`
`captured video image to segregate the portion of interest from the remaining
`
`- 27 -
`
`U.S. Patent No. 9,083,850
`
`portion, to provide a “less complex implementation.” (Garrison, 6:29–44.)
`
`49. Figure 15 of Garrison, which I have reproduced below, illustrates an
`
`example architecture for implementing the image processing techniques described
`
`by Garrison.
`
`
`
`(Garrison, Figure 15.) During operation, CCD camera 513 first captures a video
`
`image “having a long depth of field (i.e., one that is substantially infinite).” (Id. at
`
`9:29–31.) The captured video image is processed by image processing module
`
`1516, which includes image segregation logic 1520 to segregate the video image
`
`
`
`
`
`
`
`
`into a target portion and a remaining portion, and blurring logic 1526 to blur the
`
`- 28 -
`
`U.S. Patent No. 9,083,850
`
`remaining portion of the video image, as described above. (Id. at 9:34–63.)
`
`C. Mayhew––Controlling the functions of a camera to adjust depth
`of field.
`50. Mayhew is directed to providing “depth of field fade capabilities in
`
`moving imaging systems through synchronized operation of the optical elements of
`
`a camera system.” (Mayhew, 3:17–20.) Mayhew describes a “motion picture
`
`system” that “includes a computer controller, a computer controlled lens, a
`
`computer controlled aperture, and a computer controlled adjustable shutter having
`
`a shutter angle.” (Mayhew, 3:24–27.) In Mayhew, “the operation of the aperture,
`
`the shutter, and the lens are synchronized by the controller to produce a depth of
`
`field fade without a visible luminance shift in a film exposed by the camera.” (Id.
`
`at 3:27–30.) This system is illustrated in Figure 1 of Mayhew, which I have
`
`reproduced below. Figure 1 includes “computer/controller 30 coupled to camera 40
`
`having lens 20,” and “[t]he functions of the camera and the camera lens are
`
`controlled by the operation of the computer/controller.” (Id. at 5:32–35.) Some
`
`embodiments of Mayhew provide “for the remote control of camera functions
`
`through the use of camera control software protocol via a computer interface.” (Id.
`
`at 2:22–25.) These embodiments allow a controller, such as computer/controller
`
`30, to alter the depth of field of the camera remotely.
`
`
`
`
`
`
`
`
`
`- 29 -
`
`U.S. Patent No. 9,083,850
`
`
`
`(Mayhew, Figure 1.)
`
`51. Mayhew explains that “the area within which a point will be imaged
`
`within acceptable focus is the depth of field, and is bounded at both extremes by
`
`the point at which the image of a point would create a circle greater than the
`
`defined accepted circle of confusion value.” (Mayhew, 4:47–51.) Mayhew
`
`describes that two variables are of interest in the calculation of the depth of field:
`
`the hyperfocal distance and the object distance. (Id. at 4:52–5:3.) The hyperfocal
`
`distance is “the distance to the closest point that will be acceptably in focus when
`
`the lens is focused at infinity.” (Id. at 4:6–9.) The object distance is “the distance
`
`
`
`
`
`
`
`
`between the front principle point of the lens and the plane of focus.” (Mayhew,
`
`- 30 -
`
`U.S. Patent No. 9,083,850
`
`6:55–56.)
`
`52. Mayhew first describes that a change in hyperfocal distance alters the
`
`depth of field: “As mentioned above, by changing the diameter of the aperture, the
`
`hyperfocal distance is changed. For example, changing the f/stop from f/2 to f/4
`
`would result in halving the hyperfocal distance. Using the changed value of the
`
`hyperfocal distance in the equations for near and far limit of depth of field above, it
`
`is shown that the near and far limits of the depth of field are also changed.” (Id. at
`
`5:3–9.) “[T]he diameter of the aperture is related to the f/stop of the camera, and
`
`thereby the focal distance and the depth of field of image produced by the camera.”
`
`(Mayhew, 5:37–42.) The computer/ controller of Mayhew remotely controls the
`
`size of the aperture as illustrated in Figure 2 of Mayhew, which I have reproduced
`
`below.
`
`
`
`
`
`
`
`
`
`- 31 -
`
`U.S. Patent No. 9,083,850
`
`
`
`(Mayhew, Figure 2.)
`
`53. Mayhew discusses that a “user of camera system 10 can produce a
`
`depth of field fade by inputting a signal via an interface to computer/controller 30.”
`
`(Mayhew, 5:59–61.) Then, “[c]omputer/controller 30 outputs a first signal to
`
`aperture 100 in accordance with the desired change in depth of field.” (Id. at 5:61–
`
`63.) Consequently, a change in the lens aperture changes the f-stop, which in turn
`
`alters the depth of field.
`
`54.
`
`In addition to the hyperfocal distance that Mayhew describes,
`
`Mayhew also teaches that changes in the object distance will alter the depth of
`
`field. (Mayhew, 7:8–10.) Here, Mayhew explains that “[c]omputer/controller 30 is
`
`connected to camera 40. Computer/controller 30 produces lens control signals,
`
`which alter the distance between lens 20 and the fi
Accessing this document will incur an additional charge of $.
After purchase, you can access this document again without charge.
Accept $ Charge
Still Working On It
This document is taking longer than usual to download. This can happen if we need to contact the court directly to obtain the document and their servers are running slowly.
Give it another minute or two to complete, and then try the refresh button.
A few More Minutes ... Still Working
It can take up to 5 minutes for us to download a document if the court servers are running slowly.
Thank you for your continued patience.
This document could not be displayed.
We could not find this document within its docket. Please go back to the docket page and check the link. If that does not work, go back to the docket and refresh it to pull the newest information.
Your account does not support viewing this document.
You need a Paid Account to view this document. Click here to change your account type.
Your account does not support viewing this document.
Set your membership
status to view this document.
With a Docket Alarm membership, you'll
get a whole lot more, including:
- Up-to-date information for this case.
- Email alerts whenever there is an update.
- Full text search for other cases.
- Get email alerts whenever a new case matches your search.
One Moment Please
The filing “” is large (MB) and is being downloaded.
Please refresh this page in a few minutes to see if the filing has been downloaded. The filing will also be emailed to you when the download completes.
Your document is on its way!
If you do not receive the document in five minutes, contact support at support@docketalarm.com.
Sealed Document
We are unable to display this document, it may be under a court ordered seal.
If you have proper credentials to access the file, you may proceed directly to the court's system using your government issued username and password.
Access Government Site
