(12) United States Patent
`Ono et al.
`
`USOO6314137B1
`(10) Patent No.:
`US 6,314,137 B1
`(45) Date of Patent:
`Nov. 6, 2001
`
`(54) VIDEO DATA COMPRESSION SYSTEM,
`VIDEO RECORDING/PLAYBACK SYSTEM,
`AND WIDEO DATA COMPRESSION
`ENCODING METHOD
`(75) Inventors: Koichi Ono, Yokosuka; Hideo
`Nishijima; Takayuki Kanesaki, both of
`Hitachinaka; Tadasu Horiuchi;
`Nobuyoshi Tsukiji, both of Yokohama,
`all of (JP)
`
`(73) Assignee: Hitachi, Ltd., Tokyo (JP)
`* Y Not
`Subj
`y disclai
`h
`f thi
`Otice:
`ubject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 0 days.
`
`(21) Appl. No.: 09/050,568
`(22) Filed:
`Mar. 30, 1998
`(30)
`Foreign Application Priority Data
`Mar. 31, 1997
`(JP) ................................................... 9-080082
`(51) Int. Cl." ................................................... H04N 7/12
`(52) U.S. Cl. ..................
`... 375/240; 375/240.01
`(58) Field of Search ..................................... 348/402,409,
`348/14.1, 14.08, 159,385.1, 705, 176
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`6/1979 Dischert ............................... 348/176
`4,158,208
`4,961.211 * 10/1990 Tsugane et al. .
`... 348/14.08
`5,223,949
`6/1993 Honjo ................................... 358/426
`5,371.535 * 12/1994 Takizawa ...
`348/14.1
`5,488,482 * 1/1996 Ueda et al. ..
`348/402
`5,579,060
`11/1996 Elberbaum ........................... 348/705
`FOREIGN PATENT DOCUMENTS
`
`KOKAI
`8/1987 (JP).
`62-17928O
`KOKAI
`3/1989 (JP).
`64-65989
`* cited by examiner
`Primary Examiner-Chris Kelley
`Assistant Examiner Allen Wong
`(74) Attorney, Agent, or Firm-Mattingly, Stanger &
`Malur, P.C.
`ABSTRACT
`(57)
`A video data compression/playback System efficiently com
`presses a plurality of Video data pieces through inter-frame
`predictive-encoding. Mixed video data VSMIr composed of
`time-divided video data received by video input terminals
`are divided Video data pieces each of which corresponding
`to Video input terminals and the Video data pieces are written
`into memories respectively by a memory control circuit for
`recording mode. The mixed video data written into the
`memories is read, a Specified number of frames at a time. An
`MPEG encoding circuit performs inter-frame predictive
`encoding for each specified number of frames.
`
`3,580,998
`3,582,542
`
`5/1971 Hammond et al. .................. 348/159
`6/1971 Smierciak ......................... 348/385.1
`
`13 Claims, 20 Drawing Sheets
`
`5
`
`26
`
`11
`
`
`
`SYNCHRONIZATIO
`SWITCHING
`CIRCUIT
`
`
`
`
`
`
`
`VIDEO
`DECODER
`CIRCUIT
`
`s
`
`WSMr MEMORY 1.
`MEMORY 27
`TE:
`
`WSMO
`
`MPEG
`ENCODING
`CIRCUIT
`
`13
`
`FRAME SWITCHER |
`24
`
`CAMERA CODE
`RETERMINATION
`i CIRCUIT
`DISPLAY
`SPL - 10
`
`O-H
`
`-----------------
`CAMERA CODE
`ADDITION
`23r;
`
`MEMORY CONTROL
`CIRCUIT FOR RECORDING MODE
`VSMIp
`1. MEMORY 1Ps
`2. MEMORY 2P -
`MEMORY 3P
`YE59 vSMOp.
`MEMORY 4P
`ENCODER
`CIRCUIT
`--
`
`19
`MEMORY CONTROL CRCUIT
`FOR PLAYBACK MODE
`
`14
`
`
`
`BSr
`
`RECORONG/
`PLAYBACK
`MODULE
`
`Amazon / WAG Acquisition
`Exhibit 1019
`Page 1
`
`

`

`Amazon / WAG Acquisition
`Exhibit 1019
`Page 2
`
`

`

`U.S. Patent
`
`Nov. 6, 2001
`
`Sheet 2 of 20
`
`US 6,314,137 B1
`
`
`
`NOILICJCIV
`1100HIO
`
`V LInOHIO DI IL
`NOÏOETES DI ] Õ??IN DI ]
`
`|CIV/EH|-?NIHOL?NAS
`
`
`
`~ TOHI NOO || TOHINOO
`
`
`
`JLIT KOBIO
`
`]][TOHIKO|
`NOLLVHENE!!!) ?
`
`68
`
`?S?TINOILOETES
`
`OEGIA
`
`8 LITTOHIO
`
`Amazon / WAG Acquisition
`Exhibit 1019
`Page 3
`
`

`

`U.S. Patent
`
`Nov. 6, 2001
`
`Sheet 3 of 20
`
`US 6,314,137 B1
`
`FIG.3
`
`1 FRAME
`seamlapa
`
`VS1
`
`
`
`Amazon / WAG Acquisition
`Exhibit 1019
`Page 4
`
`

`

`U.S. Patent
`
`Nov. 6, 2001
`
`Sheet 4 of 20
`
`US 6,314,137 B1
`
`FIG. 4
`
`
`
`FIG. 5
`
`
`
`53
`
`54
`
`MAGNTUDE
`COMPARATOR
`
`LATCH
`CIRCUT
`
`SYNC
`SEPARATION
`CIRCUIT
`
`
`
`TIMING SIGNA
`GENERATION
`CIRCUIT
`
`Amazon / WAG Acquisition
`Exhibit 1019
`Page 5
`
`

`

`U.S. Patent
`
`Nov. 6, 2001
`
`Sheet 5 of 20
`
`US 6,314,137 B1
`
`
`
`CT60
`
`FT-CK Q
`
`61
`
`62
`
`67
`----
`?
`i-DEGH
`-DEG
`DEG
`-DEG
`
`MWRR
`MWR2R
`MWR3R
`MWR4R
`MRR1 R
`
`El MR2
`Egil
`MRF
`MRR4R
`DEG
`till
`M
`MR2R
`MR3R
`MR4R
`68
`D-RES
`
`CCODE
`
`COINCIDENCE
`DETECTION CIRCUIT
`
`
`
`
`
`CONCDENCE
`DETECTION CIRCUIT
`
`
`
`COINCIDENCE
`DETECTION CIRCUIT
`
`COINCIDENCE
`DETECTION CIRCUIT
`
`MW1R
`
`MW2R
`
`MW3R
`
`MW4R
`
`Amazon / WAG Acquisition
`Exhibit 1019
`Page 6
`
`

`

`Amazon / WAG Acquisition
`Exhibit 1019
`Page 7
`
`

`

`Amazon / WAG Acquisition
`Exhibit 1019
`Page 8
`
`

`

`U.S. Patent
`
`Nov. 6, 2001
`
`Sheet 8 of 20
`
`US 6,314,137 B1
`
`
`
`
`
`
`
`
`
`
`
`TOHINOO
`ELVE
`
`
`
`?NIHEJCIHOBH
`
`NO?LVHEINES)
`
`ECJOO
`
`WHEWWO
`
`LI?IOHIO
`
`Amazon / WAG Acquisition
`Exhibit 1019
`Page 9
`
`

`

`U.S. Patent
`
`Nov. 6, 2001
`
`
`
`
`
`
`
`
`
`
`
`
`
`[0-7]
`
`Sheet 9 of 20
`
`US 6,314,137 B1
`
`(TXETXÈXT?XIX EX EXETXETXÈXETXÈX EX EX EX E XEIX EX E XIX EXC Edald
`
`??XTXTXTXTXTXTXTXTXTXEXTXTXTXTXTXIX–X a… §§
`
`
`
`
`
`Amazon / WAG Acquisition
`Exhibit 1019
`Page 10
`
`

`

`U.S. Patent
`
`Nov. 6, 2001
`
`Sheet 10 0f 20
`
`US 6,314,137 B1
`
`Å HOWEWN
`
`HE-J-J.[18]
`
`LITTOHIKO
`
`00||
`
`50NI\\ECIEJOEB
`
`B?.HTMLOId
`
`LITTO HIO
`
`
`NOI LVSNEdWOO
`LITTOHIKO
`NOLLOW
`
`86
`
`
`
`
`
`
`CITOH AVTEQ
`|I/QOHIO
`daGOOO
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Amazon / WAG Acquisition
`Exhibit 1019
`Page 11
`
`

`

`Amazon / WAG Acquisition
`Exhibit 1019
`Page 12
`
`

`

`U.S. Patent
`
`Nov. 6, 2001
`
`Sheet 12 of 20
`
`US 6,314,137 B1
`
`
`
`
`
`
`
`
`
`
`
`? (j? (); () i (?); () i
`
`dIWSA
`
`dEGOOO
`
`Amazon / WAG Acquisition
`Exhibit 1019
`Page 13
`
`

`

`U.S. Patent
`
`Nov. 6, 2001
`
`Sheet 13 of 20
`
`US 6,314,137 B1
`
`FIG. 14
`
`123
`
`VIDEO
`
`124
`
`125
`
`126
`
`129
`
`VIDEO
`ENCODER
`
`CAMERA CODE
`SEPARATION
`CIRCUIT
`
`
`
`MEMORY
`CONTROL
`CIRCUIT
`
`127
`
`128
`
`Amazon / WAG Acquisition
`Exhibit 1019
`Page 14
`
`

`

`U.S. Patent
`
`Nov. 6, 2001
`
`Sheet 14 of 20
`
`US 6,314,137 B1
`
`
`
`
`
`
`
`
`
`
`
`
`
`©NICJOONE
`LITIKOHIO
`5DEc{WN
`BECJOOBC]
`LITTOHIO
`OEC?IA
`
`NOLL\/ZINOHHONÁS
`
`JLI[\OHIO
`
`5DNÍHO LINAS
`
`Amazon / WAG Acquisition
`Exhibit 1019
`Page 15
`
`

`

`U.S. Patent
`
`US 6,314,137 B1
`
`O
`t
`O
`
`
`
`
`
`
`
`
`
`
`
`1][10H10
`
`AWTIEQ
`
`Amazon / WAG Acquisition
`Exhibit 1019
`Page 16
`
`

`

`U.S. Patent
`
`Nov. 6, 2001
`
`Sheet 16 of 20
`
`US 6,314,137 B1
`
`LITTOH?O ( L/(\O 8 NO|1\/SNEdWOO I LOO -}
`NOILOWN O L U LITO N
`
`VSA
`
`
`
`
`
`
`
`
`
`99 ||
`
`D
`
`fit
`
`uld
`U
`
`8 TEIS
`
`O
`
`CO
`
`an
`
`5.
`
`CC
`l
`
`l
`
`TEIS
`
`ÅBOWEJ W ºd i ÁHOWNEW
`
`\/d
`
`IESA
`
`ÅEJOWNEW E
`
`TOHINOO A HOWEWN
`.]][\OHIO
`09_10
`
`Amazon / WAG Acquisition
`Exhibit 1019
`Page 17
`
`

`

`U.S. Patent
`
`Nov. 6, 2001
`
`Sheet 17 of 20
`
`US 6,314,137 B1
`
`
`
`| GTV/EH I ELI?JNA
`
`£ OVERH 8 E LIHAW
`
`?nON N TIES
`
`??O-? – TES
`
`?nog 8 TES
`
`Amazon / WAG Acquisition
`Exhibit 1019
`Page 18
`
`

`

`U.S. Patent
`
`Nov. 6, 2001
`
`Sheet 18 of 20
`
`US 6,314,137 B1
`
`NOLLV/SNECHWOO
`
`NO|JLOW O 1
`
`LITTOHIO
`
`LTÀO C18
`
`LTIO CJE
`
`
`
`
`
`
`
`AHOWNEW
`
`„LITTOHIKO
`
`TOHI NOO
`
`09 LO
`
`Amazon / WAG Acquisition
`Exhibit 1019
`Page 19
`
`

`

`U.S. Patent
`
`Nov. 6, 2001
`
`Sheet 19 of 20
`
`US 6,314,137 B1
`
`as a a
`s
`ve al
`up a
`
`s
`also e s a
`
`ve
`fa.
`ques as its a
`
`eye
`al
`use is a
`t
`
`s
`
`as is a
`O
`v
`AL
`anese a un un
`s
`
`uso a
`Se
`es
`
`-
`
`so
`S
`te
`a.
`a
`so
`s
`
`as a on
`s
`
`as as on as
`
`s
`cessa
`s
`Cl
`as a
`q
`L
`as
`
`was
`SF
`
`as a so
`
`aes up a
`
`so us as a
`
`98.
`
`ope does do
`
`aus
`
`as a so as as as a
`
`usa a
`
`a sa as
`
`aume s as so
`
`use asso
`
`as as as a n as
`
`
`
`|HAVET +
`
`IESA
`
`09 LO
`
`I OVERH | 3. LIHNA
`
`Eld/, id
`
`CESA
`
`ºd CJVE!!!!
`
`€d E. LIENA
`
`
`
`CJ? TEIS
`
`??OOS
`
`Amazon / WAG Acquisition
`Exhibit 1019
`Page 20
`
`

`

`U.S. Patent
`
`US 6,314,137 B1
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`66
`
`dEGOOO
`
`86
`
`Amazon / WAG Acquisition
`Exhibit 1019
`Page 21
`
`

`

`US 6,314,137 B1
`
`1
`VIDEO DATA COMPRESSION SYSTEM,
`VIDEO RECORDING/PLAYBACK SYSTEM,
`AND WIDEO DATA COMPRESSION
`ENCODING METHOD
`
`BACKGROUND OF THE INVENTION
`
`2
`A high intra-frame correlation in the image means closer
`two-dimensional frequency components, thus increasing the
`encoding efficiency and reducing the amount of data while
`ensuring the picture image quality.
`However, an image composed of fine patterns results in a
`lower intra-frame correlation and therefore decreases the
`compression efficiency. Thus, a compression ratio of up to
`10:1 is required to ensure a reasonable JPEG image.
`Next, MPEG will be described.
`In addition to the technique used by JPEG described
`above, MPEG uses an inter-frame correlation to reduces the
`amount of data. More specifically, it finds changes between
`every two frames and performs DCT on the changes. This is
`called inter-frame predictive encoding.
`For slow-moving video with little or no change between
`frames, MPEG produces a very small amount of data.
`Even for fast-moving video, MPEG provides an
`extremely high predictive encoding efficiency because it
`detects motion vector between frames and motion compen
`sation for them.
`AS a result, the image quality is not degraded at all even
`when the compression ratio is Several Scores to one and,
`therefore, a recording/playback system using the MPEG
`compression algorithm can record Video data longer in time
`than JPEG.
`
`SUMMARY OF THE INVENTION
`Time lapse VTRS, which record analog video Signals on
`magnetic tapes, have been used in most conventional video
`Surveillance Systems. It is also possible to configure a video
`Surveillance System using digital recording System
`explained above.
`That is, a System with only one camera digitizes the
`output Video signals, reduces the amount of data by com
`pressing data with MPEG, and records compressed data on
`a disk or tape.
`A system with multiple cameras has an MPEG compres
`Sion unit and a recording/playback unit installed for each
`camera to record digitized video signals Sent from each
`Caca.
`The problem is that a video surveillance system with
`multiple MPEG compression encoding units is very expen
`sive because the MPEG compression encoding unit is very
`expensive. It is therefore preferable that, as with the con
`ventional Video Surveillance System, the System has only one
`compression encoding unit which compresses video signals
`Sent from multiple cameras for recording.
`However, compressing digitized mixed video Signals gen
`erated by a frame Switcher with MPEG poses the following
`problem.
`In most cases, Video generated by Separate cameras has no
`correlation. This means that the amount of changes among
`Video obtained by Separate cameras is much larger than the
`amount of changes among Video obtained by the same
`Caca.
`Therefore, a frequent Switch among multiple cameras in
`the time divided manner affects inter-frame predictive
`encoding. Video data cannot be compressed efficiently for
`recording.
`This invention seeks to Solve the problems associated
`with the prior art described above. It is an object of this
`invention is to provide a Video data compression System,
`Video recording/playback System, and Video data compres
`Sion encoding method which efficiently compress multiple
`Video data items through predictive encoding.
`
`1O
`
`15
`
`1. Field of the Invention
`This invention relates to a Video data compression System,
`and more particularly to a compression encoding System for
`use in a video recording unit which Sequentially Switches
`Video data obtained from a plurality of Video cameras Such
`as those used in a Video Surveillance System.
`2. Description of the Related Art
`Conventionally, a Video Surveillance System which
`records Video from multiple video cameras (hereafter simply
`called cameras) onto a long-time recording time-lapse VTR
`has been used. This System uses a Switcher to Switch video
`from multiple cameras in a time-dividing manner.
`Earlier patent disclosures dealing with this type of Video
`Surveillance Systems are found in Japanese Patent Laid
`Open Application No. Sho 64-65989 and Japanese Patent
`Laid-Open Application No. Hei 5-73312.
`The system disclosed in Japanese Patent Laid-Open
`25
`Application No. Sho 64-65989 synchronizes multiple cam
`eras with the reference Signal to eliminate the discontinuity
`of Video signals at Switching time.
`It also synchronizes the VTR recording timing with the
`Video signal Switching Signal to minimize unstable operation
`time. This results in a shorter Video Switching period,
`making high-density recording possible.
`In addition, detecting at the time of playback, the camera
`identification Signal Superimposed on the Video signal
`allows only the desired video to be selectively monitored.
`The system disclosed in Japanese Patent Laid-Open
`Application No. Hei 5-73312 switches timely between a
`2-System Video Selection means and a 2-system Video signal
`Synchronization means to enable a Video Surveillance
`System, which has cameras not under control of external
`Synchronization Signals or each controlled by its own Syn
`chronization method, to Synchronize Video signals for
`proper Switching.
`A System which Switches multiple cameras in a time
`division manner to combine their video signals into one
`mixed video signal is called a frame Switcher. Many Such
`Systems are commercially available.
`Today, digital video recording and playback Systems
`which record and play back digital image data have become
`popular.
`50
`In general, image data to be recorded digitally is com
`pressed to prevent the amount of data from increasing and
`therefore to allow the recording medium to be used for
`recording for a longer period of time.
`55
`Well-known image data compression techniques include
`JPEG (Joint Photographic Experts Group) and MPEG
`(Moving Pictures Experts Group).
`Because these techniques are described in detail in many
`books, the following briefly gives the Summary of their
`algorithms.
`First, JPEG will be described
`JPEG divides an image into multiple small blocks and
`converts each block into two-dimensional frequency com
`ponents through DCT (Discrete Cosine Transform). It then
`65
`reduces the amount of data through non-linear quantization
`and entropy encoding.
`
`35
`
`40
`
`45
`
`60
`
`Amazon / WAG Acquisition
`Exhibit 1019
`Page 22
`
`

`

`US 6,314,137 B1
`
`15
`
`3
`To solve the above problems, a first embodiment of this
`invention is a video data compression System which com
`preSSes mixed video data generated by mixing a plurality of
`Video data items in a time-divided manner, wherein the
`mixed video data is compressed by performing inter-frame
`or inter-field predictive encoding for each of the plurality of
`Video data items.
`For example, Such a System comprises first Storage means
`for Storing the mixed Video data; first control means for
`controlling writing and reading the mixed video data to and
`from the first Storage means, and predictive encoding means
`for performing predictive encoding among a plurality of
`images of the mixed Video data.
`The first control means writes the mixed video data into
`the first Storage means, the mixed video data being divided
`into the plurality of Video data pieces, and Sequentially reads
`the mixed video data, a plurality of images at a time, for each
`of the plurality of video data items from the first storage
`means and wherein
`the predictive encoding means performs predictive encod
`ing among a plurality of images of Video data Sequentially
`read, a plurality of images at a time, from the first Storage
`CS.
`In addition, when the mixed Video data is composed of the
`25
`plurality of Video data items, each of the plurality of Video
`data items Sequentially appearing for a predetermined num
`ber of images, the predictive encoding means performs
`predictive encoding on a plurality of images appearing at an
`interval of a number of images, the number being a product
`of the predetermined number of images, the number of the
`plurality of Video data items, and a natural number.
`The first embodiment of this invention compresses mixed
`Video data, composed of a plurality of Video data items,
`through predictive encoding for each of the plurality of
`Video data items, thus efficiently compressing the mixed
`Video data.
`A Second embodiment of this invention is a Video data
`compression System which compresses a plurality of Video
`data items, comprising video data acquisition means for
`Sequentially acquiring a predetermined number of Video data
`images from the plurality of Video data items on a frame
`basis or on a field basis, and predictive encoding means for
`compressing the mixed video data by performing predictive
`encoding among the images of Video data images Sequen
`tially acquired by the Video data acquisition means.
`For example, the Video data acquisition means comprises
`first Storage means for Storing the plurality of Video data
`items, and first control means for controlling writing and
`reading the plurality of Video data items to and from the first
`Storage means.
`The first control means divides each of the plurality of
`Video data items at an interval of a predetermined number of
`images, writes the divided Video data into the first Storage
`means, and Sequentially reads the Video data, a plurality of
`images at a time, for each of the plurality of Video data
`pieces from the first Storage means and
`the predictive encoding means performs predictive encod
`ing among the plurality of images of Video data Sequentially
`read, a plurality of images at a time, from the first Storage
`CS.
`The second embodiment of this invention sequentially
`acquires a predetermined number of Video data images from
`a plurality of Video data items and compresses the Video data
`by performing predictive encoding among a plurality of
`images of the acquired Video data, thus compressing the
`plurality of video data items efficiently.
`
`35
`
`4
`BRIEF DESCRIPTION OF THE DRAWINGS
`FIG. 1 is a block diagram showing the outline of a video
`data recording/playback System used in a first embodiment
`of this invention.
`FIG. 2 is a diagram showing the outline of the Synchro
`nization switching circuit 5 shown in FIG. 1.
`FIG. 3 is a timing diagram showing the operation of the
`synchronization switching circuit 5 shown in FIG. 2.
`FIG. 4 is a diagram explaining the waveform of the Signal
`used for a camera code included in a Video signal output
`from the synchronization Switching circuit 5 shown in FIG.
`2.
`FIG. 5 is a diagram showing the outline of the camera
`code determination circuit 12 shown in FIG. 1.
`FIG. 6 is a diagram showing the outline of the memory
`control circuit for recording mode 13 shown in FIG. 1.
`FIG. 7 is a timing diagram showing the operation of the
`memory control circuit for recording mode 13 shown in FIG.
`6.
`FIG. 8 is a timing diagram showing the operation of the
`memory control circuit for recording mode 13 shown in FIG.
`6.
`FIG. 9 is a diagram showing the outline of the MPEG
`encoding circuit 11 shown in FIG. 1.
`FIG. 10 is a timing diagram showing the operation of the
`MPEG encoding circuit 11 shown in FIG. 9.
`FIG. 11 is a diagram showing the outline of the MPEG
`decoding circuit 15 shown in FIG. 1.
`FIG. 12 is a diagram showing the outline of the memory
`control circuit for playback mode 21 shown in FIG. 1.
`FIG. 13 is a timing diagram showing the operation of the
`memory control circuit for playback mode 21 shown in FIG.
`12.
`FIG. 14 is a diagram showing the outline of the display
`control circuit 24 shown in FIG. 1.
`FIG. 15 is a block diagram showing the outline of a video
`data recording/playback System of a Second embodiment of
`this invention.
`FIG. 16 is a diagram showing the outline of the MPEG
`encoding circuit 131 shown in FIG. 15.
`FIG. 17 is a diagram showing the outline of the picture
`reordering circuit 141 shown in FIG. 16.
`FIG. 18 is a timing diagram showing the operation of the
`picture reordering circuit 141 shown in FIG. 17.
`FIG. 19 is a diagram showing the outline of the image
`memory circuit 142 shown in FIG. 15.
`FIG. 20 is a timing diagram showing the operation of the
`image memory circuit 142 shown in FIG. 19.
`FIG. 21 is a diagram showing the outline of the MPEG
`decoding circuit 132 shown in FIG. 15.
`DETAILED DESCRIPTION OF PREFERRED
`EMBODIMENTS
`A first embodiment according to this invention will now
`be described with reference to the attached drawings.
`FIG. 1 is a block diagram showing the outline of a video
`data recording/playback System used in the first embodiment
`according to this invention.
`In the figure, numbers 1 to 4 refer to Video input terminals,
`number 5 refers to a Synchronization Switching circuit,
`numbers 6-9 and numbers 16-19 refer to memory, number
`11 refers to an MPEG encoding circuit, number 12 refers to
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`Amazon / WAG Acquisition
`Exhibit 1019
`Page 23
`
`

`

`US 6,314,137 B1
`
`15
`
`S
`a camera code determination circuit, number 13 refers to a
`memory control circuit for recording mode, number 14
`refers to a recording/playback module, number 15 refers to
`an MPEG decoding circuit, number 21 refers to a memory
`control circuit for playback mode, number 23 refers to a
`camera code addition circuit, number 24 refers to a display
`control circuit, number 25 refers to a Video output terminal,
`number 26 refers to a video decoder circuit, and number 27
`refers to a video encoder circuit. Number 10 refers to a frame
`Switcher composed of the Synchronization Switching circuit
`5 and the display control circuit 24.
`The Video input terminals 1-4 receive analog video
`Signals from the cameras not shown in the figure.
`The synchronization Switching circuit 5 converts the four
`analog video signals received by the Video input terminals
`1-4 to digital signals, Synchronizes and Sequentially
`Switches the four Video signals, and outputs them as one
`mixed Video signal. In addition, the circuit adds camera
`codes each identifying the four video Signals to the corre
`sponding part of the mixed video signal.
`The video decoder circuit 26 converts the mixed video
`Signal from the Synchronization Switching circuit 5 to Video
`data VSMIr which is composed of three signals: a luminance
`Signal and two color difference signals composed of the
`25
`number of picture elements corresponding to the MPEG
`encoding format.
`The camera code determination circuit 12 detects the
`camera codes in the mixed video signal output from the
`Synchronization Switching circuit 5.
`Memory (1R) 6 to memory (4R) 9 store the corresponding
`part of the video data VSMIr.
`The memory control circuit for recording mode 13 con
`trols memory containing the video data VSMIr from the
`Video decoder circuit 26 according to the camera codes
`determined by the camera code determination circuit 12.
`In this embodiment, the video data VSMIr is controlled
`according to the camera codes as follows. That is, the
`memory (1R) 6 to the memory (4R) 9 are controlled such
`that the Video data corresponding to the Video signal entered
`from the video input terminals 1 is stored in the memory
`(1R) 6, the Video data corresponding to the video signal
`entered from the video input terminals 2 is stored in the
`memory (2R)7, the video data corresponding to the video
`Signal entered from the Video input terminals 3 is Stored in
`the memory (3R) 8, and the video data corresponding to the
`Video Signal entered from the Video input terminals 4 is
`stored in the memory (4R) 9.
`The memory control circuit for recording mode 13 also
`Sequentially accesses the memory (1R) 6 to the memory
`(4R) 9 to read any frames of video data VSMIristored in each
`memory.
`In this way, the video data VSMIr from the video decoder
`circuit 26 is converted to video data VSMOr where sequence
`of frames has been changed. The video data VSMOr will
`consist a Sequence of frames each of which corresponding to
`the Signals received by Video input terminals 1 to 4.
`The MPEG encoding circuit 11 encodes the video data
`VSMOr using MPEG format and outputs it as an MPEG bit
`Stream BSr.
`At this time, the MPEG encoding circuit 11 adds camera
`code data to the bit Stream BSr according to the instruction
`generated by the memory control circuit for recording mode
`13. The camera code data is that of the video signals from
`which VSMOr was generated, which was in turn converted
`to the bit stream BSr.
`
`35
`
`6
`The recording/playback module 14 records the bit stream
`BSr output from the MPEG coding circuit 11 on a recording
`medium Such as a disk or tape. It also plays back the bit
`Stream from the recording medium and outputs the played
`back bit stream BSp.
`The MPEG decoding circuit 15 decodes the bit stream
`BSp played back by the recording/playback module 14 and
`converts it to video data VSMIp composed of the luminance
`Signal and the color difference Signals. It also detects the
`camera codes added to the bit stream BSp.
`The memory (1P) 16 to the memory (4P) 19 each store the
`corresponding part of the video data VSMIp output from the
`MPEG decoding circuit 15.
`The memory control circuit for playback mode 21 con
`trols memory to contain the video data VSMIp output from
`the MPEG decoding circuit 15 according to the camera
`codes detected by the MPEG decoding circuit 15.
`In this embodiment, the video data VSMIp decoded by the
`MPEG decoding circuit 15 is controlled according to the
`camera codes as follows. That is, the memory (1P) 16 to the
`memory (4P) 19 are controlled such that the video data
`corresponding to the Video signal entered from the Video
`input terminals 1 is stored in the memory (1P) 16, the video
`data corresponding to the Video signal entered from the
`video input terminals 2 is stored in the memory (2P) 17, the
`Video data corresponding to the Video Signal entered from
`the video input terminals 3 is stored in the memory (3P) 18,
`and the Video data corresponding to the Video signal entered
`from the video input terminals 4 is stored in the memory
`(4P) 19.
`The memory control circuit for playback mode 21 also
`accesses the memory (1P) 16 to the memory (4P) 19
`Sequentially to read Video data VSMIp Sequentially, one
`frame at a time. This way, it generates video data VSMOp
`composed of frames generated from the Video data corre
`sponding to the Video input terminals 1-4, with a frame from
`a terminal Sequentially followed by a frame from the next.
`The video encoder circuit 27 converts the video data
`VSMOp to the composite video signal.
`The camera code addition circuit 23 adds the correspond
`ing camera code to the composite Video signal, converted by
`the Video encoder circuit 27, during the Vertical blanking
`interval according to the instruction from the memory con
`trol circuit for playback mode 21.
`In a similar manner as the reproduction process of the
`conventional frame Switcher, the display control circuit 24
`determines the camera code and outputs the video signal
`from the video output terminal 25 so that only the video of
`the desired camera may be Selectively displayed.
`The video output terminal 25 is connected to a monitor,
`not shown in the figure, on which Video is displayed.
`Next, the components of the Video data recording/
`playback system shown FIG. 1 are described in more detail.
`First, the synchronization switching circuit 5 will be
`described.
`FIG. 2 is a diagram showing the outline of the Synchro
`nization Switching circuit 5.
`In this figure, number 31 refers to a video selection circuit
`A, number 32 refers to a video selection circuit B, number
`33 refers to an input timing Signal generation circuit, num
`bers 34 and 35 refer to A/D converters, numbers 36 and 37
`refer to switching circuits, number 38 refers to a FIFO (First
`IN First Out) memory A, number 39 refers to a FIFO
`memory B, number 40 refers to a write control circuit A,
`number 41 refers to a write control circuit B, number 42
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`Amazon / WAG Acquisition
`Exhibit 1019
`Page 24
`
`

`

`7
`refers to a read control circuit A, number 43 refers to a read
`control circuit B, number 44 refers to an output Selection
`circuit, number 45 refers to a reference timing Signal gen
`eration circuit, and number 47 refers to a camera code
`addition circuit.
`The Video Selection circuit A 31 Selects one of analog
`video signals VS1-VS4 entered from the video input termi
`nals 1-4 according to a control Signal SA from the reference
`timing Signal generation circuit 45.
`The video selection circuit B32 selects one of analog
`video signals VS1-VS4 entered from the video input termi
`nals 1-4 according to a control Signal SB from the reference
`timing Signal generation circuit 45.
`The A/D converter 34 converts the analog video signal
`selected by the video selection circuit A 31 to the digital
`Signal.
`The A/D converter 35 converts the analog video signal
`selected by the video selection circuit B 32 to the digital
`Signal.
`The video signal output from the A/D converter 34 is
`written into the FIFO memory A38 according to the control
`signal MWA sent from the write control circuit A40, and the
`video signal VOA written into the FIFO memory A38 is
`read according to the control Signal MRA Sent from the read
`control circuit A 42.
`The video signal output from the A/D converter 35 is
`written into the FIFO memory B39 according to the control
`signal MWB sent from the write control circuit B 41, and the
`video signal VQB written into the FIFO memory B 39 is
`read according to the control signal MRB sent from the read
`control circuit B 43.
`The output selection circuit 44 selects one of the video
`signal VQA read from the FIFO memory A38 and the video
`signal VOB read from the FIFO memory B 39 according to
`the control signal SQ sent from the reference timing Signal
`generation circuit 45.
`The reference timing Signal generation circuit 45 uses an
`internal crystal oscillator and So on to generate the 4fsc clock
`Signal with the frequency that is four times as high as that of
`the color Sub-carrier. It Supplies control Signals generated
`based on the 4fsc clock signal.
`The camera code addition circuit 47 generates camera
`codes, based on the Video Selection information from the
`video selection circuit A31, video selection circuit B32, and
`output Selection circuit 44, and adds the camera codes to the
`output video signal VO from the output selection circuit 44
`during the vertical blanking interval.
`The input timing Signal generation circuit 33 Separates the
`horizontal Sync signal and the vertical Sync signal from the
`video signals VS1-VS4 received by the video input termi
`nals 1-4. Based on the Separated horizontal Sync signal and
`the vertical Sync signal, the input timing Signal generation
`circuit generates the Start timing and the end timing of a
`frame of the Video signal. For each of the Video signals
`VS1-VS4, the input timing Signal generation circuit also
`generates the 4fsc clock signal which is in phase lock with
`the Sub-carrier of the Signal.
`The Start timing and the end timing of a frame of the
`Signal and the 4fsc clock Signal, which are generated for
`each of the analog video signals VS1-VS4, are called the
`input timing of the Signal.
`The Switching circuit 36 Selects and outputs one of input
`timings from those of each of video signals VS1-VS4,
`generated by the input timing Signal generation circuit 33,
`according to the control signal SA Sent from the reference
`timing Signal generation circuit 45.
`
`15
`
`25
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`US 6,314,137 B1
`
`8
`The Switching circuit 37 Selects and outputs one input
`timing from those of each of video signals VS1-VS4,
`generated by the input timing Signal generation circuit 33,
`according to the control signal SB Sent from the reference
`timing Signal generation circuit 45.
`The write control circuit A 40 generates the control Signal
`MWA which causes one frame of the video signal to be
`written into the FIFO memory A38 according to the control
`Signal from the reference timing Signal generation circuit 45
`and the input timing from the Switching circuit 36. The
`circuit also Supplies the 4fsc clock signal, included in the
`input timing, to the A/D converter 34.
`The write control circuit B 41 generates the control Signal
`MWB which causes one frame of the video signal to be
`written into the FIFO memory B39 according to the control
`Signal from the reference timing Signal generation circuit 45
`and the input timing from the Switching circuit 37. The
`circuit 41 also Supplies the 4fsc clock signal, included in the
`input timing, to the A/D converter 35.
`The read control circuit A 42 generates the control Signal
`MRA Specifying the timing at which the Video Signal is to be
`read from the FIFO memory A38 according to the control
`Signal from the reference timing Signal generation circuit 45.
`The read control circuit B 43 generates the control Signal
`MRB specifying the timing at which the video signal