`
`
`
`
`
`
`
`(12) United States Patent
`US 7,143,432 B1
`(10) Patent N0.:
`
`
`
`
`
`
`
`
`
`(45) Date of Patent:
`Nov. 28, 2006
`Brooks et al.
`
`
`
`US007143432B1
`
`
`
`(54) SYSTEM FOR TRANSFORMING
`
`
`
`STREAMING VIDEO DATA
`
`
`
`
`
`
`(75)
`
`
`
`
`
`
`
`
`Inventors: Roger K. Brooks, Palo Alto, CA (US);
`Stephen A- Molloy, Los Gatos, CA
`
`
`
`
`
`
`
`
`
`
`(US); Chi-Te Fengs Hsi Chill (TW);
`
`
`
`
`Qing Zhang, SulmyVales CA (Us);
`
`
`
`
`
`Yanda Ma, M1lp1tas, CA (US); Dave
`
`
`
`
`Singhal, San Jose, CA (US)
`
`
`
`
`
`
`
`
`
`
`(73) Assignee: Vidiator Enterprises Inc., Nassau (BS)
`
`
`
`
`
`
`
`
`( * ) Notice:
`Subject to any disclaimer, the term of this
`
`
`
`
`patent is extended or adjusted under 35
`
`
`
`U.S.C. 154(b) by 0 days.
`
`
`
`(21) Appl. No.: 09/502,409
`
`
`
`(22)
`
`
`
`Flled:
`
`
`
`
`
`
`Feb. 10’ 2000
`_
`_
`
`
`Related U'S' Application Data
`
`
`
`
`
`
`
`
`
`
`(60) Provisional application No. 60/157,468, filed on Oct.
`
`
`1; 1999,
`
`
`
`(51)
`
`
`
`(56)
`
`
`
`
`
`Int. Cl.
`
`
`
`(2006.01)
`H04N 7/173
`
`
`
`
`
`
`
`(52) US. Cl.
`...................... 725/105; 725/ 109; 725/114;
`
`
`
`
`
`725/115; 725/117; 725/118; 709/231; 709/235;
`
`
`
`
`709/203; 709/247; 37 5/240,24; 37 5/240,25;
`
`
`37 5/240,26; 370/437
`
`
`
`
`
`
`(58) Field of Classification Search ........... 375/24002,
`
`
`
`
`
`375/24024; 240.11; 240.01; 240.25; 240.26;
`
`
`
`
`370/420; 398, 437; 725/118, 105; 114; 109;
`
`
`
`
`
`
`
`725/115; 117; 709/247; 246, 231; 235; 203
`
`
`
`
`
`
`
`See application file for complete search history.
`
`
`References Cited
`US. PATENT DOCUMENTS
`
`
`
`
`
`5,619,995 A *
`
`5,671,319 A
`
`5,768,535 A
`
`5,903,775 A
`
`5,914,751 A
`
`
`............... 348/398
`4/ 1997 Lobodzinski
`
`
`
`
`9/1997 Ueda ;;;;;;;;;;;;;;;;;;;; 386/96
`
`
`
`
`6/1998 Chaddha et al.
`395/200.77
`
`
`
`
`5/1999 Murray ....................... 395/853
`
`
`
`6/l999 Korth ......................... 348/409
`
`
`
`
`
`
`
`
`
`
`
`
`.................... 709/246
`5,970,233 A * 10/1999 Liu et a1.
`
`
`
`
`
`6,011,868 A
`1/2000 van den Branden
`
`
`
`
`
`et a1.
`.......................... 382/233
`
`
`
`
`
`
`
`
`2/2000 Bhatt et a1.
`...... 348/441
`6,028,639 A
`
`7/2000 Guetz et a1.
`........... 375/240.11
`6,091,777 A *
`
`
`
`
`
`
`
`
`
`
`6,141,447 A * 10/2000 Linzer et a1.
`............... 382/236
`
`
`
`
`
`6,141,693 A
`10/2000 Perlman et a1.
`............. 709/236
`
`
`
`
`
`
`
`
`
`
`(Continued)
`
`
`
`Primary Examinerishawn S. An
`
`
`
`
`
`
`(74) Attorney, Agent, or Firmiseed IP Law Group PLLC
`
`
`
`(57)
`
`
`
`ABSTRACT
`
`
`
`
`
`
`
`
`
`According to one embodiment, a circuit configured to form
`an out ut Video stream includes a resolution modification
`
`
`
`
`
`
`
`circuitionfigured to receive a plurality ofVideo frames from
`
`
`
`
`
`
`
`
`
`
`
`
`
`a frame buffer, and configured to modify resolution of the
`
`
`
`
`
`
`
`
`plurality of Video frames, when the desired resolution for the
`
`
`
`
`
`
`
`output Video stream is different than a resolution of the input
`
`
`
`
`
`
`
`
`Video stream, the plurality of frames of data deriVed from an
`
`
`
`
`
`
`
`
`input Video stream, a frame reducing circuit coupled to the
`
`
`
`
`
`
`resolution reducing circuit configured to reduce a number of
`
`
`
`
`
`
`
`
`
`Video frames in the plurality of Video frames from the
`
`
`
`
`
`
`
`
`
`resolution reducing circuit, when a desired frame rate for the
`
`
`
`
`
`
`
`
`output Video stream is different than a frame rate of the input
`
`
`
`
`
`
`
`
`video stream, a depth reduction circuit coupled to the frame
`
`
`
`
`
`
`
`
`reducing circuit configured to reduce bit depth of the plu-
`
`
`
`
`
`
`
`
`
`rality ofVideo frames from the frame reducing circuit, when
`
`
`
`
`
`
`
`
`
`
`a desired bit depth for the output Video stream is different
`
`
`
`
`
`
`
`
`
`
`than a bit depth 0f the lan Video stream, and a rate
`
`
`
`
`
`
`
`reduction circuit coupled to the depth reduction circuit,
`
`
`
`
`
`
`
`
`
`configured to scale the plurality of Video frames from the
`
`
`
`
`
`
`
`
`depth reduction circuit, in response to a desired bit rate for
`
`
`
`
`
`
`
`
`
`the output Video stream, and an encoder coupled to the rate
`
`
`
`
`
`
`
`reduction circuit, configured to encode the plurality of Video
`
`
`
`
`
`
`
`
`
`
`frames from the rate reduction circuit into the output Video
`
`
`
`stream is also contemplated
`
`17 Claims, 8 Drawing Sheets
`
`
`
`
`
`
`
`
`5 10
`520
`530
`540
`
`/
`/
`L
`/
`
`
`
`DCT
`
`
`
`
`
`Frame Rate
`
`
`
`
`
`Color De 1h
`
`
`
`
`
`/500
`
`
`
`
`
`
`
`
`
`
`Reducer { ReduceiJ
`Cropper J subsompler
`
`
`
`
`550
`/560
`/570
`
`
`
`
`
`
`
`Encoder { Encrypior
`Control
`
`Bilrole
`
`
`
`
`MPEG—1,2,4
`
`
`
`Page 1 of 23
`
`GOOGLE EXHIBIT 1006
`
`GOOGLE EXHIBIT 1006
`
`Page 1 of 23
`
`

`

`
`
`US 7,143,432 B1
`
`
`Page 2
`
`U.S. PATENT DOCUMENTS
`
`
`
`
`
`
`6,141,709 A
`
`
`6,160,544 A
`
`
`6336395 B1
`
`
`6,275,536 B1
`6,292,834 B1
`
`
`6,298,385 B1
`
`
`6,343,313 B1
`
`
`
`
`
`
`
`10/2000 Cutter _........................ 710/100
`
`
`
`
`
`
`12/2000 HayaShl etal'
`345/327
`
`
`
`
`
`”001 sez‘m et 31'
`““““ 345/328
`
`
`
`
`
`
`8/2001 Chen etal.
`.. 375/240.25
`9/2001 Ravi et al. ............ 709/233
`
`
`
`
`
`
`10/2001 Sparks et al.
`709/233
`.....
`
`
`
`
`
`
`1/2002 Salesky et al.
`............. 709/204
`
`
`
`
`
`
`6,398,245 B1
`
`
`6,490,627 B1
`
`
`
`
`6,505,299 B1
`
`
`6,525,746 B1
`
`6,567,986 B1*
`
`6795 863 131*
`’
`’
`
`
`
`
`
`................ 280/228
`6/2002 Gruse et al.
`
`
`
`
`.....
`12/2002 Kalra et al.
`709/231
`
`
`
`
`
`
`
`
`
`“2003 Zeng et 31.
`..
`713/160
`
`
`
`
`2/2003 Lau etal.
`345/725
`
`
`
`
`................. 725/118
`5/2003 Ward etal.
`
`
`”004 Doty Jr
`709/231
`' “““““““““““
`’
`
`
`
`
`
`
`
`
`
`
`
`* cited by examiner
`
`
`
`
`
`Page 2 of 23
`
`Page 2 of 23
`
`

`

`
`U.S. Patent
`
`
`
`
`
`Nov. 28, 2006
`
`
`
`
`
`Sheet 1 of 8
`
`
`
`US 7,143,432 B1
`
`
`
`SAMPLE NETWORK TOPOLOGY
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`750
`
`
`
`240
`
`
`
`
`
`
`
`I Gateway is optional; used I
`
`
`
`for archiving and/or
`
`
`
`
`further video processing.
`
`
`
`
`2,0
`
`
`
`
`
`
`
`56Kbit
`
`160x120/
`5fPS/COIOF/
`
`MPEG4
`120 E
`
`
`
`
`_
`14 4Kbps
`i 5fps/160x120/
`
`
`color/MPEG4
`
`
`Gateway
`L ————————————————————— _l
`
`
`
`
`WAP Server +
`Gateway Q
`
`
`
`’50
`
`
`
`\
`
`
`
`
`
`220
`
`
`
`230
`
`
`
`1.90
`
`
`
`
`
`10Kbit/wireless
`
`
`b/w 64‘48/
`
`
`'
`bitmap 3fps
`I ’80
`
`Cellphone/
`microbrowser/
`
`WAP
`
`
`
`
`
`
`
`
`
`
`
`730 Q.;\ I
`
`\/
`
`
`PDA, WinCE
`
`
`I40
`
`384Kbit/DSL
`
`3:23180/10fps/coi0r
`.
`
`El
`
`:
`
`
`
`
`Page 3 of 23
`
`Page 3 of 23
`
`

`

`
`U.S. Patent
`
`
`
`
`
`Nov. 28, 2006
`
`
`
`
`Sheet 2 of 8
`
`
`
`US 7,143,432 B1
`
`510
`
`
`
`
`
`300
`
`
`/
`
`0
`35
`
`
`
`MONITOR
`
`
`
`,
`
`.370
`
`
`
`330
`
`
`
`320
`
`
`
`
`
`
`
`
`
`
`
`
`DISK DRIVE
`
`
`
`
`
`RANDOM
`
`ACCESS
`
`MEMORY
`
`
`
`
`PROCESSOR I
`
`
`
`A l
`
`
`
`
`NETWORK
`INTERFACE
`
`
`PROCESSING
`UNIT
`
`
`
`
`KEYBOARD
`
`
`
`‘
`
`350
`
`
`
`340
`
`
`
`330
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Page 4 of 23
`
`Page 4 of 23
`
`

`

`893M3:28
`
`
`
`US 7,143,432 B1
`
`
`
`m6%
`
`0
`
`Mmm,
`
`3mhS
`
`v.RV
`
`
`
`V580EomtmV388:9;
`
`3.28.282
`
`QMU
`
`
`
`
`
`:55men:_mL06EV\\nm‘"
`
`Na:_
`
`EmEoomm:
`
`53.5
`
`$5.253
`
`82>uEEogm
`
`
`
`
`
`
`
`
`
`Page 5 of 23
`
`Page 5 of 23
`
`
`
`

`

`
`
`nma
`
`P8m\\
`
`
`
`t3m.3%awnSm,
`
`N
`
`m2
`
`u
`
`
`
`US 7,143,432 B1
`
`may§§an
`
`.228m53:052.78%225
`
`$305
`
`w“2%
`
`
`
`
`
`8,$2.3m$0331SEEomnam538m£30ES20m9:9:BoVo
`
`QMU
`
`
`
`
`
`
`
`
`
`Page 6 of 23
`
`Page 6 of 23
`
`
`

`

`
`U.S. Patent
`
`
`
`
`
`Nov. 28, 2006
`
`
`
`
`Sheet 5 of 8
`
`
`
`US 7,143,432 B1
`
`
`
` REQUEST VIDEO DATA FROM
`
`
`
`
`
` 600
`
`
`
`A VIDEO SOURCE
`
`
`
`
`
`
`
`
`
`
`DETERMINE BANDWIDTH
`
`
`
`6/0
`LIMITATIONS AND FORMAT
`
`
`REQUIREMENTS
`
`
`
`
`
`
`
`
`
`
`
`
`
`
` VIDEO SOURCE
`
`
`
`
`
`
`CAN PROVIDE
`
`
`VIDEO?
`
`
`
`
`N0
`
`
`
`
`
`
`
`
`REDIRECT REQUEST TO A
`
`
`GATEWAY COMPUTER
`
`
`
`
`
`
`
`DETERMINE BANDWIDTH
`
`
`
`LIMITATIONS AND FORMAT
`
`REQUIREMENTS
`
`
`
`
`
`
`
`
`
`
`
` GATEWAY CAN
`
`
`
`
`PROVIDE VIDEO?
`
`
`
`
` COMMUNICATE WITH VIDEO
`
`
`
`
`
`SOURCE TO OBTAIN INPUT
`
`
`VIDEO DATA
`
`
`
`
`560
`
`
`
`
`FIG. 5A
`
`
`
`Page 7 of 23
`
`Page 7 of 23
`
`

`

`
`U.S. Patent
`
`
`
`
`
`Nov. 28, 2006
`
`
`
`
`Sheet 6 of 8
`
`
`
`US 7,143,432 B1
`
`
`
`
`BUFFER INPUT VIDEO DATA
`
`
`
`
`
`
`
`
`
`TRANSCODE INPUT VIDEO
`
`
`
`DATA TO THE DESIRED
`
`
`OUTPUT VIDEO
`
`
`
`FORMAT INTO TCP/IP
`
`PACKETS
`
`
`
`670
`
`580
`
`590
`
`700
`
`
`
`
`
`
`
`
`
`IN FRAME BUFFER
`
`
`
`
`720
`
`
`
`
`
`
`
`
`
`
`
`
`SEND TCP/IP PACKETS To
`
`
`THE REQUESTER
`
`
`
`DECODE TCP/IP PACKETS
`TO RECOVER VIDEO DATA
`
`
`
`
`STREAM
`
`
`
`
`
`
`
`DECOMPRESS/PLAY VIDEO
`
`
`DATA STREAM
`
`
`
`7 ’0
`
`
`
`
`
`
`
`FIG. 58
`
`Page 8 of 23
`
`Page 8 of 23
`
`

`

`
`U.S. Patent
`
`
`
`
`
`Nov. 28, 2006
`
`
`
`
`Sheet 7 of 8
`
`
`
`US 7,143,432 B1
`
`
`
`
`INPUT VIDEO DATA STORED
`
`
`
`IN FRAME BUFFER
`
`
`
`
`
`
`CROP INPUT VIDEO DATA
`
`
`
`500
`
`
`
`IN>0UT
`
`8/0
`
`
` INPUT COLOR
`
`
`
`
`
`DEPTH COMPARED
`
`
`
`
`
`TO OUTPUT COLOR
`
`
`DEPTH
`
`OUT>IN
`
`
`
`
`
`820
`
`
`
`
`
`DOWNSCALE COLOR DEPTH
`
`
`
`
`OF INPUT VIDEO DATA
`
`
`
`
`
`
`UPSCALE COLOR DEPTH OF
`
`
`
`INPUT VIDEO DATA
`
`
`
`830
`
`
`
`
`
`
`
`
`
`
`
`
`
`
` INPUT RESOLUTION
`
`OUT>IN
`IN>0UT
`
`
`COMPARED TO OUTPUT
`
`
`
`RESOLUTION
`
`
`840
`
`
`
`
`
`850
`
`
`
`
`
`DOWNSCALE RESOLUTION
`
`
`
`
`OF INPUT VIDEO DATA
`
`
`
`UPSCALE RESOLUTION OF
`
`
`
`INPUT VIDEO DATA
`
`
`
`860
`
`
`
`
`
`
`
`
`
`
`870
`
` INPUT FRAM
`
`
`
`
`IN>0UT
`
`
`RATE COMPARED
`
`
`
`TO OUTPUT FRAME
`
`
`RATE
`
`
`
`
`0UT>IN
`
`
`
`880
`
`
`
`
`
`
`REDUCE FRAME RATE OF
`
`
`
`INPUT VIDEO DATA
`
`
`
`
`
`
`INCREASE FRAME RATE OF
`
`
`
`INPUT VIDEO DATA
`
`
`
`890
`
`
`
`FIG.
`
`
`
`6A
`
`
`
`Page 9 of 23
`
`Page 9 of 23
`
`

`

`
`U.S. Patent
`
`
`
`
`
`Nov. 28, 2006
`
`
`
`
`Sheet 8 of 8
`
`
`
`US 7,143,432 B1
`
`
`”MI”
`
`
`
`900
`
`INPUT BIT RAT
`
`
`
`OUT>IN
`
`COMPARED TO
`
`OUTPUT BIT RATE
`
`
`
`
`
`
`
`
`
`
`.920
`
`
`
`
`ENCODING
`FORMAT?
`
`
`QUICKTIME
`
`
`
`
`
` DECREASE BIT RATE OF
`DECREASE BIT RATE OF
`
`
`
`
`
`
`
`
`
`
`INPUT VIDEO DATA
`INPUT VIDEO DATA
`
`
`
`
`
`
`
`
`
`
`
`ENCODE INPUT DATA
`ENCODE INPUT DATA
`
`
`
`
`
`
`
`
`STREAM INTO *.MPG
`STREAM INTO *.MOV
`
`
`
`
`
`
`
`
`FORMAT
`FORMAT
`
`
`
`
`
`
`
`
`
`
`
`
`
`9/0
`
`940
`
`Page 10 of 23
`
`
`
`
`
`
`
`
`
`ENCODE INPUT DATA
`
`
`
`STREAM INTO *.AV|
`
`
`
`FORMAT
`
`
`
`
`
`
`ENCRYPT DATA STREAM
`
`
`
`
`
`
`
`FIG.
`
`
`
`615’
`
`
`
`Page 10 of 23
`
`

`

`
`
`US 7,143,432 B1
`
`2
`
`1
`
`SYSTEM FOR TRANSFORMING
`
`STREAMING VIDEO DATA
`
`
`
`
`
`
`
`CROSS-REFERENCES TO RELATED
`
`APPLICATIONS
`
`
`
`
`
`
`
`
`
`
`
`
`The present invention disclosure claims priority to Pro-
`
`
`
`
`
`
`
`
`visional US. patent application Ser. No. 60/157,468, filed
`Oct. 1, 1999, entitled Internet Camera Video Producer. This
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`application is herein by incorporated by reference for all
`purposes.
`
`BACKGROUND OF THE INVENTION
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`The present invention relates to devices for providing
`
`
`
`
`
`
`media across a computer network. In particular, the present
`
`
`
`
`
`
`
`invention relates to systems configured to adapt
`input
`
`
`
`
`
`
`
`streams of video data to meet desired parameters for output
`streams of video data.
`
`
`
`
`
`
`
`
`
`
`
`The legend of the Tower of Babel tells us that humans
`
`
`
`
`
`
`
`once spoke a common language. However, divine interven-
`
`
`
`
`
`
`
`
`tion soon foiled human’s plans in the building of the tower
`
`
`
`
`
`
`
`
`
`to the sky. This was done by making humans speak in
`
`
`
`
`
`
`
`
`different languages. As a result, the legend goes, humans did
`not understand each other, humans misunderstood others’
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`intentions, and ultimately humans moved away from each
`other. Ever since that day, humans have been benefited and
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`plagued by language and cultural differences.
`
`
`
`
`
`
`
`Translating between different languages has also been an
`
`
`
`
`
`
`
`
`issue considered in popular fiction.
`In Douglas Adam’s
`
`
`
`
`
`
`
`well-known Hitchhiker’s Guide to the Galaxy (1979), the
`
`
`
`
`
`
`
`problem of different languages was solved by the introduc-
`tion of a “Babel fish.” Adams described the Babel fish as a
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`“small, yellow and leechlike, and probably the oddest thing
`in the Universe. It feeds on the brainwave energy received
`
`
`
`
`
`
`
`
`not from its own carrier but from those around it. It absorbs
`
`
`
`
`
`
`
`
`
`
`all unconscious mental frequencies from this brainwave
`
`
`
`
`
`
`
`energy to nourish itself with. It then excretes into the mind
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`of its carrier a telepathic matrix formed by combining the
`
`
`
`
`
`
`
`conscious thought frequencies with nerve signals picked up
`
`
`
`
`
`
`
`
`
`
`from the speech centers of the brain which has supplied
`
`
`
`
`
`
`
`
`them. The practical upshot of all this is that if you stick a
`
`
`
`
`
`
`
`
`
`Babel fish in your ear you can instantly understand anything
`
`
`
`
`
`
`
`
`said to you in any form of language.” While this, of course,
`may not really exist, a commercial translation service has
`
`
`
`
`
`
`
`actually been named after the Babel fish.
`
`
`
`
`
`
`
`On the Internet, Altavista.com provides a translation
`
`
`
`
`
`
`service that translates text in a web page from one selected
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`language to another selected language. The tool is called the
`“Babel fish.” To use this tool, the user enters a web address
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`and then specifies the language direction, e.g. English to
`French. In response, the Babel fish will translate the text that
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`appears on the page into the new language, all other aspects
`
`
`
`
`
`
`
`
`
`
`
`of the page will remain the same. The quality of the
`
`
`
`
`
`
`
`
`translations may vary with actual usage. For example,
`
`
`
`
`
`
`
`
`
`translating the phrase “To jump the gun” to French and back
`
`
`
`
`
`
`
`
`
`
`
`into English returns the phrase “To connect the gun.” As can
`be seen, the quality of such services are not quite ideal.
`
`
`
`
`
`
`
`
`
`
`
`Further, such services do not address non-text data such as
`
`
`
`
`
`
`
`
`audio and visual (media) data.
`
`
`
`
`
`On the web, other types of data than text are also
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`displayed to users. Such data include media such as images,
`sounds, video, and the like. With such data, instead of being
`
`
`
`
`
`
`
`
`
`
`in different languages, the data are stored and transmitted
`
`
`
`
`
`
`
`
`
`into different formats. Most of the different media formats
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`are incompatible. Currently, there are a multitude of stan-
`
`
`
`
`
`
`
`
`
`dards or formats for each of them, for example, images may
`
`10
`
`
`
`15
`
`
`
`20
`
`25
`
`
`
`30
`
`
`
`35
`
`
`
`40
`
`
`
`45
`
`
`
`50
`
`
`
`55
`
`
`
`60
`
`
`
`65
`
`
`
`Page 11 of 23
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`be transmitted in formats including *.jpg, *.gif, *.bmp,
`
`
`
`
`
`
`
`*.pcx, and the like; sounds may be transmitted in formats
`
`
`
`
`
`
`
`
`
`including *.wav, *.mp3, *.aiff, and the like; and video may
`
`
`
`
`
`
`
`be transmitted in formats including *.avi, *.mov, *.rm,
`
`
`
`
`
`
`*.mpg, *.asf, vivo and the like.
`To view or hear data in any of the above media formats
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`requires an appropriate viewing (translation) application
`
`
`
`
`
`
`
`program. That is, an application program is required to
`
`
`
`
`
`
`
`
`
`
`decode the transmitted data and output
`the data to the
`
`
`
`
`
`
`
`computer of the requester. The requesting computer must
`
`
`
`
`
`
`
`have preinstalled many different viewer applications. This is
`
`
`
`
`
`
`
`
`
`
`so that the computer can receive, decode, and display data
`
`
`
`
`
`
`stored in the many different media formats.
`
`
`
`
`
`
`
`One drawback to requiring multiple viewers is that this
`
`
`
`
`
`
`
`
`solution is not appropriate for all devices connected to the
`
`
`
`
`
`
`
`
`web. In particular, it is not appropriate for future wireless
`devices, or the like. Such devices may include cellular
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`telephones, wireless PDAs, network appliances (refrigera-
`tors, toasters, ovens, smart houses), wrist watches, wearable
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`computers, and the like. Because many of these devices will
`
`
`
`
`
`
`
`have lower amounts of memory and performance compared
`
`
`
`
`
`
`
`
`
`to desktop computers,
`these devices will not be able to
`
`
`
`
`
`
`
`display a large number of viewers. As a result, these devices
`
`
`
`
`
`
`
`
`will not be able to play many different media formats.
`
`
`
`
`
`
`
`
`One solution proposed to address this problem has been to
`
`
`
`
`
`
`
`
`standardize upon one particular format. For example, all
`wireless devices in the future would be able to receive and
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`output data stored in one particular media format, such as
`
`
`
`
`
`
`
`MPEG-4 (Motion Pictures Expert Group), or the like.
`A drawback to this solution is that in theory, this solution
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`is reasonable, however, in practice, it is not likely to happen.
`
`
`
`
`
`
`
`
`
`The primary reason is that there are many competing media
`
`
`
`
`
`
`
`
`formats available, and most are backed by separate compa-
`
`
`
`
`
`
`
`
`
`nies. For example, Windows pushes the *avi media format,
`
`
`
`
`
`
`
`
`Apple pushes the *.mov media format, Real Networks
`
`
`
`
`
`
`
`
`
`pushes the *.rm format, Vivo Software pushes its own
`format, and the like. It is doubtful that such companies will
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`agree on a single media format.
`Another drawback is that even if there is one standardized
`
`
`
`
`
`
`
`media format, different requesting devices will have differ-
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`ent performance, resolutions, bandwidth, and the like. For
`
`
`
`
`
`
`
`
`example, a PDA may have the ability to display 80x60 pixel
`24-bit color images, however a wrist watch may have the
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`ability to display only 40x30 pixel 8-bit gray scale images,
`and the like. Because many future wireless network devices
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`or appliances will have different performance, bandwidth,
`and the like, the source of the media will have to store the
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`data in just as many data files. For example, if there are 100
`
`
`
`
`
`
`
`
`different telephones with web access, each telephone with its
`
`
`
`
`
`
`
`
`
`own performance, bandwidth factors, and the like, to sup-
`
`
`
`
`
`
`
`
`
`
`port all these telephones the media source will have to store
`the data in 100 different data files. This is highly inefficient
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`and/or impractical to implement, not to mention impossible
`for live data.
`
`
`
`
`
`
`
`
`
`
`
`Typically, media sources, such as web sites, provide only
`
`
`
`
`
`
`
`
`a limited number of media formats and performance char-
`
`
`
`
`
`
`
`
`acteristics. For example, a typical site may only offer a
`media clip in the Quicktime format at 320x240 resolution or
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`160x120 resolution. If a requesting device cannot process
`Quicktime format,
`the user is out of luck. As another
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`example, if the requesting device can only play 80x60
`resolution Quicktime movies, the media source is wasting its
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`output bandwidth sending the requesting device 160x120
`resolution Quicktime movies.
`
`
`
`
`Page 11 of 23
`
`

`

`
`
`US 7,143,432 B1
`
`4
`
`
`
`
`3
`
`
`
`
`
`
`
`Thus what is needed in the industry are improved systems
`
`
`
`
`
`
`
`for providing requesting devices with media in the format
`
`
`
`
`
`
`and performance appropriate and/or requested for these
`
`
`requesting devices.
`SUMMARY OF THE INVENTION
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`The present invention relates to systems for transcoding
`
`
`
`
`
`
`
`
`and transforming video streams. In particular, the present
`
`
`
`
`
`
`
`invention relates to apparatus for adapting input streams of
`
`
`
`
`
`
`
`
`video data to meet desired parameters for output streams of
`
`
`
`
`
`
`
`
`video data. On the fly adaptation to desired output param-
`
`
`
`
`
`
`
`
`
`eters may be made with respect to display size, frame rate,
`
`
`
`
`
`
`
`
`bit-depth, bit rate, encoding format, and the like.
`
`
`
`
`
`According to one embodiment, a circuit configured to
`
`
`
`
`
`
`
`form an output video stream includes a resolution modifi-
`
`
`
`
`
`
`
`cation circuit configured to receive a plurality of video
`
`
`
`
`
`
`
`
`frames from a frame buffer, and configured to modify
`
`
`
`
`
`
`resolution of the plurality of video frames, when the desired
`
`
`
`
`
`
`
`
`resolution for the output video stream is different than a
`
`
`
`
`
`
`
`resolution of the input video stream, the plurality of frames
`of data derived from an input video stream, and a frame
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`reducing circuit coupled to the resolution reducing circuit
`
`
`
`
`
`
`
`configured to reduce a number of video frames in the
`
`
`
`
`
`
`
`
`plurality of video frames from the resolution reducing
`
`
`
`
`
`
`
`
`
`
`circuit, when a desired frame rate for the output video stream
`is different than a frame rate of the input video stream. The
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`circuit also includes a depth reduction circuit coupled to the
`
`
`
`
`
`
`
`frame reducing circuit configured to reduce bit depth of the
`
`
`
`
`
`
`
`
`
`plurality of video frames from the frame reducing circuit,
`
`
`
`
`
`
`
`
`
`when a desired bit depth for the output video stream is
`
`
`
`
`
`
`
`
`different than a bit depth of the input video stream, and a rate
`
`
`
`
`
`
`
`reduction circuit coupled to the depth reduction circuit,
`
`
`
`
`
`
`
`
`
`configured to scale the plurality of video frames from the
`
`
`
`
`
`
`
`
`depth reduction circuit, in response to a desired bit rate for
`
`
`
`
`
`
`
`
`
`the output video stream. An encoder coupled to the rate
`
`
`
`
`
`
`
`reduction circuit, configured to encode the plurality of video
`frames from the rate reduction circuit into the output video
`
`
`
`
`
`
`
`
`
`
`
`
`
`stream is also contemplated.
`
`
`
`
`
`
`According to another embodiment, a circuit for reducing
`
`
`
`
`
`
`bandwidth of an incoming video stream includes a cropping
`
`
`
`
`
`
`
`portion configured to receive a first video frame, a second
`video frame, and a third video frame from a frame buffer,
`
`
`
`
`
`
`
`
`
`and configured to determine the resolution of the first video
`
`
`
`
`
`
`
`frame, the second video frame, and the third video frame, the
`
`
`
`
`
`
`
`
`
`
`first video frame, the second video frame, and the third video
`
`
`
`
`
`
`
`
`
`
`frame derived from the incoming video stream, and a
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`sampling portion coupled to the cropping portion, config-
`
`
`
`
`
`
`
`ured to receive a desired output resolution, and configured to
`subsample the first video frame, the second video frame, and
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`the third video frame to respectively form a first sampled
`video frame, a second sampled video frame, and a third
`
`
`
`
`
`
`
`
`sampled video frame. The circuit may also include a frame
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`rate reducing portion coupled to the sampling portion,
`
`
`
`
`
`
`
`
`configured to receive a desired output frame rate, and
`
`
`
`
`
`
`
`
`configured to drop the second sampled video frame, and a
`
`
`
`
`
`
`
`
`depth reducing portion coupled to the frame rate reducing
`
`
`
`
`
`
`
`portion, configured to receive a desired output bit-depth, and
`
`
`
`
`
`
`
`
`configured to reduce bit depth of the first sampled video
`
`
`
`
`
`
`
`
`frame and the third sampled video frame to respectively
`form a first reduced video frame and a third reduced video
`
`
`
`
`
`
`
`
`
`frame.
`
`
`
`
`
`
`
`for
`According to yet another embodiment, a circuit
`
`
`
`
`
`
`dynamically reducing bandwidth of an input video stream to
`
`
`
`
`
`
`
`meet bandwidth requirements for an output video stream
`
`
`
`
`
`
`includes a receiving portion configured to receive frames of
`
`
`
`
`
`
`
`
`data derived from the input video stream, a processing
`
`10
`
`
`
`15
`
`
`
`20
`
`25
`
`
`
`30
`
`
`
`35
`
`
`
`40
`
`
`
`45
`
`
`
`50
`
`
`
`55
`
`
`
`60
`
`
`
`65
`
`
`
`Page 12 of 23
`
`
`
`
`
`
`
`
`
`portion coupled to the receiving portion, configured to
`
`
`
`
`
`
`
`receive a bandwidth requirement
`for the output video
`
`
`
`
`
`
`
`stream, and configured to reduce bandwidth used by the
`
`
`
`
`
`
`frames of data in response to the bandwidth requirement to
`
`
`
`
`
`
`
`form bandwidth reduced frames of data, and an encoding
`
`
`
`
`
`
`portion coupled to the processing portion, configured to
`
`
`
`
`
`
`
`
`
`receive an encoding format for the output video stream, and
`
`
`
`
`
`
`configured to encode bandwidth reduced frames of data to
`
`
`
`
`
`form the output video stream.
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`A more complete appreciation of the invention and many
`
`
`
`
`
`
`of the attendant advantages thereof will be readily obtained
`as the same becomes better understood by reference to the
`
`
`
`
`
`
`
`
`
`
`
`
`following detailed description when considered in connec-
`
`
`
`
`
`
`tion with the accompanying drawings, wherein:
`
`
`
`
`
`
`FIG. 1 illustrates a block diagram of a usage scenario
`
`
`
`
`
`according to an embodiment of the present invention;
`
`
`
`
`
`FIG. 2 is a block diagram of a typical gateway computer
`
`
`
`
`
`and its interface to client computers according to an embodi-
`
`
`
`
`ment of the present invention;
`
`
`
`
`
`FIG. 3 illustrates a block diagram of an embodiment of
`
`
`
`the present invention;
`
`
`
`
`
`
`FIG. 4 illustrates a block diagram of a transcoding com-
`
`
`
`
`
`pound according to an embodiment of the present invention;
`
`
`
`
`
`
`FIGS. 5A and 5B illustrates an overview flow diagram
`
`
`
`
`
`
`according to an embodiment of the present invention; and
`FIGS. 6A and 6B illustrates a more detailed embodiment
`
`
`
`
`
`
`
`
`
`of the present invention.
`DESCRIPTION OF THE SPECIFIC
`
`
`EMBODIMENTS
`
`
`
`
`
`
`
`
`
`
`
`FIG. 1 illustrates a block diagram according to an embodi-
`
`
`
`
`
`
`ment of the present invention. FIG. 1 illustrates a gateway
`
`
`
`
`
`
`
`computer 100, and plurality of computer systems 1107150
`
`
`
`
`
`
`
`
`coupled to each other via a computer network 160. In the
`
`
`
`
`
`
`present embodiment, a video camera 170 is coupled to
`
`
`
`
`
`
`
`computer system 110, and a network appliance 180 is
`
`
`
`
`
`
`
`
`coupled via a wireless network 190 to computer system 150.
`
`
`
`
`
`
`
`Computer systems 1107150 are coupled to computer net-
`work 160 via network connections 2007240.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`In the present embodiment, computer network 160 is the
`Internet. In alternative embodiments of the present inven-
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`tion, computer network 160 may be any computer network,
`such as an intranet, a computer network, a local area
`
`
`
`
`
`
`
`
`network, an intemet, and the like. Computer network 160
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`provides data communication among computer systems
`
`
`
`
`
`
`
`1107150 and gateway computer 100. Data communication
`may include transfer of HTML based data, textual data, form
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`submissions, plug-in programs or viewers, applets, audio
`
`
`
`
`
`
`
`
`
`data, video data, and the like. Although computer network
`
`
`
`
`
`
`
`
`
`160 is illustrated as a single entity, as is the case with the
`Internet, it should be understood that computer network 160
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`may actually be a network of individual computers and
`servers.
`
`
`In the present embodiment, network connections 1907240
`
`
`
`
`
`
`
`have typical maximum bandwidth characteristics that are
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`known a priori. In FIG. 1, for example, network connection
`
`
`
`
`
`
`
`
`200 is shown as a 1.5 mega bit per second (Mbps) TI
`correction, as is common with businesses; network connec-
`
`
`
`
`
`
`tion 210 is shown as a 56 kilobit per second (kbps) connec-
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`tion as is common with home computers, set top boxes, and
`the like; network connection 220 is shown as a 14.4 kbit
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`(kbps) connection to personal digital assistants (PDAs), such
`as PalmOS devices, WindowsCE devices, and the like. In
`
`
`
`
`
`
`
`
`
`
`Page 12 of 23
`
`

`

`
`
`US 7,143,432 B1
`
`5
`
`
`
`
`
`
`
`
`this example, network connection 230 is shown as a 384
`
`
`
`
`
`
`
`kbps digital subscriber line (DSL) connection as is common
`
`
`
`
`
`
`
`with small businesses or power users. In alternative embodi-
`
`
`
`
`
`
`
`ments, network connection 230 may be smaller or greater
`
`
`
`
`
`
`
`
`
`than 384 kbps, for example 1 mbps downstream and 500
`
`
`
`
`
`
`
`kbps upstream, or the like. In embodiments of the present
`
`
`
`
`
`
`
`
`invention, other speeds of network connections are envi-
`
`
`
`
`
`
`
`sioned. Further,
`in practice, many network connection
`
`
`
`
`
`
`
`
`
`speeds may vary greatly with network traffic, time of day,
`and the like.
`
`
`
`In FIG. 1, network connection 190 is shown as a 10 kbps
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`connection as is currently planned for typical network
`
`
`
`
`
`
`
`
`
`appliances. The connection rate may vary, and may increase
`
`
`
`
`
`
`
`in alternative embodiments of the present invention. For
`
`
`
`
`
`example, alternative embodiments of network appliances
`
`
`
`
`
`
`
`
`may include wireless modems that may range in speeds of
`
`
`
`
`
`
`
`
`
`
`up to 128 kbps, or the like. It is believed that slower rates
`
`
`
`
`
`
`
`may be more widely utilized in typical network appliances
`
`
`
`
`
`
`
`to keep the cost of such appliances down.
`
`
`
`
`
`
`
`
`In the present embodiment, computer systems 110, 120,
`
`
`
`
`
`
`
`
`140, and 150 are embodied as typical personal computers
`
`
`
`
`
`
`
`
`such as those available from companies such as HP, Com-
`
`
`
`
`
`
`
`
`
`
`paq, IBM, and the like. Such personal computers are typi-
`
`
`
`
`
`
`
`cally powered by microprocessors such as the Athlon pro-
`
`
`
`
`
`
`
`cessor available from AMD and include operating systems
`such as Windows98 from Microsoft. In alternative embodi-
`
`
`
`
`
`
`
`
`
`
`
`
`
`ments, other personal computers such as those available
`
`
`
`
`
`
`
`
`from Apple or Dell, may also be used. Computer systems
`
`
`
`
`
`
`
`
`110 and 140 are typically desktop computers. Computer
`
`
`
`
`
`
`
`system 120 may be a desktop computer, a laptop computer,
`a television set top box, such as from WebTV Networks,
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`game consoles such as the Dreamcast, a network computer,
`
`
`
`
`
`
`
`or other types of units incorporating processors, microcon-
`trollers, ASICs, and the like.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Computing systems 110, 120, 140, and 150 are typically
`connected to computer network 160 via local area networks,
`
`
`
`
`
`
`
`
`via dial-up modems, ISDN, DSL, cable modems, satellite
`
`
`
`
`
`
`
`
`modems, or the like.
`
`
`
`
`
`
`
`
`
`
`
`In the present embodiment, computer system 130 typi-
`
`
`
`
`
`
`
`cally includes PDAs or other portable computing platforms.
`
`
`
`
`
`
`
`
`Such PDAs may operate on a variety of operating system
`
`
`
`
`
`
`
`platforms including PalmOS, WindowsCE, or the like. Fur-
`
`
`
`
`
`
`
`
`ther, such PDAs operate on a variety of processors. Such
`
`
`
`
`
`
`
`
`devices are typically coupled to computer network 160 via
`telephone lines, or other wire-based network connections.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`In the present embodiment, network appliance 180 may
`
`
`
`
`
`
`include wireless telephones including cellular technology,
`
`
`
`
`
`
`CDMA, TDMA, and other technologies. In other examples,
`
`
`
`
`
`
`
`network appliances may include kiosks, wrist watches,
`
`
`
`
`
`
`pocket or portable displays or terminals, wearable comput-
`
`
`
`
`
`
`
`ers, retinal implants, surveillance equipment, kitchen appli-
`ances, and the like.
`
`
`
`
`
`
`
`
`
`
`
`These devices are typically coupled to computer network
`
`
`
`
`
`
`
`
`
`160 via computing system 150 and wireless network 190. In
`
`
`
`
`
`
`one embodiment of the present invention, computing system
`
`
`
`
`
`
`
`150 is a wireless application protocol server (WAP) that
`
`
`
`
`
`
`
`
`
`provides data to and from network appliance 180 in the WAP
`format. In alternative embodiments, other standard and/or
`
`
`
`
`
`
`
`
`
`
`
`
`proprietary formats may also be used.
`
`
`
`
`
`
`
`In the present embodiment, computer systems 1207140
`
`
`
`
`
`
`
`
`and network appliance 180 include application software that
`communicates using the HTTP, TCP/IP, and/or RTP/RTSP
`
`
`
`
`
`
`