USOO71434.32B1
`
`(12) United States Patent
`Brooks et al.
`
`(10) Patent No.:
`(45) Date of Patent:
`
`US 7,143,432 B1
`Nov. 28, 2006
`
`(54) SYSTEM FOR TRANSFORMING
`STREAMING VIDEO DATA
`
`(75) Inventors: Roger K. Brooks, Palo Alto, CA (US);
`Stephen A. Molloy, Los Gatos, CA
`(US); Chi-Te Feng, Hsi Chih (TW);
`Qing Zhang, Sunnyvale, CA (US);
`Yanda Ma, Milpitas, CA (US); Dave
`Singhal, San Jose, CA (US)
`
`(73) Assignee: Vidiator Enterprises Inc., Nassau (BS)
`(*) Notice:
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 0 days.
`(21) Appl. No.: 09/502.409
`
`(22) Filed:
`
`Feb. 10, 2000
`O
`O
`Related U.S. Application Data
`(60) Provisional application No. 60/157,468, filed on Oct.
`1, 1999.
`
`(51) Int. Cl.
`(2006.01)
`HO)4N 7/173
`(52) U.S. Cl. ...................... 725/105: 725/109; 725/114;
`725/115; 725/117; 725/118; 709/231; 709/235;
`709/203; 709/247; 375/240.24; 375/240.25;
`375/240.26; 370/437
`(58) Field of Classification Search ........... 375/240.02,
`375/240.24, 240.11, 240.01, 240.25, 240.26;
`370/420,398,437; 725/118, 105,114, 109,
`725/115, 117; 709/247, 246, 231, 235, 203
`See application file for complete search history.
`References Cited
`U.S. PATENT DOCUMENTS
`
`(56)
`
`5,619.995 A * 4, 1997 Lobodzinski ............... 348,398
`5,671.319 A
`9/1997 Ueda ........................... 386.96
`5,768,535 A
`6, 1998 Chaddha et al.
`395/2OO.77
`5,903,775 A
`5/1999 Murray ....................... 395/853
`5,914,751 A
`6/1999 Korth ......................... 348,409
`
`
`
`5,970,233 A * 10/1999 Liu et al. .................... TO9,246
`6,011,868 A
`1/2000 van den Branden
`et al. .......................... 382,233
`2/2000 Bhatt et al. ................. 348/441
`6,028,639 A
`6,091,777 A * 7/2000 Guetz et al. ........... 375,240.11
`6,141,447 A * 10/2000 Linzer et al. ............... 382,236
`6,141,693 A 10/2000 Perlman et al. ............. TO9,236
`
`(Continued)
`Primary Examiner Shawn S. An
`(74) Attorney, Agent, or Firm—Seed IP Law Group PLLC
`
`(57)
`
`ABSTRACT
`
`According to one embodiment, a circuit configured to form
`an output video stream includes a resolution modification
`Norigid to receive a plurality of video frames from
`a frame buffer, and configured to modify resolution of the
`plurality of video frames, when the desired resolution for the
`output video stream is different than a resolution of the input
`video stream, the plurality of frames of data derived from an
`input video stream, a frame reducing circuit coupled to the
`resolution reducing circuit configured to reduce a number of
`video frames in the plurality of video frames from the
`resolution reducing circuit, when a desired frame rate for the
`output video stream is different than a frame rate of the input
`video stream, a depth reduction circuit coupled to the frame
`reducing circuit configured to reduce bit depth of the plu
`rality of video frames from the frame reducing circuit, when
`a desired bit depth for the output video stream is different
`than a bit depth of the input video stream, and a rate
`reduction circuit coupled to the depth reduction circuit,
`configured to scale the plurality of video frames from the
`depth reduction circuit, in response to a desired bit rate for
`the output video stream, and an encoder coupled to the rate
`reduction circuit, configured to encode the plurality of video
`frames from the rate reduction circuit into the output video
`stream is also contemplated.
`
`17 Claims, 8 Drawing Sheets
`
`-
`
`-so
`
`-st
`
`-so
`-
`
`Cropper
`
`DCT
`Subsampler
`
`Frome Rote
`Reducer
`
`Color Depth
`Reducer
`
`550
`
`Bitrate
`Control
`
`- -se
`Encoder -
`Encryptor
`
`MPEG-1,2,4
`
`IPR2018-01413
`Sony EX1005 Page 1
`
`

`

`US 7,143.432 B1
`Page 2
`
`U.S. PATENT DOCUMENTS
`
`6,141,709. A 10 2000 Cutter...................... T10/100
`6, 160,544. A 12/2000 Hayashi et al.
`... 345,327
`6,236,395 B1
`5/2001 Sezan et al. ................ 345,328
`6,275,536 B1
`8/2001 Chen et al. ............ 375/240.25
`6,292,834 B1
`9/2001 Ravi et al. .................. TO9,233
`6.298,385 B1
`10/2001 Sparks et al. .....
`... 709,233
`6,343,313 B1
`1/2002 Salesky et al. ............. TO9.204
`
`6/2002 Gruse et al. ................ 280/228
`6,398,245 B1
`12/2002 Kalra et al. ..
`TO9,231
`6,490,627 B1
`1/2003 Zeng et al. ..
`T13,160
`6,505,299 B1
`2/2003 Lau et al. ....
`6,525,746 B1
`345,725
`6,567,986 B1* 5/2003 Ward et al. ................. 725/118
`6,795,863 B1* 9/2004 Doty, Jr
`TO9,231
`- -
`v. « » « » « » « » « » « » « » « » « » « » «
`
`
`
`* cited by examiner
`
`IPR2018-01413
`Sony EX1005 Page 2
`
`

`

`U.S. Patent
`
`Nov. 28, 2006
`
`Sheet 1 of 8
`
`US 7,143,432 B1
`
`SAMPLE NETWORK TOPOLOGY
`
`
`
`
`
`Gateway is optional; used
`for archiving and/or
`further video processing.
`
`
`
`160
`
`56 Kbit
`160x120/
`5fps/color/
`MPEG4
`120
`
`Gateway
`
`- - - - - - - - - - - - - - - - - - - - - - -
`
`210
`
`240
`
`f50
`
`WAP Server --
`Gateway P
`
`10Kbit/wireless
`b/w 64x48/
`bitmap 5fps
`Cellphone/
`microbrowser/
`WAP
`
`f80
`
`
`
`m
`St. 14.4Kbps
`5fps/160x120/
`color/MPEG4
`
`f30
`
`
`
`&S
`PDA, WinCE
`
`220
`
`230
`
`f90
`
`f40
`
`384.Kbit/DSL
`6:0/10fps/color
`D
`
`SS
`
`IPR2018-01413
`Sony EX1005 Page 3
`
`

`

`U.S. Patent
`
`Nov. 28, 2006
`
`Sheet 2 of 8
`
`US 7,143,432 B1
`
`310
`
`MONITOR
`
`300
`/
`
`360
`
`370
`
`380
`
`320
`
`PROCESSOR
`
`
`
`
`
`
`
`RANDOM
`ACCESS
`MEMORY
`
`DISK DRIVE
`
`NETWORK
`INTERFACE
`
`PROCESSING
`UNIT
`
`KEYBOARD
`
`350
`
`340
`
`330
`
`Roer
`mRR
`
`O
`
`DE
`EE
`
`IPR2018-01413
`Sony EX1005 Page 4
`
`

`

`U.S. Patent
`U.S. Patent
`
`Nov. 28, 2006
`
`Sheet 3 of 8
`
`US 7,143,432 B1
`US 7,143,432 B1
`
`RV
`
`SQ.
`
`3‘
`
`:55men:
`
`8925mm:
`
`53.5
`
`+05253
`
`82>mEEogm
`
`V.280Eomtm
`
`V
`
`388:9;
`
`3.28.282
`
`
`
`
`
`
`
`
`
`3:28P3
`
`m6%
`§ 5)IH
`
`|PR2018—O1413
`
`Sony EX1005 Page 5
`
`IPR2018-01413
`Sony EX1005 Page 5
`
`
`
`
`
`
`

`

`U.S. Patent
`U.S. Patent
`
`N
`
`6m
`
`4
`
`US 7,143,432 B1
`M7,SU
`
`1B
`
`.9550MLofitocmim.78¢:£25
`
`3965
`
`
`
`
`
`ww§§fin
`
`3%\\
`
`
`8,526%.833133.53%m:38L28sumwas;SoV.385
`
`
`
`
`
`
`
`
`
`
`nIu,wEm
`
`|PR2018—O1413
`
`Sony EX1005 Page 6
`
`IPR2018-01413
`Sony EX1005 Page 6
`
`
`

`

`U.S. Patent
`
`Nov. 28, 2006
`
`Sheet S of 8
`
`US 7,143,432 B1
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`REQUEST WIDEO DATA FROM
`A WIDEO SOURCE
`
`600
`
`DETERMINE BANDWIDTH
`LIMITATIONS AND FORMAT
`REQUIREMENTS
`
`
`
`610
`
`
`
`
`
`
`
`
`
`VIDEO SOURCE
`CAN PROVIDE
`VIDEO?
`
`NO
`
`REDIRECT REQUEST TO A
`GATEWAY COMPUTER
`
`
`
`DETERMINE BANDWIDTH
`LIMITATIONS AND FORMAT
`REQUIREMENTS
`
`GATEWAY CAN
`PROVIDE WIDEO?
`
`COMMUNICATE WITH WDEO
`SOURCE TO OBTAIN INPUT
`WIDEO DATA
`
`660
`
`FIG. 5A
`
`IPR2018-01413
`Sony EX1005 Page 7
`
`

`

`U.S. Patent
`
`Nov. 28, 2006
`
`Sheet 6 of 8
`
`US 7,143,432 B1
`
`BUFFER INPUT WIDEO DATA
`IN FRAME BUFFER
`
`670
`
`
`
`
`
`
`
`
`
`TRANSCODE INPUT WIDEO
`DATA TO THE DESRED
`OUTPUT WIDEO
`
`FORMAT INTO TCP/IP
`PACKETS
`
`SEND TCP/IP PACKETS TO
`THE REQUESTER
`
`DECODE TCP/IP PACKETS
`TO RECOVER WIDEO DATA
`STREAM
`
`
`
`DECOMPRESS/PLAY WIDEO
`DATA STREAM
`
`680
`
`690
`
`700
`
`710
`
`720
`
`FIG. 5B
`
`IPR2018-01413
`Sony EX1005 Page 8
`
`

`

`U.S. Patent
`
`Nov. 28, 2006
`
`Sheet 7 of 8
`
`US 7,143,432 B1
`
`INPUT WIDEO DATA STORED
`IN FRAME BUFFER
`
`CROP INPUT WIDEO DATA
`
`800
`
`NXOUT
`
`INPUT COLOR
`DEPTH COMPARED
`TO OUTPUT COLOR
`DEPTH
`
`OUT>IN
`
`820
`
`DOWNSCALE COLOR DEPTH
`OF INPUT WIDEO DATA
`
`UPSCALE COLOR DEPTH OF
`INPUT WIDEO DATA
`
`830
`
`
`
`340
`
`
`
`NXOUT
`
`
`
`INPUT RESOLUTION
`COMPARED TO OUTPUT
`RESOLUTION
`
`OUTXN
`
`850
`
`DOWNSCALE RESOLUTION
`OF INPUT WIDEO DATA
`
`UPSCALE RESOLUTION OF
`INPUT VIDEO DATA
`
`860
`
`
`
`
`
`
`
`870
`
`INXOUT
`
`INPUT FRAM
`RATE COMPARED
`TO OUTPUT FRAME
`RATE
`
`OUTXIN
`
`880
`
`REDUCE FRAME RATE OF
`INPUT WIDEO DATA
`
`INCREASE FRAME RATE OF
`INPUT WIDEO DATA
`
`890
`
`FIG. 6A
`
`IPR2018-01413
`Sony EX1005 Page 9
`
`

`

`U.S. Patent
`
`Nov. 28, 2006
`
`Sheet 8 of 8
`
`US 7,143,432 B1
`
`NXOUT
`
`INPUT BIT RAT
`COMPARED TO
`OUTPUT BIT RATE
`
`900
`
`OUTXN
`
`DECREASE BIT RATE OF
`INPUT WIDEO DATA
`
`
`
`
`
`DECREASE BIT RATE OF
`INPUT WIDEO DATA
`
`920
`
`ENCODNG
`FORMAT2
`
`QUICKTIME
`
`910
`
`
`
`
`
`
`
`
`
`
`
`
`
`940
`
`ENCODE INPUT DATA
`STREAM INTO MPG
`FORMA
`
`
`
`ENCODE INPUT DATA
`STREAM INTO *. MOW
`FORMAT
`
`ENCODE INPUT DATA
`STREAM INTO AV
`FORMAT
`
`
`
`ENCRYPT DATA STREAM
`
`FIC. 6B
`
`IPR2018-01413
`Sony EX1005 Page 10
`
`

`

`1.
`SYSTEM FORTRANSFORMING
`STREAMING VIDEO DATA
`
`CROSS-REFERENCES TO RELATED
`APPLICATIONS
`
`The present invention disclosure claims priority to Pro
`visional U.S. patent application Ser. No. 60/157,468, filed
`Oct. 1, 1999, entitled Internet Camera Video Producer. This
`application is herein by incorporated by reference for all
`purposes.
`
`BACKGROUND OF THE INVENTION
`
`The present invention relates to devices for providing
`media across a computer network. In particular, the present
`invention relates to systems configured to adapt input
`streams of video data to meet desired parameters for output
`streams of video data.
`The legend of the Tower of Babel tells us that humans
`once spoke a common language. However, divine interven
`tion soon foiled human’s plans in the building of the tower
`to the sky. This was done by making humans speak in
`different languages. As a result, the legend goes, humans did
`not understand each other, humans misunderstood others
`intentions, and ultimately humans moved away from each
`other. Ever since that day, humans have been benefited and
`plagued by language and cultural differences.
`Translating between different languages has also been an
`issue considered in popular fiction. In Douglas Adam’s
`well-known Hitchhiker's Guide to the Galaxy (1979), the
`problem of different languages was solved by the introduc
`tion of a “Babel fish.” Adams described the Babel fish as a
`“small, yellow and leechlike, and probably the oddest thing
`in the Universe. It feeds on the brainwave energy received
`not from its own carrier but from those around it. It absorbs
`all unconscious mental frequencies from this brainwave
`energy to nourish itself with. It then excretes into the mind
`of its carrier a telepathic matrix formed by combining the
`conscious thought frequencies with nerve signals picked up
`from the speech centers of the brain which has supplied
`them. The practical upshot of all this is that if you stick a
`Babel fish in your ear you can instantly understand anything
`said to you in any form of language.” While this, of course,
`may not really exist, a commercial translation service has
`actually been named after the Babel fish.
`On the Internet, AltaVista.com provides a translation
`service that translates text in a web page from one selected
`language to another selected language. The tool is called the
`“Babel fish.” To use this tool, the user enters a web address
`and then specifies the language direction, e.g. English to
`French. In response, the Babel fish will translate the text that
`appears on the page into the new language, all other aspects
`of the page will remain the same. The quality of the
`translations may vary with actual usage. For example,
`translating the phrase "To jump the gun' to French and back
`into English returns the phrase "To connect the gun. As can
`be seen, the quality of Such services are not quite ideal.
`Further, such services do not address non-text data Such as
`audio and visual (media) data.
`On the web, other types of data than text are also
`displayed to users. Such data include media Such as images,
`sounds, video, and the like. With such data, instead of being
`in different languages, the data are stored and transmitted
`into different formats. Most of the different media formats
`are incompatible. Currently, there are a multitude of stan
`dards or formats for each of them, for example, images may
`
`2
`be transmitted in formats including *.jpg, *.gif, *.bmp,
`*.pcX, and the like; sounds may be transmitted in formats
`including *.wav, *.mp3, *.aiff, and the like; and video may
`be transmitted in formats including *.avi, *.mov, *.rm,
`*.mpg, *.asf, vivo and the like.
`To view or hear data in any of the above media formats
`requires an appropriate viewing (translation) application
`program. That is, an application program is required to
`decode the transmitted data and output the data to the
`computer of the requester. The requesting computer must
`have preinstalled many different viewer applications. This is
`so that the computer can receive, decode, and display data
`stored in the many different media formats.
`One drawback to requiring multiple viewers is that this
`Solution is not appropriate for all devices connected to the
`web. In particular, it is not appropriate for future wireless
`devices, or the like. Such devices may include cellular
`telephones, wireless PDAs, network appliances (refrigera
`tors, toasters, ovens, Smart houses), wrist watches, wearable
`computers, and the like. Because many of these devices will
`have lower amounts of memory and performance compared
`to desktop computers, these devices will not be able to
`display a large number of viewers. As a result, these devices
`will not be able to play many different media formats.
`One solution proposed to address this problem has been to
`standardize upon one particular format. For example, all
`wireless devices in the future would be able to receive and
`output data stored in one particular media format, Such as
`MPEG-4 (Motion Pictures Expert Group), or the like.
`A drawback to this solution is that in theory, this solution
`is reasonable, however, in practice, it is not likely to happen.
`The primary reason is that there are many competing media
`formats available, and most are backed by separate compa
`nies. For example, Windows pushes the *avi media format,
`Apple pushes the *.mov media format, Real Networks
`pushes the *.rm format, Vivo Software pushes its own
`format, and the like. It is doubtful that such companies will
`agree on a single media format.
`Another drawback is that even if there is one standardized
`media format, different requesting devices will have differ
`ent performance, resolutions, bandwidth, and the like. For
`example, a PDA may have the ability to display 80x60 pixel
`24-bit color images, however a wrist watch may have the
`ability to display only 40x30 pixel 8-bit gray scale images,
`and the like. Because many future wireless network devices
`or appliances will have different performance, bandwidth,
`and the like, the source of the media will have to store the
`data in just as many data files. For example, if there are 100
`different telephones with web access, each telephone with its
`own performance, bandwidth factors, and the like, to Sup
`port all these telephones the media source will have to store
`the data in 100 different data files. This is highly inefficient
`and/or impractical to implement, not to mention impossible
`for live data.
`Typically, media Sources, such as web sites, provide only
`a limited number of media formats and performance char
`acteristics. For example, a typical site may only offer a
`media clip in the Quicktime format at 320x240 resolution or
`160x120 resolution. If a requesting device cannot process
`Quicktime format, the user is out of luck. As another
`example, if the requesting device can only play 80x60
`resolution Quicktime movies, the media Source is wasting its
`output bandwidth sending the requesting device 160x120
`resolution Quicktime movies.
`
`US 7,143,432 B1
`
`10
`
`15
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`IPR2018-01413
`Sony EX1005 Page 11
`
`

`

`US 7,143,432 B1
`
`3
`Thus what is needed in the industry are improved systems
`for providing requesting devices with media in the format
`and performance appropriate and/or requested for these
`requesting devices.
`
`SUMMARY OF THE INVENTION
`
`4
`portion coupled to the receiving portion, configured to
`receive a bandwidth requirement for the output video
`stream, and configured to reduce bandwidth used by the
`frames of data in response to the bandwidth requirement to
`form bandwidth reduced frames of data, and an encoding
`portion coupled to the processing portion, configured to
`receive an encoding format for the output video stream, and
`configured to encode bandwidth reduced frames of data to
`form the output video stream.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`A more complete appreciation of the invention and many
`of the attendant advantages thereof will be readily obtained
`as the same becomes better understood by reference to the
`following detailed description when considered in connec
`tion with the accompanying drawings, wherein:
`FIG. 1 illustrates a block diagram of a usage scenario
`according to an embodiment of the present invention;
`FIG. 2 is a block diagram of a typical gateway computer
`and its interface to client computers according to an embodi
`ment of the present invention;
`FIG. 3 illustrates a block diagram of an embodiment of
`the present invention;
`FIG. 4 illustrates a block diagram of a transcoding com
`pound according to an embodiment of the present invention;
`FIGS. 5A and 5B illustrates an overview flow diagram
`according to an embodiment of the present invention; and
`FIGS. 6A and 6B illustrates a more detailed embodiment
`of the present invention.
`
`DESCRIPTION OF THE SPECIFIC
`EMBODIMENTS
`
`FIG. 1 illustrates a block diagram according to an embodi
`ment of the present invention. FIG. 1 illustrates a gateway
`computer 100, and plurality of computer systems 110–150
`coupled to each other via a computer network 160. In the
`present embodiment, a video camera 170 is coupled to
`computer system 110, and a network appliance 180 is
`coupled via a wireless network 190 to computer system 150.
`Computer systems 110–150 are coupled to computer net
`work 160 via network connections 200–240.
`In the present embodiment, computer network 160 is the
`Internet. In alternative embodiments of the present inven
`tion, computer network 160 may be any computer network,
`Such as an intranet, a computer network, a local area
`network, an internet, and the like. Computer network 160
`provides data communication among computer systems
`110–150 and gateway computer 100. Data communication
`may include transfer of HTML based data, textual data, form
`Submissions, plug-in programs or viewers, applets, audio
`data, video data, and the like. Although computer network
`160 is illustrated as a single entity, as is the case with the
`Internet, it should be understood that computer network 160
`may actually be a network of individual computers and
`SWCS.
`In the present embodiment, network connections 190–240
`have typical maximum bandwidth characteristics that are
`known a priori. In FIG. 1, for example, network connection
`200 is shown as a 1.5 mega bit per second (Mbps) TI
`correction, as is common with businesses; network connec
`tion 210 is shown as a 56 kilobit per second (kbps) connec
`tion as is common with home computers, set top boxes, and
`the like; network connection 220 is shown as a 14.4 kbit
`(kbps) connection to personal digital assistants (PDAs). Such
`as PalmOS devices, WindowsCE devices, and the like. In
`
`The present invention relates to systems for transcoding
`and transforming video streams. In particular, the present
`invention relates to apparatus for adapting input streams of
`Video data to meet desired parameters for output streams of
`Video data. On the fly adaptation to desired output param
`eters may be made with respect to display size, frame rate,
`bit-depth, bit rate, encoding format, and the like.
`According to one embodiment, a circuit configured to
`form an output video stream includes a resolution modifi
`cation circuit configured to receive a plurality of video
`frames from a frame buffer, and configured to modify
`resolution of the plurality of video frames, when the desired
`resolution for the output video stream is different than a
`resolution of the input video stream, the plurality of frames
`of data derived from an input video stream, and a frame
`reducing circuit coupled to the resolution reducing circuit
`configured to reduce a number of video frames in the
`plurality of video frames from the resolution reducing
`circuit, when a desired frame rate for the output video stream
`is different than a frame rate of the input video stream. The
`circuit also includes a depth reduction circuit coupled to the
`frame reducing circuit configured to reduce bit depth of the
`plurality of video frames from the frame reducing circuit,
`when a desired bit depth for the output video stream is
`different than a bit depth of the input video stream, and a rate
`reduction circuit coupled to the depth reduction circuit,
`configured to scale the plurality of video frames from the
`depth reduction circuit, in response to a desired bit rate for
`the output video stream. An encoder coupled to the rate
`reduction circuit, configured to encode the plurality of video
`frames from the rate reduction circuit into the output video
`stream is also contemplated.
`According to another embodiment, a circuit for reducing
`bandwidth of an incoming video stream includes a cropping
`portion configured to receive a first video frame, a second
`video frame, and a third video frame from a frame buffer,
`and configured to determine the resolution of the first video
`frame, the second video frame, and the third video frame, the
`first video frame, the second video frame, and the third video
`frame derived from the incoming video stream, and a
`sampling portion coupled to the cropping portion, config
`ured to receive a desired output resolution, and configured to
`subsample the first video frame, the second video frame, and
`the third video frame to respectively form a first sampled
`Video frame, a second sampled video frame, and a third
`sampled video frame. The circuit may also include a frame
`rate reducing portion coupled to the sampling portion,
`configured to receive a desired output frame rate, and
`configured to drop the second sampled video frame, and a
`depth reducing portion coupled to the frame rate reducing
`portion, configured to receive a desired output bit-depth, and
`configured to reduce bit depth of the first sampled video
`frame and the third sampled video frame to respectively
`form a first reduced video frame and a third reduced video
`frame.
`According to yet another embodiment, a circuit for
`dynamically reducing bandwidth of an input video stream to
`meet bandwidth requirements for an output video stream
`includes a receiving portion configured to receive frames of
`data derived from the input video stream, a processing
`
`10
`
`15
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`IPR2018-01413
`Sony EX1005 Page 12
`
`

`

`US 7,143,432 B1
`
`10
`
`15
`
`5
`this example, network connection 230 is shown as a 384
`kbps digital Subscriber line (DSL) connection as is common
`with small businesses or power users. In alternative embodi
`ments, network connection 230 may be smaller or greater
`than 384 kbps, for example 1 mbps downstream and 500
`kbps upstream, or the like. In embodiments of the present
`invention, other speeds of network connections are envi
`Sioned. Further, in practice, many network connection
`speeds may vary greatly with network traffic, time of day,
`and the like.
`In FIG. 1, network connection 190 is shown as a 10 kbps
`connection as is currently planned for typical network
`appliances. The connection rate may vary, and may increase
`in alternative embodiments of the present invention. For
`example, alternative embodiments of network appliances
`may include wireless modems that may range in speeds of
`up to 128 kbps, or the like. It is believed that slower rates
`may be more widely utilized in typical network appliances
`to keep the cost of Such appliances down.
`In the present embodiment, computer systems 110, 120,
`140, and 150 are embodied as typical personal computers
`Such as those available from companies such as HP. Com
`paq, IBM, and the like. Such personal computers are typi
`cally powered by microprocessors such as the Athlon pro
`cessor available from AMD and include operating systems
`such as Windows98 from Microsoft. In alternative embodi
`ments, other personal computers such as those available
`from Apple or Dell, may also be used. Computer systems
`110 and 140 are typically desktop computers. Computer
`system 120 may be a desktop computer, a laptop computer,
`a television set top box, such as from WebTV Networks,
`game consoles such as the Dreamcast, a network computer,
`or other types of units incorporating processors, microcon
`trollers, ASICs, and the like.
`Computing systems 110, 120, 140, and 150 are typically
`connected to computer network 160 via local area networks,
`via dial-up modems, ISDN, DSL, cable modems, satellite
`modems, or the like.
`In the present embodiment, computer system 130 typi
`cally includes PDAs or other portable computing platforms.
`Such PDAs may operate on a variety of operating system
`platforms including PalmOS, WindowsCE, or the like. Fur
`ther, such PDAs operate on a variety of processors. Such
`devices are typically coupled to computer network 160 via
`telephone lines, or other wire-based network connections.
`45
`In the present embodiment, network appliance 180 may
`include wireless telephones including cellular technology,
`CDMA, TDMA, and other technologies. In other examples,
`network appliances may include kiosks, wrist watches,
`pocket or portable displays or terminals, wearable comput
`ers, retinal implants, Surveillance equipment, kitchen appli
`ances, and the like.
`These devices are typically coupled to computer network
`160 via computing system 150 and wireless network 190. In
`one embodiment of the present invention, computing system
`150 is a wireless application protocol server (WAP) that
`provides data to and from network appliance 180 in the WAP
`format. In alternative embodiments, other standard and/or
`proprietary formats may also be used.
`In the present embodiment, computer systems 120–140
`and network appliance 180 include application software that
`communicates using the HTTP, TCP/IP, and/or RTP/RTSP
`protocols. These communication protocols are well known,
`thus no description is given herein. The application Software
`is typically embodied as a web browser (client), in one
`embodiment. Further, the software is typically able to dis
`play *.gif, and/or *.jpg format images. For computer sys
`
`55
`
`6
`tems 120 and 140, the web browser software may be
`embodied as Netscape Navigator 4.x, Microsoft's Internet
`Explorer 5.x, or the like. In alternative embodiments of the
`present invention, other transfer and communication proto
`cols may also be used, for example IPX, or the like. Further,
`different web client software may be used in other embodi
`mentS.
`In the present embodiment, it is envisioned that video data
`will be transferred from computing system 110 to computer
`systems 120–150 and onto network appliance 180. The
`video data may also be provided by gateway computer 100.
`In one example, video data is stored on computing system
`110 in a variety of formats including MPEG1, MPEG2, and
`MPEG4, as are well known in the art. In alternative embodi
`ments other video formats are envisioned and may include
`the Windows *.avi format, the Quicktime *.mov format, or
`the like. In other embodiment, streaming video formats may
`be used to provide video data, for example formats from
`RealNetworks, Microsoft, Apple, or the like. In the present
`embodiment, the streaming video may be from a stored
`video archive, or from a live video camera 170, or the like.
`Whether archived or live, the video is typically output by
`computing system 110 onto computer network 160.
`As illustrated in the examples in FIG. 1, computing
`systems 120–140 and network appliance 180 are all coupled
`to computer network 160 with different bandwidth limited
`connections. Further, computing systems 120-140 and net
`work appliance 180 typically have different processing
`power, display capabilities, memory, operating systems, and
`the like. As a result of these differences, each system have
`different abilities to receive, process, and display video data.
`In the example in FIG. 1, the bandwidth of network
`connection 230 between computing system 140 and com
`puter network 160 is a DSL connection. As illustrated in
`FIG. 1, because the bandwidth is relatively large, network
`connection 230 is capable of providing computing system
`140 with enough video data to display up to a 640x480 pixel
`color image at 10 frames per second (fps) using an MPEG1
`format. In alternative embodiments, other configurations are
`envisioned, for example, 320x240 monochromatic image at
`30 fps, or the like.
`In the example in FIG. 1, the bandwidth of network
`connection 210 between computing system 120 and com
`puter network 160 is limited to 56K by the modem. As
`illustrated in FIG. 1, because the bandwidth is relatively
`Small, network connection 210 is capable of providing
`computing system 120 with enough video data to display up
`to a 160x120 color image at 5 fps using an MPEG4 format.
`As above, in alternative embodiments, other video configu
`rations are envisioned, for example, a 80x60 4-bit image at
`25 fps, or the like.
`Still further, in the example in FIG. 1, the bandwidth of
`network connection 220 between computing system 130 and
`computer network 160 is limited to 14.4 kbps by the modem.
`As illustrated in FIG. 1, because the bandwidth is small,
`network connection 220 is capable of providing computing
`system 130 with enough video data to display up to a
`160x120 256 color (8-bit color) image at 5fps. As above, in
`alternative embodiments, other video configurations are
`envisioned, for example, a 80x60 16 gray scale (4-bit) image
`at 10 fps, or the like.
`Also in the example in FIG. 1, the bandwidth of wireless
`network 190 between network appliance 180 and computer
`network 160 is limited to 10 kbps. As illustrated in FIG. 1,
`because the bandwidth is very small, wireless network 190
`is capable of providing network appliance 180 with enough
`Video data to display up to a 64x48 black and white image
`
`25
`
`30
`
`35
`
`40
`
`50
`
`60
`
`65
`
`IPR2018-01413
`Sony EX1005 Page 13
`
`

`

`US 7,143,432 B1
`
`7
`at 3 fps. As above, in alternative embodiments, other video
`configurations are envisioned, for example, a 32x24 black
`and white image at 10 fps, or the like.
`In the present embodiment, gateway computer 100 is
`coupled to computer network 160 and is configured to
`provide video data to computer systems 120-140 and net
`work appliance 180. In particular, in the present embodi
`ment, gateway computer 100 is configured to receive video
`data from computer system 110 and to provide video data to
`each device according to that device's bandwidth limita
`tions, and in the output format desired. In this example,
`gateway computer 100 delivers a stream of video data to
`computer system 120 that represents a 160x120 resolution
`color image at 5 fps, in the MPEG4 format; gateway
`computer 100 delivers a stream of video data to network
`appliance 180 that represents a 64x48 resolution black and
`white image at 3 fps, in a custom format; and the like.
`Further details regarding gateway computer 100 are given
`below.
`The diagram in FIG. 1 is merely an illustration which
`should not limit the scope of the claims herein. One of
`ordinary skill in the art would recognize many other varia
`tions, modifications, and alternatives.
`FIG. 2 is a block diagram of a typical gateway computer
`300 according to an embodiment of the present invention.
`Gateway computer 300 typically includes a monitor 310, a
`computer 320, a keyboard 330, a graphical input device, a
`processing unit 340, a network interface 350, and the like.
`In the present embodiment, a graphical input device is
`typically embodied as a computer mouse, a trackball, a track
`pad, wireless remote, and the like. Graphical input devices
`typically allow the users to graphically select objects, icons,
`text and the like output on monitor 310 in combination with
`a CUSO.
`Processing unit 340 is typically embodied as a high
`bandwidth PC bus, such as the PCI and the like, plug-in card
`into computer 320. In the present embodiment, processing
`unit 340 provides much of the functionality that will be
`described below. Presently, processing unit 340 is a plug-in
`board, that is not yet currently available from Luxxon
`Corporation, the assignee of the present invention. In alter
`native embodiments of the present invention, the function
`ality provided by processing unit 340 may be implemented
`on a host computer 320 as software. In such a case, little
`additional hardware is typically needed.
`Embodiments of network interface 350 include an Ether
`net card, a mo