US007295608B2
`
`(12) United States Patent
`Reynolds et al.
`
`(10) Patent N0.:
`(45) Date of Patent:
`
`US 7,295,608 B2
`Nov. 13, 2007
`
`(54)
`
`(76)
`
`(21)
`
`(22)
`
`(65)
`
`(60)
`
`(51)
`
`(52)
`(58)
`
`(56)
`
`SYSTEM AND METHOD FOR
`COMMUNICATING MEDIA SIGNALS
`
`Inventors: Jodie Lynn Reynolds, 13389 Folsom
`Blvd. #303-303, Folsom, CA (US)
`95630; Robert Walter Ingraham, 127
`Honey Cook Cir., Folsom, CA (US)
`95630
`
`Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 920 days.
`
`Appl. No.:
`
`10/256,866
`
`Filed:
`
`Sep. 26, 2002
`
`Prior Publication Data
`
`US 2004/0045030 A1
`
`Mar. 4, 2004
`
`Related US. Application Data
`
`Provisional application No. 60/325,483, ?led on Sep.
`26, 2001.
`
`Int. Cl.
`(2006.01)
`H04N 7/12
`US. Cl. ........... .. 375/240.01; 375/240; 375/240.12
`Field of Classi?cation Search .............. .. 375/240,
`375/240.01, 240.12, 240.1, 240.08; 348/1412,
`348/412.1; 725/105
`See application ?le for complete search history.
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`5,481,297
`5,517,246
`5,539,908
`5,596,659
`5,649,030
`5,684,714
`5,822,465
`6,002,720
`6,031,939
`
`* *
`
`1/1996
`5/1996
`7/1996
`1/1997
`7/1997
`11/1997
`10/1998
`12/1999
`2/2000
`
`Cash et a1. ............ .. 348/1412
`
`Suzuki .................. .. 348/4121
`
`Chen et a1. ............... .. 395/700
`Normile et a1.
`Normile et a1.
`Yogeshwar et a1.
`Normile et a1.
`Yurt et a1.
`Gilbert et a1.
`
`6,085,236 A
`
`7/2000 Lea .......................... .. 709/220
`
`6,115,755 A
`
`9/2000 Krishan .................... .. 709/250
`
`6,157,965 A 12/2000 Mohammed et a1. ........ .. 710/8
`
`6,195,692 B1
`6,212,302 B1
`6,243,676 B1
`
`6,252,544 B1
`6,349,151 B1
`
`2/2001 Hsu ......................... .. 709/219
`4/2001 Honsinger et a1.
`6/2001 Witteman ................. .. 704/243
`
`6/2001 Ho?berg ............... .. 342/3571
`2/2002 Jones et a1.
`
`(Continued)
`FOREIGN PATENT DOCUMENTS
`
`EP
`
`0889471 B1
`
`6/2004
`
`(Continued)
`OTHER PUBLICATIONS
`
`Ostman, Charles; Sentience on Demand, as an Online Commodity;
`1997, 1998; WWW.biota.org/ostman/sentl.htm;pp. 1-11.
`
`Primary ExamineriAllen Wong
`(74) Attorney, Agent, or F irmiKory D. Christensen; Stoel
`Rives LLP
`
`(57)
`
`ABSTRACT
`
`A media streaming system for streaming media signals is
`provided. The media streaming system takes a library of
`separate and distinct CODECs that are provided as a search
`able CODEC library and used in determining speci?c char
`acteristics in the media signal to identify similar sections of
`the signal. The media streaming system uses a computer
`implemented intelligence system, such as an arti?cial intel
`ligence mechanism to learn and capture the unique charac
`teristics of a sinal as the signal is being streamed. The media
`streaming system also compresses and decompresses the
`media signal as the signals are streamed from a source media
`to a destination device.
`
`35 Claims, 15 Drawing Sheets
`
`Baseline
`Snaplho!
`505
`
`M | ‘1::
`
`Medi
`Buffer
`515
`
`HTC EXHIBIT 1023
`
`Page 1 of 45
`
`

`

`US 7,295,608 B2
`Page 2
`
`US. PATENT DOCUMENTS
`
`3/2002 Vargo er 31-
`6356545 B1
`6,356,589 B1* 3/2002 Gebler et a1. .......... .. 375/240.1
`6,356,668 B1
`3/2002 Honsinger et a1.
`6,421,726 B1
`7/2002 Kenner et a1. ............ .. 709/225
`6,587,638 B1
`7/2003 Watanabe et 31.
`6,624,761 B2
`9/2003 Fallon ....................... .. 341/51
`6,968,006 B1* 11/2005 Puri et a1. ............ .. 375/240.08
`7,130,472 B2 10/2006 IiZuka et a1.
`
`2002/0062482 A1* 5/2002 Bolle et a1. ............... .. 725/105
`2002/0097917 A1
`7/2002 Nelson et a1.
`2003/0133501 A1
`7/2003 Ueda et a1.
`
`FOREIGN PATENT DOCUMENTS
`
`W0
`WO
`
`4/1999
`9918728 Al
`02087255 A2 10/2002
`
`* cited by examiner
`
`Page 2 of 45
`
`

`

`U.S. Patent
`
`Nov. 13, 2007
`
`Sheet 1 0f 15
`
`US 7,295,608 B2
`
`Live Video
`
`F100
`
`Splitter
`
`$105
`
`1 1 O
`l
`
`Microsoft
`Med iaTM
`Encoder
`
`1 15
`l
`
`RealTM
`Networks
`Encoder
`
`120
`L
`
`Quick?mem
`Encoder
`
`\l/
`
`125
`
`.
`Med'a Player
`1
`130
`
`RealTM
`Player
`l
`1 35
`
`QTTM
`Player
`W
`140
`
`FIG. 1A
`(Prior Art)
`
`Page 3 of 45
`
`

`

`U.S. Patent
`
`Nov. 13, 2007
`
`Sheet 2 0f 15
`
`US 7,295,608 B2
`
`145~
`
`Intermediate File
`
`On machine or
`RAID
`
`150'»
`
`Real Encoder
`Media Encoder
`I
`I
`IPC (lnterprocess Communication)
`
`QT Encoder
`I
`
`<—— On single large
`machine
`
`125
`
`.
`Medla Player
`I
`130
`
`RealTM
`Player
`I
`135
`
`QTTM
`Player w.
`l
`140
`
`FIG. 1B
`(Prior Art)
`
`Page 4 of 45
`
`

`

`U.S. Patent
`
`Nov. 13, 2007
`
`Sheet 3 0f 15
`
`US 7,295,608 B2
`
`REAL VIDEO
`
`K’ CONTENT
`
`F
`Transcoder/Server MIS Media Decoder
`
`\
`Real Decoder
`
`M/S Media Encoder
`
`Real Encoder
`
`125
`
`159
`
`Request from a
`Microsoft Media Player
`for Real Video Content
`
`FIG. 10
`(Prior Art)
`
`Page 5 of 45
`
`

`

`U.S. Patent
`
`Nov. 13, 2007
`
`Sheet 4 0f 15
`
`US 7,295,608 B2
`
`W8 Media
`Encoder
`151
`
`M/S Media
`Encoder
`152
`
`M/S Media
`Encoder
`153
`
`M/S Media
`Encoder
`154
`
`Real
`Decoder
`155
`
`Real
`Decoder
`157
`
`Real
`Decoder
`156
`
`Real
`Decoder
`158
`
`i
`150
`
`F l G. 1 D
`(Prior Art)
`
`Page 6 of 45
`
`

`

`
`
`WOMISNJ9YIOYOMISN
`
`
`
`SSO|SJIM
`
`ya}UODisonbewdLIHdV¥M|ysonbeydvM|asegeuoud
`
`
`
`
`JaAles—__—_—>Aemeye__uonels
`
`dV¥M(JWLH/IWM)}49}009yUaJUOD99asuodsaydiLHesuodsey
`
`
`
`Nov. 13, 2007
`
`Sheet 5 of 15
`
`US 7
`
`2
`
`295,608 B2
`
`4)Old
`
`(Uy10d)
`
`U.S. Patent
`
`JOyOULd}U|
`
`SSO[OIIMS|IQOIN
`
`Page 7 of 45
`
`Page 7 of 45
`
`

`

`U.S. Patent
`
`Nov. 13, 2007
`
`Sheet 6 0f 15
`
`US 7,295,608 B2
`
`200
`
`Live Video
`
`F210
`
`Live Buffer
`
`215
`
`—— MS Codec Thread @ 32 Kb
`—— MS Codec Thread @ 100 Kb
`-—— MS Codec Thread @ 40 Kb
`
`'
`|
`|
`I
`
`MS player, 32 Kb - HTTP
`Network ——— MS player 100 Kb - MMS
`—— Real Player 40 Kb - RTSP
`
`l
`I
`
`|
`I
`
`~220
`
`FIG. 2
`
`Page 8 of 45
`
`

`

`U.S. Patent
`
`Nov. 13, 2007
`
`Sheet 7 0f 15
`
`US 7,295,608 B2
`
`310
`
`300
`
`Live Buffer (Cache)
`
`__
`
`'——
`
`Video 3
`Video4
`
`V)4
`V)3
`V1
`1
`1
`1
`1
`311 312 313 314
`
`........ ..
`
`l _ _ _ _ _ _ — _ _ _ _ — _ _|
`
`l
`|
`1
`
`-—
`Network ——
`'_
`
`|
`P220
`|
`
`l_ _ _ _ _ _ _ _ _ _ _ _ __ _ J
`
`FIG. 3
`
`Page 9 of 45
`
`

`

`U.S. Patent
`
`Nov. 13, 2007
`
`Sheet 8 of 15
`
`US 7,295,608 B2
`
`LNA
`
`diueuAq
`
`OSP
`
`dWol
`
`Sbp
`
`puea91A0q
`
`YIOMON
`
`Jayoweleg
`
`einseey|
`
`O9F
`
`apon
`
`GOP
`
`BAIQDOYLbP
`sonba=uD,+TTsehand
`aiqeyndexy
`
`':JeAe7]WIOMION
`
`iObyJOYISSE[D
`
`|poteJake7YIOMJEN
`
`Jou]yoysdeus
`GerGLP
`
`YOsssa00dd
`
`O€P1Ndjno
`
`suljeseg
`
`jeunon
`
`OMION
`
`ssao0ld
`
`JOsselduo0y
`
`Ayieno
`
`puepueys
`
`0@P
`
`obey]
`
`JOSSBIOId
`
`OLY
`
`OOP
`
`AOYNOS
`
`EIPE
`
`oor
`
`omumeuAg
`
`Jaheld
`
`GOV
`
`Page 10 of 45
`
`Page 10 of 45
`
`
`
`
`
`
`
`

`

`U.S. Patent
`
`Nov. 13, 2007
`
`Sheet 9 0f 15
`
`US 7,295,608 B2
`
`P3950
`
`www
`
`1.
`
`mEEmw
`
`@ENE
`
`Page 11 of 45
`
`

`

`U.S. Patent
`
`Nov. 13, 2007
`
`Sheet 10 of 15
`
`US 7,295,608 B2
`
`
`
`puenio4JaSald
`
`
`
`OLS810)$SegJou,
`
`
`
`EdwH|eoy2emMl
`
`gosOES
`
`
`
`Gurweayss|so,aut!S}
`
`ON
`
`G‘Old
`
`03009
`
`S@uEq!)
`
`Ost
`
`taeee
`
`quiseitGPGUOneISOssy
`
`ONAN
`
`SJOJOWBIEY
`
`peesMeN
`
`JOJBIAUBS
`
`sss
`
`
`
`YOMION[BINEN|
`
`JeAIany
`
`ogs
`
`ejdwes
`
`
`
`Obsyolees
`
`waped
`
`
`Ssval01So1607
`
`dWuo}}e19}}auedwoy
`
`
`0SSGes
`
`payue|dalg
`
`OLSSpeaS
`
`euyjeseg
`
`yoysdeus
`
`gos
`
`ozsShs
`
`BIDaN
`
`eyed
`
`Page 12 of 45
`
`Page 12 of 45
`
`
`
`
`
`
`
`
`
`
`

`

`U.S. Patent
`
`Nov. 13, 2007
`
`Sheet 11 0f 15
`
`US 7,295,608 B2
`
`600
`
`280 Kb/S - VHS
`400 Kb/S - DVD
`
`Local
`Content Server
`610
`
`900 Kb/S - HDTV 5
`5
`g
`
`5
`g
`g
`
`_
`
`Cable/DSL
`Modern 630
`
`Cable/DSL
`Router 640
`
`S t_T B0
`8 Op X
`
`Content Server
`620
`
`60
`
`FIG. 6
`
`Page 13 of 45
`
`

`

`U.S. Patent
`
`Nov. 13, 2007
`
`Sheet 12 0f 15
`
`US 7,295,608 B2
`
`700 [
`
`Wired or
`wire|ess
`
`Nextel
`IDEN
`Repeater
`710
`
`Nextel
`Base Station
`and IP
`Gateway
`
`Colocated
`Server/
`Transcoder
`740
`
`4
`
`4-8 KBlsx‘x
`
`“~._‘
`
`Motorola
`
`Phone
`720
`
`Jogzna
`
`Pocket
`PC
`730
`
`<———>
`.
`Serial
`Cable
`30 Kb/s
`
`FIG. 7
`
`Page 14 of 45
`
`

`

`U.S. Patent
`
`Nov. 13, 2007
`
`Sheet 13 of 15
`
`US 7,295,608 B2
`
`yus}U04y
`
`J8NIaS
`
`Wweadys
`
`008
`
`JOJ@UJ9}U|
`
`
`
`YIOMJONJOUIO
`
`ysenbeydilH
`
`—————>
`
`
`<+_——_
`
`
`/OSPlA9esuodseydiLH
`
`OIpnyyua}u0D
`segPIAPp
`
`oepiApueolpny
`
`
`dMpueJenlesysenbeydVM‘|sSala.I\Aououd
`
`udIssadWogjVayIGOW
`MONO|di/dOLAyewndoGz
`
`
`Kemaye+——_—_—_-esegO18
`
`‘Aemayeg‘sesmouq‘yamod
`yoeys——_——__*yual|D+——dM‘oye
`
`j090}01qWOHEId
`‘Aoua}e|
`
`yUug}UODPesuodseydVM028
`pessaidwoo
`——>__|uoneis
`
`8Sls
`
`SSO|OIIN
`
`YIOMION
`
`olpne/oapiA
`
`yu9}U09
`
`Page 15 of 45
`
`Page 15 of 45
`
`
`
`
`

`

`U.S. Patent
`
`Nov. 13, 2007
`
`Sheet 14 0f 15
`
`US 7,295,608 B2
`
`Message to cellular
`customer in Cell 1
`901
`
`Backhaul
`communication
`when customer
`leaves Cell 1 area
`903
`
`\_
`
`\~~.__
`
`Central Of?ce
`
`Cell 2
`
`_“~\
`~“~‘__
`
`.
`“Lu-<1"
`
`_,.-"'
`"e"
`
`incoming message to Message to cellular
`cellular customer
`customer in Cell 2
`
`1;‘
`
`‘“~~___
`
`“~~__~~~
`
`Cell 3
`
`““~~_
`
`Depiction of backhaul communications resulting from handoff of cellular
`communication when recipient transits into a second cellular area
`
`FIG. 9
`
`Page 16 of 45
`
`

`

`U.S. Patent
`
`Nov. 13, 2007
`
`Sheet 15 0f 15
`
`US 7,295,608 B2
`
`1001
`
`1000
`A/
`
`1003
`
`g
`g
`
`g
`g
`
`Cable/DSL
`Modem
`
`Set-Top Box
`
`
`
`("W1 l Browsing via the Set-Top Box
`
`1001
`
`1005
`Proxy Game /
`Sewer
`
`1003
`
`Set-Top Box — CKIAbCLZLDrEL
`
`Gina's/5T?‘- —— Set-Top Box
`
`1004b
`
`Interactive Gaming
`
`FIG. 10
`
`Page 17 of 45
`
`

`

`US 7,295,608 B2
`
`1
`SYSTEM AND METHOD FOR
`COMMUNICATING MEDIA SIGNALS
`
`RELATED APPLICATION
`
`This application is a non-provisional application of “Rey
`nolds et al. a provisional application entitled “System and
`Method for Communicating Media Signals”, application
`number 60/325,483, ?led on Sep. 26, 2001.
`
`FIELD OF THE INVENTION
`
`This invention is a system and method for communicating
`media signals betWeen source and destination devices. More
`speci?cally, it is a system and method for compressing and
`decompressing streaming and static media signals for effi
`ciently communicating those signals betWeen source and
`destination devices using arti?cial intelligence mechanisms.
`
`BACKGROUND OF THE INVENTION
`
`The ability to ef?ciently communicate streaming and
`static media betWeen remotely located devices is a signi?
`cant need that has emerged exponentially With the advent of
`netWorked communications such as the Internet. This need
`has been recently addressed With substantial development
`resources on a WorldWide scale.
`The term “media” is herein intended to mean information
`that may be communicated in the form of a signal from a
`source device to a destination device for use by the desti
`nation device; and, Where used herein, media is generally
`contemplated to comprise either streaming or static media
`signals. For the purpose of this disclosure, the term “use” as
`applied to a destination device’s operation on media signals
`is intended to include playing (e.g. sounds, images, video),
`processing (e.g. telemetry data), or any other use or opera
`tion that is the intended purpose of the media signal.
`The terms “streaming media” are herein intended to mean
`media signals that comprise information intended to be
`communicated to and used by a destination device in a
`temporal, streaming fashion. The term “streaming” as
`applied to streaming media signals is herein intended to
`include signals communicated and processed in a continuous
`manner over time, or signals that may be communicated in
`a series of discrete packets, pieces, or blocks that are
`interrelated and may be thereafter used by the destination
`device in a continuous, interrelated fashion. Examples of
`streaming media signals for the purpose of this disclosure
`therefore include, Without limitation, the folloWing types of
`media: video, audio, audio combined With video, and data
`strings such as temporal telemetry. The terms “streaming
`media” are most typically used by reference to digitiZed
`forms of data representing the subject media.
`The terms “static media” are herein intended to generally
`mean media that is not “streaming” as de?ned above. Static
`media signals are of the type that generally may be com
`municated and are intended to be used as a single packet,
`block, or piece. Static media therefore may include for
`example, Without limitation the folloWing: a discrete image,
`an individual and relatively temporally short video clip, a
`sound or sound bite, or a piece or block of information such
`as telemetry information. It is contemplated, hoWever, that
`such a “single piece” of static media may be of suf?cient
`magnitude to consist of a plurality of smaller pieces or
`sub-parts, such as for example regions or pixels of an overall
`image, individual frames that together form a video clip,
`digital bits that together comprise a sound, a group of sounds
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`that comprise a sound bite, or bits of information that
`together comprise a larger block of information.
`Streaming media generally includes data ?les that are
`signi?cantly larger than static media ?les, and also often
`represent many more variables over the temporal commu
`nication of such ?les than experienced With most static
`media ?les. Therefore, the ability to ef?ciently compress
`streaming media for appropriate communication to destina
`tion devices for use is often a much more complex and
`dif?cult to achieve goal. Accordingly, much of this disclo
`sure is provided by reference speci?cally to streaming media
`communication, and the present invention has been
`observed to provide signi?cant bene?ts for such communi
`cation. HoWever, Where streaming media is speci?cally
`referenced herein With respect to this background, and
`further With respect to the many bene?ts of the present
`invention herein disclosed, static media is also further con
`templated Where appropriate according to one of ordinary
`skill.
`Many different “type-speci?c” media systems have been
`in use for quite a long time for transmitting speci?c types
`(eg video, audio, image, voice, etc.) of streaming and static
`media signals betWeen sources and remote destinations.
`Typical examples of such type-speci?c media systems
`include television transmission systems, telephone line sys
`tems, and radio transmission systems, and every television,
`telephone, and radio is therefore a receiving device for
`media. Accordingly, the needs for ef?cient communication
`of streaming and static media touch upon many diverse
`communications industries, including for example the tele
`phone, television, movie, music, and more recently interac
`tive gaming industries.
`Moreover, many medial communications systems, includ
`ing the various long-standing type-speci?c systems, are also
`“format speci?c”, Wherein the subject media signals are
`communicated in a particular format such that the source,
`transmission channel, and destination device must be spe
`ci?cally compliant to Work Within that format. Examples of
`format speci?c media systems include for example encoded
`cable television systems that Work only for certain types of
`media and only delivered in particular encoded formats from
`the cable carrier. Therefore, these systems, in hardWare and
`softWare, are generally dedicated to only the type and format
`of media to be provided by the content provider.
`Society’s needs have outpaced the abilities of these dedi
`cated, content-speci?c and format-speci?c systems. In par
`ticular, these dedicated systems are not structured to accom
`modate the ever increasing client demand, real-time, for
`speci?ed streaming media. Still further, technology devel
`opments in the recently interconnected World has tempted
`the palate of society for the ability to pull, receive, push, and
`send multiple types of media in multiple formats using one
`device. Moreover, content providers need to be able to
`deliver many different media signals to many different types
`of devices in their clients’ of?ces, living rooms, and hands.
`Individuals and corporations also desire to communicate
`With each other using various different formats and using
`various different respective devices.
`Accordingly, a signi?cant industry has emerged for deliv
`ering streaming and static media over the centraliZed net
`Work of the Internet. Content delivery companies are cur
`rently delivering a Wide range of streaming media, from live
`horse racing and entertainment to medical telemetry and
`education, over the Internet, and in video and audio formats.
`According to one published report from DFC Intelligence,
`video streaming on the Internet greW 215 percent in 2000 to
`over 900 million total streams accessed. This includes
`
`Page 18 of 45
`
`

`

`US 7,295,608 B2
`
`3
`broadband streams, Which made up almost 29 percent of
`total accesses. This same report also estimates that as much
`as 15 percent of available stream inventory is noW being
`exploited With in-stream advertising. In another report pub
`lished by Internet researcher Jupiter Media Metrix, business
`spending alone on streaming video technology Will balloon
`from one-hundred forty million (US$140M) US dollars in
`2000 to nearly three billion (U S$3B) US dollars by 2005 as
`companies turn to electronic interaction in communicating
`With employees, consumers and other businesses.
`Still further, the population explosion and increasing
`number of people transmitting on these systems has severely
`impacted the available bandWidth for available information.
`Therefore, the ability to stream media efficiently, using
`limited bandWidth resources and limited available transmis
`sion speeds, is of increased societal importance.
`Compression/Decompression Algorithms (“CODECS”)
`In vieW of the exponential demand for communicating the
`different types of media, various compression/decompres
`sion systems (“CODEC(s)”) have been developed over
`many years, and have in particular become the recent topic
`of signi?cant research and development. Speci?c types of
`CODECS and systems for managing the operation of
`CODECS With respect to communicating streaming and
`static media signals have been developed for speci?c types
`of media, including for example still-frame images such as
`graphics and photographs, and streaming media.
`Image CODECS
`Various different types of static media CODECS have
`been developed, and a Wide variety of these CODECS are
`Widely knoWn and used. One speci?c type of static media
`that has been the topic of particular attention includes
`images (though a long series of interrelated image frames
`such as in video context is generally treated as streaming
`media due to more complex variables, eg siZe and temporal
`relationship betWeen frames, that signi?cantly impact appro
`priate compression/decompression needs). Examples of
`static media CODECing is therefore herein exempli?ed by
`reference to certain speci?c types of conventional image
`CODEC technologies and methods.
`The tWo most common ?le formats for graphic images on
`the World Wide Web are knoWn as “GIF” and “JPEG”
`formats, generally considered the respective standards for
`draWings (e.g. line art) and photographs, and are further
`described together With other image compression modalities
`for the purpose of further understanding as folloWs.
`“JPEG” is an acronym for “Joint Photographic Experts
`Group”, and is a graphic image ?le that complies With ISO
`standard 10918. Commonly used for photograph compres
`sion/ decompression, a J PEG ?le is created by choosing from
`a range of compression qualities, or, as has also been
`described, by choosing from one of a suite of compression
`algorithms. In order to create a JPEG ?le, or convert an
`image from another format to JPEG, the quality of image
`that is desired must be speci?ed. In general, because the
`highest quality results in the largest ?le, a trade-off may then
`be made, as chosen by the user, betWeen image quality and
`image siZe. The JPEG mode of compression generally
`includes 29 distinct coding processes although a JPEG
`implementer may not use them all. A JPEG image is
`typically given a name suf?x “.jpg”.
`“GIF” is an acronym for “Graphics Interchange Format”,
`and is generally considered the de facto standard form of
`draWing image compression/decompression for Internet
`communication. GIF formatting uses a compression algo
`rithm knoWn as the LZW algorithm, Which Was developed
`by Abraham Lempel, Jacob Ziv, and Terry Welch and made
`commercially available by Unisys Corporation (though in
`general such algorithm has been made publicly available
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`60
`
`65
`
`4
`Without requiring fee-bearing licenses). More speci?cally, a
`“LZW” compression algorithm takes each input sequence of
`bits of a given length (eg 12 bits) and creates an entry in a
`table, sometimes called a “dictionary” or “codebook”, for
`that particular bit pattern. The entry consists of the pattern
`itself and a shorter code. As input is read, any pattern that has
`been read before results in the substitution of the shorter
`code, effectively compressing the total amount of input to
`something smaller. Earlier approaches, knoWn as LZ77 and
`LZ78, did not include the look-up table as part of the
`compressed ?le. HoWever, the more recent LZW algorithm
`modality does include the table in the ?le, and the decoding
`program that decompresses the ?le for vieWing is able to
`build the table itself using the algorithm as it processes the
`encoded input. The GIF format uses the 2D raster data type
`(associated With display screens using raster lines) and is
`encoded in binary.
`TWo versions of GIF formats include GIF 87a, and more
`recently GIF89a that alloWs for “animated GIF” ?le cre
`ation, or short sequences of images Within a single GIF ?le
`that are played in sequence to present movement or change
`in the image (either in an endless loop or through a pro
`gression that reaches an end). GIF89A also alloWs for, and
`also for “interlaced GIF”, Which is a GIF image that arrives
`and is displayed by the receiver ?rst as a fuZZy outline of an
`image that is gradually replaced by seven successive Waves
`of bit streams that ?ll in the missing lines until full resolution
`is reached. Interlaced GIF alloWs, for example, a vieWer
`using 14.4 Kbps and 28.8 Kbps modems to observe a briefer
`Wait-time before certain information in a subject image may
`be processed, such as for example to make decisions (eg to
`click on the image to execute an operation such as a link).
`By presenting Waves of resolution ?lling image
`sequences, interlaced GIF is similar to “Progressive JPEG”,
`Which describes an image created using the JPEG suite of
`compression algorithms that Will “fade in” in successive
`Waves. While the progressive JPEG is often observed to be
`more appealing Way to deliver an image at modern connec
`tion speeds, users With faster connections may not likely
`notice a difference.
`“PNG” or “Portable NetWork Graphics” format has been
`more recently developed for image compression and that, in
`time, has been publiciZed to replace the GIF format for
`Internet use (though not generally the JPEG format alloWing
`siZe/quality trade-offs). This format has been developed for
`public consumption and development. Similar to GIF, PNG
`is considered a “lossless” compression format, and therefore
`all image information is restored When a compressed ?le is
`decompressed during vieWing. HoWever, PNG formatted
`?les are generally intended to be from 10 to 30 percent more
`compressed than With a GIF format. Further aspects of PNG
`?le formats are provided as folloWs: (i) color transparency
`may not be limited to only one color, but the degree of
`transparency may be controlled (“opacity”); (ii) “interlac
`ing” of images is improved versus standard GIF; (iii)
`“gamma correction” is enabled, alloWing for “tuning” of
`images in terms of color brightness required by speci?c
`display manufacturers; (iv) images can be saved using true
`color, palette, and gray-scale formats similar to GIF; and (v)
`“animation” is generally not supported, though PNG is
`generally considered extensible and therefore softWare may
`be layered to provide for such scriptable image animation.
`“TIFF” is an acronym for “Tag Image File Format”, and
`is a common format for exchanging raster graphics (or
`“bitmap”) images betWeen application programs, such as for
`example graphics used for scanner images. A TIFF ?le is
`usually given a name suf?x of “.tif” or “.ti?‘”, and had
`generally been developed in the mid-1980’ s With the support
`of Adobe SoftWare, Microsoft, and HeWlett-Packard. TIFF
`?les can be in any of several classes, including gray scale,
`
`Page 19 of 45
`
`

`

`US 7,295,608 B2
`
`5
`color palette, or RGB full color, the descriptions and differ
`ences of Which are further developed elsewhere herein this
`disclosure. TIFF ?les may also include ?les With JPEG,
`LZW, or CCITT Group 4 standard run-length image com
`pression, Which are also further described elseWhere herein.
`As one of the most common graphic image formats, TIFF
`?les are typically used in desktop publishing, faxing, 3-D
`applications, and medical imaging applications.
`V1deo CODECS
`Vldeo compression has been the topic of intense devel
`opment for various applications, including, for example:
`pre-recorded video (e.g. “video-on-demand”), teleconfer
`encing, and live video (e.g. broadcasts). “Desk-top” com
`puters, Wireless devices, conventional televisions, and high
`de?nition televisions are examples of the different types of
`receiving devices that an e?icient video compression system
`must serve.
`In general, video CODEC algorithms operate on either or
`both of an individual, frame-by-frame basis, and/or on a
`“temporal compression” basis Wherein each frame is the
`most common video compression algorithms in conven
`tional use are based on several mathematic principles,
`including the following: Discrete Cosine Transforms
`(“DCT”), Wavelet Transforms and Pure Fractals.
`“Discrete Cosine Transforms” or “DCT’s” are by far the
`most popular transforms used for image compression appli
`cations. In general, DCT is a technique for representing
`Waveform data as a Weighted sum of cosines. The DCT is
`similar to the discrete Fourier transform: it transforms a
`signal or image from the spatial domain to the frequency
`domain. The DCT helps separate the image into parts (or
`spectral sub-bands) of differing importance (With respect to
`the image’s visual quality). Reasons for its popularity
`include not only good performance in terms of energy
`compaction for typical images but also the availability of
`several fast algorithms. DCTs are used in tWo international
`image/video compression standards, JPEG and MPEG.
`“Wavelet transforms” are generally mathematical algo
`rithms that convert signal data into a set of mathematical
`expressions that can then be decoded by a destination
`receiver device, such as for example in a manner similar to
`Fourier transform. Wavelets have been observed to enhance
`recovery of Weak signals from noise, and therefore images
`processed in this manner can be enhanced Without signi?
`cant blurring or muddling of details. For this reason, Wavelet
`signal processing has been particularly applied to X-ray and
`magnetic-resonance images in medical applications. In
`Internet communications, Wavelets have been used to com
`press images to a greater extent than is generally possible
`With other conventional methods. In some cases, the Wave
`let-compressed image can be as small as about 25 percent
`the siZe of a similar quality image using the more familiar
`JPEG format, Which is discussed in further detail elseWhere
`in this disclosure. Thus, for example, a photograph that
`requires 200 Kb and takes a minute to doWnload in JPEG
`format may require only 50 Kb and take only 15 seconds to
`doWnload in Wavelet-compressed format. A Wavelet-com
`pressed image ?le is often given a name suf?x “.Wif”, and
`either the receiver (e.g. Internet broWser on a computer
`receiver) must support these format speci?c ?les, or a
`plug-in program Will be required to read such ?le.
`Fractal image compression is a modern technique of lossy
`image coding that provides several improvements over
`existing Fourier series compression schemes. Edge depic
`tion is improved since, When modeled as a step function,
`edges require a large number of Fourier series terms to
`properly depict. Other advantages of fractals include fast
`decoding time and scale independence. Fractal compression
`is based on Mandelbrot sets Which take advantage of a self
`similar, scaling dependent, statistical feature of nature (Man
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`6
`delbrot, 1983). Fractal compression and decompression
`involves a clustering approach to ?nd regions Which shoW
`the same characteristics as a sample region independent of
`rotation and scale. The fractal image compresses images as
`recursive equations and instructions about hoW to reproduce
`them. The equations describe the image in terms of the
`relationships betWeen its components. The reduction in
`storage need is due to the fact that fractal compression saves
`equations and instructions instead of a pixel representation
`of the image.
`“MPEG” is an acronym for Moving Picture Experts
`Group and has come to be used synonymously With certain
`evolving video and audio compression standards promul
`gated therefrom. In general, to use MPEG video ?les, a
`personal computer is required With suf?cient processor
`speed, internal memory, and hard disk space to handle and
`play the typically large MPEG ?le, usually given the name
`suf?x “.mpg”. A speci?ed MPEG vieWer or client softWare
`that plays MPEG ?les must be available on the client
`system, and generally can be doWnloaded shareWare or
`versions of commercial MPEG players from various sites on
`the Web. The modes of operation for MPEG formatted
`media are herein described by reference to these sequentially
`evolved standards as folloWs.
`More speci?cally, MPEG-1 standard Was designed for
`coding progressive video generally at a transmission rate of
`about 1.5 Mbps. This Was generally designed for the speci?c
`application for Video-CD and CD-I media. MPEG-1 audio
`layer-3 (“MP3”) has also evolved from early MPEG Work.
`“MPEG-2” is a standard generally designed for coding
`interlaced images at transmission rates above 4 Mbps, and
`Was generally intended for use With digital TV broadcast and
`digital versatile disk. Though it is generally observed that
`many MPEG-2 players can handle MPEG-1 data as Well, the
`opposite is not generally observed to be true and MPEG-2
`encoded video is generally incompatible With MPEG-1
`players. Yet another progressive standard, “MPEG-3”, has
`also been proposed for use With high de?nition television
`(“HDTV”), though in general MPEG-3 has merged With
`MPEG-2 Which is generally believed to meet the HDTV
`requirements. Finally, an “MPEG4” standard has also been
`most recently developed and is intended to provide a much
`more ambitious standard to address speech and video syn
`thesis, fractal geometry, and computer visualization, and has
`further been disclosed to incorporate arti?cial intelligence in
`order to reconstruct images.
`MPEG-1 and -2 standards de?ne techniques for com
`pressing digital video by factors varying from 25:1 to 50:1.
`This compression is achieved according to these standards
`generally using ?ve different compression techniques: (i)
`discrete cosine transform (DCT), Which is a frequency
`based transform; (ii) “quantization”, Which is a technique for
`losing selective information, e. g. lossy compression, that can
`be acceptably lost from visual information; (iii) “Huffman”
`coding, Which is a technique of lossless compression that
`uses code tables based on statistics about the encoded data;
`(iv) “motion compensated predictive coding”, Wherein dif
`ferences in What has changed betWeen an image and its
`preceding image are calculated and only the differences are
`encoded; and (v) “bi-directional prediction”, Wherein some
`images are predicted from the pictures immediately preced
`ing and folloWing the image.
`Further more detailed examples of commercially avail
`able video compression technologies include: Microsoft
`Media PlayerTM (available from Microsoft Corporation),
`RealPlayerTM or RealSystem G2TM (commercially available
`from Real NetWorksTM), Apple’s QuickTimeTM (commer
`cially available from SorensonTM); and “VDO”. The
`Microsoft Media PlayerTM is generally believed to apply the
`MPEG standard of CODEC for compression/decompres
`
`Page 20 of 45
`
`

`

`US 7,295,608 B2
`
`7
`sion, whereas the others have been alleged to use proprietary
`types of CODECS. Standard compression algorithms, such
`as MPEG4, have made their way into the hands of devel-
`opers who are building embedded systems for enterprise
`streaming, security, and thelike.
`One example of a more recenteffort to provide streaming
`video solutions over Wireless and I