WORLD INTELLEcTUAL_ PROPERTY ORGANIZATION
`International Bureau
`
`
`
`INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT)
`
`(51) Imernational Patent C13S5ifi°3fi°“ 6 ‘
`
`(11) International Publication Number:
`
`WO 99/10836
`
`G06K 9/36, G06F 9/30, 5/20, G01S 15/02 A1
`
`_
`_
`_
`(43) International Publication Date:
`
`4 March 1999 (04.03.99)
`
`(21) International Application Number:
`
`PCT/IL97/00288
`
`(22) International Filing Date:
`
`27 August 1997 (27.08.97)
`
`(71) Applicant (far all designated States except US): GEO INTER-
`ACTIVE MEDIA GROUP, LTD. [IL/IL]; Corazin Street 1,
`53583 Givataim (IL).
`
`(72) Inventors; and
`CARMEL, Sharon
`(75) InventorsIApplicants (for US only):
`[IL/IL]; Kakal Street 25, 53225 Givataim (IL). DABOOSH,
`Tzur [IL/IL]; Ben—Gurion Street 236/4, 53224 Givataim
`(IL). REIFMAN, Eli
`[IL/IL]; Nordau Street 19, 75265
`Rishon le Zion (IL). SHANI, Neftali
`[IL/IL]; Hamishna
`Street 6, 62917 Tel Aviv (IL).
`
`(74) Agents: COLB, Samford, T. et al.; Sanford T. Colb & Co.,
`P.O. Box 2273, 76122 Rehovot (IL).
`
`(81) Designated States: AL, AM, AT, AT (Utility model), AU, AZ,
`BA, BB, BG, BR, BY, CA, CH, CN, CU, CZ, CZ (Utility
`model), DE, DE (Utility model), DK, DK (Utility model),
`EE, EE (Utility model), ES, F1, F1 (Utility model), GB,
`GE, GH, HU, IL, IS, JP, KE, KG, KP, KR, KZ, LC, LK,
`LR, LS, LT, LU, LV, MD, MG, MK, MN, MW, MX, NO,
`NZ, PL, PT, RO, RU, SD, SE, SG, SI, SK, SK (Utility
`model), SL, TJ, TM, TR, TT, UA, UG, US, UZ, VN, YU,
`ZW, ARIPO patent (GH, KE, LS, MW, SD, SZ, UG, ZW),
`Eurasian patent (AM, AZ, BY, KG, KZ, MD, RU, TJ, TM),
`European patent (AT, BE, CH, DE, DK, ES, FI, FR, GB,
`GR, IE, IT, LU, MC, NL, PT, SE), OAPI patent (BF, BJ,
`CF, CG, CI, CM, GA, GN, ML, MR, NE, SN, TD, TG).
`
`Published
`With international search report.
`
`(54) Title: REAL~TIME MULTIMEDIA TRANSMISSION
`
`(57) Abstract
`
`A method for conveying a multimedia sequence from a source computer (24) to a destination computer (26), including, providing an
`ordered series of data frames (120, 122, 124, 126) at the source computer (24), each data frame corresponding to an image (40, 42, 44)
`in the sequence and including data representing one or more media associated with the image, producing a table (104) including pointers
`(114), each pointer associated with one frame in the series, transmitting the table (104) to the destination computer (26), and responsive
`to selection of one of the pointers (114) in the table (104), transmitting at least a sub—series of the series of data frames, said sub—series
`beginning with the frame with which the selected one of the pointers is associated.
`
`Page 1 of 69
`
`HTC EXHIBIT 1025
`
`HTC EXHIBIT 1025
`
`Page 1 of 69
`
`

`

`FOR THE PURPOSES OF INFORMATION ONLY
`
`Codes used to identify States party to the PCT on the front pages of pamphlets publishing international applications under the PCT.
`
`Albania
`Armenia
`Austria
`Australia
`Azerbaijan
`Bosnia and Herzegovina
`Barbados
`Belgium
`Burkina Faso
`Bulgaria
`Benin
`Brazil
`Belarus
`Canada
`Central African Republic
`Congo
`Switzerland
`Cote d’lvoire
`Cameroon
`China
`Cuba
`Czech Republic
`Germany
`Denmark
`Estonia
`
`ES
`FI
`FR
`GA
`GB
`GE
`GH
`GN
`GR
`HU
`IE
`IL
`IS
`IT
`JP
`KE
`KG
`KP
`
`KR
`KZ
`LC
`LI
`LK
`LR
`
`Spain
`Finland
`France
`Gabon
`United Kingdom
`Georgia
`Ghana
`Guinea
`Greece
`Hungary
`Ireland
`Israel
`Iceland
`Italy
`Japan
`Kenya
`Kyrgyzstan
`Democratic People's
`Republic of Korea
`Republic of Korea
`Kazakstan
`Saint Lucia
`Liechtenstein
`Sri Lanka
`Liberia
`
`LS
`LT
`LU
`LV
`MC
`MD
`MG
`MK
`
`ML
`MN
`MR
`MW
`MX
`NE
`NL
`NO
`NZ
`PL
`PT
`RO
`RU
`SD
`SE
`SG
`
`Lesotho
`Lithuania
`Luxembourg
`Latvia
`Monaco
`Republic of Moldova
`Madagascar
`The former Yugoslav
`Republic of Macedonia
`Mali
`Mongolia
`Mauritania
`Malawi
`Mexico
`Niger
`Netherlands
`Norway
`New Zealand
`Poland
`Portugal
`Romania
`Russian Federation
`Sudan
`Sweden
`Singapore
`
`SI
`SK
`SN
`SZ
`TD
`TG
`TJ
`TM
`TR
`TT
`UA
`UG
`US
`UZ
`VN
`YU
`ZW
`
`Slovenia
`Slovakia
`Senegal
`Swaziland
`Chad
`Togo
`Tajikistan
`Turkmenistan
`Turkey
`Trinidad and Tobago
`Ukraine
`Uganda
`United States of America
`Uzbekistan
`Viet Nam
`Yugoslavia
`Zimbabwe
`
`Page 2 of 69
`
`Page 2 of 69
`
`

`

`W0 99/10836
`
`PCT/IL97/00288
`
`REAL-TIME MULTIMEDIA TRANSMISSION
`
`RELATED APPLICATIONS
`
`This application is 21 Continuation—in-Part of U.S.
`
`Patent Application O8/594,890, which is assigned to the
`
`5
`
`assignee of
`
`the present patent application and whose
`
`disclosure is incorporated herein by reference.
`
`FIELD OF THE INVENTION
`
`The present
`
`invention relates generally to data
`
`transmission,
`
`and. particularly to fast
`
`transmission of
`
`l0
`
`large multimedia files.
`
`BACKGROUND OF THE INVENTION
`
`The advent of powerful computers and large storage
`
`disks,
`
`such as CD-ROMS,
`
`in recent years, has brought with
`
`it
`
`a
`
`surge
`
`in production and
`
`use
`
`of
`
`sophisticated
`
`l5 multimedia
`
`applications,
`
`including
`
`full
`
`length video
`
`movies,
`
`interactive sound and animation.
`
`In recent years with the growth in Internet use,
`
`multimedia applications have been transferred to Internet
`
`web sites. However, most Internet users are connected to
`
`20
`
`the web through public telephone lines, which have a slow
`
`average rate of data transfer,
`
`commonly referred to as
`
`bandwidth,
`
`typically about 1-3 Kbyte/secondd Therefore,
`
`when these users connect
`
`to web sites that
`
`include large
`
`multimedia applications,
`
`they must wait
`
`long periods
`
`25
`
`until they can view the contents of the site. The result
`
`is that sites must often limit
`
`their non—text contents
`
`to, at most, a few still images.
`
`In order to reduce the amount of
`
`time required for
`
`transmission of multimedia applications,
`
`attempts
`
`are
`
`30 made
`
`to reduce the amount of data transmitted. by the
`
`are used to
`compression methods
`applications. Various
`minimize the amount of data which is to be transferred.
`
`Such compression methods include the well—known JPEG and
`
`MPEG standards, which are described,
`
`for
`
`example,
`
`in
`
`35
`
`ISO/IEC WGll N050l,
`
`July
`1
`
`1993,
`
`published
`
`by
`
`the
`
`Page 3 of 69
`
`Page 3 of 69
`
`

`

`W0 99/10836
`
`PCT/IL97/00288
`
`International Standards Organization,
`
`and in "JPEG Still
`
`Image Data Compression Standard", William B. Pennebaker
`
`and Joan L. Mitchell, Van Nostrand Reinhold, New York,
`
`which are incorporated herein by reference. However, even
`
`5 with
`
`these
`
`compression methods,
`
`current
`
`nmltimedia
`
`applications require a data transfer rate of about 50-150
`
`Kbytes/second to provide true real—time multimedia. Also,
`
`more
`
`sophisticated
`
`compression. methods
`
`may
`
`require
`
`complex
`
`decompression
`
`software
`
`packages
`
`and
`
`special-
`
`lO
`
`purpose CPUS, which are not available to most users.
`
`Page 4 of 69
`
`Page 4 of 69
`
`

`

`W0 99/10335
`
`.
`
`PCT/IL97/00288
`
`SUMMARY OF THE INVENTION
`
`It
`
`is an object of
`
`some aspects of
`
`the present
`
`invention to provide improved methods of multimedia data
`
`organization,
`
`to allow fast data transfer of nmltimedia
`
`5
`
`applications.
`
`It is another object of some aspects of the present
`
`invention
`
`to
`
`provide
`
`apparatus
`
`and methods
`
`for
`
`development
`
`of multimedia
`
`applications
`
`for
`
`Internet
`
`users.
`
`10
`
`It
`
`is yet another object of
`
`some aspects of
`
`the
`
`present
`
`invention
`
`to
`
`provide
`
`improved methods
`
`for
`
`compressing images and data for transmission and storage.
`
`It
`
`is
`
`a
`
`further object of
`
`some
`
`aspects of
`
`the
`
`present
`
`invention to provide apparatus
`
`and methods
`
`for
`
`15
`
`developing multimedia applications suitable for
`
`transfer
`
`at a predetermined bandwidth.
`
`It is still a further object of some aspects of the
`
`present
`
`invention to provide methods
`
`for estimating the
`
`size and transfer rate of a multimedia application.
`
`20
`
`In some aspects of the present
`
`invention, multimedia
`
`information is organized in a file for transmission,
`
`such
`
`that a recipient may randomly access parts of
`
`the file,
`
`and
`
`there
`
`is
`
`no
`
`need
`
`to
`
`transmit
`
`the whole
`
`file
`
`sequentially.
`
`25
`
`In
`
`some
`
`preferred
`
`embodiments
`
`of
`
`the
`
`present
`
`invention,
`
`a
`
`transmitting
`
`computer
`
`organizes
`
`the
`
`multimedia
`
`information
`
`in
`
`frames,
`
`each
`
`frame
`
`corresponding to a portion of
`
`the
`
`information to be
`
`presented at a given point
`
`in time. A file containing the
`
`30
`
`information includes
`
`a
`
`sequential
`
`listing of
`
`the data
`
`forming each frame
`
`and a direct
`
`access
`
`table, which
`
`contains an ID for each frame and one or more pointers to
`
`the
`
`data
`
`in
`
`the
`
`frame.
`
`The
`
`frames
`
`are preferably
`
`organized in the file sequentially according to the order
`
`35
`
`in which the frames are likely to be displayed,
`
`and the
`
`table is preferably placed at the beginning of the file.
`
`3
`
`Page 5 of 69
`
`Page 5 of 69
`
`

`

`‘W0 99/ 10836
`
`.
`
`PCT/IL97/00288
`
`When
`
`the recipient accesses
`
`the file,
`
`the access
`
`table is transmitted initially,
`
`and the recipient may
`
`then
`
`request
`
`transmission
`
`of
`
`any desired
`
`frame
`
`or
`
`sequence of
`
`frames
`
`in the file. Preferably,
`
`for
`
`this
`
`5
`
`purpose, as soon as the table and an initial frame in the
`
`file
`
`have
`
`been
`
`received,
`
`the
`
`recipient's
`
`computer
`
`displays an image of
`
`the initial
`
`frame including one or
`
`more “hot spots,” as are known.
`
`in the art, which are
`
`linked to invoke corresponding frame IDs or pointers in
`
`10
`
`the
`
`table. Each
`
`frame
`
`is preferably displayed as
`
`it
`
`arrives at
`
`the recipient's computer, without waiting for
`
`the entire file. Thus, after the initial
`
`frame has been
`
`received and displayed,
`
`the recipient will generally be
`
`able to View a continuous
`
`stream of nmltimedia images,
`
`15 without having to wait for the entire multimedia file to
`
`download to his computer as in applications known in the
`
`art.
`
`received frame
`
`is
`
`stored in a
`
`Preferably,
`
`each
`
`memory of
`
`the recipient's computer
`
`for
`
`further access.
`
`20
`
`Thus,
`
`if
`
`a
`
`frame or parts of
`
`a
`
`frame
`
`appear or are
`
`invoked by a user twice or more in an application,
`
`there
`
`is no need to transmit
`
`the frame or parts a second time.
`
`Rather,
`
`the frame is accessed in the memory according to
`
`the access table. Preferably,
`
`the frames are stored as
`
`25
`
`they
`
`are
`
`received
`
`in
`
`a
`
`compressed
`
`format
`
`and
`
`are
`
`decompressed
`
`when
`
`required.
`
`Alternatively
`
`or
`
`additionally,
`
`the frames are stored in a non—compressed
`
`format, which is ready for immediate display.
`
`Preferably,
`
`the size of
`
`the table is dynamically
`
`30
`
`adjusted according to the size of the file, and the data
`
`size of the pointers in the table is chosen according to
`
`the size of the file. Preferably,
`
`in a file smaller than
`
`256 bytes,
`
`the pointers are one byte long,
`
`in a
`
`larger
`
`file which is smaller than 64K the pointers are two bytes
`
`35
`
`long, etc.
`
`Page 6 of 69
`
`Page 6 of 69
`
`

`

`‘WO 99/10836
`
`PCT/IL97/00288
`
`Preferably,
`
`based
`
`on
`
`the
`
`table
`
`the
`
`recipient
`
`computer does not need to continuously check for a file
`
`delimiter, but
`
`rather counts
`
`the frames
`
`received. until
`
`their number matches the number of
`
`frames in the table.
`
`5 Alternatively,
`
`the number of
`
`frames and/or the length of
`
`the file is specifically stated in a header of the file.
`
`In addition,
`
`according to the difference between the
`
`pointers of two consecutive frames,
`
`the size of the first
`
`frame is known, and there is no need to have a delimiter
`
`10
`
`after the information of each frame.
`
`In some aspects of
`
`the present
`
`invention,
`
`the data
`
`of
`
`the frames, and/or other multimedia information, are
`
`stored in a compact
`
`form so as to reduce the size of the
`
`file.
`
`In preferred embodiments of the present
`
`invention,
`
`15
`
`the size of multimedia files may be reduced in over 90%,
`
`and on
`
`the
`
`average at
`
`a
`
`ratio of
`
`about
`
`1:80.
`
`Such
`
`compression
`
`rates
`
`allow fast
`
`transmission.
`
`of
`
`the
`
`information over relatively slow communication channels.
`
`In a preferred embodiment of
`
`the present
`
`invention,
`
`20
`
`each frame comprises
`
`a number of separate layers which
`
`when. displayed. or otherwise actuated together
`
`form.
`
`the
`
`frame. These layers include one or more media which carry
`
`visual or
`
`audio
`
`information in any
`
`suitable
`
`format,
`
`preferably in standard formats known in the art. Each of
`
`25
`
`the media comprises any number of primary objects,
`
`such
`
`as vectors or bitmaps, which form the media. Preferably,
`
`the table includes pointers to each of
`
`the media in the
`
`file. Identical media which appear in more than one frame
`
`are kept
`
`in the file only in the first
`
`frame in which
`
`30
`
`they appear.
`
`In the rest of the frames they are referred
`
`to by an ID code and coordinates. When reading the file,
`
`the recipient computer will refer to the table to find
`
`the addresses of the media. Alternatively,
`
`the media may
`
`be referenced directly by a pointer to their position in
`
`35
`
`the file.
`
`Page 7 of 69
`
`Page 7 of 69
`
`

`

`‘W0 99/10836
`
`PCT/IL97/00288
`
`In preferred embodiments of
`
`the present
`
`invention,
`
`coordinates used in identifying the locations of visual
`
`elements,
`
`layers, visual media and visual objects,
`
`are
`
`dynamically sized according to the range of pixels over
`
`5 which the element may move. An element having positions
`
`up to 256 pixels preferably uses one byte coordinates,
`
`while elements having up to 64K possible positions use
`
`two byte coordinates.
`
`In elements having a
`
`‘very large
`
`range,
`
`three byte
`
`coordinates may
`
`also be
`
`employed.
`
`10 Alternatively or additionally, other sizes such as half
`
`bytes and 12-bit coordinates may be used for specifying
`
`coordinates
`
`in
`
`elements
`
`of
`
`appropriate
`
`size.
`
`Such
`
`adaptively—sized coordinates are useful
`
`in reducing the
`
`volume of data that must be transmitted and in increasing
`
`15
`
`the
`
`speed with which
`
`the media
`
`are
`
`processed
`
`and
`
`displayed by the recipient computer.
`
`Preferably,
`
`the
`
`coordinates
`
`of
`
`each
`
`element
`
`are
`
`stated with respect
`
`to the region in which the element
`
`may be positioned rather than to the view of
`
`the entire
`
`frame. For example,
`
`the coordinates of objects in one of
`
`20
`
`the media may be
`
`specified relative to the top left
`
`corner
`
`(or any other suitable point) of a region in the
`
`frame
`
`occupied
`
`by
`
`the medium,
`
`and
`
`the medium is
`
`positioned within the frame according to the coordinates
`
`25
`
`of
`
`the top left corner of
`
`the region that
`
`it occupies
`
`relative to the frame.
`
`It
`
`is noted that most media are
`
`relatively small,
`
`and therefore the savings of storage
`
`space
`
`in using dynamic-sized coordinates
`
`is
`
`typically
`
`close to 50%. Preferably,
`
`in choosing coordinate systems
`
`30
`
`for media which do not occupy the entire frame,
`
`the
`
`coordinate system is chosen such that
`
`the data volume of
`
`the
`
`coordinates
`
`is
`
`small.
`
`For
`
`example,
`
`a
`
`coordinate
`
`system having up to 256 pixels is preferably chosen such
`
`that the coordinates occupy only one byte.
`
`35
`
`In preferred embodiments of
`
`the present
`
`invention,
`
`each
`
`object
`
`and/or medium has
`
`a
`
`dynamically
`
`sized
`
`Page 8 of 69
`
`Page 8 of 69
`
`

`

`‘W0 99/ 10836
`
`PCT/IL97/00288
`
`property list.
`
`In prior art methods,
`
`each object has
`
`a
`
`property list of
`
`fixed length, which may either contain
`
`properties which are usually not used,
`
`and therefore
`
`waste memory and/or
`
`transmission. bandwidth, or contain
`
`5
`
`very few properties and thus limits the flexibility of
`
`the user in defining properties.
`
`Preferably,
`
`the property list is divided into blocks
`
`according
`
`to
`
`the
`
`complexity
`
`and/or
`
`frequency
`
`of
`
`occurrence of
`
`the properties.
`
`A first block includes
`
`10
`
`those properties which are defined for every object. A
`
`second block includes properties which are used commonly
`
`but not always. Further blocks include properties which
`
`are used more seldom in reducing occurrence. Preferably,
`
`the properties in the more
`
`seldom. blocks are required
`
`15
`
`only if the properties in the more
`
`frequent blocks are
`
`used” At
`
`the end. of
`
`each. block there is a
`
`flag" which
`
`signals whether there are further property blocks for the
`
`object.
`
`Thus,
`
`the
`
`size
`
`of
`
`the
`
`property
`
`list
`
`is
`
`dynamically sized according to the properties in use.
`
`20
`
`Alternatively or additionally,
`
`the property list is
`
`ordered dynamically according to the occurrence of given
`
`properties
`
`in the
`
`file.
`
`Those properties which
`
`are
`
`altered or differ most often from one object
`
`to another
`
`are positioned at
`
`the top of
`
`the list, while those used
`
`25
`
`seldom. are positioned at
`
`the bottom. of
`
`the list. The
`
`property list of each object
`
`is truncated of properties
`
`which are not defined, and therefore their values are set
`
`according to a predetermined default. The property list
`
`is preferably terminated by
`
`a
`
`termination code,
`
`30
`
`alternatively,
`
`the number of entries
`
`in the
`
`list
`
`or
`
`is
`
`stated before the list. Alternatively,
`
`each property in
`
`the list is accompanied by its definition, and only those
`
`properties having values different
`
`from the default are
`
`listed.
`
`35
`
`In
`
`some
`
`preferred
`
`embodiments
`
`of
`
`the
`
`present
`
`invention, media comprising bitmaps
`
`are
`
`compressed in
`
`7
`
`Page 9 of 69
`
`Page 9 of 69
`
`

`

`PWO 99/10836
`
`PCT/IL97/00288
`
`order
`
`to reduce
`
`the memory
`
`required. Preferably,
`
`the
`
`bitmap is first filtered using a
`
`low—pass blur filter,
`
`and a compression transform operation,
`
`such as a discrete
`
`cosine transform (DCT), preferably in accordance with the
`
`5
`
`JPEG standard.:mentioned above,
`
`is then. applied.
`
`to the
`
`filtered bitmap. When such bitmaps are to be transmitted
`
`over
`
`the Internet,
`
`it has been found that filtering the
`
`bitmap before applying the transform results in reduction
`
`of the required memory and/or bandwidth required to store
`
`10
`
`or transmit
`
`the bitmap by about 2-4 times, substantially
`
`without
`
`loss of
`
`image quality,
`
`relative to conventional
`
`JPEG transformed bitmaps. Preferably,
`
`the user may define
`
`areas which are not
`
`to be filtered and/or areas which may
`
`undergo stronger filtering.
`
`15
`
`In
`
`some
`
`preferred
`
`embodiments
`
`of
`
`the
`
`present
`
`invention,
`
`a multimedia authoring tool primarily designed
`
`for Internet applications allows a user
`
`to conveniently
`
`create multimedia applications
`
`on
`
`a
`
`source
`
`computer,
`
`particularly for
`
`transmission over
`
`the
`
`Internet.
`
`The
`
`easy creation of web
`
`20
`
`authoring tool preferably allows
`
`sites
`
`and
`
`automatic
`
`translation and
`
`compression
`
`of
`
`multimedia applications to a form suitable for real—time
`
`display
`
`by
`
`recipient
`
`sites,
`
`as
`
`described
`
`above.
`
`Preferably, at any desired time during the creation of
`
`25
`
`the application,
`
`the user of
`
`the authoring tool may
`
`receive an estimate of the transmission rate required for
`
`smooth real—time appearance of a multimedia application,
`
`based on the data content and predicted compression ratio
`
`of the frames in the application. Thus,
`
`there is no need
`
`30
`
`to
`
`perform a
`
`time-consuming
`
`full
`
`compression
`
`of
`
`a
`
`multimedia
`
`application
`
`in
`
`order
`
`to
`
`receive
`
`a
`
`good
`
`estimate of the required transmission rate. Also, use of
`
`the estimate allows more efficient compression of
`
`the
`
`file by knowledge of the resulting file size.
`
`35
`
`In a preferred embodiment of the present
`
`invention,
`
`a multimedia
`
`software
`
`development
`
`tool
`
`performs
`
`an
`
`Page 10 of 69
`
`Page 10 of 69
`
`

`

`- W0 99/ 10836
`
`PCT/IL97/00288
`
`estimate of the memory required for each object. Bitmaps
`
`are estimated according to estimates of
`
`the
`
`required
`
`memory based on statistical samplings of the bitmaps.
`
`In
`
`some
`
`preferred
`
`embodiments
`
`of
`
`the
`
`present
`
`5
`
`invention,
`
`the user defines a required transmission rate
`
`or bandwidth for a multimedia application,
`
`and the file
`
`representing the application is automatically compressed
`
`according to the transmission rate or bandwidth.
`
`In order
`
`to create an application that can be transmitted at a low
`
`10
`
`transmission rate or narrow bandwidth, strong compression
`
`is used, at the cost of image sharpness, while for higher
`
`transmission rates, better
`
`sharpness
`
`is
`
`achieved.
`
`In
`
`authoring and compression tools known
`
`in the art,
`
`the
`
`user
`
`must
`
`generally
`
`specify
`
`desired
`
`compression
`
`15
`
`parameters,
`
`and
`
`can
`
`determine whether
`
`the
`
`file
`
`compression accordingly achieved is sufficient
`
`to match
`
`the available bandwidth only by trial
`
`and error. The
`
`present
`
`invention
`
`enables
`
`the
`
`user
`
`to
`
`specify
`
`the
`
`available bandwidth
`
`and
`
`a
`
`frame
`
`rate, whereupon
`
`the
`
`20
`
`required
`
`compression
`
`parameters
`
`are
`
`automatically
`
`determined.
`
`by‘
`
`the source computer
`
`‘based. on statistical
`
`estimation of
`
`the compression ratio and pre—calculation
`
`of the file size.
`
`Preferably,
`
`a recipient connecting to a site created
`
`25
`
`using
`
`the
`
`authoring
`
`tool
`
`of
`
`the present
`
`invention
`
`receives upon. entering‘
`
`the site a
`
`software applet
`
`for
`
`decompressing the data. Preferably,
`
`the transmission time
`
`of
`
`the applet
`
`is typically less than fifteen seconds,
`
`more preferably less than ten seconds.
`
`30
`
`In
`
`some
`
`preferred
`
`embodiments
`
`of
`
`the
`
`present
`
`invention,
`
`an initial
`
`frame,
`
`including images or nenus
`
`for
`
`invoking
`
`selected parts
`
`of
`
`the
`
`application,
`
`is
`
`transmitted to the recipient upon entering the site. The
`
`recipient may then request a specific part of
`
`the file,
`
`35 which is
`
`immediately transmitted upon
`
`request.
`
`If
`
`no
`
`request
`
`is made,
`
`the file is transmitted sequentially as
`
`9
`
`Page 11 of 69
`
`Page 11 of 69
`
`

`

`WO 99/10836
`
`PCT/IL97/00288
`
`described above, until
`
`such a request
`
`is made, or until
`
`the entire file is transmitted.
`
`It
`
`is
`
`noted
`
`that
`
`the
`
`compression methods
`
`of
`
`preferred embodiments of the present
`
`invention are simple
`
`5
`
`to implement and are especially easy for decompression.
`
`The simplicity of decompression ensures that a receiving
`
`computer does not require heavy CPU use for decompression
`
`resulting
`
`in
`
`slow performance
`
`of
`
`the
`
`computer
`
`in
`
`displaying the multimedia application.
`
`10
`
`There
`
`is therefore provided.
`
`in accordance with a
`
`preferred embodiment of
`
`the present
`
`invention,
`
`a method
`
`for
`
`conveying
`
`a multimedia
`
`sequence
`
`from a
`
`source
`
`computer to a destination computer,
`
`including:
`
`providing an ordered series of data frames at
`
`the
`
`15
`
`source
`
`computer,
`
`each data
`
`frame
`
`corresponding to an
`
`image in the sequence and including data representing one
`
`or more media associated with the image;
`
`producing a
`
`table including pointers,
`
`each pointer
`
`associated with one frame in the series;
`
`20
`
`transmitting the table to the destination computer;
`
`and
`
`responsive to selection of one of
`
`the pointers in
`
`the table,
`
`transmitting at
`
`least
`
`a
`
`sub-series of
`
`the
`
`series of data frames, said sub—series beginning with the
`
`25
`
`frame with. which.
`
`the selected. one of
`
`the pointers
`
`is
`
`associated.
`
`Preferably, producing the table includes defining
`
`pointers having a data size normalized to the size of a
`
`data file containing the series of frames.
`
`30
`
`Preferably, defining the pointers includes defining
`
`pointers of a minimal size required to distinctly address
`
`every frame in the file.
`
`Preferably, defining the pointers includes defining
`
`one—byte pointers when the ordered series includes less
`
`35
`
`than 256 bytes.
`
`10
`
`Page 12 of 69
`
`Page 12 of 69
`
`

`

`‘W0 99/1 0836
`
`PCT/IL97/00288
`
`Preferably, defining the pointers includes defining
`
`two—byte pointers when the ordered series includes less
`
`than 65,536 bytes.
`
`Preferably,
`
`the method also includes transmitting at
`
`5
`
`least an initial
`
`frame in the series to the destination
`
`computer and associating one or more of
`
`the pointers in
`
`the table with the initial
`
`frame, wherein transmitting
`
`the
`
`sub—series
`
`includes
`
`transmitting
`
`the
`
`sub—series
`
`responsive
`
`to a
`
`selection made with respect
`
`to the
`
`10
`
`initial frame.
`
`Preferably, producing the table includes producing
`
`pointers which indicate respective media in one or more
`
`of the frames.
`
`Preferably,
`
`transmitting the sub-series
`
`responsive
`
`15
`
`to selection includes invoking the media indicated by a
`
`selected pointer.
`
`Preferably, providing an ordered series of
`
`frames
`
`includes
`
`creating
`
`a
`
`first
`
`frame
`
`including
`
`data
`
`representing an object in the corresponding image.
`
`20
`
`Preferably, providing the ordered series
`
`includes
`
`creating additional
`
`frames after the first
`
`frame, which
`
`include frame data indicative of
`
`the coordinates of
`
`the
`
`object.
`
`Preferably, creating the additional
`
`frames
`
`includes
`
`25
`
`creating additional
`
`frames which.
`
`include a pointer
`
`to
`
`data representing the object in the first frame.
`
`Preferably, creating the additional
`
`frames
`
`includes
`
`creating
`
`additional
`
`frames
`
`which
`
`include
`
`an
`
`ID
`
`corresponding in the table to a pointer
`
`to the data
`
`30
`
`representing the object in the first frame.
`
`Preferably,
`
`transmitting
`
`the
`
`table
`
`includes
`
`transmitting over a computer network.
`
`Preferably,
`
`transmitting
`
`the
`
`table
`
`includes
`
`transmitting over the Internet.
`
`11
`
`Page 13 of 69
`
`Page 13 of 69
`
`

`

`‘ WO 99/10836
`
`PCT/IL97/00288
`
`Preferably,
`
`the method also includes transmitting to
`
`the destination computer
`
`a
`
`JAVA applet operative
`
`to
`
`produce sensible material
`
`from the ordered series.
`
`Preferably,
`
`creating the series
`
`includes creating
`
`5
`
`data objects representing audio data.
`
`Alternatively’ or additionally,
`
`creating‘
`
`the series
`
`includes creating data objects representing visual data.
`
`Alternatively or additionally,
`
`creating the data
`
`objects includes creating at
`
`least
`
`some
`
`segments which
`
`10
`
`include vector objects.
`
`Alternatively or additionally,
`
`creating the data
`
`objects includes creating at
`
`least
`
`some
`
`segments which
`
`include compressed bitmaps.
`
`Preferably, providing the ordered series of data
`
`15
`
`frames
`
`includes creating coordinates of objects within
`
`the frame, relative to sub—elements of the frame, wherein
`
`creating coordinates of objects includes normalizing the
`
`data
`
`size
`
`of
`
`the
`
`coordinates
`
`responsive
`
`to
`
`pixel
`
`dimensions of the sub—elements.
`
`also
`
`provided
`
`in
`
`accordance with
`
`a
`
`20
`
`There
`
`is
`
`preferred embodiment of
`
`the present
`
`invention,
`
`a method
`
`for
`
`conveying
`
`a multimedia
`
`sequence
`
`from a
`
`source
`
`computer
`
`to a destination computer,
`
`including creating a
`
`series of data frames at
`
`the source computer, each frame
`
`25
`
`corresponding to an image in the sequence, and the series
`
`including coordinates of objects within sub—elements of
`
`the frames, wherein for each coordinate a data size of
`
`the
`
`coordinate
`
`is
`
`normalized
`
`responsive
`
`to
`
`pixel
`
`dimensions of the sub~element,
`
`and transmitting at
`
`least
`
`30
`
`one of
`
`the data segments
`
`to the destination computer,
`
`wherein the
`
`segments
`
`are displayed responsive
`
`to the
`
`coordinates.
`
`Preferably,
`
`normalizing
`
`the
`
`data
`
`size
`
`of
`
`the
`
`coordinates includes normalizing the data size responsive
`
`35
`
`to pixel dimensions of a layer in the frame.
`
`12
`
`Page 14 of 69
`
`Page 14 of 69
`
`

`

`’WO 99/10836
`
`PCT/IL97/00288
`
`Alternatively or additionally, normalizing the data
`
`size of
`
`the coordinates
`
`includes normalizing the data
`
`size responsive to pixel dimensions of an object.
`
`There
`
`is
`
`further provided in accordance with a
`
`5
`
`preferred embodiment of
`
`the present
`
`invention,
`
`a method
`
`for displaying an image sequence on a computer display,
`
`including:
`
`choosing
`
`a
`
`coordinate
`
`system having
`
`desired
`
`dimensions
`
`for
`
`at
`
`least
`
`some of
`
`the
`
`images
`
`in the
`
`10
`
`sequence;
`
`defining a medium to appear in the at
`
`least
`
`some of
`
`the images;
`
`setting" position coordinates of
`
`the mediunx in.
`
`the
`
`image,
`
`the
`
`coordinates
`
`having
`
`a data
`
`size
`
`that
`
`is
`
`15
`
`normalized.
`
`to the dimensions of
`
`the coordinate system;
`
`and
`
`displaying the at
`
`least
`
`some of
`
`the images on the
`
`computer display, by rendering the object at the position
`
`coordinates thereof.
`
`20
`
`Preferably, choosing the coordinate system includes
`
`choosing a coordinate system having a number of pixels
`
`such that the data size required for addressing any point
`
`in the system is substantially minimized.
`
`Preferably, choosing the coordinate system includes
`
`25
`
`choosing a coordinate system having up to 256 pixels.
`
`Preferably, choosing the coordinate system includes
`
`choosing a coordinate system smaller than at least one of
`
`the images.
`
`Preferably, displaying the at
`
`least
`
`some of
`
`the
`
`30
`
`images
`
`includes creating and transmitting‘
`
`a data file
`
`including the object and the position coordinates from a
`
`source computer
`
`to a destination computer, where the at
`
`least some of the images are displayed.
`
`There
`
`is
`
`further provided in accordance with a
`
`35
`
`preferred embodiment of
`
`the present
`
`invention,
`
`a method
`
`of
`
`transmitting image data from a source computer
`
`to a
`
`13
`
`Page 15 of 69
`
`Page 15 of 69
`
`

`

`two 99/10836
`
`PCT/IL97/00288
`
`destination computer,
`
`including low-pass
`
`filtering the
`
`image data,
`
`compressing the data according to a block-
`
`compression method,
`
`and transmitting the compressed data
`
`to the destination computer.
`
`5
`
`Alternatively,
`
`the method
`
`includes
`
`storing
`
`the
`
`compressed data in the storage medium.
`
`Preferably,
`
`compressing
`
`the
`
`data
`
`includes
`
`compressing the data using a JPEG compression algorithm.
`
`Preferably,
`
`low—pass
`
`filtering includes convolving
`
`10
`
`the image data with a blur filter kernel.
`
`There
`
`is
`
`further provided in accordance with a
`
`preferred embodiment of
`
`the present
`
`invention,
`
`a method
`
`of
`
`transmitting a multimedia
`
`sequence
`
`from a
`
`source
`
`computer to a destination,
`
`including providing an ordered
`
`15
`
`series of data frames at
`
`the source computer,
`
`the series
`
`having a predetermined frame rate,
`
`indicating a bandwidth
`
`at which the series of data frames is to be transmitted,
`
`compressing
`
`the data
`
`frames
`
`at
`
`a
`
`compression
`
`ratio
`
`determined responsive to the indicated bandwidth and the
`
`20
`
`predetermined frame rate, and transmitting the compressed
`
`frames.
`
`Preferably, providing’
`
`the series includes creating
`
`the series responsive to the indicated bandwidth.
`
`Preferably, providing the series includes importing
`
`25
`
`the series from another computer.
`
`Preferably,
`
`providing the
`
`series
`
`of data
`
`frames
`
`includes
`
`receiving a
`
`series of data
`
`frames having a
`
`predetermined data size, and wherein compressing the data
`
`frames
`
`includes
`
`compressing the data at
`
`a compression
`
`30
`
`ratio generally equal
`
`to or
`
`less than.
`
`a
`
`ratio of
`
`the
`
`indicated bandwidth to a product of
`
`the frame data size
`
`times the frame rate.
`
`Preferably,
`
`compressing the data
`
`frames
`
`includes
`
`selecting a desired compression ratio, and estimating the
`
`35
`
`bandwidth that will be required to transmit
`
`the frames
`
`14
`
`Page 16 of 69
`
`Page 16 of 69
`
`

`

`[W0 99/1 0336
`
`PCT/IL97/00288
`
`compressed
`
`at
`
`the
`
`desired
`
`compression
`
`ratio,
`
`in
`
`comparison to the indicated bandwidth.
`
`Preferably,
`
`estimating
`
`the
`
`bandwidth
`
`includes
`
`displaying a graphical
`
`comparison. of
`
`the estimated. and
`
`5
`
`indicated bandwidths over time.
`
`Preferably,
`
`transmitting
`
`the
`
`compressed
`
`frames
`
`includes
`
`transmitting at
`
`least
`
`a portion of
`
`a
`
`frame
`
`before
`
`a
`
`respective
`
`time
`
`slot
`
`thereof when
`
`it
`
`is
`
`estimated.
`
`that
`
`the
`
`required.
`
`transmission
`
`bandwidth
`
`10
`
`exceeds the indicated bandwidth during the time slot.
`
`Preferably,
`
`estimating
`
`the
`
`bandwidth
`
`includes
`
`calculating an
`
`estimate without
`
`compressing the data
`
`frames.
`
`Preferably,
`
`compressing the data
`
`frames
`
`includes
`
`15
`
`setting coefficients
`
`for
`
`compressing bitmap images
`
`to
`
`achieve the determined compression ratio.
`
`Preferably,
`
`compressing the data
`
`frames
`
`includes
`
`normalizing the data size of coordinates of objects in
`
`the frames.
`
`20
`
`Preferably,
`
`compressing the data
`
`frames
`
`includes
`
`storing in a file containing the frame sequence only one
`
`description of each of the media in the data frames.
`
`Preferably,
`
`estimating
`
`the
`
`bandwidth
`
`includes
`
`statistically sampling the media in the frames.
`
`25
`
`There
`
`is
`
`further provided in accordance with a
`
`preferred embodiment of
`
`the present
`
`invention,
`
`a method
`
`of displaying an estimate of
`
`the bandwidth required for
`
`transmitting a multimedia
`
`sequence
`
`over
`
`a
`
`network,
`
`including:
`
`30
`
`providing an ordered series of data frames having a
`
`predetermined frame rate;
`
`estimating" the bandwidth that will be
`
`required to
`
`transmit
`
`the
`
`frames when
`
`compressed
`
`according
`
`to
`
`a
`
`predetermined compression scheme; and
`
`35
`
`displaying on a computer display the estimate of the
`
`required bandwidth.
`
`15
`
`Page 17 of 69
`
`Page 17 of 69
`
`

`

`‘W0 99/10836
`
`PCT/IL97/00288
`
`Preferably,
`
`estimating
`
`the
`
`bandwidth
`
`includes
`
`displaying a graphical
`
`comparison
`
`over
`
`time
`
`of
`
`the
`
`estimated
`
`bandwidth
`
`and
`
`an
`
`indicated
`
`bandwidth
`
`characteristic of the network.
`
`5
`
`Preferably,
`
`estimating
`
`the
`
`bandwidth
`
`includes
`
`subtracting from the
`
`estimate of
`
`a
`
`first
`
`frame
`
`the
`
`difference between.
`
`the bandwidth. of
`
`a
`
`second. preceding
`
`frame and the indicated bandwidth.
`
`Preferably,
`
`estimating
`
`the
`
`bandwidth
`
`includes
`
`10
`
`calculating an
`
`estimate without
`
`compressing
`
`the data
`
`frames.
`
`Preferably,
`
`estimating
`
`the
`
`bandwidth
`
`includes
`
`statistically sampling media in the frames.
`
`Preferably, estimating the bandwidth includes adding
`
`15
`
`to the estimate of
`
`a first
`
`frame
`
`the bandwidth of
`
`a
`
`medium referenced in the first
`
`frame but not described
`
`therein.
`
`Preferably the method includes altering at least one
`
`of the data f