WO 97/22201
`
`PCT/US9G/19226
`
` seTue
`
`
`< SUCCESSFUL”
`
`
`
`
`
`[VIDEO/AUDIO TRANS-
`J
`IMISSION AND PLAYBACK |
`
`
`
`
` REQUEST EVALUATION
`
`
`REQUEST CAN
`ISEND BACK DENY
`NO »MESSAGE ALONG WITH
`
`BE GRANTED 2 >
` YES
`
`
`
`
`SEND BACK GRANT
`MESSAGE
`
`
`FORK OFF A SEPARATE
`
`PROCESS TO HANDLE
`
`THE REQUEST
`
`____|
`
`
`
`
`SUBSTITUTE SHEET (RULE 26)
`
`IPR2022-01227
`IPR2022-01227
`EXHIBIT 1003 - PAGE 01301
`EXHIBIT 1003 - PAGE 01301
`
`|
`
`| i
`
`ACCEPTREQUEST
`FROM CLIENTS
`
`|
`
`r
`
`|
`
`|
`
`
`
`Le
`
`

`

`WO 97/2220)
`
`PCT/US96/19226
`
`M28
`
`
`
`
`
`
`
`AGE FROM SERVER
`
`
`
`
`
`fE
`
`ND
`
`ON
`REQUEST ~\NO
`“GRANTED? a—
`
` YYES
`
`|RESULT=SUCCESS
`
`
`
`DISPLAY ERROR MESS-
`
`{i{t
`
`SUBSTITUTE SHEET (RULE 26)
`
`IPR2022-01227
`IPR2022-01227
`EXHIBIT 1003 - PAGE 01302
`EXHIBIT 1003 - PAGE 01302
`
`

`

`WE 97/2226
`
`PCT/US96/19226
`
`10/28
`
`Size AND SETUP BUFF |
`
`SETUP UDPCONNECTION
`
`ESTIMATE RTT.
`
`CALCULATE BUFFER]
`
`
`
` DECODING AND DISPLAY
`
`VIDEO AND AUDIO DATA
`
`
`
`
`[CPU CONGESTION
`| DETECTION
`
`
`NETWORK ©CONGESTION
`
`
`DETECTION
`
`
`“DETECTED
`
`_-~CONGESTION
`yes [SEND ADAPTATION
`mI MESSAGE TO SERVER
`
`
`
`[PROCESS USER COMMAND
`
`|
`
`
`
`SUBSTITUTE SHEET (RULE 26)
`
`IPR2022-01227
`IPR2022-01227
`EXHIBIT 1003 - PAGE 01303
`EXHIBIT 1003 - PAGE 01303
`
`

`

`WO 97/22204
`
`PCT/US96/19226
`
`N/28
`
`[READ IN VIDEO AUDIO
`
`
`|PARSE INFORMATION
`
`
`
`
`ssmN,
`a ~
`_~ ADAPTATION ~~
`“MESSAGE?
`
`
`
`~
`CLIENT PLAY
`
`BACK COMMAND
`
`SUBSTITUTE SHEET (RULE 26)
`
`IPR2022-01227
`IPR2022-01227
`EXHIBIT 1003 - PAGE 01304
`EXHIBIT 1003 - PAGE 01304
`
`

`

`WO 97/22201
`
`PCT/US96/19226
`
`12/28
`
`
`
`SUBSTITUTE SHEET (RULE 26)
`
`IPR2022-01227
`IPR2022-01227
`EXHIBIT 1003 - PAGE 01305
`EXHIBIT 1003 - PAGE 01305
`
`

`

`WO 97/22201
`
`PCT/US96/1 9226
`
`13/28
`
`OES
`
`
`
`aeae a,apes
`
`Save Js.
`
`ao
`Oe
`°
`:
`“
`pennetitece anhbearamineeet ode tentiienteninrioeenorenmttceatin aide
`VosatprojectarSystemsReisartie Croup,DeptofCS,UIUC, For informationandcomments
`send to Ehigang Chen(achen@xs.iiuewhe),See~MongTan(sran@cealuciaeds) orProfessorRoy.
`Campbati(camphelliiiesaiuc.etu)..
`"
`“
`o
`va
`oe
`
`SUBSTITUTE SHEET (RULE 26)
`
`IPR2022-01227
`IPR2022-01227
`EXHIBIT 1003 - PAGE 01306
`EXHIBIT 1003 - PAGE 01306
`
`

`

`WO O7/22205
`
`POTATS96/19226
`
`14/28
`
`= D
`
`ain Gener
`emos
`
`aenntert
`joc
`
`PEMRI
`
`Vosaicproject at System Reseorth Crone, Devtof CS,ULC. Forinformmation and contmnatts
`send to Ziigung Chenlichen@csasiec.ote), See~MengTan(ston@esxluec.edie).or ProfesarRey:
`Camphedicampbhedl@csuiteaan), ET De8Bas ee ee
`
`: en -
`. a -
`:
`:
`if Homel Bemedt Spend Seu ASA Cion
`PEPSI
`POONA
`hateaoe
`
`New Window] Close Window)”
`
`“Tt
`
`on Cg
`
`SUBSTITUTE SHEET(RULE 28)
`
`IPR2022-01227
`IPR2022-01227
`EXHIBIT 1003 - PAGE 01307
`EXHIBIT 1003 - PAGE 01307
`
`

`

`WO 97/2220
`
`POPSIG9226
`
`or ee hee)
`
`iSNN
`
`oN
`ero
`se
`ote,
`See
`ENR easCE
`AGES
`BSae
`
`“SandbeZhigang Chea(shengs.eiec.eak),SeenMong TanisianGes.nine.eet)orProjector Boy
`
`|WARLORDRN BNRRAOREETREERA INERARND
`VosaieprojectatSestem: ResearchGroup,DekotCS,ULOC.Forinformationandcomments:
`
`i
`
`Ramee:
`
`ayiA cemzaaty
`Seat 9
`
`oe
`
`‘
`Nese
`
`SUBSTITUTE SHEET (RULE 26)
`
`IPR2022-01227
`IPR2022-01227
`EXHIBIT 1003 - PAGE 01308
`EXHIBIT 1003 - PAGE 01308
`
`

`

`WG 97/22205
`
`PCT/AUSI6/1 9226
`
`16/28
`
` aio sae
`
`
`
`whee cwie esene Am:
`Vosalc projectwt SystemResearch Crowe,LieplofCS,UCCBarinfornintion ancamunents
`send to Thigany Chen(ackenipcrwiseedt), See~Mang Tan{stan@esuduc.eda} ap Projessar Roy
`
`Homel Retoadf Open.| Save As.) Clone] Mew Wirew) Close W
`AS Caw
`
`
`
`
`
`
`
`
`
`Ml
`Esc
`
`xe G.15D
`
`SUBSTITUTE SHEET (RULE 26)
`
`IPR2022-01297
`IPR2022-01227
`EXHIBIT 1003 - PAGE 01309
`EXHIBIT 1003 - PAGE 01309
`
`

`

`WO O7/27201
`
`PCYT/DS96/19226
`
`7/28
`
`
`
`SUBSTITUTE SHEET (RULE 26)
`
`IPR2022-01227
`IPR2022-01227
`EXHIBIT 1003 - PAGE 01310
`EXHIBIT 1003 - PAGE 01310
`
`

`

`WO 97/2220
`
`PCT/US96/1 9226
`
`18/28
`
`
`
`
`
`Cites onthe obiectsvarithDashing outlinetnthe abe’ 6widen atreani witsTestyoutoe
`
`anotherpage. This isextensionofhyperlinklatevideostreame. The flethat specityesthe
`anchos information is resided on the server skle. We have tools for making thet tink files
`for our video server. See here for more cetadls.
`
`
` Backch{ +arent +teh Peinasi!Soper... Sweas.f ¢fone Newwindow}©C1asewanton!
`
`
`
`
`
`
`
`
`:
`
`
`
`FIG. 15F
`
`SUBSTITUTE SHEET (RULE 26)
`
`IPR2022-01227
`IPR2022-01227
`EXHIBIT 1003 - PAGE 01311
`EXHIBIT 1003 - PAGE 01311
`
`

`

`WO O7/222618
`
`POT/USIG/1 9226
`
`.i.; Clone} New Window} Close Windew
`
`SUBSTITUTE SHEET (RULE 26)
`
`IPR2022-01227
`IPR2022-01227
`EXHIBIT 1003 - PAGE 01312
`EXHIBIT 1003 - PAGE 01312
`
`19/28
`
`‘ P:¢iy
`
`

`

`WO 97/22201
`
`PCT/US96/19226
`
`(FPS)
`
` herneee
`
`
`
`FRAMERATE
`
`
`
`
`
`FRAME NUMBER x 10
`
`SUBSTITUTE SHEET (RULE 26}
`
`IPR2022-01227
`IPR2022-01227
`EXHIBIT 1003 - PAGE 01313
`EXHIBIT 1003 - PAGE 01313
`
`

`

`WO 97/2226)
`
`PCT/US96/19226
`
`21/28
`
`SEMANTIC
`DESCRIPTION
`|ANNOTATION |
`
`SEMANTIC.
`[DESCRIPTION
`ANNOTATION 2 |
`
`
`SEMANTIC
`| DESCRIPTION
`| ANNOTATION n
`
`
`
`
`
`
`INHERENT PROPERTIES
`STRUCTURAL. INFORMATION
`
`
`
`
`
`PHYSICAL REPRESENTATION
`
`SUBSTITUTE SHEET (RULE 25}
`
`IPR2022-01227
`IPR2022-01227
`EXHIBIT 1003 - PAGE 01314
`EXHIBIT 1003 - PAGE 01314
`
`

`

`WO 97/22201
`
`PCT/US96/19226
`
`22/28
`
`CLIPS
`
`
`ENGINEERINGCOLLEGE
`
`
`OVERVIEW (FRAMES 1-6355)
`
`ISHOTS
`
`/RANPUSOVERVEEWC-Ta08)~
`
`
`
`sr(9655-1000)
`
`
`COMPUTER SCIENCE
`DEPARTMENT (FRAMES 6356 -
`12003)
`
`INTERVIEW WITH A UNDERGRADUATE
`
`
`
`SUBSTITUTE SHEET (RULE 26)
`
`IPR2022-01227
`IPR2022-01227
`EXHIBIT 1003 - PAGE 01315
`EXHIBIT 1003 - PAGE 01315
`
`

`

`WO 97/22201
`
`POCT/US96/19226
`
`
`
`enRSIAUIN|SLOHS|aera
`
`
`
`
`
`“SNSONIMABNIONS‘oninMaIAW3BAOSANYO]
`
`
`
`
`
`ShidiNYO
`
`
`
`
`
`ENOL“9G‘Layd3dSOOF‘Oninoaos-
`
`OOPS
`
`
`
`
`
`
`
`
`‘MNQL‘Luvd3dSO‘Onin,bS96~1008
`av]WWNOLLOYAYLSNI
`
`
`
`ASIAMSLN'Luvd3dSo‘OnnOOO-SS386SAVAGVYONSONNYHLIMMSIAdSLNI
`
`ANSGOALS
`
`HNOLBYANO
`
`SUBSTITUTE SHEET (RULE 26)
`
`IPR2022-01227
`IPR2022-01227
`EXHIBIT 1003 - PAGE 01316
`EXHIBIT 1003 - PAGE 01316
`
`
`
`

`

`WO 97/2223}
`
`PCT/US96/19226
`
`24/28
`
`LNVd3dSOMRIAYSAO(D3
`
`uvS[omo23
`peapeereAnanNRRTRTE
`
`/
`
`SIAOW
`
`MOGNIM
`
`SUBSTITUTE SHEET (RULE 26)
`
`IPR2022-01227
`IPR2022-01227
`EXHIBIT 1003 - PAGE 01317
`EXHIBIT 1003 - PAGE 01317
`
`
`
`

`

`WO 07/22201
`
`PCT/US8G/19226
`
`25/28
`
`
`
`.
`ma,
` Media Beameer
`nannnnnaAnannnes
`
`erro
`@ Voesgio> LortEe
`Navigate Annatate
`
`fie Options
`
`
`penen
`
`
`nmRSEERAAImI2iAN ‘
`
` ‘
`
`Socument Tithe:
`
`Gecument URL:
`
`eames weeneenen:
`
`:
`
`4
`
`?
`i
`” Forinformation andcomments
`sendio Drigang CChen(zchenas wine.-acu), beeMong 7Tanjstan@es.arae eds) oy Professor
`:
`7 Rey CaripbelténmeinBes auc ede}
`
`Cinge Window:
`
`
`SUBSTITUTE SHEET (RULE 26)
`
`IPR2022-01227
`IPR2022-01227
`EXHIBIT 1003 - PAGE 01318
`EXHIBIT 1003 - PAGE 01318
`
`

`

`WO 87/22201
`
`POCT/AUS9G/19226
`
`26/28
`
` ‘s: Vogaie: Cort LMuCUs Media Browser
`
`Fite Options
`fMavigete Annotate
`Bocunsent Tithe:
`
`
`
`Embeded Links within Video Streams -
`
`
`
`
`j
`
`
`
` Becurment URL:
`
`FeEeINEERASNEI
`Srenrenennmnriniinmninittriiatenniiorwettestin
`
`=
`
`
`
`en
`Rew]BAYPLAYYi Fe oTapy Quy
`:
`ann
`~
`x
`oo
`
`
`» Click on the objects oyith festiing outline in the shove video stream oil! jeed you io enother page
`j
`This ig exterwion of hyweriink Into video streams. The [le thet sgecifves the anchor informationis
`'
`segided on the server cide. We have tools for making that link Miles for our video server. See bere
`i
`for more detalis.
`|
`
`
`|
`2
`
`
`
` I Clonal NewWingdow)Clogs Window!
`
`
`
`one
`
`acerwi sti!
`
`
`
`Jb enemenie es
`
`FIG.22NSGe
`
`SUBSTITUTE SHEET (RULE 28)
`
`IPR2022-01227
`IPR2022-01227
`EXHIBIT 1003 - PAGE 01319
`EXHIBIT 1003 - PAGE 01319
`
`

`

`WO 97/2720
`
`PCT/US96/19226
`
`
`
`
`
`W3uUYYNVGUNNOWNVHOLOORYBIAOW/
`
`
`
`
`
`Co
`
`7/28
`
`AIAOWYSHLO
`
`
`__LYvd3dSOGNY399T1090ONINSSNIONSOnin|MGIAXSAO
`SND
`
`SUBSTITUTE SHEET (RULE 28)
`
`IPR2022-01227
`IPR2022-01227
`EXHIBIT 1003 - PAGE 01320
`EXHIBIT 1003 - PAGE 01320
`
`
`

`

`WO 97/22201
`
`POT/US96/19226
`
`28/28
`
`—_A
`
`Foang
`
`-NoLLISod;
`
`‘—
`
`OGSWVeS
`
`L{'i
`
`veOld
`
`OOSAVGS
`
`iSavas
`
`SUBSTITUTE SHEET (RULE 26)
`
`IPR2022-01227
`IPR2022-01227
`EXHIBIT 1003 - PAGE 01321
`EXHIBIT 1003 - PAGE 01321
`
`
`
`
`

`

`PCT
`
`WORLD INTELLECTUAL PROPERTY ORGANIZATION
`International Bureau
`
`
`
`Ped
`
`Published
`With muernational search report.
`
`|
`
`iii
`
`
`4H.
`CAMPBELL, Roy,
`(7U72) Applicants and Inventors:
`{US/US}, University of Hinois
`at Champaign-Urbama,
`Depr. of Computer Science, 1304 W. Springfield, Urbana,
`iL 61801 (US). TAN, See-Mong [SG/US]; University of
`Ulinois at Champaign-Urbana, Dept. of Computer Science,
`1304 W. Springfield, Urbana, TL 61801 (US). XTE, Bong
`[Ch/NOE University of Hiinois at Champaign-Urbana,
`Dept. of Computer Science, 1304 W. Springfield, Urbana,
`HL 6180]
`(US). CHEN, Zhigang [CN/USh University of
`Hilinois at Champaign-Crbana, Depr. of Computer Science,
`1304 W. Springfield, Urbana, IL 61801 CUS).
`
`INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT)
`
` (51) international Patent Classification 6 ;
`(1) International Publication Number:
`WO 97722261
`39 June 1997 (19.06.97}
`i
`¥ 5/06, HOAN 7/16, 9/79
`(43) International Publication Date:
`
`
`
`
`{81} Designated States: CN. IP, KR, RU, US, European patent (AT,
`
`(23) international Application Number: PCT/AUS86/19226
`BE, CH, DE, DK, ES, Fi, FR, GB, GR, TE, IT, LU, MC,
`
`12 December 1996 (12.12.96)
`
`NL, PT, S&).
`(22) International Filing Date:
`
`
`
`(3%) Priority Data:
`
`{2 December 1995 (12.12.95)&V/00853} US
`
`
` {88} Date of publication of the international search repert:
`|
`
`
`2 October 1997 (62.10.97) |
`
`
`
`
` RELIABLE TCP CHANNEL FOR
`FOR DATA AND FEEDBACK
`
`|
`
`| |
`
`{74} Agemts: BERNSTEIN, Frank, L. et al.; Sughrue, Mion, Zinn,
`Macpeak & Seas, Suite 800, 2100 Pennsylvania Avenue,
`
`N.W., Washington, DC 20037-3202 (US).
`
`(34) Tide: METHOD AND SYSTEM POR TRANSMITTING REAL-TIME VIDEO
`
`PLAYBACK COMMAND
`
`UNRELIABLE UDP CHANNEL
`
`
`
`(S7} Abstract
`
`The architecture of numerous networks, including the Internet with its World Wide Web (WWW) browsers and servers (550), support
`full file transfer for document retrieval. In order for the WWW to support continuous media, it is mecessary to transmit video and audic on
`demand and in real-time, as well as new protocols for real-time data. The invention extends the architecture of the WWWto encompass the
`dynamic, real-time information space of video and audio. The inventive method, called Vosaic, short for Video Mosaic, incorporates real-
`time video and audio into standard hypertext pages and which are displayed in place. The invention includes a real-time protocol, called a
`video datagram protece] (VDP), for handling real-time video over the WWW. VDP minimizes inter-frame jitter and dynarnically adapts to
`the client (500) CPU load and network congestion.
`
`
`
`
`
`
`
`IPR2022-01227
`IPR2022-01227
`EXHIBIT 1003 - PAGE 01322
`EXHIBIT 1003 - PAGE 01322
`
`
`
`
`
`
`
`
`
`
`
`

`

`
`
`(eeRRNAnnA
`
`FOR THE PURPOSES OF INFORMATION ONLY
`
`Codes used to identify States party to the PCT on the front pages of pamphlets publishing international
`applications under the PCT.
`
`GB
`Gk
`GN
`GR
`HU
`iz
`if
`a?
`RE
`KG
`KP
`
`|
`
`AM
`Armenis
`Unieed Kingdom
`MW
`Malawi
`AT
`Austria
`Georgia
`MX
`Mexico
`AU
`Australia
`Guines
`NE
`Niger
`BS
`Barbados
`Greece
`NL
`Netherlands
`BE
`Belgium
`Hungary
`NO
`Norway
`SF
`Burkina Faso
`Iweland
`RZ
`New Zealand
`BG
`Bulgaria
`Tealy
`PL
`Poland
`Bi
`Benin
`Japa
`PY
`Porugal
`BR
`Brazil
`Kenya
`RO
`Romania
`BY
`Belarus
`Kyrgystan
`RU
`Russian Federation
`CA
`Canada
`Democratic Peopk’s Republic
`SB
`Sudan
`cE
`Central African Republic
`of Kors
`SE
`Sweden
`ce
`Congo
`KR
`Republic of Korea
`8G
`Singapore
`cH
`Switzerland
`KZ
`Kazakhstan
`St
`Slovenia
`ci
`Cie dlvolse
`u
`Liechtenstein
`8K
`Stovakia
`cM
`Cameroon
`LEK
`Sri Lanka
`BN
`Senegal
`CN
`China
`LR
`Liberia
`SZ
`Swaziland
`cs
`Crechoslovakia
`LY
`Lithuania
`TD
`Chad
`CZ
`Czech Republic
`LU
`Luxembourg
`TG
`Togs
`BE
`Germany
`LY
`Latvia
`TI
`Tajikistan
`DK
`Denmark
`MC
`Monaco
`vr?
`Trinidad and Tobago
`BE
`Estonia
`MD
`Repablic of Moldova
`vA
`Ukraine
`ES
`Spain
`MG
`Madagascar
`UG
`Uganda
`Fi
`Finland
`ML
`Mali
`US
`United States of America
`i
`FR
`Fravice
`REN
`Mongolia
`Us
`Uzbekistan
`GA
`Gabon
`MR
`Mauritania
`VN
`Vier Nam
`al
`IPR2022-01227
`IPR2022-01227
`EXHIBIT 1003 - PAGE 01323
`EXHIBIT 1003 - PAGE 01323
`
`:
`i
`i
`
`i
`
`

`

`INTERNATIONAL SEARCH REPORT
`
`International application Ne.
`PCTAUS9G6/19226
`
`
`
`
`
`
`A.
`CLASSIFICATION OF SUBIECT MATTER
`
`
`|
`IPC(6)
`:GOGF 5/66, HOSN 7/16, 9/79
`i US CL :364/314; 348/7; 295/200.03; 370/79
`
`
`
`
`
`iAccording to International Patent Classification (PC) or to both national classification and IPC
`
`
`
`[B.
`PIELDS SEARCHED
`
`Minimum documentation searched (classification system followed by classification symbols)
` Us: 364/514; 348/7;, 395/200.03; 37O/79
`
`Documentation searched other than minimum documentation te the extent that such documents are included in the fields searched
`
`
`
` Electronic data base consulted during the international search (name of data base and, where practicable, search terms used)
`
`
`ic.
`
`
`DOCUMENTS CONSIDERED TO BE RELEVANT
`
`
`
` Citation af document, with indication, where appropriate, of the relevant passages
`| US, 5,442,396, A (HOOPER ET AL.} 15 August 1995, col.|
`
`
`
`
`
`
`i US, 5,247,363, A (SUN ET AL.} 21 September 1993, col. 3.
`|
`line 32 to cal. 5, line 45.
`
`
`
`
`US, 5,537,408, A (BRANSTAD ET AL.) 16 July 1996, see
`
`the whale referance.
`
`
`
`US, 5,187,754, A (CURRIN ET AL.) 16 February 1993, see}
`the whole reference.
`
`
`
`
`US, 5,557,320, A (KRESS) 17 September 1996, see the!
`whole reference.
`
`
`
`
`| [x]
`Further documenis are listed in the continustion of Box C. CI
`See patent family annex.
`Spocial categories of cited documents:
`|
`any das eon
`ster documentpuibtiakedafterthe=~wnt
`
`ate ancl mot an Soatiictwith te appletberche fo understand the
`f “AS
`fe
`documentdefining the geacra state of the ent whick is aot considered
`
`
`prieciple or theory undertying the invention
`s@ be of particuby relevance
`
`
`aye
`.
`:
`.
`.
`document of particular relevance: the claimed invention cmanet be
`
`
`E
`earlier documemtpublished on or efter the iniemationsl filing date
`comeidened sowel or cxanot be considered to involve an mventivestop
`
`
`“lL
`document which may throw doube on priority claam(s) or whick iz
`when the document is nken akone
`
`sited to cateblih the publication date of another citation or atkes
`
`
`special reason (os apecified}
`document of particular relevance; the claimed invention cannot be
`¥
`i
`osapidered to invalve am ieventive step when the document is
`
`
`document referring to an ond dinclosuse, use, eahibitice or other
`§ OY
`combined with ome or sar: other auch document, such combination
`means
`5
`being obvions io a person killed in the ext
`
`
`
`
`document published griar te the intesnationnl filing date but inter than
`document member of the same patent fanily
`the prionty date claimed
`
`i Date of the actual completion of the international search
`
`
`18 JUNE 1997
`
` Name and mailing address of the ISA/US
`Commissioner of Patents and Trademarks
`
`Box POP
`
`Washington, B.C. 20231
`
` “01227
`Facsimile No.
`(703) 305-3230
`Form PCTHSA/216 (second sheetsuly 1992)e
`
`¥
`
`4, line 36 to col. G6, line 57, col. 8-13.
`
`
`Y,£
`
`US, 5,610,841, A (TANAKA ET AL.) 11 March 1997. col.
`
`11, Ines 12-35, col. 18-26.
`
`Y
`
`PY, P
`
`A
`
`A
`
`“x”
`
`+g°
`
`IPR2022-01227
`EXHIBIT 1003 - PAGE 01324
`EXHIBIT 1003 - PAGE 01324
`
`

`

`INTERNATIONAL SEARCH REPORT
`
`
`
`
`
`international application No.
`POTAIS9G/19226
`
`iA,E
`
`US, 5,633,683, A (ROSENGREN ET AL.) 27 May 1997, see the
`whole reference.
`
`[33-35
`
`
`
`LAB US, 5,634,064, A (WARNOCK ET AL.) 27 May 1997, see the|33-35
`whole reference,
`
`IPR2022-01227
`IPR2022-01227
`EXHIBIT 1003 - PAGE 01325
`EXHIBIT 1003 - PAGE 01325
`
`

`

`INTERNATIONAL SEARCH REPORT
`
` international application Na.
`
`
`PCTSUSS6/ 19226
` Box 1 Observations where certain claims were found unsearcheble (Coatinuation of Kem 1 of first sheel}
`
`
`‘This international report has not been established in respect of certain claims under Auticle 12}(a) forthe following reasons:
`
`1. | Cisims Nos.:
`because they relate to subject matter not required to be searched by this Authority, namely:
`
`co
`
`Claims Mos.:
`because they relate to parts of the international application that do not comply with the preseribed requirements to such
`an extent that no meaningful international search can be carried cut, specifically:
`
`: 3.i| Claims Nos.:
`because they are dependent claims and are net drefied in accordance with the second and third sentences of Rule 6.4(a).
`
`This international Searching Authority found multiple inventions in this international application, as follows:
`
`ii.
`i
`
`As all required additional search foos were timely paid by the applicant, this international search report covers all searchable
`claims.
`
`2. [| Agsall searchable claims could be searched without effortjustifying an additional fee, this Authority did not invite payment
`of any additional fee.
`3. Cy As only sume ofthe requiredadditional search feea were timely paid by the applicant, this international search repert covers
`only those claims for which fees wore paid, specifically claims Nos.
`
`i
`
`F
`
`i4.C| Mo required additional search fees were timely paid by the applicant. Consequently, this international search report is
`i
`restricted to the invention first mentioned in the claims; it is covered by claims Nos.:
`
`Remark on Protest
`
`J The additional search fees were accompanied by the applicant's protest.
`
`Form PCTHSA/210 (continuation of first sheet(1)(uly 1992)y
`
`IPR2022-01227
`IPR2022-01227
`EXHIBIT 1003 - PAGE 01326
`EXHIBIT 1003 - PAGE 01326
`
`

`

`
`
`HON 7/i4
`21 August 1997 (21.08.97)
`(43) International Publication Date:
`epeeonnnnnnnnannannnannnnnnntnnsneannaenneennnannnans
`ee.
`
`
`
`(23) International Application Number:
`
`PCTITLOT/00055
`
`(22) International Filing Date:
`
`13 February 1997 (13.02.97)
`
`(30) Priority Data:
`Li7i33
`119655
`08/788 830
`
`14 February 1996 (14.02.96)
`20 November 1996 (20.11.96)
`6 January 1997 (06.01.97)
`
`IL,
`IL
`US
`
`WORLD INTELLECTUAL PROPERTY ORGANIZATION
`PCT
`International Bureau
`.
`INTERNATIONAL APPLICATION PUBLISHED UNDER THE PATENT COOPERATION TREATY (PCT)
`
`
`WwWoO STf3gSSi
`(Sl) International Patent Classification © ;
`(1) International Publication Number:
`
`
`
`
`
`
`(81) Designated States: AL, AM, AT, AT (Utility model}, AU, AZ,
`BA, BB, BG, BR, BY, CA, CH, CN, CU, CZ, C2 (Utlity
`mode), DE, DE (Utility modeD, DE, DK (Uullty made),
`EE, EE (Utility model}, ES, F1, PI (Utility model), GB, GE,
`U, iL, IS, IP, KE, KG, KP, KR, KZ, LC, LK, LR, LS, LT,
`BU, LV, MD, MG, MK, MN, MW, MX, NO, NZ, PL, PT,
`RO, RU, SD, SE, SG, SI, SK, SK (Utility model, TY, TM,
`TR, TT, UA, UG, US, UZ, VN, YU, ARIPO patent (KE,
`LS, MW, SD, §Z, UG), Eurasian patent (AM, AZ, BY, KG,
`KZ, MD, RU, TI, TM}, European patent (AT, BE, CH, DE,
`DK, ES, Fl, PR, GB, GR, TE, IF, LU, MC, NL, PT, SE),
`OAPI patent (BP, BI, CF, CG, Cl, CM, GA, GN, ML, MR,
`NE, 8N, TD, TG),
`
`(71) Applicant (for all designated States except US): OLIVR COR: |
`PORATION LID. {ILL}, Kanfei Nesharim Sreet 24
`95464 Ferusaiom CL).
`| Published
`(72) baventor; and
`(78) Inveator/Applicant (for US only}: GUEDALIA, Jacob, Leon |
`With international search report.
`{EL/IL}, Haporzim Street 24, 92541 Jerusalem (IL).
`i
`Before the expiration of the time limit for amending the
`
`i
`claims and ta be republished in the event af the receipt of |
`|
`amendments,
`
`i
`
`(74) Agents: COLB, Sanford, T. et al; Sanford T. Colb & Co, P.O.
`Box 2273, 76122 Rehovet (IL).
`
`
`Jaananentnnernnneneronearest
`: METHOD AND SYSTEMS FOR PROGRESSIVE ASYNCHRONOUS TRANSMISSION OF MULTIMEDIA DATA
`
`
`Production Workstation
`
`
`
`input Images
`
`
`
`wn M8
`Enceder
`5
`
`
`
`/
`2a?
` Compression
`Encoded
`
`
`| Transmit
`
`_{ Goniroter
` ,} Partial Frarne
`
`Sudtractor
`
`
`‘ |
`
`(S57) Abstract
`
`
`
`A system for trangmitting digital data representing an original over plural transmission links (101), at least some of which havelimited
`bandwidth, inchading a digital data source storing digital data representing the original, a digital data receiver receiving data representing
`the original via one of the plural transmission links having limited bandwidth and a digital wansmitter for transmitting the digital data
`representing the original to the receiver over a transmitting link having a limited bandwidth in plural blocks (76) which are sequentially
`transmitted at a rate determined by the limited bandwidth, each block being an incomplete collection of data which includes parts of multiple
`frames (72}, receipt of subsequent blocks bythe receiver being used to cumulatively improve the quality of the digital data viewed by the
`
`||
`oo
`
`receiver.
`
`IPR2022-01227
`IPR2022-01227
`EXHIBIT 1003 - PAGE 01327
`EXHIBIT 1003 - PAGE 01327
`
`

`

`
`FOR THE PURPOSES OF INFORMATION ONLY
`
`Codes used to identify States party to the PCT on the front pages of pamphlets publishing international
`applications under the PCT.
`
`ROW
`MR
`NE
`NL
`NO
`RZ
`PL
`Pr
`RO
`RU
`
`SE
`8G
`Si
`3K
`SN
`
`Malawi
`Mexico
`Niger
`Netherlands
`Norway
`New Zealand
`Poland
`Portugal
`Romania
`Russian Federation
`Sudan
`Sweden
`Singapore
`Slovenia
`Slovakia
`Senegal
`Swaziland
`Chad
`Tago
`Tajikistan
`Trinidad and Tobago
`Ukraine
`Uganda
`United Seates of Amesica
`Uztekistan
`Yies Nam
`
`
`
`IPR2022-01227
`IPR2022-01227
`EXHIBIT 1003 - PAGE 01328
`EXHIBIT 1003 - PAGE 01328
`
`AM
`At
`AU
`BB
`BE
`BF
`3G
`Bs
`aR
`BY
`CA
`cr
`CG
`cH
`ce
`cM
`CN
`cs
`Cf
`BE
`
`'
`
`i |
`
`to
`
`Armenia
`United Kingdom
`Austria
`Georgia
`Australia
`Guinea
`Barbados
`Greece
`Belgium
`Hungary
`Burkina Faso
`Ireland
`Kaly
`Bulgasta
`Benin
`Japan
`Brazil
`Kenya
`Belarus
`Kyrgystan
`Canada
`Democratic People's Republic
`of Korea
`Central African Republic
`Congo
`Republic of Korea
`Switzerland
`Kazakhstan
`Cte d'fvoire
`Liechtenstein
`Cameroon
`Sri Lanka
`China
`Liberia
`Lithuania
`Czechoslovakia
`Czech Republic
`Luzembourg
`Latvia
`Germany
`Denmark
`Monaco
`EE
`Bsronia
`Republic of Moldova
`gS
`Spain
`Madagascar
`FE
`Mali
`Fintand
`FR
`France
`MN
`Mongolia
`MR
`Gabon
`Mauritania
`GA
`
`
`GB
`GE
`GN
`GR
`HU
`IE
`if
`dP
`KE
`KG
`KP
`
`KZ
`LS
`Le
`LR
`Le
`LY
`LY
`MC
`MB
`MG
`
`

`

`WO 97/30531
`
`PCTALGTH/OD035
`
`METHOD AND SYSTEMS FOR PROGRESSIVE ASYNCHRONOUS
`
`TRANSMISSION OF MULTIMEDIA DATA
`
`FIELD OF THE INVENTION
`
`The present
`
`invention relates to methods and systems for encoding digital
`
`multimedia data for transmussion over a network.
`
`BACKGROUND OF THE INVENTION
`
`The following U.S. Patents have been found in a U.S. Patent Search and are
`
`believed to be generally relevant to the field of the invention:
`
`4,897,867
`5,119,188
`§,122,873
`5,195,092
`§,220,420
`5,236,199
`§,251,209
`5,265,248
`$,283,819
`§,328,423
`$351,276
`§,363,482
`5,420,572
`$420,801
`5,438,658
`5,487,167
`3,495, 376
`§, 508,940
`§,519,438
`5,553,221
`
`1/90
`6/92
`5/92
`3/93
`6/93
`8/93
`10/93
`1i/93
`1/94
`6/94
`9/94
`11/94
`5/95
`5/95
`8/95
`1/96
`2/96
`4/96
`5/96
`9/96
`
`Foster et al.
`McCalley et al.
`Golin
`Wilson et al.
`Hoarty et al.
`Thompson, Jr.
`Jurkevich et al,
`Moulios et al.
`Ghck et al.
`Lewis
`Doll, Ir. et al.
`Victor et al.
`Dolin, Fr. et al.
`Dockteret al.
`Fitzpatrick et al.
`Dinallo et al.
`Ritchey
`Rossmere et al.
`Anderson
`Reimeret al.
`
`SUBSTITUTE SHEET (RULE 26)
`
`IPR2022-01227
`IPR2022-01227
`EXHIBIT 1003 - PAGE 01329
`EXHIBIT 1003 - PAGE 01329
`
`

`

`WO 97/30551
`
`PCTALSFOSS
`
`§ 553,222
`$557,538
`5,561,791
`5,564,001
`5,377,180
`5,877,258
`5,581,783
`
`9/96
`O/96
`10/96
`106
`11/96
`11/96
`12/96
`
`Milne et al.
`Retter et al.
`Mendelson et al.
`Lewis
`Reed
`Cruz et al.
`Ohashi
`
`When using various media such as video, audio, text and iMages, a user generally
`retrieves the media from a storage device or “server® connected via a network to many
`computers or users. The server downloads the media to the network and transmits it to
`the user at the user's request.
`There are two basic limitations involved in such data retrieval: delay between the
`time that a user requests the data and the time when the server downloads it to the
`network, and bandwidth limitations on data throughput and rate of data transmission.
`The present invention relates to the second limitation.
`One example of such a system includes a CD ROM drive and personal computer
`which may be located at the same site. Another example includes a network connecting
`Internet servers and users’ personal computers. Such networks are installed in order tc
`facilitate convenient data transmission between users and data distribution from the
`server to the users’ computers.
`typically
`When a user retrieves the media from a storage device or server,
`connected via a network, the aforesaid bandwidth limitations affect the ammount of time
`required to transmit a video frame from the server to the user, and thus limit the video
`frame rate. Moreover, when dealing with object movies and panoramas the files being
`transmitted are extremely large, so that overcoming bandwidth limitations is a critical
`enabling factor, even for high bandwidth networks.
`
`Currently, two methods are employed to overcome bandwidth limitations: The first
`is to compress the video frame sequence, thereby speeding up transmission time at the
`cost of additional downstream processing to decompress the frames prior to display.
`The second is to copy the entire sequence to an intermediate storage device, such as a
`user's hard disk, to which the user has higher bandwidth access, at the cost of delaying
`the viewing of the video until the entire sequence has been delivered.
`
`SUBSTITUTE SHEET (RULE 26)
`
`IPR2022-01227
`IPR2022-01227
`EXHIBIT 1003 - PAGE 01330
`EXHIBIT 1003 - PAGE 01330
`
`

`

`WO F7/30331
`
`PCTALSTMOOSS
`
`Known network applications involve streaming data from a server to a client
`computer {hereinafter also referred to as “client"). "Streaming" refers to serial or parallel
`transmission of digital data between two computers, by transmitting sequences ofbit
`packets. For example,
`installation executables on a network server stream files to a
`client computer performing the installation. Servers with large amounts of memory are
`used to archive digital movies, which are streamed to a client computer for viewing upon
`demand. Digital video is broadcast from cable stations to subscribers using streaming.
`Internet browsers, such as Netscape and Microsoft Explorer, are used to stream data
`from a server on the web to a client.
`Internet web sites can contain enormous databases,
`such as phone directories for all of the cities in the U.S., photographs from art galleries
`and museums around the world, voluminous encyclopedias, and even copies of all
`patents ever issued by the U.S. Patent & Trademark Office. Clients using the Internet
`can search these databases and then request the server to download specific information.
`This request initiates a streaming event.
`In view of the multitude of bandwidths present in complex client/server systems
`today and the large amounts of data necessary to produce compelling audio, video and
`imaging presentations, there is a pressing demand for scalable data representation of
`multimedia data, so that it can be delivered for on-line interactive playback in such a
`form that the transmission rate can accommodate the client bandwidth. The Internet is
`
`an example of a complex system where clients with many different bandwidth constraints
`demand multimedia data from server sites. There are several low bandwidth rates for
`modem transmission over phone lines, higher rates for framerelay lines, higher rates for
`ISDN lines, even higher rates for T1 lines, etc.
`
`SUBSTITUTE SHEET (RULE 26}
`
`IPR2022-01227
`IPR2022-01227
`EXHIBIT 1003 - PAGE 01331
`EXHIBIT 1003 - PAGE 01331
`
`

`

`WO 97/3083
`
`PCTELS7MOGSS
`
`SUMMARY OF THE INVENTION
`
`invention seeks to provide an improved method and system for
`The present
`transmitting digital data representing the original over plural transmission links at least
`
`some of which have limited bandwidth.
`
`The present invention relates to scalable encoding, which enables two or more
`clients, connected to a server by lines having differing bandwidth, to begin playing the
`multimedia data on-line, and both at the same time, almost immediately after the start of
`streaming, but the lower bandwidth client receives lower quality media than the higher
`bandwidth chent at first. As the media is replayed in the foreground and the bandwidthis
`freed, more data streams in via background, and the quality of the media is enhanced.
`Without scalable encoding, a client of an Internet application must wait until the
`
`requested dats arrives, at whatever rate its network line provides. A client with a 14.4
`
`Kbs modem line, for example, would have to wait twice as long as a client with a 28.8
`Kbs modem line. Moreover, if the data was encoded for 28.8 Kbs on-line playback, the
`14.4 Kbs chent would never be able to achieve live playback, since there would be an
`
`ever-increasing lag in the data stream. Conversely, if the data were encoded for 14.4
`Kbs on-line playback,
`the 28.8 Kbs client would receive unnecessarily poor quality
`media.
`
`On account of the progressive nature of the data representation in accordance with
`the present invention, the additional data block arriving in a 14.4 Kbs stream combines
`
`with the previous data block which arrived in a 14.4 Kbs stream, to produce a 28.8 Kbs
`streamed version, all that is being sent is the incremental data necessary for the upgrade.
`The progressive form of the encodingitself provides the ability to achieve scalability.
`Another shortcoming of non-scalable encoding as in the prior art is the inability to
`preview a video sequence. Often a client would like to play a quick preview of a video
`clip, before deciding whether or not to download it. The scalable representation of the
`
`present invention can be used to deliver the video in a preview mode, as the first data
`
`blocks.
`
`Ifthe client continues to download the video afler previewing, the first data
`
`SUSSTITUTE SHEET (RULE 26}
`
`IPR2022-01227
`IPR2022-01227
`EXHIBIT 1003 - PAGE 01332
`EXHIBIT 1003 - PAGE 01332
`
`

`

`WO 97/30551
`
`PCT/ELS7/O0085
`
`block already transmitted is progressively integrated with additional data blocks to create
`the full viewing video.
`.
`The present invention can also be applied to enhance delivery of large still images
`for multi-resolution gazing. Current technology transmits such images as large files, and
`camies Out extensive computations for sub-sampling to lower resolution and zooming in
`to areas of interest for gazing. This makes it very time consuming to interact with large
`images, and as a result it is currently impractical to produce high resolution images for
`Internet browsing. When dealing with large images, producers simply sub-sample them
`to fit entirely within a computer monitor screen, and store the resulting low resolution
`images on web servers. Using the technology of the present invention, producers can
`deliver high resolution images over the Internet for rapid interactive gazing.
`The present invention seeks to provide a scalable representation of multimedia
`data, enabling the data to be (a) progressively streamed, (b) transmitted asynchronously
`to clients at different bandwidths and (c) played back interactively on-line.
`The
`representation is
`two-dimensional, with one dimension
`(block number)
`being
`characterized by progressiveness in quality, and the second dimension (frame mumber)
`being characterized by interactivity.
`The representation comprises data blocks which are integrated with one another to
`produce successively higher bandwidth versions ofthe media, the data blocks comprising
`encoded frames. The first data block corresponds to the lowest bandwidth, and enables
`the chent with this bandwidth to play back the media on-line at the lowest quality. The
`second data block, when integrated with the first block, corresponds to the next higher
`bandwidth, and enables the client with this bandwidth to play back the media on-line at
`the next highest quality, and similarly for each successive data block.
`Moreover, a client with the lowest bandwidth who played the media at the low
`quality and freed the bandwidth can continue in background to receive successive data
`blocks and integrate them with previously received data blocks, resulting in successively
`higher quality media each time it
`is
`replayed.
`The modular
`form of the data
`representation thus makes it possible to both accommodate different bandwidths and
`progressively update media quality. A production tool makesit possible for a producer
`to control modularity and quality settings,
`
`SUS STITUTE SHEET (RULE 26}
`
`IPR2022-01227
`IPR2022-01227
`EXHIBIT 1003 - PAGE 01333
`EXHIBIT 1003 - PAGE 01333
`
`

`

`WO S7/30851
`
`PCT/DLS700055
`
`in accordance with the present invention there is provided a method for providing
`on-line virtual reality movies, comprising inputting a cyclic movie sequence into an
`encoder, determining the number of portions that each frame of said movie is divided
`into, and forming partial frames, specifying hot-spots and independent objects for
`interaction within a partial frame, transmitting the partial frames part by part to a user's
`asynchronous database, and displaying said frames on a user's interface.
`There is also provided in accordance with a preferred embodiment of the present
`invention a system for producing virtual reainy (VR) movies comprising an encoder for
`preparing the VR. movie for transmission