United States Patent [19]
`Glaser et al.
`
`[54] AUDIO-ON-DEMAND COMMUNICATION
`SYSTEM
`
`[75]
`
`Inventors: Robert D. Glaser. Seattle. Wash.;
`Mark O'Brien. Cambridge. Mass.;
`Thomas B. Bouten. Seattle. Wash.;
`Randy Glen Goldberg. Princeton. N.J.
`
`[73) Assignee: RealNetworks, Inc.. Seattle. Wash.
`
`[21) Appl. No.: 347,582
`
`Nov. 30, 1994
`
`[22) Filed:
`Int. CL 6
`.............................. H04H 1/02; G06F 13/42
`[51]
`[52) U.S. Cl . ................................ 395/200.61; 395/200.49;
`395/200.67; 370/528
`[58) Field of Search ......................... 395/200.13. 200.09.
`395/200.14. 200.17. 806. 807. 849. 850.
`200.61. 200.49. 200.67. 200.66; 370/231.
`476. 486. 496. 528; 364/514 A. 514 C;
`375/257. 369. 370; 345/302
`
`[56]
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`3,882,538
`4,253,157
`4,504,705
`4,506,387
`4,581,484
`4,611,277
`4,658,093
`4,827,256
`4,845,756
`4,899,299
`4,905,094
`4,941,123
`4,975,691
`4,987,529
`4,999,806
`
`5/1975 Lowe.
`2/1981 Kirschner et al. .
`3/1985 Pilloud.
`3/1985 Walter.
`4/1986 Bendig.
`9/1986 Kemppainen et al ..
`4/1987 Hellman.
`5/1989 Yokoyama.
`7/1989 Seen et al ..
`2/1990 MacPhail.
`2/1990 Pocock et al ..
`7/1990 Thompson.
`12/1990 Lee.
`1/1991 Craft et al ..
`3/1991 Chernow et al ..
`
`(List continued on next page.)
`
`FOREIGN PATENT DOCUMENTS
`
`309298
`WO93/16557
`WO94/14273
`
`3/1989 European Pat. Off ..
`8/1993 WIPO.
`6/1994 WIPO.
`
`I 1111111111111111 11111 IIIII IIIII IIIII IIIII IIIII IIIII IIIII 111111111111111111
`US005793980A
`[111 Patent Number:
`[451 Date of Patent:
`
`5,793,980
`Aug. 11, 1998
`
`OTHER PUBLICATIONS
`
`PCT/US95/14765 PCT International Search Report dated
`Apr. 11. 1996.
`
`Stephen J. Bigelow. Modem Communications Standards.
`Electronics Now. pp. 35-42. Sep. 1994.
`
`PCT Written Opinion Mailed Nov. 14. 1996 International
`Application Number PCT/US95/14765.
`
`C. Topolcic; "experimental Internent Stream Protocol. Ver(cid:173)
`sion 2 (ST-II)": Internet Working Group Request for Com(cid:173)
`ments RFC-1190. Oct. 1990.
`
`Primary Examiner-Mark H. Rienhart
`Attorney, Agent, or Firm-Knobbe. Martens. Olson & Bear.
`LLP
`
`[57)
`
`ABSTRACT
`
`An audio-on-demand communication system provides real(cid:173)
`time playback of audio data transferred via telephone lines
`or other communication links. One or more audio servers
`include memory banks which store compressed audio data.
`At the request of a user at a subscriber PC. an audio server
`transmits the compressed audio data over the communica(cid:173)
`tion link to the subscriber PC. The subscriber PC receives
`and decompresses the transmitted audio data in less than
`real-time using only the processing power of the CPU within
`the subscriber PC .. According to one aspect of the present
`invention. high quality audio data compressed according to
`lossless compression techniques is transmitted together with
`normal quality audio data. According to another aspect of
`the present invention. metadata, or extra data. such as text.
`captions. still images. etc., is transmitted with audio data and
`is simultaneously displayed with corresponding audio data.
`The audio-on-demand system also provides a table of con(cid:173)
`tents indicating significant divisions in the audio clip to be
`played and allows the user immediate access to audio data
`at the listed divisions. According to a further aspect of the
`present invention. servers and subscriber PCs are dynami(cid:173)
`cally allocated based upon geographic location to provide
`the highest possible quality in the communication link.
`
`6 Claims, 17 Drawing Sheets
`
`I
`
`L' .\I•\
`Q,\l,\
`, -~ - -
`['l,H.D,,
`,~~---,
`
`1
`
`(~U..\ _J
`['1-',1,'\
`
`(~A 1 J.
`
`['i.'\T,\
`
`l~;T~
`
`I
`
`I
`I I
`
`5LIBSCRl8ER
`
`----- ~ Tl\ 1\NSMI T
`.\0,NllWl £1}1..~F
`
`Petitioners' Exhibit 1018
`Page 0001
`
`

`

`5,793,980
`Page 2
`
`OTHER PUBLICATIONS
`
`5,001580
`5,051.822
`5.057.932
`5,109,482
`5,132,99'2
`5,195,09'2
`
`3/1991 Aranovsky et al ..
`9/1991 Rhodes .
`10/1991 Lang .
`4/1992 Bohrman .
`7/199'2 Yurt et al ..
`3/1993 Wilson et al. .......................... 370/528
`
`5,237,322
`5,247,347
`5,253,341
`5,262,875
`5,282,028
`5.283,819
`5,289,545
`5,297,249
`
`8/1993 Heberle.
`9/1993 Litteral et al ..
`10/1993 Rozmainth et al.
`ll/1993 Mincer et al ..
`l/1994 Johnson et al ..
`2/1994 Glick et al ..
`2/1994 Jestice .
`3/1994 Berstein et al ..
`
`Petitioners' Exhibit 1018
`Page 0002
`
`

`

`5,793,980
`5,793,980
`
`ww
`
`Or©
`
`Petitioners’ Exhibit 1018
`Page 0003
`
`CENTER
`
`AUDIO
`
`CONTROL
`
`Ne
`
`8\
`
`U.S. Patent
`US. Patent
`
`Aug. 11, 1998
`Aug. 11, 1998
`
`Sheet 1 of 17
`Sheet 1 of 17
`
`Petitioners' Exhibit 1018
`Page 0003
`
`

`

`U.S. Patent
`
`Aug. 11, 1998
`
`Sheet 2 of 17
`
`5,793,980
`
`-210
`
`LIVE
`SOURCE
`
`-- _/
`/----215
`
`/
`
`RECORDED
`SOURCE
`
`A/D CONVERTOR
`
`DIGITAL
`,,---220 SOURCE
`
`225
`
`1 1
`
`-
`
`COMPRESSOR - - - -~
`
`-
`
`-
`
`-
`
`-
`
`'2
`1. I,
`
`i ~ -
`- - /~ 2~ -
`I
`-
`-
`-
`I I
`ARCHIVAL
`I I
`STORAGE
`I
`I
`-....----11
`I I
`I . - - - - ' - - -~
`~~r.5
`I I
`I
`PRIMARY
`.,,-------240
`I I
`I
`SERVER
`I
`L __ -_-_-_-_ -_- -_-_-_-_-_-_-_-_-_-_J_J/_.--250
`
`DISI, STORAGE
`
`1
`I
`
`NET TRANSPORT
`
`TEMPORARY
`STORAGE
`
`PROXIMATE
`SERVER
`
`SUBSCRIBER
`P.C.
`
`260---/
`
`270
`
`NET
`TRANSPORT
`
`/
`110/
`
`110
`
`SUBSCRIBER
`P,C.
`
`_,,,..-----265
`/260
`.---'----'--'-I---.
`, - - -L I_ ' - - - - ,
`PROXIMATE
`TEMPORARY
`SERVER
`STORAGE
`
`NET
`TRANSPORT
`
`SUBSCRIBER
`P.C.
`
`270
`
`110
`
`Petitioners' Exhibit 1018
`Page 0004
`
`

`

`272 -----
`
`t
`
`'
`
`'
`
`FLOW
`CONTROL
`
`282 ----_,
`
`--,1
`
`TCP /IP
`PROTOCOL
`
`2?2 ----~ ~--
`
`TCP/IP
`PROTOCOL
`
`274
`
`.:c..,/"'76
`
`/·2
`
`I
`
`284-,,"
`
`MODEM
`
`ERROR
`",,
`' CORRECTING
`MODEM
`
`l
`
`l
`
`"--,, TELEPHONE
`LINES
`
`286-
`
`TELEPHONE
`LINES
`
`2,78
`
`',"
`
`,280
`
`I'--,,
`
`!
`
`ERROR
`CORRECTING
`MODEM
`
`t
`FLOW
`CONTROL
`
`t
`
`288 -~
`',
`
`MODEM
`
`290
`
`TCP /IP
`PROTOCOL
`
`294 - , ,
`'\
`\
`
`HIGH SPEED
`NETWORI-..
`(T1 OR
`OTHER FAST
`TRANSPORT)
`
`TCP M-l I
`285 --lJ
`1-1 PROT~
`
`FIG 2B
`
`FIG 2C
`
`FIG 20
`
`•
`
`~ • 7JJ.
`~ ....
`~ = ....
`
`>
`{J
`'"""
`'""" ~
`
`'""" !
`
`~ ro
`
`ro -~
`
`~
`~
`....:I
`
`Ul
`,_.
`....;a
`~ ,_.
`\0
`QC
`0
`
`Petitioners' Exhibit 1018
`Page 0005
`
`

`

`U.S. Patent
`
`Aug. 11, 1998
`
`Sheet 4 of 17
`
`5,793,980
`
`1/0
`
`SUBSCRIBER PC
`BUFFERS
`
`/310
`i
`/-315
`
`i
`
`300~' 312
`
`---~~LJ)
`
`AUDIO
`CON TR OL ----------.--- RE CE I VER
`CENTER
`/
`
`~
`
`CPU
`~-322~324 /326
`SCRATCH
`BUFFER ._
`DECODER
`,,
`'--,_32·0 /335
`
`_-, __ -;;-2-----.
`'"
`
`130/
`
`WAVE DRIVER
`
`L-,-------.3. ''7
`..JL
`
`I DAC
`L.----342
`
`~
`~
`~
`
`FIG, 3
`
`Petitioners' Exhibit 1018
`Page 0006
`
`

`

`U.S. Patent
`
`Aug. 11, 1998
`
`Sheet 5 of 17
`
`5,793,980
`
`FIG 4,..;/
`
`BEGIN
`
`SEND MESSAGE
`TO DESIGNATED
`ADDRESS
`
`NO
`
`C
`
`B
`
`404
`
`SEND REQUEST
`MESAGE TO
`SERVER WITH
`CUP NAME
`
`,---405
`
`/
`
`SEND MESSAGE
`TO DESIGN A TED
`ADDRESS
`
`YES
`
`,---408
`
`07
`
`NOED TH
`
`NO
`
`END
`
`SEND BEGIN
`MESSAGE
`TO SERVER
`
`FLUSH BUFFERS AND
`IGNORE MESSAGES
`FROM SERVER UNTIL
`NEW DATA COMES:
`ATTEND TO ANY
`PENDING MESSAGES
`FROM SUBSCRIBER
`
`YES
`
`SEND A
`SEEi\
`MESSAGE
`\"
`' - 4 l4
`
`Petitioners' Exhibit 1018
`Page 0007
`
`

`

`U.S. Patent
`
`Aug. 11, 1998
`
`Sheet 6 of 17
`
`5,793,980
`
`(D)
`
`(c)
`
`A)
`
`,,.--4..-;....,l
`
`SEND A
`PAUSE MESSAGE
`TO THE
`SERVER
`
`FILL
`BUFFERS
`
`SEND A STOP
`MESSAGE TO THE
`SERVER
`
`YES
`
`,/4 .24
`
`HAS
`THE USER
`INITIATED A
`STOP MESSAGE
`
`SEND AN
`END MESSAGE
`TO THE SERVER
`
`HAS
`,,,,.-4215
`THE USER
`INITIATED AN
`END CONNECTION
`
`SEND MESSAGE
`TO DESIGNATED
`ADDRESS
`
`NO
`
`YES
`,-439
`
`/
`
`/440
`
`DECOMPRESS
`A
`SUFFER
`
`LOAD
`WAVE
`DRIVER
`
`445
`
`//_45,_,7
`~-_.__~~
`
`SET
`~-------< DROP-OUT
`FLAG
`
`4B
`
`Petitioners' Exhibit 1018
`Page 0008
`
`

`

`U.S. Patent
`
`Aug. 11, 1998
`
`Sheet 7 of 17
`
`5,793,980
`
`500
`
`BEGIN
`
`PLAY PART
`OF BLOCI,
`
`YES
`
`IS A
`BLOCK BEING
`PLAYED
`
`5/0
`
`520
`
`NO
`
`NO
`
`BLOCK
`IN INPUT
`QUEUE
`?
`
`530
`
`YES
`
`540
`
`DEQUEUE A
`BLOCK SO
`THAT IT GETS
`PLAYED
`
`FIG 5
`
`Petitioners' Exhibit 1018
`Page 0009
`
`

`

`U.S. Patent
`
`Aug. 11, 1998
`
`Sheet 8 of 17
`
`5,793,980
`
`BEGIN
`~--------------1D
`
`NO
`
`HAS
`HE SUBSCRIBE
`UESTED
`
`CALL UP
`REQUESTED
`CLIP
`
`.SEND
`INITIALIZATION
`DATA
`
`NO
`
`625
`
`HAS
`THE SERVER
`DETECTED A
`STOP MARKER
`
`YES
`
`630
`
`HAS
`HE SERVER
`RECEIVED AN
`ACKNOWLEDGE FROM
`THE SUBSCRIBER
`?
`
`640
`
`HAS
`SERVER RECEIVED
`A SEEi, SIGNAL
`
`SEND SEEh
`ACl,NOWLDGE
`MESSAGE
`
`GO TO ADDRESS
`REQUESTED BY
`SEEh. MESSAGE
`
`C
`
`®
`
`Petitioners' Exhibit 1018
`Page 0010
`
`

`

`U.S. Patent
`
`Aug. 11, 1998
`
`Sheet 9 of 17
`
`5,793,980
`
`YES
`
`HAS
`ERVER RECEIVED
`A PAUSE
`MESSAGE
`
`HAS
`SERVER RECEIVED
`A STOP
`MESSAGE
`
`YES
`
`HAS
`SERVER RECEIVED
`AN END
`MESSAGE
`
`YES
`> - - - - -~
`
`END
`
`TRANSMIT AUDIO
`DATA TO SUBSCRIBER
`
`650
`
`678
`
`680
`
`INCREMENT
`
`i
`
`HA
`M D
`NO
`- - - OCKS BE
`
`SEND TIME
`MESSAGE TO
`SUBSCRIBER
`
`RESET
`
`i
`
`690
`
`FIG 6B
`
`Petitioners' Exhibit 1018
`Page 0011
`
`

`

`U.S. Patent
`
`Aug. 11, 1998
`
`Sheet 10 of 17
`
`5,793,980
`
`BEGIN
`
`710
`
`LOAD DATA
`INTO BUFFER
`
`YES
`
`NO
`
`YES
`
`INDICATE
`TIME ON 1 - - - - - - - i
`DISPLAY
`
`FLUSH
`BUFFERS
`
`YES
`
`745
`
`,,,,,-- /-;-JO
`
`LENGT
`OR TABLE OF
`CONTENTS BLOCK
`
`~TS
`
`INDICATE LENGTH
`OR TABLE OF
`CONTENTS ON
`VISUAL DISPLA 'l
`
`755
`
`£-YES
`
`NO
`
`750
`
`NO
`
`RETURN
`
`FIG 7
`
`Petitioners' Exhibit 1018
`Page 0012
`
`

`

`8(7,·"J
`
`-
`- file
`@JD URL I file:
`
`[ RealAudio]
`Spyglass Mosaic -
`Windows
`~dit
`~ovigote
`!::!elp
`-
`
`/ / /CI/PN/DEMO.HTM
`
`Rondy extols the virtues of PN's audio-on-demand technology
`Commander Will Riker is sarcastic yet menacing
`Captain Picard odresses the Replicator
`
`I •
`
`I
`
`...
`...
`"'
`•
`
`i - - -
`
`-I
`Sound PlaybacJ..
`t Pause I I Stop 11
`! ~ewind I [Ploy:
`'-860 \5 It' '-82t'
`BJO
`]:#]
`1 . . 11 I
`J
`850~-· ~
`.~
`Length (eec): I 2497 Position (sec):~ l
`
`fforword
`
`,5-L-,..,
`
`/
`
`'
`!
`,- --S4G7
`
`,-·
`
`/
`
`b"'?f.}
`
`.•BP.
`7
`
`(
`
`-I
`
`BBC-- /
`
`Display
`-~--·--·-
`
`-·-·
`
`Remote Clips
`
`These clips are t;,eing kept on a PNA server on halcyon.cam
`accessed via the Internet, Both the WinSock and the server on
`be started before these clips can be retrieved.
`
`Dove Letterman and Dee-Dee exchange )'\JkS
`Notions of the World
`
`batter: Tino Martine;
`balls:
`0
`strikes:
`0
`outs:
`
`1
`
`Mariners
`
`0 8 1
`
`1 0
`
`Royols
`
`2 1 0
`
`0 1
`
`inning:
`
`t"p 6th
`R H E
`------ ::!
`------ 4
`
`4 8
`
`8
`
`1
`
`::--,~
`
`/\
`<
`
`'
`
`N
`
`Remote Clips
`
`Mariners vs, Royals
`
`This clip begins in the top of the 6th inning, A text display window
`is included. showing the game's progress.
`
`.. ;
`
`I
`j
`
`I
`
`!
`
`~ ...
`I•
`
`: Dismiss Display I
`
`FIC 84
`
`•
`
`~ • 00
`~ .....
`~ = .....
`
`> = ~ --"' ....
`
`~
`00
`
`~ a .... -~ ....
`
`-..,I
`
`Ol
`-..
`.....:a
`~ -..
`\0
`~ =
`
`Petitioners' Exhibit 1018
`Page 0013
`
`

`

`U.S. Patent
`
`Aug. 11, 1998
`
`Sheet 12 of 17
`
`5,793,980
`
`-
`--· ·~
`-
`
`---
`file
`
`Custom - C: \NETMANAG\ TCPIP,CFG
`~ervices
`!£,tertace
`t!_elp
`Disconnect
`~etup
`-
`'>
`--
`.... -
`~ R J..:;
`PPPO - COMI, 57600 baud
`
`Interface:
`
`'Y
`
`...
`...
`...
`
`---··
`
`Dial:
`
`3826245
`
`IP Address:
`
`198.137,231.20
`
`SLJbnet Mask:
`
`255.255.255,0
`
`Host Name:
`
`boutell
`
`Domain Name:
`
`NAME
`
`PPPO
`
`-
`
`-I
`I Stop
`
`TYPE
`PPP
`
`IP
`
`DOMAIN
`
`198.137,231.20
`
`--
`
`I 'Y A
`Custom - C: \NETMANAG\ TCPIP,CFG
`I I Save
`I I Print
`I I Copy
`I I Clear I I Close
`t -
`
`I
`
`•
`
`-
`
`--
`
`I ..
`
`login: boutell
`boutell
`Password:********
`
`•I I
`
`F/C, 8B
`
`Petitioners' Exhibit 1018
`Page 0014
`
`

`

`,..-,L .... /~--,
`
`/~---- t.2 {'
`RECORDED I DIGITIZED
`LIVE
`SOURCE
`SOURCE
`SOURCE
`_ __,_ _ _,__ __ '" _- 2 18 /
`A/D CONVERTOR
`
`.,-,. -,~
`
`I
`
`TIME
`
`TRANSMISSION LINE
`
`.o·c
`
`--SEND H.Q. DATA
`,
`
`~ - - - - - - - _ _ . / ' - - " - · · - - - - . , - · - ·~
`
`BUFFERS
`~'15
`[ill~ NIN IN IN IN IN IN IN IN IN IN IN IN IN IN IN IN IN IN IN IN IN IN IN~
`
`I
`L -
`
`I H
`
`l~~N~l~Nl~N~i-Nl~N~IN-i~N~IN_I_N_IN_I_N-IN_I_N_IN_i_N_IN_I_N_IN~I-N-IN_I_N_IN_I_N~IN~
`
`-- .,;;£()
`
`COMPRESSOR~•••---
`'------~-~225
`
`[ H
`
`I
`
`N m ..
`H
`N
`N
`N
`
`9,X'
`
`l.!ik7
`
`MEMORY
`
`I
`
`H
`H
`
`~
`
`H
`H
`
`SERVER
`
`.,,-920
`
`19.30
`
`SWITCH CONTROLLER
`
`\;~~D1O CONTROL CENTER
`
`FIG 9
`
`l -Id l•l•l•JNJNJNJNJNJNJNJNJNJNJNJNJNJNJNINJNJNJN]Nj--
`d
`
`-- SEND N DATA
`I H
`I
`
`• •
`•
`JN IN I• l•l•l•l •IN INl•l•l •I NI+ H--
`
`1
`
`[ill [ill --l
`Q
`[ffi [ffi --l
`Q
`[ill (El --l
`Q
`
`DATA PATH----
`
`• •
`.
`
`I
`
`I
`
`I
`
`1
`
`H
`
`I
`
`INININININ~
`I
`I
`INININININININ~
`I
`I
`INININININININININ~
`I
`I
`
`H
`
`H
`
`H
`
`H
`
`H
`
`H
`
`~ • rJl
`•
`~ = ~
`~ ~
`
`>
`~
`
`~
`~
`"'
`
`~ !
`
`~ a
`
`~ v-
`~
`~
`--.J
`
`01
`._.
`....:J
`~ ._.
`\0
`00
`0
`
`DECOMPRESSOR
`
`WAVE
`DRIVER
`
`B
`
`Petitioners' Exhibit 1018
`Page 0015
`
`

`

`AUDIO CONTROL CENTER
`AUDIO DATA
`
`/..---1005
`
`/
`
`SUBSCRIBER PC
`AUDIO BUFFERS
`
`/1.-"i..."li.,;
`
`~·
`
`,..,
`"--
`
`3
`
`4
`
`5
`
`6
`
`120'--,
`
`\
`
`I sw1TcH coNTROLLE1~ t-1 - . ----i /
`',1020 -i-\-
`
`:
`
`:
`
`i
`
`1-r - - -
`0
`
`l3'0 '-
`
`J 15---._ ,,
`
`1050
`~ - - - - - - , ,---i;- 1
`:
`:
`:_ _ _ _ .:
`
`_R_E_c_E...,.1v"TE_R_
`
`2
`
`3
`
`4
`
`5
`
`6
`
`,)
`
`1uJO
`
`JCC-'
`
`t----"1010
`
`/0, 70-~'-,
`
`. ..____
`
`'11{)
`
`SWITCH
`coN TROLLER
`
`I
`I
`
`\105'0
`
`METADATA
`
`METADATA BUFFERS
`
`FIG 10
`
`~
`00 •
`~ = ....
`~ = ....
`
`> = ~ --"' -!
`r.J') =-a -.,:.
`~ -.....
`
`01
`~ -...J
`
`~ ~ \0
`00 =
`
`Petitioners' Exhibit 1018
`Page 0016
`
`

`

`U.S. Patent
`
`Aug. 11, 1998
`
`Sheet 15 of 17
`
`5,793,980
`
`AUDIO
`CONTROL
`CENTER
`
`/130
`/
`i
`
`, '--
`1,./
`
`L,.---
`
`310
`
`.•
`
`l,......- 1..--- 1110
`
`AUDIO
`BUFFER
`I
`CPU
`I
`HIGH
`QUALITY
`BUFFER
`
`SUBSCRIBER PC
`
`FIC !!
`
`/L
`
`AUDIO
`CONTROL
`CENTER
`
`(130
`
`/-
`
`AUDIO
`BUFFER
`l
`
`CPU
`
`1----- 315
`i,,✓
`
`L-/ L--
`
`310
`
`I
`ADVANCE _,,,- 1..-- /.. '210
`BUFFER
`
`SUBSCRIBER PC
`
`Petitioners' Exhibit 1018
`Page 0017
`
`

`

`U.S. Patent
`
`Aug. 11, 1998
`
`Sheet 16 of 17
`
`5,793,980
`
`AUDIO SERVER
`
`240
`\
`
`!33t1
`
`-------
`
`DATA
`DATA
`DATA
`DATA
`DATA
`DATA
`DATA
`DATA
`DATA
`DATA
`DATA
`DATA
`DATA
`
`J.300
`
`J.320
`
`I DATA I DATA
`
`i
`
`DATA
`
`J .3 .30~
`
`- - - - TRANSMIT
`ACKNOWLEDGE
`
`SUBSCRIBER
`PC
`
`DATA
`DATA
`DATA
`DATA
`
`.315
`
`Petitioners' Exhibit 1018
`Page 0018
`
`

`

`U.S. Patent
`
`Aug. 11, 1998
`
`Sheet 17 of 17
`
`5,793,980
`
`-1 B'I TE
`
`I B\TE
`
`VARIABLE
`
`REQUEST I LENGTH I FILE NAME
`l4LY' 1 B\TE
`~
`I PAUSE
`I
`I BYTE ', ___ 14t14
`
`I
`
`1 BYTE
`
`BEGIN
`
`1 BYTE
`
`\
`
`-
`
`l4i)2
`
`FROM CLIENT
`TO SERVER
`
`I E\TRAS 01, I
`l4C6 _,, 1 BYTE
`
`I EXTRAS NO I
`
`4 BYTES
`
`\
`·-----14l78
`
`1 BYTE
`
`TIME
`
`-1 BYTE
`
`TIME DATA
`
`4 BYTES
`
`\
`
`· 1410
`
`FIG 14
`
`TIME DATA
`
`2 BYTES
`
`I
`--142C
`
`I 6 TIME + I
`6 TIME DATA
`~1425
`~ - -~ - - - - - - - -~
`1 BYTE
`2 BYTES
`
`~' _6_T_I M_E_-~l~_6_TI_M_E _D_A_T_A __ ~ ',...__ 14.30
`VARIABLE
`I BYTE
`I BYTE
`I LENGTH I AUDIO DATA
`1 BYTE
`
`1-------, 1435
`
`FROM
`SERVER
`TO
`CLIENT
`
`AUDIO
`
`1 BYTE
`I SEEK ACK
`1 BYTE
`
`' l44o 4 BYTES
`
`TIME DATA
`
`4 BYTES
`
`1 BYTE
`
`1 BYTE
`
`TIME DATA
`"'·
`''--1455
`
`4 BYTES
`
`ROWS
`
`I COLUMNS I
`
`·1 BYTE
`
`VARIABLE
`
`LENGTH
`
`1 BYTE
`
`SIZE
`
`·1 BYTE
`
`TEXT
`
`TIME DATA
`
`ROWS
`
`I TEXT DATA I
`
`',
`-'--- /460
`
`Petitioners' Exhibit 1018
`Page 0019
`
`

`

`5.793.980
`
`1
`AUDIO-ON-DEMAND COMMUNICATION
`SYSTEM
`
`BACKGROUND OF THE INVENTION
`
`2
`bilities would be needed. The system of the present inven(cid:173)
`tion overcomes these difficulties so that real-time audio-on(cid:173)
`demand is available to the average consumer on an
`unmodified personal computer.
`In order to overcome the aforementioned difficulties. the
`system of the present invention employs an audio compres(cid:173)
`sion algorithm which provides audio compression on the
`order of 22:1. As is well known in the art. audio data in
`digitized format requires large amounts of memory space. It
`10 has been found that. in order to transmit digitized audio data
`so that a high quality audio signal is generated in real time.
`a data rate on the order of 22 kilobytes per second is
`typically necessary. However. current data rates achievable
`by most average cost modems on a reliable basis. fall in the
`15 range of 1.8 kilobytes (14.4 kilobits) per second.
`Consequently. the real-time, audio-on-demand system of the
`present invention provides a form of audio compression
`which allows digitized audio data to be transmitted over a
`conventional 14.4 kilobits per second modem connection.
`20 For purposes of practical implementation. it is preferable to
`use less than the maximum possible modem bandwidth
`when transmitting data. It has been found that very good
`performance can be obtained if the data transmission rate is
`about 1 kilobyte per second. Assuming a required data rate
`25 of 22 kilobytes per second and a transmission bandwidth of
`approximately 1 kilobyte per second. an audio compression
`of approximately 22 to 1 is required. Audio compression
`algorithms which may be used in accordance with the
`teachings of the present invention to provide audio com-
`30 pression on the order of 22:1 are well known in the art. The
`EWfIA IS-54 standard. which is herein incorporated by
`reference, discloses an algorithm description such that one
`of ordinary skill in the art could implement a compression
`algorithm suitable for use in the present invention.
`35 Advantageously. a preferred embodiment of the algorithm
`employs an adaptation of the IS-54 VSELP cellular com(cid:173)
`pression algorithm compatible with the IS-54 VSELP cel(cid:173)
`lular compression algorithm availiable from MOTOROLA.
`Of course. it should be understood that in order to facilitate
`the compression and transmission of digitized audio data. it
`may be advantageous to convert the compression algorithm
`from hexadecimal to binary (i.e .. from ASCII data format to
`binary data format). Another preferred embodiment of the
`invention utilizes the code excited linear predication ( CELP)
`45 coder. version 3.2. available from NTIS. U.S. Department of
`Commerce. 5285 Port Royal Rd.. Springfield. Va .• 22161
`(telephone number 703-487-4650). Another preferred
`embodiment implements the well known GSM coding algo-
`rithm available through the European standards committee.
`Yet another preferred implementation uses a LPC-10 based
`coder described in a publication entitled "Digital Processing
`of Speech Signals," by L. R. Rabiner and R. W. Schafer,
`published by Prentice Hall. 1978. The aforementioned pub(cid:173)
`lic documents are herein incorporated by reference.
`Although the required data rates are achievable by means
`of the improved audio compression algorithm described
`above. certain difficulties are still inherent in a system which
`provides real time audio-on-demand without specialized
`software. Further difficulties are encountered in computer
`60 systems which run high power applications programs such
`as computer systems which run in a MICROSOFT WIN(cid:173)
`DOWS environment. Specifically. it is still necessary to
`decompress and translate the audio data received into a
`format compatible with WINDOWS. This poses particular
`65 problems since a WINDOWS environment typically
`requires a great deal of processing power so that much of a
`CPU's time is spent in supporting the WINDOWS software.
`
`5
`
`1. Field of the Invention
`The present invention relates to multimedia computer
`communication systems and, in particular. to communica(cid:173)
`tion systems which provide Audio-On-Demand services.
`2. Description of the Related Art
`In recent years. the computer industry has observed an
`increasing demand for versatility in the personal computer
`market. The average consumer is less interested in high
`computer performance such as increased memory and clock
`rates than in the everyday usefulness of a personal computer
`system. For example. parents may be interested in educa(cid:173)
`tional computer programs for their children which instruct
`using both visual and audio media. As a result, there has
`been an increasing demand for personal computers and
`computer networks which have multimedia capabilities.
`Among the most desirable multimedia capabilities are
`those associated with the transmission of audio information.
`A number of uses have been contemplated for transmission
`of audio information. For example, a user may want access
`to music or news. or may want to have a book read to them
`over their computer. Also. transmission of audio data pro(cid:173)
`vides much needed access to valuable information for visu(cid:173)
`ally impaired persons. Such multimedia communication
`systems which provide subscribers with selectable audio
`information are commonly called audio-on-demand sys(cid:173)
`tems.
`U.S. Pat. No. 5.132.992 issued to Yurt. et al .. discloses an
`audio and video transmission and receiving system. The
`audio and video-on-demand system disclosed by Yurt, et al.,
`distributes video and/or audio information to multiple sub(cid:173)
`scriber units from a central source material library. Digital
`signal processing is used to compress data within the source
`material library so that such data can be transmitted over
`standard communication links such as a cable or satellite
`broadcast channel. or a standard telephone line to a receiver
`specified by subscriber service. The receiver subscriber unit
`includes a decompressor for decompressing data sent from
`the source materials library and playing back the decom(cid:173)
`pressed data by means of an audio or visual display.
`Although known audio-on-demand communication sys(cid:173)
`tems offer many significant benefits, such systems are still
`subject to a number of significant limitations. For instance,
`significant difficulties are encountered when attempting to
`provide real time audio playback over narrowband commu- 50
`nication links such as a standard telephone line.
`
`40
`
`SUMMARY OF THE INVENTION
`
`The present invention provides a real-time, audio-on(cid:173)
`demand system which may be implemented using only the 55
`processing capabilities of the CPU within a conventional
`personal computer. As detailed above, a number of signifi(cid:173)
`cant difficulties arise when attempting to provide real-time
`audio-on-demand. It has been found that these difficulties
`are exacerbated when the subscriber receiving unit is a
`conventional personal computer having an Intel 486
`microprocessor. or processors of equivalent power. as a
`central processing unit. Of course. higher power processors
`could be used. but such systems would become prohibitively
`expensive and would not be available to the mainstream
`personal computer user. In order to compensate for lack of
`processing power, special hardware or other additional capa-
`
`Petitioners' Exhibit 1018
`Page 0020
`
`

`

`5,793.980
`
`3
`To overcome this difficulty, the system of the present inven(cid:173)
`tion continually monitors requests issued by application
`programs which run concurrently with the audio-on-demand
`system of the present invention. In this manner. requests
`issued by the applications programs are processed rather
`than ignored in the system of the present invention.
`Furthermore. data buffers of reasonable size should be
`allocated within the dynamic random access memory
`(DRAM) of a conventional 486 Intel based personal com(cid:173)
`puter in order to avoid deleterious effects on computer 10
`performance. Thus. typically, buffer memories are allocated
`within the DRAM to have on the order of approximately 16
`or 32 kilobytes of storage. If digitized audio data is trans(cid:173)
`mitted and received within the data buffer at too fast a rate,
`the buffers would overflow causing the loss of significant
`portions of data and audio dropout. As is well known in the 15
`art. audio dropout is a phenomena wherein audio playback
`terminates for some noticeable time period and then resumes
`after this delay. On the other hand. if data was transmitted
`too slowly. then the buffers would empty out again resulting
`in significant dropout and degradation of audio quality. 20
`Thus. a number of significant difficulties are encountered
`when attempting to implement a real time audio-on-demand
`system within a 486 CPU based personal computer system.
`or other similar personal computer systems. Thus. the
`present invention provides a method of monitoring and 25
`regulating the flow of data between the server and the
`subscriber unit which insures that the buffers are constantly
`maintained at or near maximum capacity.
`In a further aspect of the invention. audio quality degra(cid:173)
`dation may be compensated for through the data flow 30
`regulation of the present invention. This flow regulation
`constantly maintains the buffers at or near maximum capac-
`ity so that. in the event of a delay in the communication link.
`the subscriber unit can continue to play back audio already
`stored in the buffers until new audio data begins to arrive 35
`again, Also. the present invention employs a method of
`transmitting high quality audio data compressed using a
`lossless compression algorithm or a compression algorithm
`having a compression ratio which requires transmission at a
`rate greater than real time. at selected intervals so that brief 40
`passages of higher quality audio signals are produced at
`playback. In one embodiment. the user may select when a
`high quality passage is to be sent so that important pieces of
`audio data are played back clearly.
`In another aspect of the invention increased control over 45
`received audio data is provided for by transmitting selected
`significant portions of an audio clip being transmitted in
`anticipation that the user may desire to move immediately to
`a new position in the audio clip.
`In addition, versatility is added to the audio-on-demand 50
`system of the present invention by transmission of limited
`extra data. or "metadata." interleaved with the transmitted
`audio data. The metadata may include text, captions, still
`image data, high quality audio data. etc.. and includes
`information so as to allow the subscriber to synchronize the
`metadata with significant events in the audio data. The
`metadata is correlated with the audio data to provide a
`combined audio and visual experience.
`Furthermore. the present invention advantageously pro(cid:173)
`vides dynamic allocation of server/subscriber pairs to insure 60
`the best possible quality of communication links between
`the server and the subscriber.
`
`55
`
`4
`FIG. 2A is a more detailed schematic block diagram
`showing the main functional elements of the audio-on(cid:173)
`demand system of the present invention.
`FIGS. 2B-2D are schematic block diagrams showing the
`5 main functional elements of alternate embodiments of the
`net transports depicted in FIG. 2A.
`FIG. 3 is a schematic block diagram showing the main
`functional elements of a receiving subscriber audio unit such
`as a subscriber personal computer.
`FIGS. 4A and 4B together depict a control flow diagram
`showing the general method employed by the audio-on(cid:173)
`demand system of the present invention to provide real time
`audio decoding within the CPU of the receiver subscriber
`audio unit.
`FIG. 5 is a subcontrol flow diagram showing the general
`operation of the wave driver of FIG. 3.
`FIGS. 6A and 6B together depict the general flow of
`control employed within the audio server of the present
`invention.
`FIG. 7 depicts a control flow diagram which details the
`method employed within the read data subroutine block of
`FIG. 4B.
`FIG. SA depicts the various displays observed on the
`video screen of the subscriber personal computer as the user
`selects an audio clip to be played from a menu. and selects
`various options while the audio clip is being played
`FIG. 8B depicts the various displays observed on the
`video screen of the subscriber personal computer as the user
`dials the server. logs into the server system. and initiates a
`disconnect.
`FIG. 9 is a schematic representation of an exemplary data
`transaction between a server and a subscriber unit which
`illustrates method used in the high quality transmission
`mode of the present invention.
`FIG. 10 is a simplified block diagram which depicts the
`main functional elements of an audio-on-demand system
`that provides real-time playback of audio data in addition to
`metadata which can be displayed in synchronism with
`corresponding audio data.
`FIG. 11 is a simplified block diagram which depicts the
`main functional elements of an audio-on-demand system
`that provides audio playback of selected portions of high
`quality audio data in real-time.
`FIG. 12 is a simplified block diagram which depicts the
`main functional elements of an audio-on-demand system
`that provides a table of contents indicating significant divi(cid:173)
`sions within a requested audio clip. and which provides for
`immediate playback of audio data at the divisions specified
`in the table of contents.
`FIG. 13 is a schematic representation of the method used
`in accordance with the present invention to manage the flow
`of data blocks from the server to the subscriber PC.
`FIG. 14 illustrates the data structures of various data
`messages transmitted between the server and the subscriber
`PC in accordance with the teachings of the present inven(cid:173)
`tion.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`FIG. 1 shows a simplified schematic block diagram of an 65
`audio-on-demand system constructed in accordance with the
`present invention.
`
`DEfAII..ED DESCRIPTION OF THE
`PREFERRED EMBODIMENT
`FIG. 1 shows a simplified schematic block diagram of an
`"audio-on-demand" system constructed in accordance with
`the present invention. The system 100 comprises a sub(cid:173)
`scriber personal computer (PC) 110 ( e.g .. an IBM PC having
`a 486 Intel Microprocessor). having a video display 115. The
`
`Petitioners' Exhibit 1018
`Page 0021
`
`

`

`5.793.980
`
`5
`subscriber PC 110 connects to an audio control center 120
`over telephone lines 130 via a modem 140.
`In operation. a user calls the audio control center 120 by
`means of the modem 140. The audio control center 120
`transmits a menu of possible selections over the telephone
`lines 130 to the personal computer 110 for display on the
`video display 115. The user may then select one of the
`available options displayed on the video display 115 of the
`computer 110. For example. the user may opt to listen to a
`song or hear a book read. Once the audio data has been
`transmitted. the modem 140 disconnects from the audio
`control center 120.
`FIGS. 2A-2D and FIG. 3 are schematic block diagrams
`which show. in greater detail. the main functional elements
`of the audio-on-demand system 100 of the present invention
`which provides a real time audio-on-demand system in
`conjunction with the subscriber PC 110 which comprises a
`standard microprocessor based personal computer system.
`In the context of the present invention. the term "standard"
`personal computer system should be understood to mean
`that the system includes a microprocessor of equivalent or
`greater processing power than an INTEL 486 microproces(cid:173)
`sor ( although not necessarily compatible with an INTEL 486
`microprocessor). a random access memory (RAM). an inter(cid:173)
`nal or external modem which transmits data in the approxi(cid:173)
`mate range of 9.6 Kbps to 14.4 Kbps. and some kind of
`sound card or sound chip which serves as a digital-to-analog
`convertor. Such a system is advantageously capable of
`running MICROSOFf WINDOWS software. Of course, it
`should be understood that a "standard" personal computer
`system should not be simply understood to be an IBM
`compatible computer. In practice any kind of workstation or
`personal computing system ( e.g .. a SUN MICROSYSTEMS
`workstation, an APPLE computer. a laptop computer, etc.)
`which includes the above described features may be under(cid:173)
`stood to be broadly encompassed under the expression
`"standard" computer system.
`A more detailed block diagram of the audio-on-demand
`system 100 of the present invention is depicted in FIG. 2A.
`The audio control center 120 is shown in FIG. 2A to 40
`comprise a live audio source 210 and a recorded audio
`source 215. In one embodiment, the live audio source may
`simply comprise a person talldng into a microphone or some
`other source of live audio data like a baseball game, while
`the recorded audio source 215 may comprise a tape recorder, 45
`a compact disk. or any other source of recorded audio
`information. Both the live audio source 210 and the recorded
`audio source 215 serve as inputs to an analog-to-digital
`converter 220. The analog-to-digital converter 220 may, in
`one embodiment. comprise a Roland® RAP 10 analog-to-
`digital converter available with the Roland® audio produc(cid:173)
`tion card. The analog-to-digital converter 220 provides
`inputs to a digital compressor 225. Of course. it should be
`understood that some audio data input into the audio control
`center 120 may already be in digital form, as represented by 55
`a digitized audio source 218, and, therefore, may be input
`directl