United States Patent [19]
`Mattaway et al.
`
`[54] GRAPHIC USER INTERFACE FOR
`INTERNET TELEPHONY APPLICATION
`
`[75]
`
`Inventors: Shane D. Mattaway, Boca Raton;
`Glenn W. Hutton, Miami; Craig B.
`Strickland, Tamarac, all of Fla.
`
`[73] Assignee: NetSpeak Corporation, Boca Raton,
`Fla.
`
`[21] Appl. No.: 08/721,316
`
`[22] Filed:
`
`Sep. 25, 1996
`
`Related U.S. Application Data
`
`[63] Continuation-in-part of application No. 08/533,115, Sep. 25,
`1995
`[60] Provisional application No. 60/025,415, Sep. 4, 1996, and
`provisional application No. 60/024,251, Aug. 21, 1996.
`••.•••.••.••.••.•••••••••.••.••••••••••••••••••••••.•• G06F 17/00
`Int. CI.6
`[51]
`[52] U.S. CI. .............................................................. 709/227
`[58] Field of Search ......................... 395/200.57,200.58;
`709/227,228; 370/352, 355, 357, 389,
`395
`
`[56]
`
`5,095,480
`5,150,360
`5,166,931
`5,204,669
`5,224,095
`5,291,554
`5,309,433
`5,309,437
`5,321,813
`5,357,571
`5,400,335
`5,425,028
`5,430,709
`5,430,727
`5,432,846
`5,442,633
`5,452,296
`5,455,854
`5,457,683
`5,457,738
`
`References Cited
`u.s. PATENT DOCUMENTS
`3/1992 Fenner.
`9/1992 Perlman et al. .
`11/1992 Riddle.
`4/1993 Dorfe et aI ..
`6/1993 Woest et al. .
`3/1994 Morales.
`5/1994 Cidon et al. .
`5/1994 Perlman et al. .
`6/1994 McMillen et al. .
`10/1994 Banwart .
`3/1995 Yamada.
`6/1995 Britton et al. .
`7/1995 Galloway.
`7/1995 Calion.
`7/1995 Norio.
`8/1995 Perkins et al. .
`9/1995 Shimizu.
`10/1995 Dilts et al. .
`10/1995 Robins.
`10/1995 Sylvan.
`
`111111
`
`1111111111111111111111111111111111111111111111111111111111111
`US006009469A
`[11] Patent Number:
`[45] Date of Patent:
`
`6,009,469
`Dec. 28, 1999
`
`5,463,625 10/1995 Yasrebi.
`5,465,286 11/1995 Clare et al. .
`5,469,500 1111995 Satter et al. .
`5,475,741 12/1995 Davis et al. .
`5,479,411 12/1995 Klein.
`4/1996 Sestak et al. .
`5,509,058
`5/1996 Green.
`5,517,494
`6/1996 Danneels et al. .
`5,524,110
`6/1996 Braun et al. .
`5,524,141
`6/1996 Nilakantan et al. .
`5,526,489
`7/1996 Pinard et al. .
`5,533,110
`8/1996 Maroteaux et al. .
`5,544,303
`8/1996 Brockmeyer et al. .
`5,546,582
`
`FOREIGN PATENT DOCUMENTS
`
`A2 0445402 1111991 European Pat. Off ..
`8/1993 European Pat. Off ..
`A2 0556012
`WO 9219054 10/1992 WIPO.
`
`OTHER PUBLICAfIONS
`
`Internetworking with Tep/lp, vol. 1, Second Edition, Prin(cid:173)
`ciples, Protocols, and Architecture, by Douglas E. Comer.
`VocalTec Internet Phone (TM) Version 2.5, www.cox.sm(cid:173)
`u.edu/class/mis6386/people/stort/iphone25.exe, Feb. 1995.
`Weinberg, Netscape Conference and Cooltalk Meeting
`Room, \v\vw.q5.com, Feb. 22, 1996.
`Gull, Re: Getting IP address of PPP-connected Mac,
`<jgull-0304951005350001@pm012-11.dialip.mich.net>,
`Apr. 3, 1995.
`Internet
`Gull,
`Re:
`Mac?,
`for
`Phone
`>jgull-1704950116450001@pm049-28.dialip.mich.net>,
`Apr. 17, 1995.
`
`Primary Examiner-Ellis B. Ramirez
`Attorney, Agent, or Firm-Kudirka & lobse, LLP
`
`[57]
`
`ABSTRACT
`
`A communication utility for establishing real-time, point(cid:173)
`to-point communications between processes over a com(cid:173)
`puter network includes apparatus for querying a server as to
`the network protocol address of another client process, and
`apparatus for directly establishing a communication link
`with the client process upon receipt of the network protocol
`address from the server. In one embodiment, the utility
`includes a sophisticated user interface having features simi(cid:173)
`lar to typical telephony hardware but implementing greater
`flexibility with software.
`
`18 Claims, 27 Drawing Sheets
`
`,--~-~-------------
`
`INTERNET
`
`28
`
`:
`I
`
`:
`I
`I
`I
`
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`I
`
`L __________________ l
`
`I
`
`22
`
`SONY EXHIBIT 1009- Page 1
`
`

`
`"" = = 'C
`
`Q',
`
`'C
`
`"'"' Q',
`""
`
`'"'" N
`"'"" Q
`.....
`~ t!,)
`
`t!,)
`
`-...l
`
`o t!,) r
`
`'-C
`'-C
`"'""
`'-C
`",1:10
`N
`
`""'" ~ = ""'"
`
`~
`~
`•
`'J).
`
`o •
`
`PROCESSING J
`
`UNIT
`
`~
`
`/
`
`SECOND
`
`22\
`
`,
`I
`I
`,
`,
`I
`I
`I
`I
`I
`
`,
`
`I
`I
`I
`I
`I
`
`I
`
`L _________________ _
`
`"'-34
`
`DATABASE
`
`'-32
`
`TIMER I
`'-30
`
`....
`"""'-
`
`---.. PROCESSOR
`
`~
`
`CONNECTION SERVER
`
`r26
`
`.III.
`
`~ MAIL SERVER ......
`_ ....
`
`POP 3
`
`oj.
`
`I
`I
`
`I
`I
`I
`I
`I
`I
`I
`I
`I
`
`,
`,
`
`I
`I
`I
`
`r-----~------------~
`
`INTERNET -28
`
`24,
`
`FIG.]
`
`SONY EXHIBIT 1009- Page 2
`
`,--~-------------------,
`
`12
`
`_ ___ I
`I
`I
`I
`.... 1
`I
`...
`
`,
`
`I
`I
`I
`
`I ,
`
`I
`I
`I
`
`DEVICE
`OUTPUT
`
`'---
`
`MEMORY
`
`18-"
`
`4-"
`
`DEVICE
`INPUT
`
`t---
`
`PROCESSOR r-
`
`L_
`I
`I
`I
`I
`I
`I
`I
`: 1
`I
`I
`I
`I
`: FIRST PROCESSING UNIT
`
`

`
`I:: r24
`
`I
`I
`I
`1
`
`: INTERNET ---28 --I
`
`,-----
`
`IlL SERVER
`I
`I
`
`POP 3
`
`MA
`
`o t!,) r
`
`'-C
`'-C
`"'""
`'-C
`",1:10
`N
`
`~
`~
`•
`'J).
`
`""'" ~ = ""'"
`
`o •
`
`"" = = 'C
`
`Q',
`
`'C
`
`"'"' Q',
`""
`
`Q
`N
`.....
`~ t!,)
`
`t!,)
`
`-...l
`'""" N
`
`22
`
`PROCESSING
`
`SECOND
`
`UNIT
`
`3-5 SECONDS
`POLLS EVERY
`
`-------"
`I
`I
`I
`I
`1
`I
`1
`
`L_____
`I
`I
`I
`1
`I
`
`FIG. 2
`
`SONY EXHIBIT 1009- Page 3
`
`I
`{
`
`I
`: <CONNECTREQUEST> :
`
`_____
`I
`I
`I <CONNECTOK>
`I
`I
`
`___
`
`_________
`
`___
`
`MEMORY
`
`L. _ _ _ _ _
`1 16
`1
`1
`:
`
`I
`I
`I
`I
`I
`I
`I
`-------------------,
`
`DEVICE
`
`18
`
`INPUT
`
`PROCESSOR
`
`12 ,--~
`
`1
`1
`I
`I 14
`I
`1
`I
`1
`:
`
`

`
`u.s. Patent
`
`Dec. 28, 1999
`
`Sheet 3 of 27
`
`6,009,469
`
`N
`N
`
`~ (9
`
`o~
`Z(I)I-
`0(1)-
`U WZ
`WU::>
`(1)0
`CC
`0...
`
`N
`N
`
`'V
`N
`
`,----
`I ,
`
`I
`I
`
`~ A
`« u
`
`~
`-!
`
`V
`
`l ____
`
`----,
`I
`I
`I
`I
`t- I
`~
`WI
`ZI
`0::1 ~
`WI
`t-I ~
`Z
`-
`
`I
`I
`I
`I
`____ I
`
`N ....-
`
`N ....-
`
`,
`
`;
`
`I---=r-- ----
`I ~
`I
`(9
`I
`I
`0
`I OA
`I wO
`(1)-
`(1)0
`I
`I W=>
`I 0::«
`a.
`'r
`I
`I ~
`, U
`0
`1
`l __ V __ '----
`
`l-
`W
`Z
`0::
`W
`l-
`-
`Z
`
`"
`
`~
`
`(9
`Z
`t-(I)t-
`(1)(1)-
`O:::W Z
`u::U::>
`0
`0...
`
`SONY EXHIBIT 1009- Page 4
`
`

`
`u.s. Patent
`
`Dec. 28, 1999
`
`Sheet 4 of 27
`
`6,009,469
`
`r 36
`
`40
`
`38
`
`44
`
`50
`
`48
`
`X
`
`ITC
`
`I I
`I I
`I I
`
`VOL II
`MIC II
`WAV II
`D
`
`FIG. 5
`
`52
`
`r 36
`
`3R
`
`SHANE
`195.201.00.137
`13
`
`• IDLE
`ITJ00~
`8] [I] [I] ~
`0[I]~ L3
`~~~
`jSPDljMSGl1 DIR I
`I RCLIIENDIISNDI
`I FWD II MUTI HLD
`9 52
`
`48
`
`50
`
`X
`
`ITC
`
`FIG. 6
`
`SONY EXHIBIT 1009- Page 5
`
`

`
`u.s. Patent
`
`Dec. 28, 1999
`
`Sheet 5 of 27
`
`6,009,469
`
`START THE POINT-TO-POINT
`INTERNET PROTOCOLS
`
`INITIATE PRIMARY INTERNET
`PROTOCOL
`
`54
`
`56
`
`58
`
`60
`
`IS THE CONNECTION SERVER IN A
`RESPONSIVE CONDITION?
`
`NO
`
`62
`
`PERFORM PRIMARY
`INTERNET PROTOCOL
`
`INITIATE SECONDARY
`INTERNT PROTOCOL
`
`FIG. 7
`
`START THE PRIMARY
`POINT-TO-POINT INTERNET
`PROTOCOL
`
`TIMESTAMP AND STORE E-MAIL
`ADRESSES AND IP ADRESSES OF
`LOGGED-IN UNITS IN A DATABASE
`
`RECEIVE QUERY FROM FIRST UNIT
`WHETHER A SPECIFIED SECOND
`UNIT IS LOGGED-IN
`
`RETRIEVE IP ADDRESS FROM
`DATABASE IF THE SECOND UNIT IS
`LOGGED-IN
`
`SEND RETRIEVED IP ADRESS TO
`FIRST UNIT TO ESTABLISH POINT-
`TO-POINT CONNECTION
`
`FIG. 8
`
`64
`
`66
`
`68
`
`70
`
`72
`
`SONY EXHIBIT 1009- Page 6
`
`

`
`u.s. Patent
`
`Dec. 28, 1999
`
`Sheet 6 of 27
`
`START THE SECONDARY
`POINT-TO POINT INTERNET PROTOCOL
`
`6,009,469
`
`74
`
`V
`
`~
`GENERATE AN E-MAIL SIGNAL, V
`
`INCLUDING A SESSION NUMBER AND A
`FIRST IP ADRESS CORRESPONDING TO A
`FIRST PROCESSING UNIT
`
`76
`
`i
`
`TRANSMIT THE E-MAIL SIGNAL AS A
`<CONNECTREQUEST> SIGNAL
`TO THE INTERNET
`
`78
`
`80
`
`V
`+
`DELIVER THE E-MAIL SIGNAL THROUGH V
`
`THE INTERNET USING A MAIL SERVER TO
`A SECOND PROCESSING UNIT
`
`~
`EXTRACT THE SESSION NUMBER AND V
`
`THE FIRST IP ADRESS FROM THE E-MAIL
`SIGNAL
`
`82
`
`~
`SEND THE SESSION NUMBER AND A V
`
`SECOND IP ADRESS CORRESPONDING
`TO THE SECOND PROCESSING UNIT TO
`THE FIRST PROCESSING UNIT THROUGH
`THE INTERNET
`
`84
`
`~
`
`VERIFY THE SESSION NUMBER
`RECEIVED FROM THE SECOND
`PROCESSING UNIT
`
`86
`
`V
`
`~
`ESTABLISH A POINT-TO-POINT INTERNET V
`
`COMMUNICATION LINK BETWEEN THE
`FIRST AND SECOND PROCESSING UNITS
`USING THE FIRST AND SECOND IP
`ADDRESSES
`
`88
`
`FIG. 9
`
`SONY EXHIBIT 1009- Page 7
`
`

`
`"" = = 'C
`
`Q',
`
`'C
`
`"'"' Q',
`""
`
`-..J
`'""" N
`
`Q
`-..J
`.....
`CI'1 =- t!,)
`
`t!,)
`
`" 1000
`
`Figure 10
`
`1022
`
`WP2
`
`1012
`
`WP1
`
`'-__ ," .,,1018
`
`1014 ..... f. __ ~
`
`SONY EXHIBIT 1009- Page 8
`
`'-C
`'-C
`"'""
`'-C
`",1:10
`N
`t!,) r
`0
`
`""'" ~ = ""'"
`
`~
`~
`•
`'J).
`0 •
`
`1024
`
`IV~I
`
`1022~~ I 1020
`
`Global Server
`
`NetSpeak
`
`1026 '(iSP)
`
`Provider
`Service
`Internal
`
`

`
`"" = = 'C
`
`Q',
`
`'C
`
`"'"' Q',
`""
`
`'"'" N
`
`-...l
`
`Q
`IX:;
`
`.....
`~ t!,)
`
`t!,)
`
`'-C
`'-C
`'""'"
`'-C
`
`o t!,) r
`
`",IX:;
`N
`
`""'" ~ = ""'"
`
`~
`~
`•
`'J).
`
`o •
`
`rl12
`o
`
`CONTROLLER
`
`DISK ;.
`
`,150
`
`FIXED DISK DRIVE
`,.
`r 151
`~
`
`....
`
`T r 146 t::
`8
`c
`
`..
`
`r141
`
`r----
`
`0
`
`r100
`
`CONTROLLER
`~
`,145
`
`~
`,140
`
`I ROM I CONTROLLER
`
`RAM
`
`r115
`
`SYSTEM
`~
`,110
`
`CD ROM DRIVE
`
`DISKETIE DRIVE
`
`CONTROLLER
`INTERRUPT
`
`r135
`
`r--
`
`CONTROLLER
`
`BUS
`
`~125
`
`CONTROLLER
`
`MEMORY
`
`120
`
`,..
`
`197
`
`CONTROLLER
`..-
`
`AUDIO
`
`~
`
`NETWORK
`
`~~ It'
`
`19C
`
`A~ L
`
`ADAPTE~
`
`COMMUNICATION
`
`170
`
`191
`
`C
`
`155
`CONTROLLER ~
`
`CONTROLLER
`
`VIDEO
`
`.~
`
`CONTROLLER
`" DMA
`
`'\
`
`MOUSE
`
`KEYBOARD AND
`
`165~
`
`160~
`
`BUS
`
`195
`
`Figure 11 (PRIOR ART)
`
`157
`
`Itt
`
`V.
`I
`/ .... · .. ·· ...... :1 ~~
`
`II •• II
`
`II •• II II •
`
`II
`
`II
`
`,. II •
`
`•
`
`.. Il10..............
`
`'}f
`
`105
`
`CPU
`
`,.
`
`SONY EXHIBIT 1009- Page 9
`
`

`
`u.s. Patent
`
`Dec. 28, 1999
`
`Sheet 9 of 27
`
`6,009,469
`
`:>
`
`I
`
`~ 1214
`
`/1212
`
`AID
`
`/1216
`INPUT
`BUFFER
`
`/1222
`
`...
`
`DSP
`
`offL
`
`/1224
`
`ROM
`
`V
`BUS
`INTERFACE
`
`l/1228
`
`/1218
`
`D/A
`
`..
`
`/1220
`.. OUTPUT
`BUFFER
`
`r
`
`/1226
`
`RAM
`
`...
`
`~1210
`
`...
`
`...
`
`...
`
`..
`
`..
`
`~~
`
`FIGURE 12 (PRIOR ART)
`
`~ 1230
`0>
`
`SONY EXHIBIT 1009- Page 10
`
`

`
`"" = =
`
`Q',
`
`"'"' Q',
`""
`'C
`
`'C
`
`'""" = c
`.....
`rJ'l =(cid:173)
`
`('t)
`('t)
`
`.....:t
`N
`~
`
`'"""
`",C/O
`N
`!"l
`('t)
`~
`
`I,C)
`I,C)
`I,C)
`
`~
`~
`•
`'J).
`
`""'" ~ = ""'"
`
`o •
`
`Figure 13 A
`
`Windows
`UNIX
`Macintosh
`
`~1324
`•
`
`..
`
`SONY EXHIBIT 1009- Page 11
`
`, pxxx.cdc
`
`sp.cdc
`
`lJ
`
`\1320
`
`--.. wpgsm.cdc
`
`L.."
`plug & play
`
`codecs
`
`Windows gui
`X-Windows 9
`Macintosh g.
`
`.,.........1316
`V1314
`
`V1312
`
`I-
`
`....
`
`~
`
`Mel API
`
`am
`
`me
`
`322 .... sockets
`
`~26 "-wapi
`
`em
`
`ui
`
`gui
`
`~
`
`1300
`
`db
`
`1318
`
`1 110,
`
`

`
`u.s. Patent
`
`Dec. 28, 1999
`
`Sheet 11 of 27
`
`6,009,469
`
`WAPI
`
`1328
`
`1326
`
`1322
`
`1336
`
`1338
`
`1332
`
`comm
`state
`machine
`
`WAPI
`
`sockets
`
`1334
`
`RTP
`TCP ~----------~
`UDP
`
`IP
`
`1330
`
`network
`
`FIGURE 13 B
`
`SONY EXHIBIT 1009- Page 12
`
`

`
`u.s. Patent
`
`Dec. 28, 1999
`
`Sheet 12 of 27
`
`6,009,469
`
`1400
`
`14021 f
`
`1
`
`1402A ---...-+--L....~ Roger WileD
`14028
`rwilco@isp.com
`
`00
`3/6
`
`1406
`
`1408·
`
`""
`
`I /'"1402
`vV
`./
`TSP--l-........---1402E
`01:31
`1402D
`1402G
`1402H
`
`1404
`
`1402C--t-~10:31
`Q L1 TALK
`1402F
`1402J ,.
`[2J00~ "
`000~
`000~
`D00~
`§]~~
`~ IMUTI ~
`0~§]
`V
`
`Figure 14
`
`SONY EXHIBIT 1009- Page 13
`
`

`
`u.s. Patent
`
`Dec. 28, 1999
`
`Sheet 13 of 27
`
`6,009,469
`
`1500 .f
`
`1512
`
`CONNECTION
`SERVER
`
`DATABASE
`SEARCH
`ENGINE
`
`1516F
`
`DATABASE
`
`1516
`
`1518
`
`INFORMATION ..-_ __ __ Pj
`SERVER
`
`DATABASE
`SERVER
`
`1514
`
`1508
`NETWORK INTERFACE CARD
`
`1510
`
`1522
`
`1524
`
`1526
`
`Figure 15A
`
`SONY EXHIBIT 1009- Page 14
`
`

`
`u.s. Patent
`
`Dec. 28, 1999
`
`Sheet 14 of 27
`
`6,009,469
`
`r-------------------------------l
`I
`I
`I
`I
`I I
`I
`r .....
`I
`I
`I
`I
`I
`I
`I
`
`I
`I
`I
`I
`I
`I
`I
`
`.... 1500
`
`:
`I
`I
`
`DATABASE
`
`1516F
`
`DATABASE
`~AA~
`ENGINE
`
`1512
`
`CONNECTION
`SERVER
`
`I
`I
`I
`I
`I
`I
`I
`L _ _ __ _ __ _ __ _
`
`INFORMATION
`SERVER
`
`~SP 1~2 _~
`
`~ 1530
`
`1536
`
`:
`I
`f
`
`I
`I
`I
`I
`I
`I
`I
`
`1516
`
`1518
`
`DATABASE
`SERVER
`
`1514
`
`ROUTER
`
`1522
`
`1524
`
`1526
`
`Figure 158
`
`SONY EXHIBIT 1009- Page 15
`
`

`
`"" = = 'C
`
`Q',
`
`'C
`
`"'"' Q',
`""
`
`Figure 15C
`
`.....:t
`N
`~
`c
`'""" Ul
`.....
`rJ'l =- ('t)
`
`('t)
`
`'"""
`",C/O
`N
`('t) r
`~
`
`I,C)
`I,C)
`I,C)
`
`1530
`
`Internet )-
`
`~
`
`1528
`
`/
`
`/
`
`\
`
`-
`
`'-
`
`""
`
`Global Server
`
`NetSpeak
`
`f. .. .:'\ /1546
`
`1544
`
`SONY EXHIBIT 1009- Page 16
`
`""'" ~ = ""'"
`
`~
`~
`•
`'J).
`
`o •
`
`I /15000
`
`I -"' 1500C
`
`• 15008 I
`
`Global Server
`
`NetSpeak
`
`Global Server
`
`NetSpeak
`
`Global Server
`
`NetSpeak
`
`

`
`"" = = 'C
`
`Q',
`
`'C
`
`"'"' Q',
`""
`
`.....:t
`N
`~
`c
`'""" ~
`.....
`rJ'l =(cid:173)
`
`('t)
`('t)
`
`'"""
`",C/O
`N
`!"l
`('t)
`~
`
`I,C)
`I,C)
`I,C)
`
`""'" ~ = ""'"
`
`~
`~
`•
`'J).
`
`o •
`
`Figure 16A
`
`NO
`
`1620
`
`1618
`
`<CONNECT ACK>
`
`SEND
`
`r«.>
`
`1614
`
`1612
`
`ADDRESS TO DATABASE
`
`FORWARD E·MAIL
`
`EXTRACT E-MAIL
`
`ADDRESS
`
`1622
`
`<OFF LINE>
`MESSAGE OR
`SEND ERROR
`
`1615
`
`DATA
`
`UNRECOGNIZED
`
`DISREGARD
`
`SONY EXHIBIT 1009- Page 17
`
`

`
`u.s. Patent
`
`Dec. 28, 1999
`
`Sheet 17 of 27
`
`6,009,469
`
`START
`
`1630
`NO
`
`DISGARD
`UNRECOGNIZED
`DATA
`
`1635
`
`EXTRACT SEARCH
`CRITERIA
`
`SUPPLY SEARCH
`CRITERIA TO DATABASE
`
`SEND
`<INFO ACK>
`
`1634
`
`1636
`
`1638
`
`YES
`
`NO
`
`YES
`
`Figure 168
`
`SONY EXHIBIT 1009- Page 18
`
`

`
`"" = =
`
`Q',
`
`"'"' Q',
`""
`'C
`
`'C
`
`Figure 17A
`
`110 <CALLER ACK>
`
`11C <CALLER OK>
`
`.....:t
`N
`~
`C
`'""" C/O
`.....
`rJ'l =(cid:173)
`
`('t)
`('t)
`
`'"""
`",C/O
`N
`!"l
`('t)
`~
`
`I,C)
`I,C)
`I,C)
`
`""'" ~ = ""'"
`• ;p
`o •
`
`'J).
`
`.4~
`
`E
`N
`0
`H
`P
`B
`E
`W
`
`.,..
`...
`.,..
`...
`,.
`...
`
`...
`
`".
`
`I
`
`8
`
`12 <END>
`
`11B <AUDIO>
`
`11A <AUDIO>
`
`10 <ANSWER>
`
`THEN
`
`90 <CALL ACK>
`
`OR
`
`....
`L..I
`
`......
`
`~
`
`....
`... -
`
`"'-9C <ANSWER MACH>
`
`OR
`
`9B <BUSY>
`
`OR
`
`9A <REJECT>
`
`8 <CALL>
`
`....
`
`...
`....
`
`E
`N
`0
`H
`p
`B
`E
`W
`
`I
`
`1536
`
`-...
`
`-....
`•
`
`...
`...
`...
`
`"lI"'"
`
`-...
`...
`
`7B <OFF LINE>
`
`7 A <CONNECT ACK>
`
`6 <CONNECT REQ>
`
`5 <ONLINE ACK>
`
`4 < REGISTRATION>
`
`3 <USER INFO>
`
`2 <USER INFO REQ>
`
`1 <ONLINE REQ>
`
`...
`....
`
`...
`t...
`
`...
`t...
`
`~~
`
`R
`E
`V
`R
`E
`S
`
`L
`A
`B
`0
`L
`G
`
`1500
`
`I
`
`SONY EXHIBIT 1009- Page 19
`
`

`
`u.s. Patent
`
`Dec. 28, 1999
`
`Sheet 19 of 27
`
`6,009,469
`
`3:wmc.xozw
`
`A
`I-
`A ~
`a:
`0
`w
`1\
`1\
`1\
`0
`0
`0
`0
`0
`a: <
`11. m
`11.
`11.
`-
`-
`z ~6
`z
`z
`0
`0
`V
`- -
`11.
`11.
`V
`V
`u.
`Z
`Z
`< m
`V ~ ~ ~ -
`z
`(.)
`V
`V
`~ N
`~
`
`~
`..--
`e
`
`:::J
`.~
`La:
`
`."
`
`0
`0
`It)
`~ (!l..Jom<..J
`
`"
`
`., ..
`
`(l)Wa:>Wa:
`
`SONY EXHIBIT 1009- Page 20
`
`

`
`"" = = 'C
`
`Q',
`
`'C
`
`"'"' Q',
`""
`
`N = C
`.....
`rJ'l =(cid:173)
`
`('t)
`('t)
`
`.....:t
`N
`~
`
`Figure 18A
`
`1806
`
`.
`
`1804
`
`1802
`
`UICollection* Parent
`UIVirtualObject* Children 0
`
`UICollection* Parent
`GUIControl* Sibling
`
`UIControl* Sibling
`long Value
`
`UIColiection
`
`I
`-f-is a
`
`I
`
`UIControl
`I
`
`GUIControl
`
`r
`
`-f-is a
`
`SONY EXHIBIT 1009- Page 21
`
`'"""
`",C/O
`N
`!"l
`('t)
`~
`
`I,C)
`I,C)
`I,C)
`
`virtual boot ValuedateNewValue(long NewValue)
`
`virtual long GetValue()
`virtual boot SetValue(long NewValue)
`
`UIVirtualObject
`
`UIVirtualControl
`
`""'" ~ = ""'"
`
`~
`~
`•
`'J).
`
`o •
`
`-r-is a
`
`/1812
`
`short Type
`short 10
`
`UIVirtualBase
`
`

`
`u.s. Patent
`
`Dec. 28, 1999
`
`Sheet 21 of 27
`
`6,009,469
`
`L 1804
`UIControl
`
`GUIControl* Sibling
`UIColiection* Parent
`
`1828
`
`UIButton
`
`+isa
`
`/1826
`
`UISlider
`
`1824
`
`)
`UI Scroller
`
`UlTab
`
`UIDisplay
`
`UI ListBox
`
`1822
`
`UIComboBox
`
`\1818
`
`UIEditBox
`
`1820
`
`\ 1814
`
`\ 1816
`
`Figure 18B
`
`SONY EXHIBIT 1009- Page 22
`
`

`
`u.s. Patent
`
`Dec. 28, 1999
`
`Sheet 22 of 27
`
`6,009,469
`
`UIButton
`
`Figure 18C
`
`........... 1848
`
`I
`UIPushButton
`
`- -is a
`
`UIPlayRun
`
`I
`UIToggle
`
`\1842
`
`\ 1844
`
`\ 1846
`
`UIColiection
`
`GUIVirtualObject* ChiidrenD
`UIColiection* Parent
`
`Figure 180
`
`+isa
`
`UICall
`
`UILine
`
`.......... 1806
`
`1
`UIPopUp
`
`void Show()
`void HideO
`
`\1836
`
`V 1834
`
`I
`UIPhone
`
`1832 .........
`
`SONY EXHIBIT 1009- Page 23
`
`

`
`"" = =
`
`Q',
`
`"'"' Q',
`""
`'C
`
`'C
`
`.....:t
`N
`~
`c
`~
`.....
`rJ'l =(cid:173)
`
`('t)
`('t)
`
`'"""
`",C/O
`N
`!"l
`('t)
`~
`
`I,C)
`I,C)
`I,C)
`
`~
`~
`•
`'J).
`
`""'" ~ = ""'"
`
`o •
`
`*tasks
`
`Figure 19C
`
`jobs
`
`*parties
`
`Figure 198
`
`*job
`
`lines
`
`job queue
`
`line array
`
`/
`
`~-~---
`
`Figure 19A
`
`v
`
`,---~-
`
`SONY EXHIBIT 1009- Page 24
`
`AE_FILE
`AE_SOCKET
`AE_ITCSPKR
`AE_SPKR
`AE_ITCMIC
`AE_MIC
`
`src/dst
`
`AE_USEME
`AE_FlllME
`AE_STOP
`AE_START
`AE_ClOSE
`AE_INIT
`
`cmd
`
`fileSize
`filelen
`fileType
`fileHandle
`
`... task()
`taskO
`
`flags
`spkr
`mic
`
`extend
`
`*buf
`*job
`state
`dst
`src
`cmd
`
`task
`
`0
`
`RemoveTaskO
`CreateTeskO
`RemovePartyO
`
`AddPartyO
`
`... jobO
`jobO
`
`LoadPartyO
`
`-party()
`part yO
`
`partyRec
`socket
`session
`
`state
`
`*prevJob
`*nextjob
`*outTask
`*inTask
`*party[ ]
`nparties
`
`state
`type
`
`id
`
`\l
`
`party
`
`~
`
`job
`
`RemoveCaliO
`CreateCall()
`
`duration
`
`state
`
`*job
`
`\1)
`
`line
`
`

`
`u.s. Patent
`
`Dec. 28, 1999
`
`Sheet 24 of 27
`
`6,009,469
`
`SOCKET
`
`2002
`
`2006
`
`ACCUMULATE
`INTO BUFFER
`
`PLACE EMPTY
`BUFFER ON
`MIC SAMPLING
`QUEUE
`
`2034
`
`PLACE EMPTY
`BUFFER ON
`WinSock QUEUE
`
`PLACE EMPTY
`BUFFER IN
`"free" POOL
`
`2036
`
`2038
`
`Figure20A
`
`SONY EXHIBIT 1009- Page 25
`
`

`
`'"""
`",C/O
`N
`!"l
`('t)
`~
`
`I,C)
`I,C)
`I,C)
`
`""'" ~ = ""'"
`
`~
`~
`•
`'J).
`
`o •
`
`SONY EXHIBIT 1009- Page 26
`
`return
`
`from WinSock
`
`Async read
`
`from II poo III
`Obtain msg
`
`2045
`
`v
`... (f\
`
`file
`
`~
`
`mic
`
`

`
`"" = = 'C
`
`Q',
`
`'C
`
`"'"' Q',
`""
`
`.....:t
`N
`~
`c
`~
`N
`.....
`rJ'l =- ('t)
`
`('t)
`
`'"""
`",C/O
`N
`!"l
`('t)
`~
`
`I,C)
`I,C)
`I,C)
`
`""'" ~ = ""'"
`
`~
`~
`•
`'J).
`0 •
`
`Figure20C
`
`return
`
`speaker
`
`2068
`
`~( return ')
`
`free II pool I!
`in Mel msg
`
`place msg buffer I
`
`mixing
`
`buf
`
`........
`
`socket .-
`
`NoT /2064
`
`•
`
`(l
`
`from file
`rea~
`
`2066
`
`.( return
`
`•
`
`I
`
`II
`
`../ ..J __ ....... ~ socket I pla.ce msg buffer
`
`free II pool II
`In WinSock
`
`other
`
`2058
`
`START "'\
`
`SONY EXHIBIT 1009- Page 27
`
`

`
`u.s. Patent
`
`Dec. 28, 1999
`
`Sheet 27 of 27
`
`6,009,469
`
`START
`
`mic
`
`file
`
`Cancel pending
`async read
`
`2078
`
`2080
`set src = NONE
`
`Figure20D
`
`Cancel sampling
`from mic
`
`Discard pending
`data in mix
`buffer
`
`SONY EXHIBIT 1009- Page 28
`
`

`
`6,009,469
`
`1
`GRAPHIC USER INTERFACE FOR
`INTERNET TELEPHONY APPLICATION
`
`RELATED APPLICATIONS
`
`2
`E-Mail Online ... MULTIMEDIAWORLD.VOL.2.NO. 9,
`Aug. 1995, p. 52. Using such Voice E-Mail software, a user
`may create an audio message to be sent to a predetermined
`E-mail address specified by the user.
`Generally, devices interfacing to the Internet and other
`online services may communicate with each other upon
`establishing respective device addresses. One type of device
`address is the Internet Protocol (IP) address, which acts as
`a pointer to the device associated with the IP address. A
`10 typical device may have a Serial Line Internet Protocol or
`Point-to-Point Protocol (SLIP/PPP) account with a perma(cid:173)
`nent IP address for receiving E-mail, voice mail, and the like
`over the Internet. E-mail and voicemail is generally intended
`to convey text, audio, etc., with any routing information
`such as an IP address and routing headers generally being
`15 considered an artifact of the communication, or even gib(cid:173)
`berish to the recipient.
`Devices such as a host computer or server of a company
`may include multiple modems for connection of users to the
`Internet, with a temporary IP address allocated to each user.
`20 For example, the host computer may have a general IP
`address "XXX.XXX.XXX," and each user may be allocated
`a successive IP address of XXX.XXX.XXX.10,
`XXX.XXX.XXX. 11 , XXX.XXX.XXX.12, ttl'. Such [tm(cid:173)
`porary IP addresses may be reassigned or recycled to the
`25 users, for example, as each user is successively connected to
`an outside party. For example, a host computer of a company
`may support a maximum of 254 IP addresses which are
`pooled and shared between devices connected to the host
`computer.
`Permanent IP addresses of users and devices accessing the
`Internet readily support point-to-point communications of
`voice and video signals over the Internet. For example,
`real-time video teleconferencing has been implemented
`using dedicated IP addresses and mechanisms known as
`reflectors. Due to the dynamic nature of temporary IP
`addresses of some devices accessing the Internet, point-to-
`point communications in real-time of voice and video have
`been generally difficult to attain.
`The ability to locate users having temporary or dynami(cid:173)
`cally assigned Internet Protocol address has been difficult
`without the user manually initiating the communication.
`Accordingly, spontaneous, real-time communications with
`such users over computer networks have been impractical.
`Further, it is desirable to have a communication utility which
`contains familiar features and functions to current commu-
`nication utility such as telephones and cellular telephones. It
`is even further desirable to utilize the current graphic user
`interface technology associated with computer software in a
`manner to achieve a more flexible interface to a such a
`communication utility, without the limitations associated
`50 with hardware.
`Accordingly, a need exists for a way to determine whether
`computer users are actively connected to a computer net(cid:173)
`work.
`A further need exists for a way to obtain the dynamically
`55 assigned Internet Protocol address of a user having on-line
`status with respect to a computer network, particularly the
`Internet.
`An even further need exists for a method and apparatus by
`which to establish real-time, point-to-point communications
`60 over a computer network using a communication utility
`having an interface which combines the familiar aspects of
`current hardware communication utilities but which allows
`for the flexibility associated with graphic user interfaces.
`SUMMARY OF THE INVENTION
`The above deficiencies in the prior art and previously
`described needs are fulfilled by the present invention which
`
`5
`
`This application is a continuation-in-part of U.S. patent
`application Ser. No. 08/533,115 (Pending) entitled Point-to(cid:173)
`Point Internet Protocol, by Glenn W. Hutton, filed Sep. 25,
`1995, commonly assigned, the subject matter of which is
`incorporated herein by reference.
`To the extent that any matter contained herein is not
`already disclosed in the above-identified parent application
`a location claims priority to U.S. provisional patent appli(cid:173)
`cation 601025,415 entitled Internet Telephony Apparatus and
`Method by Mattaway et aI., filed Sep. 4, 1996, and U.S.
`provisional patent application Ser. No. 60/024,251 entitled
`System and Methods for Point-To-Point Communications
`Over a Computer Network, by Mattaway et a1., filed Aug.
`21,1996.
`In addition, this application is one of a number of related
`applications filed on an even date herewith and commonly
`assigned, the subject matters of which are incorporated
`herein by reference, including the following:
`U.S. patent application Ser. No. 08/719,894, entitled
`Directory Server For Providing Dynamically Assigned
`Network Protocol Addresses, by Mattaway et a1.;
`U.S. patent application Ser. No. 08/719,554, entitled
`Point-to-point Computer Network Communication
`Utility Utilizing Dynamically Assigned Network Pro-
`tocol Addresses, by Mattaway et a1.;
`U.S. patent application Ser. No. 08/719,640, entitled
`Method And Apparatus For Dynamically Defining Data
`Communication Utilities, by Mattaway et a1.;
`U.S. patent application Ser. No. 08/719,891, entitled 35
`Method And Apparatus For Distribution And Presen(cid:173)
`tation Of Multimedia Data Over A Computer Network,
`by Mattaway et a1.;
`U.S. patent application Ser. No. 08/719,898, entitled
`Method And Apparatus For Providing Caller Identifi- 40
`cation Based Out-going Messages In A Computer Tele(cid:173)
`phony Environment, by Mattaway et al.;
`U.S. patent application Ser. No. 08/718,911, entitled
`Method And Apparatus For Providing Caller Identifi(cid:173)
`cation Based Call Blocking In A Computer Telephony 45
`Environment, by Mattaway et al.; and
`U.S. patent application Ser. No. 08/719,639, entitled
`Method And Apparatus For Providing Caller Identifi(cid:173)
`cation Responses In A Computer Telephony
`Environment, by Mattaway et al.
`
`30
`
`FIELD OF THE INVENTION
`
`The present invention relates, in general, to data process(cid:173)
`ing systems, and more specifically, to a method and appa(cid:173)
`ratus for facilitating audio communications over computer
`networks.
`
`BACKGROUND OF THE INVENTION
`
`The increased popularity of on-line services such as
`AMERICA ONLINP", COMPUSERVE®, and other ser(cid:173)
`vices such as Internet gateways have spurred applications to
`provide multimedia, including video and voice clips, to
`online users. An example of an online voice clip application
`is VOICE E-MAIL FOR WINCIM and VOICE E-MAIL 65
`FOR AMERICA ONLINpM, available from Bonzi
`Software, as described in "Simple Utilities Send Voice
`
`SONY EXHIBIT 1009- Page 29
`
`

`
`6,009,469
`
`4
`FIG. 16A is a flowchart illustrating the process steps
`performed by the connection server in accordance with the
`present invention;
`FIG. 16B is a flowchart illustrating the process steps
`performed in accordance with the information server of the
`present invention;
`FIGS. 17 A-B are schematic block diagrams illustrating of
`the packet transfer sequence in accordance with the com(cid:173)
`munication protocol of the present invention;
`FIGS. 18A-D are conceptual block diagrams illustrating
`user interface and graphic user interface objects utilized by
`the communication utility of the present invention;
`FIGS. 19A-C are conceptual block diagrams illustrating
`the event manager and media engine objects utilized by the
`communication utility of the present invention; and
`FIGS. 20A-D illustrate process steps performed by the
`media engine function of the communication utility in
`accordance with the present invention.
`
`DETAILED DESCRIPTION
`
`3
`provides a virtual communications utility displayable on
`computer system interfaces which enables real-time, point(cid:173)
`to-point communications over computer networks. Accord(cid:173)
`ing to one embodiment of the present invention, a computer
`program product for use with a computer system having a 5
`display and an audio transducer comprises a computer
`usable medium having computer readable code means
`embodied therein comprising program code means for gen(cid:173)
`erating a user interface, program code means responsive to
`user input commands for establishing a point-to-point com- 10
`munication link with another computer over a network and
`program code means responsive to audio data from the audio
`transducer for transmitting the audio data over the commu(cid:173)
`nication link.
`According to another embodiment of the present 15
`invention, a computer program product for use with a
`computer system comprises a computer usable medium
`having computer readable program code means embodied
`thereon comprising code means for transmitting from a
`client process to a server a query a'S to whether a second 20
`client process is connected to the computer network, pro(cid:173)
`gram code means for receiving the network protocol address
`of the second process from the server, and program code
`means responsive to the network protocol address of the
`second client process for establishing a point-to-point com- 25
`munication link between the first client process and the
`second client process.
`
`Referring now in specific detail to the drawings, with like
`reference numerals identifying similar or identical elements,
`as shown in FIG. 1, the present disclosure describes a
`point-to-point network protocol and system 10 for using
`such a protocol.
`In an exemplary embodiment, the system 10 includes a
`BRIEF DESCRIPTION OF THE DRAWINGS
`first processing unit 12 for sending at least a voice signal
`The features of the invention will become more readily
`30 from a first user to a second user. The first processing unit
`apparent and may be better understood by referring to the
`12 includes a processor 14, a memory 16, an input device 18,
`following detailed description of an illustrative embodiment
`and an output device 20. The output device 20 includes at
`of the present invention, taken in conjunction with the
`least one modem capable of, for example, 14.4 Kilobit-per(cid:173)
`accompanying drawings, in which:
`second communications and operatively connected via
`FlG. 1 illustrates, in block diagram format, a system for
`35 wired and/or wireless communication connections to the
`the disclosed point-to-point Internet protocol;
`Internet or other computer networks such as an Intranet, i.e.,
`a private computer network. One skilled in the art would
`FlG. 2 illustrates, in block diagram format, the system
`understand that the input device 18 may be implemented at
`using a secondary point-to-point Internet protocol;
`least in part by the modem of the output device 20 to allow
`FlG.3 illustrates, in block diagram format, the system of
`40 input signals from the communication connections to be
`FIGS. 1-2 with the point-to-point Internet protocol estab(cid:173)
`received. The second processing unit 22 may have a
`lished;
`processor, memory, and input and output devices, including
`FlG. 4 is another block diagram of the system of FIGS.
`at least one modem and associated communication
`1-2 v.rith audio communications being conducted;
`connections, as described above for the first processing unit
`FlG. 5 illustrates a display screen for a processing unit;
`FlG. 6 illustrates another display screen for a processing 45 12. In an exemplary embodiment, each of the processing
`units 12, 22 may execute the WEBPHONE® Internet tele(cid:173)
`unit;
`phony application available from NetSpeak Corporation,
`FlG. 7 illustrates a flowchart of the initiation of the
`Boca Raton, Fla., which is capable of performing the dis(cid:173)
`point-to-point Internet protocols;
`closed point-to-point Internet protocol and system 10, as
`FlG. 8 illustrates a flowchart of the performance of the
`50 described herein.
`primary point-to-point Internet protocols;
`The first processing unit 12 and the second processing
`FlG. 9 illustrates a flowchart of the performance of the
`unit 22 are operatively connected to the Internet 24 by
`secondary point-to-point Internet protocol;
`communication devices and software known in the art, such
`FlG. 10 illustrates schematically a computer network over
`as an Internet Service Provider (ISP) or an Internet gateway.
`which the present invention may be utilized;
`The processing units 12, 22 may be operatively intercon(cid:173)
`FlG. 11 is a block diagram of a computer system suitable
`nected through the Internet 24 to a connection server 26, and
`for use with the present invention;
`may also be operatively connected to a mail server 28
`FlG. 12 is a block diagram of an audio processing card
`associated with the Internet 24.
`suitable for use with the computer system of FIG. 10;
`The connection server 26 includes a processor 30, a timer
`FlGS. 13 A-B are schematic block diagrams of the 60
`32 for generating time stamps, and a memory such as a
`elements comprising the inventive computer network tele(cid:173)
`database 34 for storing, for example, E-mail and Internet
`phony mechanism of the present invention;
`Protocol (IP) addresses of logged-in units. In an exemplary
`FlG. 14 is a screen capture illustrating an exemplary user
`embodiment, the connection server 26 may be a SPARC 5
`interface of the present invention;
`server or a SPARC 20 server, available from SUN
`FlG. 15 is a schematic diagram illustrating the architec- 65
`MICROSYSTEMS, INC., Mountain View, Calif., having a
`central processing unit (CPU) as processor 30, an operating
`ture of the connection server apparatus suitable for use with
`system (OS) such as UNIX, for providing timing operations
`the present invention;
`
`55
`
`SONY EXHIBIT 1009- Page 30
`
`

`
`6,009,469
`
`5
`
`5
`such as maintaining the timer 32, a hard drive or fixed drive,
`as well as dynamic random access memory (DRAM) for
`storing the database 34, and a keyboard and display and/or
`other input and output devices (not shown in FIG. 1). The
`database 34 may be an SQL database available from
`ORACLE or INFORMIX.
`In an exemplary embodiment, the mail server 28 may be
`implemented with a Post Office Protocol (POP) Version 3
`mail server and the Simple Mail Transfer Protocol (SMTP),
`including a processor, memory, and stored programs oper(cid:173)
`ating in a UNIX environment, or, alternatively, another OS,
`to process E-mail capabilities between processing units and
`devices over the Internet 24.
`In the illustrative embodiment, the POP protocol is uti(cid:173)
`lized to retrieve E-mail messages from mail server 28 while
`the SMTP protocol is used to submit E-mail message to
`Internet 24.
`The first processing unit 12 may operate the disclosed
`point-to-point Internet protocol by a computer program
`described hereinbelow in conjunction with FIG. 6, which
`may be implemented from compiled and lor interpreted
`source code in the C++ programming language and which
`may be downloaded to the first proces