Ulllted States Patent [19]
`Mattaway et al.
`
`[54] GRAPHIC USER INTERFACE FOR
`INTERNET TELEPHONY APPLICATION
`
`[75] Inventors: Shane D. Mattaway, Boca Raton;
`Glenn W- Hutton, Miami; Craig B-
`Strickland, Tamarac, all of Fla.
`
`[73] Assignee: NetSpeak Corporation, Boca Raton,
`Fla.
`
`[21] Appl. No.: 08/721,316
`[22]
`Filed:
`Sep. 25, 1996
`
`US006009469A
`[11] Patent Number:
`[45] Date of Patent:
`
`6,009,469
`Dec. 28, 1999
`
`5,463,625 10/1995 Yasrebi .
`5465286 11/1995 Clare 9 91-
`5,469,500 11/1995 Satter et al. .
`2E1; et a1‘ '
`5,509,’05s
`4/1996 Sestak et al. .
`5,517,494
`5/1996 Green .
`5,524,110
`6/1996 Danneels et al. .
`5,524,141
`6/1996 Braun et al. .
`5,526,489
`6/1996 Nilakantan et a1. .
`5,533,110
`7/1996 Pinard et al. .
`5,544,303
`8/1996 Maroteaux et al. .
`5,546,582
`8/1996 Brockmeyer et al. .
`FOREIGN PATENT DOCUMENTS
`
`-
`-
`Related U'S' Apphcatlon Data
`
`A2 0445402 11/1991 European Pat. Off. .
`A2 0556012 8/1993 European Pat. Off. .
`
`[63]
`
`Continuation-in-part of application No. 08/533,115, Sep. 25,
`1995
`[60] Provisional application No. 60/025,415, Sep. 4, 1996, and
`provisional application No- 60/024,251, Aug- 21, 1996-
`
`[51]
`
`Int. Cl? .................................................... .. G06F 17/00
`
`
`U-S- Cl- ............................................................ .. [58] Field of Search ....................... .. 395/200.57, 200.58;
`709/227, 228; 370/352, 355, 357, 389,
`395
`
`[56]
`
`Ct d
`R f
`e erences l e
`
`5,095,480
`
`5’2O4’669
`5,224,095
`5,291,554
`
`U-S- PATENT DOCUMENTS
`3/1992 Fenner.
`girllcrlrlljn et al' '
`4/1993 Dorfe e't a1_ _
`6/1993 Woest et al. .
`3/1994 Morales .
`551E511 a? eatl'a'L '
`6/1994 McMillen et a1__
`5’321’813
`5’357’571 10/1994 Banwart _
`5,400,335
`3/1995 Yamada .
`5,425,028
`6/1995 Britton et al. .
`5,430,709
`7/1995 Galloway -
`5,430,727
`7/1995 CaTTOH-
`574327846
`7/1995 Non? '
`5,442,633
`8/1995 Perkins et al. .
`5452 296
`9/1995 Shimizu.
`5:455:85,‘ 10/1995 D?ts et a1_ _
`5,457,683 10/1995 Robins.
`5,457,738 10/1995 Sylvan .
`
`W0 9219054 10/1992 WIPO '
`OTHER PUBLICATIONS
`
`Internetworking with T CP/IR vol. 1, Second Edition, Prin
`' les Protocols and Architecture by Douglas E. Comer.
`“P ’
`’
`’
`.
`VocalTec Internet Phone (TM) Version 2.5, www.coXsm
`
`u'edu/class/mis63s6/peOp1e/StOrt/iphOne25‘6X6, Weinberg, Netscape Conference and Cooltalk Meeting
`Room, www.q5.com, Feb. 22, 1996.
`Gull, Re: Getting IP address of PPP—connected Mac,
`<jgull—0304951005350001@pm012—11.dialip.mich.net>,
`Apr. 3, 1995.
`Mac‘),
`for
`Phone
`Internet
`Gull,
`Re:
`>jgull—1704950116450001@pm049—28.dialip.mich.net>,
`Apr- 17,1995
`Primary Examiner—Ellis B. Ramirez
`Attorney, Agent, or Firm—Kudirka & Jobse, LLP
`
`[57]
`ABSTRACT
`A communication utility for establishing real-time, point
`to-point communications between processes over a com
`puter network includes apparatus for querying a server as to
`the network protocol address of another client process, and
`apparatus for directly establishing a communication link
`with the client process upon receipt of the network protocol
`address from the server. In one embodiment, the utility
`includes a sophisticated user interface having features simi
`1
`t
`t
`-
`1 t 1 h
`h d
`b t -
`1
`t-
`t
`ar oypica' e ep ony ar ware u imp emen ing grea er
`?exibility With software.
`
`18 Claims, 27 Drawing Sheets
`
`,
`
`:
`i
`,
`:
`l
`i
`:
`1
`l
`L _______________________ __:
`
`| |
`i
`{
`I
`|
`:
`
`22
`_\
`SECOND
`PROCESSING
`UNIT
`
`INTERNET 28
`
`‘F
`POP 3
`‘
`' MAIL SERVER ‘
`
`26
`
`CONNECTION SERVER
`, PROCESSOR 4
`,
`
`30
`
`32
`
`DATABAS E
`
`I
`I
`I
`I
`I
`
`I
`I
`l
`I
`I
`I
`I
`l
`I
`I
`I
`I
`I
`l
`I
`I
`l
`I
`I
`I
`I
`I
`I
`l
`l
`I
`
`SONY EXHIBIT 1001- Page 1
`
`

`

`U.S. Patent
`
`Dec. 28, 1999
`
`Sheet 1 0f 27
`
`6,009,469
`
`0200mm
`
`.U2_wwwOOmn_
`
`.223
`
`ml
`
`mOwwwOOwE
`
`oml/
`
`Y
`
`mOwwwOOmm +
`
`mm)?
`
`P3950
`
`mO_>wQ
`
`klww
`
`~ HUN“
`
`SONY EXHIBIT 1001- Page 2
`
`

`

`U.S. Patent
`
`Dec. 28, 1999
`
`Sheet 2 0f 27
`
`6,009,469
`
`mm
`
`
`
`>mm>mijn.
`
`szOowwmLm
`
`‘llIl'
`
`0200mm
`
`Oz_mwmoOmm
`
`.EZD
`
`
`
`mm>mmw..=<_>_
`
`_mmHmzmmc.2_
`
`________._L
`
`
`
`AthDmehomz200v
`
`AkaOmzzOOV
`
`v
`
`
` SONY fiXH R i
`
`
`
`SONY EXHIBIT 1001- Page 3
`1001— Page 3
`
`
`
`

`

`U.S. Patent
`
`Dec. 28, 1999
`
`Sheet 3 0f 27
`
`6,009,469
`
`ozoomm5mm
`0258095025300.“.
`
`:2:9:23
`
`
`
`
`Oz_wmm00mn.Oz_wmm_00n__652
`
`
`02086.7565ommmmmmzoov“5m:
`
`t2:.223
`
`Hmzmmhz
`
`
`
`
`
`
`SONY EXHIBIT 1001- Page 4
`SONY *ZXH R i 100‘]— Page 4
`
`
`

`

`U.S. Patent
`
`Dec. 28, 1999
`
`Sheet 4 0f 27
`
`6,009,469
`
`ROGER
`91.103.001.027
`C2
`0343 13
`O IN USE
`
`46
`
`[36
`
`SHANE
`195.201.00.137
`L1
`0024
`13
`O IDLE
`
`~38
`
`EE1§IEI
`EEJEHEQ.
`$51K!
`‘ [5%133
`
`42
`
`@IEHE/
`EEIHEH'EI
`@[@@f
`
`44
`
`48
`
`FIG. 6
`
`SONY EXHIBIT 1001- Page 5
`
`

`

`U.S. Patent
`
`Dec. 28, 1999
`
`Sheet 5 0f 27
`
`6,009,469
`
`START THE POINT-TO-POINT
`INTERNET PROTOCOLS
`I
`INITIATE PRIMARY INTERNET
`PROTOCOL
`l
`IS THE CONNECTION SERVER IN A
`RESPONSIVE CONDITION?
`
`54
`
`56
`
`58
`
`60—\ YESl
`
`PERFORM PRIMARY
`INTERNET PROTOCOL
`
`lNO
`INITIATE SECONDARY
`INTERNT PROTOCOL
`
`62
`
`FIG. 7
`
`START THE PRIMARY
`POINT-TO-POINT INTERNET
`PROTOCOL
`
`64
`
`‘
`
`I
`
`66
`
`TIMESTAMP AND STORE E-MAIL
`ADRESSES AND IP ADRESSES OF
`LOGGED-IN UNITS IN A DATABASE
`
`I
`
`RECEIVE QUERY FROM FIRST UNIT /_ 68
`WHETHER A SPECIFIED SECOND i
`UNIT IS LOGGED-IN
`l
`RETRIEVE IP ADDRESS FROM
`DATABASE IF THE SECOND UNIT IS
`LOGGED-IN
`l
`SEND RETRIEVED IP ADRESS TO
`FIRST UNIT TO ESTABLISH POINT
`TO—POINT CONNECTION
`
`70
`
`72
`
`FIG. 8
`
`SONY EXHIBIT 1001- Page 6
`
`

`

`U.S. Patent
`
`Dec. 28, 1999
`
`Sheet 6 0f 27
`
`6,009,469
`
`74
`START THE SECONDARY f
`POINT-TO POINT INTERNET PROTOCOL
`I
`
`76
`GENERATE AN E-MAIL SIGNAL,
`INCLUDING A SESSION NUMBER AND A f
`FIRST IP ADRESS CORRESPONDING TO A
`FIRST PROCESSING UNIT
`
`I
`TRANSMIT THE E-MAIL SIGNAL AS A
`<CONNECTREQUEST> SIGNAL
`TO THE INTERNET
`
`I
`
`I
`80
`DELIVER THE E-MAIL SIGNAL THROUGH f
`THE INTERNET USING A MAIL SERVER TO
`A SECOND PROCESSING UNIT
`
`78
`
`82
`
`
`
`EXTRACT THE SESSION NUMBER AND THE FIRST IP ADRESS FROM THE E-MAIL
`
`SIGNAL
`
`I
`84
`SEND THE SESSION NUMBER AND A
`SECOND IP ADRESS CORRESPONDING f
`TO THE SECOND PROCESSING UNIT TO
`THE FIRST PROCESSING UNIT THROUGH
`I
`THE INTERNET
`
`I
`
`-
`
`.
`
`86
`
`VERIFY THE SESSION NUMBER V
`RECEIVED FROM THE SECOND
`PROCESSING UNIT
`
`I
`
`FIG. 9
`
`88
`ESTABLISH A POINT-TO-POINT INTERNET
`COMMUNICATION LINK BETWEEN THE f
`FIRST AND SECOND PROCESSING UNITS
`USING THE FIRST AND SECOND IP
`ADDRESSES
`
`SONY EXHIBIT 1001- Page 7
`
`

`

`U.S. Patent
`
`Dec. 28, 1999
`
`Sheet 7 0f 27
`
`6,009,469
`
`x8352
`32mm :326
`
`
`\ ® 82 1/ E:
`
`HESS
`
`8.3mm
`
`3235
`
`m _
`
`m was
`
`coop //
`
`2.2.
`
`22 @
`
`2 mémt
`
`I; a 32
`
`SONY EXHIBIT 1001- Page 8
`
`

`

`U.S. Patent
`
`Dec. 28, 1999
`
`Sheet 8 0f 27
`
`6,009,469
`
`SP
`
`\
`
`
`
`|\ l\_ a > |\
`
`
`
`
`
`02 L ‘ mi |\ 9: |\
`
`9. w o l\ 556% m9
`
`
`$105,200 $30520 _ * v65
`$3258 _>_<m
`
`
`mam
`
`
`
`it.‘ moEn= h h QEQQE
`
`
`
`
`
`T210 v5.5 nwxm mama 20m Q0 wzmo wtwxwa m E052
`
`
`
`
`
`
`
`
`@ rrl?. SP |\-&
`
`
`@3 S: |\. 2: mwjomkzoo
`mm? Elk N3 |\ W
`
`
`0Q
`
`mm? Iv
`
`052
`
`|\ v m: *
`
`20m
`
`‘
`
`|l|. HII $305200
`o: _ mwjoEzoo
`
`mm?
`
`l‘ 4
`
`Nm_‘|v . . . . . . . . . . . . . . . ..
`
`I s n o o a I n u I - u - n o.
`
`on?
`
`V mm?
`
`mam
`
`SONY EXHIBIT 1001- Page 9
`
`

`

`U.S. Patent
`
`Dec. 28, 1999
`
`Sheet 9 0f 27
`
`6,009,469
`
`4}
`U
`
`BUS
`INTERFACE
`
`/1228
`
`1214
`
`1230
`
`1212
`
`ND
`/1216
`INPUT
`BUFFER
`
`/1222
`
`DSP
`
`/1224
`
`/1213
`—* D/A
`/1210
`122 0
`4——>
`/
`OUTPUT
`' BUFFER
`
`'
`
`1—>
`
`1226
`/
`
`——> RAM
`
`ROM —-—>
`
`v
`
`FIGURE 12 (PRIOR ART)
`
`SONY EXHIBIT 1001- Page 10
`
`

`

`U.S. Patent
`
`Dec. 28, 1999
`
`Sheet 10 0f 27
`
`6,009,469
`
`unudwfi?
`
`Emmsou:_>>-x
`
`:52585332
`
`525022
`:5535055.2.2.
`
`952.53
`
`52:
`
`<22:3
`
`
`
` SONY *ZXH R i
`
`
`
`SONY EXHIBIT 1001- Page 11
`1001— Page 11
`
`

`

`U.S. Patent
`
`Dec. 28, 1999
`
`Sheet 11 0f 27
`
`6,009,469
`
`1328
`
`1326
`
`1322
`
`1336
`
`1338
`
`1332
`
`FIGURE 13 B
`
`SONY EXHIBIT 1001- Page 12
`
`

`

`U.S. Patent
`
`Dec. 28, 1999
`
`Sheet 12 0f 27
`
`6,009,469
`
`14921
`
`I
`1
`
`1402A “~~(Roger Wilco
`1402B
`n~i|co@isp.c0m
`
`1400
`
`# /1402
`
`14
`
`-
`
`Q/G?~ 1402D
`1031
`02C
`
`1402F km TALK \ A “14026
`1402] r
`J 5 14021-1
`
`-
`
`1406 4
`
`;
`
`1408 -4
`
`1
`
`4
`
`7
`
`*
`
`2
`
`5
`
`8
`
`3
`
`6
`
`9
`
`L1
`
`L2
`
`L3
`
`O
`
`#
`
`L4
`
`RCL
`
`END
`
`SND
`
`DND
`INE
`
`MUT
`DIR
`
`HLD
`MSG
`
`V
`
`> 1404
`
`J
`
`} 1410
`
`[E] ? W@E®@[FD©UD© X
`
`Figure 14
`
`SONY EXHIBIT 1001- Page 13
`
`

`

`U.S. Patent
`
`Dec. 28, 1999
`
`Sheet 13 0127
`
`6,009,469
`
`/1516A
`
`/1516B
`
`/1s1ec
`
`D
`
`/1s1e
`
`1 1 E
`
`/ 5 s
`
`.200
`
`DATABASE /1516F
`SEARCH
`ENGINE
`
`DATABASE
`
`/1512
`
`\‘516
`
`1518 \
`
`CONNECTION
`SERVER
`
`INFORMATION
`SERVER
`
`V DATABASE
`7
`v
`SER ER
`
`\1514
`
`'
`
`r
`
`\1soa
`
`r
`
`r
`
`\1510
`
`NETWORK INTERFACE CARD
`
`\1s20
`
`1532
`
`I
`FIREWALLSERVER /1522
`
`\1sss
`
`1 538
`
`/
`
`1534
`
`l
`ROUTER
`l
`CSU/DSU
`
`/‘524
`
`/1526
`
`1538
`
`Figure 15A
`
`SONY EXHIBIT 1001- Page 14
`
`

`

`U.S. Patent
`
`Dcc.28,1999
`
`Sheet 14 0f 27
`
`6,009,469
`
`DATABASE
`
`DATABASE
`SEARCH
`ENGINE
`
`SERVER
`
`CONNECTION
`SERVER
`
`INFORMATION
`SERVER
`
`DATABASE
`
`. ®
`
`1530
`
`
`
`1534
`
`Figure 158
`
`
`
`
`
`
`SONY EXHIBIT 1001- Page 15
`SONY *ZXH R i 100‘]— Page 15
`
`

`

`U.S. Patent
`
`Dec. 28, 1999
`
`Sheet 15 0f 27
`
`6,009,469
`
`Doomw
`
`Oocmr
`
`mccmw
`
`xmuamuoz
`
`atom.320
`
`xmaamfiz
`
`32mm.320
`
`2819...:
`
`52mm.320
`
`wvmw@
`
`$26
`
`Nvmw@
`
`cmmw
`
`mwm..
`
`@
`
`xmonmsz
`
`
`
`3me.320
`
`<0:w
`
`0m.9:9...
`
`
`
` SONY *ZXH R i
`
`
`
`SONY EXHIBIT 1001- Page 16
`1001— Page 16
`
`

`

`U.S. Patent
`
`Dec. 28, 1999
`
`Sheet 16 0f 27
`
`6,009,469
`
`<2 959E
`
`5..
`
`m0 m0<wwm2
`
`
`
`AwZj uEOv
`
`
`
`mOmmw 02mm
`
`\
`
`was
`
`SONY EXHIBIT 1001- Page 17
`
`

`

`U.S. Patent
`
`Dec. 28, 1999
`
`Sheet 17 0f 27
`
`6,009,469
`
`<|NFOREQ>
`RECEIVED
`
`DISGARD
`UNRECOGNIZED
`DATA
`
`| \1635
`
`EXTRACT SEARCH
`CRITERIA
`
`1634
`
`SUPPLY SEARCH / 1636
`CRITERIA TO DATABASE
`I
`SEND
`<INFO ACK>
`
`1638
`
`<INFO ABORT
`RECEIVED
`
`No /1s42
`SEND <|NFO>
`
`END
`
`Figure 168
`
`SONY EXHIBIT 1001- Page 18
`
`

`

`U.S. Patent
`
`iDcc.28,1999
`
`Sheet 18 0f 27
`
`6,009,469
`
`wmmwmum_,
`
`IIIIIIIIIEEIHIHIHIHIHIHIHIIIIIIIIIII
`
`A._._<Ovm
`
`AhUmwmmv<m
`
`A>m3mvmm
`
`m0
`
`m0
`
`
`
`A¥U<.._.—<Uvamm0
`
`zmzp
`
`Amm3w2<v2.
`
`
`
`A1032mm>>mz<vOm
`
`A0_n_=<v<:
`
`AO_DD<Vm:
`
`ADvaNF
`
`gmmmzozm
`
`
`
`AOmm0&2.mmmDvN
`
`
`
`AOmmMEI—20v_.
`
`AZO_._.<m._.m_Gm_mvv
`
`A0u2_mmeVn
`
`
`
`Av_U<mz_._ZOvm
`
`
`
`AOmm.FUmZZOUvm
`
`
`
`A¥0<HUMZZOUV<~.
`
`
`
`AMZE"EOVmh
`
`040m<4 mmm>mm
`
`Av_Omm..:.<0v0:.
`
`
`
`Av_0<Ema.._<0vaI.
`
`Eh2:5
`
`
`
`SONY
`
`
` fiXH R i
`
`SONY EXHIBIT 1001- Page 19
`1001— Page 19
`
`
`
`
`
`
`
`

`

`U.S. Patent
`
`Dec. 28, 1999
`
`Sheet 19 0f 27
`
`6,009,469
`
`BmmcLIOZuJ
`MM
`
`
`
`AQmm OmZ_v _.
`
`ADM-2V <m
`
`AOuZ_v mm
`
`m“ h msmt
`
`(5400114.: (DLUE>LLID'.'
`
`SONY EXHIBIT 1001- Page 20
`
`

`

`U.S. Patent
`
`Dec. 28, 1999
`
`Sheet 20 0f 27
`
`6,009,469
`
`
`
`
`
`NEF
`
`8833.55
`
`
`
`2:;team
`
`a.:27.
`
`woww
`
`05..
`
`«fl
`
`nom_nO_m:t_>_D
`
`_ohcoo_m:t_>5
`
`
`
`$332502mcosmamauwSonEst;
`
`ant;
`
`3:50.20 c3323mac.
`
`
`
`33.15262m:o=o:.a>>>ozflm6o:_a>noon_m::_>
`
`co_uom=00_3
`
`6:50.:
`
`
`
`
`
`LewsoaatSSEwan.2262.005a85:5
`
`
`
`9.35LEEooSc
`
`Epin—2232.005
`
`
`
`m:=£mLEEOOS
`
`w:_m>mac.
`
`wear
`
`
`
`vow?New—1
`
`3:939$
`
`
`
`SONY
`
`
` fiXH R i
`
`SONY EXHIBIT 1001- Page 21
`1001— Page 21
`
`
`
`
`
`
`
`

`

`U.S. Patent
`
`Dec. 28, 1999
`
`Sheet 21 0f 27
`
`6,009,469
`
`1 804
`
`UlControl
`
`GUIControl* Sibling
`
`1828
`
`1 822
`
`UlCollection* Parent
`
`
`
`
`
`
`
`1814
`
`Figure 188
`
`
`
`SONY fiXH
`
`SONY EXHIBIT 1001- Page 22
`i 1001— Page 22
`
`

`

`U.S. Patent
`
`Dcc.28,1999
`
`Sheet 22 0f 27
`
`6,009,469
`
`Figure 180
`
`
`
`
`
`
`is a
`
`UlPushButton
`
`UlPlayRun
`
`UlToggle
`
`1842
`
`1844
`
`1846
`
`
`
`
`UlCollection
`
`
`
`GUIVirtualObject* Childrenl]
`UlCollection* Parent
`
`Figure 180
`
`
`
`1806
`
`
`
`
`void ShowO
`
`void Hide()
`
` 1834
`
`1832
`
`
`
`
`
`
`SONY EXHIBIT 1001- Page 23
`SONY *ZXH R i 100‘]— Page 23
`
`

`

`U.S. Patent
`
`Dec. 28, 1999
`
`Sheet 23 0f 27
`
`6,009,469
`
`mmo._oum<
`
`Efimm<
`aofiumE
`
`m:._.__u_nm<
`
`msmmaumz
`
`=2.m<
`
`uEu
`
`uEuOxwfi
`
`95
`
`fit
`
`28m
`
`no_.
`
`So...
`
`:xmwotxm<
`
`Egoom|m<
`m.__u_mm<
`
`55mm<
`
`U__20._._lw<
`
`U=2Im<
`
`33.2:
`
`:32:
`
`352:
`
`5%0E.
`
`man:
`
`
`
`97.3
`
`9.22:
`
`530.6
`
`5:25
`
`265:2:
`
`35mD3.3
`
`553m
`
`uoxuom
`
`our—3L3
`
`2:3
`
`o>taaa
`
`2:883
`
`8..
`
`+D¢
`
`0%:E
`
`35m
`
`3:30.:
`
`xmmhcrH:59
`
`xwabzo.
`
`nowxm:*
`
`92595.
`
`33
`
`one?
`
`o>taauu<
`
`o>tmnm>oEmm
`
`$3585
`
`oxmmkgoEmm
`
`wumwmG2.:
`
`c0383.
`
`nor
`
`
`
`930236
`
`9.89053.
`
`<2Bani
`
`\Ill'llfl'llllj
`
`32:.33>9...“6::
`
`02$sz22
`
`8:59
`
`9.22.mm“Miami
`
`
`
`SONY
`
`
` fiXH R i
`
`SONY EXHIBIT 1001- Page 24
`1001— Page 24
`
`
`

`

`U.S. Patent
`
`Dcc.28,1999
`
`Sheet 24 0f 27
`
`6,009,469
`
`SOCKET
`
`2004
`
`
`2006
`
`MIC@‘5
`
`NO
`
`
`
`
`SPEAKER
`
`® ®
`SPEAKER
`
`
`WRITE TO
`FILE HANDLE
`
`
`BUFFER
`
`EMPTY?
`
`20”
`
`SOCKET
`
`
`
`IS
`PARTYU]
`ONLINE?
`
`
`
`2°30
`
`SOCKET
`
`2032
`
`
`
`PLACE EMPTY
`MIC
`
`
`BUFFER ON
`
`ENABLED
`MIC SAMPLING
`
`
`?
`
`
`QUEUE
`
`2034
`
`PLACE EMPTY
`BUFFER ON
`
`WinSock QUEUE
`
`
`PLACE EMPTY
`
`BUFFER IN
`
`"free" POOL
`
`
`2°36
`
`2038
`
`Figure 20A
`
`
`
`
`
`
`SONY EXHIBIT 1001- Page 25
`SONY *ZXH R i 100‘]— Page 25
`
`

`

`U.S. Patent
`
`Dec. 28, 1999
`
`Sheet 25 0f 27
`
`6,009,469
`
`meow meow
`
`Loon:So:mEEEmm
`
`3:.£2300::222m.
`
`
`
`.23..oc>m<
`
`gnaw—EsE0:
`
`
`
`9::59:0
`
`Loon...E0:
`
`Neon
`
`a
`
`oeom
`
`0.:o_E
`
`usecoc>m<
`
`2EEa:
`
`Nmow
`
`
`
`omenevow
`
`mam93mm
`
`memmm
`
`.mEBEOz
`
`SEN
`
`mm;
`
`
`
`
`
`
`SONY EXHIBIT 1001- Page 26
`SONY *ZXH R i 100‘]— Page 26
`
`
`
`

`

`U.S. Patent
`
`Dec. 28, 1999
`
`Sheet 26 0f 27
`
`6,009,469
`
`Loon...mm:
`
`wmom
`
`
`
`xuomcts55:3memama“9.00m.650
`
`omenemom
`
`icon:3:
`
`53mmEmomata_omow0gen3:332:
`
`
`mmE.05.Emm;«mom
`
`%.a32:3
`
`-2:so:
`
`02tam:
`
`._.m<._.w
`
`V.
`
`Dam9:919
`
`«9.00m
`
`
`
`Exam“?mmou
`
`
`
`SONY
`
`
` fiXH R i
`
`SONY EXHIBIT 1001- Page 27
`1001— Page 27
`
`
`

`

`U.S. Patent
`
`Dec. 28, 1999
`
`Sheet 27 0f 27
`
`6,009,469
`
`
`
`Discard pending
`data in mix
`
`buffer
`
`Figure 200
`
`
`
`SONY fiXH
`
`SONY EXHIBIT 1001- Page 28
`i 1001— Page 28
`
`

`

`6,009,469
`
`1
`GRAPHIC USER INTERFACE FOR
`INTERNET TELEPHONY APPLICATION
`
`RELATED APPLICATIONS
`
`This application is a continuation-in-part of US. patent
`application Ser. No. 08/533,115 (Pending) entitled Point-to-
`Point Internet Protocol, by Glenn W. Hutton, filed Sep. 25,
`1995, commonly assigned, the subject matter of which is
`incorporated herein by reference.
`To the extent that any matter contained herein is not
`already disclosed in the above-identified parent application
`a location claims priority to US. provisional patent appli-
`cation 60/025,415 entitled Internet Telephony Apparatus and
`Method by Mattaway et al., filed Sep. 4, 1996, and US.
`provisional patent application Ser. No. 60/024,251 entitled
`System and Methods for Point-To-Point Communications
`Over a Computer Network, by Mattaway et al., filed Aug.
`21, 1996.
`In addition, this application is one of a number of related
`applications filed on an even date herewith and commonly
`assigned,
`the subject matters of which are incorporated
`herein by reference, including the following:
`US. patent application Ser. No. 08/719,894, entitled
`Directory Server For Providing Dynamically Assigned
`Network Protocol Addresses, by Mattaway et al.;
`US. patent application Ser. No. 08/719,554, entitled
`Point—to—point Computer Network Communication
`Utility Utilizing Dynamically Assigned Network Pro-
`tocol Addresses, by Mattaway et al.;
`US. patent application Ser. No. 08/719,640, entitled
`Method And Apparatus For Dynamically Defining Data
`Communication Utilities, by Mattaway et al.;
`US. patent application Ser. No. 08/719,891, entitled
`Method And Apparatus For Distribution And Presen-
`tation Of Multimedia Data Over A Computer Network,
`by Mattaway et al.;
`US. patent application Ser. No. 08/719,898, entitled
`Method And Apparatus For Providing Caller Identifi-
`cation Based Out—going Messages In A Computer Tele—
`phony Environment, by Mattaway et al.;
`US. patent application Ser. No. 08/718,9II, entitled
`Method And Apparatus For Providing Caller Identifi-
`cation Based Call Blocking In A Computer Telephony
`Environment, by Mattaway et al.; and
`US. patent application Ser. No. 08/719,639, entitled
`Method And Apparatus For Providing Caller Identifi-
`cation Responses In A Computer Telephony
`Environment, by Mattaway et al.
`FIELD OF THE INVENTION
`
`The present invention relates, in general, to data process-
`ing systems, and more specifically, to a method and appa-
`ratus for facilitating audio communications over computer
`networks.
`
`BACKGROUND OF THE INVENTION
`
`The increased popularity of on-line services such as
`AMERICA ONLINETM, COMPUSERVE®, and other ser—
`vices such as Internet gateways have spurred applications to
`provide multimedia,
`including video and voice clips,
`to
`online users. An example of an online voice clip application
`is VOICE E-MAIL FOR WINCIM and VOICE E-MAIL
`FOR AMERICA ONLINETM, available from Bonzi
`Software, as described in “Simple Utilities Send Voice
`
`10
`
`20
`
`L)m
`
`30
`
`2
`E-Mail Online”, MULTIMEDIA WORLD, VOL. 2, NO. 9,
`Aug. 1995, p. 52. Using such Voice E-Mail software, a user
`may create an audio message to be sent to a predetermined
`E-mail address specified by the user.
`Generally, devices interfacing to the Internet and other
`online services may communicate with each other upon
`establishing respective device addresses. One type of device
`address is the Internet Protocol (IP) address, which acts as
`a pointer to the device associated with the IP address. A
`typical device may have a Serial Line Internet Protocol or
`Point—to—Point Protocol (SLIP/PPP) account with a perma—
`nent IP address for receiving E—mail, voicemail, and the like
`over the Internet. E—mail and voicemail is generally intended
`to convey text, audio, etc., with any routing information
`such as an IP address and routing headers generally being
`considered an artifact of the communication, or even gib-
`berish to the recipient.
`Devices such as a host computer or server of a company
`may include multiple modems for connection of users to the
`Internet, with a temporary IP address allocated to each user.
`For example,
`the host computer may have a general IP
`address “XXXXXXXXX,” and each user may be allocated
`a successive IP address of XXX.XXX.XXX.10,
`XXX.XXX.XXX.II, XXXXXXXXXTZ, etc. Such tem-
`porary IP addresses may be reassigned or recycled to the
`users, for example, as each user is successively connected to
`an outside party. For example, a host computer of a company
`may support a maximum of 254 IP addresses which are
`pooled and shared between devices connected to the host
`computer.
`Permanent IP addresses of users and devices accessing the
`Internet readily support point-to-point communications of
`voice and video signals over the Internet. For example,
`real-time video teleconferencing has been implemented
`using dedicated IP addresses and mechanisms known as
`reflectors. Due to the dynamic nature of temporary IP
`addresses of some devices accessing the Internet, point-to-
`point communications in real-time of voice and video have
`been generally difficult to attain.
`The ability to locate risers having temporary or dynami-
`cally assigned Internet Protocol address has been difficult
`without
`the user manually initiating the communication.
`Accordingly, spontaneous, real-time communications with
`such users over computer networks have been impractical.
`Further, it is desirable to have a communication utility which
`contains familiar features and functions to current commu-
`~ nication utility such as telephones and cellular telephones. It
`is even further desirable to utilize the current graphic user
`interface technology associated with computer software in a
`manner to achieve a more flexible interface to a such a
`communication utility, without
`the limitations associated
`with hardware.
`
`L»UI
`
`40
`
`50
`
`Accordingly, a need exists for a way to determine whether
`computer users are actively connected to a computer net-
`work.
`
`A further need exists for a way to obtain the dynamically
`assigned Internet Protocol address of a user having on-line
`status with respect to a computer network, particularly the
`Internet.
`
`60
`
`An even further need exists for a method and apparatus by
`which to establish real-time, point-to-point communications
`over a computer network using a communication utility
`having an interface which combines the familiar aspects of
`current hardware communication utilities but which allows
`
`for the flexibility associated with graphic user interfaces.
`SUMMARY OF THE INVENTION
`
`The above deficiencies in the prior art and previously
`described needs are fulfilled by the present invention which
`
`
`
`SONY
` R I
`SONY EXHIBIT 1001- Page 29
`1001— Page 29
`
`

`

`6,009,469
`
`3
`provides a virtual communications utility displayable on
`computer system interfaces which enables real-time, point-
`to-point communications over computer networks. Accord-
`ing to one embodiment of the present invention, a computer
`program product for use with a computer system having a
`display and an audio transducer comprises a computer
`usable medium having computer readable code means
`embodied therein comprising program code means for gen-
`erating a user interface, program code means responsive to
`user input commands for establishing a point-to-point com-
`munication link with another computer over a network and
`program code means responsive to audio data from the audio
`transducer for transmitting the audio data over the commu-
`nication link.
`
`According to another embodiment of the present
`invention, a computer program product for use with a
`computer system comprises a computer usable medium
`having computer readable program code means embodied
`thereon comprising code means for transmitting from a
`client process to a server a query as to whether a second
`client process is connected to the computer network, pro-
`gram code means for receiving the network protocol address
`of the second process from the server, and program code
`means responsive to the network protocol address of the
`second client process for establishing a point-to-point com-
`munication link between the first client process and the
`second client process.
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`The features of the invention will become more readily
`apparent and may be better understood by referring to the
`following detailed description of an illustrative embodiment
`of the present
`invention,
`taken in conjunction with the
`accompanying drawings, in which:
`FIG. 1 illustrates, in block diagram format, a system for
`the disclosed point-to-point Internet protocol;
`FIG. 2 illustrates, in block diagram format, the system
`using a secondary point—to—point Internet protocol;
`FIG. 3 illustrates, in block diagram format, the system of
`FIGS. 1—2 with the point-to-point Internet protocol estab-
`lished;
`FIG. 4 is another block diagram of the system of FIGS.
`1—2 with audio communications being conducted;
`FIG. 5 illustrates a display screen for a processing unit;
`FIG. 6 illustrates another display screen for a processing
`unit;
`FIG. 7 illustrates a flowchart of the initiation of the
`point-to-point Internet protocols;
`FIG. 8 illustrates a flowchart of the performance of the
`primary point—to—point Internet protocols;
`FIG. 9 illustrates a flowchart of the performance of the
`secondary point-to-point Internet protocol;
`FIG. 10 illustrates schematically a computer network over
`which the present invention may be utilized;
`FIG. 11 is a block diagram of a computer system suitable
`for use with the present invention;
`FIG. 12 is a block diagram of an audio processing card
`suitable for use with the computer system of FIG. 10;
`FIGS. 13 A—B are schematic block diagrams of the
`elements comprising the inventive computer network tele-
`phony mechanism of the present invention;
`FIG. 14 is a screen capture illustrating an exemplary user
`interface of the present invention;
`FIG. 15 is a schematic diagram illustrating the architec-
`ture of the connection server apparatus suitable for use with
`the present invention;
`
`10
`
`20
`
`L)m
`
`30
`
`L»UI
`
`40
`
`50
`
`60
`
`4
`illustrating the process steps
`FIG. 16A is a flowchart
`performed by the connection server in accordance with the
`present invention;
`illustrating the process steps
`FIG. 16B is a flowchart
`performed in accordance with the information server of the
`present invention;
`FIGS. 17A—B are schematic block diagrams illustrating of
`the packet transfer sequence in accordance with the com-
`munication protocol of the present invention;
`FIGS. 18A—D are conceptual block diagrams illustrating
`user interface and graphic user interface objects utilized by
`the communication utility of the present invention;
`FIGS. 19A—C are conceptual block diagrams illustrating
`the event manager and media engine objects utilized by the
`communication utility of the present invention; and
`FIGS. 20A—D illustrate process steps performed by the
`media engine function of the communication utility in
`accordance with the present invention.
`
`DETAILED DESCRIPTION
`
`Referring now in specific detail to the drawings, with like
`reference numerals identifying similar or identical elements,
`as shown in FIG. 1,
`the present disclosure describes a
`point-to-point network protocol and system 10 for using
`such a protocol.
`In an exemplary embodiment, the system 10 includes a
`first processing unit 12 for sending at least a voice signal
`from a first user to a second user. The first processing unit
`12 includes a processor 14, a memory 16, an input device 18,
`and an output device 20. The output device 20 includes at
`least one modem capable of, for example, 14.4 Kilobit-per-
`second communications and operatively connected Via
`wired and/or wireless communication connections to the
`Internet or other computer networks such as an Intranet, i.e.,
`a private computer network. One skilled in the art would
`understand that the input device 18 may be implemented at
`least in part by the modem of the output device 20 to allow
`input signals from the communication connections to be
`received. The second processing unit 22 may have a
`processor, memory, and input and output devices, including
`at
`least one modem and associated communication
`
`connections, as described above for the first processing unit
`12. In an exemplary embodiment, each of the processing
`units 12, 22 may execute the WEBPHONE® Internet tele—
`phony application available from NetSpeak Corporation,
`Boca Raton, Fla., which is capable of performing the dis-
`closed point-to-point Internet protocol and system 10, as
`described herein.
`
`The first processing unit 12 and the second processing
`unit 22 are operatively connected to the Internet 24 by
`communication devices and software known in the art, such
`as an Internet Service Provider (ISP) or an Internet gateway.
`The processing units 12, 22 may be operatively intercon-
`nected through the Internet 24 to a connection server 26, and
`may also be operatively connected to a mail server 28
`associated with the Internet 24.
`
`The connection server 26 includes a processor 30, a timer
`32 for generating time stamps, and a memory such as a
`database 34 for storing, for example, E—mail and Internet
`Protocol (IP) addresses of logged-in units. In an exemplary
`embodiment, the connection server 26 may be a SPARC 5
`server or
`a SPARC 20 server, available from SUN
`MICROSYSTEMS, INC, Mountain View, Calif, having a
`central processing unit (CPU) as processor 30, an operating
`system (OS) such as UNIX, for providing timing operations
`
`
`
`SONY
` R l
`SONY EXHIBIT 1001- Page 30
`1001— Page 30
`
`

`

`@009A69
`
`5
`such as maintaining the timer 32, a hard drive or fixed drive,
`as well as dynamic random access memory (DRAM) for
`storing the database 34, and a keyboard and display and/or
`other input and output devices (not shown in FIG. 1). The
`database 34 may be an SQL database available from
`ORACLE or INFORMIX.
`
`In an exemplary embodiment, the mail server 28 may be
`implemented with a Post Office Protocol (POP) Version 3
`mail server and the Simple Mail Transfer Protocol (SMTP),
`including a processor, memory, and stored programs oper-
`ating in a UNIX environment, or, alternatively, another OS,
`to process E—mail capabilities between processing units and
`devices over the Internet 24.
`
`In the illustrative embodiment, the POP protocol is uti-
`lized to retrieve E-mail messages from mail server 28 while
`the SMTP protocol is used to submit E-mail message to
`Internet 24.
`
`The first processing unit 12 may operate the disclosed
`point-to-point Internet protocol by a computer program
`described hereinbelow in conjunction with FIG. 6, which
`may be implemented from compiled and /or interpreted
`source code in the C++ programming language and which
`may be downloaded to the first processing unit 12 from an
`external computer. The operating computer program may be
`stored in the memory 16, which may include about 8 MB
`RAM and/or a hard or fixed drive having about 8 MB of
`available memory. Alternatively, the source code may be
`implemented in the first processing unit 12 as firmware, as
`an erasable read only memory (EPROM), etc. It is under-
`stood that one skilled in the art would be able to use
`
`programming languages other than C++ to implement the
`disclosed point-to-point network protocol and system 10.
`The processor 14 receives input commands and data from
`a first riser associated with the first processing unit 12 though
`the input device 18, which may be an input port connected
`by a wired, optical, or a wireless connection for electromag—
`netic transmissions, or alternatively may be transferable
`storage media, such as floppy disks, magnetic tapes, com-
`pact disks, or other storage media including the input data
`from the first user.
`
`The input device 18 may include a user interface (not
`shown) having, for example, at least one button actuated by
`the user to input commands to select from a plurality of
`operating modes to operate the first processing unit 12. In
`alternative embodiments, the input device 18 may include a
`keyboard, a mouse, a touch screen, and/or a data reading
`device such as a disk drive for receiving the input data from
`input data files stored in storage media such as a floppy disk
`or, for example, an 8 mm storage tape. The input device 18
`may alternatively include connections to other computer
`systems to receive the input commands and data therefrom.
`The first processing unit 12 may include a Visual interface
`for use in conjunction with the input device 18 and output
`device 20 similar to those screens illustrated in FIGS. 5—6,
`discussed below.
`It
`is also understood that alternative
`devices may be used to receive commands and data from the
`user, such as keyboards, mouse devices, and graphical user
`interfaces (GUI) such as WINDOWSTM 3.1 available form
`MICROSOFT Corporation, Redmond, Wash, and other
`operating systems and GUIs, such as OS/2 and OS/2 WARP,
`available from IBM CORPORATION, Boca Raton, Fla.
`Processing unit 12 may also include microphones and/or
`telephone handsets for receiving audio voice data and
`commands, speech or voice recognition devices, dual tone
`multi-frequency (DTMF) based devices, and/or software
`known in the art to accept voice data and commands and to
`operate the first processing unit 12.
`
`10
`
`20
`
`L)m
`
`30
`
`L»UI
`
`40
`
`50
`
`60
`
`6
`In addition, either of the first processing unit 12 and the
`second processing unit 22 may be implemented in a personal
`digital assistant (FDA) providing modem and E-mail capa-
`bilities and Internet access, with the PDA providing the
`input/output screens for mouse interactions or for touch-
`screen activation as shown, for example, in FIGS. 5—6, as a
`combination of the input device 18 and output device 20.
`For clarity of explanation, the illustrative embodiment of
`the disclosed point-to-point Internet protocol and system 10
`is presented as having individual functional blocks, which
`may include functional blocks labeled as “processor” and
`“processing unit”. The functions represented by these blocks
`may be provided through the use of either shared or dedi—
`cated hardware,
`including, but not
`limited to, hardware
`capable of executing software. For example, the functions of
`each of the processors and processing units presented herein
`may be provided by a shared processor or by a plurality of
`individual processors. Moreover, the use of the functional
`blocks with accompanying labels herein is not to be con-
`strued to refer exclusively to hardware capable of executing
`software. Illustrative embodiments may include digital sig-
`nal processor (DSP) hardware, such as the AT&T DSP16 or
`DSP32 C, read-only memory (ROM) for storing software
`performing the operations discussed below, and random
`access memory (RAM) for storing DSP results. Very large
`scale integration (VLSI) hardware embodiments, as well as
`custom VLSI circuitry in combination with a general pur-
`pose DSP circuit, may also be provided. Any and all of these
`embodiments may be deemed to fall within the meaning of
`the labels for the functional blocks as used herein.
`
`The processing units 12, 22 are capable of placing calls
`and connecting to other processing units connected to the
`Internet 24, for example, via dialup SLIP/TPP lines. In an
`exemplary embodiment, each processing unit assigns an
`unsigned long session number, for example, a 32-bit long
`se