`
`[19]
`
`[11] Patent Number:
`
`6,009,469
`
`Mattaway et al.
`
`[45] Date of Patent:
`
`Dec. 28, 1999
`
`US006009469A
`
`54] GRAPHIC USER INTERFACE FOR
`INTERNET TELEPHONY APPLICATION
`
`75]
`
`Inventors: Shane D. Mattaway, Boca Raton;
`Glenn W- Hutton» Miami; Craig 3-
`Strickland, Tamarac, all of Fla.
`
`73] Assignee: NetSpeak Corporation, Boca Raton,
`Fla.
`
`21] APPL N0-3 08/721,316
`
`22]
`
`Filed:
`
`Sep. 25, 1996
`
`63]
`
`60]
`
`-
`-
`Related U's' Apphcatwn Data
`Continuation—i11—part of application No. 08/533,115, Sep. 25,
`1995
`Provisional application No. 60/025,415, Sep. 4, 1996, and
`provisional application No. 60/024,251, Aug. 21, 1996.
`5
`
`.................................................... .. G06F 17/00
`Int. Cl.
`51]
`709/227
`U S C]
`7
`' """"""""""""""""""""""""""""""""""
`‘
`‘
`5-]
`58] Field of Search ....................... .. 395/200.57, 200.58;
`709/227, 228; 370/352, 355, 357, 389,
`395
`
`56]
`
`References Cited
`
`5,095,480
`5 150 360
`’
`’
`5
`
`5,224,095
`5,291,554
`
`.
`
`.
`
`.
`
`U5 PATENT DOCUMENTS
`3/1992 Fennel’.
`9/1992 Perlman et al.
`-
`11/1992 Rddl
`.
`4/1993 Dlorfeeet 31.
`6/1993 VVocst ct al.
`3/1994 Morales.
`E;1l°nI11a:’1t(::1'a'L .
`6/1994 MCMHICH et 31”
`5,321,813
`10/1994 Banwart .
`5;357;571
`3/1995 Yamada .
`5,400,335
`6/1995 Britton et a1..
`5,425,028
`7/1995 GQHOWQY ~
`5,430,709
`7/1995 Ca11F’11~
`5,430,727
`7/1995 Non‘) ‘
`5>432>846
`8/1995 Perkins et al’ ’
`5’442’633
`9/1995 Shimizu '
`5’452’296
`5,455,854 10/1995 Dilts et al. .
`5,457,683
`10/1995 Robins.
`5,457,738 10/1995 Sylvan .
`
`5,463,625
`5,465,286
`5,469,500
`55’:;g=’,7:H
`55093058
`5:517;494
`5,524,110
`5,524,141
`5,526,489
`5,533,110
`5,544,303
`5,546,582
`
`10/1995 Yasfebi ~
`11/1995 Clare 6191 ~
`.
`11/1995 Satter et al.
`'
`ft a1’
`4/1996 Sestak et al. .
`5/1995 (jreen .
`6/1996 Danneels et al.
`6/1996 Braun et al. .
`6/1996 Nilakantan et al. .
`7/1996 Pinard et al. .
`8/1996 Marotcaux ct al. .
`8/1996 Brockmeyer et al.
`
`.
`
`.
`
`FOREIGN PATENT DOCUMENTS
`
`European Pat. Off.
`11/1991
`A2 0445402
`European Pat. Off.
`8/1993
`A2 0550012
`W0 9219054 10/1992 W IPO '
`(JTHER PUBLICATIONS
`
`.
`.
`
`Intemetworking with TCP/IP, vol. 1, Second Edition, Prin-
`ciples, Protocols, and Architecture, by Douglas E. Comer.
`h
`.
`-
`7
`VocalTcc Intcrnct P onc (TM) Version 2.5, wWw.cox.sm-
`u.edu/class/m1s6386/people/stort/1phone25.eXe, Feb. 1995.
`Weinberg, Netscape Conference and Ce01ta1k Meeting
`Room, www.q5.com, Feb. 22, 1996.
`Gull, Re: Getting IP address of PPP—conncctcd Mac,
`<jgull—0304951005350001@pm012—11.dialip.mich.net>,
`Apr. 3, 1995.
`Mac'?,
`for
`Phone
`Internet
`Gull,
`Re:
`>jgu11—1704950116450001@pm049—28.dia1ip.mich.nct>,
`Apr. 17, 1995.
`.
`.
`.
`.
`Prtmarv Exammer—Ell1s B. Ramirez
`~
`,
`.
`Attorney, Agent, or Fzrm—Kud1rka & Jobse, LLP
`_
`ABSTRACT
`[371
`A communication utility for establishing real-time, point-
`to-point communications between processes over a com-
`puter network includes apparatus for querying a server as to
`the network protocol address of another client process, and
`apparatus for directly establishing a communication link
`with the client process upon receipt of the network protocol
`address from the server. In one embodiment,
`the utility
`includes a sophisticated user interface having features simi-
`lar to t
`ical tele hon hardware but im lementin
`reater
`flexibillltly with sd)ftwa)r,e
`P
`g g
`‘
`
`18 Claims, 27 Drawing Sheets
`
`T
`
`24
`12-\
`|l_——|7N§1'Ei_-“\’i:1E‘iT"2;"""
`;"F“..;;T'p,;o'e‘E;.;.'N2;n'n'.;“““““‘i
`1|
`r
`1
`PROCESSOR
`‘NWT
`1
`I
`POP 3
`2
`DEVICE
`{
`:
`’ MAILSERVER
`} 14_x
`18_x
`:
`I
`I
`,
`
`'
`I
`:'
`MEMORY
`L glé\T/59;? 471* E
`26
`[
`: 164
`20-)‘
`g
`I
`L _______________________ _ _.
`;
`CONNECTION SERVER
`|_i_, PROCESSOR
` _
`\— 3o
`TIMER
`32
`DATABASE
`
`
`22\
`PRSOEC(IE%';li3NG
`“N”
`
`Page 1 of 67
`
`LG Electronics Exhibit 1001
`
`
`
`U.S. Patent
`
`Dec.28, 1999
`
`Sheet 1 of 27
`
`6,009,469
`
`ozoomm
`
`oz_wmmoomn_
`
`:2:
`
`
`
`mm>mmw.__<_>_mn_On_
`
`
`
`mm_>mm_mzocbmzzoo
`
`N.©~n~
`
`Page 2 of 67
`
`
`
`
`
`
`U.S. Patent
`
`Ezmm:z_
`
`__________.
`
`9_W_1_mm,_K“
`
`
`5.mm>mmw4_<_2
`
`fimmaommfiomzzoov
`
`6,009,469
`
`mm
`
`ozoomm
`
`oz_mmmoomn_
`
`C23
`
`m
`
`_
`
`70.mozoommma
`a>mm>mmjoa%a----
`_H
`__
`¢_o5mzzoov
`
`______._—._______
`
`Page 3 of 67
`
`
`
`
`
`
`U.S. Patent
`
`Dec. 28, 1999
`
`Sheet 3 of 27
`
`6,009,469
`
`
`
`_._<_._o_oommmmmmzoovn
`
`
`oz_mmm_ooEozammoom
`ozoomwAo_o:<_Bx:_
`
`._.mZmm_._.Z_
`
`:2::2:
`
`NN
`
`__‘|''|IIL
`§I\~N.
`
`ozoommBx:
`
`
`
`oz_wmmooEoz_wmmoon_
`
`
`
`:2:.:2:
`
`Page 4 of 67
`
`
`
`
`
`U.S. Patent
`
`Dec. 28, 1999
`
`Sheet 4 of 27
`
`6,009,469
`
`f 36
`
`40
`
`ROGER
`
`91.103.001.027
`
`38
`
`0343
`C2
`O IN USE
`
`13
`
`
`ANE
`.201.00.137
`0024
`
`13
`
`L1
`
`O IDLE
`
`42
`
`
`
`
`IE
`
`
`
`44
`
`
`IIJE
`
`48
`
`50
`
`
`
`Page 5 of 67
`
`
`
`U.S. Patent
`
`Dec. 28, 1999
`
`Sheet 5 of 27
`
`6,009,469
`
`START THE POINT—TO—POINT
`
`
`
`INTERNET PROTOCOLS
`
`
`54
`
`
`
`INITIATE PRIMARY INTERNET
`
`PROTOCOL
`
`IS THE CONNECTION SERVER IN A
`
`RESPONSIVE CONDITION?
`
`56
`
`58
`
`
`
`NO
`
`62
`
`INITIATE SECONDARY
`INTERNT PROTOCOL
`
`Y
`
`ES
`
`
`
`PERFORM PRIMARY
`
`INTERNET PROTOCOL
`
`FIG. 7
`
`60
`
`64
`
`68
`
`START THE PRIMARY
`
`POINT—TO—PO|NT INTERNET
`PROTOCOL
`
`TIMESTAMP AND STORE E-MAIL
`
`
`
`
`
` 66
`ADRESSES AND IP ADRESSES OF
`
`LOGGED-IN UNITS IN A DATABASE
`
`RECEIVE QUERY FROM FIRST UNIT
`
`
`
`WHETHER A SPECIFIED SECOND
`UNIT IS LOGGED-IN
`
`
`
`RETRIEVE IP ADDRESS FROM
`
`DATABASE IF THE SECOND UNIT IS
`
`
`
`72
`
` 70
`LOGGED-IN
`
`
`
`SEND RETRIEVED IP ADRESS TO
`
`
`FIRST UNIT TO ESTABLISH POINT-
`
`TO—POINT CONNECTION
`
`FIG. 8
`
`Page 6 of 67
`
`
`
`U.S. Patent
`
`Dec. 28, 1999
`
`Sheet 6 of 27
`
`6,009,469
`
`START THE SECONDARY
`POINT—TO POINT INTERNET PROTOCOL
`
`GENERATE AN E-MAIL SIGNAL,
`INCLUDING A SESSION NUMBER AND A
`
`FIRST PROCESSING UNIT
`
`FIRST IP ADRESS CORRESPONDING TO A
`
`TRANSMIT THE E-MAIL SIGNAL AS A
`
`<CONNECTREQUEST> SIGNAL
`TO THE INTERNET
`
`DELIVER THE E-MAIL SIGNAL THROUGH
`
`THE INTERNET USING A MAIL SERVER TO
`
`A SECOND PROCESSING UNIT
`
`
`
`
`
`EXTRACT THE SESSION NUMBER AND
`THE FIRST IP ADRESS FROM THE E-MAIL
`SIGNAL
`
`74
`
`75
`
`78
`
`80
`
`82
`
`84
`
`86
`
`88
`
`
`
`SEND THE SESSION NUMBER AND A
`
`
`
`SECOND IP ADRESS CORRESPONDING
`
`TO THE SECOND PROCESSING UNIT TO
`
`THE FIRST PROCESSING UNIT THROUGH
`THE INTERNET
`
`
`
`.
`-
`VERIFY THE SESSION NUMBER
`RECEIVED FROM THE SECOND
`PROCESSING UNIT
`
`
`
`
`ESTABLISH A POINT—TO—POINT INTERNET
`COMMUNICATION LINK BETWEEN THE
`
`FIRST AND SECOND PROCESSING UNITS
`
`
`
`
`
`USING THE FIRST AND SECOND IP
`ADDRESSES
`
`
`
`
`
`FIG. 9
`
`Page 7 of 67
`
`
`
`U.S. Patent
`
`Dec. 28, 1999
`
`Sheet 7 of 27
`
`82
`
`fie_.
`
`coo.
`
`v_8..__mm.z
`
`
`
`._w>._ww_mno_G
`
`_m.EoE_
`
`mo_>._mw
`
`.mu_>E..._
`
`Page 8 of 67
`
`2939$
`
`
`
`
`6,009,469
`
`92F5.moan:hF0L=m..l
`
`0:VI
`
`on.o-.n.o---.-u.-
`
`_.mm._.mm?
`
`72:nmmjowfizoo
`OmD_><_>_DDZ<Om<Om>m_v_m_m._._O~.._._.ZOOw.ommE<n_<mm.:oE.zoommjomhzoomw:os_%ZO_._.<O_ZD_>=>_OO
`
`
`
`
`E:mmm_»z_
`
`
`
`U.S. Patent
`
`%
`
`our
`
`2.xmam:smPm>mmfiM9;oi
`
`O09mm._._oEzoommjomhzoos_<m
`%om.mm.:oEzoowmmjompzooEmam
`
`
`
`
`0:mmjomhzoo
`
`
`
`mm?
`
`B_
`
`)0
`
`N3nisx.
`
`
`
`
`
`m_>_mov_m_oomxmm>_moEOE00
`
`81.
`
`em.
`
`Er
`
`Tmmjoyfizoo
`
`>mo2m2
`
`Page 9 of 67
`
`
`
`
`
`
`
`
`U.S. Patent
`
`Dec. 28, 1999
`
`Sheet 9 of 27
`
`6,009,469
`
`
` 1214
`
`1220
`
`OUTPUT
`BUFFER
`
`1230
`
`BUS
`INTERFACE
`
`1216
`
`INPUT
`BUFFER
`
`
`
`FIGURE 12 (PRIOR ART)
`
`Page 10 of 67
`
`
`
`U.S. Patent
`
`999100:
`
`72ml.001MehS
`
`6,009,469
`
`2_:mm.sou:_>>w_:mm_sou:_>>-xD_:m_._moE_oms_
`
`22
`
`Page 1 1 of 6?
`
`;moE_oas_
`
`m26u:_>>
`
`x_z:
`
`<.22:9...
`
`
`
`U.S. Patent
`
`Dec. 28, 1999
`
`Sheet 11 0127
`
`6,009,469
`
`wAp|
`
`1328
`
`process
`
`1334
`
`Page 12 of 67
`
`network
`
`FIGURE 13 B
`
`
`
`U.S. Patent
`
`Dec. 28, 1999
`
`Sheet 12 of 27
`
`6,009,469
`
`1400
`
`1402A
`
`14023
`
`14020
`1402F
`1402J
`
`1406
`
`1408
`
`1402
`
`1402E
`
`14020
`14029
`1402“
`
`1404
`
`
`BillBill
`
`
`
`
`
`£1? ’W@Er9@£fn©Lm@
`
`X
`
`Figure 14
`
`Page13of6T
`
`
`
`U.S. Patent
`
`Dec. 28, 1999
`
`Sheet 13 of 27
`
`6,009,469
`
`1500
`
`
` DATABASE
`1516B
`l1516C
`l1516D
`i1516E
`
`DATABASE
`SEARCH
`ENGNE
`
`1 515F
`
`CONNECTION
`
`INFORMATION
`
`SERVER SEE
`
`
`DATABASE
`SERVER
`
`NETWORK INTERFACE CARD
`
`1520
`
`1532
`
`I @
`
`1536
`
`1530
`
`1533
`
`I @ 1534
`
`FIREWALL SERVER
`
`ROUTER
`
`1522
`
`‘524
`
`csulosu
`
`1526
`
`@ 1533
`
`Figure 15A
`
`Page 14 of 6?
`
`
`
`U.S. Patent
`
`Dec. 28, 1999
`
`Sheet 14 of 27
`
`6,009,469
`
`DATABASE
`
`DATABASE
`SEARCH
`ENGINE
`
`COSNENRECTION
`VER
`
`INFORMATION
`SERVER
`
`DATABASE
`SERVER
`
`I @
`
`1530
`
`
`
`1534
`
`Figure 15B
`
`Page 15 of 67
`
`
`
`U.S. Patent
`
`Dec. 28, 1999
`
`Sheet 15 of 27
`
`6,009,469
`
`8.:Q
`
`$23
`
`N¢m_.Q
`
`noomw
`
`Sam.
`
`mocfl
`
`xmuamuoz
`
`
`
`.o>._om.320
`
`xmanmflmz
`
`
`
`.w>._wm_mno_w
`
`
`
`._o>._mm.320
`
`._mwnwaoz%
`
`Page 16 of 67
`
`camp
`
`32
`
`@
`
`xmwnmfiz
`
`
`
`._m>._mw_mno__@
`
`<8—.
`
`cm.2:9...
`
`
`
`U.S. Patent
`
`Dec. 28, 1999
`
`Sheet 16 of 27
`
`6,009,469
`
`am<cmmm_a
`
`nmuzooummzn
`
`<._.<D
`
`<2Ezmt
`
`
`
`ACMEpomzzoov
`
`n..
`
`om>_mumm
`
`
`
`.__<s_-m»o<mbm_
`
`mmmmoo<
`
`
`
`.__<_2-mn_I<>>m0u_
`
`
`
`mw<m<»<aO...mmumoa<
`
`azmm
`
`
`
`Av_U<PUMZZOUV
`
`
`
`EOEENazmm
`
`E0mo<mmm_2
`
`
`
`AwZ_._n_u_Ov
`
`«mm:
`
`Page 17 of 67
`
`
`
`
`
`
`
`
`
`
`U.S. Patent
`
`Dec. 28, 1999
`
`Sheet 17 of 27
`
`6,009,469
`
`START
`
`
`
`
`
`
`<lNFO‘REQ>
`RECEIVED
`?
`
`YES
`
`DISGARD
`
`UNRECOGNIZED
`DATA
`
`1635
`
`EXTRACT SEARCH
`CRITERIA
`
`SUPPLY SEARCH
`CRITERIA TO DATABASE
`
`SEND
`<lNFO ACK>
`
`1634
`
`1636
`
`1638
`
`<|NFO ABORT
`RECEIVED
`
` YES
`
`
`
` 1644
`
`DD
`
`SEND <lNFO>
`
`NO
`
`1632
`
`NO
`
`POWER OFF
`7
`
`was
`
`W Figure 16B
`
`Page 18 of 67
`
`
`
`U.S. Patent
`
`Dec. 28, 1999
`
`Sheet 18 of 27
`
`6,009,469
`
`$5mm._._<ovU:
`
`
`
`¢_o<mm._._<ovQ:
`
`E.m:=m.E
`
`mam—Smr
`
`;uJmn.::ozu.1
`
`A._._<0va
`
`fiomwmmv<m
`
`A>m:mvmm
`
`E0
`
`
`
`A_._U<_2mEsmz<vomE0
`
`
`
`Av_U<.._._<Ov8E0
`
`zmzp
`
`A..m=smz<v2.
`
`Ao_n_:<v<3
`
`AO_DD<vm:
`
`AozmvNw
`
`gmmmzozm
`
`
`
`AcmmO..._2_mmmavN
`
`Aommmz_._zov_.
`
`Azo:<Ew_ummv...
`
`
`
`AOn_Z_mmmavm
`
`
`
`0.9..mz_._zovm
`
`
`
`Aommpomzzoovo
`
`
`
`Av.O<Bmzzoov<5
`
`
`
`AmZ_._n_u_Ovmh
`
`O4Om<4 mmm>mm
`
`Page 19 of 67
`
`
`
`
`
`
`
`
`
`
`U.S. Patent
`
`Dec. 28, 1999
`
`Sheet 19 of 27
`
`6,009,469
`
`1536
`
`;Lumn.::::OzLu
`
`ABORT>
`3C<|NFO> on4<|NFO
`2<|NFOACK>
`1<lNFOREQ>
`
`3B<|NFO>
`3A<INFO>
` 1500
`
`L'J...lOm<_l
`
`(DllJD'Z>l.LlE
`
`
`
`Figure17B
`
`Page 20 of 67
`
`
`
`U.S. Patent
`
`2
`
`.mhS
`
`6,009,469
`
`
`
`
`
`
`
`
`
`..uE9_mn_.:o_.8__oo_:Em._mn_.co_.8__oo_:m:__n_w,._o=:oo_:Mm_ceu__..o..8.3o_§:_>_:m:__.___m._2Eoo_:c33>95.
`
`
`
`
`
`
`
`
`
`
`
`W.=o_.8__oo_:_o=:ou_:_2Eoo_:w
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`_m:t_>Hco:_m>~o..o9.2_m:t_>WAo:_m>.swzm:o=o:_m>>>wz3aUo:_a>noonoo,Aw:_m>.swzm:o:m:_m>.mwEon_m:t_>
`
`
`
`M..8.3o_~_E_>_:_eEoo_m::_>_:
`
`82on:mm_
`
`
`
`mm:D.tocm
`
`
`
`o..._>htocm
`
`$mm_§t_>_:
`
`Page 21 of 6?
`
`wear2::Namw
`
`<2m::m.E
`
`
`
`
`
`U.S. Patent
`
`Dec. 28, 1999
`
`Sheet 21 of 27
`
`6,009,469
`
`1804
`
`
`
`1828
`
`1822
`
`Page 22 of 67
`
`UIComboBox
`
`
`
`
`1814
`
`Figure 18B
`
`
`
`GUIContro|* Sibling
`UlCo||ection* Parent
`
`
`
`
`
`U.S. Patent
`
`Dec. 28, 1999
`
`Sheet 22 of 27
`
`6,009,469
`
`Figure 180
`
`
`
`is a
`
`UlPushButton
`
`UlP|ayRun
`
`UlToggle
`
`
`
`
`
`1342
`
`1844
`
`1846
`
`Ulcollection
`
`GUlVirtualObject* Childrenl]
`UlCol|ection* Parent
`
`
`
`Figure 18D
`
`1806
`
`void Show_()
`
`void Hide()
`
` 1834
`
`1832
`
`Page 23 of 67
`
`
`
`U.S. Patent
`
`Dec. 28, 1999
`
`Sheet 23 of 27
`
`6,009,469
`
`
`
`Illll.o__2_a:o_=o€,...__a3._o_...s_..
`
`
`
`
`
`um“m.Sm.E22
`
`$_.§_.
`
`
`
`9.3..mm.0.39....
`
`
`5_n_mo:mm<2:.o._o_-
`5_._m:m<o~_mo___Ea
`
`
`
`
`
`o_s_2._:m<5.3.:llll
`
`o_s_m<2:5:._o_.>§_.
`
`Ev_oom:m<5%o€&%<
`
`
`
`m.__n_nm<mas.o>:mn_o>o:_mm
`
`ofisa
`
`
`
`oxmflo._mm.rm>oEom
`
`c._wo._.2mu._o
`
`<2..u:=m.....
`
` |||J
`
`63036no_>m._._a9.:
`
`
`
`~mEo._mucflxoo>tmn_zv_mm._.~:o..
`
`
`
`
`
`
`
`
`
`
`ms_mm:-m<29.».oom>:&$_:a.,._c
`=3...2:3._£#.__..o=mom>oEmm
`
`32.N:H_>tmn.o__moo.mm:o
`
`
`mmo._omm<
`
`:2.mi0DAnglvG_uEuxmfi3.3an...m:__
`
`m_2._.E:m<an.938Seano...
`
`Efimm<use«Em229¢
`
`
`n_o._.wmm<o._m:o_wmmm25couflzu
`
`Page 24 of 67
`
`
`
`
`U.S. Patent
`
`Dec. 28, 1999
`
`Sheet 24 of 27
`
`6,009,469
`
`2004
`
`2006
`
`SOCKET E
`
`NO
`
`
`
`SPEAKER
`
`0
`
`2 14
`0
`
`SOCKET
`
`
`
`WRITE TO
`spsmcen
`wnm: TO
`
`FILE HANDLE
`
`
`
`LOOP FOR ALL PARTIES
`
`WRITE TO
`PARTY[1].SOCKET
`
`0 Q
`
`
`
`IS
`
`pAR1'y[1]
`
`QNUNE?
`
`2023
`m
`
`
`
`PLACE EMPTY
`
`BUFFER N
`MIC SAMPSNG
`QUEUE
`
`2034
`
`
`
`“"0
`ENABLED
`7
`
`2032
`
`2030
`
`SOCKET
`
`MIC
`
`PLACE EMPTY
`BUFFER ON
`
`Winsock QUEUE
`
`
`PLACE EMPTY
`
`BUFFER IN
`"free" POOL
`
`
`
`2036
`
`2038
`
`Figure 20A
`
`Page 25 of 67
`
`
`
`U.S. Patent
`
`Dec. 28, 1999
`
`w0.,
`
`w
`
`Eflnoo_Em_na:m_M\mm;M.,..xm- tam.:o:>m<mm...
`mommm
`
`
`
`
`
`
`
`4,man...33......
`
`6«wow8888
`
`
`
`
`
`o_EE9:.._ooq..E9:m=__.E_$
`
`
`
`umo.o:>w<
`
`
`
`xoow:_>>EO._u
`
`
`
`mmEEmfio
`
`
`
`.__oon_..E9:
`
`$8_m:._3:_02ESEN
`
`
`
` 2.8$8@o___u_.__
`
`88
`
`._.m_<._.w
`
`Page 26 of 67
`
`
`
`
`
`U.S. Patent
`
`Dec. 28, 1999
`
`Sheet 26 of 27
`
`6,009,469
`
`xoowc_>>:_
`
`5:32mmEmomiumxoom.550
`
`
`
`
`
`:_°oQ=00‘;
`
`
`
`
`.53mm._=nEwen.omom0
`mme.05.c_mm_> S8
`
`
`
`
`___o8___mm...
`
`Sam._9_mmn_m2:
`
`newm§m.E9
`
`V.
`
`222.5
`
`.oxmonm
`
`02cam.
`
`-2:E9:
`
`0
`uoxoom
`
`._3_amam38
`
`wmcm
`
`mmomemom
`
`._.m<._.m
`
`Page 27 of 67
`
`
`
`
`U.S. Patent
`
`Dec. 28, 1999
`
`Sheet 27 of 27
`
`6,009,469
`
`Discard pending
`data in mix
`
`buffer
`
`Page 28 of 67
`
`Figure 20D
`
`
`
`6,009,469
`
`1
`GRAPHIC USER INTERFACE FOR
`INTERNET TELEPHONY APPLICATION
`
`RELATED APPLICATIONS
`
`This application is a continuation-in-part of U.S. patent
`application Ser. No. 08/533,115 (Pending) entitled Point—to—
`Point Internet Protocol, by Glenn W. Hutton, filed Sep. 25,
`1995, commonly assigned, the subject matter of which is
`incorporated herein by reference.
`To the extent that any matter contained herein is not
`already disclosed in the above-identified parent application
`a location claims priority to U.S. provisional patent appli-
`cation 60/025,415 entitled Internet Telephony Apparatus and
`Method by Mattaway et al., filed Sep. 4, 1996, and U.S.
`provisional patent application Ser. No. 60/024,251 entitled
`System and Methods for Point-To-Point Communications
`Over a Computer Network, by Mattaway et al., filed Aug.
`21, 1996.
`In addition, this application is one of a number of related
`applications filed on an even date herewith and commonly
`assigned,
`the subject matters of which are incorporated
`herein by reference, including the following:
`U.S. patent application Ser. No. 08/719,894, entitled
`Directory Server For Providing Dynamically Assigned
`Network Protocol Addresses, by Mattaway et al.;
`U.S. patent application Ser. No. 08/719,554, entitled
`Point-to-point Computer Network Communication
`Utility Utilizing Dynamically Assigned Network Pro-
`tocol Addresses, by Mattaway et al.;
`U.S. patent application Ser. No. 08/719,640, entitled
`Metl1odAnd Apparatus For Dynamically Defining Data
`Communication Utilities, by Mattaway et al.;
`U.S. patent application Ser. No. 08/719,891, entitled
`Method And Apparatus For Distribution And Presen-
`tation Of Multimedia Data Over A Computer Network,
`by Mattaway et al.;
`U.S. patent application Ser. No. 08/719,898, entitled
`Method And Apparatus For Providing Caller Identifi-
`cation Based Out-going Messages In A Computer Tele-
`phony Environment, by Mattaway et al.;
`U.S. patent application Ser. No. 08/718,911, entitled
`Method And Apparatus For Providing Caller Identifi-
`cation Based Call Blocking In A Computer Telephony
`Environment, by Mattaway et al.; and
`US. patent application Ser. No. 08/719,639, entitled
`Method And Apparatus For Providing Caller Identifi-
`cation Responses In A Computer Telephony
`Environment, by Mattaway et al.
`FIELD OF THE INVENTION
`
`The present invention relates, in general, to data process-
`ing systems, and more specifically, to a method and appa-
`ratus for facilitating audio communications over computer
`networks.
`
`BACKGROUND OF THE INVENTION
`
`The increased popularity of on-line services such as
`AMERICA ONIINETM, COMPUSERVE®, and other ser-
`vices such as Internet gateways have spurred applications to
`provide multimedia,
`including video and voice clips,
`to
`online users. An example of an online voice clip application
`is VOICE E—MAIL FOR WINCIM and VOICE E—MAIL
`FOR AMERICA ONLINETM, available from Bonzi
`Software. as described in “Simple Utilities Send Voice
`
`10
`
`15
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`E-Mail Online”, MULTIMEDIA WORLD, VOL. 2, NO. 9,
`Aug. 1995, p. 52. Using such Voice E-Mail software, a user
`may create an audio message to be sent to a predetermined
`E-mail address specified by the user.
`Generally, devices interfacing to the Internet and other
`online services may communicate with each other upon
`establishing respective device addresses. One type of device
`address is the Internet Protocol (IP) address, which acts as
`a pointer to the device associated with the IP address. A
`typical device may have a Serial Line Internet Protocol or
`Point-to-Point Protocol (SLIP/PPP) account with a perma-
`nent IP address for receiving E-mail, voicemail, and the like
`over the Internet. E-mail and voicemail is generally intended
`to convey text, audio, etc., with any routing information
`such as an IP address and routing headers generally being
`considered an artifact of the communication, or even gib-
`berish to the recipient.
`Devices such as a host computer or server of a company
`may include multiple modems for connection of users to the
`Internet, with a temporary IP address allocated to each user.
`For example,
`the host computer may have a general IP
`address “XXX.XXX.XXX,” and each user may be allocated
`a successive IP address of XXX.XXX.XXX.10,
`XXX.XXX.XXX.11, XXX.XXX.XXX.12, etc. Such tem-
`porary IP addresses may be reassigned or recycled to the
`users, for example, as each user is successively connected to
`an outside party. For example, a host computer of a company
`may support a maximum of 254 IP addresses which are
`pooled and shared between devices connected to the host
`computer.
`Permanent IP addresses of users and devices accessing the
`Internet readily support point-to-point communications of
`voice and video signals over the Internet. For example,
`real-time video teleconferencing has been implemented
`using dedicated IP addresses and mechanisms known as
`reflectors. Due to the dynamic nature of temporary IP
`addresses of some devices accessing the Internet, point-to-
`point communications in real-time of voice and video have
`been generally difficult to attain.
`The ability to locate users having temporary or dynami-
`cally assigned Internet Protocol address has been dillicult
`without
`the user manually initiating the communication.
`Accordingly, spontaneous, real-time communications with
`such users over computer networks have been impractical.
`Further, it is desirable to have a communication utility which
`contains familiar features and functions to current commu-
`nication utility such as telephones and cellular telephones. It
`is even further desirable to utilize the current graphic user
`interface technology associated with computer software in a
`manner to achieve a more flexible interface to a such a
`communication utility, without the limitations associated
`with hardware.
`
`Accordingly, a need exists for a way to determine whether
`computer users are actively connected to a computer net-
`work.
`
`A further need exists for a way to obtain the dynamically
`assigned Internet Protocol address of a user having on-line
`status with respect to a computer network, particularly the
`Internet.
`
`An even further need exists for a method and apparatus by
`which to establish real-time, point-to-point communications
`over a computer network using a communication utility
`having an interface which combines the familiar aspects of
`current hardware communication utilities but which allows
`
`for the flexibility associated with graphic user interfaces.
`SUMMARY OF THE INVENTION
`
`The above deficiencies in the prior art and previously
`described needs are fulfilled by the present invention which
`
`Page 29 of 67
`
`
`
`6,009,469
`
`3
`provides a virtual communications utility displayable on
`computer system interfaces which enables real-time, point-
`to-point communications over computer networks. Accord-
`ing to one embodiment of the present invention, a computer
`program product for use with a computer system having a
`display and an audio transducer comprises a computer
`usable medium having computer readable code means
`embodied therein comprising program code means for gen-
`erating a 11ser interface, program code means responsive to
`user input commands for establishing a point-to-point com-
`munication link with another computer over a network and
`program code means responsive to audio data from the audio
`transducer for transmitting the audio data over the commu-
`nication link.
`
`According to another embodiment of the present
`invention, a computer program product for use with a
`computer system comprises a computer usable medium
`having computer readable program code means embodied
`thereon comprising code means for transmitting from a
`client process to a server a query as to whether a second
`client process is connected to the computer network, pro-
`gram code means for receiving the network protocol address
`of the second process from the server, and program code
`means responsive to the network protocol address of the
`second client process for establishing a point-to-point com-
`munication link between the first client process and the
`second client process.
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`The features of the invention will become more readily
`apparent and may be better understood by referring to the
`following detailed description of an illustrative embodiment
`of the present
`invention,
`taken in conjunction with the
`accompanying drawings, in which:
`FIG. 1 illustrates, in block diagram format, a system for
`the disclosed point-to-point Internet protocol;
`FIG. 2 illustrates, in block diagram format, the system
`using a secondary point-to-point Internet protocol;
`FIG. 3 illustrates, in block diagram format, the system of
`FIGS. 1-2 with the point-to-point Internet protocol estab-
`lished;
`FIG. 4 is another block diagram of the system of FIGS.
`1-2 with audio communications being conducted;
`FIG. 5 illustrates a display screen for a processing unit;
`FIG. 6 illustrates another display screen for a processing
`unit;
`FIG. 7 illustrates a flowchart of the initiation of the
`point-to-point Internet protocols;
`FIG. 8 illustrates a flowchart of the performance of the
`primary point-to-point Internet protocols;
`FIG. 9 illustrates a flowchart of the performance of the
`secondary point-to-point Internet protocol;
`FIG. 10 illustrates schematically a computer network over
`which the present invention may be utilized;
`FIG. 11 is a block diagram of a computer system suitable
`for use with the present invention;
`FIG. 12 is a block diagram of an audio processing card
`suitable for use with the computer system of FIG. 10;
`FIGS. 13 A—B are schematic block diagrams of the
`elements comprising the inventive computer network tele-
`phony mechanism of the present invention;
`FIG. 14 is a screen capture illustrating an exemplary user
`interface of the present invention;
`FIG. 15 is a schematic diagram illustrating the architec-
`ture of the connection server apparatus suitable for use with
`the present invention;
`
`10
`
`15
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`4
`illustrating the process steps
`FIG. 16A is a flowchart
`performed by the connection server in accordance with the
`present invention;
`illustrating the process steps
`FIG. 16B is a flowchart
`performed in accordance with the information server of the
`present invention;
`FIGS. 17A—B are schematic block diagrams illustrating of
`the packet transfer sequence in accordance with the com-
`munication protocol of the present invention;
`FIGS. 18A—D are conceptual block diagrams illustrating
`user interface and graphic user interface objects utilized by
`the communication utility of the present invention;
`FIGS. 19A—C are conceptual block diagrams illustrating
`the event manager and media engine objects utilized by the
`communication utility of the present invention; and
`FIGS. 20A—T) illustrate process steps performed by the
`media engine function of the communication utility in
`accordance with the present invention.
`
`DETAILED DESCRIPTION
`
`Referring now in specific detail to the drawings, with like
`reference numerals identifying similar or identical elements,
`as shown in FIG. 1,
`the present disclosure describes a
`point-to-point network protocol and system 10 for using
`such a protocol.
`In an exemplary embodiment, the system 10 includes a
`first processing unit 12 for sending at least a voice signal
`from a first user to a second user. The first processing unit
`12 includes a processor 14, a memory 16, an input device 18,
`and an output device 20. The output device 20 includes at
`least one modem capable of, for example, l4.4 Kilobit-per-
`second communications and operatively connected via
`wired and/or wireless communication connections to the
`Internet or other computer networks such as an Intranet, i.e.,
`a private computer network. One skilled in the art would
`understand that the input device 18 may be implemented at
`least in part by the modem of the output device 20 to allow
`input signals from the communication connections to be
`received. The second processing unit 22 may have a
`processor, memory, and input and output devices, including
`at
`least one modem and associated communication
`
`connections, as described above for the first processing unit
`12. In an exemplary embodiment, each of the processing
`units 12, 22 may execute the WEBPHONE® Internet tele-
`phony application available from NetSpeak Corporation,
`Boca Raton, Fla., which is capable of performing the dis-
`closed point-to-point Internet protocol and system 10, as
`described herein.
`
`The first processing unit 12 and the second processing
`unit 22 are operatively connected to the Internet 24 by
`communication devices and software known in the art, such
`as an Internet Service Provider (ISP) or an Internet gateway.
`The processing units 12, 22 may be operatively intercon-
`nected through the Internet 24 to a connection server 26, and
`may also be operatively connected to a mail server 28
`associated with the Internet 24.
`
`The connection server 26 includes a processor 30, a timer
`32 for generating time stamps, and a memory such as a
`database 34 for storing, for example, E-mail and Internet
`Protocol (IP) addresses of logged-in units. In an exemplary
`embodiment, the connection server 26 may be a SPARC 5
`server or a SPARC 20 server, available from SUN
`MICROSYSTEMS, INC., Mountain View, Calif., having a
`central processing unit (CPU) as processor 30, an operating
`system (OS) such as UNIX, for providing timing operations
`
`Page 30 of 67
`
`
`
`6,009,469
`
`5
`such as maintaining the timer 32, a hard drive or fixed drive,
`as well as dynamic random access memory (DRAM) for
`storing the database 34, and a keyboard and display and/or
`other input and output devices (not shown in FIG. 1). The
`database 34 may be an SQL database available from
`ORACLE or INFORMIX.
`
`In an exemplary embodiment, the mail server 28 may be
`implemented with a Post Oflice Protocol (POP) Version 3
`mail server and the Simple Mail Transfer Protocol (SMTP),
`including a processor, memory, and stored programs oper-
`ating in a UNIX environment, or, alternatively, another OS,
`to process E-mail capabilities between processing units and
`devices over the Internet 24.
`
`In the illustrative embodiment, the POP protocol is uti-
`lized to retrieve E-mail messages from mail server 28 while
`the SMTP protocol is used to submit E-mail message to
`Internet 24.
`
`The first processing unit 12 may operate the disclosed
`point—to—point Internet protocol by a computer program
`described hereinbelow in conjunction with FIG. 6, which
`may be implemented from compiled and /or interpreted
`source code in the C++ programming language and which
`may be downloaded to the first processing unit 12 from an
`external computer. The operating computer program may be
`stored in the memory 16, which may include about 8 MB
`RAM and/or a hard or fixed drive having about 8 MB of
`available memory. Alternatively, the source code may be
`implemented in the first processing unit 12 as firmware, as
`an erasable read only memory (EPROM), etc. It is under-
`stood that one skilled in the art would be able to use
`
`programming languages other than C++ to implement the
`disclosed point-to-point network protocol and system 10.
`The processor 14 receives input commands and data from
`a first user associated with the first processing unit 12 though
`the input device 18, which may be an input port connected
`by a wired, optical, or a wireless connection for electromag-
`netic transmissions, or alternatively may be transferable
`storage media, such as floppy disks, magnetic tapes, com-
`pact disks, or other storage media including the input data
`from the first user.
`
`The input device 18 may include a user interface (not
`shown) having, for example, at least one button actuated by
`the user to input commands to select from a plurality of
`operating modes to operate the first processing unit 12. In
`alternative embodiments, the input device 18 may include a
`keyboard, a mouse, a touch screen, and/or a data reading
`device such as a disk drive for receiving the input data from
`input data files stored in storage media such as a fioppy disk
`or, for example, an 8 mm storage tape. The input device 18
`may alternatively include connections to other computer
`systems to receive the input commands and data therefrom.
`The first processing unit 12 may include a visual interface
`for use in conjunction with the input device 18 and output
`device 20 similar to those screens illustrated in FIGS. 5-6,
`discussed below.
`It
`is also understood that alternative
`
`devices may be used to receive commands and data from the
`user, such as keyboards, mouse devices, and graphical user
`interfaces (GUI) such as WINDOWSTM 3.1 available form
`MICROSOFT Corporation, Redmond, Wash., and other
`operating systems and GUIs, such as OS/2 and OS/2 WARP,
`available from IBM CORPORATION, Boca Raton, Fla.
`Processing unit 12 may also include microphones and/or
`telephone handsets for receiving audio voice data and
`commands, speech or voice recognition devices, dual tone
`multi—frequency (DTMF) based devices, and/or software
`known in the art to accept voice data and commands and to
`operate the first processing unit 12.
`
`10
`
`15
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`6
`In addition, either of the first processing unit 12 and the
`second processing unit 22 may be implemented in a personal
`digital assistant (PDA) providing modem and E-mail capa-
`bilities and Internet access, with the PDA providing the
`input/output screens for mouse interactions or for touch-
`screen activation as shown, for example, in FIGS. 5-6, as a
`combination of the input device 18 and output device 20.
`For clarity of explanation, the illustrative embodiment of
`the disclosed point-to-point Internet protocol and system 10
`is presented as having individual functional blocks, which
`may include functional blocks labeled as “processor” and
`“processing unit”. The functions represented by these blocks
`may be provided through the use of either shared or dedi-
`cated hardware,
`including, but not limited to, hardware
`capable of executing software. For example, the functions of
`each of the processors and processing units presented herein
`may be provided by a shared processor or by a plurality of
`individual processors. Moreover, the use of the functional
`blocks with accompanying labels herein is not to be con-
`strued to refer exclusively to hardware capable of executing
`software. Illustrative embodiments may include digital sig-
`nal processor (DSP) hardware, such as the AT&T DSP16 or
`DSP32 C, read-only memory (ROM) for storing software
`performing the operations discussed below, and random
`access memory (RAM) for storing DSP results. Very large
`scale integration (VLSI) hardware embodiments, as well as
`custom VLSI circuitry in combination with a general pur-
`pose DSP circuit, may also be provided. Any and all of these
`embodiments may be deemed to fall within the meaning of
`the labels for the functional blocks as used herein.
`
`The processing units 12, 22 are capable of placing calls
`and connecting to other processing units connected to the
`Internet 24, for example, via dialup SLIP/PPP lines. In an
`exemplary embodiment, each processing unit assigns an
`unsigned long session number, for example, a 32-bit long
`sequence in a *.ini file for each call. Each call may be
`assigned a successive session number in sequence, which
`may be used by the respective processing unit to associate
`the call with one of the SLIP/PPP lines,
`to associate a
`<ConnectOK> response signal with a <Connect Request>
`signal, and to allow for multiplexing and demultiplexing of
`inbound and outbound conversations on conference lines, as
`explained hereinafter.
`For callee (or called) processing units with fixed IP
`addresses, the caller (or calling) processing unit may open a
`“socket”, i.e. a file handle or address indicating where data
`is to be sent, and transmit a <Call> command to establish
`communication with the callee utilizing, for example, data-
`gram services such as Internet Standard network layering as
`well as transport layering, which may include a Transport
`Control Protocol (TCP) or a User Datagram Protocol (UDP)
`on top of the IP. Typically, a processing unit having a fixed
`IP address may maintain at least one open socket and a
`called processing unit waits for a <Call> command to assign
`the open socket to the incoming signal. If all lines are in use,
`the callee processing unit sends a BUSY signal or message
`to the caller processing unit. As shown in FIG. 1,
`the
`disclosed point-to-point Internet protocol and system 10
`operate when a