`
`[19]
`United States Patent
`[45] Date of Patent: Dec. 28, 1999
`Mattaway et al.
`
`[11] Patent Number:
`
`6,009,469
`
`
`
`[54] GRAPHIC USER INTERFACE FOR
`INTERN E'l‘ TELEPHONY APPLICATION
`
`[75]
`
`Inventors: Shane D. Mattaway, Boca Raton,
`Glenn W. Hutton, Miami; Craig B.
`Strickland, Tamarac, all of Fla.
`
`[73] Assignee: NetSpeak Corporation, Boca Raton,
`Fla.
`
`[21] Appl. No.: 08/721,316
`
`[22]
`
`Filed:
`
`Sep. 25, 1996
`
`Related US. Application Data
`
`[63] Continuationin-patt of application No. 08/533,115, Sep. 5,
`1995
`Provisional application No. 60/025,415, Sep. 4, 1996, and
`provisional application No. 60/024,251, Aug. 21, 1996.
`
`[60]
`
`Int. Cl.6 ........... G06F 17/00
`[51]
`
`[52] us. Cl. ................... 709/227
`
`[58] Field of Search
`.. 395/20057, 20058;
`709/227, 228; 370/352, 355, 357, 389,
`395
`
`[56]
`
`References Cited
`U.S. PATENT DOCUMENTS
`
`,
`
`3/1992 Fenner.
`5,095,480
`9/1992 Pcrlman et al,
`5,150,360
`11/1992 Riddle .
`5,166,931
`4/1993 Doric ct al. .
`5,204,669
`6/1993 Woes! ct a|_ .
`5,224,095
`3/1994 Morales.
`5,291,554
`5/1994 Cidon et al. .
`5,309,433
`5/1994 Perlman et al. .
`$309,437
`6/1994 McMillen ct al_ ,
`5,321,813
`10/1994 Banwart.
`5,357,571
`3/1995 Yamada.
`5,400,335
`6/1995 Britten et a1 _
`5,425,028
`7/1995 Galloway.
`5,430,709
`7/1995 Callon.
`5,430,727
`5,4323% 7/1995 Norio.
`5,442,633
`8/1995 Perkins et al. ,
`5,452,296
`9/1995 Shimizu.
`5,455,854
`10/1995 Dilts et ai. .
`5,457,683
`10/1995 Robins.
`5,457,738
`10/1995 Sylvan.
`
`10/1995 Yasrebi.
`5,463,625
`.
`11/1995 Clare ct a1.
`5,465,286
`5,469,500 11/1995 Satter eta]. .
`5,475,741
`12/1995 Davis et al.
`.
`5,479,411
`12/1995 Klein .
`5,509,058
`4/1996 Sestnk ct al.
`5,517,494
`5/1996 Green ,
`5,524,110
`6/1996 Danneels et al. .
`5,524,141
`6/1996 Btaun el al. .
`5,526,489
`6/1996 Nilukantan ct al_ .
`5,533,110
`7/1996 Pinard et al. .
`5,544,303
`8/1996 Maroteaux et al. .
`.
`5,546,582
`8/1996 Brockmeyer et al.
`FOREIGN PATENT DOCUMENTS
`
`.
`
`European Pat. 00'. .
`A2 0445402 11/1991
`A2 0556012
`8/1993 European Pat. 011.
`.
`W0 9219054 10/1992 WIPO I
`
`OTHER PUBLICATIONS
`
`[meme/working with TCP/IP, vol. 1, Second Edition, Prin-
`ciples, Protocols, and Architecture, by Douglas E. Comer.
`\bcal'l‘ec Internet Phone
`) Version 25, www.cox.sm-
`u.edu/class/mi56386/pcople/stortfiphoneZSexe, Feb. 1995.
`Weinberg, Netscape Conference and Cooltalk Meeting
`Room, www.q5.com, Feb. 22, 1996.
`Gull, Re: Getting IP address of PPP—oonnected Mac,
`<jgu11—0304951005350001@prn012-11.dialip.mich.net>,
`Apr. 3, 1995.
`Mac?,
`for
`Phone
`[ntemet
`Gull,
`Re:
`>jguIl-1704950116450001@pm049—28.dialip.mich.net>,
`Apr. 17, 1995.
`Primary Examiner—Ellis B. Ramirez
`Attorney, Agent, or Firm—Kudirka & Jobse, LLP
`
`[57}
`
`ABSTRACT
`
`A communication utility for establishing real-time, point-
`to-point communications between processes over a com-
`putcr network includes apparatus for querying a server as to
`the network protocol address of another client process, and
`apparatus for directly establishing a communication link
`with the client process upon receipt of the network protocol
`address from the server in one embodiment,
`the utility
`includes a sophisticated user interface having features simi-
`lar to typical telephony hardware but implementing greater
`flexibility with software.
`
`18 Claims, 27 Drawing Sheets
`
`
`
`PROCESSING
`UNIT
`
`Page 1 of 67
`
`LG Electronics Exhibit 1001
`
`
`
`
`
`
`
`
`
`
`
`
`
`US. Patent
`
`Dec. 28, 1999
`
`Sheet 1 of 27
`
`6,009,469
`
`onoww
`
`Oz_wmm00mn_
`
`t2:
`
`
`
`mm>mmm20:.0mzzoo
`
`,mac;
`
`mmhmzmm»z_
`
`512.
`
`w0_>mo
`
`.mm>mmw4.52
`
` wr
`
`N65%
`
`Page 2 of 67
`
`
`
`
`
`
`
`US. Patent
`
`Dec. 28, 1999
`
`Sheet 2 of 27
`
`6,009,469
`
`VN
`
`«tea
`
`mm>mmw4.52
`
`NN
`
`
`
`>mw>mw._._0n_
`
`wozoommmé
`
`‘Iull'-
`
`0200mm
`
`02—mmm00mm
`
`.52:
`
`._______._
`
`lllllllllllll.—
`mmEszz.
`
`N65%
`
`AkwMDOwEbmzzOOV
`
`AXOPOMZZOOV
`
`.Sntbo
`
`wo_>m_o
`
`3
`
`Page 3 of 67
`
`
`
`
`
`
`
`
`
`U. S. Patent
`
`Dec. 28, 1999
`
`Sheet 3 of 27
`
`6,009,469
`
`0200mm
`
`Oz.mww00mn_
`
`._._ZD
`
`ozooww
`
`02—wwm00mm
`
`.523
`
`_
`
`omwwmmmzoov"
`
`34590 AO_QD<
`hwy—E
`
`NF
`
`...w~.=....
`
`02—wwm00m
`
`.EZD
`
`02—wwm00m
`
`.22:
`
`Page 4 of 67
`
`
`
`
`
`
`US. Patent
`
`Dec. 28, 1999
`
`Sheet 4 of 27
`
`6,009,469
`
`SHANE
`195.201.00.137
`L1
`0024
`
`38
`
`42
`
`48
`
`
`
`
`ROGER
`
`91.103.001.027
`
`
`0343
`
`C2
`
`
`
`
`HE] an
`BEE
`
`50
`
`Page 5 of 67
`
`
`
`US. Patent
`
`Dec. 23,1999
`
`Sheet 5 of 27
`
`6,009,469
`
`START THE POINT-TO-POINT
`INTERNET PROTOCOLS
`
`f 54
`
`0'10)
`
`PERFORM PRIMARY
`INTERNET PROTOCOL
`
`INITIATE SECONDARY
`INTERNT PROTOCOL
`
`
`
`62
`
`
`
`INITIATE PRIMARY INTERNET
`PROTOCOL
`
`IS THE CONNECTION SERVER IN A
`RESPONSIVE CONDITION?
`
`
`
`
`
`YES
`NO
`
`
`FIG. 7
`
`
`
`STA ' T THE PRIMARY
`POINT-TO-POINT INTERNET
`
`
`PROTOCOL
`
`
`
`64
`
`66
`
`
`
`TIMESTAMP AND STORE E-MAIL
`ADRESSES AND IF ADRESSES OF
`
`
`LOGGED-IN UNITS IN A DATABASE
`
`
`
`-' MIRT "
`.
`"
`. c
`
`WHETHER A SPECIFIED SECOND
`UNIT IS LOGGED-IN
`
`
`
`
`-
`'
`-
`'V'
`- AVIDR' SF'O'
`DATABASE IF THE SECOND UNIT IS
`
`
`LOGGED-IN
`
`
`
`SEND - "’ -IEVE IP ADRESS 0
`FIRST UNIT TO ESTABLISH POINT-
`TO-POINT CONNECTION
`
`FIG. 8
`
`68
`
`
`
`7o
`
` 72
`
`
`
`
`
`
`60
`
`Page 6 of 67
`
`
`
`US. Patent
`
`Dec. 28, 1999
`
`Sheet 6 of 27
`
`6,009,469
`
`START THE SECONDARY
`POINT-TO POINT INTERNET PROTOCOL
`
`74
`
`GENERATE AN E-MAII. SIGNAL.
`
`INCLUDING A SESSION NUMBER AND A
`FIRST IP ADRESS CORRESPONDING TO A
`FIRST PROCESSING UNIT
`
`/‘ 76
`
`78
`
`80
`
`82
`
`84
`
`86
`
`88
`
`
`
`,
`TRANSMIT THE E-MAIL SIGNAL AS A
`<CONNECTREQUEST> SIGNAL
`TO THE INTERNET
`
`DELIVER THE E-MAIL SIGNAL THROUGH
`THE INTERNET USING A MAIL SERVER TO
`A SECOND PROCESSING UNIT
`
`EXTRACT THE SESSION NUMBER AND
`THE FIRST IF’ ADRESS FROM THE E-MAIL
`SIGNAL
`
`SEND THE SESSION NUMBER AND A
`SECOND IP ADRESS CORRESPONDING
`TO THE SECOND PROCESSING UNIT TO
`
`THE FIRST PROCESSING UNIT THROUGH
`THE INTERNET
`
`VERIFY THE SESSION NUMBER
`RECEIVED FROM THE SECOND
`PROCESSING UNIT
`
`ESTABLISH A POINT-TO-POINT INTERNET
`COMMUNICATION LINK BETWEEN THE
`FIRST AND SECOND PROCESSING UNITS
`USING THE FIRST AND SECOND IP
`ADDRESSES
`
`FIG. 9
`
`Page 7 of 67
`
`
`
`U
`
`ta
`
`t
`
`9m892mD
`
`u.m7m
`
`6,009,469
`
`h60°FSl/
`
`cue.
`
`w82
`
`P320m.326S”€390:
`
`352:.
`
`33.3
`
`823.5
`
`22:95
`
`Page 8 of 67
`
`
`
`
`U.S. Patent
`
`Dec. 28, 1999
`
`Sheet 8 of 27
`
`6,009,469
`
`of
`
`2:3;o!
`mwjoEzooa.25.
`
`xwa$30528$39.583‘555
`
`0:
`
`292923250089><20oz<359%
`
`m2
`
`
`
`N9
`
`5
`
`"2mg55omximama20m8
`
`E2:I\)w:)-5r
`
`om.
`
`
`
`
`
`mmE<o<mmjomhzoo5.305200mmnoz
`
`at'22I/5—m2
`.mmjomhzoo
`:Ifi.mmmm
`
`mar
`
`
`
`Eng.:95»:.0.59l
`
`on.
`
`O_OD<
`
`mNJAOmHZOU
`
`>102m=2
`
`mquomhzoo
`
`mm?
`
`mam
`
`mijmHzOo
`
`we
`
`.EDmmwkZ
`
`mquomkzoo
`
`Duo
`
`mow
`
`Page 9 of 67
`
`
`
`
`
`
`
`
`
`US. Patent
`
`Dec. 28, 1999
`
`Sheet 9 of 27
`
`6,009,469
`
`1214
`
`FIGURE 12 (PRIOR ART)
`
`Page 10 of 67
`
`
`
`netaPS.
`
`nf00
`
`6,009,469
`
`.canUP
`
`.1...
`
`03.503?aE.35aman
`928.353man:
`
`tI
`
`m:6maoufiak
`
`wrap
`
`Page 11 of 67
`
`58:.an
`
`252.53
`
`52:
`
`<223mm
`
`
`
`
`US. Patent
`
`Dcc.28, 1999
`
`Sheet 11 of 27
`
`6,009,469
`
`WAPI
`
`1328
`
`process
`
`comm
`
`state
`
`1334
`
`machine
`
`network
`
`FIGURE 13 B
`
`Page 12 of 67
`
`
`
`US. Patent
`
`Dec. 28, 1999
`
`Sheet 12 01-27
`
`6,009,469
`
`1402
`
`1402E
`
`1402D
`1402G
`1402H
`
`1404
`
`1402A
`
`14028
`
`1402C
`
`1402F
`1 402J
`
`1 408 £1? W©©®h©tm© X
`
`1406
`
`Figure 14
`
`Page 13 of 67
`
`
`
`US. Patent
`
`l)cc.28, 1999
`
`Sheet 13 0f 27
`
`6,009,469
`
`1500
`
`I15“!
`
`DATABASE
`SEARCH
`ENGINE
`
`15160
`
`1513!:
`
`15160
`
`1516B
`
`DATABASE
`
`CONNECTION
`
`SERVER
`
`
`DATABASE
`SERVER
`
`
`
`
`NETwORK INTERFACE CARD
`
`1520
`
`1532
`
`1530
`
`HREWALL SERVER
`
`‘522
`
`1 524
`
`1538 ® 1539
`
`ISP
`
`I Q 1534
`
`Figure 15A
`
`Page 14 of 67
`
`
`
`US. Patent
`
`Dec. 28, 1999
`
`Sheet 14 0f 27
`
`6,009,469
`
`DATABASE
`
`SEARCH
`
`
`
`-ATABASE_NGINE
`
`ssnven
`
`CONNECTION
`SERVE"
`
`'NFSOERRHQQEON
`
`DATABASE
`
`I @ '534
`
`Figure 158
`
`Page 15 of 67
`
`
`
`US. Patent
`
`Dec. 28, 1999
`
`Sheet 15 of 27
`
`6,009,469
`
`xmamsz
`
`atom.326
`
`cmmw
`
`xaoamaoz atom.32“.
`
`0m.23.5
`
`samamfiz
`
`xaonwaz
`
`320m.320
`
`
`
`.5wa.305
`
`Page 16 of 67
`
`
`
`US. Patent
`
`Dec. 28, 1999
`
`Sheet 16 of 27
`
`6,009,469
`
`<29:9...
`
`are
`
`ozwm
`
`
`
`A¥U<POWZZOOV
`
`\woz“mo526..
`
`mm>.
`
`30—.
`
`nmgmmwa
`
`owN—Zmuoommz:
`
`<._.<a
`
`N3—
`
`@5—
`
`823mm
`
`_
`
`6mmhomzzoov
`
`
`
`.=<STmPU<§
`
`wmwmoo<
`
`
`
`.=<E.wnun—55:0”.
`
`
`
`ww<n<h<0O...mmmman<
`
`
`
`mOmmm02mm
`
`m0m0<wmm2
`
`
`
`ANZ...“EOV
`
`Num—
`
`Page 17 of 67
`
`
`
`
`
`
`
`
`US. Patent
`
`Dec. 28, 1999
`
`Sheet 17 of 27
`
`6,009,469
`
`
`
`
`DISGARD
`UNRECOGNIZED
`
`
` <INFO-REQ>
`DATA
`
`
`
`
`1 635
`
`RECEIVED
`?
`
`YES
`
`EXTRACT SEARCH
`CRITERIA
`
`SUPPLY SEARCH
`CRITERIA TO DATABASE
`
`SEND
`<INFO ACK>
`
`1634
`
`1636
`
`1638
`
`YES
`
`
`
`<INFO ABORT
`RECEIVED
`
`
`
`
`
`MORE
`<INFO>
`?
`
`N°
`
`1632
`
`N0
`
`POWER OFF
`?
`
`YES
`
`@ Figure 168
`
`Page 18 of 67
`
`
`
`US. Patent
`
`Dec. 28, 1999
`
`Sheet 18 of 27
`
`6,009,469
`
`8383x
`
`gmmmxozm
`
`AI_.._<UVa
`
`Arum1mmv<m
`
`A>w3mvumm0
`
`E0
`
`
`Axo<
`
` j<0von5A1052mm3m2<vUm
`
`sz
`
`gummxozm
`
`AOmm9.2.mmeVN
`
`
`
`Amem2_._ZOv_.
`
`AZOF<EPQOMIvv
`
`
`
`AOkZ.mmmsvm
`
`
`
`A¥U<MZSZOVm
`
`
`
`ACMEhumzzoovo
`
`
`
`A¥0<homzzoov<~u
`
`
`
`Amz:“EOVmh
`
`040m<4 (DILIC>I.IJII
`
`A20mm44<0v0:.
`
`
`
`A¥U<mm..._<ova:
`
`EF9:95
`
`Page 19 of 67
`
`
`
`
`
`
`
`
`
`
`US. Patent
`
`Dec. 28, 1999
`
`Sheet 19 of 27
`
`6,009,469
`
`mz0Ia.mm>>
`
`oqom<4 mmm>mm
`
`ACNEOuz_vv
`
`A¥O<Ouz_vN
`
`AOuz_v<0
`
`AOn—Z_vmm
`
`A0u2_vOm
`
`
`
`Ahm0m<Ouz_veco
`
`m:h"2%.:
`
`Page 20 of 67
`
`
`
`US. Patent
`
`Dec. 28, 1999
`
`Sheet 20 0f 27
`
`6,009,469
`
`acezzo
`LSEOEESS
`
`«newn—
`
`£282.85
`
`
`
`222.2252.85
`
`
`
`9:35._o=:oo_:w
`
`
`o:_m>9.0.
`
`9:36._o==oos
`
`
`
`wearvow_.woww
`
`<2939$
`
`
`
`$202502m:o=o=.u>3022muo:_u>noonfiat;
`
`
`
`
`
`
`
`$332502acoaasfizomSon.a:t_>
`
`
`
`coaaaow93.fists
`
`Nwww
`
`mJim—«3:23
`
`a::23
`
`n::05
`
`
`
`.oosoaatss32:83:52:
`
`Page 21 of 67
`
`
`
`
`
`
`
`
`US. Patent
`
`Dcc.28, 1999
`
`Sheet 21 of 27
`
`6,009,469
`
`1804
`
`
`
`
`
`GUIControl" Sibling
`UlCollection" Parent
`
`
`
`1 828
`
`1822
`
`Figure 183
`
`Page 22 of 67
`
`
`
`US. Patent
`
`Dec.28, 1999
`
`Sheet 22 of 27
`
`6,009,469
`
`UlToggle
`
`1842
`
`1844
`
`1846
`
`
`
`
`Figure 180
`
`UlPIayHun
` UlPushButton
`
`
`
`
`
`
`GUIVirtualObject" Childrenfl
`UlCoIlectlon’ Parent
`
`Figure 180
`
`
`
`1 806
`
`vold ShowO
`vold Hideo
`
`1834
`
`1832
`
`Page 23 of 67
`
`
`
`US. Patent
`
`Dec. 28, 1999
`
`Sheet 23 of 27
`
`6,009,469
`
`029:9...32
`
`933-
`
`mafia.
`
`m223E
`
`<22:9...
`
`968..
`
`\|||l|\/||||II.|||J
`
`26:633
`
`>852...
`
`$30m<
`
`
`
`5320NMqugum.3.2.
`
`E<5Hu<use25m
`
`Inor
`
`m§4m|m<an«988
`mama—ME25mcom—Ea
`no.5m<95Simon
`
`
`
`Emmobuu<own
`
`0.22.13:32:
`5515352:
`0.:m<2:5:
`
`558:3ban
`
`Baum?”mu:
`
`
`
`9.23
`
`
`
`
`
`225:2:9:383
`
`Esau2.2.39E2...
`
`.3.96.3.
`
`no_
`
`+D¢
`
`25E
`
`85m
`
`3.9.2.:
`
`xmahcrH3.2..
`
`numb—.0.
`
`8?»...
`
`90.529
`
`
`
`one.
`
`one...
`
`oEanuu<
`
`963225:
`
`9.8585
`
`9.3.5380:
`
`29.».G0::
`
`20:92.
`
`nor
`
`
`
`9.8235
`
`9.32050:
`
`Page 24 of 67
`
`
`
`
`US. Patent
`
`Dec. 28, 1999
`
`Sheet 24 of 27
`
`6,009,469
`
`2°”
`
`MIX MO
`MIXING BUFFER
`
`K
`
`SPEAKER
`
`LOOP FOR ALL PARTIES
`
`
`WRITETO
`PAR I YI1).SOCKET
`
`IS
`pAaTvm
`ONUNE?
`
`Mo
`
`0 Q
`
`2030
`
`SOCKET
`
`2032
`
`MIC
`
`"'0
`EMBLE”
`7
`
`N0
`
`YES
`
`PLACE EMPTY
`BUFFER 0N
`MIC SAMPLING
`GUEUE
`
`
`
`2034
`
`PLACE EMPTY
`BUFFER 0N
`WInSock OUEUE
`
`
`
`PLACE EMPTY
`
`-£3.‘--"§3$’i
`
`- w
`
`W)
`
`2°36
`
`2°38
`
`Figure 20A
`
`2°“
`
`Page 25 of 67
`
`
`
`U
`
`m,waP
`
`D
`
`9m
`
`n
`
`6,009,469
`
`mm;m:858RES...02
`Sbush
`n,x3955E0:.30....E2.—m.93..
`
`
`
`uc>m<was5530
`
` Nmowomow3cm
`
`
`on».coc>m<mmEEaEO0::032m
`
`2:.EatLoon.EatmEEEmm
`
`3cm
`
`mow«Sat
`
`Page 26 of 67
`
`
`
`US. Patent
`
`Dec. 28, 1999
`
`Sheet 26 of 27
`
`6,009,469
`
`wmow
`
`8:33.:mafia
`
`x085;5
`
`Loon...mo...
`
`umxoom.550
`
`$83.8
`
`.
`
`
`
`_.00a:3....
`
`$5655w53%;,\$8
`
`.5:mmEwow.umm;=5ta_88amoon59.3%0.:
`
`.555
`
`
`
`08.93mm
`
`32
`
`2:E9:
`
`é
`
`.93QOnoon
`
`Page 27 of 67
`
`
`
`
`
`
`US. Patent
`
`Dec. 28, 1999
`
`Sheet 27 of 27
`
`6,009,469
`
`data in mix
`
`Discard pending
`
`Figure 200
`
`Page 28 of 67
`
`
`
`6,009,469
`
`1
`GRAPHIC USER INTERFACE FOR
`INTERNET TELEPHONY APPLICATION
`
`RELATED APPLICATIONS
`
`This application is a continuation-in-part of US. patent
`application Ser. No. 08/533,115 (Pending) entitled Point-to.
`Point Internet Protocol, by Glenn W_ Hutton, filed Sep. 25,
`1995, commonly assigned. the subject matter of which is
`incorporated herein by reference.
`To the extent
`that any matter contained herein is not
`already disclosed in the above-identified parent application
`a location claims priority to U.S_ provisional patent appli-
`cation 60/025,415 entitled Internet Telephony Apparatus and
`Method by Mattaway et al., filed Sep. 4, 1996, and US.
`provisional patent application Ser. No. 60/024,251 entitled
`System and Methods for Point-To-Point Communications
`Over a Computer Network, by Mattaway ct al., filed Aug.
`21, 1996.
`In addition, this application is one of a number of related
`applications filed on an even date herewith and commonly
`assigned,
`the subject matters of which are incorporated
`herein by reference, including the following:
`US. patent application Ser. No. 08/719,894, entitled
`Directory Server For Providing Dynamically Assigned
`Network Protocol Addresses, by Mattaway et al.;
`US. patent application Ser. No. 08/719,554, entitled
`Point-to-point Computer Network Communication
`Utility Utilizing Dynamically Assigned Network Pro-
`tocol Addresses. by Mattaway et at;
`US. patent application Ser. No, 08/719,640, entitled
`Method And Apparatus For Dynamically Defining Data
`Communication Utilities, by Mattaway et al.,
`US. patent application Ser. No. 08/719,891, entitled
`Method And Apparatus For Distribution And Presen-
`tation 0f Multimedia Data Over A Computer Network,
`by Mattaway et II.)
`US. patent application Ser. No. 08/719,898, entitled
`Method And Apparatus For Providing Caller Identifi-
`cation Based Out-going Memges In A Computer Tele-
`phony Environrnent, by Mattaway et al.;
`US. patent application Ser. No. 08/718,911, entitled
`Method And Apparatus For Providing Caller Identifi-
`cation Based Call Blocking In A Computer Telephony
`Environment, by Mattaway et al.; and
`US. patent application Ser. No. 08/719,639, entitled
`Method And Apparatus For Providing Caller Identifi-
`cation Responses In A Computer Telephony
`Environment, by Mattaway et al.
`FIELD OF THE INVENTION
`
`The present invention relates, in general, to data proces.
`ing systems, and more specifically, to a method and appa-
`ratus for facilitating audio communications over computer
`networks.
`
`BACKGROUND OF THE INVENTION
`
`Ihe increased popularity of on-line services such as
`AMERICA ONLINETM, COMPUSERVE®, and other ser-
`vices such as Internet gateways have spurred applications to
`provide multimedia,
`including video and voice clips, to
`online users. An example of an online voice clip application
`is VOICE E-MAIL FOR WINCIM and VOICE E-MAII.
`FOR AMERICA ONLINE”, available from Bonzi
`Software, as described in “Simple Utilities Send Voice
`
`5
`
`10
`
`15
`
`20
`
`30
`
`35
`
`4o
`
`45
`
`50
`
`55
`
`65
`
`Page 29 of 67
`
`2
`E-Mail Online", MULTIMEDIA WORLD, VOL. 2, N0. 9,
`Aug. 1995, p. 52. Using such Voice E-Mail software, a user
`may create an audio message to be sent to a predetermined
`E-mail address specified by the user.
`Generally, devices interfacing to the Internet and other
`online services may communicate with each other upon
`establishing respective device addresses. One type of device
`address '5 the Internet Protocol (IP) address, which acts as
`a pointer to the device associated with the IP address. A
`typical device may have a Serial Linc Internet Protocol or
`Point-to-Point Protocol (SLIP/PPP) account with a penna-
`nent IP address for receiving Email, voicemail, and the like
`over the Internet. E-mail and voieemail is generally intended
`to convey text, audio, etc, with any routing information
`such as an IP address and routing headers generally being
`considered an artifact of the communication, or even gib-
`berish to the recipient.
`Devices such as a host computer or server of a company
`may include multiple modems for connection of users to the
`Internet, with a temporary IP address allocated to each user.
`For example,
`the host computer may have a general IP
`address “XXX.XXX.XXX," and each user may be allocated
`a successive IP address of XXX.XXX.XXX.10,
`XXXXXXXXXJI, XXX.XXX.XXX.12, etc, Such tem-
`porary IP addresses may be reassigned or recycled to the
`users, for example, as each user is successively connected to
`an outside party. For example, a host computer of a company
`may support a maximum of 254 IP addresses which are
`pooled and shared between devices connected to the host
`computer.
`Permanent IP addresses of users and devices accessing the
`Internet readily support point-to-point communications of
`voice and video signals over the Internet. For example,
`real-time video teleconferencing has been implemented
`using dedicated IP addresses and mechanisms known as
`reflectors. Due to the dynamic nature of temporary IP
`addresses of some devices accessing the Internet, point-to-
`point communications in reaLtime of voice and video have
`been generally difficult to attain.
`The ability to locate users having temporary or dynami-
`cally assigned Internet Protocol address has been difficult
`without the user manually initiating the communication.
`Accordingly, spontaneous, real-time communications with
`such users over computer networks have been impractical.
`Further, it is desirable to have a communication utility which
`contains familiar features and functions to current commu-
`nication utility such as telephones and cellular telephones. It
`is even further desirable to utilize the current graphic user
`interface technology associated with computer software in a
`manner to achieve a more flexible interface to a such a
`communication utility, without the limitations associated
`with hardware.
`Accordingly, a need exists for a way to determine whether
`computer users are actively connected to a computer not-
`work.
`A further need exists for a way to obtain the dynamically
`assigned Internet Protocol address of a user having on-line
`status with respect to a computer network, particularly the
`Internet.
`An even further need exists for a method and apparatus by
`which to establish real-time, point-to-point communications
`over a computer network using a communication utility
`having an interface which combines the familiar aspects of
`current hardware communication utilities but which allows
`for the flexibility associated with graphic user interfaces,
`SUMMARY OF THE INVENTION
`
`The above deficiencies in the prior art and previously
`described needs are fulfilled by the present invention which
`
`
`
`6,009,469
`
`3
`provides a vtrtual communications utility displayable on
`computer system interfaces which enables real-time, point-
`to-point communications over computer networks. Accord-
`ing to one embodiment of the present invention, a computer
`program product for use with a computer system having a
`display and an audio transducer comprises a computer
`usable medium having computer readable code means
`embodied therein comprising program code means for gen-
`erating a user interface, program code meats responsive to
`user input commands for establishing a point-to-point com-
`munication link with another computer over a network and
`program code means responsive to audio data from the audio
`transducer for transmitting the audio data over the commu-
`nication link.
`
`According to another embodiment of the present
`invention, a computer program product
`for use with a
`computer system comprises a computer usable medium
`having computer readable program code means embodied
`thereon comprising code means for transmitting from a
`cltcnt process to a server a query as to whether a second
`client process is connected to the computer network, pro-
`gram code means for receiving the network protocol adiress
`of the second process from the server, and program code
`means responsive to the network protocol address of the
`second client process for establishing a point-to-point com-
`munication link between the first client process and the
`second client process.
`BRIEF DESCRIPTION OF THE DRAWINGS
`The features of the invention Wlll become more readily
`apparent and may be better understood by referring to the
`following detailed description of an illustrative embodiment
`of the present
`invention.
`taken in conjunction with the
`accompanying drawings, in which:
`FIG. 1 illustrates, in block diagram format. a system for
`the disclosed pcint-to-point Internet protocol;
`FIG. 2 illustrates, in block diagram format, the system
`using a secondary point-to-point Internet protocol,
`FIG. 3 illustrates, in block diagram format, the system of
`FIGS. 1-2 with the point-to-point Internet protocol estab-
`lished;
`FIG. 4 is another block diagram of the system of FIGS.
`1—2 with audio communications being conducted;
`FIG. 5 illustrates a display screen for a processing unit;
`FIG. 6 illustrates another display screen for a processing
`unit;
`FIG. 7 illustrates a flowchart of the initiation of the
`point-to-point Internet protocols;
`FIG. 8 illustrates a flowchart of the performance of the
`primary point-to-point Internet protocols;
`FIG. 9 illustrates a flowchart of the performance of the
`secondary point-to-point lntemet protocol;
`FIG. 10 illustrates schematically a computer network over
`which the present invention may be utilized;
`FIG. 11 is a block diagram of a computer system suitable
`for use with the present invention;
`FIG. 12 is a block diagram of an audio processing card
`suitable for use with the computer system of FIG. 10;
`FIGS. 13 A—B are schematic block diagrams of the
`elements comprising the inventive computer network tele.
`phony mechanism of the present invention;
`FIG. 14 is a screen capture illustrating an exemplary user
`interface of the present invention;
`FIG. 15 is a schematic diagram illustrating the architec-
`ture of the connection server apparatus suitable for use with
`the present invention;
`
`4
`illustrating the process steps
`FIG. 16A is a flowchart
`performed by the connection server in accordance with the
`present invention;
`illustrating the process steps
`FIG. 16B is a flowchart
`performed in accordance with the information sewer of the
`present invention;
`FIGS. l7A—B are schematic block diagrams illustrating of
`the packet transfer sequence in accordance with the com-
`munication protocol of the present invention;
`FIGS. l8A—D are conceptual block diagrams illustrating
`user interface and graphic user interface objects utiliud by
`the communication utility of the present invention;
`FIGS. 19A—C are conceptual block diagrams illustrating
`the event manager and media engine objects utilized by the
`communication utility of the present invention, and
`FIGS. 20A—D illustrate process steps performed by the
`media engine function of the communication utility in
`accordance with the present invention.
`
`10
`
`15
`
`DETAILED DESCRIPTION
`
`Rcfcmng now in spectfic detail to the drawings, with like
`reference numerals identifying similar or identical elements,
`as shown in FIG. 1,
`the present disclosure describes a
`pointvto-point network protocol and system 10 for using
`such a protocol.
`ln an exemplary embodiment, the system 10 includes a
`first processing unit 12 for sending at least a voice signal
`from a first user to a second user. The first processing unit
`12 includes a processor 14, a memory 16, an input device 18,
`and an output device 20. The output device 20 includes at
`least one modem capable of, for example. 14.4 Kilobit-per—
`second communications and operatively connected via
`wired and/or wireless communication connections to the
`Internet or other computer networks such as an Intranet, Le,
`a private computer network. One skilled in the art would
`understand that the input device 18 may be implemented at
`least in part by the modem of the output device 20 to allow
`input signals from the communication connections to be
`received. The second processing unit 22 may have a
`processor, memory, and input and output devices, including
`at
`least one modem and associated communication
`connections, as described above for the first processing unit
`12. In an exemplary embodiment, each of the processing
`units 12, 22 may execute the WEBPHON'FJD Internet tclc-
`phony application available from NetSpeak Corporation,
`Boca Raton, Fla, which is capable of performing the dis-
`closed point-to-point Internet protocol and system 10, as
`described herein.
`
`The first processing unit 12 and the second processing
`unit 22 are operatively connected to the Internet 24 by
`communication devices and software known in the art, such
`as an lntemet Service Provider (ISP) or an lntemet gateway.
`The processing units 12, 22 may be operatively intercon-
`nected through the lntemet 24 to a connection server 26, and
`may also be operatively connected to a mail server 28
`associated with the lntemet 24.
`The connection server 26 includes a processor 30, a timer
`32 for generating time stamps, and a memory such as a
`database 34 for storing, for example, E-mail and lntemet
`Protocol (IP) addresses of logged-in units. In an exemplary
`embodiman the connection server 26 may be a SPARC 5
`server or a SPARC 20 server, available from SUN
`MICROSYSTEMS, INC, Mountain View, Caltf.. having a
`central processing unit (CPU) as processor 30, an operating
`system (OS) such as UNIX. for providing timing operations
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`Page 30 of 67
`
`
`
`
`
`
`
`
`
`
`
`
`
`6,009,469
`
`6
`In addition, either of the first processing unit 12 and the
`second processing unit 22 may be implemented in a personal
`digital assistant (PDA) providing modem and E-mail capa-
`bilities and Internet access, with the PDA providing the
`input/output screens for mouse interactions or for touch-
`screen activation as shown, for example, in FIGS. 5—6, as a
`combination of the input device 18 and output device 20.
`For clarity of explanation, the illustrative embodiment of
`the disclosed potnt-to-point Internet protocol and system 10
`is presented as having individual functional blocks, which
`may trrclude functional blocks labeled as “processor" and
`“processing unit”. The functions represented by these blocks
`may be provided through the use of either shared or dedi-
`cated hardware,
`including, but not
`limited to, hardware
`capable of executing software. For example, the functions of
`each of the processors and processing untts presented herein
`may be provided by a shared processor or by a plurality of
`individual processors. Moreover, the use of the functional
`blocks with accompanying labels herein is not to be con-
`strued to refer exclusively to hardware capable of executing
`software. Illustrative embodiments may include digital sig-
`nal processor (DSP) hardware, such as the AT&T DSPIG or
`DSP32 C, read-only memory (ROM) for storing software
`performing the operations discussed below, and random
`access memory (RAM) for storing DSP results. Very large
`scale integration (VLSI) hardware embodiments, as well as
`custom VLSI circuitry in combination with a general pur-
`pose DSP circuit, may also be provided. Any and all of these
`embodiments may be deemed to fall within the meaning of
`the labels for the functional blocks as used herein.
`The processing units 12, 22 are capable of placing calls
`and connecting to other processing units connected to the
`Internet 24, for example, via dialup SLIP/PPP lines. In an
`exemplary embodiment, each processing unit assigns an
`unsigned long session number, for example, a 32bit long
`sequence in a “.ini file for each call. Each call may be
`assigned a successive session number in sequence, which
`may be used by the respective processing unit to associate
`the call with one of the SLIP/PPP lines,
`to associate a
`<ConnectOK> response signal with a <Connect Request>
`signal, and to allow for multiplexing and demultiplexing of
`inbound and outbound conversations on conference lines, as
`explained hereinafter.
`For callee (or called) processing units with fixed IP
`addresses, the caller (or calling) processing unit may open a
`“socket”, i.e. a tile handle or address indicating where data
`is to be sent, and transmit a <Call> command to establish
`communication with the caller: utilizing, for example, data-
`gram services such as Internet Standard network layering as
`well as transport layering, which may include a Transport
`Control Protocol (1‘CP) or a User Datagram Protocol (UDP)
`on top of the IP. Typically, a processing unit having a fixed
`IP address may maintain at least one open socket and a
`called processing unit waits for a <Call> command to assign
`the open socket to the incoming signal. If all lines are in use,
`the callee processing unit sends a BUSY signal or message
`to the caller processing unit. As shown in FIG. 1, the
`disclosed point-to-point Internet protocol and system 10
`operate when a callee procming unit does not have a fixed
`or predetermined IP address. In the exemplary embodiment
`and without loss of generality, the first prooeming unit 12 is
`the caller processing unit and the second processing unit 22
`is the callee processing unit. When either of processing units
`12, 22 logs on to the Internet via a diaLup connection, the
`respective unit
`is provided a dynamically allocated IP
`address by an Internet service provider.
`Upon the first user initiating the poinLto-point Internet
`protocol when the first user is logged on to the Internet 24.
`
`5
`such as maintaining the ttmer 32, a hard drive or fixed drive,
`as well as dynamic random access memory (DRAM) for
`storing the database 34, and a keyboard and display and/or
`other input and output devices (not shown in FIG. 1). The
`database 34 may be an SOL database available from
`ORACLE or INFORMIX,
`In an exemplary embodiment, the mail server 28 may be
`implemented with a Post Oflice Protocol (POP) Version 3
`mail server and the Simple Mail Transfer Protocol (SMTP),
`including a processor, memory, and stored programs oper-
`ating in a UNIX environment, or, alternatively, another OS,
`to process E-mail capabilities between processing units and
`devices over the Internet 24,
`In the illustrative cmbodtment, the POP protocol is uti.
`lized to retrieve E—mail messages from mail server 28 while
`the SMTP protocol is used to submit E-mail message to
`Internet 24.
`The first processing unit 12 may operate the disclosed
`poinLto-point Internet protocol by a computer program
`described hereinbelow in conjunction with FIG. 6, which
`may be implemented from compiled and /or interpreted
`source code in the C++ programming language and which
`may be downloaded to the first processing unit 12 from an
`external computer. The operating computer program may be
`stored in the memory 16, which may include about 8 MB
`RAM and/or a hard or fixed drive having about 8 MB of
`available memory. Alternatively, the source code may be
`implemented in the first processing unit 12 as firmware, as
`an erasable read only memory (EPROM), etc. It is under.
`stood that one skilled in the art would be able to use
`programming languages other than OH to implement the
`disclosed point-to-point network protocol and system 10.
`The processor 14 receives input commands and data from
`a first user associated with the first processtng unit 12 though
`the input device 18, which may be an input port connected
`by a wired, optical, or a wireless connection for electromag-
`netic transmisions, or alternatively may he transferable
`storage media, such as floppy disks, magnetic tapes, com.
`pact disks, or other storage media including the input data
`from the first user.
`The input device 18 may include a user interface (not
`shown) having, for example, at least one button actuated by
`the user to input commands to select from a plurality of
`operating modes to operate the first processing unit 12. In
`alternative embodiments, the input device 18 may include a
`keyboard, a mouse, a touch screen, and/or a data reading
`device such as a disk drive for receiving the input data from
`input data files stored in storage media such as a floppy disk
`or, for example, an 8 mm storage tape. The input device 18
`may alternatively include connections to other computer
`systems to receive the input commands and data therefrom.
`The first processing unit 12 may include a visual interface
`for use in conjunction with the input device 18 and output
`device 20 similar to those screens illustrated in FIGS. 5—6,
`discussed below.
`It
`is also understood that alternative
`devices may be used to receive commands and data from the
`user, such as keyboards, mouse devices, and graphical user
`interfaces (GUI) such as WINDOWS‘m 3.1 available form
`MICROSOFT Corporation, Redmond, Wash, and other
`operating systems and GUIs, such as 08/2 and 08/2 WARP,
`available from IBM CORPORATION, Boca Raton, Fla.
`Processing unit 12 may also include microphones and/or
`telephone handsets for receiving audio voice data and
`commands, speech or voice recognition devices, dual tone
`multi-frcquency (DTMF) based devices, and/or sofiwarc
`known in the art to accept voice data and commands and to
`operate the first processing unit 12.
`
`10
`
`15
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`Page 31 of 67
`
`
`
`6,009,469
`
`7
`the first processing untt 12 automatically transmits tls asso-
`ciated E-mail address and its dynamically allocated IP
`address to the connection sewer 26. The connection server
`26 then stores these addresses in the database 34 and time
`stamps the stored addresses using timer 32. The first user
`operating the first processing unit 12 is thus established in
`the database 34 as an active on-linc party available for
`communication using the disclosed point-to-point Internet
`protocol. Similarly, a second user operating the second
`processing unit 22, upon connection to the lntemet 24
`through an Internet service provider, is processed by the
`c