US007233589B2
`
`(12) United States Patent
`Tanigawa et a].
`
`(10) Patent N0.:
`(45) Date of Patent:
`
`US 7,233,589 B2
`Jun. 19, 2007
`
`(54) COMMUNICATION SYSTEM AND
`COMMUNICATION METHOD
`
`(75) Inventors: Keiko TanigaWa, Kawasaki (JP);
`Kazuma Yumoto, Hachiouji (JP);
`Kenta Shiga, Yokohama (JP); Toru
`Hoshi, Yokohama (JP)
`
`(73)
`
`Assignee: Hitachi, Ltd., Tokyo (JP)
`
`(*)
`
`Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 1170 days.
`
`(21)
`
`(22)
`
`(65)
`
`Appl. No.: 10/231,262
`
`Filed:
`
`Aug. 30, 2002
`
`Prior Publication Data
`
`OTHER PUBLICATIONS
`
`Nikkei Communication, Nov. 5, 2001, pp. 106-113.
`R. Perera, “Instant Messaging Gets Down to Business”, PC
`WORLD.COM, (http://www.pcworld.com) , Nov. 15, 2001.
`Mahy et al, “A Multi-Party Application Framework for SIP”,
`SIPPING Working Group Internet Draft, Feb. 2002.
`“Instant Messaging”, PC WORLD.COM, (http://wwwpcworld.
`com), May 22, 2000.
`M. Goldberg, “Generation IM”, Yahoo ! Internet Life, Apr. 2002,
`pp. l-6.
`D. Willmott, “Which IM Program Is for You?”, Yahoo ! Internet
`Life, Apr. 2002.
`J. Caplan, “amazing IM Tricks ! ”, Yahoo ! Internet Life, Apr. 2002.
`J. Caplan, “Amazing IM Tricks ! ”, Yahoo ! Internet Life, Apr. 2002.
`* cited by examiner
`
`Primary ExamineriPhirin Sam
`(74) Attorney, Agent, or F irmiMattingly, Stanger, Malur &
`Brundidge, PC.
`
`US 2004/0001480 A1
`
`Jan. 1, 2004
`
`(57)
`
`ABSTRACT
`
`Foreign Application Priority Data
`(30)
`Jun. 4, 2002
`
`(JP)
`
`........................... .. 2002-162940
`
`(51) Int. Cl.
`(2006.01)
`H04L 12/66
`(52) US. Cl. ..................................... .. 370/352; 370/493
`(58) Field of Classi?cation Search ................... .. None
`See application ?le for complete search history.
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`6/2002
`2002/0073142 A1 *
`l/2003
`2003/0021290 A1 *
`2003/0126213 Al* 7/2003
`2003/0149774 Al* 8/2003 McConnell et al. ...... .. 709/227
`2003/0210770 Al* ll/2003 Krejcarek .............. .. 379/8817
`
`Moran ...................... .. 709/203
`
`The usage of communication systems is improved. An IM
`server 4 manages presence information regarding each of IM
`clients and usable media and user information such that each
`of the IM clients can obtain these kinds of information. In
`order to perform text chatting, the IM server 4 manages a
`connection between each of the IM clients participating the
`chat and the IM server 4, merges text from each of the
`participating IM clients and distributes the result to each of
`the participating IM clients. In order to perform voice
`chatting, an AP server 5 manages a connection between each
`of the IM clients participating the chat and an MD server 6,
`mixes voice from each of the participating IM clients except
`for a focused IM client and distributes the result to the
`focused participating IM clients. This processing is per
`formed for each of the participating IM clients.
`
`12 Claims, 19 Drawing Sheets
`
`APSERVER
`
`5\I\.=
`
`IMSERVER
`
`3
`
`1
`
`I
`
`PUBLICLY
`SWITCHED
`TELEPHONE
`NETWORK
`
`1 1
`
`1:1
`Um FIXED
`TELEPHONE
`
`2
`
`DIO
`[:3 COMMUNI
`T N
`
`1 O
`\I\
`
`9
`
`VR SERVER
`
`IP NETWORK
`
`RADIO
`TERMINAL
`
`7-1
`[P Q TERMlNAL
`
`7-2
`f TERM'NM-
`
`TERMINAL
`
`/
`
`GOOGLE EXHIBIT 1014
`
`Page 1 of 35
`
`

`

`U.S. Patent
`
`Jun. 19, 2007
`
`Sheet 1 of 19
`
`US 7,233,589 B2
`
`2053
`
`4232500
`
`mm>mwmMS xmozcmz
`
`xmosz—zn:
`
`“202nm?
`
`mzoEEEg35:2»
`BEBuzosmsn.
`NumP_.m«mimmn2
`
`x8352mw>¢mms:
`
`TEE
`
`Page 2 0f 35
`
`Page 2 of 35
`
`
`
`
`

`

`U.S. Patent
`tnetaP
`
`n.HJ
`Jun. 19, 2007
`
`Sheet 2 0f 19
`m
`
`US 7,233,589 B2
`2B
`3
`5,
`
`9m?_dzmmcmwemwe5
`sgazmwwmu
`
`
`2,Equwzn§z<mammwooE2350w(:5
`ofifimmmmzzmsmmwfiamfimfima=9anEa20:18quSmEEmoissoo
`
`
`hm.ng
`
`
`mgmmmwfimmoflmwmEu505m:
`
`3&he
`
`
`
`wvmm>mmms—
`
`2N7,mS0£6352UEg....a
`
`
`mam.
`
`22:6"
`
`
`
`62.-mzmmm<
`
`29.5..on
`
`mNGE
`
`Page 3 0f 35
`
`
`
`
`
`2058Emsmgzsz20:82on
`
`Page 3 of 35
`
`
`

`

`U.S. Patent
`
`Jun.19,2007
`
`Sheet3 0f19
`
`US 7,233,589 B2
`
`O¢v‘
`
`I-I°.l.m.D-o.mr<
`
`:GEMdddi
`
`
`
`
`$5225:>83mozmmmmzooEmzoHzmzo238%
`
`mMVwmuFMVmm?W”?dWfiMm?NW?Fm?
`
`
`$325.2mummmmon:
`
`4hzmrh3<
`
`em:
`
`Page 4 0f 35
`
`
`
`8..MES.53stngzOF<2mO....z.mozmwmmm
`
`
`
`
`
`Page 4 of 35
`
`
`
`
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jun. 19, 2007
`Jun. 19, 2007
`
`Sheet 4 0f 19
`Sheet 4 of 19
`
`US 7,233,589 B2
`US 7,233,589 B2
`
`—2%”;me
`
`n—_
`
`ww
`
`:Il XHOMJBN dI
`
`mm
`
`pan—z. 29an-wz_._m5mmm<mo
`meoai
`
`P¢
`
`mmmSmmwn_<
`
`
`
`mfim
`
`moSwn
`
`._<2mm._.xm
`
`mo<mprw
`
`mu_>mn
`
`:0555sz
`
`N¢
`
`meu<a
`
`
`
`02—N>:_<z<Han—SO
`
`oz<<55
`
`.azameOwE
`
`023(me
`
`
`
`zoFmO¢zQEOn—
`
`wz_._m5mwm<
`
`20.551
`
`
`
`
`
`2950102.22%zofiowzzoo
`
`TEE
`
`Page 5 0f 35
`
`Page 5 of 35
`
`
`
`
`
`

`

`U.S. Patent
`
`Jun. 19
`
`3
`
`2007
`
`Sheets 0f19
`
`US 7,233,589 B2
`
`0
`.vm
`
`
`mozfimuzoo85>389mEma5265:83
`
`
`
`
`
`$55.22mmmwmwmfl29.553:56.22mmmwmmood.$522
`
`
`
`mummum.vmmmumNmmram
`
`DOE
`
`Page 6 0f 35
`
`.u.
`
`§.***.§.*tn
`
`
`
`mamWEE.kg;zozbmzzou
`
`Page 6 of 35
`
`
`
`
`

`

`9,1n.
`
`9
`
`US 7,233,589 B2
`
`wz_._mzmmm<m593
`n2958
`
`
`hmsagage5E6SNmI.2H<Dmung—mmwwmwmmeggN.9..22an_ZOFMOLmwznsdfi
`
`6wawEoozézoo0255w.mMm9.595%
`299.85.3
`
`o29A.2.389:29an
`
`
`
`205.9.Mwznmsmmm<mmooQ/mOzfimwszoo(II.Pm.on56>32
`2058Jmagnumoznfizz,‘
`e2056".
`
`w2_._mzwmm<mo62.5?onm555.85>so.
`u20:5”.
`E.552$8.Ema
`
`
`U
`
`s”m0.”.
`
`Page 7 0f 35
`
`WViv
`
`w?
`
`SE85%.5&5
`
`
`magmamagma$58
`
`$5.05Ea
`
`—xxo‘shmz
`
`a:
`
`:il )l‘dOMlEIN cl!
`
`he
`
`N.22.szn:
`
`
`
`me
`
`Page 7 of 35
`
`
`
`

`

`U.S. Patent
`
`Jun. 19, 2007
`
`Sheet 7 0f 19
`
`US 7,233,589 B2
`
`FIGT
`
`631
`
`632
`
`(provision name=“Taro " >
`633
`(user account-1' " client B "
`password= " z9vBx7 '' 5 634
`addr= " m,***_*** " />
`(imsrv addr'; " =l==l==l=?**.*** " />
`<voipsrv addr= “ =I=**.***.*=I==I=* " />
`(/provision)
`
`PROFILE DATA 886
`
`Page 8 of 35
`
`

`

`U.S. Patent
`
`Jun. 19, 2007
`
`Sheet 8 0f 19
`
`US 7,233,589 B2
`
`QUE
`
`O
`3:1 NOIlHOd ONI'IBWiSSV lEMOVd
`
`V gm
`
`Page 9 of 35
`
`

`

`U
`
`MP
`
`t
`
`7002
`
`f09
`
`9
`
`2B
`
`S”m0—“—
`
`133
`9,20.521zonmon.m.63:2mewzzmfimwfl$582.”.
`
`
`
`Jm.on8.9..
`.$885w.20522.9EE%azamgmmmfiooznmswmmuwm
`
`
`
`mmm205305308“.dfizoo:5
`
`mu9$55352..mm,a3MW%W:mammm2,mN7OmSEmoEmzwWUflmworms
`$565BeEozmz4/w.v\5&8Suz—4<zmmhxwwe
`
`
`mama$53
`
`
`
`$55.8.0.;
`
`
`
`om:0.55m
`
`503m3;
`
`mzoTEES.
`
`mx102,er
`
`055.
`
`Page 10 0f 35
`
`29:62:21.8
`
`NVEQSEZ
`
`Page 10 of 35
`
`
`
`

`

`U.S. Patent
`
`Jun. 19, 2007
`
`Sheet 10 of 19
`
`US 7,233,589 B2
`
`2mg
`
`@m222me
`055.wEgo
`u<z=2mw_.n:u22.0
`
`U04.
`
`$2.
`
`Siam...
`
`wwzozmmd
`(leL
`2%2$59
`UElazingn:
`EE0mug—um
`
`m>0fig“.
`
`as
`
`mac—w
`
`3.00..EEme
`emmsmmwm«.5._<z_smm:.n:
`
`
`_<«5:05:ES.
`
`Page 11 0f 35
`
`385
`
` x0;
`
`I=0=.<m_o:w<m
`.95
`
`
`
`om~E015<260..82m
`
`5r.
`n5mozmwmmn.
`
`ZOEQZO
`
`ohmwzcmmIBu5wzChmw
`zOF<uE=bzmooFm
`
`mac5
`
`szEozoo
`
`
`
`mozwwmuzOo._.mmo55
`
`Page 11 of 35
`
`
`
`
`
`

`

`U.S. Patent
`
`Jun. 19, 2007
`
`Sheet 11 0f 19
`
`US 7,233,589 B2
`
`3.-_____gamma-Illa:
`
`gal:5:5533
`
`Nu
`
`figIGE
`
`0:950:.
`
`m225me2“:2,E.2235:1..35.313a:
`.u25:".u3%es
`
`0.9m22.55n:m«agen.EsauSmgmmwwmmmflmmmmunpwww¢mmbmmm<23.0
`lam—awn.@mmEE@samc
`
`m—opw
`5:>88Inezas6a:scam
`
`II
`Hall.H5%I025%
`rfifi
`
`$200
`
`Em
`
`
`
`3.55Rm:8».2:085>h35Iamsome
`
`Page 12 0f 35
`
`Page 12 of 35
`
`
`
`
`

`

`tnetaP
`
`70029a1
`
`hS
`
`21
`
`US 7,233,589 B2
`
`mup0E
`
`
`
`T52:“gem.th
`
`mEgan—a
`
`9.56m:5saw__isas.m;u<EuH50>:55£82Eu:_@555angm:_5E5“oEEO
`
`
`
`o“Beamefiflafiume25..guess2.20:”{Eu
`m.2.233«32.5JCzfiogéo833.\\
`
`v5“DEB mmP383:82...Bummums:
`
`\/\BF
`
`NEE”.8?2E32.,2:8
`
`Page 13 0f 35
`
`Page 13 of 35
`
`

`

`U.S. Patent
`
`Jun. 19, 2007
`
`Sheet 13 of 19
`
`US 7,233,589 B2
`
`9W5:
`
`mfg...—
`
`99Eézfimfla.w2.2.".n.“5:0
`m2%afig:
`
`.nlu,D32259EE3<z_z~_m:3“=9;E22.2%:a.
`
`,wemm>zmmmmsmmmmmmimmEmswmmEézfimmha. fipzmwflmmE«55
`m>a:n?s:<E._
`
`322m
`
`osuy
`
`UcNo—m
`
`J=410No.9—
`
`Ima.mm
`02225m:
`
`552:0mo_o>
`
`Swag”.
`
`.385
`
`02E.<:0wgox._mu:5
`
`HowzzOUmE
`
`Page 14 0f 35
`
`Page 14 of 35
`
`
`
`
`
`
`

`

`U.S. Patent
`
`Jun. 19, 2007
`
`Sheet 14 0f 19
`
`US 7,233,589 B2
`
`
`
`OZFEIOwo_o>
`
`wz_mw
`
`mafia
`
`.5:>003
`
`EFOZ
`
`nines;Eh320—”.
`
`9.390$.._<z_2mmha._m#2.".n.£23.<E230“Ego“—uco=u{IIJ\||\m>95.n35:
`
`
`n=o>.2mm:mummmm>mmmmmw>mwwA.Mm>mmm-T”.Hpic—mn”mmNmmzoImm._m_._._.N22.25;.n:EmN22.5%.“:a442.2me-
`
`
`
`
`
`on;Fm#8395.
`
`\.
`
`Page 15 0f 35
`
`
`
`hm...>n_n_:m
`
`
`
`ham—30mm2:0mu_o>
`
`«Em82m
`
`Page 15 of 35
`
`
`
`
`
`

`

`U.S. Patent
`
`Jun. 19, 2007
`
`Sheet15 0f19
`
`US 7,233,589 B2
`
`E
`
`é]m
`.1ENEAlaEmEIv
`
`F
`
`n
`
`Scum
`
`
`
`02:.me0282
`
`kww30mm
`
`EFOZ
`
`hm:>035.aoww
`
`oz_._._.<_._oH20)
`
`Emzcwm
`
`(Scum
`
`28m>83Emacmz?
`
`“5mozmflE
`
`.a8
`
`2m.me
`
`«comm
`
`85615..
`
`25388m%
`
`w.520u2%
`
`m.2%
`
`n«5%
`
`Sgfifliammfln:as,E22.szn:953533mmmmmémmEmsfim
`
`
`
`
`ScumHm:zmbenfluEEma
`z>ozn_<3<Eo=u
`
`9N5:
`
`2.6K
`
`any.
`
`N.»27.5mm._.n=
`
`Page 16 0f 35
`
`Page 16 of 35
`
`
`
`
`
`
`

`

`U.S. Patent
`
`Jun. 19, 2007
`
`Sheet 16 of 19
`
`US 7,233,589 B2
`
`2“>920E
`
`Siam:
`
`
`
`
` hmeOmm@ma£39.mEEEmpcwwQm.22..2w_m._.21:51.€2.59..52mmSmmmomm>wmmmmm>mwwvmm>mmwms.._<z_zm_m_._.n=
`muuEma—oI.—Eu=o_(JIKm>92ad.2—25.
`300.0.49—._<Z_S_mm._.n:mu:o“22.0<gum—u
`
`oENm
`
`709.5
`
`.5:
`
`9:ch
`
`(nu-gum
`
`225w22085
`
`
`
`
`jlrmmaommmesowmmozmwwwflmmm8%
`
`.52ScamIP02Comm
`
`ZOEQZOOmSum
`
`
`
`
`
`I295656.".20520.5(;Im8d<omeaomm
`
`mozmzmmzooHaI:on
`llzonazg
`
`Page 17 0f 35
`
`Page 17 of 35
`
`
`
`
`
`
`
`

`

`U.S. Patent
`
`Jun. 19, 2007
`
`Sheet17 0f19
`
`US 7,233,589 B2
`
`E
`
`8.9.2
`
`2.0.“—
`
`0€30
`
`
`
`n”n”.E:23”HUmmZOIn—mjw...UE3%.sz2%>TZEEEE,WEESEE
`
`oaé@Pmmwi.min95%2&9me33%EmbmmmgammaQua
`
`Page 18 0f 35
`
`maoomw
`
`lazOmMmm
`.II
`
`firl
`szmwfwnomxc_
`02x5Pmm20mm
`
`320-0529
`
`$2.55
`
`
`
`588£948“
`a:.anmmsmm
`
`mo_0>
`
`E52
`
`.5muzfimE
`
`«comm
`
`-
`
`wasEBQEPanama
`
`oHaoEE259"88l:39
`IE2“?
`
`Page 18 of 35
`
`
`
`
`
`
`

`

`U
`
`n.m
`
`1f0001w.
`
`US 7,233,589 B2
`
`9%II
`020mme
`.
`Illa
`
`
`-muoEEmmi
`
`I‘V'ISH
`
`1H
`
`025w
`
`0HEP,_2.s.M:0.“.
`:w.@EHEUHEECmhwmznznmdxh£255n__NN3.2%:n__t:2m:-a3._-mmfiqfim
`
`
`
`531$n:m«isa“5%amwwfiw@mememam/“mmEmbwmm55%MG9:050u#5:".EN.
`
`scamIN53mE52
`,.amuzmmmmn.w28m>88Ewaomm
`22%Eng”.m5...
`
`%9EE-_S=§
`
`Page 19 0f 35
`
`NEE—zoo0
`zifim
`
`36mm
`
`Page 19 of 35
`
`
`
`

`

`nH
`
`2
`
`a
`
`US 7,233,589 132
`
`Lnlavwatom—on.atom—u9.3.PE»>2.9.2mmwGE
`J“u_HHugVtMUSmmzoEmEFhflmflgEEE325mm:_9:25me%>-w-4._n.m992ézimm:n__mEgoo2%2«mu/mum
`mmwfimmmmwfimwwEbmmm<2%
`
`w.52%
`
`0momma_0.
`732.868“.~85
`
`omammmuomawIIIIAmomms.
`
`
`m:62
`
`Swami
`
`805
`
`EmE_
`
`Page 20 0f 35
`
`Page 20 of 35
`
`
`
`

`

`US 7,233,589 B2
`
`1
`COMMUNICATION SYSTEM AND
`COMMUNICATION METHOD
`
`BACKGROUND OF THE INVENTION
`
`The present invention relates to a communication tech-
`nology such as Instant Messaging (IM).
`A communication technology called IM is gathering
`attentions. Sometimes, a matter to talk with the other party
`may not need a rapid response through a telephone. How-
`ever, if an e-mail containing the matter is sent through an
`E-mail, it is not clear when the other party reads (that is,
`when his/her reply is obtained). IM has a time characteristic
`that is intermediate between the telephone and the e-mail. In
`IM, a current status (which may be called presence infor-
`mation), such as online or oflline or the ability to respond,
`of each party is published. Thus, in response to presence
`information of friends using the same IM system over the
`Internet, a tool to be used for communication with friends,
`such as a chat and an e-mail, can be selected.
`Presently, IM systems are released from many venders,
`and IM clients, each of which is application software to use
`a respective IM system, are distributed. However, there is no
`compatibility among IM systems of the respective venders.
`As a result, communication cannot be performed between
`the IM clients of the different venders. Thus, the standard-
`ization is started from a need for the interconnectivity. In
`Instant Messaging & Presence Protocol (IMPP) working
`group of Internet Engineering Task Force (IETF), the archi-
`tecture, the message integration and the security (such as
`authentication and encryption) relating to IM are being
`discussed. A model is defined for Require for Comment
`(RFC) 2778 while a protocol requirements are defined for
`RFC 2779.
`
`Conventionally, the multi-party connection (group chat)
`in IM is mainly used just for fun(hobby use), such as
`gossiping. However, recently, study for using the group chat
`for business has been started. Here,
`it is noted that the
`text-based group chat may be not enough for the business
`uses in many cases. For example, inputting Japanese lan-
`guage takes time. In addition, in order to express a compli-
`cated matter or a shade of meaning, which is hard to be
`understood from a document, the voice support is needed. In
`the one-to-one (Peer-to-Peer) technology, the expansion into
`communication using multimedia has been started. The first
`reference (Nikkei Communication, Nov. 5, 2001, pgs. 106 to
`113) proposes such usage of multimedia that a meeting is
`held by using a voice chat or by exchanging files.
`
`SUMMARY OF THE INVENTION
`
`When a communication technology is applied to the
`business purpose, it is desirable that not only text but also
`voice can be handled easily in addition to participation in a
`conference without leaving his/her desk and/or participation
`in conference even from the outside of his ofiice.
`
`However, the conventional IM systems are not ready for
`a group chat using multimedia. By using the system dis-
`closed in the reference 1, switching between text and voice
`can be performed in a one-to-one chat. However, no con-
`sideration is given to the switching between text and voice
`in a group chat.
`There is a television conference system as a technology
`for implementing a group chat other than the IM system.
`However,
`the conventional
`television conference system
`requires the buddies to gather in a room such as a television
`conference room at a scheduled conference time. In some
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`
`cases, users must call to a predetermined virtual conference
`room. That is, the conventional television conference system
`lacks flexibility.
`The present invention was made in view of these circum-
`stances. It is an object of the present invention to improve
`handling (dealing) of a communication system. More spe-
`cifically, it is an object of the present invention to achieve
`group chat using multimedia. For example, the switching
`between a group chat through electronic documents and a
`group chat through voice and/or the switching between a
`one-to-one chat through voice and a group chat through
`voice can be handled flexibly.
`the present invention
`In order to achieve the objects,
`implements a chat between/among IM clients by using an
`IM presence management server which manages presence
`information indicating a state of each of the IM clients; a
`VoIP communication connection management server which
`manages a connection for voice chatting using VoIP (Voice
`over Internet Protocol); and a media server which imple-
`ments multi-party voice communication by mixing voice
`data.
`
`For example, the IM presence management server man-
`ages, as to each of the IM clients, presence information of
`the IM client, information regarding media (including text
`and voice) which can be used for a chat by the IM client, and
`user information of the IM client, and in accordance with an
`instruction from an IM client, provides the IM client in
`question with the presence information, usable media infor-
`mation and user information of each of the IM clients set as
`
`buddies of the IM client in question.
`In accordance with information of each of IM clients
`
`(called as text participating clients) participating in a text
`chat, which is notified from the IM client, the IM presence
`management server manages a connection between/among
`the text participating clients and the IM presence manage-
`ment server, merges text data sent from each of the text
`participating clients and distributes the result to each of the
`text participating clients.
`The VoIP communication connection management server
`manages a connection between/among each of IM clients
`(called as voice participating clients) and the media server in
`accordance with information of the voice participating cli-
`ents participating in a voice chat, which is notified from the
`IM client or the IM presence management server.
`The media server performs processing for mixing voice
`data sent from each of the voice participating clients except
`for one of the voice participating clients set as a focused
`client and for distributing the result data thus mixed to the
`focused client. These processing is performed as focused
`clients on all of the voice participating clients.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`FIG. 1 is a schematic diagram of an IM-VoIP (Voice over
`Internet Protocol) interconnecting system, which is a com-
`munication system to which one embodiment of the present
`invention is applied;
`FIG. 2 is a schematic configuration diagram of an IM
`server 4 shown in FIG. 1;
`FIG. 3 is a diagram showing an example of a presence
`information management table 488 used by a connection
`managing portion 487 shown in FIG, 2 for managing pres-
`ence information;
`FIG. 4 is a schematic configuration diagram of an AP
`server 5 shown in FIG. 1;
`
`Page 21 0f 35
`
`Page 21 of 35
`
`

`

`US 7,233,589 B2
`
`3
`FIG. 5 is a diagram showing an example of a VoIP
`connection management table 586 used by a connection
`managing portion 585 shown in FIG. 4 for managing voice
`chat;
`FIG. 6 is a schematic configuration diagram of an IP
`terminal 7 shown in FIG. 1;
`FIG. 7 is a diagram showing an example of profile data
`686 required when an IM client participates in an IM;
`FIG. 8 is a schematic configuration diagram of an MD
`server 6 shown in FIG. 1;
`FIG. 9 is a schematic configuration diagram of a VR
`server 10 shown in FIG. 1;
`FIG. 10 is a diagram for explaining an operation for
`switching from a text chat to a voice chat, which is a first
`operational example of an IM-VoIP interconnecting system
`according to one embodiment of the present invention:
`FIG. 11 is a diagram for explaining an operation for
`switching from a text chat to a voice chat, which is a first
`operational example of an IM-VoIP interconnecting system
`according to one embodiment of the present invention;
`FIG. 12 is a diagram showing an example of a user
`interface of an IM client, which is displayed in an IP
`terminal 7;
`FIG. 13 is a diagram for explaining a variation example
`of the first operational example shown in FIG. 11;
`FIG. 14 is a diagram for explaining a variation example
`of the first operational example shown in FIG. 11;
`FIG. 15 is a diagram for explaining an operation for using
`a text chat during a voice chat, which is a second operational
`example of an IM-VoIP interconnecting system according to
`one embodiment of the present invention;
`FIG. 16 is a diagram for explaining an operation for using
`a text chat during a voice chat, which is a second operational
`example of an IM-VoIP interconnecting system according to
`one embodiment of the present invention.
`FIG. 17 is a diagram for explaining an operation for
`switching from a one-to-one voice chat to a multi-party
`voice chat, which is a third operational example of an
`IM-VoIP interconnecting system according to one embodi-
`ment of the present invention;
`FIG. 18 is a diagram for explaining an operation for
`switching from a one-to-one voice chat to a multi-party
`voice chat, which is a third operational example of an
`IM-VoIP interconnecting system according to one embodi-
`ment of the present invention; and
`FIG. 19 is a diagram for explaining an operation for
`closing a conference room, which is a fourth operational
`example of an IM-VoIP interconnecting system according to
`one embodiment of the present invention.
`
`DESCRIPTION OF THE PREFERRED
`EMBODIMENTS
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`Embodiments of the present invention will be described
`below.
`
`55
`
`FIG. 1 is a schematic diagram of an IM-Voice over
`Internet Protocol (VoIP) interconnecting system, which is a
`communication system to which one embodiment of the
`present invention is applied.
`As shown in FIG. 1, an IP network 1 is connected to an
`IM presence managing Server (called as IM server herein-
`after) 4 which manages presence information of an IM
`client, a VoIP communication connection managing server
`(called as AP server hereinafter) 5 which manages connec-
`tion for a voice chat using VoIP, a media server (called as
`MD server hereinafter) 6 which implements multi-party
`voice speech by mixing voice data, a plurality of IP termi-
`
`60
`
`65
`
`4
`nals 7-1 to 7-3 in each of which an IM client is installed, and
`a voice relay server (called as VR server hereinafter) 10
`which performs voice relay among a radio communication
`network 2, a publicly switched telephone network (PSTN) 3
`and the IP network 1. Here, a radio terminal 9, such as a
`mobile telephone,
`in which an IM client
`is installed,
`is
`connected to the radio communication network 2. A fixed
`
`telephone 11 in which an IM client is installed is connected
`to the publicly switched telephone network 3.
`In the above-described configuration, the IP terminals 7-1
`to 7-3 can perform a multi-media chat using the IM server
`4, the AP server 5 and the MD server 6 by independently or
`combination with the VoIP telephone 8, the radio terminal 9
`or the fixed telephone 11. For example, in a group or in
`one-to-one, a chat through electronic documents such as text
`(called as text chat hereinafter) and a chat through voice
`(called as voice chat hereinafter) can be performed.
`Next, each of devices included in the IM-VoIP intercon-
`necting system will be described.
`First of all, the IM server 4 will be described.
`As described above, the IM server 4 manages presence
`information of the IM clients. A schematic configuration of
`the IM server 4 is shown in FIG. 2.
`
`As shown in FIG. 2, the IM server 4 is achieved by
`configuring functional blocks 481 to 487 shown within a
`balloon 48 in FIG. 2 when a CPU 41 executes a predeter-
`mined program loaded onto a memory 42 in a general-
`purpose computer system including the CPU 41,
`the
`memory 42, an external storage device 42, such as an HDD,
`an input device 44, such as a keyboard, a mouse and a pen,
`an output device 45, such as a speaker and a display, an IP
`network interface (IF) 46 which performs communication
`over the IP network 1, and a bus 47 for connecting these
`devices 41 to 46. The program for configuring the functional
`blocks 481 to 487 may be directly loaded from a storage
`medium (not shown) such as a CD-ROM, through a reading
`device (not shown), or from the IP network 1 to the memory
`42 through the IP network IF 46. Altematively, the program
`may be stored in the external memory device 43 and then
`may be loaded to the memory 42.
`A packet deassembling portion 481 receives an IP packet
`addressed to itself from the IP network 1. Then, the packet
`deassembling portion 481 processes the IP header and the
`TCP/UDP header of the packet and extracts a payload.
`An analyzing and data/command distributing portion 482
`analyzes the content of the payload processed by the packet
`deassembling portion 481 and distributes the content to a
`data processing portion 483 as data or a command process-
`ing portion 484 as a command.
`The data processing portion 483 performs processing
`required for IM on data distributed by the analyzing and
`data/command distributing portion 482. For example, for
`each conference, a predetermined amount of latest text data
`items received from each of IM clients participating in the
`conference is held. Then,
`the held text data items are
`merged, and then data for text chat is created therefrom.
`The command processing portion 484 performs process-
`ing required for IM in accordance with the content of the
`command distributed by the analyzing and data/command
`distributed portion 482. For example, in accordance with a
`conference setting request command from an IM client, the
`command processing portion 484 performs processing
`required for setting a new conference. Further in accordance
`with a conference participating request command from an
`IM client, the command processing portion 484 performs
`processing required for calling the other IM clients to
`participate in the conference.
`
`Page 22 0f 35
`
`Page 22 of 35
`
`

`

`US 7,233,589 B2
`
`5
`An output data creating portion 485 creates output data in
`accordance with results of the processing by the data pro-
`cessing portion 483 and/or the command processing portion
`484.
`
`A packet assembling portion 486 assembles an IP packet
`by adding a TCP/UDP header and an IP header to the output
`data created by the output data creating portion 485. The
`packet assembling portion 486 sends the IP packet to the IP
`network 1.
`
`A connection managing portion 487 manages presence
`information of IM clients. In accordance with the presence
`information, the connection managing portion 487 controls
`each of the functional blocks 481 to 486 and relays between
`connections with the IM clients. Thus, the chat between the
`IM clients can be achieved.
`
`Incidentally, the packet deassembling portion 481 and the
`packet assembling portion 406 may be configured for
`example, in hardware with a use of such as integrated logic
`IC within the IP network IF 46.
`
`FIG. 3 shows an example of a presence information
`management table 488, which is used by the connection
`managing portion 487 for managing presence information.
`The presence information managing table 488 is stored in
`the external storage device 43, for example.
`As shown in FIG. 3, one record 440 includes a field 431
`for registering an account name of a user of an IM client, a
`field 432 for registering an address of the IM client, a field
`433 for registering a nickname of the user of the IM client,
`a field 434 for registering an authentication key for using the
`IM-VoIP interconnecting system, a field 435 for registering
`presence information of the IM client, a field 436 for
`registering a medium (text chat and voice chat), which can
`be used by IM client for a chat, a field 437 for registering an
`address (IP address or DNS and a port number) of a
`conference room in which the IM client participates, a field
`438 for registering a nickname of the conference room, and
`a field 439 for registering account names of the other IM
`clients with whom the IM client can chat.
`Here, when a terminal in which an IM client is installed
`is an IP terminal, an IP address or a DNS of the terminal is
`registered, as an address of the IM client, in the field 432.
`When the terminal in which the IM client is installed is a
`
`mobile telephone and/or a fixed telephone, a telephone
`number and an IP address or a DNS of a VR server 10 which
`
`voice relays between the mobile telephone and/or the fixed
`telephone and the IP network 1 are registered therein.
`Any one of information (OFF) indicating that the IM
`client is oflline, information (idle) indicating that the IM
`client is online but idle, information (text) indicating that the
`IM client is text-chatting, information (voice) indicating that
`the IM client is voice-chatting, and information (text/voice)
`indicating that the IM client is both text chatting and voice
`chatting is registered in the field 435 as presence informa-
`tion.
`
`An IP address assigned to the conference room is regis-
`tered in the field 437 as an address of the conference room.
`
`When a user uses a plurality of IM clients, for example,
`when the client terminal 7-1 is used for text-chatting and the
`VoIP telephone 8 is used for voice-chatting, a record 440 is
`registered for each of them.
`The AP server 5 will be described next.
`
`As described above, the AP server 5 manages a connec-
`tion for voice chatting using VoIP. A schematic configuration
`of the AP server 5 is shown in FIG. 4.
`
`As shown in FIG. 4, the AP server 5 is achieved by
`configuring functional blocks 581 to 585 shown within a
`balloon 58 in FIG. 4, when, a CPU 41 executes a predeter-
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`6
`mined program loaded onto a memory 42 in a computer
`system having the same hardware configuration as that of
`the IM server 4. The program for configuring the functional
`blocks 581 to 585 may be directly loaded from a storage
`medium (not shown) such as a CD-ROM, through a reading
`device (not shown), or from the IP network 1 to the memory
`42 through the IP network IF 46. Altematively, the program
`may be stored in an external memory device 43 and then
`may be loaded to the memory 42.
`A packet deassembling portion 581 receives an IP packet
`addressed to itself from the IP network 1. Then, the packet
`deassembling portion 581 processes an IP header and a
`TCP/UDP header of the packet and extracts a payload.
`An analyzing and processing portion 582 analyzes the
`content of the payload processed by the packet deassembling
`portion 581 and performs processing required for a voice
`chat in accordance with the content of the command stored
`
`in accordance with a voice chat
`therein. For example,
`requesting command from an IM client, processing required
`for calling the other parties of the conference in which the
`IM client participates to participate in a voice-chat is per-
`formed.
`
`An output data creating portion 583 creates output data in
`accordance with a result of the processing by the analyzing
`and processing portion 582.
`A packet assembling portion 584 assembles an IP packet
`by adding a TCP/UDP header and an IP header to the output
`data created by the output data creating portion 583. The
`packet creating portion 584 sends the created IP packet to
`the IP network 1.
`
`A connection managing portion 585 performs connection
`management of a voice chat. In other words, the connection
`managing portion 585 controls each of the functional blocks
`581 to 584 to manage a connection between an IM client and
`the MD server 6. Thus, voice chatting among IM clients
`through the MD server 6 can be achieved.
`Incidentally, the packet deassembling portion 581 and the
`packet assembling portion 586 may be configured for
`example, in hardware with a use of such as integrated logic
`IC within the IP network IF 46.
`
`FIG. 5 shows an example of a VoIP connection manage-
`ment table 586, which is used by the connection managing
`portion 585 for managing a voice chat. The VoIP connection
`managing table 586 is stored in the external storage device
`43, for example.
`As shown in FIG. 5, one record 540 includes a field 531
`for registering an account name of a user of an IM client, a
`field 532 for registering an address of the IM client, a field
`533 for registering a nickname of the user of the IM client,
`a field 534 for registering information regarding a method of
`encoding voice signals carried in a VoIP packet, a field 535
`for registering an address (IP address or DNS and a port
`number called as “address for voice chat” hereinafter) of the
`MD server 6, which is used for a voice chat in which the IM
`client participate, and a field 536 for registering a nickname
`of the conference room.
`Next, the IP terminal 7 will be described.
`As described above, an IM client to be used for chatting
`is installed in the IM terminal 7. A schematic configuration
`of the IP terminal 7 is shown in FIG. 6.
`
`As shown in FIG. 6, the IP terminal 7 is achieved by
`configuring functional blocks 781 to 794 shown within a
`balloon 78 in FIG. 6, when, a CPU 41 executes a predeter-
`mined program (IM client program) loaded onto a memory
`42 in a computer system having the same hardware con-
`figuration as that of the IM server 4. The program for
`configuring the functional blocks 781 to 794 may be directly
`
`Page 23 0f 35
`
`Page 23 of 35
`
`

`

`US 7,233,589 B2
`
`7
`loaded from a storage medium (not shown) such as a
`CD-ROM, through a reading device (not shown), or from
`the IP network 1 to the memory 42 through the IP network
`IF 46. Altematively,
`the program may be stored in the
`external memory device 43 and then may be loaded to the
`memory 42.
`A packet deassembling portion 781 receives an IP packet
`addressed to itself from the IP network 1. Then, the packet
`deassembling portion 781 processes a IP header and a
`TCP/UDP header of the packet and extracts a payload.
`An analyzing and voice packet/data/command distribut-
`ing portion 782 analyzes the content of the payload pro-
`cessed by the packet deassembling portion 781 and distrib-
`utes the content to a voice packet deassembling portion 785
`as a voice packet (VoIP packet), a data processing portion
`783 as data, or a command processing portion 784 as a
`command.
`
`The data processing portion 783 analyzes a type of the
`data distributed by the analyzing and voice packet/data/
`command distributing portion 782 and performs processing
`in accordance with the type. Then,
`the data processing
`portion 783 notifies the result to an output data creating
`portion 786.
`The command processing portion 784 analyzes the con-
`tent of the command distributed by the analyzing and voice
`packet/data/command distributing portion 782 and performs
`processing in accordance with the content. Then, the com-
`mand processing portion 784 notifies the result to the output
`data creating portion 786.
`The output data creating portion 485 creates output data
`in accordance with results of the processing by the data
`processing portion 483 and the command processing portion
`484. The created data is output from a display and/or speaker
`of the o