US007233589B2
`
`(12) United States Patent
`Tanigawa et a].
`
`(10) Patent N0.:
`(45) Date of Patent:
`
`US 7,233,589 B2
`Jun. 19, 2007
`
`(54) COMMUNICATION SYSTEM AND
`COMMUNICATION METHOD
`
`(75) Inventors: Keiko TanigaWa, Kawasaki (JP);
`Kazuma Yumoto, Hachiouji (JP);
`Kenta Shiga, Yokohama (JP); Toru
`Hoshi, Yokohama (JP)
`
`(73)
`
`Assignee: Hitachi, Ltd., Tokyo (JP)
`
`(*)
`
`Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 1170 days.
`
`(21)
`
`(22)
`
`(65)
`
`Appl. No.: 10/231,262
`
`Filed:
`
`Aug. 30, 2002
`
`Prior Publication Data
`
`OTHER PUBLICATIONS
`
`Nikkei Communication, Nov. 5, 2001, pp. 106-113.
`R. Perera, “Instant Messaging Gets Down to Business”, PC
`WORLD.COM, (http://www.pcworld.com) , Nov. 15, 2001.
`Mahy et al, “A Multi-Party Application Framework for SIP”,
`SIPPING Working Group Internet Draft, Feb. 2002.
`“Instant Messaging”, PC WORLD.COM, (http://wwwpcworld.
`com), May 22, 2000.
`M. Goldberg, “Generation IM”, Yahoo ! Internet Life, Apr. 2002,
`pp. l-6.
`D. Willmott, “Which IM Program Is for You?”, Yahoo ! Internet
`Life, Apr. 2002.
`J. Caplan, “amazing IM Tricks ! ”, Yahoo ! Internet Life, Apr. 2002.
`J. Caplan, “Amazing IM Tricks ! ”, Yahoo ! Internet Life, Apr. 2002.
`* cited by examiner
`
`Primary ExamineriPhirin Sam
`(74) Attorney, Agent, or F irmiMattingly, Stanger, Malur &
`Brundidge, PC.
`
`US 2004/0001480 A1
`
`Jan. 1, 2004
`
`(57)
`
`ABSTRACT
`
`Foreign Application Priority Data
`(30)
`Jun. 4, 2002
`
`(JP)
`
`........................... .. 2002-162940
`
`(51) Int. Cl.
`(2006.01)
`H04L 12/66
`(52) US. Cl. ..................................... .. 370/352; 370/493
`(58) Field of Classi?cation Search ................... .. None
`See application ?le for complete search history.
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`6/2002
`2002/0073142 A1 *
`l/2003
`2003/0021290 A1 *
`2003/0126213 Al* 7/2003
`2003/0149774 Al* 8/2003 McConnell et al. ...... .. 709/227
`2003/0210770 Al* ll/2003 Krejcarek .............. .. 379/8817
`
`Moran ...................... .. 709/203
`
`The usage of communication systems is improved. An IM
`server 4 manages presence information regarding each of IM
`clients and usable media and user information such that each
`of the IM clients can obtain these kinds of information. In
`order to perform text chatting, the IM server 4 manages a
`connection between each of the IM clients participating the
`chat and the IM server 4, merges text from each of the
`participating IM clients and distributes the result to each of
`the participating IM clients. In order to perform voice
`chatting, an AP server 5 manages a connection between each
`of the IM clients participating the chat and an MD server 6,
`mixes voice from each of the participating IM clients except
`for a focused IM client and distributes the result to the
`focused participating IM clients. This processing is per
`formed for each of the participating IM clients.
`
`12 Claims, 19 Drawing Sheets
`
`APSERVER
`
`5\I\.=
`
`IMSERVER
`
`3
`
`1
`
`I
`
`PUBLICLY
`SWITCHED
`TELEPHONE
`NETWORK
`
`1 1
`
`1:1
`Um FIXED
`TELEPHONE
`
`2
`
`DIO
`[:3 COMMUNI
`T N
`
`1 O
`\I\
`
`9
`
`VR SERVER
`
`IP NETWORK
`
`RADIO
`TERMINAL
`
`7-1
`[P Q TERMlNAL
`
`7-2
`f TERM'NM-
`
`TERMINAL
`
`/
`
`GOOGLE EXHIBIT 1014
`
`Page 1 of 35
`
`

`

`U.S. Patent
`
`Jun. 19, 2007
`
`Sheet 1 of 19
`
`US 7,233,589 B2
`
`
`
`TWNIWYALTWNIAMSLcldl
`
`USAYNSSVA
`
`SYOMLINdi
`
`ANOHdSaTSL
`
`O00)
`
`daxid
`CIC)
`hoo
`
`JNOHd3ST14L
`
`QSHOLIMS
`
`SOM!NUAAUISWI
`
`LL
`
`Cc
`
`©HANNAdiv
`
`Sls
`
`Page 2 of 35
`
`Page 2 of 35
`
`
`
`
`
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jun. 19, 2007
`Jun. 19, 2007
`
`Sheet 2 0f 19
`Sheet 2 of 19
`
`US 7,233,589 B2
`233,589 B2
`US 7
`2
`
`|MYOMLAN
`
`dl
`
`ai MYOMLAN di
`
`
`
`sna
`
`dv
`
`YMAACHSA
`
`AndNl
`“WNY3Xa
`
`JDVYOLS
`
`
`
`saneNdoAYOWAN
`
`LIxOVd
`
`ONMEINASSVSa0
`
`LayoVvd
`
`ONIZAWNYV
`
`NGONY
`
`ONVNWOS
`
`NOILYOd
`
`ONTLNGIMLSIC
`
`Sv
`
`NOILLYO.
`
`SV
`
`INdtNo
`
`SOAs
`
`SSIAZO
`
`VL¥G
`
`ONISSSOOUdiNdLNG
`
`NOIYOd
`
`ov
`
`YLvd
`
`ONILVSYO
`ONTaWASSV
`
`NOLLYOd
`
`
`
`
`
`NOLLYOdLNANSAOVNVINNOLLOANNOO
`
`6Old
`
`Page 3 of 35
`
`Page 3 of 35
`
`
`
`

`

`U.S. Patent
`
`Jun. 19, 2007
`
`Sheet 3 of 19
`
`US 7,233,589 B2
`
` “pyJaCoOay
`
`“oadO'a'V
`
`=nynsado'a'yv
`
`6erBEDLEVSEYSEVpercepZEVLev
`accra}.S2OGaN}Sasa
`
`80TIGVLLNSWEOVNYWNOILVWHOSNISONSSIYd
`
`
`
`
`sravsn|=NASSud4NG|inarioTNaND|LNNODO¥
`van|-SSWYNDOIN|SaSSaUCay
`
`&ols
`
`SAD
`
`Page 4 of 35
`
`Page 4 of 35
`
`
`
`
`

`

`U.S. Patent
`U.S. Patent
`
`Jun. 19, 2007
`Jun. 19, 2007
`
`Sheet 4 0f 19
`Sheet 4 of 19
`
`US 7,233,589 B2
`US 7,233,589 B2
`
`LXYCMIAN
`
`di
`
`dl MYOMLANdi
`
`
`
`srg
`
`LY
`
`GMANSdV
`
`OPov
`
`JOU
`
`indinoNddAYOWSIN
`
`AOIAIO
`
`8S
`
`NOUMOd
`
`voila
`—>}oynawassyvad-—>)NBSPONSoNONNEVESSSY
`
`
`
`LavoVdONIZATWNYIndinoLFAIVd
`NOILYOdNOILYOdNOULWOd
`
`
`
`
`
`
`
`NOLLUOdONISVNVIWNOILOSNNOD
`
`Page 5 of 35
`
`Page 5 of 35
`
`
`
`

`

`U.S. Patent
`
`Jun. 19
`
`, 2007
`
`Sheet 5 of 19
`
`US 7,233,589 B2
`
`9&S
`
`FINAYIANOD
`
`SSNVNXOIN
`
`sesvesgeszesLes
`tole°Aor
`
`
`SasssaydavS3INYN
`
`ANSITOINNODDY
`
`QINIGOQONA
`
`eekSickdolaioe
`
`
`
`
`
`98JTEV1LNSNSOVNYWNOLLOSNNOD
`
`Page 6 of 35
`
`Page 6 of 35
`
`
`
`
`

`

`|YYOMLAN
`
`dl
`
`dl HHOMLSIN di
`
`
`
`sng
`
`Lv
`
`2TWNIWYALdl
`
` vy
`
`ONTTENISSY
`
`
`
`NOLLYOdaNvunios
`
`1ay0Vd
`
`
`
`La4O¥dJDIOA
`
`ONNAWSSSY
`
`NOWLYOd
`
`ONITVSYO
`
`NOILYOd
`
`
`
`LAOVdSICA
`
`NANI
`WNYSDG
`
`SDIAA
`
`SJOIASG
`
`JOVYOLSNdd
`
`ONISSSOOUd
`
`NOLLYOd
`
`viva
`
`ONISSSOOUd
`
`CNVWWNOD
`
`NOILYOd
`
`ONILVAYO
`
`NOILYOd
`
`U.S. Patent
`
`Jun. 19, 2007
`
`Sheet 6 of 19
`
`US 7,233,589 B2
`
`9Dis
`
`BL
`
`97
`
`NOILYOd
`ONTIENASSVsa
`Ladd
`
`Sv
`
`LNdLNO
`
`3olAag
`
`JIIOACNV
`
`
`
`ONIZATYNYNOILYOd
`
`ONMEWSSSy3d
`
`NOILYOd
`
`any
`
`
`
`ONIZAIWNV
`
`NOILYOd
`
`LN3A4
`
`ONLLYSANODWd
`
`NOILYOd
`
`Page 7 of 35
`
`Page 7 of 35
`
`
`
`
`
`
`

`

`U.S. Patent
`
`Jun. 19, 2007
`
`Sheet 7 0f 19
`
`US 7,233,589 B2
`
`FIGT
`
`631
`
`632
`
`(provision name=“Taro " >
`633
`(user account-1' " client B "
`password= " z9vBx7 '' 5 634
`addr= " m,***_*** " />
`(imsrv addr'; " =l==l==l=?**.*** " />
`<voipsrv addr= “ =I=**.***.*=I==I=* " />
`(/provision)
`
`PROFILE DATA 886
`
`Page 8 of 35
`
`

`

`U.S. Patent
`
`Jun. 19, 2007
`
`Sheet 8 0f 19
`
`US 7,233,589 B2
`
`QUE
`
`O
`3:1 NOIlHOd ONI'IBWiSSV lEMOVd
`
`V gm
`
`Page 9 of 35
`
`

`

`U.S. Patent
`
`Jun. 19, 2007
`
`Sheet 9 of 19
`
`US 7,233,589 B2
`
`OIG
`
`NOLLOd
`
`ONNENASSY
`
`NOLLNOd
`
`¥Waqd09N3
`
`SYpy
`
`zsnao=2==2LMYOMISN|26(})(]()=di|22a
`
`IOVYOLSndoANONWN,orNd.LNOINdniTWNYALXSov
`~O}UAAMASUA8°|ve=Ly<2
`
`
`3OA30JOIAZ0
`B6
`
`
`
`NOLLWOd‘ONMaNZSSvad|-—>)TEN=sSvac ¥adoozdON}
`
`
`LaNOVdAION
`
`
`NOLLVOINNINHIOD
`
`éMHOMLAN
`
`Oldva
`
`SNOHdTSL
`
`€YHOMLAN
`
`CAHOLIMS
`
`ATONGNd
`
`Page 10 of 35
`
`Page 10 of 35
`
`
`
`

`

`U.S. Patent
`
`Jun. 19, 2007
`
`Sheet 10 of 19
`
`US 7,233,589 B2
`
`tyson
`
`aziolis
`
`pfoBIVNINHALEL
`Eyweyo=}wuayo
`olagvy=WNINALll
`3SNOHdI1LJ
`
`,awaysqpwerp
`dio,b-2IWNINALdl
`co—
`LOLYSANSS9YAANAS
`uAra
`boCLWHIWMALclSsansvuayuasyjuayD
`
`600RSSNOLLIGNCO
`daZIMOHLNYNFOO1—€00iS
`
`SONSY3S3NOO013S|iyO10LS
`NOLVOISILON8001S21S43NOAYP|
`
`
`z001S
`
`NID01
`
`df3ON3S3e8d
`
`SNILLSS
`
`aie}
`
`Page 11 of 35
`
`Page 11 of 35
`
`
`
`
`

`

`U.S. Patent
`
`Jun. 19, 2007
`
`Sheet 11 of 19
`
`US 7,233,589 B2
`
`=One|—PLOLS—.Ooowe,i(Jcv50,-=BiceWNWal3ayreWNLoyWANS9MAASSUaANAS—pNaANESCeedi
`
`wey4wayanole}
`{wasn>Bf—
`apaeeas
`oyeueywrLOI
`
`
`“JOONd‘‘XINLZ0LS9701S
`
`QNOdS3u
`
`
`
`1SZ0LS
`
`
`
`isiAganaSLOLS
`
`LVHDSION
`
`VvO0ZzOLS
`
`
`
`solsACCOR
`
`1S"Aggna
`
`Page 12 of 35
`
`Page 12 of 35
`
`
`
`

`

`U.S. Patent
`
`Jun. 19, 2007
`
`Sheet 12 of 19
`
`US 7,233,589 B2
`
`
`
`(——)eul]330:gqWel|)
`
`(NTOweHD
`
`epnpuly
`
`
`
`
`¢Buuioos1azojagpnogepayya]aAeyomsoUNOsoy)MOH
`
`
`BEL‘apoOSUNOSay]pussaseeig
`“nodupi4230)pIOU|BuIYYOWOSBAY|"HHA
`
`YELPSHY)Wlyroge|)y}e}OYBIEN$1
`
`
`
`{Guyy24yo89/004YayMsaMpio?
`
`obOlas
`
`<uoIpys>
`
`Page 13 of 35
`
`Zel
`
`40
`
`>WUalD»DWaID>WWaID
`
`“UE
`
`>HI
`
`Page 13 of 35
`
`

`

`U.S. Patent
`
`Jun. 19, 2007
`
`Sheet 13 of 19
`
`US 7,233,589 B2
`
`oeuey
`
`6IWNINYSLeddio,orddi
`E)jusyoyelp
`OOva=IWNINYALdl
`
`INLLLVHSDSOIOA
`gwayo=qwey
`Ss
`
`
`
`O}BRANESSAMASSUAANIS—PUaNES2WNIWYALdlIVow
`elSls
`=rONES
`
`ONIXINISSN
`LOANNOOSIG|SHOAaSZOLS
`acioistsiaes
`
`Liols
`
`
`
`LYHOSOIOA
`
`ISSNhO3Y
`
`PLOoLS
`
`LS
`
`Page 14 of 35
`
`Page 14 of 35
`
`
`
`
`
`
`

`

`U.S. Patent
`
`Jun. 19, 2007
`
`Sheet 14 of 19
`
`US 7,233,589 B2
`
`aTre9020Jgozoisavesau]—cinAVYSTae2$$Frevozols
`
`
`CNELaiaoeHeeObuaeasOUSAMIS SyamMaSYrUa/H3SohNEAL
`
`
`pw=gue,——y———"Wiouey
`—ariOis—oOtofobLoee
`
`“svviSls
`
`oyeuey
`
`Page 15 of 35
`
`ONIXIA
`Sanda
`
`isnAgana
`
`AgILON
`
`
`
`L311AGONY
`
`
`
`ogiais‘1S4i03u
`
`
`
`+saf3eTWDSOI0A
`
`ONISS
`
`
`
`SOOKE[Ruzors|—ezois
`
`eeaR]is
`
`
`
`Page 15 of 35
`
`
`
`
`
`
`

`

`U.S. Patent
`
`Jun. 19, 2007
`
`Sheet 15 of 19
`
`US 7,233,589 B2
`
`&)yuays4wep
`
`oyeuey
`
`SiSid
`
`
`
`
`.el3strI-ZTYNIWMALdlelAWAdi6TWNIAMSLdion
`
`
`oava‘TWNIWISLdi3yusyoqwayoof-anNaS8bauas$balesyaSyywera
`=UUJOfL1002S
`
`Oae}
`
`sie]ofaoa|__|
`
`QNILIESaAldIdGACD
`
`Lo0zSissndad
`
`
`Sanday
`
`SNILLSSONIXIN
`
`
`
`ONILLVHOADIOA
`
`AdILON
`
`ishAgond9002S
`
`10SDNAS3ud
`
`
`
`soosAgandisanosa
`
`ron!
`
`[anSONSSHed
`
`CaZMOHLAW
`
`NSOeo0es
`
`Page 16 of 35
`
`Page 16 of 35
`
`
`
`
`
`

`

`U.S. Patent
`
`Jun. 19, 2007
`
`Sheet 16 of 19
`
`US 7,233,589 B2
`
`8rzocs
`
`VEzOzS
`
`9102¢8
`
`
`
`AGQN8LSSNO3Y
`
`1S}
`
`
`
`NOILOSNNOOSIG
`
`
`
`_——\SeNILIVHO[AsILON
`
`SOYWAN
`
`9c02S
`
`IysDA
`
`:6TWNINYAL
`(CI
`
`e-L
`
`_)wall
`OldWy
`TWNINEALell
`Jjwaya
`
`oyeuey
`
`9}'Sis
`
`83tra
`Cl
`
`dion
`“LTYNIAELdl
`UEOlues9BSAUAS
`
`ego
`
`aw
`
`dv
`
`|.GYAAN3S
`japYSAYASWi
`
`st0cSC=LV2-2TWNINYALdi
`
`YWwarp
`
`O1e}
`
`Page17 of 35
`
`Page 17 of 35
`
`
`
`
`
`

`

`U.S. Patent
`
`Jun. 19, 2007
`
`Sheet 17 of 19
`
`US 7,233,589 B2
`
`oyeuey
`
`
`
`
` NOT>>zonesNI-901Clwe,OO1.(LJLL.-LV6TWHwatoea“ToteasSuauaSSUAS=pUAAUASBh.
`5quay4qualyoO18}
`
`Page 18 of 35
`
`¥eoorsas9NILLAS
`aNods=4|iINO)
`Z}Dis
`(HONO'AD
`LOOES_ishLawToare
`ag>|4SNS
`(HONG
`
`ONXINLS3N
`a1SOaeNI-30
`CSZROHLNY
`
`ONILLYHO
`
`FQIOA
`
`ICNOdS34]
`
`Vé00es
`
`Page 18 of 35
`
`
`
`
`
`

`

`U.S. Patent
`
`Jun. 19, 2007
`
`Sheet 18 of 19
`
`US 7,233,589 B2
`
`
`
`oyeuRy
`
`6WNINYAL2dio,=b-ZWWNINYALdi2-2WNINUALdi
`PHyaAYaSoeepyySeeoeMAandiWlvate
`fo2.0.022LBEV
`
`OLUSANSSDUSAYSSSYSANgS
`
`Page 19 of 35
`
`
`
`9N
`
`ILiSS
`
`ozoesS
`
`%B
`
`TIVO
`
`
`
`WROILIGGYLS4N038
`
`ZLacs
`
`
`
`AQGndLs3and3y
`
`‘10SONAS3Ud
`
`amSfFons
`|~~XI
`31SWdN09
`
`ONILLSSONIXIN
`
`
`
`
`
`Page 19 of 35
`
`
`
`
`
`
`

`

`U.S. Patent
`
`Jun. 19, 2007
`
`Sheet 19 of 19
`
`US 7,233,589 B2
`
`—_—I
`
`1ysoA
`
`cyeuey
`
`y00rS
`
`
`2§oerIHLWNINYALdli(J|.[.”(AVNIWMALdl6TYNINRELLdenhs2-2TYNIdOlavu‘IWNINYALdiJqoljoqweroObUUsaa$YEAYESyCEAUAS
`
`Hwa4waysose}
`\yquaryo
`
`
`LitaOL|
`ONISSSION—|a—-*SOUS
`
`6}Did
`
`L00vS
`
`QNISSIOON|,
`
`Z00rS
`
`f00rSLitaOL
`
`issn0sy
`
`Page 20 of 35
`
`Page 20 of 35
`
`
`
`
`
`

`

`US 7,233,589 B2
`
`1
`COMMUNICATION SYSTEM AND
`COMMUNICATION METHOD
`
`BACKGROUND OF THE INVENTION
`
`The present invention relates to a communication tech-
`nology such as Instant Messaging (IM).
`A communication technology called IM is gathering
`attentions. Sometimes, a matter to talk with the other party
`may not need a rapid response through a telephone. How-
`ever, if an e-mail containing the matter is sent through an
`E-mail, it is not clear when the other party reads (that is,
`whenhis/her reply is obtained). IM hasa time characteristic
`that is intermediate between the telephone andthe e-mail. In
`IM,a current status (which may be called presence infor-
`mation), such as online or offline or the ability to respond,
`of each party is published. Thus, in response to presence
`information of friends using the same IM system over the
`Internet, a tool to be used for communication with friends,
`such as a chat and an e-mail, can be selected.
`Presently, IM systems are released from many venders,
`and IM clients, each of which is application software to use
`a respective IM system,are distributed. However,there is no
`compatibility among IM systemsof the respective venders.
`As a result, communication cannot be performed between
`the IM clients of the different venders. Thus, the standard-
`ization is started from a need for the interconnectivity. In
`Instant Messaging & Presence Protocol (IMPP) working
`group of Internet Engineering Task Force (IETF), the archi-
`tecture, the message integration and the security (such as
`authentication and encryption) relating to IM are being
`discussed. A model is defined for Require for Comment
`(RFC) 2778 while a protocol requirements are defined for
`RFC 2779.
`
`Conventionally, the multi-party connection (group chat)
`in IM is mainly used just for fun(hobby use), such as
`gossiping. However,recently, study for using the group chat
`for business has been started. Here,
`it is noted that the
`text-based group chat may be not enough for the business
`uses in many cases. For example, inputting Japanese lan-
`guage takes time. In addition, in order to express a compli-
`cated matter or a shade of meaning, which is hard to be
`understood from a document, the voice support is needed. In
`the one-to-one (Peer-to-Peer) technology, the expansion into
`communication using multimedia has been started. Thefirst
`reference (Nikkei Communication, Nov. 5, 2001, pgs. 106 to
`113) proposes such usage of multimedia that a meeting is
`held by using a voice chat or by exchangingfiles.
`
`SUMMARY OF THE INVENTION
`
`When a communication technology is applied to the
`business purpose, it is desirable that not only text but also
`voice can be handled easily in addition to participation in a
`conference without leaving his/her desk and/or participation
`in conference even from the outside of his office.
`
`2
`cases, users must call to a predetermined virtual conference
`room. Thatis, the conventional television conference system
`lacks flexibility.
`The present invention was made in view of these circum-
`stances. It is an object of the present invention to improve
`handling (dealing) of a communication system. More spe-
`cifically, it is an object of the present invention to achieve
`group chat using multimedia. For example, the switching
`between a group chat through electronic documents and a
`group chat through voice and/or the switching between a
`one-to-one chat through voice and a group chat through
`voice can be handled flexibly.
`the present invention
`In order to achieve the objects,
`implements a chat between/among IM clients by using an
`IM presence management server which manages presence
`information indicating a state of each of the IM clients; a
`VoIP communication connection management server which
`manages a connection for voice chatting using VoIP (Voice
`over Internet Protocol); and a media server which imple-
`ments multi-party voice communication by mixing voice
`data.
`
`For example, the IM presence management server man-
`ages, as to each of the IM clients, presence information of
`the IM client, information regarding media (including text
`and voice) which can be used for a chat by the IM client, and
`user information of the IM client, and in accordance with an
`instruction from an IM client, provides the IM client in
`question with the presence information, usable media infor-
`mation and user information of each of the IM clients set as
`buddies of the IM client in question.
`In accordance with information of each of IM clients
`
`(called as text participating clients) participating in a text
`chat, which is notified from the IM client, the IM presence
`management server manages a connection between/among
`the text participating clients and the IM presence manage-
`ment server, merges text data sent from each of the text
`participating clients and distributes the result to each of the
`text participating clients.
`The VoIP communication connection management server
`manages a connection between/among each of IM clients
`(called as voice participating clients) and the media serverin
`accordance with information of the voice participating cli-
`ents participating in a voice chat, which is notified from the
`IM client or the IM presence managementserver.
`The media server performs processing for mixing voice
`data sent from each of the voice participating clients except
`for one of the voice participating clients set as a focused
`client and for distributing the result data thus mixed to the
`focused client. These processing is performed as focused
`clients on all of the voice participating clients.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`FIG. 1 is a schematic diagram of an IM-VoIP (Voice over
`Internet Protocol) interconnecting system, which is a com-
`However, the conventional IM systemsare not ready for
`munication system to which one embodimentofthe present
`a group chat using multimedia. By using the system dis-
`invention is applied;
`closed in the reference 1, switching between text and voice
`FIG. 2 is a schematic configuration diagram of an IM
`can be performed in a one-to-one chat. However, no con-
`server 4 shown in FIG. 1;
`sideration is given to the switching between text and voice
`FIG. 3 is a diagram showing an example of a presence
`in a group chat.
`information management table 488 used by a connection
`There is a television conference system as a technology
`managing portion 487 shown in FIG, 2 for managing pres-
`for implementing a group chat other than the IM system.
`ence information;
`However,
`the conventional
`television conference system
`
`requires the buddies to gather in a room suchasatelevision FIG. 4 is a schematic configuration diagram of an AP
`conference room at a scheduled conference time. In some
`server 5 shown in FIG. 1;
`
`60
`
`65
`
`Page 21 of 35
`
`Page 21 of 35
`
`

`

`3
`FIG. 5 is a diagram showing an example of a VoIP
`connection management table 586 used by a connection
`managing portion 585 shownin FIG. 4 for managing voice
`chat;
`FIG. 6 is a schematic configuration diagram of an IP
`terminal 7 shownin FIG.1;
`FIG. 7 is a diagram showing an example ofprofile data
`686 required when an IMclient participates in an IM;
`FIG. 8 is a schematic configuration diagram of an MD
`server 6 shown in FIG. 1;
`FIG. 9 is a schematic configuration diagram of a VR
`server 10 shown in FIG. 1;
`FIG. 10 is a diagram for explaining an operation for
`switching from a text chat to a voice chat, whichis a first
`operational example of an IM-VoIP interconnecting system
`according to one embodiment of the present invention:
`FIG. 11 is a diagram for explaining an operation for
`switching from a text chat to a voice chat, whichis a first
`operational example of an IM-VoIP interconnecting system
`according to one embodiment of the present invention;
`FIG. 12 is a diagram showing an example of a user
`interface of an IM client, which is displayed in an IP
`terminal 7;
`FIG. 13 is a diagram for explaining a variation example
`of the first operational example shownin FIG. 11;
`FIG. 14 is a diagram for explaining a variation example
`of the first operational example shownin FIG. 11;
`FIG.15 is a diagram for explaining an operation for using
`a text chat during a voice chat, which is a second operational
`example of an IM-VoIP interconnecting system according to
`one embodiment of the present invention;
`FIG.16 is a diagram for explaining an operation for using
`a text chat during a voice chat, which is a second operational
`example of an IM-VoIP interconnecting system according to
`one embodiment of the present invention.
`FIG. 17 is a diagram for explaining an operation for
`switching from a one-to-one voice chat to a multi-party
`voice chat, which is a third operational example of an
`IM-VoIP interconnecting system according to one embodi-
`ment of the present invention;
`FIG. 18 is a diagram for explaining an operation for
`switching from a one-to-one voice chat to a multi-party
`voice chat, which is a third operational example of an
`IM-VoIP interconnecting system according to one embodi-
`ment of the present invention; and
`FIG. 19 is a diagram for explaining an operation for
`closing a conference room, which is a fourth operational
`example of an IM-VoIP interconnecting system according to
`one embodiment of the present invention.
`
`DESCRIPTION OF THE PREFERRED
`EMBODIMENTS
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`Embodiments of the present invention will be described
`below.
`
`55
`
`FIG. 1 is a schematic diagram of an IM-Voice over
`Internet Protocol (VoIP) interconnecting system, which is a
`communication system to which one embodiment of the
`present invention is applied.
`As shown in FIG. 1, an IP network 1 is connected to an
`IM presence managing Server (called as IM server herein-
`after) 4 which manages presence information of an IM
`client, a VoIP communication connection managing server
`(called as AP server hereinafter) 5 which manages connec-
`tion for a voice chat using VoIP, a media server (called as
`MDserver hereinafter) 6 which implements multi-party
`voice speech by mixing voice data, a plurality of IP termi-
`Page 22 of 35
`
`60
`
`65
`
`US 7,233,589 B2
`
`4
`nals 7-1 to 7-3 in each of which an IM clientis installed, and
`a voice relay server (called as VR server hereinafter) 10
`which performs voice relay among a radio communication
`network2, a publicly switched telephone network (PSTN)3
`and the IP network 1. Here, a radio terminal 9, such as a
`mobile telephone,
`in which an IM client
`is installed,
`is
`connected to the radio communication network 2. A fixed
`
`5
`
`telephone 11 in which an IM clientis installed is connected
`to the publicly switched telephone network 3.
`In the above-described configuration, the IP terminals 7-1
`to 7-3 can perform a multi-media chat using the IM server
`4, the AP server 5 and the MD server 6 by independently or
`combination with the VoIP telephone8, the radio terminal 9
`or the fixed telephone 11. For example, in a group or in
`one-to-one, a chat through electronic documents such as text
`(called as text chat hereinafter) and a chat through voice
`(called as voice chat hereinafter) can be performed.
`Next, each of devices included in the IM-VoIP intercon-
`necting system will be described.
`First of all, the IM server 4 will be described.
`As described above, the IM server 4 manages presence
`information of the IM clients. A schematic configuration of
`the IM server 4 is shownin FIG.2.
`
`As shown in FIG. 2, the IM server 4 is achieved by
`configuring functional blocks 481 to 487 shown within a
`balloon 48 in FIG. 2 when a CPU 41 executes a predeter-
`mined program loaded onto a memory 42 in a general-
`purpose computer system including the CPU 41,
`the
`memory 42, an external storage device 42, such as an HDD,
`an input device 44, such as a keyboard, a mouse and a pen,
`an output device 45, such as a speaker and a display, an IP
`network interface (IF) 46 which performs communication
`over the IP network 1, and a bus 47 for connecting these
`devices 41 to 46. The program for configuring the functional
`blocks 481 to 487 may be directly loaded from a storage
`medium (not shown) such as a CD-ROM,through a reading
`device (not shown), or from the IP network 1 to the memory
`42 through the IP network IF 46. Alternatively, the program
`may be stored in the external memory device 43 and then
`may be loaded to the memory 42.
`A packet deassembling portion 481 receives an IP packet
`addressed to itself from the IP network 1. Then, the packet
`deassembling portion 481 processes the IP header and the
`TCP/UDPheader of the packet and extracts a payload.
`An analyzing and data/commanddistributing portion 482
`analyzes the content of the payload processed by the packet
`deassembling portion 481 and distributes the content to a
`data processing portion 483 as data or a commandprocess-
`ing portion 484 as a command.
`The data processing portion 483 performs processing
`required for IM on data distributed by the analyzing and
`data/command distributing portion 482. For example, for
`each conference, a predetermined amountof latest text data
`items received from each of IM clients participating in the
`conference is held. Then,
`the held text data items are
`merged, and then data for text chat is created therefrom.
`The command processing portion 484 performs process-
`ing required for IM in accordance with the content of the
`command distributed by the analyzing and data/command
`distributed portion 482. For example, in accordance with a
`conference setting request command from an IM client, the
`command processing portion 484 performs processing
`required for setting a new conference. Further in accordance
`with a conference participating request command from an
`IM client, the command processing portion 484 performs
`processing required for calling the other IM clients to
`participate in the conference.
`
`Page 22 of 35
`
`

`

`US 7,233,589 B2
`
`5
`An output data creating portion 485 creates output data in
`accordance with results of the processing by the data pro-
`cessing portion 483 and/or the commandprocessing portion
`484.
`
`A packet assembling portion 486 assembles an IP packet
`by adding a TCP/UDP headerandan IP headerto the output
`data created by the output data creating portion 485. The
`packet assembling portion 486 sends the IP packet to the IP
`network 1.
`
`A connection managing portion 487 manages presence
`information of IM clients. In accordance with the presence
`information, the connection managing portion 487 controls
`each of the functional blocks 481 to 486 and relays between
`connections with the IM clients. Thus, the chat between the
`IM clients can be achieved.
`Incidentally, the packet deassembling portion 481 and the
`packet assembling portion 406 may be configured for
`example, in hardware with a use of such as integrated logic
`IC within the IP network IF 46.
`
`FIG. 3 shows an example of a presence information
`management table 488, which is used by the connection
`managing portion 487 for managing presence information.
`The presence information managing table 488 is stored in
`the external storage device 43, for example.
`As shown in FIG. 3, one record 440 includes a field 431
`for registering an account nameofa user of an IM client, a
`field 432 for registering an address of the IM client, a field
`433 for registering a nicknameofthe user of the IM client,
`a field 434 for registering an authentication key for using the
`IM-VoIP interconnecting system, a field 435 for registering
`presence information of the IM client, a field 436 for
`registering a medium (text chat and voice chat), which can
`be used by IM client for a chat, a field 437 for registering an
`address (IP address or DNS and a port number) of a
`conference room in which the IM client participates, a field
`438 for registering a nicknameof the conference room, and
`a field 439 for registering account names of the other IM
`clients with whom the IM client can chat.
`Here, when a terminal in which an IMclient is installed
`is an IP terminal, an IP address or a DNSofthe terminal is
`registered, as an address of the IM client, in the field 432.
`Whenthe terminal in which the IM client is installed is a
`
`mobile telephone and/or a fixed telephone, a telephone
`number and an JP address or a DNS ofa VRserver 10 which
`voice relays between the mobile telephone and/orthe fixed
`telephone and the IP network 1 are registered therein.
`Any one of information (OFF) indicating that the IM
`client is offline, information (idle) indicating that the IM
`client is online but idle, information (text) indicating that the
`IM client is text-chatting, information (voice) indicating that
`the IM client is voice-chatting, and information (text/voice)
`indicating that the IM client is both text chatting and voice
`chatting is registered in the field 435 as presence informa-
`tion.
`
`An IP address assigned to the conference room is regis-
`tered in the field 437 as an address of the conference room.
`
`Whena user uses a plurality of IM clients, for example,
`whenthe client terminal 7-1 is used for text-chatting and the
`VoIP telephone 8 is used for voice-chatting, a record 440 is
`registered for each of them.
`The AP server 5 will be described next.
`
`Asdescribed above, the AP server 5 manages a connec-
`tion for voice chatting using VoIP. A schematic configuration
`of the AP server 5 is shown in FIG.4.
`
`As shown in FIG. 4, the AP server 5 is achieved by
`configuring functional blocks 581 to 585 shown within a
`balloon 58 in FIG. 4, when, a CPU 41 executes a predeter-
`Page 23 of 35
`
`20
`
`25
`
`30
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`6
`mined program loaded onto a memory 42 in a computer
`system having the same hardware configuration as that of
`the IM server 4. The program for configuring the functional
`blocks 581 to 585 may be directly loaded from a storage
`medium (not shown) such as a CD-ROM,through a reading
`device (not shown), or from the IP network 1 to the memory
`42 through the IP network IF 46. Alternatively, the program
`may be stored in an external memory device 43 and then
`may be loaded to the memory 42.
`A packet deassembling portion 581 receives an IP packet
`addressed to itself from the IP network 1. Then, the packet
`deassembling portion 581 processes an IP header and a
`TCP/UDPheader of the packet and extracts a payload.
`An analyzing and processing portion 582 analyzes the
`contentof the payload processed by the packet deassembling
`portion 581 and performs processing required for a voice
`chat in accordance with the content of the commandstored
`
`in accordance with a voice chat
`therein. For example,
`requesting commandfrom an IM client, processing required
`for calling the other parties of the conference in which the
`IM client participates to participate in a voice-chat is per-
`formed.
`An output data creating portion 583 creates output data in
`accordance with a result of the processing by the analyzing
`and processing portion 582.
`A packet assembling portion 584 assembles an IP packet
`by adding a TCP/UDP headerandan IP headerto the output
`data created by the output data creating portion 583. The
`packet creating portion 584 sends the created IP packet to
`the IP network 1.
`
`A connection managing portion 585 performs connection
`managementof a voice chat. In other words, the connection
`managing portion 585 controls each of the functional blocks
`581 to 584 to manage a connection between an IM client and
`the MD server 6. Thus, voice chatting among IM clients
`through the MD server 6 can be achieved.
`Incidentally, the packet deassembling portion 581 and the
`packet assembling portion 586 may be configured for
`example, in hardware with a use of such as integrated logic
`IC within the IP network IF 46.
`
`FIG. 5 shows an example of a VoIP connection manage-
`ment table 586, which is used by the connection managing
`portion 585 for managing a voice chat. The VoIP connection
`managing table 586 is stored in the external storage device
`43, for example.
`As shown in FIG. 5, one record 540 includes a field 531
`for registering an account nameof a user of an IM client, a
`field 532 for registering an address of the IM client, a field
`533 for registering a nickname ofthe user of the IM client,
`a field 534 for registering information regarding a method of
`encoding voice signals carried in a VoIP packet, a field 535
`for registering an address (IP address or DNS and a port
`numbercalled as “address for voice chat” hereinafter) of the
`MDserver6, which is used for a voice chat in which the IM
`client participate, and a field 536 for registering a nickname
`of the conference room.
`Next, the IP terminal 7 will be described.
`As described above, an IM client to be used for chatting
`is installed in the IM terminal 7. A schematic configuration
`of the IP terminal 7 is shown in FIG.6.
`As shown in FIG. 6, the IP terminal 7 is achieved by
`configuring functional blocks 781 to 794 shown within a
`balloon 78 in FIG. 6, when, a CPU 41 executes a predeter-
`mined program (IM client program) loaded onto a memory
`42 in a computer system having the same hardware con-
`figuration as that of the IM server 4. The program for
`configuring the functional blocks 781 to 794 may bedirectly
`
`Page 23 of 35
`
`

`

`US 7,233,589 B2
`
`7
`loaded from a storage medium (not shown) such as a
`CD-ROM,through a reading device (not shown), or from
`the IP network 1 to the memory 42 through the IP network
`IF 46. Alternatively,
`the program may be stored in the
`external memory device 43 and then may be loaded to the
`memory 42.
`A packet deassembling portion 781 receives an IP packet
`addressed to itself from the IP network 1. Then, the packet
`deassembling portion 781 processes a IP header and a
`TCP/UDPheader of the packet and extracts a payload.
`An analyzing and voice packet/data/commanddistribut-
`ing portion 782 analyzes the content of the payload pro-
`cessed by the packet deassembling portion 781 and distrib-
`utes the content to a voice packet deassembling portion 785
`as a voice packet (VoIP packet), a data processing portion
`783 as data, or a command processing portion 784 as a
`command.
`
`The data processing portion 783 analyzes a type of the
`data distributed by the analyzing and voice packet/data/
`command distributing portion 782 and performs processing
`in accordance with the type. Then,
`the data processing
`portion 783 notifies the result to an output data creating
`portion 786.
`The command processing portion 784 analyzes the con-
`tent of the commanddistributed by the analyzing and voice
`packet/data/commanddistributing portion 782 and performs
`processing in accordance with the content. Then, the com-
`mandprocessing portion 784 notifies the result to the output
`data creating portion 786.
`The output data creating portion 485 creates output data
`in accordance with results of the processing by the data
`processing portion 483