lllllllilllllllllllllillllllllllllllllllH
`
`US 20040001480A1
`
`(iQ) United States
`(I?) Patent Application Publication (io> Pub. No.: US 2004/0001480 Al
`Jan. 1,2004
`Tanigawa et al.
`(43) Pub. Date:
`
`(54) COMMUNICATION SYSTEM AND
`COMMUNICATION METHOD
`
`(76) Inventors: Keiko Tanigawa, Kawasaki (JP);
`Kuzunut Yumoto, Ilachiouji (JP);
`Kenta Shiga. Yokohama (JP); Torn
`Hoshi, Yokohama (JP)
`
`Correspondence Address:
`MATTINGLY, STANGER & MALUR, I’.C.
`I SOO DIAGONAL ROAD
`SUITE 37(1
`ALEXANDRIA, VA 22314 (US)
`
`(21) Appl. No. 10/231,262
`
`Piled:
`
`Aug. 30. 2002
`
`(30)
`
`Foreign Application Priority Data
`
`Jun. 4. 2002 (JP)
`
` 2002-162940
`
`Publication Classification
`
`(51) lot. CT,7
`
`H04L 12/66
`
`(52) U.S. Cl
`
`
`
`...................... 370/352
`
`(57)
`
`ABSTRACT
`
`1’he usage of communication systems is improved. An IM
`server 4 manages presence information regarding each of IM
`clients and usable media and user information such that each
`of the IM clients can obtain these kinds of information. In
`order Io perform text chatting, the IM server 4 manages a
`connection between each of the IM clients participating the
`chat and the IM server 4, merges text from each of the
`participating IM clients and distributes the result to each of
`the participating IM clients. In order to perform voice
`chatting, an Al3 server 5 manages a connection between each
`of the IM clients participating the chat and an MD server 6.
`mixes voice from each of the participating IM clients except
`for a focused IM client and distributes the result to the
`focused participating IM clients This processing is per­
`formed for each of the participating IM clients.
`
`Epic Games Ex. 1008
`Page 1
`
`

`

`Patent Application Publication Jan. 1,2004 Sheet lot’19 US 2004/0001480 Al
`
`Epic Games Ex. 1008
`Page 2
`
`

`

`Patent Application Publication Jan. 1,2004 Sheet 2 of 19 US 2004/0001480 Al
`
`WSSWER4
`
`Epic Games Ex. 1008
`Page 3
`
`

`

`Patent Application Publication .Ian. 1, 2004 Sheet 3 of 19 US 2004/0001480 Al
`
`Epic Games Ex. 1008
`Page 4
`
`

`

`Patent Application Publication .Ian. 1, 2004 Sheet 4 of 19 US 2004/0001480 Al
`
`Epic Games Ex. 1008
`Page 5
`
`

`

`Patent Application Publication .Jan. 1,2004 Sheet 5 of 19 US 2004/0001480 Al
`
`Epic Games Ex. 1008
`Page 6
`
`

`

`Patent Application Publication Jan. 1,2004 Sheet 6 of 19 US 2004/0001480 Al
`
`Epic Games Ex. 1008
`Page 7
`
`

`

`Patent Application Publication Jan. 1,2004 Sheet 7 of 19 US 2004/0001480 Al
`
`FIG7
`
`631
`
`_«T «S /632
`,
`provision name- Taro > /
`<user account= “ dient B ”/ 333
`password= “ z9y8x7 ’ L 534
`addr= 0 ***,***.*** ” />/ c
`<imsrv addr= “ ***.***.*** ” /> /
`<voipsrv addr= “ ***.***.**** ” />
`</provision>
`
`PROFILE DATA 686
`
`Epic Games Ex. 1008
`Page 8
`
`

`

`Patent Application Publication
`
`Jan. 1,2004 Sheet 8 of 19 US 2004/0001480 Al
`
`Epic Games Ex. 1008
`Page 9
`
`

`

`Patent Application Publication Jan. 1,2004 Sheet 9 of 19 US 2004/0001480 Al
`
`Epic Games Ex. 1008
`Page 10
`
`

`

`Patent Application Publication Jan. 1, 2004 Sheet 10 of 19 US 2004/0001480 Al
`
`Epic Games Ex. 1008
`Page 11
`
`

`

`Patent Application Publication Jan. 1,2004 Sheet 11 of 19 US 2004/0001480 Al
`
`Epic Games Ex. 1008
`Page 12
`
`

`

`Patent Application Publication Jan. 1,2004 Sheet 12 of 19 US 2004/0001480 Al
`
`Epic Games Ex. 1008
`Page 13
`
`

`

`Patent Application Publication Jan. 1,2004 Sheet 13 of 19 US 2004/0001480 Al
`
`Epic Games Ex. 1008
`Page 14
`
`

`

`Patent Application Publication Jan. 1,2004 Sheet 14 of 19 US 2004/0001480 Al
`
`Epic Games Ex. 1008
`Page 15
`
`

`

`Patent Application Publication Jan. 1,2004 Sheet 15 of 19 US 2004/0001480 Al
`
`Epic Games Ex. 1008
`Page 16
`
`

`

`Patent Application Publication .Jan. 1, 2004 Sheet 16 of 19 US 2004/0001480 Al
`
`Epic Games Ex. 1008
`Page 17
`
`

`

`Patent Application Publication Jan. 1,2004 Sheet 17 of 19 US 2004/0001480 Al
`
`Epic Games Ex. 1008
`Page 18
`
`

`

`Patent Application Publication Jan. 1,2004 Sheet 18 of 19 US 2004/0001480 Al
`
`Epic Games Ex. 1008
`Page 19
`
`

`

`Patent Application Publication Jan. 1,2004 Sheet 19 of 19 US 2004/0001480 Al
`
`Epic Games Ex. 1008
`Page 20
`
`

`

`US 2004/(XX)1480 Al
`
`Jan. 1.2004
`
`COMMUNICATION SYSTEM AND
`COMM UN ICATION M ETH<)D
`
`BACKGROUND OF THE INVENTION
`[0001] The present invention relates to a communication
`technology such as Instant Messaging (IM).
`[0002] A communication technology called IM is gather­
`ing attentions. Sometimes, a matter to talk with the other
`party may not need a rapid response through a telephone.
`However, if an e-mail containing the matter is sent through
`an E-mail, it is not clear when the other party reads (that is,
`when his/her reply is obtained). IM has a time characteristic
`that is intermediate between the telephone and the e-mail. In
`IM, a current status (which may be called presence infor­
`mation), such as online or offline or the ability to respond,
`of each party is published. Thus, in response to presence
`information of friends using the same IM system over the
`Internet, a tool to be used for communication with friends,
`such as a chat and an e-mail, can be selected.
`[0003] Presently, IM systems are released from many
`venders, and IM clients, each of which is application soft­
`ware to use a respective IM system, arc distributed. How­
`ever, there is no compatibility among IM systems of the
`respective venders. As a result, communication cannot be
`performed between the IM clients of the different venders.
`Thus, the standardization is started from a need for (he
`interconnectivity. In Instant Messaging & Presence Protocol
`(IMPP) working group of Internet Engineering Task Force
`(IETF), tile architecture, the message integration and the
`security (such as authentication and encryption) relating to
`IM are being discussed. A model is defined for Require for
`Comment (RFC) 2778 while a protocol requirements are
`defined for RFC 2779.
`[0004] Conventionally, the multi-party connection (group
`chat) in IM is mainly used just for liin(hobby use), such as
`gossiping. However, recently, study for using the group chat
`for business has been started. Here, it is noted that the
`text-based group chat may be not enough for the business
`uses in many cases. For example, inputting Japanese lan­
`guage takes time, hi addition, in order to express a compli­
`cated matter or a shade of meaning, which is bard to be
`understood from a document, Ibc voice support is needed. In
`the one-to-one (Peer-to-Peer) technology, the expansion into
`communication using multimedia has been started. The first
`reference (Nikkei Communication. Nov. 5,2001, pgs. 106 Io
`113) proposes such usage of multimedia that a meeting is
`held by using a voice chat or by exchanging files.
`SUMMARY OF THE INVENTION
`[0005] When a communication technology is applied to
`the business purpose, it is desirable that not only text but also
`voice can be handled easily in addition to participation in a
`conference without leaving his/her desk and/or participation
`in conference even from the outside of his office.
`[0006] However, the conventional IM systems are not
`ready for a group chat using multimedia. By using the
`system disclosed in the reference 1, switching between text
`and voice can be performed in a one-to-one chat. However,
`no consideration is given to the switching between text and
`voice in a group chat.
`[0007] There is a television conference system as a tech­
`nology for implementing a group cbal other than the IM
`
`system. However, the conventional television conference
`system requires the buddies to gather in a room such as a
`television conference room al a scheduled conference time.
`In some cases, users must call to a predetermined virtual
`conference room. That is, the conventional television con­
`ference system lacks flexibility.
`[0008] The present invention was made in view of these
`circumstances. It is an object of the present invention to
`improve handling (dealing) of a communication system.
`More specifically, it is an object of the present invention to
`achieve group chat using multimedia. For example, the
`switching between a group chat through electronic docu­
`ments and a group chat through voice and/or the switching
`between a one-to-one chat through voice and a group chat
`through voice can be handled flexibly.
`[0009] In order to achieve the objects, the present inven­
`tion implements a chat between/among IM clients by using
`an IM presence management server which manages pres­
`ence information indicating a stale of each of the IM clients;
`a Voll’ communication connection management server
`which manages a connection for voice chatting using VoIP
`(Voice over Internet Protocol); and a media server which
`implements multi-party voice communication by mixing
`voice data.
`[0010] For example, the IM presence management server
`manages, as to each of the IM clients, presence information
`oflhe IM client, information regarding media (including text
`and voice) which can be used for a chat by the IM client, and
`user information oflhe IM client, and in accordance with an
`instruction from an IM client, provides the IM client in
`question with the presence information, usable media infor­
`mation and user information of each of the IM clients set as
`buddies of the IM client in question.
`[0011J In accordance with information of each of IM
`clients (called as text participating clients) participating in a
`text chat, which is notified from the IM client, the IM
`presence management server manages a connection
`between/among the text participating clients and the IM
`presence management server, merges text data sent from
`each of the text participating clients and distributes the result
`to each of the text participating clients.
`[0012] The VoIP communication connection management
`server manages a connection between/among each of IM
`clients (called as voice participating clients) and the media
`server in accordance with information of the voice partici­
`pating clients participating in a voice chat, which is notified
`from the IM client or the IM presence management server.
`[0013] Hie media server performs processing for mixing
`voice data sent from each of the voice participating clients
`except for one of the voice participating clients set as a
`focused client and for distributing the result data thus mixed
`to the focused client. These processing is performed as
`focused clients on all of the voice participating clients.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`[0014] FIG. 1 is a schematic diagram of an IM-VolP
`(Voice over Internet Protocol) interconnecting system,
`which is a communication system to which one embodiment
`of the present invention is applied;
`[0015] FIG. 2 is a schematic configuration diagram of an
`IM server 4 shown in FIG. 1;
`
`Epic Games Ex. 1008
`Page 21
`
`

`

`US 2004/0001480 Al
`
`Jan. 1, 2004
`
`2
`
`[0016] FIG. 3 is a diagram showing an example of a
`presence information management table 488 used by a
`connection managing portion 487 shown in FIG, 2 for
`managing presence information;
`[0017] FIG. 4 is a schematic configuration diagram of an
`AP server 5 shown in FIG. 1;
`[0018] FIG. 5 is a diagram showing an example of a VoIP
`connection management table 586 used by a connection
`managing portion 585 shown in FIG. 4 for managing voice
`chat;
`[0019] FIG. 6 is a schematic configuration diagram of an
`IP terminal 7 shown in FIG. 1;
`[0020] FIG. 7 is a diagram showing an example of profile
`data 686 required when an IM client participates in an IM;
`[0021] FIG. 8 is a schematic configuration diagram of an
`MD server 6 shown in FIG. 1;
`[0022] FIG. 9 is a schematic configuration diagram of a
`VR server 10 shown in FIG. 1;
`[0023] FIG. 10 is a diagram for explaining an operation
`for switching from a text chat to a voice chat, which is a first
`operational example of an IM-VoIP interconnecting system
`according to one embodiment of the present invention:
`[0024] FIG. 11 is a diagram for explaining an operation
`for switching from a text chat to a voice chat, which is a first
`operational example of an IM-VoIP interconnecting system
`according to one embodiment of the present invention;
`[0025] FIG. 12 is a diagram showing an example of a user
`interface of an IM client, which is displayed in an IP
`terminal 7;
`[0026] FIG. 13 is a diagram for explaining a variation
`example of the first operational example shown in FIG. 11;
`[0027] FIG. 14 is a diagram for explaining a variation
`example of the first operational example shown in FIG. 11;
`[0028] FIG. 15 is a diagram for explaining an operation
`for using a text chat during a voice chat, which is a second
`operational example of an IM-VoIP interconnecting system
`according to one embodiment of the present invention;
`[0029] FIG. 16 is a diagram for explaining an operation
`for using a text chat during a voice chat, which is a second
`operational example of an IM-VoIP interconnecting system
`according to one embodiment of the present invention.
`[0030] FIG. 17 is a diagram for explaining an operation
`for switching from a one-to-one voice chat to a multi-party
`voice chat, which is a third operational example of an
`IM-VoIP interconnecting system according to one embodi­
`ment of the present invention;
`[0031] FIG. 18 is a diagram for explaining an operation
`for switching from a one-to-one voice chat to a multi-party
`voice chat, which is a third operational example of an
`IM-VoIP interconnecting system according to one embodi­
`ment of the present invention; and
`[0032] FIG. 19 is a diagram for explaining an operation
`for closing a conference room, which is a fourth operational
`example of an IM-VoIP interconnecting system according to
`one embodiment of the present invention.
`
`DESCRIPTION OF THE PREFERRED
`EMBODIMENTS
`[0033] Embodiments of the present invention will be
`described below.
`[0034] FIG. 1 is a schematic diagram of an IM-Voice over
`Internet Protocol (VoIP) interconnecting system, which is a
`communication system to which one embodiment of the
`present invention is applied.
`[0035] As shown in FIG. 1, an IP network 1 is connected
`to an IM presence managing Server (called as IM server
`hereinafter) 4 which manages presence information of an IM
`client, a VoIP communication connection managing server
`(called as AP server hereinafter) 5 which manages connec­
`tion for a voice chat using VoIP, a media server (called as
`MD server hereinafter) 6 which implements multi-party
`voice speech by mixing voice data, a plurality of IP termi­
`nals 7-1 to 7-3 in each of which an IM client is installed, and
`a voice relay server (called as VR server hereinafter) 10
`which performs voice relay among a radio communication
`network 2, a publicly switched telephone network (PSTN) 3
`and the IP network 1. Here, a radio terminal 9, such as a
`mobile telephone, in which an IM client is installed, is
`connected to the radio communication network 2. A fixed
`telephone 11 in which an IM client is installed is connected
`to the publicly switched telephone network 3.
`[0036] In the above-described configuration, the IP termi­
`nals 7-1 to 7-3 can perform a multi-media chat using the IM
`server 4, the AP server 5 and the MD server 6 by indepen­
`dently or combination with the VoIP telephone 8, the radio
`terminal 9 or the fixed telephone 11. For example, in a group
`or in one-to-one, a chat through electronic documents such
`as text (called as text chat hereinafter) and a chat through
`voice (called as voice chat hereinafter) can be performed.
`[0037] Next, each of devices included in the IM-VoIP
`interconnecting system will be described.
`[0038] First of all, the IM server 4 will be described.
`[0039] As described above, the IM server 4 manages
`presence information of the IM clients. A schematic con­
`figuration of the IM server 4 is shown in FIG. 2.
`[0040] As shown in FIG. 2, the IM server 4 is achieved by
`configuring functional blocks 481 to 487 shown within a
`balloon 48 in FIG. 2 when a CPU 41 executes a predeter­
`mined program loaded onto a memory 42 in a general-
`purpose computer system including the CPU 41, the
`memory 42, an external storage device 42, such as an HDD,
`an input device 44, such as a keyboard, a mouse and a pen,
`an output device 45, such as a speaker and a display, an IP
`network interface (IF) 46 which performs communication
`over the IP network 1, and a bus 47 for connecting these
`devices 41 to 46. The program for configuring the functional
`blocks 481 to 487 may be directly loaded from a storage
`medium (not shown) such as a CD-ROM, through a reading
`device (not shown), or from the IP network 1 to the memory
`42 through the IP network IF 46. Alternatively, the program
`may be stored in the external memory device 43 and then
`may be loaded to the memory 42.
`[0041] A packet deassembling portion 481 receives an IP
`packet addressed to itself from the IP network 1. Then, the
`packet deassembling portion 481 processes the IP header
`and the TCP/UDP header of the packet and extracts a
`payload.
`
`Epic Games Ex. 1008
`Page 22
`
`

`

`US 2(X)4/(XX)1480 Al
`
`Jan. 1, 2004
`
`3
`
`[0042] An analyzing and data,command distributing por­
`tion 482 analyzes the content of the payload processed by
`the packet deassembling portion 481 and distributes the
`content to a data processing portion 483 as data or a
`command processing portion 484 as a command.
`
`[18)43] Ihe data processing portion 483 performs process­
`ing required for IM on data distributed by the analyzing and
`data command distributing portion 482. For example, for
`each conference, a predetermined amount of latest text data
`items received from each of IM clients participating in the
`conference is held. Then, the held text data items arc
`merged, and then data for text chat is created therefrom.
`[0044] fhe command processing portion 484 performs
`processing required for IM in accordance with the content of
`the command distributed by the analyzing and data/com-
`mand distributed portion 482. For example, in accordance
`with a conference setting request command from an IM
`client, the command processing portion 484 performs pro­
`cessing required for setting a new conference. Further in
`accordance with a conference participating request com­
`mand from an IM client, the command processing portion
`484 performs processing required for calling the other IM
`clients to participate in the conference.
`[0045] An output data creating portion 485 creates output
`data in accordance with results of the processing by the data
`processing portion 483 and or the command processing
`portion 484.
`
`[0046] A packet assembling portion 486 assembles an IP
`packet by adding a TCP/UDP header and an IP header to the
`output data created bv the output data creating portion 485.
`The packet assembling portion 486 sends the IP packet to the
`IP network 1.
`[0047] Aconnection managing portion 487 manages pres­
`ence information of IM clients. In accordance with the
`presence information, the connection managing portion 487
`controls each of the functional blocks 481 Io 486 and relays
`between connections with the IM clients, rhus, the chat
`between the IM clients can be achieved.
`[0048] Incidentally, the packet dcasscmbling portion 481
`and the packet assembling portion 406 may be configured
`for example, io hardware with a use of such as integrated
`logic IC within the IP network 11 46.
`[0049] FIG. 3 shows an example of a presence informa­
`tion management table 488. which is used by the connection
`managing portion 487 for managing presence information.
`The presence information managing table 488 is stored in
`the external storage device 43. for example.
`[0050] As shown in FIG. 3. one record 440 includes a field
`431 for registering an account name of a user of an IM client,
`a field 432 for registering an address of the IM client, a field
`433 for registering a nickname of the user of the IM client,
`a field 434 for registering an authentication key for using the
`IM-VolP interconnecting system, a field 435 for registering
`presence information of the IM client, a field 436 for
`registering a medium (text chat and voice chat), which can
`be used by IM client lor a chat, a field 437 for registering an
`address (IP address or DNS and a port number) of a
`conference room in which the IM client participates, a field
`438 for registering a nickname of the conference room, and
`
`a field 439 for registering account names of the other IM
`clients with whom the IM client can chat.
`[0051] Here, when a terminal in which an IM client is
`installed is an IP terminal, an IP address or a DNS of the
`terminal is registered, as an address of the IM client, in the
`field 432. When the terminal in which the IM client is
`installed is a mobile telephone and/or a fixed telephone, a
`telephone number and an IP address or a DNS of a VR server
`10 which voice relays between the mobile telephone and/or
`the fixed telephone and the IP network 1 are registered
`therein.
`[0052] Any one of information (OFF) indicating that the
`IM client is offline, information (idle) indicating that the IM
`client is online but idle, information (text) indicating that the
`IM client is text-chatting, information (voice) indicating that
`the IM client is voice-chatting, and information (text/voicc)
`indicating that the IM client is both text chatting and voice
`chatting is registered in the field 435 as presence informa­
`tion.
`[0053] An IP address assigned to the conference room is
`registered in the field 437 as an address of the conference
`room.
`[0054] When a user uses a plurality of IM clients, for
`example, when the client terminal 7-1 is used for text­
`chatting and the VoIP telephone 8 is used for voice-chatting,
`a record 440 is registered for each of them.
`[0055] The AP server 5 will be described next.
`[0056] As described above, the AP server 5 manages a
`connection for voice chatting using VoIP. A schematic
`configuration of the AP server 5 is shown in FIG. 4.
`[0057] As shown in FIG. 4. the AP server 5 is achieved by
`configuring functional blocks 581 to 585 shown within a
`balloon 58 in FIG. 4, when, a CPU 41 executes a predeter­
`mined program loaded onto a memory 42 tn a computer
`system having the same hardware configuration as that of
`the IM server 4. The program for configuring (he functional
`blocks 581 to 585 may be directly loaded from a storage
`medium (not shown) such as a CD-ROM, through a reading
`device (not shown), or from the IP network 1 Io the memory
`42 through the IP network IF 46 Alternatively, the program
`may be stored in an external memory’ device 43 and then
`may be loaded to the memory 42.
`[0(158] A packet deassembling portion 581 receives an IP
`packet addressed to itself from the IP network 1 Then, the
`packet dcasscmbling portion 581 processes an IP header and
`a TCP/UDP header of the packet and extracts a payload.
`[0059] An analyzing and processing portion 582 analyzes
`the content of the payload processed by the packet deas­
`sembling portion 581 and performs processing required for
`a voice chat in accordance with the content of the command
`stored therein. For example, tn accordance with a voice chat
`requesting command from an IM client, processing required
`for calling the other parties of the conference in which the
`IM client participates to participate in a voice-chat is per­
`formed.
`[0060] An output data creating portion 583 creates output
`data in accordance with a result of the processing by the
`analyzing and processing portion 582.
`
`Epic Games Ex. 1008
`Page 23
`
`

`

`US2(X)4/(XX)1480 Al
`
`4
`
`Jan. 1,2004
`
`[0061 ] A packet assembling portion 5X4 assembles an IP
`packet by adding aTCP/UDP header and an IP header to the
`output data created by the output data creating portion 583.
`The packet creating portion 584 sends the created IP packet
`to the IP network 1.
`[0062] A connection managing portion 585 performs con­
`nection management of a voice chat. In other words, the
`connection managing portion 585 controls each of lhe
`functional blocks 581 to 584 Io manage a connection
`between an IM client and the Ml) server 6. Thus, voice
`chatting among IM clients through the Ml) server 6 can be
`achieved.
`[0063] Incidentally, the packet deassembling portion 581
`and the packet assembling portion 586 may be configured
`for example, in hardware with a use of such as integrated
`logic 1C within the IP network IF 46
`[0064] FIG. 5 shows an example of a VoIP connection
`management table 586, which is used by the connection
`managing portion 585 for managing a voice chat. The VoIP
`connection managing table 586 is stored in the external
`storage device 43, for example.
`[0065] As shown in FIG. 5, one record 540 includes a field
`531 for registering an account name of a user of an IM client,
`a field 532 for registering an address of the IM client, a field
`533 for registering a nickname of the user of the IM client,
`a field 534 for registering information regarding a method of
`encoding voice signals carried in a VoIP packet, a field 535
`for registering an address (IP address or DNS and a port
`number called as “address for voice chat" hereinafter) of the
`Ml) server 6, which is used for a voice chat in which the IM
`client participate, and a field 536 for registering a nickname
`of the conference room.
`[0066] Next, the IP terminal 7 will be described.
`[0067] As described above, an IM client to be used for
`chatting is installed in the IM terminal 7. A schematic
`configuration of the IP terminal 7 is shown in FIG. 6.
`[0068] As shown in FIG. 6. the IP terminal 7 is achieved
`by configuring functional blocks 781 to 794 shown within a
`balloon 78 in FIG. 6, when, a CPU 41 executes a predeter­
`mined program (IM client program) loaded onto a memory
`42 in a computer system having the same hardware con­
`figuration as that of the IM server 4. The program for
`configuring the functional blocks 781 to 794 may be directly
`loaded from a storage medium (not shown) such as a
`CD-ROM. through a reading device (not shown), or from
`the IP network 1 Io lhe memory 42 through the IP network
`IF 46. Alternatively, the program may be stored in the
`external memory device 43 and then may be loaded to the
`memory 42.
`[0069] A packet deassembling portion 781 receives an IP
`packel addressed to itself from the IP network 1. Then, lhe
`packet deassembling portion 781 processes a IP header and
`a TCP/UDP header of the packet and extracts a payload.
`[0070] An analyzing and voice packel,data/command dis­
`tributing portion 782 analyzes the content of the payload
`processed by the packet deassembling portion 781 and
`distributes the content to a voice packet deassembling por­
`tion 785 as a voice packel (VoIP packet), a data processing
`portion 783 as data, or a command processing portion 784
`as a command.
`
`[0071] The data processing portion 783 analyzes a type of
`the data distributed by the analyzing and voice packet/data
`command distributing portion 782 and performs processing
`in accordance with the type. Then, the data processing
`portion 783 notifies the result to an output data creating
`portion 786.
`[0072] The command processing portion 784 analyzes the
`content of the command distributed by the analyzing and
`voice packet/data command distributing portion 782 and
`performs processing in accordance with lhe content. Then,
`lhe command processing portion 784 notifies the result to
`the output data creating portion 786.
`10073] The output data creating portion 485 creates output
`data in accordance with results of the processing by the data
`processing portion 483 and the command processing portion
`484. The created data is output from a display and/or speaker
`of the output device 45.
`[0074] A voice packet deassembling portion 785 performs
`processing of the header of lhe voice packet distributed by
`the analyzing and voice packel/dala/command distributing
`portion 782, thereby to extract a payload.
`[0075] A decoder 787 decodes encoded voice data stored
`in the payload extracted by the voice packet deassembling
`portion 785.
`[0076] A D/A converting portion 788 converts digital
`voice data decoded by lhe decoder 787 into analog voice
`signals. The analog voice signals are output from lhe speaker
`of the output device 45.
`[0077] An AD converting portion 790 converts analog
`voice signals sent from a microphone of the input device 44
`to digital voice data.
`[0078] An encoder 791 encodes digital voice data output
`from the AD converting portion 790
`[0079] A voice packet assembling portion 793 assembles
`voice packets (VolP-packets) of encoded voice data, which
`is output from the encoder 791.
`[0080] An event analyzing portion 789 analyzes an
`instruction (event) input by a user through the input device
`44 and sends the analysis result to a data/command creating
`portion 792.
`[0081] A data/command creating portion 792 creates data
`or a command in accordance with the analysis result from
`the event analyzing portion 789. For example, when an
`event analyzed is an instruction regarding a request for
`starting a voice chat, a voice chat request command is
`created. When an event analyzed is a receipt of a message of
`a text ebnt, data representing lhe message is created.
`[0082] A packet assembling portion 794 assembles IP
`packets of voice packets assembled by lhe voice packel
`assembling portion 793, IP packets of data created by lhe
`data/command creating portion 792. and IP packets of
`commands created by the data/command creating portion
`792. And the packet assembling portion 794 sends the IP
`packets to lhe IP network 1.
`[0083] Incidentally, the packet deassembling portion 781
`and the packet assembling portion 794 may be configured
`for example, in hardware with a use of such as integrated
`logic 1C within the IP network If 46. lhe D/A converting
`
`Epic Games Ex. 1008
`Page 24
`
`

`

`US2004/(XM)1480A1
`
`5
`
`Jan. I. 2(M)4
`
`portion 780 and the A/D converting portion 790 are config­
`ured in hardware within the output device 45 and the input
`device 44 (or interfaces with these devices), respectively.
`[0084] FIG, 7 shows an example of a profile data 686
`required for an IM client to participate in IM. The profile
`data 686 may be stored in the external storage device 43, for
`example.
`[0085] Asshown in FIG. 7, it nickname of a user of the IM
`client is registered in a lag 631. An account name, an
`authentication key and an address of the user of the IM client
`arc registered in a lag 632. An address of the IM server 4 is
`registered in a lag 633. An address of the AP server 5 is
`registered in a tag 634. In this example, the profile data is in
`XML (extensible Markup Language) form. However, pro­
`file data may he in a table form or a text file form.
`[0086] Basically, existing devices may be used as the VoIP
`telephone 8, the radio terminal 9 and the fixed telephone 11.
`These devices hold profile data 686 shown in FIG. 7 and use
`the profile data 686 to participate in IM.
`[0087] When a text chat is only performed in the IP
`terminal 7 and a voice chat is performed by using the VoIP
`telephone 8, the radio terminal 9 and/or the fixed telephone
`11. the IP terminal 7 does not need functional blocks
`(functional blocks 785. 787. 788, 790, 791 and 793) for the
`voice chat within a balloon 78.
`[0088] Next, the MD server 6 will be described.
`[0089] As described above, the MD server 6 mixes voice
`data for multi-party voice speech (communication). FIG. 8
`shows a schematic configuration of the MD server 6.
`[0090] As shown in FIG. 8, the MD server 6 is achieved
`by configuring functional blocks 881 to 888 shown within a
`balloon 88 in FIG. 8, when, a CPU 41 executes a predeter­
`mined program loaded onto a memory 42 in a computer
`system having the same [text missing or illegible when
`tiled] as that of the IM server 4. Like the case of the IM
`server 4. the program for configuring the functional blocks
`001 Io 888 may be directly loaded from a storage medium
`(not shown) such as a CD-ROM, through a reading device
`(not shown), or from the IP network 1 to the memory 42
`through the IP network IF 46. Alternatively, the program
`may be stored in an external memory device 43 and then
`may be loaded to the memory 42.
`[0091] A packet dcassembling portion 881 receives an IP
`packet addressed to itself from the IP network 1 Then, the
`packet dcassembling portion 881 processes an IP header and
`a TCP/UDP header of the packet and extracts a payload.
`[0092] An analyzing and voice packet/'command distrib­
`uting portion 882 utalyzcs the content of the payload
`processed by the packet deassembling portion 881 and
`distributes the content to a voice packet deassembling por­
`tion 885 as a voice packet (VoIP packet) or a command
`processing portion 883 as a command.
`[0093] A command processing portion 883 analyzes the
`content of the command distributed by the analyzing and
`voice packel/command distributing portion 882 an