us) United States
`a2) Patent Application Publication (10) Pub. No.: US 2003/0216178 Al
`(43) Pub. Date: Nov.20, 2003
`
`Danieli et al.
`
`US 20030216178A1
`
`(54) USE OF MULTIPLE PLAYER REAL-TIME
`VOICE COMMUNICATIONS ON A GAMING
`DEVICE
`
`(76)
`
`Inventors: Damon V. Danieli, Clyde Hill, WA
`(US); Roxana Gabriela Arama,
`Seattle, WA (US)
`
`Correspondence Address:
`LAW OFFICES OF RONALD M. ANDERSON
`Suite 507
`600 - 108th Avenue N.E.
`Bellevue, WA 98004 (US)
`
`(21) Appl. No.:
`
`10/147,578
`
`(22)
`
`Filed:
`
`May 16, 2002
`
`Publication Classification
`
`1S) Wat Oc ccscicsesataciniacninnanmin ROSLST
`
`108
`
`(SOT POSES cccccs cnsusnstrarteaareavcvestiansucinatse GROSS
`
`(57)
`
`ABSTRACT
`
`Agame console capable of communicating with other game
`consoles over a link or network is provided with a head-
`phone and microphone for cach player who will engage in
`voice communication, Verbal communications directed to
`one or more other players are converted ta pulse code
`modulated (PCM) digital data and are encoded and com-
`pressed in real-time, producing dala packets that are trans-
`mitted Lo another game console. The compressed dala pack-
`ets are decompressed and decoded, producing PCM data that
`are converted lo an analog signal thal drives a headphone of
`the intended recipient. Players can selectively mute voice
`communications to and from a specific other player. The
`PCM data can be encoded in a round-robin fashion that
`reduces the number of encoders required. A predefinedlevel
`of computing resources is used for voice communication to
`
`Epic Games Ex. 1027
`Epic GamesEx. 1027
`Page 1
`Page 1
`
`avoid aversely affecting the quality of game play.
`
`

`

`Patent Application Publication Noy. 20,2003 Sheet 1 of 13
`
`US 2003/0216178 Al
`
`FIG, 2
`
`Epic Games Ex. 1027
`Epic GamesEx. 1027
`Page 2
`Page 2
`
`

`

`Patent Application Publication Novy. 20,2003 Sheet 2 of 13
`
`US 2003/0216178 Al
`
`‘VOL
`
`QSSsaYdNoo
`BOL €Old
`
`NOILOATSSNOILOATAS
`
`ANISNAANISNA
`
`|SIOSNOD
`
`OFLvel
`
`VLVdWOd
`
`ViVdWOd
`
`ANOHdOVAH
`
`SANOHdOXYDIN
`
`dassayddWwoo
`
`
`
`O91WVAYLSSIONIS
`
`YSaQOONA
`
`VLVG
`
`MYOMLAN
`
`NANANO
`
`MYOMLAN
`MYOMLAN
`
`éANANO
`
`L3NSNO
`
`Epic Games Ex. 1027
`Epic GamesEx. 1027
`Page 3
`Page 3
`
`
`
`
`
`

`

`Patent Application Publication Noy, 20,2003 Sheet 3 of 13
`
`US 2003/0216178 Al
`
`144
`
`146
`
`CONSOLE1
`
`MICROPHONE
`
`HEADPHONE
`
`102
`
`PCM DATA
`
`PCM DATA
`
`COMPRESSED DATA CONSOLE2
`
`160
`
`NETWORK
`
`of
`
`COMPRESSED DATA
`
`COMPRESSED DATA
`
`FIG. 4
`
`Epic Games Ex. 1027
`Epic GamesEx. 1027
`Page 4
`Page 4
`
`

`

`US 2003/0216178 Al
`
`pz0Z
`
`YSGOONS
`
`Patent Application Publication Noy. 20,2003 Sheet 4 of 13
`
`GSLOANNOOSIYAAVIdHOV]ooa00gqooz
`
`
`YSTIOULNOOarIaaSVS.VIASIQENODOLyaAWwid||vaAwid||ysaAWId||YSAWIdPY2002
`
`|FIOSNOOSSAV1MNOMLANeal
`
`
`
`ZorYYOMLANYYOMLANYYOMLSN
`
`v0e SIOSNOD€FIOSNOODZJ1IOSNOO
`
`
`YWYOMLSNOLGSLOSNNOOMYOMLAN
`
`GL‘SINOLSINab“OINVbOIW
`
`14OdLSNYSHISAg902
`LANIINdNlLONILNdNi
`
`
`
`WOusWOudswoudWOud
`
`YAAVId|]YSAWId|]MSAWId|]YSAWIdYaAVIdYAWId|_YSAVIdJegiz
`
`
`9Z0ZqZ0z202POG!YaLNOYLNdLNO
`
`
`Oob|H3GOONS|YyOdanand||YyOsS3NaNoD|}YOsaNsaNo
`
`
`YSLNOULNdNIeal"waq00ad
`
`pozz20ZZq0zzB0cé
`
`vlZ“4JIOSNODcb@"“4370SNO9JIOSNOD
`ANALNOJ}LNALAOJ]LAdLNOF]LAdLNO
`
`06)gO6hOGL
`
`
`
`
`SIFIOSNODHOVa
`
`OLL
`
`QLZqgLe
`
`gol
`
`§DId
`
`Epic Games Ex. 1027
`Epic GamesEx. 1027
`Page 5
`Page 5
`
`
`
`
`

`

`Patent Application Publication Noy. 20,2003 Sheet 5 of 13
`
`
`
`NOILVALLOVNOLLVAILLOVNOILVAILOVNOLLVAILOV
`
`
`
`
`
`
`
`
`
`PELZSDIOAADIONADIOAASIOA
`
`
`
`VLVdWOdSELCGElZRELZ
`
`
`
`
`
`?ANOHdOUDIN€JNOHdONDINZANOHdOHNDINLtSNOHdOXDIN
`
`
`
`PLLZOLLZaLLZELLZ
`
`
`
`
`
`
`
`NOILOSL3q0NOILOSLSq0NOILOSL30NOILOSLSqG
`
`US 2003/0216178 Al
`
`GLZ
`
`WHLINYODTVWHLINOOTVWHLIYODTVWHLINOS1V
`
`WHLIYODIViNOILVZILINOId
`
`
`
`(0)ALINOIMd
`
`
`
`
`
`dassaydwoo‘FOIOAONIAVHLONSVMYVW
`
`,09L
`
`
`
`SYSGOONSTSaTIVeVdOML
`
`(¢)ALINOId(Z)ALIMOIYd
`
`
`
`
`(L)ALIMOId902
`
`ii
`
`
`
`PAbeeveQLLZsane
`
`
`
`
`
`WLWaALIMOIYdHDIHSAVHTIMASHL
`
`YYOMLANOL9DIA
`
`
`
`SWVYsLXSNNIDSNIGOONSYOS
`
`Epic Games Ex. 1027
`Epic GamesEx. 1027
`Page 6
`Page 6
`
`
`
`

`

` Fe
`
`9c2
`
`‘dals
`
`
`
`YO'13yDVdGSGOONS‘tb
`
`
`
`
`
`YO/ONY“SWISHL
`
`
`
`
`
`
`
`‘O13'LaAWOWdJONASSGIAOYd
`
`Patent Application Publication
`
`Nov. 20, 2003
`
`Sheet 6 of 13
`
`US 2003/0216178 Al
`
`YAAVIdHOWSHOSSWVWSYLSSLVOIONVS40LSTWOusG3aLVNIWNS
`
`
`
`AGGS.LHOSSuvYSAV1dHOWSYOsSWVSYLSSDIOAONINIVINSS@“AIOSNODNO
`
`
`
`TIALIWOIdLSSHOIHSAWHAVIVWWSYLSSDIOAANOL%
`
`
`
`TIIMSWS‘CSYS0NSe3dOLCALYVLSWVSYLS¥4122%
`
`
`NIVLYS9ALNWOLASOOHDAVWAIOSNOSNOHSA1dHOWS|b
`
`SuvSINVSYLSCGSLNIN‘SIANNVHOONINSLSITNO31dO3d
`
`
`
`S3ILINOlddGSNIS30Y3SN8XSVINSNILAWAldd¥
`
`
`
`
`‘SSIUYOlddCANISS0-essn
`GaZINOYHONAS
`Tez-LAGLNOSAGIAONdANANDHOWS
`
`
`
`
`
`
`ONVWASYMANANDAPGAAdAOYdLAWoWdSNOIASHYdalbe
`
`
`
`
`J3ax9VdLNG“LaNOVdSAWHLNSSOC1!LVHLLNAMNDSSISILON‘2
`
`
`
`
`‘'S1ayNoVdSNOIASYdYSHLOWONSNOILVANALLY
`
`
`LXANNISCIAGUdONYAdO9D“13NOVdGAGOONA
`
`
`
`SDIOAOF'OOLONISSIN.SVMLANSVdSNOIASHdJI2°
`
`LNdLNoS32N3SNd
`
`‘OSdl“SNANONINSSSAVHOTNOHS
`
`
`
`dOSINNIOASHLASLLVWON'HSINISOLLI¥OsLIVIA
`
`i
`
`TivSWYWSYLSLSS0NO1YWSHOLASOOHDAVAAWS£2
`
`
`
`
`
`
`
`YO/ONV‘HSLW1GALYWLSLVHLSWVSYLSHSHLO
`
`
`
`
`INS1SALWNIAITS“SWNI10AASSLAXDVdONILINSAYY30NO
`
`SLEWOVd
`
`
`
`YO/ONY‘SAILSHL
`
`
`
`SANnaNdAgGACIAOYdS13x0vdJOLaSansV39TIMLINs3aY
`
`
`
`
`
`
`
`
`
`“JINV9“TANNVHOHOWSNOG3NIS3QS3ILINOlddYSHLO#2
`
`
`
`FDIOAANVANIVLNOSLASEnsNISLaNoWVdTV
`
`
`
`
`
`SHSVIN
`
`ALVOOTIV ZAdAL
`YOCSHOVadSISLA4OVdGAd003dAOYSgINNN“XVAHLTLNN“Lb
`
`
`YaAWTdLNSYHNDYOsSLANDVddOLSI]GSYSCHOLISIALt
`
`
`
`Q30003040YaLNNODLNSWAYONchkbga00930
`
`YSL4V‘CUNSYYNDSSIWOO3)YSAW1dLXSNOLSAOWZL
`
`
`
`YOsSLEWDVdJONASJSN'NIVAAYSLOISONIGOD3dA!‘Zz
`
`
`
`
`
`
`‘3YOIS8N3SOH9DLONLSITJOGVSHalbbb
`
`39OL1S1740GWAHSSOOHOLh’'t
`COHL3SWdSdNOXIW8300940-SHa00930
`“GAHOVAYSISLSI1SLEMOVdSLVGIGNVOONSAHL
`
`
`
`“SSMIdSNdGIOAVOLHAG0930TaTIVeVd
`SNIDNAO€¢
`TENNVHOAldd¥
`ALVOOTIV ce
`
`QOHLSWYSAV1dHOWEHOd3000308XIN-SHADO9IRd
`
`
`
`
`Gaxsa0uO401S!I1NISLAMOVdNLSHisLOATSS"YSAVIdHOWSHOS
`
`
`
`
`YAAWIdHOVEHOSMSVWTENNVH9ONAdd401SSAVHTIMHALLVHODHOVA'SINVAYLS3SIOAJOSLESENS?NISLINSSY
`
`
`
`
`IN31SSSN'SLNAW313NNVHLSS37SNIVLNOOLSI141“SLaxoVd
`
`
`O1S1SNoVdLNSLSIXANISSOHLSOGVSLSNIWLWGasSaydWOd
`AINOOLNSLSITOLSSOOHSAVWFIOSNODVNOHAAW1dHOWS
`
`
`
`“MSVTANNWHO¥SAVHGINOHSONYSTANNWHOJWOS
`
`
`
`(SWVAULSLNSYH344IG)SL3NOVdSLYGIGNVWO
`
`\NIgodad
`
`
`
`
`
`“O13‘GS1V13ay310"
`
`
`
`
`
`‘SIWHLIMOOTYSHL
`
`S.LaxoVd
`
`
`
`{$1140OW3SHSLVNIWINA2b
`
`"NIVSV0YSAVId‘€HSAVId
`
`“ALAINLONoi
`
`PNIGOIsS0
`
`
`
`LddALANIONS
`
`£Ol
`
`‘SAMIdSNddGIOAV
`
`Epic Games Ex. 1027
`Epic GamesEx. 1027
`Page 7
`Page 7
`
`
`
`
`
`‘S|WHLINOOT¥SHL
`
`
`
`

`

`Patent Application Publication Noy, 20,2003 Sheet 7 of 13
`
`US 2003/0216178 Al
`
`240
`244
`242
`
`
`
`
`
`
`ENCODED STREAM 1}
`|ENCODED STREAM 2] @ @ @ |ENCODED STREAM N
`
`
`COMPRESSED
`COMPRESSED
`257
`DATA
`DATA
`
`
` CHOOSE 2 ENCODED STREAMS FOR EACH HEADSET
`
`
`
`
`ENCODED
`ENCODED
`
`
`STREAM 4.1
`STREAM 4.2
` ENCODED
`
`ENCODED
`
`STREAM 1.1
`
`DATA
`
`2 STREAMS
`MIXING DECODER
`
`
`
`
`PCM DATA
`
`248
`
`
`
`
`DATA
`
`168
`
`
`
`2 STREAMS
`MIXING DECODER
`
`
`PCM DATA
`
`
`
`HEADSET4
`
`250
`
`
`
`FIG. 8
`
`Epic Games Ex. 1027
`Epic GamesEx. 1027
`Page 8
`Page 8
`
`

`

`Patent Application Publication Noy. 20,2003 Sheet 8 of 13
`
`US 2003/0216178 Al
`
`242
`
`244
`
`
`ENCODED STREAM 1]
`|ENCODED STREAM 2] @ @ @ |ENCODED STREAMN
`COMPRESSED
`ago
`COMPRESSED
`DATA
`x DATA
`
`
`
`CHOOSE MAXIMUM 4 ENCODED STREAMS, MINIMUM 1 FOR EACH
`LISTENER (IF VOICE ADDRESSED TO LISTENER)
`
`COMPRESSED
`DATA
`
`
`
`262
`
`COMPRESSED
`DATA
`
`FOUR PARALLEL DECODERS
`
`PCM DATA
`
`ea. [fete
`
`
`
`266
`
`PCM DATA
`
`248
`
`PCM DATA
`
`HEADSET1
`
`@#ees
`
`FIG. 9A
`
`Epic Games Ex. 1027
`Epic GamesEx. 1027
`Page 9
`Page 9
`
`

`

`Patent Application Publication Noy. 20,2003 Sheet 9 of 13
`
`US 2003/0216178 Al
`
`2-STREAM
`MIXER FOR|
`USER1
`
`2-STREAM
`|MIXER FOR|
`USER 2
`
`2-STREAM
`|MIXERFOR|
`USER 3
`
`2-STREAM
`|MIXER FOR
`USER 4
`
`4-STREAM PARALLEL DECODER
`
`288
`
`
`
`250
`USER 4
`USER 3
`USER 2
`USER1
`248
`HEADPHONE || HEADPHONE ||HEADPHONE || HEADPHONE
`
`252
`
`953
`
`FIG. 9B
`
`Epic Games Ex. 1027
`Epic GamesEx. 1027
`Page 10
`Page 10
`
`

`

`Patent Application Publication Noy, 20,2003 Sheet 10 0f13
`
`US 2003/0216178 Al
`
`336 PROCESS NEXT
`
`SPEAKER
`
`
`
`
`DON'T PROCESS
`VOICE
`
`GET PCM DATA
`FROM MIC
`
`[338
`
`COMPRESS PCM
`SPEECH INTO
`COMPRESSED
`DATA
`
`APPLY ANY
`SELECTED VOICE
`EFFECTS WHILE
`
`340
`
`349
`
`COMPRESSING
`VOICE PERIPHERAL
`
`SEND COMPRESSED a
`DATA TO INTENDED
`RECIPIENTS HAVING
`
`PROCESS NEXT
`SPEAKER
`
`348
`
`386™
`
`NETWORK
`
`346
`
`358™
`
`350
`
`RECEIVE
`>| COMPRESSED
`DATA
`
`i
`
`QUEUE
`COMPRESSED
`DATA
`
`354™
`
`COMPRESSED
`DATA INTO PCM
`DATA
`
`
`
`ADD ANY SELECTED
`ENVIRONMENTAL
`EFFECTS
`
`MIX VOICE STREAMS
`INTO ONE OUTPUT
`VOICESTREAM
`
`FIG 1 0
`
`360™)
`
`SEND PCM DATA TO
`
`EARPIECE
`
`Epic Games Ex. 1027
`Epic GamesEx. 1027
`Page 11
`Page 11
`
`

`

`Patent Application Publication Noy, 20,2003 Sheet 11 of 13
`
`US 2003/0216178 Al
`
`346
`
`TWORK
`
`VOICE STREAM N
`
`350a
`
`350b
`
`NETWORK CODE RECEIVES
`COMPRESSED DATA SENT
`TO CONSOLE FROM OTHER
`CONSOLES IN GAME
`
`QUEUE COMPRESSED DATA
`TO HANDLE NETWORK
`JITTER (QUEUE 1)
`
`RECEIVE COMPRESSED
`DATA
`NETWORK CODE RECEIVES
`COMPRESSED DATA SENT
`TO CONSOLE FROM OTHER
`CONSOLES IN GAME
`
`
`351b
`
`QUEUE COMPRESSED DATA
`TO HANDLE NETWORK
`
`
`
`
`
`
`
`
`
`
`JITTER (QUEUE N)
`QUEUES FORA VOICE FRAME
`
`364
`
`COMPRESSED PACKETS
`
`SYNCHRONIZE QUEUES. GET ENCODED PACKETS FROM ALL
`
`SELECTION ENGINE|366
`
`COMPRESSED
`PACKETS
`
`COMPRESSED
`PACKETS
`
`DECODING ENGINE
`(DECOMPRESS
`COMPRESSED DATA INTO
`
`368
`
`PCM DATA) PCM DATA
`PERIPHERAL N
`
`VOICE
`PERIPHERAL 1
`
`ate
`
`372
`
`VOICE
`
`Epic Games Ex. 1027
`Epic GamesEx. 1027
`Page 12
`Page 12
`
`

`

`Patent Application Publication Noy, 20,2003 Sheet 12 0f13
`
`US 2003/0216178 Al
`
`
`START
`
`(FOR EACH VOICE FRAME)
`SET VOICEDETECTED
`PROPERTY FOR
`
`
`
`
`ASSEMBLE ARRAY OF 4 PCM
`PCMPACKET[PRIORITIES[0]] &
`
`PCMPACKET[PRIORITIES[1]]
`PACKETS (USE PCM SILENCE
`
`
`PACKET FOR ANY PLAYER W/O
`ON FALSE
`
`
`COMMUNICATOR)
`
`
`
`PARALLEL ENCODE
`
`
`PCMPACKET[PRIORITIES[2]] &
`PCMPACKET/PRIORITIES[3]]
`
`
`
`PRIORITIES(I) = |,
`NO WHERE | =0, 1,2, 3
`
`
`
`
`USE PRIORITIES
`ARRAY GENERATED IN
`
`
`
`PREVIOUS LOOP
`
`
`PCMSTREAMINDEX=
`
`PRIORITIES[PRIORITIESINDEX]
`
`
`
`VOICE ACTIVATION
`DETECTION (MARK PACKETS
`
`
`TO INDICATE WHETHER THEY
`
`
`VOICE
`DETECTED te
`HAVE VOICE)
`
`
`CMPACKET[PCMSTREAM;
`
`INDEX]?
`
`
`
`
`
`
`
`
`MOVE
`
`
`PRIORITIES[PRIORITIESINDEX]
`AT END OF PRIORITIES ARRAY
`
`
`
`AND SHIFT ALL OTHER
`NO
`
`PRIORITIESINDEX < 4 &&
`
`ELEMENTS AFTER IT ONE PLACE
`
`ENCODEOPS< 2?
`FORWARD
`
`PRIORITIESINDEX + 1
`
`
`FIG, 12
`
`PRIORITIESINDEX=
`
`Epic Games Ex. 1027
`Epic GamesEx. 1027
`Page 13
`Page 13
`
`PRIORITIES INDEX =0
`ENCODEOPS= 0
`
`NO
`
`ENCODEOPS=ENCODEOPS + 1
`
`

`

`Patent Application Publication Noy, 20,2003 Sheet 13 0f 13.
`
`US 2003/0216178 Al
`
`
`
`
`PLAYERS
`
`FRIENDS
`
`430
`
`MY STATUS/
`VOICE OPTIONS
`434
`436
`STATUS WRIWWG|128
`
`AVENGER
`BEAR
`ay} O
`oy) O
`OQ
`O
`o|O
`
`IS CURRENTLY ONE OF
`YOUR GAMERS & HAS A
`COMMUNICATOR
`
`OSELECT
`
`OBACK
`
`FIG.13
`
`™
`
`
`GAME.REALM
`
`
`
`LEESLL.
`SEND FEEDBACK
`
`452
`
` 450
`
`
`FIG. 13A
`
`
` GAME.REALM
`SEND FEEDBACK
`LLLESERIESLLL)
` 444
`
`FIG. I3B
`
`Epic Games Ex. 1027
`Epic GamesEx. 1027
`Page 14
`Page 14
`
`

`

`US 2003/0216178 Al
`
`Nav. 20. 2003
`
`USE OF MULTIPLE PLAYER REAL-TIME VOICE
`COMMUNICATIONS ON A GAMING DEVICE
`
`FIELD OF THE INVENTION
`
`‘The present invention generally relates to commu-
`(Q00L]
`nication between players of an electronic game: and more
`specifically, pertains to a multiplayer electronic game sys-
`tem that facilitates voice communication between players
`using one or more multiplayer electronic gaming devices,
`including voice communication over a network that conveys
`data between the multiplayer electronic gaming devices thal
`are coupled together to enable the player to participate in a
`game.
`
`BACKGROUND OF THE INVENTION
`
`[0002] When playing a non-electronic game with one or
`more other people, for example, a card game such as bridge,
`the social interaction arising through verbal communication
`between the players during the game typically adds much to
`ihe enjoyment of the game. Verbal communication is also
`ofien an element of gameplay, since comments made by a
`player to an opponent during a game can have the effect of
`causing the opponent to lose concentration and perform
`poorly, while comments made to team members can provide
`encouragement,
`thereby improving their quality of play.
`Verbal communication between persons playing games is
`thus clearly an important element of the gaming experience.
`
`is so
`[0003] The verbal repartee between players that
`important to game play was initially missing when players
`first began to play electronic games over the Internet and
`other network links, Players at different sites were generally
`not able to communicate with each other, because their
`personal computers (PCs) only communicated data related
`to the play of a game over the network. The loss of the verbal
`communication and related social interaction that is such an
`Important aspect of games played by people at the same
`location thus caused games played over the Internet to be
`less interesting. To address this problem, hardware and
`software solutions were developed that support vaice com-
`munications between PCs over the Internet or other network
`during game play. At about the same time, techniques were
`developed to convey voice over the Internet or other net-
`works (i.ec., voice over
`IP)
`to enable communications
`belween parties connected by the network without incurring
`the cost af conventional telephone long distance calls. This
`workresulted in the creation of various protocols supporting,
`voice over IP communication, including the H.323 specifi-
`cation, Session Initiation Protocol
`(SIP), and Media Gate-
`
`way Conirol Protocol! Txamples 0
`
`systems that provide voice communication between con-
`nected PCs during game play include Microsoft Corpora-
`tion’s SIDE WINDER GAME VOICE™, Mindmaker,
`Inc.'s GAME COMMANDER 27 device and software,
`TEAMSOUND™software, GameSpy Industries’ ROGER
`WILCO™software, and Alienware Technology’s FIRST
`CONTACT™ software, The voice communication provided
`by these products greatly adds to the enjoyment of playing
`games on PCs that are connected over the Internet or other
`networks. Some of these systems operate in peer-to-peer
`
`mode, in which voice data are transferred over the network
`directly between PCs, while others require a voice server
`that receives the voice data from one game player's PCand
`forwards the voice data over the network to one or more
`other PCs that are connected to the network for playing the
`game,
`
`[0004] Since these systems provide communication for
`only one player per PC, each PCproduces its own network
`stream of voice data, and this network stream is directed Lo
`other PC of each other player (or to the voice server, which
`then directs the voice data to the PC of each other player).
`This approach thus produces a substantial network data
`overhead.
`
`(0005) Currently, none of the pnor art systems for PC
`game voice communication enable multiple players per PC
`in a game played over the Internet or other network and
`therefore, the prior art does not support multiplayer-per-PC
`voice communication funetionality. Also,
`if such multi-
`player PC systems were developed using existing voice
`communication protocols for several players on a PC, they
`would likely require an excessive amount of computational
`resources. Allocating the required resources to voice com-
`munication for all players of the game on a PC might well
`have an adverse effect on the quality of game play, unless the
`PC had a very fast processor, lots of memory, and a fast
`video driver card.
`
`In contrast io PCs, dedicaled game consoles ofien
`[0006]
`do not havethe processing power and available memory of
`a powerlul PC, so this problem is of even greater concern in
`developing a scheme to supporl voice communications by
`multiple players on cach game console. li would be desirable
`to allocate a fixed level to the requirements for memory and
`other computer resources needed for voice communication,
`independent of the number of players who are capable of
`voice communication on the game console, as appropriate
`for the game functionality and design, so that the resources
`required for voice processing are not allowed to increase
`beyond a definedlimit as the number of players participating
`in voice communication changes.
`It would also be advan-
`lageous to enable voice communications between multiple
`players for each instance of a game at a site, and to enable
`each player to control with whom the player verbally
`communicates (both speaking and listening), and to combine
`all of the voice data from that instance of the game into a
`single network dala stream in order to reduce the network
`bandwidth required for yoice communications between mul-
`liplayer game consoles. It would further be desirable to share
`certain resources, such as a voice data encoder or decoder,
`between multiple players for a single game instanceat a sile,
`
`it
`[0007] As the quality of game graphics improves,
`becomes more important to maintain other features relating
`to realism. One such fealure ts the ability to provide lip syne
`or other viseme (lip position) information with the voice
`data during game play,
`to enable the lips of a graphic
`character displayed in a game to move in synchronization
`with the words of a player who is represented by and
`controlling the graphic character in the game display. How-
`ever, existing voice communication systems typically do not
`convey dala to enable lip syne, and as a result, the player
`receiving the voice communication will not see the lips of
`the character in the game corresponding to the player who is
`speaking move in syne with the speaker’s words.
`
`Epic Games Ex. 1027
`Epic GamesEx. 1027
`Page 15
`Page 15
`
`

`

`US 2003/0216178 Al
`
`Nav. 20. 2003
`
`bo
`
`[0008] While voice communication is generally a desir-
`able feature, if misused by a specific player, it may become
`annoying and reduce ihe enjoyment of games by other
`players. The priorart also does not enable a player to block
`an annoying player from talking or listening to the player
`during any game in which an annoying player is a partici-
`pant, regardless of the game console used by the annoying
`player. Different people have varying degrees of tolerance
`
`for annoying behavior of others, However,
`
`in games, A player may
`
`« upset because the player feels a specific other player uses
`excessive profanity or sexually explicit language, or uses
`language or makes comments that the player feels to be
`derogatory or socially unacceptable.
`
`
`
`[0009] A parent may also want to block a child from
`parlicipating
`in voice communication during game play
`to
`
`
`
`1s voice communi-
`
`cation should be stored on an online game service so that tt
`remains in effect if the child connects to the online game
`service from a different game console. The prior art game
`voice communication systems do not permit blocking verbal
`communications by a selected player, such as a child,
`participating in games using a multiplayer game console.
`
`Should any player's verbal conduct while playing
`[0010]
`games over the Internet or other network become so egre-
`gious (based upon the number of complaints received from
`other players) as to warrant il, an online game service should
`be able to prevent that player from participating in voice
`communication while playing games through the online
`vame service for a period oftime, and if further justified by
`the continued receipt of complaints about
`that player's
`verbal behavior, to ban the player from using voice com-
`munication permancotly, The current voice communication
`systems do not enable this level of control to be applied at
`each online game service so that a player is banned regard-
`less of an alias used or the garne console through which the
`player participates in game play through the online game
`service.
`
` ifferent architecture, wil
`
`sysiem resources. Most game consoles enable multiple
`players on a single instance of a game, 1e., on the same
`console. When gaming consoles are interconnected over the
`Internet or other network, they should preferably be able to
`provide voice communications for cach of multiple players
`on the game console. Use of conventional techniques that
`have been developedto enable voice communication during,
`game play on PCs will be unacceptable on game consoles,
`because of the more limited computing resources and the
`need to accommodate voice communications for multiple
`players. Accordingly, there is clearly a need for a method
`and system to enable voice communication for games played
`on one or more multiplayer game consoles, which addresses
`the issues noted above.
`
`SUMMARY OF THE INVENTION
`
`[0012] As noted above, voice communication between one
`person playing a game on a PC connected over a network
`and one or more other players of the game, cach using a
`separate PCis well known. In contrast, the present invention
`enables more than one person playing a game on a game
`console to verbally communicate locally and/or over a
`network with one or more other people playing the game. To
`avoid creating too much of a demand for computing
`resources to support verbal communication as more people
`join in the game, the present invention enables a designer of
`the game to establish a predefined limit on the computing
`resources that will be employed to support verbal commu-
`nication between players of the game.
`
`[0013] Specific hardware is provided for use wilh each
`multiplayer game console to support verbal communication
`between players of a game. Each player who ts verbally
`communicating is provided with a headset thal includes a
`microphone (more generally, an audio sensor) and a head-
`phone (more generally, an audio transducer). The headset is
`coupled to a voice communication module, which is either
`attached to a game controller in one embodiment, or inte-
`erally included in a game controler in another embodiment.
`An input signal from the microphone is compressed and
`digitized by an encoder in the multiplayer game console.
`The digitized and compressed signal
`is
`then conveyed
`through a voice communication channel to another player
`who is an intended recipient. A decoder decompresses the
`compressed signal to produce an output signal thal is applied
`to the headphone of the other player, to produce an audible
`sound corresponding to the sound originally produced and
`incident on the microphone of the player from whom the
`voice communication was conveyed. As used herein and in
`the claims that follow the term “voice communication” is
`intended to be synonymous and inierchangeable with the
`term “verbal communication.” Also, it will be understood
`that “verbal communication” or “voice communication”are
`
`intended to encompass the conveyance of spoken words and
`phrases, as well as other sounds or utterances such ds grunts,
`screams, cic. that are produced by a player participating in
`a game.
`
`[0014] A level of computing resources allocated to pro-
`cessing verbal communications on the multiplayer game
`console is preferably predefined and fixed so that
`it
`is
`independent of the number of players using voice commu-
`nication while playing the game, This feature ensures thal an
`increase in the qumber of players who are verbally commu-
`nicating during a game does not adversely impact upon the
`quality of other aspects of the game play,
`
`[O01S] Data comprising the digital compressed signal are
`in a specific format when conveyed to one or more intended
`recipients playing the game on one or more game consoles.
`The digital compressed signal
`ts conveyed over either a
`direct
`link or through a network, such as the Internet.
`Depending upon the game, each player can be enabled to
`select an intended recipient of a verbal communication from
`among the other players of the game. For example, members
`of a team can verbally communicate with cach other and
`with a team leader over one voice channel, but the team
`leader may be able to selectively communicate with the team
`members over thal same voice channel or with another team
`leader over a different voice channel,
`
`Epic Games Ex. 1027
`Epic GamesEx. 1027
`Page 16
`Page 16
`
`

`

`US 2003/0216178 Al
`
`Nov. 20. 2003
`
`Preferably, the step of encoding includes the step of
`[0016]
`converting the input signal from an analog signal to a digital
`signal, and compressing the digital signal to produce a
`compressed digital signal in the appropriate format. Simi-
`larly, the step of decoding preferably includes the steps of
`decompressing the compressed signal io produce a decom-
`pressed digital signal, and converting the decompressed
`digital signal to the output signal used to drive the audio
`transducer.
`
`[0017] The format used for the compressed signal com-
`prises a plurality of audio data packets. Each audio data
`packet extends over an audio time frame. Preferably, a
`predefined number of encoding instances are operative on
`the game console during each audio time frame. The pre-
`defined number is. preferably less than a total number of
`players using the game console and producing verbal ultter-
`ances to verbally communicate with other players, The
`verbal ullerances of players on the game console are
`encoded in such a manner that if more players on the game
`console than the predefined number of encoding instances
`are speaking in successive audio time frames, a round robin
`selection is applied in choosing the verbal utterances thal are
`encoded in successive audio time frames. The game being
`played determines the predefined number of encoding
`instances active al one time.
`
`Parallel decoding of encoded data streams received
`[0018]
`from other multiplayer game consoles can be employed in
`conjunction with one or more mixers lo produce the output
`signal that drives the players’ headphones.
`
`execuling an instance of a game, Verbal communication
`input and output devices are included for each player who
`will be verbally communicating during a game, and each
`verbal communication input and output device has a sound
`sensor that produces an input signal indicative of sound
`incident on the sound sensor, and a sound transducer that
`produces an audible sound in response to an output signal
`that is applied to the sound transducer. The machine instruc-
`tions in the memory cause the processor
`to carry out
`functions generally consistent with the sieps of the method
`described above to enable verbal communication.
`
`BRIEF DESCRIPTION OF THE DRAWING
`PIGURES
`
`[0023] The foregoing aspects and many of the altendant
`advantages of this invention will become more readily
`appreciated as the same becomes better understood by
`reference to the following detailed description, when Laken
`in conjunction with the accompanying drawings, wherein:
`[0024] FIG. 1 is a schematic isometric view of a multi-
`player game console and voice communication system in
`accord wilh the present invention;
`
`[0025] FIG. 2 is a block diagram ofthe multiplayer game
`console and voice communication module of FIG. 1;
`
`[0026] FIG. 3 is a functional block diagram of a multi-
`player game console with voice communication capability;
`
`[0027] FIG. 4 is a functional block diagram illustrating
`two multiplayer game consoles coupled in point-to-point
`communication over a network;
`
`[0028] FIG. 5 is a block diagram illustrating a first mul-
`tiplayer game console coupled in communication with three
`olher multiplayer game consoles over a network;
`
`[0029] FIG. 6 is functional block diagram illustrating
`prioritization encoding for a plurality of players on a game
`console having two parallel encoders;
`
`[0019] Selected players can be assigned to one or more
`channels, and players who are assigned lo a common chan-
`nel are enabled to selectively verbally communicate with
`each other. In one embodiment, each player is assigned to
`one or more listener channels on which the player can
`receive a verbal communication from other players.
`In
`addition, each player is assigned to one or more talker
`channels over which a verbal utterance is conveyed to other
`players, The verbal utterances heard by a specific player on
`(0030) FIG. 7 is a logic diagram illustrating the steps
`the multiplayer game console are determined by logically
`employedby the present invention in selecting packets from
`combining (i.c., ANDing) the listener channel of the specilic
`a queue to decode on a multiplayer game console;
`player with the talker channel of the player making the
`verbal utterance.
`[0031] FIG.8is a functional block diagram illustrating a
`Type 1 decoding engine used in the multiplayer game
`console;
`
`[0020] The method also includes the step of providing oral
`synchronization data for controlling an oral portion (e.g., the
`mouth) of an animated graphie character in the game, The
`oral synchronization data are used to control the oral portion
`of the animated graphic character that is displayed during
`the game, so that it moves in synchronization with the verbal
`uilerance of the player who controls the animated graphic
`character.
`
`(Q021] A player is enabled to prevent verbal communica-
`tion with a selected other player. Verbal communication with
`the selecied player can be prevented in just a current game
`being played, or can be prevented in any game in which both
`the player and the selected other player are participating.
`
`[0022] A further aspect of the present invention is directed
`lo a system that enables verbal communication between
`players who are playing a game. The system includes a
`multiplayer game console having a processor and a memory,
`In the memory are stored machine instnictions for causing
`(he processor to carry out a plurality of functions, including
`
`FUG. 9A is a functional block diagram illustrating
`[0032]
`a Type 2 decoding engine used in the multiplayer game
`console;
`
`[0033] FIG. 9B is a block diagram illustrating details of
`ihe mixers and 4-stream parallel decoder of FIG. 9A;
`
`[0034] FIG. 10 is a logic diagram ilustrating further
`details of the steps for transmitting and receiving encoded
`packets of sound data over a network;
`
`[0035] FIG. 11 is a functional block diagram showing
`how voice streams are received, queued, and decoded tor
`cach player on a multiplayer game console;
`[0036] FIG. 12 isa flow chartillustrating the steps carried
`out for round robin encoding of sound packets;
`
`FUG. 13 is an exemplary user interfacefor selecting
`[0037]
`voice options in a game that employs voice communication
`berween players; and
`
`Epic Games Ex. 1027
`Epic GamesEx. 1027
`Page 17
`Page 17
`
`

`

`US 2003/0216178 Al
`
`Nov. 20. 2003
`
`[0038] FIGS. 13A and 13B illustrate two different options
`that are selectable by a player to control or preclude voice
`communication with another player in a multiplayer game,
`in accord with the present invention.
`
`DESCRIPTION OF THE PREFERRED
`EMBODIMENT
`
`[0039] Exemplary Gaming System for Practicing the
`Present Invention
`
`[0040] As shown in FIG. 1, an exemplary electronic
`gaming system LM)
`includes a game console 102 and
`support for up to four user input devices, such as controllers
`1)4¢@ and 104b,. Game console 102 is equipped with an
`internal hard disk drive (not shown in this Figure) and a
`portable media drive 106 thal supports various forms of
`portable optical storage media, as represented by an optical
`storage disc 108. Examples of suitable portable storage
`media include DVD dises and compact disk-read only
`memory (CD-ROM) discs, In this gaming system, game
`programs are preferably distributed for use with the game
`console on DVD dises, bul itis also contemplated that other
`storage media might instead be used, or that games and other
`programs can be downloaded over the [nternet or other
`network.
`
`[0041] On a front face of game console 102 are four
`connectors 110 that are provided for electrically connecting
`lo the controllers.
`[t
`is contemplated that other types of
`connectors or wireless connections might alternatively be
`employed. A power button 112 and a dise tray eject button
`114 are also positioned on the frout face of game console
`102. Power button 112 controls application of electrical
`powerto the game console, and eject button 114 alternately
`opens and closes a tray (not shown) of portable media drive
`106 to enable insertion and extraction of storage disc LOS so
`that the digital data on it can be read und loaded into memory
`or stored on the hard drive for use by the game console.
`
`[0042] Game console 102 connects to a television or other
`display monilor or screen (not shown) via audio/visual
`(A/V)interface cables 120. A power cable plug 122 conveys
`electrical power to the game console when connected to a
`conventional alternating current
`line source (nol shown).
`Game console 102 may be further provided with a data
`connector 124 to transfer data through an Lthernet connec-
`lion to.a network and/or the Internet, or through a broadband
`connection. Alternatively, it is contemplated that a modem
`(not shown) may be employed to transfer data to a network
`and/or the Internet. As yet a further alternative, ihe game
`console can be directly Linked to another game console via
`an Ethernet cross-over cable (not shown).
`
`(0043) Each controller 104¢ and 1046 is coupled to game
`console 102 via a lead (or in another contemplated embodi-
`ment, alternatively,
`through a wireless interface),
`In the
`illustrated implementation,
`th