(19) United States
`(12) Patent Application Publication
`EPSTEIN
`
`(10) Pub. N0.: US 2010/0137049 A1
`(43) Pub. Date:
`Jun. 3, 2010
`
`US 20100137049A1
`
`(54) INTERACTIVE GUITAR GAME DESIGNED
`FOR LEARNING T0 PLAY THE GUITAR
`
`Publication Classi?cation
`
`(51) Int. Cl.
`
`(76) Inventor:
`
`
`
`Joseph Charles EPSTEIN, New York, NY (US)
`
`0
`
`(
`
`~
`
`)
`
`Correspondence Address:
`WILSON, soNsINI, GOODRICH & R0sATI
`650 PAGE MILL ROAD
`PALO ALTO, CA 94304-1050 (US)
`
`(21) Appl. N0.:
`
`12/623,368
`
`(22) Filed:
`
`Nov. 20, 2009
`
`Related US. Application Data
`
`(60) Provisional application No. 61/117,053, ?led on Nov.
`21, 2008.
`
`(52) as. C]. ............................................. .. 463/7; 463/35
`
`ABSTRACT
`(57)
`An interactive game designed for learning to play a guitar. A
`guitar may be connected to a computer or other platform,
`capable of loading music and displaying notes and chords and
`other feedback and visual learning aids on a display screen,
`alloWing a user to read music and play along. The goal of the
`software or interactive game engine is for players to learn hoW
`to play a guitar. Users may operate the game in a number of
`modes With different goals, playing mini-games throughout
`the levels of the game. The game provides feedback and
`statistics to help users learn hoW to play the guitar.
`
`Display /
`Audio Output /\-150
`
`Guitar/ Musical
`Instrument
`
`Guitar/ Musical
`Input
`
`(
`
`130
`
`110
`
`Computer/Platform
`
`Game
`Engine\
`\
`120
`
`Storage
`\ 7
`1
`121
`
`Network
`
`?
`140
`
`\ 100
`
`Petitioners Ex. 1004 Page 1
`
`

`

`Patent Application Publication
`
`Jun. 3, 2010 Sheet 1 0f 22
`
`US 2010/0137049 A1
`
`Display /
`Audio ()utput X150
`
`Guitar/ Musical
`Instrument
`
`I
`
`110
`
`Computer/Platform
`
`Gultarl Muslcal
`Input
`
`130
`
`Game
`
`Engine\2
`
`\
`
`\
`120
`
`Storage
`
`7/
`
`\
`121
`
`Network
`
`F
`
`140
`
`\\ 100
`
`FIG. 1
`
`Petitioners Ex. 1004 Page 2
`
`

`

`Patent Application Publication
`
`Jun. 3, 2010 Sheet 2 0f 22
`
`US 2010/0137049 A1
`
`205
`
`206
`Signal
`Processing
`
`5on9
`/\ library
`
`V
`
`Com uter
`p
`
`Game /V2O8
`manager
`
`v
`
`Miriam
`292/
`'0
`
`Game
`I
`
`notes
`
`of
`
`"
`
`9
`/\/2(3
`
`mix;
`9
`
`Audio
`output
`
`20“
`
`FIG. 2
`
`Petitioners Ex. 1004 Page 3
`
`

`

`Patent Application Publication
`
`Jun. 3, 2010 Sheet 3 0f 22
`
`US 2010/0137049 A1
`
`o5 \/\ mcom \xonoxsw
`
`
`
`
`26E >31
`
`
`
`an \1k 282 59w
`
`08 \KEEQ @2225:
`
`
`262
`
`ovm \/ $303
`
`
`
`\muo_>_ 260C926
`
`m GE
`
`\
`
`:2623 A
`
`
`26E wEmw 26E Em_>_
`
`I
`
`<Om. H
`
`mow \) mEmw
`K cozmSmccou
`
`On»
`
`Any
`
`
`
`wt?w @960
`
`Petitioners Ex. 1004 Page 4
`
`

`

`Patent Application Publication
`
`Jun. 3, 2010 Sheet 4 0f 22
`
`US 2010/0137049 A1
`
`<% w
`
`Petitioners Ex. 1004 Page 5
`
`

`

`Patent Application Publication
`
`Jun. 3, 2010 Sheet 5 0f 22
`
`US 2010/0137049 A1
`
`(1.0m
`
`508
`
`509
`
`Baa
`
`507
`$CRE
`3 6,863
`EQMUS
`2X
`Note Streak
`‘i4
`
`FIG. 5
`
`Petitioners Ex. 1004 Page 6
`
`

`

`Patent Application Publication
`
`Jun. 3, 2010 Sheet 6 0f 22
`
`US 2010/0137049 A1
`
`606
`
`608
`
`sa
`19,959
`BONUS
`2%
`Nate Streak
`
`2B
`
`. ........................ ..
`
`FIG. 6
`
`Petitioners Ex. 1004 Page 7
`
`

`

`Patent Application Publication
`
`Jun. 3, 2010 Sheet 7 0f 22
`
`US 2010/0137049 A1
`
`148,429
`
`24 112 FIG. 7
`ETREJEK
`
`Petitioners Ex. 1004 Page 8
`
`

`

`Patent Application Publication
`
`Jun. 3, 2010 Sheet 8 0f 22
`
`US 2010/0137049 A1
`
`
`
`
`
`mmmiwm mash? met. a m?cmzm Maw 55%: a5;
`
`
`
`
`
`
`
`wmhm NWF _.
`
`Petitioners Ex. 1004 Page 9
`
`

`

`Patent Application Publication
`
`Jun. 3, 2010 Sheet 9 0f 22
`
`US 2010/0137049 A1
`
`FIG. 9
`
`902
`
`Petitioners Ex. 1004 Page 10
`
`

`

`Patent Application Publication
`
`Jun. 3, 2010 Sheet 10 0f 22
`
`US 2010/0137049 Al
`
`N
`O
`O
`T-
`
`‘_
`
`g
`
`‘-
`
`C)
`CD
`O
`?
`
`Muitipiayar
`
`
`
`Sie’zgie may-e?‘
`
`O
`
`LL
`
`N
`P
`C)
`T
`
`F
`\_
`
`o
`
`v
`
`CD
`\
`O
`P
`
`Petitioners Ex. 1004 Page 11
`
`

`

`Patent Application Publication
`
`Jun. 3, 2010 Sheet 11 0f 22
`
`US 2010/0137049 A1
`
`Petitioners Ex. 1004 Page 12
`
`

`

`Patent Application Publication
`
`Jun. 3, 2010 Sheet 12 0f 22
`
`US 2010/0137049 A1
`
`FIG. 12
`
`Petitioners Ex. 1004 Page 13
`
`

`

`Patent Application Publication
`
`Sm
`
`U
`
`mmEmm
`
`
`
`2w,hmfiacmmamaMED“mPom/x“mafia
`
`
`
`nmEmm”to;mmmmgME35w?wEmw3m?“
`
`
`“3ngmmmaB90%2”aEmamto82mw>;c,.u.m
`
`
`
`OC$me52mm
`
`m2.QE
`
`
`
`
`
` 0,mEmmaBmeméEmaWMmMWCmmmmom?.EEm5E§mDm8262mmEmma5..
`
`Petitioners Ex. 1004 Page 14
`
`
`
`meme:67”.959xonmxsw
`
`35EEwE253mm
`
`@302
`
`Petitioners Ex. 1004 Page 14
`
`
`
`
`
`

`

`Patent Application Publication
`
`US 2010/0137049 A1
`
`moi
`
`wm IT Paw: mm 652
`‘ 322w 5mm
`
`
`
`wmsi 33m
`
`
`
`, Em cm waging
`
`P03
`
`Petitioners Ex. 1004 Page 15
`
`

`

`Patent Application Publication
`
`Jun. 3, 2010 Sheet 15 0f 22
`
`US 2010/0137049 A1
`
`Home
`
`Story
`
`Open
`Strings
`
`Animation
`Interstitial
`
`t
`
`Fret View
`Open Strings ewmm Retry
`- Picklng
`
`Pass
`
`M'MFaiIR
`
`Animation
`Interstitial
`
`Nab.
`
`Animation
`Interstitial
`
`Timeline View
`Open Strings
`
`mM‘FaiL
`
`— In Time
`on“
`Pass
`t
`Animation
`Interstitial
`
`Retry
`
`“em-WM
`
`I
`C
`j/FOUSG
`
`'eW
`y,
`//
`Practice
`“Wm
`Animation /
`Interstitial
`
`t
`
`Fret View
`Frets & The Neck
`
`Fretted
`Strings
`
`Animation
`Interstitial
`l
`
`Retry
`
`Animation
`Interstitial
`
`FIG. 15A
`
`Petitioners Ex. 1004 Page 16
`
`

`

`Patent Application Publication
`
`Jun. 3, 2010 Sheet 16 0f 22
`
`US 2010/0137049 A1
`
`Timeline View
`Frets & The Neck
`— In Time
`
`Retry
`
`ere-WW
`
`I
`C
`avr.0uSe
`lew
`
`I ~~~~~~~~~~~~~~~~~~~ to
`Pass
`“““““ “Fail _______ a
`It
`~~~~~~~~~~~~
`
`?u
`
`t
`//
`Practice
`
`- - Animatlon
`
`
`Interstitial
`
`Animation //
`Interstitial
`
`Open
`Chords
`
`Fret View
`Open Strings
`- Ring Test
`
`Retry
`
`Pass
`I
`Animation
`Interstitial
`
`Animation
`Interstitial
`
`Timeline View
`Open Chords
`— In Time
`
`t ----- WW
`
`Retry
`
`I
`C
`i/F‘mse
`‘6W
`
`,
`
`5W mmmmmmmmmmmmmmmmmm \
`jt’
`Pass
`MM‘FaiI rrrrrrr N
`//
`t,
`~~~~~~~~~~~~~ M
`Practice
`Animation
`Animation ///
`Interstitial
`Interstitial
`
`chaptem
`Complete
`
`l
`
`FIG. 155
`
`Petitioners Ex. 1004 Page 17
`
`

`

`Patent Application Publication
`
`Jun. 3, 2010 Sheet 17 0f 22
`
`US 2010/0137049 A1
`
`52»
`
`@575
`
`wwwwwwwwwww J
`
`wwzmm m
`
`@P GE
`
`
`
`
`
`‘$30 :Eim “$5
`
`,» A.
`
`wag $2
`
`mmmmm
`
`88 S8
`
`Q08
`
`Petitioners Ex. 1004 Page 18
`
`

`

`Patent Application Publication
`
`Jun. 3, 2010 Sheet 18 0f 22
`
`US 2010/0137049 A1
`
`1553;“; gamgg
`
`b43535 B1355
`
`"0000 6%
`V
`
`hiss
`iginilig?mgg H
`O I O I
`
`1
`
`1702
`
`3/5
`
`M1118 games
`1
`
`“55 3955
`1 1
`
`Main; ggama?
`1
`
`...OIԤ? OOOOEIQ
`
`bass. E0551
`1
`1
`
`_
`
`mm games
`M153 31935
`L
`.1706‘
`l 1
`000 a
`
`1707
`V
`
`111m; games 1705
`
`was sass
`
`i 1
`Level? 2;
`(5000.& 1/5
`
`FIG. 17
`
`Petitioners Ex. 1004 Page 19
`
`

`

`Patent Application Publication
`
`Jun. 3, 2010 Sheet 19 0f 22
`
`US 2010/0137049 A1
`
`BNYQE.
`
`i \ T‘ \ ‘ 3.
`
`\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\M.
`
`\\\\\\\\\\\\\\\\\\\\\\\\\\\\\
`
`.
`
`Petitioners Ex. 1004 Page 20
`
`

`

`Patent Application Publication
`
`Jun. 3, 2010 Sheet 20 of 22
`
`US 2010/0137049 A1
`
`
`
`mEmmfigm
`
`gamma.
`
`‘\
`
`f$33252\/roar
`
`\\_
`
`2.®_u_
`
`moor
`
`Egmfiyum
`
`35EEmaQEflwm
`
`
`
`mEmmEmmi
`
`
`
`:flwmummmmmamw
`
`mmmfim
`
`Nomw
`
`Petitioners Ex. 1004 Page 21
`
`Petitioners Ex. 1004 Page 21
`
`
`
`

`

`Patent Application Publication
`
`Jun. 3, 2010 Sheet 21 of 22
`
`US 2010/0137049 A1
`
`yer S 525
`5555i {5555
`. 53555 5555 553555
`
`FIG. 20
`
`5 555355 5555 5 555:5H5555
`F5 5555 5'. 55535 55555
`
`Petitioners Ex. 1004 Page 22
`
`Petitioners Ex. 1004 Page 22
`
`

`

`Patent Application Publication
`
`Jun. 3, 2010 Sheet 22 0f 22
`
`US 2010/0137049 A1
`
`2101
`
`
`
`
`
`Network
`
`(Internet)
`
`
`
`
`“m“
`\i‘
`g
`
`Pumfigé emu
`
`mm tea: newt?
`pumhaeett merge
`
`2102
`
`‘
`
`\1
`
` Wk
`
`2103
`
`2104
`
`At Emit {315:}1%:1y
`
`titayed
`
`eertge if} he
`
`FIG. 21
`
`Petitioners Ex. 1004 Page 23
`
`Petitioners Ex. 1004 Page 23
`
`

`

`US 2010/0137049 A1
`
`Jun. 3, 2010
`
`INTERACTIVE GUITAR GAME DESIGNED
`FOR LEARNING TO PLAY THE GUITAR
`
`CROSS-REFERENCE
`
`[0001] This application claims the benefit of US. Provi-
`sional Application No. 61/117,053 filed NOV. 21, 2008 which
`application is incorporated herein by reference in its entirety.
`
`BACKGROUND
`
`[0002] Learning to play a musical instrument can be an
`enjoyable and rewarding experience. However, to develop
`proficiency in playing a musical instrument requires dedica-
`tion, many hours of practice, and the application of proper
`playing techniques and methods.
`[0003] Conventional learning tools and sources of instruc-
`tional information for learning to play a musical instrument
`include music teachers, music books, audio tapes or compact
`disks (CDs), and video tapes. While each of these offers
`certain advantages, each is also limited in the quality of
`instruction or the manner in which the information is pre-
`sented.
`
`For example, a music teacher provides personally-
`[0004]
`tailored instruction, however the cost of the instruction is
`relatively expensive and each session is necessarily limited in
`both time and depth. Additionally, the student may be limited
`to learning the playing style and habits (both good and bad) of
`the particular teacher, which may limit the student’s creativity
`and spontaneity. The practice material assigned by the
`instructor is also static and is therefore unable to accommo-
`
`date and adjust to the student’s individual progress or profi-
`ciency.
`[0005] Music books provide a relatively low-cost alterna-
`tive or adjunct to a music teacher. However, the material
`presented by music books is necessarily static and if the
`student has difficulty reading and interpreting music, particu-
`larly the tempo and rhythm, then music books provide a
`limited instructional capability. Additionally, music books
`are not interactive and provide no feedback to the student,
`making it difficult to develop a “feeling” for the music which
`is necessary to progress beyond purely technical or mechani-
`cal playing. Audio tapes or compact discs combine recorded
`songs and musical passages with instructional commentary
`and playing tips. But still, they lack visual instruction and
`feedback, which can make it difficult for the student to
`develop a feeling for the music.
`[0006] What is needed is an effective way to provide inter-
`active method and system for learning and practicing a musi-
`cal instrument, which provides both audio and visual feed-
`back, and an integrated learning approach.
`
`SUMMARY OF INVENTION
`
`[0007] The invention provides for an interactive guitar
`game designed for learning to play the guitar. Various aspects
`ofthe invention described herein may be applied to any of the
`particular applications set forth below. The invention may be
`applied as a standalone game engine system or as a compo-
`nent of an integrated software solution. It shall be understood
`that different aspects of the invention can be appreciated
`individually, collectively or in combination with each other.
`[0008] Unless specifically stated otherwise, as apparent
`from the following discussions, it is appreciated that through-
`a, cc
`out the specification, discussions utilizing terms such as “pro-
`cessing,” “computing,
`calculating,” “determining,” or the
`
`like, may refer in whole or in part to the action and/or pro-
`cesses of a processor, computer or computing system, or
`similar electronic computing device, that manipulate and/or
`transform data represented as physical, such as electronic,
`quantities within the system’s registers and/or memories into
`other data similarly represented as physical quantities within
`the system’s memories, registers or other such information
`storage, transmission or display devices. It will also be appre-
`ciated by persons skilled in the art that the term “players” or
`“users” referred to herein can be individuals as well as cor-
`
`porations and other legal entities, such as schools or other
`educational
`institutions. Furthermore,
`the processes pre-
`sented herein are not inherently related to any particular com-
`puter, processing device, article or other apparatus. An
`example of a structure for a variety of these systems will
`appear from the description below. In addition, embodiments
`of the present invention are not described with reference to
`any particular processor, programming language, machine
`code, etc. It will be appreciated that a variety ofprogramming
`languages, machine codes, etc. may be used to implement the
`teachings of the invention as described herein.
`[0009] Other goals and advantages of the invention will be
`further appreciated and understood when considered in con-
`junction with the following description and accompanying
`drawings. While the following description may contain spe-
`cific details describing particular embodiments of the inven-
`tion, this shouldnot be construed as limitations to the scope of
`the invention but rather as an exemplification of preferable
`embodiments. For each aspect of the invention, many varia-
`tions are possible as suggested herein that are known to those
`of ordinary skill in the art. A variety of changes and modifi-
`cations can be made within the scope ofthe invention without
`departing from the spirit thereof.
`
`INCORPORATION BY REFERENCE
`
`[0010] All publications and patent applications mentioned
`in this specification are herein incorporated by reference to
`the same extent as if each individual publication or patent
`application was specifically and individually indicated to be
`incorporated by reference.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`Some of the features of the invention are described
`[0011]
`as set forth in the following figures and description. A better
`understanding ofthe features and advantages ofthe invention
`will be obtained by reference to the following detailed
`description that sets forth illustrative embodiments provided
`in accordance with the invention.
`
`FIG. 1 illustrates an example of the various compo-
`[0012]
`nents of the interactive game system, in accordance with an
`embodiment of the invention.
`
`FIG. 2 shows a flowchart of how information flows
`[0013]
`while playing the interactive game, in accordance with an
`embodiment of the invention.
`
`FIG. 3 shows an example of the overall game archi-
`[0014]
`tecture, in accordance with an embodiment of the invention.
`[0015]
`FIG. 4 shows one example of a graphical represen-
`tation of a screenshot of a moving notation on a screen, in
`accordance with an embodiment of the invention.
`
`FIG. 5 shows another example of how notes may
`[0016]
`scroll across a display screen, in accordance with an embodi-
`ment of the invention.
`
`Petitioners Ex. 1004 Page 24
`
`Petitioners Ex. 1004 Page 24
`
`

`

`US 2010/0137049 A1
`
`Jun. 3, 2010
`
`FIG. 6 shows an example of Visual feedback dis-
`[0017]
`played on a display screen, in accordance with an embodi-
`ment of the invention.
`
`FIG. 7 illustrates an example of a timeline view of
`[0018]
`notation displayed on a display screen, in accordance with an
`embodiment of the invention.
`
`FIG. 8 illustrates an example of a carousel view of
`[0019]
`notation displayed on a display screen, in accordance with an
`embodiment of the invention.
`
`FIG. 9 illustrates an example ofa guitar neck view of
`[0020]
`notes displayed on a display screen, in accordance with an
`embodiment of the invention.
`
`FIG. 10 shows examples of various scrolling meth-
`[0021]
`ods for the tablature notation or notes on a display screen, in
`accordance with an embodiment of the invention.
`
`FIG. 11 shows an example of displaying a visual
`[0022]
`graphical representation of a guitar on the screen, in accor-
`dance with an embodiment of the invention.
`
`FIG. 12 shows an example of a display where the
`[0023]
`actual fingering of a chord or note is displayed, in accordance
`with an embodiment of the invention.
`
`FIG. 13 illustrates a flowchart of what happens dur-
`[0024]
`ing Jukebox (or Song Play) mode of the game, in accordance
`with an embodiment of the invention.
`
`FIG. 14 illustrates a flowchart of what happens dur-
`[0025]
`ing Story mode of the game, in accordance with an embodi-
`ment of the invention.
`
`FIGS. 15A and 15B illustrate and example of a
`[0026]
`flowchart of what may happen during Story mode of the
`game, in accordance with an embodiment of the invention.
`[0027]
`FIG. 16 illustrates an example of the level structure
`of each level of the game, in accordance with an embodiment
`of the invention.
`
`FIG. 17 illustrates one example of completion indi-
`[0028]
`cators as used to indicate completion of levels, sub-levels and
`mini-games, in accordance with an embodiment ofthe inven-
`tion.
`
`FIG. 18 illustrates an example of a view for two
`[0029]
`players playing simultaneously on the same machine,
`in
`accordance with an embodiment of the invention.
`
`FIG. 19 shows a flowchart of the game when Mul-
`[0030]
`tiplayer (or Online) mode is selected, in accordance with an
`embodiment of the invention.
`
`FIG. 20 shows an example of how a user’s statistics
`[0031]
`may be displayed, in accordance with an embodiment of the
`invention.
`
`FIG. 21 shows an example of how the game engine
`[0032]
`may interact over a network such as the Internet, in accor-
`dance with an embodiment of the invention.
`
`DETAILED DESCRIPTION OF INVENTION
`
`In the following detailed description, numerous spe-
`[0033]
`cific details are set forth in order to provide a thorough under-
`standing of the invention. However it will be understood by
`those of ordinary skill in the art that the invention may be
`practiced without these specific details. In other instances,
`well-known methods, procedures, components and circuits
`have not been described in detail so as not to obscure the
`invention. Various modifications to the described embodi-
`
`ments will be apparent to those with skill in the art, and the
`general principles defined herein may be applied to other
`embodiments. The invention is not intended to be limited to
`
`the particular embodiments shown and described.
`
`[0034] An aspect of the invention provides an interactive
`game engine for learning to play a guitar. In one embodiment
`of the invention, any guitar, including a MIDI guitar, an OSC
`guitar, or a guitar using any other format or protocol,
`is
`connected to a computer or other platform. The computer or
`platform is capable of loading music, such that it can be
`displayed on a display screen in multiple ways, and allows the
`user to read the music andplay along. The goal ofthe software
`or interactive game engine is for players to learn how to play
`a guitar. It is understood that any references herein to specific
`file formats do not limit the scope ofthe invention, but merely
`provide examples of file formats that may be used in the
`implementation of particular embodiments of the invention,
`and various other file formats, protocols, etc. are contem-
`plated herein.
`[0035]
`FIG. 1 illustrates an embodiment of the various
`components of the system, with a computer or platform 100
`which runs a game engine 120, a guitar or musical instrument
`110 providing a guitar or musical instrument input 130, and
`video and audio outputs 150. The system may also be option-
`ally connected to a network 140. The guitar 110 may be any
`type of guitar (or other musical instrument), which the user
`can play, and may provide an input into the computer or
`platform 100, identifying which strings and frets are being
`played by the user or the velocity or timing with which the
`user plays certain notes or chords. The input may be a com-
`posite audio signal or an isolated or discrete audio signal. The
`game engine 120 may use and process this information, and
`may be coupled with a computer or other platform having a
`storage or memory 121.
`[0036] Referring to FIG. 2, one example of how informa-
`tion flows during game-play is shown. A song may be selected
`from a song library 201. A song interpreter 202 may produce
`a game representation of notes 210, to be displayed by the
`note manager 209 on the graphics display 211. At the same
`time, the song provides audio output 204. Viewing this graph-
`ics display and hearing the audio output, a user or player may
`play either a note or chord on the guitar 205. The signal
`processing module 206 may convert the analog or digital
`audio signal produced by the guitar into a digital signal read
`in by the computer 207 or other platform. The note manager
`209 may process the analog or digital audio signal (such as a
`MIDI signal or signal of another format), and compare it
`against the selected song. The signal may be a composite
`signal, or the signal may be an isolated or discrete signal. The
`analog or digital audio signal may be taken directly from a
`standard guitar out (by way of a 1A" cable from the guitar to an
`input that the computer can handle (such as a 1/8" or USB) or
`from a hex pickup (six separated isolated channels of audio).
`The audio signal may be processed and used to provide feed-
`back on the location, positioning, attack, velocity and action
`as the user plays their guitar. Thus, the system may process
`the signal by polyphonic pitch detection (for composite signal
`parsing), or may process isolated/discrete signals. Data
`derived from the comparison may be fed back to the game
`manager 208 and further processed to update the graphics
`display 211. Further, a song video 203 which corresponds to
`the song selected from the song library 201 may be played in
`the background via the graphics display 211.
`[0037] The song may be a compressed archive of the fol-
`lowing resources: a description XML file giving information
`about
`the song’s title, author, and the location of the
`MusicXML file (or file of any other format). The file may be
`a standard file format used for describing both the tonal rep-
`
`Petitioners Ex. 1004 Page 25
`
`Petitioners Ex. 1004 Page 25
`
`

`

`US 2010/0137049 A1
`
`Jun. 3, 2010
`
`resentation of the song, as well as formatting information.
`The file may also provide information about chord names,
`additional MP3 tracks (or other types of tracks), or the Video
`file to play in the background. When the file is read in, it may
`be converted into both a graphical representation (for dis-
`play), as well as a MIDI representation (or representation in
`any other format). The note manager 209 may display the
`notes graphically, depending on notations that may be speci-
`fied in the file. The note manager may display the notes to the
`graphics display 211 in time with the audio output 204 of the
`song selected from the song library 201. The notes may be
`displayed in tablature, a form of musical notation which tells
`players where to place their fingers on the guitar. The notes
`may also be displayed in modern notation which uses a five-
`line staff. It is understood that various types of file formats,
`including but not limited to the XML or MusicXML file
`formats, are contemplated.
`[0038]
`FIG. 3 shows the choices to the user which are
`available when the game starts, and also displays the overall
`game architecture. The game allows for four different modes
`to be selected from the game mode selection menu 304:
`Jukebox (or Song Play) mode 310, Story mode 320, Multi-
`player (or Online) mode 330, and Arcade (or Diagnostic)
`mode 340. The game also may have a game configuration
`module 303, which allows the user to set certain game pref-
`erences such as: volume, game difficulty, library repository
`options, and network configuration.
`[0039] Referring to FIG. 4, one example of a graphical
`representation is shown. The image presented in FIG. 4 is a
`snapshot of a moving video image which is shown on a
`display screen to a user. The moving image is a moving (or
`scrolling) notation which prompts the user to play certain
`chords or notes, such as indicated by a symbol 401 directed to
`the chord name or displaying tablature of the strings to be
`played, and the manner in which to play them (i.e. the length
`of the chord in the song). In FIG. 4, the user is prompted to
`play the G-chord (the 15‘ string at the 3rd fret, the 5‘17 string at
`the 2"“ fret, and the 6’11 string at the 3rd fret, with the 2"“, 3rd,
`and 4th strings open). As the song plays, the display moves,
`such that the notation moves and progresses in time with the
`song. In the example given in FIG. 4, the chords or notes move
`from the top-right side of the screen to the bottom-left. As
`shown in FIG. 4, upcoming notes 403 are also displayed and
`move from the top-right side of the screen towards the bot-
`tom-left. When the notes approach the hit-line or hit-area 402,
`the user is supposed to play the notes.
`[0040] The moving (or scrolling) notation is sometimes
`referred to as a “note highway” and as the song progresses,
`symbols which may be colored markers or gems indicate
`notes that travel across the screen in time with the music. The
`
`symbols’ colors and positions may match those ofthe frets on
`a guitar. Once the symbol reaches a “hit zone,” the player
`must play the indicated note.
`[0041] As the user listens to the song and watches the
`display on a display screen in any version of the views
`described herein, the user may attempt to play the chords and
`notes that he or she is prompted to play. Depending on the
`user’s accuracy with regard to which fret and strings are
`actually played, the velocity with which each note or chord is
`played, and the timing ofthe user’s play, the game engine may
`calculate a score or provide the user with other feedback as to
`his or her accuracy. As shown in FIG. 4, the score 404 may be
`shown as a numerical score.
`
`[0042] Referring to FIG. 5, another example of how notes
`may scroll across the display screen for a user to play is
`shown. The six strings of the guitar 501 may be displayed
`across the screen. As a song plays, notes or tab bubbles 502
`may move across the screen from the right-side to the left- side
`in time with the song. The hit bar/zone 503 may indicate when
`the notes 502 are supposed to be played. When the notes 502
`reach the hit bar 503, the user is supposed to play the notes
`502. The notes are either shown as individual notes with their
`
`duration of play or duration bar, 505 for example, or as
`chords, 502 for example. If a chord 502 is being played, the
`name of the chord 504 may also be displayed. Further, the
`stroke-direction may be shown in the duration bar by the
`strum indicator 506, by the arrows indicating which direction
`the user is supposed to stroke the chord. A power-up indicator
`may also be displayed for game play. For example, if a user is
`able to play a certain number of notes accurately, a power-up
`indicator may display a note that the user may triple their
`score if they get the next X number of notes correct. Thus, a
`power-up indicator may display some kind of score modifier
`that may become available to a user.
`[0043] As the user listens to the song and watches the
`display on the display screen, the user may attempt to play the
`chords and notes that he or she is prompted to play. Depend-
`ing on the user’s accuracy with regard to which fret and
`strings are actually played, the velocity with which each note
`or chord is played, the direction of the strokes, and the timing
`of the user’s play, the game engine may calculate a score and
`score modifiers 507 or provide the user with other feedback
`508 as to his or her accuracy and performance, in the form of
`a success meter, for example. Further, to immerse the user or
`player in the music, a movie or video 51 0 may be played in the
`background.
`[0044]
`In FIG. 6, a more detailed example of what the
`visual feedback that a user experiences is shown. Whenever a
`user plays a correct note, points 601 may be rewarded along
`with a textual representation of the score 602. Thus, the game
`engine may provide both a numeric indicator for points
`scored 601 and a qualitative indicator for a correctly or incor-
`rectly played note 602. For example, the textual representa-
`tion of the qualitative indicator 602 may be words of encour-
`agement or praise such as, “Awesome!” as shown in FIG. 6.
`The notes or chord 603 may visually appear to float from the
`block to which they were attached, and eventually fade and
`disappear as 604. Thus, the graphical indicator for the cor-
`rectly played note 603 may fade and disappear 604. As the
`chords and notes are played and the feedback is presented to
`the user, the performance or success meter 607 (or score and
`score modifiers 605) may be updated. In addition, the actual
`note played by the user may also be indicated. Further, the
`game may provide for a background audience audio feed-
`back, which provides audio output mimicking an audience
`watching the user’s performance. As the user plays a particu-
`lar song, the volume of the background audiences may be
`updated in accordance with his or her performance. Thus, the
`game engine may provide new audio and video feedback
`based on the success or failure of the user.
`
`[0045] Many different views of the scrolling notation are
`possible. For example, FIG. 7 illustrates an example of a
`timeline view of the scrolling notation, in accordance with an
`embodiment of the invention. As shown in the timeline view,
`the notes 703, 705 may scroll from the right side 701 to the left
`side 702. There may also be a video in the background 707
`and an audio track timed to the scrolling notes 703, 705.
`
`Petitioners Ex. 1004 Page 26
`
`Petitioners Ex. 1004 Page 26
`
`

`

`US 2010/0137049 A1
`
`Jun. 3, 2010
`
`Before play begins, the user may be given a one bar count off,
`derived from the tempo and time signature of an audio file. As
`the notes reach a region towards the left side ofthe screen (the
`hit area 704), the user may play those notes on their guitar.
`The user may receive a score 706, which is updated as the user
`plays along, to reflect how close to the actual hit line the user
`played the note, or ifthe user missed the note all together. The
`score may also reflect a note streak (how many notes the user
`has correctly and accurately played in a row), the number of
`hits (how many notes the user has played correctly in the song
`or exercise) and a accuracy rating (the percentage ofnotes that
`the user has played correctly in the song or exercise). Depend-
`ing on the user’s action, the view may also give the user visual
`feedback by changing the representation of the note. As the
`user’s score changes, different crowd noises may be provided
`as audio feedback.
`
`Ifthere are no notes on the screen, the interface may
`[0046]
`display the next offscreen note on the far right ofthe screen so
`as to alert the user of an upcoming note. The interface may
`also display certain metrics regarding the user’s performance,
`including a score and a current note streak 706. Additionally,
`the interface may display a timeline for a whole song, broken
`into sections, showing the user where they are and giving a
`general sense as to how the user did on a particular section of
`a song. For example, a user’s progress may be tracked by a
`progress bar 708, which indicates how far along the user is in
`the song, or ifthe user is currently playing in the introduction,
`chorus, bridge, a solo breakout, etc. The interface elements
`may be done as 2D overlays onto a 3D space. The lines
`representing the guitar may be 3D curves starting at the right
`side far back 702, expanding and coming nearer as they
`approach the left side ofthe screen 701. The inside ofthe note
`bubble 703 may contain the number of the fret to be pressed.
`A bubble containing the note or chord name may also scroll
`with any note or chord above the strings.
`[0047]
`FIG. 8 illustrates an example of a carousel view of
`the scrolling notation, in accordance with an embodiment of
`the invention. In the carousel view, note bubbles with frets
`803 (or other note symbols) may also be displayed, in addi-
`tion to note or chord names above the strings 805. Rather than
`scrolling from left to right, however, the notes may circle
`around clockwise and rotate. Thus, the notes will repeat, after
`the carousel completes each rotational cycle. The carousel
`800 may be semi-transparent or translucent such that some of
`the notation is displayed in the foreground (or a front portion)
`801, and some of the notation is displayed in the background
`(or a back portion) 802. Because the carousel 800 may be
`translucent, notation displayed in the background may be
`viewable by the user through the foreground. As the notation
`rotates, the notation displayed in the foreground 803 is rotated
`to the background 802, and vice versa (i.e. the notation dis-
`played in the foreground may travel along the carousel to the
`background, and vice versa, in a circular manner). If the note
`is played by the user within the time that the note symbol is
`displayed within a hit zone 804, the note may change in color
`in a certain number of stages, or other real-time feedback of
`the user’s accuracy with regard to the notes and timing may be
`provided to the user, such as in a score box 806. For example,
`there may be three stages, and when a user plays a note within
`the hit zone 804, then the note may be change in color (i.e.
`from the color indicating stage 1 to the color indicating stage
`2. Alternatively, if the user misses a note, then the note may
`change in color (i.e. from the color indicating stage 2 to t