a2) United States Patent
`US 7,081,579 B2
`(0) Patent No.:
`Jul. 25, 2006
`(45) Date of Patent:
`Alcalde et al.
`
`US007081579B2
`
`METHOD AND SYSTEM FOR MUSIC
`RECOMMENDATION
`
`(56)
`
`References Cited
`
`(54)
`
`(75)
`
`Inventors: Vicen¢ Gaitan Alcalde, Castella del
`Valles (ES); Carlos Maria Lépez
`Ullod, Saragossa (ES); Antonio Trias
`Bonet, Sant Cugat del Valles (ES);
`Antonio Trias Llopis, San Cugat del
`Valles (ES); Jestis Sanz Marcos,
`Barcelona (ES); Daniel Caldentey
`Ysern, Barcelona (ES); Dominic
`Arkwright, Barcelona (ES)
`
`U.S. PATENT DOCUMENTS
`
`4/1997 Cluts ..........
`... 84/609
`5,616,876 A
`
`8/1998 Timis et al.
`....
`... 84/609
`5,792,971 A
`6/1999 Blum et al. wo... 707/1
`5,918,223 A *
`
`9/2001 Suzuki et al. ..........
`... 84/626
`6,284,964 Bl
`
`.......... 84/667
`1/2002 Gjerdingen et al.
`2002/0002899 Al
`2002/0147628 Al* 10/2002 Specter et al... 705/10
`.. 700/94
`2003/0045953 Al*
`3/2003 Weare
`
`2/2004 Kruse 0... eee eee 713/176
`2004/0039913 Al*
`
`* cited by examiner
`
`Primary Examiner—MarlonT. Fletcher
`Assistant Examiner—Jianchun Qin
`(74) Attorney, Agent, or Firm—Whiteford, Taylor &
`Preston; Jeffrey C. Maynard; Gregory M. Stone
`
`(57)
`
`ABSTRACT
`
`An artificial intelligence song/music recommendation sys-
`tem and method is provided that allows music shoppers to
`discover new music. The system and method accomplish
`these tasks by analyzing a database of music in order to
`identify key similarities between different pieces of music,
`and then recommends pieces of music to a user depending
`upon their music preferences. An embodiment enables a user
`to evaluate a new song’s similarity to songs already estab-
`lished as commercially valuable.
`
`20 Claims, 5 Drawing Sheets
`
`analysis machine
`
`analyst wrapper
`
`output data
`
`EX1034
`Roku V. Media Chain
`U.S. Patent No. 9,715,581
`
`(73)
`
`Assignee: Polyphonic Human Media Interface,
`S.L., Barcelona (ES)
`
`Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`US.C. 154(b) by 98 days.
`
`(21)
`
`Appl. No.: 10/678,505
`
`(22)
`
`Filed:
`
`Oct. 3, 2003
`
`(65)
`
`(60)
`
`(51)
`
`(52)
`(58)
`
`Prior Publication Data
`
`Jun. 10, 2004
`US 2004/0107821 Al
`Related U.S. Application Data
`
`Provisional application No. 60/415,868, filed on Oct.
`3, 2002.
`Int. Cl.
`
`(2006.01)
`GI10H 7/00
`US. C1. ec ecccceccecseeeeeseeteeeeeeeenees 84/608; 700/94
`Field of Classification Search.......... 84/604—609,
`84/623, 626, 660, 661, 693, 697-699; 708/403;
`707/1; 700/94; 705/10; 713/176; 434/307 A
`See application file for complete search history.
`
`Analysis stage
`
`songfile archive
`
`Polyphonic
`
`EX1034
`Roku V. Media Chain
`U.S. Patent No. 9,715,581
`
`

`

`Analysis stage
`
`output data
`
`songfile archive
`
`Polyphonic
`
`analysis machine
`
`analyst wrapper
`
`Figure 1
`
`yuajed“SN
`
`9007‘sz“THE
`gJO[Jooys
`
`7d6LS‘°180°LSN
`
`

`

`U.S. Patent
`
`Jul. 25, 2006
`
`Sheet 2 of 5
`
`US 7,081,579 B2
`
`xrsyyoZOZOQeHtTaeK
`
`
`
`2HSOTASpYe4lTo
`
`IMJZOToTOQeHTACH
`
`
`
`ShsoTASpQU9TTO
`
`
`
`ULVOTASp4WeTTo
`
`TeyZOToToZeHTACH
`
`~ainsiy
`
`e[buts
`
`SPITASPWOTTI
`
`xaq40JoFoReHTAEH
`
`
`
`TH9oTAapQUueTTS
`
`
`
`APAAGYIPAGASUT
`
`
`
`easeqeqepoTsnw
`
`settsfuos9
`
`
`
`uotjyeinbtjuodrzeares
`
`

`

`U.S. Patent
`
`Jul. 25, 2006
`
`Sheet 3 of 5
`
`US 7,081,579 B2
`
`ZeyZoxozoQehtacn
`
`
`
`eHsoTASpyueTTS
`
`xsy_O50XoZeHTAcH
`
`
`
`uesOoTASpyeTTO
`
`z3eazesSTItyTOW
`A9YZOFOZ0zeHTACHAepAIesJTuUCYdATOd
`
`
`SH9OTASPQueTTOcreQeurequT‘HUTd/adob
`
`
`
`
`uoTyeANHtjuce
`
`
`
`SPIOTASPAUSTTO
`
`xeYZOToTOQVeHTACH
`
`
`
`TH90TAapqueTTO
`
`
`
`ABAADSPAOASUT
`
`
`
`SeTTZMros9aseqeqepoTsaw
`
`¢dns
`
`

`

`U.S. Patent
`
`Jul. 25
`
`, 2006
`
`Sheet 4 of 5
`
`US 7,
`
`081,579 B2
`
`pdns]
`
`oO
`
`050
`
`oO
`
`O
`
`LL#49}SN)9HYO
`Oo”%
`
`O
`
`>0
`
`oO
`
`O
`
`O
`
`J#sa\Sny9WY
`
`O09
`
`B38°S8ORRBGo
`
`

`

`Jul. 25
`
`, 2006
`
`Sheet 5 of 5
`
`US 7,
`
`081,579 B2
`
`Cl#wyOOa
`
`OO
`
`LL#40}SNI9WY
`
`e3%O
`
`U.S. Patent Samsiy
`
`Lit2SOPRB0O99
`
`O
`
`OR
`Bae°
`
`

`

`US 7,081,579 B2
`
`1
`METHOD AND SYSTEM FOR MUSIC
`RECOMMENDATION
`
`CROSS REFERENCE TO RELATED
`APPLICATION
`
`This application is based upon and claims benefit of
`copending and co-owned U.S. Provisional Patent Applica-
`tion Ser. No. 60/415,868 entitled “Method and System for
`Music Recommendation”, filed with the U.S. Patent and
`Trademark Office on Oct. 3, 2002 bythe inventors herein,
`the specification of which is incorporated herein by refer-
`ence.
`
`BACKGROUND OF THE INVENTION
`
`1. Field of the Invention
`The invention disclosed herein relates generally to meth-
`ods and systems for analyzing and using digital music
`compositions, and moreparticularly to a method and system
`for determining the characteristics of a musical composition
`by analyzing its digital composition, and recommending
`particular musical compositions to users based upon the
`relative comparability of a user’s desired musical charac-
`teristics and the musical characteristics of a collection of
`
`digital music.
`2. Background of the Invention
`Historically, what is pleasing to the human ear has not
`changed since man began making sounds. Patterns in music
`that are pleasing to the human ear have not changed much,
`if at all, since the timesof the classical composers. What has
`changedare styles, performances, the instruments used, and
`the way music is produced and recorded, but a compelling
`melodyis still compelling and a series of random notesstill
`sounds random. For example,
`the dictionary describes
`melody asa series of notes strung together in a meaningful
`sequence. Unfortunately, some sequences sound meaningful
`and make up a beautiful song and other sequencesjust sound
`like noise.
`
`While the numberof possible melody patterns combined
`with all of the other variables in recorded music allow for a
`seemingly infinite number of combinations, the patterns that
`we find pleasing have not changed. That
`is not to say
`everything has been invented, however. So far, every new
`style of music that has comeinto being: country, rock, punk,
`grunge etc. have all had similar mathematical patterns. The
`hits in those genres have all come from the same ‘hit’
`clusters that exist today and anything that has fallen outside
`of such ‘hit’ clusters has rarely been successfully on the
`charts for its musical qualities.
`
`SUMMARY OF THE INVENTION
`
`It is an object of the present invention to provide a method
`and system for measuring the characteristics of a musical
`composition, and establishing a collection of digital musical
`compositions that may be sorted based upon such charac-
`teristics.
`
`It is another object of the present invention to provide a
`method and system for determining a preferred musical
`characteristic profile for a music listener.
`It is another object of the present invention to enable a
`method and system to compare new digital music files with
`historical commercially successful songs.
`In accordance with the above objects, an artificial intel-
`ligence song/music recommendation system and method is
`providedthat allows music shoppers to discover new music.
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`2
`The system and method accomplish these tasks by analyzing
`a database of music in order to identify key similarities
`between different pieces of music, and then recommends
`pieces of music to a user depending upon their music
`preferences.
`In an alternate embodiment, a system and methodis
`provided to compare specific new songs to a database of
`music to establish a score according to the new song’s
`similarities with current ‘hit’ songs. A user can provide a
`digital music file of an original song that can be analyzed
`and comparedto a ‘hit’ song database. The system provides
`a numerical score depending upon mathematical similarities
`to songs in the current ‘hit’ song database.
`To provide users with music recommendations, the sys-
`tem employs a numberof analysis functions. First, a “Music
`Taste Test” function learns a user’s music preferences via a
`series of binary choice questions, and delivers lists and/or
`personalized song recommendations to the user based on
`this
`information. Recommendations are prioritized and
`listed in order of closest song match on a theoretical multi-
`dimensional grid. Next, a “More Like This” contextual
`matching function delivers song recommendations to the
`user based on a specific song as the defined starting point,
`and delivers songs that are most similar. Lastly, a “My
`Personal Preferences” taste matching function allows the
`user to select and rate songs from the overall database of
`songs in the system, and personalized recommendationsare
`received based on the personalized ratings.
`To provide users with an analysis of an original song, the
`system analyzes a music data file with extra parameters
`relating to commercial success of the music in the database.
`Using this extra dimension, new songs can be compared
`with the database to see how well such new songfits into the
`current market and to identify potential hits.
`
`DESCRIPTION OF THE DRAWINGS
`
`Other objects, features, and advantages of the present
`invention will become more apparent from the following
`detailed description of the preferred embodiments andcer-
`tain modifications thereof when taken together with the
`accompanying drawings, in which:
`FIG. 1 is a schematic view of a system architecture for the
`system of the instant invention.
`FIG. 2 is an exemplary single server configuration using
`the system of the instant invention.
`FIG. 3 is an exemplary multiple server configuration
`using the system of the instant invention.
`FIG.4 is a view of a portion of the universe of analyzed
`songs illustrating somehit clusters.
`FIG. 5 is a view of a portion of the universe of analyzed
`songs shown in FIG. 4, with a collection of new songs
`superimposed thereon.
`
`DETAILED DESCRIPTION OF THE
`PREFERRED EMBODIMENTS
`
`The invention summarized above and defined by the
`enumerated claims may be better understood by referring to
`the following description, which should be read in conjunc-
`tion with the accompanying drawings. This description of an
`embodiment, set out below to enable one to build and use an
`implementation of the invention, is not intended to limit the
`enumerated claims, but
`to serve as particular examples
`thereof. Those skilled in the art should appreciate that they
`may readily use the conception and specific embodiments
`disclosed as a basis for modifying or designing other meth-
`
`

`

`US 7,081,579 B2
`
`3
`ods and systems for carrying out the same purposesof the
`present invention. Those skilled in the art shouldalso realize
`that such equivalent assemblies do not depart from thespirit
`and scope of the invention in its broadest form.
`The method and system of the instant invention enable the
`analysis and processing of digital music in orderto establish
`a description of a numberof characteristics of the music, and
`likewise enable recommendinga collection of music having
`particular characteristics to a user who has established a
`desired musical characteristic profile.
`In an alternate
`embodiment, the method and system of the instant invention
`enable identification of new songs that have similar charac-
`teristics with songs already established as commercial suc-
`cesses.
`
`The first step performed by the system is to analyze an
`existing digital music file in order to create a descriptive
`profile for the musical characteristics of the song. Preferably,
`a database containing a library of digital music is provided.
`Such music database may bea part of the system employing
`the other functions described herein, or may be a remote
`resource that is maintained by a third party digital music
`company through remote, secure access. Access to such a
`third party digital music library may be provided, assigned,
`and protected under terms established by the particular third
`party database provider using known technology.
`Whether provided as an element of the analysis and
`recommendation system or as a third party remote resource,
`the database preferably includesdigital musicfiles formatted
`as uncompressed linear PCM (“Pulse Code Modulation’)
`audio data. If stored in the database in a compressed format,
`the digital music files are decompressed using a standard
`tool for the compression scheme in use. The database may
`be modified, such as by adding additional digital music files
`to the database on a regular, periodic basis, such as weekly
`or monthly,to fit the particular business needs and objectives
`of the database provider. New analysis (as described herein)
`may also be done on a periodic basis similar to database
`updates, such that new music provided to the database may
`be included in the music recommendation functions
`described herein. Any newly generated analysis data from
`newly added digital music files is simply added at the end of
`the analysis data file, which is described in greater detail
`below.
`
`FIG. 1 depicts a schematic view of a system architecture
`for enabling the transfer and processing of digital music files
`to an automated digital music file analysis tool in order to
`generate an output file that, as described in greater detail
`below, serves as a descriptor of the musical characteristics of
`the particular musical composition that was analyzed. As
`shown in FIG.1, an “analyst wrapper” script receives digital
`music files from the music database or a songfile archive,
`preferably copies the particular digital music file to a loca-
`tion on a computer network on which the analysis system is
`stored, decompresses the digital music file when necessary,
`and passes the linear PCM file to the analysis engine for
`processing. (In FIG. 1, the analysis engine is labeled “HMI
`analyst” for Human Media Interface.) The output of such
`analysis engineis directed to an outputtext file that, in turn,
`may be used by the recommendation utilities described
`below.
`
`During the initial analysis performed by the system, all
`songs that are to be analyzed are processed in the same way,
`in series. The particular list of songs to be processed may
`vary depending upon the application. For instance, for a
`particular retailer, only their music catalogue is analyzed.
`For a generalized music recommendation system, all songs
`available in the digital music database are analyzed. The
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`4
`purposeofthe initial analysis performed by the system is to
`analyze a variety of physical parameters of the music stored
`in the target digital music database. Such physical param-
`eters describe quantifiable characteristics of music that may
`be mathematically modeled to create a descriptive, elec-
`tronic “footprint” for each song. Moreover, the analyzed
`parameters are based on humanperception, and the system
`is referred to as a Human Media Interface (HMI) system.
`The characteristics have been identified to be the ones that
`
`produced the strongest reaction in testers. Often the listener
`detects the characteristics unconsciously. In general, the mix
`of parameters is more important than any individual param-
`eter. To implement the methods described herein, the system
`particularly analyzes one or more of the following charac-
`teristics for each musical composition: brightness, band-
`width, volume, tempo, rhythm,low frequency, noise, octave,
`and how these characteristics change over time, as well as
`length of the audio data. Not all of the characteristics
`necessarily provide distinctions in the music. Combinations
`of someor all of these characteristics may be employed
`without departing from the spirit and scope of the instant
`invention.
`
`the
`In order to measure each of these characteristics,
`digital music file is divided into “chunks” which are sepa-
`rately processed in order to measure the characteristics for
`each such “chunk.” “Chunk”size is fixed and selected for
`
`optimizing performance over a test sample of songs so as to
`provide an appropriately representative sample for each
`parameter of interest. Once the data from the digital music
`file is divided into such “chunks,” the value for each
`parameter in each chunk is measured. Parameters are mea-
`sured overall “chunks” and averaged. The values that are
`derived through such measurements are, as described in
`greater detail below, used, in turn, to establish mean values
`and standard deviations among the parameter data so as to
`develop a profile for the entire songfile.
`In order to measure the parameters noted above, the data
`from the digital music file is decomposed using Fast Fourier
`Transform (FFT)
`techniques. The Fourier transform,
`in
`essence, decomposes or separates a waveform or function
`into FFT coefficients of different frequency that sum to the
`original waveform. The Fourier transform identifies or dis-
`tinguishes the different frequency FFT coefficients and their
`respective amplitudes. Specific details concerning the appli-
`cation of FFT techniques can be found in P. Duhamel and M.
`Vetterli, Fast Fourier Transforms: A Tutorial Review And A
`State Of The Art, Chapter 19: Signal Processing, pp.
`259-299 (Elsevier Science Publishers B.V. 1990), which is
`incorporated herein by reference thereto.
`Using such FFT techniques, each digital music file is
`decomposed into a defined set of FFT coefficients, which are
`grouped and separated in order to determine the different
`parameters to be calculated for each song. To accomplish
`this task, raw data, in the form of the digital music file in
`PCMformat, is directed to a script that decompressesthefile
`(where necessary), divides the data into “chunks,” and
`applies FFT techniques to the data in each “chunk”to, in
`turn, establish the FFT coefficients for each “chunk.” Once
`those coefficients are established for each “chunk,” particu-
`lar coefficients are chosen in order to calculate a numerical
`
`value for each of the parameters of interest in each “chunk.”
`Moreparticularly, the specific parameters for each chunk are
`calculated, using the FFT coefficients, based on the follow-
`ing principles:
`
`

`

`US 7,081,579 B2
`
`5
`Brightness is quantified as a spectral histogram built from
`the FFT.
`
`Bandwidth is quantified as the variance of the spectral
`histogram built from the FFT.
`Tempo is quantified as a measure obtained using a “Beat
`Tracker” algorithm (based upon autocorrelation val-
`ues). The Beat tracker algorithm calculates how many
`beats per second are in a “chunk.”
`Volume is quantified as the average change in the bit
`sample amplitudes at lag 1. Lag 1 refers to consecutive
`beats.
`Low Frequency is quantified as spectral power weighted
`with ten inverse frequency.
`Noise is quantified as the sum of all the FFT coefficients
`normalized to the maximum FFT coefficient.
`Octave is quantified as the temporal mean of Cepstrum
`(inverse of the logarithmic FFT). The Octave is calcu-
`lated for each chunk and averaged.
`File Size is quantified as the numberof bytes ofthefile.
`
`In other words, FFT coefficients provide data needed to
`calculate all the parameters listed, exceptfor file size. Thus,
`the result of the analysis of the data from the digital music
`file is several parameter values based on the number of
`particular “chunks.” In order to derive a final parameter
`value for the song as a whole, the mean is taken from all
`“chunks” within the same parameter.
`Compiling each value for all the parameters measured
`results in an output that is referred to herein as a “parameter
`vector,” which parameter vector is formatted as a text file
`(with values
`separated by semicolons). An exemplary
`parameter vector may appear as follows:
`78221904926; 1; 01; 41.018945; 26.088065; 38.450639;
`23.616234;
`5337.3284;
`266.848675;
`118.7692;
`76.484095; 116.6154; 73.836892; 0.00007
`Note: The actual ordering of values in the set is immate-
`rial so long as it is known and consistent.
`
`The bold section of the sample parameter vector noted above
`is an example of song identification information, including
`the UPC, disk number, and track number. The other data is
`sample parameter value information (e.g., tempo, volume,
`etc.). Based upon the particular database, other generally
`available information may optionally be provided and may
`be useful in the music recommendation functions described
`in greater detail below. Such other information may include,
`by way of example: (i) retailer/store-specific media identi-
`fication; (41) set count (number of disks in set); (111) album
`title; (iv) artist name; (v) music genre; (vi) record label: (vii)
`track title; (viii) track artist; (ix) track duration; and (x)
`copyright date.
`The system will recognize almost all values for any
`parameter as valid, except negative values. If negative
`values are found,
`it means that the digital music file is
`invalid, and the data needs to be manually discarded. Fur-
`ther, if values are consistently too high, there is the possi-
`bility of damaged data or data stemming from an invalidfile.
`Negative and “too high” parameters correspondto out layers
`in the standardized distribution of frequencies of each
`parameter. The cut-offs are defined over a numberofstan-
`dard deviations calculated in a test sample of songs. The
`analysis tool analyzes files assuming that they are standard
`digital music files and assumes all data is valid (unless
`negative values are found). Such digital music files are
`typically identified with the filename extension.wav(.) The
`analysis tool reads audio waves of a song and performs
`algorithmic analyses of certain subjective criteria,
`as
`described above.
`
`10
`
`15
`
`20
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`6
`Once the parameter vector for the entire song has been
`established as set forth above, standard mathematical analy-
`sis is used to extract samples of the most representative
`passages of the music. More particularly, the analysis tool
`divides the entire song into small sections (on the order of
`several seconds, depending upon the precision required),
`and calculates all of the parameter values for each of those
`sections using the same FFT analysis methods set forth
`above. The resulting parameter vectors thus represent the
`instantaneous value for all of the parametersat the particular
`song position specified by each section. When such section
`processing is completed, a parameter vector exists for each
`section.
`Following section processing, the system then proceedsto
`find the most representative passage (whose length varies
`depending upon the application) of the song. First, each
`parameter vector for each section is compared to the param-
`eter vector derived from the analysis of the entire song. The
`closest section’s parameter vector is chosen as the most
`representative part of the song. “Closest” is defined in terms
`of unweighted quadratic differences between each section
`and the whole song. The parameters are already “‘equalized”
`by the standardization procedure. Once the most represen-
`tative part of the song is determined, the system produces a
`song clip that includes that most representative part. In order
`to create a clip of a particular desired length (X), the system
`subtracts X/2 seconds from the position of the most repre-
`sentative section’s position in orderto obtain a clip of length
`X having the most representative part in the middle of the
`clip.
`The output from the analysis process described above for
`the complete song is stored in a plain text format,
`tab-
`delimited. The next step following analysis and profiling of
`the library of songs is to provide a recommendation engine
`to enable the system to recommendsongsto particular users
`based upon the musical characteristics of the songs as
`depicted by their parameter vectors. It should be noted that
`such recommendation functionality may be provided as an
`“in store” application running on the same server as the
`analysis component; on a server at the same location as the
`database of song parameter vectors, as illustrated in FIG. 2;
`on a remote server in communication with an application
`server at a retail location across a TCP/IP connection, as
`illustrated in FIG. 3; or the like. In each instance,
`the
`recommendation engine loads the parameter vectorfile, and
`is configured to particularly deal with the text format of the
`parameter vector file. The text data of the parameter vector
`file may be loaded during the recommendation engine
`startup by specifying so on the commandline, or a command
`may besent after startup to locate and load the data.
`Following the loading of the text data of the parameter
`vector to the recommendation engine, various mathematical
`and statistical procedures (discussed in detail below) are run
`on the loaded data to ensure that all of the data is meaning-
`ful, and to extract the essential characteristics from each
`song and its preferences profile. Such preferences profile is
`the parameter vector described above, after normalization
`has been performed based on the whole song analysis
`database (and is generated and formatted in the same man-
`ner).
`Somestatistical methods are used to improve the quality
`of the analysis information and subsequent Artificial Intel-
`ligence techniques. Data is normalized and centered by the
`system using Zero Mean Data and Standard Deviation Equal
`to Variance Equal to 1 for each parameter separately. More
`particularly, in applying Zero Mean Data, the mean value for
`each of the parameters is calculated across the entire song
`
`

`

`US 7,081,579 B2
`
`7
`database, and the individual parameters of individual songs
`are then centered using those means. In other words, the
`parameter values for each song are changed by subtracting
`the calculated mean for each parameter by the original
`parameter value of the song. Thus, if the mean wasrecal-
`culated with these new values, the mean value would be
`zero. Further, standard deviation and varianceare correlated,
`as variance equals the square of the standard deviation. In
`applying Standard Deviation Equal to Variance Equalto 1,
`the data is normalized using standard deviation equal to 1.
`To do this, the system calculates the standard deviation and
`the variance for the set, and then, changing all of the values
`for the set (dividing the Zero Mean Data already calculated
`by the variance ofthe set) in order to get standard deviation
`equal to 1 if it was recalculated using these new values.
`With centered and normalized data, correlations between
`all analysis parameters are identified and eliminated to
`reduce redundant information given by different parameters
`and getting the most unique information from each param-
`eter. To do this, the system creates a new set of parameters
`with no correlations, but with all the information included in
`the parameters analyzed by the analysis tool. These new
`parameters have no conceptual meaning (ie.,
`they are
`referred to as parameterl, parameter2, etc., instead of “vol-
`ume,” “tempo,”etc.).
`Data correlations are determined using a standard corre-
`lation matrix. The standard correlation matrix formula is as
`follows:
`
`1 M
`Vi = ad, BOOZ)
`cl
`
`V,, 18 the value of the intersection of column i and row j
`M is the number of songs in the database
`c is each particular song
`Z, and Z, are parameter number i and j respectively for
`each song ¢
`In essence, the correlation matrix multiplies, for each pair
`of parameters, all values for all registers of data, adding
`them, and then dividing by the total numberofregisters.
`The correlation matrix is then used to calculate Eigen-
`vectors and Eigenvalues. As the correlation matrix is sym-
`metric,
`standard Jacobi
`transformations
`(of symmetric
`matrices) are used to compute all Eigenvectors and Eigen-
`values. Eigenvectors and Eigenvaluesare standardstatistical
`formulae used to determine and eliminate correlations
`
`between parameters, thereby reducing repeated information
`and increasing data utility.
`Eigenvectors are important in determining the new uncor-
`related parameters from the original parameters, as illus-
`trated in the following formula:
`
`[Pi
`
`Po
`
`++ Py]=[OQ, O2
`
`+++ On]x
`
`Ey Ey2
`Ex, En.
`
`+ Ein
`+++ Eon
`
`Ey; Enz
`
`++ Ewn
`
`P, (where i goes from 1 to N) are the new parameter values
`for each song
`N is the number of parameters, and also, the number of
`eigenvectors
`
`to N) are the values for
`
`8
`O, (where i goes from 1 to N) are the original parameter
`values for each song
`E,, (where i and j go from 1
`parameter j of Eigenvector i
`Eigenvalues give the amountof uncorrelated information
`contained in each parameter. Thus, these values are used to
`weight
`the importance of parameters (or even discard
`parameters that contain no or almost no information).
`Depending on precision and speed desired, weighting and
`discarding rules are set for each customer. Weights for
`various parameters are session-dependent and user-depen-
`dent. For example,
`in a three parameters example,
`the
`parameters P,, P, and P; contain 75%, 23%, and 2% ofthe
`total amountof information, respectively. Depending on the
`specific customer needs (speed, accuracy, simplicity, etc.)
`the following weighting rules can be applied:
`Keep all parameters equally weighted (33.33%, 33.33%,
`and 33.33%)
`Weight all parameters based on the percentages above
`(75%, 23%, and 2%)
`Discard the last parameter and weight the remaining two
`as P, 77% and P, 23%
`Discard the last parameter and equally weight the remain-
`ing two (50%, 50%)
`Ete.
`
`The parameters are now uncorrelated, normalized, and
`orthogonal and weights have been set for each one, so the
`system can get maximum value from them. These new
`parameters replace, in the system, the old ones that came
`from the music analysis tool.
`Standard clustering algorithms are then employed by the
`system to locate and determine clusters within the entire
`database for further usage (specific to music recommenda-
`tion functions “More Like This” and “My Personal Prefer-
`ences,” described in greater detail below).
`After a large selection of music has been analyzed and the
`data has been uncorrelated, normalized, orthogonal, and
`weights are set, the system may provide a variety of music
`recommendation functions through interaction with a user.
`There are two keys ways to recommend music to an indi-
`vidual user. One methodis to capture a user’s own personal
`taste profile by enabling them to take a “musictaste test”. In
`this process, the user is presented with a number of binary
`choices between two short audio clips and chooses the clip
`they prefer. After a series of comparisons, it is possible to
`generate a profile for that user; the profile is analogous to a
`song’s own profile, as measured in the analysis phase. In this
`way, songs from the database that share commonalities to
`the userprofile can be identified and presented to the user to
`preview. The second way simply links a song or a group of
`songs to a selection of music that has a similar profile. This
`technique takes the individual profile of the song or songs
`and matches it to the whole catalogue of music in the
`database.
`In order to initiate the music recommendation features, a
`user interacts with a computer, website, interactive kiosk,
`in-store listening station (like those commercially available
`from ACD, Fullplay, Black Box AV, First Universal, and
`RedDotNet), or other computer-like device in which the
`recommendation functions are loaded. In FIGS. 2 and 3, user
`interaction takes place at one ofthe client devices, as shown.
`The recommendation software can be loaded directly to the
`computer/device/server, as shown in the single server con-
`figuration of FIG. 2. In other embodiments, the recommen-
`dation software can be served remotely through a LAN,
`WAN,or via an ASP environment, as shown in the multiple
`server configuration of FIG.3.
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`

`

`US 7,081,579 B2
`
`9
`Irrespective of the locality of the software, the music
`recommendation functions are preferably executed as a
`series of requests and responses handled by a daemon (or
`service) on a server. The daemonlistens for requests to its
`network address on a specific port, and communicates with
`client devices using its own protocol, referred to herein as
`Simple HMI-Music Protocol (SHMP). SHMPis a protocol
`designed to satisfy the functionality given by the system
`over any TCP/IP network, and is used to send and receive a
`A user’s profile is stored in the system while the user is
`variety of requests, including, but not exclusive to, “Music
`still connected to the system. When the session is termi-
`Taste Test Request” and “More Like This Song,” as
`nated, the user profile is eliminated. However, this profile
`described in greater detail below.
`can also be saved within the system or external database and
`A “Music Taste Test” (“MTT”) is used to determine a
`accessed at a later date, depending upon the specific retail
`user’s music preferences, and thereby make personalized
`customerand implementation environment. Thus, a user can
`song recommendations. The MTT function is implemented
`go back, continue the MTT process, and obtain refined
`through a pattern-recognizing artificial intelligence system
`recommendations. This functionality is achieved using an
`using an MTT daemon (labeled ‘mttd’ in FIGS. 2 and 3). The
`identification technologysuch as loyalty cards or cookies, as
`MTT function creates simple questions and instructions that
`descri