US007650570B2
`
`(12) United States Patent
`US 7,650,570 B2
`(10) Patent No.:
`Torrens et al.
`(45) Date of Patent:
`Jan. 19, 2010
`
`(54)
`
`(75)
`
`METHODS AND APPARATUS FOR
`VISUALIZING A MUSIC LIBRARY
`
`Inventors: Marc Torrens, Corvallis, OR (US);
`Patrick Hertzog, Lausanne (CH);
`Josep-Lluis Arcos, Bellaterra (ES)
`
`(73)
`
`Assignee: Strands, Inc., Corvallis, OR (US)
`
`(*)
`
`Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 671 days.
`
`6,434,621 B1
`6,438,579 B1
`6,487,539 B1
`6,526,411 B1
`6,532,469 B1
`
`8/2002 Pezzillo
`8/2002 Hosken
`11/2002 Aggarwal
`2/2003 Ward
`3/2003 Feldman
`
`(Continued)
`FOREIGN PATENT DOCUMENTS
`
`EP
`
`1 231 788
`
`8/2002
`
`(Continued)
`OTHER PUBLICATIONS
`
`International Search Authority/US; PCT Search Report; Date Mar.
`25, 2008; 3 Pages.
`
`(Continued)
`
`Primary ExamineriBa Huynh
`Assistant ExamineriPhenuel S Salomon
`
`(74) Attorney, Agent, or Firm7Stolowitz Ford Cowger LLP
`
`(57)
`
`ABSTRACT
`
`Visualizing and exploring a music library using metadata,
`such as genre, sub-genre, artist, and year, is provided. Geo-
`metric shapes, such as disks or rectangles, may be divided
`into sectors representing genre and each sector may be further
`divided into sub-sectors representing artists associated with
`each genre. The sector’s relative size generally reflects the
`importance of the corresponding genre within the library.
`Likewise, the sub-sector’s relative size generally reflects the
`importance ofthe corresponding artist within the genre which
`may be determined by the number ofmedia items ofthe artist.
`Marks representing each media item may be arranged and
`displayed within the geometric shape to reflect the mark’s
`corresponding genre, artist, and year. In addition, each mark
`may reflect an attribute, such as playcount, of the media item
`and each sector may reflect the mean value of an attribute of
`all media items within the sector.
`
`45 Claims, 10 Drawing Sheets
`
`194
`
`
`
`
`
`
`ldon'twant a lover
`Texas
`
`180
`
` (1959)
`
`
`Southside
`184
`182—]
`
`Apple Exhibit 4225
`
`Apple V. SightSound Technologies
`CBM2013-00020
`
`Page 00001
`
`
`70
`110
`
`E1
`
`110
`162‘
`2) Bruce Sprmgslsen
`160— E3 180‘s E 70's Mule
`156‘t) 3) 25 mt played 09'“
`158
`152m 4) Jogging playl)sl(
`154
`1544:) 5) REM. after 1990
`158“): 6) 80's pug muslc
`\110
`156+): 7) Romng smnas a
`Beatles
`
`
`
`—134
`
`132 \- G) Playcounk/
`‘0 Myra!in9./‘136
`o Added—/’
`0 Last Mayan/:1l 23
`
`
`
`
`(21)
`
`Appl. No.: 11/543,730
`
`(22)
`
`Filed:
`
`Oct. 4, 2006
`
`(65)
`
`(60)
`
`(51)
`
`(52)
`(58)
`
`(56)
`
`Prior Publication Data
`
`US 2007/0233726 A1
`
`Oct. 4, 2007
`
`Related US. Application Data
`
`Provisional application No. 60/723,865, filed on Oct.
`4, 2005.
`
`Int. Cl.
`
`(2006.01)
`G06F 3/16
`US. Cl.
`........................ 715/727; 715/728; 715/730
`Field of Classification Search ................. 715/727,
`715/728, 730
`See application file for complete search history.
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`5,583,763
`5,918,014
`6,000,044
`6,047,311
`6,134,532
`6,346,951
`6,381,575
`
`ww>>>>>,_.,_.
`
`12/1996
`6/1999
`12/1999
`4/2000
`10/2000
`2/2002
`4/2002
`
`Atcheson
`Robinson
`Chrysos
`Ueno
`Lazarus
`Mastronardi
`Martin
`
`114
`
`
`
`
`
`
`
`70
`
`
`50
`
`
`
`186
`
`Apple Exhibit 4225
`Apple v. SightSound Technologies
`CBM2013-00020
`Page 00001
`
`

`

`US 7,650,570 B2
`
`Page 2
`
`US. PATENT DOCUMENTS
`
`6,615,208 B1
`6,687,696 B2
`6,785,688 B2
`6,842,761 B2
`6,850,252 B1
`6,914,891 B2
`693L454 B2
`6938309 B2
`6,947,922 B1
`6,987,221 B2
`6,990,497 B2
`6993532 B1
`770205637 B2
`7,021,836 B2
`7:072:846 B1
`7’082’407 B1
`7,096,234 B2
`7’111’240 B2
`7’120’619 B2
`7’180’473 B2
`7,256,341 B2
`7’457’862 B2
`7,493,572 B2
`2002/0082901 A1
`2003/0120630 A1
`2003/0229537 A1
`2004/0003392 A1
`2004/0073924 A1
`
`2004/0139064 A1
`2004/0263337 A1
`2005/0060350 A1
`2005/0102610 A1
`2005/0193014 A1
`2005/0198075 A1
`2005/0203807 A1
`2005/0210101 A1
`2005/0216855 A1
`2005/0235811 A1
`2005/0276570 A1
`2006/0015904 A1
`2006/0018208 A1
`2006/0018209 A1
`2006/0020062 A1
`2006/0026263 A1
`2006/0053077 A1
`2006/0062094 A1
`2006/00747 50 A1
`2006/0095516 A1
`2006/0100978 A1
`2006/0112098 A1
`2006/0165571 A1
`2006/0173910 A1
`2006/0173916 A1
`2006/0195438 A1
`2006/0195462 A1
`2006/0195789 A1
`2006/0206811 A1
`2006/0253874 A1
`2006/0288044 A1
`2007/0043829 A1
`2007/0203790 A1
`2007/0250761 A1
`2007/0294096 A1
`
`9/2003 Behrens
`2/2004 Hofmann
`8/2004 Abajian
`1/2005 Diamond
`2/2005 Hoffberg
`7/2005 Ha
`8/2005 Deshpande
`8/2005 Ogawa et 3L --------------- 715/727
`9/2005 Glance
`1/2006 Platt ............................ 84/601
`“2006 O’Rourke
`“2006 Platt
`3/2006 Bratton
`4/2006 Bratton
`7/2006 Robinson
`7/2006 Bezos
`8/2006 Plastina
`9/2006 Crow et al.
`“22006 Drucker
`2 2007 Horie
`8/2007 Plastina
`“/2008 Hepworth
`2/2009 Card
`.
`6/2002 Dunning et a1.
`6/2003 Tunkelang
`.
`12/2003 Dunning
`.
`.
`1/2004 TraJkov1c
`4/2004 Pendakur
`.
`7/2004 Chevallier
`12/2004 Terauchi et ai.
`3/2005 Baum
`5/2005 Jie
`9/2005 Prince
`9/2005 Plastina et a1.
`9/2005 Bezos et al.
`9/2005 Janik
`9/2005 Kopra
`10/2005 Dukane
`12/2005 Reed
`1/2006 Marcus
`1/2006 Nathan
`1/2006 Drakoulis
`1/2006 Robinson
`2/2006 Raghavan
`3/2006 Mourad
`3/2006 Nathan
`4/2006 Clark
`5/2006 Wijeratne
`5/2006 Heller
`5/2006 Renshaw
`7/2006 Seon
`8/2006 McLaughlin
`8/2006 Verbeck
`8/2006 Galuten ......................... 707/4
`8/2006 Rogers
`8/2006 Rogers et al.
`9/2006 Dowdy
`11/2006 Stark
`12/2006 Kashiwagi
`2/2007 Dua
`8/2007 Torrens
`10/2007 Bradley
`12/2007 Randall
`
`............... 715/727
`
`FOREIGN PATENT DOCUMENTS
`
`EP
`JP
`JP
`JP
`KR
`
`1420388
`11-052965
`2002-108351
`2003-255958
`2002-025579
`
`5/2004
`2/1999
`4/2002
`9/2003
`4/2002
`
`WO
`WO
`wo
`wo
`W0
`
`WO2004070538
`WO2006052837
`WO2007134193
`WO2007075622
`WO2007092053
`
`8/2004
`5/2006
`5/2007
`7/2007
`8/2007
`
`OTHER PUBLICATIONS
`Cano, Pedro et al.,On the Use of FastMap for Audio Retrieval and
`Browsing, The International Conferences on Music Information
`Retrieval and Related Activities (ISMIR 2002), Paris, France, Oct.
`2002, 2 pages.
`Connell, Iain et al., Ontological Sketch Models: Highlighting User-
`System Misfits, In P. Palanque, E. O’Neill and P. Johnson, editors,
`Proceedings of Human Computer Interaction (HCI), Bath, England,
`Sep. 2003, London Springer, pp. 1-16.
`The Trustees of Indiana University, Variations2, The Indiana Univer-
`sity Digital Music Library, http://dml.indiana.edu/, last updated May
`11 2005 1 page
`Logan, Beth, Content-Based Playlist Generation: Exploratory
`Experiments, The International Conferences on Music Information
`Retrieval and Related Activities (ISMIR 2002), Paris, France, Oct.
`2002 2 pages
`Logan, Beth et al., A Music Similarity Function Based on Signal
`.
`.
`.
`.
`Analys1s, IEEE International Conference on Multimedia and Expo
`(ICME), Tokyo, Japan, Aug. 2001, IEEE Press, pp. 952-955.
`.
`.
`~~
`.
`.
`Maidin, Donncha O etal., The Best of Two Wolds: RetrieVing and
`.
`.
`.
`.
`.
`Brows1ng, Proceedings ofthe Cost G-6 Conference on Digital Audio
`Effects (DAFX-OO), Verona, Italy, Dec. 7-9, 2000, 4 pages.
`.
`.
`.
`.
`.
`Notess, Mark et al., Variations2: Toward Visual Interfaces for Digital
`M . L'b
`.
`S
`dI
`.
`1W k h
`V'
`11
`f
`11133.”. :1 3:332 ecggozntgrnationa
`or S 01’ 0“ ““3 mer aces
`lglltk Ei mm; C ’ pfes'd o
`.
`.
`dV.
`1.
`.
`if
`ampa .
`,
`ias-et
`.,
`ontent- ase . rganization an
`isua ization
`of Mus1c Archives, ACM Multimedia, Juan les Pins, France, Dec.
`2002’ PP' 570'579'
`_
`_
`_
`Pauws, Steffen et al., Pats: Realization and User Evaluation of an
`Automatic Playlist Generator, The International Conferences on
`Music Information Retrieval and Related ActiVities (ISMIR 2002),
`Pans, Francea 0“ 2002’ 9 Pages
`_
`_
`Rauber, Andreas et al., The SOM-enhanced JukeBox: Organization
`and Visualization of Mus1c Collections Based on Perceptual Models,
`Journal ofNew Mus1c Research, vol, 32, No. 2, 2003, pp. 193-210.
`Shneiderman, Ben, Tree Visualization with Tree-Maps: 2-d Space-
`FillingApproach, ACM Transactions on Graphics, vol, 11,No, 1, Jan.
`1992,1311 92-99.
`Treemap, University of Maryland, http://www.cs.umd.edu/hci1/
`treemaph last updated AUg. 5, 2003, 4 pages.
`Shneiderman, Ben, Treemaps for Space-ConstrainedVisualization of
`Hierarchies,
`http://www.cs.umd.edu/hcil/treemap-history/,
`last
`updated Apr. 28, 2006, 16 pages.
`Tzanetakis, George, et al., MARSYAS3D: A Prototype Audio
`Browser-Editor Using a Large Scale Immersive Visual and Audio
`Display, Proceedings of the 2001 International Conference on Audi-
`tory Display, Espoo, Finland, Jul/Aug. 2001, 5 pages.
`N. A. Lazar; Bayesian Empirical Likelihood; Technical Report,
`Carnegi Mellon University, Department of Statistics, 2000; 26 pages.
`S. Baluja, R. Seth, D. Sivakumar, Y. Jing, J. Yagnik, S. Kumar, D.
`Ravichandran, and M. Aly, “Video Suggestion and Discovery for
`YouTube: Taking Random Walks Through the View Graph”. In
`WWW ’08: Proceedings of the 17th international conference on
`World Wide Web, pp. 895-904, Beijing, China, 2008. ACM Press.
`A. Das, M. Datar, A. Garg, and S. Rajaram. “Google News Person-
`alization: Scalable Online Collaborative Filtering”. In WWW’07:
`Proceedings of the 16th international conference on World Wide
`Web, pp. 271-280, NewYork, NY, USA, 2007. ACM Press.
`J. Dean and S. Ghemawat, “MapReduce: Simplied Data Processing
`on Large Clusters”. Commun. ACM, 51(1):107-113, 2008.
`Y. Dempster, N. Laird, and D. Rubin. “Maximum Likelihood from
`Incomplete Data viathe EMAlgorithm”. Jour. ofthe Royal Stat. Soc.,
`Ser. B., 39:1047-1053, 1977.
`T. Hofmann. “Unsupervised Learning by Probabilistic Latent
`Semantic Analysis”. Mach. Learn., 42:177-196, 2001.
`
`Page 00002
`
`Page 00002
`
`

`

`US 7,650,570 B2
`Page 3
`
`T. Hofmann. “Latent Semantic Models for Collaborative Filtering”.
`ACM Transactions on Information Systems, 22:89-115, 2004.
`P. Indyk and J. Matousek. “Low-Distortion Embeddings of Finite
`Metric Spaces”. In Handbook of Discrete and Computational Geom-
`etry, pp. 177-196. CRC Press, 2004.
`I. Scihira. “A Characterization of Singular Graphs”. Electronic Jour-
`nal of Linear Algebra, 16:451-462, 2007.
`Alvear, Jose, “Risk-Free Trial Streaming Media Delivery Tools,”
`Streaming
`Media.com;
`WWW.streamingmedia.com/article.
`ap?id:5768, Jun. 30, 2000.
`Deshpande, Mukund, et al., “Item-Based Top-N Recommendation
`Algoriths,” ACM Transactions on Information Systems, 22:1 (Jan.
`2004), pp. 143-177.
`Pachet, Francois, A Taxonomy of Musical Genres, Content-Based
`Multimedia Information Access Conference (RIAO), Paris, Apr.
`2000, 8 pages. Not Submitted in IDS.
`Smart Computing, “The Scoop on File-Sharing Services,” Dec. 2000,
`vol. 11, Issue 12; pp. 30-33 in printed issue. Available at W.
`smartcomputing.com/editorial/article.
`asp?article:articles%F2000%Fsl l l2%2F08sl2.asp.
`
`wwwaxcessnews.com/modules/Wfsection/article.
`php?articleid:8327, Web Page, Feb. 24, 2006, Maintenance Fees,
`Digital Music Sales Triple to $1.1 Billion in 2005.
`Page,
`Web
`wwwbmi.com/news/200403/20040324b.asp,
`BMITMFigures Don’t Lie, Mar. 24, 2004, Touch Tunes Signs License
`Agreement for BMI Music in Digital Jukeboxes.
`“New Music Recommendation System is Based on FOAF Personal
`Profiling,”
`www.masternewmedia.org/musicirecommendation/
`musicirecommendationisystemiFOAF, Oct. 1, 2005.
`“Social Networking Meets Music Listening: Mecora Launches
`Radio
`2.0,”
`www.masternewmedia.org/news/2006/04/ 13/
`socialinetworkingimeetsimusicilistening.htrn, Apr. 13, 2006.
`PCT/US2006/0342 l 8; International Search Authority; PCT Interna-
`tional Search Report; Feb. 9, 2007; 3 pages.
`PCT/US06/48330; International Bureau; PCT Search Report and
`Written Opinion; Mar. 20, 2008; 10 pages.
`PCT/US2006/003795;
`International Search Report and Written
`Opinion of International Application; May 28, 2008.
`
`* cited by examiner
`
`Page 00003
`
`Page 00003
`
`

`

`U.S. Patent
`
`Jan. 19, 2010
`
`Sheet 1 of 10
`
`US 7,650,570 B2
`
`
`
`
`
`or
`
`rmmmg9:OOP\AEEEVm,
`.0;
`
`
`
`
`
`Alvm
`
`m;,mwcomEfi333%“.
`
`
`E52mc
`goggu
`«EmzwccmwN\0Van“Wm.mfim
`
`
`
`
`
`313mm»wwESO5:39:
`
`Encaccau
`
`mucmflxmuwcbuvmfi
`
`v28
`
`muowmcom@mommou
`
`
`
`@52333mm
`
`2:98qu
`
`>brzou
`
`“EEK>znmvwmé\U85.ash22flmm\5%youHmm4
`
` mcofiummmu$83020mmE35%aCéa“Ema
`
`“82mmQ8.fl
`
`
`
`i:§<33q,69;SEm“
`
`E.30»Ema,0»60..<E.“m.
`
`33%mm=Am.3amop36m
`
`
`
`
`
`
`
`:9;3052?..952EU_Emr
`CaSen/Eu3o_9>mmmmwwmKm.\3ocxm37N3;\mCMFGmCMm1m
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`\mfiwq,wBuooumm,\ammo,53:1;
`
`36:)ng.I
`
`EwwmEavg
`
`
`
`Eccwaxtmuxu:5mm5>933%on
`
`€355,051figmsmw
`
`530m3:51i
`
`mEuummm
`
`59%.4
`
`swamE33,
`
`EumwE>93"
`
`:Jmmc133V38.3:
`
`53mm5>23,>mnI
`
`
`
`mm>>0gmNWsumem‘|\\$9M”?!mmmm®®0
`
` :3ng635mG:3mmE‘63_.3“32>305xH.20030>mmGuamvmfiflmm
`
`
`
`
`
`
`,Eum.mxtofiaw.ECESE,653m5353..>mummoxmwfimwumwxwmmuwimm:no;E$33,395:me,memimm:zmm$1.0m.“$2on9:mMmmnm30>Em;0%.08<E;m4SEEmoEF_
`
`
`
`
`:mmsmlcm>fainmAHcflmm5>3,",maxzdwmSmE4WVmnwmuom
`$6»meNmmwmcfinuozgvim.)3:9quo>mus/f“8&me
`
`
`
`
`
`.5951JEumm".3>9...>mn0152:5me0M33m“:E>mbm_QIW55d.2193meQ:amm5S3>350:uwmcmgu“3.33,m.
`
`
`mEma:8,5)*0umwm2.:va0x93ww>uwc$x<2anMW
`,
`nE332"“iii/N;Ecww830w.
`
`362:4Q:43?,chmimg
`
`Vn%tim23%:an3“W38E@QEU%
`
`
`
`Page 00004
`
`Page 00004
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`

`

`U.S. Patent
`
`Jan. 19, 2010
`
`Sheet 2 of 10
`
`US 7,650,570 B2
`
`#2,
`
`o:owe
`
`323%.@5323mm?5ESEaaWWW
`
`:mmmmgam33m8Nmro:232MEw£2oar
`
`
` wmzmmmO:\awmcgmacacmANGrimm:
`.er325%SMEwar
`/Vu_m:EEaMowAm
`
`mH:
`
`
`
`mm”\bgma530
`
`mmrxczfigOfl/
`vmwaxcsoimi®‘7!Nmr
`
`
`all
`
`00¢v:
`
`Page 00005
`
`Page 00005
`
`
`
`
`

`

`U.S. Patent
`
`Jan. 19, 2010
`
`Sheet 3 of 10
`
`US 7,650,570 132
`
`82m0wrem:NEayou
`i“r\
`3.6%g£onEEEOrell/rageQ8pom:
`vmklrge5%sax>33mm?:2mg2
`Nma3(15853mya8a
`A\lcmmmmgam
`:8ng®
`
`V
`
` .23363o
`
`
`
`\lmém.EO
`
`Page 00006
`
`Page 00006
`
`
`

`

`U.S. Patent
`
`Lu
`
`f04
`
`nUa
`
`GABnUHI5,
`
`0//Vo_m=Ea8MowAmwwwmwe.825%02mma3:ONV
`
`won
`
`
`
`
`90:38:Om:NmrJoionr
`9:32..3NmrOW\4/9,(\E5%5288@mrN:1”\tcmemmgaw82mMNNmrM0:
`2wmjvymzxflma
`SNNwit
`cm85%o:My385E?C9:on\
`
`
`
`
`
`X:\
`
`noooooooo:
`
`55¢530
`
`.EEEOweiv4/N55935®om
`
`7,mm?
`
`
`S/4Um«4,4mmmcomma08'
`
`4..va?avmr
`
`
`
`Page 00007
`
`Page 00007
`
`
`
`
`

`

`U.S. Patent
`
`Jan. 19, 2010
`
`Sheet 5 of 10
`
`US 7,650,570 B2
`
`
`Nww
`
`
`.mflTplmEmEzo80(cm
`
`
`mm
`
`.mmP785;7.3.wa4
`
`
`
`_ovw,mMWJrgoomoom
`
`
`”mg?00qomm.3;\mmou<mm<\a‘
`
`
`
`
`
`
`b3.mflam:I8mmoo
`
`mmi9NJI,gm?.:
`
`
`
`
`
`
`
`rflkx:vm>mawmm§mm
`
`.EW,mo0<mm<§2mm<E,$§omm0960figw%,<momom<omtu
`83mmm$34m<<mmmmm#15Q94,,m0OE0
`cmmwmmgaw
`mmmmmmmm
`
`mmdegagmmcmg%mmmoommmmmmo<
`
`
`
`ommtmzm...:wmoooommmmQoo0000umooW9m2cgoam_owm.vm: 0000mmmmmEmmmoowm“wmmcemmczomo<Mmmomm
`
` .vmWT!232350Wmuo<mm:mmmommWo5NF0ma<Moumo
`
`
`
`
`.wmw\\‘I».0.wmyljfl¢///<\\\\\«wnmr.
`
`comNW?am_mm,mmw”Nwwmommow
`<m«$2fiau<§a95
`
`
`um:
`
`m
`
`mmm
`
`
`m<m<
`
`m<
`
`bww
`
`"om
`
`m<<mommm
`mmmmm,
`
`m<<mooo4mmm
`
`
`
`Page 00008
`
`Page 00008
`
`
`
`
`
`
`
`

`

`
`
`
`
`
`
`
`
`
`
`
`150'148’146'
` MyRating
` Playcoum
`
`
`US. Patent
`
`Jan. 19, 2010
`
`Sheet 6 0f 10
`
`US 7,650,570 B2
`
`'158'
`
`122‘
`024‘
`
`
`206322‘128‘126'
`fi‘o\m M.14\;»,:.,2‘;.133;2;'u'-£0
`
`
`(\R 38\ \ g
`
`
`
`
`
`
`.21-MVVV'VV'V v
`
`vv '7
`
`
`x
`a
`s .' c:'x'.:_~._’.v.3:57.33.'.ut.:'.:.‘_’.' Ell
`
`\
`
`
`
` v
`
`‘7
`
`a.
`
`60
`
`<
`
`1'
`
`
`
`
`
`
`-.’...,‘. u‘a-J-
`.............
`
`~PE3.‘-.-:::;-.--:=.-::.~-s-:'-.~'-'
`
`
`
`www»wvmm
`/ 66’fi95”.%’0:9% ‘¢” .’%’ \
`_
`.9 .491.
`
`3%
`\“i
`
`
`
`
`
`
`
`Page 00009
`
`Page 00009
`
`

`

`U.S. Patent
`
`Jan. 19, 2010
`
`Sheet 7 of 10
`
`US 7,650,570 B2
`
`
` .wmw«wimi9.392:V3%mg8cmemmczam8.5
`
`
`$062:flak.wmbm
`
`memmmwmgemgem
`
`
`
`.9:@56122
`
`
`
`
`
`.93.mibmw
`
`N0E
`
`ENF
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`mmm
`
`ooo
`
`000”
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`.wmw85a$3
`.vmwE29391
`aw: b:
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
` mmmmmm
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
` gmummmmkflmooo<mm
`
`
`
`3m
`
`.w:
`
`Page 00010
`
`Page 00010
`
`
`

`

`US. Patent
`
`Jan. 19, 2010
`
`Sheet 8 0f 10
`
`US 7,650,570 B2
`
`
`
`‘
`
`y”
`
`,
`
`/ 0
`
`
`
`
`%//%/2
`
`7%7é“
`\\ V
`‘~\\\\ \
`
`;
`X?
`
`
`
`.1531;
`'
`\
`,
`;_-.~j;-_ ./ (,2 \ A
`/ ” k §§\ £3
`\\\\
`
`///l
`///4 M
`,
`.
`{I
`
`
`‘A 3%
`;
`Vly/l
`/////////
`
`7
`‘ \\,
`‘
`
`,V,
`,0“
`|’*%
`A
`
`////j
`, Wfi’g LL,
`7
`,
`/
`
`%
`
`
`
`//
`
`6/4 %
`//;\
`
`/
`/
`["1 Mg};
`/
`//
`
`/; f/Zz/Z/fi
`
`3/
`4/: m&
`" '
`///
`\
`
`
`
`
`
`,x
`
`§
`V
`‘
`
`
`I}
`
`/
`
`
`
`
`
`'I“
`////;u
`
`
`
`
`L1".::'-'-
`/g ’7
`’
`.V
`i 7:; ”/5/
`\
`
`/
`
`
`
`
`
`
`
`
`
`
`%%%%
`
`353.33i‘5§
`\¢/%%%729?
`u
`
`
`77;, .i
`\ %
`’\\t\\:
`/,71‘ /§%/%
`304
`%%%%Z¥%5
`
`
`
`‘
`\
`
`
`
`
`/ véM
`
`%%l%
`o VIIW
`
`\\
`
`\
`
`
`
`Page 00011
`
`4/
`///
`
` 1/
`.
`
`
`
`
`
`
`7
`I
`
`Page 00011
`
`

`

`US. Patent
`
`Jan. 19, 2010
`
`Sheet 9 0f 10
`
`US 7,650,570 B2
`
`332
`
`334
`
`
`
`“i“
`
`
`
`
`
`u
`
`
`
`322
`\
`
`-. 22...}?w... '.'_ ”.1..... Mi:”.2".-. 21:5.
`
`302
`
`0.2 %
`”U
`
`V '.";..;; :-
`
`'-
`'5 0: 1'
`
`:3." .. ,.'._
`
`328
`
`'5' g
`.
`
`-
`
`v
`
`".. ."..".'-',"...."::'.
`5
`112'
`11:“ ;j'-'
`g
`O...
`.
`‘/
`‘
`i ..
`'
`;
`..g-:-' §g
`.
`
`/A 2
`
`V .
`
`.Q
`V
`
`g
`k
`8
`n:
`
`oo \LL. K
`Kw
`
`CO
`
` :
`
`. “’1 I"
`
`1"
`
`Page 00012
`
`

`

`U.S. Patent
`
`Jan.19,2010
`
`Sheet100f10
`
`US 7,650,570 B2
`
`fiomvow
`
`
`
`...v.‘..........
`
`................‘..
`..............‘..‘...
`..................
`...........-
`
`...........c...a
`
`................
`
`
`
` \\
` 03BE3on%§
`
`
`Page00013
`
`Page 00013
`
`
`

`

`1
`METHODS AND APPARATUS FOR
`VISUALIZING A MUSIC LIBRARY
`
`RELATED APPLICATIONS
`
`This application claims priority from US. Provisional
`Application No. 60/723,865 filed Oct. 4, 2005, which is
`hereby incorporated by reference in its entirety.
`
`BACKGROUND
`
`This application relates to methods and apparatus for pro-
`viding a graphical representation of a music library.
`Effectively organizing a music library not only allows a
`user to get a sense of music contained in the library, but also
`helps them select and play the music. However, the popularity
`of digital audio encoding together with music distribution
`channels through the Internet have allowed users to collect
`hundreds or even thousands of media items. This change in
`scale of accessible music from the traditional album to thou-
`
`sands of songs makes choosing what music to listen to at a
`particular moment a challenge.
`Digital media players, such as iTunes 10 of FIG. 1 intro-
`duced by Apple Computer, Inc., Cupertino, Calif., USA,
`allow users to play and organize digital music and video files
`using textual lists. Each item of the list may be categorized by
`track title 12, track length 14, artist 16, album 18, year
`released 20, genre 22, and composer 24. Track lists can be
`ordered alphanumerically by categories such as title 12, artist
`16, album 18, or genre 22 for example. Search bar 26 may be
`used to perform a keyword-based search by one or more
`category, such as artist 16, album 18, or track title 12. Tracks
`may also be filtered using a genre filter 28, artist filter 30, or
`album filter 32, or all three filters may be used at the same
`time. For example, the user can filter all tracks by (1) Jazz
`using genre filter 28, (2) Billie Holiday using artist filter 30,
`and (3) Lady in Satin using album filter 32. Results from the
`filters are displayed in a results window 34 and may be
`ordered alphanumerically by one or more category, such as
`artist 16, album 18, or track title 12.
`Playlists are also known to help organize and manage
`music libraries. A playlist is a subset of a library that defines
`an ordered sequence of media items to be played and are
`usually created by adding media items to the playlist one-by-
`one. A smart playlist follows a set of logical filtering criteria,
`such as all jazz from 1970 that were played in the last six
`months. Playlists may be generated in iTunes 10 using two
`different methods: (1) adding media items manually in a
`one-by-one manner; and (2) defining filtering criteria, such as
`artist 16, album 18, or track title 12, to create smart playlist 36.
`The smart playlist may automatically update when new media
`items are added to the library.
`Playlists are also known to be automatically generable.
`PATS: Realization and User Evaluation of anAutomatic Play-
`list Generator of Pauws et. al. refers to a Personalized Auto-
`
`matic Track Selection (PATS) that creates a playlist using a
`dynamic clustering method. Songs are grouped based on a
`similarity measure that selectively weighs categorization val-
`ues of songs, such as track title, year released, album, style,
`tempo, instruments used, place ofrecording, record company,
`or rhythmic foundation. The similarity measure is selective in
`the sense that one categorization value may be more impor-
`tant than another. When the user selects a song, the cluster in
`which the song is contained is presented as a playlist. An
`inductive learning algorithm is used to eliminate tracks from
`future playlists based upon user input.
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`US 7,650,570 B2
`
`2
`
`In addition, analyzing media items based on signal pro-
`cessing techniques are known to allow the user to organize
`and visualize a music library. However, these methods ana-
`lyze each media item using signal processing techniques
`without requiring categorization based on genre, artist, and
`year.
`Furthermore, visualizing search results is known to help
`users browse for digitized music. Variations2: Toward Visual
`Interfaces for Digital Music Libraries of Notess et. al. refers
`to visualizing music bibliographic data to assist music stu-
`dents and faculty members browse and search for digitized
`music. By way of example, a student may search for works by
`creator and instrumentation used. The results ofthe search are
`
`displayed with a grid-based visualization that uses icon shape
`to represent media type, such as audio, score, or video, color
`to represent the performer, and position within the grid to
`indicate both work genre (x-axis) and composer/work
`(y-axis). Hovering over an object gives details on the per-
`former, for example.
`However, each one of these references suffers from one or
`more of the following disadvantages: (1) the user does not
`have an overall feel for how many media items are in the
`music library; (2) the user cannot intuitively see what portion
`of the library represents a rock genre versus an easy listening
`genre, for example; (3) the user cannot easily see desired
`attributes about each media item in relation to the library as a
`whole, such as which media items they listen to most often;
`(4) the user cannot easily visualize, manage, or organize
`playlists; and (5) the user cannot easily rediscover media
`items in their music library or know which portion of their
`music library needs expansion.
`The present inventors have recognized a need for improved
`apparatus and methods for providing a graphical representa-
`tion of a music library.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`Features, aspects, and advantages of the present invention
`are set forth in the following description, appended claims,
`and accompanying drawings wherein:
`FIG. 1 shows a user interface for iTunes of the prior art;
`FIG. 2 shows a disk visualization ofa music library accord-
`ing to a first embodiment;
`FIG. 3 shows the disk visualization of FIG. 2 with a mean
`
`value of a playcount of all media items within a sector illus-
`trated;
`FIG. 4 shows the disk visualization of FIG. 2 having a
`graphical representation of playlists overlaid thereon;
`FIG. 5 shows a rectangular visualization of a music library
`according to a second embodiment;
`FIG. 6 shows the rectangular visualization of FIG. 5 with a
`mean value of a playcount of all media items within a sector
`illustrated;
`FIG. 7 shows the rectangular visualization of FIG. 5 having
`a graphical representation of playlists overlaid thereon;
`FIG. 8A shows a Tree-Map visualization of a music library
`according to a third embodiment;
`FIG. 8B shows a Tree-Map visualization for the rock genre
`of FIG. 8A;
`FIG. 8C shows a Tree-Map visualization for the rock and
`roll sub-genre of FIG. 8B.
`
`DETAILED DESCRIPTION OF PREFERRED
`EMBODIMENTS
`
`Throughout the specification, reference to “one embodi-
`ment,” “an embodiment,” or “some embodiments” means that
`
`Page 00014
`
`Page 00014
`
`

`

`US 7,650,570 B2
`
`3
`a particular described feature, structure, or characteristic is
`included in at least one embodiment. Thus appearances of the
`phrases “in one embodiment,” “in an embodiment,” or “in
`some embodiments” in various places throughout this speci-
`fication do not necessarily refer to the same embodiment.
`Furthermore, the described features, structures, and char-
`acteristics may be combined in any suitable manner in one or
`more embodiments. Skilled persons will appreciate that the
`invention can be practiced without one or more of the specific
`details, or with other methods, components, materials, etc. In
`some instances, well-known structures, materials, and opera-
`tions are omitted or not described in detail to aVoid obscuring
`aspects of the embodiments.
`FIG. 2 shows a disk Visualization 40 of a music library
`according to a first embodiment. Disk Visualization 40
`includes a plurality of sectors 50 representing genre of the
`music library, one or more sub-sectors 60 representing artists
`associated with the genre, radii 70 representing a time axis,
`and a plurality of marks 100 corresponding to each media
`item of the music library. Disk Visualization 40 may not only
`pr0Vide an 0VerView ofthe total number ofmedia items in the
`music library but may also pr0Vide good percentage and
`proportional 0VerViews. In addition, disk Visualization 40
`may allow one or more playlists 110 to be Visualized, man-
`aged and organized. Furthermore, disk Visualization 40 may
`facilitate redisc0Vering media items in a music library instead
`of simply enlarging it. Therefore, when the time comes to
`expand the music library, disk Visualization 40 may be helpful
`in deciding what to acquire or listen to next.
`Referring now to FIG. 2, disk Visualization 40 is diVided
`into different sectors 50 that represent each genre of the
`library. For example, the music library illustrated in the
`embodiment of FIG. 2 is categorized into nine genres: (1)
`Rock genre 112; (2) Soundtrack genre 114; (3) Easy listening
`genre 116; (4) Electronica/dance genre 118; (5) Jazz genre
`120; (6) Latin genre 122; (7) Pop genre 124; (8) AltematiVe
`and punk genre 126; and (9) R&B genre 128. The music
`library could also be classified into other genres, such as
`Classical music, Gospel, Blues, Rhythm and blues, Funk,
`Metal, Country music, Electronic music, Melodic music,
`Ska, Reggae, Dub, Hip hop, and Contemporary African
`music.
`
`The size of each sector 50 may be proportional to the
`number ofmedia items ofthe associated genre with respect to
`the whole library. Therefore, the size of each sector 50 may be
`directly proportional to the importance of the corresponding
`genre within the library. At the same time, sectors 50 may be
`split in sub-sectors 60 representing the artists of the associ-
`ated genre. Again, the size of sub-sectors 60 may be propor-
`tional to the number of media items of the artist. The radii 70
`
`of disk Visualization 40, from the center 80 to the perimeter
`90, could illustrate a time axis. In addition, the center 80 could
`represent the year of the oldest possible media item of the
`library and the perimeter 90 could represent the most recent
`media items in the library. While year may be described
`primarily in relation to a year an album was released, it is to be
`appreciated that year can include the year a media item was
`released, for example.
`In addition, although media items may be described pri-
`marily in relation to songs or music tracks, it is to be appre-
`ciated that media items can include, but are not limited to,
`songs, tracks, music CDs, m0Vies, music Videos, documents,
`books, poems, and images (e.g., photographs), for example.
`Media items may be depicted as marks 100 and can be
`arranged within disk Visualization 40 according to the media
`item’s categorization. For example, the media items of the
`library may be categorized according to genre, artist, and year
`
`10
`
`15
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`4
`
`of release. Further categorizations may include title of the
`media item, album, style or era, tempo, musicians featured in
`the media item, instruments used in the media item, total
`number of musicians, soloing musicians, composer of the
`media item, producer ofthe media item, where the media item
`was recorded, whether the media item is a liVe performance,
`record company, rhythmic foundation, and melodic/har-
`monic deVelopment.
`In one embodiment, media items
`belonging to the same album are positioned contiguously
`thereby haVing the effect of producing arcs of points 130
`representing albums. In addition, albums may be depicted in
`alphanumeric order and media items of the same album may
`be ordered in the original order of the album.
`QuantitatiVe attributes 132 may be chosen by the user. For
`example, quantitatiVe attributes 132 may include playcount
`134, rating 136, last played date 138, and added date 140. In
`addition, other quantitatiVe attributes 132 may include ratings
`and reViews assigned by critics, artists, or others, or artists
`haVing new media items forthcoming. The quantitatiVe
`attributes 132 may be depicted by marks 100 and colors may
`be used to express the exact Value for one media item in its
`associated mark 100. For example, each mark 100 could haVe
`different color or grayscale tonalities indicatiVe of quantita-
`tiVe attribute 132. By way of example, if the quantitatiVe
`attribute 132 is playcount 134, a spectrum from light blue to
`black may be used. Marks 100 colored light blue could rep-
`resent the most played media items, marks 100 colored black
`could represent the least played media items, and marks 100
`colored according to anotherpart ofthe spectrum could fall in
`between the most played and least played media items. In
`addition, each mark 100 could haVe a unique identification
`code 142, such as A, B, and C. By way of example, if the
`quantitatiVe attribute 132 is playcount 134, A could represent
`the most played media item, C could represent the least
`played media item, and B could represent a media item falling
`in between the least played and most played.
`Referring now to FIG. 3, a mean Value of all the media
`items for one genre may be used to color a corresponding
`sector 50. For example, each sector 50 could haVe different
`color or grayscale tonalities indicatiVe of the mean Value of
`the playcount 134 of all media items within that sector 50.
`Again, a spectrum from light blue to black may be used, but
`other colors would be suitable. Sectors 50 colored light blue
`could represent sectors 50 haVing the most played media
`items, sectors 50 colored black could represent sectors 50
`haVing the least played media items, and sectors 50 colored
`according to another part of the spectrum could represent
`sectors 50 haVing media items with a playcount falling
`between the most played and least played. In addition, each
`sector 50 could also be shaded with unique patterns 144 as
`shown in FIG. 3. By way of example, sectors 50 represented
`by a first unique pattern 146 could represent sectors 50 haVing
`the most played media items. In a similar manner, sectors 50
`represented by a second unique pattern 148 could represent
`sectors 50 haVing media items with a playcount falling
`between the most played and least played. Finally, sectors 50
`represented by a third unique pattern 150 could represent
`sectors 50 haVing the least played media items.
`Referring now to FIG. 4, playlists 110 are shown using the
`disk Visualization 40. Playlists 110 may be created by adding
`media items in a one-by-one manner, or they could be smart
`playlists following a set of logical filtering criteria. Media
`items of playlists 110 without any grouping logic may be
`depicted using geometric forms different from marks 100,
`which are used in general for the rest of the media items. For
`example, jogging playlist 152 could be displayed using dia-
`mond shapes 154. While, diamond shapes 154 are colored
`
`Page 00015
`
`Page 00015
`
`

`

`US 7,650,570 B2
`
`5
`black, other shapes, sizes, colors or shading could be used.
`Likewise, 25 last played playlist 156 could be represented
`using black crosses 158. Again, other shapes, sizes, colors or
`shading could also be used. Other playlists 110, including
`60’s and 70’s music playlist 160, Bruce Springsteen playlist
`162, and R.E.M. after 1990 playlist 164, may be shown as
`shaded regions since they follow a regular geometric form. In
`one embodiment, red shaded regions are used, but other col-
`ors, shading patterns, or indicia could also be used. Rolling
`Stones & Beatles playlist 166 and 80’ s pop music playlist 168
`are not highlighted in the embodiment shown in FIG. 4
`because the user has not activated the corresponding check-
`boxes 170.
`
`Disk Visualization 40 may also indicate currently playing
`media item 172. For example, currently playing media item
`172 could be displayed using a circular shape. While, the
`circular shape illustrating currently playing media item 172 is
`colored black, other shapes, sizes, colors or shading could be
`used. Furthermore, path 174 grouping media items to be
`played next could be displayed. In this manner, the user could
`get an idea of what regions of the library are going to be used
`in the current music sequence, such as playlist 110.
`The user may interact with disk Visualization 40 in a num-
`ber ofways, including naVigating media items, zooming 0Ver
`one or more sectors 50, managing playlists 110, and search-
`ing for media items. F