I lllll llllllll Ill lllll lllll lllll lllll lllll 111111111111111111111111111111111
`US007870142B2
`
`c12) United States Patent
`Michmerhuizen et al.
`
`(10) Patent No.:
`(45) Date of Patent:
`
`US 7,870,142 B2
`Jan. 11, 2011
`
`(54) TEXT TO GRAMMAR ENHANCEMENTS
`FOR MEDIA FILES
`
`(75)
`
`Inventors: Mark Michmerhuizen, Holland, MI
`(US); Mark Zeinstra, Holland, MI
`(US); Brian L. Douthitt, Holland, MI
`(US)
`
`(73) Assignee: Johnson Controls Technology
`Company, Holland, MI (US)
`
`( *) Notice:
`
`Subject to any disclaimer, the term ofthis
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 0 days.
`
`(21) Appl. No.: 11/517,832
`
`(22) Filed:
`
`Sep.8,2006
`
`(65)
`
`Prior Publication Data
`
`US 2007/0233725 Al
`
`Oct. 4, 2007
`
`Related U.S. Application Data
`
`5,214,707 A *
`5,278,547 A
`5,384,893 A *
`5,475,366 A
`
`5/1993 Fujimoto et al ............. 704/275
`111994 Suman et al.
`111995 Hutchins .................... 704/267
`12/1995 Van Lente et al.
`
`(Continued)
`
`FOREIGN PATENT DOCUMENTS
`
`DE
`
`10 2004 016 319 Al
`
`1112004
`
`(Continued)
`
`OTHER PUBLICATIONS
`
`Wikipedia-Speak & Spell (toy), retrieved Sep. 2, 2010.*
`
`(Continued)
`
`Primary Examiner-James Trujillo
`Assistant Examiner-William Spieler
`(74) Attorney, Agent, or Firm-Foley & Lardner LLP
`
`(60) Provisional application No. 60/789,176, filed on Apr.
`4, 2006.
`
`(57)
`
`ABSTRACT
`
`(51)
`
`Int. Cl.
`G06F 17130
`(2006.01)
`(2006.01)
`GJOL 13102
`(2006.01)
`GJOL 13108
`(52) U.S. Cl. ....................... 7071755; 704/258; 704/275;
`701/36
`( 58) Field of Classification Search . ... ... ... ... .. ... ... 707 /1,
`707/755; 709/219; 704/277, 1, 273, 275,
`704/257, 258, 249; 705/14; 701/1, 29, 202;
`434/167; 455/3.05, 574
`See application file for complete search history.
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`3,704,345 A * 1111972 Coker et al ................. 704/266
`4,827,520 A
`5/1989 Zeinstra
`5/1992 Suman et al.
`5,113,182 A
`
`A control system in a vehicle for extracting meta data from a
`digital media storage device over a communication link. The
`system includes a communication module for establishing a
`communication link with the digital media storage device.
`The system also includes a processing module coupled to the
`communication module. The processing module is config(cid:173)
`ured to retrieve, via the communication module, meta data
`associated with a media file from the digital media storage
`device. The meta data includes a plurality of entries, wherein
`at least one of the plurality of entries includes text data. The
`processing module is also configured to compare the text data
`of the entries with a set of data files stored in a database. The
`system also includes a memory module configured to store
`the plurality of entries retrieved from the digital media stor(cid:173)
`age device.
`
`20 Claims, 8 Drawing Sheets
`
`Establish
`Communication link
`with Digital Media
`Storage Device
`
`Identify Data File to
`Retrieve From Digital
`Media Storage Device
`
`Retrieve Data File from
`Digital Media Storage
`Device
`
`Identify Text and/or
`NumericDataofOata
`File
`
`Generate Phonemic
`Representation of Text
`and/or Numeric Data
`
`Store Data File And
`Phonemic
`Representation in
`Memory
`
`BMW EXHIBIT 1002
`
`Page 1 of 19
`
`

`

`US 7,870,142 B2
`Page 2
`
`U.S. PATENT DOCUMENTS
`
`5,479,155 A
`5,479,157 A
`5,583,485 A
`5,614,885 A
`5,614,891 A
`5,619,190 A
`5,627,529 A
`5,646,701 A
`5,661,455 A
`5,691,848 A
`5,699,044 A
`5,703,308 A *
`5,708,415 A
`5,717,387 A
`5,854,593 A
`5,903,226 A
`5,926,087 A
`6,028,537 A
`6,078,885 A *
`6,154,148 A
`6,202,008 Bl
`6,389,337 Bl *
`6,526,335 Bl *
`6,584,181 Bl*
`6,600,430 B2
`6,622,083 Bl *
`6,801,893 Bl*
`6,874,018 B2 *
`6,928,261 B2
`6,993,532 Bl
`7,024,424 Bl
`7,047,039 B2
`7,257,426 Bl
`2002/0048224 Al
`2002/0049717 Al *
`2002/0082831 Al *
`2002/0086719 Al *
`200210090596 Al *
`2002/0091706 Al
`200210107973 Al
`2002/0120925 Al
`2002/0197955 Al
`2003/0014767 Al
`2003/0032419 Al*
`2003/0093260 Al *
`2003/0109218 Al*
`2003/0110079 Al *
`2003/0137543 Al
`2003/0158737 Al*
`2003/0163823 Al
`2003/0177013 Al *
`2003/0228879 Al
`2004/0022137 Al
`2004/0033821 Al
`
`12/1995 Zeinstra et al.
`12/1995 Suman et al.
`12/1996 Van Lente et al.
`3/1997 Van Lente et al.
`3/ 1997 Zeinstra et al.
`411997 Duckworth et al.
`511997 Duckworth et al.
`7I1997 Duckworth et al.
`8/1997 Van Lente et al.
`1111997 Van Lente et al.
`12/1997 Van Lente et al.
`12/1997 Tashiro et al .................. 84/609
`111998 Van Lente et al.
`2/1998 Suman et al.
`12/1998 Dykema et al.
`511999 Suman et al.
`7I1999 Busch et al.
`212000 Suman et al.
`612000 Beutnagel ................... 704/258
`1112000 Fluharty et al.
`3/2001 Beckert et al.
`512002 Kolls . ... ... ... .. ... ... ... ... .. . 701129
`212003 Treyz et al ..................... 70111
`6/2003 Aktas et al. .............. 379/88.23
`7/2003 Minagawa et al.
`.......... 7011202
`9/2003 Knockeart et al.
`........... 704/257
`10/2004 Backfried et al.
`3/2005 Wu ............................ 709/219
`8/2005 Hasegawa et al.
`112006 Platt et al.
`412006 Platt et al.
`512006 Lalley
`8/2007 Witkowski et al.
`412002 Dygert et al.
`412002 Routtenberg et al ............ 707/1
`612002 Hwang et al. ............... 704/249
`712002 Kedia et al. ................. 455/574
`712002 Sosoka et al. ............... 434/167
`712002 Anderson et al.
`8/2002 Lennon et al.
`8/2002 Logan
`12/2002 Witkowski et al.
`112003 Stumphauzer, II
`........... 455/419
`212003 Shibasaki et al.
`5/2003 Dagtas et al.
`.. .. ... ... ... ... .. 704/ 1
`6/2003 Pourkeramati et al ...... 455/3.05
`6/2003 Weisman et al ............... 705/14
`7 /2003 Anderson et al.
`8/2003 Csicsatka .................... 704/273
`8/2003 Logan et al.
`9/2003 Falcon et al. ................ 704/275
`12/2003 Witkowski et al.
`212004 Campbell et al.
`212004 Slesak et al.
`
`3/2004 Witkowski et al.
`2004/0048622 Al
`512004 Georges et al.
`2004/0089141 Al
`6/2004 Witkowski et al.
`2004/0110472 Al
`2004/0111271 Al*
`6/2004 Tischer ....................... 704/277
`6/2004 Thielen
`2004/0117442 Al
`2004/0194611 Al* 10/2004 Kawana ....................... 84/600
`2004/0203379 Al
`10/2004 Witkowski et al.
`2005/0038813 Al
`212005 Apparao et al.
`2005/0090279 A9
`412005 Witkowski et al.
`2005/0155068 Al
`712005 Chang
`2005/0223406 Al
`10/2005 Viti to
`2005/0235326 Al
`10/2005 Viti to
`2005/0235327 Al
`10/2005 Viti to
`2005/0239434 Al
`10/2005 Marlowe
`2005/0281414 Al
`12/2005 Simon eta!.
`2006/0034481 Al
`212006 Barzegar et al.
`2006/0070102 Al
`3/2006 Viti to
`712006 Zeinstra et al.
`200610168627 Al
`3/2007 Zeinstra et al.
`2007/0061067 Al
`2007 /0082706 Al
`4/2007 Campbell et al.
`2007/0198563 Al
`8/2007 Apparao et al.
`2007/0233725 Al
`10/2007 Michmerhuizen et al.
`
`FOREIGN PATENT DOCUMENTS
`
`05290382 A * 1111993
`JP
`2004-309795 A
`JP
`1112004
`WO WO 2005/019985 A2
`3/2005
`WO WO 2007/123797 Al
`1112007
`WO WO 2007/123798 Al
`1112007
`
`OTHER PUBLICATIONS
`
`International Search Report and Written Opinion for Application No.
`PCT/US2007/008063, Sep. 25, 2007, 8 pages.
`International Search Report and Written Opinion for Application No.
`PCT/US2007/008064, Sep. 14, 2007, 8 pages.
`Gilroy, Amy, "Sony Joins In Music Streaming From Cellphones to
`Car Audio," printed from website www.twice.com on Aug. 17, 2006,
`2 pages.
`Jain, Ankit et al., "Standard boosts automotive Bluetooth streaming,"
`Automotive Design Line,
`printed
`from website www.
`automotivedesignline.com on Jun. 15, 2006, 6 pages.
`Johnston, Lisa, "ANY COM Announces Bluetooth Adapter for iPod
`nano," TWICE: This Week in Consumer Electronics, printed from
`website www.twice.com on Aug. 17, 2006, 1 page.
`Pioneer DEH-P9800BT, CD receiver with MP3/WMA playback and
`Bluetooth wireless, Crutchfield, printed from website www.
`crutchfield.com on Nov. 27, 2006, 3 pages.
`Sony MEX-BT5000, CD receiver with Bluetooth technology and
`MP3/WMA playback, Crutchfield, printed from website www.
`crutchfield.com on Nov. 27, 2006, 2 pages.
`International Search Report and Written Opinion for International
`Patent Application No. PCT/US2007/008064, mailed Sep. 14, 2007,
`7 pages.
`* cited by examiner
`
`Page 2 of 19
`
`

`

`U.S. Patent
`
`Jan. 11, 2011
`
`Sheet 1of8
`
`US 7,870,142 B2
`
`.
`(9
`LL
`
`Page 3 of 19
`
`

`

`U.S. Patent
`
`Jan. 11, 2011
`
`Sheet 2of8
`
`US 7,870,142 B2
`
`106
`
`I
`
`112
`
`rci
`©
`©
`112
`
`108
`
`114
`
`MEDIA PLAYBACK
`HANDS-FREE PHONE
`HVAC CONTROL
`CONTACT MANAGEMENT
`CALENDAR
`VEHICLE LOGS
`
`110
`
`880
`
`FIG. 2
`
`Page 4 of 19
`
`

`

`(128
`
`(106
`
`\I ~
`
`Digital Me
`dia
`Storage
`Device
`
`Audio Input
`Device
`
`-
`-
`
`~
`
`ir-126
`
`User Interface ~
`
`-
`
`/130
`
`Audio Output
`Device
`
`-
`
`-
`
`-
`
`~
`
`Communication -120
`Device
`(
`
`/122
`
`Data
`Processing
`System
`
`/124
`
`Display Driver
`
`Memory
`Device
`
`~ (11~ Digital Media
`-
`Storage
`Device
`w/ Wired Port
`
`116
`I~
`
`/132
`
`l
`
`1r
`
`.~
`
`1 •
`
`108
`
`'/104
`
`Output Display
`
`Audio System
`
`FIG. 3
`
`~
`00
`•
`~
`~
`~
`
`~ = ~
`
`116
`
`~
`
`~ := ....
`....
`0 ....
`....
`
`'"
`N
`
`1J1 =(cid:173)
`.....
`
`('D
`('D
`
`(.H
`
`0 .....
`
`QO
`
`d
`rJl
`-....l
`Oo
`-....l
`'"=
`"'"" ~
`N = N
`
`Page 5 of 19
`
`

`

`106
`
`fi18
`
`128
`
`Audio Input
`Device
`
`Text-to-Grammar
`Device
`
`peech Recognition
`Device
`
`Communication j.-120
`Device
`
`122
`
`144
`
`Mobile Phone
`
`146
`
`Personal
`Digital
`Assistant
`
`User Interface 1~
`
`.,,
`
`130
`
`Display Driver
`
`124
`
`Volatile v
`Memory
`Non- v
`Volatile
`Memory
`
`132
`
`Audio c:>utput 14
`Device
`
`.,
`
`1
`
`143
`
`......,
`
`~
`
`Database
`II
`
`11s I Digital Media v--11&
`Storage
`Device
`w/ Wired Port ,
`
`I Media Player
`
`1..,.-148
`
`v
`Remote Server
`
`154
`
`108
`
`104
`
`Output Display
`
`Audio System
`
`FIG. 4
`
`~
`00
`•
`~
`~
`~
`
`~ = ~
`
`~
`
`~ := ....
`....
`0 ....
`....
`
`'"
`N
`
`1J1 =-('D
`.....
`0 ....
`
`('D
`
`.i;...
`
`QO
`
`d
`rJl
`-....l
`Oo
`-....l
`'"=
`"'"" ~
`N = N
`
`Page 6 of 19
`
`

`

`U.S. Patent
`
`Jan. 11, 2011
`
`Sheet 5of8
`
`US 7,870,142 B2
`
`230
`
`(
`Display and Graphics
`Hardware
`
`(202
`
`(212
`
`Tactile User Interface and
`Radio Manager
`
`I Rotary GPIO Radio I Cl.
`
`Input
`10
`~ lli ~
`
`Graphical User Interface
`
`I+-
`
`t
`v---210
`GPIO I
`
`+ ,,.--214
`r
`12C Unit
`
`(222
`
`+
`
`GPIO /
`Extender
`
`,,-216
`
`t
`+ ,.,...-218
`
`Rad id
`Unit
`
`226
`
`•
`
`State Manager
`UI Context Manager f
`GUINUlfTUl/Radio
`(
`Integration
`Existing
`\ Existing
`Focus
`Handstref'
`Management
`App
`
`- 224
`
`226
`
`-
`
`228
`
`,,
`
`(232
`
`Voice User Interface
`
`Text-to-Grammar Device
`
`234
`
`236
`
`238
`
`-
`I
`r
`,,,...-
`I Speech Recognition Device t
`t-
`I Text-to-Speech Device
`'
`·~
`
`v
`--. Microphone
`
`240
`
`~ Speakers
`
`250
`I~
`
`•
`
`Music Manager
`
`USB
`Playback
`244
`
`Media
`iPod
`Playback Cataloger
`246
`248
`
`t
`
`v-242
`
`J.
`
`• (278
`
`t
`
`Mass
`Storage
`Device
`1USB)
`
`+ (260
`
`Bluetooth Device
`
`Media
`Streaming
`2§2
`
`iPod
`Control
`™
`
`•
`
`Bluetooth
`Host Stack
`m-
`
`+
`
`27~ iPod
`Media
`Player
`
`+
`
`Mobile
`Phone
`
`-~276
`
`FIG. 5
`
`V---254
`v
`
`i,r--256
`v
`i,r--258
`v
`
`(252
`
`•Ir
`Music Database
`
`ClientAPI
`
`SQL Database Engine
`
`Music Database Schema
`
`(268
`
`v -270
`
`Volatile
`Memory
`
`Non- v -272
`
`Volatile
`Memory
`
`Page 7 of 19
`
`

`

`Establish
`Communication Link
`with Digital Media
`Storage Device
`
`l
`
`Identify Data File to
`Retrieve From Digital
`Media Storage Device
`
`1,
`
`1
`Retrieve Data File from 11
`Digital Media Storage
`Device
`1
`Identify Text and/or
`Numeric Data of Data
`File
`l
`Generate Phonemic
`Representation of Text
`and/or Numeric Data
`l
`Store Data File And
`Phonemic
`Representation in
`Memory
`
`.,
`
`FIG. 6
`
`282
`
`284
`
`286
`
`BB
`
`90
`
`92
`
`302
`
`304
`
`306
`
`308
`
`310
`
`Receive Request to
`Retrieve Meta Data
`from Media Files
`
`Retrieve Meta Data
`
`Capture Meta Data in
`Media File
`
`Process Meta Data in
`Media File
`
`Store Meta Data in
`Memory
`
`FIG. 7
`
`02
`
`04
`
`06
`
`Identify Meta Data
`Entries in Each
`Media File
`
`Generate Phonemic
`Representation for
`Each Entry of
`Meta Data
`
`Store Phonemic
`Representations
`
`FIG. 8
`
`~
`00
`•
`~
`~
`~
`
`~ = ~
`
`~
`
`~ := ....
`....
`0 ....
`....
`
`'"
`N
`
`('D
`('D
`
`1J1 =(cid:173)
`.....
`O'I
`0 .....
`
`QO
`
`d
`rJl
`-....l
`Oo
`-....l
`'"=
`"'"" ~
`N = N
`
`Page 8 of 19
`
`

`

`02
`
`06
`
`I Select Media r12
`
`File for
`Playaback
`
`I
`
`Prompt User to
`Confirm
`
`Receive
`Confirmation
`
`Yes
`
`I
`
`14
`
`16
`
`I Receive Command I
`602
`to List Database ~
`Contents by Meta
`Data Entry Type
`
`Audibly List Entries
`for Meta Data
`Type
`
`604
`
`608
`
`606
`
`Receive r
`Prompt User r"'
`Receive Entry r
`Perform r°'
`
`Command
`
`for Entry to
`Playback
`
`(Track Title)
`
`Speech
`Recognition
`
`e •
`
`00
`•
`~
`~
`~
`
`~ = ~
`
`~
`
`~ := ....
`....
`0 ....
`....
`
`"'
`N
`
`('D
`
`1J1 =-('D
`.....
`-....J
`0 .....
`
`QO
`
`~r~ Perform
`
`Ye
`
`Requested Action
`
`In?
`
`I
`No
`I
`
`I
`
`No
`...
`
`c±r"" FIG. 10
`
`d
`rJl
`-....l
`Oo
`
`-....l = "' "'"" ~
`N = N
`
`I
`
`10
`
`I
`
`Play Media
`File
`
`18
`
`FIG. 9
`
`Page 9 of 19
`
`

`

`U.S. Patent
`
`Jan. 11, 2011
`
`Sheet 8of8
`
`US 7,870,142 B2
`
`Identify Media File on
`Digital Media Storage
`Device
`
`Retrieve Meta Data
`from Media File
`
`Identify Entries of Meta
`Data that Include Text
`Data
`
`708
`
`Ye
`
`Text Data
`Matches Stored
`File?
`
`No
`
`Generate Acoustic
`Baseform Based on
`Text Data from Entry
`
`702
`
`704
`
`706
`
`712
`
`714
`
`Store Entries in
`Memory
`
`FIG. 11
`
`Retrieve Acoustic
`Baseform from Data
`File
`
`710
`
`Page 10 of 19
`
`

`

`US 7,870,142 B2
`
`1
`TEXT TO GRAMMAR ENHANCEMENTS
`FOR MEDIA FILES
`
`CROSS-REFERENCE TO RELATED PATENT
`APPLICATIONS
`
`This application claims priority to U.S. Provisional Patent
`Application No. 60/789, 17 6, which was filed on Apr. 4, 2006.
`
`FIELD OF THE INVENTION
`
`The present invention relates generally to the field of data
`extraction from a digital audio/video source in a vehicle and
`in particular, a system and method for extracting and process(cid:173)
`ing meta data from media files accessible from a digital media 15
`storage device, such as a media player, in the vehicle.
`
`BACKGROUND OF THE INVENTION
`
`2
`media files stored on a digital media storage device; 3)
`retrieve meta data from a media file of the digital media
`storage device upon receiving a command from a user; 4)
`automatically retrieve meta data from a media file of the
`digital media storage device upon establishing a wireless
`communication link with the digital media storage device; 5)
`generate and/or update a database of meta data based on the
`meta data extracted from the digital media storage device,
`including converting the information to phonemic represen-
`10 tations so that a user may access and manipulate the media
`files via voice commands and speech recognition; 6) audibly
`list the meta data entries of a media file based on an audible
`command of the user; 7) generate a second database of meta
`data using the meta data extracted from a second digital media
`storage device, wherein the meta data of the first digital media
`storage device is retained across power cycles, thereby elimi-
`nating the time consuming process of extracting meta data
`each time a digital media storage device is connected to the
`system; and 8) provide for the use of multiple databases
`20 concurrently, for example, when two or more digital media
`storage devices are linked to the control module concurrently.
`
`Vehicles typically include a number of electronic systems
`such as an audio system, a mobile telephone system, a navi(cid:173)
`gation system, video entertainment system, etc. Along with
`those systems, digital media storage devices (i.e., digital
`audio/video players or MP3 players) may be setup in the
`vehicle for a user to access digital audio or video content 25
`stored on the digital media storage device. A conventional
`digital media storage device includes a data storage device
`(i.e., memory) and an embedded software application that
`allows a user to transfer media files to the player and create
`custom lists of selected media files, referred to as playlists. 30
`Media files may be transferred to the memory of the player
`from various sources, including disks (e.g., CDs, DVDs, etc.)
`and Internet sites. Most players are configured to be coupled
`to a personal computer via a connection port (e.g., USB port,
`parallel port, serial port, etc.) in order to transfer media files. 35
`The media files typically include data, referred to as meta
`data, which identifies the contents of the file. For example,
`with digital audio files, such as MP3 files, the meta data
`comprises a tagging format which may include the title of the
`song, the artist's name, the title of the album, track number, 40
`genre, etc. The tagging format for MP3 (i.e., MPEG Audio
`Layer III) files is commonly referred to as ID3.
`A digital media storage device is typically configured to
`include a memory having one or more media files stored
`therein. A digital media storage device in a vehicle may be 45
`configured to be coupled to or integrated with other vehicle
`electronic systems, such as the audio system, for playing the
`media files from the player. In one known configuration, a
`digital media storage device may be connected via an analog
`audio output (e.g., a headphone jack) from the digital media 50
`storage device to the auxiliary input of the vehicle's audio
`system, such that the playback is controlled by the controls of
`the digital media storage device. In another known configu(cid:173)
`ration, the digital media storage device's control interface is
`coupled to the control interface of the head unit of an audio 55
`system (i.e., radio), in addition to connecting the analog audio
`output of the digital media storage device to auxiliary input of
`the vehicle's audio system. In this configuration, the user can
`control the playback of the media from the head unit of the
`audio system. Yet, in another configuration, an audio system 60
`may include a database of meta data about certain media files.
`The audio system is configured to determine the meta data of
`a media file based on a unique identifier relating to that file.
`It would be advantageous to provide a system for extracting
`meta data from a digital media storage device in a vehicle that 65
`is configured to: 1) establish a communication link between a
`control module and a digital media storage device; 2) identify
`
`SUMMARY OF THE INVENTION
`
`One embodiment of the invention relates to a method for
`extracting meta data from a digital media storage device in a
`vehicle over a communication link between a control system
`of the vehicle and the digital media storage device. The
`method includes the step of identifying a media file on the
`digital media storage device. The method also includes the
`step of retrieving meta data from a media file. The meta data
`includes a plurality of entries, wherein at least one of the
`plurality of entries includes text data. The method also
`includes the step of identifying the entries of the meta data
`that include text data. The method also includes the step of
`comparing the text data with a set of stored data files, the
`stored data files being stored in a database. If the text data
`matches at least one of the stored data files, the method
`generates an acoustic baseform of the text data based on a
`phonemic representation of the stored data file. If the text data
`does not match at least one of the stored data files, the method
`generates an acoustic baseform of the text data based on a
`phonemic representation of the text data from the entry. The
`method also includes the step of storing the plurality of entries
`in a memory.
`Another embodiment of the invention relates to a control
`system in a vehicle for extracting meta data from a digital
`media storage device over a communication link. The system
`includes a communication module for establishing a commu(cid:173)
`nication link with the digital media storage device. The sys(cid:173)
`tem also includes a processing module coupled to the com(cid:173)
`munication module. The processing module is configured to
`retrieve, via the communication module, meta data associated
`with a media file from the digital media storage device. The
`meta data includes a plurality of entries, wherein at least one
`of the plurality of entries includes text data. The processing
`module is also configured to compare the text data of the
`entries with a set of data files stored in a database. The system
`also includes a memory module configured to store the plu(cid:173)
`rality of entries retrieved from the digital media storage
`device.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`FIG.1 is a perspective view ofa motor vehicle that includes
`a number of vehicle systems, including an in-vehicle control
`system according to one exemplary embodiment.
`
`Page 11 of 19
`
`

`

`US 7,870,142 B2
`
`3
`FIG. 2 is a front elevation view of the user interface of the
`in-vehicle control system of FIG. 1 according to one exem(cid:173)
`plary embodiment.
`FIG. 3 is a block diagram of the in-vehicle control system
`of FIG. 1 that includes a speech recognition system according
`to one exemplary embodiment.
`FIG. 4 is a more detailed embodiment of the in-vehicle
`control system of FIG. 3 according to one exemplary embodi(cid:173)
`ment.
`FIG. 5 is a block diagram of a control system for extracting
`meta data from a digital media storage device in a vehicle
`including the in-vehicle control system of FIG. 4 according to
`one exemplary embodiment.
`FIG. 6 is process flow diagram illustrating a method of
`generating a phonemic representation of data with the in(cid:173)
`vehicle control system of FIG. 3 according to one exemplary
`embodiment.
`FIG. 7 is a process flow diagram illustrating a method of
`retrieving and processing meta data from media files within a
`digital media storage device with the in-vehicle control sys(cid:173)
`tem of FIG. 3 according to one exemplary embodiment.
`FIG. 8 is a process flow diagram illustrating a method of
`generating a phonemic representation of data associated with
`meta data entries in a media file with the in-vehicle control
`system of FIG. 3 according to one exemplary embodiment.
`FIG. 9 is a process flow diagram illustrating a method of
`receiving a request from a user to play a media file using
`speech recognition with the in-vehicle control system of FIG.
`3 according to one exemplary embodiment.
`FIG. 10 is a process flow diagram illustrating a method of
`listing media file entries to a user with the in-vehicle control
`system of FIG. 3 according to one exemplary embodiment.
`FIG. 11 is a process flow diagram illustrating a method of
`extracting meta data from a digital media storage device using
`the in-vehicle control system of FIG. 3 according to one
`exemplary embodiment.
`
`DETAILED DESCRIPTION OF THE
`EXEMPLARY EMBODIMENTS
`
`Referring to FIG. 1, a vehicle 100 includes a number of
`subsystems for user convenience. Vehicle 100 generally
`includes a heating, ventilation, and air-conditioning (HVAC)
`system, a sound system, and an in-vehicle control system 106.
`The HVAC system and sound system may be coupled to 45
`in-vehicle control system 106, which is capable of controlling
`and monitoring both systems, automatically or by a manual
`user command. It is noted that in various exemplary embodi(cid:173)
`ments: vehicle 100, the HVAC system, and the sound system
`may be of any past, present, or future design that is capable of 50
`housing (in the case of vehicle 100) and interacting with
`in-vehicle control system 106.
`Referring to FIG. 2, one exemplary embodiment of in(cid:173)
`vehicle control system 106 is shown. In-vehicle control sys(cid:173)
`tem 106 generally includes an output display 108, one or 55
`more knobs 110, one or more pushbuttons 112, and one or
`more tactile user inputs or pushbuttons 114, which facilitate
`controlling various vehicle functions. Output display 108
`may be configured to display data related to the control of the
`vehicle functions. In one exemplary embodiment, output dis- 60
`play 108 may be a touch-screen display, while in other exem(cid:173)
`plary embodiments, may be any other non-touch sensitive
`display. In still other exemplary embodiments, output display
`108 may be ofany technology (e.g. LCD, DLP, plasma, CRT),
`configuration (e.g. portrait or landscape), or shape (e.g. 65
`polygonal, curved, curvilinear). Knobs 110 and pushbuttons
`112and114 maybe configured: (i) to control functions of the
`
`4
`HVAC system such as fan speed, cabin temperature, or rout(cid:173)
`ing of air flow, (ii) to control playback of media files over the
`sound system, (iii) to control retrieval of phone book entries,
`or (iv) to control any other desired vehicle function. Pushbut(cid:173)
`tons 114 typically allow for the selection and display of
`various functions of in-vehicle control system 106 including
`HVAC system control, sound system control, hands-free
`phone use, contact or address/phone book management, cal(cid:173)
`endar viewing/modification, and vehicle data logging. The
`10 operation of pushbutton 114 for media playback may display
`a media playback menu screen or execute commands that
`allow the user to view, select, sort, search for, and/or play
`audio or video files by tactile or oral command. The operation
`of pushbutton 114 for hands-free phone operation may dis-
`15 play a menu screen or execute commands that allows the user
`to connect in-vehicle control system 106 to a mobile phone so
`that speaking into the vehicle console of in-vehicle control
`system 106 operates the mobile phone. The operation of
`pushbutton 114 for HVAC control may display a menu screen
`20 or execute commands that allow the user to control cabin
`temperature and air flow by tactile or oral command. The
`operation of pushbutton 114 for contact management may
`display a menu screen or execute commands that allow the
`user to view, list, select, sort, search for, edit and/or dial one or
`25 more entries containing personal contact information, by use
`ofa tactile or oral command. The operation of pushbutton 114
`for calendar management may display a menu screen or
`execute commands that allow the user to view, list, select,
`sort, search for, edit and/or create one or more entries con-
`30 taining personal schedule information by tactile or oral com(cid:173)
`mand. The operation of pushbutton 114 for vehicle log man(cid:173)
`agement may display a menu screen or execute commands
`that allow the user to input, view, select and/or reset informa(cid:173)
`tion related to vehicle operation (e.g. fuel economy, engine
`35 temperature, distance to empty, etc.) by tactile or oral com(cid:173)
`mand.
`FIG. 3 is a block diagram of in-vehicle control system 106
`for extracting meta data from a digital media storage device in
`a vehicle in accordance with an exemplary embodiment. The
`40 control system enables a vehicle occupant, such as a driver, to
`extract the meta data from a digital media storage device 116,
`for example the occupant's media player, and subsequently
`generates an acoustic baseform based on a phonemic repre-
`sentation of the extracted text and/or numeric data. Other
`vehicle occupants (e.g., passengers) may also use the system
`to extract the meta data from another digital media storage
`device 116. It should be understood that although a system for
`extracting meta data will be described in detail herein with
`reference to digital media storage device 116, one or more of
`the systems and methods for extracting meta data herein may
`be applied to, and find utility in, other types of digital storage
`or audio/video devices as well. For example, one or more of
`the systems for extracting meta data may be suitable for use
`with a flash drive, a micro hard drive, a USB memory stick,
`etc., all of which may require certain device-specific hard(cid:173)
`ware and software to extract the meta data.
`The exemplary system shown in FIG. 1 enables a wireless
`and/or wired communication link 118 to be established
`between an audio system 104 of vehicle 100 and digital media
`storage device 116 (e.g., a digital media storage device with a
`Bluetooth-enabled connection) of a vehicle occupant. The
`wired connection may include a cradle for digital media stor(cid:173)
`age device 116 or a similar wired adapter, such as an analog
`audio output connection (e.g., a headphone jack). The follow(cid:173)
`ing description will refer to an exemplary control system
`using a wireless communication link between the vehicle
`audio system and digital media storage device 116 of a
`
`Page 12 of 19
`
`

`

`US 7,870,142 B2
`
`5
`vehicle occupant, however, it should be understood that meth(cid:173)
`ods and systems for retrieving and processing meta data from
`a digital media storage device described herein may be used
`in conjunction with other configurations of control systems,
`such as a wired connection to a vehicle occupant's digital
`media storage device, etc.
`In FIG. 1, the in-vehicle control system 106 is coupled to
`audio system 104 and an antenna 105. In an alternative
`embodiment, in-vehicle control system 106 may be incorpo(cid:173)
`rated into audio system 104. Antenna 105, in conjunction 10
`with a communications device 120, for example a communi(cid:173)
`cations device, within in-vehicle control system 106, enables
`two-way communication with digital media storage device
`116. Communication module 120 may be configured as a
`communication circuit including analog and/or digital com- 15
`ponents, such that communications device 120 is capable of
`transmitting and receiving short-range radio frequency
`("RF") signals in any of a variety of data transmission for(cid:173)
`mats, such as a Bluetooth communications protocol, an IEEE
`802.11 communications protocol or other personal area net- 20
`work wireless communications protocols or data formats.
`Digital media storage device 116 may include a plurality of
`communications device circuits, for example, a Bluetooth
`communications device circuit configured to communicate
`with communications device 120.
`In-vehicle control system 106 is also coupled to a user
`interface 126 and an audio input device 128. User interface
`126 may be used to receive input commands from a vehicle
`occupant via, for example, pushbuttons, switches, a keypad, a
`touch screen display, etc. Alternatively, input commands to
`in-vehicle control system 106 may include a set of audio
`signals from a vehicle occupant. For example, a vehicle occu(cid:173)
`pant may speak directly into the audio input device 128 to
`provide input commands to the in-vehicle control system 106.
`Audio input device 128 may include one or more audio input 35
`devices configured to receive an oral command from a vehicle
`occupant. The oral command may be any word or phrase that
`the occupant may speak, utter, or otherwise provide to cause
`the control system or another system to perform a function. A
`data processing module 122 in in-vehicle control system 106 40
`may be used to process various data signals, such as audio
`signals (e.g., oral input commands) received via audio input
`device 128 and to recognize words or phrases in the oral
`command. Voice recognition technologies known in the art
`may be implemented in data processing module 122. For 45
`example, data processing module 122 may comprise any
`speech recognition software or engine such as IBM Embed(cid:173)
`ded Via Voice®, manufactured by International Business
`Machines Corporation.
`In-vehicle control system 106 may also be coupled to an 50
`output display 108. Output display 108 may comprise a small
`cathode ray tube (CRT), liquid crystal display (LCD), or
`various other types of visual displays which are easily visible
`in various lighting conditions. In-vehicle control system 106
`may comprise one or more analog and/or digital electrical or 55
`electronic components, and may include a microprocessor, a
`microcontroller,
`application-specific
`integrated circuit
`(ASIC), programmable logic and/or other analog and/or digi-
`tal circuit elements configured to perform various input/out(cid:173)
`put, control, analysis and other functions described herein. 60
`In-vehicle control system 106 may also include a memory
`device 132, including volatile and non-volatile memory in
`order to, for example, store a computer program, other soft(cid:173)
`ware, or a data file to perform the functions described herein.
`Each element in vehicle 100 shown in FIG. 1 may be mounted 65
`in or coupled to the same or different interior vehicle elements
`such as a dashboard, an overhead console, an instrunient
`
`6
`panel, a visor, a rearview mirror, or other appropriate loca(cid:173)
`tions within the vehicle. Preferably, user interface 126, audio
`input device 128 and output display 108 are mounted in a
`location convenient to the vehicle occupants, in particular, the
`driver.
`Audio system 104 may include, for example,