AROAATA
`
`US 20070167187A1
`
`as) United States
`a2) Patent Application Publication (10) Pub. No.: US 2007/0167187 Al
`
` Rezvaniet al. (43) Pub. Date: Jul. 19, 2007
`
`
`(54) WIRELESS MULTIMEDIA HANDSET
`
`Publication Classification
`
`(76)
`
`Inventors: Behrooz Rezvani, San Ramon, CA
`(US); Andrea Goldsmith, Menlo Park,
`CA (US)
`
`(51)
`
`Int. Cl.
`(2006.01)
`HO4M 1/00
`(52) US. Che csecccsesssnncsecssesssnneessseessnnennsenece 455/550.1
`
`Correspondence Address:
`PERKINS COIE LLP
`P.O. BOX 2168
`MENLO PARK, CA 94026 (US)
`Appl. No:
`11/607,475
`Appl.
`No
`,
`Filed:
`Dee. 1, 2006
`
`21)
`(21)
`(22)
`
`Related U.S. Application Data
`
`A wireless multimedia handset can include one or more of
`wireless system support, a platform to handsetfeatures, and
`multiple features. The multiple features may include, by way
`of example but not limitation, multiple-antennae, multime-
`dia storage with advanced search capability, a high fidelity
`sound system, peer-to-peer networking capability, seamless
`handoff capability, instant hotspot capability, and ultra low
`(60) Provisional application No. 60/741,672, filed on Dec.|power operation such that the handsetis capable of opera-
`1, 2005.
`tion without recharging by operating solely on solarcells.
`
`(57)
`
`ABSTRACT
`
`100 a,
`
`Peer-to-Peer Networking and
`
`Audio Subsystem
`
`Microphone Array Processing
`130
`
`,
`
`132
`
`—
`MP3
`436
`
`.
`
`—_
`
`GSM/EDGE/3G and/or Wimax
`
`116
`
`Wifi (B02. 1 1.a/b/g/n) Module
`118
`
`108
`
`102
`
`104
`
`124
`Audio VO
`126
`Display
`128
`
`Applications
`106
`
`VoiP Module
`112
`
`802.15 (Bluetooth, Zigbee, UWB)
`Module
`420
`
`Antenna Algorithms Module
`422
`
`Power Management Engine
`
`1
`
`APPLE 1128
`APPLE1128
`
`1
`
`

`

`Patent Application Publication Jul. 19,2007 Sheet 1 of 12
`
`US 2007/0167187 Al
`
`
`
`(QAAN‘seqBiz‘yjoo}snig)S1°Z08
`
`
`
`
`
`ajnpow
`
`Oer
`
`
`
`
`
`SINPO-:suUWobyyeuuajuy
`
`cor
`
`Sir
`
`
`
`
`
`eulbugjueuwebeueysaModg
`
`30T
`
`lSls
`
`
`
`Bulsses0idAbiisuoYydoloI
`
`er
`
`uone||aoueD@sioN
`
`PEL
`
`cel
`
`ainpowdio
`
`cle
`
`[aeyMquINYLJOpegjoUuOD
`
`
`
`soeye}UjJas/)
`
`POT
`
`vel
`
`O/|Olpny
`
`Ocl
`
`Aeldsiq
`
`Ber
`
`suojeoyjddy
`
`90r
`
`XBUIAAJO/PUeHE/AOGA/WSD
`
`
`
`uoljuBooayyoseds
`
`188q-0}-188q
`
`
`
`
`
`
`
`ainpoywy(u/6q/eLL°Z08)YAN
`
`ainpo;
`
`OVE
`
`siNPOWOIPEdWX/INA/VSd
`
`weayshsqngoipny
`
`PLE
`
`OLE
`
`pueGupoMpN
`
`
`
`auibugyopuey
`
`cor
`
`2
`
`

`

`Patent Application Publication Jul. 19,2007 Sheet 2 of 12
`
`US 2007/0167187 Al
`
`200
`
`216
`
`202
`
`display
`
`keypad
`
`[TTT TTT 1
`j audio interface |
`\a a
`
`220
`234
` ialaiateeaate
`
`{{
`
`218
`
`WV
`
`|
`|
`|
`
`{|
`
`USB
`
`212
`
`210
`
`Microphone
`Array
`
`i\V/
`
`f 204
`Interface 226
`
`
`microprocessor CK)
`
`powersupplies
`
`baseband
`processor
`
`MIMO RF
`system
`
`VV
`|
`frDocsoee
`antenna interface
`
`DC
`
`222
`
`224
`
`battery/charger
`
`FIG. 2
`
`power
`management
`
`228
`
`
`
`3
`
`

`

`Patent Application Publication Jul. 19,2007 Sheet 3 of 12
`
`US 2007/0167187 Al
`
`300
`
`306
`
`front side
`
`302
`
`304
`back side
`
`
`cells
`
`310
`
`FIG. 3
`
`4
`
`

`

`Patent Application Publication Jul. 19,2007 Sheet 4 of 12
`
`US 2007/0167187 Al
`
`400
`
`Languagetraining
`
`Language Model
`(HMM)
`402
`
`
`Userinput to
`augment language
`model
`
`
`
`
`
`DSP Frontend
`Dictionary
`
`
`
`Translation
`for feature
`
`
`
`
`
`
`
`
`, Recognizer (HMM)KWirelessInterfacex Assistance
`
`extraction
`Engine
`406
`
`
`408
`7
`
`
`
`
`
`
`
`Extracted
`Acoustic signal
`metadata
`
`Index Engine
`410
`
`
`
`User entered
`metadata
`
`
`Search
`Action
`Criteria
`
`Search Engine
`412
`
`
`
`Dial a number
`
`
`
`Play content
`
`FIG. 4
`
`5
`
`

`

`Patent Application Publication Jul. 19,2007 Sheet 5 of 12
`
`US 2007/0167187 Al
`
`00S
`
`|
`
`80S
`
`
`
`uORe]]eoueDOY4I9
`
`JeAIo0a1
`
`----G4
`
`Aywenbysveds
`
`yUswaoueyue
`
`urewopAsuenbay
`
`Jaoueyueasiou
`
`G‘Sls
`
`N-OLS
`
`n-gog—/©95
`
`punosBbyaeq
`
`
`
`Ja]]a0uedasiou
`
`
`
`Aewesuoydosoiw
`
`v0S
`
`
`
`asiouyuarquie
`
`suoydossiu
`
`—eeeeeeeee
`
`6
`
`
`
`
`

`

`Patent Application Publication Jul. 19,2007 Sheet 6 of 12
`
`US 2007/0167187 Al
`
`SS600
`
`beam2
`
`
`noise canceller
`
`
`background
`
`
`
`ambient noise
`microphone
`
`FIG. 6
`
`7
`
`

`

`Patent Application Publication Jul. 19,2007 Sheet 7 of 12
`
`US 2007/0167187 Al
`
`700 —~»«,
`
`702
`
`User inputs a requestto initiate a
`search for one or morefiles
`
`704
`
`Search engine queries user for
`
`desiredfile type
`
`706
`
`Userinputs type offile(s) desired
`
`Start
`
`(A)
`
`708
`
`718
`
`Search engine queries userfor
`search term(s) associated with file
`types
`
`710
`
`Search engine scanslibrary forfiles
`matchingfile type and search terms
`
`Terminate search
`
`No
`
`716
`
`Yes
`
`5
`Changesearch term(s)’
`
` Search engine asks user whether
`
`714
`
`search term(s) are to be changed
`
`Yes
`
`722
`
`
`
`
`.
`.
`File sent over appropriate user
`interface on handset
`
`726
`
`Files sent over appropriate user
`nterface on handset
`
`.i
`
`
`End
`
`More than one match?
`
`Yes
`
`Requested more than one?
`
`No
`
`(A)
`
`720
`
`N
`
`0
`
`
`724
`
`Y
`
`
`
`FIG. 7
`
`8
`
`

`

`Patent Application Publication Jul. 19,2007 Sheet 8 of 12
`
`US 2007/0167187 Al
`
`800
`
`display/audio
`memory
`
`DSP &
`processor
`
`analog/RF
`power amp
`
`|
`Ir projected
`l
`consummation
`
`advanced power management
`
`== ua 0o>
`algorithms
`
`
`
`
`multi-antenna
`
`
`pwr optimization
`
`
`mostcircuits
`
`shutting down
`most functions
`
`shutting down
`
`9
`
`

`

`Patent Application Publication Jul. 19,2007 Sheet 9 of 12
`
`US 2007/0167187 Al
`
`S900
`
`VoIP call
`handoff
`
`
`
`
`
`FIG. 9
`
`10
`
`

`

`Patent Application Publication Jul. 19,2007 Sheet 10 of 12
`
`US 2007/0167187 Al
`
`1000
`
`VVV¥
`
`Ly
`
`caméra
`
`game
`console
`
`FIG. 10
`
`11
`
`11
`
`

`

`Patent Application Publication Jul. 19,2007 Sheet 11 of 12
`
`US 2007/0167187 Al
`
`1100—»
`
`1102
`
`Anew node broadcasts a request to
`join a peer-to-peer network
`
` 1106
`
`Neighbor hears request?
`
`The new node tries a different
`interface and/or waits an
`implementation-specific time
`
`Yes
`
`1108
`
`Neighboring node establishes
`connection with the new node
`
`1110
`
`Neighboring node exchanges
`information about existing network
`with the new node
`
`1112
`
`Neighboring nodeinforms other
`nodes on the peer-to-peer network
`about the new node
`
`1114
`
`The new node becomesa part of
`the established network and
`enables routing to other nodes
`
`FIG. 11
`
`12
`
`12
`
`

`

`Patent Application Publication Jul. 19,2007 Sheet 12 of 12
`
`US 2007/0167187 Al
`
`so
`
`
`
`13
`
`

`

`US 2007/0167187 Al
`
`Jul. 19, 2007
`
`WIRELESS MULTIMEDIA HANDSET
`
`BACKGROUND
`
`
`
`[0001] Wireless handsets are common devices used in
`communication systems such as cell phones, PDAs, and
`VoIP phones. Such handsetstypically include radio technol-
`ogy to access a given wireless system. For example, a cell
`phone handset must include wireless technology to access
`one of the cellular system standards, and a VoIP handset
`nust include wireless technology that can access the Inter-
`et.
`
`[0002] The foregoing examples of the related art and
`limitations related therewith are intended to beillustrative
`and not exclusive. Other limitations of the related art will
`
`become apparent to those of skill in the art upon a reading
`of the specification and a studyof the drawings.
`
`SUMMARY
`
`[0003] The following embodiments and aspects thereof
`are described and illustrated in conjunction with systems,
`tools, and methods that are meant to be exemplary and
`illustrative, not limiting in scope. In various embodiments,
`one or more of the above-described problems have been
`reducedor eliminated. while other embodimentsare directed
`to other improvements.
`
`[0004] A wireless multimedia handset can include one or
`more of wireless system support, a platform to handset
`features, and multiple features. The wireless system support
`mayinclude support for one or more of Wifi (802.11a/b/g/n),
`
`Wimax, 3G cellular, 2G cellular, GSM-EDGE,radio (e.g.
`AM/FM/XM), 802.15 (Bluetooth, UWB, and Zigbee) and
`GPS. Theplatform to handset features may include provid-
`ing a platform such that common handsetapplications (such
`as a camera capability) as well as third party applications
`such as gaming) can access one or more handset features.
`The multiple features may include, by way of example but
`ot limitation, multiple-antennae, multimedia storage with
`advanced search capability, a high fidelity sound system,
`peer-to-peer networking capability, seamless handoff capa-
`bility, instant hotspot capability, and ultra low power opera-
`ion such that the handset is capable of operation without
`recharging byoperating solely on solar cells.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`[0005] Embodiments of the inventions are illustrated in
`he figures. However,
`the embodiments and figures are
`illustrative rather than limiting; theyprovide examplesof the
`invention.
`
`FIG. 1 depicts an example of a multimedia handset
`[0006]
`device that has ultra-low power requirements.
`
`FIG. 2 depicts examples of subsystems of a mul-
`[0007]
`imedia handset device.
`
`[0008]
`
`FIG. 3 depicts a conceptual diagram of a handset.
`
`FIG. 4 depicts a flowchart of an example of a
`[0009]
`nethod for operating a handset with speech recognition
`capabilities.
`
`FIG. 5 depicts an example of a microphone array
`[0010]
`system 500 for a handset.
`
`
`
`FIG. 6 depicts an example ofsteering a microphone
`[0011]
`array towards a person that is speaking or towards multiple
`people speaking.
`TI
`
`FIG. 7 depicts a flowchart of an example of a
`[0012]
`method for locating a file or set offiles.
`
`FIG. 8 depicts an example of details of a power
`[0013]
`management algorithm embodied in a computer-readable
`medium.
`
`FIG. 9 depicts a conceptual diagram of seamless
`[0014]
`handoff of a VoIP call between a cellular and Wifi system.
`TI
`
`FIG. 10 depicts peer-to-peer networking to estab-
`[0015]
`lish direct or multihop connections with other wireless
`devices for real-time interaction and file exchange.
`
`FIG. 11 depicts a flowchart of an example of a
`[0016]
`method for neighbor discovery and joing a peer-to-peer
`network.
`
`TI
`
`
`
`
`
`TI
`
`
`FIG. 12 depicts a system that includes simulta-
`[0017]
`
`
`neous operation over different frequencies.
`
`
`DETAILED DESCRIPTION
`
`
`
`Inthe following description, several specific details
`[0018]
`are presented to provide a thorough understanding of
`embodimentsof the invention. Oneskilled in the relevantart
`will recognize, however, that the invention can bepracticed
`without one or moreofthe specific details, or in combination
`with other components,etc. In other instances, well-known
`implementationsor operations are not shownor described in
`detail to avoid obscuring aspects of various embodiments, of
`the invention.
`
`
`
`FIG. 1 depicts an example of a multimedia handset
`[0019]
`device 100 that has ultra-low power requirements. The
`device maybe used in any applicable known or convenient
`wireless network, and
`may be used in conjunction with a
`wireless server, such as the wireless server described in U.S.
`application Ser. No.
`, filed concurrently herewith
`entitled WIRELESS MEDIA SERVER SYSTEM AND
`
`METHODbyRezvani et al., and which is incorporated by
`reference. In the example of FIG. 1, the device 100 includes
`a peer-to-peer networking and handoff engine 102, a user
`interface 104, applications 106, a power management engine
`108, an audio subsystem 110, a VoIP module 112, a GPS
`
`AM/FM/XM radio module 114, a GSM/EDGE/3G and/or
`Wimax module 116, a Wifi (802.11a/b/g/n) module 118, an
`802.15 (Bluetooth, Zigbee, UWB) module 120, and an
`antennae algorithms module 122. The user interface 104
`includes a control pad or thumbwheel 124, audio I/O 126,
`and a display 128. The audio subsystem 110 includes a
`microphonearray processing engine 130, a speech recogni-
`tion engine 132, a noise cancellation engine 134, and an
`MP3engine 136.
`
`Inthe example of FIG.1, multiple wireless systems
`[0020]
`maybe integrated into the device, including various cellular
`phone standards (3G/2G/GSM/Edge), different Wifi stan-
`dards (802.11a/b/g/n), Wimax, 802.15 (Bluetooth, Zigbee,
`and/or UWB), and GPS. In an embodiment, the handset 100
`has the capability to support Voice over IP (VoIP) through
`anyof the interfaces that allow it to connectto the Internet.
`In addition, the handset 100 will have a peer-to-peer net-
`working capability that will allow any handset within range
`of other wireless devices to self-configure with them into a
`
`14
`
`14
`
`

`

`nultihop network. In an embodiment, seamless handoff of a
`given application (e.g. VoIP or music streaming) between
`different wireless interfaces is also supported.
`
`In general, the handset 100 could be implemented
`[0021]
`as any device capable of receiving and using content. For
`example, the handset could include, by way of example but
`ot limitation, a PDA, cell phone, smart phone, etc. Content
`nay include audiofiles, multimediafiles, software applica-
`ions, or any other content that is capable of playback at a
`device. Strictly speaking, any given handset maynotbe able
`o play content, though for the purposes of this application,
`which is regarding handsets, content
`is assumed to be
`playable on the handset.
`
`In an embodiment, the handset 100 can include one
`[0022]
`or moreofmultiple features such as multimedia storage with
`advanced search capability, a high fidelity sound system,
`peer-to-peer networking capability, and ultra low power
`operation such that
`the device is capable of operation
`without recharging by operating solely on solar cells. The
`andset can also support one or more wireless systems and
`provide a platform such that common handset applications
`such as a camera capability) as well as third party appli-
`cations (such as gaming) can access the devicefeatures.
`
`[0023] As used herein, algorithmic descriptions within a
`computer memoryare believed to most effectively convey
`he techniques to others skilled in the art. An algorithm is
`ere, and generally, conceived to be a self-consistent
`sequence of operations leading to a desired result. The
`operations are those requiring physical manipulations of
`physical quantities. Usually, though not necessarily, these
`quantities take the form ofelectrical or magnetic signals
`capable of being stored, transferred, combined, compared,
`and otherwise manipulated. It has proven convenient at
`imes, principally for reasons of common usage,to refer to
`hese signals as bits, values, elements, symbols, characters,
`erms, numbers, orthe like.
`
`
`
`US 2007/0167187 Al
`
`It should be borne in mind, however, that all of
`[0024]
`hese and similar terms are to be associated with the appro-
`priate physical quantities and are merely convenientlabels
`applied to these quantities. Unless specifically stated other-
`wise as apparent from the following discussion,it is appre-
`ciated that throughout the description, discussions utilizing
`ermssuchas “processing”or “computing”or “calculating”
`or “determining” or “displaying” or the like, refer to the
`action and processes of a computer that manipulates and
`ransforms data represented as physical (electronic) quanti-
`ies within the computer system’s registers and memories
`into other data similarly represented as physical quantities
`within the computer system memoriesor registers or other
`such information storage, transmission or display devices.
`
`[0025] The algorithms and displays presented herein are
`not inherently related to any particular computer architec-
`ture. The techniques may be implemented using any known
`or convenient programming language, whether high level
`(e.g., C/C++) or lowlevel (e.g., assembly language), and
`whetherinterpreted (e.g., Perl), compiled (e.g., C/C++), or
`Just-In-Time (JIT) compiled from bytecode (e.g., Java). Any
`known or convenient computer, regardless of architecture,
`should be capable of executing machine code compiled or
`otherwise assembled from any language into machine code
`that is compatible with the computer’s architecture, includ-
`ing that of embedded systems,if applicable.
`
`Jul. 19, 2007
`
`FIG. 2 depicts examples of subsystems of a mul-
`[0026]
`timedia handset device 200. In the example of FIG. 2, the
`device 200 includes a display 202, a microprocessor 204,
`memory206, a power module 208, a keypad 210, a base-
`band processor 212, a SIM 214, an audio module 216, a
`MIMORFsystem 218, and an antenna interface 220. The
`power module 208 includesa battery/charger 222, solar cells
`224, a power management engine 226, and power supplies
`228. The audio module 216 includes a microphone array
`230, a speaker 232, and a receiver 234.
`
`[0027] An antennaor array of antennae is used as part of
`the wireless system or subsystems; signals received from
`this antenna array are processed by an RF signaling block
`and a baseband block; power is supplied via DC power
`and/or throughsolar cell recharging; the device has memory
`and a USB interface;
`the user interface is via a display,
`keypad (or thumbwheel), and audio input/output from a
`microphonearray.
`
`FIG. 3 depicts a conceptual diagram of a handset
`[0028]
`300. A front side 302 of the handset 300 includes a control
`
`pad 306. A back side 304 of the handset 300 includes solar
`cells 308 and a speaker 310. The solar cells can charge the
`battery from ambientlight as well as solar light. Inside the
`device are the electronics for supporting the handset features
`(e.g. RF, analog, mixed-signal, and digital electronic cir-
`cuitry, microprocessors, antennas,etc.). The user interface to
`the device may be through speech recognition of commands,
`through a control pad, and/or through another type of
`interface such as a thumbwheel for menu navigation.
`
`FIG. 4 depicts an example of a system 400 for
`[0029]
`providing speech recognition capabilities to a handset. In the
`example of FIG. 4, the system 400 includes a language
`model 402. The language mayinclude, by way of example
`but not limitation, a set of hidden Markov models (HMMs)
`for speech data. The language model may be enhanced
`through language training. For example, a one-time-per-
`language training could be used for 500 common words.
`Any known or convenient technique may be implemented
`for the purposes of initial (or later) language training. In
`addition to language training, user input can be used to
`augmentthe language model 402. For example, a user could
`enter specific vocabulary, such as proper names,either one
`at a time, or captured as a user enters commandsor speaks
`to a voice application.
`
`Inthe example of FIG. 4, the system 400 includes
`[0030]
`a speech recognizer (HMM) 404. The speech recognizer
`may embody on a computer-readable medium a speech
`recognition algorithm that uses the HMMs. The speech
`recognizer 404 will take as input a user’s speech. In an
`embodiment, the speech recognizer 404 is designed and/or
`configured such that wireless interfaces and/or peer-to-peer
`networking capability can be used to provide additional
`input to the algorithm. Specifically, the speech recognizer
`404 has the ability to use any of the available wireless
`interfaces and/or the peer-to-peer network to connect to
`another device such as a laptop, computer, or another
`handset to expand the vocabulary base, provide translation
`assistance, or access a more powerful speech recognition
`algorithm or set of HMMs.
`
`In the example of FIG. 4, the speech recognizer
`[0031]
`404 is coupled to a DSP frontendfor feature extraction 406,
`which receives an acoustic signal as input. Noise cancella-
`
`15
`
`15
`
`

`

`US 2007/0167187 Al
`
`Jul. 19, 2007
`
`Sirtus) or AM/FM radioreceiver, via a USB high-speed data
`port, or via a wired or wireless connection to another device
`(e.g. a wireless connection to a computer, music server,
`video server, or digital camera). The library maybeparti-
`tioned by mediatype, for example there may beone partition
`of the memoryfor photos, one for video clips, one for music,
`one for phone numbers, etc. File storage will include the
`capability to add “tags” to files. The tagging is done to
`facilitate searching based on tags that the user selects for
`each media type. For example, a music file might havea tag
`or tags such asfile title, song title, artist, keywords, genre,
`album name, music sample orclip, etc. A video file might
`have a tag suchasfile title, videotitle, date, subject, location,
`artist, etc. A photo file might havea tag suchaspicturetitle,
`date, subject, location, etc.
`
`tion and beamforming algorithmscan be designedrelative to
`the speech recognition algorithm such that
`the feature
`extraction of the input speech is optimized (note that noise
`cancellation/beamforming algorithms designed independent
`of the speech recognition algorithms can degrade speech
`recognition performance by introducing undesired speech
`artifacts).
`
`In the example of FIG. 4, the speech recognizer
`[0032]
`404 is coupled to a dictionary translation assistance engine
`408 via a wireless interface. Since a handset may not have
`as much powerorstorage as a laptop, if a laptop is nearby,
`it can be used to provide translation assistance. Any appro-
`priately configured known or convenient device capable of
`communicating with the speech recognizer 404 should be
`able to provide dictionary translation assistance.
`
`In the example of FIG. 4, the speech recognizer
`[0033]
`404 may categorize recognized speech aseither a command
`(e.g. dial a specific number) or a query (e.g.
`look for a
`particular content file). Both commandsare provided to an
`index engine 410. The index engine 410 may also make use
`of metadata, such as metadata that is extracted from, by way
`of example but not limitation, musicfiles, video files, game
`files, contact info, and the like, or metadata that is entered by
`a user, such as personal photos. For commands the index
`engine 410 may or maynot use the metadatato determine an
`appropriate course ofaction, then take the appropriate action
`to satisfy the command. For queries the index engine 410
`mayor may not use the metadata to generate searchcriteria
`that is (hopefully) sufficient to enable a search engine 412 to
`locate a desired contentfile, then take the appropriate action
`(such as playing the contentfile).
`
`In an embodiment, the handset includesintelligent
`[0036]
`software for searching multimediafiles stored on the handset
`based on multiple search criteria and by the type offile of
`interest. Alternatively, a user can set up certain tags forall
`pictures taken under
`the given tagging criterion. For
`example, when using a cell phone with a camera, the user
`maydefine a certain tag or set of tags for all pictures being
`taken (e.g. “kids”, “spouse”, “vacation to Europe 2006”,
`etc.). The user need only enter this tag or set of tags once,
`and then change the tag or tags when a change is desired.
`When the picture is taken, the image will be stored in the
`memoryof the handset with the associated tags attached to
`the file. Then, after this tagging process is set up, the user
`can tag all pictures without any additional manual inputs
`(e.g.
`typing or thumbwheel
`inputs). This is particularly
`useful for a handset sinceit is relatively hard to do manual
`entry for each new file on a small-sized device. The same
`concept can also used for tagging and searching for music on
`FIG. 5 depicts an example of a microphone array
`[0034]
`the handset, whereaset of tags can be assignedto all music
`system 500 for a handset. The handset will feature a high
`downloaded.
`fidelity sound system for picking up voice signals by uti-
`lizing a microphonearray coupled with a noise cancellation
`algorithm embodied in a computer-readable medium.In an
`embodiment, a microphone array 502 includes one ambient
`oise microphone 504 located on a part of the handset
`optimized for it to pick up background ambient noise, as
`well as additional microphone elements 506-1 to 506-N
`referred to collectively as microphone elements 506) in
`different locations on the handset. Echo cancellation 508 is
`
`
`
`In an embodiment, a search engine (SE) may
`[0037]
`implement a search algorithm that has been embodied in a
`computer-readable medium. The algorithm may include a
`multistep process to locate a file or set of files of interest.
`This generalized search engine may re-use a number of
`similar functions for different kind of searches such as
`
`
`
`speech recognition, image recognition, and music recogni-
`tion. The SE interacts with the user through the user inter-
`face, which for example can be a control pad, thumbwheel,
`or via voice. In the case of voice commands, the handset
`synthesizes a voice signal to query the user, and the user’s
`voice commands are processed by a voice recognition
`engine and then sent to the SE. The noise cancellation and
`beamforming capabilities of the microphonearray, shown in
`FIG. 5, can be combined with a speech recognition engine
`to improve its performance.
`
`
`
`sed for signals transmitted from the handset to remove echo
`effects. In an embodiment, antenna elements 510-1 to 510-N
`referred to collectively as antenna elements 510) in the
`
`nicrophone array 502 have weights assigned to each
`
`
`
`antenna input. Different algorithms can be used to determine
`he weights, depending on the performance criteria,
`the
`umber of antenna elements available and their nature, and
`he algorithm complexity. For example, the weights may be
`sed to minimize ambient noise, to make the antennaarray
`FIG. 7 depicts a flowchart 700 of an example of a
`[0038]
`gain independent of frequency, to minimize the expected
`method for locatingafile or set of files. This method and
`nean square distortion orerror of the signal, or to steer the
`other methods are depicted as serially arranged modules.
`direction of the microphone array towards a person that is
`However, modules of the methods may be reordered, or
`speaking or towards multiple people speaking, as shownin
`FIG. 6.
`arranged for parallel execution as appropriate.
`In the
`example of FIG, 7, a user first initiates a search by sending
`
`a command to the SE. The SE then queries the user as to
`what type of media is desired from a given list of types,
`which may include video, music, photos, etc. Determining
`the type of media requested will narrow the search space to
`a particular partition of the multimedia storage library. Once
`the file type is identified, the SE will have a set of search
`
`Inan embodiment, the handset includes nonvolatile
`[0035]
`storage for multimedia data files, for example through a
`Flash RAM. There are many methods by which the multi-
`mediadata files may be loaded into the handset memory, for
`example via a wireless connection to the Internet, via a
`cellular telephone connection, via a satellite (e.g. XM or
`
`16
`
`16
`
`

`

`US 2007/0167187 Al
`
`Jul. 19, 2007
`
`criteria associated with that type. The search engine can
`utilize the tags users input to labelthe files as well as other
`data associated with the files. For example, a search for a
`musicfile may be by usertag or tag fragment,file title, song
`title, album name,artist name, keywords, lyrics, a hummed
`tune, popularity, similarity to other music, genre, etc.
`
`In the example ofFIG.7, the flowchart 700 begins
`[0039]
`at module 702 where a user inputs a request to initiate a
`search for one or morefiles. In the example of FIG. 7, the
`flowchart 700 continues to module 704 where a search
`engine queries the userfor a desiredfile type. In the example
`of FIG.7, the flowchart 700 continues to module 706 where
`the user inputs the typeoffile(s) desired. In the example of
`FIG.7, the flowchart 700 continues to module 708 where the
`search engine queries the user for search term(s) associated
`with the desiredfile type(s). In the example of FIG. 7, the
`flowchart 700 continues to module 710 where the search
`engine scansa library forfiles matching thefile type(s) and
`search term(s).
`
`In the example of FIG. 7, the flowchart 700 con-
`[0040]
`tinues to decision point 712 where it is determined whether
`there are more than one matches. If not (712-N), then the
`flowchart 700 continues to module 714 where the search
`engine asks the user whether the search term(s) are to be
`changed. In the example of FIG. 7,
`the flowchart 700
`continues to decision point 716 where it
`is determined
`whether the search term(s) is/are to be changed. If so
`(716-Y), then the flowchart 700 continues from module 710
`as described previously. If not (716-N), then the flowchart
`700 continues to module 718 where the search terminates
`and the flowchart 700 ends.
`
`relatively low power can be maintained for an indefinite
`time period under solar power alone,e.g., solar cells embed-
`ded in the device and aggressive power management will
`allow the device to support the given application(s) indefi-
`nitely without recharging by shutting downall nonessential
`functions except those associated with the specific applica-
`tion or applications. For example, the device may operate
`indefinitely without recharging in Wifi-only mode byshut-
`ting downall functions in the device (e.g. certain display
`features, memory access, audio processing, noise cancella-
`tion, and search algorithms) not associated with maintaining
`a low-rate Wifi connection to the Internet through one or
`more interfaces that support this connection (e.g. 802.11a/
`b/g/n); in voice-only mode the device may operate indefi-
`nitely without recharging by shutting down all functionality
`of the device not associated with making a voice call (e.g.
`certain display features, memory access, audio processing,
`noise cancellation, and search algorithms) through one or
`moreinterfaces that support such calls (e.g. 2G, 3G, GSM,
`VoIP over Wifi), etc.
`
`
`
`[0041] Returning once again to decision point 712,ifit is
`determined that there are more than one matches (712-Y),
`then the flowchart 700 continues to decision point 720 where
`it is determined whether there is more than one match.If not
`
`In an embodiment, the handset supports simulta-
`[0044]
`neous operation on the different wireless interfaces,
`i.e.
`simultaneous operation on at least two systems that may
`include Wifi (802.11a/b/g/n), Wimax, 3G cellular, 2G cel-
`lular, GSM-EDGE, radio (e.g.AM/FM/XM), 802.15 (Blue-
`tooth, UWB, and Zigbee) and GPS. These systems often
`
`operate at different frequencies. The simultaneous operation
`
`over different frequencies can be done, for example, by
`using someset of antennasfor one system and using another
`set of antennas for another system. This is illustrated in FIG.
`9 for simultaneous operation over a cellular system and a
`Wifi system. Another mechanism to support this simulta-
`neous multifrequency operation is timedivision. In addition
`to simultaneous operation, the handset can support seamless
`handoff between two systems. For example,
`the handset
`could switch a VoIP call from a wide-area wireless network
`(720-N), then the flowchart 700 continues to module 722
`such as Wimax or 3G toalocal area network such as Wifi.
`
`
`where the matching file is sent over an appropriate user
`
`
`FIG. 9 depicts a conceptual diagram of seamless handoff of
`interface on the handset and the flowchart 700 ends. If, on
`a VoIP call between a cellular and Wifi system.
`the other hand, it is determined that there are more than one
`matches (720-Y), then the flowchart 700 continues to deci-
`sion point 724 where it
`is determined whether the user
`requested more than onefile. If so (724-Y), then the flow-
`chart 700 continues to module 726 where the matching files
`are sent over an appropriate user interface on the handset,
`and the flowchart ends. If not (724-N), then the flowchart
`700 continues from module 714 as described previously.
`
`
`
`[0042] As shownin the example of FIG.2, the handset has
`a power managementalgorithm, embodied in a computer-
`readable medium,that minimizes power consumption based
`on the usage of the handset. FIG. 8 depicts an example of
`details of a power management algorithm embodied in a
`computer-readable medium. As shown in FIG. 8, compo-
`nents of the power managementalgorithm include managing
`power consumption associated with display, audio, memory,
`DSP, and/or processors to be minimized while supporting
`the applicationsin use; utilizing multiple antennas (MIMO)
`in the mostefficient way to minimize the power consump-
`tion required for wireless transmission; shutting down cer-
`tain nonessential device functionality, and turning off non-
`essential device circuitry.
`
`In an embodiment, the handset is designed such
`[0043]
`that a certain application or set of applications that require
`
`
`
`FIG. 10 depicts peer-to-peer networking to estab-
`[0045]
`lish direct or multihop connections with other wireless
`devices for real-time interaction and file exchange. A peer-
`to-peer networking protocol can make use ofall wireless
`interfaces that can establish a direct connection with other
`
`wireless devices. For example,it could use an 802.1 la/b/g/n
`interface operating in peer-to-peer mode, an 802.15 inter-
`face, a proprietary peer-to-peer radio interface, and/or an
`infrared communication link. The user mayselect to estab-
`lish peer-to-peer networks on all available interfaces simul-
`taneously, on a subset of interfaces, or on a single interface
`based on a prioritized list of possible interfaces.
`
`[0046] Alternatively, a peer-to-peer network may beestab-
`lished based ona list or set oflists of specific devices or user
`IDs that the user wishesto interact with. For example,a user
`mayset up a list of gaming partners (or multiple lists, each
`associated with a different game or partner preferences),
`such that whenever a device or user assoc