1111111111111111 IIIIII IIIII 11111 1111111111 111111111111111 IIIII IIIII 111111111111111 11111111
`US 20090164219Al
`
`c19) United States
`c12) Patent Application Publication
`Yeung et al.
`
`c10) Pub. No.: US 2009/0164219 Al
`Jun. 25, 2009
`(43) Pub. Date:
`
`(54) ACCELEROMETER-BASED CONTROL OF
`WEARABLE DEVICES
`
`(75)
`
`Inventors:
`
`King-Wah Walter Yeung, Davis,
`CA (US); Wei-Wei Vivian Yeung,
`Davis, CA (US)
`
`Correspondence Address:
`Enbiomedic
`P.O. Box 407
`Davis, CA 95617-0407 (US)
`
`(73) Assignee:
`
`ENBIOMEDIC, Davis, CA (US)
`
`(21) Appl. No.:
`
`12/337,869
`
`(22) Filed:
`
`Dec.18,2008
`
`Related U.S. Application Data
`
`(60) Provisional application No. 61/008,504, filed on Dec.
`19, 2007.
`
`Publication Classification
`
`(51)
`
`Int. Cl.
`(2006.01)
`GJ0L 19100
`(2006.01)
`G08B 21/00
`(52) U.S. Cl. ... 704/258; 704/500; 340/669; 704/E13.002
`ABSTRACT
`(57)
`
`Accelerometer-based orientation and/or movement detection
`for controlling wearable devices, such as wrist-worn audio
`recorders and wristwatches. A wrist-worn audio recorder can
`use an accelerometer to detect the orientation and/or move(cid:173)
`ment of a user's wrist and subsequently activate a correspond(cid:173)
`ing audio-recorder function, for instance recording or play(cid:173)
`back. A wearable device with a vibration mechanism can use
`this method to remind a user of an undesirable movement
`such as restless leg movement. Likewise, a talking wristwatch
`can use this method to activate audio reporting of time when
`a user moves or orients his or her wrist in close proximity to
`his or her ear. In such applications, and many others, accel(cid:173)
`erometer-based control of the wearable device offers signifi(cid:173)
`cant advantages over conventional means of control, particu(cid:173)
`larly in terms of ease of use and durability.
`
`' '
`'
`
`'
`'
`
`'
`
`'
`
`/
`
`/
`
`/
`
`/
`
`/
`
`/
`
`G
`
`Petitioner Samsung Ex-1005, 0001
`
`

`

`Patent Application Publication
`
`Jun. 25, 2009 Sheet 1 of 14
`
`US 2009/0164219 Al
`
`FIG. 1
`
`FIG. 2
`
`33
`
`30
`
`31
`
`Petitioner Samsung Ex-1005, 0002
`
`

`

`Patent Application Publication
`
`Jun. 25, 2009 Sheet 2 of 14
`
`US 2009/0164219 Al
`
`32
`42
`X-Axis 1------1•1 Analog
`Signal
`Multiplexer
`
`44
`
`46
`
`Sample(cid:173)
`And-Hold
`Circuit
`
`Analog-To(cid:173)
`Digital
`Converter
`
`Accele(cid:173)
`rometer
`
`Y-Axis
`Signal
`
`Z-Axis
`Signal
`
`50
`
`48
`
`Accelerometer(cid:173)
`Data
`Memory
`
`Recorder
`Controller
`
`62
`
`Sample(cid:173)
`And-Hold
`Circuit
`
`Analog-To- - - (cid:173)
`Digital
`Converter
`
`60
`
`Communi(cid:173)
`cation Port
`
`68
`
`User
`Interface
`
`36
`
`38
`
`Microphone Speaker
`
`64
`
`66
`
`Digital-To(cid:173)
`Analog
`Converter
`
`52
`
`Real-Time System
`Clock
`Clock
`
`FIG. 3
`
`Petitioner Samsung Ex-1005, 0003
`
`

`

`Patent Application Publication
`
`Jun. 25, 2009 Sheet 3 of 14
`
`US 2009/0164219 Al
`
`Processor
`
`I
`
`Earphone
`
`Audio
`
`Wrist(cid:173)
`band
`
`31
`
`FIG. 4
`
`Petitioner Samsung Ex-1005, 0004
`
`

`

`Patent Application Publication
`
`Jun. 25, 2009 Sheet 4 of 14
`
`US 2009/0164219 Al
`
`START
`
`74
`
`SELECT A SIGNAL COMPONENT
`(X, Y, OR Z) BY
`ANALOG MULTIPLEXER
`
`76
`
`SAMPLE AND HOLD
`ANALOG SIGNAL COMPONENT
`
`78
`
`CONVERT ANALOG SIGNAL
`COMPONENT TO DIGITAL DATUM
`
`80
`
`STORE DIGITAL DATUM
`IN ACCELEROMETER-DATA
`MEMORY
`
`WAIT FORA
`PREDETERMINED
`TIME INTERVAL
`
`88
`
`NO
`
`COMPUTE ORIENTATION AND/OR MOVEMENT
`TO ACTIVATE AN AUDIO-RECORDER FUNCTION
`ONLY WHEN A PREDETERMINED ORIENTATION
`AND/OR MOVEMENT IS DETECTED
`
`NO
`
`END
`
`FIG. 5
`
`Petitioner Samsung Ex-1005, 0005
`
`

`

`Patent Application Publication
`
`Jun. 25, 2009 Sheet 5 of 14
`
`US 2009/0164219 Al
`
`' ' ' ' ' ' '
`
`'
`'
`
`,,
`
`,,
`
`,,
`
`,,
`
`,,
`
`,,
`
`,,
`
`'
`'
`
`/
`
`G
`
`FIG. 6
`
`Petitioner Samsung Ex-1005, 0006
`
`

`

`Patent Application Publication
`
`Jun. 25, 2009 Sheet 6 of 14
`
`US 2009/0164219 Al
`
`31
`
`/
`
`/
`
`/
`
`/
`
`/
`
`/
`
`/
`
`FIG. 7
`
`Petitioner Samsung Ex-1005, 0007
`
`

`

`Patent Application Publication
`
`Jun. 25, 2009 Sheet 7 of 14
`
`US 2009/0164219 Al
`
`FIG. 8
`
`Petitioner Samsung Ex-1005, 0008
`
`

`

`Patent Application Publication
`
`Jun. 25, 2009 Sheet 8 of 14
`
`US 2009/0164219 Al
`
`170,
`~---~-----------------------------------------
`178
`180
`172
`176
`X-Axis 1-----1•1
`Analog
`Signal
`Multiplexer
`
`Sample(cid:173)
`And-Hold
`Circuit
`
`Analog-To(cid:173)
`Digital
`Converter
`
`Accele(cid:173)
`rometer
`
`Y-Axis
`Signal
`
`Z-Axis
`Signal
`
`198
`
`182
`
`174
`
`Accelerometer(cid:173)
`Data
`Memory
`
`Recorder
`Controller
`
`186
`
`Sample(cid:173)
`And-Hold
`Circuit
`
`184
`
`192
`
`Microphone Speaker
`
`Analog-To-14-----t
`Digital
`Converter
`
`188
`
`191
`
`200
`
`User
`Interface
`
`194
`
`196
`
`Digital-To(cid:173)
`Analog
`Converter
`
`Real-Time System
`Clock
`Clock
`
`FIG. 9
`
`Petitioner Samsung Ex-1005, 0009
`
`

`

`Patent Application Publication
`
`Jun. 25, 2009 Sheet 9 of 14
`
`US 2009/0164219 Al
`
`START
`
`210 SELECT A SIGNAL COMPONENT
`(X, Y, OR Z) BY
`ANALOG MULTIPLEXER
`
`212
`
`SAMPLE AND HOLD
`ANALOG SIGNAL COMPONENT
`
`214
`
`CONVERT ANALOG SIGNAL
`COMPONENT TO DIGITAL DATUM
`
`216
`
`STORE DIGITAL DATUM
`IN ACCELEROMETER-DATA
`MEMORY
`
`WAIT FORA
`PREDETERMINED
`TIME INTERVAL
`
`224
`
`NO
`
`COMPUTE ORIENTATION AND/OR MOVEMENT TO PLAY
`BACK A REMINDING AUDIO MESSAGE ONLY WHEN A
`PREDETERMINED ORIENTATION AND/OR MOVEMENT
`15 DETECTED A PREDETERMINED NUMBER OF TIMES
`
`NO
`
`END
`
`FIG. 10
`
`Petitioner Samsung Ex-1005, 0010
`
`

`

`Patent Application Publication
`
`Jun. 25, 2009 Sheet 10 of 14
`
`US 2009/0164219 Al
`
`FIG. 11
`
`FIG. 12
`
`Petitioner Samsung Ex-1005, 0011
`
`

`

`Patent Application Publication
`
`Jun. 25, 2009 Sheet 11 of 14
`
`US 2009/0164219 Al
`
`334 341b
`
`FIG. 13
`
`331
`
`FIG. 14
`
`Petitioner Samsung Ex-1005, 0012
`
`

`

`Patent Application Publication
`
`Jun. 25, 2009 Sheet 12 of 14
`
`US 2009/0164219 Al
`
`332
`342
`X-Ax is 1------1•1
`Signal
`
`Analog
`Multiplexer
`
`344
`
`346
`
`Sample(cid:173)
`And-Hold
`Circuit
`
`Analog-To(cid:173)
`Digital
`Converter
`
`Accele(cid:173)
`rometer
`
`Y-Axis
`Signal
`
`Z-Axis
`Signal
`
`350
`
`348
`
`Accelerometer(cid:173)
`Data
`Memory
`
`362
`
`Sample(cid:173)
`And-Hold
`Circuit
`
`Analog-To- - - (cid:173)
`Digital
`Converter
`
`Wristwatch
`Controller
`
`368
`
`User
`Interface
`
`336
`
`338
`
`364
`
`Microphone Speaker
`
`366
`
`Digital-To(cid:173)
`Analog
`Converter
`
`352
`
`Real-Time System
`Clock
`Clock
`
`FIG. 15
`
`Petitioner Samsung Ex-1005, 0013
`
`

`

`Patent Application Publication
`
`Jun. 25, 2009 Sheet 13 of 14
`
`US 2009/0164219 Al
`
`START
`
`374 SELECT A SIGNAL COMPONENT
`(X, Y, OR Z) BY
`ANALOG MULTIPLEXER
`
`376
`
`SAMPLE AND HOLD
`ANALOG SIGNAL COMPONENT
`
`378
`
`CONVERT ANALOG SIGNAL
`COMPONENT TO DIGITAL DATUM
`
`380
`
`STORE DIGITAL DATUM
`IN ACCELEROMETER-DATA
`MEMORY
`
`WAIT FORA
`PREDETERMINED
`TIME INTERVAL
`
`388
`
`NO
`
`COMPUTE ORIENTATION AND/OR MOVEMENT
`TO ACTIVATE AUDIO REPORTING OF TIME
`ONLY WHEN A PREDETERMINED ORIENTATION
`AND/OR MOVEMENT IS DETECTED
`
`NO
`
`END
`
`FIG. 16
`
`Petitioner Samsung Ex-1005, 0014
`
`

`

`Patent Application Publication
`
`Jun. 25, 2009 Sheet 14 of 14
`
`US 2009/0164219 Al
`
`I
`
`FIG. 17
`
`Petitioner Samsung Ex-1005, 0015
`
`

`

`US 2009/0164219 Al
`
`Jun.25,2009
`
`1
`
`ACCELEROMETER-BASED CONTROL OF
`WEARABLE DEVICES
`
`CROSS-REFERENCE TO RELATED
`APPLICATIONS
`[0001] This application claims the benefit of provisional
`application Ser. No. 61/008,504, filed Dec. 19, 2007 by the
`present inventors.
`
`FEDERALLY SPONSORED RESEARCH
`
`[0002] Not Applicable
`
`SEQUENCE LISTING OR PROGRAM
`[0003] Not Applicable
`
`BACKGROUND
`
`[0004]
`1. Field
`[0005] This application relates to using accelerometer(cid:173)
`based orientation and/or movement sensing to control wear(cid:173)
`able devices, such as wrist-worn audio recorders and wrist(cid:173)
`watches.
`[0006] 2. Prior Art
`[0007] Wrist-worn audio recorders can serve a wide range
`of uses, from recording memos and meetings for profession(cid:173)
`als to documenting dietary intake and physical activity for
`health-conscious individuals and chronicling a child's first
`words and actions for parents. Furthermore, recorded audio
`messages can be played back at preset times to remind or alert
`a user, as disclosed in U.S. Pat. No. 5,511,046 to Vanderpal
`(1996).
`[0008] These wrist-worn audio recorders, wristwatches,
`and related devices are generally controlled through one or
`more of the following means: electromechanical switches,
`voice activation, and motion activation.
`[0009] Switches are commonly used to control functions
`such as audio recording and playback on devices like elec(cid:173)
`tronic wristwatches (as described in U.S. Pat. No. 4,717,261
`to Kita et al. (1988)) and, more recently, wrist-worn watch/
`MP3 players (for example, the Xonix MP3 recorder watches
`manufactured by Xonix Electronic Watch Co., Ltd., Zhuhai,
`China). They are also used to report the time in talking
`watches. However, these switches are often small, difficult to
`operate, and not very reliable, which may lead to premature
`failure of the audio recorder or talking watch. In addition,
`using switches can be inconvenient for recording many brief
`personal messages, because the user has to remember to turn
`the audio recorder on and off in order to record each message.
`[0010] Voice-activated recording mechanisms offer more
`convenience since they facilitate convenient recording of per(cid:173)
`sonal messages only when the user wants to record, without
`relying on the user's active attention to physically tum the
`audio reorder on and off during a recording session. However,
`irrelevant audio signals in the surrounding can still tum on a
`voice-activated mechanism.
`[0011] Finally, motion activation has also been used to
`operate wristwatches. U.S. Pat. No. 3,939,640to Kahn (1976)
`describes using a motion-activated switch inside a wristwatch
`to tum on or off the illumination of a wristwatch display. A
`free-rolling heavy ball within the container of the switch
`strikes a spring to cause conduction of electrical current when
`the wrist rotates rapidly. Additionally, U.S. Pat. No. 4,115,
`995 to Brien (1978) discloses using a motion-activated switch
`secured within a wristwatch to set the display on the wrist-
`
`watch (i.e. to a desired time and date), where quick snaps of
`the wrist dislodge a metal ball from a magnet and cause
`conduction of electrical current through the ball. Although
`these motion-activated switches can be used to turn on and off
`a wrist-worn device, such as an audio recorder, the rapid
`motion required to close the electrical switches is unnatural
`and can cause muscle strain, especially after many repeated
`actions. Furthermore, it is difficult to add more control func(cid:173)
`tions to a wrist-worn device using only these simple motion(cid:173)
`activated on-off switches.
`[0012] The aforementioned mechanisms for controlling
`wrist-worn devices leave much to be desired in terms of ease
`of use and durability. Accelerometer-based methods would
`offer advantages in these senses, since control can be auto(cid:173)
`matically activated through a user's natural movements, and
`the user does not need to directly interact with the electrome(cid:173)
`chanical parts of the accelerometer.
`[0013]
`In recent years, accelerometers produced with low(cid:173)
`cost MEMS (micro-electro-mechanical systems) technology
`have been used for movement and orientation sensing. For
`example, the wristwatch described in U.S. Pat. No. 6,513,532
`B2 to Mault et al. (2003) uses an accelerometer and a button(cid:173)
`controlled audio recorder to monitor physical activity and
`record dietary consumption, respectively, and U.S. Pat. No.
`6,956,564 Bl to Williams (2005) discloses a portable hand(cid:173)
`held computer that incorporates two single-axis accelerom(cid:173)
`eters for display control and gesture recognition with small
`fingertip switches for recording speech notes. In the field of
`animal behavior monitoring, U.S. Pat. No. 7,246,033 Bl to
`Kudo (2007) and U.S. Pat. No. 6,263,836 Bl to Hollis (2001)
`describe the use of accelerometers to monitor pet activity and
`train dogs, respectively, and both incorporate an audio
`recorder for recording and playback of a human's voice.
`However, all of the above accelerometer applications require
`conventional switches to operate audio recording, so that
`there still exists a need for the application of accelerometer
`technology to control recording, playback, time reporting,
`and other functionality in wrist-worn devices.
`
`SUMMARY
`
`[0014] The use of accelerometer-based orientation and
`movement sensing to control wearable devices, such as wrist(cid:173)
`worn audio recorders and wristwatches, is illustrated through
`three embodiments.
`[0015]
`In accordance with a first embodiment, a wrist-worn
`audio recorder comprises an accelerometer for sensing the
`orientation and/or movement of the audio recorder. In this
`embodiment, the audio recorder activates audio recording
`only when it is in a predetermined orientation and/or after it
`has completed a predetermined movement. Different prede(cid:173)
`termined orientations and/or movements (i.e. sequences of
`accelerometer data) of the audio recorder can be used to
`activate other functions of the recorder, such as playback,
`rewinding, or fast forwarding.
`[0016]
`In accordance with a second embodiment, a wrist(cid:173)
`worn audio recorder comprises an accelerometer for sensing
`the orientation and/or movement of the audio recorder. In this
`embodiment, a reminding audio message can be recorded in
`accordance with the first embodiment or in any other manner.
`When the audio recorder is worn on a wrist, a leg, or another
`part of the body, it plays back a reminding audio message after
`it detects a predetermined orientation and/or movement a
`predetermined number of times.
`
`Petitioner Samsung Ex-1005, 0016
`
`

`

`US 2009/0164219 Al
`
`Jun.25,2009
`
`2
`
`[0017]
`In accordance with a third embodiment, a wrist(cid:173)
`watch comprises an accelerometer for sensing the orientation
`and/or movement of the wristwatch. In this embodiment, the
`watch gives an audio report of the time after it detects a
`predetermined orientation and/or movement.
`[0018] Although the above embodiments take the form of
`wrist-worn devices, the methods and apparatus they illustrate
`can be extended to wearable devices in general, by one skilled
`in the art and without departing from the spirit and scope of
`the invention.
`
`DRAWINGS-FIGURES
`
`[0019] FIG. 1 shows a front view of a wrist-worn audio
`recorder incorporating a three-axis accelerometer in accor(cid:173)
`dance with the first embodiment, illustrating the X- and
`Y-axes of the accelerometer.
`[0020] FIG. 2 shows a side view of the wrist-worn audio
`recorder of FIG. 1, illustrating the Y- and Z-axes of the accel(cid:173)
`erometer.
`[0021] FIG. 3 is a schematic block diagram of the wrist(cid:173)
`worn audio recorder of FIG. 1, in accordance with the first
`embodiment.
`[0022] FIG. 4 is a view of the wrist-worn audio recorder in
`accordance with the first embodiment, schematically illus(cid:173)
`trating communication between the audio recorder and a pro(cid:173)
`cessor, and between the audio recorder and an earphone.
`[0023] FIG. 5 is a flow diagram illustrating the orientation
`and/or movement detection operation of a recorder controller
`of the wrist-worn audio recorder, in accordance with the first
`embodiment.
`[0024] FIG. 6 is a graphical illustration of a possible orien(cid:173)
`tation of the wrist-worn audio recorder for activating audio
`recording, in accordance with the first embodiment.
`[0025] FIG. 7 is a graphical illustration of a possible orien(cid:173)
`tation of the wrist-worn audio recorder for activating play(cid:173)
`back, in accordance with the first embodiment.
`[0026] FIG. 8 is a graphical illustration of possible move(cid:173)
`ments of the wrist-worn audio recorder for activating rewind(cid:173)
`ing and fast forwarding, in accordance with the first embodi(cid:173)
`ment.
`[0027] FIG. 9 is a schematic block diagram of a wrist-worn
`audio recorder, in accordance with the second embodiment.
`[0028] FIG. 10 is a flow diagram illustrating the orientation
`and/or movement detection operation of a recorder controller
`of the wrist-worn audio recorder, in accordance with the
`second embodiment.
`[0029] FIG. 11 is a graphical illustration of using a wrist(cid:173)
`worn audio recorder to remind the user of eating too many
`snacks, in accordance with the second embodiment.
`[0030] FIG. 12 is a graphical illustration of using a wrist(cid:173)
`worn audio recorder to sense restless leg movement of the
`user and transmit a reminding audio message to the caregiver
`of the user, in accordance with the second embodiment.
`[0031] FIG. 13 shows a front view of a wristwatch incor(cid:173)
`porating a three-axis accelerometer in accordance with the
`third embodiment, illustrating the X- and Y-axes of the accel(cid:173)
`erometer.
`[0032] FIG.14 shows a side view ofthewristwatchofFIG.
`13, illustrating the Y- and Z-axes of the accelerometer.
`[0033] FIG. 15 is a schematic block diagram of the wrist(cid:173)
`watch of FIG. 13, in accordance with the third embodiment.
`
`[0034] FIG. 16 is a flow diagram illustrating the orientation
`and/or movement detection operation of a wristwatch con(cid:173)
`trollerof the wristwatch, in accordance with the third embodi(cid:173)
`ment.
`[0035] FIG. 17 is a graphical illustration of a possible ori(cid:173)
`entation of the wristwatch for activating audio time reporting,
`in accordance with the third embodiment.
`
`DETAILED DESCRIPTION-FIGS. 1, 2, 3, AND
`4-FIRST EMBODIMENT
`
`[0036] The use of accelerometer-based orientation and/or
`movement sensing to control a wearable device is illustrated
`in a first embodiment with a wrist-worn audio recorder. The
`wrist-worn audio recorder incorporates an accelerometer for
`sensing the orientation and/or movement (i.e. either orienta(cid:173)
`tion or movement or both) of the audio recorder, and it is
`activated to record audio messages only when it is in a pre(cid:173)
`determined orientation and/or after it has completed a prede(cid:173)
`termined movement. In this embodiment we describe the
`predetermined orientation and/or movement to be the natural
`orientation and/or movement for the user to record personal
`audio messages, so that the user does not need to actively
`remember to turn the audio recorder on and off, but the
`predetermined orientation and/or movement can be any other
`orientation and/or movement. The wrist-worn audio recorder
`may usually also contain a real-time clock, so that the
`recorder can provide additional time-keeping function, and in
`this case the recorded audio data can be synchronized with
`other time-stamped data, such as video data or accelerometer
`data from other devices.
`[0037] FIG. 1 shows a front view of a wrist-worn audio
`recorder 30 incorporating a three-axis accelerometer 32
`inside a housing 33 of audio recorder 30. Three-axis acceler(cid:173)
`ometer 32 is commonly also called a triaxial accelerometer,
`and it senses acceleration in the three orthogonal axes X, Y,
`and Z. InFIG.1, the accelerometer 32 is illustrated by a dotted
`outline, and the Z-axis points out of the figure. Wristband 31
`secures audio recorder 30 on top of the user's right wrist
`(although audio recorder 30 could also be secured on top of
`the user's left wrist). A display 34 on the front surface ofaudio
`recorder 30 shows the time of day (hours, minutes, seconds),
`the date (month, date, day), or the current status (recording,
`playback, etc.) of audio recorder 30. Alternatively, the hands
`for the hours, minutes, and seconds of an analog wristwatch
`may be used to show the time of day, in place of display 34.
`Wrist-worn audio recorder 30 can function without a real(cid:173)
`time clock, although the inclusion of a real-time clock enables
`time-stamping of the recorded audio data and facilitates data
`archiving, data searching, or time synchronization of the data
`with time-stamped data from other devices.
`[0038] A microphone 36 is used for audio recording, and a
`speaker 38 is used for playing back the recorded audio data.
`When audio recorder 30 is activated for audio recording,
`indicators 40a, 40b, 40c at the front of housing 33 may be
`used to inform the user whether the recording is started at the
`beginning, the end, or somewhere in the middle of the audio(cid:173)
`data record. Recording audio data at the beginning or between
`the beginning and the end of the audio-data record will over(cid:173)
`write previously recorded audio data. Recording at the end is
`the usual mode of audio recording and will append additional
`audio data to the end of the audio-data record. Indicators 40a,
`40b, 40c can be miniature LEDs (light-emitting diodes), and
`flashing of an indicator can be used to signal the user that
`audio recording is in process. Alternatively, the functions of
`
`Petitioner Samsung Ex-1005, 0017
`
`

`

`US 2009/0164219 Al
`
`Jun.25,2009
`
`3
`
`indicators 40a, 40b, 40c can be incorporated into display 34,
`so that separate indicators 40a, 40b, 40c are not needed.
`[0039] FIG. 2 shows a side view of wrist-worn audio
`recorder 30 with accelerometer 32 (illustrated by a dotted
`outline) mounted inside housing 33 to sense the orientation
`and/or movement of audio recorder 30. Audio recorder 30 is
`secured on top of the wrist, using wristband 31, and the X-axis
`points out of the figure. Three-axis accelerometers such as
`accelerometer 32 may be constructed with MEMS (micro(cid:173)
`electro-mechanical systems) technology using capacitance
`measurement to determine the amount of acceleration, and
`they are available from Freescale Semiconductor, Inc., of
`Austin, Tex., or other companies. The X-, Y-, and Z-axis
`signals from a three-axis accelerometer provide information
`about the accelerometer's movement (which may be deter(cid:173)
`mined by a sequence of accelerometer data, for example), and
`can also be separated into components of the vertical gravi(cid:173)
`tational acceleration G to determine orientation when the
`accelerometer is at rest, so that a three-axis accelerometer can
`serve as both an orientation and movement sensor. In this way,
`audio recorder 30 uses the X-, Y-, and Z-axis acceleration
`signals of accelerometer 32 to detect specific orientations
`and/or movements of audio recorder 30 and subsequently
`activate the corresponding audio-recorder functions. For
`example, if the user stretches his or her arm out forwards, with
`the palm of his or her hand facing vertically downwards,
`accelerometer 3 2 ( as illustrated in FIG. 2) will sense an accel(cid:173)
`eration of -9.8 meter/sec 2 (the gravitational acceleration G is
`9 .8 meter/sec2 vertically downward) along its Z-axis, because
`the Z-axis is pointing vertically upward in this orientation.
`The acceleration signals are zero along its X- and Y-axes,
`because they are orthogonal to the direction ofG. Likewise,
`an acceleration of9.8 meter/sec2 (i.e. G) will be sensed along
`the accelerometer's Z-axis if the palm is facing upwards
`instead. Other orientations of the wrist will produce accelera(cid:173)
`tion signals with different signs and magnitudes in the X-, Y-,
`and Z-axis components of accelerometer 32, and the orienta(cid:173)
`tion can be computed using standard vector analysis.
`[0040] FIG. 3 is a schematic block diagram of wrist-worn
`audio recorder 30 (FIG. 1). Each of the X-, Y- and Z-axis
`analog signal components of accelerometer 32 is selected by
`an analog multiplexer 42 at predetermined time intervals,
`under the control of a recorder controller 48. Recorder con(cid:173)
`troller 48 also activates an analog-to-digital converter 46,
`which uses a sample-and-hold circuit 44 to sample and hold
`the selected analog signal component. Analog-to-digital con(cid:173)
`verter 46 converts the sampled analog signal component to
`the corresponding digital datum and sends the digital datum
`to recorder controller 48, which stores the digital datum in an
`accelerometer-data memory 50. Although FIG. 3 shows that
`accelerometer 32 senses accelerometer signal components in
`three orthogonal axes, accelerometer 32 may sense signal
`components in a different number of axes for detection of
`different predetermined orientations and/or movements.
`Sample-and-hold circuit 44 and analog-to-digital converter
`46 are not required for an accelerometer that produces digital
`output data for the signal components, and analog multiplexer
`42 should be replaced with a digital multiplexer in this case.
`Furthermore, if accelerometer 32 is a single-axis accelerom(cid:173)
`eter that produces only one signal component, the analog or
`digital multiplexer is not needed.
`[0041] A system clock 52 provides the operation timing for
`recorder controller 48, which is usually a microprocessor.
`Recorder controller 48 can be configured to perform math-
`
`ematical computation, logic operation, timer function, stor(cid:173)
`ing and retrieving data using an audio-data memory 56 and
`accelerometer-data memory 50, and reading and sending data
`through a communication port 60, etc., as well known in the
`art. A real-time clock 54 provides time-keeping function, and
`recorder controller 48 can also use real-time clock 54 to
`time-stamp each section of recorded audio message. In cer(cid:173)
`tain designs, real-tine clock 54 is derived from system clock
`52, so that a dedicated real-time clock 54 is not required.
`[0042] A user records audio messages into microphone 36,
`and the analog audio signal from microphone 36 is converted
`to digital audio data by a sample-and-hold circuit 62 and an
`analog-to-digital converter 64. Recorder controller 48 stores
`the digital audio data in audio-data memory 56, and it can
`subsequently send the digital audio data to a digital-to-analog
`converter 66 for playback from speaker 38. Through commu(cid:173)
`nication port 60, recorder controller 48 can also send the
`recorded audio data to a processor (not shown in FIG. 3) for
`data archiving, data searching, or time synchronization of the
`data with other time-stamped data, or to an earphone (not
`shown in FIG. 3) for playback. Audio-data memory 56 and
`accelerometer-data memory 50 can be RAM (random-access
`memory), flash memory, removable memory cards, or other
`types of digital memory. A user interface 68, which includes
`display 34 (FIG. 1), indicators 40a, 40b, 40c (FIG. 1), and
`switches if necessary (not shown in FIG. 1), facilitates com(cid:173)
`munication between audio recorder 30 and the user.
`[0043] Communication port 60 facilitates communication
`between audio recorder 3 0 and a processor (not shown in FIG.
`3 ), such as a personal computer, and can send recorded audio
`data to the processor for playback, data archiving, data
`searching, or time synchronization of the data with other
`time-stamped data. Communication port 60 also facilitates
`transmission of recorded audio data to an earphone (not
`shown in FIG. 3) for playback. Communication port 60 can be
`a wired or wireless USB (Universal-Serial-Bus) port, a Blue(cid:173)
`tooth® (a digital wireless protocol) wireless communication
`port, or any other wired or wireless communication port.
`[0044] FIG. 4 illustrates communication between audio
`recorder 30 and a processor 70 through a communication link
`72, which can be a wired or wireless data link. Communica(cid:173)
`tion link 72 can be used to transmit data between audio
`recorder 30 and processor 70, or for other uses such as send(cid:173)
`ing clock-setting commands from processor 70 to audio
`recorder 30 to set real-time clock 54 (FIG. 3) of audio
`recorder 30 to an accurate real-time clock reading. Audio
`recorder 30 can also transmit recorded audio data to an ear(cid:173)
`phone 71 through a wired or wireless data link 73, such as a
`Bluetooth® data link. Processor 70 can be a personal com(cid:173)
`puter, a PDA, a cellular phone, or another digital device.
`Although FIG. 4 shows audio recorder 30, with housing 33,
`secured to the right wrist by wristband 31, audio recorder 30
`can communicate with processor 70 and earphone 71 without
`being secured to a wrist.
`[0045] Speech recognition of recorded speech notes is use(cid:173)
`ful in applications such as recording food intake or physical
`activity, because it minimizes the time and inconvenience
`otherwise involved if the user needs to manually enter this
`food and activity data into a processor such as processor 70 to
`analyze his or her energy balance or fitness status. In these
`cases, either recorder controller 48 (FIG. 3) can perform the
`speech recognition, or it can send the audio data to processor
`70 for processor 70 to perform the speech recognition. If
`audio recorder 30 performs the speech recognition, then the
`
`Petitioner Samsung Ex-1005, 0018
`
`

`

`US 2009/0164219 Al
`
`Jun.25,2009
`
`4
`
`time-stamped text messages (obtained from speech recogni(cid:173)
`tion of time-stamped audio data) can be stored in audio-data
`memory 56 (FIG. 3) along with other audio data and sent to
`processor 70 at a later time.
`
`Detection Operation-FIGS. 5, 6, 7, and 8
`
`[0046] FIG. 5 is a flow diagram illustrating the orientation
`and/or movement (i.e. sequence of acceleration data) detec(cid:173)
`tion operation of recorder controller 48 (FIG. 3). In FIG. 5,
`after analog multiplexer 42, under the control of recorder
`controller 48, selects an acceleration signal component (X, Y,
`or Z) at step 74, recorder controller 48 activates analog-to(cid:173)
`digital converter 46 (FIG. 3) to use sample-and-hold circuit
`44 (FIG. 3) to sample and hold the selected signal component
`at step 76. At step 78, analog-to-digital converter 46 converts
`the sampled analog signal component to its corresponding
`digital datum and sends the digital datum to recorder control(cid:173)
`ler 48, which stores the digital datum in accelerometer-data
`memory 50 (FIG. 3) at step 80. At step 82, recorder controller
`48 repeats this process for the next accelerometer signal com(cid:173)
`ponent, until the X, Y, and Z signal components have all been
`selected. If audio recorder 3 0 does not move too fast over each
`cycle of accelerometer-data acquisition, the acquired and
`stored X, Y, and Z signal data in each cycle are approximately
`simultaneous. Alternatively, separate data acquisition sub(cid:173)
`systems, each including a sample-and-hold circuit and an
`analog-to-digital converter, can be used for each of the three
`accelerometer-signal components to obtain more precisely
`simultaneous X, Y, and Z accelerometer signal data. At step
`84, recorder controller 48 uses the data stored in accelerom(cid:173)
`eter-data memory 50 to compute the orientation and/or move(cid:173)
`ment of audio recorder 30 and activates an audio-recorder
`function, such as audio recording, playback, or rewinding
`(the operation of the audio-recorder function is not illustrated
`in FIG. 5), only if a predetermined orientation and/or move(cid:173)
`ment is detected. At step 86, if the user does not stop the
`accelerometer-data acquisition, recorder controller 48 waits
`for a predetermined time interval at step 88 and then returns to
`step 74 to repeat the above process.
`[0047] FIG. 6 illustrates a possible orientation of audio
`recorder 30 during audio recording, with audio recorder 30
`secured on top of the wrist, using wristband 31. The user
`naturally positions audio recorder 30 in front of his or her
`mouth (typically within 8 inches of the mouth, for example),
`with the front surface of housing 33 of audio recorder 30
`facing the mouth. In this specific orientation of audio recorder
`30, the Y component of the gravitational acceleration G is
`small (the Y-axis points away from the figure and is nearly
`horizontal). Since the angle between the X-axis and the direc(cid:173)
`tion of the gravitational acceleration G is about 45 degrees,
`the X component of the gravitational acceleration G is
`approximately G cosine 45°, which is equal to 0.707 G, as
`illustrated in the vector diagram in FIG. 6. Similarly, the angle
`between the Z-axis and the gravitational acceleration G is
`about 135°, so that tile Z component of the gravitational
`acceleration G is G cosine 135°, which is -0.707 G. Recorder
`controller 48 (FIG. 3) detects this combination ofX, Y, and Z
`signal components to activate audio recording. Alternatively,
`and for more robust detection, recorder controller 48 can use
`a machine learning algorithm or other suitable method to
`detect a predetermined movement, such as the natural move(cid:173)
`ment of lifting the arm upwards from a resting position (i.e.
`
`from the side of the body or from in front of the lower end of
`the torso) and tilting the wrist towards the mouth to activate
`audio recording.
`In the first embodiment, the audio recording func(cid:173)
`[0048]