(12) United States Patent
`Fiedler
`
`USOO6804638B2
`US 6,804,638 B2
`*Oct. 12, 2004
`
`(10) Patent No.:
`(45) Date of Patent:
`
`(54) DEVICE AND METHOD FOR SELECTIVE
`RECALL AND PRESERVATION OF EVENTS
`PRIOR TO DECISION TO RECORD THE
`EVENTS
`
`(75) Inventor: Mark Fiedler, New York, NY (US)
`(73) Assignee: Recent Memory Incorporated, New
`York, NY (US)
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 0 days.
`
`(*) Notice:
`
`This patent is Subject to a terminal dis
`claimer.
`
`(21) Appl. No.: 09/884,532
`(22) Filed:
`Jun. 20, 2001
`(65)
`Prior Publication Data
`US 2003/004.0917 A1 Feb. 27, 2003
`Related U.S. Application Data
`(63) Continuation of application No. 09/201,359, filed on Apr.
`30, 1999, now Pat. No. 6,205,419.
`(60) Provisional application No. 60/148,622, filed on Aug. 12,
`1999, provisional application No. 60/133,800, filed on May
`11, 1999, provisional application No. 60/133,801, filed on
`May 11, 1999, provisional application No. 60/123,487, filed
`on Mar. 9, 1999, provisional application No. 60/123,486,
`filed on Mar. 9, 1999, and provisional application No.
`60/115,274, filed on Jan. 8, 1999.
`(51) Int. Cl. ................................................ G10L 19/00
`(52) U.S. Cl. ....................... 704/201; 704/270; 704/275;
`360/60; 360/53; 360/48; 369/32; 369/54;
`369/47; 369/58; 369/61; 369/72
`
`(56)
`
`(58) Field of Search ................................. 704/201, 231,
`704/270, 275; 360/60, 55, 57, 53, 48; 369/56,
`57, 47, 61, 32, 54, 58, 72, 48
`References Cited
`U.S. PATENT DOCUMENTS
`... 704/215
`4,331,837 A * 5/1982 Soumagne .....
`5,053,896 A * 10/1991 Sakata et al. ................. 360/71
`5,379,153 A * 1/1995 Ishii et al. .................... 360/27
`5,386,493 A * 1/1995 Degen et al. .....
`... 704/267
`5,592.586 A * 1/1997 Maitra et al. ............... 704/220
`5,617,383 A * 4/1997 Matsumoto et al. .......... 369/32
`5,724,475 A * 3/1998 Kirsten ..........
`... 386/109
`5,765,129 A * 6/1998 Hyman et al. .
`... 704/270
`5,845,240 A * 12/1998 Fielder ..........
`... 704/201
`6,205,419 B1
`3/2001 Fiedler ....................... 704/201
`* cited by examiner
`Primary Examiner Vijay Chawan
`(57)
`ABSTRACT
`A device process for event-activated recording generally
`includes the Steps of providing an acquisition buffer having
`at least one finite extent of a recording medium. Events are
`recorded in the acquisition buffer. Current events in place of
`earlier events are then recorded within the acquisition buffer.
`Action by the user or triggering System activates the pres
`ervation of all or part of the record of the most recent events
`recorded in the acquisition buffer upon the detection of one
`or more predetermined events. A portion of the record of
`events Selected by the user is preserved by reserving a
`portion of the record of events from being overwritten by
`other events, and/or by transferring the Selected portions of
`the record of events to a predetermined permanent location.
`Portable devices and applications to the recording telephone
`conversations are also disclosed.
`
`19 Claims, 12 Drawing Sheets
`
`
`
`ACQUISTION BUFFER RECORDS
`
`ACQUISTION BUFFER
`(FULLY RECORDED)
`
`ACOUSITION BUFFER
`(PARTIALLY RECORDED)
`
`ACQUISITION BUFFER
`(NOT RECORDED)
`
`

`

`U.S. Patent
`
`Oct. 12, 2004
`
`Sheet 1 of 12
`
`US 6,804,638 B2
`
`/? MICROPHONE
`
`
`
`A/D
`CONVERTER
`
`
`
`INPUT BUFFERS
`
`
`
`
`
`USER
`CONTROLS
`
`
`
`ACOUISTION
`BUFFERS
`
`
`
`
`
`
`
`PERMANENT
`STORAGE
`
`FIG. 1
`
`

`

`U.S. Patent
`
`Oct. 12, 2004
`
`Sheet 2 of 12
`
`US 6,804,638 B2
`
`ACOUISTION BUFFER RECORDS
`
`III
`
`Tib
`
`
`
`ACQUISTION BUFFER
`(FULLY RECORDED)
`
`ACQUISITION BUFFER
`(PARTIALLY RECORDED)
`
`ACOUISTION BUFFER
`(NOT RECORDED)
`
`

`

`U.S. Patent
`
`Oct. 12, 2004
`
`Sheet 3 of 12
`
`US 6,804,638 B2
`
`
`
`BLOCK
`GRAPH
`
`AMPLITUDE
`GRAPH
`
`TIME
`SCALE
`
`
`
`PLAY/STOP
`
`MOVE BACK
`BUTTON
`
`SPLT
`
`MOVE FORWARD
`BUTTON
`
`CAPTURE
`BUTTON
`
`CURSOR
`
`FIG. 3
`
`

`

`U.S. Patent
`
`Oct. 12, 2004
`
`Sheet 4 of 12
`
`US 6,804,638 B2
`
`INTIALIZE ACQUISTION BUFFERS
`WITH BEGINNING AND END
`NODESo INITIALIZE RECORDING
`CURSOR AFTER BEGINNING NODE
`OF FIRST ACQUISITION BUFFER
`
`20
`MARK THE CURRENT
`BLOCK FOR CAPTURE
`19
`YES
`
`
`
`
`
`HAS THE USER
`PRESSED
`CAPTURE OR
`CAPTURE ALL?
`18
`UPDATE DISPLAY
`AND RESET DISPLAY
`UPDATE COUNTERS
`
`
`
`
`
`
`
`HAS SAMPLE
`COUNT FOR DISPLAY
`UPDATE BEEN
`REACHED?
`
`MOVE RECORD CURSOR
`BACK TO BEGINNING OF
`SILENT INTERVAL CREATE
`AND ENTER SELENCE NODE
`
`HAS AN INTERVAL
`OF SILENCE >
`DURATION THRESHOLD
`ELAPSED?
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`2
`
`READ SAMPLE FROM
`INPUT BUFFER
`
`IS THE RECORD
`CURSOR IN A
`SILENCE NODE2
`
`HAS THE
`RECOR
`ESOR
`NODE2
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`WRITE SAMPLE AND
`INCREMENT RECORD
`CURSOR
`
`FIG. 4A
`
`

`

`U.S. Patent
`
`Oct. 12, 2004
`
`Sheet 5 of 12
`
`US 6,804,638 B2
`
`NCREMENT SLENCE
`COUNT IN CURRENT
`NODE
`
`THE AMPLTUDE
`THRESHOLD?
`
`COUNT AND PLACE
`RECORD CURSOR AFTER
`NODE
`
`
`
`
`
`
`
`IS THE SLENCE
`NODE AT THE END OF ANy NQ
`ACOUISITION BUFFER2
`
`YES
`
`MAKE THE FIRST
`ACQUISITION BUFFER
`THE CURRENT
`ACQUISITION BUFFER
`
`YES
`
`YES
`
`IS THE RECORD
`CURSOR IN THE
`LAST ACQUISTION
`BUFFER?
`NO
`10
`MAKE THE NEXT
`ACQUISTON BUFFER
`THE CURRENT
`ACQUISITION BUFFER
`
`
`
`
`
`
`
`SAVE ANY DATA IN
`THIS ACQUISTION
`BUFFER THAT IS MARKED
`FOR CAPTURE
`
`DELETE DISPLAY DATA
`REFERING TO DATA IN
`THIS ACQUISITION BUFFER
`UPDATE ORIGIN TIME AND
`ORIGIN PIXEL IN DISPLAY
`13
`LINK BEGINNING AND END
`NODES IN THIS ACQUESTION
`BUFFER PLACE RECORD
`CURSOR AFTER BEGINING
`NODE
`
`FIG. 4B
`
`

`

`U.S. Patent
`
`Oct. 12, 2004
`
`Sheet 6 of 12
`
`US 6,804,638 B2
`
`21
`
`
`
`
`
`INCREMENT CURRENT
`PXEL POINTER.
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`IS PIXEL ARRAY
`FILLED?
`
`EXTEND PXEL ARRAY
`
`WRITE TIME, PLACE,
`AMPLTUDE TO CURREN
`PXEL
`
`DISPLAY CURRENT
`PXEL
`
`
`
`DISPLAY PREVIOUS
`PIXEL
`
`CLEAR DISPLAY FROM
`HERE TO PREVIOUS
`ORIGIN
`
`ORIGIN PXEL
`REACHED?
`
`
`
`PHYSICAL LOCATION
`OF PREVIOUS ORIGIN
`REACHED?
`
`FIG. 5
`
`

`

`U.S. Patent
`
`Oct. 12, 2004
`
`Sheet 7 of 12
`
`US 6,804,638 B2
`
`30
`USER
`PRESSES
`MOVE
`BACK
`
`
`
`
`
`POSITION PLAY CURSOR
`AND DISPLAY CURSOR
`AT START OF MOST
`RECENT BLOCK
`
`
`
`COPY SAMPLE FROM
`ACQUISITION BUFFER (AT
`PLAY CURSOR) TO OUTPUT
`BUFFER AND INCREMENT
`PLAY CURSOR
`
`34
`HAS THE PLAY
`CURSOR REACHED \ YES FLUSH OUTPUT
`THE CURRENT TIME?
`BUFFERS
`
`EXIT
`PLAYBACK
`
`
`
`
`
`HAS THE PLAY
`CURSOR REACHED
`A NODE
`
`YES
`
`IS THIS NODE'S
`DURATION > MINIMUM
`ELLIPSS LENGTH2
`
`No COPY THIS NODE'S
`DURATION OF SILENCE
`TO OUTPUT BUFFER
`
`NO
`
`COPY ELLIPSIS TO
`OUPUT BUFFER
`
`HAS PLAYBACK
`OUTPUT REACHED THE
`PLACE OF THE NEXT
`PXEL?
`
`MOVE DISPLAY
`CURSOR TO NEXT
`PXEL
`
`HAS THE USER
`PRESSED A CONTROL
`BUTTON?
`
`YES
`
`FIG. 6A
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`

`

`U.S. Patent
`
`Oct. 12, 2004
`
`Sheet 8 of 12
`
`US 6,804,638 B2
`
`
`
`
`
`
`
`59
`S THIS NODE
`AT THE END OF AN
`ACOUISTION
`BUFFER2
`
`
`
`YES
`
`40
`IS THIS NODE
`AT THE END OF THE
`AST ACQUISTION
`BUFFER2
`
`42
`YES
`PLACE PLAY CURSOR
`AFTER START NODE OF
`FIRST ACQUISITION
`BUFFER
`
`
`
`
`
`
`
`
`
`4.
`PLACE PLAY CURSOR
`AFTER START NODE OF
`NEXT ACQUISTION
`BUFFER
`
`TAKE ACTION IN
`RESPONSE TO CONTROL
`BUTTON (FIG.7)
`
`47
`
`
`
`
`
`YES
`
`46
`FLUSH OUTPUT
`BUFFERS
`
`
`
`
`
`
`
`DD USER PRESS
`
`
`
`YES
`
`
`
`EXIT
`PLAYBACK
`
`FIG. 6B
`
`

`

`U.S. Patent
`
`Oct. 12, 2004
`
`Sheet 9 of 12
`
`US 6,804,638 B2
`
`
`
`
`
`
`
`
`
`MOVE
`BACK
`(TAPPED)
`
`IS THE DISPLAY
`CURSOR ON A SPACE
`BETWEEN BLOCK?
`
`IS THE DISPLAY
`CURSOR ON THE
`FIRST BLOCK?
`
`MAKE THE PREVIOUS
`BLOCK THE CURRENT
`BLOCK
`
`PLACE PLAY CURSOR AND
`DISPLAY CURSOR AT START
`OF CURRENT BLOCK
`
`
`
`
`
`RETURN TO
`PLAYBACK
`
`
`
`
`
`FIG. 7
`
`
`
`MOVE
`FORWARD
`(TAPPED)
`
`
`
`
`
`IS THE DISPLAY
`CURSOR ON THE LAST
`BLOCK?
`
`PLACE PLAY CURSOR AND
`DISPLAY CURSOR AT START
`OF NEXT BLOCK
`
`
`
`
`
`RETURN TO
`PLAYBACK
`
`
`
`
`
`EXIT
`PLAYBACK
`
`FIG. 8
`
`

`

`U.S. Patent
`
`Oct. 12, 2004
`
`Sheet 10 of 12
`
`US 6,804,638 B2
`
`
`
`
`
`
`
`
`
`MOVE PLAY CURSOR BACK
`TO JUST AFTER MOST
`RECENT NODEx=SPLT
`DURATION THRESHOLD
`
`EXTEND PXEL ARRAY
`INSERT PXEL FOR NEW
`SPACE
`
`PLACE DISPLAY
`CURSOR AT START OF
`BLOCK AFTER NEW
`SPACE
`
`
`
`
`
`
`
`
`
`RETURN TO
`PLAYBACK
`
`
`
`FIG. 10
`
`
`
`
`
`
`
`
`
`
`
`
`IS THE CURRENT
`BLOCK MARKED FOR
`CAPTURE2
`
`MARK CURRENT BUOCK
`FOR CAPTURE
`
`UNMARK CURRENT
`BLOCK FOR CAPTURE
`
`RETURN TO
`PLAYBACK
`
`
`
`
`
`FIG.9
`
`

`

`U.S. Patent
`
`Oct. 12, 2004
`
`Sheet 11 of 12
`
`US 6,804,638 B2
`
`
`
`CLIP
`
`RECEMING DEVICE
`(e.g., camera lens,
`microphone)
`
`AMBENT
`POWER SOURCE
`
`RECORDING DEVICE
`
`DATA TRANSFERPORT
`
`FIG. 11
`
`

`

`U.S. Patent
`
`Oct. 12, 2004
`
`Sheet 12 of 12
`
`US 6,804,638 B2
`
`WRST BAND
`
`MICROPHONE
`(HIDDEN)
`
`
`
`MCROPHONE
`(HIDDEN)
`
`,
`
`IIIHIII III-III-III-III IIII
`
`III
`
`IIHIIII IIII
`
`III
`M
`
`
`
`TOUCH-
`SENSIVE
`DISPLAY
`
`DATA
`TRANSFER
`PORT
`
`FIG. 12
`
`

`

`1
`DEVICE AND METHOD FOR SELECTIVE
`RECALL AND PRESERVATION OF EVENTS
`PRIOR TO DECISION TO RECORD THE
`EVENTS
`
`This is a continuation in part of U.S. patent application
`Ser. No. 09/201359, filed Apr. 30, 1991, now U.S. Pat. No.
`6,205,419 entitled “Selective Recall and Preservation of
`Continuously Recorded Data.”
`This invention is covered by the following provisional
`patent applications filed with the U.S. Patent and Trademark
`Office: Ser. No. 60/133,800 (Event-Activated Recording
`Method with Time Buffering), filed May 11, 1999; Ser. No.
`60/133,801 (Applications and Improvements for Selective
`Recording Method), filed May 11, 1999; Ser. No. 60/148,
`622 (Speech Searching), filed Aug. 12, 1999; Ser. No.
`60/123,487 (Selective Recording Method Using Flash
`Memory), filed Mar. 9, 1999; Ser. No. 60/115,274 (Data
`Capture Device), filed Jan. 8, 1999; and Ser. No. 60/123,486
`(Wrist Recorder), filed Mar. 9, 1999.
`FIELD OF THE INVENTION
`This invention relates to the mechanical, electrical, mag
`netic or other mechanized recording and reproduction of
`25
`auditory and/or visual or other data, and particularly to
`improvements to a process of recording which enables
`human users to decide whether to preserve a record of events
`that have already transpired.
`
`15
`
`BACKGROUND OF THE INVENTION
`My earlier patent, U.S. Pat. No. 5,845,240 (Selective
`Recall and Preservation of Continuously Recorded Data),
`discloses a recording method that effectively allows a user to
`decide to record an event after the event has taken place.
`This is achieved by continuously recording all ambient
`events in a continuous logical loop on a finite extent of
`recording medium and allowing users to Select for perma
`nent preservation portions of the material thus recorded,
`before they are overwritten with new material.
`The present invention comprises improvements to and
`applications of this basic technology.
`One area of technology addressed by the present inven
`tion is event-activated recording, in which a recording
`proceSS is initiated on the occurrence of Some event that is
`recognized by Some mechanism, algorithm or other auto
`mated means. Current examples include:
`(1) Voice-activated recording of speech
`(2) Sound-activated recording of Sound
`(3) In a Security System, audio and/or video recording
`activated on detection of movement
`(4) In a medical instrument, recording initiated by Some
`Significant event, e.g., an arrythmia
`A drawback common to all of these methods is the fact
`that they do not record the often-significant events leading
`up to the event that triggers the recording process. (Indeed,
`because Some time elapses between the occurrence of the
`triggering event, detection of the event, processing of the
`resulting Signal, recognition of the event as Significant, and
`finally initiation of the recording process, the triggering
`event is usually not recorded, or only partially recorded.)
`The present invention overcomes this drawback by pro
`viding an acquisition buffer (of a rewritable recording
`medium) in which current events are continuously recorded.
`When the recording process fills the acquisition buffer, it
`overwrites the record of the earliest events recorded in the
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`US 6,804,638 B2
`
`2
`buffer with current events. On detection of a triggering
`event, a predetermined portion of the current contents of the
`acquisition buffer (which may be the buffer's entire contents,
`or may correspond to a time interval that a user may specify)
`is transferred to the (permanent) recording medium normally
`used for event-activated recording. This has the effect of
`permanently preserving the events leading up to the trigger
`ing event.
`A further area of application of the present invention is
`telephony. Current methods of recording telephone conver
`sations share the drawbacks of conventional methods of
`recording: a recording process must be initiated (and, when
`applicable, consented to by parties to the conversation)
`before any of the events to be recorded have transpired.
`Accordingly, it is an objective of the present invention to
`enable the effective recording of portions of a telephone
`conversation after those portions have taken place. It is a
`further objective of the present invention to secure the
`consent of parties to the conversation as a condition of Such
`recording.
`It is a further objective of the present invention to enable
`the recording of different kinds of data on interchangeable
`media Such as flash memory.
`Portable recording devices constitute a further area of
`application of the present invention. Current portable
`recording devices share the drawbacks of conventional
`methods of recording: a recording process must be initiated
`before any of the events to be recorded have transpired. The
`present invention is embodies in two devices that overcome
`these drawbacks: a pocket device and a wrist recorder, both
`advantageously using interchangeable media Such as flash
`memory.
`
`SUMMARY OF THE INVENTION
`The present invention differs enables users effectively to
`record words that have already been spoken, or events that
`have already taken place, prior to the time that a user decides
`to record them. This is achieved by recording events on a
`finite extent of recording medium in a continuous logical
`loop (i.e., overwriting the earliest recorded data with new
`data whenever the end of the medium used for this continu
`ous recording is reached), and playing back and/or perma
`nently preserving (capturing) portions of the recorded data
`as Selected by the user.
`A further advantage of this invention is that it allows users
`to preserve the Spontaneity of their speech and actions by
`dispelling the Self-consciousness that Sets in when people
`know that they are being recorded-which happens even
`when people record themselves, triggered by the very act of
`Switching on a recording device. Moreover: Even in con
`tinuous monitoring methods, where continuous recording on
`an indeterminate Supply of recording medium is practicable,
`there is the need to go back to Search for and edit out the
`parts to be preserved, discarding the rest. The present
`invention goes far towards eliminating that need, by cap
`turing data permanently only in response to a user's explicit
`request (although this request must come within a
`predetermined, finite period from the time of the event).
`The present invention may be applied to Sound or Video
`recording, as well as recording
`(1) the Sequence of user actions (or other events) on a
`computer or other machine or on a musical instrument,
`(2) Streams of data produced by medical or weather
`instruments, or
`(3) Streams of data produced by any broadcasting means,
`including "netcasting over the Internet.
`
`

`

`US 6,804,638 B2
`
`3
`The invention may be embodied, with varying degrees of
`practicality, using any rewritable recording medium, includ
`ing electronic memory, tape, wire, or discs, both analog and
`digital. However, the current preferred medium for continu
`ous recording is digital memory, as it entails a minimum of
`wear, power consumption and bulk-especially important in
`a portable device.
`The essential components of this invention-beyond a
`minimum of appropriate hardware to detect, acquire and
`preserve data-are algorithms, which may be implemented
`as either hardware or software or both. The invention may be
`implemented as a program on a general-purpose computer,
`using either a Single-thread or a multithreaded operating
`System, as long as means of acquiring and playing back data
`are provided.
`Following is a description of an embodiment of this
`invention as a computer type program to capture audio data,
`running on an interrupt-driven computer of known type and
`using digital memory as the continuous recording medium.
`The same Scheme may be used in a dedicated device other
`than a general-purpose computer; and variations of this
`Scheme may be used to record other Streams of events, e.g.,
`Video data. In this last case, the division of the continuous
`recording into blockS Separated by intervals of Silence may
`be replaced by a separation into blocks each having the same
`base frame as detected by a Video compression algorithm.
`This embodiment uses, in place of a single circular
`acquisition buffer for Storing continuously recorded data, a
`circular array of Smaller, Separately allocated buffers. This
`arrangement affords a number of advantages on the design
`as compared with a single, larger buffer: (1) On general
`purpose computers, smaller blocks of memory are easier to
`allocate and move than larger ones; (2) Smaller-size numbers
`(e.g., 16 bits for a 64 KB buffer) may be used to keep track
`of locations in the Smaller buffers; and (3) this arrangement
`allows a simplified Scheme for keeping track of areas that
`have been overwritten with fresh data.
`
`BRIEF DESCRIPTION OF DRAWINGS
`FIG. 1 shows an overall block diagram of the parts of the
`preferred embodiment and their interrelationships.
`FIG. 2 shows the Scheme for Storing data in the acquisi
`tion buffers, using nodes to mark intervals of Silence as well
`as the beginnings and ends of buffers. The array of acqui
`Sition buffer records is shown along with three representa
`tive acquisition buffers: one fully recorded, one partially
`recorded, and one not yet recorded.
`FIG. 3 Shows the user interface, comprising the display
`and user controls, which may be embodied in a hand-held
`device or in a computer program.
`FIG. 4 is a flowchart of the continuous recording or
`acquisition process.
`FIG. 5 is a flowchart of the periodic process of updating
`the display as audio data are acquired.
`FIG. 6 is a flowchart of the process of playing back
`acquired material in response to a user's request.
`FIGS. 7-10 shows four flowcharts, detailing actions taken
`in response to various user controls.
`FIG. 11 shows a pocket recording device.
`FIG. 12 shows a wrist recording device.
`DETAILED DESCRIPTION OF THE
`PREFERRED EMBODIMENT
`FIG. 1 shows the major components and their interrela
`tionships: an analog input transducer Such as a microphone
`
`4
`feeding a continuous signal to an analog-digital converter,
`which in turn passes a train of digital Sample values to a Set
`of input buffers. From these the Sample train is continuously
`Supplied to a circular set of acquisition buffers. The contents
`of the acquisition buffers are shown on a periodically
`updated display, and parts of the contents of the acquisition
`bufferS may, by means of user controls, be played back
`through an output transducer Such as a speaker or Selected
`for preservation in permanent Storage.
`Acquisition and Storage of Audio Data
`Analog audio data are acquired through a microphone,
`telephone or similar transducer or input, converted into
`digital data at a certain Sampling frequency by an analog
`digital converter, and Stored Sequentially in a circular Series
`of input buffers. (Input is by no means limited to these
`Sources; it may also include the output of musical, medical,
`Seismic or weather instruments. Input may also be digital in
`nature, as for example a Stream of data broadcast over the
`Internet or other network, or issuing from any type of digital
`instrument.) In an arrangement typical of audio on personal
`computers, a peripheral Sound card incorporating an analog
`digital converter acquires audio Signals and Stores them
`Sequentially via direct memory access (DMA) in a queue of
`input bufferS allocated by an application program. AS each
`input buffer in turn becomes full, an interrupt signal or a
`message identifying the particular buffer is generated. When
`the application program receives this signal or message, the
`Sound data are transferred from that input buffer to acqui
`sition buffers in memory in the manner described in the
`following paragraphs. Concurrently with this transfer of
`data, audio data continue to be fed by the analog-digital
`converter into another input buffer or buffers, each of which
`in turn generates a further interrupt or message when it
`becomes full. After the contents of each buffer are trans
`ferred to the acquisition buffers, the input buffer is assigned
`to the tail end of the queue awaiting recording. A-minimum
`of two input buffers, used alternately, is required. Particu
`larly in multitasking Systems, there may be times when no
`input buffers are available for recording because all allo
`cated input buffers are still awaiting transcription, when this
`occurs, new input buffers may be allocated. The memory
`allocated for these buffers may be freed when no longer
`needed, or when the number of unrecorded but allocated
`buffers exceeds a certain number.
`An array of acquisition buffer records, illustrated in FIG.
`2, provides access (for recording, playback and transfer to
`permanent Storage) to the acquisition buffers and to the
`audio data Stored in them. Each acquisition buffer record
`contains the following data:
`tTime Number of Sampling intervals elapsed, at the begin
`ning of this buffer, Since the beginning of the program
`run. In the record that terminates the array, this value is
`ZCO.
`tEndTime Number of sampling intervals elapsed, at the
`end of this buffer, Since the beginning of the program
`.
`pWhere Pointer to the acquisition buffer holding the audio
`data. In the record that terminates the array, this value
`is null.
`The various acquisition buffers each contain, in addition
`to audio data, at least two nodes, which mark the beginning
`and the end of each buffer. Additional nodes, interspersed in
`the audio data, are used to mark intervals of Silence (as
`determined in the manner described below) within the
`buffer. The successive nodes within each acquisition buffer
`form a two-way linked list. Each node contains the follow
`ing data:
`
`15
`
`25
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`

`

`S
`tDuration Duration of Silent interval, in Sampling inter
`vals. Zero for nodes marking the beginning or the end
`of a buffer.
`nNext Number of samples of sound data between the end
`of the current node and the location of the next node.
`nPrev Number of samples of sound data between the
`beginning of the current node and the end of the
`previous node. This facilitates reverse playback as well
`as Searching backwards.
`FIG. 2 shows three acquisition buffers in various states of
`being recorded: Buffer 10 is completely filled, and has two
`Silence nodes interspersed in the audio data; these forming
`a linked list with the beginning and end nodes of the buffer.
`Buffer 11 is partially filled with audio data, and its single
`Silence node is Similarly linked with the beginning and end
`node. Buffer 10 is empty of data, containing only the
`beginning and end nodes, which also form a linked list.
`(In this acquisition buffer Scheme, the data are held
`Stationary in memory and accessed by moving cursors, as
`described below. An alternative dataflow arrangement is
`possible, in which data flow through a train of memory
`elements, all data elements moving to the next elements as
`each new Sample is read in.)
`In the recording process illustrated in FIG. 4, the con
`tinuously acquired audio data are written Successively to
`acquisition buffers, beginning with the acquisition buffer
`pointed to by the first record in the acquisition buffer table.
`When the recorded data reaches the end of that buffer (as
`indicated by reaching the node there, block 7), recording
`proceeds by writing to the acquisition buffer pointed to by
`the next record in the table. When the recording reaches the
`end of the acquisition buffer pointed to by the last acquisi
`tion buffer record, or the end of any other acquisition buffer
`except the last, recording proceeds with the acquisition
`buffer pointed to by the first record in the table, overwriting
`the existing contents of that acquisition buffer, (blocks 8-9).
`On Startup, each acquisition buffer is initialized with two
`nodes, one at the beginning of the buffer and one at the end.
`The nNext element of the beginning node points to the node
`at the end, and the nPrev element of the end node points to
`the beginning node (block 1; see also FIG. 2):
`Any interval of audio data whose absolute amplitude
`values do not exceed a certain minimal level (the amplitude
`threshold) over a certain minimal duration (the duration
`threshold), and whose length exceeds that of a node, is
`treated as an interval of Silence and is represented by a node
`in an acquisition buffer.
`(This is a simple criterion for detecting an interval of
`Silence; more Sophisticated ones are possible, including
`those involving the detection and cancellation of ambient
`noise. Moreover, other Signal-enhancement techniques, Such
`as automatic gain control, may be incorporated in the
`continuous recording process.)
`As audio data are transferred from the input buffers to the
`acquisition buffers, the Successive amplitude values in the
`audio stream are measured against the amplitude and dura
`tion thresholds (FIG. 4, block 15), and the stream is accord
`ingly Stored in the acquisition buffers as Segments of Sound
`data interspersed with silence nodes. Whenever a new
`Silence node is written in an acquisition buffer, its nPrev
`member is assigned the number of Samples written between
`this node and the previous node, and the nNext member of
`the previous node is assigned the same value. Similarly, this
`node's nNext member is assigned the number of Samples
`that may be written between this node and the node at the
`end of the acquisition buffer, and the end node's nPrev
`member is assigned the same value.
`
`15
`
`25
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`US 6,804,638 B2
`
`6
`When a Sufficient interval of silence has been detected, the
`recording cursor goes back to the beginning of the Sequence
`of amplitude values in the acquisition buffers corresponding
`to this interval and writes a Silence node there (blocks
`15–16); this may involve going back into a previous acqui
`sition buffer. Thus, if the data values at the end of an input
`buffer and those at the beginning of the next buffer fall
`within the threshold values for Silence, or if the entire
`contents of Successive buffers fall within the threshold
`values, a single node is generated with the total duration of
`the consecutive Silent intervals.
`If the beginning of a “silent Sequence is found to be
`within less than a node's length before the end node of the
`relevant acquisition buffer, the remaining Spaces are filled in
`with actual amplitude values and the beginning node of the
`next acquisition buffer is used to mark the interval of
`“Silence'; its duration is Set to the remaining duration in the
`“Silent Sequence.
`In the foregoing discussion, references to “next and
`“previous' acquisition buffers should be understood in light
`of the following: When the recording of incoming audio data
`reaches the end of an acquisition buffer, it resumes at the
`position following the beginning node of the acquisition
`buffer pointed to by the next acquisition buffer table
`element, except if the present acquisition buffer table ele
`ment is the last element in the table. In that case, the “next'
`acquisition buffer is that pointed to b the element at the
`beginning of the acquisition buffer table. Likewise, the
`acquisition buffer table element “previous” to that at the
`beginning of the table is that at the end of the table. In this
`way, recording of amplitude data proceeds in a circular
`fashion, overwriting the oldest data with fresh data.
`Event-activated recording, as described above, may be
`effected as follows: On detection of a Suitable triggering
`event, a predetermined portion of the current contents of the
`acquisition buffer (which may be the buffer's entire contents,
`or may correspond to a time interval that a user may specify)
`is transferred to the (permanent) recording medium normally
`used for event-activated recording. This has the effect of
`permanently preserving the events leading up to the trigger
`ing event.
`Alternatively, the portion of the acquisition buffer con
`tents leading up to the triggering event may be transferred to
`a Second acquisition buffer, which will be used in place of
`the original acquisition buffer for all purposes described
`below, including the presentation of data to the user for
`Selection for recall and/or preservation.
`This record of the events leading up to the triggering event
`is advantageously placed in the permanent record (or the
`Second acquisition buffer) just prior to the triggering event,
`and advantageously forms a continuous record with the
`triggering event and the events following it. It is of further
`advantage to mark the triggering event in this continuous
`record.
`Such a continuous record may be effected by:
`(1) Continuing to record in the acquisition buffer the
`triggering event and the events following the triggering
`event,
`(2) Concurrently with this continuous recording, copying
`the appropriate portions of the acquisition buffer con
`tents to the permanent record (or Second acquisition
`buffer), preserving the record of events prior to,
`including, and following the triggering event, and
`(3) Recording events directly to the permanent record (or
`Second acquisition buffer) when the copying process
`has caught up with the direct recording proceSS by
`reaching the present moment.
`
`

`

`7
`Permanent preservation of the record of events may
`alternatively be effected by:
`(1) Continuing to record in the acquisition buffer the
`triggering event and the events following the triggering
`event, and
`(2) Reserving the appropriate portions of the acquisition
`buffer contents (i.e., the record of events prior to,
`including, and/or following the triggering event) from
`being overwritten by the continuous recording process.
`Whenever a previously recorded acquisition buffer is
`about to be overwritten with fresh data, the block record is
`checked to see if any of the buffers contents have been
`Selected for capture. If So, the Selected data are written or
`appended to a file on the computers hard disk (block 11).
`This file may be in a standard sound wave file format,
`Suitable for playing through a Standard media player pro
`gram. Also, at this time, the origin time (whose value is the
`earliest time stored in the acquisition buffers) is updated to
`equal the Start time of the next acquisition buffer record
`(block 12). Thus, the contents of any acquisition buffer in the
`process of being overwritten are unavailable for recall or
`Storage.
`In general, captured data may be preserved by writing it
`to any permanent Storage medium, Such as a disk, tape or
`(non-volatile) Solid-state memory. Captured data may be
`preserved at any time between the decision to capture and
`the time that the data is overwritten; however, deferring
`Storage until the acquisition buffer is about to be overwritten
`facilitates giving the user the ability to cancel the decision to
`capture a block of data.
`When the system is to be shut down for any reason (as, for
`example, a critically low battery level), all captured data in
`the acquisition buffers are preserved prior to shutdown.
`The capture interval, i.e., the time interval that may be
`captured before the user's decision to record, is a function of
`the quantity of recording medium and the recording density
`(i.e., length of recording time Stored in a given quantity of
`recording medium). If captured data, rather than being
`transferred out of the acquisition buffer, are Stored in a newly
`reserved area of the acquisition buffer, then the capture
`interval will diminish as this area becomes fille