`
`[19]
`
`[11] Patent Number:
`
`4,682,248
`
`Schwartz
`
`.
`
`[45] Date of Patent:
`
`Jul. 21, 1987
`
`[54] AUDIO AND VIDEO DIGITAL RECORDING
`AND PLAYBACK SYSTEM
`
`[75]
`
`Inventor: David M. Schwartz, Englewood,
`C010.
`
`,
`-_
`_
`_
`[73] Ass1gnee: C0mpuS0n1cs \/Ideo Corporation,
`Palo Alto, Calif.
`,
`121] AWL N0“ 776309
`[22]
`pned;
`sep_17,19s5
`.
`.
`Related U'S'Ap1‘l‘°““°" Data
`Continuation—in—part of Ser. No. 651,111, Sep. 17, 1984,
`which is a continuation—in-part of Ser. No. 486,561,
`Apr. 19, 1983, Pat. No. 4,472,747.
`
`[63]
`
`Int. Cl.4 ........................ .. G11B 51/00; GIOL 5/02
`[51]
`[52] US. Cl. .........
`..... .. ... .. 360/32; 381/51
`[58] Field of Search .................... .. 381/51, 41; 360/32;
`353/22; 340/703, 717, 747
`References Cited
`
`[56]
`
`U'S' PATENT DOCUMENTS
`3,102,165
`8/1963 Clapper ..........
`3,236,947 2/1966 Clapper ..
`3,435,134 3/1969 Richards -
`316351O31
`3/1972 C°°k "" "
`
`V
`
`7/1973
`3,745,264
`1/1974 Myers et al‘
`3,786,201
`3,g55y517 12/1974 Jakowski et a1_
`4,o15,2g6
`3/1977 Russeu ,,,,,,,,,_,
`4,075,423
`2/1978 Martin et al.
`4,150,397 4/1979
`4311997 7/1930
`412141125
`7/1980
`4,225,885
`9/1980
`4,270,150
`5/1981 Diermann at 8]‘
`4,281,355
`7/1981 Wada et al.
`Taylor et a1_
`4,335,393
`6/1982 Pearson ......
`8/1982 Melamud
`4,345,314
`
`-
`
`381/51
`179/1
`-- 173/5
`340/17“
`1793/2;/:5
`179/1002
`179/1002
`350/32
`, 3 58/13
`. 381/50
`358/127
`- 371/33
`- 381/51
`340/146.3
`360/10
`. 360/32
`.
`358/4
`364/515
`
`DATA Anu1511I0N MOELEWIDAMI
`1'§1{5.~.'u-_‘.
`',TzIFEuc_'}
`
`01:02 KBVTESI
`1
`sac
`
`4,365,304 12/1982 Ruhman et a1. ........... 1. 364/515
`4,368,988
`1/1983 Tahara et al.
`..
`360/32
`4,375,650
`3/1983 Tiemann.
`.. 358/133
`4,387,406
`6/1983 on ............ ..
`.. 358/310
`4,389,537
`6/1983 Tsunoda et al.
`381/51
`4,389,681
`6/1983 Tanaka eta1’..
`360/27
`4,392,159
`7/1983 Lemoine 6, a]_
`358/319
`4,410,917 10/1983 Newdoll et al,
`_ 360/15
`4,411,015 10/1983 Scherl etal.
`. 382/51
`4,417,276 11/1983 Bennett et al..
`_. 358/160
`4,417,283 11/1983 Hoshimietal.
`358/310
`4,429,334
`1/1984 Hashimoto etal.
`.. 358/310
`4,432,019
`2/1984 Maier ............... ..
`.. 358/260
`4,441,201
`4/1984 Henderson etal.
`381/51
`4,455,635
`6/1984 Dieterich
`369/59
`4,458,110 7/1984 Mozer ........
`. 381/51
`4,493,106
`1/1985 Farhangi et al.
`382/41
`4,504,972
`3/1985 Scherl et al.
`. 382/51
`4,516,246
`5/1985 Kenemmh __
`_ 375/37
`4,519,027
`5/1935 Vogelsberg ._ _
`I 331/51
`4,520,401
`5/1935 Takahashi at 3,
`353/310
`4,528,585
`7/1935 Bolger
`. 353/22
`4,549,201 10/1985 Tanaka et a1.
`358/13
`Primary Examiner—Vincent P. Canney
`Attorney, Agent, or Firm—Jerry W. Berkstresser
`
`ABSTRACT
`[571
`A microcomputer system for converting an analog sig-
`nal, such as an audio or video signal representative of
`sound or video into a digltal form for storing in digital
`form in a highly condensed code and ‘for reconstructing
`the analog s1gnal from the coded digital form. The sys-
`tern includes reductive analytic means where the Orig]-
`nal digital data stream is converted to a sequential series
`of spectrograms,
`signal amplitude histrograms and
`waveform code tables. Approximately 100 times less
`storage space than previously required for the storage
`of digitized signals is thereby obtained. Additive synthe-
`-
`-
`-
`sis logic interprets the stored codes and recreates an
`".“‘P“‘ digitald"“?S"ef““1f°’;1ligi“?11°final‘? °°.“V°’1‘
`S1011
`IS nearly ldCnt1C3. to 1: C orlgma ana Og Slglla .
`3 Claims, 18 Drawing Figures
`
`@371
`PLMER Mg[:_|‘1
`
`‘L3 MBVTES I SEC
`AMAXI
`
`25 MEVTES
`MAX / SEC
`IIZE CHANNELS)
`RAM BUFFER MODULE
`1
`,— ——e-—---1
`1-
`I
`‘l.MP1.1‘1U|)E
`I
`KWAVEFORMS
`1
`, H1Sl’0GRAMS
`1
`:YAELE5
`1
`1
`1
`I
`.—
`_J
`LZ.§°_"B".E_5
`us_°_9_"»:1
`4
`»-—
`1
`V 2 1<av7:5/5:021
`wavsroam nmuzsn
`130 KBYTES 1 55:
`a cons:
`(WAC)
`I
`1
`(AVERAGE) -~\..
`, DISK RECORD ASSEMBLER
`MODULE {Dam
`F
`1
`1-4
`TABLE
`.
`__.L.._.._____l__-._
`1
`T FREQUENCIES 7,wAv: ears 1
`‘AMP
`iwnvzmw
`_
`fl
`,
`K
`"1
`1c.e.1LL0s
`1___.'_°_s2fi_D;K_RE_uE!1.-___1
`f!1IA1NTENANCE_:
`
`.
`
`11
`
`READ / wan:
`MODULE
`
`2405,00 Q, 5E:3,,,,;
`RECORDS 0111 nvsmsz
`DISKETTE ~\\
`
`,_,w;«: nets srcxuci
`~,
`.5: nerves
`
`,
`
`Apple Exhibit 1118 Page 00001
`
`
`
`U.S. Patent
`
`Jul. 21, 1987
`
`Sheet1of18
`
`4,682,248
`
`DATA AQUISITION MODULE (DAM)
`FBROAD- 1.
`U23 BAND _}
`I
`I BAND
`I
`IARRAY
`_J
`
`'
`
`pL_AyER MODULE
`
`2.6 MBYTES
`MAX. / SEC.
`
`(I28 CHANNELS)
`
`L3 MBYTES / SEC.
`(MAX)
`
`RAM BUFFER MODULE
`‘FAMPLITUDE
`«'
`(—V(IAVEFDRMS_1I
`I HISTOGRAMS
`I TABLES
`l
`I
`I
`I
`L'<:E°_*<B*Es_ J L'3_°:<B_:TE:J
`
`I II
`
`WAVEFORM ANALYZER
`8: CODER
`(WAC)
`
`2 KBYTES /SEC.
`
`I30 KBYTES /SEC.
`(AVERAGE)
`
`‘.
`
`.02 KBYTES/
`SEC.
`’
`
`2 KBYTESI
`SEC.
`
`.
`
`DISK RECORD ASSEMBLER
`MODULE (DRA)
`F *-*—“— "'"—I
`
`7'
`,WA7E—RE=s._.] WWEEEORM
`I‘ FREQUENCIES
`IAMR
`_}
`I--4
`TABLE
`J'_
`LREF,
`- "“‘ ““““ ““ —“‘ —
`‘CATALOG
`(____ ’_°'_§EC‘___°'_S_K_5§_°C_’3°__ _ __ :4] _
`'MAINTENANCE_J~
`
`“--
`
`I
`
`DISK
`R EAD / WRITE
`MODULE
`
`240.000 .O| SECOND
`RECORDS ON AVERAGE
`DISKETTE
`
`,,
`f
`/’
`MAG. DISK
`STORAGE
`
`WAVE TABLE STORAGE
`I30 KBYTES
`FIG.
`I
`
`Page 00002
`
`
`
`U.S. Patent
`
`Jul. 21, 1937
`
`Sheet 2 of 18 V 4,682,248
`
`\
`
`\
`
`\\
`\o
`3» "§_’Y“"“‘ 5
`\1
`
`_/_
`
`awvsma /
`aanmdwv
`
`Page 00003
`
`
`
`U.S. Patent
`
`Jul. 21, 1937
`
`001:103tee..nS
`
`4,682,248
`
`
`
`momom>mm_mm_mmmtrm¢
`
`m:o.>mmn_nowomoomm
`
`3‘IVC>S 118 32
`
`Page 00004
`
`
`
`«MU
`
`la2m.Jm6taP.
`
`70091
`
`Sheet 4 of 18
`
`4,682,248
`
`Em%_§u_mm_.ammmmmm.§.mmmmmE
`
`
`
`
`
`.m»._E._m2z<_._ow._m<._.s_moum><
`
`
`
`mm.u_:.zma_mm_n_Fzua_
`
`
`
`Embr:._mzz<_..o
`
`mu_.._:zmo_
`
`
`
`
`
`mo“.uo::.E2<oz<m_o<ommm._m<.rzmEm><u...m<»zmoum?
`
`
`
`
`
`
`
`muozmzmbmmu._<ummm_u_:zmE_mmEF2uo_
`
`
`
`zzozmH02mtm>:m<n_mfioz
`
`Page 00005
`
`
`
`tHCtaPS_U
`
`70091L,2M.J
`
`Sheet 5 of 18
`
`4,682,248
`
`
`
`
`
`
`
`
`
`
`
`zm.Sn.s_ooomo__a._<zmmtn._v.o<—..—.Dn.._.DO.X0...._.Dn_Z_
`
`>._%5m
`
`Egon.
`
`zofiomzzoo
`
`gEII5
`
`
`
`
`
`mm5_.._ozqmmm.u._n.z<mQ2530%
`
`.
`
`
`
`><mm<mN_:ea,6
`
`mm._._oEzoommmomrmom_nuw.m..z<
`
`
`
`
`
`
`
`m>;axmaxmammommmooE-ooo_E_)5z
`
`
`
`
`
`Eozmz,._._2D
`
`
`
`m:_om._.zOomum:>420ofimuzamuooma._<Ezuu
`
`
`
`
`
`
`
`
`
`mommmooE-oooimssz
`
`
`
`mommmoomn_-ooQEMZDZ
`
`
`
`momm.mooE-oo25552
`
`
`
`
`
`mo»<._.__umo._<:o_oQ40:m.E2<mm><mm<mN_._._0_D.m040:
`
`
`
`
`
`
`
`Page 00006
`
`
`
`Patent
`
`Jul. 21, 1987
`
`Sheet 6 of 13
`
`4,682,248
`
`IO DUAL-ELEMENT
`GUN (
`R-CO
`
`ANNING
`8 RANGE‘
`
`WNTER
`
`H
`
`DCOPY
`OR PLA
`SHELF SPACE
`LAYOUT WITH
`‘c
`RENTINVEN-
`T
`
`Page 00007
`
`
`
`‘ U.S. Patent
`
`Jul. 21, 1987
`
`Sheet7of18
`
`4,682,248
`
`‘
`
`TIME OF SWEEP.
`
`I
`
`EDGES OF PRO-
`DUCT YIELD
`fA’2ET?g§“°Y "“F°R'
`
`SIGNAL PROPORTIONAL
`TO DEPTH OF SPACE
`
`1
`
`TIME.
`
`FIG. 7 ANALOG SIGNAL REPRESENTING SHELF CONTENTS
`
`Page 00008
`
`
`
`U.S. Patent
`
`Jul. 21, 1937
`
`Sheet8of18v
`
`4,682+248
`
`V/0/:0 C0/\//L/€C70€.5
`
`//V
`
`I400/0 C’0A/A/r_=‘(’7a€5/A/
`
`.5/C?/I/4./. C0/1/0/7'/0/f//A/6‘
`
`5/GA/41. C‘0,(/£2/7'/a/1///l/6’
`
`.4/V4106" 70 0/6/3271.
`(’0/f/V¢7€5/0/t/
`
`AA/A106 76 0/G/741.
`C‘0A/VE?6/0A/
`
`,e44/av,» 4&2-5.5 Mznaey MM ear;-.925’
`
`(/55? co/1/7z7a¢ 5
`,7/1/0 0/spay
`
`_
`
`€4.’/1.//‘P44 Fracta-
`S/A/6’ 04//r (cf/Ga)
`
`{[40 04/1 4/ /1/E/1¢dx'€fl
`/an/44)
`
`£27/I7
`
`,1/wt/5?/0 C0 —/9@€€.5sa/€
`K/Vac)
`
`,4(’(’[5S /1’/5Wd/€;/ /fl?r‘°A?0(’£'S.S‘//£6‘
`
`Page 00009
`
`
`
`U.S. Patent
`
`Jul. 21, 1957
`
`Sheet9of18
`
`4,682,248
`
`>
`
`400/0 C0/I/A/E'C7D€ cw‘
`
`1//050 C04//1/éZ’mA’5,,0U7'
`
`8/6'/l/AA ca4/0/r/axt//A/6
`
`:5/GA/.44. C0/t/0/7/0A//A/G’
`
`.0/G/74/_ 7a .4A/4406
`C04/V6?sS/04/>
`
`0/6'/744 7a .4/1/4406
`Ca</Vses/cw
`
`0,02-/o,</44 comm/z,e .470
`
`0/I6-C7 /If/4/46V
`4(’(’E'.55 &'V/CE
`(‘OM/I)
`
`D/Sr
`C234//‘€041.56
`
`A/4/M5?/C ea-/46065550/F
`{A/C50)
`
`F0/I4
`
`/C271;/ER
`SOAWLV
`
`Page 00010
`
`
`
`U.S. Patent
`
`Jul. 21, 1987
`
`Sheet 10 oflé
`
`4,682,248
`
`5/GA/AL '4cou/5/7'/ou
`/MODULE
`
`VEfiF§%7!7%EE&Z7
`l|_.____.___.__|
`I______ _..__I
`
`[—
`
`EUFFEE £007‘/A/E
`
`F‘
`
`“ ‘
`
`I6‘//6'5/1’/‘ANA/\/G;|[
`L AA/ALVZEE
`
`‘ " -1
`
`‘F-52/6/-/7'A/E55
`AA/ALVZSE
`
`____ ___l_____________J
`I
`/MAGS AA/AL V256
`I 74 040522 MOLULE
`
`Page 00011
`
`
`
`VU.S. Patent
`
`Jul. 21, 1937
`
`Sheet 11 of 13
`
`4,682,248
`
`OUT
`
`SEE
`Fla 1
`
`S/GA./AL D/51°05/770V
`/W90£/LE
`
`‘
`
`‘
`
`-
`
`,
`
`.
`
`V““3;—';a;.:u;~2 ”“"
`L_T_,__j___ _____T_J
`I
`_l___
`I
`______-_l__
`—
`I
`rmMmmL7
`I
`rbmwmwmf]
`L_5_:’/\/77‘/E5/SQ
`I
`l_ 5)’/k/W/$/5 '
`___
`_____ -4
`T
`‘
`_L ______
`T '
`| F
`|F_flMWM%J
`I
`|
`L5_”’X’f_5’5_I L §’Z‘_’_’_"f*T‘i5_1
`I
`I
`F
`7
`I
`l__1_____1__ _ 1______J_
`-7
`51//-‘F5/e
`/eour/A/5
`__ _____ __ ________________I
`F JMA 55 0500052
`|
`3:" s'‘w'77-/Es/‘s—_Moo.sL
`
`4z/0/0/V/0:0
`5.s:€4e4rae
`A»/004/4'5‘
`
`0/5/c
`0/21 vs
`cm/273044 5/e
`Mona; E
`
`Page 00012
`
`
`
`
`
`U.S. PatentU.S. Patent
`
`
`
`Jul. 21, 1987Jul. 21, 1987
`
`
`
`Sheet 12 of 18Sheet 12 of 18
`
`
`
`4,682,2484,682,248
`
`Page 00013
`
`
`
`
`
`U.S. PatentU.S. Patent
`
`
`
`Jul. 21, 1937Jul. 21, 1937
`
`
`
`Sheet 13 of 13 4,682,248Sheet 13 of 13 4,682,248
`
`Page 00014
`
`
`
`U.S. Patent
`
`Jul. 21, 1987
`
`Sheet 14 of 13
`
`4,682,248
`
`,
`3.»
`%2
`Q"‘
`\u‘°
`51
`‘L
`
`%@
`E5:
`I
`Q“
`Inf
`D
`-3,
`>
`
`Fl.G.IZ
`
`W-TeATIVEUASHEDLmuss
`
`SIGNAL-SANDwl-\rraAREA6Repnasawr
`
`
`
`omu{.-DIFFERENCES\6tdALS.AREFORREFERENCES,
`
`VTlE(V524SECOND)
`
`NOTE;
`
`
`
`
`caossHATE!-IEDAREASREPRESENTRasmvaon-rzemce
`
`
`
`Page 00015
`
`
`
`U.S. Patent
`
`Jul. 21, 1987
`
`Sheet 15 of 18
`
`4,682,248
`
`DIREC -rotzv DATA
`FOE x/coco
`om max ¢
`DATA WAC?‘
`
`DIRECTORY DATA
`FOR Aumo
`K%5“F’%E 555555.“
`LI 2
`a§rAA'§$’eE57§s”§”°E
`
`Lo
`
`ENCODED VIDEO DATA
`
`ENCODED Auouo DA1-A
`
`F|G.l3.
`
`Page 00016
`
`
`
`tnetaPSU
`
`7891L,2mJ
`
`Sheet 16 of 18
`
`4,682,248
`
`m_nm1.5>w\m.m>..42<
`
`mw<:_...6I<mm¢_u
` /V...@_Iu._/nm0._.u<mI
`
`
`\
`
`\
`
`/
`
`_\
`
`\\
`
`/
`
`
`
`fimohwvwwfimwMQ.u<m5zm_u<3<
`a1o.,Am...<:.ou_$
`
`
`Page 00017
`
`
`
`U.S. Patent
`
`Jul. 21, 1987
`
`Sheet 17 of 18 4,682,248
`
`0/us(I)met.(55:FIG.re.)
`
`LBows
`
`48oHm‘§«_‘_zoouTALsCI_§I5s()
`
`0w .
`
`9> 6 D
`
`.
`<[
`
`E:
`
`TRIGTIMULUSVALUE
`
`Page 00018
`
`
`
`nu
`
`S_
`
`tHCtaD1
`
`la2.mJ
`
`7oo91
`
`Sheet 18 of 13
`
`4,682,248
`
`
`
`zmwuwuumuz4z<z3.mafiamoMU{Z423
`
`
`
`Cwd.\oq.w
`.\V.//
`
`9.:
`
`
`
`wm3<>w9.5_z._.m_E.no._<20:..&2mmmmumm
`
`JmvblM20L.(
`
`®_.w_u
`
`.Auqv
`ommn.Qmu2<2<:3
`
`
`
`.nwmm
`
`Page 00019
`
`
`
`1
`
`4,682,248
`
`AUDIO AND VIDEO DIGITAL RECORDING AND
`PLAYBACK SYSTEM
`
`BACKGROUND OF THE INVENTION
`
`This application is a continuation-in-part of U.S. pa-
`tent application Ser. No. 651,111 filed Sept. 17, 1984
`which is a continuation in part of U.S. patent applica-
`tion Ser. No. 486,561, filed Apr. 19, 1983, now US. Pat.
`No. 4,472,747, directed to Audio Digital Recording and
`Playback System of David M. Schwartz.
`Conventional recording of sound and playback is
`performed by electronic systems of the analog type.
`The sound waves from a source being recorded are
`converted to electrical signals on a one to one basis; the
`acoustic sound waves have their analogy in the electri-
`cal current generated by the microphone or pre-
`amplifier circuit such as used in a receiver, turntable or
`magnetic tape source. On playback the electrical signal
`is amplified and used to drive loudspeakers which con-
`vert the electrical signal to sound waves by the mechan-
`ical motion of an electromagnet and speaker cone.
`Conventional video recorders store the electrical
`waveforms, generated by the video camera, represent-
`ing the visual image. The most common memory de-
`vices used to store the waveforms are magnetic tape or
`disk. These devices store an analogy to the electrical
`waveforms in the form of magnetic gradients in the
`medium of magnetic particles. The waveforms may be a
`composite of the color signal or discrete red, green, and
`blue signals, depending on the system. Due to the ana-
`log nature of the system, the noise level is high and the
`results of surface defects are readily seen in the image
`when it is played back.
`Similarly, the output of conventional recording and
`playback systems consists of electrical signals in the
`form of signal waveforms either cut into a vinyl me-
`dium or imposed on magnetic particles on tape. On
`playback,
`the signal waveforms are converted into
`sound waves as described above. The accuracy of the
`reproduced sound wave is directly dependent on the
`quality of the metal or plastic disk or of the tape itself.
`Both the production of disk copies and tapes and their
`means of playback tend to degrade the quality of the
`reproduced analog signal. Noise, in the form of contam-
`ination, wear and the inherent background output of the
`medium itself is therefore unavoidably present in the
`recording and playback systems utilizing conventional
`analog to analog recording and playback technology.
`Recent developments in audio-digital sound recording
`and playback systems represent efforts to reduce or
`eliminate this noise problem. Exemplary of such devel-
`opments are the kinds of systems and equipment dis-
`closed in the following patents: U.S. Pat. Nos. Meyers
`et al, 3,786,201 issued Jan.
`15, 1974; Borne et al,
`4,075,665, issued Feb. 21, 1978; Yamamoto, 4,141,039,
`issued Feb. 20, 1979; Stockham, Jr. et al, 4,328,580 is-
`sued May 4, 1982; Tsuchiya et al, 4,348,699 issued Sept.
`7, 1982; and Baldwin, U.S. Pat. No. 4,352,129 issued
`Sept. 28, 1982, the disclosures of which are specifically
`incorporated herein by reference. These systems are
`characterized generally as taking advantage of the high
`speed operation of digital electronic computers. The
`signal waveform, representative of sound in such digital
`sound recording and playback systems,
`is frequently
`sampled to produce a serial stream of data that is trans-
`lated into a binary code that assigns a numerical value
`for each sample. This can be visualized as slicing up a
`
`2
`continuous curve into a large number of very short
`step-like segments. The process is reversed on playback
`as each numerical value of each segment is converted
`into an output voltage. When this process is done rap-
`idly enough, the fact that the signal wave form repre-
`sentative of a sound wave has been “chopped up” and
`re-assembled cannot be detected by the human car.
`When sound is recorded in digitized binary code in this
`manner, the sound, such as music,
`is only a series of
`numbers represented by magnetic tracks on a recording
`medium which, when read by the appropriate elec-
`tronic means, are either “on” or “off” with no interme-
`diate values. Such binary signals are virtually immune
`to distortion, error, and degradation with time. All
`sources of noise normally associated with analog de-
`vices are eliminated that is, there is no tape hiss, no
`tracking errors, no surface effects. Signal to noise ratios
`are limited only by the digital to analog conversion
`circuit itself and the power amplifiers, rather than the
`sensitivity of the mechanical or magnetic analog to
`analog conversion circuitry.
`These systems do, however, have several drawbacks.
`A representative system currently in use for recording
`master tapes in the record industry has excellent audio
`qualities as a result of a high speed sampling rate of 50
`KHz and good digital binary code resolution in the
`form of a 16 bit word for each sample. The problem
`with this system is that every sample must be preserved
`in mass storage for playback. The storage system thus
`must hold on the order of 4,320,000,000 bits of informa-
`tion for a 45 minute record. Storage systems of this
`capacity are large, expensive, and generally not suitable
`for a consumer product.
`Attempts to resolve the storage capacity problem
`have taken the approach of reducing the resolution of
`each sample (fewer bits per “word”) while at the same
`time reducing the sampling rate to 12 khz). Such reduc-
`tions have reduced the data storage requirement by as
`much as a factor of 4. The resulting fidelity of the out-
`put, however, is often below that acceptable for high
`fidelity sound recordings of music.
`Another approach much favored by telephone com-
`panies, employs the foregoing reduction of bits de-
`scribed above and in addition adds the restriction of
`input signal band width to that most used by talking
`voices (50 Hz to 3500 Hz). A total data reduction factor
`of about 12 is possible in this manner, again accompa-
`nied with a reduction in sound quality.
`Recent attempts at a solution to the storage problem
`and the fidelity reduction problem utilizes ultra high
`density digital storage by laser recording technology.
`This has been partially successful in that adequate play-
`ing times have been achieved with the improved stor-
`age capacity. However, the manufacturing technology
`and equipment presently necessary to create a “laser-
`burned hole”, “pit”, or “black spot” in the storage me-
`dium restricts “laser disks” or “laser fiches” to the
`“playback only” mode with no potential for in-home
`recording or erasing and editing.
`With respect to digital video recording, digital mem-
`ory devices identical to those used in conventional corn-
`puter systems have found use storing very high quality
`images. Small digital memories of 10 to 500 megabytes
`are frequently used as still frame stores for image pro-
`cessors that create special effects and enhancements.
`The digital memories tend to be small for cost reasons.
`Typically, the video images are recorded on magnetic
`
`Page 00020
`
`
`
`3
`tape as they are produced, in analog form, then small
`portions of the tape are digitized and transferred to the
`digital
`image memory for manipulation. When the
`image processing task is complete, the data in the digital
`memory is converted back into analog form and stored
`on magnetic tape.
`The digital image storage and playback systems cur-
`rently in use have two principal problems: cost and slow
`access speed. The high cost of digital memory for image
`storage is a result of the large quantities of data pro-
`duced when analog video is digitized. The wide band-
`width of the video signal consumes memory at the rate
`of 80,000,000 binary numbers (bits) per second. Slow
`access to stored images is the result of the time consum-
`ing task of locating and transferring the desired image
`from analog tape to the digital system, and then back
`again before the next segment can be processed.
`Typical present day digital video recorders are com-
`posed of an imaging system such as a video camera, a
`digitizer, digital memory for frame buffering, and a
`winchester disk or optical disk data storage subsystem.
`These recorders are restricted to non real time opera-
`tion due to the limited bandwidth of the data channel in
`and out of the storage subsystem. The fastest disk stor-
`age device will sustain an average data transfer rate of 25
`less than 10,000,000 bits per second. This is about one
`eighth the rate required to capture continuous moving
`images.
`Solutions to the above problems have been limited by
`the negative complementary nature of the relationships
`between access time, digital memory size, and tape
`transport speed.
`It is therefore an objective of the present invention to
`provide a system for high fidelity sound recording and
`playback that does not have the foregoing drawbacks
`and associated problems.
`It is therefore an objective of the present invention to
`store high quality digital video and audio data in a
`readily accessible, durable, and inexpensive form, and
`to provide a system for video and audio playback of the
`stored data.
`
`BRIEF DESCRIPTION OF THE INVENTION
`
`The present invention, using high density recording
`on a low cost magnetic media, such as a magnetic tape
`or disc or magneto-optical discs, or optical discs in a
`system having a random access memory architecture
`and a unique bit rate reduction scheme for processing
`digital audio and video data, provides a digital audio,
`video recording and playback system.
`SUMMARY OF THE INVENTION
`
`The present invention is yet another approach to a.
`solution to the storage and reproduction problems asso-
`ciated with digital audio recording and playback sys-
`tems described herein and digital video recording and
`playback systems. Good audio fidelity can be achieved
`with limited computer storage capacity by the provi-
`sion of unique electronic signal processing means
`which: (1) converts analog data to digital data stream
`samples; (2) selects portions of the samples to produce
`at least three data streams indicative of amplitude, fre-
`quency and waveform characteristics; (3) stores data
`samples indicative of waveform having a predetermined
`time duration, comparing each such sample of wave-
`form data against predetermined waveform parameters
`to select and preserve only predetermined portions, said
`waveform data samples matching the preserved por-
`
`4,682,248
`
`4
`tions with pre-existing waveform and real time data and
`generating a resultant waveform data code from such
`comparison, and then comparing the selected data from
`the data streams which are indicative of frequency and
`amplitude with the waveform data code to produce
`another data code proportional to the frequency and
`amplitude of the original analog signal, sequentially
`recording the data stream indicative of amplitude, the
`data code indicative of frequency and amplitude, and
`the data code indicative of waveform, onto a recording
`media, for subsequent playback by the processing of the
`sequentially recorded data.
`And if audio and video recording is desired, the fol-
`lowing description will apply.
`A micro computer recording system for recording
`analog audio and video signals in digital data form can
`comprise converting means for converting an analog
`audio signal into a multiplicity of digital data streams
`wherein at least one of the data streams is a relatively
`broadband reference signal representative of the ampli-
`tude of a preselected range of audio frequencies, and
`wherein another of the data streams is produced by
`filtering the analog audio signal
`to produce a data
`stream channel indicative of a plurality of discrete fre-
`quencies encompassed by the bandwidth represented by
`the first data stream; and wherein another of the digital
`data stream is a reference signal representative of the
`amplitude of the audio signal for each of plurality of
`discrete frequencies; sampling means for producing a
`sequential stream of samples in each of the digital data
`streams, selection means for selecting a predetermined
`portion of the digital data samples produced by the
`sampling means in each digital data stream; means for
`separately storing each of the selected data samples
`produced by the sampling means; means for comparing
`the reference data stream containing amplitude data
`with the reference data stream containing frequency
`data to produce frequency spectrogram data representa-
`tive of the frequency and energy of the original audio
`signal; means for comparing the histogram data with
`selected waveform parameters and producing address-
`able data representative of the waveform of the original
`input data; means for sequentially assembling and stor-
`ing the frequency spectrogram data and the amplitude
`reference data and the addressable waveform data for
`subsequent use; and converting means for converting an
`analog video signal into a multiplicity of digital data
`streams wherein the first of the digital data streams is a
`sequential time code representative of the beginning of
`each video frame, and wherein another of the digital
`data streams is produced by filtering the analog time
`domain signal to produce a data stream channel indica-
`tive of chrominance; and wherein another of the data
`streams is indicative of brightness; and wherein another
`of the digital data streams is indicative of pixel spatial
`relationships; and wherein another of the data streams is
`indicative of the temporal frame to frame relationships;
`and coding means for receiving each data stream indi-
`vidually, the coding means including means for mathe-
`matically transforming each digital data stream into
`modified data streams each capable of being subse-
`quently analyzed by comparison of the chromanance,
`brightness and spatial factors present respectfully in the
`modified data streams, and means for selecting prede-
`termined data bits from each of the modified data
`streams after comparison, in a sufficient amount to re-
`construct each chromanance, brightness and spatial
`
`Page 00021
`
`
`
`5
`factors for video presentation, and means for storing the
`digital data bits for retrieval.
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`4,682,248
`
`5
`
`10
`
`FIG. 1 is a schematic diagram of the digital recording
`and playback systems of the present invention.
`FIG. 2 is a pictorial representation of the analytical
`model of the function of the Data Acquisition module of
`FIG. 1.
`FIG. 3 contains a diagrammatic representation of the
`recorded waveform data.
`FIG. 4 is a pictorial representation of a single module
`of binary code as stored on disk, from which reproduc-
`tion will be obtained according to the system of the
`present invention.
`FIG. 5 is a diagrammatic representation of the layout
`of the electronic components used in the present inven-
`tion.
`FIG. 6 is a pictorial representation of a warehouse
`inventory system.
`FIG. 7 represents an analog signal output of the appa-
`ratus of FIG. 6.
`FIGS. 8 and 8:: together are a schematic block dia-
`gram of the digital video recorder in the system of the .
`25
`present invention.
`FIGS. 9 and 9a together are a schematic diagram of
`the software modules for the digital audio and video
`recording and playback system of the present invention.
`FIG. 10 is a diagramatic representation of the trans-
`formed digital, video signal for one video frame (VF,,)
`displayed.
`FIG. 11 is a diagramatic representation of the trans-
`formed digital video signal for the video frame (VF,,...1)
`following that depicted in FIG. 10.
`FIG. 12 is a diagrarnatic representation of the differ-
`ence between the frames depicted in FIGS. 10 and 11,
`or (VF,,)—(VF,,+1).
`FIG. 13 is a diagrammatic representation of the audio
`and video data disposition on a 55" flexible magnetic
`diskette.
`'
`FIG. 14 is a diagrammatic representation of the anal-
`ysis and synthesis factors related to a single digital video
`image picture element (pixel).
`FIG. 15 is a diagrammatic representation of the bit
`map of a digital video frame image.
`FIG. 16 is a diagrammatic representation of the en-
`coding of tri-stimulus values of a single picture element
`(pixel).
`
`DETAILED DESCRIPTION OF THE
`INVENTION
`
`The present invention provides a system for convert-
`ing input analog signals such as audio signals, and/or
`video signals into digital signals and subsequently coded
`into structured data sets for recording in condensed
`digital form; and, for reconstructing a digital data set
`similar to the original digital signal input prior to recon-
`- version to the analog form of signal.
`In its broadest sense, therefore, the recording of the
`audio signals into a digital form for subsequent playback
`is accomplished by the provision of a microcomputer
`recording system which comprises electronic compo-
`nents for converting an analog audio signal into at least
`three digital data streams, wherein the first of the digital
`data streams is a relatively broad band reference signal
`representative of the amplitude of a pre-selected range
`of audio frequencies, and the second of the data streams
`is produced by filtering the analog audio signal to pro-
`
`6
`duce at least one data stream channel indicative of a
`sampled bandwidth of frequencies narrower than the
`bandwidth represented by such first data stream, and a
`third reference data stream representative of the sam-
`pling frequency of the audio signal; sampling means for
`producing a sequential stream of data samples from
`each of the digital data streams, selection means for
`selecting a pre-determined portion of the digital data
`sample produced by the sampling means in each of the
`data streams; means for separately storing each of the
`selected digital data samples produced by the sampling
`means; means for comparing the reference signal data
`stream containing amplitude data with the second data
`stream containing frequency data to produce frequency
`spectrogram data representative of the frequency and
`amplitude of the original audio signal; means for trans-
`forming data samples of the third data stream channel
`selected from the narrower bandwidth into data repre-
`sentative of a time versus amplitude histogram for each
`bandwidth means for comparing the histogram data
`with selected waveform parameters and producing and
`storing addressable data representative of the waveform
`of the original audio input and means for sequentially
`assembling and storing the frequency spectrogram data
`and the amplitude reference data of the first data stream
`and the addressable waveform data for subsequent play-
`back use.
`In the preferred embodiment shown in FIG. 1, for
`digital audio recording and playback the input signal is
`conditioned and amplified in the first stage of the Data
`Acquisition Module (DAM). The DAM is a multichan-
`nel programmable microprocessor based device that
`utilizes standard integrated circuits to perform three
`functions:
`1. To sample at the rate of 42 Khz, hold, digitize, and
`output the broadband (20 hz to 20 Khz) audio sig-
`nal level (dc voltage) of amplitude every 0.01 sec-
`ond. Thus, 100 times every second a digital “word”
`composed of from 4 to 14 bits is created for assem-
`bly as part of a disk record file.
`. To sample, hold, digitize and output an audio fre-
`quency spectrogram every 0.01 second from a 128
`segment array of logical bandpass filters which
`sample 128 channels and are arranged logarithmi-
`cally over the overall band width used. The data
`set produced by this function may range from null
`(no signals on any channel) to (n) [(7 bit
`iden-
`tifier+(7 bit sca1er)'+(2 bit pointer)] where (n) is
`the number of channels with signal content.
`. To act as a digital storage oscilloscope loader,
`assembling strings of digitized amplitude versus
`time data‘(histograms) corresponding to the array
`of bandpass filters selected in paragraph 2, above.
`This assembled data set is produced every 0.01
`second and is the largest single data structure and
`contains time continuous listing for every active
`bandpass filter. The number of “words” in each
`string is a function of the filter center frequency
`and requires as many as 4,000 samples for a 20 Khz
`channel, or as few as five samples for a 20 hz chan-
`nel. This data set'is not sent to the file assembler as
`in paragraphs 1 and 2, above, but is loaded into a
`Random Access Memory (RAM) buffer where it is
`accessible by the Waveform Analyzer and Coder
`module.
`The function of the Waveform Analyzer and Coder
`module (WAC FIG. 1) is to be a digital numeric proces-
`sor array that is programmed to extract characteristic
`
`Page 00022
`
`
`
`7
`waveforms from the data set stored in the RAM by the
`DAM described above. The waveform data are re-
`duced to tabular form in which one period of each
`waveform codified is assigned to one wave table which
`preferably is a digitized x-y coordinate system consist-
`ing of 1,024 bytes along the x axis and an 8 bit “word"
`in each byte location to scale the y axis in 256 incre-
`ments; 127 above zero and 127 below. A set of waveta-
`bles is therefore generated for all active bandpass filter
`channels every 0.10 second. A range of 0 to 128
`(P.M.S.) tables may be generated per cycle (0.01 sec-
`ond).
`The WAC utilizes either one of several P.M.S. reduc-
`tive analytic methods to find waveforms. The first being
`the Fast Fourier Transform (FFT) and the second the
`Fast Delta Hadamard Transform'(FDHT). The two
`methods may be briefly described as follows:
`The FFT is based on the principal that almost any
`periodic function f(x) of period 2 of vibrations can be
`represented by a trigonometric series of sines and co-
`sines. The full expression in general terms is:
`
`_ .l_
`jlx) — 2”
`
`O0
`-0:
`
`W
`-06 j(v)e
`
`iw(x—v)d
`
`v
`
`,1
`
`w
`
`The algorithm for executing this equation efficiently
`was first published by Rabiner & Gold, 1974 and Op-
`penheim and Schafer, 1975.
`The FDHT is utilized for the analysis of spectral
`composition of a data set where the spectrum ‘ll is in the
`form:
`
`W) =
`
`1' f[F — (Fi + 1+ F1)/z]
`
`wher Fi is the frequency and ‘I’ i is signal intensity. In
`the present application of this method the digital output
`of the logical filters from hereinbeforenumbered para-
`graph 2, is summed at each filter and added to the next
`output until all frequencies have been sampled. At the
`last step the total output is:
`II
`11/: _E si+_/Elli
`I=0
`
`Then an estimation of the spectrum \II’ can be found by
`matrix multiplication:
`
`SB.'y,=-,ll—SB.S.;1;=11;
`
`The algorithm for implementing the FDHT was pub-
`lished in 1983 by E. E. Fenimore at Los Alamos Na-
`tional Laboratory. B-splines computational algorithms
`may also be employed to extract characteristic wave-
`forms.
`Ten timm every second the latest produced set of
`waveform tables are sent to the Disk Record Assembler
`(DRA FIG. 1).
`The Disk Record Assembler (DRA) is a program
`module that receives as input the waveform table refer-
`ences (addresses) from the WAC every 0.10 seconds
`and paragraph 2 (above) frequency spectrogram data
`sets every 0.01 seconds directly from the Data Acquisi-
`tion Module (DAM) as well as the digital word repre-
`senting the total broadband signal strength. The wave-
`
`4,682,248
`
`8
`form tables are kept in a local memory buffer in the
`DRA,so that they may be revised or discarded every
`0.10 second cycle by a subroutine which for conve-
`nience will be called Waveform Catalog Maintenance.
`Disk records (FIG. 4) for storage are variable in length
`but always follow this format: