`
`Volume 25 Number 1 1992
`
`«$6
`«x
`- ‘§.§’§‘(“{}3®\
`a§\‘ .(i::\3‘§\
`‘S5
`
`ART
`_
`mnvsnsrrv or: MARYLAND
`
`MAR IF?
`
`Journal of the
`
`International
`
`.1
`
`1 _
`
`lI
`
`I Society for the
`
`Arts, Sciences
`
`and Technology
`
` > Pergamon Press
`
`1
`
`(JXFORD - NEW YORK I
`
`I-iE.0L3I.
`
`-
`
`Page 1 of 11
`
`Harmonix Exhibit 1007
`
`
`
`EDITORIAL
`
`RUG1-LR F. MALINA: The Cultural Dimension of Space Fxploration
`
`GATEWAY
`
`DENISE EM. PENRosF., WlLI.IAM C. C.-\S'l‘F.L1.,RO(Jl£R F. MALIN.-X
`
`SPACE ART
`FR:\N(21SCt)INl-‘ANTI~‘.: Projects for the REC(‘Il"lS[I'llCli0l'l of the Firmament
`CARI. L. DEVITO: 1.zInguages, Science and the Search for Ext.1'nterI‘es1rial Intelligence
`Ml(llIAEl. L. HF.l\-'|.‘I' WITH MICHAI. REED; The Space Between the Real and the Imagined:
`Microwave Sculpture in Deep Space
`
`ARTISTS’ ARTICLES
`ATTIIA USA]! AND NURBRRT KROC): The Application of Lasers to Conipose Pictures:
`The Method of Superposifroning
`LYNN P()C:()CK—W1LI.IAMs; Toward the Automatic Generation of Visual Music
`
`ARTISTS’ NOTES
`GEORGE GESSERT: Notes toward a Radioactive Art
`CAJ.1w-\N SHEMI: Soft Painting and Its Technology
`I-lORA'(:IO ZABALA: The Image of Duplication
`BARRY TREU: My Electronic and Computer-LloIItr0I|ed Sculpture:
`Robotic Techniques Applied to Kinetic and Imeract.i\'e Sculpture
`RON.-\l.D R, l5RowN: The Use of the Knighfs Tour to Create Abst.r-.u:t Art
`
`'i‘I:cIINIeM.AR1'IcLE
`'1‘AM..\s 1.?Nl,:\'.-\R‘I"_ SIMON V\'.«\'i'F.RS AND PETER R«\_]K.«\_: NUNTIUS: A (_'-Imnpuurr S_\_'3l.enI for the
`Interactix-‘e (1orrIposition and An2I|_\_-'sis of Mus-;'Ic and Dzunre.
`
`TECHNICAL NOTE
`DANN 15.. P,-\55(i)_].-‘I AND AK}-II.l~;sH LAKJITAKIA: ('Jarpets and Rugs: An Exerctise in Numbers
`)5} TNF"
`
`|'
`
`11
`
`13
`
`17
`
`23
`
`29
`
`37
`
`43
`
`47
`
`51
`
`59
`
`69
`
`,L L! 3:.
`v- =35
`F9.‘-v'°'-:"
`Contents Co-ntirmes on Next. Page
`
`REONARDOJ
`
`I?‘
`
`Page 2 of 1 1|OURNAL OF THE INTERNATIONAL SOCIETY FOR THE ARTS. SCIENCES AND TECHNOLOG
`
`Vofume 25 Number I 1992
`
`Y
`
`Page 2 of 11
`
`
`
`THEORETICAL PERSPECTIVES ON THE ARTS, SCIENCES AND TECHNOLOGY
`DANIEL L. C()l..L!NS: Anamorphosis and the Eccentric Obsewer:
`Inverted Perspective and the Construction of the Gaze
`W. RAY CROZIER AND PAUL GRF.l:‘.NHA.l.(}H: Beyond Relativism and Formalism:
`
`The Empathy Principle
`
`ABSTRACTS
`
`MATFS R005; Two Observations
`WiLLtAM_|.JOEL: Pseudo-Stereographie lma es
`STEVE MANN: Dopp'lerDanse: Some Novel Applications of Radar
`
`X-NN HERSHMAN, CHR1s'rO1=nER_]ANNE\',]1MJOHNSON,
`CHRISTIANE GEOFFROY, 1
`A RAHMANL PALJL SERMON
`lVIARY_]E.AN KENTON, MICHAEL MCNABB. STEPHEN MOORE. Avrv
`
`WORns ON WORKS
`
`Book Reviews by RUDOLF ARNHI-;1M, DAVID CARRIER,JOYTURNER Lu!-us, ROGER F. MALINA.
`CRAIG ROLAND,_]. WILLIAM SHANK, CARTER S(‘.l-I()I..'£.
`
`CURRENT LITERATURE
`
`SPEAKERS’ NETwoRi<
`
`73
`83
`
`89
`9091
`
`93
`
`98
`
`105
`
`0
`
`Front cover: :\leksand1‘a Maficzalt, Imaginaty Inndscafae, installation of glass and human hair,
`30 X 25 X 30 cm. 19’?6. This work is One in a series of photographs in which the artist has arranged
`spatial relationships between geolnetrical elements and objects from nature.
`Bark rottm‘:Ale-ksandra Manczak. Between. Sensibility and Rationality, ink drawing, 1982. This drawing,
`inspired by a photograph depicting a banal situation, is intended to create a surprising impression
`through the unusual proportions of the ball ofrope, the Lench and other elements.
`
`l’lr'?1.i£e ever}! tffofl is nutde by the publis
`appears in thisjournal, they w-tlsh to makeit dearthat thedata and opinions appeurmgin theanti: ..
`arethesoleresporrsibiiity oftheruntn'butm'oradvertisermnremed. Accordingly, thepublishers, theerfitorint bormi and editors and
`theirrespertiueetnf;to)'m, o £tTe't”.i and agents «crept no mspamibftity orliubitity whatmerterfnrthe cmtsequenres 9} anysuch
`Enar-cu rate or -tnisteading data. ofnlniun or statement.
`
`Pagesom
`
`Page 3 of 11
`
`
`
`i\RTIST’§ AR'[‘|.CLE
`
`Toward the Automatic Generation
`of Visual Music
`
`Ly-rirri, Pocock-Williams
`
`SItrt.d.-it of color: r, like (lime tifsottnrt‘, (ire ofrt irttitrh. firm‘
`textmit and azimlir in the sou.-7 emotions toofim’ to be ex-
`pressed in words.
`
`1 .‘)l4
`—-—Wass:'ly Kritirlirislay,
`The power of music combined with the. expressi\*eness of
`ln’l:'tgEI'}-‘ can result in a unique artistic expctience. Dating as
`f:11‘ back as the age of Aristotle, there have been numerous
`attempts through recorded histong to integrate sounds and
`images [1].
`In the fifteenth and sixteenth centuries,
`Leonardo daVinci sketched his ideas about the relationship
`of music to color [2], and, more recently, artists such as
`Wassily Kandinsky and Paul Klee expressed their ideas
`through poem; and prose, as well as through paint. Klee
`explored the notation of rhythm and music in an intellec-
`tual and intuitive man net‘. as can be seen in Ancient. Sotmd,
`.-‘l{JSt".l'aCl' on Black and Posture! {Rh}-thiri.t). Kandinsk}-', on the
`other hand, often created paintings that were intended to
`be ‘heard‘ by the viewer and of which words could not
`explain [3]. as seen in his synesthetic painting Green Sound.
`
`Fig. 1. Homage to Kandinsky: Improvisations. videotape, 1939. This
`image shows a cluster of circles and tt'ianglcs——vnc-at-ulary typical
`of Eandinsky’s Bauhaus paintings.
`
`i\BSTRAC'l'
`
`ln this article. the author ore
`sents her research. involving the
`integration of sound and ‘image.
`which resulted in the design of a.
`computer-software system and the
`creation of titre computer-animated
`works. She discusses her three
`'pretteterrn'ined't:omputer-graohic
`animations that were created by
`the use of traditional computer-
`graphic techniques. She also
`details her two ‘automatically
`generated’ animations resulting
`from the creation of a computer
`system that uses rule-system tech-
`nology to assist in the automatic
`translation of sound to "image. All
`tiue animations illustrate visual
`expressions of music.
`
`511?.
`
`.
`
`l
`
`l
`
`‘
`
`'
`
`-_
`
`I
`
`Musicians have had similar
`concerns. Scriabin scored iniisic
`for an orchestra as well as for a
`‘color organ‘ for his perform-
`ance at Carnegie llall. ln addi-
`tion. there have been collabora-
`tions between nittsicizuis and
`Musician
`artists.
`l eopolcl
`Stokowslti teamed up with artist
`Thomas Wilfred to create a
`‘light show for a performattce of
`Rirrisky'—Ko1'sakov’s .S'clie!t.erri2.ride
`[-1].
`Danceixs also concern them-
`selves with the integration of
`sound and image. A dancer
`nioves through a three—dirnen-
`sional
`space,
`syiirhrtinizittg
`each step with music. Content-
`poraiy
`choreographer Kay
`Knight is iiivestigatitig the ap-
`plieatioti of computers to dance. She is currently using a
`cornputcr-graphic program, (.‘.OMPO,SE. to create graphic
`dancers and tnove them through a stage—like space I5]. In
`the future, programs like COMPOSE may also assist in the
`synchronization of dance to music I6].
`Also important in the liistor}-' ofvisual music are the works
`of early" experititental iiltnttiakei's who sought to create a
`teiriporal vistial experience that appro'.ir.lied tnusic. Film-
`051-lat‘
`l-'iscliinger and I\iorman
`niakc rs llans Richter,
`MrLarcn developed persoiial visual and temporal laiiguages
`that correspond clirectl_\-' with music.
`'l‘lieir films present
`laiigttages in motion. coupled with t'.orrespot'iding music. as
`can be seen in Rh}-tiiiit 21 [7], Allegretto [8] and .'i‘p!i.vres [9].
`Contetnporan_,' artists sttcli as _]tihn \'\-'l1itiie}-'. Sr., Lltl'l'}'
`Cuba, Robert. SIl}'(l{,'t' and lirlward '/,,a_iet: use computer.-s to
`piesent images that are closely linked to music, music con-
`cepts or sound. Wl'titne_t-' developed a lormal relatiorisliip
`between music structure and visual structure that he ex-
`presses through the use of abstract eleinetits such as points
`and lines I10]. as seen in Pei'mu.tritt'ons [ll]. lri Cu£ru£i:t.i‘.ed
`Miii.rmiimt.t 1'12], Cuba closely correlates the rnovemciit of
`ahstract shapes with alistract sounds. Snyder integrates
`sound and image by using an equal—tempei-ed l<erl.io:1i‘tl to
`control colored light [13]. Finally, in Chromas,
`'/.a_jec or-
`cliestrates "the flow of color passage in time", creating a
`visual counterpart to the correspondiiig music I14]. While
`
`Lyitti l’ortit:k \‘\-‘ill'i-.ims r_-.irtisI. ed-traitnrt. licpai-tiitt-iit of{'omputei' (‘-inpliics. Pratt
`Institute, Hi-onltltgti. .\'i'
`I
`|'2{J."i,
`l_'.S..-X.
`Received ll .-Miguel
`l‘.)fiEl.
`
`flzillf-_—. til: 1"irIted in Great Britain
`U024-094K-92 $3.00 t6 (1
`
`Ll-,t }E\3.-\Rl}(J. mi. 25-. No I. pp. ‘3Fl—:5ti. 19.192
`
`gt}
`
`Page 4 of 11
`
`
`
`Fig. 2. Ot:carrr’s
`Razor, videotape,
`1939. Computer-
`graphic imagery
`is overiaid onto
`recorded live-
`action source
`video.
`
`
`
`“It the canvas is blank. then begin :1
`cotriposition by drawing ".1 circle near
`the middle of the screen" can be trans-
`lated into computer
`languages and
`entered into the computer. This ena-
`bles an artist t.o turn the computer into
`a cirstornized tool for the generation
`and analysis of art. An example of such
`a rule system resulted from my research
`with Raymond l.aur.zarra in 1987. We
`entered rules formalizing Kandirtskfs
`compositional style.
`into a computer
`and then used them to generate origi-
`nal compositions in the style of Kandin-
`sky [24]. Another example of this is
`Harold Coherfs
`program called
`.»‘tARO.\£. which has been programmed
`to assist Cohen in the creation of im-
`agery [‘2:'3'l. In addition, Russell Kirsch
`and joan Kirsch use a computer lan-
`guage to describe to a computer the
`linear structure of Diebenkorn’s Orerrrt
`Park paintings. Vt-’l1ert
`these rules are
`invoked, a composition in the ‘Ocean-
`Park style‘ is generated [26].
`This paper presents research that in-
`vestigates the integration of sound and
`image through the use of rule-system
`technology. Tire result of this research
`is twofold, First, three ‘predetermined‘
`w'(‘. ["C
`computer—graphic
`animations
`created by rising traditional compute r-
`graphic techniques. Second,
`at com-
`puter system was created that uses rule-
`system tec.irr1olog}-'
`to assist
`in the
`automatic
`translation of sound to
`image. The translation is hased upon
`charac-
`theoretical
`and emotional
`teristics of music, with pitch and dura-
`tion being the main elements oi'analv-
`sis.
`Two ‘autornatically generated‘
`have resulted from this
`animations
`trarrslation process. While limited in
`the nnnrher oi" elements arr'.ily'.terl, all
`live animated works are first steps
`towartl a complete \‘isttal artrrlrsis and
`expression oi" llll.lSl('.
`
`THE VIDEOS
`
`Background
`I created
`Before this research began.
`arrirnatiorrs without a well—dei'tncd sys-
`tem. I would start with an already ex-
`isting piec.e of music. or I would lravc
`the music created specifically for an
`animation that
`I planned to create.
`Since music would conjure up images
`in my mind, 1 would listen to the music
`nrarty times until I had a sense of the
`overall quality,» of the corresponding im-
`ages. 1 would then write computer pro-
`grams to create the animated work.
`
`these artists have concerned themselves
`aspects of integration,
`with broader‘
`other artists have been concerned only
`with the connection between color and
`pitch [l5,l 6'1.
`Alt.lroug’tr this history iirtlicates that
`there is a strong htrman need to inte-
`grate sound and image . visual mttsic. has
`often been met with skepticism and
`indifference. A primary reason may be
`its strong dependence on tec.ltrtt)ltrg)'.
`Because the presentation of trew visual
`music was generally not possible with
`existing tools, artists have had to be
`scientists
`and
`inverrtors-building
`elaborate devices and working with com-
`plicated eqtripment-in order to real-
`ize t.heir visions. Mot'eo\'er, limitations
`in technology have occasionall_\' kept
`svnesthetic art from being shown Hill.
`This teclrnological depentlence will
`continue: however. as teclnrologv ad-
`vances. so will si'ncsthetit' art. Witlr
`computer tet'hnolog_\' lreronritrg mort-
`accessible and flexible. the tt-t:lrno|ogi—
`cal lirnitations o|'Ihe past sliould he less
`sigrrifieant in the iittture.
`.-\ttother ro:ttlblock I'or vistral nutsit‘
`rrra_\-' be its association with lilnr anti
`Au tr—!*iar'g ctrt
`vicleo. Acc.or'dirrg
`to
`Wooster. “Because Film and \'it'lco un-
`told in t.inte. arrtl laecause tltev possess
`histories and conrentiorrs quite sepa-
`rate [torn those of other \-'isttal arts, art
`critics freqttetrtlv sh_\' awav from discuss-
`ing such work" [ IR 1 ._lt ishopetl that this
`factor will be less of an issue astilrrr and
`video become nrore acceptable. A final
`concern regarding svnesthetic art has
`been the "glib comparisons” tirade be-
`tween sound and image [19] . .-Xltlrotrgh
`artists such as Klee and Kzmdinskr have
`been well retteirecl. the arts lit.et‘at1tre of
`this centun_' ignores trratrv s~_.'t1t-stlretic
`works. This resportse is not sttt'pr‘i.~;ittg_
`
`Visual mtrsic involves a very complex
`union of images and sound in order to
`create an experience that is somehow
`more expressive than the sum of its
`parts. The clrallenges involved are
`enormous, and therefore strccess has
`been limited. although the works of
`iidward Zajec and
`jolt n Wit itnev. Sit.
`Vibeke Sorensen [20] have shown that
`successful results are possible.
`My personal interest in vistral music
`first btfgarr when I read Karrdinskvs
`analogies between sound and image
`[21]. His ideas and my love of music
`prompted me to start my own explora-
`tion. Arrotlret‘
`influence was my
`husband, musician Robert Williams,
`through the many conversations we
`had regarding his music. 1 would often
`describe the irnpact ofhis work in visual
`terms for
`I could t1nderst.and and
`appreciate. the music rnore. deeply i[' I
`could sorrrehow ‘see the sound. Finally,
`nrv ['anriliarit_v with the current state of
`rttle s_\'stt-ins in the arts. as described
`ht-low. led me to believe that a \'ial)le
`2l[)pl‘l'.|2t('l1. to \'iS1t:‘tl music might incor-
`porate rtrle—si'steni technology.
`The (lert-loprnt-tit oi" formal rules b_\_'
`which artworks are created and ana-
`l_w.e(l has ot'c.urr'et:l tlrroughout history,
`from the ancient Greeks to the late
`twentietlr—t:enturv postmoder‘nist.s, with
`each art movement strictlv following its
`own aesthetic codes. For example, Was-
`sily Kantlirrskv and Paul Klee presented
`rules that fortnalized connections be-
`tween vistral entities and spiritual re-
`sponses in the l920s and 19305. as seen
`in Kantlinskifs (.‘rirr.o?mr'rt.g thr?Spirtt'.rtrtlt'n
`xlrt [221 and Klee's Ptvtt'(tgogrTcal Sketch-
`book [23].
`is possible to define
`At present, it
`rules, or instrtrctions, that can be un-
`derstood by cornpttters. Rules such as
`
`Page5of11
`l“m'r-dt-ll}it'nrm.~. T-rimrtl tilt‘ .\rrt:-tn.ttrc (it-In-t';tti:.-it tat \'isu'.tI -.\lnsit
`Til}
`
`Page 5 of 11
`
`
`
`more t'lt'an'Iatic, and. in respotise, the
`colored shapes begin to pulsate like
`visual heartbeats.
`1 min, 1989 (Fig. 2},
`()rrrtm's Razor.
`combines
`abstract
`and
`repre-
`sentational imagery. Aseries of parallel
`lines—-the predominant abstract form
`in this aniniation—-appear in the fore-
`ground of the frame. Although these
`parallel lines appear well controlled,
`they also possess a ‘uerx-‘ous‘ quality be-
`cause their movement is at times er-
`ratic. In addition, recorded live-action
`video can be seen behind the fore-
`gron or! lines. The resulting video is fast-
`paced and discordant, and the ending
`is on resolved.
`'v’i!'tucIE Reality, 3 min, 1989 (Color
`Plate 13 No. 3), also combines abstract
`and
`representational
`iinagery.
`In
`general, the. music in this animation is
`fast, and the visuals move with similar
`speed. The tempo is increased even
`more through the addition of con-
`stantly moving layered background
`scenesofocean wavesand a performing
`musician. The visuals express the ex-
`citement, flow and pace ollthe corre-
`sponding music.
`
`A similar approach was used to cre-
`ate my three ‘predetermined‘ anima-
`tions. The imageswere created through
`computer prograrns consisting of a ser-
`ies of graphics commands and sub-
`routine calls. The pieces were I'ealiz.e.d
`when these programs were executetl.
`My two ‘automatically generated‘ an-
`imations were generated via a transla-
`tion process, which is a step towards a
`formal approach for integrating sound
`and itnage.This svnesthetic integration
`does not vet refer to all the elements of
`music—sucb as timbre anti volume——
`but only to melody and pitch. The inte-
`gration is not always one-to-one, but
`nevertheless the method is a significant
`first step towards the full integration of
`image and sound.
`1 developed an abstrac.t visual ‘lan-
`guage’, based upon color, linear move-
`ment anti geometric shapes,
`to give
`graphic expression to the music. The
`abstract quality of this language is ap-
`propriate for the integration of images
`and sound clue to the abstract and struc-
`tured nature of the music it is meant I.o
`express. Changes controlled by the
`computer-graphic component include
`a variety of time-syntactive operations
`sttch as scaling. translating. pulsating
`and replicating [27], with special atten-
`tion paid to the ways in which objects
`enter, exit and cross paths [28].
`This visual langtiage. helps bring a
`new world to life. In the animations.
`there is usually one moving object vis-
`ible at any given time, with :1 stage-like
`background for the object
`to move
`around in. The objects, along with the
`ways that they move. give a glimpse iIuo
`a nonexistent world oi" abstI'at'tion and
`structure, while their lluid movement
`rellects the fluidity oi‘ music. \\’ben on
`videotape, each piece is act:omp:mietI
`by its corresponding music. rt-inf: n't'in5_r_
`the connections be-I.we.eI1 the two [25]].
`
`The Automatically Generated
`Animations
`ThcF37rst Generrttttm, 3 min, 1989 (Fig. 3) ,
`is mv first ‘generated' animation. The
`repetitive anti unstable quality of the
`notes in the music create an impression
`of machine parts; hence the visuals ex-
`press a similar quality. Diagonal lines
`and piston—like movement. of windows
`into the recorded live—actiou video eru-
`phasize the mootl of the ntttsic.
`The Mvxr Ceitmition, 112‘; min, 1989
`(Fig. 4). is rnv second ‘_t=;enerated' ani-
`mation. The images are entirely ab-
`strtuit, and thev express the wliintsiral
`mood of the music through somewhat '
`llIlt‘X|')(‘t‘.I("(l tI‘.0vt'-m('I‘It. A lnoving, grav
`sttrfatre, a moving point and a static
`stain-painti1'ig—lilte
`snrl'ac.e. constittttc
`the visual vocabtIlan__'.
`
`AQTOMATICALLY
`GENERATED ANIMATIONS:
`METHOD AND
`IMPLEMENTATION
`
`Sound-to-image integration consists of
`five major steps: (1) music-data input.
`(‘.21 musicdata analysis according to a
`predefined strategy, (3) decision-making
`according to a predefined set of rules_.
`(4)
`animation generation and (5)
`transfer to videotape.
`
`The Predetermined Animations
`Hornrtgc to Kt! rtt1'.th,t'k}I.'
`ltnfJrom'.s'rrIiIJrI.
`1
`min, 1989 (Fig. 1). takes its \'{l("ttl)tllill‘)'
`from Kartdinskvls l'Sat1haus paintings,
`with c.ircles, rectangles and triangles as
`the primary shapes seen. These shapes,
`however. are shown predominantly in
`tones of gray, with only a few of them
`containing color. Music [or this video is
`subdued, and the corresponding im-
`ages and motion express ‘it similar
`mood. Throughout the first hallo!" the
`animation, circles. rectangles and trian-
`gles emerge graduallv out of a black.
`vertical field. l-lalfwav through the ani-
`mation, the mood of the music becomes
`
`Fig. 3. Was First Generation, videotape,
`1989. This sequence of frames shows a
`progression of images from the author's
`‘automatically generated’ animation.
`
`These live steps enable the automat-
`ic generalion of an animated work that
`directlv correlates to music. I selected
`an Amiga computer for this research
`because of its ease of use with video, its
`atlordabiiitv and the flexibility of avail-
`able software for animation and sound.
`With the exception of Director. an ani-
`ination—programrning language. and
`SoundScape Pro MIDI Studio 150], a
`music—computer-software package,
`I
`
`Page6of11
`
`P.-marl: ll'it'h:tm.I. '|'on'-.nI't! Ilu- .-\uIoniaIic (Sent-rantm ol'\"ist1al i\|IIs1t'
`
`31
`
`Page 6 of 11
`
`
`
`wrote all 01" the st'1i'twai‘e that I will dis-
`russ.
`
`allows
`software
`input
`Mus-ic—data
`music to be entered into a computer.
`Sottndhcape Pro MIDI Studio, the pri-
`music—(lata
`[1'l.?l.l‘_\-'
`software used for
`input, allows a musician to enter music
`into the computer. play the music
`through the coinputcr’s speakers and
`‘save’ the music in a standard format.
`Once the music is entered in, it is
`then ‘extracted from Soundscape via
`secondary software called Extracter
`I31].
`17.:-ttracter
`is
`linked into the
`
`Soundficape environ ment and extracts
`music data that is needed for anal}-"sis in
`step 2, rnusic-data analysis. This data is
`written to a character File and Finally.
`before it is used for analysis, it is con-
`verted to an integer format via a pro-
`gram called (lonvc1'te1' and written to
`another file called Music. Pitch and du-
`ration are written to the Music file for
`each note of the melody.
`Once the Music file exists, the music
`is analyzed according to a predefined
`strategy, via Analyzer, a C computer-
`language program. In the second step
`
`Fig. 4. The Next Generation, video, 1989. Two images from the author's second ‘automat-
`ically generated’ animation.
`
`of integration, Aiialyzer reatls a l0—sec
`music phrase from the Music tile, deter-
`mines u ‘siinplificd pitch—time graph‘
`for the phrase and then searches for
`higher order patterns in the graph. Fi-
`nally, the ‘predominant higher order
`pattern‘ is determined.
`This simplified pitch-time graph is a
`graph of slope types. For example, if the
`pitches in a sequence of notes are all
`increasing,
`then those notes are re-
`placed by an UP slope type. If the
`pitches in the next sequence are all
`decreasing. alternating or are all the
`saine, then those notes are replaced by
`:1 DOWN, ALT or FLAT slope type,
`respectively. The slope types of UP,
`DOWN, ALT and FLAT were chosen as
`part of the analysis process because they
`are the basic components of any pitch-
`tirne graph and because they can he
`tutdcrstood visually.
`Therefore, assuming the order of
`pitches from low to high is as follows:
`a b c d e i‘,
`
`a l0—sec phrase of notes that looks like
`acac
`cdf
`arac
`cdf
`fdc
`[ti C
`fdc
`acac cdf
`
`would be re placed by a simplified pitch-
`time graph that looks like that seen in
`Table l.
`
`1
`
`i
`
`I
`'.
`
`T |
`
`Table 1. Simplified |:tltcl1—time
`graph created from the pitch and
`duration values from Tahie 2.
`ALT L"-P 'l'}O\~\-'.\’ ALT U!’ 'DO\-\"l\‘ ALT 'L' l‘ DU\'\«'N
`
`1
`
`I
`:
`.1
`
`it is clear that a pattern occurs within
`this phrase. The pattern is ALT UP
`DOWl\’. it occurs three times, anti it is
`the ‘predoniinant higher order pattern‘.
`Hence, Analyzer reduces the phrase to
`A] II‘ UP DOV\-“N. In essence. .-’\1121l}'7.er
`analyzes the t‘J1€'.l0Cl}-' and reduces it to
`its predominant components.
`Dependent upon the predominant
`higher order pattern of the simplified
`pitch—time graph, decisions are made
`by the program regarding the visuals
`that best suit the music with the given
`graph characteristics. In step 3, Anal}
`zer tandotnly selects an animation phrase
`from the several animation phrases that.
`have been preclassified as being visual
`expressions of music containing the
`predominant higher order pattern.
`Once a phrase is selected, step 4 follows
`and a subroutine call to 3.10-sec anima-
`tion phrase that corresponds to the 10-
`sec music phrase is written to a [iie
`
`
`
`Page 7 of 11
`l¥::'rar.':—ll-it’.liuni_~, 'l'o\x'ai-(l Ihe Aulotnatic l_}t'ner'.ilion of Visual! Mimic
`32
`
`Page 7 of 11
`
`
`
`
`
`
`Fig. 5. Graphic representation of the translation process. (above) Here, music is read
`into the computer, converted to the appropriate format and then analyzed. An anima-
`tion is generated and, (right) alongwith the corresponding music and additional
`camera video, transferred to videotape.
`
`type. One will be randomly selected for
`synchronization to a music terminal.
`During the final integration step. the
`visuals and music are combined with
`camera imagery and then transferred
`to videotape. To simplify the initial
`stages of the research, only the predom-
`inant line ofrnusic was entered into the
`computer for analysis. Other litres of
`music can be heard on the final video-
`tape, but
`they were not considered
`during the analysis process. Recorded
`live—action source video images were
`added to the animation to represent
`the lines of music that were not ana-
`lyzed.
`Figure 5 shows a graphic repre-
`sentation of the translation processjust
`described.
`Figure
`5a
`shows
`the
`sequence of steps from musicdata
`input to the creation of the animation.
`Rectangles are used to denote software,
`while a curved page symbol is used to
`denote data. A television symbol is rrsed
`to signify the resulting animation that
`has not yet been recorded on video-
`tape. Figure Fib shows that
`the final
`videotape is constructed from the
`union of the auitnation, the music and
`additional r'ccorded li\'e—action video.
`Figure 6 shows the translation process
`in the form ofari algorithm, which pre-
`sents two sets of step-by—step instruc-
`tions. The first set presents general in-
`structions such as ‘run converter’ to
`convert the data to integers. Tire ser-
`ond set ofinstructions presents the fun-
`damental steps of the 2'5LI'I'r1ly'LCT/Gt:llCl'—
`ator portion of the translation process.
`
`AN EXAMPLE
`
`further
`is
`integration process
`This
`clarified by a desc ription of the creation
`of the video for The Fast Gctrreratian.
`Before the process began, a library of
`animation terminals was created with
`each terminal classified by style and
`
`called GenAni. Steps 2 through 4 are
`repeated until all of the mtrsic has been
`analyzed.
`As previously stated, the music is an-
`alyzed in l0~sec phrases. These phrases
`are called music terminals and, as such,
`
`
`Table 2.
`Pitch and duration values for the
`first rrtttsic termi11al's musical
`notes for The First Generation.
`
`Animation Phrase
`
`Pitch
`16
`14
`16
`14
`16
`14
`15
`16
`19
`21
`22
`21
`19
`16
`14
`16
`14
`16
`14
`15
`16
`19
`21
`22
`21
`19
`16
`14
`16
`14
`16
`14
`15
`16
`19
`21
`22
`21
`19
`18
`
`Duration
`12
`12
`12
`12
`24
`12
`12
`12
`12
`12
`12
`12
`12
`12
`12
`12
`12
`24
`12
`12
`12
`12
`12
`12
`12
`12
`12
`12
`12
`12
`24
`12
`12
`12
`12
`12
`12
`12
`12
`12
`
`they are the smallest music componen ts
`considered for analysis. A similar idea
`is applied to animation phrases. That is.
`animation terminals are the smallest
`animation components, and they are
`approximately 10 sec in length. For the
`initial stages of this research, it is natu-
`ral for both types of terminals to be of
`fixed length. If the music terminals
`were allowed to be variable in length.
`then the generator software would have
`to be intelligent enough to be able to
`alter the animation terminals so that
`they could run for a variable amount of
`time. Hence, for simplicity the termi-
`nals are of Fixed length.
`-
`It should be emphasized that in step 4.
`when an animation is generated, the
`l0—sec animation terminal is not itself
`generated. Rather, a subroutine call to
`a predefined animation terminal
`(li-
`brary routine) is selected. These anima-
`tion terminals are written in Director.
`In the end,
`the entire generated-
`animated work is a collection of sub-
`routine calls wt itten into a GenAni file,
`and the resulting video is a synch ron i-
`ration of animation and music termi-
`l‘1d.lS.
`The mapping between music termi-
`rials and animation terminals is based
`upon the rnovenrent within the termi-
`nals and upon irrterpretation. For ex-
`ample. if:1 music terminal reduces to an
`alterrtatlng pattern, ALT. then the corre-
`sponding animation terminal will have
`an alternating visual quality, such as an
`u[rand—down movement. Because there
`is no absolute or scientific connection
`between an ALT music terminal and an
`alternating animation terminal, this in-
`tegration remains an interpretation.
`However, the animation and music ter-
`minals oppear synchronized when seen
`and heard together, due to the alternat-
`ing quality of each. A library of anima-
`tion terminals, classified by predorni-
`nant
`pattern
`and
`overall
`visual
`qualities, is initially built and then used
`during the generation ofan animation.
`There may be several animation termi-
`nals classiiied for each music-terminal
`
`Page8of11
`
`Prrr:or:k-li-'r'Hr'artrx, Toward tin: .-\urum:tIic f‘-r-iteration ol'\'isu'.r1 Music
`
`33
`
`Page 8 of 11
`
`
`
`higher order pattern, a decision was
`made
`regarding which :mimation—
`terrninal—library routine best expressed
`a music terminal with the given pre-
`dominant pattern. When an animation
`terminal was selected, a call
`to that
`anirnation-library routine was written
`to a Gen;-\ni file.
`The above procedure was repeated
`for each music terminal of the melody.
`When the analysis was complete, the
`Gem'\ni file was executed, and the re-
`sulting generated animation was real-
`izccl.
`As previorrsly stated, Fig. 7 presents a
`subset
`of
`thc
`animation-terminal
`library for this animation. Currently.
`there are a total of 21 terminals. The
`terminals in this figure illustrate termi-
`nals that have been classified as being
`visual rcprcscnt.ations of music termi-
`nals with an ALT UP DOVVN predomi-
`narn higher order pattern. It
`is this
`predominant pattern in the music that
`creates the impression of alternating
`nrovcrncnt of pistons or machine parts.
`and, as the figure illustrates, the anima-
`tion terminals classified for ALT UP
`DOWN all contain a pumping type of
`movement to express this sonic quality.
`Figure 8 presents one of these anima-
`tion terminals with its corresponding
`rnusic terminal and pseudo—code.
`
`'II
`II
`
`l
`II
`lI
`
`II
`
`in
`
`A
`
`General Procedure:
`Boot up SoundScapc.
`Link in Extracter.
`PlayMusicviaSoundscape and intercept thenoteswith lflxtracterastheyare played.
`The notes will be written to a character file by Extracter.
`Run Converter, which will convert the character file to an integer file, Music.
`Run Analyzer. which will analyze the music and select a corresponding animation
`Run Director on the newly generated animation file.
`phrase and write it to a file.
`Video record the animation along with the corresponding music.
`
`A A A
`
`A
`
`naly7.er/ Generator:
`‘While Music Not Over Do:
`=~
`Read a 10-52:; phrase of music.
`--
`Reduce the series of pitches for the music phrase to a series of slope types.
`Slope types are as follows: ALT, UP. DOWN. FIAT.
`the series of slope types.
`Search for Patterns that might exist in
`Determine the predominant pattern.
`Randomlyselect from a library an animation phrase that has been preclassified
`as being a visual expression of music containing the predominant pattern.
`selected animation terminal to the
`-‘- Write a subroutine call statement for the
`Ge mini file.
`End ‘While.
`
`r-
`r-
`I:
`
`in
`
`Fig. 6. An algorithmic representation of the translation process. The fust set of steps
`describes the overall translation process. The second set presents the fundamental steps
`n
`of the translation process.
`of the Analyzer/ Generator portion
`
`simplified pitch-time graph shown in
`'fablc 1 was created, and the predomi-
`nant higher order pattern for the music
`terminal was determined to be ALT UP
`l)()WN. Based upon the predominant
`
`predominant pattern. Style classifica-
`tion was irnposcd so that the rcsriltirig
`video would have a consistent \'ocahu-
`lary. Figure 7 prcscrns a subset of the
`For
`animation terminal
`library
`the
`visual style of the diagonals that was
`used for ’r“!re first Gmerrnimr. in this fig-
`ure, the animation terminals are pre-
`sented as short storyboards to illustrate
`the overall nature of each t.errr_1ina].
`During step l, the music was cnter'crl
`into the computer rrsirig SounrlScapt-.
`‘cxtractcd‘ with Exl.]‘i‘l(.‘Lt‘1',(T(ll‘l\‘(?I‘lt‘t.l to
`integcris witlr (.IorwerIt~r' and, tinallx-', \\'I'li-
`tcn to
`a music lilc in tin" Iollowing
`lorrriat‘.
`
`pitch 1 (luration l
`pitch 2 duration ‘3.
`pitch 3 duration 3
`
`pitch Ndurrrtion N
`where N is the number of nolcs in the
`melo(.l_\'.
`Pitch and duration were I":-prcscrrlt-tl
`as integers ranging lrclwccn 0 and 12?.
`A1‘l}ll}‘?.t?l‘, with a record of the duration
`of each note, read in a music terurinal,
`rlclirred as 10 sec oi" nrrrsir. The
`simplified _pitch—tiIne graph of
`the
`music terminal was then cletcrmirrcd.
`Table 5.’ shows the pitch and duration
`values for the notes in the First. rnrrsir
`tr.-rminal for the music ol‘
`"H.-r Firs!
`(_§r?rm'(I!.irIrt. From these I1tlInl‘I(*rs.
`the
`
`Fig. 7. Five anima-
`tion terminals,
`classified as
`ALT UP DOV\.’N.
`Short sections of
`animations, such
`as these, are used
`to create a com-
`
`plete animation
`for a correspond-
`ing piece of
`music. These
`animation ter-
`minals are clas-
`sified by types,
`such as ALT UP
`DOWN, so that
`they may be
`matched up with
`shortisections of
`
`Alt Up Down
`
`Subroutine #111 Phrase 1: Pistons Around
`
`
`
`
`
`music.
`
`Page9of11
`l"o.r'mIr-ll'ulI'Irrm.~,
`34
`
`I'rm'-.rr (l the .\:rIrrnI.ILiu' (Gem-r '.r1ic'rII :ni'\'isII'.tl i\l1I:<i:'
`
`Page 9 of 11
`
`
`
`The creation of predetermined ani-
`mations was an important step in the
`development of the computer system
`just described. ()ne reason is t.hat, as pre-
`determined animations were created,
`segments of these animations were
`added to tile library. hence expanding
`the contents ofthe library. An

Accessing this document will incur an additional charge of $.
After purchase, you can access this document again without charge.
Accept $ ChargeStill Working On It
This document is taking longer than usual to download. This can happen if we need to contact the court directly to obtain the document and their servers are running slowly.
Give it another minute or two to complete, and then try the refresh button.
A few More Minutes ... Still Working
It can take up to 5 minutes for us to download a document if the court servers are running slowly.
Thank you for your continued patience.

This document could not be displayed.
We could not find this document within its docket. Please go back to the docket page and check the link. If that does not work, go back to the docket and refresh it to pull the newest information.

Your account does not support viewing this document.
You need a Paid Account to view this document. Click here to change your account type.

Your account does not support viewing this document.
Set your membership
status to view this document.
With a Docket Alarm membership, you'll
get a whole lot more, including:
- Up-to-date information for this case.
- Email alerts whenever there is an update.
- Full text search for other cases.
- Get email alerts whenever a new case matches your search.

One Moment Please
The filing “” is large (MB) and is being downloaded.
Please refresh this page in a few minutes to see if the filing has been downloaded. The filing will also be emailed to you when the download completes.

Your document is on its way!
If you do not receive the document in five minutes, contact support at support@docketalarm.com.

Sealed Document
We are unable to display this document, it may be under a court ordered seal.
If you have proper credentials to access the file, you may proceed directly to the court's system using your government issued username and password.
Access Government Site