(cid:14)(cid:25)(cid:19)(cid:27)(cid:18)(cid:22)(cid:21)(cid:24)(cid:1)(cid:15)(cid:24)(cid:28)(cid:20)(cid:23)(cid:25)(cid:26)(cid:20)(cid:1)(cid:17)(cid:14)(cid:11)(cid:1)(cid:8)(cid:16)(cid:3)(cid:14)(cid:6)(cid:7)(cid:5)(cid:15)(cid:2)(cid:10)(cid:13)(cid:16)(cid:9)(cid:2)(cid:7)(cid:4)(cid:3)(cid:13)(cid:2)(cid:12)(cid:10)(cid:10)(cid:4)(cid:2)(cid:7)(cid:16)(cid:7)(cid:9)(cid:12)(cid:9)(cid:7)(cid:6)(cid:14)(cid:6)(cid:3)(cid:16)
`
`
`
`
`DECLARATION OF GORDON MACPHERSON
`
`I, Gordon MacPherson, am over twenty-one (21) years of age. I have never been
`convicted of a felony, and I am fully competent to make this declaration. I declare the following
`to be true to the best of my knowledge, information and belief:
`
`1.
`
`2.
`
`3.
`
`I am Director Board Governance & IP Operations of The Institute of Electrical and
`Electronics Engineers, Incorporated (“IEEE”).
`
`IEEE is a neutral third party in this dispute.
`
`I am not being compensated for this declaration and IEEE is only being reimbursed
`for the cost of the article I am certifying.
`
`4. Among my responsibilities as Director Board Governance & IP Operations, I act as a
`custodian of certain records for IEEE.
`
`5.
`
`I make this declaration based on my personal knowledge and information contained
`in the business records of IEEE.
`
`6. As part of its ordinary course of business, IEEE publishes and makes available
`technical articles and standards. These publications are made available for public
`download through the IEEE digital library, IEEE Xplore.
`
`7.
`
`It is the regular practice of IEEE to publish articles and other writings including
`article abstracts and make them available to the public through IEEE Xplore. IEEE
`maintains copies of publications in the ordinary course of its regularly conducted
`activities.
`
`8. The article below has been attached as Exhibit A to this declaration:
`
`A.
`
`J.F. Tilki; A.A. Beex; “Encoding a hidden auxiliary channel onto a digital
`audio signal using psychoacoustic masking”, Proceedings IEEE
`SOUTHEASTCON '97. 'Engineering the New Century', April 12 – 14,
`1997.
`
`9.
`
`I obtained a copy of Exhibit A through IEEE Xplore, where it is maintained in the
`ordinary course of IEEE’s business. Exhibit A is a true and correct copy of the
`Exhibit, as it existed on or about July 13, 2022.
`
`445 Hoes Lane Piscataway, NJ 08854
`1
`
`Sony Exhibit 1026
`Sony v. MZ Audio
`
`

`

`(cid:14)(cid:25)(cid:19)(cid:27)(cid:18)(cid:22)(cid:21)(cid:24)(cid:1)(cid:15)(cid:24)(cid:28)(cid:20)(cid:23)(cid:25)(cid:26)(cid:20)(cid:1)(cid:17)(cid:14)(cid:11)(cid:1)(cid:8)(cid:16)(cid:3)(cid:14)(cid:6)(cid:7)(cid:5)(cid:15)(cid:2)(cid:10)(cid:13)(cid:16)(cid:9)(cid:2)(cid:7)(cid:4)(cid:3)(cid:13)(cid:2)(cid:12)(cid:10)(cid:10)(cid:4)(cid:2)(cid:7)(cid:16)(cid:7)(cid:9)(cid:12)(cid:9)(cid:7)(cid:6)(cid:14)(cid:6)(cid:3)(cid:16)
`
`10. The article and abstract from IEEE Xplore shows the date of publication. IEEE
`Xplore populates this information using the metadata associated with the publication.
`
`11. J.F. Tilki; A.A. Beex; “Encoding a hidden auxiliary channel onto a digital audio
`signal using psychoacoustic masking” was published in the Proceedings of IEEE
`SOUTHEASTCON '97. 'Engineering the New Century'. IEEE SOUTHEASTCON
`'97 was held from April 12 – 14, 1997. Copies of the conference proceedings were
`made available no later than the last day of the conference. The article is currently
`available for public download from the IEEE digital library, IEEE Xplore.
`
`12. I hereby declare that all statements made herein of my own knowledge are true and
`that all statements made on information and belief are believed to be true, and further
`that these statements were made with the knowledge that willful false statements and
`the like are punishable by fine or imprisonment, or both, under 18 U.S.C. § 1001.
`
`I declare under penalty of perjury that the foregoing statements are true and correct.
`
`
`
`
`(cid:6)(cid:1)(cid:3)(cid:5)(cid:1)(cid:4)(cid:2)(cid:4)(cid:4)
`Executed on:
`
`
`
`
`
`
`
`
`
` 2
`
`

`

`(cid:14)(cid:25)(cid:19)(cid:27)(cid:18)(cid:22)(cid:21)(cid:24)(cid:1)(cid:15)(cid:24)(cid:28)(cid:20)(cid:23)(cid:25)(cid:26)(cid:20)(cid:1)(cid:17)(cid:14)(cid:11)(cid:1)(cid:8)(cid:16)(cid:3)(cid:14)(cid:6)(cid:7)(cid:5)(cid:15)(cid:2)(cid:10)(cid:13)(cid:16)(cid:9)(cid:2)(cid:7)(cid:4)(cid:3)(cid:13)(cid:2)(cid:12)(cid:10)(cid:10)(cid:4)(cid:2)(cid:7)(cid:16)(cid:7)(cid:9)(cid:12)(cid:9)(cid:7)(cid:6)(cid:14)(cid:6)(cid:3)(cid:16)
`DocuSign Envelope ID: 5F0D342E-9CF8-410C-A991-4F48A843D30F
`
`
`
`
`
`
`
`
`
`
`
`
`EXHIBIT A
`EXHIBIT A
`
`
`A-1
`A-1
`
`

`

`IEEE.org
`
`IEEE Xplore
`
`IEEE SA
`
`IEEE Spectrum
`
`More Sites
`
`Cart
`
`
`Create
`Account
`
`
`Personal
`Sign In
`
`
`
`Access provided by:
`Everything Demo User
`
`Sign Out
`
`Browse  My Settings  Help 
`
`Access provided by:
`Everything Demo User
`
`Sign Out
`
`All
`
`
`
`fl
`
`ADVANCED SEARCH
`
`Conferences > Proceedings IEEE SOUTHEASTCON... 
`
`Encoding a hidden auxiliary channel onto a digital
`audio signal using psychoacoustic masking
`Publisher: IEEE
` PDF
`Cite This
`
`   
`
`Alerts
`
`Manage Content Alerts
`
`Add to Citation Alerts
`
`J.F. Tilki ; A.A. Beex All Authors
`
`109
`Full
`Text Views
`
`5P
`
`atent
`Citations
`
`12
`Paper
`Citations
`
` D
`
`ownl
`
`PDF
`
`Abstract:We report on the development of a method of encoding an auxiliary
`channel onto a digital audio signal such that it is imperceptible to human
`observers. The encoding is ac... View more
`
` Metadata
`Abstract:
`We report on the development of a method of encoding an auxiliary channel
`onto a digital audio signal such that it is imperceptible to human observers. The
`encoding is accomplished by imposing slight and controlled changes on the
`phase spectrum of short-time signal windows. By employing principles of
`psychoacoustics the auxiliary channel is masked from human perception.
`Furthermore, the encoding and decoding are achieved via fast algorithms which
`allow real time processing. The method is applicable for any digitized audio
`signal containing voice, music, or other acoustic signals to be heard by
`humans. Possible signal sources include compact discs, digital television,
`digital radio, digital telephony, and any other source where an audio signal is in
`a digital format.
`
`A-2
`
`Abstract
`
`Authors
`
`References
`
`Citations
`
`Keywords
`
`Metrics
`
`More Like This
`
`

`

`Published in: Proceedings IEEE SOUTHEASTCON '97. 'Engineering the New
`Century'
`
`Date of Conference: 12-14 April 1997
`
`INSPEC Accession Number: 5672553
`
`Date Added to IEEE Xplore: 06
`August 2002
`
`Print ISBN:0-7803-3844-8
`
`J.F. Tilki
`
`DOI: 10.1109/SECON.1997.598705
`
`Publisher: IEEE
`
`Conference Location: Blacksburg, VA,
`USA
`
`DSP Research Laboratory The Bradley Department of Electrical Engineering,
`Virginia Polytechnic Institute and State University, Blacksburg, VA, USA
`
`A.A. Beex
`DSP Research Laboratory The Bradley Department of Electrical Engineering,
`Virginia Polytechnic Institute and State University, Blacksburg, VA, USA
`
`Authors
`
`
`
`J.F. Tilki
`DSP Research Laboratory The Bradley Department of Electrical
`Engineering, Virginia Polytechnic Institute and State University, Blacksburg,
`VA, USA
`
`A.A. Beex
`DSP Research Laboratory The Bradley Department of Electrical
`Engineering, Virginia Polytechnic Institute and State University, Blacksburg,
`VA, USA
`
`References
`
`Citations
`
`Keywords
`
`Metrics
`
`
`
`
`
`
`
`
`
`IEEE Personal Account
`
`Purchase Details
`
`Profile Information
`
`Need Help?
`
`CHANGE
`USERNAME/PASSWORD
`
`PAYMENT OPTIONS
`
`VIEW PURCHASED
`DOCUMENTS
`
`COMMUNICATIONS
`PREFERENCES
`
`PROFESSION AND
`EDUCATION
`
`US & CANADA: +1 800 678
`4333
`
`WORLDWIDE: +1 732 981
`0060
`
`TECHNICAL INTERESTS
`
`CONTACT & SUPPORT
`
`Follow
`
`  
`
`About IEEE Xplore | Contact Us | Help | Accessibility | Terms of Use | Nondiscrimination Policy | IEEE Ethics Reporting  | Sitemap |
`Privacy & Opting Out of Cookies
`A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of
`humanity.
`
`© Copyright 2022 IEEE - All rights reserved.
`
`A-3
`
`

`

`IEEE Account
`
`Purchase Details
`
`Profile Information
`
`Need Help?
`
`» Change Username/Password
`» Update Address
`
`» Payment Options
`» Order History
`» View Purchased Documents
`
`» Communications Preferences
`» Profession and Education
`» Technical Interests
`
`» US & Canada: +1 800 678 4333
`» Worldwide: +1 732 981 0060
`» Contact & Support
`
`About IEEE Xplore Contact Us
`|
`
`|
`
`Help
`
`|
`
`Accessibility
`
`|
`
`Terms of Use
`
`|
`
`Nondiscrimination Policy
`
`|
`
`Sitemap
`
`|
`
`Privacy & Opting Out of Cookies
`
`A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity.
`© Copyright 2022 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.
`
`A-4
`
`

`

`ENCODING A HIDDEN AUXILIARY CHANNEL ONTO A DIGITAL AUDIO
`SIGNAL USING PSYCHOACOUSTIC MASKING
`
`John F. Tilki and A. A. (Louis) Beex
`DSPResearchLaboratory
`TheBradley Department ofElectricalEngineering
`VlRGINLATECH
`Blacksbur& VA 24061-0111, USA
`voice: (540) 23 14877 fax: (540) 23 1-3362
`e-mail: tilki@vt.edu and beex@vt.edu
`Abstract - We report on the development of a method
`shaped quantization [13]. However, a direct cOmpaTiSOn
`of encoding an auxiliary channel onto a digital audio
`betweenthe schemes in terms ofauxihy channel data rate
`versus perceived quality of the primary signal could not be
`Signal such that it is imperceptible to human observers.
`The encoding iS accomplished by imposing slight and
`peiformd Since the other approaches seem to require
`more computations, our method could be an excellent
`controlled changes on the phase spectrum of short-time
`alternative especially at high sampling rates and where
`By employing principles of
`signal Windows
`psychoacoustics the auxiliary channel is masked from
`computational capability is a limiting tictor.
`human perception. E’urthermore, the encoding and
`D ~ C ~ f f I O N OF CODING M E m o D
`decoding are achieved via fast algontbms which allow
`real time processing. The method is applicable for any
`Encoding
`digitizes audio signal containing voice, music, or other
`It is well known that humans are relatively insensitive
`acoustic signals to be heard by humans. Possible signal
`to certain types of phase distortions. We use this fact to
`sources include compact discs, digital television, digital
`encode additional infonnaton onto the phase of selected
`radio, digital telephony, and any other source where an
`bins of an FlT. The signal bins are selected according to
`audio Signal is in a digital format.
`human sensitivity in the corresponding €iequency bands,
`and are spaced to facilitate masking by umodified
`neighboring bins in the same spectral region. In particular
`the lower frequencies (below say 2 IrHi), which typically
`contain the most energy in common audio signals, are left
`untouched. The stronger low frequency components help to
`mask the changes made in the higher €iequency region.
`Additionally, the unmodified FFT bins in the higher
`frequency region help to mask their modified neighbors.
`TheFFI’binsimmedrate ly preoeding the signal bins
`are used as references. While keeping the digital audio
`signal’smiTamplitudethesame,thephaseoftheFFTat
`the signal bins is discarded, and a new phase is assigned
`based on the neighboring reference bins, a differential phase
`change, and the digital information to be ended. For
`example, a small clockwise rotation of phase relative to the
`reference can represent a digital “1” while a counter-
`clockwise rotation of phase represents a digital “0”.
`Suppose that the FFT has a complex value of& at a
`reference bin. In polar form this value is represented as
`
`INTRODUCllON
`The ability to add extra hidden i n f o d o n to a signal
`could be useful in many applications. Audio compact discs
`could be modified to contain artist information, song titles,
`song lyrics, karaoke information, still images, and perhaps
`even video clips. In other contexts an extra channel could
`be used to transmit control information. The want and need
`for additional bandwidth is ever-increasing, and the
`capability to add extra information to digital audio can be
`quite useful even for archid and storage purposes.
`The coding method presented in this paper allows the
`addition of such an amdiary channel, and it does so in a
`manner that does not significantly degrade the perceived
`quality of the pximary audio channel. Furthermore, the
`method does not require changes in the underlying digital
`format, and so it completely maintains backward
`compatiiility with existing data formats. One can imagine
`the possiiilities for the audio CD example. Existing CD
`players could stilI play CDs encoded with the auxihy
`channel. With additional signal processing capability
`however, new CD players could provide the e n h a n d
`functonality described above.
`Other researchers have aimed at the same objective
`using methods such as subband coding coupled with
`adaptive quantization, and subtractively &thered noise-
`
`where R, is magnitude and 0, is the phase. Similarly
`thevalue ofthem at the neighboringsignal bin is
`
`-331-
`
`0-7803-3844-8/97/$10.00 0 1997 EEE
`
`Authorized licensed use limited to: Everything Demo User. Downloaded on July 13,2022 at 18:59:48 UTC from IEEE Xplore. Restrictions apply.
`
`

`

`Then if QP is the differential phase change, the new F”
`value for the signal bin is assigned as
`.E, =&e i(@,*b)
`(3)
`of the reference bin that is used
`Note that it is the
`along with the differential phase to replace the phase of the
`signal bin. Addition of the differential phase represents a
`digital one and subbadion a digital zero.
`Once the phases of the signal bins have been m&ed,
`the conjugate phase is assigned to the matching negative
`frequency bins. An inverse FFT then yields the modified
`signal block containing both the primary audio and the
`secondary channel. The process is then repeated for
`subsequent signal blocks.
`1 shows a small region of an example phase
`and its modified version after encoding. The
`phase at the reference bins is denoted by asterisks, the
`original phase at the signal bins is denoted by ‘o’, and the
`modified phase at the signal bins is denoted by ‘x’. This
`example shows a digital pattern of [0 1 0 0 11.
`For a given source sample rate the FFT size is chosen
`to be short enough in time to maimah the imperceptibility
`of the i n t r d u d phase distortions. However, the FET size
`should conversely be chosen as large as possllle under the
`above constraint such that the FFT bin frequencies are as
`dense as possible. In our limited simulations, FFT sizes
`corresponding to block lengths of 10 to 50 msec worked
`well. Smaller blocks often yielded ccgra” noise, while
`larger blocks usually created noticeable distortion of the
`primary signal.
`
`Decoding
`The signal is decoded by comparing the phase at the
`(known) signal bins to the phase at the neighboring
`reference bins, and assigning bits based on the direction of
`relative phase rotation.
`
`S y n c ~ ~ o n ~ n
`process requires perfect synchronization
`The
`with the encoded signal blocks. For a storage medium such
`as a compact disc synchronization is simple and only
`requires knowledge of the offset of the first signal block
`For real-time data streams however, synchroniaiion is
`more
`Several options are available, including
`decoding candidate blocks until an error detection criterion
`is satisfied. Error detection can be accomplished with cyclic
`redundancy checks (GRC) or other ef3icient error detection
`schemes [4]. The candidate block alignments are changed
`one sample at a time until a valid data trans” *
`ion is
`detected. The offset at the point of valid data detection
`provides synchronization for all subsequent signal blocks.
`Figure 2 shows how zero bit errors result when perfect
`synchronization is achiwed. When synchronization is
`
`incorrect by even a single sample delay or advance several
`bit errors result (238 in the case shown).
`Another possiiity for synchronization is the inclusion
`of known bit pattems in the data stream. This situation also
`requires shifts by a single sample at a time until the
`embedded bit pattern is detected. Such a “control” function
`for synchronization has been use61 in a similar context in the
`analog channel case [§I.
`SIMULATION RESULTS
`The coding method has been tested and works well.
`The example provided in the paper is based on four seconds
`of the audio component of a TV commercial, sampled
`monophonically at a rate of 44.1 kTitz, consistent with CD
`Quality audio. The lowpass nature of this audio signal is
`shown in Figure 3. (Note that the sinmid at 15.734 kHz is
`interference fiom the horizontal synchronization pulse for
`the video electron beam,) The decoded phase errors would
`be zero were it not for the quantization back to 16-bits that
`occurs after the inverse FFT. The phase errors due to
`Quantization are shown in Figure 4. A differential phase of
`d8 was used to modi@ the phase of every fourth FET bin
`above 2 IiEz in 2048 point FFT blocks. A secondary
`channel resulted with a capacity of over 5000 bits per
`second with little or no degradation in the perceived quality
`ofthe primary audio signal.
`Higher data rates can be achieved by expanding the
`frequency region used for coding and using smaller spacing
`bemeen signal bins in the FFT. However, as would be
`expect& the perceived quality of the primary si@ d e r s
`as the data rate ofthe auxiliary channel increases.
`
`Future improvements include using several increments
`of the differential phase for the forward and reverse
`rotations of phase, allowing for the coding of multiple bits
`per FFT bin. Also, prequantization of encoded FFT blocks
`should allow preampensation for quantization effects
`during the encoding process. In addition, the audio signal
`can be decomposed into subbands, with each subband
`independently used for coding. This would allow more bits
`to be encoded in selected subbands based on the relative
`influence the individual subbands have on overall
`perception.
`Finally, adaptive encoding based on local power levels
`and frequency content in the primary audio signal can
`perhaps increase the effixtive capacity of the auxiliilly
`channel. A related variation would be adaptation of the
`differential phase change based on the local frequency
`content and on some measure of phase deviation in the
`region. This could be used to compensate for the .fact that
`frequency regions with lower energy are more sensitive to
`
`-332-
`
`Authorized licensed use limited to: Everything Demo User. Downloaded on July 13,2022 at 18:59:48 UTC from IEEE Xplore. Restrictions apply.
`
`

`

`quantization effects tban regions with more energy. Note
`that the phase errors inFigure 4 are largest in the regions of
`lowest energy, as seen from comparison with Figure 3.
`CONCLUSION
`In conclusion, we have developed a method of
`SUCCeSSfllUy encoding an auxiliary channel onto a digital
`audio signal. This channel is encodedusingpsychoacoustic
`principles such that it is imperceptiile to human obsetvers.
`Both the encoding and decoding are accomplished with the
`FIT, allowing real time implementations. The coding
`process is appropriate for any application containing a
`digitized audio signal.
`
`PI
`
`r31
`
`" ~ C Z i S
`V I
`W.RTh. ten Date, L.M. van de Kerkhd, and F.F.M.
`Zijdemld,
`?Digital Audio Canying Extra
`
`ICASSP 1990, p ~ . 1097-1100.
`W O ~ O Q "
`Michael A. Genon and Peter G. Craven, "A High-Rate
`Buried-Data Channel for Audio CD," J. Audio Eng.
`Soc., Vol. 43, NO. 1/2, Jall./F&. 1995, p ~ . 3-22.
`
`A.W.J. Oomen, ME. Groenewegen, RG. Van Der
`Waal, and RN.J. Veldhuis, "A Variable-Bit-Rate
`Buried-Data Channel for Compact Disc," J. Audio Eng.
`Soc., Vol. 43, NO. 1/2, Ja/F&. 1995, pp. 23-28.
`Stephen B. Wicker, Error Control Svstems for Digital
`Communications and Storam, Englewood C m , New
`Jersey: Prentice Hall, 1995.
`John F. T W and A A (Louis) Beex, ''Encoding a
`Hidden Digital Signature onto an Audio Signal Using
`Psychoacoustic Masking,'' International Conference on
`Signal Procmsing Applications & Technolow, October
`
`7-10,1996, Boston, MA, p ~ . 476-480.
`
`r41
`
`151
`
`2oo
`
`g
`&
`i%
`
`3000
`1000
`-3000
`2000
`0
`-1000
`-2000
`Delq in Samples
`Figure 2. Block Synchronization by Error Detection.
`
`Filrtq.(ww. ls=44.11*b
`
`3 -
`20-
`10 -
`
`F f w @ n ~ y W)
`x Id
`Figure 3. Time-Averaged PSD of Audio Signal.
`
`".e
`
`I
`0.6}
`
`
`
`phase ~ m n firan auanmion vs. Frequency
`
`0
`
`I
`
`i
`
`
`
`0 0 0
`
`1
`.
`
`
`
`
`
`0
`
`I
`
`' i
`
`Original and Modified Phase SpeU?'a
`
`I
`
`'
`
`
`
`I
`
`cm a -1 y'
`
`..*"
`
`-3
`
`460
`
`Fn Bin Nunber
`470
`465
`475
`Figure 1. Example Phase Spectra.
`
`g 0.4 P - t
`
`
`
`N
`
`e.
`0.2 -
`w 3 0 -
`8
`-0.2 -
`
`0
`
`-0.4
`
`0
`0
`loo0
`600
`200
`400
`800
`FFT Bin
`Figure 4. Phase Errors fiom Quantization vs. Frequency.
`
`-333-
`
`Authorized licensed use limited to: Everything Demo User. Downloaded on July 13,2022 at 18:59:48 UTC from IEEE Xplore. Restrictions apply.
`
`