UNITED STATES PATENT AND TRADEMARK OFFICE
`_____________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`_____________
`
`SONY GROUP CORPORATION (JAPAN), SONY CORPORATION OF
`AMERICA, SONY INTERACTIVE ENTERTAINMENT LLC, SONY
`PICTURES ENTERTAINMENT INC., SONY ELECTRONICS INC., and
`VERANCE CORPORATION,
`Petitioners,
`
`v.
`
`MZ AUDIO SCIENCE, LLC,
`Patent Owner.
`_____________
`
`Case No. TBD
`Patent No. 7,289,961
`_____________
`
`DECLARATION OF RACHEL J. WATTERS
`RELATING TO EXHIBIT 1021(cid:3)
`
`Sony Exhibit 1053
`Sony v. MZ Audio
`
`

`

`Declaration of Rachel J. Watters on Authentication of Publication
`
`I, Rachel J. Watters, am a librarian, and the Head of Resource Sharing for the
`
`General Library System, Memorial Library, located at 728 State Street, Madison,
`
`Wisconsin, 53706. Part of my job responsibilities include oversight of Wisconsin
`
`TechSearch (“WTS”), an interlibrary loan departmentat the University of Wisconsin-
`
`Madison.
`
`I have worked asa librarian at the University of Wisconsin library system
`
`since 1998, starting as a graduate student employee in the Kurt F. Wendt Engineering
`
`Library and WTS,thenasa librarian in Interlibrary Loan at Memorial Library.
`
`I began
`
`professional employment at WTSin 2002 and became WT's Director in 2011. In 2019,
`I became of Head of Resource Sharing for UW-Madison’s General Library System.
`|
`
`have a master’s degree in Library and Information Studies from the University of
`
`Wisconsin-Madison. Through the course of my studies and employment, I have
`
`become well informed about the operations of the University of Wisconsin library
`
`system, which followsstandard library practices.
`
`This Declaration relates to the dates of receipt and availability of the following:
`
`Wu,C.-P., Su, P.-C., and Kuo, C.-C.J. (2000). Robust and
`efficient digital audio watermarking using audio content
`analysis. In Wong, P.W. and Delp, E.J. (Eds.) Security and
`Watermarking ofMultimedia Contents I, 24-26 January 2000,
`San Jose, California, p. 382-392.
`
`Standard operating procedures for materials at the University of Wisconsin-
`
`Madison Libraries. When a volumewasreceived by the Library, it would be checked
`
`

`

`Declaration of Rachel J. Watters on Authentication of Publication
`
`in, addedto library holdings records, and madeavailable to readers as soon afterits
`
`arrival as possible. The procedure normally took a few daysor at most 2 to 3 weeks.
`
`Exhibit A to this Declaration is true and accurate copy of the front matter of the
`
`Security and Watermarking ofMultimedia Contents II, 24-26 January 2000, San Jose,
`
`California (2000) publication, which includes a stamp on the verso page showing that
`this bookis the property of the Wendt Library at the University of Wisconsin-Madison.
`
`Exhibit A also includes an excerpt of pages 382 to 392 of that volume, showing the
`
`article entitled Robust andefficient digital audio watermarking using audio content
`
`analysis (2000).
`Attached as Exhibit B is the cataloging system record ofthe University of
`Wisconsin-Madison Libraries for its copy of the Security and Watermarking of
`
`Multimedia Contents II, 24-26 January 2000, San Jose, California (2000)publication.
`
`As showninthe “Receiving date”field of this Exhibit, the University of Wisconsin-
`
`Madison Libraries ownedthis book and hadit cataloged in the system as of June 14,
`
`2000.
`
`Membersof the interested public could locate the Security and Watermarking of
`
`Multimedia Contents II, 24-26 January 2000, San Jose, California (2000) publication
`
`after it was cataloged by searchingthe public library catalog or requesting a search
`
`through WTS. The search could be done bytitle, editor, and/or subject key words.
`
`

`

`Declaration of Rachel J. Watters on Authentication of Publication
`
`Membersofthe interested public could access the publication by locating it on the
`
`library’s shelves or requesting it from WTS.
`
`I declare that all statements made herein of my own knowledgearetrue and that
`
`all statements made on information and belief are believed to be true; and further that
`
`these statements were made with the knowledgethat willful false statements andthe like
`
`so madeare punishable by fine or imprisonment, or both, under Section 1001 ofTitle 18
`
`of the United States Code.
`
`Date: September2, 2022
`
`Memorial Library
`728 State Street
`Madison, Wisconsin 53706
`
`H¢ad of Resource Sharing
`
`

`

`
`
`
`
`
`
`EXHIBIT A
`EXHIBIT A
`
`

`

`Security and Watermarking
`of Multimedia Contents II
`
`Volume 3971
`
`Ping Wah Wong
`Edward J. Delp
`Chairs/Editors
`
`24-26 January 2000
`San Jose, California
`
`Sponsored by
`IS&T-The Society for Imaging Science and Technology
`SPIE-The International Society for Optical Engineering
`
`PROCEEDINGS OF SPIE
`
`SPIE—The International Society for Optical Engineering
`
`a
`
`

`

`YDAL—4n
`
`2><OYwo
`
`omve
`
`aA-
`
`ND
`
`SS
`
`rice
`
`coLn
`
`

`

`
`
`PROCEEDINGS OF SPIE
`
`SPIE—TheInternational Society for Optical Engineering
`
`Security and Watermarking
`of Multimedia Contents II
`
`Ping Wah Wong
`Edward J. Delp
`Chairs/Editors
`
`24-26 January 2000
`San Jose, California
`
`Sponsored by
`IS&T-The Society for Imaging Science and Technology
`SPIE-The International Society for Optical Engineering
`
`
`
`Volume 3971
`
`
`
`
`
`
`
`I]$}U9}U0DeIpawHjnwjoBuPjzewsazeMpueAyINdag
`
`
`
`|
`
`3
`S
`|
`y
`=.
`oown
`’
`@
`
`=———
`
`——
`
`——
`——
`
`— —
`
`===
`eS——
`=—S
`———
`—=
`
`|
`|
`
`I
`
`onal?
`
`=S
`at
`e
`So
`
`e—
`
`enero
`eea
`
`ISSN 0277-786x
`ISBN 0-8194-3589-9
`
`—
`
`

`

`PROCEEDINGS OF SPIE
`\@ia). SPIE—The International Society for Optical Engineering
`
`Security and Watermarking
`of Multimedia Contents II
`
`Ping Wah Wong
`Edward J. Delp
`Chairs/Editors
`
`24-26 January 2000
`San Jose, California
`
`Sponsored by
`IS&T—The Society for Imaging Science and Technology
`SPIE—The International Society for Optical Engineering
`
`Published by
`SPlE—The International Society for Optical Engineering
`
`
`
`
`
`Volume 3971
`
`SPIE is an international technical society dedicated to advancing engineering and scientific
`applications of optical , photonic, imaging, electronic, and optoelectronic technologies.
`
`
`
`

`

`Libraty
`con
`ANomposed
`of NP all Mg1688
`
`;
`
`
`
`The papers appearing in this book compose the proceedingsof the technical conference cited on
`the coverandtitle page of this volume. They reflect the authors' opinions and are published as
`presented, in the interests of timely dissemination. Their inclusion in this publication does not
`necessarily constitute endorsement by the editors or by SPIE. Papers were selected by the
`conference program committee to be presented in oral or poster format, and were subject to
`review by volumeeditors or program committees.
`
`Please use the following format to cite material from this book:
`
`Author(s), "Title of paper," in Security and Watermarking of Multimedia Contents II, Ping Wah
`Wong, EdwardJ. Delp, Editors, Proceedings of SPIE Vol. 3971, page numbers (2000).
`
`ISSN 0277-786X
`ISBN 0-8194-3589-9
`
`Published by
`SP1E—TheInternational Society for Optical Engineering
`P.O. Box 10, Bellingham, Washington 98227-0010 USA
`Telephone 360/676-3290 (Pacific Time) ¢ Fax 360/647-1445
`
`Copyright ©2000, The Society of Photo-Optical Instrumentation Engineers.
`
`Copying of material in this book for internal or personaluse,orfor the internal or personal use
`of specific clients, beyondthefair use provisions granted by the U.S. Copyright Law is authorized
`by SPIE subject to paymentof copyingfees. The Transactional Reporting Service base fee for this
`volumeis $15.00 perarticle (or portion thereof), which should bepaid directly to the Copyright
`Clearance Center (CCC), 222 Rosewood Drive, Danvers, MA 01923. Payment mayalso be made
`electronically through CCC Online at http:/(www.directory.nel/copyright/. Other copying for
`republication,resale, advertising or promotion, or any form ofsystematic or multiple reproduction
`of any material in this book is prohibited except with permission in writing from the publisher.
`The CCCfee code is 0277-786X/00/$15.00.
`
`Printed in the United States of America.
`
`
`
`

`

`
`
`Contents
`
`Conference Committee
`Introduction
`
`xi
`
`SESSION 1
`COMMUNICATIONS APPROACH TO WATERMARKING
`Improving the robustness of nonadditive watermarks through optimum detection theory
`2
`[3971-01]
`A. Piva, Univ. degli Studi di Firenze (Italy); M. Barni, Univ. degli Studi di Siena (Italy);
`F. Bartolini, V. Cappellini, A. De Rosa, Univ. degli Studi di Firenze (Italy)
`
`14
`
`24
`
`48
`
`60
`
`Detection theory and digital watermarking [3971-02]
`A. Robert, R. Knopp, Swiss Federal Institute of Technology/Lausanne
`
`Improving data hiding by using convolutional codes and soft-decision decoding [3971-03]
`J. R. Hernandez, J.-F. Delaigle, B. M. M. Macq, Univ. Catholique de Louvain (Belgium)
`
`Preprocessed and postprocessed quantization index modulation methodsfor digital
`watermarking [3971-04]
`B. Chen, G. W. Wornell, Massachusetts Institute of Technology
`
`Quantization watermarking [3971-05]
`J. J. Eggers, Univ. of Erlangen-Nuremberg (Germany); B. Girod, Stanford Univ.
`
`IMAGE WATERMARKING |
`SESSION 2
`74
`Automatic recovery ofinvisible image watermarks from geometrically distorted images
`[3971-06]
`G. W. Braudaway, F. C. Mintzer, IBM ThomasJ. Watson Research Ctr.
`
`82
`
`90
`
`99
`
`Geometric distortion correction in image watermarking [3971-07]
`M. Alghoniemy, A. H. Tewfik, Univ. of Minnesota/Twin Cities
`
`Rotation-, scale-, and translation-resilient public watermarking for images [3971-08]
`C.-Y. Lin, Columbia Univ.; M. Wu, Princeton Univ.; J. A. Bloom,I. J. Cox, M. L. Miller, NEC
`Research Institute; Y. M. Lui, Signafy, Inc.
`
`Robust watermarking based on the warping of predefined triangular patterns [3971-09]
`P. Bas, J.-M. Chassery, Lab. des Images et des Signaux (France), B. M. M. Macq, Univ.
`Catholique de Louvain (Belgium)
`
`110
`
`Smearing-desmearing transformation and the application to digital watermark [3971-10]
`T. Okahisa, K. Ohue, Ehime Univ. (Japan)
`
`

`

`SESSION 3
`AUTHENTICATION WATERMARK I
`120
`Spatial watermark for imageverification [3971-11]
`M. P. Queluz, P. Lamy, Instituto Superior Técnico (Portugal)
`
`131
`
`140
`
`152
`
`164
`
`Compression-compatible fragile and semifragile tamper detection [3971-12]
`L. M. Marvel, G. W. Hartwig, Jr., Army Research Lab.; C. G. Boncelet,Jr., Univ. of Delaware
`
`Semifragile watermarking for authenticating JPEG visual content [3971-13]
`C.-Y. Lin, S.-F. Chang, Columbia Univ.
`
`Detection of imagealterations using semifragile watermarks [3971-14]
`E. T. Lin, Purdue Univ; C.
`|. Podilchuk, Lucent Technologies/Bell Labs.; E. J. Delp,
`Purdue Univ.
`
`Distortion-bounded authentication techniques [3971-15]
`N. D. Memon, Polytechnic Univ.; P. L. Vora, Hewlett-Packard Labs.; B.-L. Yeo, EXP.com;
`M. M. Yeung, Intel Corp.
`
`VIDEO WATERMARKING I
`SESSION 4
`
`Newapproachfor transformation-invariant image and video watermarking in the spatial
`176
`domain:self-spanning patterns (SSP) [3971-16]
`J. Dittmann, T. Fiebig, R. Steinmetz, German National Research Ctr. for Information
`Technology
`
`186
`
`198
`
`209
`
`Robust frame-dependent video watermarking [3971-18]
`M. J. Holliman, W. Macy, M. M. Yeung,Intel Corp.
`
`Real-time detection of video watermark on Intel architecture [3971-19]
`Y.-K. Chen, M. J. Holliman, W. Macy, M. M. Yeung, Intel Corp.
`
`Adaptive video watermarking using motion information [3971-20]
`C.-H. Lee, Korea AdvancedInstitute of Science and Technology; H.-S. Oh, Electronics
`and Telecommunications Research Institute (Korea); H.-K. Lee, Korea AdvancedInstitute
`of Science and Technology
`
`DATA HIDING
`SESSION 5
`
`218
`
`228
`
`237
`
`Embeddingdigital watermarks in halftone screens [3971-21]
`S.-G. Wang, K. T. Knox, Xerox Corp.
`
`Data hiding for halftone images [3971-22]
`M. S. Fu, O. C. Au, Hong Kong Univ. of Science and Technology
`
`Data hiding and watermarking in JPEG-compressed domain by DC coefficient modification
`[3971-23]
`P. H. W. Wong,O. C. Au, J. W. C. Wong, Hong Kong Univ. of Science and Technology
`
`

`

`
`
` SESSION 6 IMAGE SECURITY SYSTEMS
`
`
`
`246
`
`264
`
`274
`
`Overview of multimedia content protection in consumerelectronics devices [3971-24]
`A. M. Eskicioglu, Thomson Consumer Electronics; E. J. Delp, Purdue Univ.
`
`Smart Images using Digimarc's watermarking technology [3971-25]
`A. M. Alattar, Digimarc Corp.
`
`Hardware/software implementation 3-way algorithm for image encryption [3971-26]
`P. P. Dang, P. M. Chau, Univ. of California/San Diego
`
`
`SESSION 7
`IMAGE WATERMARKING II
`
`286
`
`295
`
`306
`
`314
`
`326
`
`Visual hash for oblivious watermarking [3971-27]
`J. Fridrich, SUNY/Binghamton and Mission Research Corp.
`
`Spatial-frequency composite watermarking for digital image copyright protection [3971-28]
`P.-C. Su, C.-C. J. Kuo, Univ. of Southern California and Media Fair, Inc.
`
`Secure digital watermark generation based on chaotic Kolmogorovflows [3971-29]
`J. Scharinger, Johannes-Kepler-Univ. Linz (Austria)
`
`Fundamental performancelimits of power-spectrum condition-compliant watermarks
`[3971-30]
`J. K. Su, Univ. of Erlangen-Nuremberg (Germany); B. Girod, Stanford Univ.
`
`Robust watermarking using robust coefficients [3971-31]
`T. Liang, J. J. Rodriguez, Univ. of Arizona
`
`
`SESSION 8
`ATTACK
`
`338
`
`346
`
`358
`
`371
`
`Possible counter-attacks against random geometric distortions [3971-32]
`J.-L. Dugelay, Institut Eurécom (France); F. A. P. Petitcolas, Microsoft Research Ltd. (UK)
`
`Countermeasures for unintentional and intentional video watermarking attacks [3971-33]
`F. Deguillaume, G. Csurka, T. Pun, Univ. of Geneva (Switzerland)
`
`Generalized watermarking attack based on watermark estimation and perceptual
`remodulation [3971-34]
`S. Voloshynovskiy, Univ. of Geneva (Switzerland) and Digital Copyright Technologies
`(Switzerland); S. Pereira, Univ. of Geneva (Switzerland); A. Herrigel, N. Baumgartner, Digital
`Copyright Technologies (Switzerland); T. Pun, Univ. of Geneva (Switzerland)
`
`Watermark copyattack [3971-35]
`M. Kutter, Digital Copyright Technologies (Switzerland); $. Voloshynovskiy, Digital Copyright
`Technologies (Switzerland) and Univ. of Geneva (Switzerland); A. Herrigel, Digital Copyright
`Technologies (Switzerland)
`
`SESSION 9
`AUDIO WATERMARKING
`
`
`382
`
`Robust and efficient digital audio watermarking using audio content analysis [3971-36]
`C.-P. Wu, P.-C. Su, C.-C. J. Kuo, Media Fair, Inc. and Univ. of Southern California
`
`
`
`
`
`

`

`393
`
`Audio registration and its application in digital watermarking [3971-37]
`C. Xu, J. Wu, Q. Sun, Kent Ridge Digital Labs. (
`Singapore)
`
`
`SESSION 10
`AUTHENTICATION WATERMARK II
`
`
`
`404
`
`417
`
`428
`
`438
`
`VQ-based digital signature scheme for multimedia content authentication [3971-39]
`Q. Sun, Kent Ridge Digital Labs. (Singapore); D. Zhong, S.-F. Chang, Columbia Univ.; J. Wu,
`A. D. Narasimhalu, Kent Ridge Digital Labs. (Singapore)
`
`Secret and public key authentication watermarking schemesthat resist vector quantization
`attack [3971-40]
`P. W. Wong, Hewlett-Packard Co.; N. D. Memon, Polytechnic Univ.
`
`Further attacks on Yeung-Mintzerfragile watermarking scheme [3971-41]
`J. Fridrich, M. Goljan, SUNY/Binghamton; N. D. Memon, Polytechnic Univ.
`
`Secure digital photograph handling with watermarking technique in insurance claim process
`[3971-42]
`K. Toyokawa, Nihon Univ. (Japan); N. Morimoto, S. Tonegawa, K. Kamijo, A. Koide, IBM
`Japan Ltd.
`
`
`SESSION 11
`VIDEO WATERMARKING II
`
`448
`
`455
`
`465
`
`477
`
`486
`
`Hiding video in plain sight [3971-43]
`C. P. Rosiene, Univ. of Hartford; J. A. Rosiene, Analysis and Technology
`
`Combinedvideo and audio watermarking: embedding content information in multimedia data
`[3971-44]
`J. Dittmann, M. Steinebach, German National Research Ctr. for Information Technology;
`|. Rimac, S. Fischer, Darmstadt Univ. of Technology (Germany); R. Steinmetz, German National
`Research Ctr. for Information Technology and Darmstadt Univ. of Technology (Germany)
`
`Object watermarking for for MPEG-4 video streams copyright protection [3971-45]
`M. Barni, Univ. degli Studi di Siena (Italy); F. Bartolini, V. Cappellini, N. Checcacci, Univ.
`degli Studi di Firenze (Italy)
`
`Robust compressed video watermarking [3971-46]
`J. Jing, Y. Guan, Nanyang Technological Univ. (Singapore)
`
`Quality evaluation of watermarked video [3971-47]
`W. Macy, M. J. Holliman, Intel Corp.
`
`
`
`SESSION 12
`IMAGE WATERMARKING II]
`
`Topological-ordered color table for BPCS-steganography using indexed color images
`[3971-48]
`R. Ouellette, H. Noda, M. Niimi, E. Kawaguchi, Kyushu Institute of Technology (Japan)
`
`Color opponency watermarking schemefor digital images [3971-49]
`S. Battiato, D. Catalano, G. Gallo, Univ. of Catania (Italy); R. Gennaro, IBM ThomasJ. Watson
`Research Ctr.
`
`502
`
`510
`
`vi
`
`

`

`Watermark optimization technique based on genetic algorithms [3971-50]
`C.-H. Huang,J.-L. Wu, National Taiwan Univ.
`
`Blind watermarking system for digital images in the wavelet domain [3971-51]
`A. Lumini, D. Maio, Univ. degli Studi di Bologna (Italy)
`
`Digital image watermarking on a special object: the human face [3971-52]
`H.-S. Oh, D.-H. Chang, Electronics and Telecommunications ResearchInstitute (Korea);
`C.-H. Lee, H.-K. Lee, Korea AdvancedInstitute of Science and Technology
`
`516
`
`524
`
`536
`
`545
`547
`
`Addendum
`
`Author Index
`
`

`

`Robust and Efficient Digital Audio Watermarking Using Audio
`Content Analysis
`
`Chung-Ping Wu, Po-Chyi Su and C.-C. Jay Kuo
`Media Fair, Inc., 1055 Corporate Center Dr., Ste 580
`Monterey Park, CA 91754
`and
`Department of Electrical Engineering-Systems
`University of Southern California, Los Angeles, CA 90089-2564
`E-mail:{chungpin,pochyisu,cckuo}@sipi.usc.edu
`
`ABSTRACT
`Digital audio watermarking embeds inaudible information into digital audio data for the purposes of copyright
`protection, ownership verification, covert communication, and/or auxiliary data carrying.
`In this paper, we first
`describe the desirable characteristics of digital audio watermarks. Previous work on audio watermarking, which
`has primarily focused on the inaudibility of the embedded watermark andits robustness against attacks such as
`compression and noise,
`is then reviewed.
`In this research, special attention is paid to the synchronization attack
`caused by casualaudio editing or malicious random cropping, which is a low-cost yet effective attack to watermarking
`algorithms developed before. A digital audio watermarking scheme of low complexity is proposed in this research as
`an effective way to deter users from misusingorillegally distributing audio data. The proposed schemeis based on
`audio content analysis using the wavelet filterbank while the watermark is embedd in the Fourier transform domain.
`A blind watermark detection technique is developed to identify the embedded watermark under various types of
`attacks.
`Keywords: digital watermark, blind watermark detection, audio content analysis, synchronization attack, human
`auditory system, malicious cropping attack,wavelet
`
`1. INTRODUCTION
`Digital audio watermarking, the embedding and detection of an imperceptible signalin digital audio data, has received
`increasing attention recently. Among various different uses of digital audio watermarking, copyright protection is
`the most highly demanded application. The fast growth of the Internet and the maturity of audio compression
`techniques enable the promising market of on-line music distribution. However, since the digital technology allows
`lossless data duplication,illegal copying and distribution would be mucheasier than before. This concern does make
`musical creators and distributors hesitant to step into this market quickly. Therefore, the proper content protection
`technology is the key to the emergence of this new market.
`Encryption and watermarking are the two most important content protection techniques. Encryption protects
`the content from anyone without the proper decryption key.
`It is useful in protecting the audio data from being
`intercepted during transmission. However, after the intended receiver decrypts it with the correct key, audio data
`could be illegally distributed and misused. Watermarks, on the other hand, cannot be removed from audio data
`even by the intended receiver. The embedded watermark signal permanently remainsin audio data after repeated
`reproduction andredistribution. Thus,this signal could be used to protect the copyright of audio content by playback
`prohibition, illegal copy source tracing and ownership establishment.
`Other applications of digital audio watermarking include data hiding for covert communication, auxiliary data
`embedding for audio content labeling, and modification detection for authentication. Data hiding can also be used
`to complement encryption, ie. enhancing communication security by concealing the existence of sensitive data
`transmission. Embeddedauxiliary data can carry lyrics or descriptions of the carrying audio data, or serve as links
`to external databases. Disappearance of fragile watermark could indicate unauthorized modifications and be used
`for content integrity verification.
`Different watermarking applications have different sets of requirements. Here, our discussion is focused on copy-
`right protection because it has the most stringent requirement on the watermark’s ability to survive intentional
`
`382
`
`In Security and Watermarking of Multimedia Contents Il, Ping Wah Wong, Edward J. Delp,
`Editors, Proceedings of SPIE Vol. 3971 (2000) ¢ 0277-786X/00/$15.00
`
`
`
`

`

`attacks. This is considered as one of the most challenging issues of the watermarking technology today. Users benefit
`from embedded label data while hackers do not know the existence of hidden communication data. Thus, embedded
`watermarks in these two applications are generally not subject to malicious attacks.
`This paperis organized as follows. The requirements for audio watermarking systems are described in Section
`2. Previous work on audio watermarking is reviewed in Section 3. Our current work onsalient point extraction and
`Fourier domain watermarkingis presented in Section 4. Experimental results and their analysis are given in Section
`5. Finally, concluding remarks are provided in Section 6.
`
`2. REQUIREMENTS FOR AUDIO WATERMARKING SYSTEMS
`In order for the embedded watermark to effectively protect the copyright of the digital audio data, it has been
`generally agreed!® that a good watermarking scheme should satisfy the following properties:
`
`1. The embedded watermark should not produce audible distortion to the sound quality of the original audio.
`2. The computation required by watermark embedding and detection should be low. The complexity of watermark
`detection should be especially low tofacilitate its integration into consumer electronic products.
`3. Watermark detection should be done without referencing the original audio data. This propertyis known as
`blind detection.
`
`4. The watermark should be undetectable without prior knowledge of the embedded watermark sequence. This
`property prevents attackers from reversing the embedding process to remove the watermark.
`5. The embedded watermark should be robust against commonsignal processing attacks suchas filtering, resam-
`pling and compression.
`6. The watermark should survive malicious attacks such as random cropping and noise adding. However, severe
`attacks that produce annoying noise can be ignored for the survival test.
`
`3. PREVIOUS WORK ON AUDIO WATERMARKING
`A variety of audio watermarking methods with very different characteristics have been proposed. They will be
`reviewed in this section.
`Early work on audio watermark embedding achieved inaudibility by placing watermark signals in perceptually
`insignificant regions. One popular choice was the higher frequency region,!°!? where human sensitivity declines
`compared to its peak around 1 kHz. In somesystems,'®" the watermark signalis high-pass filtered before being
`inserted into the original audio. In another system,!? the Fourier transform magnitudecoefficients over the frequency
`range from 2.4 kHz to 6.4 kHz are replaced with the watermark sequence. In these systems, inaudibility is further
`enhanced by only embedding watermarks in audio segments whose low frequency components have a higher energy
`value. The strong low frequency signals in the original audio could help to mask the embedded high frequency
`watermark signal.
`Another human ‘insensitive domain is the Fourier transform phase coefficients. Human ears are relatively insen-
`sitive to phase distortions, and especially lack the ability to perceive the absolute phase value. A scheme’ proposed
`to substitute the phase of an initial audio segment with a reference phase that represents the watermark. The phase
`of subsequent segmentsis adjusted to preserve the relative phase between segments. In another system,!? selected
`Fourier transform phasecoefficients in higher frequencies are discarded and new values are assigned based on neigh-
`boring reference coefficients. The watermark is represented by the relative phase betweenselected coefficients and
`their neighbors. The problem with watermarking schemes that hide watermark signals in perceptually insignificant
`regions is that they are less robust to signal processing and malicious attacks. Compression algorithmsdo not preserve
`these regions well so that malicious hackers could implement stronger attacks in these regions without introducing
`annoying noise.
`Anotherclass of algorithms embed watermarks as echosignals of the original audio. The inaudibility of echo
`hiding is based on the theory that-resonance is so common in our environment that human usually do not perceive
`it as noise. In these algorithms,”'# watermarksignals are actually delayed and attenuated versions of the original
`
`383
`
`

`

`signal. The watermark sequence is represented by delay amounts which are retrieved by observing autocorrelation
`peaks in the time domain’! or in the cepstrum domain.”
`~
`Recently, some researchers use a concept borrowed from spread spectrum communication and embed the water-
`mark as pseudo-random noise in the time domain. It is guaranteed by spread spectrum theory that the embedded
`watermark is statistically undetectable by hackers. Since human ears have different sensitivity to additive noise in
`different frequency bands, all proposed work uses somefilter to spectrally shape the pseudo-random (white) noise
`and achieve inaudibility. A simple band-pass filter was used in one work,!
`and a nonlinear filter was adopted in
`another.* In yet another system,!° instead of filtering white noise, a scheme was developed to generate the band-
`limited pseudo-random watermark signal. The inaudibility of the embedded watermark could be further ensured by
`utilizing the masking effects of the human auditory system. One system!®° used MPEG-I Audio Psychoacoustic
`Model 1 to spectrally shape the watermark signal while another system!’ used the masking model from MPEG-II
`AAC. Watermark detection is done by calculating the correlation between the watermarked audio signal and the wa-
`termark signal. Armed with the spread spectrum communication theory, this type of watermarking usually survives
`pretty well under distortions and attacks. However, synchronization is difficult to implement, and its computational
`cost is high.
`
`Another trend in digital audio watermarking is to combine watermark embedding with the compression or mod-
`ulation process. The integration could minimize unfavorable mutual interference between watermarking and com-
`pression, especially preventing the watermark from being removed by compression.
`In one scheme,'® ‘watermark
`embedding is performed during vector quantization. The watermark is embedded by changing the selected code
`vector or changing the distortion weighting factor used in the searching process. The need of the original audio to
`extract the watermark greatly limits the applications of this scheme. Another algorithm’? embeds watermark directly
`in the sigma delta modulation bitstream to eliminate the need of transforming it into PCM data, thereby keeping
`the computational cost low. This is important to the sigma delta modulation system, where hardware savings is
`the main goal. In another scheme,®?° watermarking is integrated with MPEG-II AAC compression. Watermarkis
`embedded by modifying selected compression coefficients such as the scale factor.
`
`4. PROPOSED ALGORITHM
`
`Although the methods describedin section 3 have their own features and properties, they share one common problem.
`That is, they are vulnerable to the synchronization attack in watermark detection. This problem could be resulted
`from casual audio editing such as cropping unwanted audio segments or intentional attacks such as randomly deleting
`or adding samples to watermarked audio data. This random sample cropping attack is very effective in interfering
`with the watermark detection process with respect to the algorithms mentioned above. This attack has a very low
`computational complexity. Besides, when donecorrectly, it would not introduce annoying noise to the underlying
`audio signals. One might argue that such a skillful attack could only be done by a few professionals and not by
`the majority of consumers. However, once a watermarking method is widely in use, it is almost certain that some
`professionals would produce and distribute attacking apparatuses so that a majority of common users would beable
`to perform the skillful attack. One method® was proposedto solve the synchronization problem, where an exhaustive
`search algorithm was used and the original audio signal was required. Consequently, its computational complexity
`is too high, and the needof original audio for watermark detection greatly limits its applications. Furthermore, it
`can only handle the casual editing attack, but not the random sample cropping attack.
`In this research, we propose a low complexity solution to the synchronization problem caused by both casual
`and malicious attacks. The solution is composed of a salient point extraction technique and a Fourier transform
`domain watermark embedding procedure. Salient point extraction through audio content analysis is done during
`both watermark embedding and detection processes so that synchronization is regained at each salient point. The
`extraction algorithm is designed such that salient points remain stable after distortion. TheFourier transform domain
`watermark embedding and detection is adopted since the frequency domain information is less effected by sample
`cropping in the time domain.
`
`One commoncharacteristic among most existing audio watermarking algorithms is that their watermark is em-
`bedded throughout the entire audio signal. However, this may not be the most efficient way to embed and detect
`watermarks. For a skilled attacker, different amount of attack could be applied to different segments of the audio
`signal to avoid introducing annoying noise. For example, randomly cropping (deleting) one sample out of every 100
`samples in high energy tonal segments of audio signals would produce noticeable noise, but the effect of doing so in
`
`384
`
`

`

`fnpnchnnfn—}—}ff]tg ttt}Ytotir
`erttha eney
`jtoo eertt
`rs)eatt
`TT eee) Uo td i
`eenoe
`7
`oat
`Leactileceal
`eT
`ptt|
`Sr
`iiHii
`Pee
`2 Aeeoto +}
`._
`Lah
`ry ——+
`
`
`
`neue finontaedietlimedlthdAtTt ——|celtealbeseedaneedhape co a asmsSoesenen
`
`ee
`tt rtee
`RS 4Creer
`eeOe
`i
`|
`c
`SEIGJA BAC,
`’
`1
`
`ele
`
`{
`
`PGE
`
`{GJAsB,|
`
`CIDJE,IF,IG4 ALB,
`
`Figure 1. Illustration of the correspondence between music notes and frequency values, and the 5-subbandpartition
`adopted in this work
`
`low energy segments would be inaudible. Thus, watermarks embedded in highly-attackable areas will face heavier
`attack and are morelikely to be destroyed. The second major contribution of this work is the introduction of “attack-
`sensitive regions” via audio content analysis. If the watermark is only embedded in attack-sensitive regions where
`little attack could be applied, the computational complexity of both watermark embedding and detection could be
`reduced.
`
`By combining techniquesof salient point extraction, attack-sensitive region identification, and Fourier transform
`domain watermark embedding and detection, we propose a complete audio watermark embedding and detection
`system for copyright protection. This system satisfies all desired properties of watermark design described earlier.
`Furthermore, it has a very low computational complexity, and it is robust to casual and intentional synchronization
`attacks. Although we incorporate the concept ofsalient point extraction and attack-sensitive regions into our own
`watermark embedding methodhere, it is our belief that other watermark embedding algorithms will benefit from
`the same concepts as well.
`
`4.1. Audio Content Analysis for Watermarking
`In our system, audio content analysis is performed for the purposes of salient point extraction and attack-sensitive
`region identification. Salient points in an audiosignal allow watermark detection to resynchronize at these locations.
`Synchronization by salient points has far less complexity than exhaustive search and makes blind watermark detection
`possible