UNITED STATES PATENT AND TRADEMARK OFFICE
`_____________
`
`BEFORE THE PATENT TRIAL AND APPEAL BOARD
`_____________
`
`SONY GROUP CORPORATION (JAPAN), SONY CORPORATION OF
`AMERICA, SONY INTERACTIVE ENTERTAINMENT LLC, SONY
`PICTURES ENTERTAINMENT INC., SONY ELECTRONICS INC., and
`VERANCE CORPORATION,
`Petitioners,
`
`v.
`
`MZ AUDIO SCIENCE, LLC,
`Patent Owner.
`_____________
`
`Case No. TBD
`Patent No. 7,289,961
`_____________
`
`DECLARATION OF RACHEL J. WATTERS
`RELATING TO EXHIBIT 1011(cid:3)
`
`Sony Exhibit 1034
`Sony v. MZ Audio
`
`

`

`Declaration of Rachel J. Watters on Authentication of Publication
`
`I, Rachel J. Watters, am a librarian, and the Head of Resource Sharing for the
`
`General Library System, Memorial Library, located at 728 State Street, Madison,
`
`Wisconsin, 53706. Part of my job responsibilities include oversight of Wisconsin
`
`TechSearch (“WTS”), an interlibrary loan department at the University of Wisconsin-
`
`Madison.
`
`I have worked as a librarian at the University of Wisconsinlibrary system
`
`since 1998, starting as a graduate student employee in the Kurt F. Wendt Engineering
`
`Library and WTS,then asa librarian in Interlibrary Loan at Memorial Library.
`
`I began
`
`professional employment at WTS in 2002 and became WTSDirector in 2011. In 2019,
`
`I became of Head of Resource Sharing for UW-Madison’s General Library System.
`
`I
`
`have a master’s degree in Library and Information Studies from the University of
`
`Wisconsin-Madison. Through the course of my studies and employment,I have
`
`become well informed aboutthe operations of the University of Wisconsin library
`
`system, which follows standard library practices.
`
`This Declarationrelates to the dates of receipt and availability of the following:
`
`Podilchuk, C.I. and Delp, E.J. (July 2001). Digital
`watermarking: algorithms and applications. [EEE Signal
`Processing Magazine, 18(4), 33-46.
`
`Standard operating procedures for materials at the University of Wisconsin-
`
`Madison Libraries. When an issue wasreceived by the Library, it would be checkedin,
`
`stamped with the date of receipt, added to library holdings records, and madeavailable
`
`

`

`Declaration of Rachel J. Watters on Authentication of Publication
`
`to readers as soonafterits arrival as possible. The procedure normally took a few days
`
`or at most 2 to 3 weeks.
`
`Exhibit A to this Declaration is a true and accurate copy of the table of contents
`
`page with library date stamp of JEEE Signal Processing Magazine (July 2001), from the
`
`University of Wisconsin-Madison Library collection. Exhibit A also includes an
`
`excerpt of pages 33 to 46 of that issue, showingthearticle entitled Digital
`
`watermarking: algorithms and applications (July 2001). Based on this information, the
`
`date stamp on the table of contents page indicates Digital watermarking: algorithms and
`
`applications (July 2001) was received by the Kurt F. Wendt Library, University of
`
`Wisconsin-Madison Libraries on August 1, 2001.
`
`Based on the information in Exhibit A,it is clear that the issue was received by |
`
`the library on or before August 1, 2001, catalogued and available to library patrons
`
`within a few daysor at most 2 to 3 weeks after August 1, 2001.
`
`Membersofthe interested public could locate the JEEE Signal Processing
`
`Magazine (July 2001) publication after it was cataloged by searching the public library
`
`catalog or requesting a search through WTS. The search could be donebytitle and/or
`
`subject key words. Membersofthe interested public could access the publication by
`
`locating it on the library’s shelves or requesting it from WTS.
`
`I declare thatall statements made herein of my own knowledgearetrue andthat
`
`all statements made on information and belief are believed to be true; and furtherthat
`
`2
`
`

`

`Declaration of Rachel J. Watters on Authentication of Publication
`
`these statements were made with the knowledgethat willful false statements and thelike
`
`so made are punishable by fine or imprisonment, or both, under Section 1001 ofTitle 18
`
`of the United States Code.
`
`Date: August 31, 2022
`
`Memorial Library
`728 State Street
`Madison, Wisconsin 53706
`
`Head of Resource Sharing
`
`

`

`
`
`
`
`
`
`EXHIBIT A
`EXHIBIT A
`
`

`

`Reebacmea
`
`RRR KKK RK RK RRR RAK EK AKER *S-DIGIT 53706
`22072488
`072001 310
`S76
`WISCONSIN UNIV
`Pl
`ENGRG LIB
`215 N RANDALL AVE
`MADISON WI 53706-1605
`
`(desert
`
`5= z
`
`

`

`
`
`rable of
`
`IEEE SIGNAL PROCESSING MAGAZINE e JULY 2001 VOL. 18, NO. 4
`
`
`
`
`
`IEEE SIGNAL PROCESSING MAGAZINE
`Aggelos Katsaggelos, Editor-In-Chief
`— Northwestern University
`
`EDITORIAL BOARD
`
`Dana H. Brooks — Northeastern University
`Tsuhan Chen — Carnegie Mellon University
`Nikolas P. Galatsanos — Illinois Institute ofTechnology
`Bernd Girod — University of Erlangen (Germany)
`Robert M. Gray — Stanford University
`Jelena Kovacevic — Bell Labs
`Chung J. Kuo — National Chung :
`Cheng Univeristy (Taiwan)
`Majid Nayeri — ERIM International
`Ashok Rao — Comsat Laboratories
`
`Ronald W.Schafer — Georgia Institute of Technology
`Thomas Sikora — Heinrich-Hertz Institute for
`
`Communication Technology (Germany)
`
`IEEE PERIODICALS
`MAGAZINES DEPARTMENT
`
`Geraldine Krolin — Senior Managing Editor
`
`Susan Schneiderman
`
`Business Development Manager
`+1 732 562 3946 Fax: +1 732 981 1855
`
`Dawn M. Melley — Editorial Director
`Robert Smrek — Production Director
`
`Cathline Tanis — Sr. Advertising
`Production Coordinator
`
`SIGNAL PROCESSING SOCIETY (2001)
`Panos E. Papamichalis — President
`Richard V. Cox — President-Elect
`Leah H. Jamieson — Past President
`Michael D. Zoltowski — Secretary
`Alfred O. Hero — Vice President, Finance
`José M.F. Moura — Vice President, Publications
`Andreas $. Spanias — Vice President, Conferences
`Russell M. Mersereau — Vice President,
`Awards and Membership
`Mercy Kowalezyk — Executive Director
`and Associate Editor
`
`SCOPE: IEEE Signal Processing Magazine
`publishes tutorial-style articles on signal processing
`research and applications.It is the primary meansof
`communicationofthe Society leadership with the
`entire membership.
`
`BPAInternational Membership applied for
`November 2000.
`
`JULY 2001
`
`IEEE SIGNAL PROCESSING MAGAZINE
`
`
`
`features
`Genomic Signal Processing
`8
`
`—Dimitris Anastassiou
`Lossless Compression of Digital Audio
`—Mat Hans and Ronald W. Schafer
`
`Digital Watermarking: Algorithms and Applications
`—Christine I. Podilchuk and Edward J. Delp
`
`21
`
`33
`
`+7 F. WENDT LIBRARY
`COLLEGE OF ENGINEERING
`AG O1 2001
`UW-MADISON, WI 5370¢
` departments
`
`From the Editor
`
`2
`
`Dates Ahead
`
`50
`
`Literature Search
`
`52
`
`ON THE COVER:
`
`Illustration: Jim Hankard
`and Janet Dudar
`
`IEEE SIGNAL PROCESSING MAGAZINE (ISSN 1053-5888) (ISPREG)is published bi-monthly
`bythe Institute ofElectrical and Electronics Engineers, Inc., 3 Park Avenue, 17th Floor, NewYork,
`NY 10016-5997 (+1 212 419 7900). Responsibility for the contents rests upon the authors and not
`the IEEE,the Society, oritsmembers. Annual subscriptions: $20 per memberperyear included in
`Societyfee. Nonmembersubscriptionsavailable upon request. Copyright and Reprint Permissions:
`Abstracting is permitted with credit to the source. Libraries are permitted to photocopy beyondthe
`limits ofU.S. Copyright Lawforprivate use ofpatrons: (1) those post-1977articlesthat carrya code
`at the bottomofthe first page, provided the per-copyfee indicatedin the codeis paid throughthe
`Copyright Clearance Center, 222 RosewoodDrive, Danvers, MA 01923;(2) pre-1978 articles with-
`outfee. Instructors are permitted to photocopyisolated articles for noncommercial classroomuse
`withoutfee. Forall other copying, reprint, or republication permission, write to IEEE Service Cen-
`ter, 445 Hoes Lane,Piscataway, NJ 08855-1331. Copyright© 2001 bythe Institute ofElectrical and
`Electronics Engineers, Inc. All rights reserved. Periodicals postage paid at NewYork, NY, andat ad-
`ditional mailing offices. Postmaster: Send address changes to IEEE SP Magazine, IEEE, P.O. Box
`1331, 445 Hoes Lane, Piscataway, NJ 08855-1331.
`
`
`
`4
`President's Message
`Janet Dudar — Art Director
`Debra Cantillo — Associate Art Director
`Book Beat
`54
`Forum=6
`Fran Zappulla — Staff Director,
`Advertisers Index
`56
`New Products
`47
`Publishing Operations
`
`

`

`
`CORP. Christine I. Podilchuk
`
`and EdwardJ. Delp
`
`igital watermarking of multimedia content
`has becomea very active research area over
`the last several years. A general framework
`for watermark embedding and detection/de-
`codingis presented here along with a review of some of
`the algorithmsfor different media types described in the
`literature. We highlight someofthe differences based on
`application such as copyright protection, authentication,
`tamperdetection,and data hiding as well as differences in
`technology and system requirements for different media
`types such as digital images, video, audio andtext.
`
`Introduction
`
`The success ofthe Internet, cost-effective and popular dig-
`ital recording and storage devices, and the promise of
`higher bandwidth and quality of service (QoS) for both
`wired andwireless networks has madeit possible to create,
`replicate, transmit, and distribute digital content in an ef-
`fortless way. The protection and enforcementofintellec-
`tual property rights for digital media has become an
`important issue. In 1998, Congress passed the Digital
`Millenium Copyright Act (DMCA)which makesit illegal
`to circumventany technological measure that protects an
`owner’sintellectual property rights ofdigital content. The
`headline news regarding Napster madethe general public
`awareofthe issues regarding intellectual property rights
`and the impact of current technology.
`In recentyears, the research community has seen much
`activityin the area ofdigital watermarkingas an additional
`
`IEEE SIGNAL PROCESSING MAGAZINE
`1053-5888/01/$10.00 ©200 IEEE
`
`33
`
`
`
`©1998CORBIS
`
`
`
`
`
`

`

`
`
`There should be no perceptible
`difference between the
`watermarkedandoriginal signal,
`and the watermark should be
`difficult to removeoralter without
`damaging the hostsignal.
`
`tool in protecting digital content and manyexcellent .pa-
`pers have appeared in special issues [1], [2], as well as
`dedicated conferences and workshops[3]-[5]. New com-
`panies dedicated to watermarking technology are emerg-
`ing and products like Digimarce’s MediaBridge are
`appearing [6]. Unlike encryption, which is useful for
`transmission but does not provide a way to examine the
`original data in its protected form, the watermark re-
`mains in the contentin its original form and doesnotpre-
`vent a user from listening to, viewing, examining, or
`manipulating the content. Also, unlike the idea of
`steganography, where the method of hiding the message
`may be secret and the message itself is secret,
`in
`watermarking, typically the watermark embedding pro-
`cess is known and the message (exceptforthe useofa se-
`cret key) does not haveto besecret. In steganography,
`usually the messageitself is of value and must be pro-
`tected through clever hiding techniques and the “vessel”
`for hiding the messageis not of value. In watermarking,
`the effective coupling of message to the “vessel,” which is
`the digital content, is of value and the protection of the
`contentis crucial. Watermarkingis the direct embedding
`of additional information into the original content or
`hostsignal. Ideally, there should be no perceptible differ-
`ence between the watermarked and originalsignal [7],
`[8] and the watermark shouldbe difficult to remove oral-
`ter without damagingthe host signal. In some instances,
`the amount of information that can be hidden and de-
`tected reliablyis important. It is easy to see that the re-
`quirements of imperceptibility, robustness, and capacity
`conflict with each other. For instance, a straightforward
`wayto provide an imperceptible watermark is to embed
`the watermark signal into the perceptually insignificant
`portionof the host data. However, this makes the water-
`mark vulnerable to attack becauseit is fairly easy to re-
`moveoralter the watermark withoutaffecting the host
`signal. To provide a robust watermark, a goodstrategyis
`to embed the watermark signal into the significant por-
`tion of the host signal. This portion of the host data is
`highlysensitive to alterations, however, and mayproduce
`veryaudible orvisible distortions in the host data. Appli-
`cations for digital watermarking include copyright pro-
`tection, fingerprinting, authentication, copy control,
`tamperdetection, and data hiding applications such as
`broadcast monitoring. Watermarking algorithms have
`
`been proposed for audio, still images, video, graphics,
`and text, and excellent review articles on multimedia
`watermarking can be foundin [9]-[13].
`Visible watermarks which do notinterfere with the in-
`telligibility of the host signal have also been proposed
`[14]. In this article, we limit the scope of our review to
`transparent marking techniques. Transparent
`watermarking techniques can be fragile, robust, or
`semifragile. Fragile watermarks donotsurvive lossytrans-
`formationsto the original hostsignal and their purposeis
`tamperdetection ofthe original signal. There are many
`effective ways to insert a fragile watermark into digital
`content while preserving the imperceptibility require-
`ment. Placing the watermark informationinto thepercep-
`tually insignificant portions of the data guarantees
`imperceptibility and provides fragile marking capabili-
`ties. For instance, early watermark techniquesforstill im-
`age data propose inserting watermark information into
`the least significant bits of the pixel values. This results in
`an imperceptible mark whichcandetect lossy transforma-
`tions performed on the watermarked content. For secu-
`rity applications and copyright protection, robust
`watermarking techniques have been proposed. Here the
`technical challengeis to provide transparency and robust-
`ness which are conflicting requirements. Ideally, an effec-
`tive, robust watermarking scheme provides a mark that
`can only be removed whenthe original content is de-
`stroyed as well. The degree of robustness and distortion
`necessaryto alter the value ofthe original content can vary
`for different applications. Typically, many of the applica-
`tions for copyright protection involve relatively high
`quality original content and the imperceptibility criterion
`is critical for such applications. The authors in [15] and
`[16] were the first to describe that in order for a
`watermarking technique to be robust, the watermark
`should be embeddedin theperceptually significant portion
`ofthe data. Sometypical distortions orattacks thatdigital
`watermarking schemes are expected to survive include
`resampling, rescaling, compression, linear and nonlinear
`filtering, additive noise, A/D and D/A conversion, and
`transcoding. Applications for robust watermarking in-
`clude copyright protection where each copy gets a unique
`watermark (commonly referred to as a fingerprint) to
`identify the end-userso thattracing is possible for cases of
`illegal use; authentication, where the watermark can rep-
`resent a signature and copy control for digital recording
`devices. Within the class of robust watermarking tech-
`niques there are several different constraints on encoder
`and decoderdesign which depends ontheparticular ap-
`plication. The differences are discussed in detail later in
`this paper. Semifragile watermarking techniques differ-
`entiate between lossy transformationsthatare “informa-
`tion preserving” and lossy transformations which are
`“information altering.” Lossy transformations include
`any signal processing step that alters the original signal
`values andis not invertible. For example, in authentica-
`tion applications it maybe desirable to have a watermark
`
`
`
`34
`
`IEEE SIGNAL PROCESSING MAGAZINE
`
`JULY 2001
`
`
`

`

`that can distinguish between a lossy transformation
`such as compression whichdoesnotalter the integrity
`of the contentandan alteration whichdoesalter the in-
`tegrity, such as manipulating or replacing objects
`within the scene.
`Requirements and design ofwatermarking techniques
`are impacted bythedifferent types of contentin two ma-
`jor ways: imperceptibility and robustness requirements.
`Thefirst challenge is designing a watermark embedding
`algorithm which provides an imperceptible mark,thatis,
`one which doesnot noticeably degradethe original host
`signal. By taking advantage of psychovisual and psycho-
`auditory properties, we can design effective watermark-
`ing schemes which remain transparent under particular
`conditions [7], [8], [17]-[22]. Ideally, the marking algo-
`rithm should be adapted by using perceptual models ap-
`propriate for the different media types. The perceptual
`models used for representations of continuous tone im-
`ages are not appropriate for text or graphics. The other
`factor for designing watermarking schemes for multime-
`dia is the type of degradations that the watermark ts ex-
`pected to survive and system requirements for media
`specific applications. Forinstance, it may be desirable for
`a still image watermarking techniqueto beable to survive
`JPEG compression and photocopying while for some
`video watermarking applications, it may be important to
`do watermark embedding anddetectioninreal time on a
`compressed bit stream.
`In the next section we describe watermarking for dif-
`ferent media types including an overview of some sample
`algorithms proposedintheliterature. This is followed by
`a description of a general framework for watermark em-
`bedding and watermark detection and decoding, outlin-
`ing someofthe differences for different applications. We
`then review some work on modeling the general
`watermarking problem and drawing parallels to commu-
`nication and information theory to help understand the
`fundamental properties and limitations of a watermark-
`ing system. This workis very useful for future algorithm
`design and helping to define open areas of research.
`Lastly, we review and summarize future directionsin this
`new andexciting area.
`
`Media Requirements
`Here weexplore the requirements for watermarking sys-
`temsdesignedfor different media types and review some
`of the algorithms coveredin theliterature.
`
`Image Watermarking
`Many techniques have been developed for the
`watermarking of still
`image data. For grey-level or
`color-image watermarking, watermark embedding tech-
`niques are designedto insert the watermarkdirectly into
`the original image data, such as the luminance or color
`components or into some transformed version of the
`original data to take advantageofperceptual properties or
`
`robustness to particular signal manipulations. Require-
`ments for image watermarking include imperceptibility,
`robustness to commonsignal processing operations, and
`capacity. Commonsignal processing operations which
`the watermark should survive include compression (such
`as JPEG), filtering, rescaling, cropping, A/D and D/A
`conversion, geometric distortions, and additive noise.
`Capacity refers to the amount of information (or pay-
`load) that can be hidden in the host image and detected
`reliably under normal operating conditions. Many of the
`watermarking techniques are additive, where the water-
`marksignal is added directly to the host signal or trans-
`formed host signal. The watermark may be scaled
`appropriately to minimize noticeable distortions to the
`host. Perceptual models may be used to determine and
`adaptthe watermarkscale factor appropriately to the host
`data. The watermarkitselfis a function of the watermark
`information,a secret or public key and perhapsthe origi-
`nal host data. Some examples of watermark information
`includes a binary sequence representing a serial number
`or credit card number, a logo, a picture, or a signature.
`Manyofthe current watermarking techniques insert one
`bit of information over manypixels or transform coeffi-
`cients and useclassical detection schemes to recover the
`watermark information. These types of watermarking
`techniquesare usually referred to as spread-spectrum ap-
`proaches, due to their similarity to spread-spectrum com-
`munication systems. Forstill image watermarking,
`watermark embeddingis applieddirectly to the pixelval-
`ues in the spatial domainor to transform coefficients in a
`transform domain such as the discrete cosine transform
`(DCT) or discrete wavelet transform (DWT). Water-
`mark detection usually consists of some preprocessing
`step (which mayinclude removalofthe original hostsig-
`nal ifit is available for detection) followed by a correlation
`operator. Moredetails on watermark embedding and de-
`tection appearlater. Spatial-domain watermarking tech-
`niques for image data include [23]-[28]. Some of the
`earliest techniques [23], [29], [28] embed m-sequences
`into the least significant bit (LSB) of the data to provide
`an effective transparent embedding technique.
`M-sequences are chosen due to their good correlation
`properties so that a correlation operation can be used for
`watermark detection. Furthermore, these techniques are
`computationally inexpensive to implement. Such a
`schemewasfirst proposed in [23] and extendedto two di-
`mensions in [29]. In [28]
`the authors reshape the
`m-sequence into two-dimensional watermark blocks
`which are addedanddetected on a block-by-blockbasis.
`The block-based method, referred to as variable-w
`two-dimensional watermark (VW2D) is showntobe ro-
`bust to JPEG compression. This technique has also been
`shownto be an effective fragile watermarking scheme
`whichcan detectimage alterations on a block basis [30].
`Other early work [31] suggests using check sums for LSB
`watermark embedding.
`
`JULY 2001
`
`IEEE SIGNAL PROCESSING MAGAZINE
`
`35
`
`
`
`
`
`

`

`Several spatial-domain watermarking techniques for
`images are proposed in [25]. One technique consists of
`embeddinga texture-based watermark into a portion of
`the image with similar texture. Theidea hereis that due to
`the similarity in texture,it will be difficult to perceive the
`watermark. The watermarkis detectedusing a correlation
`detector. Another technique described as the patchwork
`methoddivides the imageinto two subsets A and B where
`the brightness of one subset is incremented by a small
`amountandthebrightnessofthe otherset is decremented
`by the same amount. The incremental brightnesslevelis
`chosenso that the changein intensity remains impercep-
`tible. The location of the subsets is secret and assuming
`certain properties for image data, the watermarkis easily
`located by averaging the difference between the valuesin
`the two subsets. It is assumed that, on average, without
`the watermark, this value will go to zero for image data.
`In the example wherethepixels in Set A are incremented
`by one and the pixels in set B are decremented by one,
`with N locationsin the set, the expected value of the sum
`of differences between the sets is given by 2N. For
`nonwatermarked data, this value should go to zero. A
`variation of this approach is described in.[27], where
`more information can beinserted in the host signal. An-
`other spatial-domain technique is proposed in [32],
`wherethe blue componentof an image in RGB formatis
`watermarked to ensure robustness while remaining fairly
`insensitive to human visual system (HVS) factors.
`Transform domain watermarkingis useful for taking
`advantage of perceptualcriteria in the embedding pro-
`cess, for designing watermarking techniques which are
`robust to common compression techniques, and for di-
`rect watermark embedding of compressedbit streams. A
`common transform framework for images is the
`block-based DCT whichis a fundamental building block
`of current image coding standards such as JPEG and
`video coding standards such as the MPEG video coders
`[33] and the ITU H.26xfamily ofcodecs. Oneofthefirst
`block-based DCT watermarking techniqueis proposedin
`[34]. The DCT is performed on 8x8 blocks of data, a
`pseudorandomsubset ofthe blocks are chosen anda trip-
`
`let ofmidrange frequenciesare slightly altered to encode a
`binary sequence. This is a reasonable heuristic
`watermarking approach since watermarks inserted in the
`high frequencies are vulnerable to attack whereas the low
`frequency components are perceptually significant and
`sensitive to alterations. One of the most influential
`watermarking works [15], [16] was first to describe how
`spread spectrum principles borrowed from communica-
`tion theory can be used in the context of watermarking.
`The published results show that the techniqueis very ef-
`fective both in terms of image quality and robustness to
`signal processing and attempts to remove the watermark.
`The techniqueis motivated by both perceptual transpar-
`ency and watermark robustness. One ofthe significant
`contributions in this work is the realization that the wa-
`termark should be inserted in the perceptually significant
`portion ofthe image in orderforit to be robust to attack.
`A DCT is performed on the whole image and the water-
`mark is inserted in a predetermined range of low fre-
`quency components minus the DC component. The
`watermarkconsists of a sequence of real numbers gener-
`ated from a Gaussian distribution which is added to the
`DCT-coefficients. The watermark signal is scaled accord-
`ing to the signal strength ofthe particular frequency com-
`ponent. Thisis a reasonable and simple wayto introduce
`sometype ofperceptual weightinginto the watermarking
`scheme. The watermark embedding algorithm could be
`described as
`
`X =S(1+aW)
`
`(1)
`
`
`
`whereS is the original host signal, X is the watermarked
`signal, and W is the watermark consisting of a random,
`Gaussian distributed sequence. a is a scaling factor which
`the authors suggestto set to 0.1 to provide a goodtrade-
`offbetween imperceptibility and robustness. Referring to
`Fig. 1 for a block diagram of a general watermarking sys-
`tem, the secret key is used to generate the random se-
`quence W in this case. Also note that this particular
`algorithm addressesthe case ofwatermark detection where
`you wouldlike to detect whether a particular watermark
`is Or is not present in the host signal at the receiver. The
`
`Secret Key K
`
`Watermarked
`Signal X —
`
`Attack/
`Distortion
`Channel
`
`Distorted
`Watermark
`Signal Y
`
`A
`Recovered Message M
`
`Watermark
`Detection/
`Identification
`
`Message M
`
`Watermark
`Insertion
`
`_ Original Host S
`
`
`A 1. Block diagram of a watermarking system.
`
`36
`
`IEEE SIGNAL PROCESSING MAGAZINE
`
`JULY 2001
`
`

`

`| SINVESTIGATION WILL BE NOTIFIED.
`
`
`where W is the extracted watermark from
`the received, possibly distorted signal Y.
`The authors show that the similarity mea-
`sure is also normally distributed so that a
`highsimilarity value is extremely unlikely
`for W #W.Otherpostfiltering operations
`could be performed to undo possible dis-
`tortions, improve performance, and get a
`better similarity measure. Moredetails on
`improving detection results can be found
`later.
`A variationonthis ideais variable length
`DCT-based watermarking [35], where the
`DCT coefficients are sorted by magnitude
`and only the w largest coefficients are
`marked that correspondto a userspecified
`percentofthe total energy. This allows the
`user to trade off imperceptibility and ro-
`bustness to attack. Other DCT-based
`watermarking schemes use more elaborate
`models of the human visual system to in-
`corporate an image adaptive watermark of
`maximum strength subject to the imper-
`ceptibility criterion [17], [7], [8]. Two im-
`age-adaptive watermarking schemes are
`described in [19] and [7], which are based
`on a block-based DCT framework and
`wavelet framework. The perceptual models
`used here can be described in termsof three
`different properties of the human visual
`system that have been studied in the con-
`image-adaptive quantization table [36]. A similar model
`text of image coding: frequency sensitivity, luminance
`was developed for wavelet-based compression using only
`sensitivity, and contrast masking [36]. Frequencysensi-
`frequency sensitivity to derive perceptual weights for
`tivity describes the human eye’s sensitivity to sine wave
`each of the subbands [37]. This model was used for a
`gratings at various frequencies. This componentonly de-
`wavelet-based watermarking scheme [19], [7]. Unlike
`pends on the modulation transfer function (MTF) of the
`compression, where the amountof perceptual informa-
`eye andis independentofthe image data. Luminance sen-
`tion that can be incorporated into the encoderis limited
`sitivity measurestheeffect of the detectability threshold
`to the amountof side information that is necessary to
`of noise on a constant background. For the human visual
`transmit this information to the decoder, all of the per-
`system,this is a nonlinear function and dependson local
`ceptual information can be utilized in a watermarking
`image characteristics. Contrast masking refers to the
`scheme. For instance, in JPEG, we are limited to one
`detectability of one signal in the presence of anothersig-
`quantization matrix for the entire image which cannot
`nal and the effect is strongest when both signalsare ofthe
`take full advantageof local visual threshold characteris-
`samespatial frequency, orientation, and location. A com-
`tics. The image dependent masking thresholds are used to
`bination of the three componentsresults injust noticeable
`determine the location and maximum strength ofthe wa-
`distortion (JND) thresholds for the entire image. These
`termark signal that can be tolerated in every location of
`models werefirst developed to design more efficient im-
`the host image underthe constraint of imperceptibility at
`age compression schemes than -waveform techniques
`somespecified viewing condition. Examples of the im-
`alone could provide. This modelwasderived forthe base-
`age-adaptive watermarks describedin [7] areillustrated
`line mode of JPEG and showedasignificant improve-
`mentin compression performance whenused to derive an
`in Fig.2.
`
`;
`
`isoto. OFFENSE PUNISHABLE BY FINES
`S10,000,
`IMPRISONMENT FOR 20 YEARS, OR
`SDEATH IF SUCH A THEFT CAUSES AN ACCIDENT
`MIING IN LOSS OF LIFE.
`THE FEDERAL BURE|
`
`
`
`
`watermarkdetector for this scheme [15] is
`
`described by theet measure
`sim(W,w)=et
`
`i.
`
`S DAMAGING OR DISABLING ANY AIRCRAFT LOCA
`<3
`THESE PREMISES BY STEALING RADIOS, NAVI
`
`
`
`2. Watermarked images(first, third row) and corresponding image-adaptive wa-
`termarks using perceptual models (second, fourth row).
`
`JULY 2001
`
`IEEE SIGNAL PROCESSING MAGAZINE
`
`37
`
`€
`

`

`
`
`A 3. Watermark examplefor different viewing distances.
`
`Note how the watermark structureis similar to the lo-
`image properties. Fig. 3 illustrates several water-
`cal
`marked images and the corresponding watermarkto the
`right of the watermarked image. The three examples
`show the watermarked imagefor different viewing condi-
`tions where the top image corresponds to a viewing dis-
`tance of four times the image height, the middle image
`correspondsto a viewing distance of two times the image
`height and the bottom image corresponds to a viewing
`distance of one times the image height. Modifying the
`viewing conditions of the watermark embedding algo-
`rithm allows for a tradeoff between imperceptibility and
`robustnessto certain types of attacks. These examples are
`forillustrative purposes and the viewing conditions here
`are not based on viewing printed images.
`Two DCT-based approaches were described in [38]
`and [39] where watermark detection does not require the
`original image. The methodin[40] is an extensionof the
`method proposed in [19] and [7] to the case where the
`original host signalis not available for watermark detec-
`tion. This is an important feature for some applications
`suchas authentication and will be covered in moredetail
`later. Another block-based frequency domain technique
`describedin [41] is based oninserting a watermark into
`
`the phase componentsof the image data using the same
`motivation as in [16], that for the watermarkto be robust
`to attack, it must be embedded in the perceptually signifi-
`cant portion of the data. It has beenestablished that for
`image data, the phase information is perceptually more
`important than the magnitude data. Other novel ap-
`proaches for watermarking image data include
`fractal-based approaches [42], [43] and geometric fea-
`ture based watermarking [44]. In [44], salient points in
`an image are found and warpedaccordingto a denseline
`pattern representing the watermark and generated ran-
`domly. Detection consists of determining whethera sig-
`nificantly large numberofpoints are within the vicinity of
`the line patterns.
`The type of distortions or attacks that image
`watermarking techniquesare designedto survivefall into
`two broad categories—noise type distortions. like com-
`pression and geometrical distortions which cause loss of
`synchronization for detection, such as resamplingand ro-
`tations. Watermarking schemesfor tamper detection and
`tamper estimation to be able to differentiate between
`lossy attacks whichalter the information andlossy attacks
`which do notalter the information [45]-[47] have also
`been proposed.
`
`Document Watermarking
`Much ofthe early work on recognizing the potential
`problemswith intellectual propert