HHAPOCESIING SOCIElY
`
`leetye )
`Pinitsi[< Code
`with DSP
`
`tek ek kk kK RR KK RK RK KAAS DIGIT 53706
`22072488
`072001 310
`S76
`WISCONSIN UNIV
`Pi
`ENGRG LIB
`215 N RANDALL AVE
`MADISON WI 53706-1605
`
`TadaLeneUDofseatUDvvsuldorsoe/UaUlaslDvanalelaveeldelelolll
`
`July 2001 m Vol.18, No. 4
`ISSN 1053-5888
`
`Sony Exhibit 1011
`Sony Exhibit 1011
`Sony v. MZ Audio
`Sony v. MZ Audio
`
`

`

`rable of
`
`IEEE SIGNAL PROCESSING MAGAZINE e JULY 2001 VOL. 18, NO. 4
`
`features
`Genomic Signal Processing
`—Dimitris Anastassiou
`
`Lossless Compression of Digital Audio
`—Mat Hansand Ronald W. Schafer
`
`Digital Watermarking: Algorithms and Applications
`—Christine I. Podilchuk and Edward J. Delp
`
`21
`
`33
`
`JULY 2001
`
`IEEE SIGNAL PROCESSING MAGAZINE
`Aggelos Katsaggelos, Editor-In-Chief
`— Northwestern University
`
`EDITORIAL BOARD
`
`Dana H. Brooks — Northeastern University
`Tsuhan Chen — Carnegie Mellon University
`Nikolas P. Galatsanos — Illinois Institute ofTechnology
`Bernd Girod — University of Erlangen (Germany)
`Robert M. Gray — Stanford University
`Jelena Kovacevic — Bell Labs
`Chung J. Kuo — National Chung
`Cheng Univeristy (Taiwan)
`Majid Nayeri — ERIM International
`Ashok Rao — Comsat Laboratories
`
`Ronald W.Schafer — Georgia Institute of Technology
`Thomas Sikora — Heinrich-Hertz Institute for
`
`Communication Technology (Germany)
`
`IEEE PERIODICALS
`MAGAZINES DEPARTMENT
`
`Geraldine Krolin — Senior Managing Editor
`
`Susan Schneiderman
`
`Business Development Manager
`+1 732 562 3946 Fax: +1 732 981 1855
`
`DawnM. Melley — Editorial Director
`Robert Smrek — Production Director
`
`Cathline Tanis — Sr. Advertising
`Production Coordinator
`
`Janet Dudar — Art Director
`Debra Cantillo — Associate Art Director
`
`Fran Zappulla — Staff Director,
`Publishing Operations
`
`SIGNAL PROCESSING SOCIETY (2001)
`Panos E. Papamichalis — President
`Richard V. Cox — President-Elect
`Leah H. Jamieson — Past President
`Michael D. Zoltowski — Secretary
`Alfred O. Hero — Vice President, Finance
`José M.F. Moura — Vice President, Publications
`Andreas $. Spanias — Vice President, Conferences
`Russell M. Mersereau — Vice President,
`Awards and Membership
`Mercy Kowalczyk — Executive Director
`and Associate Editor
`
`SCOPE: IEEE Signal Processing Magazine
`publishes tutorial-style articles on signal processing
`research and applications.It is the primary meansof
`communicationofthe Society leadership with the
`entire membership.
`
`BPAInternational Membershipapplied for
`November2000.
`
`
`
`£. WENDT LIBRARY
`KURT F. WEN
`RING
`COLLEGE OF ENGINEE
`AUG O41 2001
`UW-MADISON, WI 5970¢
` departments
`
`From the Editor
`
`President's Message
`Forum
`New Products
`
`2
`
`4
`6
`47
`
`Dates Ahead
`
`Literature Search
`
`Book Beat
`
`Advertisers Index
`
`50
`
`52
`
`54
`
`56
`
`ON THE COVER:
`
`Illustration: Jim Hankard
`and Janet Dudar
`
`IEEE SIGNAL PROCESSING MAGAZINE (ISSN 1053-5888) (ISPREG)is published bi-monthly
`bythe Institute ofElectrical and Electronics Engineers, Inc., 3 Park Avenue, 17th Floor, NewYork,
`NY10016-5997 (+1 212 419 7900). Responsibilityfor the contentsrests uponthe authors and not
`the IEEE,the Society, oritsmembers. Annual subscriptions: $20 per memberperyear includedin
`Society fee. Nonmembersubscriptions available upon request. Copyright and Reprint Permissions:
`Abstracting is permitted with credit to the source. Libraries are permitted to photocopy beyondthe
`limits ofU.S. Copyright Lawforprivate use ofpatrons: (1) those post-1977articlesthat carrya code
`at the bottomofthe first page, providedthe per-copyfee indicatedin the codeis paid throughthe
`Copyright Clearance Center, 222 RosewoodDrive, Danvers, MA 01923;(2) pre-1978 articles with-
`outfee. Instructors are permitted to photocopyisolated articles for noncommercial classroomuse
`withoutfee. Forall other copying, reprint, or republication permission, write to IEEE Service Cen-
`ter, 445 Hoes Lane,Piscataway, NJ 08855-1331. Copyright©2001 bythe Institute of Electrical and
`Electronics Engineers, Inc. All rights reserved. Periodicals postage paid at NewYork, NY, andat ad-
`ditional mailing offices. Postmaster: Send address changes to IEEE SP Magazine, IEEE, P.O. Box
`1331, 445 Hoes Lane, Piscataway, NJ 08855-1331.
`
`IEEE SIGNAL PROCESSING MAGAZINE
`
`
`
`
`

`

`
`
`Digital
`Watermarking:
`Alg
`or‘thms
`pplications
`an
`
`and EdwardJ. Delp
`
` Christine I. Podilchuk
`
`igital watermarking of multimedia content
`has becomea very active research area over
`the last several years. A general framework
`for watermark embedding and detection/de-
`codingis presented here along with a review of some of
`the algorithmsfor different media types described in the
`literature. We highlight someofthe differences based on
`application such as copyright protection, authentication,
`tamperdetection,and data hiding as well as differences in
`technology and system requirements for different media
`types such as digital images, video, audio andtext.
`
`Introduction
`
`The success ofthe Internet, cost-effective and popular dig-
`ital recording and storage devices, and the promise of
`higher bandwidth and quality of service (QoS) for both
`wired andwireless networks has madeit possible to create,
`replicate, transmit, and distribute digital contentin an ef-
`fortless way. The protection and enforcementofintellec-
`tual property rights for digital media has become an
`important issue. In 1998, Congress passed the Digital
`Millenium Copyright Act (DMCA)which makesit illegal
`to circumventany technological measure that protects an
`owner’sintellectual property rights ofdigital content. The
`headline news regarding Napster madethe general public
`awareofthe issues regarding intellectual property rights
`and the impact of current technology.
`In recentyears, the research community has seen much
`activityin the area ofdigital watermarkingas an additional
`
`
`
`
`
`©1998CORBISCORP.
`
`JULY 2001
`
`IEEE SIGNAL PROCESSING MAGAZINE
`1053-5888/01/$10.00© 200 IEEE
`
`33
`
`
`
`
`
`
`
`

`

`
`
`There should be no perceptible
`difference between the
`watermarkedandoriginal signal,
`and the watermark should be
`difficult to removeor alter without
`damagingthe hostsignal.
`
`tool in protecting digital content and manyexcellent pa-
`pers have appeared in special issues [1], [2], as well as
`dedicated conferences and workshops[3]-[5]. New com-
`panies dedicated to watermarking technology are emerg-
`ing and products like Digimarce’s MediaBridge are
`appearing [6]. Unlike encryption, which is useful for
`transmission but does not provide a way to examine the
`original data in its protected form, the watermark re-
`mains in the contentin its original form and doesnotpre-
`vent a user from listening to, viewing, examining, or
`manipulating the content. Also, unlike the idea of
`steganography, where the method of hiding the message
`may be secret and the message itself is secret,
`in
`watermarking, typically the watermark embedding pro-
`cess is known and the message (exceptforthe use ofa se-
`cret key) does not haveto besecret. In steganography,
`usually the messageitself is of value and must be pro-
`tected through clever hiding techniques and the “vessel”
`for hiding the messageis notof value. In watermarking,
`the effective coupling of message to the “vessel,” which is
`the digital content, is of value and the protection of the
`contentis crucial. Watermarkingis the direct embedding
`of additional information into the original content or
`hostsignal. Ideally, there should be no perceptible differ-
`ence between the watermarked and original signal [7],
`[8] and the watermark shouldbedifficult to remove oral-
`ter without damagingthe host signal. In some instances,
`the amountof information that can be hidden and de-
`tected reliablyis important. It is easy to see that the re-
`quirements of imperceptibility, robustness, and capacity
`conflict with each other. For instance, a straightforward
`wayto provide an imperceptible watermark is to embed
`the watermark signal into the perceptually insignificant
`portionof the host data. However, this makes the water-
`mark vulnerable to attack becauseit is fairly easy to re-
`moveoralter the watermark withoutaffecting the host
`signal. To provide a robust watermark, a goodstrategyis
`to embed the watermark signal into the significant por-
`tion of the host signal. This portion of the host data is
`highlysensitive to alterations, however, and mayproduce
`veryaudible orvisible distortions in the host data. Appli-
`cations for digital watermarking include copyright pro-
`tection, fingerprinting, authentication, copy control,
`tamperdetection, and data hiding applications such as
`broadcast monitoring. Watermarking algorithms have
`
`been proposed for audio, still images, video, graphics,
`and text, and excellent review articles on multimedia
`watermarking can be foundin [9]-[13].
`Visible watermarks which do notinterfere with the in-
`telligibility of the host signal have also been proposed
`[14]. In this article, we limit the scope of our review to
`transparent marking techniques. Transparent
`watermarking techniques can be fragile, robust, or
`semifragile. Fragile watermarks donotsurvive lossytrans-
`formationsto the original hostsignal and their purposeis
`tamperdetection of the original signal. There are many
`effective ways to insert a fragile watermark into digital
`content while preserving the imperceptibility require-
`ment. Placing the watermark informationinto thepercep-
`tually insignificant portions of the data guarantees
`imperceptibility and provides fragile marking capabili-
`ties. For instance, early watermark techniquesforstill im-
`age data propose inserting watermark information into
`the least significant bits of the pixel values. This results in
`an imperceptible mark whichcandetect lossy transforma-
`tions performed on the watermarked content. For secu-
`rity applications and copyright protection, robust
`watermarking techniques have been proposed. Here the
`technical challengeis to provide transparency and robust-
`ness which are conflicting requirements. Ideally, an effec-
`tive, robust watermarking scheme provides a mark that
`can only be removed whenthe original content is de-
`stroyed as well. The degree of robustness and distortion
`necessaryto alter the value ofthe original content can vary
`for different applications. Typically, manyofthe applica-
`tions for copyright protection involve relatively high
`quality original content and the imperceptibility criterion
`is critical for such applications. The authors in [15] and
`[16] were the first to describe that in order for a
`watermarking technique to be robust, the watermark
`should be embeddedin theperceptually significant portion
`ofthe data. Sometypical distortions orattacks thatdigital
`watermarking schemes are expected to survive include
`resampling, rescaling, compression, linear and nonlinear
`filtering, additive noise, A/D and D/A conversion, and
`transcoding. Applications for robust watermarking in-
`clude copyright protection where each copy gets a unique
`watermark (commonly referred to as a fingerprint) to
`identify the end-userso thattracing is possible for cases of
`illegal use; authentication, where the watermark can rep-
`resent a signature and copy control for digital recording
`devices. Within the class of robust watermarking tech-
`niques there are several different constraints on encoder
`and decoderdesign which depends ontheparticular ap-
`plication. The differences are discussed in detail later in
`this paper. Semifragile watermarking techniques differ-
`entiate between lossy transformationsthatare “informa-
`tion preserving” and lossy transformations which are
`“information altering.” Lossy transformations include
`any signal processing step that alters the original signal
`values andis not invertible. For example, in authentica-
`tion applications it maybe desirable to have a watermark
`
`34
`
`IEEE SIGNAL PROCESSING MAGAZINE
`
`JULY 2001
`
`
`
`

`

`
`
`that can distinguish between a lossy transformation
`such as compression whichdoesnotalter the integrity
`of the contentandanalteration which doesalter the in-
`tegrity, such as manipulating or replacing objects
`within the scene.
`Requirements and design ofwatermarking techniques
`are impacted bythe different types of contentin two ma-
`jor ways: imperceptibility and robustness requirements.
`Thefirst challenge is designing a watermark embedding
`algorithm whichprovides an imperceptible mark,thatis,
`one which does not noticeably degrade the original host
`signal. By taking advantage of psychovisual and psycho-
`auditory properties, we can design effective watermark-
`ing schemes which remain transparent under particular
`conditions [7], [8], [17]-[22]. Ideally, the marking algo-
`rithm should be adapted by using perceptual models ap-
`propriate for the different media types. The perceptual
`models used for representations of continuous tone im-
`ages are not appropriate for text or graphics. The other
`factor for designing watermarking schemes for multime-
`dia is the type of degradations that the watermark ts ex-
`pected to survive and system requirements for media
`specific applications. Forinstance, it may be desirable for
`a still image watermarking techniqueto beableto survive
`JPEG compression and photocopying while for some
`video watermarking applications, it may be important to
`do watermark embedding anddetectioninreal time on a
`compressed bit stream.
`In the next section we describe watermarking for dif-
`ferent media types including an overview of some sample
`algorithms proposedintheliterature. This is followed by
`a descriptionof a general framework for watermark em-
`bedding and watermark detection and decoding, outlin-
`ing someofthe differences for different applications. We
`then review some work on modeling the general
`watermarking problem and drawingparallels to commu-
`nication and information theory to help understand the
`fundamental properties and limitations of a watermark-
`ing system. This work is very useful for future algorithm
`design and helping to define open areas of research.
`Lastly, we review and summarize future directionsin this
`new andexciting area.
`
`Media Requirements
`Here weexplore the requirements for watermarking sys-
`temsdesignedfor different media types and review some
`of the algorithms covered in theliterature.
`
`Image Watermarking
`Many techniques have been developed for the
`watermarking of still
`image data. For grey-level or
`color-image watermarking, watermark embedding tech-
`niques are designedto insert the watermarkdirectly into
`the original image data, such as the luminance or color
`components or into some transformed version of the
`original data to take advantageofperceptual properties or
`
`robustness to particular signal manipulations. Require-
`ments for image watermarking include imperceptibility,
`robustness to commonsignal processing operations, and
`capacity. Commonsignal processing operations which
`the watermark should survive include compression (such
`as JPEG), filtering, rescaling, cropping, A/D and D/A
`conversion, geometric distortions, and additive noise.
`Capacity refers to the amount of information (or pay-
`load) that can be hidden in the host image and detected
`reliably under normal operating conditions. Many of the
`watermarking techniques are additive, where the water-
`marksignal is added directly to the host signal or trans-
`formed host signal. The watermark may be scaled
`appropriately to minimize noticeable distortions to the
`host. Perceptual models may be used to determine and
`adaptthe watermarkscale factor appropriately to the host
`data. The watermarkitselfis a function of the watermark
`information,a secret or public key and perhapsthe origi-
`nal host data. Some examples of watermark information
`includes a binary sequence representing a serial number
`or credit card number, a logo, a picture, or a signature.
`Manyofthe current watermarking techniques insert one
`bit of information over manypixels or transform coeffi-
`cients and useclassical detection schemes to recover the
`watermark information. These types of watermarking
`techniquesare usually referred to as spread-spectrum ap-
`proaches,due to their similarity to spread-spectrum com-
`munication systems. Forstill image watermarking,
`watermark embeddingis applieddirectly to the pixelval-
`ues in the spatial domainor to transform coefficients in a
`transform domain such as the discrete cosine transform
`(DCT) or discrete wavelet transform (DWT). Water-
`mark detection usually consists of some preprocessing
`step (which mayinclude removalofthe original hostsig-
`nal ifit is available for detection) followed by a correlation
`operator. Moredetails on watermark embedding and de-
`tection appearlater. Spatial-domain watermarking tech-
`niques for image data include [23]-[28]. Some of the
`earliest techniques [23], [29], [28] embed m-sequences
`into the least significant bit (LSB) of the data to provide
`an effective transparent embedding technique.
`M-sequences are chosen due to their good correlation
`properties so that a correlation operation can be used for
`watermark detection. Furthermore, these techniques are
`computationally inexpensive to implement. Such a
`schemewasfirst proposed in [23] and extendedto two di-
`mensions in [29]. In [28]
`the authors reshape the
`m-sequence into two-dimensional watermark blocks
`which are addedanddetected on a block-by-blockbasis.
`The block-based method, referred to as variable-w
`two-dimensional watermark (VW2D) is shownto be ro-
`bust to JPEG compression. This technique has also been
`shownto be an effective fragile watermarking scheme
`which can detectimagealterations on a block basis [30].
`Otherearly work [31] suggests using check sums for LSB
`watermark embedding.
`
`JULY 2001
`
`IEEE SIGNAL PROCESSING MAGAZINE
`
`35
`
`
`
`

`

`let ofmidrangefrequencies are slightly altered to encode a
`binary sequence. This is a reasonable heuristic
`watermarking approach since watermarksinserted in the
`high frequencies are vulnerable to attack whereas the low
`frequency componentsare perceptually significant and
`sensitive to alterations. One of the most influential
`watermarking works [15], [16] wasfirst to describe how
`spread spectrum principles borrowed from communica-
`tion theory can be used in the context of watermarking.
`The published results show that the techniqueis very ef-
`fective both in terms of image quality and robustness to
`signal processing and attempts to remove the watermark.
`The techniqueis motivated by both perceptual transpar-
`ency and watermark robustness. One of the significant
`contributions in this workis the realization that the wa-
`termark should beinserted in the perceptually significant
`portion ofthe imagein orderforit to be robustto attack.
`A DCT is performed on the whole imageandthe water-
`mark is inserted in a predetermined range of low fre-
`quency components minus the DC component. The
`watermark consists of a sequence ofreal numbers gener-
`ated from a Gaussian distribution which is added to the
`DCT-coefficients. The watermark signal is scaled accord-
`ing to the signal strength ofthe particular frequency com-
`ponent. Thisis a reasonable and simple way to introduce
`sometype ofperceptual weightinginto the watermarking
`scheme. The watermark embedding algorithm could be
`described as
`
`
`
`Several spatial-domain watermarking techniques for
`images are proposed in [25]. One technique consists of
`embeddinga texture-basedwatermark into a portion of
`the imagewith similar texture. Theidea hereis that due to
`the similarity in texture,it will be difficult to perceive the
`watermark. The watermarkis detected using a correlation
`detector. Another technique described as the patchwork
`methoddivides the image into two subsets A and B where
`the brightness of one subset is incremented by a small
`amountandthebrightnessofthe otherset is decremented
`by the same amount. The incremental brightnesslevelis
`chosen sothat the changein intensity remains impercep-
`tible. The location of the subsets is secret and assuming
`certain properties for image data, the watermarkis easily
`located by averagingthe difference betweenthevaluesin
`the two subsets. It is assumed that, on average, without
`the watermark,this value will go to zero for imagedata.
`In the example wherethe pixels in Set A are incremented
`by one andthe pixels in set B are decremented by one,
`with N locationsin the set, the expected value of the sum
`of differences between the sets is given by 2N. For
`nonwatermarked data, this value should go to zero. A
`variation of this approach is described in.[27], where
`more information can be inserted in the host signal. An-
`other spatial-domain technique is proposed in [32],
`where the blue componentof an image in RGB formatis
`watermarked to ensure robustness while remaining fairly
`insensitive to human visual system (HVS) factors.
`Transform domain watermarkingis useful for taking
`advantage of perceptual criteria in the embedding pro-
`cess, for designing watermarking techniques which are
`whereS is the original host signal, X is the watermarked
`robust to common compression techniques, and for di-
`signal, and W is the watermark consisting of a random,
`rect watermark embedding of compressedbit streams. A
`Gaussian distributed sequence. o is a scaling factor which
`common transform framework for images is the
`the authors suggestto set to 0.1 to provide a goodtrade-
`block-based DCT which is a fundamental building block
`offbetween imperceptibility and robustness. Referring to
`of current image coding standards such as JPEG and
`Fig. 1 for a block diagram of a general watermarking sys-
`video coding standards such as the MPEGvideo coders
`tem, the secret key is used to generate the random se-
`[33] and the ITU H.26xfamily ofcodecs. One ofthefirst
`quence W in this case. Also note that this particular
`block-based DCT watermarkingtechniqueis proposedin
`algorithm addressesthe case ofwatermark detection where
`[34]. The DCT is performed on 8x8 blocks of data, a
`you wouldlike to detect whether a particular watermark
`pseudorandomsubsetofthe blocks are chosen andatrip-
`is Or is not presentin the host signal at the receiver. The
`
`
`
`
`
`X=S(1+0W)
`
`(1)
`
`
`
`A 1. Block diagram of a watermarking system.
`
`36
`
`IEEE SIGNAL PROCESSING MAGAZINE
`
`JULY 2001
`
`
`

`

`watermarkdetector for this scheme [15] is
`described by the similarity measure
`sim(W,W)=
`vW-W
`
`(2)
`
`
`
`
`where W is the extracted watermark from
`the received, possibly distorted signal Y.
`The authors show that the similarity mea-
`sure is also normally distributed so that a
`highsimilarity value is extremely unlikely
`for W #W.Otherpostfiltering operations
`could be performed to undo possible dis-
`tortions, improve performance, and get a
`better similarity measure. Moredetails on
`improving detection results can be found
`later.
`A variationonthis ideais variable length
`DCT-based watermarking [35], where the
`DCT coefficients are sorted by magnitude
`and only the w largest coefficients are
`marked that correspondto a user specified
`percentofthe total energy. This allows the
`user to trade off imperceptibility and ro-
`bustness to attack. Other DCT-based
`watermarking schemes use more elaborate
`models of the human visual system to in-
`corporate an image adaptive watermark of
`maximum strength subject to the imper-
`ceptibility criterion [17], [7], [8]. Two im-
`age-adaptive watermarking schemes are
`described in [19] and [7], which are based
`on a block-based DCT framework and
`wavelet framework. The perceptual models
`used here can be described in termsof three
`different properties of the human visual
`system that have been studied in the con-
`image-adaptive quantization table [36]. A similar model
`text of image coding: frequency sensitivity, luminance
`was developed for wavelet-based compression using only
`sensitivity, and contrast masking [36]. Frequencysensi-
`frequency sensitivity to derive perceptual weights for
`tivity describes the human eye’s sensitivity to sine wave
`each of the subbands [37]. This model was used for a
`gratingsat various frequencies. This componentonly de-
`wavelet-based watermarking scheme [19], [7]. Unlike
`pends on the modulationtransfer function (MTF) of the
`compression, where the amountof perceptual informa-
`eye andis independentofthe image data. Luminance sen-
`tion that can be incorporatedinto the encoderis limited
`sitivity measurestheeffect of the detectability threshold
`to the amountof side information that is necessary to
`of noise on a constant background. For the human visual
`transmit this information to the decoder, all of the per-
`system,this is a nonlinear function and dependson local
`ceptual information can be utilized in a watermarking
`image characteristics. Contrast masking refers to the
`scheme. For instance, in JPEG, we are limited to one
`detectability of one signal in the presence of anothersig-
`quantization matrix for the entire image which cannot
`nal and the effect is strongest when both signals are ofthe
`take full advantage of local visual threshold characteris-
`samespatial frequency, orientation, and location. A com-
`tics. The image dependent masking thresholds are used to
`bination of the three componentsresults injust noticeable
`determine the location and maximum strength ofthe wa-
`distortion (JND) thresholds for the entire image. These
`termark signal that can be tolerated in every location of
`models werefirst developed to design more efficient im-
`the host image underthe constraint of imperceptibility at
`age compression schemes than -waveform techniques
`somespecified viewing condition. Examples of the im-
`alone could provide. This modelwasderived for the base-
`age-adaptive watermarks described in [7] areillustrated
`line mode of JPEG and showedasignificant improve-
`mentin compression performance whenused to derive an
`in Fig. 2.
`
`2. Watermarked images(first, third row) and corresponding image-adaptive wa-
`termarks using perceptual models (second, fourth row).
`
`JULY 2001
`
`IEEE SIGNAL PROCESSING MAGAZINE
`
`37
`
`
`
`te FEDERAL OFFENSE PUNISHABLE BY FINES
`$10,000,
`IMPRISONMENT FOR 20 YEARS, OR
`DEATH IF SUCH A THEFT CAUSES AN ACCIDENT
`pamem
`SWAGTING IN LOSS OF LIFE,
`THE FEDERAL BURE
`
`SSINVESTIGATION WILL BE NOTIFIED.
`
`
`

`

`
`
`
`
`A 3. Watermark examplefor different viewing distances.
`
`Note how the watermark structureis similar to the lo-
`image properties. Fig. 3 illustrates several water-
`cal
`marked images and the corresponding watermarkto the
`right of the watermarked image. The three examples
`show the watermarked imagefor different viewing condi-
`tions where the top image correspondsto a viewing dis-
`tance of four times the image height, the middle image
`corresponds to a viewing distance of two timesthe image
`height and the bottom image corresponds to a viewing
`distance of one times the image height. Modifying the
`viewing conditions of the watermark embedding algo-
`rithm allows for a tradeoff between imperceptibility and
`robustnessto certain types of attacks. These examples are
`forillustrative purposes and the viewing conditions here
`are not based on viewing printed images.
`Two DCT-based approaches were described in [38]
`and [39] where watermark detection does not require the
`original image. The methodin [40] is an extensionof the
`method proposed in [19] and [7] to the case where the
`original host signalis not available for watermark detec-
`tion. This is an important feature for some applications
`suchas authentication and will be covered in moredetail
`later. Another block-based frequency domain technique
`describedin [41] is based oninserting a watermark into
`
`the phase componentsof the image data using the same
`motivation as in [16], that for the watermarkto be robust
`to attack, it must be embedded in the perceptually signifi-
`cant portion of the data. It has beenestablished that for
`image data, the phase information is perceptually more
`important than the magnitude data. Other novel ap-
`proaches for watermarking image data include
`fractal-based approaches [42], [43] and geometric fea-
`ture based watermarking [44]. In [44], salient points in
`an image are found and warpedaccordingto a dense line
`pattern representing the watermark and generated ran-
`domly. Detection consists of determining whethera sig-
`nificantly large numberofpoints are within the vicinity of
`the line patterns.
`The type of distortions or attacks that image
`watermarking techniquesare designedto survivefall into
`two broad categories—noise type distortions like com-
`pression and geometrical distortions which cause loss of
`synchronization for detection, such as resamplingandro-
`tations. Watermarking schemesfor tamper detection and
`tamper estimation to be able to differentiate between
`lossy attacks whichalter the information andlossy attacks
`which do notalter the information [45]-[47] have also
`been proposed.
`
`Document Watermarking
`Much ofthe early work on recognizing the potential
`problemswith intellectual property rights of digital con-
`tent and addressingthese issues with early watermark-
`ing techniques was in the area of document water-
`marking [48]-[50]. These techniques were devised for
`watermarkingelectronic versions oftext documents which
`are in some formatted version suchas postscript or PDF.
`Mostofthis workis based on hiding the watermarkinfor-
`mationinto the layout and formatting of the document
`directly. In [48]-[50], the authors develop document
`watermarking schemesbasedonline shifts, wordshifts as
`well as slight modifications to the characters. These tech-
`niques are focused on watermarking the binary-valued
`text regions ofa document. Watermark detectionconsists
`ofpostprocessing steps to try to remove noise andcorrect
`for skew. These techniques are quite effective against
`some commonattacks such as multigenerational photo-
`copying. The authorspointoutthat optical characterrec-
`ognition can removethe layout information and,for such
`schemes, remove the watermark information. An open
`area of research remains in how to formulate formatdevi-
`ationsin a perceptual framework.Fig.4 illustrates an ex-
`ample from [49] of word shift coding; (a) shows where
`the space has been added before the word “for,” and (b)
`contains the unwatermarked and watermarked versions
`in their naturalstate to illustrate that the wordshift is not
`noticeable. Fig. 5 shows an example from [49] on charac-
`ter alteration for watermark embedding.
`
`38
`
`IEEE SIGNAL PROCESSING MAGAZINE
`
`JULY 2001
`
`
`
`
`

`

`
`
`Graphics Watermarking
`There has been some work oneffective watermarking of
`graphics, motivated in part by such standards as
`MPEG-4. In [51], the authors address watermarking
`three-dimensional polygonal models. The work in [52]
`addresses the watermarking of facial animation parame-
`ters as defined by the MPEG-4 standard. The watermark
`is embeddeddirectly into the parameters and can be ex-
`tracted from the watermarked parameters directly or
`from video sequences rendered using the parameterbit
`stream where the parameters are estimated using a
`model-based approach. One bit of watermark informa-
`tion is embedded ina blockoffacial animation parameter
`(FAP) data using a pseudonoise sequence that is gener-
`ated fromthesecret key. The authorslimit the amount of
`deviation the watermark signal has on the FAPs empiri-
`cally to minimizevisible distortion. For instance, global
`FAPslike head rotationare limited to deviate by 1% of
`their dynamic range while local FAPssuch aslip motionis
`limited to 3%. Watermark detection can be donedirectly
`on the watermarked FAPs througha traditional correla-
`tion detector. The authors demonstrate that they are able
`to recover the watermark information withouterror us-
`ing both the FAPsdirectly or by estimating them from a
`rendered sequence. They also show that their methodis
`robust to moderate compression using MPEG-2. In gen-
`eral, watermarking of graphics data remains aninterest-
`ing research topic since our understanding of perceptual
`models in this domainis notyet fully recognized.
`
`detection in the drive and watermark detection within the
`application (within the MPEG decoder). The drive-based
`solutionhas the advantage that as long as the watermark
`exists, pirated content cannotleave the drive in playback
`modeorrecording mode. There is some added complex-
`ity with detection inthe drive versus detection in the ap-
`plication, for example, a partial decode of the MPEG bit
`stream is necessary. Watermark detection in the MPEG
`decoder is not as secure as the drive-based solution and
`some addedfeatures that have been suggested for detec-
`tion in the application include a protocol to recognize a
`compliant device, a bidirectional link with authentica-
`tion, encryption and data integrity, and a protocol be-
`tweensource and sink whichinformsthe drive whetherto
`stop transmitting data. Advantages of application-based
`detection are the ability to provide a more complex detec-
`tor and the flexibility of extending the scheme to other
`data types. The other unique requirement for DVD appli-
`cations is copy generation management,thatis, the abil-
`ity to detect the copy once state and changeit to a copy no
`morestate after