(19) United States
`(12) Patent Application Publication (10) Pub. No.: US 2008/0235200 A1
`Washington et al.
`(43) Pub. Date:
`Sep. 25, 2008
`
`US 20080235200A1
`
`(54) SYSTEM AND METHOD FOR IDENTIFYING
`CONTENT
`
`(75) Inventors:
`
`Richard G- Washington, Marble
`Falls, TX (US); Clifford L. Hall,
`Austin’ TX (Us)
`
`Cones Ondence Address_
`p
`'
`BAKER BOTTS L‘L‘P‘
`2001 ROSS AVENUE’ SUITE 600
`DALLAS’ TX 75201-2980 (Us)
`
`(73) Assignee;
`
`Ripcode, Inc” Richardson, TX
`(US)
`
`_
`(21) Appl' NO"
`
`11/688’93 6
`
`(22) Filed:
`
`Mar. 21, 2007
`
`Publication Classi?cation
`
`(51) Int. Cl.
`(2006.01)
`G06F 7/00
`(52) US. Cl. ................................... .. 707/4; 707/E17 .109
`
`ABSTRACT
`(57)
`A method for processing media ?les includes receiving a ?rst
`signature. The ?rst signature describes a ?rst characteristic of
`a ?rst media ?le. The method also includes determining
`Whether the ?rst signature matches any of a ?rst plurality of
`stored signatures and, in response to determining that the ?rst
`signature matches one or more of the ?rst plurality of stored
`signatures, requesting a second signature based on the ?rst
`media ?le. The second signature describes a second charac
`teristic of the ?rst media ?le. The method also includes deter
`mining Whether the second signature matches any of a second
`plurality of stored signatures and, in response to determining
`that the second signature matches one or more of the second
`plurality of stored signatures, initiating a remedial action
`associated With the ?rst media ?le.
`
`32
`
`Si 18g
`
`TRANSCODEHS
`
`18f
`
`l__l
`18a
`18d
`IT
`180
`40\El WUNSGC
`¢'J\36b
`m5 “\36a
`SIGNATURE
`I FL‘
`_
`SERVER
`|-_-;13<\34c Q
`LT‘ 34b
`
`18a
`
`CONTENT
`MANAGEMENT
`SERVER
`
`TRANSCODERS
`
`iESTAEéJJ/EIDL
`STORE
`
`SUBMITTED
`CONTENT
`STORE
`
`22
`
`26
`
`PROTECTED
`SIGNATURE
`STORE
`
`Page 1 of 22
`
`

`

`Patent Application Publication
`
`Sep. 25, 2008 Sheet 1 of 5
`
`US 2008/0235200 A1
`
`Qm_._.om_.5mE
`
`.__<zm_>_3_._H
`
`mEo._.m
`
`m_E:<zu_m
`
`Eimm
`
`E5Eoxn_
`
`mEE<za_m
`
`#55
`
`8E_>_m5m
`
`EH28
`
`#55
`
`N.U~,_..~
`
`
`
`m$o8mz<E:,msma<z<s_
`
`Ezmm
`
`EH28
`
`DE
`
`
`
`Page 2 of 22
`
`Page 2 of 22
`
`
`
`
`

`

`Patent Application Publication
`
`Sep. 25, 2008 Sheet 2 0f 5
`
`US 2008/0235200 A1
`
`Page 3 of 22
`
`

`

`Patent Application Publication
`
`Sep. 25, 2008 Sheet 3 of 5
`
`US 2008/0235200 A1
`
`
`
`
`
`M:flames.m_o<uEm_._.z_vEo>>Ez
`
`mm.
`
`Sm
`
`m_momSNamNew
`
`
`
`
`
`
`
`
`
`wma__%m._p_m#_E<zo_wmE:<zw_wumaxmmgéowz_m$ooE...Q9gzoamEm:am:
`-5:
`
`
`58o_>_5:85.3:82In
`
`«m
`
`NewSNSNmomENNmmSN
`
`8
`
`M...0N5~
`
`3m
`
`
`
`mommmooxmEm_8mz<E
`
`fl
`
`
`
`
`
`
`
`zo:<s__>5mzo_H<H2m_>_wm_mzoE...N:<:8oz_:omEmwz_$_3mozamsazo_5<.Em
`
`
`
`
`
`
`
`
`Edam_>_<moEm_:5<Ez8z<_mm:<a4<mE_>_Efl<om><_”_w
`
`V.U~h~
`
`Page 4 of 22
`
`Page 4 of 22
`
`

`

`Patent Application Publication
`
`Sep. 25, 2008 Sheet 4 0f 5
`
`US 2008/0235200 A1
`
`SIGNATURE
`sERvER
`
`I
`I
`
`-
`I
`
`I
`I
`
`NETWORK
`INTERFACE
`
`MODULE
`
`\ F__3___
`
`l FIRST- LEVEL
`' COMPARISON
`MODULE
`
`MAPPING
`MODULE
`
`MAPPING
`MODULE
`
`MAPPING
`MODULE
`
`f
`
`340
`
`[316
`
`—_
`
`Vi
`
`r3183
`
`318b
`\314 I
`
`SECOND-LEVEL
`COMPARISON
`MODULE \308
`
`MODULE \310
`
`E
`
`FIG. 5
`
`Page 5 of 22
`
`

`

`Patent Application Publication
`
`Sep. 25, 2008 Sheet 5 0f 5
`
`US 2008/0235200 A1
`
`@
`
`CONTENT MANAGEMENT SERvER
`(CMS) RECEIvES UPLOAD REQUEST
`
`4OO\
`
`402\
`
`CONTENT MANAGEMENT SERvER TRANSMITS
`SUBMHTED CONTENT FILE T0 TRANSCODER
`
`404\ TRANSCODER DECODES SUBMITTED CONTENT FILE
`
`4O6\
`
`TRANSCODER GENERATES FIRST-LEVEL
`SIGNATURES BASED ON SUBMITTED CONTENT FILE
`
`408
`
`TRANSCODER TRANSMITS FIRST-LEVEL
`SIGNATURES TO SIGNATURE SERVER
`
`DO
`FIRST-LEVEL SIGNATURES
`MATCH?
`
`412/
`
`SIGNATURE SERvER IDENTIFIES
`MATCHING PROTECTED CONTENT FILES
`
`41 4/
`
`SIGNATURE SERVER RETRIEvES SECOND
`LEvEL SIGNATURES ASSOCIATED WITH
`IDENTIFIED PROTECIED CONTENT FILES
`
`416/
`
`SIGNATURE SERvER REQUESTS SECOND-LEVEL
`SIGNATURE FOR SUBMITTED CONTENT FILE
`L
`TRANSCODER GENERATES AND TRANSMITS SECOND-
`418/ LEvEL SIGNATURE FOR SUBMITTED CONTENT FILE
`
`SECOND-LEVEL SIGNATURES
`
`422
`
`SIGNATURE SERvER INITIATES REMEDIAL ACTION
`
`6 (
`
`FIN‘IVSH )
`
`=7
`SIGNATURE SERVER
`NOTIFIES
`TRANSCDDER THAT \
`REQUEST CAN
`424
`BE COMPLETED
`@
`TRANSCDDER
`ENCODES CONTENT
`FROM SUBMITTED \426
`CONTENT FILE
`I
`TRANSCDDER
`STORES SUBMITTED
`CONTENT FILE IN \428
`CONTENT STORE
`
`Page 6 of 22
`
`

`

`US 2008/0235200 A1
`
`Sep.25,2008
`
`SYSTEM AND METHOD FOR IDENTIFYING
`CONTENT
`
`TECHNICAL FIELD OF THE INVENTION
`
`[0001] This invention relates in general to multimedia-con
`tent delivery systems, and more particularly, to a method and
`system for identifying protected content on a content delivery
`system.
`
`BACKGROUND OF THE INVENTION
`
`[0002] The rapid growth in Internet usage has given users
`access to a Wide range of sources for text, audio, video, and
`multimedia content provided in many different formats. At
`the same time, the costs of producing content have plum
`meted, alloWing end-users to produce and distribute a sub
`stantial amount of media content. As a result, Websites that
`offer free content-hosting services, such as YouTube and
`MySpace, have become popular both With amateur content
`providers and With an ever-groWing audience.
`[0003] The exponential groWth in the use of content-shar
`ing Websites and netWorks has made it increasingly di?icult
`to monitor user activity. The distribution of copyrighted and
`otherWise protected content has become a common problem
`for such Websites, as users mix protected content in With the
`user-generated content intended to be distributed on such
`Websites. Similarly, many such Websites and netWorks pro
`hibit the distribution of pornographic, explicit, or in?amma
`tory content. In fact, the operators of content-sharing sites and
`netWorks may face laWsuits from copyright holders and com
`plaints from offended users if protected and/or prohibited
`content is not identi?ed and removed. Nonetheless, policing
`the distribution of such ?les can be di?icult, time-consuming,
`and expensive. Given the exponential groWth in the amount of
`user-uploaded content available on such content-sharing
`sites, traditional approaches to applying audio/video content
`detection techniques are no longer effective. Implementation
`of process-intensive approaches that require in-depth analy
`sis of the content and/or transmission of massive amounts of
`signature data Would result in system con?gurations that are
`economically unviable due to cost and complexity.
`
`SUMMARY OF THE INVENTION
`
`[0004] In accordance With the present invention, the disad
`vantages and problems associated With content delivery sys
`tems have been substantially reduced or eliminated. In par
`ticular, a content-delivery system is disclosed that provides
`?exible techniques for identifying protected content.
`[0005] In accordance With one embodiment of the present
`invention, a method for processing media ?les includes
`receiving a ?rst signature that describes a ?rst characteristic
`of a ?rst media ?le. The method also includes determining
`Whether the ?rst signature matches any of a ?rst plurality of
`stored signatures and, in response to determining that the ?rst
`signature matches one or more of the ?rst plurality of stored
`signatures, requesting a second signature based on the ?rst
`media ?le. The second signature describes a second charac
`teristic of the ?rst media ?le. The method also includes deter
`mining Whether the second signature matches any of a second
`plurality of stored signatures and, in response to determining
`that the second signature matches one or more of the second
`plurality of stored signatures, initiating a remedial action
`associated With the ?rst media ?le.
`
`[0006] Technical advantages of certain embodiments of the
`present invention include the ability to identify content that is
`protected, prohibited, and/or otherWise Worthy of special pro
`cessing on a content-delivery system. Additionally, particular
`embodiments may provide for the optimiZed use of time and
`processing resources in identifying the relevant content. Par
`ticular embodiments of the content delivery system may also
`include ?exible and customizable techniques for addressing
`the use of protected or prohibited content that may limit the
`need for human involvement. Other technical advantages of
`the present invention Will be readily apparent to one skilled in
`the art from the folloWing ?gures, descriptions, and claims.
`Moreover, While speci?c advantages have been enumerated
`above, various embodiments may include all, some, or none
`of the enumerated advantages.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`[0007] For a more complete understanding of the present
`invention and its advantages, reference is noW made to the
`folloWing description, taken in conjunction With the accom
`panying draWings, in Which:
`[0008] FIG. 1 illustrates a content-delivery system capable
`of identifying and managing the use of protected content;
`[0009] FIG. 2 is a conceptual illustration of a multi-level
`signature analysis process that may be utiliZed by particular
`embodiments of the content-delivery system;
`[0010] FIG. 3 illustrates in further detail a transcoder that
`may be included in particular embodiments of the content
`delivery system;
`[0011] FIG. 4 is a ?owchart illustrating an example opera
`tion of the transcoder in generating a content signature in
`accordance With a particular embodiment;
`[0012] FIG. 5 illustrates in further detail a signature server
`that may be included in particular embodiments of the con
`tent-delivery system; and
`[0013] FIG. 6 is a ?oWchart illustrating certain aspects of an
`example operation of the content-delivery system shoWn in
`FIG. 1.
`
`DETAILED DESCRIPTION OF THE INVENTION
`
`[0014] FIG. 1 illustrates a particular embodiment of a sys
`tem 10 for delivering media content to clients 12a-c from
`submitted content store 22. System 10 includes clients 12a-c,
`content sources 14a-c, content management server 16,
`transcoders 18a-d and e-g; signature server 20, submitted
`content store 22, protected signature store 24, and protected
`thumbnail store 26, and a netWork 28. Signature server 20
`receives submitted content ?les 30 from users through con
`tent management server 16 and transcoders 18a-d and ana
`lyZes submitted content ?les 30 to determine their contents.
`By minimizing the time and system resources utiliZed in
`analyZing submitted content ?les 30, particular embodiments
`of signature server 20 provide e?icient techniques for identi
`fying and managing protected content.
`[0015] In general, Within system 10, content sources 1411-0
`and clients 12a-c connect to content management server 16
`through netWork 28. Content management server 1 6 connects
`to submitted content store 22 and manages access to submit
`ted content store 22. Additionally, content management
`server 16 connects to transcoders 18a-d and forWards submit
`ted content ?les 30 to transcoders 18a-d for transcoding.
`Transcoders 18a-d are coupled to signature server 20 and
`transmit content signatures for submitted content ?les 30 to
`
`Page 7 of 22
`
`

`

`US 2008/0235200 A1
`
`Sep.25,2008
`
`signature server 20 for analysis. Similarly, transcoders 18e-g
`are also coupled to signature server 20 and transmit content
`signatures for protected content ?les 32 to signature server
`20. Based on a comparison of these content signatures, sig
`nature server 20 determines whether submitted content ?les
`30 include protected content and, if so, initiates an appropri
`ate remedial action, such as refusing to upload submitted
`content ?les 30 or notifying a human operator 42.
`[0016] More speci?cally, clients 12a-c display content
`retrieved from submitted content store 22 to users, such as
`subscribers to a web site or content-sharing service. Clients
`12a-c may each represent any type of device appropriate to
`display one or more type of content utiliZed in system 10.
`Examples of clients 12a-c may include, but are not limited to,
`computers, video-enabled telephones, media players (such as
`audio- and/or video-capable iPods), televisions, and portable
`communication devices. In general, however, clients 12a-c
`may include any appropriate combination of hardware, soft
`ware, and/or encoded logic suitable to provide the described
`functionality. Clients 12a-c may couple to network 28
`through a dedicated connection, wired or wireless, or may
`connect to network 28 as needed to access media content. For
`example, clients 12, such as portable media players, may
`connect temporarily to network 28 to download submitted
`content ?les 30 but then disconnect before displaying content
`from the submitted content ?les 30. Although FIG. 1 illus
`trates, for purposes of example, a particular number and type
`of clients 12, alternative embodiments of system 10 may
`include any appropriate number and suitable type of clients
`12.
`[0017] Content sources 14 provide media content, such as
`submitted content ?les 30, to system 10. Content from con
`tent sources 14 is uploaded through network 28 to submitted
`content store 22 and made available for display by clients 12.
`For example, media content such as video and audio ?les may
`be entered into a content source 14a, such as a computer, and
`sent over network 28 to be stored in submitted content store
`22 for subsequent access by users through clients 12. Content
`sources 14 may include any form of media generation and/or
`capture devices, such as personal computers, video cameras,
`camera-enabled telephones, audio recorders, and/ or any other
`device capable of generating, capturing, or storing media
`content.
`[0018] Although FIG. 1 shows only content sources 14a-c,
`it will be understood that system 10 can accommodate very
`large numbers of content sources 14 and large numbers of
`submitted content ?les 30. Although the description below
`focuses on embodiments of system 10 in which content origi
`nates from content sources 14, system 10 may utiliZe content
`that originates at and/or is provided to content store 12, or
`system 10 generally, in any appropriate manner. For example,
`content store 12 may include, or be con?gured to accept,
`detachable storage media, such as compact discs (CDs) or
`digital video discs (DVDs). In such embodiments, content
`may be introduced into system 10 as a result of detachable
`storage media containing submitted content ?les 30 being
`coupled to or accessed by content store 12. More generally,
`however, content may be provided to system 10 in any appro
`priate manner.
`[0019] Network 28 represents any form of communication
`network supporting circuit-switched, packet-based, and/or
`any other suitable type of communication. Although shown in
`FIG. 1 as a single element, communication network may
`represent one or more separate networks, including all or
`
`parts of various different networks that are separated and
`serve different groups of clients 12. Network 28 may include
`routers, hubs, switches, gateways, call controllers, and/ or any
`other suitable components in any suitable form or arrange
`ment. In general, network 28 may comprise any combination
`of public or private communication equipment such as ele
`ments of the public-switched telephone network (PSTN), a
`global computer network such as the intemet, a local area
`network (LAN), a wide-area network (WAN), or other appro
`priate communication equipment.
`[0020] Content management server 16 processes requests
`from content sources 14 to upload submitted content ?les 30
`to submitted content store 22 and from clients 12 to download
`submitted content ?les 30 from submitted content store 22.
`Content management server 16 may additionally authenticate
`users, execute content search requests, and/ or otherwise
`facilitate interaction between users and the content-provision
`services offered by system 10. In particular embodiments,
`content management server 16 may be responsible for initi
`ating transcoding and/or signature analysis of submitted con
`tent ?les 30 when uploaded by content sources 14.
`[0021] Transcoders 18 convert or modify submitted content
`?les 30 to a type and/or format appropriate for transmission
`to, storage on, and/ or display by a particular client 12.
`Transcoders 18 may modify the media content by translating,
`transcoding, transrating, encoding, rendering, and/or pro
`cessing or otherwise modifying the relevant content to the
`requirements of a particular client 12. As speci?c examples,
`transcoders 18 may modify submitted content ?les 30 by
`changing the codec, bit-rate, associated communication pro
`tocol, type of storage medium, compression, and/or digital
`rights management information of the relevant content. Par
`ticular embodiments of system 10 may only support ?xed
`input and/or output formats and, as a result, may not include
`any transcoders 18. Additionally, in particular embodiments,
`transcoding may be done independently from the signature
`matching performed by signature server 24. As a result,
`transcoding may be performed outside the signature-analysis
`datapath.
`[0022] Additionally, FIG. 1 illustrates a particular embodi
`ment of system 10 in which transcoders 18a-d are responsible
`for generating at least a portion of the content signatures used
`by signature server 20 to compare submitted content ?les 30
`to protected content ?les 32. In the illustrated example, after
`decoding submitted content ?les 30 and before re-encoding
`submitted content ?les 30, transcoders 18a-d generate one or
`more lightweight content signatures (shown in FIG. 1 as
`?rst-level submitted signatures 34a-c) that can be quickly
`generated using a minimal amount of processing resources.
`Transcoders 18a-d may then transmit these lightweight sig
`natures to signature server 20 to be used as in the ?rst level of
`multi-level signature analysis.
`[0023] Furthermore, in particular embodiments, one or
`more transcoders 18 may be responsible for generating con
`tent signatures for protected content ?les 32 received by sys
`tem 10. Such transcoders 18 may be dedicated solely to
`signature generation for protected content ?les 32 or may be
`con?gured to process both submitted content ?les 30 and
`protected content ?les 32 as needed. In the illustrated embodi
`ment, system 10 includes transcoders 18e-g that are dedicated
`to generating content signatures for protected content ?les 32.
`More speci?cally, in the illustrated embodiment, transcoders
`18e-g generate lightweight content signatures for protected
`content ?les 32 that utiliZe the same algorithms as those used
`
`Page 8 of 22
`
`

`

`US 2008/0235200 A1
`
`Sep.25,2008
`
`by transcoders 18a-d. These lightweight content signatures
`are shown in FIG. 1 as ?rst-level protected signatures 3611-0.
`[0024] Signature server 20 compares content signatures
`generated from submitted content ?les 30, such as ?rst-level
`submitted signatures 34, to content signatures generated from
`protected content ?les 32, such as ?rst-level protected signa
`tures 36, to determine whether submitted content ?les 30
`represent or include protected content. If signature server 20
`determines that a suf?ciently high level of similarity exists
`between a particular submitted content ?le 30 and one or
`more protected content ?les 32, signature server 20 may
`initiate an appropriate remedial action, such as refusing an
`upload or download request, or notifying a supervisor 42, to
`prevent the submitted content ?le 30 from being used and/or
`misused on system 10. In particular embodiments, signature
`server 20 may also be responsible for maintaining informa
`tion describing protected content ?les 32 in protected signa
`ture store 24 and protected thumbnail store 26. Additionally,
`as described further below, signature server 20 may utiliZe
`multi-level signature analysis techniques to provide more
`e?icient use of the processing resources available to system
`10.
`[0025] In general, content management server 16,
`transcoders 18, and signature server 20 may each represent
`any appropriate combination of hardware, software, and/or
`encoded logic suitable to provide the described functional
`ities. In particular embodiments, each of content management
`server 16, transcoders 18, and signature server 20 represents
`a physically separate server programmed to operate as
`described herein. Alternatively, in particular embodiments,
`any of content management server 16, transcoders 18, and
`signature server 20 may share or represent the same physical
`components. For example, in particular embodiments, sys
`tem 10 may include a server that houses a plurality of digital
`signal processor (DSP) groups that are collectively capable of
`providing the functionality described for both transcoders 18
`and signature server 20. As a result, transcoders 18 and sig
`nature server 20 may be housed in a single physical compo
`nent. More generally, however, the functionality provided by
`content management server 16, transcoders 18, and signature
`server 20 may in particular embodiments be divided among
`the various physical components of system 10 in any appro
`priate manner.
`[0026] Submitted content store 22 stores submitted content
`?les 30 transmitted to submitted content store 22 by transcod
`ers 18, while protected signature store 24 and protected
`thumbnail store 26 store information relating to protected
`content ?les 32, as described further below. Submitted con
`tent store 22, protected signature store 24, and protected
`thumbnail store 26 may represent or include any appropriate
`type of memory devices. Moreover, stores 22, 24, and 26 may
`comprise any collection and arrangement of volatile or non
`volatile, local or remote devices suitable for storing data, such
`as for example random access memory (RAM) devices, read
`only memory (ROM) devices, magnetic storage devices, opti
`cal storage devices, or any other suitable data storage devices.
`In particular embodiments, submitted content store 22 repre
`sents a storage area network (SAN) to which submitted con
`tent ?les 30 are uploaded. Such a SAN may receive and store,
`for example, video and sound ?les from a plurality of differ
`ent sources, the ?les having a variety of different formats and
`characteristics. Additionally, content thumbnails are only one
`example of the types of content signatures that may be uti
`liZed in particular embodiments of system 10. As a result, in
`
`particular embodiments, protected thumbnail store 26 may be
`replaced by storage for alternative types of signatures or
`omitted altogether.
`[0027] Submitted content ?les 30 represent media content
`submitted by users for storage on content provision system
`10, while protected content ?les 32 represent media content
`used by content provision system 10 to identify protected
`content and/ or other content of interest. In general, submitted
`content ?les 30 and protected content ?les 32 may represent
`media content structured in any appropriate manner.
`Examples of content ?les 30 and 32 include Moving Picture
`Experts Group (MPEG), Windows Media Video (WMV),
`Audio Video Interleave (AVI), and Quicktime video ?les;
`audio content such as Waveform audio (WAV), MPEG-l
`Audio Layer 3 (MP3), and/or Windows Media Audio (WMA)
`?les; image data such as Joint Photographic Experts Group
`(JPEG) or Tagged Image File Format (TIFF) ?les; and/or
`content of any other appropriate type or format. For example,
`a particular embodiment of content provision system 10 may
`be speci?cally con?gured to support such formats as
`Microsoft DV, Video for Windows, DirectShow, QuickTime,
`MPEG-2, MPEG-4, Windows Media, DivX, MP3, PCM
`WAV, AVISynth script, Audio Compression Manager (ACM),
`Macromedia Flash, RealV1deo, VOB (DVD-Video image),
`Windows bitmap (BMP), TGA, TIFF, Portable Network
`Graphics (PNG), and JPEG and, when requested, process,
`modify, or convert the stored media for output as one or more
`of an MPEG-2, MPEG-4, or SDI-encoded video stream to
`clients 12.
`[0028] In operation, content management server 16,
`transcoders 18, and signature server 20 interact to satisfy
`requests to upload submitted content ?les 30 from content
`sources 14 and to retrieve submitted content ?les 30 for cli
`ents 12. In the illustrated embodiment, content is uploaded
`from content sources 14 to system 10 in the form of submitted
`content ?les 30. More speci?cally, in the illustrated example,
`a client 12 transmits a submitted content ?le 30 to content
`management server 16 for storage on submitted content store
`22 and subsequent use by other users.
`[0029] To illustrate this process, FIG. 1 shows an example
`in which a user attempts to upload a submitted content ?le 30,
`here submitted content ?le 30a, to submitted content store 22.
`As shown in FIG. 1, the user attempts to upload submitted
`content ?le 3011 by transmitting submitted content ?le 30a to
`content management server 16. For example, a user may
`attempt to upload submitted content ?le 30a to a content
`sharing website as part of a Hypertext Transfer Protocol
`(HTTP) POST operation.
`[0030] In response, content management server 16 initiates
`a signature analysis process to determine whether the submit
`ted content ?le 30a represents and/ or includes protected con
`tent. As part of this process, content management server 16
`transmits submitted content ?le 30a to a selected one of
`transcoders 18a-d. In particular embodiments, the selected
`transcoder 18 decodes submitted content ?le 30a from its
`original media format to raw content (e.g., raw video). In the
`illustrated embodiment, the selected transcoder 18 then gen
`erates one or more content signatures based on the raw con
`tent from submitted content ?le 3011. Although FIG. 1 illus
`trates an embodiment of system 10 in which transcoders 18a-f
`generate ?rst-level and second-level content signatures, in
`alternative embodiments, signature server 20, other compo
`nents of system 10, or components external to system 10 may
`instead be responsible for generating one or more of the
`
`Page 9 of 22
`
`

`

`US 2008/0235200 A1
`
`Sep.25,2008
`
`content signatures utilized by system 10. For example, the
`copyright holder for protected content ?les 32 may generate
`content signatures for protected content ?les 32 and transmit
`these to system 10 for use in signature analysis.
`[0031] The generated content signatures each describe one
`or more characteristics of submitted content ?le 3011. Each
`content signature may represent a ?le, a collection of one or
`more values, a binary indication of Whether a particular con
`dition is satis?ed by the corresponding submitted content ?le
`30, and/or any appropriately structured information that
`describes characteristics of the corresponding content ?les.
`Examples of these content signatures may include, but are not
`limited to, image histograms, grayscale values, chroma val
`ues, frequency domain representations of the image (e.g., a
`Wavelet representation), the results of object identi?cation
`algorithms (e. g., an indication of Whether a face Was detected
`at a particular location Within the content ?le or foliage Was
`detected in the background of the content ?le), the results of
`other pattern recognition algorithms, and/or any other appro
`priate description of the contents of the corresponding con
`tent ?les. Additionally, in particular embodiments, these con
`tent signatures may represent a portion of the relevant content
`?le itself. Moreover, for multimedia content, each content
`signature may represent characteristics of the video portion of
`the content, the audio portion, or both. An example technique
`for generating a particular type of ?rst-level signature are
`discussed in greater detail beloW With respect to FIG. 4.
`[0032] Additionally, as noted above, the content signatures
`generated by transcoders 18 may represent part of the ?rst
`level in a multi-level signature analysis process. Moreover,
`these content signatures may be generated using a set of
`lightWeight signature-generating algorithms that can be
`quickly generated and/or generated using a limited amount of
`processing capacity. Thus, in the illustrated example, the
`selected transcoder 18 generates a plurality of ?rst-level sub
`mitted signatures 34a-c based on submitted content ?le 30a
`and transmits these ?rst-level submitted signatures 34 to sig
`nature server 20 for analysis.
`[0033] Meanwhile, at any appropriate time While system 10
`is operational, system 10 may receive protected content ?les
`32 containing protected content. As used in this description
`and the claims that folloW, “protected content” may include
`any form of copyrighted, restricted-use, or licensed content,
`or any content users of system and/or the general public are
`not authorized to use. In particular embodiments, “protected
`content” may also include pornographic, explicit, and/or
`offensive content, or content that users may be prohibited
`from using or disseminating on system 10 for any other rea
`son.
`[0034] In the illustrated embodiment, as system 10 receives
`protected content ?les 32, protected content ?les 32 are trans
`mitted to one or more transcoders 18 responsible for process
`ing protected content (transcoders 18e-g in FIG. 1). Similar to
`transcoders 18a-d, transcoders 18e-g decode protected con
`tent ?les 32 and generate ?rst-level protected signatures 36
`from the resulting raW video. Transcoders 18e-g utilize the
`same ?rst-level signature generation algorithms as transcod
`ers 18a-d use to generate ?rst-level submitted signatures 34.
`Additionally, in particular embodiments, transcoders 18e-g
`may generate one or more second level content signatures for
`protected content ?les 32, such as protected thumbnails 40,
`for each protected content ?le 32. Each protected thumbnail
`40 represents a portion of the corresponding protected con
`tent ?les 32, such as one or more frames. In particular
`
`embodiments, each protected thumbnail 40 represents a time
`average of multiple (e.g., ?ve) consecutive frames of the
`corresponding protected content ?les 32. Transcoders 18e-g
`may transmit these ?rst-level protected signatures 36 and
`protected thumbnails 40 to signature server 20. Signature
`server 20 may then store ?rst-level protected signatures 36 in
`protected signature store 24 and protected thumbnails 40 in
`protected thumbnail store 26.
`[0035] As a result, When signature server 20 receives ?rst
`level submitted signatures 34 from one of transcoders 18a-d,
`signature server 20 may compare each ?rst-level submitted
`signature 34 for submitted content ?le 30a to a corresponding
`set of ?rst-level protected signatures 36 maintained by signa
`ture server 20. For example, if submitted content signature
`3411 represents gray scale values extracted from submitted
`content ?le 30a, signature server 20 compares submitted con
`tent signature 34a to a set of protected content signatures 36,
`each representing gray scale values generated from a different
`protected content ?le 32 recognized by system 10. Similarly,
`if submitted content signature 34b represents chroma values
`extracted from submitted content ?le 30a, signature server 20
`compares submitted content signature 34b to another set of
`protected content signatures 36, each representing chroma
`values generated from a different protected content ?le 32
`recognized by system 10. Signature server 20 may perform
`these comparisons in any appropriate manner based on the
`con?guration and capabilities of signature server 20. One
`example of hoW such comparisons may be implemented in
`particular embodiments of signature server 20 is discussed in
`greater detail beloW With respect to FIG. 4.
`[0036] Signature server 20 then determines, based on the
`comparison betWeen ?rst-level submitted signatures 34 and
`their corresponding ?rst-level protected signatures 36,
`Whether submitted content ?le 30 is suf?ciently similar to one
`or more protected content ?les 32 to Warrant remedial action
`and/or further investigation. For example, as noted above,
`signature server 20 may utilize a multi-level technique for
`signature analysis and may perform multiple different levels
`of signature comparisons. As a result, in response to deter
`mining that ?rst-level submitted signatures 34 are suf?ciently
`similar to one or more corresponding set of ?rst-level pro
`tected signatures 36, signature server 20 may generate addi
`tional content signatures from submitted content ?le 30a. For
`example, in the illustrated example, signature server 20 gen
`erates or retrieves an additional signature, such as submitted
`thumbnail 38, that is created using a signature generation
`algorithm different from those used to generate ?rst-level
`submitted signatures 34.
`[0037] In particular embodiments, the addit