`US007460731B2
`
`c12) United States Patent
`Senftner et al.
`
`(10) Patent No.:
`(45) Date of Patent:
`
`US 7 ,460, 731 B2
`Dec. 2, 2008
`
`(54) PERSONALIZING A VIDEO
`
`(75)
`
`Inventars: Blake Senftner, Burbank, CA (US); Liz
`Ralston, Burbank, CA (US); Miles
`Lightwood, Glendale, CA (US); Todd
`Shifflett, Venice, CA (US)
`
`(73) Assignee: Flixor, Inc., Burbank, CA (US)
`
`( *) Notice:
`
`Subject to any disclaimer, the term ofthis
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 0 days.
`
`(21) Appl. No.: 11/840,834
`
`(22) Filed:
`
`Aug. 17, 2007
`
`(65)
`
`Prior Publication Data
`
`US 2008/0019576 Al
`
`Jan.24,2008
`
`(52) U.S. Cl. ....................................... 382/284; 382/103
`(58) Field of Classification Search ................. 382/100,
`382/103, 305, 284, 282-283; 358/537-538,
`358/540, 452-453; 707/1-7, 104.1; 348/584,
`348/586,589,590, 591, 598, 600, 722,578,
`348/36, 39, 135, 140--143, 169-172; 345/723;
`386/52, 55,95,54, 53
`See application file for complete search history.
`
`(56)
`
`References Cited
`
`U.S. PATENT DOCUMENTS
`
`6,795,818 Bl*
`912004 Lee ............................... 707/3
`6,826,292 Bl* 1112004 Tao et al. .................... 382/103
`6,970,639 Bl* 1112005 McGrath et al.
`.............. 386/52
`7,230,653 Bl*
`6/2007 Overton et al. . . . . . . . . . . . . . . 348/ 5 84
`* cited by examiner
`Primary Examiner-Sherali Ishrat
`(74) Attorney, Agent, or Firm-Fish & Richardson P.C.
`
`Related U.S. Application Data
`
`(57)
`
`ABSTRACT
`
`(63)
`
`(60)
`
`Continuation of application No. PCT/US2006/
`036150, filed on Sep. 14, 2006.
`
`Provisional application No. 60/717,852, filed on Sep.
`16, 2005, provisional application No. 60/717,937,
`filed on Sep. 16, 2005, provisional application No.
`60/717,938, filed on Sep. 16, 2005.
`
`(51)
`
`Int. Cl.
`G06K 9136
`G06K 9100
`
`(2006.01)
`(2006.01)
`
`Processes and apparatus for personalizing video through par(cid:173)
`tial image replacement are disclosed. Personalization may
`include partial or füll replacement ofthe image of an actor, an
`object or both. Personalization may also include insertion or
`replacement of an object, and füll or partial replacement of
`the background and/or sound track. A video preparation pro(cid:173)
`cess may be used to create a library of personalization-ready
`videos.
`
`101Claims,11 Drawing Sheets
`
`Digital video
`
`2D Image of
`new actor
`
`r------- -------1
`
`1
`
`r------ ------
`' 1
`
`1
`
`Track original
`actor's position,
`orientation, and
`expression
`
`210
`
`220
`
`230
`
`240
`
`1
`
`1
`
`Render image of
`new actor over
`Replace
`frames prepared
`background behind
`by 220. in
`key portions of the 1--1--t--+1
`position,
`image ofthe
`orientation and
`original actor
`expression from
`210
`
`1
`:
`1
`1
`
`ldentify and isolate
`non-replaced skin 1--1--+--1>1
`areas
`
`Alter non~replaced
`skin areas to
`match skin
`appearance
`of new actor
`
`:
`1
`
`ldentify and track
`mumination,
`
`200 \
`
`shading, shadows,
`1::, ~ reflections
`
`!:',
`
`Recreate
`illumination,
`shading, shadows,
`
`reflections J
`
`:
`
`1
`
`1
`1
`1
`1
`---------·------
`
`1
`_______ j
`
`Personalized
`video
`
`Create model of
`:
`new actor
`1
`(3D geometry,
`:
`demographic
`1
`profile,and
`:
`1
`personal
`1
`1nformation)
`1 L------- - - - - - - -1
`i------- ------,
`
`1LJ
`
`1
`
`110
`
`100
`
`320
`
`330
`
`340
`
`300
`
`1
`
`Petitioner Samsung 1006
`
`
`
`U.S. Patent
`
`Dec. 2, 2008
`
`Sheet 1of11
`
`US 7 ,460, 731 B2
`
`Digital video
`
`20 lmageof
`new actor
`
`r - - - - - -
`1
`1
`1
`
`- - - - - - -,
`1
`1
`1
`t
`
`Track original
`actor's position,
`orientation, and
`expression
`
`r------- -------,
`1
`1
`Create model of
`1
`1
`new actor
`1
`1
`:
`(30 geometry,
`:
`demographic
`1
`:
`profile, and LJ : ' 100
`personal
`1
`information)
`1
`1
`L------- -------1
`r-------
`
`110
`
`Replace
`background behind
`key portions of the i---r----r-111'1
`image ofthe
`original actor
`
`1
`1
`Render image of
`1
`1
`new actor over
`1
`~_._-Pi frames prepared
`by 220, in
`position,
`orientation and
`expression from
`210
`
`ldentify and isolate
`non-replaced skin 1---+----i-!PI
`areas
`
`Alter non-replaced
`skin areas to
`match skin
`appearance
`of new actor
`
`320
`
`330
`
`340
`
`210
`
`220
`
`230
`
`240
`
`1
`1
`1
`1
`
`ldentify and track
`illumination,
`shading, shadows,
`reflections
`
`200~
`
`1
`1
`
`FIG. 1
`
`Recreate
`illumination,
`shading, shadows,
`
`reflections L-/300
`
`1
`_______ J
`
`Personalized
`video
`
`2
`
`
`
`U.S. Patent
`
`Dec. 2, 2008
`
`Sheet 2of11
`
`US 7,460,731 B2
`
`Pose and
`expression
`
`320 ~ --_ :ara---------_ :;~e d:~ -------~·:~n dara
`
`30 Model of
`new actor
`
`Transform 30
`model
`
`Convert to 20
`image and scale
`
`FIG.2
`
`------1
`1
`1
`1
`1
`
`Insert 20 image
`into frame
`of video clip
`
`To
`330
`
`3
`
`
`
`U.S. Patent
`
`Dec. 2, 2008
`
`Sheet 3of11
`
`US 7 ,460, 731 B2
`
`Model of new
`actor as
`created by a
`prior use of
`process 100
`
`Model of2"d
`new actor as
`created by a
`prior use of
`process 100
`
`Model of N1
`h
`new actor as
`created by a
`prior use of
`process 100
`
`,---- ---------- ------------- --,
`
`FIG. 3
`
`User controlled N-dimensional
`morph of the 3D geometry and
`parameters that define a new actor
`
`1
`1
`1
`1
`1
`1
`1
`
`110A
`
`1
`1
`1
`1
`
`L_J-100A
`
`1
`---------------~
`
`Hybrid model of new actor in identical
`data format as new actor models
`created by use of process 100
`
`4
`
`
`
`U.S. Patent
`
`Dec. 2, 2008
`
`Sheet 4of11
`
`US 7 ,460, 731 B2
`
`Digital video
`
`r------
`1
`1
`1
`1
`
`Track location for
`object placement
`
`30 model of
`new object
`,-------
`
`1
`1
`1
`1
`t - - - . - - - -o - - iP I
`
`Insert image of
`new object
`
`350
`
`200A r~ -------------_:
`
`ldentify and track
`illumination,
`shading, shadows,
`reflections
`
`1
`1
`1
`
`Recreate
`illumination,
`shading, shadows,
`reflections
`
`1
`
`1
`1
`
`'
`1
`
`340A
`
`~ - - - - - - - - - - - - - - ~\_ 300A
`
`Personalized
`video
`
`1
`1
`1
`1
`1
`1
`1
`1
`:
`1
`1
`1
`
`250
`
`240A
`
`FIG.4
`
`5
`
`
`
`U.S. Patent
`
`Dec. 2, 2008
`
`Sheet 5of11
`
`US 7 ,460, 731 B2
`
`Digital video
`
`r------
`1
`1
`1
`1
`1
`
`Track position and
`orientation of
`original object
`
`Remove image of
`original object and
`replace
`background
`
`ldentify and track
`illumination,
`shading, shadows,
`reflections
`
`1
`1
`
`255
`
`260
`
`2408
`
`2008
`
`FIG. 5
`
`1
`
`1
`1
`
`~--------------!
`
`1
`1
`1 - - - -1 - - -1 - - -
`
`30 model of
`replacement
`object
`,-------
`' 1
`
`Insert image of
`replacement object
`
`Recreate
`illumination,
`shading, shadows,
`reflections
`
`1
`1
`1
`1
`
`1
`1
`
`"----------------'
`Personalized
`video
`
`1
`\
`
`360
`
`3408
`
`3006
`
`6
`
`
`
`U.S. Patent
`
`Dec. 2, 2008
`
`Sheet 6of11
`
`US 7 ,460, 731 B2
`
`Digital video
`
`,------
`
`1
`1
`1
`
`Track camera
`location
`
`New
`background
`image
`
`Substitute at least
`a portion of the
`1---.----+--19'! background image
`area
`1
`1
`1
`1
`
`r------- ------,
`1
`1
`1
`1
`1
`1
`
`370
`
`1
`1
`1
`1
`
`: _______ -------~
`
`Personalized
`video
`
`3ooc
`
`265
`
`270
`
`200C
`
`FIG.6
`
`Separate
`foreground and
`background image
`areas
`
`1
`
`_r~ -------------_i
`
`7
`
`
`
`U.S. Patent
`
`Dec. 2, 2008
`
`Sheet 7of11
`
`US 7 ,460, 731 B2
`
`! 1008
`
`New actor
`voice sample
`r------ -------,
`1
`1
`1
`1
`1
`'
`1
`1
`1
`1
`
`ldentify key
`attributes of new
`actor's voice
`
`1
`1
`
`'------- -------
`,-------
`
`Digital video
`,-------
`
`1
`1
`1
`1
`1
`
`1
`1
`1
`1
`1--....,._---1--111M
`
`lsolate original
`actor's dialog
`
`Transform original
`actor's dialog
`
`180
`
`380
`
`390
`
`:
`1..---------------'
`
`\_ '
`
`Personalized
`video
`
`3000
`
`280
`
`290
`
`2000
`
`FIG. 7
`
`Substitute or add
`lsolate
`background audio 1---.-------111M background audio
`elements
`elements
`1
`1
`
`1
`1
`1
`1
`
`1
`1
`
`rl
`
`_)
`
`~-- - - -
`
`:
`
`- - - ______ I
`
`8
`
`
`
`U.S. Patent
`
`Dec. 2, 2008
`
`Sheet 8of11
`
`US 7 ,460, 731 B2
`
`FIG.8
`
`410~
`
`Requestor
`
`450
`
`455
`
`Digital
`Video
`
`415
`
`200
`
`Video
`Preparation
`Process
`
`Request
`
`Personalized
`Video
`
`2DDigital
`lmage(s) +
`other info
`
`425
`
`Actor
`Modeling
`Process
`
`Actor
`Model
`
`100
`
`435
`
`465
`
`470
`
`Selection
`
`Selection
`
`Video
`Library
`
`Prepared
`Digital
`Video
`
`Personalization
`Process
`
`Actor
`Model
`
`300
`
`Actor
`Model
`Library
`
`440
`
`9
`
`
`
`U.S. Patent
`
`Dec. 2, 2008
`
`Sheet 9of11
`
`US 7,460,731 B2
`
`410~
`
`'
`
`FIG. 9
`
`Requestor
`
`550
`
`~420
`
`'~~~
`
`Personalized
`Advertising
`Video
`
`2D Digital
`lmage(s) +
`other info
`
`425
`
`Actor
`Modeling
`Process
`
`100
`
`450
`
`455
`
`Suppliers of
`Digital Video
`
`Digital
`Video
`
`200
`
`465
`
`Video
`Preparation
`Process
`
`Prepared
`Digital
`Video
`
`Selection
`
`Selection
`
`Video
`Library
`
`Prepared
`Digital
`Video
`
`300
`
`470
`
`Personalization
`Process
`
`3D
`Product
`Model
`
`3D Product
`Modeling
`process
`
`Selection
`
`Product
`Model
`Library
`
`510
`
`520
`
`Actor
`Model
`
`Actar
`Model
`Library
`
`440
`
`540
`
`Demograph ic
`1 nformation
`
`Advertising
`Strategy
`
`530
`
`10
`
`
`
`U.S. Patent
`
`Dec. 2, 2008
`
`Sheet 10 of 11
`
`US 7,460,731 B2
`
`FIG. 10
`
`20 Digital
`lmage(s) +
`other info
`
`640
`
`Personalized
`Video
`Clip
`
`600
`
`680
`
`\/ideo
`Library
`
`610
`
`620
`
`Processor
`
`Storage
`Medium
`
`630
`
`Memory
`
`11
`
`
`
`U.S. Patent
`
`Dec. 2, 2008
`
`Sheet 11 of 11
`
`US 7 ,460, 731 B2
`
`FIG.11
`
`Personalized
`Video
`
`Requestor
`...
`L::::.
`1 1
`1 1
`'1
`1 1
`
`'1
`
`720
`
`Memory
`
`Processor
`
`Storage
`Medium
`
`700
`
`730
`
`740
`
`710
`
`II Personalizable
`~Video
`
`760
`
`Video
`Library
`
`750
`
`12
`
`
`
`US 7,460,731 B2
`
`1
`PERSONALIZING A VIDEO
`
`PRIORITY CLAIMS AND RELATED
`APPLICATION INFORMATION
`
`This non-provisional patent applicationis, under 35 U.S.C.
`§ 120, a continuing patent application of and claims priority to
`the co-pending PCT Application No. PCT/US2006/036150
`entitled "PERSONALIZING A VIDEO" and filed on Sep. 14,
`2006 (PCT Publication No. W02007/035558) which, under
`35 U.S.C. §119(e)(l), claims priorities from the following
`prior U.S. patent applications: Application No. 60/717,852
`filed on Sep. 16, 2005 and entitled "Facial image replace(cid:173)
`ment;" Application No. 60/717,937 filed Sep. 16, 2005 and
`entitled "Customized product marketing images;" Applica(cid:173)
`tion No. 60/717,938 filed on Sep. 16, 2005 and entitled "Call
`and message notification."
`Tue above referenced PCT application and U.S. patent
`applications are incorporated by reference in their entirety as
`part ofthe specification ofthe present application.
`
`NOTICE OF COPYRIGHTS AND TRADE DRESS
`
`A portion of the disclosure of this patent document con(cid:173)
`tains material which is subject to copyright protection. This
`patent document may show and/ or describe matter which is or
`may become trade dress ofthe owner. The copyright and trade
`dress owner has no objection to the facsimile reproduction by
`anyone of the patent disclosure as it appears in the Patent and
`Trademark Office patent files or records, but otherwise 30
`reserves all copyright and trade dress rights.
`
`2
`depict the representation of the passage of time within one or
`more settings. The number of images, the rate of display of
`the images, and the dimensions ofthe images is irrelevant. Yet
`the images are still conventionally identified as comprised of
`lines and pixels, even though throughout the various steps of
`the processes disclosed herein, the number oflines and pixels
`per frame may be statistically re-sampled, as necessary, for
`the precision of accuracy required by various steps.
`Each frame of a digital video is therefore comprised of
`10 some total number of pixels, and each pixel is represented by
`some number ofbits of information indicating the brightness
`and color ofthe respective portion ofthe image. Throughout
`the plurality of methods that a digital video may be created, all
`are essentially a series of images, represented as a series of
`15 frames composed of lines and pixels. Various means of rep(cid:173)
`resenting a digital video in bits and bytes exist, but at some
`level all can be referenced as frames, lines and pixels.
`A digital video requires a display medium to view the
`frames in sequence. A display medium is typically electronic,
`20 such as a TV, computer and monitor, a cellular phone or a
`personal digital assistant (PDA). These devices receive or
`possess the digital video in the form of a file, and display the
`frames in sequence to the user. Other potential display medi(cid:173)
`ums that are not electronic exist, which are also a method for
`25 a userto experience a digital video. Examples ofthese medi(cid:173)
`ums are 1) printed holograms of the nature found on credit/
`debit cards and collectable sports cards, 2) digital paper that
`employs chemical and other non-electronic image encoding
`methods, 3) simple printed flipbooks.
`
`SUMMARY
`
`TECHNICAL FIELD
`
`This disclosure relates to systems, processes and apparatus
`for manipulating, processing and producing video creating
`personalized videos through partial image replacement.
`
`BACKGROUND
`
`Digital image recording, storage, and synthesis are now
`widely used in television, motion pictures, and video games.
`A digital video is essentially a series of digital photographs,
`commonly termed "frames," of a scene taken at periodic
`intervals. A digital video may be recorded using a digital
`video camera, may be created by digitizing an analog video
`recording or by digitizing a motion picture film recording,
`may be created by rendering 2D and 3D computer graphics,
`or even a hybrid combination of all the above with analogue
`or digital compositing of the various elements to realize a
`final digital video. To give the viewer an impression of
`smooth, continuous motion, digital or analog video images
`are typically comprised of 25, 30, 60, or more frames every
`second. However, the number of frames per second should
`not be viewed as a limiting factor to identify a digital video; 55
`some digital video formats support as few as 1 frame every N
`seconds, or even a variable frame rate as necessary to achieve
`the effect of perceived motion while attempting to reduce the
`final storage size ofthe resulting digital video. Whatever the
`frame rate, each frame may be divided into a plurality of
`horizontal lines, and each line is typically divided into a
`plurality of picture elements, commonly termed "pixels", per
`line. Standard broadcastvideo recordings in the United States
`have 525 lines per frame, and HDTV recordings have 1080
`lines per frame. However, for the purposes of this description,
`the term "digital video" is meant in the broader sense of
`simply a series of images that when viewed in sequence
`
`Apparatus, systems and techniques for providing person(cid:173)
`alized digital video in various applications are described. One
`35 or more target images, such as an actor and an object, in an
`original digital video can be replaced based on user prefer(cid:173)
`ences to produce a personalized digital video. Such a person(cid:173)
`alized video can be used for advertising a product or service
`by inserting one or more images associated with the product
`40 or service in the personalized video.
`In one implementation, a computer-implemented process
`for providing personalized digital video can include selecting
`a target in original digital video to be replaced by a target
`replacement, wherein the target is a portion or an entirety of
`45 an actor or an object other than an actor in the original digital
`video; analyzing each frame of the original digital video to
`track a change in the selected target in the original digital
`video to capture data on the selected target, wherein the
`captured data includes at least information on a position,
`50 orientation and size of the selected target in the original
`digital video; and replacing the selected target with an image
`that resembles a continuation of a scene adjacent to the target
`in the original digital video to produce altered digital video in
`which the selected target is removed.
`In another implementation, a computer-implemented pro-
`cess for providing personalized digital video can include
`storing altered digital video files based on original digital
`video files in a video library. At least one target in an original
`video file is removed in a corresponding altered digital video
`60 file and is substituted by an image that resembles a continu(cid:173)
`ation of a scene adjacent to the target in a frame ofthe original
`digital video file, and the target is a portion or an entirety of an
`actor or an object other than an actor in the original digital
`video file. This method includes receiving, via a computer in
`65 communication with a network, a request from a user for
`personalizing a user selected altered digital video file by
`replacing a target in a corresponding original digital video file
`
`13
`
`
`
`US 7,460,731 B2
`
`15
`
`3
`with a user target replacement identified by the user; and
`retrieving, from the video library, data on the target that is
`removed from the user selected altered digital video file. The
`data includes at least information on a position, orientation
`and size ofthe target in the original digital video file for the
`user selected altered digital video file. This method further
`includes applying the retrieved data on the target, frame by
`frame, to transform the user target replacement received from
`the user into a modified user target replacement that acquires
`characteristics of the target in the corresponding original
`digital video file; and inserting the modified user target
`replacement at a position of the target in each frame of the
`user selected altered digital video file in which the target
`appears in the original digital video file to substantially match
`at least the position, orientation and size ofthe selected target
`in the original digital video file to produce a personalized
`digital video file.
`In another implementation, a system for providing person(cid:173)
`alized digital video can include a network for communica- 20
`tions; and a video library to store altered digital video files
`based on original digital video files. At least one target in an
`original video file is removed in a corresponding altered
`digital video file and is substituted by an image that resembles
`a continuation of a scene adjacent to the target in a frame of 25
`the original digital video file, and wherein the target is a
`portion or an entirety of an actor or an object other than an
`actor in the original digital video file. This system includes at
`least one computer in communication with the network and
`the video library. The computer is operable to interface with 30
`a user via the network and to receive a request from the user
`for personalizing a user selected altered digital video file by
`replacing a target in a corresponding original digital video file
`with a user target replacement identified by the user. The
`computer is operable to retrieve from the video library data on 35
`the target that is removed from the user selected altered digital
`video file, where the data includes at least information on a
`position, orientation and size of the target in the original
`digital video file for the user selected altered digital video file.
`This computer is also operable to apply the retrieved data on 40
`the target, frame by frame, to transform the user target
`replacement received from the user into a modified user target
`replacement that acquires characteristics of the target in the
`corresponding original digital video file, and to insert the
`modified user target replacement at a position of the target in 45
`each frame of the user selected altered digital video file in
`which the target appears in the original digital video file to
`substantially match at least the position, orientation and size
`ofthe selected target in the original digital video file to pro(cid:173)
`duce a personalized digital video file for the user.
`In another implementation, a computer-implemented pro(cid:173)
`cess for providing personalized video can include providing a
`user selected altered digital video file by replacing a target in
`a corresponding original digital video file with a user target
`replacement identified by the user; and obtaining data on the 55
`target that is removed from the user selected altered digital
`video file. The data includes at least information on a position,
`orientation and size ofthe target in the original digital video
`file for the user selected altered digital video file. This process
`includes applying the obtained data on the target, frame by 60
`frame, to transform the user target replacement into a modi(cid:173)
`fied user target replacement that acquires characteristics of
`the target in the corresponding original digital video file; and
`inserting the modified user target replacement at a position of
`the target in each frame of the user selected altered digital 65
`video file in which the target appears in the original digital
`video file to substantially match at least the position, orien-
`
`4
`tation and size of the selected target in the original digital
`video file to produce a personalized digital video file.
`In another implementation, a process is provided for per(cid:173)
`sonalizing an original digital video containing an image com(cid:173)
`prising an original background scene and a foreground
`including an original actor. This process includes tracking the
`position, orientation, and expression of the original actor;
`replacing at least a key portion of the original actor with an
`image that continues the background scene; inserting a new
`10 actor into the video, the new actor substantially matching the
`position, orientation and expression of the replaced portion of
`the original actor; recreating illumination and shading effects
`on the new actor; and recreating shadows and reflections of
`the new actor.
`In another implementation, a process for personalizing a
`video can include providing a video library of a plurality of
`prepared videos, each of the prepared videos resulting from a
`video preparation process; providing an actor model library
`of one or more new actor models where each of the models
`resulting from an actor modeling process; selecting a video
`from the video library; selecting a new actor model from the
`actor model library; and applying a personalization process to
`create a personalized version ofthe selected video using the
`selected new actor model.
`In another implementation, a process is provided for cre(cid:173)
`ating a personalized version of an original digital video and
`the original video contains an image of an original actor. This
`process includes replacing at least a portion ofthe image of
`the original actor with an image of a new actor; and inserting
`an image of a new object into the video.
`In yet another implementation, a computing device is pro(cid:173)
`vided to create a personalized version of an original digital
`video containing an image comprising a background scene
`and a foreground including an image of an original actor. This
`computing device includes a processor, a memory coupled
`with the processor, a storage medium having instructions
`stored thereon which when executed cause the computing
`device to perform actions. These actions include comprising
`personalizing a video where the video contains an image
`including an image of an original actor, tracking the position,
`orientation, and expression ofthe original actor; replacing at
`least a key portion of the original actor with an image that
`continues the background scene; inserting a new actor into the
`video where the new actor substantially matches the position,
`orientation and expression of the replaced portion of the
`original actor; recreating illumination and shading effects on
`the new actor; and recreating shadows and reflections ofthe
`new actor.
`These and other implementations and associated features
`50 are described in greater detail in the attached drawings, the
`description and the claims.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`FIG. 1 is a flow chart of a process to create a personalized
`digital video.
`FIG. 2 is an expansion ofthe process ofFIG. 1.
`FIG. 3 is a flow chart of optional steps in the process to
`create a personalized video.
`FIG. 4 is a flow chart of optional steps in the process to
`create a personalized video.
`FIG. 5 is a flow chart of optional steps in the process to
`create a personalized video.
`FIG. 6 is a flow chart of optional steps in the process to
`create a personalized video.
`FIG. 7 is a flow chart of optional steps in the process to
`create a personalized video.
`
`14
`
`
`
`US 7,460,731 B2
`
`5
`FIG. 8 is a flow chart of another process for providing
`personalized videos.
`FIG. 9 is a flow chart of a process for providing personal(cid:173)
`ized advertising.
`FIG. 10 is a block diagram of a computer apparatus.
`FIG. 11 is a block diagram of another computer apparatus.
`
`DETAILED DESCRIPTION
`
`10
`
`6
`Depending upon the quality of the desired replacement,
`and the impact ofthis desired quality upon the storyline ofthe
`original video, essential elements ofthe replaced actors inter(cid:173)
`pretation andmeaning within the context ofthe storyline may
`be altered. For example, by replacing a muscular, heroic male
`in a video with that of a petite female, the storyline remains
`essentially unaltered, but the interpretation ofthe storyline is
`drastically altered for effect. In order for such alteration to
`occur, the replacement ofthe face and portions ofthe head is
`not enough to achieve this result; in this situation a complete
`removal of the original actor is executed, their key motions
`are preserved in a secondary storage medium, and then ref(cid:173)
`erenced for the animation and insertion of the petite female' s
`digital double.
`Between the two extremes of face/head with connecting
`skin and a complete digital double lies an infinite range of
`degrees which actor replacement could be executed. Note that
`in all instances ofthis infinite range, the storyline and essen(cid:173)
`tial actions of the actors in the original video may be unal-
`20 tered.
`A personalized video can be personalized in various ways.
`In one example, a personalized video is a video sequence that
`is altered via one or more actor replacements. In another
`example, a personalized video is a video sequence that does
`25 not include any actor replacements-and includes non-actor
`objects and/or set pieces within the video which have been
`alte red and/or replaced with othero bjects and/ or set pieces. In
`yet another example, a personalized video is a video sequence
`that is altered via a combination of (1) at least one actor
`30 replacement or alternation and (2) at least one object replace(cid:173)
`ment or altemation. The objects and/or set pieces that have
`been altered and/or replaced may be selected based a user
`criterion, such as some demographic or other information
`about the receiver and/or viewer of the digital video. For
`35 example, a video advertisement for a pizza restaurant could
`receive personalization such that the video advertisement
`appears to take place at the specific local neighborhood fran(cid:173)
`chise of the video' s viewer. In this instance, no actor replace(cid:173)
`ment has occurred, but the video is personalized due to the
`40 local world-space context ofthe new advertisement having a
`personal connection with the viewer-it now takes place in
`their neighborhood. A further example of a non-actor replac(cid:173)
`ing personalized video would be a video depicting a character
`drinking some brand of soda, where the brand of soda has
`45 been altered to be a brand of soda that appeals to the demo(cid:173)
`graphic of the viewer. Different viewers, being members of
`different demographic populations, would receive different
`alterations for the brand of soda being consumed by the
`video's character-making each video personalized to fit to
`50 the demographic tastes of each viewer.
`Certain theater systems use stereopsis to provide an illu(cid:173)
`sion of a three-dimensional (3D) image. These systems
`present separate images or film 2D channels for each ofthe
`viewer's eyes. The two image channels may be presented on
`55 a common surface and separated at the viewer's eyes by
`specialized glasses with polarized or colored lenses. Other
`optical techniques may be employed such that each image
`channel is only presented to the appropriate eye. Although the
`discussion in this disclosure is primarily directed to person-
`60 alizing conventional (2D) videos, the personalization tech(cid:173)
`niques disclosedherein could be also applied to the two image
`channels of a stereographic 3D display system.
`A personalized video may be provided in a range of pos(cid:173)
`sible formats including, but not limited to, the following:
`(a) A Digital Rights Management free format that allows
`for downloading and free exchange and trading ofthe video.
`In advertising applications, due to the potential presence of
`
`the embodiments and
`this description,
`Throughout
`examples shown should be considered as exemplars, rather
`than limitations on the apparatus and methods disclosed or
`claimed.
`Throughout this description, the terms "digital video clip", 15
`"video clip", "clip" and "digital video" all connote a digital
`encoding of a series of images with the intent to view the
`images in sequence. There is no implied limitation on a digital
`video's duration, or the final medium which a digital video
`may be expressed. Examples of a digital video include, but
`are not limited to, the following: a portion of a current or
`classic movie or television show, an entire film or TV show, an
`advertisement, a music video, or a specialty clip made spe(cid:173)
`cifically for personalization (for example, a clip that can be
`personalized to show the new actor with a "celebrity friend").
`A single frame image, e.g., one from any of the previous
`examples, can be considered a digital video for some imple(cid:173)
`mentations within the context ofthis specification. Likewise,
`a single image composed via a collage technique between still
`frame images and other digital image manipulation and cre(cid:173)
`ation techniques can also be considered a digital video in
`some implementations within the context of this specifica(cid:173)
`tion. A digital video may be generated via various methods.
`For example, a digital video may have been recorded using a
`digital video camera, may have been digitized from an analog
`video camera or film recording, may have been retrieved from
`a digital medium such as a DVD, may have been created by
`combining multiple still images placed on a timeline for
`animation purposes, may have been created by a composite
`process employing any ofthe above and other processes not
`described here, or otherwise.
`Tue creation of personalized video is a combination of
`multiple fields that in totality allow for the alteration of video
`sequences such that individuals are able to replace the par(cid:173)
`ticipants of an original video with themselves, their friends,
`their family members or any individuals, real or imagined,
`which they have images depicting. This replacement of par(cid:173)
`ticipants in an original video may only require, but is not
`limited to, the replacement of the face, portions of the head
`and/or connecting skin as visible in the original video due to
`framing ofthe view and/or occluding individuals and objects
`in the video sequence blocking the view ofthe entire replaced
`individuals' bodies, costumes and/or wardrobe wom by the
`role depicted by the replaced character within the storyline of
`the video sequence and so forth. Depending upon the content
`of the storyline depicted within the original video, the
`replacement of participants within the video may include
`portions of their other visible skin, such as hands, arms, legs
`and so on.
`Furthermore, the replacement desired may be carried to an
`extreme with the essential removal ofthe original actor in the
`video sequence, the removal of their shadow, reflection and
`visual influence upon other objects in the scene with a com(cid:173)
`pletely synthesized version (a "digital double") ofthe replac(cid:173)
`ing individual and the addition of their characteristic shadow, 65
`reflectionandothervisual influences uponotherobjects in the
`scene.
`
`15
`
`
`
`US 7,460,731 B2
`
`7
`product placements, advertisers may benefit from having per(cid:173)
`sonalized videos exchanged, traded and displayed in as many
`locations as possible.
`(b) A Digital Rights Management enabled format that only
`allows the download and playback on specific devices by
`specific individuals, such as the purchase of a personalized
`video for playback on specific devices.
`( c) 3gp, 3gpp, mv4, giforotherpublic orproprietary digital
`video or di
Accessing this document will incur an additional charge of $.
After purchase, you can access this document again without charge.
Accept $ ChargeStill Working On It
This document is taking longer than usual to download. This can happen if we need to contact the court directly to obtain the document and their servers are running slowly.
Give it another minute or two to complete, and then try the refresh button.
A few More Minutes ... Still Working
It can take up to 5 minutes for us to download a document if the court servers are running slowly.
Thank you for your continued patience.
This document could not be displayed.
We could not find this document within its docket. Please go back to the docket page and check the link. If that does not work, go back to the docket and refresh it to pull the newest information.
Your account does not support viewing this document.
You need a Paid Account to view this document. Click here to change your account type.
Your account does not support viewing this document.
Set your membership
status to view this document.
With a Docket Alarm membership, you'll
get a whole lot more, including:
- Up-to-date information for this case.
- Email alerts whenever there is an update.
- Full text search for other cases.
- Get email alerts whenever a new case matches your search.
One Moment Please
The filing “” is large (MB) and is being downloaded.
Please refresh this page in a few minutes to see if the filing has been downloaded. The filing will also be emailed to you when the download completes.
Your document is on its way!
If you do not receive the document in five minutes, contact support at support@docketalarm.com.
Sealed Document
We are unable to display this document, it may be under a court ordered seal.
If you have proper credentials to access the file, you may proceed directly to the court's system using your government issued username and password.
Access Government Site