US008650591B2
`
`(12) United States Patent
`Prieto
`
`(10) Patent N0.:
`(45) Date of Patent:
`
`US 8,650,591 B2
`Feb. 11, 2014
`
`(54)
`
`VIDEO ENABLED DIGITAL DEVICES FOR
`EMBEDDING USER DATA IN INTERACTIVE
`APPLICATIONS
`
`(58) Field of Classi?cation Search
`USPC ............................. .. 725/32i38; 709/2l7i232
`See application ?le for complete search history.
`
`(76)
`
`Inventor: Yolanda Prieto, Coral Gables, FL (US)
`
`(56)
`
`References Cited
`
`(*)
`
`Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 220 days.
`
`(21)
`
`(22)
`
`(65)
`
`(60)
`
`(51)
`
`(52)
`
`Appl. No.: 13/042,955
`
`Filed:
`
`Mar. 8, 2011
`
`Prior Publication Data
`US 2011/0225610A1
`Sep. 15,2011
`
`Related US. Application Data
`
`Provisional application No. 61/311,892, ?led on Mar.
`9, 2010.
`
`Int- Cl-
`H04N 7/025
`US. Cl.
`USPC .............................. .. 725/36; 725/34; 709/231
`
`(2006-01)
`
`U.S. PATENT DOCUMENTS
`
`7,076,392 B1*
`8,024,768 B2 *
`2008/0192736 A1*
`
`702/119
`7/2006 Wilson ...... ..
`725/146
`9/2011 Berger et a1. ..
`8/2008 Jabri et a1. .................. .. 370/352
`
`* cited by examiner
`
`Primary Examiner * Annan Shang
`(74) Attorney, Agent, or Firm * Michael J. Buchenhorner
`
`ABSTRACT
`(57)
`Connecting vieWers in the broadcasting and gaming experi
`ence for the purpose of enhancing the entertainment experi
`ence of a user is a fast growing technical area. It seeks to
`engage the vieWer in a media that is visually rich Which
`consequently provides a neW and unique form of enhancing
`the user experience.
`
`20 Claims, 3 Drawing Sheets
`
`User input data J
`
`m
`
`r
`
`Game Device \
`
`‘ w
`
`1
`
`,
`
`1
`j
`
`Home Data
`NeiworkModem
`
`1-1-9
`
`. a
`
`.
`
`DIgEtBI Media
`Player/Recorder
`£95
`
`1
`
`_
`
`W
`m r/
`PC
`
`data
`I
`I
`We“
`
`‘
`
`1
`
`'
`
`\
`’
`
`/ Userinputdatavia
`,4
`
`conneciedcamera
`
`Synchronizad?anverged Data
`
`-------------------- -
`
`W1 1.188111
`
`528"‘ @5512??? modi?ed
`p / 1
`
`3
`
`‘\
`\'\
`1
`imemetléezwork
`j
`Metadaiza
`I/
`,\
`\\ 116 /v/
`
`.
`
`Uni-channeior
`'9Mu1t;-channe|Set<-''">
`Tran Rm
`Lu
`
`Image
`Substitution
`190
`
`1
`
`Petitioner Samsung 1001
`
`

`

`US. Patent
`
`Feb. 11,2014
`
`Sheet 1 013
`
`US 8,650,591 B2
`
`,/7
`User input data i/yjl
`
`199
`
`W’
`w
`
`‘
`
`I‘
`
`v
`Q GameDevzce 3”“
`3mg“;
`1
`1
`
`I’ W
`
`l9 DigitalMediag
`
`Mayer/Recorden
`.192
`3
`
`I
`homeData e‘
`Network Modem
`112
`
`I
`
`>
`
`
`
`
`
`Synchronized Camerged Data sértiammmiiiizmf?f v W! user :npu/
`
`
`
`Uni-channei or létviulti-channeiSetf'm")I
`
`.
`.
`Usennputéatawa
`
`El connectedcamera
`i
`
`Userinputéatavia
`\i connected camera
`
`
`
`/ \
`
`
`
`
`
`‘ internet?etwork Metaéata
`
`
`
`\ if’ _j Tnn Rm: L13
`
`
`
`
`
`‘\
`
`116 //
`
`'mage
`Substitution
`190
`
`FIG. 1
`
`2
`
`

`

`US. Patent
`
`Feb. 11,2014
`
`Sheet 2 of3
`
`US 8,650,591 B2
`
`4 User input data via
`\1 connected camera
`
`i
`
`150
`
`User Input
`
`Gaming
`
`Wireless TX
`
`evice
`
`1
`
`Image Substitution
`
`190
`
`.
`
`.
`
`D E 1 User Input data V13
`N connected camera
`
`FIG. 2
`
`3
`
`

`

`US. Patent
`
`Feb. 11,2014
`
`Sheet 3 of3
`
`US 8,650,591 B2
`
`wired or wireless
`transmission
`
`150
`
`Image Substitution
`
`FIG. 3
`
`4
`
`

`

`US 8,650,591 B2
`
`1
`VIDEO ENABLED DIGITAL DEVICES FOR
`EMBEDDING USER DATA IN INTERACTIVE
`APPLICATIONS
`
`CROSS-REFERENCE TO RELATED
`APPLICATIONS
`
`This application claims priority from Provisional Patent
`Application No. 61/31 1,892, Which is incorporated by refer
`ence herein.
`
`10
`
`STATEMENT REGARDING FEDERALLY
`SPONSORED-RESEARCH OR DEVELOPMENT
`
`None.
`
`INCORPORATION BY REFERENCE OF
`MATERIAL SUBMITTED ON A COMPACT DISC
`
`None.
`
`20
`
`FIELD OF THE INVENTION
`
`The invention disclosed broadly relates to the ?eld of data
`base administration and more particularly relates to the ?eld
`of altering index objects in tables.
`
`25
`
`BACKGROUND OF THE INVENTION
`
`Vast research and implementation are currently underWay
`to connect content from the Internet With active broadcast
`services. For example, by the use of Widgets, content may be
`connected to services, thus synchronizing to programs
`selected by the user or vieWer. Additionally, current technol
`ogy provides viable means of searching, Within a data stream
`and metadata, objects of interest Which further alloWs the
`capability of editing, substituting, and enhancing the identi
`?ed data in the original. data stream or metadata.
`
`30
`
`35
`
`SUMMARY OF THE INVENTION
`
`40
`
`Brie?y, according to an embodiment of the invention a
`method for generating an edited video data stream from an
`original video stream Wherein generation of said edited video
`stream comprises a step of: substituting at least one object in
`a plurality of objects in said original video stream by at least
`a different object. According to another embodiment, an
`interactive broadcast TV system enhances the user experi
`ence by alloWing said user the capability of providing his/her
`object data for use in the substitution of an object data in a
`broadcast video by said user object data.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`To describe the foregoing and other exemplary purposes,
`aspects, and advantages, We use the folloWing detailed
`description of an exemplary embodiment of the invention
`With reference to the draWings, in Which:
`FIG. 1 shoWs a block diagram of a digital system, accord
`ing to an embodiment of the present invention;
`FIG. 2 shoWs a block diagram of a television and gaming
`device, according to an embodiment of the invention; and
`FIG. 3 shoWs a simpli?ed illustration of a video image
`substitution on a gaming device, according to an embodiment
`of the invention.
`While the invention as claimed can be modi?ed into alter
`native forms, speci?c embodiments thereof are shoWn by Way
`
`50
`
`55
`
`60
`
`65
`
`2
`of example in the draWings and Will herein be described in
`detail. It should be understood, hoWever, that the draWings
`and detailed description thereto are not intended to limit the
`invention to the particular form disclosed, but on the contrary,
`the intention is to cover all modi?cations, equivalents and
`alternatives falling Within the scope of the present invention.
`
`DETAILED DESCRIPTION
`
`Referring to FIG. 1, We discuss a video enabled input
`digital device such as an interactive television (“i TV”) 102,
`camcorder 104, camera-enabled personal device 106, gaming
`device 108, and the like, Which is operable for taking a cap
`tured video to be uploaded, or inputted by a user, for the
`purpose of inserting the video content into another video,
`graphics, image sequence selected by the user. This produces
`a neW video sequence 190 Which is subsequently broadcasted
`or played by the digital device. In addition to video, the user
`may insert any digital data such as text, audio, or images, to
`personalize the metadata 116 that has been created resulting
`from the synchronization of Internet content With broadcast
`data. In this embodiment the internet netWork is capable of
`editing the metadata 116 in order to insert the user input data
`to generate a synchronized converged data 190.
`For example, the user may have a video sequence Which
`he/she desires to insert in place of an actress or particular
`avatar in a game or broadcast for the purpose of having a
`participative experience in the game or broadcast program.
`The user Will have the capability of choosing and informing
`the video enabled digital device Which actor, avatar, back
`ground, or region of the vieWing data to be displayed he or she
`desires to edit, enhance or substitute With the user provided
`data.
`In one embodiment of this invention, the digital device is
`enabled With a camera input Which may be hardWare con
`nected to the device or Wirelessly connected to said digital
`device. The digital device may be at least a digital TV, a
`gaming console, a digital media player/recorder, or set top
`box, Which may co-exist as part of a netWork With the capa
`bility of all being connected to a netWork via the Internet. The
`connections may be Wired, Wireless, further managed by an
`in-home broadcast netWork or a home data netWork. The
`digital device is capable of receiving image, video, or text
`data from a user Wirelessly, or the device itself is image
`enabled With a mounted imaging device.
`Once the digital device receives the user input signal,
`Which may be in the form of an image, video, graphics, text,
`or any other digital data form, the digital device may further
`process the image to enhance it, compress it, or otherWise
`modify it to make it compatible With the digital device’s data
`requirements. The digital device’s processed input data is
`then sent through shared connections to a system that sup
`ports internet services, TV broadcast services, gaming ser
`vices, e-book services, in-home data net leading to a con
`verged IP netWork solution Where content and services
`provided by these services may be uni?ed and synchronized
`to provide a converged interfaced data such that the user input
`is also merged.
`An application of this embodiment is to provide the user
`With the enhanced experience of vieWing himself as part of a
`broadcast shoW Which he may subsequently desire to record
`via the digital media recorder. Such a netWork system alloWs
`the digital device to transmit the user’s request and data to a
`netWork Where his request is processed and his data inserted
`per the instruction provided by the user into the video data
`that is to be broadcast. Referring noW to FIG. 3 We shoW a user
`input 150 of a photo image of the user used to replace the face
`
`5
`
`

`

`US 8,650,591 B2
`
`3
`of the image shown on the device 108. The user transmits the
`photo image 150 by Wired or Wireless means to the device
`108. The image substitution is performed and the device 108
`shoWs the substituted image 190.
`Another embodiment of this invention alloWs for a stand
`alone digital device to receive input digital data from a user or
`from a Wired or Wireless imaging device for the purpose of
`creating a modi?ed converged data stream processed by the
`input device such that said converged data stream is the result
`of the input device’s original stream modi?ed by the user
`inputted or imaging device inputted data. One application of
`this embodiment can be a gaming device Where the user
`enters a video or image of himself for its immersion into a
`game or graphics data stream that has been created by the
`imaging device. Said imaging device is capable of receiving
`the user’s data Wirelessly or through a camera or video device
`that may be part of the digital device.
`The resulting converged data stream Will then contain the
`user’ s image data in the game data. This Will alloW the user to
`become part of the game With his or her data image. The
`user’ s input data, Which may be an image or video of himself,
`is added to a particular structure that is denoted by the user.
`The user inputted data Will contain, in addition to the user
`image data, instructions to indicate Which actor, avatar, or
`structure the user Will like to have his data incorporated into.
`In this example, the resulting graphics or video game is one in
`Which the vieWer Will see his image as part of the game.
`The received digital data form is additionally formatted
`such that an instruction is conveyed in the message data to
`explain hoW and Where the digital data is to be inserted into
`the converged interface data. This process requires the input
`device to perform video, image analysis and processing in
`order to recogniZe the actor or structure the user requests to
`have substituted by his inputted data. The process further
`requires the steps of background analysis, face detection and
`recognition, image and video motion detection and analysis
`to folloW through the input device’s motion sequence With the
`user’s inputted data to produce a resulting stream Where the
`user inputted data becomes part of the input device’s stream
`as if it Were originally its oWn.
`In FIG. 1 the User Data Device (UDD) 106 is an image
`capable digital device enabled With Wired and Wireless com
`munication capability. Further, the UDD 106 is capable of
`performing at least the folloWing functions: image and video
`capture, graphics processing, image and video processing
`such as image and video enhancements, data compression,
`data storage, data displaying, receiving and transmitting digi
`tal data from other and to other devices respectively. An
`example of UDD’s input can be image or video data. Addi
`tionally the UDD 106 can consist of a data entry means 112
`that alloWs the user to specify a set of instructions that indi
`cate hoW the input data is to be manipulated. One goal is to
`generate an output data stream that consists of the input data
`plus a set of instructions that indicates to another receiving
`device such as a PC 120, a TV 102, or a gaming device 108,
`hoW the input data is to be used or embedded in an existing
`stream of the receiving device.
`The UDD 106, TV 102, PC120, Game Device (GD) 108,
`unichannel and multi-channel set top box 114 (STB), digital
`media player/recorder 104 (DMPR) are all capable of com
`pressing, decompressing, and transcoding data for those
`image, video, and graphic formats that need conversion to
`another coding standard.
`Consider the case Where the input data is a video sequence
`of a user Which the user desires to embed in a broadcast TV
`program. The user desires to enhance his or her experience by
`vieWing a modi?ed broadcast program Which substitutes his
`
`50
`
`55
`
`60
`
`65
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`4
`image as a substitute for an actor or actress in the original
`broadcast program. The data path here de?ned in this example
`is not limited to this particular data How; instead, it is intended
`to shoW a capability in the system shoWn in FIG. 1. The user
`image or video data can be captured by the UDD device 106
`and transmitted Wired or Wireless to a television 102 (TV). In
`addition to the captured user image data, the user sends to the
`TV instructions regarding Which actor, actress, or structure
`item he desires to replace or substitute in the original pro
`gram. The user image data can be in a compressed data format
`prior to transmission to the TV 102.
`Since the TV 102 may also be image and video enabled, the
`user may select to use the TV camera input to capture the
`desired video data rather than using the UDD 106. In this
`case, the TV 102 may be equipped With the appropriate entry
`means to alloW the user to enter the appropriate instructions
`regarding hoW the user input video is to be embedded in the
`existing broadcast data sequence. The embedding instruc
`tions and the user input video data are then transmitted to a
`home data netWork modem 110 (HDNM) that has access to
`internet netWork metadata 116. The metadata 116 itself con
`tains program information that is transmitted or broadcasted
`by the TV 102. The internet netWork devices have the capa
`bility of processing the user image or video data and the set of
`instructions that indicate hoW said user input data is to be
`embedded into the metadata 116 to produce a modi?ed broad
`cast bit stream.
`To accomplish the embedding process, the internet net
`Work devices are capable of performing at least the folloWing
`functions: receiving user input data and instructions, per
`forming image and video analysis such as face recognition
`and detection, image and video data portioning, image and
`video enhancement, ?ltering, texture analysis, data compres
`sion and decompression, motion detection and estimation,
`motion correction to adapt the motion of the user input
`sequence With that of the original metadata to be broadcasted,
`error analysis, etc. Once the user input data has been correctly
`embedded into the data to be broadcasted, the internet net
`Work devices send the resulting modi?ed data to the TV 102
`to be broadcast.
`A similar data How may be con?gured using for example a
`gaming device 108 instead of a television 102. In this con
`?guration the metadata 11 6 in the internet netWork consists of
`graphics data rather than broadcast data.
`It is evident that each of the equipments or devices in FIG.
`1 may have the capability of on-the-?y or off-line modifying
`for the user input data as stand-alone devices. This implies
`that in this type of con?guration there is no need to access the
`metadata in the Internet netWork. Instead, the received user
`input data can be processed by the particularly user enabled
`device to produce a modi?ed bitstream that contains the
`embedded user de?ned input data substituting the portion of
`the original data stream With the user input data as speci?ed
`by the user. For these stand alone devices the interface from
`the user can be through the UDD, or the user input data can be
`captured directly by the stand alone device.
`The stand alone devices can be equipped With image or
`video capturing means, data entry devices such as a keyboard,
`displaying devices, Wired and Wireless communicating
`means for receiving and transmitting user input data and
`instructions, memory storage, computing devices, data
`manipulation capabilities such as performing image and
`video analysis such as face recognition and detection, image
`and video data portioning, image and video enhancement,
`?ltering, texture analysis, data compression and decompres
`sion, motion detection and estimation, motion correction to
`adapt the motion of the user input sequence With that of the
`
`6
`
`

`

`US 8,650,591 B2
`
`5
`original metadata to be broadcasted, error analysis, data
`transcoding such as converting a particular codec standard to
`another in order to conform the image captured data to the
`particular standard in the stand alone device.
`In another embodiment of this invention the user is capable
`of generating a set of programming instructions or Widgets
`for the purpose of blending the user de?ned video With the
`metadata 116 or internet content, Whereby the internet net
`Work devices are capable of interpreting said set of instruc
`tions to produce a modi?ed synchronized converged data
`stream. The programming instructions can be stored in a
`computer readable storage medium such as a CDROM or
`DVD-ROM, or other suitable media device. The user de?ned
`data that has been attached to the programming instructions is
`embedded in the original metadata to create the modi?ed
`synchronized converged data stream. A PC 120, connected as
`shoWn in FIG. 1, may be used to produce the set of program
`ming instructions and for attaching the user input data to the
`set of instructions. In the con?guration Where the stand alone
`devices are used independently of the internet content meta
`data 116, the PC transmits the user data and programming
`instructions directly to the stand-alone device con?guration
`for further processing as described in the paragraph above.
`Another embodiment of this invention is shoWn in FIG. 2.
`Consider the example Where the user desires to embed his or
`her created avatar, image, 2-dimensional (2-D) or three-di
`mensional (3D) video sequence, graphics sequence, or
`another form of digital input data 150 into an existing video
`game to create a modi?ed converged neW sequence for dis
`play. The user is capable of modifying and thus personalizing
`the existing video game according to his or her preferences.
`The user can even generate a set of programming instructions
`or Widgets for the purpose of blending the user de?ned input
`data 150 With the existing graphics data in the gaming device
`108. Said programming instructions and user data may be
`transmitted Wired or Wirelessly to the gaming device via a PC
`120 or a user input data device 106.
`The gaming device 108, TV 102, set-top box 114, user
`input data device 106 may be equipped With a data entry
`device 112 such as but not limited to: a keyboard, optical
`reader, scan device, display capability and device, projection
`capability and device, Wired and/ or Wireless communication
`capability and device, internet netWork capability, memory
`storage. The device 108 has computing means to perform data
`manipulation capabilities such as performing image and
`video analysis such as face recognition and detection, image
`and video data portioning, image and video enhancement,
`?ltering, texture analysis, data compression and decompres
`sion, motion detection and estimation, motion correction to
`adapt the motion of the user input sequence With that of the
`original graphical data of the gaming device, error analysis,
`data transcoding such as converting a particular codec stan
`dard to another in order to conform the image captured data to
`the particular standard in the stand alone device. In the case
`Where a user inputs a video sequence 150 into an interactive
`TV 102, the video data 150 is transmitted via the Internet to a
`server or broadcast station Where the image substitution 190
`takes place. The substituted image is then broadcast back to
`the TV 102.
`The embodiments described in this invention use mainly
`video or images as the user de?ned input. HoWever, user
`de?ned input for embedding in an existing data format to
`create the user experience that he or she desires is not limited
`to video or image data that has been captured by the user. It
`may consist of sequences that may have been previously
`doWnloaded for the internet or other sources. It may addition
`ally consist of text data, voice data, sound data, a neW back
`
`20
`
`25
`
`30
`
`35
`
`40
`
`50
`
`55
`
`60
`
`65
`
`6
`ground that permits modifying the vieWing scene Which is to
`be broadcasted or displayed by a stand-alone device for the
`purpose of personalizing his or her vieWing experience.
`Another embodiment of this invention alloWs the user to
`embed his or her voice or a user selected voice and substitute
`it for a voice already in a video broadcast, a game, a graphics
`sequence, or a digital device such as a talking e-book. It is
`then a requirement that all these stand alone devices as Well as
`the internet netWork devices are capable of performing voice
`recognition, voice and audio processing to alloW embedding
`a neW audio or speech and substituting it for another existing
`voice.
`Referring again to FIG. 2 there is shoWn an embodiment
`presenting examples of digital input devices Which are
`capable of merging the user input data into their original
`streams to produce a modi?ed converged stream 190 accord
`ing to the user demands and the user’s provided data.
`What has been shoWn and discussed is a highly-simpli?ed
`depiction of a programmable computer apparatus. Those
`skilled in the art Will appreciate that a variety of alternatives
`are possible for the individual elements, and their arrange
`ment, described above, While still falling Within the scope of
`the invention. Thus, While it is important to note that the
`present invention has been described in the context of a fully
`functioning data processing system, those of ordinary skill in
`the art Will appreciate that the processes of the present inven
`tion are capable of being distributed in the form of a computer
`readable medium of instructions and a variety of forms and
`that the present invention applies equally regardless of the
`particular type of media actually used to carry out the distri
`bution. Examples of media include ROMs, DVD-ROMs, and
`transmission-type media, such as digital and analog commu
`nication links, Wired or Wireless communications links using
`transmission forms, such as, for example, radio frequency
`and light Wave transmissions. The media make take the form
`of coded formats that are decoded for use in a particular data
`processing system.
`This invention may be easily adapted to hand-held or por
`table devices: Prompting is especially useful for small hand
`held devices, such as cell phones, Which Would otherWise
`require (non-intuitive) complex or multiple ?elds, screen dis
`plays, etc. Altematively, the device can “Walk” or “guide” the
`user through using friendly intuitive methodsiprompting
`the user for appropriate responses or programmed to antici
`pate (learning the -end-user’s habits and behaviors) the most
`commonly used keyWord identi?ers and defaulting as such by
`default so as to minimize the questioning and prompting. This
`can be done by auto-generating the data in the ?elds. This
`provides for a more user friendly and intuitive interface and a
`more pleasant experience for a user. OtherWise, a longer
`learning curve is required if the same Were used With multiple
`other specialized ?elds and/or screens for each possible sce
`nario or behavior that Would be required for each keyWord
`that may be entered in addition to ordinary data When one or
`multiple like or differing ?elds are simultaneously displayed
`or maintained in either the foreground or background.
`When using less de?nitive means of entering data and
`commands Where the user is more detached from the elec
`tronic or computing device, there is a further need for the user
`to be able to easily indicate and the computer or device to
`recognize obscurity or ambiguous entries that may have mul
`tiple meanings and to delineate and con?rm, either though
`solicitation or for the user to indicate using more explicit
`means, Whether data is to be interpreted as a function, com
`mand or simply as ordinary data.
`Therefore, While there has been described What is presently
`considered to be the preferred embodiment, it Will understood
`
`7
`
`

`

`US 8,650,591 B2
`
`7
`by those skilled in the art that other modi?cations canbe made
`Within the spirit of the invention. The above descriptions of
`embodiments are not intended to be exhaustive or limiting in
`scope. The embodiments, as described, Were chosen in order
`to explain the principles of the invention, shoW its practical
`application, and enable those With ordinary skill in the art to
`understand hoW to make and use the invention. It should be
`understood that the invention is not limited to the embodi
`ments described above, but rather should be interpreted
`Within the full meaning and scope of the appended claims.
`
`I claim:
`1. An interactive media apparatus for generating a display
`able edited video data stream from an original video data
`stream, Wherein at least one pixel in a frame of said original
`video data stream is digitally extracted to form a ?rst image,
`said ?rst image then replaced by a second image resulting
`from a digital extraction of at least one pixel in a frame of a
`user input video data stream, said apparatus comprising:
`an image capture device capturing the user input video data
`stream;
`an image display device displaying the original video
`stream;
`a data entry device, operably coupled With the image cap
`ture device and the image display device, operated by a
`user to select the at least one pixel in the frame of the user
`input video data stream to use as the second image, and
`further operated by the user to select the at least one pixel
`to use as the ?rst image;
`Wherein said data entry device is selected from a group of
`devices consisting of: a keyboard, a display, a Wireless
`communication capability device, and an external
`memory device;
`a digital processing unit operably coupled With the data
`entry device, said digital processing unit performing:
`identifying the selected at least one pixel in the frame of
`the user input video data stream;
`extracting the identi?ed at least one pixel as the second
`image;
`storing the second image in a memory device operably
`coupled With the interactive media apparatus;
`receiving a selection of the ?rst image from the original
`video data stream;
`extracting the ?rst image;
`spatially matching an area of the second image to an area
`of the ?rst image in the original video data stream,
`Wherein spatially matching the areas results in equal
`spatial lengths and Widths betWeen said tWo spatially
`matched areas; and
`performing a substitution of the spatially matched ?rst
`image With the spatially matched second image to
`generate the displayable edited video data stream
`from the original video data stream.
`2. The interactive media apparatus of claim 1 Wherein the
`digital processing unit is further capable of performing:
`computing motion vectors associated With the ?rst image;
`and
`applying the motion vectors to the second image extracted
`from the user input video data stream, Wherein the gen
`erated displayable edited video data stream resulting
`from the substitution maintains an overall motion of the
`original video data stream.
`3. The interactive media apparatus of claim 1 Wherein the
`digital processing unit is further capable of extracting the at
`least one pixel from the user entering data in the data entry
`display device.
`
`20
`
`25
`
`30
`
`35
`
`40
`
`45
`
`50
`
`55
`
`60
`
`65
`
`8
`4. The interactive media apparatus of claim 3 Wherein the
`digital processing unit is further capable of extracting the at
`least one pixel from the user pointing to a spatial location in
`a displayed video frame.
`5. The interactive media apparatus of claim 1 Wherein the
`digital processing unit is further capable of extracting the at
`least one pixel from the user selecting from a displayed list of
`items identifying content in the video.
`6. The interactive media apparatus of claim 1, Wherein the
`second image is transmitted Wirelessly to the interactive
`media apparatus.
`7. The interactive media apparatus of claim 1 Wherein the
`digital processing unit is capable of processing video games.
`8. The interactive media apparatus of claim 1, Wherein the
`substitution performed by the digital processing device
`replaces at least a face of a ?rst person from the original video
`data stream by at least a face of a second person from the user
`input video data stream.
`9. The interactive media apparatus of claim 1 further com
`prising an audio processing unit performing audio processing
`corresponding to the video data streams.
`10. The interactive media apparatus of claim 9, Wherein the
`audio processing unit is additionally capable of performing
`audio processing to replace audio corresponding to the ?rst
`image in the original video stream With audio corresponding
`to the second image in the user input video data stream.
`11.Amethod for generating a displayable edited video data
`stream from an original video data stream, Wherein at least
`one pixel in a frame of the original video data stream is
`digitally extracted to form a ?rst image, said ?rst image then
`replaced by a second image resulting from a digital extraction
`of at least one pixel in a frame of a user input video data
`stream, said method comprising:
`capturing a user input video data stream by using a digital
`video capture device;
`using a data entry device operably coupled With the digital
`video capture device and a digital display device, select
`ing the at least one pixel in the frame of the input video
`data stream;
`Wherein the data entry device is selected from a group of
`devices consisting of: a keyboard, a display, a Wireless
`communication capability device, and an external
`memory device; and
`using a digital processing unit operably coupled With the
`data entry device, performing:
`identifying the selected at least one pixel in the frame of
`the input video stream;
`extracting the identi?ed at least one pixel as the second
`image;
`storing the second image in a memory device operably
`coupled With the digital processing unit;
`receiving a selection of the ?rst image from the user
`operating the data entry device;
`extracting the ?rst image from the original video data
`stream;
`spatially matching an area of the second image to an area
`of the ?rst image in the original video data stream,
`Wherein spatially matching the areas results in equal
`spatial lengths and Widths betWeen said tWo spatially
`matched areas;
`performing a substitution of the spatially matched ?rst
`image With the spatially matched second image to
`generate a the displayable edited video data stream
`from the original video data stream;
`computing motion vectors associated With the ?rst
`image; and
`
`8
`
`

`

`US 8,650,591 B2
`
`9
`applying the motion vectors to the second image,
`Wherein the generated displayable edited video data
`stream resulting from the substitution maintains an
`overall motion of the original video data stream.
`12. An interactive media apparatus in an internet enabled
`broadcast netWork system for generating a displayable edited
`video data stream from an original broadcast video data
`stream, Wherein at least one pixel in a frame of the original
`broadcast video data stream is digitally extracted to form a
`?rst image, said ?rst image is substituted by a second image
`resulting from a digital extraction of at least one pixel in a
`fram