ARAVAUYAAA
`
`US 20010034668A1
`
`as) United States
`a2) Patent Application Publication (10) Pub. No.: US 2001/0034668 Al
`Whitworth
`(43) Pub. Date:
`Oct. 25, 2001
`
`
`(54) VIRTUAL PICTURE HANGING VIA THE
`INTERNET
`
`(76)
`
`Inventor: Brian L. Whitworth, Malibu, CA (US)
`
`Correspondence Address:
`Brian L. Whitworth
`3003 Sequit Dr.
`Malibu, CA 90265 (US)
`,
`(21) Appl. No.:
`
`09/770,150
`
`(22)
`
`Filed:
`
`Jan. 27, 2001
`
`Related U.S. Application Data
`
`(63) Non-provisional of provisional
`60/178,863, filed on Jan. 29, 2000.
`
`application No.
`
`Publication Classification
`
`
`ecceecceseseesneeneceeccenennnnesees GO6F 17/60
`(SL) Unt, C07 acne
`(52) US. C1 eeceeeeceessssssssessnessssnsccssnseesnnneecsnnseersseeeees 705/27
`
`(57)
`ABSTRACT
`:
`:
`oo
`A method for viewing images of one or more items which a
`user may be interested in buying and displaying candidate
`items with one or more imagesprovided by the user, such as
`a room image, an image of a person, or an image ofa car.
`Methodsfor replacing, recoloring, orfilling all or part of a
`user’s image are provided, such as could be used to simulate
`the appearance of installed paint, carpet, or wallpaper.
`Methods are also disclosed where at least one image is
`animated, moves, morphs, grows, or otherwise changes.
`Lighting conditions, size, scale, and perspective of one or
`more images can be adjusted.
`
`111
`109
`50
`107
`
`
`HTML OR JAVA
`KEYBOARD OR
`SELLER'S
`
`
`
`SITE PROGRAM
`MODEM
`DIGITAL CAMERA
`
`
`
`
`HTML OR JAVA
`
`
`AUTHORING
` SELLER'S
`SOFTWARE
`
`COLOR SCANNER
`404
`
`
`COMPUTER
`MONITOR
`INFO ON SELLERS
`
` INTERNETSITE
`AND ITEMS TO BE
`
`
`
`SOLD
`
`
`MODEM, CABLE
`POTENTIAL BUYER'S
`
`MODEM OR LAN DIGITAL CAMERA
`
`
`
`
`SERVER
`139
`INTERNET
`
`CAPABLE
`POTENTIAL BUYER'S
`
`
`KEYBOARD OR
`COMPUTER
`COLOR SCANNER
`
`MODEM
` 125
`
`PURCHASEINFO ON
`ARTWORK
`
`anonen
`
` 137
`
`
`POTENTIAL BUYER'S
`
`PREFERENCES
`
`
`
`
` 135
`
`VIEWS OF
`
`131
`ARTWORKS ON
`
`BACKGROUND(S)
`COMPUTER
`MONITOR
`
`
`PRINTER
`
`Zynga Ex. 1016,p. 1
`Zynga v. IGT
`IPR2022-00199
`
`Zynga Ex. 1016, p. 1
` Zynga v. IGT
` IPR2022-00199
`
`

`

`Patent Application Publication Oct. 25,2001 Sheet 1 of 14
`
`US 2001/0034668 A1
`
`407
`
`105
`
`50
`
`4
`
`HTML OR JAVA
`SITE PROGRAM
`
`109
`
`111
`
`MODEM
`
`KEYBOARD OR
`SELLER'S
`DIGITAL CAMERA
`
`
`413
`
`|
`HTML OR JAVA
`AUTHORING
`SOFTWARE
`
`SELLER'S
`COLOR SCANNER
`
`101
`
`
`103|eS sor 121
`r=]
`'V
`
`404
`
`COMPUTER
`
`11
`
`
`
`INTERNETSITE
`
`AND ITEMS TO BE
`
`MODEM, CABLE
`MODEM OR LAN
`
`POTENTIAL BUYER'S
`DIGITAL CAMERA
`
`
`
`
`
`MONITOR INFO ON SELLERS
`
`
`8
`All
`SERVER
`
`139 INTERNET
`
`
`POTENTIAL BUYER'S
`
`
`CAPABLE
`
`
`KEYBOARD OR
`COLOR SCANNER
`
`
`COMPUTER
`MODEM
`
`125
`
`
`
`
`
`POTENTIAL BUYER'S
`
`
`
`131
`
`COMPUTER
`MONITOR
`
`
`
`VIEWS OF
`
`
`
`ARTWORKSON
`BACKGROUND(S)
`
`FIG.
`
`1
`
`PRINTER
`
`Zynga Ex. 1016, p. 2
`Zynga v. IGT
`IPR2022-00199
`
`137
`
`
`
`PURCHASE INFO ON
`ARTWORK
`
`INTERNET
`
`BROWSER
`
`PREFERENCES
`
`
`
` 135
`
`Zynga Ex. 1016, p. 2
` Zynga v. IGT
` IPR2022-00199
`
`

`

`Patent Application Publication Oct. 25,2001 Sheet 2 of 14
`
`US 2001/0034668 A1
`
`RAW DIGITAL IMAGE OF
`ITEM
`
`
`
`ITEM IMAGE &
`ASSOCIATED DATA
`
`201
`
`207
`
`
`
`nichrat CAMERA
`ELLER'S
`
`
`
`
`
`SELLER'S
`
`
`COLOR SCANNER
`
`
`
`
`INFO ON SELLERS AND
`
`ITEMS TO BE SOLD
`
`
`203
`
`205
`
`209
`
`213
`
`217
`
`Zynga Ex. 1016, p. 3
`Zynga v. IGT
`IPR2022-00199
`
`
`
`
`
`IMAGEIN
`
`PROPER
`
`SPECTRUM?
`
`
`
` NO
`RESIZE
`
`
`IMAGE FOR
`
`DISPLAY?
`
`
`
`
`"
`COLOR CORRECTION TO
`SUNLIGHT SPECTRUM,
`OPTIONAL
`
`NO
`
`RESIZING, OPTIONAL
`
`
`219
`
`YES
`
`ATTACHED
`
`SCANNED IMAGE WITH
`COLOR CORRECTION,
`RESIZED, INFO ON
`SELLER AND ITEM
`
`Zynga Ex. 1016, p. 3
` Zynga v. IGT
` IPR2022-00199
`
`

`

`Patent Application Publication Oct. 25,2001 Sheet 3 of 14
`
`US 2001/0034668 A1
`
`RAWDIGITAL IMAGE OF
`ROOM
`
`
`
`ROOM IMAGE&
`ASSOCIATED DATA
`
`303
`
`305
`
`309
`
`
`
`
`OenALCAMER.
`POTENTIAL BUYER’
`
`
`
`COLOR SCANNER
`
`
`POTENTIAL BUYER'S
`
`
`
`
`POTENTIAL BUYER'S
`PREFERENCES
`
`
`7
`313
`
`COLOR CORRECTION TO
`
`IMAGEIN No_|SUNLIGHT SPECTRUM,
`
`
`PROPER
`OPTIONAL
`
`SPECTRUM?
`
`
`
`
`
`
`
`RESIZING, OPTIONAL|247
`RESIZE
`NO
`
`
`IMAGE FOR
`
`DISPLAY?
`
`
`319
`
`YES
`
`
`
`REUSABLE SCANNED
`
`
`IMAGE WITH COLOR
`CORRECTION, RESIZED,
`INFO ON BUYER AND
`
`
`ROOM ATTACHED
`
`Zynga Ex. 1016, p. 4
`Zynga v. IGT
`IPR2022-00199
`
`Zynga Ex. 1016, p. 4
` Zynga v. IGT
` IPR2022-00199
`
`

`

`Patent Application Publication Oct. 25,2001 Sheet 4 of 14
`
`US 2001/0034668 A1
`
`GO TO WEBSITE WITH
`IMAGES OF ITEMS FOR
`
`401
`
`SALE
`
`
` VIEW ART
`
`IMAGES WITH
`
`ROOM IMAGE?
`
`
`YES
`
`
`
`
`
`UPLOAD ROOM IMAGE
`AND BUYERINFO, MAKE
`ANY ADJUSTMENTS
`
`405
`
`NO
`
`
`VIEW EVERY
`IMAGE AT
`
`WEBSITE?
`
`NO
`
`
`
` PRIORITIZE
`
`419
`
`CANDIDATE
`
`IMAGES?
`YES
`
`
`
`MODIFY ROOM
`
`IMAGE?
`
`
`YES
`
`PROCEED TO STEP 501
`
`407
`
`409
`
`413
`
`|
`SET POSITION FOR
`ARTWORKIMAGE TO
`DISPLAY
`
`,
`
`
`SET ROOM IMAGE AS
`PAGE BACKGROUND,
`
`LOWER LAYER, OR FRAME
`
`
`
`NARROW POSSIBLE
`
`
`
`IMAGES FOR REVIEW
`IMAGESBY CRITERIA
`
`SORT CANDIDATE
`
`PROCEED TO
`STEP 601
`
`423
`
`Zynga Ex. 1016, p. 5
`Zynga v. IGT
`IPR2022-00199
`
`Zynga Ex. 1016, p. 5
` Zynga v. IGT
` IPR2022-00199
`
`

`

`Patent Application Publication Oct. 25,2001 Sheet 5 of 14
`
`US 2001/0034668 A1
`
` 506
`
`
`
`BUY OR BID ON
`
`
`ENLARGE/POSITION
`ITEM(S) OF INTEREST
`
`4ST ITEM(S)?
`
`
` NO
`
`
`YES
`
`BUY ORBID ON
`
`| ARTWORK >
`
`
` SAVE AS
`
`CANDIDATE FOR
`
` DONE SHOPPING ON
`
`LATER REVIEW
`THIS WEBSITE
`
`509
`
` DISPLAY NEXT
`
`
`
`ARTWORKIMAGE(S)
`WITH ROOM IMAGE
`
`
`517
`YES
`
`519
`
`
`
`
`
`FIG.5| GOTOSTEP505 GOTOSTEP501 |
`
`
`
`Zynga Ex. 1016, p. 6
`Zynga v. IGT
`IPR2022-00199
`
`Zynga Ex. 1016, p. 6
` Zynga v. IGT
` IPR2022-00199
`
`

`

`Patent Application Publication Oct. 25,2001 Sheet 6 of 14
`
`US 2001/0034668 A1
`
`OBTAIN DIGITAL IMAG
`OF ROOM WITH COLOR
`CORRECTION AND
`RESIZING FROM USER OR
`PLUG-IN
`
`
`
`
`
`
`
`
`
`
`CHOOSE PORTION OF
`
` ROOM TO BE MODIFIED
`
`609
`
`
`
`CLICK ON PORTION OF
`OUTLINE PORTION OF
`
`
`
`ROOM TO BE MODIFIED
`
`ROOM TO BE MODIFIED REPLACE ORIGINAL
`
`
`
`AREA WITH COLOR OR
`PATTERN WITH PAINT,
`CARPET, OR WALLPAPER
`ADJ. FOR ROOM LIGHT
`
`
`
`
`
`Zynga Ex. 1016, p. 7
`Zynga v. IGT
`IPR2022-00199
`
`Zynga Ex. 1016, p. 7
` Zynga v. IGT
` IPR2022-00199
`
`

`

`Patent Application Publication Oct. 25,2001 Sheet 7 of 14
`
`US 2001/0034668 A1
`
`BUY OR BID ON
`
`706
`
`SIMULATE
`INSTALLATION OF
`ITEM
`
`707
`Yes
`BUYORBIDON=|YES BUY OR
`
`703
`ARTWORK
`|
`RID?
`
`
`
`
`
`"7 ng|MAYBENO
`709
`YES
`
`
` MAYBE
`4ST ITEM(S)?
`
`
`THIS WEBSITE
`
`SAVE AS
`CANDIDATE FOR
`LATER REVIEW
`
`DONE SHOPPING ON
`
`711
`
`NO
`
`NO 721
`
`YES
`
`
`
`DISPLAY NEXT
`
`ARTWORKIMAGE(S)
`WITH ROOM IMAGE
`|
`REQUEST SAMPLES
`
`MAYBE
`
`YES
`
`717
`
`719
`
`723
`
`GO TO STEP 701
`
`GO TO STEP 703
`
`GO TO STEP 705
`
`Zynga Ex. 1016, p. 8
`Zynga v. IGT
`IPR2022-00199
`
`Zynga Ex. 1016, p. 8
` Zynga v. IGT
` IPR2022-00199
`
`

`

`Patent Application Publication Oct. 25,2001 Sheet 8 of 14
`
`US 2001/0034668 A1
`
`IMAGE WAS TAKEN
`
` 801 INPUT DIGITAL ROOM IMAGE
`
` 8037TAKE PC TO ROOM WHERE
`
`
`
` 805
`
`
`COLOR
`
`
`SET WHITE BALANCE
`
`CORRECT
`
`IMAGE?
`
`
`DON'T
`PRINT OR
`
`
`
`
`EMAIL
`
`DISPLAY CANDIDATEITEM(S)
`WITH ROOM IMAGE
`
`ADJUST MONITORDISPLAY
`TO MATCH ROOM
`
`INTERNET SHOPPING
`
`
`
`SAVE ROOM IMAGE
`
`813
`
`
`
`807
`
`
`
`815
`
`
`
`
`PRINT
`EMAIL
`
`
`UNMODIFIED?
`
`UNMODIFIED?
`
`
`
`
`
`PRINTSCREEN AND
`
`
`PRINT FROM
`MODIFY IMAGE IN
`EDIT IMAGE AND
`BROWSER
`EDITOR, PRINT
`PASTE INTO EMAIL
`
`ATTACH FILE OR
`
`
`
`FROM EDITOR
`
`
`
`PASTE INTO EMAIL
`
`
`Zynga Ex. 1016, p. 9
`Zynga v. IGT
`IPR2022-00199
`
`Zynga Ex. 1016, p. 9
` Zynga v. IGT
` IPR2022-00199
`
`

`

`Patent Application Publication
`
`Oct. 25,2001 Sheet 9 of 14
`
`US 2001/0034668 A1
`
`
`
`SAOYNOSANVS.LNdNIFOVGSYINOSY6‘Did
`
`
`
`SAOVWNIYAH.LONOGIVTYSAOJuvHOIHMSS9VNI
`
`
`
`
`
`‘ASOduNd
`
`SNISOOHDYOONIANG
`
`ONIdIULSWIL‘SWIY
`
`
`
`LuVdINSWS0V1d44
`
`
`
`SONINd‘SSdv¥a
`
`SIWHIVH
`
`uiVH
`
`ANIDNA30AuNLOld
`
`YayVvaM
`
`
`SWOHNIWOOY
`
`JIWOHSHLYO4YYOMLYY
`
`JOldsO
`
`
`
`301440YOdWHOMLYY
`
`
`
`SWOHNIWOOY
`
`
`
`dOV4S.YAAN
`
`
`
`SONI1¢‘SAdVua
`
`SFTALSYIVH
`
`
`JOVAS\YSANG
`
`SLONGOUdSYVOYIVH
`
`YVOS.YdANG
`
`SOLNVONIZINOLSND
`
`
`
`
`
`SONVH"WOSN‘JOv4
`
`NAS
`
`WOON‘NNS|
`
`
`YaTIAS“ANNVWI
`
`
`
`YATIAS‘LSILYV
`
`
`
`YATISS“NNW
`
`
`
`Y3TIAS“SNNV
`
`
`
`YITS“4NNVA
`
`aYNLINYNs
`
`YATIAS“ANNVN
`
`SOINOYLO313
`
`YSHOYVSS
`
`YaMOND
`
`
`
`JIDJAYLCALVIN
`
`NOS9SYNS
`
`JIDYIVHGALYWINY
`
`u3d34ud
`ONIAVTdTYWINY
`
`YaAnd
`
`
`
`AQO€LNSYYND
`
`
`
`AQOdS.YSANg
`
`AWOHNIWOO’
`
`AWOH40AdISLNO
`
`SLNAWAAOdillSWOH
`
`AWOH40AdIsino
`AYMAAIYGNIVO
`
`WOOudag
`
`SLAYNV19‘SLASHS}
`
`AWOHNiWOO’
`SOINOYLOA13SWOH
`
`AWIOHNiWOO’
`
`SAISILOATIOO“FYNLAINDS
`
`TACOWO/MANIZVOVN
`
`SYAAODANIZVOVWAV
`
`100dAdYIVHO
`LISIAOLLYOS3YYOT3.LOH
`
`dOYOMOVEOLLNVNOY
`
`
`
`ONILVG'SSLISSSTONIS
`
`AINOH40SCISLNO
`
`
`
`JOVAS.Y3SANg
`
`QUYVAMOVE
`
`ONIdVOSANY
`
`SLNVTdiYIVH
`
`SLdd
`
`
`AQO€dsxIS3d
`
`
`NOILINLAN‘GOudSSANLI4
`
`Zynga Ex. 1016, p. 10
`Zynga v. IGT
`IPR2022-00199
`
`Zynga Ex. 1016, p. 10
` Zynga v. IGT
` IPR2022-00199
`
`
`
`
`
`
`
`
`

`

`
`YaTIAS“ANNVNNuall¥dYOYO109SdOLYSLNNODMAN
`
`
`
`Y3aTIAS“ANNVAYO109LNIVdYVO
`
`
`
`
`YATIAS“ANNVN&usll¥dYOYOO}ONIHLOTOONIHLOTO
`
`
`JOYNOSJYNLXALYOCalsIGOWJOADVIINISOOHOYOONIANE
`
`
`
`
`
`
`40109‘NYSLLYdVaYdVTWNIDIXO‘SSOduNd
`
`
` QSQ0VIdaJUVSSYNLXALYO/GNVSYO100SYSHMSSOVINI
`
`
`
`
`
`NYaLIVdYOYO1OO}=SLASHSSLSYNV14'SLASHS
`
`
`
`
`
`
`
`SSOUNOSCNVSLAdNIJOVIGSYINDSY
`
`Patent Application Publication
`
`Oct. 25, 2001 Sheet 10 of 14
`
`US 2001/0034668 A1
`
`Ob“Sld
`
`
`
`
`
`“1d3u‘40109GINOs
`
`HLIMG30V1d34
`
`YTS“NNW
`
`
`
`Yas“ANNVW
`
`
`
`YaT13$“ANNVW
`
`
`
`YaTIAS“ANNVN
`
`
`
`Y3ATIS“ANNVW
`
`Nasilvd
`
`SYNLXal
`
`YO109
`
`YO109
`
`
`
`
`
`ONINOOTS“FTL“LAdHWO
`
`
`
`ONILHOM‘SdMAN
`
`Zynga Ex. 1016, p. 11
`Zynga v. IGT
`IPR2022-00199
`
`Zynga Ex. 1016, p. 11
` Zynga v. IGT
` IPR2022-00199
`
`
`
`
`

`

`Patent Application Publication Oct. 25,2001 Sheet 11 of 14
`
`US 2001/0034668 Al
`
`
`
`
`FIG. 11
`
`oe
`
`t=Lu2
`<=
`
`
`
`
`
`w
`S
`a
`
`Zynga Ex. 1016, p. 12
`Zynga v. IGT
`IPR2022-00199
`
`5
`
`592ma=aa
`
`oom
`2oe]
`~=z
`==
`=~
`
`8= i
`
`
`
`
`
`oea
`2=
`aa——
`
`Qe=
`
`>=
`
`o
`=I
`
`5
`xr
`
`4)
`2,
`(CSo
`
`21
`
`08>
`wy
`Ofis
`'
`
`2
`=
`g
`oO
`s/t5uw
`5
`'
`ul
`
`¥
`9
`a
`ii +3
`—|—-——,
`
`Zynga Ex. 1016, p. 12
` Zynga v. IGT
` IPR2022-00199
`
`

`

`Patent Application Publication Oct. 25,2001 Sheet 12 of 14
`
`US 2001/0034668 Al
`
` FIG. 12
`
`karencameyart.com
`|ré}http://w0ww,
` avories
`JoolsHep ?]ahGeStopRefreshHomeHistory
`a>-ix)
`itViewFi
`Forward
`|FileEdit + Back
`
` rf@Internet
`
`Done
`
`Zynga Ex. 1016, p. 13
`Zyngav. IGT
`IPR2022-00199
`
`Zynga Ex. 1016, p. 13
` Zynga v. IGT
` IPR2022-00199
`
`

`

`Patent Application Publication
`
`Oct. 25, 2001 Sheet 13 of 14
`
`
`
`deysjoo[sayoaeymarpFapy[
`
`?:dyyIEssauppy MDI
`
`woopefaweoquareymai?
`
`
`
`
`SWOLYsaysydayspiemoyyorgYBOe»
`
`
`US 2001/0034668 A1
`
`Zynga Ex. 1016, p. 14
`Zynga v. IGT
`IPR2022-00199
`
`Zynga Ex. 1016, p. 14
` Zynga v. IGT
` IPR2022-00199
`
`
`

`

`Patent Application Publication Oct. 25,2001 Sheet 14 0f 14
`
`US 2001/0034668 Al
`
`FIG. 14 r|®Internet
`
`
`2a=ax|
`
`Fyhttp://www.karencameyartcomPyon
`
`
`a&StopRefreshHomeHistory
`¢€.%.9 Forward
`
`(3)
`
`
`
`
`
`ViewFavoritesToolsHel
`
`FileEdit
`Back
`|e |
`
`
`
`@]Done
`
`Zynga Ex. 1016, p. 15
`Zynga v. IGT
`IPR2022-00199
`
`Zynga Ex. 1016, p. 15
` Zynga v. IGT
` IPR2022-00199
`
`

`

`US 2001/0034668 Al
`
`Oct. 25, 2001
`
`VIRTUAL PICTURE HANGING VIA THE
`INTERNET
`
`BACKGROUNDOF THE INVENTION
`
`[0001]
`
`1. Field of the Invention
`
`invention relates to a system and
`[0002] The present
`method for viewing one or more images of items which the
`viewer may be interested in buying and displaying such
`images with an image of where it might be placed or
`mounted. For example, artwork may be displayed with an
`image of a user’s room. Methods are included for: simulat-
`ing the look of paint or carpet using a current image of a
`room;trying on virtual clothing or hairstyles using an image
`of the wearer; for combining images from the user and the
`website to create intermediate images; and for animated
`images which simulate movement or changes overtime.
`
`[0003]
`
`2. Description of the Related Art
`
`[0004] A shopping trip via the web currently takes a user
`through what has become the world’s largest shopping mall,
`complete with cars, clothes, artwork, paint, drapes, and even
`live animals. A great manysites display two-dimensional
`images of the items for sale. A few even display three-
`dimensional images, such as cars at Toyota.com, where a 3D
`rendering can be rotated to any viewing position. However,
`even the Toyota.com site will not show what the vehicle
`would look like in your driveway, or what it would look like
`with you driving the car. Thus, potential buyers using the
`web cannot currently see how artwork might look in their
`ownliving room, how a hairstyle might look on their own
`head, or how new paint might look on the walls of their own
`house.
`
`[0005] Another problem which often occursis that an item
`looks very different in the home or on the wearer than it
`appeared on the website. A common example would be
`finding that a particular paint looks vastly different than
`expected on the walls of your home than on the website, duc
`to lighting of the room or colors of walls, drapes, carpet and
`furniture.
`
`[0006] Methods of Displaying Images of Items for Sale
`
`[0007]
`
`1. Hangable Artwork, Draperies.
`
`[0008] These items are easily displayed as two-dimen-
`sional digitized images. Except for black and white artwork,
`the images are usually in color. The images may be dis-
`played one at a time or manyon one page in “galleries”. A
`typical example of gallery display is used by eBay.com,
`where a dozen or more images are displayed on a single
`page, with some description of each artwork, against a white
`background.
`
`[0009]
`
`2. Sculpture, Furniture.
`
`for a complete
`(0010] Unlike artwork and draperies,
`impression of the look of sculpture it is usually necessary to
`show more than one view or image.
`
`[0011]
`
`3. Clothing and Hairstyles.
`
`[0012] These may be displayed on mannequins, models,
`average people, or in isolation. Clothing and hairstyles are
`not currently displayed on imagesof the potential buyers on
`the web.
`
`[0013]
`
`4. Paint, Carpet, Wallpaper.
`
`are usually displayed as color chips,
`[0014] These
`samples, or swatches. Occasionally, paint, carpet, or wall-
`paper will be displayed in model rooms, but not the rooms
`of the actual buycrs. Dulux’s www.dulux.com is one such
`site, sponsored by a manufacturer of paints andstains.
`
`[0015]
`
`5. Automobile Customization.
`
`[0016] Custom wheels, paints, body molding and other
`customparts are currently displayed either in isolation, or on
`a model car, bul nol on an image of the potential buyer’s
`actual car.
`
`(0017] Skilled computer users with photo retouching or
`drawing software can sometimes download or copy images
`from the web anddigitally overlay them on images oftheir
`room, car, face, etc. However,
`this method is not well-
`adapted to looking at large numbers of potential purchases.
`For example, if a user wanted to see what 200 different
`artworks would looklike overlaid on an imageoftheir living
`room,it would be necessary to copy 200 the images from the
`web. This would take time and disk storage space on the
`user’s computer. In addition, when the images are copied,
`certain functionality is usually lost. For cxample, in some
`galleries clicking on an image will produce a description of
`the image or allow the user to purchase the image. A copy
`of just the image taken into a photo-retouching program will
`not work in this way;it is simply a picture at that point, not
`a hyperlink.
`
`[0018] Viewing Problems.
`
`(0019] The lighting conditions used to photograph the
`items for sale can have an effect on the color of the images.
`Sites which offer itemsfor sale by third parties will typically
`receive images photographed under a variety of photo-
`graphic and lighting conditions which aftect the apparent
`colors of the image. For example, incandescentlighting will
`impart more red than daylight, while fluorescent will impart
`more blue. If the item for sale is photographed under
`different light than where the item will be used or displayed,
`the buyer may be in for a surprise. This problem is particu-
`larly irritating for paint, carpet, and wallpaper, since they are
`not easily removed or returned after installation.
`
`(0020] The color of surrounding items can strongly affect
`the perceived colors of the item being purchased. Because
`the seller does not know the current color schemes of
`potential purchasers’ rooms, cars, or faces, this problem
`exists even when the seller carefully photographs items
`under controlled conditions. Dulux, the paint manufacturer,
`has a website which can be used to demonstrate this at
`www.dulux.com. This website was meant for a different
`purpose, to demonstrate the look of various paint colors in
`a model home. However,using the “colour schemer”to vary
`the colors of paint on the walls also shows howexactly the
`same artwork and furniture appear very different when the
`colors of walls and other objects in the model home are
`changed.
`
`[0021] Size can also be a problem, especially on the
`internet where images usually do not include other items
`which help give a perception ofrelative size. Though many
`sites will give dimensions for such items as artwork or
`furniture,
`the buyer may very well open the box for a
`recently purchased item and say, “It looked a lot bigger on
`my computer monitor”.
`
`Zynga Ex. 1016, p. 16
`Zynga v. IGT
`IPR2022-00199
`
`Zynga Ex. 1016, p. 16
` Zynga v. IGT
` IPR2022-00199
`
`

`

`US 2001/0034668 Al
`
`Oct. 25, 2001
`
`[0022] Another problem is that one person may like an
`item they are about to buy, but their spouse, friends, or
`children will have quite a different impression. For example,
`if a person allows someone else to buy artwork for their
`living room,
`the person who doesn’t buy the art may
`encounter what is referred to as the “blind date problem”.
`[0023] Allowing someone else to select artwork can be
`considerably worse than allowing them to select a blind date.
`The artwork may cost considerably more than a bad night on
`the town and, unlike a bad blind date,
`the undesirable
`artwork may not leave their house for years. The artwork
`mayhang overthe fireplace for a decade or more, mocking
`the poor soul who allowed their friend or spouse to choose
`something so inappropriate and so far from their owntastes.
`[0024] Purchasing via the web maypartially alleviate this
`problem. Whenshoppingatart galleries in the real world,it
`would be necessary to bring the artwork or a photo of the
`artwork to the purchaser, or bring the purchaser to the
`candidate artwork. On the web,it is possible to simply have
`the potential purchaser look at the image on a computer
`monitor (after someone else has done the hard work of
`narrowing the candidates). Unfortunately, even with artwork
`purchased on the web, it is still difficult to visualize what it
`will look like in a particular room. Whether an image is
`displayed on a monitoror printed on a color printer, making
`color and size matches using current methods involves
`considerable skill and luck on the part of the user.
`[0025] Accordingly, none of the prior art satisfies the
`objectives of the present invention, and nonce showsthe basic
`features of the invention as described herein. More back-
`ground information can be found in the following refer-
`ences, the contents of which are incorporated by reference.
`[0026] Accordingly,it is an object of the present invention
`to provide a method for displaying an item which may be
`purchased via the internet by combining such an image with
`an image of the place whereil will be used by the purchaser.
`[0027]
`It is another object of the present
`invention to
`provide a method for viewing an imageof artwork which the
`viewer may beinterested in buying and superimposing such
`an image on an image of a room where it might be placed
`or mounted.
`
`invention to
`It is another object of the present
`[0028]
`provide a method for simulating the look of paint or carpet
`using a current image of a room whereit might be used.
`[0029]
`It is another object of the present
`invention to
`provide a method for simulating the look of actual clothing
`or hairstyles when worn bya particular individual by using
`an image of the clothing, or hairstyle and one or more
`images of the individual.
`invention to
`[0030]
`It is another object of the present
`provide a method for simulating the look of actual automo-
`bile customization using an image of a modification or
`accessory and a current image of the potential buycr’s own
`car.
`
`invention to
`It is another object of the present
`[0031]
`provide a method for color correction which makes images
`of items which may be purchased more representative of
`their appearance in a particular sctting.
`RELATED PROVISIONAL APPLICATION
`
`[0032] The applicant claims priority from Provisional
`Patent Application No. 60/178,863 dated Jan. 29, 2000. Said
`
`PPA contains color versions of FIGS. 11-14 of this applica-
`tion and color prior art from: www.victoriassecret.com,
`www.hairdos.com, www.tswnet.com, www.americanhome-
`decorating.com, and www.dulux.com.
`
`SUMMARYOF THE INVENTION
`
`an exemplary preferred
`accordance with
`In
`[0033]
`embodimentof the present invention, virtual picture hang-
`ing, a method adapted to: display an image of one or more
`pieces of artwork which the viewer may be interested in
`buying and displaying such an image with an image of a
`room where it might be placed or mounted.
`
`It is further shown how the same approach can be
`[0034]
`used for other applications, such as virtual paint matching
`and virtual dress up dolls.
`
`In another aspect of the current invention, the same
`[0035]
`methods can be employed for virtual furniture arrangement
`by simply substituting images of furniture for images of
`artwork.
`
`In another aspect of the current invention, the same
`[0036]
`methods can be employed for virtual windowtreatments by
`simply substituting images of drapes or blinds for images of
`artwork.
`
`In another aspect of the current invention, similar
`[0037]
`methods can be employedfor virtual paint samplesor carpet
`samples.
`
`In another aspect of the current invention, the same
`[0038]
`methods can be employed for virtual dress up dolls, by
`substituting images of the potential clothing purchasers for
`images of rooms where artwork will be hung and by
`substituting images of clothing for images of artwork.
`
`In another aspect of the current invention, methods
`[0039]
`similar to those used in virtual picture hanging can be
`adapted for automotive accessories.
`
`In another aspect of the current invention, methods
`[0040]
`similar to those used in virtual picture hanging can be
`adapted for displaying items which change over time, such
`as landscaping, with an image supplied bythe user.
`
`In another aspectof the current invention, methods
`[0041]
`similar to those used in virtual picture hanging can be
`adapted for “morphing” between an image supplicd by the
`user and an image supplied by the website.
`
`DESCRIPTION OF THE DRAWINGS
`
`features and advantages of the
`[0042] Other objects,
`wvention will becomereadily apparent upon referenceto the
`following detailed description when considered in conjunc-
`tion with the accompanying drawings, in whichlike refer-
`ence numerals designate like parts throughout
`the FIGs.
`thereof, and wherein:
`
`[0043] FIG. 1 is a high level, functional flowchart of an
`exemplary preferred system according to the present inven-
`tion.
`
`FIG.2 is a flowchart showing methodsof getting
`[0044]
`digital images of artwork to the website, correcting color and
`size, and attaching relevant data regarding images.
`
`Zynga Ex. 1016, p. 17
`Zynga v. IGT
`IPR2022-00199
`
`Zynga Ex. 1016, p. 17
` Zynga v. IGT
` IPR2022-00199
`
`

`

`US 2001/0034668 Al
`
`Oct. 25, 2001
`
`FIG.3 is a flowchart showing methods of getting
`[0045]
`digital images of a room where artwork might be placed into
`the website.
`
`[0046] FIG. 4 is a flowchart showing a method for dis-
`playing one or more images of artwork with a room image.
`
`FIG.5 is a flowchart showing a method for decid-
`[0047]
`ing whether to purchase artwork displayed with a room
`image.
`
`[0048] FIG. 6 is a flowchart showing the use of color
`and/or pattern replacement for simulating paint, carpet,tile,
`flooring, wallpaper and flooring.
`
`[0049] EIG.7 is a flowchart showing a method for decid-
`ing whether to purchase paint, carpet, tile, wallpaper and
`flooring.
`
`FIG.8 is a flowchart showing methodsof printing
`[0050]
`or emailing one or more images of candidate artwork with
`an image of a room where it might be placed.
`
`[0051] FIG. 9 is a chart showing required image inputs
`and sources for applications where one image is overlaid on
`another image.
`
`[0052] FIG. 10 is a chart showing which portions of a
`buyer’s image are modified, and in what manner, to simulate
`carpet, paint, wallpaper and other items which replace part
`of the current buyer’s room image.
`
`[0053] FIG. 11 is a sample screen imagefor displaying a
`single artwork overlaid on a background of a room image.
`
`[0054] FIG. 12 is a sample screen image for displaying
`multiple images of artwork overlaid on a background of a
`room image.
`
`[0055] FIG. 13 is a sample screen image for displaying a
`single artwork in a frame next to a room image.
`
`[0056] FIG. 14 is a sample screen image for displaying
`multiple images of artwork in frames next to a room image.
`
`DETAILED DESCRIPTION OF THE
`PREFERRED EMBODIMENTS
`
`[0057] The exemplary preferred embodiment, virtual pic-
`ture hanging, is adapted to: display one or more images of
`paintings or other artwork and an image of a room where a
`particular purchaser might hang said paintings or artwork;
`make color corrections which provide a more accurate
`portrayal of how a particular artwork would look in a
`particular setting; allow the user to sort through a numberof
`possible pieces of artwork to display; and provide methods
`for printing, or emailing displays of how artwork would look
`in a particular settings.
`
`[0058] Major Inputs and Outputs
`
`[0059] Referring to FIG. 1, a system 50, according to the
`present invention includes an internet site 101. The internet
`website 101 is programmed or modified via a keyboard or
`modem109, data is stored on a server or other data storage
`device, such as a hard drive, DVD drive, or floppy disk 103,
`and can be viewed via a monitor 104. The internet site 101
`is programmed using, an internet-compatible authoring soft-
`ware, such as html or Java, 105. The program for a specific
`internet site is in an html or Java program 107. Images of
`itemsfor sale are input via the seller’s digital camera 111, or
`
`the seller’s color scanner 113. For each item, it is often
`necessary to input other data onthe sellers or the items to be
`sold such as size, name of artwork, current owner, lighting
`conditions under which the picture was taken, or asking
`price 115.
`
`Information for the internet site 101 stored on the
`(0060]
`server 103 is accessed by users via a modem, cable modem,
`or local area network (LAN) 117 and an internet capable
`computer 119. Software running on the internet capable
`computer 119 includes an internet browser 127, such as
`Netscape Communicator or Microsoft Internet Explorer.
`Images of the internet user’s home, apartment, office, body,
`car, or other place where they might use an piece of artwork
`is input using the potential buyer’s digital camera 121 or the
`potential buyer’s color scanner 123. If the potential buyer
`docsnot have a digital camcra or scanner, images mightalso
`be input indirectly using someone else’s digital camera or
`scanner and sending the digital image to the potential buyer
`using a modem 139. The digital camera 121 does not need
`to be a still frame camera; a video camera can also provide
`the necessary images. A keyboard or modem 139, is also
`used to input other information on a potential buyer’s
`preferences 125. Such preferences might include size of art,
`desired price range, lighting in which artwork would be
`displayed, or shipping time.
`
`Images of artworks and one or more images of
`[0061]
`places where a purchaser might put the artwork are com-
`bined to create viewsof artwork on background(s) 129. Such
`viewsare displayed on a computer monitor 131 and maybe
`printed using a printer 133, or sent to someoneelse via email
`135. If the user decides to purchase a particular artwork,
`required purchase information for artwork 137 is exchanged.
`
`[0062] Other methodsof inputting programming data may
`be substituted for the keyboard or modem 109, such as
`internet downloads, or reading, from a floppy or DVD disk.
`Other methodsofstoring data maybe substitutedfor the data
`storage device 103, such as a compactdisk or magnetic tape.
`Other methods for outputting data may be substituted for the
`printer 121, such as a monitor, projector, plotter, or internet
`upload. Other methods of inputting images may be substi-
`tuted for digital cameras 111 and 121 or scanners 113 and
`123.
`
`[0063] The system 50 is further adapted to print, display,
`or save, as desired, a virtual picture hanging.
`
`[0064] Putting Digital Images of Artwork on the Website
`
`[0065] FIG. 2 shows the major steps in getting digital
`images of artwork onto a website.
`
`image of item 201 is typically
`[0066] A raw digital
`obtained using a seller’s digital camera 203, or a seller’s
`color scanner 205. This raw digital image 201 and informa-
`tion on sellers and items to be sold 209 are combined to
`create an image of the item with associated data 207. Such
`data might include size, name of artwork, current owner,
`lighting conditions under which the picture was taken, or
`asking price.
`
`[0067] Because of difficulties in matching colors caused
`by photography underdifferent lighting conditions, there is
`an optional step to see if the image is in the right spectrum
`211. Excessive red can be adjusted for using a color cor-
`rection to sunlight spectrum 213. Fluorescent, incandescent,
`
`Zynga Ex. 1016, p. 18
`Zynga v. IGT
`IPR2022-00199
`
`Zynga Ex. 1016, p. 18
` Zynga v. IGT
` IPR2022-00199
`
`

`

`US 2001/0034668 Al
`
`Oct. 25, 2001
`
`halogen, andotherartificial light sources affect the apparent
`colors of the artwork in different way when compared to
`sunlight. For example, if the photo of the artwork was taken
`under incandescentlighting, the artwork will appear to have
`more red and less blue than midday sunlight. While digital
`images can theoretically be digitally converted from their
`appearance under one light source directly to their appear-
`ance under another light source, it is simplest to bring all
`artwork to the spectrum of sunlight. Sunlight’s exact spec-
`tral values are very well-known and well-documented. Addi-
`tionally, if sunlight spectrum is the standard, all artists and
`owners theoretically have free access to using the standard
`light source to photograph their artwork. An assortment of
`color correction methods currently exist. Examples of such
`methods are the Correct Tint function in Microsoft Picture
`It! 2.0 software and the Hue Shift function in Paint Shop Pro
`6.0 from Jase Software. After any color correction to sun-
`light spectrum 213, proceed to resize the image for display
`215.
`
`If photos are taken under sunlight, or light sources
`[0068]
`very close to sunlight, the image is already in the proper
`spectrum. In that case, proceed to the next optional step,
`resizing the digital image for display on the website 215. The
`website can also decide to display images without size
`modifications and skip steps 215 and 217 for all images,
`proceeding directly to 219.
`
`[0069] There are two likely reasons for resizing. The first
`reason is that the website may wish to create “thumbnails”
`of many imagesin a consistentsize, regardless of the size of
`the original artwork. Thumbnail programs are common for
`display of many small images on a single page of internet
`content. The user will usually be able to click on a thumbnail
`and see a larger, more detailed image.
`
`[0070] The sccond reason is not commonin current inter-
`net practice, but is particularly useful for this embodiment.
`Digital images will come to the website in different resolu-
`tions and photos will be taken from different distances. The
`width of one pixel on a particular digital image may corre-
`spond to a centimeter on a particular artwork. One pixel on
`another digital image may correspond to half a centimeter,
`or two centimeters, on the real artwork. The website may
`wish to have images set to a particular scale, such as one
`pixel on the digital image corresponds to one centimeter on
`the real artwork. While calibrating sizes in this fashion is not
`commonfor internetdisplay, it is easily done with inexpen-
`sive image manipulation software. One example of a pro-
`gram which can conveniently resize images using several
`different methods is Paint Shop Pro 6.0 from Jasc Software.
`If no resizing is