(12) United States Patent
`(12) United States Patent
`Rhoads
`Rhoads
`
`(54)
`(54)
`
`(75)
`(75)
`
`(73)
`(73)
`
`* )
`
`(21)
`(21)
`(22)
`(22)
`
`(63)
`(63)
`
`(60)
`(60)
`
`(51)
`(51)
`(52)
`(52)
`(58)
`(58)
`
`(56)
`(56)
`
`LINKING OF COMPUTERS BASED ON
`LINKING OF COMPUTERS BASED ON
`OPTICAL SENSING OF DIGITAL DATA
`OPTICAL SENSING OF DIGITAL DATA
`
`Inventor: Geoffrey B. Rhoads, West Linn, OR
`Inventor: Geoffrey B. Rhoads, West Linn, OR
`(US)
`(US)
`Assignee: Digimarc Corporation, Tualatin, OR
`Assignee: Digimarc Corporation, Tualatin, OR
`(US)
`(US)
`Subject to any disclaimer, the term of this
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 0 days.
`U.S.C. 154(b) by 0 days.
`This patent is Subject to a terminal dis
`This patent is subject to a terminal dis-
`claimer.
`claimer.
`
`Notice:
`Notice:
`
`Appl. No.: 09/342,689
`Appl. No.: 09/342,689
`Filed:
`Jun. 29, 1999
`Filed:
`Jun. 29, 1999
`Related U.S. Application Data
`Related U.S. Application Data
`
`Continuation-in-part of application No. 09/130,624, filed on
`Continuation-in-part of application No. 09/130,624, filed on
`Aug. 6, 1998, which is a continuation of application No.
`Aug. 6, 1998, which is a continuation of application No.
`08/508,083, filed on Jul. 27, 1995, now Pat. No. 5,841,978,
`08/508,083, filed on Jul. 27, 1995, now Pat. No. 5,841,978,
`and a continuation-in-part of application No. 09/314.648,
`and a continuation-in-part of application No. 09/314,648,
`filed on May 19, 1999, which is a continuation-in-part of
`filed on May 19, 1999, which is a continuation-in-part of
`application No. 09/292,569, filed on Apr. 15, 1999.
`application No. 09/292,569, filed on Apr. 15, 1999.
`Provisional application No. 60/134,782, filed on May 19,
`Provisional application No. 60/134,782, filed on May 19,
`1999.
`1999.
`Int. Cl. .................................................... G06F 13/00
` GO6F 13/00
`Int. Cl.'
`709/217; 709/313; 380/4
`U.S. Cl.
` 709/217; 709/313; 380/4
`Field of Search ..................................... 709/217, 219,
`Field of Search
` 709/217, 219,
`709/227, 230, 250, 313, 328, 329; 380/4,
`709/227, 230, 250, 313, 328, 329; 380/4,
`9, 49
`9, 49
`
`References Cited
`References Cited
`
`U.S. PATENT DOCUMENTS
`U.S. PATENT DOCUMENTS
`8/1990 Gorog.
`8/1990 Gorog .
`
`4,947.028
`4,947,028
`
`IIIIIIII 111111111119111111 IIIII
`
`USOO6311214B1
`(10) Patent No.:
`US 6,311,214 B1
`US 6,311,214 B1
`(to) Patent No.:
`(45) Date of Patent:
`*Oct. 30, 2001
`(45) Date of Patent:
`*Oct. 30, 2001
`
`5,053,956
`5,053,956
`5,262,860
`5,262,860
`5,288,976
`5,288,976
`5,385,371
`5,385,371
`5,463,209
`5,463,209
`5,495,581
`5,495,581
`
`10/1991 Donald et al..
`10/1991 Donald et al. .
`11/1993 Fitzpatricket al..
`11/1993 Fitzpatrick et al. .
`2/1994 Citron et al. .
`2/1994 Citron et al. .
`1/1995 Izawa.
`1/1995 Izawa .
`10/1995 Figh et al..
`10/1995 Figh et al. .
`2/1996 Tsai .
`2/1996 Tsai .
`(List continued on next page.)
`(List continued on next page.)
`FOREIGN PATENT DOCUMENTS
`FOREIGN PATENT DOCUMENTS
`0493091
`7/1992 (EP).
`0493091
`7/1992 (EP) .
`WO95/14289
`5/1995 (WO).
`W095/14289
`5/1995 (WO)
`.
`WO95/20291
`7/1995 (WO).
`W095/20291
`7/1995 (WO) .
`WO96/27259
`9/1996 (WO).
`W096/27259
`9/1996 (WO) .
`WO96/36163
`11/1996 (WO).
`W096/36163
`11/1996 (WO) .
`WO97/02522
`1/1997 (WO).
`W097/02522
`1/1997 (WO) .
`WO97/43736
`11/1997 (WO).
`W097/43736
`11/1997 (WO) .
`WO98/03923
`1/1998 (WO).
`W098/03923
`1/1998 (WO) .
`WO98/51036
`11/1998 (WO).
`W098/51036
`11/1998 (WO) .
`WO99/57623
`11/1999 (WO).
`W099/57623
`11/1999 (WO) .
`OTHER PUBLICATIONS
`OTHER PUBLICATIONS
`U.S. application No. 60/000,442, Hudetz, filed Jun. 22,
`U.S. application No. 60/000,442, Hudetz, filed Jun. 22,
`1995.
`1995.
`
`(List continued on next page.)
`(List continued on next page.)
`Primary Examiner Viet D. Vu
`Primary Examiner—Viet D. Vu
`(74) Attorney, Agent, or Firm William Y. Conwell;
`(74) Attorney, Agent, or Firm—William Y. Conwell;
`Digimarc Corporation
`Digimarc Corporation
`(57)
`ABSTRACT
`(57)
`ABSTRACT
`A printed object, Such as an item of postal mail, a book,
`A printed object, such as an item of postal mail, a book,
`printed advertising, a business card, product packaging, etc.,
`printed advertising, a business card, product packaging, etc.,
`is Steganographically encoded with plural-bit data. When
`is steganographically encoded with plural-bit data. When
`Such an object is presented to an optical Sensor, the plural-bit
`such an object is presented to an optical sensor, the plural-bit
`data is decoded and used to establish a link to an internet
`data is decoded and used to establish a link to an internet
`address corresponding to that object.
`address corresponding to that object.
`
`23 Claims, 2 Drawing Sheets
`23 Claims, 2 Drawing Sheets
`
`10
`10
`
`
`
`
`
`be-doop
`
`NETWORK
`
`•
`
`22
`
`COMPUTER
`COMPUTER
`—
`1
`MEMORY
`
`- | MEMORY
`
`18
`18
`
`16
`
`12
`12
`
`
`
`14
`
`24
`
`2O
`20
`
`DISPLAY
`DISPLAY
`
`26
`26 —)
`
`DISH-Blue Spike-408
`Exhibit 1016, Page 0001
`
`

`

`US 6,311,214 B1
`US 6,311,214 B1
`Page 2
`Page 2
`
`U.S. PATENT DOCUMENTS
`U.S. PATENT DOCUMENTS
`
`5,496,071
`Walsh.
`3/1996
`3/1996 Walsh .
`5,496,071
`Meske, Jr. et al. .
`5,530,852
`6/1996
`6/1996 Meske, Jr. et al. .
`5,530,852
`Cooperman et al. .
`5,613,004
`3/1997
`3/1997 Cooperman et al. .
`5,613,004
`5,640,193
`Wellner.
`6/1997
`6/1997 Wellner .
`5,640,193
`5,659,164
`Schmid et al. .
`8/1997
`8/1997
`Schmid et al. .
`5,659,164
`5,673,316
`Auerbach et al. .
`9/1997
`.
`9/1997
`Auerbach et al.
`5,673,316
`5,721,788
`Powell et al. .
`2/1998
`2/1998
`Powell et al. .
`5,721,788
`Wagner.
`5,742,845
`4/1998
`4/1998 Wagner .
`5,742,845
`5,761,606
`WolZien.
`6/1998
`6/1998 Wolzien .
`5,761,606
`Bloomberg.
`5,761,686
`6/1998
`6/1998 Bloomberg .
`5,761,686
`Hidary et al..
`5,774,664
`6/1998
`6/1998 Hidary et al. .
`5,774,664
`5,774,666
`Portuesi.
`6/1998
`6/1998 Portuesi .
`5,774,666
`5,778,102
`Sandford, II et al. .
`7/1998
`7/1998 Sandford, II et al. .
`5,778,102
`Cragun et al. .
`5,804,803
`9/1998
`9/1998
`Cragun et al. .
`5,804,803
`Kogan et al. .
`5,809,317
`9/1998
`9/1998
`Kogan et al. .
`5,809,317
`5,818,441
`Throckmorton et al. .
`10/1998
`10/1998
`Throckmorton et al. .
`5,818,441
`5,822,432
`Moskowitz et al. .
`10/1998
`10/1998 Moskowitz et al. .
`5,822,432
`5,838,458
`Tsai .
`11/1998
`11/1998 Tsai .
`5,838,458
`5,848,413 *
`Wolff ...................................... 707/10
`12/1998
`707/10
`5,848,413 * 12/1998 Wolff
`5,857.038
`1/1999 Owada et al. .
`Owada et al. .
`1/1999
`5,857,038
`5,872,589
`Morales.
`2/1999
`5,872,589
`2/1999 Morales .
`5,892,900
`Ginter et al. .
`4/1999
`5,892,900
`4/1999 Ginter et al. .
`5,900,608
`5/1999
`Iida.
`5,900,608
`5/1999 Iida .
`5,903,729
`5/1999
`Reber et al. .
`5,903,729
`5/1999 Reber et al. .
`5,905,248 *
`5/1999
`Russell et al. .................. 235/462.27
`235/462.27
`5,905,248 *
`5/1999
`Russell et al.
`5,913.210
`Call.
`6/1999
`5,913,210
`6/1999
`Call .
`5,915,027
`Cox et al. .
`6/1999
`5,915,027
`6/1999 Cox et al. .
`5,918,214
`Perkowski.
`6/1999
`5,918,214
`6/1999 Perkowski .
`5,932,863
`Rathus et al. .
`8/1999
`5,932,863
`8/1999 Rathus et al. .
`5,933,829
`Durst et al. .
`8/1999
`5,933,829
`8/1999 Durst et al. .
`5,938,726
`Reber et al. .
`8/1999
`5,938,726
`8/1999 Reber et al. .
`5,940,595
`Reber et al. .
`8/1999
`5,940,595
`8/1999 Reber et al. .
`5,978.773
`Hudetz et al. .
`11/1999
`5,978,773
`11/1999 Hudetz et al. .
`5,986,651
`Reber et al. .
`11/1999
`5,986,651
`11/1999 Reber et al. .
`6,012,102
`Shachar.
`1/2000
`6,012,102
`1/2000 Shachar .
`6,052,486
`Knowlton et al. .
`4/2000
`6,052,486
`4/2000 Knowlton et al. .
`
`6,081,827
`6,081,827
`6,098,106
`6,098,106
`6,108,656
`6,108,656
`6,148,331
`6,148,331
`
`6/2000 Reber et al. .
`6/2000 Reber et al. .
`8/2000 Philyaw et al..
`8/2000 Philyaw et al. .
`8/2000. Durst et al. .
`8/2000 Durst et al. .
`11/2000 Parry.
`11/2000 Parry .
`OTHER PUBLICATIONS
`OTHER PUBLICATIONS
`IBM Technical Disclosure Bulletin 96A 61092, published
`IBM Technical Disclosure Bulletin 96A 61092, published
`Jan. 1, 1996.
`Jan. 1, 1996.
`“Distributing Uniform Resource Locators as Bar Code
`"Distributing Uniform Resource Locators as Bar Code
`Images.” IBM Technical Disclosure Bulletin, No. 39, No. 1,
`Images," IBM Technical Disclosure Bulletin, No. 39, No. 1,
`pp. 167–168, 96A 60059, published Jan. 1, 1996.
`pp. 167-168, 96A 60059, published Jan. 1, 1996.
`Frequently Asked Questions about Digimarc Signature
`Frequently Asked Questions about Digimarc Signature
`Technology, Aug. 1, 1995, 9 pages.
`Technology, Aug. 1, 1995, 9 pages.
`Bartlett, et al., “An Overview of HighWater FBI Technol
`Bartlett, et al., "An Overview of HighWater FBI Technol-
`ogy, Posted on Internet Mar. 22, 1996, 12 pages.
`ogy," Posted on Internet Mar. 22, 1996, 12 pages.
`Digimarc presentation at RSA Conference, Jan. 1996, 4
`Digimarc presentation at RSA Conference, Jan. 1996, 4
`pages.
`pageS.
`“Digital Watermarks What Are They'?” Digimarc Corpora
`"Digital Watermarks What Are They?" Digimarc Corpora-
`tion, 1997.
`tion, 1997.
`Seybold Report on Internet Publishing, vol. 1, No. 4, Dec.
`Seybold Report on Internet Publishing, vol. 1, No. 4, Dec.
`1996.
`1996.
`Seybold Report on Publishing Systems, vol. 25, No. 6, 1996.
`Seybold Report on Publishing Systems, vol. 25, No. 6, 1996.
`Bethoney, “A Lasting Way For Artists To Leave Their
`Bethoney, "A Lasting Way For Artists To Leave Their
`Mark, PCWeek, Dec. 11, 1996.
`Mark," PCWeek, Dec. 11, 1996.
`Digital Media Monthly, Aug., 1996 (excerpt re Highwater
`Digital Media Monthly, Aug., 1996 (excerpt re Highwater
`FBI).
`FBI).
`Digimarc Press Releases (various), 1996-1998.
`Digimarc Press Releases (various), 1996-1998.
`Simone, “A Digita Watermark for Images," PC Magazine
`Simone, "A Digita Watermark for Images," PC Magazine
`Dec. 18, 1996.
`Dec. 18, 1996.
`“Copyright Protection for Digital Images, Digital Finger
`"Copyright Protection for Digital Images, Digital Finger-
`printing from FBI,” Highwater FBI brochure, 1995, 4 pages.
`printing from FBI," Highwater FBI brochure, 1995, 4 pages.
`“Highwater FBI Limited Presentation, Image Copyright
`"Highwater FBI Limited Presentation, Image Copyright
`Protection Software,” FBI Ltd brochure, Jul. 1995, 17 pages.
`Protection Software," FBI Ltd brochure, Jul. 1995, 17 pages.
`* cited by examiner
`* cited by examiner
`
`DISH-Blue Spike-408
`Exhibit 1016, Page 0002
`
`

`

`U.S. Patent
`U.S. Patent
`
`Oct. 30, 2001
`Oct. 30, 2001
`
`Sheet 1 of 2
`Sheet 1 of 2
`
`US 6,311,214 B1
`US 6,311,214 B1
`
`1O
`
`NETWORK
`
`18
`18
`
`16
`
`
`
`
`
`20
`20
`
`12
`
`14
`
`or be-doop or
`
`/
`
`\
`
`
`'---
`
`22
`
`COMPUTER
`COMPUTER
`
`- | MEMORY
`
`1 MEMORY
`
`'•- 24
`
`V
`DISPLAY
`DISPLAY
`
`26 -)
`26
`
`FIG. 1
`FIG. 1
`
`DISH-Blue Spike-408
`Exhibit 1016, Page 0003
`
`

`

`U.S. Patent
`U.S. Patent
`
`Oct. 30, 2001
`Oct. 30, 2001
`
`Sheet 2 of 2
`Sheet 2 of 2
`
`US 6,311,214 B1
`US 6,311,214 B1
`
`- COMPUTER
`
`COMPUTER
`
`(-- 24
`24
`
`- 28
`or— 28
`
`BEDOOP
`BEDOOP
`DECODER
`DECODER
`2 J
`32
`
`V
`BEDOOP
`BEDOOP
`REGISTRY
`REGISTRY
`OS X34
`O.S
`34
`33-7
`33 _}
`
`OTHER
`| MAGING
`1
`IMAGING
`PROGRAMs
`' PROGRAMS
` J
`L
`
`a
`
`r
`
`i_.
`
`- -
`
`H =T
`
`H BEDOOP AP
`BEDOOP AP
`H BEDOOP AP
`H BEDOOP AP
`40
`40
`36
`
`- - - - - - - 36
`H BEDOOP AP FP.- BROWSER
`— —+
`BEDOOP AP
`BROWSER
`BEDOOPAP
`BEDOOP AP
`
`30
`30
`
`•
`
`•
`
`•
`
`+
`
`MASTER
`MASTER
`REGISTRATION
`REGISTRATION
`SERVER
`SERVER
`
`48 b
`(- 48b
`r48cJ.
`48C
`+
`CLASS
`CLASS
`CLASS
`CLASS
`CLASS
`CLASS
`SERVER
`SERVER
`SERVER
`SERVER SERVER SERVER
`
`•
`
`•
`
`•
`
`+
`
`48a
`48a
`
`46
`46
`Y
`CLASS
`CLASS
`SERVER
`SERVER
`
`•
`
`•
`
`•
`
`r 58
`58
`FURTHER
`FURTHER
`SERVER
`SERVER
`- 59
`59
`
`
`
`50
`\,.._ • • •
`
`__
`DNS
`
` +
`DNS
`
`DNS
`
`•
`
`•
`
`•
`
`DNS
`
`•
`
`•
`
`•
`
`•
`
`•
`
`•
`
`DNS
`
`DNS
`
`+
`DNS
`
`•
`
`•
`
`+
`DNS
`
`•
`
`•
`
`•
`
`1
`1
`
`•
`
`.
`
`FIG. 2
`FIG. 2
`
`DNS LEAF H - - - - - -
`DNS LEAF
`NODE SERVER
`NODE SERVER
`56
`
`DISH-Blue Spike-408
`Exhibit 1016, Page 0004
`
`

`

`RELATED APPLICATION DATA
`RELATED APPLICATION DATA
`This application is a continuation-in-part of copending
`This application is a continuation-in-part of copending
`application Ser. No. 09/130,624, filed Aug. 6, 1998, which
`application Ser. No. 09/130,624, filed Aug. 6, 1998, which
`is a continuation of application Ser. No. 08/508,083 filed on
`is a continuation of application Ser. No. 08/508,083 filed on
`Jul. 27, 1995, (now U.S. Pat. No. 5,841,978). This applica
`Jul. 27, 1995, (now U.S. Pat. No. 5,841,978). This applica-
`tion is also a continuation-in part of copending application
`tion is also a continuation-in part of copending application
`Ser. No. 09/314,648, filed May 19, 1999 (attached as Appen
`Ser. No. 09/314,648, filed May 19, 1999 (attached as Appen-
`dix A). This application is also a continuation-in-part of
`dix A). This application is also a continuation-in-part of
`copending provisional application 60/134,782, also filed
`copending provisional application 60/134,782, also filed
`May 19, 1999 (attached as Appendix B). This application is
`May 19, 1999 (attached as Appendix B). This application is
`also a continuation-in-part of copending application Ser. No.
`also a continuation-in-part of copending application Ser. No.
`09/292,569, filed Apr. 15, 1999, which claims priority to
`09/292,569, filed Apr. 15, 1999, which claims priority to
`application Ser. No. 60/082,228, filed Apr. 16, 1998.
`application Ser. No. 60/082,228, filed Apr. 16, 1998.
`
`FIELD OF THE INVENTION
`FIELD OF THE INVENTION
`The present invention relates optical user interfaces that
`The present invention relates optical user interfaces that
`Sense digitally-encoded objects. The invention further
`sense digitally-encoded objects. The invention further
`relates to Systems using Such optical interfaces to control
`relates to systems using such optical interfaces to control
`computers, and to navigate over or act as portals on net
`computers, and to navigate over or act as portals on net-
`WorkS.
`works.
`
`35
`35
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`BRIEF DESCRIPTION OF THE DRAWINGS
`FIG. 1 is a block diagram showing one embodiment of the
`FIG. 1 is a block diagram showing one embodiment of the
`present invention.
`present invention.
`FIG. 2 is another block diagram showing an embodiment
`FIG. 2 is another block diagram showing an embodiment
`of the present invention.
`of the present invention.
`
`40
`40
`
`1
`1
`LINKING OF COMPUTERS BASED ON
`LINKING OF COMPUTERS BASED ON
`OPTICAL SENSING OF DIGITAL DATA
`OPTICAL SENSING OF DIGITAL DATA
`
`US 6,311,214 B1
`US 6,311,214 B1
`
`2
`2
`disclosure concentrates on objects which have Some form of
`disclosure concentrates on objects which have some form of
`digital message attached thereto, Some aspects of the inven
`digital message attached thereto, some aspects of the inven-
`tion may apply to objects which have no Such thing, where
`tion may apply to objects which have no such thing, where
`the prior arts of pattern recognition and gestural input can be
`the prior arts of pattern recognition and gestural input can be
`borrowed in combination with this invention to effect yet a
`5 borrowed in combination with this invention to effect yet a
`broader array of applications.
`broader array of applications.
`"Bedoop.” The Sound that a refrigerator might make,
`"Bedoop." The sound that a refrigerator might make,
`outfitted with a simple camera/processor unit/net
`outfitted with a simple camera/processor unit/net
`connection, as the ten year old holds up the empty milk
`connection, as the ten year old holds up the empty milk
`carton and a ping goes out to the local grocery Store, adding
`10 carton and a ping goes out to the local grocery store, adding
`the item to an accumulating delivery list. The Sound that
`the item to an accumulating delivery list. The sound that
`might be heard echoing over and over inside Internet cafes
`might be heard echoing over and over inside Internet cafes
`as heretofore computerphobes take their first skeptical Steps
`as heretofore computerphobes take their first skeptical steps
`onto the world wide web. The Sound heard at the fast food
`onto the world wide web. The sound heard at the fast food
`counter as the repeat customer holds up their Sandwich card
`15
`15 counter as the repeat customer holds up their sandwich card
`ticking off their latest meal, hoping for the Sirens to go off
`ticking off their latest meal, hoping for the sirens to go off
`for a S500 prize given to the lucky customer of the week.
`for a $500 prize given to the lucky customer of the week.
`Blue Sky Scenarios abound.
`Blue sky scenarios abound.
`This invention is therefore about powerful new user
`This invention is therefore about powerful new user
`interfaces to computers involving optical input. These new
`20 interfaces to computers involving optical input. These new
`user interfaces extend into the everyday World in ways that
`user interfaces extend into the everyday world in ways that
`a mouse and keyboard never could. By enabling everyday
`a mouse and keyboard never could. By enabling everyday
`objects to communicate their identities and functions to
`objects to communicate their identities and functions to
`ever-attendant devices, not only will the world wide web be
`ever-attendant devices, not only will the world wide web be
`25
`given an entirely new dimension, but basic home and office
`25 given an entirely new dimension, but basic home and office
`BACKGROUND AND SUMMARY OF THE
`BACKGROUND AND SUMMARY OF THE
`computing may be in Store for Some fundamental advances
`computing may be in store for some fundamental advances
`INVENTION
`INVENTION
`as well.
`as well.
`“Bedoop.” That might be the sound that someone might
`"Bedoop." That might be the sound that someone might
`These and a great many other features of the present
`These and a great many other features of the present
`hear as they lazily place a magazine advertisement in front
`hear as they lazily place a magazine advertisement in front
`invention will be more readily apparent from the following
`30 invention will be more readily apparent from the following
`of their desktop camera. Magically, the marketing and Sales
`of their desktop camera. Magically, the marketing and sales
`detailed description, which proceeds with reference to the
`detailed description, which proceeds with reference to the
`Web Site associated with the ad is displayed on their com
`accompanying drawings.
`web site associated with the ad is displayed on their com-
`accompanying drawings.
`puter. More information? Want to buy now? Look at the full
`puter. More information? Want to buy now? Look at the full
`product line'? No problem.
`product line? No problem.
`“Bedoop.” That might be the same sound when that same
`"Bedoop." That might be the same sound when that same
`Someone places their credit card in front of their desktop
`someone places their credit card in front of their desktop
`camera. Instantly, the product displayed on the web page is
`camera. Instantly, the product displayed on the web page is
`purchased. Behind the Scenes, a Secure purchase link is
`purchased. Behind the scenes, a secure purchase link is
`initiated, transmitting all requisite information to the Vendor.
`initiated, transmitting all requisite information to the vendor.
`Twist the credit card clockwise and the purchaser chooses
`Twist the credit card clockwise and the purchaser chooses
`overnight delivery.
`overnight delivery.
`So goes an exemplary embodiment of the invention
`So goes an exemplary embodiment of the invention
`further described in this application. Though this example is
`further described in this application. Though this example is
`rather Specific, it nevertheless alludes to an indescribably
`rather specific, it nevertheless alludes to an indescribably
`vast array of applications possible when a digital camera or
`vast array of applications possible when a digital camera or
`other optical Sensing device is turned into a general purpose
`other optical sensing device is turned into a general purpose
`user interface device with an intuitive power that very well
`user interface device with an intuitive power that very well
`might rival the mouse and the keyboard.
`might rival the mouse and the keyboard.
`The centerpiece of the invention is that an object or paper
`The centerpiece of the invention is that an object or paper
`product So-Scanned contains digital information that can be
`product so-scanned contains digital information that can be
`quickly read and acted upon by an appropriately configured
`quickly read and acted upon by an appropriately configured
`device, computer or appliance. The preferred embodiment
`device, computer or appliance. The preferred embodiment
`envisions that this digital information is aesthetically hidden
`envisions that this digital information is aesthetically hidden
`on objects. These objects have been previously and pro
`on objects. These objects have been previously and pro-
`actively marked with the digital information, using any of
`actively marked with the digital information, using any of
`the broad ranges of printing and processing techniques
`the broad ranges of printing and processing techniques
`which are available on the market and which are widely
`which are available on the market and which are widely
`described in the open literature and patent literature Sur
`described in the open literature and patent literature sur-
`rounding digital watermarking.
`rounding digital watermarking.
`Be this as it may, though the invention concentrates on flat
`Be this as it may, though the invention concentrates on flat
`object applications wherein the digital information is often
`object applications wherein the digital information is often
`imperceptibly integrated into the object, it is certainly not
`imperceptibly integrated into the object, it is certainly not
`meant to be So limited. Objects can be three dimensional in
`meant to be so limited. Objects can be three dimensional in
`nature and the information more visually overt and/or pre
`nature and the information more visually overt and/or pre-
`existing (i.e., not "pro-actively embedded, or not even be
`existing (i.e., not "pro-actively" embedded, or not even be
`“digital, per se). Different implementation considerations
`"digital," per se). Different implementation considerations
`attach to these variants. Likewise, though the bulk of this
`attach to these variants. Likewise, though the bulk of this
`
`DETAILED DESCRIPTION
`DETAILED DESCRIPTION
`Basically, the technology detailed in this disclosure may
`Basically, the technology detailed in this disclosure may
`be regarded as enhanced Systems by which users can interact
`be regarded as enhanced systems by which users can interact
`with computer-based devices. Their simple nature, and
`with computer-based devices. Their simple nature, and
`adaptability for use with everyday objects (e.g., milk
`45 adaptability for use with everyday objects (e.g., milk
`45
`cartons), makes the disclosed technology well Suited for
`cartons), makes the disclosed technology well suited for
`countleSS applications.
`countless applications.
`Due to the great range and variety of Subject matter
`Due to the great range and variety of subject matter
`detailed in this disclosure, an orderly presentation is difficult
`detailed in this disclosure, an orderly presentation is difficult
`to achieve. AS will be evident, many of the topical Sections
`50 to achieve. As will be evident, many of the topical sections
`50
`presented below are both founded on, and foundational to,
`presented below are both founded on, and foundational to,
`other Sections. For want of a better rationale, the Sections are
`other sections. For want of a better rationale, the sections are
`presented below in a more or less random order. It should be
`presented below in a more or less random order. It should be
`recognized that both the general principles and the particular
`recognized that both the general principles and the particular
`details from each Section find application in other Sections as
`55 details from each section find application in other sections as
`55
`well. To prevent the length of this disclosure from balloon
`well. To prevent the length of this disclosure from balloon-
`ing out of control, the various permutations and combina
`ing out of control, the various permutations and combina-
`tions of the features of the different Sections are not exhaus
`tions of the features of the different sections are not exhaus-
`tively detailed. The inventors intend to explicitly teach such
`tively detailed. The inventors intend to explicitly teach such
`combinations/permutations, but practicality requires that the
`60 combinations/permutations, but practicality requires that the
`60
`detailed synthesis be left to those who ultimately implement
`detailed synthesis be left to those who ultimately implement
`Systems in accordance with Such teachings.
`systems in accordance with such teachings.
`Basic Principles-Refrigerators and Clutter
`Basic Principles—Refrigerators and Clutter
`Referring to FIG. 1, a basic embodiment 10 of the present
`Referring to FIG. 1, a basic embodiment 10 of the present
`invention includes an optical Sensor 12, a computer 14, and
`65 invention includes an optical sensor 12, a computer 14, and
`65
`a network connection 16 to the internet 18. The illustrated
`a network connection 16 to the internet 18. The illustrated
`optical Sensor 12 is a digital camera having a resolution of
`optical sensor 12 is a digital camera having a resolution of
`
`DISH-Blue Spike-408
`Exhibit 1016, Page 0005
`
`

`

`US 6,311,214 B1
`US 6,311,214 B1
`
`3
`4
`3
`4
`When the page was originally printed, tiny droplets of ink
`320 by 200 pixels (color or black and white) that stares out,
`When the page was originally printed, tiny droplets of ink
`320 by 200 pixels (color or black and white) that stares out,
`grabbing frames of image data five times per Second and
`or toner were distributed across the paper in a pattern So light
`grabbing frames of image data five times per second and
`or toner were distributed across the paper in a pattern so light
`Storing Same in one or more frame buffers. These frames of
`as to be essentially un-noticeable, but which Steganographi
`storing same in one or more frame buffers. These frames of
`as to be essentially un-noticeable, but which steganographi-
`image data are analyzed by a computer 14 for the presence
`cally encoded the page with a plural-bit binary number (e.g.,
`image data are analyzed by a computer 14 for the presence
`cally encoded the page with a plural-bit binary number (e.g.,
`of Bedoop data. (Essentially, Bedoop data is any form of
`64 bits). A database (e.g., maintained by the operating
`of Bedoop data. (Essentially, Bedoop data is any form of
`5 64 bits). A database (e.g., maintained by the operating
`digital data encoding recognized by the System 10-data
`digital data encoding recognized by the system 10—data
`System, the Excel program, the printer driver, etc.) stored
`system, the Excel program, the printer driver, etc.) stored
`which, in many embodiments, initiates Some action.) Once
`which, in many embodiments, initiates some action.) Once
`part of this number (e.g., 24bits, termed a Univeral Identifier
`part of this number (e.g., 24 bits, termed a Univeral Identifier
`detected, the System responds in accordance with the
`detected, the system responds in accordance with the
`or UID) in association with the path and file name at which
`or UID) in association with the path and file name at which
`detected Bedoop data (e.g., by initiating Some local action,
`detected Bedoop data (e.g., by initiating some local action,
`the electronic version of the file was Stored, the page number
`the electronic version of the file was stored, the page number
`or by communication with a remote computer, Such as over
`or by communication with a remote computer, such as over
`within the document, and other useful information (e.g.,
`10 within the document, and other useful information (e.g.,
`the internet, via an online Service Such as AOL, or using
`the internet, via an online service such as AOL, or using
`author of the file, creation date, etc.).
`author of the file, creation date, etc.).
`point-to-point dial-up communications, as with a bulletin
`point-to-point dial-up communications, as with a bulletin
`The Steganographic encoding of the document, and the
`The steganographic encoding of the document, and the
`board System.
`board system.
`updating of the database, can be performed by the Software
`updating of the database, can be performed by the software
`Consider the milk carton example. The artwork on a milk
`Consider the milk carton example. The artwork on a milk
`application (e.g., Excel). This option can be selected once by
`application (e.g., Excel). This option can be selected once by
`carton can be adapted to convey Bedoop data. In the
`carton can be adapted to convey Bedoop data. In the
`the user and applied thereafter to all printed documents (e.g.,
`15 the user and applied thereafter to all printed documents (e.g.,
`preferred embodiment, the Bedoop data is Steganographi
`15
`preferred embodiment, the Bedoop data is steganographi-
`by a user Selection on an "Options' drop-down menu), or
`by a user selection on an "Options" drop-down menu), or
`cally encoded (e.g., digitally watermarked) on the carton.
`cally encoded (e.g., digitally watermarked) on the carton.
`can be presented to the user as part of the Print dialog
`Numerous digital watermarking techniques are known-all
`can be presented to the user as part of the Print dialog
`Numerous digital watermarking techniques are known—all
`window and Selected (or not) for each print job.
`of which convey data in a hidden form (i.e., on human
`window and selected (or not) for each print job.
`of which convey data in a hidden form (i.e., on human
`When Such a printed page is later presented to the camera,
`inspection, it is not apparent that digitally encoded data is
`When such a printed page is later presented to the camera,
`inspection, it is not apparent that digitally encoded data is
`present). Exemplary techniques operate by Slightly changing
`the computer automatically detects the presence of the
`20 the computer automatically detects the presence of the
`present). Exemplary techniques operate by slightly changing
`the luminance, or contours, of Selected points on artwork or
`encoded data on the page, decodes Same, consults the
`the luminance, or contours, of selected points on artwork or
`encoded data on the page, decodes same, consults the
`text printed on the carton, or Splatter tiny droplets of ink on
`database to identify the file name/location/page correspond
`text printed on the carton, or splatter tiny droplets of ink on
`database to identify the file name/location/page correspond-
`the carton in a Seemingly random pattern. Each of these
`the carton in a seemingly random pattern. Each of these
`ing to the UID data, and opens the identified file to the
`ing to the UID data, and opens the identified file to the
`techniques has the effect of changing the local luminance at
`correct page (e.g., after launching Excel). This application is
`techniques has the effect of changing the local luminance at
`correct page (e.g., after launching Excel). This application is
`areas across the carton-luminance changes that can be
`areas across the carton—luminance changes that can be
`one of many "paper as portal” applications of the Bedoop
`25 one of many "paper as portal" applications of the Bedoop
`25
`detected by the computer 14 and decoded to extract the
`detected by the computer 14 and decoded to extract the
`technology.
`technology.
`encoded digital data. In the case of a milk carton, the data
`encoded digital data. In the case of a milk carton, the data
`The foregoing are but two of myriad applications of the
`The foregoing are but two of myriad applications of the
`may serve to identify the object as, e.g., a half gallon carton
`may serve to identify the object as, e.g., a half gallon carton
`technology detailed herein. In the following discussion a
`technology detailed herein. In the following discussion a
`of Alpenrose brand Skim milk.
`of Alpenrose brand skim milk.
`great many other applications are disclosed (some
`great many other applications are disclosed (some
`The FIG. 1 apparatus can be integrated into the door of a
`The FIG. 1 apparatus can be integrated into the door of a
`groundbreaking, a few gimmicky). However, regardless of
`30 groundbreaking, a few gimmicky). However, regardless of
`refrigerator and used to compile a shopping list. Milk
`refrigerator and used to compil