(12) United States Patent
`Rhoads
`
`1111111111111111111111111111111121111111111111111111111110111111
`US 6,286,036 B1
`(to) Patent No.:
`(45) Date of Patent:
`*Sep. 4, 2001
`
`AUDIO- AND GRAPHICS-BASED LINKING
`TO INTERNET
`
`Inventor: Geoffrey B. Rhoads, West Linn, OR
`(US)
`
`Assignee: Digimarc Corporation, Tualatin, OR
`(US)
`
`Notice:
`
`Subject to any disclaimer, the term of this
`patent is extended or adjusted under 35
`U.S.C. 154(b) by 0 days.
`
`This patent is subject to a terminal dis-
`claimer.
`
`(21) Appl. No.: 09/464,307
`
`(22) Filed:
`
`Dec. 15, 1999
`
`Related U.S. Application Data
`
`(62) Division of application No. 09/130,624, filed on Aug. 6,
`1998, which is a continuation of application No. 08/508,083,
`filed on Jul. 27, 1995, now Pat. No. 5,841,978.
`
` GO6F 13/00
`Int. C1.7
`(51)
` 709/217; 709/313; 380/4
`(52) U.S. Cl.
` 709/217, 219,
`(58) Field of Search
`709/227, 313, 321; 707/10; 380/4, 28
`
`(56)
`
`References Cited
`
`OTHER PUBLICATIONS
`
`"Multimedia Mixed Object Envelopes Supporting a Gradu-
`ated Fee Scheme via Encryption," IBM Technical Disclo-
`sure Bulletin: vol. 37, No. 03, Mar. 1994, p. 413.
`Mintzer et al., "Safeguarding Digital library Contents and
`Users Digital Watermarking," D—Lib Magazine, Dec.1997:
`ISSN 1082-9873.
`"Copyright Protection for Digital Images, Digital Finger-
`printing from FBI," Highwater FBI brochure, 1995, 4 pages.
`"High Water FBI Limited Presentation Image Copyright
`Protection Software," FBi Ltd brochure, Jul., 1995, 17
`pages.
`U.S. application No. 60/000,442, Hudetz, filed Jun. 20,
`1995, Method and Apparatus for Interfacing with Remote
`Computers.
`Dautzenberg, "Watermarking Images," Department of
`Microelectronics and Electrical Engineering, Trinity Col-
`lege Dublin, 47 pages, Oct. 1994.
`Bruckstein, A. M.; Richardson, T. J., A holographic trans-
`form domain image watermarking method, Circuits, Sys-
`tems, and Signal Processing vol. 17, No. 3 pp. 361-389,
`1998. This paper includes an appendix containing an internal
`memo of Bell Labs, which according to the authors of the
`paper, was dated Sep. 1994.
`Szepanski, "A Signal Theoretic Method for Creating Forg-
`ery—Proof Documents for Automatic Verification," Proceed-
`ings 1979 Carnahan Conference on Crime Countermeasures,
`May 16, 1979, pp. 101-109.
`
`U.S. PATENT DOCUMENTS
`
`(List continued on next page.)
`
`3,845,391
`3,984,624
`4,238,849
`
`10/1974 Crosby .
`10/1976 Waggener .
`12/1980 Gassmann .
`
`(List continued on next page.)
`
`FOREIGN PATENT DOCUMENTS
`
`493091
`94/27228
`95/14289
`95/20291
`96/27259
`96/36163
`
`1/1992 (EP) .
`11/1994 (WO) .
`5/1995 (WO) .
`7/1995 (WO) .
`9/1996 (WO) .
`11/1996 (WO) .
`
`Primary Examiner—Viet D. Vu
`(74) Attorney, Agent, or Firm Wiliam Y Conwell;
`Digimarc Corporation
`
`(57)
`
`ABSTRACT
`
`Address information is obtained from audio or from a
`graphic and used to link to the internet. The address infor-
`mation can be a literal URL, or may be an index to a
`database from which a corresponding URL can be obtained.
`
`8 Claims, 18 Drawing Sheets
`
`
`
`-f1002
`
`
`
`1006
`
`1012
`
`1008
`
`.
`
`WEB SITE
`DEVELOPMENT TOOL
`WWW SITE
`
`1004
`
`1006
`
`1010
`
`NFLE Ex. 1017 - Page 1
`
`NLFE 1017 - Page 1
`
`

`

`US 6,286,036 B1
`Page 2
`
`U.S. PATENT DOCUMENTS
`
`1/1982 Maxemchuk .
`4,313,197
`1/1983 Leventer et al. .
`4,367,488
`4/1983 Warner .
`4,379,947
`4/1983 Leventer et al. .
`4,380,027
`7/1983 Lundy et al. .
`4,395,600
`1/1984 Moses et al. .
`4,425,642
`7/1985 Lofberg
`4,528,588
`10/1985 Greenberg .
`4,547,804
`6/1987 Hustig et al. .
`4,672,605
`6/1987 Tetrick et al. .
`4,675,746
`6/1988 Bliithgen .
`4,750,173
`2/1989 Broughton et al. .
`4,807,031
`8/1989 Best .
`4,855,827
`12/1989 Earnest .
`4,888,798
`3/1990 Rushforth et al. .
`4,908,836
`8/1990 Gorog .
`4,947,028
`11/1990 O'Grady et al. .
`4,969,041
`12/1990 Shear .
`4,977,594
`6/1991 Johnson .
`5,023,907
`10/1991 Viyella .
`5,053,956
`4/1992 Gilhousen et al. .
`5,103,459
`5/1992 Durst .
`5,113,518
`8/1992 Corbin .
`5,138,712
`9/1992 Veldhuis et al. .
`5,146,457
`9/1993 DeJean et al. .
`5,243,423
`11/1993 Fitzpatrick .
`5,262,860
`2/1994 Citron .
`5,288,976
`6/1994 Preuss et al. .
`5,319,735
`1/1995 Izawa .
`5,385,371
`6/1995 Powers .
`5,428,731
`10/1995 Figh .
`5,463,209
`2/1996 Bfalogh .
`5,493,677
`2/1996 Ananda .
`5,495,411
`2/1996 Tsai .
`5,495,581
`3/1996 Walsh .
`5,496,071
`6/1996 Kunt et al. .
`5,524,933
`6/1996 Meske .
`5,530,852
`8/1996 Ananda .
`5,548,645
`9/1996 Ross .
`5,553,143
`11/1996 Plum .
`5,579,479
`5,612,943 * 3/1997 Moses et al.
`5,613,004 * 3/1997 Cooperman et al.
`5,629,980
`5/1997 Stefik .
`5,634,012
`5/1997 Stefik .
`5,636,276
`6/1997 Brugger .
`
`6/1997 Stefik .
`5,638,443
`6/1997 Wellner .
`5,640,193
`7/1997 Barton .
`5,646,997
`7/1997 Saito .
`5,646,999
`7/1997 Kawakami et al. .
`5,652,626
`7/1997 Peterson .
`5,652,714
`8/1997 Brouwer .
`5,657,462
`8/1997 Schmid .
`5,659,164
`9/1997 Sizer, II .
`5,663,766
`2/1998 Stefik .
`5,715,403
`2/1998 Powell.
`5,721,788
`3/1998 France .
`5,734,119
`6/1998 Erickson .
`5,765,152
`6/1998 Wolosewicz .
`5,774,452
`5,782,692 + 7/1998 Stelovsky
`5,801,687
`9/1998 Peterson .
`5,809,317
`9/1998 Kogan .
`5,818,441
`10/1998 Throckmorton .
`5,828,325
`10/1998 Wolosewicz et al. .
`5,845,281
`12/1998 Benson .
`5,848,413 * 12/1998 Wolff
`5,848,424
`12/1998 Scheinkman .
`5,852,673
`12/1998 Young .
`5,857,038
`1/1999 Owada et al. .
`5,872,589
`2/1999 Morales .
`5,893,910
`4/1999 Martineau .
`5,920,861
`7/1999 Hall.
`5,920,878
`7/1999 DeMont .
`5,926,822
`7/1999 Garman .
`5,932,863
`8/1999 Rathus .
`5,974,141
`10/1999 Saito .
`5,983,218
`11/1999 Suyeda-Mahmoo .
`5,991,876
`11/1999 Johnson .
`
`OTHER PUBLICATIONS
`
` 463/1
`
` 707/10
`
`Szepanski, "Additive Binary Data Transmission for Video
`Signals," Papers Presented at Conf. Of Comm. Engineering
`Soc. Sep. 30—Oct. 3, 1980, Technical Reports vol. 74, pp.
`342-352.
`Tirkel et al, "Electronic Water Mark," DICTA-93, Macqua-
`rie University, Sydney, Australia, Dec., 1993, pp. 666-673.
`Kurak et al., "A Cautionary Note On Imaging Downgrad-
`ing," 1992 IEEE, pp. 153-159.
`
`* cited by examiner
`
` 369/124
` 380/28
`
`NFLE Ex. 1017 - Page 2
`
`NLFE 1017 - Page 2
`
`

`

`U.S. Patent
`
`Sep. 4, 2001
`
`Sheet 1 of 18
`
`US 6,286,036 B1
`
`FIG. 1
`
`12
`DIGITAL
`NUMBER 8
`(SIGNAL
`LEVEL) 4
`
`•
`
`•
`
`•
`
`•
`
`'4-BITS'
`
`0
`
`•
`I
`0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
`SAMPLE NUMBER (INDEX)
`
`FIG. 4
`
`104
`
`100
`
`EXPOSE AND STEP
`
`NFLE Ex. 1017 - Page 3
`
`NLFE 1017 - Page 3
`
`

`

`U.S. Patent
`
`Sep. 4, 2001
`
`Sheet 2 of 18
`
`US 6,286,036 B1
`
`FIG. 2
`
`OBTAIN OR CREATE ORIGINAL
`DIGITAL SIGNAL OR IMAGE
`
`ESTIMATE ROUGH OFFSET
`AND RMS NOISE
`
`CHOOSE N OR N-BIT
`IDENTIFICATION WORD, E.G. 32
`
`GENERATE N-BIT
`IDENTIFICATION WORD
`3
`GENERATE OR SYNTHESIZE N "RANDOM" INDEPENDENT
`SIGNALS WITH ROUGHLY GAUSSIAN DISTRIBUTION
`ABOUT SOME MEAN VALUE, WHERE SIGNALS HAVE
`EQUAL EXTENT AND DIGITAL SPACING OF
`ORIGINAL DIGITAL SIGNAL OR IMAGE
`
`APPLY DIGITAL FILTER WHICH ATTENUATES
`BOTH LOW AND HIGH FREQUENCIES, LEAVING
`MIDDLE-RANGE FREQUENCIES LARGELY INTACT
`
`CONDENSE N RANDOM SIGNALS TO A LOWEST
`ACCEPTABLE BIT VALUE IF MEMORY OR
`STORAGE SPACE IS AT A PREMIUM
`4
`ADD ALL RANDOM IMAGES TOGETHER WHICH HAVE
`A CORRESPONDING '1' IN THEIR ASSOCIATED
`BIT-PLACE-VALUE OF THE N-BIT IDENTIFICATION WORD,
`CALL THIS THE BASE COMPOSITE SIGNAL OR IMAGE
`
`EXPERIMENT VISUALLY WITH GAIN AND GAMMA APPLIED
`TO BASE COMPOSITE SIGNAL OR IMAGE, ADDING THIS TO
`ORIGINAL DIGITAL SIGNAL OR IMAGE, AND DETERMINING
`THE ACCEPTABLE PERCEIVED NOISE LEVEL
`
`APPLY FOUND GAIN AND GAMMA TO BASE COMPOSITE,
`ADD TO ORIGINAL, THEN CALL THIS
`THE DISTRIBUTABLE SIGNAL OR IMAGE
`
`STORE AWAY AND SECURE ORIGINAL SIGNAL OR IMAGE,
`ALONG WITH N-BIT IDENTIFICATION WORD AND
`THEN RANDOM SIGNALS
`
`SELL OR DISTRIBUTE THE DISTRIBUTABLE SIGNAL OR IMAGE
`
`NFLE Ex. 1017 - Page 4
`
`NLFE 1017 - Page 4
`
`

`

`U.S. Patent
`
`Sep. 4, 2001
`
`Sheet 3 of 18
`
`US 6,286,036 B1
`
`FIG. 3 OBTAIN DIGITAL OR NON-DIGITAL COPY
`OF SUSPECT SIGNAL OR IMAGE
`
`DIGITIZE IF NOT ALREADY DIGITAL
`
`CUT AND MASK PORTION OF SIGNAL OR IMAGE
`BELIEVED TO BE SUSPECT
`(ONLY IF ENTIRE SIGNAL OR IMAGE IS NOT SUSPECT)
`
`PROCURE ORIGINAL DIGITAL SIGNAL OR
`IMAGE AND CUT AND MASK TO ROUGHLY
`THE SAME LOCATION OR SEQUENCE
`
`VISUALLY RESCALE AND REGISTER THE CUT-OUT
`SUSPECT SIGNAL TO THE CUT-OUT ORIGINAL SIGNAL
`
`RUN THROUGH SEARCH PROGRAM WITH MEAN
`SQUARED ERROR AS CRITERIA AND X OFFSET, Y OFFSET,
`AND SCALE AS THE THREE VARIABLES
`
`APPLY X OFFSET, Y OFFSET, AND SCALE TO CUT-OUT SUSPECT,
`THEN RESAMPLE ONTO EXACT GRID AND CUT-OUT
`OF ORIGINAL SIGNAL
`
`RUN THROUGH SEARCH PROGRAM WITH MEAN
`SQUARED ERROR AS CRITERIA AND DC OFFSET, GAIN, AND
`GAMMA AS THE THREE VARIABLES; APPLY TO SUSPECT
`
`SUBTRACT ORIGINAL FROM SUSPECT,
`GIVING DIFFERENCE SIGNAL OR IMAGE
`
`STEP THROUGH ALL N RANDOM INDEPENDENT SIGNALS, MASKED
`AS ORIGINAL AND CROSS-CORRELATED WITH DIFFERENCE SIGNAL
`IN IMMEDIATE NEIGHBORHOOD OF REGISTRATION POINTS
`
`FIND 0 AND 1 LEVEL BY AVERAGING FIRST FOUR 0101 CODE VALUES
`
`ASSIGN EITHER A 0 OR A 1 TO EACH CROSS-CORRELATION RESULT
`DEPENDING ON PROXIMITY TO THE AVERAGES OF PREVIOUS STEP
`
`CHECK RESULT AGAINST SECURED IDENTIFICATION NUMBER
`
`PROSECUTE IF IT MATCHES? OR AT LEAST SEND
`A NASTY LETTER DEMANDING RECOMPENSE
`
`NFLE Ex. 1017 - Page 5
`
`NLFE 1017 - Page 5
`
`

`

`U.S. Patent
`
`Sep. 4, 2001
`
`Sheet 4 of 18
`
`US 6,286,036 B1
`
`FIG. 5
`CODE WORD
`(e,g. 01101001)
`
`INPUT
`SIGNAL
`
`FIG. 6
`
`REAL-TIME
`ENCODER
`
`IDENTIFICATION-
`CODED OUTPUT
`SIGNAL
`b.. KEY DATA
`(OPTIONAL)
`
`ANALOG
`NOISE
`SOURCE
`
`ND
`
`222
`
`206
`
`224
`
`202
`
`7
`FIRST
`SCALER
`
`208
`210 226
`V j
`SECOND
`SCALER
`
`204
`
`LOOKUP
`TABLE
`LTh
`220
`
`214
`
`MEMORY
`
` OUTPUT
`
`INPUT
`
`
`
`218
`
`L
`
`ADDER
`SUBTRACTER
`
`232
`
`230
`
`_
`
`_
`
`212
`
`234
`
`J
`
`01011000
`
`216
`
`NFLE Ex. 1017 - Page 6
`
`NLFE 1017 - Page 6
`
`

`

`U.S. Patent
`
`Sep. 4, 2001
`
`Sheet 5 of 18
`
`US 6,286,036 B1
`
`NOISE
`SOURCE
`
`H
`
`NOISE
`STORE
`
`206
`
`CODE 1
`
`0
`
`CODE 2-N
`
`242
`
`(-- 202
`
`INPUT>
`
`REAL
`TIME
`ENCODER
`
` OUTPUT
`
`234
`
`•
`0 0 1 0 0 0 0 0
`0 0 1 0 0 0 0 1
`•
`•
`0 0 1 0 0 0 1 0
`
` 0 ►
`0
`1
`0
`0
`0
`1
`1
`0
`0
`1
`•
`0
`0
`1
`0
`0
`0
`0
`1
`0
`0
`1
`•
`0
`1
`•
`0 0 1 0 0 1 1 0
`0 0 1 0 0 1 1 1
`0 1 0 f 1 0 0 0
`
`1ST
`THROUGH
`NTH CODE
`WORDS
`
`REAL
`TIME
`ENCODER
`
`248
`___/- 202 f
`H A/D -- OUTPUT
`
`t NOISE
`
`CODE
`
`FIG. 7
`
`FIG. 8
`
`A/D
`
`NOISE
`SOURCE
`
`250
`
`A RESET
`
`SYNC
`DETECTOR
`
`0 0 1 0 0 0 0 0
`0 0 1 0 0 0
`1
`-•~~
`-40---1
`0 0 1 0 0 0 1 0
`0 0 1 0 0 0 1 1
`0 0 1 0 0 1 0 0
`INCREMENT
`
`NFLE Ex. 1017 - Page 7
`
`NLFE 1017 - Page 7
`
`

`

`U.S. Patent
`
`Sep. 4, 2001
`
`Sheet 6 of 18
`
`US 6,286,036 B1
`
`FIG. 9A
`400
`TIME
`
`0.0
`SECONDS
`
`FIG. 9B
`NORMALIZE
`402
`—20dB
`FREQUENCY —40dB
`
`FIG. 9C
`BORDER
`CONTINUITY
`404
`
`1.0
`SECONDS
`
`0HZ
`
`50KHZ
`
`DETAIL OF MATCH AT BORDER;
`CONTINUOUS TO mth DERIVITIVE
`
`FIG. 10
`
`ROM; 504
`STANDARD NOISE SIGNATURES
`
`AUDIO
`
`(
`500
`
`COPYRIGHT
`DETECTION FLAG
`
`502
`
`FLAG VALID VALID
`
`508
`
`506
`
`NFLE Ex. 1017 - Page 8
`
`NLFE 1017 - Page 8
`
`

`

`U.S. Patent
`
`Sep. 4, 2001
`
`Sheet 7 of 18
`
`US 6,286,036 B1
`
`FIG. 11
`
`599N
`
`600
`
`STANDARD NOISE
`SIGNATURE
`
`504
`
`READ OUT AT125% Li- 604
`NORMAL RATE
`
`PITCH CHANG ED H 606
`NOISE SIGNAL
`
`INPUT AUDIO
`SIGNAL
`
`SIGNAL-PITCH CANCELLED
`NOISE SIGNAL
`
`y-- 6013
`
`602
`
`r r
`
`TIME AVERAGED
`POWER SIGNAL
`PsIG
`
`TIME AVERAGED
`POWER SIGNAL P.C. NOISE
`SIGNAL P S-PCN
`
`y- 610
`
`POWER DIFFERENCE SIGNAL
`-
`S-PCN P SIG
`-
`POUT
`614-Th
`--- CASE 1. I
`OS
`
`612
`— 613
`
`
`
`FIG. 12 624
`600
`
`SIGNAL ONE
`.055 DIGITAL DELAY
`
`.05s DIGITAL DELAY
`
`.05s DIGITAL DELAY
`
`.055 DIGITAL DELAY
`
`620
`
`I
`I
`10s 15s
`5s
`
`wwi vv\A
`
`616 Th
`"—CASE 2:
`
`
`Os
`5s
`lOs 15s
`ti 4 SECOND BEATS
`618
`AUDIO VALID
`FLAG VALID
` SIGNAL`-- 508
`COUNTER
`599 Laj BEAT J 622
`I - DETECTIONr-
`
`599
`
`F
`
`502
`
`DETECTION
`FLAG
`
`1
`
`NFLE Ex. 1017 - Page 9
`
`NLFE 1017 - Page 9
`
`

`

`U.S. Patent
`
`Sep. 4, 2001
`
`Sheet 8 of 18
`
`US 6,286,036 B1
`
`FIG. 13
`
`x700
`
`FRAME#: 12183
`DISTRIBUTION LOT: REGION 14
`
`ENCRYPTION/SCAMBLING
`-r- ROUTINE #28, 702
`
`PSEUDO-RANDOM MASTER SNOWY IMAGE
`(SCALED DOWN AND ADDED TO FRAME 12183)
`
`NFLE Ex. 1017 - Page 10
`
`NLFE 1017 - Page 10
`
`

`

`U.S. Patent
`
`Sep. 4, 2001
`
`Sheet 9 of 18
`
`US 6,286,036 B1
`
`FIG. 14
`
`,
`720
`
`,„
`
`DIFFERENCE
`
`--j
`
`EMBEDDED
`
`MEAN-REMOVED HISTOGRAMS OF
`DIFFERENCE SIGNAL AND KNOWN EMBEDDED
`CODE SIGNAL
`
`EMBEDDED
`
`722,,..,
`
`DIFFERENCE
`
`THRESHOLDING
`
`MEAN-REMOVED HISTOGRAMS OF
`FIRST DERIVATIVES (OR SCALER GRADIENTS
`IN CASE OF AN IMAGE)
`
`NFLE Ex. 1017 - Page 11
`
`NLFE 1017 - Page 11
`
`

`

`U.S. Patent
`
`Sep. 4, 2001
`
`Sheet 10 of 18
`
`US 6,286,036 B1
`
`STEP Z FRAMES
`A
`
`750
`
`RENDER LOGO FRAME
`
`ENCRYPT/SCRAMBLE
`
`700
`
`702
`
`FIG. 15
`
`OUTPUT MASTER SNOWY
`IMAGE FRAME
`
`704
`
`HIGH BRIGHTNESS MASTER
`SNOWY MOVIE
`
`752
`
`7-\_
`54
`
`
`
`756
`
`MPEG
`
`COMPRESSED MASTER
`SNOWY MOVIE
`
`758
`
`760
`
`DIFFERENCE
`
`"CHEAP MASTER
`SNOWY MOVIE"
`
`SCALE DOWN
`A
`
`770
`
`764
`
`ORIGINAL MOVIE
`
`ADD
`
`_7-- 766
`768
`
`762
`
`762
`
`DISTRIBUTABLE
`MOVIE
`
`SIDE-BY-SIDE
`VIEWING
`
`ORIGINAL MOVIE
`
`NFLE Ex. 1017 - Page 12
`
`NLFE 1017 - Page 12
`
`

`

`wawa •sn
`
`81 Jo II lamIS
`
`HI 9£0'98V9 Sa
`
`JOE'S IMAGE
`
`JOE'S IMAGE
`
`JOE'S IMAGE
`
`JOE'S IMAGE
`
`JOE'S IMAGE
`
`JOE'S IMAGE
`
`N
`
`JOE'S IMAGE
`
`JOE'S IMAGE
`
`JOE'S IMAGE
`
`JOE'S IMAGE
`
`cI offEd - L tot .X1 aldm
`
`JOE'S IMAGE
`
`JOE'S IMAGE
`
`JOE'S IMAGE
`
`JOE'S IMAGE
`
`JOE'S IMAGE
`
`JOE'S IMAGE
`
`JOE'S IMAGE
`
`JOE'S IMAGE
`
`JOE'S IMAGE
`
`JOE'S IMAGE
`
`JOE'S IMAGE
`
`JOE'S IMAGE
`
`DATA STREAM
`
`HEADER
`
`001101011011101001010...
`
`...JOE'S IMAGE...
`
`,802
`
`,800
`
`FIG. 16
`
`NLFE 1017 - Page 13
`
`

`

`U.S. Patent
`
`Sep. 4, 2001
`
`Sheet 12 of 18
`
`US 6,286,036 B1
`
`FIG. 17
`96 BIT LEADER STRING, 820
`
`"SHADOW CHANNEL", 828
`
`64 BIT LENGTH
`
`32 BIT DATA WORD SIZE
`
`DATA...
`
`822
`
`824J
`
`826J
`
`UNIVERSAL EMPIRICAL DATA FORMAT
`
`FIG. 18
`
`850
`
`854
`
`864
`
`NFLE Ex. 1017 - Page 14
`
`NLFE 1017 - Page 14
`
`

`

`U.S. Patent
`
`Sep. 4, 2001
`
`Sheet 13 of 18
`
`US 6,286,036 B1
`
`FIG. 19
`866
`
`\
`
`FIG. 20
`BRIGHTNESS
`PROFILE, 874
`
`CENTER POINT OF
`RING, 872
`
`1
`
`NOMINAL DISTANCE
`TO CENTER OF OUTER
`RING WIDTH, 870
`
`NOMINAL DISTANCE
`TO CENTER OF OUTER
`RING WIDTH, 870
`
`876
`
`NFLE Ex. 1017 - Page 15
`
`NLFE 1017 - Page 15
`
`

`

`U.S. Patent
`
`Sep. 4, 2001
`
`Sheet 14 of 18
`
`US 6,286,036 B1
`
`FIG. 21A
`
`900, C 2C C
`2C 4C 2C
`C 2C C
`
`FIG. 21B
`
`WHERE C = 1/16
`
`2
`
`6
`
`2
`
`6
`
`•••
`
`3
`
`7
`
`3
`
`7
`
`4
`
`0
`
`5
`
`1
`
`6
`
`2
`
`C 2C C
`
`4 2C 4C 2C 6
`C 2C C
`
`0
`
`1
`
`2
`
`7
`
`3
`
`7
`
`3
`
`0
`
`4
`
`0
`
`4
`
`FIG. 23
`
`1
`
`El
`6
`
`5
`
`CABLE, 964,
`TO DATA LINE,
`966
`
`960
`
`962
`
`NFLE Ex. 1017 - Page 16
`
`NLFE 1017 - Page 16
`
`

`

`U.S. Patent
`
`Sep. 4, 2001
`
`Sheet 15 of 18
`
`US 6,286,036 B1
`
`940
`
`FIG. 22
`
`z
`
`DIGITAL IMAGE TAKEN
`OF CUSTOMER
`
`942
`
`N
`
`COMPUTER
`PROCESSES NEW
`ENCODED IMAGE
`
`952
`
`PRINTS
`(
`PROCESSED
`IMAGE ONTO
`PLASTIC CARD
`954
`
`950
`
`NFLE Ex. 1017 - Page 17
`
`NLFE 1017 - Page 17
`
`

`

`U.S. Patent
`
`Sep. 4, 2001
`
`Sheet 16 of 18
`
`US 6,286,036 B1
`
`FIG. 24
`
`970
`
`972
`
`ORIGINAL DIGITAL IMAGE WITH
`BARCODE AND FIDUCIALS
`ADDED
`
`COMPUTER GENERATES MASTER
`SNOWY IMAGE WHICH IS
`GENERALLY ORTHOGONAL TO
`ORIGINAL IMAGE AT LEFT
`
`950
`
`COMBINED TO FORM PERSONAL CASH CARD
`
`NFLE Ex. 1017 - Page 18
`
`NLFE 1017 - Page 18
`
`

`

`Juaied 'S11
`
`XT JO LI WIN
`
`HI 9£0'98V9 Sa
`
`10. CENTRAL NETWORK CHECKS RESULTS AGAINST MASTER
`
`12. CENTRAL NETWORK DEBITS MERCHANT ACCOUNT,
`
`CREDITS CARD ACCOUNT
`
`11. CENTRAL NETWORK SENDS FINAL APPROVAL OR DENIAL
`
`DOT PRODUCT RESULTS TO CENTRAL NETWORK
`
`9. READER TRANSMITS THE TWENTY-FOUR
`
`STORES RESULT
`RESULTANT PATTERN AND CARD SCAN,
`8B. READER PERFORMS DOT PRODUCT OF
`
`PATTERNS
`
`8A. READER ADDS TOGETHER SET OF ORTHOGONAL
`8. READER STEPS THROUGH THE TWENTY-FOUR SETS
`
`SETS OF RANDOM NUMBERS
`
`4. READER SENDS ID, (PIN), MERCHANT INFORMATION, AND
`3. READER CALLS CENTRAL ACCOUNT DATA NETWORK, HANDSHAKES
`
`1. READER SCANS IMAGE ON CARD, STORES IN
`
`MEMORY, EXTRACTS PERSON'S ID
`
`2. OPTIONAL: USER KEYS IN PIN NUMBER
`
`TYPICAL TRANSACTION STEPS
`
`FIG 25
`
`REQUESTED TRANSACTION AMOUNT TO CENTRAL NETWORK
`
`5. CENTRAL NETWORK VERIFIES ID, PIN, MERCHANT INFO,
`
`AND ACCOUNT BALANCE
`
`6. IF OK, CENTRAL NETWORK GENERATES TWENTY-FOUR
`
`TO A SET OF 64K ORTHOGONAL SPATIAL PATTERNS
`WHERE THE RANDOM NUMBERS ARE INDEXES
`SETS OF SIXTEEN DISTINCT RANDOM NUMBERS,
`
`7. CENTRAL NETWORK TRANSMITS FIRST OK, AND THE
`
`61 offEd - LIOI 'xa aldN
`
`NLFE 1017 - Page 19
`
`

`

`U.S. Patent
`
`Sep. 4, 2001
`
`Sheet 18 of 18
`
`US 6,286,036 B1
`
`FIG. 26
`
`982
`
`1006
`
`FIG. 27
`
`1002
`
`x-1012
`
`1 008
`
`WEB SITE
`DEVELOPMENT TOOL I
`WWW SITE
`
`1004
`
`1006
`
`1010
`
`BROWSER
`WWW SITE
`
`NFLE Ex. 1017 - Page 20
`
`NLFE 1017 - Page 20
`
`

`

`1
`AUDIO- AND GRAPHICS-BASED LINKING
`TO INTERNET
`
`RELATED APPLICATION DATA
`
`This application is a division of application Ser. No.
`09/130,624, filed Aug. 6, 1998, which is a continuation of
`application Ser. No. 08/508,083, filed Jul. 27, 1995 (U.S.
`Pat. No. 5,841,978). The subject matter of the present
`application is also related to that disclosed in applicant's
`four applications filed May 8, 1995: application Ser. No.
`08/436,098 (U.S. Pat. No. 5,636,292); Ser. No. 08/436,099
`(U.S. Pat. No. 5,710,834); Ser. No. 08/436,134 (U.S. Pat.
`No. 5,748,763); and Ser. No. 08/438,159 (U.S. Pat. No.
`5,850,481), and Ser. No. 154,866, filed Nov. 18, 1993
`(abandoned), Ser. No. 215,289, filed Mar. 17, 1994
`(abandoned), Ser. No. 327,426, filed Oct. 21, 1994 (U.S. Pat.
`No. 5,768,426), and PCT/US94/13366, filed Nov. 16, 1994.
`The disclosures of these prior applications are incorporated
`herein by reference.
`
`TECHNICAL FIELD
`
`The present invention relates to the field of computer
`network navigation, such as navigation and routing of
`instructions on the Internet. More particularly, the invention
`presents a new method for automatically linking from one
`computer site to another, also known as a "link" or "hot link"
`method.
`
`BACKGROUND AND SUMMARY OF THE
`INVENTION
`
`The World Wide Web (WWW) is a standardized graphic
`interface and network protocol to the Internet which is
`rapidly increasing in popularity and usage. Among many
`reasons for the growth in the WWW is the core feature that
`allows a user to immediately "link" from one given web site
`to another, merely by pointing to an object and clicking a
`button, where the object pointed to is usually a highlighted
`or otherwise clearly distinguished line of text or image.
`Computerized documents containing such links are often
`referred to as "hypertext."
`Generally speaking, creators of a web site set up a site or
`page using web site development tools, such as the Web-
`Force tools sold by Silicon Graphics. At a more mundane
`level, programmers use a language called Hypertext Markup
`Language (HTML) to generate the instructions necessary for
`a web site to function properly. Currently, the basic manner
`with which a "hot link" from one site to another is imple-
`mented is by specifying within the programming language
`or the web site development tools which graphical object
`presented to a web site visitor will be the visual link to
`another web site. Once the object is specified, the program-
`ming tools associate with this object a URL address, which
`is the Internet or web address of the web site to which the
`object points. Summarizing in a very general way, the
`creation of a graphical "hot link" and its underlying URL
`address needs to be programmed in a non-automated man-
`ner.
`Traditional methods of implementing hot links generally 60
`employ a "header file" that contains the URL address. The
`header file is attached to the graphical object. Alternatively,
`a database management system is set up, whereby a graphi-
`cal object has an index number attached such that a database
`of URL addresses can be searched using the index value. 65
`Both of these traditional approaches suffice in a well defined
`network system where everyone agrees to abide by identical
`
`50
`
`55
`
`NFLE Ex. 1017 - Page 21
`
`US 6,286,036 B1
`
`5
`
`10
`
`15
`
`20
`
`25
`
`30
`
`40
`
`45
`
`2
`protocols and to use the same header files and/or database
`procedures. Moreover, these methods require agreement on
`how this information is transferred when going from one
`system to another.
`In the case of the Internet, however, and the current World
`Wide Web, there is a huge range of graphical objects that are
`representable, and the ideal of a universal data file format is
`far from being realized. Instead, a multitude of file formats
`are used, and most of them do not have a simple means
`whereby a URL address could become attached in a way that
`would also facilitate, by not conflicting with, the continued
`development of standards to attach URL addresses.
`It is desirable, therefore, to find a linking method whereby
`a given object can effectively comprise both a graphical
`representation to a user and the URL address, thereby to
`serve as a hot link. In such a way, a web site developer need
`only include a pointer to the object (often an object with
`which the developer is accustomed to using), and the under-
`lying tools and web site browsers will recognize the object
`as a hot link. One way to provide such a system would be
`to associate URL addresses directly with a graphical object,
`and, preferably, provide some indication that this object is in
`the hot link class. The steganographic linking method of the
`present invention addresses this goal. The invention pro-
`vides a common sense method whereby all web browsers
`and web tools can easily attach (i.e., embed) URL addresses
`to graphical objects. The method easily integrates into the
`current system in a way that does not require sweeping
`changes to well-entrenched file formats and transmission
`protocols.
`Once steganographic methods of "hot link" navigation
`take hold, then, as new file formats and transmission pro-
`tocols develop, more traditional methods of "header-based"
`information attachment can enhance the basic system built
`35 by a steganographic-based system. In this way, stegano-
`graphic implementation of the present invention pays due
`heed to the huge installed base of file formats existing today,
`paving the way toward simpler attached information imple-
`mentations. Steganographic methods will retain one differ-
`ential property in that, at least for more robust forms of
`steganography, address and index information can survive
`going into and out of the digital and network domain.
`Another aspect of this invention pertains to unauthorized
`use and outright piracy of proprietary source material which,
`since time immemorial, has been a source of lost revenue,
`confusion, and artistic corruption.
`These historical problems have been compounded by the
`advent of digital technology. With it, the technology of
`copying materials and redistributing them in unauthorized
`manners has reached new heights of sophistication, and
`more importantly, omnipresence. Lacking objective means
`for comparing an alleged copy of material with the original,
`owners and litigation proceedings are left with a subjective
`opinion of whether the alleged copy is stolen, or has been
`used in an unauthorized manner. Furthermore, there is no
`simple means of tracing a path to an original purchaser of the
`material—something which can be valuable in tracing where
`a possible "leak" of the material first occurred.
`A variety of methods for protecting commercial material
`have been attempted. One is to scramble signals via an
`encoding method prior to distribution, and descramble prior
`to use. This technique, however, is of little use in mass
`market audio and visual media, where even a few dollars
`extra cost causes a major reduction in market, and where the
`signal must eventually be descrambled to be perceived, and
`thus can be easily recorded.
`
`NLFE 1017 - Page 21
`
`

`

`US 6,286,036 B1
`
`3
`Another class of techniques relies on modification of
`source audio or video signals to include a subliminal iden-
`tification signal, which can be sensed by electronic means.
`Examples of such systems are found in U.S. Pat. No.
`4,972,471 and European patent publication EP 441,702, as
`well as in Komatsu et al, "Authentication System Using
`Concealed Image in Telematics," Memoirs of the School of
`Science & Engineering, Waseda University, No. 52, p.
`45-60 (1988) (Komatsu uses the term "digital watermark"
`for this technique). These techniques have the common
`characteristic that deterministic signals with well defined
`patterns and sequences within the source material convey
`the identification information. For certain applications this is
`not a drawback. But in general, this is an inefficient form of
`embedding identification information for a variety of rea-
`sons: (a) the whole of the source material is not used; (b)
`deterministic patterns have a higher likelihood of being
`discovered and removed by a would-be pirate; and (c) the
`signals are not generally 'holographic' in that identifications
`may be difficult to make given only sections of the whole.
`(Holographic' is used herein to refer to the property that the
`identification information is distributed globally throughout
`the coded signal, and can be fully discerned from an
`examination of even a fraction of the coded signal. Coding
`of this type is sometimes termed "distributed" herein.)
`Among the cited references are descriptions of several
`programs which perform steganography described in one
`document as "... the ancient art of hiding information in
`some otherwise inconspicuous information." These pro-
`grams variously allow computer users to hide their own
`messages inside digital image files and digital audio files. All
`do so by toggling the least significant bit (the lowest order
`bit of a single data sample) of a given audio data stream or
`rasterized image. Some of these programs embed messages
`quite directly into the least significant bit, while other
`"pre-encrypt" or scramble a message first and then embed
`the encrypted data into the least significant bit.
`Our current understanding of these programs is that they
`generally rely on error-free transmission of the of digital
`data in order to correctly transmit a given message in its
`entirety. Typically the message is passed only once, i.e., it is
`not repeated. These programs also seem to "take over" the
`least significant bit entirely, where actual data is obliterated
`and the message placed accordingly. This might mean that
`such codes could be easily erased by merely stripping off the
`least significant bit of all data values in a given image or
`audio file. It is these and other considerations which suggest
`that the only similarity between our invention and the
`established art of steganography is in the placement of
`information into data files with minimal perceptibility. The
`specifics of embedding and the uses of that buried informa-
`tion diverge from there.
`Another cited reference is U.S. Pat. No. 5,325,167 to
`Melen. In the service of authenticating a given document,
`the high precision scanning of that document reveals pat-
`terns and "microscopic grain structure" which apparently is
`a kind of unique fingerprint for the underlying document
`media, such as paper itself or post-applied materials such as
`toner. Melen further teaches that scanning and storing this
`fingerprint can later be used in authentication by scanning a
`purported document and comparing it to the original finger-
`print. Applicant is aware of a similar idea employed in the
`very high precision recording of credit card magnetic strips,
`as reported in the Feb. 8, 1994, Wall Street Journal, page Bl,
`wherein very fine magnetic fluxuations tend to be unique
`from one card to the next, so that credit card authentication
`can be achieved through pre-recording these fluxuations
`
`4
`later to be compared to the recordings of the purportedly
`same credit card.
`Both of the foregoing techniques appear to rest on the
`same identification principles on which the mature science
`5 of fingerprint analysis rests: the innate uniqueness of some
`localized physical property. These methods then rely upon a
`single judgement and/or measurement of "similarity" or
`"correlation" between a suspect and a pre-recording master.
`Though fingerprint analysis has brought this to a high art,
`10 these methods are nevertheless open to a claim that prepa-
`rations of the samples, and the "filtering" and "scanner
`specifications" of Melen's patent, unavoidably tend to bias
`the resulting judgement of similarity, and would create a
`need for more esoteric "expert testimony" to explain the
`15 confidence of a found match or mis-match. It is desirable to
`avoid this reliance on expert testimony and to place the
`confidence in a match into simple "coin flip" vernacular, i.e.,
`what are the odds you can call the correct coin flip 16 times
`in a row. Attempts to identify fragments of a fingerprint,
`20 document, or otherwise, exacerbate this issue of confidence
`in a judgment. It is desirable, therefore, to objectively apply
`the intuitive "coin flip" confidence to the smallest fragment
`possible. Also, storing unique fingerprints for each and every
`document or credit card magnetic strip, and having these
`25 fingerprints readily available for later cross-checking,
`should prove to be quite an economic undertaking. It would
`be preferred to allow for the "re-use" of noise codes and
`"snowy images" in the service of easing storage require-
`ments.
`30 Despite the foregoing and other diverse work in the field
`of identification/authentication, there still remains a need for
`a reliable and efficient method for performing a positive
`identification between a copy of an original signal and the
`original. Desirably, this method should not only perform
`35 identification, it should also be able to convey source-
`version information in order to better pinpoint the point of
`sale. The method should not compromise the innate quality
`of material which is being sold, as does the placement of
`localized logos on images. The method should be robust so
`40 that an identification can be made even after multiple copies
`have been made and/or compression and decompression of
`the signal has taken place. The identification method should
`be largely uneraseable or "uncrackable." The method should
`be capable of working even on fractional pieces of the
`45 original signal, such as a 10 second "riff" of an audio signal
`or the "clipped and pasted" sub-section of an original image.
`The existence of such a method would have profound
`consequences on piracy in that it could (a) cost effectively
`monitor for unauthorized uses of material and perform
`"quick checks"; (b) become a deterrent to unauthorized uses
`when the method is known to be in use and the consequences
`well publicized; and (c) provide unequivocal proof of
`identity, similar to fingerprint identification, in litigation,
`with potentially more reliability than that of fingerprinting.
`In accordance with exemplary embodiments of the
`invention, the foregoing and additional objects are achieved
`by embedding an imperceptible identification code through-
`out a source signal. In the preferred embodiment, this
`60 embedding is achieved by modulating the source signal with
`a small noise signal in a coded fashion. More particularly,
`bits of a binary identification code are referenced, one at a
`time, to control modulation of the source signal with the
`noise signal.
`The copy with the embedded signal (the "encoded" copy)
`becomes the material which is sold, while the original is
`secured in a safe place. The new copy is nearly identical to
`
`55
`
`50
`
`65
`
`NFLE Ex. 1017 - Page 22
`
`NLFE 1017 - Page 22
`
`

`

`US 6,286,036 B1
`
`6
`FIGS. 22-26 detail aspects of a security card according to
`one embodiment of the present invention.
`FIG. 27 is a diagram, illustrating the aspect of the inven-
`tion that provides a network linking method using informa-
`tion embedded in data objects that have inherent noise.
`
`5
`
`DETAILED DESCRIPTION OF PREFERRED
`EMBODIMENT
`
`5
`the original except under the finest of scrutiny; thus, its
`commercial value is not compromised. After the new copy
`has been sold and distributed and potentially distorted by
`multiple copies, the present disclosure details methods for
`positively identifying any suspect signal against the original.
`Among its other advantages, the preferred embodiments'
`use of identification signals which are global (holographic)
`and which mimic natural noise sources allows the maximi-
`zation of identification signal energy, as opposed to merely
`having it present 'somewhere in the original material.' This
`allows the identification coding to be much