000001
`
`Symantec 1016
`IPR of U.S. Pat. No. 7,757,298
`
`

`
`5,761,655
`Page 2
`
`OTHER PUBLICATIONS
`
`Misuo Nagamachi. “An Image Technology Expert System
`and Its Application to Design Consultation”. International
`Journal of Human-Computer Interaction, 3(3). (1991). pp.
`267-279.
`F. Jamrik et al.. “A Visual Technique for Communicating
`Data”. Elsevier Science Publishers B.l/. (North Holland)
`IFIP, 1992. pp. 397-410.
`Unnamed Document with pp. 904-915. no date.
`Masaomi Oda. “Context Dependency Effect in the Forma-
`tion of Image Concepts and its Application” IEEE, 1991. pp.
`1673-1678.
`Robert P. Futrelle et al.. “Understanding Diagrams in Tech-
`nical Documents". Computer; Jul. 1992. pp. 75-78.
`T. Gevers et al.. “Enigma: An Image Retrieval System”.
`IEEE, 1992. pp. 697-700.
`Mariano P. Consens et al.. “Visual Queries and Querying
`Visualizations”. Sigmond Record, vol. 21. No. 1. Mar. 1992.
`pp. 39-46.
`
`A. Del Bimbo et al.. “3—D Visual Query Language for Image
`Databases”. Journal of Visual Languages and Computing,
`1992. 3. 257-271.
`
`Steve Rosenthal. “Will picons. Micons or dicons supersede
`icons"’. MacWEEK, vol: v3. Sep. 26. 1989. Issue: n34 p.
`32(1).
`
`Kazuyuki Tsuda et al.. “IconicBrowser: An Iconic Retrieval
`System for Object—Oriented Databases".
`IEEE,
`1989.
`130-137.
`
`Shi-Kuo Chang. “A Visual Language Compiler for Infor-
`mation Retrieval by Visual Reasoning”. IEEE, Transactions
`on Sofiware Engineering, vol. 16. No. 1. Oct. 1990. pp.
`1136-1149.
`
`Daniel A. Keim et al.. “Visual Query Specification in a
`Multimedia Database System". IEEE, ‘I992. pp. 194-200.
`Shi—kuo Chang.
`“Visual Reasoning For
`Information
`Retrieval From Very Large Databases”. IEEE, 1989. pp.
`1-6.
`
`000002
`
`000002
`
`

`
`QMU
`
`n._
`
`4<o_Eo
`
`txommvi,m{N
`
`om..GE
`
`umama
`
`vaa2;$5>n_n_O.I
`
`2,mm
`
`
`
`._,W%__FoE>$mmamam_>_mn_omE:E559%:
`
`,.m
`
`556.,1
`
`5,om<om>$_
`
`mo_:n_<mo>S%amode
`
`
`
`mo<u_$:,__w.$.:,__E
`
`8
`
`
`
`
`.0.mo<u_EE_m_oEEE_$5.300xmawE0252xmozfiz._<o_Eo
`
`
`mm>_¢o
`mm8Sxommvi,m
`
`m
`
`
`
`000003
`
`
`

`
`U.S. Patent
`
`Jun. 2, 1998
`
`Sheet 2 of 36
`
`5,761,655
`
`W 8°
`
`FIG. 2
`
`ALLOCATE THUMBNAIL
`DATABASE RECORD AND CLEAR.
`
`, USING GIVEN FILE ID, OPEN
`IMAGE FILE AND READ HEADER.
`
`EXTRACT LIST OF RGB COLORS
`USED AND COPY INTO RECORD
`
`39
`
`34
`
`85
`
`
`
`
`
`
`
`OWIdth = ORIGINAL IMAGE WIDTH
`OHeight = ORIGINAL IMAGE HEIGHT
`TWidth = MAX THUMBNAIL WIDTH
`TI-Ieight = MAX THUMBNAIL HEIGHT
`
` 0Width > TI/Ifidth
`OR
`0HeIght > THeight
`?
`
`NO
`
`
`
`
`
`OWidth / OHeighI
`
`TWIdth /’THeight
`.9
`
`I
`
`
`
`
`THeight = THeight I
`OHeighI / Owidth
`
`
`TwIdIh = TVIfIdth *
`Owidth /OHeIght
`
`F94
`
`
`THeight = OHeIght
`OWIdth / OWidth
`
`ALLOCATE BUFFER TO HOLD OW'Idth
`I OI-Ieight BYTE PIXEL VALUES.
`
`I
`,
`
`'
`I
`
`READ ORIGINAL IMAGE INTO
`ALLOCATED BUFFER.
`
`REDucE Owidth BY OHeight
`IMAGE TO TWidth BY THeight
`
`SAVE Owidth, OHeight, TWidth
`
`IMAGE INTO RECORD. (FIG. 4)
`ANDTHeightINRECORD
`
`100
`
`I02
`
`I04
`
`106
`
`EXTRACT 8 DOMINANT COLORS
`AND SAVE IN RECORD
`(FIG. 6)
`
`ADD THUMBNAIL REOORD AND
`INDICES TO DATABASE.
`
`DE-ALLOCATE BUFFERS.
`
`® M
`
`‘O3
`
`“O
`
`1 12
`
`000004
`
`000004
`
`

`
`U.S. Patent
`
`Jun. 2, 1993
`
`Sheet 3 of 36
`
`5,761,655
`
`FIG. 3
`
`@ 120
`
`
`‘I22
`
`
`
`OWidIh = ORIGINAL WIDTH.
`OHeIght : ORIGINAL HEIGHT.
`TWidth = THUMBNAIL WIDTH.
`THeIghI = THUMBNAIL HEIGHT.
`OBufI = ORIGINAL PIXEL DATA.
`TBUII‘ = THUMBNAIL PIXEL DATA.
`
`
`
`
`
` Deltax = OWidth I 2.
`
` 124
`DeItaY = OHeight / 2.
`ORow = O. OCoI = 0.
`TRow = 0. T00! = 0.
`
`‘
`
`COPY PIXEL VALUE AT
`OROW,OCOL OF ORIGINAL TO
`TFIOW,TCOL OF THUMBNAIL
`
`Xlnc = 1 + (OWidth - De|taX)
`/ TWidth
`
`
`
`DeItaX = DeltaX + (XInc *
`TWidth) - OWidth
`
`TRow = TFIow + 1
`ORow = OFIow + XInc
`
`134
`
`TRow < TWidth?
`
`NO
`
`Ylnc = 1 + (OHeight -
`De|taY) / THeight
`
`Delta Y = DeItaY + (YInc *
`THeight) - OHeight
`
`TCoI = TCoI + 1
`OCol = OCo| + Ylnc
`
`142
`
`TCoI < THeight'?
`
`NO
`
`®
`
`000005
`
`000005
`
`

`
`U.S. Patent
`
`Jun. 2, 1998
`
`Sheet 4 of 36
`
`5,761,655
`
`FIG. 4
`
`@ 120
`
`\/122
`
`.4124
`
`136
`
`
`
`OWidth = ORIGINAL WIDTH.
`OHeight = ORIGINAL HEIGHT.
`
`
`TWidth = THUMBNAIL WIDTH.
`
`THeighI = THUMBNAIL HEIGHT.
`OBuff = ORIGINAL PIXEL DATA.
`TBuff = THUMBNAIL PIXEL DATA.
`
`
`
`
`
`
`
` DeItaX = OWidth / 2.
`DeltaY = OHeight / 2.
`
`ORow = O. OCoI = 0.
`TRow = 0. TCoI = 0.
`
`
`
`
`
`Ylnc = 1 + (OHeight -
`De|taY) /THeight
`
`XInc = 1 + (OWidIh ~ DeItaX)
`/TWidth
`
`PROCESS PIXEL
`(FIG. 5)
`
`DeItaX = Deltax + (Xlnc *
`TWIdth) - OWIdth
`
`TRow = TRow + 1
`ORow = ORow + Xlnc
`
`134
`
`V128
`
`V 146
`
`V I30
`
`132
`
`View < TWIdtht'?
`
`YES
`
`NO
`
`De1taY = DeItaY + (Ylnc *
`THeight) - OHeight
`
`TCOI = TCoI +1
`OCoI = 0Co| + Ylnc
`
`138
`
`140
`
`142
`
`TCoI < THeight?I
`
`YES
`
`NC
`
`w 0
`
`00006
`
`000006
`
`

`
`U.S. Patent
`
`Jun. 2, 1998
`
`Sheet 5 of 36
`
`5,761,655
`
`FIG. 5
`
`PROcEss PIXEL V150
`START
`
`Col = OCOI.
`CLEAR AverageRGB.
`
`GET PIXEL AT ROw,Co|.
`USE PIXEL AS INDEX INTO
`ORIGINAL IMAGE RGB GOLOR
`TABLE AND GET Pixe|FlGB VALUE.
`
`ADD PIxeIRGB RED, GREEN
`AND BLUE VALUES TO
`AverageRGB.
`
`Row < OROw + Xlnc'?
`
`NO
`
`Co! = Col + 1
`
`I52
`
`“I54
`
`“I55
`
`158
`
`I50
`
`I54
`
`Col < OCOI + Yinc?
`
`166 Y
`
`ES
`
`L45
`
`NO
`
`.
`.
`DIvIdeAverageRGB RED,
`GREEN AND BLUE COMPONENTS
`BY (XInc * Ytnc)
`
`FIND THE RGI3 ENTRY IN COLOR
`TABLE WHICH MOST CLOSELY
`MATCHES AverageRGB.
`
`
`
`
`SAVE INDEX TO BEST MATCHING
`COLOR TABLE ENTRY AS PIXEL
`VALUE AT TROw,TCO| IN
`THUMBNAIL IMAGE.
`
`
`
`
`168
`
`170
`
`172
`
`‘I74
`
`END
`RETURN
`
`OOOOO7
`
`000007
`
`

`
`U.S. Patent
`
`Jun. 2, 1998
`
`Sheet 6 of 36
`
`5,761,655
`
`FIG. 6
`
`A 68
`'
`
`Row > 2 MAX
`Row?
`
`YES
`
`Row = O;
`COLUMN = COLUMN + Blocksize
`
`SAVE WITH THE IMAGE 8 OF THE
`MATCH TABLE RGB ENTRIES WITH
`HIGHEST COUNTS.
`
`3-O-8
`
`ADD RGB VALUE TO MATCH TABLE ~52
`WITH INITIAL COUNT OF 1.
`INCREMENT COUNT.
`
`OOOOO8
`
`58
`
`NO
`
`64
`
`W 4°
`
`CLEAR MATCH TABLE.
`COUNT = 0;
`
`ROW = 0; COLUMN = 0
`
`I
`2
`
`CUIR = Row; CUTC = COLUMN;
`RED :0; GREEN = 0; BLUE 2 0
`
`I FOR PIXEL AT ICUTR, CurC):
`3 RED = RED + PIXEL RED
`I GREEN = GREEN + PIXEL GREEN
`I BLUE = BLUE + PIXEL BLUE
`
`‘NCREMENT CUIR
`
`CurR : Row +
`Blocksize?
`
`52
`
`CurR = ROW; INCREMENT CUTC
`
`CUTC = COLUMN
`+ Blocksize?
`
`YES
`
`i RED = RED / B|ockSI'ze"2;
`I GREEN = GREEN / BIockSize"2;
`I BLUE = BLUE/B|ockSIze’\2;
`
`
`
`60
`
`RGB VALUE WITHIN
`TOLERANCE OF ANY MATCH
`
`RGB ENTRY?
`
`YES
`INCHEMENT THE MATCHING RGB
`ENTRY.
`
`
`
`000008
`
`

`
`U.S. Patent
`
`Jun. 2, 1998
`
`Sheet 7 of 36
`
`5,761,655
`
`FIG. 7A
`
`FIELD
`DATA
`DESCRIPTION
`TYPE
` A
`
`Res
`
`
`
`180
`
`I
`
`
`
`1
`
`.3
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`R68
`R68
`
`THUMBNAIL HEIGHT
`
`(3 BYTE HGB‘S)
`
`RGB
`
`PIXEL DATA
`
`W X H BYTES
`
`DATA
`TYPE
`
`
`
`
`
`DATA
`TYPE
`LONG
`
`o..7 OF RG8
`
`LONG
`
`FIG. 78
`
`FIELD
`DESCRIPTION
`RED COMPONENT
`GREEN COMPONENT
`BLUE COMPONENT
`
`
`
`1 92
`
`FIG. 7C
`
`1%
`
`.4. CD00
`
`FIELD
`DESCRIPTTON
` FILE ID
`
`ORIGINAL WIDTH
`ORIGINAL HEIGHT
` l\.')Sl’\)
`RECORD #
`
`DOMINANT COLORS
`
`000009
`
`000009
`
`

`
`Jun. 2, 1998
`
`Sheet 8 of 36
`
`5,761,655
`
`
`
`
`
`RanaW3....................................LH.....................".......................
`
`
`
`
`
`Scammmsx$96$8
`
`..............................................1.
`
`
`
`
`
`
`u...%m._coomsmmogo‘$350...m§3Eo.o.._m..ws.qEoo...m._2:QEoU
`Ivmm_mmam.T140$!!!mIDO?!
`..6..w....................2"".....................JW........................
`
`
`
`§_._._.m._..............¢.._m,.,._._._._K....._..a._2_2saw5
`
`
`ém..........................................
`....................................................................1
`
`
`
`
`
`u.........
`
` U.S.Patent
`
`
`
`
`_m>mEwmmmmE_®m._qmc_
`
`
`
`EEHfimfiw
`:m‘OE
`
`000010
`
`000010
`
`
`
`
`
`
`
`

`
`U.S. Patent
`
`Jun. 2, 1998
`
`Sheet 9 of 36
`
`5,761,655
`
`m>0-
`
`:4--0
`0
`CC
`
`FIG.9
`
`00001 1
`
`000011
`
`

`
`U.S. Patent
`
`Jun. 2, 1998
`
`Sheet 10 of 36
`
`5,761,655
`
`
`
`
`
`XmasCh—ee'r“
`
`
`
`
`-_Ma-keAMerfiE>""_Lu""'-H" mg?
`'-
`
`
`|nspire®ImaoeRetrieval
`
`
`BeMme
`
`u . . . . . . . . _ . . . . . . . .
`. . . . . . . . . . . . . . . . . . ,
`
`n . . . . . . . . . . . . . . . . .i
`I . . . . . . . . . . . . . . . . . ,
`
`I . . . . . . . . . . . . . . . . . I
`' . . . . . . . . . . . . . . . . _ ,
`
`- . . . . . . . . _ . . . . . . ..
`. . . . . . . . . . . . . . _ . . -,
`
`
`
`210
`
`. . . . . . . _ . . . . _ _ . . _ _ !
`
`| . _ . . . _ . . . . . . _ _ . _ -,
`
`WorkToo|s#2
`
`
`
`WorkTools#1
`
`I . . . . . . . . . . . . . _ _ . . I
`
`c _ _ _ . . . . . . _ . . _ _ . _ _I
`
`I . . _ _ _ _ . . . _ _ _ _ . . _ . I
`
`I _ _ . . . . . . . _ _ . . . . . --
`
`5 5
`
`000012
`
`000012
`
`

`
`U.S. Patent
`
`Jun. 2, 1993
`
`Sheet 11 of 36
`
`5,761,655
`
`FIG. 11
`
`C230
`
`I
`
`I
`
`Firstvis = O
`GridX,GrIdY = DEFAULT GRID
`VISCOIS = MAX WIDTH / GridX
`VisROws = MAX HE|GHT/ GridY
`
`ALLOCATE THUMBNAIL CACHE
`FOR VisROws ' VISCQIS * 3
`IMAGES OR AS MANY AS
`MEMORY ALLOWS. PLACE
`POINTERS IN CacheTable
`
`‘V232
`
`/234
`
`DISPLAY ANY CHANGED THUMB— V935
`NAILS. (FIG. 12)
`
`CHECK FOR USER INPUT
`
`«233
`
`240
`
`YES
`
`LINE DOWN
`REQUEST?
`
`LINE UP
`REQUEST?
`
`N0
`PAGE DOWN
`REQUEST?
`NO
`
`PAGE UP
`REQUEST?
`
`.. 244
`
`I
`
`YES
`
`246 _
`
`248
`
`YES
`
`ADJUST FirstVis BASED ON
`REQUEST.
`
`
`
`
`
`TOGGLE HIGHLIGHT STATE OF
`SELECTED IMAGE. IF HIGH-
`LIGHTED, SHOW ATTRISUTES.
`
`
`
`
`
`955 T
`
`TOGGLE HIGHLIGHT STATE OF
`
`IMAGE ATTRISUTE
`
`
`
`250 \
`
`YES
`
`UPDATE QUALIFY CRITERIA
`BASE 0“
`
`SELECTED ATTRIBUTES
`
`264
`®
`
`252
`
`000013
`
`I‘ 254
`NO
`SELECT TMAGE
`
`ATTRTEUTET
`
`NO
`258
`RETURN
`TO FILE QUALIFY
`S
`‘NO
`CHECK CACHE (FIG. 13)
`
`I
`
`I
`
`l
`
`,
`1
`
`000013
`
`

`
`U.S. Patent
`
`Jun. 2, 1993
`
`Sheet 12 of 36
`
`5,761,655
`
`FIG. 12A
`
`
`
`272
`
`
`
`270
`
`
`
`
`
`.
`
`.
`
`274
`
`CaChe'I'abIe = LIST OF POINTERS TO THUMBNAILS
`IDTabIe = LIST OF ID'S FOR THUMBNAILS TO SHOW
`VISROWS = NUMBER OF VISIBLE Rows
`VISCOIS = NUMBER OF VISIBLE COLUMNS
`Grid)-I : PIXEL WIDTH OF DISPLAY GRID CELL
`I GridY = PIXEL HEIGHT OF DISPLAY GRID CELL
`I FIISIVIS = INDEX OF FIRST VISIBLE THUMBNAIL
`
`
`
`
`
`
`
`
`I
`
`ThUmbIndeX = FIISIVIS
`
`Row = 0
`COI = 0
`
`SEARCH CacheTabIe FOR A
`MATCHING ENTRY
`
`IMAGE
`FOUND IN CACHE
`'?
`
`NO
`
`
`
`0 304
`COPY PIXEL DATA AND
`IDTab|e INDEX INTO
`CACHE ENTRY
`
`306
`
`DISPLAY PIXEL DATA AT
`Row * Gn'dX, COI * Gn'dY
`
`308
`
`’
`
`310
`
`ROW >
`
`YES
`
`» 312
`
`Col = Col +1
`Row :0
`
` 314
`
`YES
`
`316
`
`
`
`FIND CACHE ENTRY WITH
`LOWEST AND HIGHEST
`IDTabIe INDEX
`
`230
`
`HIGHEST
`INDEX VISIBLE
`:2
`
`232 NO
`
`
`
`RE-USE CACHE ENTRY WHICH V234
`CONTAINED HIGHEST INDEX.
`
`
`
`RE-USE CACHE ENTRY WHICH
`CONTAINED LOWEST INDEX.
`
`USE INDEX TO GET THUMB-
`NAIL ID FROM IDTabIe
`
`READ THUMBNAIL RECORD
`INTO MEMORY.
`
`288
`
`290
`
`E
`
`FIG- 723
`
`
`
`THUMBNNLS
`SHOWN AT FULL
`SIZE?
`
`300
`
`YES
`
`
`
`CACHED THUMBNAIL STRUCTURE
`(POINTED To BY CacheTabIe ENTRIES)
`FIELD
`DATA
`DESCRIPTION
`TYPE
`IDTabIe INDEX —INT
`
`
`
`THeIght BYTES
`PIXEL DATA
`TWidIh '
`BEING DISPLAYED (FIG. 3)
`
`
`
`REDUCE THUMBNAIL: TO SIZE
`
`000014
`
`000014
`
`

`
`U.S. Patent
`
`Jun. 2, 1998
`
`Sheet 13 of 36
`
`5,761,655
`
`FIG. 13
`
`\ , 332
`
`V 334
`
`£62
`
`GET POINTER TO ENTRY FRoM
`
`- 335
`
`
`
`THUMB INDEX <
`
`
`
`Firstvis - (VisRows *
`VisCo|s)?
`
`
`
`
`
`THUMB INDEX >
`Firstvis + (VisFIows '
`VisCo|s ' 2)?
`
`
`
`
`THUMB INDEX <
`Firstvis?
`
`
`
`# OF ENTRIES <
`(VIsFIows * VisCoIs *
`2)?
`
`
`I. 346
`MAFIK ENTRY AS EMPTY
`
`
`
`I
`
`Cachelndex = Cachelndex + 1
`350
`
`v 348
`
`YES
`
`Cachelndex < #
`OF ENTRIES?
`
`, 356
`FIND EMPTY
`ANY
`CacheTabIe SLOT
`VISIBLE IMAGES NOT
`IN CACHE?
`AND ADD IMAGE
`
`TO CACHE
`NO
`362
`FIND EMPTY
`ANY
`Ca(;he‘|'ab|e SLOT
`IMAGES IN NEXT PAGE
`NOT IN CACHE?
`AND ADD IMAGE
`
`TO CACHE
`
`
`
`
`
`" 352
`
`
`
`
`
`
`
`
`
`
`
`
`A 358 No
`
`
`
`ANY
`FIND EMPTY
`
`IMAGES IN PREV PAGE
`CacheTabIe SLOT
`
`NOT IN CACHE?
`AND ADD IMAGE
`
`TO CACHE
`
`NO
`
`354
`
`NO
`
`NO
`
`YES
`
`YES
`
`YES
`
`V370
`
`000015
`
`Ii
`
`I III
`
`I
`
`000015
`
`

`
`3U
`
`11
`
`556.,
`
`,.m8%,5,Eom:
`.,_._.E_>>Eam_o<s__Hxmaz
`
`m_o<_>__
`
`
`
`..I.S_>>E5mo<s__. Iz9E:22E98wHHHm.mm:/mHn.Ema:
`
`
`
`
`
`8:05/¢_%EaomasP3GE
`N>5n_m_oamm$:z_o._mN8E$o<_>__no5:mm_o<_>__
`
`
`xmaz_Eom:.,_._E_>>ESm_o<s__E54/ mo<s__Emm955NEmN9am_%%.LN:om_9E:ogMQ8Lmmomo8:5mmE,__o¢m>s%_n_
`
`
`
`«Wq9ms:mHxmaz_
`
`000016
`
`
`

`
`U
`
`e
`
`%
`
`5,761,655
`
`QM2GE
`
`t\omms.._.O8
`
`
`..E §nUmEo3>$_
`_om_mw
`
`
`
`
`
`cooumccmom_m::m_2oEm_>_mic$._.
`
`«mmSam8m._oimE_<8m._€n_m:o=mo__&<29Eumccmomhm:_§._m.s_
`
`
`
`mc_$mc_9m_
`
`D-V8
`
`
`
`9___3om.s:ms__m_E%_Eoo5:3\Jmov
`
`xszfiflNi».u......:...Bwe.
`
`
`.0:M;%_260¢mQ8wS,3O
`
`m0
`
`V3
`
`
`
`_8:m.ommmgmmm.xm_>_
`
`\.m_>_¢ow_::>_Q
`
`
`E_mn:m>>0.DE_mn=m>>m.D_H_
`
`000017
`
`
`

`
`U.S. Patent
`
`Jun. 2, 1998
`
`Sheet 16 of 36
`
`5,761,655
`
`FIG. 16
`
`/ 420
`
`426
`
`DATABASE
`FILE
`‘5
`T’
`
`
`
`‘
`
`( 422
`DATABASE
`
`1 424“,
`KEY 438\ KEYWORD
`4284
`434 MATCH
`I'M
`V LONG/
`5 FILECATEGORY LONG
`435»
`440 SUPER-KEY DATA 64 BYTES
`
`KEYWORD §D1
`INT
`“442
`‘E KEYWORD ID2
`INT
`I ORB”)
`3: WT
`4,01
`‘Lowe
`D;
`
`
`
`—;
`
`KEYWORD ID
`
`INT
`
`
`
`KEYWORD NAME 16 BYTES
`SUPER-KEY DATA 64 BYTES
`
`5?
`
`KEYWORDID
`:_ —
`,.32BYTES “L5”
`
`432 -‘
`
`FILE ID
`
`Lo~e '
`
`‘NT
`
`KEYWORDID4
`WT
`
`323YTE3
`
`000018
`
`000018
`
`

`
`U.S. Patent
`
`Jun. 2, 1998
`
`Sheet 17 of 36
`
`5,761,655
`
`FIG. 17
`
`443 «\
`KEY VIEW1
`444 .
`4
`
`
`(448
`LEAF NODES
`
`451
`449
`
`
`445 ».
`
`446 .5
`
`_
`
`I
`
`:
`
`;
`
`'§>2
`
`—|—=;
`
`
`
`KEYVIEW2
`
`
`CURKEY IN NODE um
`MBYTES
`
`
`
`
`
`453
`PARENTNODE
`55”
`
`
`W“
`
`
`
`
`KEYWORD ID FILE ID [L FILE :0
`J
`
`
`;~ 3
`
`
`__E__ -_ ‘E ;»454
`_KEYX’9”_E’PIFEE_'D_*‘ Fm
`
`
` KEYWORD ID FiLE ID
`
`NODE HEADER
`
`KEYWOFID ID FILE ID
`
`KEYWORD ID FILE ID
`
`
`
`
`
`
`
`
`
`
`
`
`KEY VIEW 3
`INT
`Lows
`
`
`
`
`
`
`
`54 BYTES
`
`
`E
`
`
`
`KEYWORD ID FILE ID
`
`KEYWORD ID FILE ID
`KEYWSREEJ .FIETD.
`4
`1
`
`i
`A}
`
`3
`%
`
`KEYWORD ID FILE ID
`
`000019
`
`LEGEND:
`
`~ - * - PARENT/LEAF LINK
`~—— VIEW POSITION
`— «— LEAF LINKED LIST
`
`C_—J V|EW'S CURRENT KEY
`
`000019
`
`

`
`U.S. Patent
`
`Jun. 2, 1993
`
`Sheet 18 of 36
`
`5,761,655
`
`FIG. 18
`
`\/ 455
`
`INITIALIZE FILE ID OUTPUT LIST
`
`455
`
`ICOUNT =0
`CURRENT FIIeID = 2A31 -1
`
`v 457
`
`480
`
`0
`
`
`MUstMaIch = 1
`NO
`
`
`AND MATCH COUNT >c
`MINIMUM MATCH
`?
`
`
`
`INIT A KEY VIEW FOR KEYWORD
`
`INITIALIZE CURRENT KEY TO FIRST
`ENTRY FOR THIS KEYWORD ID
`
`460
`
`
`FiIeID < CURRENT
`FIIeID?
`
`
`'
`
`CURRENT FiIe|D = FIIeID
`
`INCREMENT COUNT
`
`
`
`COUNT = # OF
`KEYWORDS?
`
`
`YES
`COUNT = 0. MATCH COUNT =0.
`MUstMatch = 1.
`
`5
`v 4 8
`
`
`
`
`
`
`
`ADD CURRENT FiIeID TO
`FILE ID OUTPUT LIST
`
`484
`
`COUNT = O
`CURRENT Fi|eID = 2"31 - I
`
`
`
`
`
`DID
`KEYWORD ID FOR
`THIS VIEW
`
`VALID KEY‘
`VIEW AND FiIeID <
`
`CURRENT
`
`FiIeID?
`
`
` COUNT = #
`?
`
`OF KEYWORDS
`
`YES
`
`
`
`VALID KEY VIEW
`AND KEYWORD FiIeID
`= CURRENT FiIeID
`
`'?
`
`
` YES
`ANY VALID‘
`VIEWS REMAINING
`9
`
`
`COUNT = # OF
`KEYWORDS?
`
`
`000020
`
`000020
`
`

`
`U.S. Patent
`
`Jun. 2, 1993
`
`Sheet 19 of 36
`
`5,761,655
`
`HELD
`
`DATA
`
`
`
`III
`was
`In
`
`3
`
`_NT
`
`SUPER KEYWORD 5“
`INFORMATION
`
`
`
`
`
`
`
`
`
`
`
`3
`;
`
`4
`
`E
`
`J
`
`
`
`
`
`
`
`
`INT
`
`
`
`RGB
`
`
`
`
`
`
`
`
`I I
`
`
`I KEYWORD ID2
`I KEYWORDD3
`
`
`5159} KEYWORDID4
`
`1 KEYWORD |D5
`I KEYWORD lD6
`I KEYWORD ID7
`
`KEYWORD D8
`;’ DOMINANT COLOR1
`5173' DOMINANT COLOR2
`
`I DOMINANT COLORS
`
`L DOMINANT COLOR4
`
`
`
`
`
`FIG 193
`
`FIELD
`
`519
`
`DESCRIPTION
`‘ KEYWORD ID
`RECORD#
`
`520»
`
`DATA
`
`TYPE
`
`/513
`
`LONG
`
`000021
`
`000021
`
`

`
`U.S. Patent
`
`Jun. 2, 1998
`
`Sheet 20 of 36
`
`5,761,655
`
`FIG. 20
`
`N 521
`
`
`
`RGBLISI IS LIST OF DESIRED COLORS
`RGBCOunt IS NUMBER OF COLORS
`
`
`FIleList IS INPUT LIST OF FILES
`MatchList IS OUTPUT FILE LIST
`MaxDIsIance IS COLOR TOLERANCE
`
`
`
`
`
`POINT TO START OF FiIeList
`
`GET FI|eID FROM CURRENT POSITION
`IN FiIeLIsI
`
`522
`
`523
`
`524
`
`SCAN To THUMBNAIL INDEX ENTRY
`CONTAINING Fi|eIDI
`
`525
`
`POINT TO START OF RGBS IN INDEX
`
`525
`
`GET RGBI FROM CURRENT POSITION
`IN INDEX RGBS
`
`POINT TO START OF RGBLISI
`
`
`
`GET RGB2 FROM CURRENT POSITION
`IN RGBLISI
`
`DeItaR = RGB1 RED — RGB2 RED
`DeIIaG : RGB1 GREEN - RGB2 GREEN
`DeItaB = RGB1 BLUE - RGB2 BLUE
`
`
`
`
`
`
`
`
`
`DISTANCE = DeIIaR/*2 + DeItaG/*2 +
`DeItaB/I2
`
`
`
`
`
`
`
`529
`
`530
`
`POINT TO NEXT ENTRY IN INDEX
`RGBS
`
`~ 532
`
`533
`-531
`
`END OF INDEX
`RGBS
` DISTANCE <
`YES
`
`?
`MaxDIstance?
` YES
`, 534
`NO
`ADD FIIeID TO MatchLisI
`
`i
`
`POINT TO NEXT ENTRY IN RGI3LIst
`
`535
`
`F 535
`
`
`
`
`END OF RGBLISI?
`
`NO
`
`r 533
`
`END OF FIIeLisI?
` YES
`
`POINT TO NEXT ENTRY IN FiIeLisI
`
`537
`
`
`
`NO
`
`YES
`
`000022
`
`000022
`
`

`
`U.S. Patent
`
`Jun. 2, 1998
`
`Sheet 21 of 36
`
`5,761,655
`
`FIG. 21
`
`540
`
`541
`
` Fi|eList IS INPUT LIST OF FILES
`
`
`
`
`MatchLIst IS OUTPUT FILE LIST
`DWidth IS DESIRED WIDTH
`DHeight IS DESIRED HEIGHT
`
`POINT TO START OF FiIeList
`
`
`
`
`
`
`
`543
`
`GET FiIeID FROM CURRENT POSITION
`IN Fi|eList
`
`
`
`SCAN TO THUMBNAIL INDEX ENTRY v 544
`CONTAINING FiIeID.
`
`545
`
`DWidth = INDEX
`WIDTH AND DHeight
`
`
`= INDEX HEIGHT?
`
`
`
`NO
`
`
`
`YES
`ADD FiIeID TO MatchList
`
`545
`
`POINT To NEXT ENTRY IN EIIeI_IsI
`
`547
`
`548
`
`END OF Fi|eList?
`
` NO
`
`000023
`
`000023
`
`

`
`U.S. Patent
`
`Jun. 2, 1993
`
`Sheet 22 of 36
`
`5,761,655
`
`550
`
`
`2
`
`
`Ei-
`
`l::1Hard Drive 1
`
`FIG. 22C
`
`FIG. 22B
`Root Folder :
`
`
`
`Jukebox 1
` 552
`
`
`{'3
`
`
`
`
`V°'Ume 1
`
`
`1— E
`
`
`Volume 2
`
`IL
`
`Jukebox 1
`
`
`
`554
`
`
`
`
`
`
`FIG. 22D
`Root Folder :
`
`FIG. 22E
`Root Folder:
`
`
`Volume 2
`
`
`553
`
`o File 4
`D File 5
`
`Jukebox 1
`
`l
`
`000024
`
`000024
`
`

`
`U.S. Patent
`
`Jun. 2, 1998
`
`Sheet 23 of 36
`
`5,761,655
`
`FIG. 23
`
`,572
`
`‘
`
`NAME
`
`
`
`
`
`-
`1
`
`14
`15
`
`
`
`
`
`
`
`
`/578
`
`/E, 577
`1 RECORD
`NAME’
`#
`
`576
`
`
`
`PARENTI
`so
`1
`
`
`
`
`
`/573
`, 574
`571x‘
`OTHER
`I
`RECORDEFILE PARENT,/
`FIELDS
`TYPE‘
`r,
`1, :0
`ID
`---- --
`0
`jebox
`---- --
`1
`1
`---- --
`0
`2
`2
`---- ~
`3 Iflwolume
`---- --
`4
`---- --
`5
`6 II ---- -- II
`7
`7
`---- --
`8 Ii ---- -- II
`9 El ---- -~ I!
`10
`---- --
`1
`11
`---- -1
`1 We
`---- -- I
`13
`---- -—
`14
`---- ~-
`15
`---- ~
`16
`---- --
`17
`---- ~
`18
`---- --
`19 j ---- -- I
`20
`---- -- III
`21
`21
`---- ~
`22 I ---- -- II
`23
`---- --
`
`DATABASERECORD
`
`SORTEDINDEX
`
`1’
`
`1
`
`570 /'5
`
`‘\
`575
`
`000025
`
`000025
`
`

`
` U.S.Patent
` m>mo5.nE_omE:E_>_3o>/EmosmaaNHEEEGE
`
`
`2,VE:
`
`smmimVEosmmaNms_3o>
`
`.mEoB.nE5m:6mes_.
`
`556,
`
`4EaosmmaMNB_.._
`xSaosmma
`pmm5:$_>_3o>
`
`11
`
`6$3:7,I5:5wEosmioVm_s_3o>m5;
`
`taosmma
`
`Nxommi,
`
`.m.6sEmsmmmmLEo
`
`000026
`
`

`
`U.S. Patent
`
`Jun. 2, 1998
`
`Sheet 25 of 36
`
`5,761,655
`
`FIG. 25
`
`580
`
`
`
`FOR PARENT ID/FILE NAME KEY,
`SET CURRENT KEY TO FIRST INDEX
`ENTRY FOR DESIRED PARENT ID
`
`
`
`582
`
`(JM ScanFiIes)
`
`
`
` RECORD FOUND
`
`AND PARENT ID STILL
`DESIRED ONE?
`
`
`
`
`
`
`
`
`
`READ KEY'S RECORD FROM
`533
`(JM GetFiIeRec)
`
`DATABASE
`
`
`
`DISPLAY FILE NAME FROM RECORD
`
`
`590
`
`500 (JM Scan Files) SCAN TO NEXT KEY
`
`
`
`
`
`
`000027
`
`000027
`
`

`
`U.S. Patent
`
`Jun. 2, 1998
`
`Sheet 26 of 36
`
`5,761,655
`
`620
`
`OFFSET ,/
`
`
`
`
`
`
`64
`
`NAME
`
`EIIE IDEIITIEIEEI
`PARENT ENTRY FILE IDENTIFIER
`VOLUME IDENTIFIER g
`EII—E SIZE
`FILE TYPE
`FILE ATTRIBUTES
`FILE CREATION DATE
`FILE LAST MODIFY DATE
`EIIECAIEGOHY
`EILENAIIE
`KEYWORD INDENTIFIERS
`
`MNEMONIC
`
`TYPE
`
`'
`
`File Keywords
`
`O..7 OF INT
`
`
`
`TOTAL:
`
`80
`
`FIG 26B
`'
`
`522
`OFFSET ,/’
`TYPE
`MNEMONIC
`NAME
`LONG "
`File ID
`FILE IDENTIFIER
`RECORD NUMBER _ LONG
`
`524
`TYPE
`OFFSET ,2
`NAME
`
`
`LONG n
`PARENT ENTRY FILE IDENTIFIER
`
`0..11 OF BYTES
`PARTIAL FILE NAME
` RECORD NUMBER —
`
`
`
`MNEMONIC
`
`
`
`
`
`FIG 250
`'
`
`
`
`NAME
` FILE CATEGORY
`VOLUME IDENTIFIER
`
`OFFSET ,/
`TYPE
`MNEMONIC
`LONG n
`Volume ID
`
`LONG u
`File Category
`RECORD NUMBER — LONG —g
`
`62
`
`TYPE
`NAME
`FIG- 255
`MNEMONIC
`OFFSET ,2
`2
`O..11 OF BYTES
`PARTIAL FILE NAME
`RECORD NUMBER _ LONG
`
`52
`
`000028
`
`000028
`
`

`
`U.S. Patent
`
`Jun. 2, 1998
`
`Sheet 27 of 36
`
`5,761,655
`
`NAME
`FIG- 27A
`VOLUMEIDENHFER
`
`MEDIA IDENTIFIER
`
`MEDIA SIDE FOR VOLUME
`
`LONG n
`
`540
`OFFSET ,2
`
`MNEMONIC
`VdumeID
`
`Media ID
`
`Media Side
`
`TYPE
`LONG
`
`LONG
`
`542
`OFFSET ,2
`TYPE
`MNEMONIO
`NAME
`FIG 273
`-I
`_
`
`-
`
`OFFSET W
`'TYPE
`MNEMONIC
`NAME
`FIG 270
`MEDIA IDENTIFIER ‘-
`RECORD NUMBER _
`
`/
`
`646
`OFFSET ,/
`TYPE
`MNEMONIC
`NAME
`FIG- 379
`ml
`RECORD NUMBER —
`
`000029
`
`000029
`
`

`
`U.S. Patent
`
`Jun. 2, 1993
`
`Sheet 23 of 36
`
`5,761,655
`
`FIG. 28A
`
`660
`OFFSET ,2
`TYPE
`MNEMONIO
`NAME
`T
`
`
`
`
`
`JUKEBOX ELEMENT GEOMETRY
`
`(SEE BELOW)
`
`E|eDat
`Drive Count
`
`ARRAY OF DRNE DESCRIPTORS
`
`{SEE BELOW)
`
`
`
`INT
`o..7 OF
`
`Driveinfo
`
`o..7 OF
`
`Jobnack Count
`762
`754Z-
`ARRAY OF JOBPACK DESCRIPTORS JobpackDat[]
`
`SLOT/SIDE MEDIA ID'S
`Storage Media[]
`796
`SLOT/SIDE STATUS
`Storage Statusfl O..511 OF CHAR
`
`TOTAL:
`
`
`
`
`
`
`
`
`
`3355
`
`FIG. 28B
`
`662
`OFFSET ,2
`TYPE
`MNEMONIC
`NAME
`
`
`
`
`
`ADDRESS OF FIRST TRANSPORT m
`
`
`NUMBER OF TRANSPORTS
`
`
`ADDRESS OF FIRST STORAGE SLOT
`
`
`NUMBER OF STORAGE SLOTS fl
`
`
`
`
`ADDRESS OF FIRST MAILBOX E
`
`
`NUMBER OF MAILBOXES
`
`
`ADDRESS OF FIRST DRIVE
`
`
`
`NUMBER OF DRIVES
`Transfer Count
`
`
`000030
`
`000030
`
`

`
`U.S. Patent
`
`Jun. 2, 1993
`
`Sheet 29 of 36
`
`5,761,655
`
`FIG. 28C
`
`664
`OFFSET ,2
`TYPE
`MNEMONIC
`NAME
`E
`
`
`
`
`
`
`
`TOTAL:
`
`EE
`
`FIG. 28D
`
`NAME
`FIRST STORAGE ADDRESS
`NUMBER OF STORAGE ELEMENTS
`
`MNEMONTC
`Element Addr
`Element Count
`
`
`
`
`666
`OFFSET ,2
`
`TYPE
`
`
`
`
`000031
`
`000031
`
`

`
`U.S. Patent
`
`Jun. 2, 1993
`
`Sheet 30 of 36
`
`5,761,655
`
`MNEMONIC
`
`TYPE
`
`OFFSET ,2
`
`540
`
`DRIVE NAME lD'S
`I (SEE BELOW)
`
`rive|D Count
`
`rivelDs[]
`
`178
`
`310
`
`O..3 OF
`NamelDs
`
`LONG
`
`LONG
`
`FIG. 29B
`
`NAME
`
`MNEMONIC
`
`PRODUCT IDENTIFIER NAME
`
`542
`
`OFFSET ,2
`TYPE
`0--31 OF CHAR 1
`TOTAL:
`
`000032
`
`000032
`
`

`
`U.S. Patent
`
`Jun. 2, 1993
`
`Sheet 31 of 36
`
`5,761,655
`
`FIG. 30
`
`702 N
`
`
`JUKEBOX MANAGER TOOLKIT
`
` JM
`LAYER
` j
`
`
`
`
`708 -
`
`NT
`NETWORK LAYER
`
`
`
`IMAGE
`PROCESSING
`
`DISPATCHER
`
`
`TO/FROM REMOTE
`JUKEBOX MANAGER CODE
`
`LAYER
`JUKEBOX MANAGER
`
`
`JMX
`
`
`
`UT
`UTILITIES LAYER
`
`
`
`DB
`DATABASE LAYER
`
`
`
`
`JB
`JUKEBOX DEPENDENT
`LAYER
`
`
`
` XF
`INTERFACE DEPENDENT
`LAYER
`
`
`
`718«.
`
`OS
`OPERATING SYSTEM DEPENDENT LAYER
`
`
`
`000033
`
`000033
`
`

`
`U.S. Patent
`
`Jun. 2, 1998
`
`Sheet 32 of 36
`
`5,761,655
`
`FIG. 31
`
`730
`OFFSET ,/‘
`TYPE
`MNEMONIC
`NAME
`
`
`
`732HI
`
`734aE
`
`
`
`
`
`
`
`*
`
`52
`54
`
`Iii
`
`
`'
`
`JUKEBOX IDENTIFIER
`
`=
`
`736%
`
`i 3 L
`
`x
`
`E i
`
`7385 ‘NT
`3
`
`desfivpe
`
`
`
`INT
`TOTAL‘
`
`000034
`
`000034
`
`

`
`U.S. Patent
`
`Jun. 2, 1998
`
`Sheet 33 of 36
`
`5,761,655
`
`FIG. 32
`
`740
`TYPE
`OFFSET ,2
`
`
`‘NT E
`
`NAME
`
`MNEMONIC
`
`
`
`1E
`
`
`
`
`
`
`
`742
`
`
`
`FIG. 33
`
`NAME
`
`MNEMONIC
`
`TYPE
`
`750
`OFFSET /
`
`
`
`FIG. 36
`
`MNEMOMC
`TYPE
`OFFSET
`NAME
`ChangedFlag m
`NODE DATA HAS CHANGED FLAG
`NODE IS NEWLY ALLOCATED FLAG NewFlag
`2
`-
`NODE NUMBER
`
`LONG
`
`
`
`
`
`000035
`
`000035
`
`

`
` U.S.Patent
`
`m_%5:EmE:E.9...
`
`
`
`
`
`Emtomato_zosm_z_>_m_s_<z
`
`Jun.2,1993
`
`Sheet34of36
`
`I823Ex:22I523gm:Sgmma
` omoomm9,2me:,_#_E.mm_mo%5n_><:m_amomE023:5989:22S88%..§e_§ON288%-%E_m8o2E8Eu:8E8Eo<-%§_s8m8889em88om<<-o9§_s8588<<<<-§e_588888<<<<-%_§__8o 955mm<zaozm2m_.;mEozm:wz_9m:mI_.oz<_E_o_ov22>>O_._m#2mfimssz
`
`5,761,655
`
`
`
`
`
`
`
`000036
`
`E::zm5_E:>EzmEmmi
`
`
`
`m=>_<zm._E._<_.E<m
`
`
`
`EmsszQmoomm
`
`000036
`
`

`
`U.S. Patent
`
`Jun. 2, 1998
`
`Sheet 35 of 36
`
`5,761,655
`
`NAME
`
`MNEMONIC
`
`TYPE
`
`770
`
`OFFSET /
`
`
`
`
`
`
`TT
`
`
`
`
`
`512
`
`TOTAL:
`
`772
`
`TYPE
`OFFSET /‘
`
`0..15 OF CHAR fl
`LONG
`
`LONG
`
`KeyPart
`
`
`
`
`
`
`
`INT
`
`64
`
`MNEMONIC
`
`
`
`NAME
`FIG 355
`NAME FOR THIS KEY TYPE
`ROOT NODE NUMBER
`TOTAL NUMBER KEYS
`0.7 of Key
`KEY COMPONENT DESCRIPTORS
`KeyFieIdCnt
`NUMBER OF COMPONENTS
`LENGTH OF KEY, INCLUDING REC # KeyLength
`LENGTH INCLUDING NODE POINTER EntryLength
`MAX KEYS IN NODE, UNPACKED
`MaxKeyInNode
`MAX KEYS IN LEAF, UNPACKED
`MaxKey!nLeaf
`
`
`
`
`
`
`
`
`
`
`
`
`NAME
`
`MNEMONIC
`
`TYPE
`
`774
`
`OFFSET ,2
`
`
`
`776
`
`OFFSET
`TYPE
`MNEMONIC
`NAME
`
`
`PARENTNOPEINPICATOR -I
`NUMBER OF ACTIVE KEYS IN NODE Keys|nNode
`15 BITS E
`NODE # OF PREVIOUS NODE
`LONG
`NODE NUMBER OF NEXT NODE
`LONG E
`KEY DATA
`o..5oI OF CHAR
`
`
`
`
`
`
`
`
`NextNode
`
`
`
`
`
`
`
`000037
`
`000037
`
`

`
`LI.nmPSMU
`
`H
`
`O09
`
`M
`
`1167’5
`
`556.,
`
`J_o§__o>@
`
`m,mm§_§Sn.mmE:_o>§
`
`
`
`.2”.m.ummm.wmww_mmmVMSEO33H.33.8as_
`x832.@E_%:_@mN»emzasooD..mummi
`
`
`1!mEmasqeooG%_.S...__mE_:<N80
`
`H $m.s_%D_ww@mm_\“HSE82Ha
`
`000038
`
`
`

`
`5.761.655
`
`1
`IMAGE FILE STORAGE AND RETRIEVAL
`SYSTEM
`
`This application is a continuation of application Ser. No.
`07/534.176. filed Jun. 6. 1990. now abandoned.
`
`5
`
`REFERENCE TO MICROFICHE APPENDIX
`
`A microfiche appendix including pseudocode for the
`operations of various portions of the present invention is
`attached hereto and incorporated by reference herein with a
`total number of fiche of two and a total number of frames of
`123.
`
`10
`
`BACKGROUND OF THE INVENTION
`
`1. Field of the Invention
`
`The present invention is directed to methods and appa-
`ratus for creating and retrieving image data files in a
`computer system and. more particularly.
`to a system in
`which tens of thousands or even hundreds of thousands of
`files must be created and searched for retrieval. where the
`file retrieval process requires narrowing the number of files
`presented to the user and manually choosing the desired files
`out of the set presented and where selection can be by many
`diiferent kinds of similarities thereby providing multi-
`dimensional organization of the files and also allowing the
`organization of many storage devices integrated with media
`autochangers to automatically bring off line files on line for
`access.
`
`2. Description of the Related Art
`An image data computer system can store tens of thou-
`sands or hundreds of thousands of files. Accordingly. storing
`such quantities of files can become a serious problem.
`Typically finding a file or a group of files requires two steps:
`first. narrowing the number of files to be presented to the
`user. and second. manually choosing the desired files out of
`the set
`to be presented. Most existing methods of both
`narrowing the number of files and selecting the files ignore
`important information which can be used to make the search
`process both more efficient and more natural to the user. The
`files on virtually all computer systems are organized as a
`hierarchical
`tree of directories (or folders -in APPLE
`terminology). All references to the stored files are accom-
`plished by specifying all or part of the directory hierarchy
`containing the desired file. Unfortunately. when many files.
`often numbering in the tens of thousands are present. this
`form of organizing files becomes inadequate. Files which are
`related to each other in more than one way can only be
`grouped together according to one primary similarity. List-
`ing or operating on a set of files which are similar in other
`ways is very cumbersome.
`Conventionally finding a file involves entering a file name
`or a partial file name template which is then used to search
`a part or all of the available storage devices. As files
`matching the given file name are encountered. they are
`shown in a textual list describing the exact location of the
`file. The user can then manually copy the desired file to a
`local work area or operate on the file directly. This method
`usually only works for storage media which is currently
`on-line and accessible to the computer.
`Conventional file management programs allow files to be
`shown as a list of files sorted either purely alphabetically or
`alphabetically by directory tree. Desired files can then be
`selected individually or in groups to be copied to another
`storage area.
`The situation worsens when searching for a file which is
`currently on a storage media. such as magnetic tape or a
`
`20
`
`25
`
`30
`
`45
`
`50
`
`55
`
`65
`
`2
`removable optical disk platter. which is ofl°-line and not
`available to the computer. At best. a master directory exists
`on a storage device which is always on-line. When this is the
`case. the software program. which understands the master
`directory.
`is executed and a search similar to the one
`described above is performed for all ofi°-line storage media.
`Once the desired file is found. the media containing the file
`must be manually loaded and placed on-line by the user or
`a storage librarian. Then. the file will be copied to a local
`work area for use.
`
`In many environments including CAD; engineering.
`printing pre—press. medical imaging. data acquisition. archi-
`tecture and document storage. users store many files which
`contain image data. When this is the case.
`there is an
`opportunity to find files with the additional aid of visual cues
`such as color information and miniature representations of
`the image. commonly called thumbnails. In conventional
`methods.
`the available storage is searched for all files
`containing image data. As image files are found. a miniature
`image is generated from the actual image file. The thumbnail
`images are then presented in a list on a graphic screen. along
`with other key file information such as file name and size.
`The user can then select the desired files by the thumbnail
`images. Conventional image selectors of this nature fall into
`two categories: integrated image selectors and image orga-
`nizer programs.
`Conventional programs which manipulate images. may
`include an image selector for selecting image files with
`which to work. When the user asks to select a file to work
`with. the program will scan part or all of the storage devices
`for image files which the program understands and will
`generate a thumbnail image for each file found. The desired
`image is then selected which is immediately opened and
`loaded into memory for further processing by the user.
`Existing programs which include this method of image
`selection are Studio/8 by Electronic Arts and ResEclit by
`Apple Computer Corp.
`Conventional programs designed specifically for organiz-
`ing images provide the same selection ability but typically
`save the thumbnails for each newly encountered image file
`in a master database or append the thumbnafls to the actual
`file itself. This allows more efficient presentation of images
`in the future. also organizers will often allow keywords to be
`assigned to files or groups of files. These keywords can then
`be used to select a smaller group (ie. only those tagged with
`the given keywords) of images from which to select. This is
`very important when a large number of images are being
`stored. Existing programs using this method of image selec-
`tion are Curator by Solutions International.
`Each known existing method has one or more of the
`following disadvantages: only one image can be selected at
`one time; thumbnail images can be presented in only one
`size; the size of the window used to present the thumbnail
`images is of fixed size; the thumbnail images are in black
`and white even for original color images; the thumbnails are
`generated each time the selection process is invoked; the
`images cannot be selected by key attributes such as size.
`color and age; custom selection criteria cannot be saved as
`super-keywords; where there is keyword support. images
`cannot be selected by matching m out of n keywords;
`keyword searching is very slow when more than a few files
`are being managed; files are presented in a directory hier-
`archy even when this does not represent natural boundaries
`between sets of files; organization is on a per volume basis
`and simultaneous selection across multiple volumes of
`media is not supported; only images which are currently
`on-line can be selected; and selected images can not be
`printed as a report
`
`000039
`
`000039
`
`

`
`5.761.655
`
`4
`
`3
`SUMMARY OF THE INVENTION
`
`It is an object of the present invention to generate thumb-
`nails (i.e.. thumbnail images) once and maintain them in a
`master database separate from the image files. allowing
`selection of images which are not currently on-line.
`It
`is another object of the present
`invention to have
`thumbnails that contain full color information for better
`identification and to perform automatic color analysis and
`selection by dominant colors each criteria.
`It is also an object of the present invention to provide a file
`qualification screen to be filled out before viewing thumb-
`nails to allow the number of images presented to a user to be
`reduced significantly.
`It is a further object of the invention to allow at any time
`a set of often used criteria to be saved as a super-keyword
`where super-keywords can be used like regular keywords
`when selecting files and used to automatically fill out the
`qualification screen. so that minor changes can be made
`before using it and where each user can have a custom set
`of super-keywords. allowing a personalized search process
`without cluttering the search for other users.
`It is an additional object of the present invention to allow
`selection of those image files matching in out of n selected
`keywords including super-keywords allowing for a fudge-
`factor where some relevant images may not be tagged with
`all selected keywords and where super-keywords are actu-
`ally entire templates of selection criteria which have been
`assigned a keyword name.
`It is a further object of the present invention to allow
`searching for only those images residing in a particular
`directory or directory tree of a given storage device which is
`useful if individuals have unique working directories or if
`projects are broken up into dilferent directories. or any other
`meaningful organization by directory.
`It is also an object of the invention to present offline media
`as if they are directories. or folders of a directory containing
`all media.
`
`It is an object of the invention to provide searching for
`only those files matching a certain file creation date range
`and/or only those matching a certain file size range.
`It
`is a further object of the invention to provide for
`searching for only those files containing a certain set of key
`dominant colors so if one remembers approximately what
`color an image is. only those images with particular domi-
`nant colors will be shown where desired colors could be
`selected by colored check boxes or variable color sliders.
`It is an object of the present invention to also provide
`searches for only those images of certain dimensions and/or
`only those matching a certain file name template and to
`allow single character replacement and multi-character
`replacement wildcards.
`It is a still further object of the