(19) United States
`(12) Patent Application Publication (10) Pub. No.: US 2009/0307615 A1
`(43) Pub. Date:
`Dec. 10, 2009
`Jalon et al.
`
`US 20090307615A1
`
`(54) BROWSING OR SEARCHING USER
`INTERFACES AND OTHER ASPECTS
`
`(76) Inventors:
`
`Julien Jalon, Paris (FR); Julien
`Robert, Paris (FR); Alexandre
`Carlhian, Puteaux (FR)
`Correspondence Address:
`APPLE INCABSTZ.
`BLAKELY SOKOLOFF TAYLOR & ZAFMAN
`LLP
`1279 OAKMEAD PARKWAY
`SUNNYVALE, CA 94085-4040 (US)
`(21) Appl. No.:
`12/189671
`
`(22) Filed:
`
`Aug. 11, 2008
`Related U.S. Application Data
`(60) Provisional application No. 61/059.614, filed on Jun.
`6, 2008.
`
`Publication Classification
`
`(51) Int. Cl.
`(2006.01)
`G06F 3/048
`(2006.01)
`G06F 7/06
`(2006.01)
`G06F 7/30
`(52) U.S. Cl. ...................... 715/763; 707/3; 707/E17.014
`
`ABSTRACT
`(57)
`User interfaces for browsing and/or searching are described.
`In some embodiments, the metadata of a file includes a field
`having an indicator to indicate whether a preview of the
`content of the file is available. If the preview is available, then
`a plug-in associated with the creator application of the file is
`executed to present the preview within the icon. Furthermore,
`if the preview is interactive according to the metadata, then
`various user interface control may be generated to allow users
`to manipulate the presentation of the preview. Other embodi
`ments are also described, and computer readable media and
`apparatuses are also described.
`
`
`
`GENERATEAN CON TO REPRESENT A FILE
`NAGU210
`
`PUT ONE OR MORE PREDEFINED
`DECORATIONS ONTO THE CON BASED ON
`THE FILE TYPE 213
`
`PRESENTING A PREVIEW OF THE CONTENT
`OF THE FILE WITHIN THE CON WITHOUT
`MODIFYING FORMAT OF THE CON215
`
`PROVIDING USER INTERFACE CONTROL
`BASED ON THE FILE TYPE TO ALLOW USERS
`TOMANIPULATE PRESENTATION OF THE
`PREVIEW217
`
`MAINTAINING THE FORMAT OF THE CON
`WHILE PRESENTING THE PREVIEW EN
`RESPONSE TO ACTUATION OF THE USER
`INTERFACE CONTROL-219.
`
`Roku EX1021
`U.S. Patent No. 10,334,311
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 1 of 67
`
`US 2009/0307615 A1
`
`
`
`Microprocessor(s)
`103
`
`Mass Storage
`(e.g., HardDrive)
`106
`
`
`
`
`
`Display Controller
`and
`Display Device
`104
`
`
`
`
`
`I/O
`Controller(s)
`108
`
`
`
`
`
`
`
`
`
`
`
`I/O Devices
`(e.g., Mouse Or
`Keyboard or
`Modem Or
`Network Interface Or
`CD Drive, etc.)
`109
`
`FIG. 1
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 2 of 67
`
`US 2009/0307615 A1
`
`Capture metadata from a variety
`of different application programs
`
`201
`
`
`
`Make Captured metadata available to a
`Searching facility (e.g., a file management
`System Software for Searching)
`
`
`
`Allow Searching of metadata across all
`applications having captured metadata
`
`
`
`Pogaseo escape
`
`203
`
`205
`
`Provide User Interface of search engine
`and Search results
`
`207
`
`FIG. 2
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 3 of 67
`
`US 2009/0307615 A1
`
`301
`
`331
`
`JPEG image File Metadata
`
`ISO Setting
`Flash Setting
`
`fl Stop
`
`Image Width
`Image Height
`
`Image Color Space
`
`Camera Brand Information
`
`BitS/Pixel
`
`User Added Keywords
`
`Other Fields.
`
`FIG. 3A
`
`
`
`MP3 Music File Metadata
`
`Song Play Times
`
`Other Fields.
`
`Artist
`
`Genre
`
`Album
`
`Song names
`
`FIG. 3B
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 4 of 67
`
`US 2009/0307615 A1
`
`403
`
`OS
`Kernel
`
`401
`
`411
`
`Metadata Processing
`Software
`
`(clientAP)
`
`405
`File System
`GUI SW
`
`407
`
`SW
`409
`Other
`Applications
`
`419
`
`Importers
`p
`(+ Optionally
`Exporters)
`of Metadata
`413
`
`C
`415 - Metadata
`
`
`
`File
`System
`Directory
`for
`Metadata
`4. 17
`
`7
`400
`
`Index
`of Files
`
`421
`.
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 5 of 67
`
`US 2009/0307615 A1
`
`Receive notification of change for a file (e.g., receive
`notification from OS kernel or software application of
`creation of new file or modification of existing file or
`deletion of existing file).
`
`501
`
`Determine type of file (e.g., an Acrobat PDF file or an
`RTF word processing file or a JPEG image file, etc.)
`
`503
`
`
`
`
`
`Activate capture software for determined file type
`(e.g., a plug in for the application which Creates
`the type of file)
`
`505
`
`Import appropriate metadata (for particular file type)
`into a metadata database
`
`507
`
`Store metadata in a database (e.g., a flat file
`format database)
`
`Receive search parameter inputs and perform search
`of metadata database and display results
`
`509
`
`511
`
`FIG. 5
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 6 of 67
`
`US 2009/0307615 A1
`
`
`
`603
`
`605
`
`607
`
`609
`
`611
`
`Metadata from file 1 of Application A (File A1)
`
`Metadata from file 1 of Application B (File B1)
`
`Metadata from file 2 of Application A(File A2)
`
`Metadata from file 1 of Application C (File C1)
`
`Metadata from file 2 of Application B (File B2)
`
`617
`
`Metadata from filen of Application C (File Cn)
`
`FIG. 6
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 7 of 67
`
`US 2009/0307615 A1
`
`
`
`
`
`Ol. quêS
`
`921
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 8 of 67
`
`US 2009/0307615 A1
`
`
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 9 of 67
`
`US 2009/0307615 A1
`
`N.
`o
`N.
`
`
`
`1] !! 9 !! !!
`
`Jepu!!!!
`
`
`
`a Saôeu]@ a S?SIG IeooT
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 10 of 67
`
`US 2009/0307615 A1
`
`
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 11 of 67
`
`US 2009/0307615 A1
`
`
`
`
`
`
`
`1 | 1 ~^+(O axaeM ?s?d@ ase6eu]@ a SXSIG IeooT
`
`
`
`
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 12 of 67
`
`US 2009/0307615 A1
`
`
`
`Áepoi908
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 13 of 67
`
`US 2009/0307615 A1
`
`
`
`Z08
`
`948
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 14 of 67
`
`US 2009/0307615 A1
`
`mom
`
`Em
`
`mom
`
`
`
`523900.322
`
`EEOmam
`
`cwzmmnmw
`
`9.55mam
`
`3203m
`
`
`
`528an:62
`
`@520gm
`
`28358.0
`
`2:589:89
`
`9:20mam
`
`07.3
`
`ammo
`
`
`
`33330:33.
`
`5waEma
`
`353.1
`
`85.2
`
`.55:<
`
`.653.
`
`353‘
`
`9:2o__.._
`
`Eoiuv.
`
`20253.
`
`._o53<
`
`85-3
`
`353‘
`
`35.2
`
`553.
`
`oEmcSE
`
`95:0:".
`
`25:2E
`
`9:9.2.".
`
`E2“.
`
`E2“.
`
`..:a£8.9322.:
`355600
`
`Exmmnaccoo
`
`mEmz2K2.55
`
`Eoode©mmn
`
`.oxmmm5am
`
`ER“.8Eom
`
`353‘
`
`SEEDmam
`wing.mEoEEoo5.8o>am82:B
`
`oEmc2c
`
`
`
`mEmc5.8m>wm-«55E03.N29.3gum-82:H 553232-59:Hvan.”mug—0co>ww-SaEB3433:3.893-59:3Bad«you3.893-30E3uwad
`
`
`
`
`395o>ww-80:.Hand3E8E0253-89:Bvan;
`
`unafiv29.52-.35“:80E33900.+mafia
`
`o>wm-SE:Bnun—£335....
`03m-89:a.
`
`
`
`89:52-.35":50Efl ofiun-52:Bvan.”5:29:00EozSox-50.:Ev3.3.
` 29.55.-52...“:82:H3938N553Emma-.35H3d...29.32-
`
`End29.52-.35“:39:E33.3.
`
`
`
` 19.-99:Eumaéwun353.8235a.:a5.33.833:8333.3853.883.$2aand=0.me
`
` .85”:92.E34382.21...“5Lav—«05$.Hamnio>o+bm=a3353B83822:9.
`
`
`33:96m0Qm6:8...QN
`
`‘2“com
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 15 of 67
`
`US 2009/0307615 A1
`
`600||
`
`100!
`
`
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 16 of 67
`
`US 2009/0307615 A1
`
`
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 17 of 67
`
`US 2009/0307615 A1
`
`
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 18 of 67
`
`US 2009/0307615 A1
`
`
`
`
`
`
`
`zyx64 sold uqorq.
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 19 of 67
`
`US 2009/0307615 A1
`
`
`
`
`
`þJONA 100xE , JO?09/10
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 20 of 67
`
`US 2009/0307615 A1
`
`
`
`esawong e?eyN
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 21 of 67
`
`US 2009/0307615 A1
`
`
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 22 of 67
`
`US 2009/0307615 A1
`
`
`
`

`

`Patent Application Publication
`
`US 2009/0307615 A1
`
`
`
`C_uopalas (O
`
`?ulil
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 24 of 67
`
`US 2009/0307615 A1
`
`
`
`w
`
`Z Caen)
`
`
`
`?ouees weN \b
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 25 of 67
`
`US 2009/0307615 A1
`
`
`
`
`
`
`
`
`
`
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 26 of 67
`
`US 2009/0307615A1
`
`
`
`1 407
`
`1406
`
`1405
`
`1415
`1416
`
`ociates, 11:2A
`Oct 1990, 11:20 A
`Oc, 1999, 12 A
`octe, 1999, 3:16 PM
`Octs, 1999, 3:19
`An 17, 2004, 10:21 PM
`hun 172001021 PM
`Jun 17, 2004. 10:21 PM
`un. 17.2004. 102P
`Jun 17, 2004,1021 PM
`Jun 17, 2004, 10:21 PM
`Jun 17, 2004, 1021 PM
`An 17,200 1021 PW
`in 17,200 102P
`in 17, 2004, 4:11 PM
`4Ks
`in 17,200 to:2 ra
`44Ks
`in 17, 2004, to 21 PM
`2K
`Yesterday, 212 PM
`MarS2004 SPM 12M
`Mar30, 2004, 3:59PM
`14 Me
`Apr 27, 2000, 4:5 PM
`sloka
`May 27, 2000, 4:03 PM
`2K8.
`May 27, 2000, 4:04 PM
`g2K
`20004:04 PM
`asks
`Ks
`84 k3
`8Ks
`g2ka
`8.8
`K3
`KB
`8Ks
`2gs
`92KB
`9 KB
`sk
`98 KB
`K.
`8KB
`98Ks
`3.
`
`s
`: r
`
`1403
`FIG. 14A
`
`1401
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 27 of 67
`
`US 2009/0307615 A1
`
`
`
`1405
`
`Saravara
`
`FIG. 14B
`
`1401
`
`1403
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 28 of 67
`
`US 2009/0307615 A1
`
`
`
`Oce tas, 11:02 AM
`octe 1989, 1120 AM
`Oct 1999, 11:2. A
`Ocs, 1999, 3:16 PM
`octe, 1989, 3:10 Ph.
`un. 17, 2004, 102P
`jun17.2004, 102 Pt
`in 17, 2004, 1021 PM
`Jun 17, 2004, 1021 Ph.
`Jun 17, 2004 to:21 PM
`in 17, 2004, 10:21 PM
`in 17, 2004, 101 PW
`Jun 17, 2004, 1021 PM
`han 7, 2004. 1021p
`Jun 17, 2004, 4:11 PM
`Jun 17, 2004, 102 Pl
`un 17, 2000:2P
`Yesterday, 212P
`Mar 30, 2004, S59 PM
`Atar 30, 2004, 359 PM
`Apr 27, 20on 4:51 PM
`
`Mary 27.
`May 27, 2000, 4:07 PM
`
`E
`
`rehan, 147 GB with
`
`1403
`FIG. 14C
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 29 of 67
`
`US 2009/0307615 A1
`
`1430
`
`
`
`O
`
`Smart Folder
`
`COn size: 128 x 128
`
`Large
`
`Small
`12pt
`Text size:
`Label position:
`C) Bottom
`ORight
`OSnap to grid
`OShow item info
`OShow icon preview
`Keep arranged by
`
`Background:
`OWhite
`(C) Color
`O Picture
`
`FIG. 14D
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 30 of 67
`
`US 2009/0307615A1
`
`Eagle
`
`
`
`
`
`1511
`
`1513
`
`
`
`
`
`
`
`
`
`
`
`DOCuments
`PDF Documents
`
`1505
`
`1507
`
`1509
`
`1503
`
`1502
`
`3 () Thu 10:42 AM
`
`EshowAll(3)
`Shakelt.gadget.zip
`don'tshakeit out.pdf
`don'tshakeit.pdf
`
`1501
`
`FIG. 15A
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 31 of 67
`
`US 2009/0307615A1
`
`1505
`
`1507
`
`1509
`1515
`
`1503
`
`
`
`1502
`
`3 () Thu 10:47 AM
`
`O
`
`1511
`
`1513
`
`El Show All (392)
`Top Hit O actionbutton
`Documents
`button.psd
`Action Button.dir
`Action Button.dir
`Metal buttons.psd
`animgif.txt
`animgif.txt
`bitmap.txt
`bitmap.txt
`bitmap.txt
`bitmap.txt
`g) buttonwav
`g button.Wav
`CUrSOr.txt
`CUSOr.txt
`E default.txt
`default.txt
`actionbutton
`O new buttons
`action button icon
`alex - clear button
`
`Folders
`
`FIG. 15B
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 32 of 67
`
`US 2009/0307615A1
`
`
`
`Hally
`
`Boco de
`
`No.
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 33 of 67
`
`US 2009/0307615 A1
`
`
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 34 of 67
`
`US 2009/0307615A1
`
`1615
`
`1617
`
`1619
`
`PushButton G11 (graphitopa
`PushButton G11 (graphitopa
`17 more
`p
`V OF Docu re rts
`AHG.sopdr
`. AHIG.51Opdf
`Dor X pdf
`
`1621
`
`
`
`1623
`
`sersGuide.ToPocketPCs.pdf
`. Designing AirPort Networka2.pdf
`be
`
`Source code
`GraphicaBution ave
`sunspherebutionave
`SunSphereastuttonava
`SunSpherunfolutton Java
`SunSphere.NorthButtoneva
`SunSphere.Northbuttoneva
`in
`v word Documents
`Word Documents
`?epubucosasodoc
`fcpubugao.32301.doc
`Radar 5.0.1 User manual
`Ere
`Radar 4.0 User manued
`ico Player bits
`Requ r Chang ee to t AAP (8
`2
`
`
`
`
`
`
`
`GD
`
`GD
`8
`
`GO
`
`GD
`-
`Gd
`8
`
`CD
`
`November 2002
`November 2002
`
`November 2002
`September 2002
`May 2002
`November 2001
`October 2001
`
`May 2001
`May 2001
`May 2001
`May 2001
`May 2001
`May 2001
`
`November 2002
`January 2002
`2000
`June 2000
`June 2000
`
`FIG. 16A
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 35 of 67
`
`US 2009/0307615A1
`
`1625
`
`1611
`
`1601
`
`V Folders
`actionbutton
`new buttons
`action button inco
`alex - clear button
`Help Button
`19 ?hor9.
`V HTML
`16Buttonsis.html
`BButtons.html
`16Buttons4.html
`16Suttons.htm
`16Buttons.htm
`Omore.
`v images
`buttonsymbols.gif
`
`.
`
`- buttonlayogi
`PushButton G11 (graphiepc
`107 more
`PDFDocterts
`AHIGSopdf
`Norton disk editor Xpdf
`Reader.pd
`MacUsersGuideoPocketPC pdf
`Designing AirPort Networks2.pdf
`
`GraphicsButtonave
`SunSpherebutton.java
`SunSphereasutionav
`SunSphereinfobutonova
`SunSpherenorthButton Java
`
`facultugs oazodoc
`Reder 5.0, Uee manual
`Radar.0 User manual
`CDPlayer bit.
`Required Charges to the AAP (8)
`
`1614A
`1613
`
`
`
`61 5
`
`1 61 7
`
`1619
`
`1621
`
`
`
`
`
`1623
`
`
`
`
`
`392 Results Obutton
`
`(8)
`
`s
`GD
`CD
`GD
`GO
`
`GD
`
`CD
`
`
`
`EEE
`GD
`
`1605
`
`1607
`
`1609
`
`1 627
`
`January 5 4:45pm
`April 2003
`February 2003
`December 2002
`November 2002
`
`.
`
`Nowerber 1999
`November 1999
`November 1999
`Nowe?tber 1999
`Nowerber 1999
`
`November 2002
`
`November 2002
`November 2002
`.
`.
`November 2002
`September 2002
`May 2002
`November 2001
`October 2001
`
`May 2001
`May 2001
`May 2001
`May 2001
`May 2001
`
`November 2002
`January 2002
`August 20o
`Ano 2000
`in 2000
`
`FIG. 16B
`
`.
`
`.
`
`GD
`GD
`
`GD
`
`GD
`GD
`
`GD
`GD
`GD
`GD
`
`GD
`GD
`GD
`GD
`GD
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 36 of 67
`
`US 2009/0307615 A1
`
`
`
` couz sunt elep pesnisen - ?ufi?Adoo – sp?owÁex|
`
`
`
`
`1894 /
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 37 of 67
`
`US 2009/0307615 A1
`
`September 193:55pm
`2003
`Action Buttond C June 2003
`March 2003
`
`GD
`GD
`
`
`
`1811
`
`1815
`
`- -
`- -
`Novoryber 2002
`
`buttonlayer.gif
`November 2002
`
`PushBu...te).pct
`November 2002
`
`1812
`1819-si
`
`V POF documents
`
`. Norton disk editor X.pdf .
`... Reader.pdf
`S. MacUsersGuideoPocketPCs.pdf
`. Designing AirPort Networks2.pdf
`8 more,
`1 821 VSource code
`
`1823
`
`Required Changes to the AAP (8)
`
`June 2000
`
`GD
`
`FIG. 18A
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 38 of 67
`
`US 2009/0307615A1
`
`1825
`
`
`
`1814
`1811
`
`1813
`
`1815
`
`18171, in
`
`L
`
`N
`
`button...ols.gif
`November 2002
`
`button...ace.gif
`November 2002
`
`a UV
`
`buttonlayer.gif
`November 2002
`
`PushBu...te).pc.
`November 2002
`
`PushBu.pct
`Newerber 2002
`
`1001...Button
`Newerser 2002
`
`re-arr
`
`1203...button 1204...button
`November 2002
`November 2002
`
`1818A
`
`al
`
`s
`
`(9)
`
`1205...button
`November 2002
`
`12 18...button
`November 2002
`
`1219...button
`Nowember 2002
`
`1222...button 135E...button
`November 2002
`
`1385...uttons 1400...Button
`November 2002
`November 2002
`
`18 1B,...editor 162 BRaised
`Nowenber 2002
`November 2002
`
`FIG. 18B
`
`A.
`v
`4.
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 39 of 67
`
`US 2009/0307615 A1
`
`1925
`
`o O O
`
`1911
`
`V Imran Chaudhri
`CD Player bits
`Required changes to the AAP (e)
`on Text
`1 91 3 V No Authors
`a man
`MAP-Imrangmail.apple.com
`it Drafts (mrandapple.com).mbox
`Fimran
`Sent Messages (imraneapple.com), mbox
`...
`
`
`
`
`
`- 1901
`
`1903
`
`June 2000
`June 2000
`May 2000
`
`Thursday 4:39 pm
`Thursday 3:50pm
`June 17208 art
`June 152:18 pm
`June 10:5:23 pm
`
`GD
`GD
`
`GD
`GD
`GD
`GD
`
`1 907
`
`1915-Eta He 1908
`pminutes ostrodoc
`November 2002
`GD
`pimeline-oazodoc
`November 2002
`GD
`minutes-orsodoc
`November 2002.
`G D
`
`
`
`WWDC 2000
`
`May 2000
`
`GD
`
`FIG. 19A
`
`1925A
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 40 of 67
`
`US 2009/0307615A1
`
`
`
`cdplayer bits
`Required changes to the AAP (B)
`floonTad
`
`Search: inran
`
`June 2000
`June 2000
`
`Keywords
`Copyright --
`Last used date May 2000
`
`pl minutes (81701.doc
`plimeline 082201.doc
`minutes. 073101.doc
`woc 2000-H. Labs
`mran
`window pix forimran
`window pix forman
`ran
`Help for imran
`system preferences wirrran.psd
`
`140 - Aqua Overview 5 14-imran
`140. Aqua overview 6/14-imran
`40-Aqua Overview 5/14-imran
`140. Aqua Owrview 514-man
`140-Aqua Overview 514-man
`imran
`MAP-Imanomaloappin.com
`Junk (mrandapple.com.mbox
`Sent Messages (mraneapoleoninboc
`Drafts (imrandapple.com).mbox
`
`List
`imran
`
`November 2002
`November 2002
`Nkwember 2002
`Nowember 202
`May 2000
`May 2003
`July 24:36 pm
`July 7 11:42 an
`Thursday 4:39 pm
`January 2000
`January 2001
`
`December 1999
`October 2001
`May 2000
`May 2000
`May 2000
`May 2000
`may 2000
`September 5 1:21 pm
`Thursday 3:50pm
`May 245:29 pm
`June 105:23 pm
`June 17208 am
`April 29 11:45pm
`May 241:21 pm
`June 15216 pm
`
`FIG. 19B
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 41 of 67
`
`US 2009/0307615A1
`
`1925
`0 0
`
`1930
`
`1903
`
`Search: te
`
`VFuture
`1932-YE
`
`. . . . .
`
`.
`
`
`
`1934
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`19361
`
`
`
`1938
`
`1940
`
`conciscoid
`outkiped
`Weathergadgatzip
`a Weathergedget
`chat
`30 mora
`Vlast week
`a volumes
`a Java
`suities
`Applications
`A7 more
`Vast month
`a 2434
`
`.
`January 4 3:38pm
`January 43:35pm
`January 4 3:34pm
`January 43:27 pm
`January 43:27 pm
`
`Today 11:12am
`
`Today 11:12am
`Today 11:12am
`Today 11:12 an
`
`Yesterdays3 pm
`
`.
`
`GD
`
`GD
`
`CD
`
`GD
`GD
`GD
`GD
`.
`GD
`
`
`
`Tuesday 6:48pm
`T Tuesday 6:48pm
`Tuesday 6:45pm
`Tuesday 64 pm
`.
`June 14 1137 pm
`
`.
`
`.
`
`.
`
`.
`
`FIG. 19C
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 42 of 67
`
`US 2009/0307615A1
`
`1925
`O O
`
`1950
`
`
`
`1903
`
`Search: te
`
`
`
`
`
`
`
`
`
`
`
`v Today
`1952- a Han
`in?opist
`Today 11:12 am
`GD
`...
`asdasd iman
`Today 11:12am
`GD
`SignaturesbyAccountpist
`today 11:12am
`CD
`Messagesorting past backup
`Today 11:12am
`CD
`more
`VYesterday
`1954-a-Ha
`CD
`Weathergedgetip
`Yesterday 9:16pm
`GD
`sweather gadget
`Yesterday 9:16 pm
`GO
`iTunes Music Libraryn
`Yesterday 8:24.pn
`GD
`Tunes 4Music Library
`Yesterdays:24 pm
`more
`.
`.
`.
`.
`Vlast week
`.
`.
`.
`1956-two canna Hirtatism - G
`incarcians
`note ion
`did
`A screwspacema
`June 1:05am
`GD
`12 Anopenetter to Nycna"
`June 16 11:04 am
`GD
`Shazanne
`June 1s 11:04 am
`GD
`10 more
`V Later 1958-and-aan HD
`webshare.gif
`August 75:33pm
`CD
`apachopbg
`May 2001.
`Gid
`
`GO r 2001
`
`FIG. 19D
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 43 of 67
`
`US 2009/0307615 A1
`
`
`
`L_,5?ETTEGGEAT
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 44 of 67
`
`US 2009/0307615A1
`
`Display a system wide Menu for inputting
`Search queries
`
`2001
`
`InputSearch, and as Search query is inputted,
`perform the search (begin searching and
`displaying search results before user finishes
`inputting Search query)
`
`Perform search through files, metadata,
`emails within an email program, address book
`entries within an address book, Calendar entries
`within a calendar program, etc.
`
`2003
`
`2005
`
`Display an abbreviated (incomplete) list of hits
`(if there are more than a certain number of
`hits), sorted by relevance and segregated h- 2007
`into groups (Categories)
`
`Receive Command to display all hits
`
`2009
`
`
`
`
`
`
`
`
`
`Display a search results window with ability
`to display 2 different types of view (e.g., icon
`and list views) within the same closeable,
`resizable, moveable Window
`
`2011
`
`FIG. 20
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 45 of 67
`
`US 2009/0307615A1
`
`RECEIVE USER'S SEARCH INPUT
`
`PRESENT (EG, DISPLAY) SEARCHRESULTS BASED
`ONUSERS SEARCH INPUT AND WITHIN THE
`RESULTS, PRESENTRESIZABLE ORZOOMABLE
`OR PAGABLE CONTENT
`
`2901
`
`2903
`
`RECEIVE USER SELECTION TO RESIZEOR
`ZOOM OR PAGE THROUGH THE CONTENT
`
`2905
`
`DISPLAY (OR OTHERWISE PRESENT) RESIZED
`OR ZOOMED OR PAGED THROUGH
`CONTENT WITHOUT LAUNCHING
`APPLICATION WHICH CREATED
`THE CONTENT
`
`2907
`
`
`
`
`
`
`
`
`
`
`
`FIG. 21
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 46 of 67
`
`US 2009/0307615A1
`
`
`
`3001
`
`About Xcode Tools.pdf
`
`Compatibility with Mac OS XVersions
`The Xcode 3.0 developer tools preview for Mac OS X 10.5 pay run exclusively on the pre
`release version of Mac OSX05, with which it is distributed. The Xcode 30 preview supports
`development for Mac OS X 10.3 E. Universal applications for Mac OS X 10.4 (Tiger) and Mac OS
`X 10.5 Preview (Leopard) using the Mac OS X SDK support.
`Please read the 'Software License Agreement for Xcode Tools Seed" document for special terms
`applying to this preview release.
`What's New
`Xcode 3.0 developer tools Ey. for Mac OS X 10.5 (Leopard) introduces many new features to
`improve develSPS productivity, provides greater debugging Control, and leveragés Leopard's new
`system-wide 64-bit capabilities.
`Developer Productivity
`Collection, properties, and for each
`Full support for
`2.0: New features include g
`language syntax to the Objective-C programming language. Note that Objective-C 2.0 is only compatible
`with the Mac OS X 10.5 (Leopard) preview.
`
`• New editor: New features include improved performance when loading large files, and streamline of
`the edits/buildldebug, process. For instance, errors and warnings can now be viewed inline within your
`code in message "bubbles", allowing youtg
`view errors and warnings beside the code you need to
`correct. Additionally, a new feature called Code focus combines new code folding features along with
`visual cues to highlight the block structure of your code.
`
`3002
`
`Eiring for Objective-C. Refactoring facilitates wide-scale restructuring of your project and class
`Structure,
`
`o Project S; Another new feature, enables you to save your complete project state, or restore
`your project to a previous state. Together with refactoring, these provide a safe mechanism to make
`
`dramatic changes to your entire project.
`• New interface Builder: This new yesign has an improved use interface, a new extensible library
`model and PE new Mac OS X 10.5 features such as Layerkit. For this preview, the previous
`version of IB is included as "interface Builder Tiger.app.
`Debugging Control
`Xray; Uses a collection of instruments to monitor and visualize your application and the OS over time,
`which are presented as a series of tracks similar to
`E. Band. Xray also allows you to Create your
`S.S." dtrace instruments. This enables you to quickly spot prob?ems and relationships within your
`application
`• Rewind. Allows you to step backward in your code while ESS; With Rewind, there is no need to 3003
`restart your entire debugging session simply because you stepp too far,
`3004
`Debugging Tooltip. Any text editor window can now display variable data as a tooltip while debugging
`
`3005
`FIG.22A
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 47 of 67
`
`US 2009/0307615 A1
`
`
`
`3010
`
`Universal applications for american PowerPC articles
`Please see the complete Xcode 3.0 preview release notes for more detailed information on the above
`features, and the many other new features in this preview release.
`Dashcode for Clashboard Widget Development
`Dashcode; A new application that includes everything you need to create Dashboard widgets
`within a single environment. Dashcode provides a number of templates, starting you off with a valuable,
`fully-functioning widget. From there you can easily change the look and controls within your widget using
`the design canvas. For mroe advanced developers you can write new code with the programmer's editor
`with Code Sense, and even debug your widget with the JavaScript debugger. Dashcode will take care of
`the packaging and deployment of your widget to Dashboard once you've perfected it.
`The Dashcode installer packages including at the top level of the Xcode Tools folder in the Leopard
`preview installation DVD, and will install into the DevelopertApplications folder.
`installing Xcode Developer Tools
`The Xcode.Tools.mpkg contains several separate sub-packages in the Custom instal panel that allow
`flexibility in installing and updating:
`O Developer Tools Software(DeveloperTools.pkg). This package contains Xcode, gob, and the other
`tools necessary for Mac OS X development.
`GCC 4.0.1 (goca.0.pkg) 'This is the latest version of goc. This compiler is necessary for 64-bit
`development, using the CWARFdebugging format, and development of Universal application.
`e GCC 33 (
`3-pig) - This package contains the gcc 3.3 compiler,
`Software Development Kits. These packages provide the header files and link libraries necessary for
`Mac OS X development. A development requires the Mac OS X SDK. This package provides the
`AP for basic Mac OS X technologies such as Carton, Cocoa, and corrmand-line tools. The other
`SOK packages provide AP for extended Macintosh technologies, such as QuickTime, OpenGL, etc.
`Revelope tools - These packages are used when developing J2SE5.0 applications for
`s
`Developer Documentation (Devdocumentation.pkg) - This package contains the core Mac OS X
`developer documentation for technologies such as Carbon, Cocoa, CuickTime, WebKit, etc.
`Big Examples ERE) - This
`econtains example code for a wide variety of
`Mac OS X technologies. Note that
`may require any of the above Software
`Development Kits to be installed before they will comple.
`... WebObjects (WebObjects,mpkg) - These package are used for WebOjects 5.4 development.
`
`Cross Development (MacOS 1039.pkg, MacOSX194u.pkg, MacOSX.O.5.pkg). These packages
`are for Mac OS X v10.4 (Panther). MacOS X v10.4 Es
`and Mac OS X 10.5 (Leopard) to allow
`cross development. These can be installed together or independently, and can be installed
`separately,
`ES tools (CHUD. pkg). This
`contains the Computer Hardware Understanding Developer
`Tools. For more information see http:lldeveloperapple.com/tools/performanced.
`installation instructions for Xcode 3.0
`
`30 12B
`
`a DPIE -
`3615. 3017 Yao19 302
`FIG.22B
`
`4
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 48 of 67
`
`US 2009/0307615A1
`
`
`
`3030
`
`Search Kit Reference
`
`QuickTime Kit Programming Guilde
`
`baceonear AAAsphaphanahas bee-sprus his sons banangoro
`anwy worn arand C- rapeus wers as weapua Asarwee flu
`- m rwisers wasmaseae arramaoa a wom
`a harwob at the sease
`anara unawants
`assaw as a
`
`1 wa? upri and
`
`GraphKit
`Reference Guide
`
`Archdiocturu and Performance Group
`Apple Computa, Inc.
`May 2004
`
`FIG.22C
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 49 of 67
`
`US 2009/0307615 A1
`
`
`
`GENERATEAN CON TO REPRESENT A FILE
`INA, GU210
`
`PUT ONE OR MORE PREDEFINED
`DECORATIONS ONTO THE CON BASED ON
`THE FILE TYPE213
`
`PRESENTING A PREVIEW OF THE CONTENT
`OF THE FILE WITHIN THE CONWITHOUT
`MODIFYING FORMAT OF THE CON215
`
`PROVIDING USER INTERFACE CONTROL
`BASED ON THE FILE TYPE TO ALLOW USERS
`TOMANIPULATE PRESENTATION OF THE
`PREVIEW 217
`
`MAINTAINING THE FORMAT OF THE CON
`WHILE PRESENTING THE PREVIEW EN
`RESPONSE TO ACTUATION OF THE USER
`INTERFACE CONTROL-219.
`
`F.G. 23
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 50 of 67
`
`US 2009/0307615 A1
`
`«N.9“.
`
`
`
`ZOO.kwwDOmm
`
`
`
` m3N270<5.0m".<k<o
`
`
`
`.._.<D200.213.51
`
`02.03402.
`
`.VZ<m20§<m00mo
`
`
`
`.._.<n_<._.m_2m0<s=
`
`".0w>_h<0_oz_
`
`
`
`m.w_>m_w_n_<mwrhm
`
`‘Nm._m<.=<><
`
`movm270341
`
`novNZOEm<D
`
`
`
`
`
`200.10“..mh<00mmmhz_
`
`ZOO.z<".0(.55
`
`mar.<GzFmemmEmm
`
`0gm
`
`
`
`ZOO.om<>>m0w
`
`
`
`O._.<H<D
`
`._.Zm_30
`
`ZO.._.<03n_n_<
`
`mwvw
`
`._.Zm_._0
`
`ZO_._.<0_.~n_n_<
`
`wovw
`
`
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 51 of 67
`
`US 2009/0307615 A1
`
`
`
`
`
`
`
`
`
`
`
`
`
`RECEIVE ICON DATA OF A FILE FROMA
`BACKGROUND PROCESS251O.
`
`
`
`ANALYZE THE CON DATATO
`DETERMINE FA PREVIEW OF THE
`ONTENT OF THE FILES AVAILABL
`2520
`
`
`
`PREVIEW NOT
`AVAILABLE
`
`GENERATE AN CON
`WITHOUTA PREVIEW
`WITHIN THE ICON 2529
`
`
`
`PREVIEW
`AVAILABLE
`
`GENERATE AN CONUSABLE TO PRESENTA
`PREVIEW WITHIN THE ICON 2523
`
`PRESENTA PREVIEW OF THE CONTENT OF
`THE FILE WITHIN THE CON WITHOUT
`MODIFYING FORMAT OF THE ICON 2525
`
`FIG. 25
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 52 of 67
`
`US 2009/0307615 A1
`
`
`
`wa
`
`2614
`
`FG. 26A
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 53 of 67
`
`US 2009/0307615 A1
`
`
`
`2622
`
`FIG. 26B
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 54 of 67
`
`US 2009/0307615 A1
`
`
`
`264
`
`2622
`
`FIG. 26C
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 55 of 67
`
`US 2009/0307615 A1
`
`
`
`2622
`
`FIG. 26D
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 56 of 67
`
`US 2009/0307615A1
`
`
`
`2710
`
`2715
`
`in 35.1.
`00:06
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 57 of 67
`
`US 2009/0307615 A1
`
`
`
`2725
`
`272O
`
`OOD5
`FIG. 27B
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 58 of 67
`
`US 2009/0307615 A1
`
`
`
`27.30
`
`2735
`
`FIG. 27C
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 59 of 67
`
`US 2009/0307615A1
`
`2700
`
`
`
`2730
`
`FIG. 27D
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 60 of 67
`
`US 2009/0307615A1
`
`
`
`2730
`
`Zn as
`
`FIG. 27E
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 61 of 67
`
`US 2009/0307615 A1
`
`RECEIVE ICON DATA OF A FILE, THE CON
`DATA INCLUDING FILE TYPE INFORMATION,
`BADGE DATA, AND ANIMAGE 2810
`
`SELECT DECORATIONS FOR THE CON
`BASED ON THE FILE TYPE
`282
`
`GENERATE BADGE USING THE BADGE DATA
`2814
`
`GENERATE AN CON OF THE FILE
`2816
`
`
`
`
`
`DISPLAY THE IMAGE, THE BADGE, AND THE
`DECORATIONS ON THE CON
`2818
`
`FIG. 28
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 62 of 67
`
`US 2009/0307615A1
`
`CREATE AN CONREPRESENTING A FOLDER
`2900
`
`ARE ALL FILES WITHIN THE FOLDER
`CREATED BY APPLICATIONS FROM
`THE SAME VENDOR
`2910
`
`DISPLAY THE CON
`OF THE FOLDER
`WITHOUT ANY BADGE
`299
`
`CREATE ABADGE INDICATIVE OF THE
`VENDOR
`293
`
`DSPLAY THE BADGE ON THE CON OF THE
`FOLDER
`2915
`
`FIG. 29
`
`
`
`
`
`
`
`
`
`
`
`
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 63 of 67
`
`US 2009/0307615 A1
`
`
`
`s :
`
`FIG. 3OA
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 64 of 67
`
`US 2009/0307615A1
`
`
`
`3060
`
`FIG. 3OB
`
`-
`
`3074
`
`XCODE
`
`3072
`
`FIG. 3OC
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 65 of 67
`
`US 2009/0307615 A1
`
`G009.
`
`
`
`-}O LEIS
`
`
`
`SEN]] [[OH
`
`'9'E)
`
`(SNI-?nºld
`
`ERHV/NAI–JOS
`HOLVHEINES)
`
`
`
`MAE HAER?d
`
`(NOWEVO)
`
`6008
`
`
`
`
`
`
`
`ERHOW HO ENO
`
`SNOLIVO|Tc|c}\f
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 66 of 67
`
`US 2009/0307615 A1
`
`
`
`

`

`Patent Application Publication
`
`Dec. 10, 2009 Sheet 67 of 67
`
`US 2009/0307615 A1
`
`mm.07.
`
`
`
`Handmw>mwm
`
`
`
`MI...20w.__n_ml...noZOO—MI...wIU<O
`
`Edd
`
`
`mahommwmm<Ohwe“.<"—0
`
`
`
`4.33225.<KOBZ<>>m_>mmn_<wozmwm02<hw7=MI...
`
`._.Zm_._om>Fowawmmmi...50m“.hwmaomz<O...
`
`
`
`
`
`m2m20¢wmmwzwiofizhzmfio
`
`
`
`“andMET—052
`
`m2...Emmmmm9zo:.<o:&<moEmmo<:2583083
`2.63.5<msomxw
`
`
`
`
`
`
`
`
`
`m562.10225Sort;265mm,".
`
`
`
`
`
`“can20.502“?«.955
`
`
`
`>>m_>wmn_<“—0mozdFmZ.Z<m._.<_._.z<...wz_
`
`
`
`
`
`m0“.mw>mmw<20202w<o02F<mw2w0
`
`
`
`
`
`mz=..o<§._.2m_._0:05
`
`
`
`
`
`
`

`

`US 2009/03076.15 A1
`
`Dec. 10, 2009
`
`BROWSNG OR SEARCHING USER
`INTERFACES AND OTHER ASPECTS
`
`for data files, but this search does not also include searching
`through metadata for those files.
`
`0001. This application claims the benefit of U.S. Provi
`sional Patent Application No. 61/059.614, filed on Jun. 6,
`2008.
`
`BACKGROUND OF THE INVENTION
`
`0002 Modern data processing systems, such as general
`purpose computer systems, allow the users of such systems to
`create a variety of different types of data files. For example, a
`typical user of a data processing system may create text files
`with a word processing program Such as Microsoft Word or
`may create an image file with an image processing program
`such as Adobe's PhotoShop. Numerous other types of files are
`capable of being created or modified, edited, and otherwise
`used by one or more users for a typical data processing sys
`tem. The large number of the different types of files that can
`be created or modified can present a challenge to a typical
`user who is seeking to find a particular file which has been
`created.
`0003 Modern data processing systems often include a file
`management system which allows a user to place files in
`various directories or subdirectories (e.g. folders) and allows
`a user to give the file a name. Further, these file management
`systems often allow a user to find a file by searching for the
`file's name, or the date of creation, or the date of modification,
`or the type of file. An example of Such a file management
`system is the Finder program which operates on Macintosh
`computers from Apple Inc. of Cupertino, Calif. Another
`example of a file management system program is the Win
`dows Explorer program which operates on the Windows oper
`ating system from Microsoft Corporation of Redmond, Wash.
`Both the Finder program and the Windows Explorer program
`include a find command which allows a user to search for files
`by various criteria including a file name or a date of creation
`or a date of modification or the type of file. However, this
`search capability searches through information which is the
`same for each file, regardless of the