(19) United States
`(12) Patent Application Publication (10) Pub. No.: US 2009/0106674 A1
`Bray et al.
`(43) Pub. Date:
`Apr. 23, 2009
`
`US 20090106674A1
`
`(54) PREVIEWING USER INTERFACES AND
`OTHER ASPECTS
`(76) Inventors:
`Cedric Bray, Vincennes (FR):
`Scott Forstall, Mountain View, CA
`(US); Yan Arrouye, Mountain
`View, CA (US)
`Correspondence Address:
`BLAKELY SOKOLOFF TAYLOR & ZAFMAN
`LLP
`1279 OAKMEAD PARKWAY
`SUNNYVALE, CA 94085-4040 (US)
`
`(21)
`
`Appl. No.:
`
`11/876,750
`
`(22)
`
`Filed:
`
`Oct. 22, 2007
`
`Publication Classification
`
`(51) Int. Cl.
`(2006.01)
`G06F 3/00
`(52) U.S. Cl. ........................................................ 71.5/762
`(57)
`ABSTRACT
`User interfaces for browsing, searching, and/or previewing
`are described. In one embodiment, a method includes call a
`preview generator to generate an initial preview in response to
`receiving an input, receiving a content to be displayed from
`the preview generator, displaying the content as the initial
`preview, and displaying and Scrolling concurrently an
`enhanced preview to a next item in a representation of items
`in a search result in response to an input to directionally move.
`Other embodiments are also described, and computer read
`able media and apparatuses are also described.
`
`Capture metadata from a variety
`of different application programs
`
`2O1
`
`
`
`
`
`
`
`
`
`Make Captured metadata available to a
`Searching facility (e.g., a file management
`system software for searching)
`
`203
`
`Allow searching of metadata across all
`applications having captured metadata
`
`205
`
`Provide User Interface of search engine
`and Search results
`
`2O7
`
`Roku EX1005
`U.S. Patent No. 10,334,311
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 1 of 81
`
`US 2009/0106674 A1
`
`
`
`Microprocessor(s)
`103
`
`ROM
`107
`
`Mass Storage
`(e.g., HardDrive)
`106
`
`Bus(es) 102
`
`
`
`
`
`Display Controller
`and
`Display Device
`104
`
`
`
`
`
`
`
`IO
`Controller(s)
`108
`
`
`
`
`
`
`
`
`
`
`
`
`
`I/O Devices
`(e.g., MOUSe Or
`Keyboard or
`Modem or
`Network interface or
`CD Drive, etc.)
`109
`
`FIG. 1
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 2 of 81
`
`US 2009/0106674 A1
`
`Capture metadata from a variety
`of different application programs
`
`2O1
`
`
`
`Make Captured metadata available to a
`Searching facility (e.g., a file management
`system software for searching)
`
`
`
`2O3
`
`Allow searching of metadata across all
`applications having captured metadata
`
`205
`
`Provide User Interface of search engine
`and Search results
`
`2O7
`
`FG.2
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 3 of 81
`
`US 2009/0106674 A1
`
`301
`
`331
`
`JPEG image File Metadata
`
`ISO Setting
`Flash Setting
`
`fl Stop
`
`Image Width
`Image Height
`
`Image Color Space
`
`Camera Brand Information
`
`BitS/Pixel
`
`User Added Keywords
`
`Other Fields.
`
`FIG. 3A
`
`
`
`MP3 Music File Metadata
`
`Song Play Times
`
`Other Fields.
`
`Artist
`
`Genre
`
`Album
`
`Song names
`
`F.G. 3B
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 4 of 81
`
`US 2009/0106674 A1
`
`403
`
`OS
`Kernel
`
`Metadata Processing
`Software
`
`Fle
`System
`Directory
`for
`Metadata
`
`419
`
`
`- 40 5
`File System
`GUI SW
`
`407
`
`
`
`Other
`Applications
`
`Importers
`(+ Optionally
`Exporters)
`of Metadata
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`400
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 5 of 81
`
`US 2009/0106674 A1
`
`
`
`Receive notification of change for a file (e.g., receive
`notification from OS kernel of Software application of
`Creation of new file or modification of existing file or
`deletion of existing file).
`
`501
`
`Determine type of file (e.g., an Acrobat PDF file oran
`RTF word processing file or a JPEG image file, etc.)
`
`503
`
`Activate capture software for determined file type
`(e.g., a plug in for the application which creates
`the type of file)
`
`505
`
`import appropriate metadata (for particular file type)
`into a metadata database
`
`507
`
`Store metadata in a database (e.g., a flat file
`format database)
`
`Receive search parameter inputs and perform search
`of metadata database and display results
`
`509
`
`511
`
`F.G. 5
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 6 of 81
`
`US 2009/0106674 A1
`
`
`
`603
`
`605
`
`607
`
`609
`
`611
`
`Metadata from file 1 of Application A (File A1)
`
`Metadata from file 1 of Application B (File B1)
`
`Metadata from file 2 of Application A (File A2)
`
`Metadata from file 1 of Application C (File C1)
`
`Metadata from file 2 of Application B (File B2)
`
`617-- Metadata from file n of Application C (File Cn)
`
`F.G. 6
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 7 of 81
`
`US 2009/0106674 A1
`
`
`
`S?uatundoqQ
`
`seiniola [[]]
`
`Sae?
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 8 of 81
`
`US 2009/0106674 A1
`
`Kepol
`
`
`
`| 1G! 1
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 9 of 81
`
`US 2009/0106674 A1
`
`
`
`101 HF, FIEFS
`
`C-5) , g?,
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 10 of 81
`
`US 2009/0106674 A1
`
`
`
`0
`1
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 11 of 81
`
`US 2009/0106674 A1
`
`
`
`
`
`/ | | ~~~~ “+`o
`
`E
`
`S.
`
`3.
`
`;
`
`: EEE ES
`
`Es:
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 12 of 81
`
`US 2009/0106674 A1
`
`
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 13 of 81
`
`US 2009/0106674 A1
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`š?asse
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 14 of 81
`
`US 2009/0106674 A1
`
`646
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 15 of 81
`
`US 2009/0106674 A1
`
`
`
`
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 16 of 81
`
`US 2009/0106674 A1
`
`
`
`
`
`
`
`
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 17 of 81
`
`US 2009/0106674 A1
`
`
`
`· 880 || ||
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 18 of 81
`
`US 2009/0106674 A1
`
`
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 19 of 81
`
`US 2009/0106674 A1
`
`
`
`90 || ~
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 20 of 81
`
`US 2009/0106674 A1
`
`
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 21 of 81
`
`US 2009/0106674 A1
`
`
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 22 of 81
`
`US 2009/0106674 A1
`
`
`
`OZ|
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 23 of 81
`
`US 2009/0106674 A1
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 24 of 81
`
`US 2009/0106674 A1
`
`
`
`
`
`?oueas weN \,
`
`
`
` Œ] [5] [5] | 41
`
` [5] Œ [8] Q [5] [×]
`
`
`[5]
`F
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 25 of 81
`
`US 2009/0106674 A1
`
`
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 26 0f 81
`
`US 2009/0106674 A1
`
`140?
`
`1405
`
`May 27. 2000. 4:10 PM
`
`Adub...imag
`TIFFC .me
`TIFF owe
`TIFFC .me
`TIFF C...
`TIFF C... :
`TEFF C...
`TEFF C...
`
`pubiiupg
`publéc.;pg
`pumicjpg
`
` aubécwg
`
`pubficjpg
`pubiic.jpg
`pubéiculpg
`Dnbéicépg
`pubiicgpg
`publicjgg
`guulicpg
`pumicwg
`publécwg
`1311:200ij
`
`' 3.p9d
`I 1de
`I 8930'
`I 9.53mi
`3 ABGenarakPrafsMefiuieié!
`a ABLandscape.a§fE
`g ABLDAPPrefsModule.fl§
`{g ABMs[Wolframisonwrefshicaiuéeisi
`[a ABPhoneFarmaiprefsMadufem
`ABPonratt lift
`ABTemeau-aPrafsMoa’ulexJ?
`B ABVCaru‘PrefsModuteJif
`I?) AddreasBmx,acns
`I2] acache‘pbgvf
`m Finch frff
`a backumcns
`davisjpg
`dichonampsd
`E diclbnarybmwserpsd
`dockpsd
`I DSCFDDO‘UPG
`I DSCFOOOZJPG
`fl DSCFODOEJPG
`“ DSCFBGOSJPG
`fl DSCFGOONPG
`I DSCFBDOEJP$
`I DSCFGOOQJPG
`I DSCFGDOWJPG
`I DSCFGDGHJPG
`I DSCFGBMZJPG
`I DSCFOOD'ISJPG
`‘ DSCFOOGMJPG
`fl DSCFOQmSJPS
`. DSCFGOmSJPG
`I Dscmwnpa
`I QSCFGQ’B‘EMPG
`‘ QSCFOOOTWPG
`fl DSCFQOEZOJPG
`E DSCFUOGZHPG
`‘ DSCFOOGRJPG
`
`Oc18,!99§.?1102 AM
`0616 1999. 13;2" AM
`OCH}. 1999,1114 AM
`Oct6,1999.3116 PM
`0:215, 1909. 3:19 PM
`Jun 17'. 2004. 10.21 PM
`Jim 17. 2004. 15:21 PM
`Jan17. 2004. 10:21 PM
`Jun 17. 2004. 10:21 PM
`Jun1?.20€34.
`10:21 PM
`Jun1?.200t€,
`18:21 PM
`Jun 112004. 10:21 PM
`Jun17.2004, 10:21PM
`”3132004. 10:21 Fifi
`Jun 17. 200:2. 4:11 PM
`Jun1?.2f304. 10.21 PM
`Jun 1?, 2004. 10:21PM
`Yeeierday. 2.12 PM
`Mar 30. 2004. 3:59 PM
`Mar 30. 2004, 3:59 PM
`A9? 27. 2000, 4:51 PM
`May 27. 2000. 4.03 PM
`May 21. 2000. 6:04 PM
`May 27. 2000, 4:04 PM
`May 27. 2000, 4:05 PM
`May 27. 2000, 4:05 PM
`May 27. 2000. 4:06 PM
`May 2?, 2000. 4:06 PM
`May 27. 2000. 4:06 PM
`May 27. 2060. 4:56 FM
`May 2?. 2000. 4:0?r PM
`May 27. 2090. 4:67 PM
`May 2?. 2000. 4:6? PM
`May 27. 2000, 4:58 PM
`May 212000.413 PM
`May 27, 2000, 4.08 ?M
`May 27. 2000. 4.09 PM
`May 27, 2000. 4:09 PM
`May 27. 2000. 4:69 PM
`May 2?, 2000. 4:10 PM
`
`78 hams. ‘> 4.7 GB auaiiabée
`
`1403
`
`1401
`
`FIG.14A
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 27 of 81
`
`US 2009/0106674 A1
`
`
`
`This Week's images
`
`davis.g
`
`S{{{2i.
`
`Sé
`
`Sis
`
`SCitats
`
`ESCO2:
`
`E.
`
`S.
`
`SC336.
`
`S
`
`J {
`
`s
`
`d
`
`S.
`
`WN
`
`sixt:373
`
`2.
`
`2.
`
`CSC,020.JPG
`
`Sii.2126
`
`3.
`
`s C 02 4. gic
`
`E.
`
`}SCO233
`
`as
`
`SR
`
`GSct2tic
`
`SCC03E3
`
`ESCO(35g.
`
`s s
`SC33}{
`
`$ itsis. 4 GB awaiiate
`
`.
`Eys
`
`3 C
`
`2
`
`---
`SCG
`
`Se
`
`E.
`
`SS SS
`Sect343G
`
`1401
`
`1403
`
`FG, 14B
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 28 of 81
`
`US 2009/0106674 A1
`
`
`
`1405
`
`Show Wiew Optics
`ice Search Criteria
`Save Seart:
`
`is 8es
`. .
`
`.
`
`. .
`
`.
`
`.
`
`F. ABGeneralPrefs Modula.tif
`ABLanciscage.tif
`E AB, APPreisblockie.tif
`E ABMai:NotificatichPre?sMcduiet:
`ABPhoneFormatPre?sModule.tif
`EAEPartrait.tiff
`2 ABTemplater refs&cdui.e.tif
`ABWCardPrsfshodule.tif
`Acci essecoi.cs
`E. apache pt.gif
`Back.tiff
`F. lackup, cris
`Edavis.jpg
`a dicticinary psc
`dictionarybrowser.asd
`dock.psd
`SCF001.J. G.
`SCF2RG
`ESCO3Cl3:
`SCF0005.FG
`DSCFC37.jFG
`SCF0008.JPG
`SCF0(SJPG
`- SCF001(PG
`- ESCF0(1;;
`- SCFG}(12G
`SCF00013.JPG
`SCF){C} 14.JPG
`DSCF008:15 PG
`SCFO}{16. G
`... SCF(Qia
`- DSCF009 i8; Ps
`SCF0(33.j PG
`DSCF00020.jpg
`ESCFC?)2.jFG
`SCF0023.JPG
`
`Oct 6, 1893, 11.92 Asi
`Ost 6, 1993, 12 AA
`s: 6, 1939, 1:24 Aisi
`cc 6, 1999, 3.6 F:
`{}ct 6, 1993, 3, 19 A
`ius 17, 2034, 3:2
`in
`Jut 17, 2004. 13:2; Ph.
`jug 17, 2004, 13:21 FM
`jith 7, 2004. 6.2 FM
`jiFi ?, 2004, t{}:t PA
`Jr. 7, 2004, 10:2t FM
`Jin f7, 2004, 10:21 FA
`Jugi 7. 2004, 10:21 PAA
`Lii, 7, 2004. 102 &
`dur: 1,2304, 4:1 iP.
`Jar: 17, 24, 3.2 Psi
`Jirt 1,2304, 2; Phi
`Yesterday, 2:2PM
`Maf 30, 2004.3:59 PM
`Mar 30, 2004, 3:59 PM
`Apr 27, 2000, 4:51 Psi
`May 27. 2030, 4:03 PM
`May 27, 208, 4:04. PA
`May 27, 2000. 4:04 aka
`May 27, 2030. 4:5 PM
`Aday 27, 2000, 4:05 Pat
`May 27, 2000, 4:06 P&
`May 27, 2008, 4:06 Psi
`May 27, 2000, 4:06 P.
`May 27, 2000, 408 FM
`May 27, 2000, 4:07 PM
`sday 27, 2300, 4.07 Phi
`May 27, 2000, 4:07 Piwi
`May 27, 2000, 4:08 Pi,
`May 27, 2000, 4:38 PM
`May 27, 2000, 4:8
`May 27, 2000, 4:09 PM
`May 27, 2003. 4.9 PM
`May 27, 2003, 4:09 Phi
`May 27, 2000, 4:30 PM
`May 27, 2006, 4:10 Phi
`
`Adob... if tag
`Libic.jpg
`pubic.jpg
`ght::it: gig
`public.jpg
`public.jpg
`public.jpg
`pubic.jpg
`piattic.jpg
`pusic.jpg
`tattic.jpg:
`public.jpg
`glitti.jgg
`ublic.jpg
`guttie.jpg
`...tigg
`glitic.jpg
`public.jpg
`public.jpg
`public.g
`
`8 it:F3s. 4 G3 awaiiate
`
`1403
`FIG. 14C
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 29 of 81
`
`US 2009/0106674 A1
`
`- 1430
`
`O
`
`Smart Folder
`
`Con size: 128 x 128
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`Large
`:
`
`Smal
`12pt
`Text size:
`Label position:
`(C) Bottom O Right
`OSnap to grid
`OShow item info
`OShow icon preview
`RM Keep arranged by
`Name
`Background:
`O White
`(C) Color
`O Picture
`
`
`
`FIG. 14D
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 30 of 81
`
`US 2009/0106674 A1
`
`
`
`1502
`
`X () Thu 10:42 AM
`
`1511
`
`1513
`
`Documents E. Shakelt.gadget.zip
`
`PDF DOCUrments
`
`don'tshakeit out.pdf
`don'tshakeit.pdf
`
`FIG. 15A
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 31 of 81
`
`US 2009/0106674 A1
`
`1502
`
`1505
`
`3 () Thu 10:47 AM
`
`o
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`1511
`
`
`
`1513
`
`1507
`
`1509
`
`1515
`
`1503
`
`Top Hit
`
`Documents
`
`--- Folders
`
`Show All (392) - -
`O actionbutton
`button.psd
`Action Button.dir
`Action Button...dir
`Metal buttons.psd
`animgif.txt
`animgif.txt
`bitmap.txt
`bitmap.txt
`bitmap.txt
`bitmap.txt
`g button.waw
`g button, wav
`Cursor.txt
`CurSOr.txt
`default.txt
`default.txt
`actionbutton
`O new buttons
`action button icon
`alex - clear button
`FG. 15B
`
`
`
`
`
`
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 32 of 81
`
`US 2009/0106674 A1
`
`
`
`éd és (66 (6)
`
`g
`s
`
`(9) (5
`
`?
`
`(9) (6
`
`(9 (5. (S 6 (6
`
`i
`
`(si?ts (s
`
`&
`g
`
`-
`
`3.
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 33 of 81
`
`US 2009/0106674 A1
`
`
`
`9.
`
`s
`
`f
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 34 of 81
`
`US 2009/0106674 A1
`
`
`
`1614
`
`61 1
`
`outcapsd
`
`inri
`1612-E is
`1613
`action::it:::
`new buttons
`
`Search button
`
`392 Resuits
`
`Sept."rber 133.55 gift
`f's 233
`is 2003
`March 2033.
`
`cate 22
`-
`33: Lay 5 4:45 pr:
`
`February 2:03
`Decenter 2032
`:Noverber 2002
`
`November 1999
`lovenborgs ""
`were 999
`
`G) Kili
`()
`late
`G. People
`()
`Fiatist
`is wins...
`()
`series"
`ae
`C.
`s
`
`Search:
`Everywhere
`ge
`testo work
`Stats
`Fl
`
`161 5 r
`
`1 6 7
`
`16Blateris8.5tri
`
`agweisel 999
`
`e-F-2
`M}w 9-138-232
`Ne'er 22
`syster 202
`overber 2002
`Nshyer'sse? 2002
`
`butteririlacs.gif
`buttgabackgrourd gif
`
`rushetitie?; 31 graphite).pct
`ignore.
`------------------. . .
`.
`
`()
`
`1619
`
`EAHS.spdf
`Nortar isz Eito Xipif
`
`TNovember 22
`Septe:Tiber 2002
`
`Newater 2)
`Gctober 21
`
`1621
`
`E sunsphereButton java. . . .
`Esseers fostcajava
`E. SunSpheresortstoniawa
`
`.
`
`.
`
`.
`
`May 2001
`day 20
`
`G).
`G. G
`()
`
`January 22
`A:ig Jsi 200
`its 2000
`
`F.G. 16A
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 35 of 81
`
`US 2009/0106674 A1
`
`1625
`
`
`
`1611
`
`1614A
`
`1613
`
`Search butto
`
`fartiary 5 4:45 inn
`April 2003
`Fsbruary 2303
`
`1601
`
`1603
`
`392 Resists (C3 utton
`:
`
`1 61 5 -
`
`R SEis8.3:
`s SBasssss
`s
`ns.html"
`
`live city: 999
`were: 999
`
`O
`
`Flat st
`
`Within Group sort by;
`Sierra
`Cate
`Kic
`recipe
`Search:
`everywhere
`in
`
`1805
`
`607
`
`1609
`
`1627
`
`1619
`
`1 62
`
`1623
`
`1617
`
`v. ..
`
`i
`butcrbackground.gif".
`butterlayer.gif
`a PushButto: G1 graphitect
`
`AHG.5:1.pdf
`Norton Disk Epiter xpdf
`a featerpdf ---
`
`we?ter 22
`we'er 2002
`Mweiler 302
`N}ve':e 2002
`Niverber 3032
`
`awarb: 2203
`32ptember 2002
`
`{esigring AirPor: Networks2.pdf
`
`(thi? 20
`
`W Source code
`tripfisBi;t of awe
`SfS:sfeBitor.jaya
`Sir Shefesastutor'.;awa
`suspiereifssutungava
`SunSahereotButton.java
`
`fe Librugs (323.01.doc
`Rada: 5.8,
`ises arisai
`Radar 40 User manua
`is GC Player its
`
`May 200t
`&lay 2001
`&lay 2301
`
`3:voriter 2002
`
`August 2000:
`sne 28t
`June 2C(st
`
`FIG. 16B
`
`()
`
`O
`
`{
`
`6)
`{)
`()
`()
`()
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 36 of 81
`
`US 2009/0106674 A1
`
`
`
`1894 A
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 37 of 81
`
`US 2009/0106674 A1
`
`
`
`1814
`
`1 8 1 1
`
`irri
`
`anings.txt
`
`1813 W Fciders
`
`refixillars
`again bl:en:ric
`alex - clear buttor
`He's Biation
`
`1 81 5 whil
`R 3Buttgas3.htm
`a 1dButtonso.html
`15Buies-4.3 tra
`siftiss-stris
`Stassif
`ifics.
`
`1817 - Y flages
`
`...)
`
`Search: buttor:
`
`392 Resuits
`
`--
`
`Jute 3.
`the
`war: 2003
`1Y
`fit
`Ceretter 22
`
`O EP By:
`sate
`eggie
`Fialisi
`Withirs group Sort by:
`
`ato
`
`Piece
`
`Search;
`Everywhege
`sia
`
`daruary 5 4:45
`a?: lar
`
`f
`
`Feruary 2003
`December 2:32
`over:bar2C2
`
`Ngwere 999
`Nowatter 1999
`Noster 1399
`Nireater 999
`Ntswe?ter 1999
`
`EE-- 1831
`
`exist:
`
`1818
`
`button.cs.gif
`Newssta 232
`
`button...ace.gif
`Nyetter 202
`
`button...nd.gif
`Navesite 262
`
`bistoriayer.gif
`Yerse 2002
`
`Pushai...tepc:
`Novariba: 23?2
`
`1821
`
`1823
`
`critugs asido:
`E. Ratar 5.3.1 ise hai: a
`Racia aserrancial
`EC Player bits
`
`,,,
`
`overritor 2002
`Sagterber 22.2
`May 2002
`::::: 20
`
`iwei ba? 2)2
`Jarliary 302
`A gust2COC
`trait
`it's 86.
`
`FIG. 18A
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 38 of 81
`
`US 2009/0106674 A1
`
`
`
`Group. By;
`Kiard
`take
`easie
`Fia: is
`
`Within group Sort by:
`N:8
`at:
`Kif:
`ecske.
`
`Search:
`everywhers
`! E:e
`tests work)
`scratch;
`:
`
`actier button inco
`3:ax iear tutor,
`
`V HTML.
`isButtoiss.html
`A Suttons.htri
`
`is?uary 2003
`8:8:38 2002
`Nic:Weir 22
`
`figvaribar 1999
`November 1939
`Na:vertist 1999
`
`buttan...ace.gif
`button.cis.gif
`Neweiser 22 Nswere 22
`
`tutor...fhd.gif
`Nowse C2
`
`button layer.gif
`Nevert&f 2032
`
`PushBu...te.pct
`Now sibs 22
`
`O
`
`PushBu... pet
`(wπt 22
`
`001. Button
`with
`2002
`
`122., butt or
`Nawsite: 20
`
`12 (3...button
`- wetter 2002
`
`2 (34. Dutton
`NewEfter 28.32
`
`12 C3...button
`Nygmire 222
`
`12:8... button
`higwernber 202
`
`2.19.8 tten
`Neveriter}:
`
`1222...button
`iwetter 22
`
`35E button
`Nowsfief 292
`
`45...Button
`1365. ittons
`it::::::: 22 Nawether 2f}{3.
`
`53C..tters
`Mits:::::: 2;
`
`1828Raised
`Mwar:8:20:2
`
`F.G. 18B
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 39 of 81
`
`US 2009/0106674 A1
`
`1925
`
`
`
`Search iri: a
`
`w infa Chaifi
`1911- is
`Fer Test
`1913-YA
`skiiP-star:grias.aspie.corn
`Erafts met:appie isotr ribox
`E. 3:
`&eit 'essages it far:3::::::...F.si:
`
`1915 - W. Sarah A. Brody
`pagendagaaeedoc
`EifriteS 81 Gide
`p: irrelite (822.doc
`talutes (31 cide.
`
`1917
`
`AwBC 2000. Hii.abs
`
`a
`day 2009
`
`Eirsday 4,38 fr;
`Thirsay 3.5i
`gst
`8
`isfits 52:8:
`ise i3.5.23 it:
`
`Nwere: 3.2
`N::::Fai
`ovember at
`SwsFE)}.
`
`Xiay 23.
`
`)
`
`Wii Sigt Suri by:
`arts:
`late
`id
`i.e.:pfg
`
`St:
`Everwhere
`isis
`this: Wi:rk
`... i. s:Fic:
`33
`
`()
`
`T
`
`()
`
`FIG. 19A
`
`1925A
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 40 of 81
`
`US 2009/0106674 A1
`
`Within Group Sort by:
`ise
`Cats
`Kitsi
`Paopis
`Search:
`Everywhere
`-its
`teste (work)
`Ec
`si
`
`
`
`
`
`
`
`
`
`
`
`G
`
`(i)
`G
`
`(i)
`(a)
`
`
`
`
`
`
`
`
`
`
`
`
`
`1925
`
`O &
`
`. .
`cc Player is .
`Required Charges to the AAP is:
`
`.
`
`minutes 731 doc
`woc acco. H. Labs
`infar
`
`Searc imar
`
`.
`
`. June 20c
`
`Kaywa Fids
`{i}{Tight
`last 1333 date
`
`way 2000
`
`November 2002
`Nevertier 2002
`were 23&
`wit::ir 82
`?tay 2008
`Piay 2a3
`
`January 231
`
`Keywords - -
`Copyright
`- -
`ast usec date
`jily 200
`Height
`$538
`is 248
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`140 - Aqua Oyarview 5:4-irrain
`34. Aqua overviews 14-irran
`40. Aqua Cve wigw 54 -FEarl
`4- Acta was view 5:14 if:;ar.
`Emran
`IMAP, Frrarigraticiapsia, first
`
`Sent Massages (infangagpie.cert.rboe
`Grafts firirangeapple.com),n-box
`Suthwest Airlines ceasess awe Coffirratign
`List
`Simran
`
`Sighterta 5.2 pr:
`Thursday 3:53 pm
`
`Egins 13.5.23 aft:
`Eise if 2:06 arm
`April 2g 11:45 pm
`hiay 24:21 pt?,
`June 152.18 pm
`
`FIG. 19B
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 41 of 81
`
`US 2009/0106674 A1
`
`1925
`
`
`
`- 1930
`
`- 903
`
`Searchite
`
`1932 F--forces
`
`i
`
`905
`
`ja:iiiary 3.36 pr.
`a: yaf
`3.3
`a:Eary 33.34.
`
`Everywere
`3ts
`East work
`
`sale
`
`1 908
`
`1926
`
`1934
`
`W Tasia
`F
`assig&ogist.acks:
`SOFF13BE.C308 E5.9F-3003938E331CiAGPerscrwcf
`-
`last E. Totiay it;2 at:
`
`g
`
`1 945
`
`M
`1946
`
`Appie Compute, irc.
`
`1 938
`
`as Wei
`System Preferences
`Wirtas
`
`Litities
`Applications
`
`1940
`
`ray 11:12 at
`aday 1:13 an:
`i:;ay 11:12 at
`
`Yeaterday 9:18
`westerday
`on
`Yesteriay 8.31 pm
`
`Tisscay 5.5 p?
`Tessay B:48 pr;
`
`Tesday 8:45
`usesday 3,
`
`it
`
`D
`g
`
`GD
`
`F.G. 19C
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 42 of 81
`
`US 2009/0106674 A1
`
`
`
`& O
`
`1952
`
`v today
`Picture as
`
`as: far:
`Sislatures A^{33}rt-lis;
`Messagesorting estacker
`
`1954-
`
`ouge.
`
`weather gadget
`
`3 Crawlspace.T.4 a
`1: An Open letter to NY(...m4a
`11 Shazam!, in 4a
`
`1958-ia
`Xmassioges
`
`Searchite
`
`T.
`today 12am
`G. K.
`
`Ex:y $1.2 a?
`Eaciay 11:2 art.
`cay 11:2 an
`
`D
`g
`
`st is
`
`Withir Group Sort by:
`life
`
`Perciela
`
`Everywhere
`Home
`Eest :work
`Sciacit
`}
`
`Yesterday 8:18 pm
`Yesterday 8:24 pm
`
`15:1
`usta 8.34 aft:
`- is 16 1:4 at
`
`a
`way acci
`
`FIG. 19D
`
`1925A
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 43 of 81
`
`US 2009/0106674 A1
`
`
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 44 of 81
`
`US 2009/0106674 A1
`
`Display a system wide Menu for inputting
`search queries
`
`2001
`
`InputSearch, and as search query is inputted,
`perform the search (begin searching and
`displaying Search results before user finishes
`inputting search query)
`
`Perform search through files, metadata,
`emails within an email program, address book
`entries within an address book, Calendar entries
`within a Calendar program, etc.
`
`Display an abbreviated (incomplete) list of hits
`(if there are more than a certain number of
`hits), Sorted by relevance and segregated
`into groups (Categories)
`
`2003
`
`2005
`
`2007
`
`Receive Command to display all hits
`
`2009
`
`Display a search results window with ability
`to display 2 different types of view (e.g., icon
`and list views) within the same closeable,
`resizable, moveable window
`
`
`
`2011
`
`FIG. 20
`
`
`
`
`
`
`
`
`
`
`
`
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 45 of 81
`
`US 2009/0106674 A1
`
`
`
`
`
`RECEIVE USERS SEARCH NPU
`
`PRESENT (E.G., DISPLAY) SEARCH RESULTS BASED
`ONUSERS SEARCH NPUT AND WITH IN THE
`RESULTS, PRESENT RESIZABLE ORZOOMABLE
`OR PAGABLE CONFEN
`
`290
`
`2903
`
`RECEIVE USER SELECTION TO RESEZEOR
`AOOM OR PAGE THROUGH THE CONTENT
`
`2.905
`
`DISPLAY (OR OTHERWISE PRESENT) RESIZED
`OR ZOOMED OR PAGED THROUGH
`CONTENT WITHOUT LAUNCHNG
`APPLICATION WHECH CREATED
`THE CONTENT
`
`29C 7
`
`FIG. 21
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 46 of 81
`
`US 2009/0106674 A1
`
`
`
`(x)
`
`About Xcode Tools.pdf
`
`300
`
`Compatibility with Mac OS X Versions
`the Xcode 3.0 developertoois preview for Mac OS X 10.5 Leopard) will fun exclusively on the pre
`release version of Mac OS X 10.5 with which it is distributed. The Xcode 30 preview suppoits
`development for Mac OS X 10.3 Panther). Universal applications for Mac OS X 10.4 Tiger) and Mac OS
`X 10.5 Preview (Leopard; using the Mac OS X SDK support.
`Please read the "Software License Agreement for Xcode Tools Seed" document for special terms
`applying to this preview release,
`What's New
`Xcode 3.3 developer topis preview for Mac OS X 10.5 (Leopard) introduces many new features to
`improve developé productivity, provides greater debugging Control, and leverages Leopard's new
`system-wide 64-bit capabilities.
`Developer Productivity
`* Full support for 3. 2.0. New features include garbage collection, properties and for each
`SR's syntax to the Objective-C programming language. Note that Objective-C 2.0 is only compatible
`with the Mac OS X 10.5 (leopard) preview.
`
`* New editor: New features include improved performance when icacing large files, and streamine of
`the edits buildidebug process, for instance, errors and warnings car; now be viewed inline within you
`code in message "bubbles", allowing you g easily view errors and warnings beside the code you need to
`correct. Additionally, a few feature called Code focus combifies new code folding features along with
`visual cues to highlight the block structure of your code.
`
`3002
`
`Refering for Objective-C. Refactoring facilitates wide-scale restructuring of your groject and cass
`structure.
`
`Project Snapshots; Another new feature, enables you to save your SERE state, or restore
`your project to a previous state, Together with refactoring, these provide a safe mechanism to make
`
`dramatic changes to your entire project,
`* New Interface Builder. This new versign has an ingroved user interface, a new extensible library
`rode? angsupports new Mac OS X 10.5 features such as Layer Kit. For this preview, the previous
`version of 18 is included as "interface Builder Tiger app".
`Debugging Control
`Xiay. Uses a collection of instruments to monitor and visualize your application and the OS over time,
`which are presented as a series of tracks" similar to SS Band, E. also allows you to create your
`own custom c trace instruinents. This enables you to quickly spot problems and relationships within your
`application
`• Rewind. Aiows you to step backward in your code while ESSES With Rewind, there is no need to 3003
`festart your entire debugging session simply because you; stepp too fa.
`3004
`Debugging Tooltip, Any text editor window can now display variable data as a tooltip while debugging
`
`Y3005
`FIG. 22A
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 47 of 81
`
`US 2009/0106674 A1
`
`
`
`3010
`
`About Xcode Tools.pdf
`
`Universal app licatars for atherican PowerPC articles
`Please see the complete Xcode 3.0 preview release notes for more detailed information on the above
`features, and the gany other new features in this preview release.
`Dashcode for Clashboard Widget Development
`* Dashcode. A new appiication that includes everything you need to create Dashboard widgets
`with in a single savior:Tient. Dashcode provides a ruimber of templates, starting you of with a valuable,
`fully-functioning widget. Fro?ia there you can easily chaige the look and controls within you; widget using
`the design canvas. For Faroe advanced developers you can write new code with the programmer's editor
`with Code Sense, and ever debugyour widget with the JavaScript debugger, Dashcode wili take care of
`the packaging and deployment of your widget to Dashboard once you've perfected it.
`The Dashcode installer package is including at the top level of the Xcode Tools folder in the leopard
`preview installation DVD, and will instal into the Developer Applications toider,
`installing Xcode Developer Tools
`The Xcode tools.mpkg contains several separate sub-packages in the Custom instal pare that affow
`flexibility in instating and updating:
`
`* Developer Toois Software (DeveloperToois.pkg) - This package Contains Xcode, gob, and the other
`tools necessary for Mac OS X development.
`GCC40, 1 (goc4, G. pkg) . This is the latest version of goc. This compiler is recessary for 64-bit
`eyepient, using the OWARF debugging format, and development of Universal application.
`GCC 33 kgCC3.3.pkg). This package intains the gcc 3.3 compiler,
`Software levelopment Kits. These packages provide the header files and link libraries necessary for
`Mac OS X development. All development requires the Mac OS X SDK. This package provides the
`AP for basic Mac OS X technologies such as Carbon, Cocoa, and command-line tools. The other
`SDK packages provide API for extended Macintosh technologies, such as QuickTime, OpenGL, etc.
`J2SE50 Developer tools - These packages are used when developing 2SE 5.0 applications for
`Macintosh.
`Developer Documentation (DewDocumentation.pkg) - this package contains the core Max OS X
`developer documentation for technologies such as Carbor, Cocoa, QuickTirae, WebKit, etc.
`Developef Exaraples (evexamples.pkg) - This package contains exampfe code for a wide variety of
`Mac OS X technologies. Note that these examples may require any of the above Software
`Development Kits to be installed before they will compile.
`WebObjects WebObjects. In pkg) - These package are used fo: WebOjects 5.4 developinent,
`
`Cross Development (MacOSX 10.3.9.3 kg. MacQSX10.4 upkg, MacOSX10.5. pkg) - these packages
`are for Mac OS X v10.4 (Panther), Mac OS X v10.4 Tiger), and Mac OS X 10.5 (Leopard to allow
`Cross development. These can be installed together grindependently, and can be installed
`separately.
`CHUi tools CHUD, pkg) - This package conta is the Computer Hardware Understanding Developer
`Tools. For more information see http:li developerapple, coni toolsiperformancel,
`instaliation instructions for Xcode 3.0
`
`3015. 3017 Y3019 302
`FIG. 22B
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 48 of 81
`
`US 2009/0106674 A1
`
`
`
`Ale?s acjas Egels
`fini:3xt::ss said:nstallaroa
`cre -ss-listia: e.
`
`Search it Reference
`
`GraphKit
`Reference Guide
`
`QuickTime Kit Programming Guide
`
`Aschitecture and Perfcfmance Group
`Apple Computer, irc.
`
`stay 2004
`
`FIG. 22C
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 49 of 81
`
`US 2009/0106674 A1
`
`
`
`DISPLAY WINDOW CONTAINING USER INTERFACE (UI)
`OF FILE SYSTEM (E.G. HFS) ORU OF SEARCH
`SOFTWARE (WINDOW MAYBE MOVABLE,
`RESIZABLE CLOSEABLE AND OVERLAPPING WITH
`OTHER WINDOWS)
`
`DISPLAY USER SELECABLE VIEW OPIONS FOR
`DISPLAYING FILES AND FOLDERS (E.G. CON VIEW,
`LIST WIEW, COLUMN VIEW OR COVER FLOW
`VIEW)
`
`23O
`
`23O3
`
`DISPLAY SCROLLINGUI OBJECTS (EG. ARROWS OR -3'
`SCROLLBAR) TO ALLOWSCROLLING OF ONE OR
`MOREVIEWS (CONCURRENTLY OR SEPARATELY)
`
`RECEIVE USER COMMANOS FROM COVER FLOW
`VIEW (OR OTHER VIEWS) TO PERFORM FILE
`SYSTEM OPERATIONS (E.G. CREATE NEW FOLDER
`NHFS) AND PERFORM THESE OPERATIONS
`
`2307
`
`DISPLAY SER SELECTABLE OPTIONS TO SORT
`ES ANDOR FLOW WEWS
`
`2309
`
`FIG. 23
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 50 of 81
`
`US 2009/0106674 A1
`
`$ $ $ ? '
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 51 of 81
`
`US 2009/0106674 A1
`
`
`
`
`
`|
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 52 of 81
`
`US 2009/0106674 A1
`
`
`
`
`
`
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 53 of 81
`
`US 2009/0106674 A1
`
`U []
`
`
`
`
`
`CIVZ "SOI-l „
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 54 of 81
`
`US 2009/0106674 A1
`
`
`
`
`
`Ozez
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 55 of 81
`
`US 2009/0106674 A1
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`~ £ 1832
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 56 of 81
`
`US 2009/0106674 A1
`
`
`
`© @ @ § ¶ ¡ ¿] [55] * @ @ @ @ : £ € © ?
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 57 of 81
`
`US 2009/0106674 A1
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 58 of 81
`
`US 2009/0106674 A1
`
`
`
`
`
`
`
`
`
`
`
`
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 59 of 81
`
`US 2009/0106674 A1
`
`
`
`DSPLAY WINDOW HAVING USER INTERFACE OF
`COVER FLOW WIEW (AND OPTIONALLY ALSO DISPLAY
`LIST VIEW WITHIN SAME WINDOW)
`
`RECEIVE USER'S COMMAND (EG, THROUGH USER'S
`SELECTION OF AUSER INTERFACE OBJECT) TO
`ZOOM OR PAGE THROUGH OR SCROLL THROUGH
`DOCUMENT SHOWN IN COWER FLOW WEW
`
`25O1
`
`2503
`
`RESPOND OUSERS COMMAND O ZOOM OR
`PAGE ROUGH OR SCROL THROUGH A DOCUMENT
`IN THE COWER FLOW WEW
`
`2505
`
`F.G. 26A
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 60 of 81
`
`US 2009/0106674 A1
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`st
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 61 of 81
`
`US 2009/0106674 A1
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`---- | No. © @ § ¶ ¡ ¿ E § @ @ @ @ § € 9. O £
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 62 of 81
`
`US 2009/0106674 A1
`
`$ $ $Z
`
`
`
`
`
`
`
`
`
`
`
`?S]|BMW 0|mp3?OS
`
`un0}\ \L uno),
`
`saunae
`
`
`
`
`
`C19Z "91- ),,,,,
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 63 of 81
`
`US 2009/0106674 A1
`
`
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 64 of 81
`
`US 2009/0106674 A1
`
`
`
`
`
`
`
`W77779 NACI C?H=ï,1)
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 65 of 81
`
`US 2009/0106674 A1
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`| WCHquêuoduloo
`@@ @@@
`
`@@ @@@
`
`
`
`---- No. $3
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 66 of 81
`
`US 2009/0106674 A1
`
`g; ? EZ -----w «» - (?
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 67 of 81
`
`US 2009/0106674 A1
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`£ | 22|----GÐ GEJ QIEL?IÐ
`
`Ozez
`
`

`

`Patent Application Publication
`
`Apr. 23, 2009 Sheet 68 of 81
`
`US 2009/0106674 A1
`
`Receive user's search input or receive
`user's request to present items 4
`
`24O1
`
`- - - - -------------------------------
`
`------
`Y
`8
`. . .
`.
`.
`.
`-
`Present items, showing an initial preview
`for each item
`
`2403
`
`Receive user selection for an enhanced -
`preview of at least one item
`
`2405
`
`-.
`Display enhanced preview of an item in a
`representation of items
`
`24O7
`
`---
`- - - - - -
`Replace enhanced preview with the initial
`preview and display, sequentially, an
`enhanced preview of a next item in a 4
`repr