NCUAUACAATA
`
`US 20040001104A1
`
`as) United States
`a2) Patent Application Publication (10) Pub. No.: US 2004/0001104 Al
`Sommereretal. Jan. 1, 2004 (43) Pub. Date:
`
`
`
`(54) RESOURCE BROWSER SESSIONS SEARCH
`
`(52) U.S. C1. eee ecssseseecsecnecneeeneess 345/811; 345/854
`
`(75)
`
`Inventors: Ralph Sommerer, Cambridge (GB);
`Robert Tucker, Cambridge (GB);
`Natasa Milic-Frayling, Cambridge
`(GB); Jurij Leskovec, Horjul (SD
`
`Correspondence Address:
`LEE & HAYES PLLC
`421 W RIVERSIDE AVENUE SUITE 500
`SPOKANE, WA 99201
`
`(73) Assignee: Microsoft Corporation
`(21) Appl. No.:
`10/186,906
`
`(22)
`
`Filed:
`
`Jun. 28, 2002
`
`Publication Classification
`
`CSL) Tt C0 eecccccceeecccssssssssnnsecceesnnnnnnseecsesnnnees G09G 5/00
`
`(57)
`
`ABSTRACT
`
`A browser sessions search tool allows a user to search
`through records of previously viewed resource pages, even
`across multiple browser sessions. Each visit to a resource
`page results in creation of a visit data structure that refer-
`ences the resource page content. A rich assortmentof data is
`recorded for each visited resource page, suchas full text and
`layout data, time of access and navigation type that lead to
`the page. In one embodiment, color aspects of each resource
`pageare storedto assist in a search, although other recorded
`resource page characteristics may additionally or alterna-
`tively include without limitation other visual, audio, navi-
`gational, or metadata aspects
`
`
`ins = raw cas
`
`
`
`
`
`
`
` “cag BePES ‘wort vee
`
`fat 1e2
`
`
`Batam a
`
`Seifticr iy
`
`steals tae ee
`Health tesiducaon *
`
`
`
`tarataarene cahenonieve
`
`
`1
`
`APPLE 1028
`
`APPLE 1028
`
`1
`
`

`

`Patent Application Publication
`
`Jan. 1, 2004 Sheet 1 of 10
`
`US 2004/0001104 Al
`
`voye.2
`
`sogenasAg
`
`1
`
`a’
`
`We
`
`rrr
`
`BLY
`
`ORIoF
`
`apeese
`
`a87OPP
`oryHines
`
`
`
`SHONQUDIEPSIABSO}!Wpps
`
`SRDUEGEeReeReARSvinFueEE
`
`RpGsBBHBhOEE
`
`é
`
`wae#
`
`50azaesesBBPFabyOyty
`
`
`
`BRAUNMESoy4giesteey
`
`AOLSOS
`
`
`
`‘OldBet
`
`pelpane
`
`
`
`apnengpchiner
`
`
`
`wipes)RdgVeyBee
`
`aEEonea2EeTR
`
`atedgedbbAube
`
`hdPeDagotUSWAPheORES
`
`ipaLederecg)engTes
`
`+aeedAoey
`PALAIEBS|SOERaeFE
`
`yee
`
`PoeEege
`
`
`
`
`
`
`
`
`
`on
`
`\
`
`
`
`
`
`puratons|uatiybeslof.seer)
`
`iP-haaN
`
`2
`
`
`
`
`
`
`
`
`
`
`
`
`
`

`

`Patent Application Publication
`
`Jan. 1, 2004 Sheet 2 of 10
`
`US 2004/0001104 Al
`
`
`
`SaARsatiressibey“weNASLPRIS,iin
`
`
`
`tadweitaseSages
`
`
`
`éOld
`
`ZOz—
`
`
`
`meRapinMU
`ebMDSBa
`
`SOREBESEFSerre,
`
`pesseotRah2ngBaEB
`
`eriemmnd-neeamranstonSarseoaoesar
`se:og8teuy,PRAPPne
`a:|Regmgeeda
`
`
`
`adffBo8ReelGaPMALOMelduae
`
`3
`
`
`
`
`
`
`
`
`
`
`

`

` 1angsan”arfayap
`zfgMISBLT,FLLAN|rs
`Onis1piesemirenaeewanaPSRS;
`
` FERDRT:iShales"SHUSTGETROPe
`
`
`
` wyyanwine7Togshasegsopowasgoer1
` 26h2WNBEEATatFRG
`:
`
`
`
`
`
`Patent Application Publication
`
`Jan. 1, 2004 Sheet 3 of 10
`
`US 2004/0001104 Al
`
`PaSgege|
`
`2REET
`
`seuelah
`
`Stl|
`
`Lage
`
`€Sls
`
`
`
`:prametnen|sassyPEARSON>ESATOL,
`
`aweoo
`srpaceonse
`
`Anne
`
`xvToRtaus|
`
`WS
`
`petasai
`legtTayBg
`
`AVTEESVAL
`
`IETSTINK
`
`aLADei
`
`Fetetyiseong
`
`+sxrtegsiergenenesvarggeIERPec|
`
`PTLMPP?Sor
`
`wateSE|
`
`Ole
`
`00€
`
`COE
`
`80¢
`
`4
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`

`

`Patent Application Publication
`
`Jan. 1, 2004 Sheet 4 of 10
`
`US 2004/0001104 Al
`
`vOlas
`
`
`
`e6eyaounosey
`
`807
`
`
`
`JoBbeuewuonebiaen
`
`90+
`
`OOF
`
`YUILedAH JeBeuey
`
`SUOISSESJESMOlg
`
`M@IABIg
`
`abbeysoinosey
`
`SINPOYWYQeIS
`
`vl
`
`giNpowW
`
`ZLP
`
`
`
`aiNpo-yJoyebiaen
`
`OLY
`
`5
`
`
`

`

`suolejouuy
`
`90S
`
`
`
`(s)adA]}USAQ
`
`abeaoinosey
`
`soent)Yeas
`
`ZLS
`
`soinosey
`SJOINUSP}JazAjeuyJUdAZ
`
`
`80SJeBeueywuoneblaen
`
`OLSvOS
`
`Patent Application Publication
`
`Jan. 1, 2004 Sheet 5 of 10
`
`US 2004/0001104 Al
`
`G‘Sls
`
`c0G
`
`
`
`yUusAquonebien
`
`00¢
`
`6
`
`
`

`

`Patent Application Publication
`
`Jan. 1, 2004 Sheet 6 of 10
`
`US 2004/0001104 Al
`
`9‘Sls
`
`
`
`aBeiojsejyeq
`
`0z9
`
`
`
`aBbeysounosey
`
`yuajuoy
`
`zg
`
`
`
`sebeu|ileuquunyL
`
`v29
`
`
`
`aoInosayJOUIO
`
`sjusweagobeg
`
`929
`
`
`
`Seolpuyejyeq
`
`829
`
`
`
`1x8]JUeWINDOG
`
`xepu|
`
`oe9
`
`
`
`xepuy[FeUqUINY|
`
`ee9
`
`
`
`JUBJUODJ9YIO
`
`seolpu|
`
`eg
`
`eoinosey
`
`iaBeuep
`
`Z09
`
`abeaoinosey
`
`Japeo7
`
`709
`
`
`
`JezAjeuyjnoAe7
`
`909
`
`JOJIEIXAXOL
`
`809
`
`JazAjeuydN
`
`O19
`
`
`
`JOXOPU]1X9
`
`ZL
`
`reuquinu|
`
`J0}2J9U98¢)
`
`rg
`
`
`
`BWYOSJO|OD
`
`JezAjeuy
`
`919
`
`/weuquinu|
`
`
`
`aBbeyaoinosay
`
`
`
`Bulxepu]Jojo9
`
`8L9
`
`7
`
`

`

`Patent Application Publication
`
`Jan. 1, 2004 Sheet 7 of 10
`
`US 2004/0001104 Al
`
`90q
`
`U9]U0D
`
`ayua]u0D!90qLS!\\=—-'\fpoyjueUCD\\|A
`
`\
`
`Z2°9ls
`
`0}(s)joy
`
`(s)juawno0q—9b2
`
`
`
`THNe6eyaseg
`
`FEUGUINY|0}JOY
`
`yusuodwoy
`
`(s)7HN
`
`OL
`
`obL
`
`vlL
`
`adA|Aen
`
`eed0}Joy
`
`TNebegaseg
`
`90/
`
`L0L
`
`
`
`[duisoui}Zoz
`
`vol
`
`USIA~~
`
`002
`
`8
`
`
`
`
`
`
`

`

`Patent Application Publication
`
`Jan. 1, 2004 Sheet 8 of 10
`
`US 2004/0001104 Al
`
`8Sls
`
`—ze
`
`sjusuuNooq
`
`vOs
`
`So01NOsay
`
`PeaiyoiyYMebed
`SdINOSAYpoaalyay
`
`
`ZG|90qaAlyoy
`
`eqiooqaalyoy
`
`Lq}ooqanyoy
`USIA[PAY
`
`panlyouy
`
`
`
`aoinosaypayepdn
`
`Aiqissogymobeg
`
`
`
`soinoseypajyepdn
`
`s]UsWINDOG
`
`JEASINOYBAIT
`
`USIA
`
`
`
`OGI90qaAl4youy
`
`ebed
`
`ysonbey
`
`008
`
`9
`
`
`
`
`
`
`
`
`

`

`Patent Application Publication
`
`Jan. 1,2004 Sheet 9 of 10
`
`US 2004/0001104 Al
`
`
`
`Event Monitor
`
`902
`
`
`Resource Page
`Downloader
`
`904
`
`
`Resource Page
`Analyzer
`906
`
`
`
`Archived Data Storage
`
`
`
`
`
`Browser Module
`900
`
`
`
`
`Resource Page Content
`910
`
`Resource Page Analysis
`Results
`912
`
`Navigation Information
`914
`
`920
`
`922
`
`
`
`~ 908
`
`
`
`
`
`
`916
`
`
`
`
`
`SF 928
`
`
`
`
`Resource Page
`Navigation Module
`
`
`
`930
`
`
`
`
`Text Search Engine
`
`Image Search Engine
`
`
`
`924
`
`Trail Generator
`
`Map Generator
`
`_| or926
`
`Hyperlink Preview
`Module
`932
`
`Browser Sessions
`Search Module
`936
`
`FIG. 9
`
`10
`
`10
`
`

`

`Patent Application Publication
`
`Jan. 1, 2004 Sheet 10 of 10
`
`US 2004/0001104 Al
`
`
`YauYTV9O1|_me)SNOWUSHLO|eT#)eiefzeTi|||||
`YOMLAN|TidasWOOaNoy|YSIGCYVH|jg]WW8OOUd
`|WvudOudasovausin)||ZOVEUSINIZovayaunt|WN“sowBe|)MMOMIAN140dSAM
`
`
`
`2-7|NOLWOMdd¥“yee,|WweDOUd|STENSAStseowaHio|(NOLWONdd¥JONILWYado
` SWYYDOdlneNOILVOMdd¥oe||SmeWaISKS”|[||WALSASLINNONILVH3dO|OIAONISSIOONd||8y|solg!7=|wz9pz©(WOW)||:AMOWSNWSISAS||—-eeeEEEJ
`
`
`|)oq|SAUSaINGOW
`|SWYYOONd:Sandon
`
`
`
`
`Lt|\oo
`(0)dela6¢\7\“__2TTTa-|Fe|==Atwivd
`YaLdvay
`-
`
`
`\ge&3geo
`\on
`
`€ C(-
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`
`11
`
`11
`
`
`
`
`
`
`
`

`

`US 2004/0001104 Al
`
`Jan. 1, 2004
`
`RESOURCE BROWSER SESSIONS SEARCH
`
`RELATED APPLICATIONS
`
`[0001] The application is related to U.S. patent application
`
`Ser. No.
`, entitled “NAVIGATING A RESOURCE
`BROWSER
` SESSION’[Docket
`No. MS#192137.1/
`40062.186US01], and U.S. patent application Ser. No.
`
`, entitled “HYPERLINK PREVIEW UTILITY AND
`METHOD”[Docket No. MS#192135.1/40062.185US01],
`both of which are specifically incorporated herein for all that
`they disclose and teach.
`
`TECHNICAL FIELD
`
`[0002] The invention relates generally to resource brows-
`ers, and more particularly to searching through resources
`previously visited by a user.
`
`BACKGROUND OF THE INVENTION
`
`[0003] Using a browser, a user mayvisit a large number of
`websites in various browser sessions. At each web site, a
`user mayalso visit multiple web pages during an individual
`browser session. In some cases, a description and an address
`(e.g., the Uniform Resource Locator or URL) for a web page
`visited during a browser session may be saved in a sequen-
`tial, stack-based “history” list, possibly allowing a user to
`return to a previously visited web page by selecting its
`description from the history list. A user can open the history
`list to select previously viewed resources, which are usually
`identified by the title or address of the web page. A user can
`also “bookmark”a currently viewed web page in a Favorites
`list to facilitate selection of the bookmarked resource from
`the Favorites list for a later visit.
`
`[0004] Browsers can also be usedto traverse a file system,
`and a history list can be used to return to a previously visited
`directory or file within the file system. Generally, browsers
`may besaid to browse resources, whether on the Web, in a
`file system, or in some other type of data storage.
`
`[0005] Existing session historylists present disadvantages
`that limit their usefulness. By recording only portions of a
`single browser session (e.g., because of the nature of the
`stack-like recording mechanism), much of a user’s recent
`browsing experience is lost both during a given browser
`session. In addition, existing history lists do to not retain
`history information across multiple browser sessions. Even
`if the web page of interest is recorded in a historylist, the
`user may not remember the nameor address of previously
`viewed web page and may, therefore, find it difficult to
`identify the desired web page in the history list.
`
`which limits the usefulness of such prior approaches. For
`example, the user cannot reconstruct the navigations steps
`within a browser session from this information.
`
`[0007] Likewise, bookmarking does not fully address a
`user’s needs because the web page in question may not have
`seemed important when it was previously viewed and,
`therefore, the user may not have designated it for inclusion
`in the Favorites list at that time. Thus, the web page may be
`lost to the user forever, unless the web page addressisstill
`recorded in a history list or the user can remember how to
`get back to the desired web page (e.g., by remembering the
`URL or remembering how to navigate back to the web
`page).
`
`[0008] Unfortunately, users frequently forget previous
`navigation patterns, addresses, pagetitles, or search param-
`eters and, instead, remember key words, colors, sounds, or
`layout characteristics on a web page. During an exemplary
`browser session, a user may not be able to remember the
`complicated URLto that e-commerce web page having that
`attractive blue tie in the middle of the page, but may be able
`to quickly select it from a subset of previously visited web
`pages having the color blue in the middle of the page.
`However, existing solutions fail to provide functionality to
`support this kind of interaction.
`SUMMARYOF THE INVENTION
`
`[0009] Embodiments of the present invention solve the
`discussed problemsby providing a sophisticated search tool
`for searching through a more complete set of recently visited
`resources within a single browser session or across multiple
`browser sessions. The record of previously visited resources
`need not be truncated by the stack-like operation of a
`standard session history list. Instead, all (or a configurable
`portion thereof) recently visited resource pages may be
`searched based on navigational details, content, metadata,
`resource page color scheme, and other visual and non-visual
`aspects of the resource pages.
`
`[0010] The search results may also be visually associated
`with previous navigational sequences to provide quick
`access to a previous navigational experience. For example,
`a user can search for previously visited web pages resulting
`from a web search and the navigational sequences associated
`with the web search results will be displayed to allow the
`user to pick up where he orsheleft off.
`
`[0011] Each visit to a resource page results in creation of
`a visit data structure that references (directly or indirectly)
`the resource page content and associated annotation data.
`Generally, a visit data structure references data associated
`with a previously visited resource page. A rich assortment of
`data may be recorded for each visited resource page, such as
`[0006] Some browsersalso provide global history archive,
`full text and layout data. In one embodiment, color aspects
`of each web pageare stored to assist in a search. Resource
`which include all the URL’s visited by the user merging the
`page content may also be displayed in search results, such as
`browsing information from multiple browser sessions. Such
`a thumbnail image of the resource page. Furthermore, in
`browsers also apply various strategies for saving informa-
`contrast to typical browser session history lists, navigation
`tion relating to previously visited web pages for further
`reuse. In responseto the search over such saved information,
`branchesare not truncated. Therefore, all previously viewed
`resource pages can be recorded in archive data storage(i.e.,
`the user typically obtains only a URL oratitle of the
`resource in the result list. In the Web environment, such
`to the extent of an available or designated storage limit).
`limited results would not typically satisfy a user because the
`Archived resource page content is indexed and annotated to
`user is often unaware of the URL of the site that they
`be searchable by text, color, and other visual, audio, navi-
`previously browsed. In addition, the title of a resource is
`gational, or metadata aspects,
`thereby allowing users to
`frequently uninformative or inaccurate. Furthermore,
`the
`search the rich record of their browsing experience during
`saved information does not include navigation information,
`the browsing session.
`
`12
`
`12
`
`

`

`US 2004/0001104 Al
`
`Jan. 1, 2004
`
`invention,
`the present
`In implementations of
`[0012]
`articles of manufacture are provided as computer program
`products. One embodiment of a computer program product
`provides a computer program storage medium readable by a
`computer system and encoding a computer program that
`searches a plurality of previously visited resource pages
`accessible via a network coupled to the computer system.
`Another embodiment of a computer program product may
`be provided in a computer data signal embodied in a carrier
`wave by a computing system and encoding the computer
`program that searches a plurality of previously visited
`resource pages accessible via a network coupled to the
`computer system.
`
`[0013] The computer program product encodes a com-
`puter program for executing on a computer system a com-
`puter process for searching a plurality of previously visited
`resource pages accessible via a network coupled to the
`computer system. Visit data structures associated with each
`of the resource pages are recorded in archive storage. The
`visit data structures are annotated with information relating
`to at least one searchable characteristic of an associated
`
`resource page. A browser sessions search parameter is
`received specifying a targeted search characteristic. The visit
`data structures in archive storage are searched to identify at
`least one of the plurality of previously visited resource pages
`having the searchable characteristic that satisfies the browser
`sessions search parameter.
`
`In another implementation of the present invention,
`{0014]
`a method of searching a plurality of previously visited
`resource pages accessible via a network coupled to the
`computer system is provided. Visit data structures associated
`with each of the resource pages are recorded in archive
`storage. The visit data structures are annotated with infor-
`mation relating to at least one searchable characteristic of an
`associated resource page. A browsersessions search param-
`eter is received specifying a targeted search characteristic.
`The visit data structures in archive storage are searched to
`identify at least one of the plurality of previously visited
`resource pages having the searchable characteristic that
`satisfies the browser sessions search parameter.
`
`In yet another embodiment of the present inven-
`[0015]
`tion, a browsersessions search tool for searching a plurality
`of previously visited resource pages accessible via a network
`coupled to the computer system is provided. An archival
`storage stores visit data structures recorded in association
`with each of the resource pages. One or more annotations
`associated with one or more of the visit data structures
`
`specify at least one searchable characteristic of an associated
`resource page. A browser sessions search query interface
`receives a browser sessions search parameter specifying a
`targeted search characteristic. A browser sessions search
`result search identifies at
`least one of the plurality of
`previously visited resource pages having the searchable
`characteristic that satisfies the browser sessions search
`parameter.
`
`[0016] These and various other features as well as other
`advantages, which characterize the present invention, will be
`apparent from a readingofthe following detailed description
`and a review of the associated drawings.
`
`BRIEF DESCRIPTION OF THE DRAWINGS
`
`[0017] FIG. 1 depicts an exemplary user interface for a
`browser sessions search tool
`in an embodiment of the
`present invention.
`
`[0018] FIG. 2 also depicts an exemplary user interface for
`a browser sessions search tool in an embodiment of the
`
`present invention.
`
`[0019] FIG. 3 depicts an exemplary user interface for a
`browser sessions search tool
`in an embodiment of the
`present invention.
`
`[0020] FIG. 4 depicts an architectural diagram of a
`browser session navigation tool in an embodiment of the
`present invention.
`
`[0021] FIG. 5 illustrates functionality of a navigation
`manager in an embodimentof the present invention.
`
`FIG.6 illustrates functionality of a resource page
`[0022]
`manager in an embodimentof the present invention.
`
`FIG.7 illustrates a resource data view of a visit in
`[0023]
`an embodiment of the present invention.
`
`[0024] FIG. 8 illustrates two exemplary requests for a
`resource page in an embodimentof the present invention.
`
`[0025] FIG. 9 illustrates processes and services of a
`browser session navigation tool in an embodiment of the
`present invention.
`
`[0026] FIG. 10 illustrates an exemplary system useful for
`implementing an embodimentof the present invention.
`
`DETAILED DESCRIPTION OF THE
`INVENTION
`
`[0027] A browser sessions search tool allows a user to
`search through records of previously viewed resource pages,
`even across multiple browser sessions. Each visit
`to a
`resource page results in creation of a visit data structure that
`references the resource page content. A rich assortment of
`data is recorded for each visited resource page, such as full
`text and layout data. In one embodiment, color aspects of
`each resource page are stored to assist in a search, although
`other recorded resource page characteristics may addition-
`ally or alternatively include without limitation other visual,
`audio, navigational, or metadata aspects
`
`[0028] Resource page content may also be displayed in
`search results, such as in a thumbnail image of the resource
`page. Furthermore, in contrast to typical browser history
`lists, navigation branches are not truncated. Therefore, all
`previously viewed resource pages are recorded in archive
`data storage (i.e., to the extent of an available or designated
`storage limit). Archived resource page content are indexed
`and annotated to be searchable by text, color, and other
`visual, audio, navigational, or metadata aspects,
`thereby
`allowing users to search the rich record of their browsing
`experience during the browsing session.
`
`[0029] FIG. 1 depicts an exemplary user interface for a
`browser sessions search tool
`in an embodiment of the
`present invention. A search period region 100 configures the
`time period over which previously visited web pages should
`be searched. A text box 102 provides a simple way of
`specifying common time periods, such as “today”, “last
`week”, “this month’, etc. If more detailed and/or more
`
`13
`
`13
`
`

`

`US 2004/0001104 Al
`
`Jan. 1, 2004
`
`complicated time periods are desired, the user can press a
`button 101 next to the text box 102, which opens a dialog
`box through which arbitrary dates/times may be input for
`either end of the desired time period. In alternative embodi-
`ments, the time period can be configured for distinct periods
`prior to the present date, for multiple periods, or with a finer
`granularity (e.g., down to the minute or hour).
`
`[0030] An Options region 104 (i.c., a browser sessions
`search query interface) configures the search parameters. A
`check box 106 allows a user to specify a text search
`parameter for the search to locate in previously visited web
`pages. The text search parameteris specified in text box 108
`and mayinclude a natural language query or a Boolean type
`query with knownlogical operators, including AND, OR,
`quotes, wild cards, etc. A drop down box 110 allowsa user
`to limit the portion of each web page to be searched. In the
`illustrated search,
`the “web page content” is specified.
`Therefore, a user can search the complete text body of the
`webpage(e.g., the HTMLof the document). However,other
`portions of the web page may also be searched, including
`link anchors, headings, captions, query terms associated
`with a search result page, etc., and whatever logical units
`(e.g.,
`introductions, abstracts,
`reference lists, etc.)
`the
`resources pages may contain.
`
`[0031] Acheck box 112 allowsa user to specify a color for
`the searchto locate in previously visited web pages. See also
`the discussion of FIG. 3. A user may not rememberspecific
`text from the sought-after web page, but may have a vague
`recollection of a color displayed on the web page or some
`other visual aspect of the web page. For example, a user may
`remember the background color of the web page or a
`particular color in an image or font. A search for a visual
`aspect of a sought-after web page allows a user to search
`based on an alternative search parameter, which may be
`applied in combination with or in place of the text search
`parameter. Other visual aspects may include without limi-
`tations frames, border colors, buttons and other controls and
`their colors, sizes, styles, etc.
`
`In another embodiment, other visual aspects may
`[0032]
`additionally or alternatively be specified as search param-
`eters. For example, a user may specify one or more locations
`or regions on the web page at which he or she remembers
`seeing the text, color, or other visual aspect. The location or
`regions may be specified by coordinates, coordinate ranges,
`frameidentifiers, or other spatial parameters. In one embodi-
`ment, regions are specified by quadrants of the web page
`(e.g., Northeast, Northwest, Southeast, Southwest), although
`other regional specifications are contemplated within the
`scope of the present invention. In yet another embodiment,
`a user may additionally or alternatively specify an audio
`aspect as a search parameter. For example, a user may search
`for the previously visited web page having the combination
`of a blue animation and soundeffects.
`
`In yet another embodiment, non-visible and non-
`[0033]
`audio aspects of previously visited resource pages can be
`searched. Exemplary aspects include without limitation web
`search parameters associated with one or more web pages,
`the time of access, the resource page navigated from, the
`resource page navigated to, image labels, metadata, adver-
`tisements included within a resource page, forms, hyper-
`links, web pagetitles, resource pages marked as favorites,
`
`and web pages pointed to by links previewed using the
`Hyperlink Preview Utility but not actually viewed in the
`main browser window.
`
`[0034] The search button 114initiates the search based on
`the currently set parameters. The search results region 115
`(i.e., a browser sessions search results interface) displays the
`results 122 of the last search. In one embodiment, each
`search result includes a thumbnail image of the previously
`viewed web page, not a newly retrieved version of the web
`page. The search results region 115 also displaysthetitle and
`address of the web pages resulting from the search. If more
`results are found than fit in the search results region 115,
`PREVIOUSbutton 118 and NEXTbutton 120 allow a user
`
`to traverse through multiple pages of search results. The
`currently selected result 124 (“the current result”), as indi-
`cated by the dark (red) border around the thumbnail image,
`is also displayed at a larger size in the “Contents” window
`126. Thevisits region 128 displays the numberof times the
`web page has been visited during the search period, along
`with date and time of each visit and a thumbnail image of
`each previously visited web page.
`
`[0035] FIG. 2 also depicts an exemplary user interface for
`a browser sessions search tool in an embodiment of the
`present invention. The “Navigation” window 200 illustrates
`the navigation sequences associated with the web pages
`shown in the visits region 202. As illustrated, the currently
`selected search result 206 is shown in the navigation
`sequence 204. This feature allows a userto retrieve a desired
`previously executed navigation sequence from archive
`based on a target search characteristic on only one of the
`resource pages in the sequence.
`
`[0036] FIG. 3 depicts an exemplary user interface for a
`browser sessions search tool
`in an embodiment of the
`
`present invention. As discussed previously, a checkbox box
`300 allows the user to select a search based, atleast in part,
`on a color, which may be selected from the colors 302. The
`color 304 is shownas selected for the current search. Search
`results 306 include previously visited web pages containing
`the color 304.
`
`In one embodiment, a browser sessions search tool
`[0037]
`uses the color index of the previously visited web pages to
`find those that match the specified color 304. The web pages
`that contain the specified color are displayed in the search
`results region 306 in order, from the web page 308, the best
`match (e.g. the page with the greatest number of matching
`pixels) for the color search at the top of the search results
`region 306 to the web page 310,the lowest scoring page(e.g.
`the page with the lowest number of matching pixels) at the
`bottom of the search results region 306. The results of the
`color analysis can also be saved as an annotation ofthe visit
`data structure.
`In another embodiment,
`the color query
`specification and matching can be limited to specific loca-
`tions or regions of each web page, such as the side bars,
`header, footer, etc. In one embodiment, the color analysis for
`each web page may be computed based on the color char-
`acteristics of the thumbnail image instead of the archived
`web page content. Alternatively, the color specification of
`the page may involve parsing the HTMLcode for color
`specifications of the page elements (e.g., backgrounds,fonts,
`tables, lines, etc.) and embedded images.
`
`In yet another embodiment, a user may search
`[0038]
`previously visited web pages based on previous search
`
`14
`
`14
`
`

`

`US 2004/0001104 Al
`
`Jan. 1, 2004
`
`communicated from the browser 400 to underlying services
`404 to effect browser sessions search functionality.
`
`queries, such as those search queries entered into a web
`search engine. For example, each archived web page asso-
`ciated with a web search maybe annotated with the search
`[0043] One such service is a navigation manager 406,
`parameters used in the search. Therefore, if a user performs
`which creates and stores the rich record of a user’s naviga-
`two websearches(e.g., one based on “Monty Python” and
`tion during a browser session.
`In one embodiment,
`the
`the other based on “Monty Hall”) and navigates through
`navigation record includes information regarding navigation
`someofthe results of each search, the user can at a later time
`events and objects, including the type and time stamp of a
`perform a browser sessions search based on “Python” to
`navigation event,
`local and remote references to objects
`display those web pages navigated from the results of the
`(e.g., URLs and local paths to of resource pages), and any
`“Monty Python” web search. Moreover, the “depth of navi-
`metadata associated with the navigation (e.g., search que-
`gation” associated with such annotation may be configured
`ries, user annotations, device or environmentspecific param-
`by the user, such that a single web page or even a webtrail
`eters, etc.). The navigation record is stored, in part, for use
`(i.e., no longer than a specified depth) resulting from a given
`in displaying various views (suchasatrail or a map) of the
`websearch can be returned by the browsersessions search.
`navigation during a browsersession.
`
`[0039] A.user mayalso select a topic or specify a label in
`association with a displayed resource page. A topic identifier
`or label is then stored as an annotation with association with
`the data structure representing the visit (i.e., the visit data
`structure). In this manner, such annotations can also be
`searched by the browserssessionssearch tool. For example,
`after performing a web search, a user can then rank the
`various search results as he or she reviews the web pages
`identified by the search results (e.g., from 1 to 10, with 10
`being the most interesting). Thereafter, the user can search
`the user-specified labels (i.e., the rankings) of the previously
`viewed resource pages to consolidate the most relevant web
`pages.
`
`[0040] For that purpose, a topic or label can be associated
`with a particular page (or, alternatively, a certain visit to a
`particular page), whichis then stored as an attribute in a page
`data structure or a visit data structure. Several pages,visits,
`and/or sequences of visits can share the same label, and
`therefore constitute a “set” of pages or visits. For example,
`to “save” a page or set of pages, a user can label all desired
`pages using the same token. The label can then be used as
`a selector on which a user can search in the browser session
`search tool. That is, the search tool can search for the label
`in the annotations of the archived resource pages and visit
`data structures. The result list for this search is displayed to
`the user, who can browse throughthe list of resource pages.
`
`[0044] Another such service is a resource page manager
`408, which performs loading and analyses of the resource
`pages accessed through the browser 400. The types of
`analysis can include without limitation page layout analysis,
`text content processing, thumbnail imagecreation, and color
`scheme analysis. The resource page manager 408 also
`manages storage of archive data so that other modules 416
`can access, further analyze, and present the analysis to the
`user in various forms. For example, a resource page navi-
`gator module 410 accesses the archive data to present the
`browser session navigation bar and/or map viewstothe user.
`The resource browser session navigation functionality is
`described in further detail in U.S. patent application Ser. No.
`
`,
`entitled
`“NAVIGATING A RESOURCE
`BROWSER
`SESSION’[Docket
`No. MS#192137.1/
`40062.186US01].
`
`[0045] Another module may perform a thumbnail color
`analysis in a browser sessions search module 414 to allow a
`user to search a page with a specific color characteristic
`and/or another visual aspect. As the resource browser gen-
`erates visit data structures for each visit to a resource page,
`each visit data structure may be annotated with the results of
`text analysis, color schemeanalysis, layout analysis, natural
`language analysis, navigation analysis, metadata analysis,
`page elements analysis, and other types of resource page
`analysis. For example, as a result of one or more of these
`analyses, a user may use the browser sessions search module
`414 to search for all previously visited resource pages
`navigated from a designated resource page containing a
`designated shadeof the color red. See the discussion of FIG.
`3 for more details of this kind of search.
`
`It should be understood that the underlying data
`[0041]
`(e.g., annotations, content, etc.) collected or accessible by a
`browser sessions search utility also supports searches for
`groups of visits. For example, a user may search for all Web
`trails or browser session that contain resource pages having
`blue pixels. The individualvisit data structures of a Webtrail
`[0046]
`‘Yet another module may include a hyperlink pre-
`may be related and reconstructed throughatrail identifier.
`view module 412 to allow a user to preview a web page
`Likewise, the individual visit data structures of a browser
`associated with a hyperlink in a miniature preview window.
`session may berelated and reconstructed through a session
`Such functionality is described in further detail in US.
`
`identifier. As long as one resource page (or a configurable
`patent application Ser. No.
`, entitled “HYPERLINK
`numberof resource pages) within the trail or session satisfies
`PREVIEW UTILITY AND METHOD”[Docket No.
`the search criteria, the trail or session may be returned as a
`MS#192135.1/40062.185US01].
`search result. As an alternative embodiment,trail or session
`level annotations may be recorded in the archive and shared
`among multiple visit data structures (e.g., by virtue of a
`reference from each visit data structure).
`
`[0042] FIG. 4 depicts an architectural diagram of a
`browser sessions search tool
`in an embodiment of the
`present invention. A browser 400 (e.g., a web browser)
`displays resource pages 402 (e.g., web pages,files, directo-
`ries, computers, etc.) in a main browser window. Browser
`and user interface events from active browser windowsare
`
`[0047] FIG. 5 illustrates functionality of a navigation
`manager in an embodiment of the present
`invention. A
`navigation event 500 is detected by the navigation manager
`502 (e.g., receiving a navigation event in an event handler).
`In one embodiment, the navigation event 500 may include
`witho