5116/2014
`
`INT(It'NI!l ~ fi· ( H J \'1
`
`http://www .searchenginew atch.com'w ebrrasters/features.htm
`
`Search Engine Features For VVebmasters
`
`W9YDge~moohmo
`
`247 captures
`6 Jul 98 - 19 Sep 13
`
`.1 ... ..1
`I I I I II I
`I I
`Search Engine Features For Webmasters
`
`J-*11' 111 11•1 111111111111111•11-11
`
`JUL APR OCT
`
`... 17 ...
`
`I
`
`I
`
`1998
`
`1999
`
`2000
`
`The search engine features cha1t below is designed primarily for webmasters
`who care about how search engines index their sites. It provides a summary of
`important fuctors and features that can a:ffuct how sites are indexed and ranked.
`Full explanations of items can be found in1111ediately below the comparison
`chart. You may aJso find .information on the SearchEngine Features For
`Searchers page to be of interest. Also see the Search Engine Display Chart for
`a summary ofhow search engines display their results.
`
`More detailed information about the search engines listed on the chart is
`available to site subscribers. Visit the subscription page to learn more about
`how you can support the site and also gain access to this expanded infonnation.
`
`Note: This chart covers AltaVis ta, Excite, lnk:tomi, Infoseek, Lycos and Northern Light.
`Excite also covers the Excite-powered services of AOL NetFind, Nets cape Search and
`WebCrawler, unless these services are nan1ed individually . The same is true for In1:tomi,
`which includes HotBo t and MSN Search . Some data for Goog le is also listed . This chart is
`as of Apri15, 1999.
`
`Crawling
`
`Yes
`
`No
`
`Notes
`
`Deep Crawl
`
`AltaVISta,
`lnk:tomi, NLight
`
`Excite, lnfoseek, Web Crawler only
`Lycos , WebCrawler
`lists home pages
`
`In stant Indexing
`
`Frames Support
`
`Image Maps
`
`Password
`Protected Sites
`
`robots .txt
`
`Meta Robots Tag
`
`AltaVista,
`Info seek,
`MSN Search
`
`AltaVista,
`NLight
`
`Excite, lnk:tomi,
`Lycos , NLight,
`Go ogle
`
`Pages will appear
`within a day or two
`after submission
`
`Excite, lnktomi,
`Infoseek, Lycos
`
`Lycos pro vides
`lin'lited support
`
`AltaVista,
`Jnfoseek, NLight
`
`Excite,
`Ink.'tomi, Lycos
`
`AltaVista, lnktomi
`
`Excite, Info seek,
`Lycos , NLight
`
`All
`
`All
`
`n/a
`
`n/a
`
`Link Popularity
`Helps Deep Crawl
`
`Excite, Ink:tomi, AltaVista, lnfoseek,
`Lycos
`NLight
`
`Learns Frequency AltaVista, Infoseek
`
`Excite, Ink:tomi,
`Lycos , NLight
`--See Ch!l!<killi: Yll!.![ .l.!BL & S!l!l[!<b E!.:atU[!<S Qjart -
`
`URL Status Check
`
`Indexing
`
`Full Body Text
`
`Yes
`
`All
`
`No
`
`n!a
`
`Notes
`
`Son1e stop words
`may not be inde>~::d
`
`Stop Words
`
`AltaVista, Excite,
`Ink:tomi, Lycos ,
`Go ogle
`
`lnfoseek, NLight
`
`Meta Description
`
`All but ...
`
`Lycos , NLight
`
`EXHIBIT 2072
`Face book, Inc. eta/.
`v.
`Software Rights Archive, LLC
`CASE IPR2013-00480
`
`https :/lweb.archi I.e. org lweb/19990417060242Jhttp:N.ww. searcheng i newatch. comlwebmas ters/features .html
`
`1/6
`
`

`

`5/16/2014
`
`Meta Keywords
`
`All but...
`
`Search Engine Features For Webmasters
`Excite, Lycos,
`
`NLight
`Excite, Inktomi,
`NLight
`
`
`
`AltaVista,
`Infoseek, Lycos
`
`Inktomi
`
`Others
`
`ALT text
`
`Comments
`
`Stemming
`Ranking
`Meta Tags
`Boost Ranking
`Reviewed Status
`Boosts Ranking
`Link Popularity
`Boosts Ranking
`Direct Hit
`Boost Ranking
`Spam
`
`-- See Search Features Chart --
`No
`AltaVista, Excite,
`Lycos, NLight
`AltaVista, Inktomi,
`Lycos, NLight
`Inktomi, Lycos,
`NLight
`
`Yes
`
`Infoseek, Inktomi
`
`Excite, Infoseek
`
`AltaVista, Excite,
`Google, Infoseek
`
`HotBot
`
`Yes
`
`Others
`
`No
`Excite, Inktomi,
`Lycos, NLight
`
`Very important
`with Infoseek
`Most important
`at Google
`
`
`
`Notes
`
`
`
`
`
`Notes
`
`
`
`
`
`
`
`Meta Refresh
`
`AltaVista, Infoseek
`
`Invisible Text
`
`Others
`
`Excite
`
`Tiny Text
`
`AltaVista, Inktomi,
`Lycos
`
`Excite, Infoseek,
`NLight
`
`Crawling
`
`This section covers factors related to how well search engines crawl web sites.
`
`Deep Crawl
`
`The search engines shown doing deep crawls will list many pages from a web
`site, even if the pages are not explicitly submitted to them. The others will
`usually list far fewer pages from a site. In general, the larger a search engine's
`index is, the more likely it will many pages per site. See the Search Engine Sizes
`page for the latest index sizes at the major search engines.
`
`Instant Indexing
`
`At an instant indexing search engine, usually any page you submit will appear
`within a day or two after submission.
`
`Frames Support
`
`This shows which search engines can follow frame links. Those that can't will
`probably miss listing much of your site. Be sure to read the Search Engines And
`Frames page for tips on overcoming the problems with frames and search
`engines.
`
`Image Maps
`
`This shows which search engines can follow client-side image maps. As with
`frames, those search engines that can't follow image maps will probably missing
`listing much of your site.
`
`Password Protected Sites
`https://web.archive.org/web/19990417060242/http:/www.searchenginewatch.com/webmasters/features.html
`
`2/6
`
`

`

`5/16/2014
`
`Search Engine Features For Webmasters
`
`Some search engines can enter a password protected site, if you arrange for
`them to have a user name and password. Why do this? You may want people
`to discover you have content that matches their query. They'll still need to fill
`out the appropriate registration information at your site to access it, but at least
`they'll know it exists.
`
`robots.txt
`
`The robots.txt file is a means for webmasters to keep search engines out of
`their sites. Site subscribers have access to a page that explains this in more
`detail. More information about robots.txt can also be found on the Robots
`Exclusion Standard page, located at:
`
`http://info.webcrawler.com/mak/projects/robots/exclusion.html
`
`Meta Robots Tag
`
`This is a special meta tag that allows site owners to specify that a page shouldn't
`be indexed. It is ideal for those who cannot create a robots.txt file.
`
`To keep spiders out, simply add this text between your header tags on each
`page you don't want indexed:
`
`<META NAME="ROBOTS" CONTENT="NOINDEX">
`
`You do not need to use variations of this tag to help your page get indexed.
`They are unnecessary. Nor do you need to use this tag if you already use a
`robots.txt file.
`
`Site subscribers have access to a page that explains the meta robots tag in more
`depth.
`
`Link Popularity Helps Deep Crawl
`
`All search engines can determine the popularity of a page by analyzing how
`many links there are to it from other pages. Some engines use this as a means to
`determine which pages they will include in the index. This is NOT the same as
`ranking a page better for having good link popularity. That is explained further
`below. Also see the Measuring Link Popularity page for tips on measuring your
`site's popularity according to different engines.
`
`Learns Frequency
`
`A number of search engines can learn how often pages change. Pages that
`change often may be visited more frequently.
`
`URL Status Check
`
`There are various ways to find whether an exact page is listed at the different
`search engines. The Checking Your URL page explains this in great detail,
`while the Search Assistance Features page also provides some coverage.
`
`Indexing
`
`This section explains what gets indexed when search engines spider a page.
`
`https://web.archive.org/web/19990417060242/http:/www.searchenginewatch.com/webmasters/features.html
`
`3/6
`
`

`

`5/16/2014
`
`Search Engine Features For Webmasters
`
`Full Body Text
`
`All of the major search engines say they index the full visible body text of a
`page, though some will not index stop words or exclude copy deemed to be
`spam (explained further below).
`
`Stop Words
`
`Some search engines either leave out words when they index a page or may not
`search for these words during a query. These "stop words" are excluded as a
`way to save storage space or to speed searches.
`
`Meta Description & Meta Keywords
`
`Shows which search engines support the meta description and meta keywords
`tags, as explained on the How Meta Tags Work page. This does NOT mean
`that using these tags gives pages a ranking boost. That is covered in a separate
`section, below.
`
`ALT Text / Comments
`
`This shows which search engines index ALT text associated with images or text
`in comment tags.
`Stemming
`
`Some search engines will search for variations of a word based on its stem. For
`example, entering "swim" might also find "swims" and maybe "swimming,"
`depending on the search engine. This is explained in more detail on the while the
`Search Assistance Features page.
`
`Ranking
`
`Most search engines use the location and frequency of keywords on a web
`page as the basis of ranking it in response to a query. The exact mechanism is
`slightly different for each engine.
`
`In addition to location and frequency, some engines may give a page a
`relevancy boost based on other factors. These usually can help a little, but they
`don't guarantee a boost to the top. Some major factors are listed below.
`
`Also see the How Search Engines Rank Web Pages article for a more in-depth
`look at how relevancy is determined, and the Search Engine Design Tips page
`for helpful advice about optimizing your pages for search engines.
`
`Meta Tags Boost Ranking
`
`Some search engines that support the meta description and keywords tag will
`also give pages an extra boost if search terms appear in these areas. Not all
`search engines that support the tags also give a ranking boost.
`
`Reviewed Status Boosts Rankings
`
`Some search engines also review sites or list them in an associated directory.
`They may also give a boost to sites that have been listed in this way.
`
`https://web.archive.org/web/19990417060242/http:/www.searchenginewatch.com/webmasters/features.html
`
`4/6
`
`

`

`5/16/2014
`
`Search Engine Features For Webmasters
`Link Popularity Boosts Rankings
`
`As described above, all search engines can determine the popularity of a page
`by analyzing how many links there are to it from other pages. Some engines
`give pages with lots of links, or links from important web sites, a relevancy
`boost.
`
`Direct Hit Boosts Rankings
`
`Direct Hit is a system that measures what users click on from search results in
`order to refine relevancy rankings. This shows which search engines use this as
`a factor. For more information, see the article below:
`
`HotBot Integrates Popularity Into Top Results
`The Search Engine Report, March 3, 1999
`Spam
`
`All major search engines penalize sites that attempt to "spam" the engines in
`order to improve their position. One common technique is "stacking" or
`"stuffing" words on a page. This is where a word is repeated many times in a
`row. If the search engines spot a spamming technique, they may downgrade a
`page's ranking or exclude it from listings altogether. The items below cover
`design elements that could cause a spam penalty. More in depth information
`specific to each search engine is available to site subscribers.
`
`Meta Refresh
`
`Some site owners create target pages that automatically take visitors to different
`pages within a web site. See the What Is A Bridge Page article for more
`information about this.
`
`The meta refresh tag is one typical way of doing this. Some search engines will
`refuse to index a page with a high meta refresh rate. Infoseek will not index
`pages with any redirection, whatsoever.
`
`Invisible Text
`
`This is the technique of placing text on a page in the same color as the
`background, making it invisible to human viewers. Many search engines either
`refuse to index this text or will not index any page containing invisible text.
`
`Tiny Text
`
`This is the technique of placing text on a page in a small font size. Pages that are
`predominantly heavy in tiny text may be dismissed as spam. Or, the tiny text
`may not be indexed. As a general guideline, try to avoid pages where the font
`size is predominantly smaller than normal.
`
` Go
` Choose Another Page
`or use the site map if you can't run JavaScript.
`You may also search the site.
`
`Click here to tell a friend about this site!
`https://web.archive.org/web/19990417060242/http:/www.searchenginewatch.com/webmasters/features.html
`
`5/6
`
`

`

`5/16/2014
`
`Search Engine Features For Webmasters
`
`By Danny Sullivan
`Search Engine Watch
`http://searchenginewatch.com/
`Copyright © 1996-99 Internet.com LLC
`http:/ /www. internet.com
`
`https://web.archive.org/web/19990417060242/http:/www.searchenginewatch.com/webmasters/features.html
`
`6/6
`
`

`

`5/16/2014
`
`Search Engine Features For Webmasters
`
`http://w w w .searchenginew atch.com/w ebmasters/features.html
`
`Go
`
`247 captures
`6 Jul 98 - 19 Sep 13
`
`OCT MAY JUN
`
`11
`
`1999
`
`2000
`
`2001
`
`Search Engine Features For Webmasters
`
`The search engine features chart below is designed primarily for webmasters
`who care about how crawler-based search engines index their sites. It
`provides a summary of important factors and features that can affect how sites
`are indexed and ranked. Full explanations of items can be found immediately
`below the comparison chart.
`
`Human-powered search engines, like Yahoo, are not listed on this chart
`because they do not crawl the web to create their listings. See How Search
`Engines Work for an explanation of the differences between crawler-based and
`human-powered services.
`
`Also see the Search Engine Display Chart for a summary of how search engines
`display their results. The Major Search Engines page has links to the search
`engines on this chart. You may also find information on the Search Engine
`Features For Searchers page to be of interest.
`
`More detailed information about the search engines below is available to
`Search Engine Watch "site subscribers." This information is just one of the many
`benefits that site subscribers receive. Click here to learn more about becoming
`a site subscriber.
`
`This chart is as of Feb. 2, 2000. It covers the crawler-based portions of AltaVista, Excite,
`FAST Search, Go (Infoseek), Google, Lycos and Northern Light. It also covers Inktomi
`results that form portions of AOL Search, HotBot and MSN Search. Excite covers
`portions of Excite-owned Magellan and WebCrawler.
`
`
`
`Crawling
`
`Deep Crawl
`
`Instant Indexing
`
`Yes
`AltaVista, FAST,
`Google, Inktomi,
`NLight
`
`AltaVista
`(pages appear
`within days)
`
`AltaVista, FAST,
`
`Frames Support
`
`Image Maps
`
`Google,
`NLight
`
`AltaVista,
`Go, NLight
`
`No
`
`Notes
`
`Excite, Go, Lycos
`
`Excite, FAST,
`Google, Go,
`Inktomi, Lycos,
`NLight,
`
`Excite, Inktomi,
`Go, Lycos
`
`Excite, FAST,
`Google,
`Inktomi, Lycos
`
`
`
`
`
`
`
`
`
`robots.txt
`
`All
`
`
`Google may not
`n/a
`All
`Meta Robots Tag
`support -- checking
`https://web.archive.org/web/20000511205630/http:/www.searchenginewatch.com/webmasters/features.html
`
`n/a
`
`1/6
`
`

`

`5/16/2014
`
`Search Engine Features For Webmasters
`support -- checking
`
`Inktomi, Lycos
`
`AltaVista, Go
`
`
`
`
`
`AltaVista, Excite,
`FAST, Go,
`NLight
`Excite, FAST,
`Google, Inktomi,
`Lycos, NLight
`-- See Checking Your URL & Search Features Chart --
`Yes
`No
`Notes
`Some stop words
`may not be indexed
`
`All
`
`n/a
`
`Link Popularity
`Helps Deep Crawl
`
`Learns Frequency
`
`URL Status Check
`Indexing
`
`Full Body Text
`
`Stop Words
`
`AltaVista, Excite,
`Inktomi, Lycos,
`Google
`
`FAST, Go, NLight
`
`FAST, Google,
`Lycos, NLight
`Excite, FAST,
`Google, Lycos,
`NLight
`Excite, FAST,
`Google, Inktomi,
`NLight
`
`Others
`
`
`
`
`
`
`
`
`
`
`
`-- See Search Features Chart --
`No
`AltaVista, Excite,
`FAST, Google,
`Lycos, NLight
`AltaVista, Excite,
`FAST, Google,
`Inktomi, Lycos,
`NLight
`
`Notes
`
`
`
`
`
`Meta Description
`
`All but...
`
`Meta Keywords
`
`All but...
`
`ALT text
`
`Comments
`
`Stemming
`Ranking
`
`Meta Tags
`Boost Ranking
`
`AltaVista,
`Go, Lycos
`
`Inktomi
`
`Yes
`
`Go, Inktomi
`
`Reviewed Status
`Boosts Ranking
`
`Go
`
`Link Popularity
`Boosts Ranking
`
`Direct Hit
`Boost Ranking
`Spam
`
`Meta Refresh
`
`AltaVista, Excite,
`FAST, Google, Go,
`Inktomi, NLight
`
`HotBot, Lycos
`
`Yes
`
`AltaVista,
`Go, Lycos
`
`Invisible Text
`
`Others
`
`Tiny Text
`
`AltaVista, Inktomi,
`Lycos
`
`Lycos
`
`Others
`
`No
`
`Very important
`at Google
`
`
`
`Notes
`
`Excite, FAST,
`Google, Inktomi,
`NLight
`Excite, FAST,
`Google
`Excite, FAST,
`Google,
`Go, NLight
`
`
`
`
`
`
`
`Crawling
`
`This section covers factors related to how well search engines crawl web sites.
`
`Deep Crawl
`
`The search engines shown doing deep crawls will list many pages from a web
`site, even if the pages are not explicitly submitted to them. The others will
`https://web.archive.org/web/20000511205630/http:/www.searchenginewatch.com/webmasters/features.html
`
`2/6
`
`

`

`5/16/2014
`
`Search Engine Features For Webmasters
`site, even if the pages are not explicitly submitted to them. The others will
`usually list far fewer pages from a site. In general, the larger a search engine's
`index is, the more likely it will list many pages per site. See the Search Engine
`Sizes page for the latest index sizes at the major search engines.
`
`Instant Indexing
`
`At an instant indexing search engine, usually any page you submit will appear
`within a day or two after submission.
`
`Frames Support
`
`This shows which search engines can follow frame links. Those that can't will
`probably miss listing much of your site. Be sure to read the Search Engines And
`Frames page for tips on overcoming the problems with frames and search
`engines.
`
`Image Maps
`
`This shows which search engines can follow client-side image maps. As with
`frames, those search engines that can't follow image maps will probably missing
`listing much of your site.
`
`robots.txt
`
`The robots.txt file is a means for webmasters to keep search engines out of
`their sites. Site subscribers have access to a page that explains this in more
`detail. More information about robots.txt can also be found on the Robots
`Exclusion Standard page, located at:
`
`http://info.webcrawler.com/mak/projects/robots/exclusion.html
`
`Meta Robots Tag
`
`This is a special meta tag that allows site owners to specify that a page shouldn't
`be indexed. It is ideal for those who cannot create a robots.txt file.
`
`To keep spiders out, simply add this text between your header tags on each
`page you don't want indexed (click on the picture if you want to copy and past
`the HTML for your own use):
`
`You do not need to use variations of this tag to help your page get indexed.
`They are unnecessary. Nor do you need to use this tag if you already use a
`robots.txt file.
`
`Site subscribers have access to a page that explains the meta robots tag in more
`depth.
`
`Link Popularity Helps Deep Crawl
`
`All search engines can determine the popularity of a page by analyzing how
`many links there are to it from other pages. Some engines use this as a means to
`determine which pages they will include in the index. This is NOT the same as
`https://web.archive.org/web/20000511205630/http:/www.searchenginewatch.com/webmasters/features.html
`
`3/6
`
`

`

`5/16/2014
`
`Search Engine Features For Webmasters
`ranking a page better for having good link popularity. That is explained further
`below. Also see the Measuring Link Popularity page for tips on measuring your
`site's popularity according to different engines.
`
`Learns Frequency
`
`A number of search engines can learn how often pages change. Pages that
`change often may be visited more frequently.
`
`URL Status Check
`
`There are various ways to find whether an exact page is listed at the different
`search engines. The Checking Your URL page explains this in great detail,
`while the Search Assistance Features page also provides some coverage.
`
`Indexing
`
`This section explains what gets indexed when search engines spider a page.
`
`Full Body Text
`
`All of the major search engines say they index the full visible body text of a
`page, though some will not index stop words or exclude copy deemed to be
`
`spam (explained further below).
`
`Stop Words
`
`Some search engines either leave out words when they index a page or may not
`search for these words during a query. These "stop words" are excluded as a
`way to save storage space or to speed searches.
`
`Meta Description & Meta Keywords
`
`Shows which search engines support the meta description and meta keywords
`tags, as explained on the How Meta Tags Work page. This does NOT mean
`that using these tags gives pages a ranking boost. That is covered in a separate
`section, below.
`
`ALT Text / Comments
`
`This shows which search engines index ALT text associated with images or text
`in comment tags.
`
`Stemming
`
`Some search engines will search for variations of a word based on its stem. For
`example, entering "swim" might also find "swims" and maybe "swimming,"
`depending on the search engine. This is explained in more detail on the Search
`Assistance Features page.
`
`Ranking
`
`Most search engines use the location and frequency of keywords on a web
`page as the basis of ranking it in response to a query. The exact mechanism is
`slightly different for each engine.
`https://web.archive.org/web/20000511205630/http:/www.searchenginewatch.com/webmasters/features.html
`
`4/6
`
`

`

`5/16/2014
`
`slightly different for each engine.
`
`Search Engine Features For Webmasters
`
`In addition to location and frequency, some engines may give a page a
`relevancy boost based on other factors. These usually can help a little, but they
`don't guarantee a boost to the top. Some major factors are listed below.
`
`Also see the How Search Engines Rank Web Pages article for a more in-depth
`look at how relevancy is determined, and the Search Engine Design Tips page
`for helpful advice about optimizing your pages for search engines.
`
`Meta Tags Boost Ranking
`
`Some search engines that support the meta description and keywords tag will
`also give pages an extra boost if search terms appear in these areas. Not all
`search engines that support the tags also give a ranking boost.
`
`Reviewed Status Boosts Rankings
`
`Some search engines also review sites or list them in an associated directory.
`They may also give a boost to sites that have been listed in this way.
`
`Link Popularity Boosts Rankings
`
`As described above, all search engines can determine the popularity of a page
`by analyzing how many links there are to it from other pages. Some engines
`give pages with lots of links, or links from important web sites, a relevancy
`boost.
`
`Direct Hit Boosts Rankings
`
`Direct Hit is a system that measures what users click on from search results in
`order to refine relevancy rankings. This shows which search engines use this as
`a factor. For more information, see the article below:
`
`HotBot Integrates Popularity Into Top Results
`The Search Engine Report, March 3, 1999
`
`Spam
`
`All major search engines penalize sites that attempt to "spam" the engines in
`order to improve their position. One common technique is "stacking" or
`"stuffing" words on a page. This is where a word is repeated many times in a
`row. If the search engines spot a spamming technique, they may downgrade a
`page's ranking or exclude it from listings altogether. The items below cover
`design elements that could cause a spam penalty. More in depth information
`specific to each search engine is available to site subscribers.
`
`Meta Refresh
`
`Some site owners create target pages that automatically take visitors to different
`pages within a web site. See the What Is A Bridge Page article for more
`information about this.
`
`The meta refresh tag is one typical way of doing this. Some search engines will
`refuse to index a page with a high meta refresh rate. Go will not index pages
`with any redirection, whatsoever.
`https://web.archive.org/web/20000511205630/http:/www.searchenginewatch.com/webmasters/features.html
`
`5/6
`
`

`

`5/16/2014
`
`with any redirection, whatsoever.
`
`Search Engine Features For Webmasters
`
`Google doesn't worry much about meta refresh tags or the items below
`because its link popularity ranking system pretty much defeats spam attempts.
`
`Invisible Text
`
`This is the technique of placing text on a page in the same color as the
`background, making it invisible to human viewers. Many search engines either
`refuse to index this text or will not index any page containing invisible text.
`
`Tiny Text
`
`This is the technique of placing text on a page in a small font size. Pages that are
`predominantly heavy in tiny text may be dismissed as spam. Or, the tiny text
`may not be indexed. As a general guideline, try to avoid pages where the font
`size is predominantly smaller than normal.
`
` Go
` Choose Another Page
`or use the site map if you can't run JavaScript.
`You may also search the site.
`
`Free Newsletter!
`Enter your email address below to get a
`monthly newsletter about search engines
`
` Subscribe
`Learn more about the newsletter
`
`Like This Site? Click Here To Tell A Friend!
`
`By Danny Sullivan
`Search Engine Watch
`http://searchenginewatch.com/
`Copyright © 1996-2000 internet.com Corp.
`
`https://web.archive.org/web/20000511205630/http:/www.searchenginewatch.com/webmasters/features.html
`
`6/6
`
`

`

`5/16/2014
`
`Search Engine Features For Webmasters
`
`http://searchenginew atch.com/w ebmasters/features.html
`
`Go
`
`247 captures
`6 Jul 98 - 19 Sep 13
`
`NOV
`
`2000
`
`DEC
`
`14
`
`2001
`
`SEP
`
`2003
`
`Search Engine Features For Webmasters
`
`The search engine features chart below is designed primarily for webmasters
`who care about how crawler-based search engines index their sites. It provides
`a summary of important factors and features that can affect how sites are
`indexed and ranked. Full explanations of items can be found immediately below
`the comparison chart.
`
`Human-powered search engines, like Yahoo, are not listed on this chart because
`they do not crawl the web to create their listings. See How Search Engines
`Work for an explanation of the differences between crawler-based and human-
`powered services.
`
`See the Search Engine Features For Searchers page for a summary of how
`search engines display their results and other information that may be of interest.
`
`More detailed information about the search engines below is available to Search
`Engine Watch "site subscribers." This information is just one of the many benefits
`that site subscribers receive. Click here to learn more about becoming a site
`subscriber.
`
`This chart is as of July 2, 2001. It covers the crawler-based portions of AltaVista, Excite,
`FAST Search, Google and Northern Light. It also covers Inktomi results that form portions
`of AOL Search, HotBot and MSN Search. Excite covers portions of Excite-owned Magellan
`and WebCrawler. FAST Search covers crawler data used by Lycos. The Major Search
`Engines page has links to the search engines on this chart.
`
`
`
`Crawling
`Deep Crawl
`
`Frames Support
`
`Image Maps
`
`Yes
`All but...
`
`All but...
`
`AltaVista,
`NLight
`
`robots.txt
`
`All
`
`Meta Robots Tag
`
`All but Excite
`
`Link Popularity
`Helps Deep Crawl
`
`All
`
`No
`Excite
`
`Excite, FAST
`
`Excite, FAST,
`Google,
`Inktomi
`
`n/a
`
`n/a
`
`n/a
`
`Learns Frequency AltaVista, Inktomi
`
`Excite, FAST,
`Google, NLight
`
`Notes
`
`
`
`
`
`
`
`
`
`
`
`
`AltaVista, Inktomi,
`FAST (coming
`9/01)
`URL Status Check
`-- See Checking Your URL & Search Features Chart --
`https://web.archive.org/web/20011214200842/http:/searchenginewatch.com/webmasters/features.html
`
`Paid Inclusion
`
`Excite, Google
`
`
`
`1/6
`
`

`

`5/16/2014
`
`Indexing
`
`Full Body Text
`
`Stop Words
`
`Yes
`
`All
`
`Search Engine Features For Webmasters
`No
`Notes
`Some stop words
`may not be indexed
`
`n/a
`
`AltaVista, Excite,
`Inktomi, Google
`
`FAST, NLight
`
`Google,
`NLight
`Excite, FAST,
`Google, NLight
`Excite, FAST,
`Inktomi, NLight
`
`Others
`
`
`
`
`
`
`
`
`
`
`
`Meta Description
`
`All but...
`
`Meta Keywords
`
`All but...
`
`ALT text
`
`Comments
`
`Stemming
`Ranking
`
`Meta Tags
`Boost Ranking
`
`Link Popularity
`Boosts Ranking
`Direct Hit
`Boost Ranking
`Spam
`
`AltaVista,
`Google
`
`Inktomi
`
`Yes
`
`Inktomi
`
`All
`
`HotBot
`
`Yes
`
`Meta Refresh
`
`AltaVista
`
`Invisible Text
`
`Others
`
`Tiny Text
`
`AltaVista, Inktomi,
`Google
`
`Crawling
`
`-- See Search Features Chart --
`No
`AltaVista, Excite,
`FAST, Google,
`NLight
`
`Notes
`
`
`
`n/a
`
`Others
`
`No
`Excite, FAST,
`Google, Inktomi,
`NLight
`
`Excite, FAST
`
`Excite, FAST,
`NLight
`
`Very important
`at Google
`
`
`
`Notes
`
`
`
`
`
`
`
`This section covers factors related to how well search engines crawl web sites.
`
`Deep Crawl
`
`The search engines shown doing deep crawls will list many pages from a web
`
`site, even if the pages are not explicitly submitted to them. The others will usually
`list far fewer pages from a site. In general, the larger a search engine's index is,
`the more likely it will list many pages per site. See the Search Engine Sizes page
`for the latest index sizes at the major search engines.
`
`Frames Support
`
`This shows which search engines can follow frame links. Those that can't will
`probably miss listing much of your site. Be sure to read the Search Engines And
`Frames page for tips on overcoming the problems with frames and search
`engines.
`
`Image Maps
`
`This shows which search engines can follow client-side image maps. As with
`frames, those search engines that can't follow image maps will probably missing
`https://web.archive.org/web/20011214200842/http:/searchenginewatch.com/webmasters/features.html
`
`2/6
`
`

`

`5/16/2014
`
`Search Engine Features For Webmasters
`frames, those search engines that can't follow image maps will probably missing
`listing much of your site.
`
`robots.txt
`
`The robots.txt file is a means for webmasters to keep search engines out of their
`sites. Search Engine Watch members have access to the How To Block Search
`Engines page, that covers the robots.txt file in more detail. More information
`about robots.txt can also be found here:
`
`The Web Robots Pages: The Robots Exclusion Protocol
`http://www.robotstxt.org/wc/exclusion.html#robotstxt
`
`Meta Robots Tag
`
`This is a special meta tag that allows site owners to specify that a page shouldn't
`be indexed. It is explained more on the How Meta Tags Work page and on the
`More About The Meta Robots Tag page, available to Search Engine Watch
`members. More details can also be found here:
`
`The Web Robots Pages: The Robots META tag
`http://www.robotstxt.org/wc/exclusion.html#meta
`
`Link Popularity Helps Deep Crawl
`
`All search engines can determine the popularity of a page by analyzing how many
`links there are to it from other pages. Some engines use this as a means to
`determine which pages they will include in the index. This is NOT the same as
`ranking a page better for having good link popularity. That is explained further
`below. Also see the Measuring Link Popularity page for tips on measuring your
`site's popularity according to different engines.
`
`Learns Frequency
`A number of search engines can learn how often pages change. Pages that
`change often may be visited more frequently.
`
`Paid Inclusion
`
`Shows whether a search engine offers a program where you can pay to be
`guarantee that your pages will be included in its index. This is NOT the same as
`paid placement, which guarantees a particular position in relation to a particular
`search term. See the Buying Your Way In page for more about paid inclusion
`programs.
`
`URL Status Check
`
`There are various ways to find whether an exact page is listed at the different
`search engines. The Checking Your URL page explains this in great detail, while
`the Search Assistance Features page also provides some coverage.
`
`Indexing
`
`This section explains what gets indexed when search engines spider a page.
`
`Full Body Text
`
`All of the major search engines say they index the full visible body text of a page,
`https://web.archive.org/web/20011214200842/http:/searchenginewatch.com/webmasters/features.html
`
`3/6
`
`

`

`5/16/2014
`
`Search Engine Features For Webmasters
`All of the major search engines say they index the full visible body text of a page,
`though some will not index stop words or exclude copy deemed to be spam
`(explained further below).
`
`Stop Words
`
`Some search engines either leave out words when they index a page or may not
`search for these words during a query. These "stop words" are excluded as a
`way to save storage space or to speed searches.
`
`Meta Description & Meta Keywords
`
`Shows which search engines support the meta description and meta keywords
`tags, as explained on the How Meta Tags Work page. This does NOT mean
`that using these tags gives pages a ranking boost. That is covered in a separate
`section, below.
`
`ALT Text / Comments
`
`This shows which search engines index ALT text associated with images or text
`in comment tags.
`
`Stemming
`
`Some search engines will search for variations of a word based on its stem. For
`example, entering "swim" might also find "swims" and maybe "swimming,"
`depending on the search engine. This is explained in more detail on the Search
`Assistance Features page.
`
`Ranking
`
`Most search engines use the location and frequency of keywords on a web page
`as the basis of ranking it in response to a query. The exact mechanism is slightly
`different for each engine.
`
`In addition to location and frequency, some engines may give a page a relevancy
`boost based on other factors. These usually can help a little, but they don't
`guarantee a boost to the top. Some major factors are listed below.
`
`Also see the How Search Engines Rank Web Pages article for a more in-depth
`look at how relevancy is determined, and the Search Engine Design Tips page
`for helpful advice about optimizing your pages for search engines.
`
`Meta Tags Boost Ranking
`
`Some search engines that support the meta description and keywords tag will
`also give