Digital Equipment Offers Web Browsers Its 'Super Spider' - The New York Times
`
`€1Jt ~r\tr ~ Jork €hnts
`""
`.....,
`This copy is for your personal. noncormnercral use only. You can order presentation-ready copies for
`distribution to your colleagues. clients or customers. please click here or use the "Reprints" tool that appears
`next to any article. Visit www.nytreprints.com for samples and additionalrnformation. Order a reprint of this
`article now. »
`
`December 18, 1995
`
`Digital Equipment Offers Web Browsers Its 'Super
`Spider' ·
`
`By PETER H. LEWIS
`
`If you build a better spider, the World Wide Web will beat a path to your door. At least, that is the
`hope of the Digital Equipment Corporation, which released a high-speed system for finding
`information on the Internet's rapidly expanding World Wide Web multimedia service on Friday.
`
`A prototype of the new search system, formally called Alta Vista but known popularly as a "super
`spider," was made available to the Web-browsing public at no charge.
`
`Allan L. Jennings, Digital's manager of advanced technology business development, said the
`company would decide later if it would eventually charge for the searching software, offer
`subscriptions, or make it available free and recoup its investment through advertising or
`sponsorship. It is expected that Alta Vista will draw large crowds to Digital's Web site and act as a
`showcase for Digital's own Internet products.
`
`Digital asserts that its new software is "an order of magnitude" faster than current methods for
`finding information on the Web, which now consists of more than 30 million electronic pages of
`information stored on tens of thousands of different computers around the world, with thousands
`of new pages being added each day.
`
`Alta Vista also scans more than 13,000 different discussion forums on the Usenet network.
`
`Attempts to find just the right piece of information in the expanding maze of Internet computers
`can be frustrating. More than a dozen companies now offer Web-searching systems -- they are
`known as spiders, crawlers, wanderers and other similar names-- and each searches in different
`ways, with varying degrees of accuracy and comprehensiveness.
`
`Digital is positioning Alta Vista as the fastest and most precise information agent on the Web. The
`software program creates complete indexes of every word on every Web page or Usenet news
`group it encounters, allowing the spider to make highly targeted searches.
`
`The Alta Vista searcher also enhances its performance by sending out what Digital calls a "brood
`of spiders," technically known as threads, to scan Web sites much more quickly. A super spider
`search can consist of as many as 1,000 simultaneous threads.
`
`http://www.nytimes.com/1995/12/18/business/digital-equipment-offers-web-browsers-its-super-spider.html?pagewanted=print
`
`1/2
`
`

`
`Digital Equipment Offers Web Browsers Its 'Super Spider' -The New York Times
`Samuel H. Fuller, vice president of corporate research at Digital's Palo Alto, Calif., research center,
`said the software runs particularly quickly on Digital's Alpha computers, which are based on a
`microprocessor that processes twice as much information at a time, 64 bits, as the 32-bit
`computers now widely used as Web servers. The search is conducted in vast indexes managed by
`the Alpha computer, so searches are conducted quickly regardless of the speed of a user's
`computer.
`
`"It's a great showcase for the Alpha, which most people agree is a fast chip and a good computer,"
`said Matthew Koll, president of Personal Library Software of Rockville, Md., a leading maker of
`information retrieval software for the Web. "Lots of existing products would run faster on Alphas."
`
`But, Mr. Koll added, speed is less important than the spider's ability to locate and deliver the most
`relevant information needed by the user.
`
`Mr. Fuller of Digital acknowledged the risk of millions of spiders clogging the Internet and
`potentially overwhelming the network connections of small Web information providers. He said
`Digital intends to detect the capacity of each Web site the spider visits and avoid a disabling drain
`on that site.
`
`Brian Pinkerton, creator of a popular Web search tool called WebCrawler and now a consultant to
`America Online Inc., said the greater drain on the Internet's capacity is not the search itself, or
`'
`even the indexing of pages, but rather the number of search queries that are conducted.
`
`A year ago, Mr. Pinkerton said, Webcrawler was asked to conduct 30,000 search requests a day.
`Today, the volume has increased to 2.5 million requests a day, he said.
`
`The super spider is on the Web at http:/ /www.altavista.digital.com.
`
`~g_Qyr]ght 2015 The ~w _Y.grls_limes Co~QY. Home
`Bac!U_o_.I9..l2
`
`Pr~acy Pq.!!_gy
`
`Search
`
`_Q_orr~tion~ XML
`
`H~Q Contaf.!J)..§.
`
`http://www.nytimes.com/1995/12/18/business/digital-equipment-offers-web-browsers-its-super-spider.html?pagewanted=print
`
`212