An Analysis of Issues Facing World Wide Web Servers
`
`A Thesis presented
`
`by
`
`Stephen Lee Manley
`
`to
`
`Computer Science
`
`in partial fulfillment of the honors requirements
`
`for the degree of
`
`Bachelor of Arts
`
`Harvard College
`
`Cambridge, Massachusetts
`
`April 7, 1997
`
`Google Inc.
`GOOG 1018
`CBM of U.S. Patent No. 6,286,045
`
`i
`
`

`
`Abstract
`The World Wide Web has captured the public’s interest like no other computer application
`
`or tool. In response, businesses have attempted to capitalize on the Web’s popularity. As a
`
`result propaganda, assumption, and unfounded theories have taken the place of facts, sci-
`
`entific analysis, and well-reasoned theories. As with all things, the Web’s popularity
`
`comes with a price for the first time the computer industry must satisfy exponentially
`
`increasing demand. As the World Wide Web becomes the “World Wide Wait” and the
`
`Internet changes from the “Information Superhighway” to a “Giant Traffic Jam,” the pub-
`
`lic demands improvements in Web performance.
`
`The lack of cogent scientific analysis prevents true improvement in Web conditions.
`
`Nobody knows the source of the bottlenecks. Some assert that the server must be the prob-
`
`lem. Others blame Internet congestion. Still others place the blame on modems or slow
`
`Local Area Networks. The Web’s massive size and growth have made research difficult,
`
`but those same factors make such work indispensable.
`
`This thesis examines issues facing the Web by focusing on traffic patterns on a variety
`
`of servers. The thesis presents a method of categorizing different Web site growth patterns.
`
`It then disproves the theory that CGI has become an important and varied tool on most
`
`Web sites. Most importantly, however, the thesis focuses on the source of latency on the
`
`Web. An in-depth examination of the data, leads to the conclusion that the server cannot be
`
`a primary source of latency on the World Wide Web.
`
`The thesis then details the creation of a new realistic, self-configuring, scaling Web
`
`server benchmark. By using a site’s Web server logs, the benchmark can create a model of
`
`the site’s traffic. The model can be reduced by a series of abstractions, and scaled to pre-
`
`dict future behavior. Finally, the thesis shows the benchmark models realistic Web server
`
`traffic, and can serve as a tool for scientific analysis of developments on the Web, and their
`
`effects on the server.
`
`1
`
`

`
`Table of Contents
`
`1 Introduction
`
`2 Related Work
`
`2.1 Web Server Traffic Analysis
`
`2.2 Modeling of WWW Traffic
`
`2.3 Analysis of Recent Developments in Web Traffic
`
`2.4 Benchmarking
`3 Analysis of Web Server Access Logs
`
`3.1 Site characterization
`
`3.2 General Analysis
`
`3.3 Differentiating Among Sites
`
`3.4 CGI
`
`3.5 Persistent Connections
`
`3.6 Caching
`
`3.7 Latency
`4 Design of a Web Server Benchmark
`
`4.1 Philosophy of Benchmarking
`
`4.2 Modeling Traffic from Server Logs
`
`4.3 Statistical Basis for Model
`
`4.4 Reducing the Model
`
`4.5 Changing the Traffic Patterns
`
`4.6 Scaling the Model to Predict Future Traffic pattern
`
`4.7 Run Time Environment
`
`4.8 Conclusion
`
`2
`
`4
`
`8
`
`8
`
`12
`
`13
`
`15
`18
`
`19
`
`21
`
`25
`
`32
`
`35
`
`40
`
`46
`54
`
`54
`
`56
`
`57
`
`59
`
`60
`
`62
`
`64
`
`67
`
`

`
`3
`5 Experimental Proof of the Accuracy of the Benchmark-
`
`ing Model
`
`5.1 Experimental Setup
`
`5.2 Analysis of the Basic Model
`
`5.3 Analysis of the Reduced Models
`
`5.4 Analysis of Generating Specific Types of Traffic
`
`5.5 Analysis of Scaling
`
`5.6 Results
`
`5.7 Conclusion
`6 Conclusion
`
`6.1 Future Work
`
`6.2 Availability
`
`6.3 Appreciation
`
`69
`
`69
`
`70
`
`71
`
`74
`
`74
`
`76
`
`78
`80
`
`81
`
`82
`
`82
`
`

`
`Chapter 1
`
`Introduction
`
`Computer scientists have been looking at exponential growth patterns for years. Exponen-
`
`tial growth in computing power has extended computer functionality in ways that few peo-
`
`ple thought possible. Only four years ago, an Intel 486, 33MHz processor was considered
`
`the top-of-the-line. Now, the Intel Pentium Pro runs at 200 MHz -- 6 times as fast. Four
`
`years ago, machines ran with 4 MB of RAM. Now, retailers sell standard PCs with 16 MB
`
`of RAM. The advances in computer power have made a desktop machine more powerful
`
`than the supercomputers of the past. Advances in technology enable someone to send text,
`
`pictures, movies, or sounds to another person thousands of miles away.
`
`After years of enjoying exponential growth in computing power, the computer com-
`
`munity now faces exponential growth in demand. The Internet and World Wide Web
`
`(WWW or Web) have captured the world's imagination like no other computer product or
`
`service. The Internet, nicknamed “The Information Superhighway” enables any two com-
`
`puters in the world to send information to one another. Popular World Wide Web browsers
`
`such as Netscape or Microsoft Internet Explorer give users a simple to use Graphical User
`
`Interface (GUI). With this simple but powerful interface, a user can download a file from
`
`another computer with only a click of the mouse. The simplicity of the Web promotes the
`
`construction of Web sites, groups of Web pages, for purposes ranging from Kellogg's
`
`advertising their cereals to high school and college students setting up their resumes. The
`
`WWW has enabled everybody to become a publisher, with the HyperText Markup Lan-
`
`4
`
`

`
`5
`
`guage (HTML) facilitating the layout of text, pictures, animated pictures, sounds, and
`
`movies onto one Web page. Moreover, advances in technology continue to extend the
`
`functionality of the Internet. The Common Gateway Interface (CGI) enables somebody
`
`making a Web site to create a program that behaves differently for each client that
`
`accesses the site. Java allows the user to download a program from the server to his/her
`
`own computer. The WWW is simple to use and is becoming increasingly powerful.
`
`The simplicity, power, and ubiquity of the Web sparked, and continue to fuel, its tre-
`
`mendous growth. Table 1 shows the growth of the Internet and the much more rapid
`
`growth of the Web from December 1991 until March 1995 [8]. Perhaps it is the greatest
`
`testament to the growth of these services, that in the past two years nobody has even
`
`attempted to quantify the amount of traffic on the Internet or the Web. The best estimates
`
`on the number of Web pages and servers come from the Alta Vista search engine run by
`
`the Digital Electronic Corporation. Alta Vista claims to have indexed over thirty-one mil-
`
`lion WWW pages on over 476,000 WWW sites.
`
`Date
`
`Total Traffic (GB)
`
`WWW
`Percentage
`0.00
`1900
`12/91
`0.02
`4300
`12/92
`2.21
`8200
`12/93
`15.98
`16,000
`12/94
`23.88
`20,329
`3/95
`Table 1: Internet and Web growth from 12/91 to 3/95
`Perhaps the greatest indication of the Web’s continued growth
`comes from the fact that nobody even attempts to determine the
`traffic currently on the Internet.
`
`The exponentially increasing demand for services due to the World Wide Web has led
`
`to great concerns about scaling computing resources to meet future needs. America Online
`
`Users can scarcely login to their system. Most users experience tremendous latencies in
`
`using the WWW during the day, leading to the nickname - “World Wide Wait.” As a
`
`result, companies and individuals continually ask, “How can the Internet and World Wide
`
`Web effectively scale with demand?” Microsoft and Netscape are trying to make their
`
`WWW server software increasingly powerful. Silicon Graphics, Incorporated and Sun
`
`claim that their only their machines have the requisite power needed to run a WWW
`
`

`
`6
`
`server. Internet Service Providers buy hundreds of megabits of bandwidth to serve their
`
`clients' needs. Researchers continue to refine the HyperText Transfer Protocol (HTTP)[1],
`
`which enables servers to send Web pages to clients. While the developments are exciting,
`
`and seem to hold great promise, all of the proposed enhancements to the Web suffer from
`
`an identical flaw. How can anybody solve a problem, without knowing exactly what the
`
`problem is? Companies spend millions of dollars and thousands of work hours on improv-
`
`ing WWW performance. Every hacker claims to know how to improve web performance,
`
`but nobody has even identified the source of WWW performance degradation.
`
`The nature of the World Wide Web has made it an extremely difficult phenomenon to
`
`measure. The distributed and extensive nature of Web makes it difficult to understand all
`
`of the issues involved. Those who possess interesting information on client behavior
`
`refuse to distribute it. Users don't want their behavior and access patterns made public, for
`
`obvious reasons. Webmasters refuse to make access logs from their servers available, due
`
`to possible security holes as well as privacy concerns. Finally, by the time the information
`
`is released and the analysis completed, the conclusions have become obsolete. With a lack
`
`of current and widely available general information, one has a very difficult time analyz-
`
`ing the World Wide Web.
`
`The goal of this work is to conduct scientific analysis on the effects of the server on
`
`WWW scaling. Servers create logs of data, known as server, access, or Web logs, that
`
`detail the requests sent to the server. Access logs can reveal the type of workload being
`
`placed on a server, as well as its ability to handle such a load. From a series of these logs,
`
`one can also see trends in the type of traffic on that server. Therefore, to understand the
`
`role of the server's effect on the handling of increased WWW traffic, as well as its limita-
`
`tions and assets in handling a greater workload, I analyze a variety of WWW logs. While
`
`an analysis of the traffic patterns handled by a server is useful, one would like to actively
`
`test the server, to monitor its behavior as it runs. Benchmarks were created for the very
`
`reason of simulating realistic behavior, in a controlled environment. To gather interesting
`
`data about the server, the benchmark must realistically model the type of load experienced
`
`by a server. To this end, I have constructed a self-configuring WWW benchmark. The
`
`

`
`benchmark analyzes a set of server logs to create a model of the site’s traffic, which
`
`becomes the basis of the benchmark’s activity.
`
`7
`
`Thesis Roadmap
`Chapter 2 discusses other research into the access patterns to WWW servers and the
`
`history of Web server benchmarking. Chapter 3 presents an analysis of server traffic pat-
`
`terns for a variety of sites. Chapter 4 details the design of the self-configuring, scaling,
`
`WWW server benchmark. Chapter 5 explores the quality of the benchmark’s approxima-
`
`tion of actual WWW traffic patterns. Finally, Chapter 6 concludes the work, presenting the
`
`most important results, and their importance in the issue of improving the scalability of the
`
`Web.
`
`

`
`Chapter 2
`
`Related Work
`
`8
`
`A survey of the relevant work in measuring Web site traffic and server performance
`
`leads to one inescapable conclusion. No real analysis of the Web has been conducted in
`
`the past two years. Basing all of the current developments on traffic patterns from two
`
`years ago can be considered nothing short of disastrous. The work presented laid a good
`
`foundation for a more in-depth analysis of Web traffic patterns. Such work never occurred.
`
`The chapter discusses the research into the fields that pertain to Web server perfor-
`
`mance. Section 2.1 examines statistical analyses of the traffic experienced by a variety of
`
`WWW servers. Section 2.2 considers current attempts at statistical modeling of WWW
`
`traffic. Section 2.3 covers research that measure the impact of new developments on Web
`
`server performance. Section 2.4 discusses the range of benchmarks currently available.
`
`2.1 Web Server Traffic Analysis
`Research into realistic traffic patterns has been extremely limited. The research efforts
`
`have been divided between proxy servers and primary Web servers. A proxy server acts as
`
`a mediator between primary Web servers and users. All user requests for a given area,
`
`such as Harvard University, can be funneled through a proxy server, which then passes the
`
`request to the intended primary server. Much of the analysis of servers has been conducted
`
`to help justify the viability and effectiveness of other research. The difficulty in conduct-
`
`ing accurate and effective analyses have been limited by the difficulty in finding accessible
`
`Web logs, the enormous variety of issues to explore, and the rapidly changing nature of the
`
`Web.
`
`Information about the behavior of users and servers is not freely available to research-
`
`ers. The Web pages that any given user accesses are known only to that user. Only when
`
`

`
`9
`
`user requests are directed through a proxy can one collect information on the behavior of
`
`real users. Even in these situations, users obviously do not want information about their
`
`Web surfing habits released. As a result, proxy server logs remain unavailable. On the
`
`other side of the connection, webmasters are unwilling to distribute information about a
`
`server. Some claim that access to server logs could lead to the discovery and exploitation
`
`of security holes. Others do not wish to identify those who access their site, as they feel
`
`their customers would feel violated and not return. Some ISPs don't want to release infor-
`
`mation on the number of requests and bytes of data they handle, as they fear it might make
`
`clients unwilling to subscribe to them for service. Furthermore, those who have gained
`
`access to logs, or have done research in the past, have seen the conditions of the Web
`
`change so rapidly, that their conclusions are outdated. Still, despite the difficulties inherent
`
`in analyzing Web traffic, other researchers have laid the groundwork, and posed interest-
`
`ing questions that should continue to be analyzed.
`
`2.1.1 Caching
`
`Pitkow and Recker used WWW logs from the Georgia Tech main university server,
`
`from January 1, 1994 to March 31, 1994 to test the effectiveness of caching schemes[12].
`
`In the three months, the server handled approximately 305,000 requests, with an average
`
`of 3500 requests a day. The researchers were attempting to model the likelihood that a file
`
`would be accessed, based on the previous weeks' traffic. Their model, based on human
`
`memory patterns, could quite accurately predict the likelihood that a file would be
`
`accessed during the coming week. Less than a year later, this research was obsolete.
`
`By 1995 researchers looking at WWW caching were examining ways to reduce the
`
`bandwidth consumption of the Internet, as traffic had increased exponentially in the time
`
`since the Georgia Tech research was completed. No longer were people concerned with
`
`whether a file would be accessed, but with how often it was accessed, and the concurrent
`
`bandwidth consumption. Bestavros and Gwertzman both examined the benefits of server-
`
`side caching. Their primary conclusion from WWW server logs was that, as users surf the
`
`Web, most requests download a small number of unique bytes. From this data, they advo-
`
`cate that a server can effectively cache its most popular files in a variety of proxy caches,
`
`thus reducing latency and bandwidth consumption on the Internet.
`
`

`
`10
`
`Bestavros and Gwertzman both use Web server access logs to detail the potential ben-
`
`efits of server-side caching. Bestavros examined a series of 1994 and 1995 logs from the
`
`Boston University(BU) Computer Science Department server. His support for the effec-
`
`tiveness of caching is enhanced by the conclusion that 0.5% of the files account for 69% of
`
`requests, and that 91% of requests download a mere 10% of unique bytes on the server. He
`
`contends that this behavior is indicative of the entire Internet because the logs of the Roll-
`
`ing Stones Fan Page, from Nov. 1, 1994 to Feb. 19, 1995 exhibit similar patterns[2].
`
`Gwertzman derives similar results from the December 1994 logs of the NCSA site, the
`
`main Harvard University site, the Harvard Division of Engineering and Applied Sciences
`
`site, and a Harvard computer club server[8]. A small number of bytes and files account for
`
`the majority of sites' requests. Interestingly, Gwertzman's figures show a significant dis-
`
`parity between the most popular sites, and common sites. The Harvard sites actually han-
`
`dle less traffic than that seen by the Georgia Tech site - between 1400 and 2800 requests a
`
`day. The NCSA site processed over 600,000 requests a day - over half a year's traffic on
`
`the other servers listed by Gwertzman. Still despite the enormous disparity in the types of
`
`sites, Gwertzman and Bestavros both claim that their extremely focused analysis of a lim-
`
`ited number of sites gives sufficient justification for their theories. My detailed examina-
`
`tion of over ten sites in chapter 3 will discuss issues ignored by Gwertzman and Bestavros,
`
`as well as the current needs for caching.
`
`An analysis of proxy server logs helps characterize user behavior. Proxy server logs
`
`reveal access patterns of a subset of users to all sites, rather than all users to one Web site,
`
`so that the entire behavior pattern of a variety of users can be seen. Duane Wessels set up a
`
`proxy server at the University of Colorado at Boulder and analyzed the traffic patterns
`
`from October 31, 1994 to December 31, 1994[16]. Over the 62 day period, almost 12,000
`
`requests were generated - of which only 7.8% were handled by the proxy server's cache.
`
`This result surprised Wessels because he assumed that the people at one college would all
`
`tend to access the same pages more often. With the greater number of Web sites and users,
`
`this number it seems unlikely that this number would increase. The type of traffic was also
`
`detailed : 99% of requests were to GET files, while 1% were to POST data to the remote
`
`server. Most of the time, users merely downloading information, and did not send any-
`
`thing to the server. Of the requests, 42% were for HTML files, and another 23% of
`
`

`
`11
`
`requests were for index files, HTML files or directory listings. Only 12.5% of requests
`
`were for image files, and another 3% of requests were due to CGI. Surprisingly, most
`
`requests from the University of Colorado resulted in text being returned. With the prolifer-
`
`ation of images on the Web, this seemed unusual. Wessels does note that the University of
`
`Colorado used text-based browsers, which would decrease the quantity of image traffic to
`
`the site's users. Still, his analysis provides interesting data. First, while Gwertzman and
`
`Bestavros show that most Web sites show a significant skewing of requests for a small per-
`
`centage of files, the same skewing cannot be seen in the requests emerging from a site.
`
`Moreover, over 65% of requests were for HTML and index files. The data show that in
`
`1994, different sites handle different levels of traffic, that HTML was a primary source of
`
`traffic, and that requests to a site's Web server showed much greater skewing than requests
`
`through a proxy server. Wessels work into analyzing proxy server logs forms a model for
`
`my own analysis. He used the logs, not to further an agenda, but to assess the current types
`
`of Web traffic.
`
`2.1.2 Network Behavior
`
`Jeffrey Mogul conducted the most comprehensive analysis of server behavior, record-
`
`ing information on a set of busy servers as they ran[10]. Mogul's analysis of three servers
`
`used during the California elections of 1994 was facilitated by his instrumentation of serv-
`
`ers to record the number of concurrently open HTTP requests, connection duration, PCB
`
`(process control block) statistics, and packet transmission statistics. The three servers han-
`
`dled 1.6 million requests in a 10 day period, with 1 million requests coming on one day.
`
`Combined, the servers handled as many as 60 requests per second. Some requests required
`
`more than 1000 seconds to complete, but most were handled in under one second. The
`
`PCB table, which stores information on each open connection, grew as large as 1297
`
`entries, 1049 of which were in TIME_WAIT - a state to ensure proper closure of the con-
`
`nection. At this time, only 100 connections were alive. Over 90% of requests used fewer
`
`than 25 packets between client and server. The fact that most Web requests are small
`
`means that they can be handled more quickly than the infrastructure for the connection can
`
`be discarded. Nearly 70% of requests were for GIF formatted images, while only 30% of
`
`requests were for HTML files. Other types of traffic were negligible. In this data, one can
`
`

`
`12
`
`see the more expected pattern of images dominating the types of requests. Mogul's in-
`
`depth analysis of a busy Internet server in 1994, yielded information about the need for
`
`maintaining PCB (Protocol Control Block -- the infrastructure of the connection) tables
`
`effectively, to cope with large numbers of open connections, and even larger numbers of
`
`closed, but not destroyed, connections. His work described the increasingly prominent
`
`role of images in Web traffic. While Mogul makes no claim that the site he analyzes is
`
`indicative, he still only looks to one site. Furthermore, like Wessels' work, the data and
`
`analysis suffer from over two years of aging. Therefore, in Chapter 3, I extend Mogul's
`
`analysis of connections, type of traffic, burstiness, and types of traffic on a server to cur-
`
`rent data, for a wide variety of sites. I also provide a means of characterizing sites to help
`
`alleviate the myth that all sites exhibit similar characteristics.
`
`2.2 Modeling of WWW Traffic
`Researchers have begun to statistically model the traffic patterns found on the World
`
`Wide Web. A current theory in the field is that Web traffic from a set of clients at a site, fits
`
`the model of self-similarity. By describing Web traffic as self-similar, one looks at the
`
`inherent burstiness of Web traffic. Burstiness is a characteristic by which many requests
`
`arrive at once, followed by stretches of time with few or no connections. The bursts,
`
`whether examined on a time scale of one second, one minute, or one hour, look similar. If
`
`Web traffic were self-similar, current burstiness would correlate to future burstiness --
`
`long-range dependence. Crovella and Bestavros use the BU client traces detailed in Sec-
`
`tion 2.1.1 to show similar patterns of burstiness over four orders of magnitude of time - 1
`
`to 1000 seconds[4]. They note the importance of user “think time” when they read the
`
`Web page they have downloaded. The work concludes that WWW traffic is self-similar,
`
`but it cannot explain the cause of the self-similarity. The authors conjecture that the self-
`
`similar behavior can be traced to the basic hierarchical structure of Web sites, and the
`
`effects on the user. For my benchmark, I use the idea of modeling Web traffic, but rather
`
`than using a mathematical model at the request level, I create a heuristic model on a user
`
`level. Still, the fundamental ideas of the importance of user idle time, and burstiness play a
`
`role in any accurate model of Web traffic. Furthermore, like Crovella and Bestavros, I con-
`
`

`
`tend that future WWW traffic patterns can be reasonably predicted, and use such methods
`
`to form the basis of a benchmark that scales its load to model the future.
`
`13
`
`2.3 Analysis of Recent Developments in Web Traffic
`Other levels of research have gone beyond measurement and estimation of traffic pat-
`
`terns. In an attempt to identify performance limitations on systems, researchers have
`
`examined the effects of CGI traffic on a server, as well as the potential benefits of persis-
`
`tent connections. Some people have theorized that CGI traffic causes a server's perfor-
`
`mance to degrade because the server starts entirely new processes to execute the task. On
`
`the other hand, people have proclaimed that persistent connections can ease the burden
`
`placed on servers. Originally, Web browsers made a separate connection to the server for
`
`each downloaded file. As many Web pages contain tens of small files -inlined images- and
`
`the cost of setting up a connection is large, the performance impact of this process would
`
`seem to be noticeable to users. Moreover, as Mogul noted in his analysis of the California
`
`election server, the operating system needed to deal with thousands of closed, but not
`
`deleted, connections. Thus, researchers have attempted to quantify the benefits of persis-
`
`tent connections. Section 2.3.1 describes the work done to analyze the impact of CGI on
`
`server performance. Section 2.3.2 discusses the analyses of persistent connections.
`
`2.3.1 CGI
`
`The analysis of CGI focused on the process of handling one CGI request. Edward and
`
`Rees offer a quantitative analysis of a CGI request that details the amount of time spent in
`
`different stages of the request[5]. The work used a CGI script with trivial functionality to
`
`isolate the effects of a server handling CGI, rather than the script itself. Their platform of
`
`measurement was CERN httpd, one of the originally popular Web servers. Their data indi-
`
`cate that the most time-consuming operation is the transfer of request data to the CGI
`
`script. On the other hand, their conclusions focus on optimization of process creation -
`
`which consumed only one-tenth of the time required to do the IPC. The study's limited and
`
`contradictory nature, and the non-trivial amount of CGI traffic run on servers led to a fur-
`
`ther analysis of the Edwards and Rees work. In chapter 3, I emphasize the insignificance
`
`of CGI on most servers.
`
`

`
`14
`
`2.3.2 Persistent Connections
`
`The original analysis of persistent connections examined the probability of reuse of
`
`open connections. Mogul ran the server access logs from the popular California election
`
`server, as well as that of a popular corporate site, through a simulator to estimate the effec-
`
`tiveness of persistent connections[9]. The logs revealed that persistent connections could
`
`significantly reduce the number of open connections, by reusing the open connections.
`
`The logs showed that over 50% of users retrieved at least 10 files per open connection. On
`
`average, one connection would support six requests. His results revealed that with a time-
`
`out of sixty seconds, connections were reused fairly often, while also enabling the maxi-
`
`mum number of open connections to remain low. Mogul also focused on the importance of
`
`allowing users to only open one connection at a time, rather than the Netscape driven stan-
`
`dard of four connections per user. The age of his data leads to concerns about the accuracy
`
`of the results. Furthermore, in the construction of the new HTTP protocol, browsers are
`
`allowed to open two persistent connections. Therefore, the work needs to be updated to
`
`examine the effects of such a decision.
`
`Currently, the World Wide Web Consortium(W3C) is conducting an analysis of persis-
`
`tent connections[11]. Rather than using historical logs to simulate behavior, the W3C
`
`chose to implement a client and a server that implement persistent connections. They then
`
`analyzed the performance weaknesses of their implementations. At first, they discovered
`
`that the latency incurred with one persistent connection lagged significantly behind that of
`
`the standard type of request. They concluded that the parallelism of 4 open connections
`
`enabled the client to receive data more quickly than the one open connection - with future
`
`requests not being produced until after current requests. They concluded that only by pipe-
`
`lining requests can persistent connections achieve reasonable latency. By pipelining
`
`requests, servers receive multiple requests from the client before they have even fulfilled a
`
`request. In this way, the transportation of data is not serialized, and performance is
`
`enhanced. On the other hand, in validating a simple page with multiple files, persistent
`
`connections generated only one-tenth of the packets as non-persistent connections. Since
`
`validating pages usually results in a response that the file is unmodified, little data is sent
`
`to the user. This research validates the potential benefits of persistent connections, as well
`
`as detailing a potential performance limitation. However, the authors do not give evidence
`
`

`
`15
`
`of the potential role of persistent connections in real traffic patterns. Their research is
`
`focuses on analysis of assumed situations. Thus, while their results detail the effects of an
`
`actual implementation of persistent connections, their tests suffer from a lack of realism.
`
`They refer to Mogul's research, but understand the limitations of the study. In chapter 3, I
`
`update the research that Mogul began. In chapter 4, I describe a benchmark that can use
`
`persistent connections.
`
`2.4 Benchmarking
`Properly designed and implemented, a benchmark enables researchers, such as the
`
`W3C, to accurately test new developments. Poorly constructed, a benchmark creates
`
`faulty impressions of needs in a given community. Web server research has suffered from
`
`just such a crisis. Benchmarks have either ignored the inherent differences in the traffic
`
`handled by a variety of sites, or have completely ignored the considerations of realism in
`
`generating traffic. Currently, developments in benchmarking technology have attempted to
`
`address some of the flaws of older benchmarks. Section 2.4.1 describes the first Web
`
`server benchmark - Silicon Graphics, Incorporated's WebStone. Section 2.4.2 looks at
`
`SPEC's venture into Web server analysis. Section 2.4.3 considers recent developments
`
`from Lotus and Harvard, with DBench.
`
`2.4.1 WebStone
`
`WebStone filled the complete void of Web server benchmarks, and despite its flaws,
`
`many of its features have rightfully become standards. The general approach of WebStone
`
`is to create a master process that spawns and gathers data from multiple client pro-
`
`cesses[15]. The master spawns multiple clients that repeatedly open one connection to the
`
`server and download one file. The key parameters of the benchmark are fileset, number of
`
`client processes, and the length of the test run. Since it is impossible to quantify WWW
`
`traffic in one way, these parameters are configurable. The theory, then, is that users can
`
`configure the benchmark to simulate the behavior experienced by their sites. Unfortu-
`
`nately, the lack of a well-researched and reasonable standardized load leads to difficulties
`
`in comparing WebStone numbers. In addition, there is no suitable means of deriving a
`
`suitable configuration for a particular site. As is often the case, individuals frequently have
`
`

`
`16
`
`misconceptions about the inner workings of their systems. The configurations used and
`
`reported often generate traffic that has little to no resemblance of real WWW traffic. As an
`
`attempt to provide a standard for comparison and testing, WebStone attempted to offer a
`
`“standard load” with a fileset of five files ranging from 500 bytes to 5 megabytes, varying
`
`by an order of magnitude. The configuration requires running with 20 to 100 users,
`
`increasing by 10 users every 10 minutes. WebStone's final limitation arises from its focus
`
`on throughput. Even with an accurate portrayal of a site in the configuration logs, the
`
`behavior of the WebStone clients is so unlike real browser behavior that the model would
`
`still fail. Since WebStone focuses on the number of bytes that can be transferred, portray-
`
`ing accurate user behavior is not salient. Instead, the benchmark chooses to try to make as
`
`many requests as possible in a short time. Unfortunately, throughput can be as much a
`
`measure of the networking hardware of the system, as performance of the server software
`
`and operating system.
`
`2.4.2 SPECWeb
`
`SPECWeb '96 and '97[13,14] attempt to rectify the lack of comparability of statistics,
`
`while retaining some of the structural aspects of WebStone. SPECWeb also builds upon
`
`the client-master model, but rather than spawning off clients that randomly download files
`
`as rapidly as possible, SPEC designed the benchmark to scale the load it generates to
`
`stress test a server. The file set is determined by the goal for operations per second on the
`
`server. Furthermore, client process behavior is also constrained by this goal. Thus, clients
`
`will remain idle for some amount of time to ramp up the generated load over time. Thus,
`
`the SPEC benchmark attempts to show the results of increasing the load on a server, until
`
`it can handle no more. SPECWeb controls the load generation of its processes with a finer
`
`granularity than WebStone because it controls how clients behave, not just the number of
`
`clients. The metric reported by SPECWeb is a report of the number of operations (connec-
`
`tions) that the server can execute per second. This metric helps show the ability of the
`
`server to handle requests, not merely its ability to send bytes over the network. While
`
`SPEC initially bases its definition of users from analysis of Web server logs, the definition
`
`degrades as users are controlled primarily by the load they generate, rather than approxi-
`
`mation to the log. Like WebStone, the