Bridging the Bandwidth Gap:
`Measures to Maintain Bandwidth Availability
`Cynthia A. Murnan
`Oberlin College
`148 West College St., Oberlin, OH 44074-1532
`440-775-8774
`cynthia.murnan@oberlin.edu
`
`
`ABSTRACT
`With ever-increasing bandwidth requirements due to bandwidth-
`intensive applications such as Napster, Gnutella, Audiogalaxy,
`various Internet games and so forth, we were faced with
`determining how to bridge the gap between the amount of
`bandwidth we provided to our clients and the amount our clients
`consumed in meeting their individual needs and desires.
`This paper will provide details on the issues we confronted in using
`a bandwidth-shaping tool and in deciding to increase our total
`amount of bandwidth. This paper will also discuss the factors we
`dealt with in developing an overarching policy for handling network
`usage.
`
`KEYWORDS
`Bandwidth, bandwidth-shaping, DS-3, IP address, Packeteer,
`PacketShaper®, policies, priorities, Resnet, T-1, web caching.
`
`1. INTRODUCTION
`As with most colleges and universities, we have been faced with
`determining how to provide sufficient bandwidth to meet both the
`academic and administrative needs of faculty and staff, mainly
`during daytime hours, and the needs and desires, academic and
`otherwise, of the students in the residence halls, mainly during
`nighttime hours. Since Oberlin College is a private, liberal arts
`institution that greatly values a free and open educational
`environment, we sought to resolve bandwidth issues in the most
`positive manner. That is to say, we decided not to block access to
`certain applications, but merely control the amount of bandwidth
`being used to ensure maximum benefit and availability to all.
`
`the procurement of additional bandwidth
`We’ve combined
`capability with policies and guidelines for network usage, along
`with the use of various network monitoring and control devices in
`an attempt to develop the bridge between what is available and what
`is desired for bandwidth.
`
`
`Permission to make digital or hard copies of all or part of this work for
`personal or classroom use is granted without fee provided that copies
`are not made or distributed for profit or commercial advantage and that
`copies bear this notice and the full citation on the first page. To copy
`otherwise, republish, to post on servers, or to redistribute to lists
`requires prior specific permission and/or a fee.
`SIGUCCS ’01, October 17-20, 2001, Portland, Oregon, USA.
`Copyright 2001 ACM 1-58113-382-0/01/0010…$5.00.
`
`
`2. BACKGROUND
`For some time, we had been functioning sufficiently with two T-1s
`for Internet access. (A T-1 is a data communications link that
`provides a transfer rate of 1.544 Mbps.) During the 1999-2000
`academic year, we noticed rising requirements for bandwidth and,
`accordingly, ordered an additional T-1, bring our total bandwidth
`available to approximately 4.5 Mbps. The third T-1 became
`operational in May 2000. Since the summer months are times of
`decreased network activity (we have no summer classes), it wasn’t
`until the start of the 2000-2001 academic year that we noticed a
`further extraordinary rise in bandwidth usage.
`Figure 1 shows a graph from our Multi-Router Traffic Grapher
`(MRTG) data. (The MRTG is a tool to monitor the traffic load on
`network links. MRTG generates HTML pages containing images
`that provide a visual representation of this traffic. [1]) Figure 1 (left
`side depicts most recent month) shows the decreased activity during
`the summer months (June-August) after our procurement of the
`additional T-1 in mid-May 2000, followed by a rapid rise in
`bandwidth usage at the start of the Fall semester in September 2000.
`In November, we realized an almost maximum use of bandwidth
`(close to the total 4.5 Mbps). In Figure 1, the solid block represents
`inbound traffic; the solid line indicates outbound traffic. We
`attributed the rise, especially in November, with the increased
`number of computers acting as servers (mainly sharing music or
`mp3 files).
`
`
`
`Figure 1. MRTG data
`One of our initial strategies to alleviate insufficient bandwidth was
`to move the Residential Network, Resnet, to its own T-1 in order to
`keep its traffic from impacting the entire network. This allowed for
`greater access throughout the rest of the campus network, but
`proved to be greatly insufficient for Resnet users.
`At this time, we urged clients to be cognizant of the issues with
`bandwidth availability and asked that they be diligent and proactive
`in helping to control bandwidth usage. We requested they:
`• turn off servers in residence halls when not in use,
`• close web sites that continuously updated when not in use
`(such as weather.com and radio stations),
`• close Internet mail clients, such as yahoo and hotmail, when
`not in use.
`
`115
`
`Cloudflare - Exhibit 1090, page 115
`
`

`

`Our bandwidth issues continued. Therefore, in the beginning of the
`2000-01 academic year, we quickly began to discuss acquiring
`additional bandwidth capability. We decided upon discontinuing
`the three T-1s and acquiring a fractional DS-3 (Digital Signal Level
`3), which equated to approximately 12 Mbps of data transmission
`rate. (Procurement of a DS-3 allows us the room for future
`expansion, up to the full DS-3, or approximately 45 Mbps.
`Additional cost, as well as bandwidth requirements, will be strong
`factors in determining future expansion.) The installation of the
`DS-3 took longer than anticipated, due mainly to cabling issues.
`This forced us to focus for a time on other measures to enhance
`bandwidth availability.
`2.1 Web Caching
`When we began to realize the need for bandwidth exceeded our
`capabilities, we also looked at the use of web caching devices as a
`means to alleviate problems. Web caching reduces traffic by storing
`frequently-used web pages locally. Since the page is stored locally
`at the user’s site, the next user desiring access receives the cached
`page and thus does not have to use the bandwidth required to reach
`the actual server for that web page. After testing this capability with
`three different products, we concluded that, while they did
`significantly decrease the amount of traffic, other problems were
`generated that could not be easily resolved. Thus, after several
`months, we postponed further research in this arena.
`2.2 Bandwidth-Shaping
`In conjunction with our web caching research, we also began to
`look into the use of bandwidth-shaping tools. At this time, our
`clients, especially students, were clamoring for more bandwidth
`availability. As the students began using more bandwidth-intensive
`applications, such as music-sharing applications, we saw continual
`limits to our bandwidth availability. Daytime academic and
`administrative functions began to be negatively impacted. Many
`clients reported extreme slowness or a complete inability to access
`web sites throughout the day.
`After initial investigations, we quickly saw the opportunities
`presented by use of a bandwidth-shaper and, after thorough
`investigation, decided to proceed with the procurement of one such
`tool, Packeteer, Inc.’s PacketShaper®. PacketShaper®
`is a
`bandwidth-management
`tool
`that “…discovers and classifies
`applications, analyzes their performance, enforces policy-based
`bandwidth allocation, and generates reports on the results.” [2].
`Thus, while still progressing
`towards acquiring additional
`bandwidth, we pursued the parallel path of exploring the capabilities
`of a bandwidth-shaping tool.
`3. PACKETSHAPER®
`We experimented a great deal with the Packeteer product in order to
`reach
`the best solution for Oberlin College. This product
`automatically detects many different types of traffic, such as http,
`ftp, smtp, Napster, Real Audio, Doom, Quake, AppleTalk and LPR.
`Traffic can be classified in many different ways, including by
`application, port number, and IP or MAC address.
`We set up PacketShaper® in the following manner. First, we
`prioritized traffic based on time of day. Prior to getting the DS-3 in
`place, we used the following plan: From 7 A.M. to 5 P.M., we gave
`priority
`to
`the
`traffic
`to and from campus academic and
`administrative IP addresses, with Resnet receiving a maximum of
`1.5 Mbps during this time; from 5 P.M. to 7 A.M., we gave priority
`to Resnet traffic, up to a maximum of 3 Mbps.
`
`Next, we looked at how to prioritize the types of traffic. We decided
`to set priorities from 0 to 7, with 1 being the lowest priority we
`would use, meaning traffic of that type would have access to
`bandwidth when no other traffic type desired it. The scheme we
`developed is depicted in Table 1.
`Table 1. Prioritization Scheme
`PRIORITY TYPE OF TRAFFIC
`0
`Unused (would receive no bandwidth)
`1
`Unknown ports; excessive bandwidth users;
`various streaming media (i.e., Napster, Gnutella)
`and Internet games (i.e., Quake, Doom)
`Various streaming media (i.e., Real Audio,
`QuickTime); miscellaneous other types
`Resnet ftp and http
`Campus ftp
`Campus http
`Smtp
`Reserved (only used by network administrator as
`required)
`
`2
`
`3
`4
`5
`6
`7
`
`
`As shown in the table, applications that could not be automatically
`detected by the Packeteer product, and thus classified as unknown,
`were given a low priority. Additionally, clients found to be
`consuming an inordinate amount of bandwidth were also placed, by
`IP address and offending application type or port number, in the
`lowest priority. We found these clients to be mainly students on
`Resnet whose computers were functioning as servers. Since all
`residence halls were configured with a 10 Mbps shared capability,
`users whose computers acquired large amounts of available
`bandwidth could monopolize the entire bandwidth available to a
`residence hall unless we intervened. PacketShaper® allowed us to
`graphically view these problem areas in real-time and limit the total
`amount of bandwidth these clients could acquire at any given time
`by changing their priority.
`With the use of PacketShaper®, we began to notice significant
`benefits almost immediately. Figure 2 shows MRTG data through
`April 2001 (left side of graph). In February 2001, we had much of
`the key constructs of the Packeteer product in place. Notice that the
`solid block and line are almost equivalent at this point in time,
`showing that we were able to better control incoming and outgoing
`traffic. In fact, we were able to identify and reduce the number of
`computers accounting for excessive bandwidth use, mainly music-
`sharing servers, which thereby decreased the requests for service
`from incoming traffic (solid block). We found that many clients
`who had used music-sharing applications were unaware that their
`computers were then acting as servers.
`
`
`
`Figure 2. MRTG data showing PacketShaper® in place
`Using PacketShaper®, along with our other network monitoring
`and control tools, such as our Smart Switch Router and our Sniffer
`product [3], we have been able to identify problem areas as they
`arise and take action quickly to restore available bandwidth to
`viable levels. The only clients negatively impacted are those whose
`
`116
`
`Cloudflare - Exhibit 1090, page 116
`
`

`

`IP addresses end up with lowered prioritization. These users are
`contacted by our Residential Computer Coordinators (student
`employees who resolve computer problems in the residence halls),
`who assist the user in determining the reason for acquiring the
`unusually large amount of bandwidth, if unknown, and ensure that
`the issue gets resolved. Only then is a user removed from the lowest
`prioritization level.
`We also discovered that, with PacketShaper®, we can watch for
`“Top Talkers”. These are depicted as the IP addresses and DNS host
`names for those clients whose computers acquire the highest
`amount of bandwidth over a given time. The data is presented via a
`web page. Table 2 provides a sample.
`Table 2. Top Talkers
` (Note: IPs shown are not the actual addresses)
`No. DNS Name
`IP Address
`Usage Classify
`1 DHCPP1070
`132.162.000.07
`21%
`▲
`2 DHCPP7971
`132.162.000.10
`9%
`▲
`3 DHCPP6728
`132.162.000.05
`7%
`▲
`4 DHCPP1059
`132.162.000.03
`7%
`▲
`5 DHCPP8289
`132.162.000.01
`3%
`▲
`6 DHCPP6993
`132.162.000.08
`2%
`▲
`7 DHCPP5357
`132.162.000.09
`2%
`▲
`8 DHCPP8866
`132.162.000.04
`2%
`▲
`9 DHCPP7782
`132.162.000.06
`1%
`▲
`10 DHCPP4545
`132.162.000.02
`1%
`▲
`
`
`The last column in the table allows us, by clicking on ▲, to see
`what type of traffic is causing the problem from that particular IP
`address. Thus, while Top Talker number 1 is using 21% of the total
`bandwidth, this may just be Napster traffic to/from this computer,
`not http or ftp. Since we expressly do not want to hinder any client’s
`ability to perform academic work, we therefore would use
`PacketShaper® in this example to set a lower priority for Napster
`traffic from this user, but not other types of traffic.
`4. CLIENT ISSUES
`Several clients, particularly Resnet users, began to determine
`“work-arounds” for what they perceived to be the problem – our
`attempt to restrict their bandwidth usage. One client acquired IP
`addresses of non-Resnet computer systems and modified the IP
`address of his computers to use these instead. Student IP addresses
`normally fall within a specific range, designated by registration via
`DHCP (Dynamic Host Configuration Protocol). With non-Resnet IP
`addresses, some students determined they could work around the
`daytime restriction for Resnet traffic.
`These IP addresses then usually showed up in the PacketShaper®
`reports as being excessive bandwidth users. An investigation into
`their use allowed us to pinpoint the actual client system and resolve
`the problem.
`The other main issue with clients concerned the ability of any one
`Resnet user to acquire maximum bandwidth and retain control of
`that bandwidth, effectively blocking access by other users within
`that residence hall. Even after we procured the fractional DS-3, one
`Resnet client was found to be using 8 Mbps, out of a total of 9
`Mbps, for several hours during one evening. This we found to be a
`recurring situation, which we resolved by determining the IP
`
`address of said user’s computer and placing it at a lower
`prioritization level.
`Figure 3 depicts an example of a Resnet client who had acquired an
`excessive amount of bandwidth. This client effectively had blocked
`other users in his residence hall from gaining access to the Internet.
`His computer was receiving a great deal of incoming traffic, mainly
`caused by functioning as a server for music files. This graphic was
`viewed using our Sniffer product. Pinpointing the IP address with
`Sniffer allowed us to then put the offending IP address in a lower
`priority using our PacketShaper®.
`
`
`
`Figure 3. Sniffer graphic showing excessive inbound traffic
`(Note: IP of Resnet client is not the actual address)
`
`5. FRACTIONAL DS-3
`At the same time that we investigated the possibilities with the
`Packeteer bandwidth-shaping tool, we proceeded with our plans to
`procure a
`fractional DS-3. As noted above, even with
`PacketShaper®, we experienced some users being able to establish
`control over large amounts of bandwidth until we intervened.
`By April 2001, we had the fractional DS-3, providing 12 Mbps
`capability, in place. Figure 4 shows the immediate rise is bandwidth
`usage given this increased capability, but maximum usage still
`remained well below the total available capacity. The graph in
`Figure 4 uses Cricket, which is a program that was “…developed to
`help network managers visualize and understand the traffic on their
`networks” and which displays the data graphically via a web
`interface. [4]
`
`Figure 4. Cricket graph depicting DS-3 capacity
`Thus, with both the bandwidth-shaping tool and the fractional DS-3
`in place, our ability to provide continuous sufficient bandwidth to
`the majority of users was realized.
`
`
`
`117
`
`Cloudflare - Exhibit 1090, page 117
`
`

`

`At this point, it should be noted that the ability to provide sufficient
`bandwidth necessitates continuous monitoring and control. The
`combination of bandwidth capability and bandwidth- shaping has
`proven to be vital for our campus. Also, in order to maintain the
`best possible capability, we have addressed corresponding issues
`with policy statements and guidelines for network usage.
`6. POLICIES
`In conjunction with use of our bandwidth-shaper and procurement
`of additional capacity, we reviewed our acceptable use policies. As
`stated earlier, Oberlin College greatly values a free and open
`academic environment. Thus, we concluded that we did not want to
`rewrite our existing broad policy to include a lot of statements of
`what was not allowed.
`We did start by thinking that perhaps we should disallow all servers
`in residence halls, or disallow servers that the owner did not have
`the express written permission of the Center for Information
`Technology (CIT). This led us into a discussion of how we would
`administer this process. We concluded that, in most cases, it was not
`necessary to disallow servers. In fact, we decided to allow any
`server (with the exception of DNS, DHCP and BOOTP servers)
`unless it became a problem. Instead, we would follow the constructs
`established within our Packeteer product to manage our bandwidth
`and limit the ability of any one computer (server or not) to gain an
`excessive amount of the total available bandwidth.
`We did include a few new or modified statements in our updated
`“Policy for the Acceptable Use of Information Technology
`Resources” [5] in order to provide maximum benefit to the majority
`of clients. These specifically include the following:
`• Users should not interfere with, interrupt, or obstruct the ability
`of others to use the network or other CIT resources.
`• Only computers that have been registered through DHCP may
`be connected to Resnet, unless otherwise authorized and
`established by CIT. Users must not attempt to circumvent this
`process.
`Excessive or improper use of network resources that inhibits or
`interferes with use by others is prohibited and will be cause for
`action by CIT, which may include restricting, limiting, or
`disabling network access.
`• Users who connect computers to the network that act as
`servers have the additional responsibility to respond to any use
`of their server that is found to be in violation of this Policy.
`In no case shall the following types of servers be connected to
`the network: DNS, DHCP, BOOTP, or any other server that
`manages network addresses.
`Our policies, therefore, are intended to allow users to perform the
`functions they desire while ensuring that other users have the ability
`to use all Information Technology resources, notably access to the
`Internet.
`
`7. CURRENT ENVIRONMENT
`We have had the fractional DS-3 in place since April 2001, giving
`us a total of 12 Mbps bandwidth capacity. While total usage has
`gone up, we have not yet reached the limits.
`
`•
`
`•
`
`Our configuration of PacketShaper® has remained basically as it
`was prior to acquisition of the DS-3. Thus, Table 1, above, remains
`an accurate representation of our prioritization scheme.
`Additionally, the prioritization based on time of day has remained
`the same, except the quantities, due to the additional bandwidth
`capability, have changed. We now support the following: From 7
`A.M. to 5 P.M., campus academic and administrative IP addresses
`receive priority access, with Resnet receiving a maximum of
`6Mbps; from 5 P.M. to 7 A.M., Resnet IP addresses receive priority
`access, up to a maximum of 9 Mbps.
`The Packeteer product
`is emplaced within our network
`configuration as depicted in Figure 4 below. In this way, it can best
`be used to monitor and control traffic between the router to the
`Internet and our campus Smart Switch Router.
`
`
`Campus
`LAN
`
`PacketShaper
`
`DS-3
`
`WAN
`
`Router
`
`Smart
`Switch
`Router
`Figure 5. Campus network environment
`
`
`Additionally, during the summer of 2001, all buildings that were not
`previously so configured, including residence halls, were upgraded
`from a shared 10 Mbps to a switched 10/100 Mbps capability. Thus,
`instead of sharing 10 Mbps per Ethernet segment, each port has up
`to 100 Mbps dedicated capability. Once the 2001-2002 academic
`year begins, we expect this capability will greatly reduce the
`occurrences of one client acquiring all the bandwidth available for
`his/her residence hall, effectively locking out all other users in that
`location.
`We have also now removed the temptation for Resnet users to
`change their DHCP-registered IP addresses to non-Resnet IP
`addresses
`in an attempt
`to circumvent our PacketShaper®
`prioritization schemes. We have been able to use the capabilities of
`our 802.1Q switch control program to prevent any non-Resnet IP
`address from accessing the campus network from within Resnet.
`Further, we use the Smart Switch Router to prevent any non-
`registered Resnet user from accessing the Internet.
`
`8. CONCLUSION
`We remain committed to bridging any gap between the available
`bandwidth and the bandwidth required or desired by our clients. As
`we are confronted with more and more applications of the
`streaming-media type, including audio and video, we expect to see
`demand for bandwidth increase and our limits once again pushed to
`the maximum. We have been very pleased with, and encouraged by,
`the capabilities provided by our bandwidth-shaping
`tool,
`PacketShaper®. Use of this product helped put control back in the
`situation so that we in CIT no longer feel helpless where bandwidth
`usage is concerned.
`PacketShaper®, combined with our more succinct policies which
`specifically focus on ensuring bandwidth availability for all, should
`help us as we try to maintain that bandwidth bridge well into the
`future.
`
`118
`
`Cloudflare - Exhibit 1090, page 118
`
`

`

`9. REFERENCES
`[1] Multi Router Traffic Grapher.
`http://people.ee.ethz.ch/~oetiker/webtools/mrtg
`[2] Packeteer, Inc. “Four Steps to Application Performance Across
`the Network; A technical Product Overview for
`PacketShaper®”. Packeteer, Inc., Cupertino, CA, 2001
`http://www.packeteer.com/products/packetshaper/
`[3] Sniffer Technologies http://www.sniffer.com
`[4] Cricket home page http://cricket.sourceforge.net
`[5] Center for Information Technology, Oberlin College, “Policy
`for the Acceptable Use of Information Technology
`Resources”. http://www.oberlin.edu/cit/policies
`
`119
`
`Cloudflare - Exhibit 1090, page 119
`
`