A Telecommunications Buildings/Power Infrastructure
`In A New Era of Public Networking
`
`1-1
`
`Nicholas Osifchin
`International Power Strategies
`USA
`
`Abstract:
`The growth of data communications and the Internet and the on-
`set of convergence following worldwide divestitures and deregu-
`lation spawned a new era in networking and increased the num-
`ber of competitive carriers and service providers. The new era
`network will be a convergent circuit switch and packet Public
`Network (PN) that will be able to transport and process voice,
`video, data and Internet traffic at higher speeds and lower cost
`than today’s PSTN. Co-location mandated by the government to
`enhance competition is deemed to be expensive and restrictive by
`competitive carriers and service providers. This together with
`the exploding growth of the Internet, second lines and data
`communications spawned a new outsourcing industry compris-
`ing carrier hotels or telehousing and Internet Data Centers.
`These businesses render space for telecommunications equip-
`ment in buildings that they refurbish or construct to replicate a
`CO environment. It is impacting power in two ways. It has in-
`creased the demand for higher capacity CO power plants and
`energized the debate over the preferred powering for telecom-
`munications; non-stop DC systems used in telephone offices or
`UPS protected AC systems used in data communications. Non-
`stop operation is key to Quality of Service which is a major con-
`cern as the PSTN evolves to the PN. The trend to outsourcing
`presents ILECs a timely opportunity join to convert a cost center
`into a profit center by consolidating their huge investment in
`COS and POPS into business units that would lease equipment
`space to both incumbents and competitors.
`
`1.0 Introduction
`
`The first Intelec in 1978 addressed the emerging digital,
`optical and wireless systems and the need for energ efficient
`and cost effective power systems and architectures to assure
`the expected high reliability and dependability of the Public
`Switched Telephone Network (PSTN).
`
`The term “telephone“ is giving way to telecommunica-
`tions, which denotes the converged voice, data, video and
`Internet services offered by a growing number of Service Pro-
`viders (SPs). Though over eighty percent of worldwide
`communications is still carried by the PSTN via Incumbent
`Local Exchange Carriers (ILECs), a new era of networking is
`on the horizon.
`
`A new Public Network (PN) is emerging with the growth
`of multimedia, network convergence and the availability of
`virtually unlimited, accessible and affordable bandwidth is a
`result of growing deployment of fiber .optic cable, packet
`
`0-7803-6407-41oQlbl0.00 92OOO IEEE
`
`1
`
`switching, Digital Subscriber Line (DSL) and sophisti-
`cated network operating and management software
`systems. It is deemed to be largely responsible for the
`explosive growth of the Internet and a principle driver
`to the new PN.
`
`The new PN will be a converged seamless circuit
`switch and packet or cell-based network with the po-
`tential to transport and process voice, video and data
`traffic at higher speeds and at lower costs than today’s
`PSTN. Elements of this new PN comprising wireline,
`wireless and coaxial networks are already being de-
`ployed.
`
`Cable Systems are ubiquitous and have high penetra-
`tion rates in developed regions worldwide. They are
`expected to increase their share of Plain Old Telephone
`Service (POTS) and Internet and multimedia traffic as
`this broadband media is upgraded to bi-directional op-
`eration.
`
`Wireless systems, particularly cellular, are being
`deployed at double-digit annual rates and are the media
`of choice in areas where traditional wireline infrastruc-
`tures don’t exist or are inadequate. In Europe and Japan
`cellular phone connections are forecast to outnumber
`wired telephones by 2003.
`
`The number of deployed packet networks is small
`compared to the circuit switch PSTN and Internet users
`still rely on the PSTN for access. However fastest
`growing and most competitive telecommunication
`service providers as shown in Fig 1 use packet net-
`works, aided by advances in Voice Over Internet Proto-
`col (VOIP).
`
`0
`
`Contributors to this growth include:
`0 Enhanced communications services
`Packet Networks
`Digital Subscriber Line (DSL)
`The Internet
`0 Competitive Access Providers (CAP)
`0 Competitive Local Exchange Carriers (CLEC),
`Interexchange Carriers (IXC)
`Internet Service Providers (ISP) .
`0 Outsourcing equipment space needs
`
`0
`
`GTL 1011
`IPR of U.S. Pat. No. 7,529,357
`
`

`

`1-1
`
`avoiding the cost of a high-bandwidth connection
`through a local exchange carrier.
`
`The equipment space is subdivided into secured areas
`called equipment cages that are sized to the clients
`needs as shown in Fig 2 and includes the following
`building subsystems:
`
`0
`
`Unintermptable Power with AC UPS, battery
`backup and generators
`Some facilities provide 48V DC power
`0 Electrical and lightning protection
`HVAC compatible with electronic equipment oper-
`ating specifications
`Raised Floors, risers and ducts for fiber optic and
`copper cable
`Fire detection and suppression systems
`Security to protect tenants against internal and ex-
`ternal intrusion
`
`0
`
`0
`
`Equipment
`Raised Floor
`Cage \ Air Conditioning?
`
`I
`
`Data
`Center
`
`Operations ’ Security
`
`Center
`
`Check Point
`
`Fiber Oitic Cable
`ILEC’
`Equipment Network Vault
`
`Fig 2. Typical Carrier Hotel Equipment Space
`
`Large tenants generally lease floors of space or
`groups of gages. Small tenants rent shelves on equip-
`ment racks, cabinets or equipment cages. The level of
`support is tailored to the customer’s requirements. This
`ranges from providing basic power and building sub-
`systems to equipment operations and maintenance sup-
`port.
`
`As IXCs congregate at a carrier hotel it becomes at-
`tractive to ILECs and CLECs to establish a Point Of
`Presence for connecting with and handing off traffic to
`IXCs and ISPs. Though originally aimed at acquiring
`POTS lines, CLECs now seem more interested in ac-
`quiring ISP traffic; at times willing to forgo POTS-only
`subscribers.
`
`I
`
`25%
`45%
`65%
`Compound annual growth rate 1996-2002
`
`Fig 1 Growth of Service Providers and Carriers
`
`There are presently about 140 million Internet users in
`North America served by 6000 ISPs and there are more than
`400 CLECs. In Europe the number of users is projected to
`grow to about 69 million and the number of ISPs to about
`2000 by 200 1. Internet users worldwide are growing at 72%
`per year.’
`
`In the United States (and happening worldwide), ILECs are
`mandated to accept competitive carriers to co-locate their
`switching and transmission equipment in ILEC’s Central Of-
`fices (CO) and Points Of Presence (POP) for a negotiated fee.
`CO-location occurs in two ways; physical when CLECs in-
`stall, operate and service their equipment in ILEC building
`and virtual when CLECs pay the ILECs to purchase and in-
`stall their equipment in ILEC CO space.
`
`CLECs and ISPs often perceive co-location in ILEC build-
`ings to be expensive and equipment installation criteria too
`stringent as presently practiced. This perception and the ex-
`ponential growth of DSL has given rise to a new industry;
`outsourcing equipment space.
`
`2.0 Alternatives To CO-Location In ILEC Facilities
`
`The new entrepreneurs, termed carrier hotels or telehous-
`ing, refurbish existing buildings or construct new ones that
`replicate Central Offices (CO) and Points of Presence (POP)
`and rent space to competitive providers and carriers. These
`facilities are usually located close to existing ILEC buildings
`to accommodate distance sensitive DSLs and affordable ac-
`cess to fiber optic networks.
`
`2.1 Carrier Hotels
`
`Carrier hotels lease equipment space to IXCs and ISPs that
`buy and resell each other’s services. CO-located IXCs inter-
`connect with one another using premises-based cabling, thus
`
`’ International Engineering Consortium Study
`
`2
`
`

`

`Carrier hotels in the larger cities typically serve upwards of
`thirty service providers in a single building. This has raised
`concerns about security and risk of potentially large financial
`liabilities resulting from breaches of privacy and service out-
`ages. To minimize this risk carriers are installing two POPS in
`each Local Access and Transport Area (LATA).
`
`Global corporations and carriers increasingly are out-
`sourcing their communications, equipment space and power
`needs that involve highly reliable 24x7~365 non-stop mis-
`sion-critical services. This is giving rise to a market for up-
`scale CO-like facilities that provide a Quality of Service
`(QoS) equal to or exceeding the legendary POTS. Interest-
`ingly, a number of CLECs and IXCs are upgrading their fa-
`cilities to attract such outsourcing companies.
`
`These upscale Carrier Hotels have become too expensive
`and restrictive for small to medium size SPs giving rise to
`another alternative, termed telehousing, to fill the needs of
`these providers.
`
`2.2 Telehousing
`
`Unlike carrier hotels, telehousing typically only provides its
`tenants basic co-location facilities and services These include
`protected equipment power, modem pools, voice and data
`switches, transmission equipment, and in-building connec-
`tivity to ILEC and CLEC local loops. However they do not
`allow interconnection between tenants installed equipment. A
`typical equipment space layout is shown below in Fig 3.
`
`Cable Racks
`CAP, CLEC, IPS,
`Equipment
`
`Return Air 4
`
`1-1
`
`The success of this industry is attracting commercial
`realty firms who team with telecommunications con-
`sultants to construct and operate telehousing sites.
`These are usually
`located
`in vacated commercial
`buildings in urban areas that are near telephone COS
`and have ready access to wide area fiber optic net-
`works.
`
`2.3 Internet Data Centers
`
`The trend to outsourcing and the growth of the Inter-
`net into e-commerce has led to the development of
`Internet Data Centers or IDCs. These centers which are
`comparable to large traditional COS are mostly new
`buildings built specifically for housing telecommunica-
`tions equipment. They are constructed to meet or ex-
`ceed the New Equipment Building Standards or NEBS
`for “telephone buildings” and are equipped with pro-
`tected power systems ranging to 40 Megawatts. These
`sites are connected to global arrays of IDCs by very
`broadband fiber optic networks. The site shown in Fig
`4 is one of twenty four sites similar sites in United
`States, Europe and the far east that are connected to a
`global fiber optic network. Most of the equipment space
`in IDCs is occupied by Servers as contrasted to tradi-
`tional COS where switches dominate the space. The
`IDC may be the CO of the new PN.
`
`Power Supply
`Wide Area Fiber
`Distribution & Protection Network Interface
`
`Outsourcer’s
`Server & Data Storage
`
`Rnhed Floor
`Plenum Air
`
`Earthquake Bracing
`Where Required
`
`Fig 3 Typical Telehousing Layout
`
`Telehousing sites own transmission equipment for con-
`necting to ILEC terminating equipment and wide area net-
`work carriers equipment that is located in the same building.
`Since the tenants are near the ILEC access points they are not
`charged the cost of a local loop. Instead they pay a modest
`access fee for connecting to the telehousing transmission
`equipment. The tenants can choose their own carriers because
`the telehousing sites are carrier neutral.
`
`Fig 4 Typical Internet Data Center
`
`Internet’s voracious use of servers and electronic
`storage is enticing manufacturers of servers and proces-
`sors to set up separate divisions or partner with global
`network carriers to build, furnish and operate IDCs.
`
`Major telecom companies worldwide are also pursu-
`ing the outsourcing business and establishing data cen-
`
`3
`
`

`

`1-1
`
`ters to serve the large corporate data communications needs
`and e-commerce; particularly business-to-business commerce.
`
`3.0 Observations
`
`Most telehosting companies claim to satisfy telecommuni-
`cations equipment specifications for power, protection, cool-
`ing and connectivity to local exchanges, and wide area and
`private networks.
`
`Virtually all of the more than 4000 ISPs in the United
`States are co-locating or outsourcing their equipment space to
`an ILEC or a telehosting company. Renting equipment space
`that meets the unique requirements of a growing population of
`service providers is proving to be an attractive opportunity to
`capitalize on the worldwide growth of data networking, the
`Internet and e-commerce.
`
`Carrier hotel and telehousing rental and leasing costs are
`comparable for like services but vary with geographic loca-
`tion and access to local and wide area network connection
`points. Rental fees are higher in metropolitan areas since real
`estate costs are a big part of the cost of operations. Rental
`schedules typically favor larger tenants and range from about
`$US 800 per month for a 200cm high x 60cm wide x 74cm
`deep equipment rack to as low as $US 240 per square meter
`per month for space in modules of 100 square meters or
`larger.
`
`Most of the larger carrier hotels and telehousing companies
`claim that they meet or exceed Telcordia’s New Equipment
`Standards (NEBS) specifications for building subsystems and
`central office power. This includes power supplies and distri-
`bution, generators, UPSs, battery backup, electrical protection
`and protection against man-made and natural disasters.
`
`Not withstanding these claims, implementation to date is
`raising issues and concerns regarding compliance with NEBS
`and Quality of Service (QoS). High-end ISPs demand “non-
`stop” reliable and dependable service, which translates to
`“equivalent CO power and protection against fire, earth-
`quakes, flooding,” etc.
`
`CLECs that have these concerns use carrier hotels and tele-
`housing only as an option for additional space rather than an
`alternative to physical or virtual co-location in an ILEC cen-
`tral office. Some of the larger CLECs are starting to build
`their own NEBS-compliant central office type facilities.
`
`4.0 Telehosting Categories
`
`Businesses worldwide are cutting costs to increase profits
`and enhance shareholder value. They are concentrating their
`capital and labor resources on their core business and
`outsourcing non-core functions or restructuring cost centers.
`Non-communications companies tend to outsource their tele-
`
`4
`
`communications networks or functions. Telecommuni-
`cations service providers, e.g. ISPs, outsource their
`equipment space requirements that include building
`supporting services and telecommunications power.
`
`Assuming that these trends will continue, a signifi-
`cant portion of datacom and Internet service provider’s
`equipment eventually will reside in some form of tele-
`housing or carrier hotel. As this industry matures it will
`undergo consolidation that will result in fewer but
`larger hosting companies that will serve fewer than 400
`ISPs. In North America, the remaining hosting compa-
`nies could reasonably fall into three categories.
`
`4.1 Category I
`
`Large global companies that operate NEBS compli-
`ant CO-like building sites strategically located close to
`major public and private network COS and POPs would
`be connected to redundant wideband optical networks.
`These sites would be equipped with AC and DC power
`with UPS, generators and battery backup and building
`and power subsystem monitoring and management
`systems. Building and power connected to internal as
`well tenants network or enterprise management systems
`would be a key feature of this category. The target
`market for this sector would be large international ISPs
`and long distance operators and global service provid-
`ers. Their heavy mission-critical traffic and the inter-
`national scope of operations place a high priority on
`dependable and reliable non-stop and carrier-neutral
`service.
`
`4.2 Category 2
`
`Companies that provide comparable but less exten-
`sive building facilities and support services than those
`in Category 1 but are also located close to ILEC COS
`and POPs. They serve a more diverse market of me-
`dium to large size CLECs, ISPs, IXCs and POPs. These
`hosting companies would provide equipment for inter-
`connecting tenants within the building and to on-
`premises POPs making it attractive to hand off traffic to
`each other and to the POPs. They would offer a compa-
`rable level of non-stop service as in Category 1 but be-
`cause of the diversity of their customers they would
`offer a range of space environments and services tai-
`lored to the customer’s needs.
`4.3 Category 3
`Companies that mostly serve a diverse group of small
`to medium size ISPs and niche service providers in
`their own buildings as well as in leased space in cus-
`tomer’s office buildings and centers. This category of-
`fers customers more flexibility in rental space that can
`be as small as rack of equipment in a cabinet and op-
`
`

`

`1-1
`diverse, expensive, and often precise equipment in the
`same facility. Compliance to NEBS is becoming a ma-
`jor factor as ISPs move more and more equipment out
`to POPs that are often located in rental facilities.
`
`In the United States the biggest ILECs are the re-
`gional Bell operating companies or RBOCs and they
`have the largest number of NEBS compliant buildings
`and power systems and the expertise to provide the QoS
`demanded by multimedia system and e-commerce op-
`erators. RBOCs are the largest segment of the tele-
`communications industry. As a group they own about
`20,000 equipment buildings and major transmission
`stations comprising over 500 million square meters of
`space.
`ILECs provide end user access and POPs to
`virtually all of the nation’s ISPs. The group uses annu-
`ally about 20 billion kWh of electricity in their facilities
`making them one of the nation’s top purchasers of
`electric power.
`
`Discounting the recent mergers and acquisitions a
`typical RBOC would own about 3500 buildings and
`stations with about 87 million square meters of equip-
`ment space and use more than 3 billion kWh of elec-
`tricity annually (NTT uses about 4.5 billion kWh). An
`RBOC equipment buildings business unit that included
`supporting systems and power would lease space to its
`RBOC operating telecommunication units as well as
`competitors. It would dwarf the existing telehousing
`and carrier hotel industry but would not be a threat to
`category 2 and 3 companies.
`
`Such a business unit would include all CO and
`transmission equipment buildings together with Net-
`work Power and the associated subsystems: HVAC,
`security, fire suppression and building management
`systems that are connected to customer’s network sys-
`tem management systems.
`
`From an accounting viewpoint it would be turning a
`cost center, which is critical to the RBOC mission but
`not a core business into a profit center. This new busi-
`ness might aptly be called a “TeleCommunications
`Power/ Infrastructure Utility” or TCPIIU.
`
`A variant of this proposition already exits in NTT
`Power and Building Facilities Inc., which was formed
`when NTT was restructured. They are pioneering the
`basic notion of telehousing and the concept of Total
`Power Management. Mr. Naoyuki Takehiro of NTT is
`participating in this conference and will discuss this
`project in his presentation.
`
`6.0 Implications For Power
`
`erations support that could include maintenance of the cus-
`tomer’s equipment. It would be particularly attractive to very
`small ISPs and niche SPs and carriers because flexible rental
`arrangements and an on-site LEC presence would save cus-
`tomers the cost of a local loop. Most startups fall into this
`category.
`
`Companies that would fall into the three categories de-
`scribed above already exist though they are aimed at specific
`markets.
`
`For example one company is focused on Internet Service
`Providers and operates twenty two networked Internet Data
`Centers throughout the United States and Europe that com-
`prise 225 thousand square meters of equipment space that
`meets NEBS specifications. It is planning to open additional
`networked centers in China and other areas of Asia.
`
`Another company founded on the assumption that the in-
`cumbents would not give CLECs access to their COS, leases
`out equipment space to regional carriers in its 48VDC pow-
`ered NEBS-ruggedized sites. It is among the fastest growing
`companies.
`
`If this trend continues the presently fragmented hosting
`company buildings and facilities may evolve into a “shadow
`network” that caters to Internet, data an video customers. The
`larger companies are already allied with global optical net-
`work operators that are helping to build the infrastructure for
`the new Public Network.
`
`5.0 Equipment Buildings In The New PN
`
`The occurrence and the form of the new PN may be debat-
`able but the number of competing service providers continues
`to grow as they outsource their equipment space and power
`needs to non-ILEC co-location sites that function as their
`c o s .
`
`ILECs huge investment in equipment building and power
`infrastructures provides the environment that enables ILECs
`to deliver dependable and reliable non-stop service while
`adapting to a succession of diverse switching and transmis-
`sion systems. The total equipment rental space in telehousing
`and carrier hotels very small compared to ILECs space but it
`is growing at a faster rate. This may be a good time for the
`ILECs to appraise their investment from a new business per-
`spective.
`
`Network equipment buildings and power are unique infra-
`structures that are essentially an extension of the network.
`They adhere to Telcordia’s Network Equipment Building
`Standards (NEBS) for fault tolerance that address every pos-
`sible CO hazard from electrical shielding to powering to acts
`of God such as earthquakes and fires. They are designed to
`handle problems associated with housing a large amount of
`
`5
`
`

`

`1-1
`The evolution to the new PN is already impacting power.
`
`The addition of second lines modems, DSLs routers, packet
`switches and collocation equipment needed to provide the
`growing Internet traffic together with the longer duty cycle of
`Internet users can increase CO power up to three fold. The
`deployment of state-of-the-art fiber (electro) optic networks to
`support the growth can result in comparable increases in
`power.
`
`This upsurge in CO power is resulting in a growing market
`for higher capacity (3000 Amps to 10,000 amp) plants that is
`expected to grow at 20% to 25% annually for several years2.
`This is a reversal of a ‘90s trend in CO power away from
`large “ferro” plants and to smaller modular SMR plants.
`
`On the other hand the demand for the SMR plants in remote
`exchange plant and transmission nodes is increasing as in-
`cumbents and competing exchange and long distance carriers
`deploy fiber networks to provide broadband access in the ex-
`change plant.
`
`The building blocks of the new PN are modems, routers,
`hubs, packet switches, servers and DSLs. These network
`components typically are powered by utility AC and backed
`up with AC Unintermptible Power System (UPS) that provide
`up to 8 hours battery reserve. Virtually all of these products
`are now available with a 48 Vdc option and thus compatible
`with the CO environment.
`
`AC UPS power systems have adequately powered data com-
`munications operations that required “computer grade”3 reli-
`ability commensurate with level of risk acceptable to the op-
`erator. However users and operators of the Internet, particu-
`larly e-commerce, are starting to expect the non-stop “telecom
`grade” Quality Of Service.
`
`However most ISPs and equipment hosting companies that
`serve primarily small customers, only provide AC power
`backed up with redundant UPSs, batteries and in some cases
`redundant generators. The companies contend this practice is
`less costly than equivalent telecom DC power and provides
`comparable unavailability with acceptable risk.
`
`Managers of telehousing facilities perceive AC power dis-
`tribution and protection to be more amenable to the telehous-
`ing equipment cage layout and lower cost than telecom type
`of DC power. This is probably more a measure of differences
`between telecom and datacom power engineering cultures and
`a lack of expertise in Central Office DC power systems.
`
`Packaged power systems are widely used in data communi-
`cations.
`These comprise modular and scalable AC power
`’ NJ Power Group lntemal Study Report
`’ Term coined by Jaques Poulin EE Times 5/22/00
`
`6
`
`supplies, battery backup and network monitor and man-
`agement software. These are available in capacities
`suitable for Category 2 or 3 companies as defined
`above.
`
`Companies that fit the Category 1 description offer
`DC and AC power that is often designed to RBOC and
`NEBS specifications and some claim to satisfy the
`99.999% reliability demanded by virtually all carriers.
`Service Providers equivalent terminology is non-stop
`24hrslday, 7dayslweek and 365dayslyear operation or
`“24x7~365”.
`
`Comparative reliability studies Intelec study groups
`and others show that Telecom 48v DC power systems
`are at least 20 times more reliable than AC UPS sys-
`tems for typical datacom applications. However tele-
`hosting and IDC operators are installing both DC and
`AC UPS systems in accordance with their assessment
`of cost, risk and Quality of Service. There is no adverse
`result in mixing AC and DC powered equipment but the
`reliability of the group will be lower than if totally DC
`powered.
`
`There is sufficient evidence that the cost of bandwidth
`and access will reach parity for all carriers and service
`providers (SP) as the new era of public networking
`matures. The customers will then differentiate service
`providers by reliability, dependability and service that
`are significantly impacted by power and protection
`systems.
`
`The AC-DC debate is still alive and AC UPS presently
`power most datacom facilities but DC power is gaining
`preference for mission-critical applications where QoS
`is a critical issue as in the new era Public Network.
`
`7.0 Conclusions
`
`No one is expecting a sweeping change to a new in-
`frastructure that would involve scrapping a half trillion
`dollar investment in the existing PSTN. The new PN
`will evolve as the demand warrants the investment to
`upgrade or replace existing infrastructures.
`
`The Internet has spawned a multitude of carriers,
`service providers and content managers. They are in-
`creasingly outsourcing their equipment space and data
`communications needs to telehosting and Internet Data
`Center companies giving rise to global “shadow net-
`works” such facilities that may become the new central
`offices of the PN.
`
`The impact of the Internet in the short term is an
`increased demand for larger CO power plants to ac-
`
`

`

`1-1
`
`commodate the growth of second lines and the addition of
`new equipment such as modems packet switches and DSLs.
`The continuing trend to move distribute CO functions is in-
`creasing the demand for scalable modular SMR plants that
`have CO levels of unavailability.
`
`The importance of reliable power in the new PN and its
`contribution to the quality of service will assure a continuing
`debate as to whether telecom or AC UPS is the preferred
`power for the PN. Most telehosting sites and IDCs presently
`use UPSs and DC where mandated.
`
`ILECs own the largest inventory of equipment buildings
`that are ideally suited for the PN. They can convert these as-
`sets into a business unit that would rent space to both incum-
`bent and competing carriers. Such a move might also mitigate
`government pressures on ILECs to form separate Internet re-
`lated businesses. A more encompassing variant of this propo-
`sition is already in existence in thew NTT Power and Building
`Facilities Inc.
`
`