Gigabytels Data Communications with the POLO Parallel Optical Link
`
`Kenneth H. Hahn, Kirk S. Giboney, Robert E. Wilson, Joseph Straznicky, Eric G. Wong,
`Michael R. Tan, Ronald T. Kaneshiro, David W. Dolfi, Erwin H. Mueller, * Alan E. Plotts, * Dale D. Murray, * Joseph E.
`**
`Marchegiano, Bruce L. Booth, ** Barton J. Sano, ' Bindu Madhaven' Barath Raghavan,' Anthony F.J. Levi'
`Hewlett-Packard Laboratories
`3500 Deer Creek Rd.
`Palo Alto, CA 94303
`
`1. Abstract
`The progress in the development of the 10 channel
`is
`POLO (Parallel Optical Link Organization) module
`described. The POLO program is a consortium of Hewlett-
`Packard, AMP, Du Pont, SDL, and the University of
`Southern California to develop low cost, high performance
`parallel optical data links for computer clusters, multimedia,
`and switching systems. The design and initial performance
`of the 1st generation POLO module (POLO-1) have been
`previously reported [ 11. In this paper, we discuss the overall
`results of the POLO-1 module as well as the design and
`implementation of the 2nd generation (POLO-2) parallel
`optical data link.
`
`2. Introduction
`Demand for interconnect bandwidth has continued
`to increase in computing and switching systems. Evolving
`communications standards such as ATM, Fiber Channel, and
`SCI require serial data rates approaching and often exceeding
`1 Gb/s. High performance processors today have clock
`speeds of 300 MHz. As clock speeds and bus widths
`continue to increase, aggregate internal bandwidths of high
`performance processors will be in the multi-Gbyte/s range.
`As a result, the performance of computer and
`communications networks are increasingly limited by the
`bandwidth-length and bandwidth-density product limitations
`of electrical interconnects. For example, in the telephone
`central office environment, electrical interconnects between
`high capacity switching systems are creating a serious
`bottleneck in terms of the sheer bulk of the cable required,
`the limited backplane real estate available for connections,
`and
`the resultant EM1 created by large electrical cable
`bundles [2]. Optical fibers in ribbon form have much higher
`density as well as lower attenuation and skew than electrical
`cables.
`
`Given the constraints of electrical interconnections,
`optical interconnect solutions at Gbyte/s data rates and
`distances greater than several meters will be commercially
`competitive.
`Parallel optical links also offer several
`advantages over serial optical links. The input and output
`data is inherently in parallel format, which reduces latency of
`mux/demux functions and simplifies system integration. A
`much smaller footprint is possible than with multiple serial
`links. Parallel optical links also amortize packaging costs
`
`over multiple channels, reducing the overall module cost per
`channel in comparison with serial optical links.
`
`3. 1st Generation POLO Module Results (POLO-1)
`Figure 1 shows a schematic of the POLO-1 module.
`The key components integrated into the package have been
`extensively described previously, including vertical cavity
`surface emitting lasers (VCSELs) [3] and PolyguideTM
`polymer optical waveguides [4].
`
`Polyguide
`
`Ceramic Package
`
`Connector housing w
`
`Figure 1. Schematic of POLO-1 module
`
`Transceiver Electronics Interface
`Figure 2 shows the design of the optical-electrical
`interface. The VCSELAnGaAs PIN detector arrays are
`packaged in a 122 pin ceramic package with the transceiver
`ICs.
`Polyguide waveguides couple light between the
`VCSEL/PIN detector arrays and ribbon fiber using 45" out-
`of-plane mirrors and fiber-to-waveguide connectors. The
`ceramic package features impedance controlled traces and
`integrated resistors for termination of input ECL signals.
`The use of 45" optical interface allows the VCSELs and PIN
`detectors to be packaged in close proximity to the transceiver
`ICs, allowing control of electrical parasitics and GHz
`bandwidth operation.
`Because
`the waveguides are
`multimode, simultaneous alignment of 10 channels to
`VCSEL and PIN detector arrays is possible with loose
`alignment tolerances.
`
`0-7803-3286-5196 $4.00 81996 IEEE
`
`301
`
`1996 Electronic Components and Technology Conference
`
`Authorized licensed use limited to: Riva Laughlin. Downloaded on May 01,2023 at 18:04:16 UTC from IEEE Xplore. Restrictions apply.
`
`Ex. 1024
`CISCO SYSTEMS, INC. / Page 1 of 7
`
`

`

`Polyguide
`
`Ele
`Interface
`
`MCM Substrate
`
`Figure 2.Optical-electrical interface
`
`fabricated with
`ICs
`receiver
`Transmitter and
`Hewlett-Packard’s HP25 bipolar process are used in the
`POLO module. The transmitter IC contains 10 laser drivers
`that use common reference voltages to set the VCSEL pre-
`bias and modulation currents. Several versions of the
`receiver IC are used, including arrays of latched digital
`receivers,
`unlatched
`digital
`receivers,
`and
`analog
`transimpedance amplifiers for linear testing. The latched
`receiver uses 9 data and 1 clock channel, where the data is
`synchronized to the clock at the receiver output. All
`receivers are dc-coupled and do not require encoded data for
`operation. Figure 3 shows one of the 10 channel receivers.
`
`23.3960 ns
`20.8960 ns
`18.3960 ns
`Figure 4. Eye pattern of 980 nm VCSEL at 622 Mb/s
`
`An attractive feature of VCSELs is their ability to
`scale to higher data rates. Modulation of greater than 3 Gb/s
`per channel has been successfully demonstrated. Figure 5
`shows the frequency response of a 980 nm VCSEL at two
`bias currents, showing a small signal 3 dB electrical
`frequency response of 6.5 GHz at the larger bias.
`
`Figure 3. 10 channel receiver IC
`
`Vertical Cavity Suface Emitting Lasers
`Discrete 980 nm bottom
`emitting VCSELs
`operating in multiple transverse modes are used in the
`POLO-1 module. We have previously shown that such large
`area VCSELs emit in multiple transverse modes, leading to
`reduced coherence [SI. This reduces the susceptibility of the
`multimode fiber link to modal noise, making these sources
`ideal for such applications. The threshold currents of the 20
`um diameter VCSELs are 3 - 4 mA. The lasers are typically
`pre-biased near threshold to guarantee a high extinction ratio
`for all channels, and modulated to peak output power of - 2
`intensity noise and reflection
`mW. The low relative
`in
`sensitivity of the VCSELs allows Gb/s data rates
`multimode fiber links with low BER. More recently, we
`have characterized top emitting VCSELs at 850 nm for use in
`the POLO module.
`Figure 4 shows an eye diagram of a 980 VCSEL
`biased below threshold and driven with a PRBS sequence at
`622 Mblsec. The eye is open, and the BER is < 1 O-I3.
`
`Frequency (GHz)
`
`0
`
`Figure 5. Frequency response of 980 nm VCSEL at two
`bias currents
`
`Polymer Waveguides and Ribbon Fiber Connector
`the
`The use of polymer waveguides allows
`waveguide design
`to be easily
`tailored
`to
`system
`requirements, including waveguide dimensions, pitch, and
`numerical aperture. For example, the waveguide pitch is 360
`pm at the PIN detector interface and 500 pm at the VCSEL
`interface, but a smooth taper allows a waveguide pitch of 250
`pm at the ribbon fiber interface. The width and numerical
`aperture of the polymer waveguide are optimized to increase
`coupling efficiencies and optical alignment tolerances at each
`interface.
`The Polyguide waveguides are assembled with an
`MT-style ferrule and aligned to the VCSEL and PIN detector
`arrays on the ceramic package. To test the waveguide-ribbon
`fiber interface, the POLO-1 module uses an optical connector
`that does not incorporate the full push/pull latch mechanism.
`
`302
`
`1996 Electronic Components and Technology Conference
`
`Authorized licensed use limited to: Riva Laughlin. Downloaded on May 01,2023 at 18:04:16 UTC from IEEE Xplore. Restrictions apply.
`
`Ex. 1024
`CISCO SYSTEMS, INC. / Page 2 of 7
`
`

`

`Figure 6 shows the waveguide losses, including coupling,
`propagation, and mirror losses, of a single Polyguide circuit.
`The total optical
`loss between
`the VCSELs and PIN
`detectors, including connector and coupling losses, is < 6 dB.
`Figure 7 shows a Polyguide waveguide circuit before
`assembly with the MT-style ferrule.
`
`Module Performance Characterization
`Figure 8 shows the assembled POLO-1 module on
`an evaluation board. The laser driver and receiver ICs are
`mounted on the ceramic substrate and wirebonded. After the
`VCSELs and PIN detectors are die-attached and wirebonded,
`Polyguide waveguides are aligned and attached for optical
`interface to ribbon fiber.
`
`" 1
`
`' 2 ' 3 ' 4 ' 5 ' 6 ' 7 ' 8 ' 9 '10
`Channel
`Figure 6. Loss of 3 cm waveguide (including mirror,
`coupling, and propagation losses)
`
`Figure 7. 10 channel Polyguide polymer waveguide
`
`Figure 8. Assembled POLO module on board
`
`The module is then mounted on an evaluation board
`for characterization. Because electrical interface to the
`POLO module is differential ECL, 40 SMA connections are
`required to operate all transmitter and receiver channels of a
`module simultaneously. Supply voltages of -5 and -3 volts
`are required for transmitter and receiver operation. An
`is also required
`additional -2 volt supply
`for ECL
`termination. Figure 9 shows the POLO-I module on the
`evaluation board.
`
`Figure 9. POLO-1 module on evaluation board
`
`303
`
`1996 Electronic Components and Technology Conference
`
`Authorized licensed use limited to: Riva Laughlin. Downloaded on May 01,2023 at 18:04:16 UTC from IEEE Xplore. Restrictions apply.
`
`Ex. 1024
`CISCO SYSTEMS, INC. / Page 3 of 7
`
`

`

`Table 1 summarizes the measured performance of
`the POLO-1 module. The use of low skew ribbon fiber
`[6] allows
`with < 1 ps/m channel-to-channel skew
`maximum interconnect lengths of up to 300 m with
`
`synchronous operation. Although the temperature range of
`operation
`has not been
`rigorously
`characterized,
`preliminary measurements have been encouraging.
`
`I Data rate per channel
`
`I
`
`I 0-622Mbls
`
`Differential ECL, latched or unlatched
`................................................................................................................................................................................................................................................................Q(cid:1)
`Ceramic leadframe
`M C M package
`4 cm
`Module width
`-~
`.........................................................................
`" ..............................................................................................................................................
`980 nm
`Wavelength
`Disconnectable MT housing
`Connector .........
`.......
`" ...............................................
`" .................................................................................................
`.- ~" .....................
`Optical interface
`62.5/125 graded index ribbon fiber
`1 < 2 W or 100 mwlchannel
`Power dissipation
`
`.............
`
`1.1""
`
`-.-""-.I-."
`
`...... "" ..... ...................................
`
`............................................
`
`i
`
`" ....... "
`
`"" ....... i
`
`3
`
`" .............................................
`
`.......................
`
`.......
`
`"
`
`I
`
`I
`
`To test BER with worst-case crosstalk conditions, all
`10 Tx and Rx channels of one module are operated in
`loopback mode, where the transmitter and receiver of one
`module are connected by a single ribbon fiber. A multi-
`channel data generator is used to modulate the 10 transmitter
`channels with independent PRBS streams. Figure 10 shows
`the eye patterns of all 10 channels in simultaneous operation
`at 622 Mbls at receiver output.
`
`..............
`
`.............. . . . . . . . . . . . . . .
`
`. . . . . .
`
`with 400 m of low-skew ribbon fiber. While some pattern
`dependent jitter is observed, the eyes are clearly open at 622
`Mbis. The rise and fall times are < 500 ps, and channel-to-
`channel skew (excluding ribbon fiber skew) is < 100 ps. The
`is typically > 1 ns. Figure 11
`phase margin for BER <
`shows 10 simultaneous output eye patterns of the module on a
`single oscilloscope trace. The observed accumulated jitter
`across all 10 channels is - 500 ps.
`The specified maximum data rate is 622 Mbls per
`channel; however, operation at data rates up to 1 Gbls has
`been demonstrated with reduced eye margins.
`
`Figure 10. Output eye patterns of ~ n ~ a t c h e d module at 622
`Mbls per channel
`
`The BER for each channel was < 10+", and an
`extended measurement of one channel resulted in BER < 1 O-I4
`
`Figure 11. Output eye patterns accumulated for 10
`channels at 622 MDIS per channel
`
`Similar results have been obtained with the latched
`version of the POLO-1 module. A 622 MHz clock signal
`synchronizes the 9 output data channels to eliminate any
`accumulated skew at the receiver output. Figure 12 shows the
`output eye patterns of a latched module at 622 Mbls per
`channel.
`
`304
`
`1996 Electronic Components and Technology Conference
`
`Authorized licensed use limited to: Riva Laughlin. Downloaded on May 01,2023 at 18:04:16 UTC from IEEE Xplore. Restrictions apply.
`
`Ex. 1024
`CISCO SYSTEMS, INC. / Page 4 of 7
`
`

`

`. . . . .
`
`. . . . .
`
`D
`
`......... .I.. ...
`
`...... .-.- ... -.
`.. ..i
`.... ~..- __. . ,... .-.._ ......
`
`:..
`
`......
`
`.... J .
`
`.
`
`
`
`....
`
`...
`
`.+
`
`Figure 12. Output eye patterns of latched POLO-1
`module at 622 Mb/s per channel
`
`4.2nd Generation POLO Module (POLO-2)
`The second generation of POLO module (POLO-2)
`will incorporate several key modifications, as summarized in
`Table 2. POLO-2 will accommodate both 980 nm bottom
`emitting and 850 nm top emitting VCSELs. At 850 nm,
`monolithic arrays of VCSELs will be used. GaAs MSM or Si
`PIN detectors will be used in the receiver. Differential ECL
`signaling and dc-coupled electrical
`interface will be
`maintained. Two versions of the receiver (with and without
`output latch) will be available.
`The ceramic package footprint will be reduced from
`4 x 4 cm to less than 2.5 x 2.5 cm to allow an assembled
`module width of 1 inch. Since the reduced package footprint
`will limit the number of pins in a standard leadframe package,
`the use of ball grid arrays (BGA) is necessary for electrical
`interface. Standard BGA technology with 50 mil pitch is
`used. Finally, the module will operate at a data rate of 1 Gb/s
`per channel. With use of low skew ribbon fiber cables, it is
`expected that link lengths of up to 300 m can be accomodated
`without skew compensation.
`
`".l.l_" ............ _,.-"-.l*-"""
`..I." ............
`Number of channels
`I Length
`Data rate per channel
`MCM package
`Module width
`Wavelength
`
`I Optical connector
`
`p?-'
`
`-..
`...
`...............................................
`
`~
`
`........ " l..l.ll.." _ .............. ........... "
`" .... " .......
`................................
`1
`i
`n
`d
`
`I 622 Mb/s
`1 Ceramic leadframe
`1 1.6 inch
`1 980nm
`1 Disconnectable MT housing
`
`POLO-2
`"- ..
`....................................................
`1 0 T x a n d lORx
`
`"
`
`"
`
`........
`-
`
`I-."
`
`1 Gb/s
`Ceramic BGA
`1 inch
`850/980 nm
`I Push-pull connector
`
`I
`
`POLO-2 will also feature push-pull ribbon fiber
`connectors from AMP (figure 13). This connector is based
`on the precision molded MT array ferrule housed inside a
`push-pull SC style housing.
`
`Figure 13. Push/pull ribbon fiber connector
`
`The ribbon fiber cable uses 62.51125 pm fiber and
`meets the requirements of GR-001435 Generic Requirements
`for Multi-Jiber Optical Connectors for Type IR Media
`(Ribbonized Fiber enclosed in reinforced jacket). The design
`and construction of the push/pull connector is also in
`accordance with the optical, environmental, and mechanical
`testing requirements of the same Bellcore generic requirement
`specifications.
`The uniformity of the insertion loss across 10
`channels of the module will be kept below 0.6 dB throughout
`the service life, which includes 200 durability mating cycles.
`The optical insertion loss for the interface will be less than 2
`dB at the end of the service life.
`Figure 14 shows the design of the assembled POLO-
`2 module. The module housing will provide a receptacle for
`the push-pull. To prevent the transfer of any mechanical
`loads from the ribbon fiber cable to the internal module
`components, the module housing will mount rigidly to the
`printed circuit board.
`
`305
`
`1996 Electronic Components and Technology Conference
`
`Authorized licensed use limited to: Riva Laughlin. Downloaded on May 01,2023 at 18:04:16 UTC from IEEE Xplore. Restrictions apply.
`
`Ex. 1024
`CISCO SYSTEMS, INC. / Page 5 of 7
`
`

`

`hardware interface with generic bus architectures, such as
`the PCI or other “open” bus standards.
`
`0 A preliminary microarchitecture of the link adapter chip
`data path and controllers has been designed, along with
`specifications for the VLSI library cells needed from the
`media access controller, VCI RAM, and data path. A
`schematic of the link adapter chip is shown in Figure 15.
`
`I
`
`i
`
`.
`
`POLOModula
`
`
`
`I
`
`Figure 15. Schematic of the link adapter chip with
`projected 1 GHz clocking
`
`6. Acknowledgments
`The support of AFWA under contract number MDA
`972-94-3-0038 and the guidance of Dr. Anis Husain from
`ARPA is gratefully acknowledged.
`
`7. References
`Kenneth H. Hahn and David. W. Dolfi, “POLO: A
`gigabyteis parallel optical link,” SPIE Optoelectronic
`Interconnects and Packaging, volume CR62, pp. 393-
`404, 1996.
`Gary J. Grimes, Stephen R. Peck, Byung H. Lee, “User
`perspectives on intrasystem optical interconnection in
`SONETBDH
`transmission
`terminals,” 1992 IEEE
`Global Telecommunications Conference, pp. 20 1-207,
`IEEE, New York, 1992.
`M.R. Tan, K.H. Hahn, Y.M. Houng, and S.Y. Wang,
`“SELs for short distance optical links using multimode
`fibers,” Conference on Lasers and Electro-optics 1995,
`pp. 54-55, Optical Society of America, Washington D.C.,
`1995.
`[4] B.L. Booth, “Polymers for integrated optical waveguides,”
`in Polymers for Lightwave and Integrated Optics (C.P.
`
`Figure 14. Schematic of POLO-2 module
`
`5. Network Interface
`A prototype POLO module with evaluation board
`has been successfully integrated into a Gbis experimental
`workstation network at the University of Southern California
`(USC). The network uses experimental high speed network
`interface boards called Jetstream, which were developed at
`Hewlett-Packard Laboratories, Bristol [7]. One each of these
`into a Hewlett-Packard 700 series
`boards are inserted
`workstation, two of which form the two nodes of the network.
`Eight channels (4 Tx, 4 Rx) of the POLO module, each
`running at 1 Gb/s, were exercised between
`the
`two
`workstations, which were connected via 500 m of low-skew
`fiber ribbon. The POLO module successfully transmitted and
`received multi-Gb/s data packets error-free in this network.
`In addition, a number of network
`tests and
`comparisons have been performed. A maximum sustained
`application-to-application
`throughput of 230 Mb/s was
`measured for this configuration. This is below the theoretical
`maximum throughput of the HP workstation SGC bus, and is
`due to limitations imposed by the Memory and System Bus
`controller within the workstations. A full speed POLO
`network is expected to have at least three orders of magnitude
`greater throughput than Ethernet. In order to demonstrate the
`potential utility of the POLO module, high quality medical
`image data has been successfully
`transmitted over the
`network. In this experiment, an image stream is fed directly
`from the main memory of one workstation via the network to
`the main memory of the second workstation, and from there
`to the graphics frame buffer and display. The result was a
`dramatic increase over a conventional Ethernet network in the
`speed and flexibility of rendering the image.
`In addition to systems results, work has proceeded
`on the next generation (following Jetstream) host interface
`hardware,
`including preliminary
`results on a test die
`containing several critical circuit components. Future plans
`for network interface of the POLO module to the USC
`network include the following developments:
`
`0 A link adapter board, presently under development, will
`replace the Jetstream board functions. This board will
`contain a CMOS link adapter chip which directly
`to external
`interfaces
`to
`the POLO module and
`synchronous FIFO buffers. This will allow the use of the
`
`306
`
`1996 Electronic Components and Technology Conference
`
`Authorized licensed use limited to: Riva Laughlin. Downloaded on May 01,2023 at 18:04:16 UTC from IEEE Xplore. Restrictions apply.
`
`Ex. 1024
`CISCO SYSTEMS, INC. / Page 6 of 7
`
`

`

`I
`
`Wong, ed.), Academic Press, New York, 1993; B.L.
`Booth, “Optical interconnection polymers,” in Polymers
`for Lightwave and Integrated Optics. Technoloay and
`Applications (L.A. Hornak, ed.), Marcel Dekker, New
`York, 1993.
`K.H. Hahn, M.R. Tan, Y.M. Houng, and S.Y. Wang,
`“Large area multi-transverse mode VCSELs for modal
`noise reduction in multimode fibre systems,” Elec. Lett.,
`vol. 29, pp. 1482-1483, August 1993.
`S. Siala, A.P. Kanjamala, R.N. Nottenburg, and A.F.J.
`Levi, “Low skew multimode ribbon fibres for parallel
`optical communication,” Elec. Lett., vol. 30, pp. 1784-
`1786, October
`A. Edwards et al., “User-space protocols deliver high
`performance to applications on a low-cost Gb/s LAN,”
`ACM SIGCOMM, 1994.
`
`* Optical Interconnection Systems, AMP Inc., Harrisburg,
`Pennsylvania 17105-3608
`**
`Central Research and Development Laboratories, E.I.
`Du Pont De NeMours and Company, Wilmington, Delaware
`19880-0357
`Department of Electrical Engineering, University of
`Southern California 90089-1 1 12
`+
`
`307
`
`1996 Electronic Components and Technology Conference
`
`Authorized licensed use limited to: Riva Laughlin. Downloaded on May 01,2023 at 18:04:16 UTC from IEEE Xplore. Restrictions apply.
`
`Ex. 1024
`CISCO SYSTEMS, INC. / Page 7 of 7
`
`