680 DEPLOYMENT CONSIDERATIONS Figure 13.8 The future telecommunications network. (a) Network topology showing a meshed long-haul backbone with metro collector rings. (b) Architecture of a typical backbone node showing an OXC, OLT, IP router, SONET add/drop multiplexer, and an MSP. (c) A node on a metro ring served by an MSP. The MSP is used to deliver a variety of services including voice, private lines, and data services. 13.2 Designing the Transmission Layer 681 Like SONET rings, most MSPs are deployed in ring configurations and include built-in restoration capabilities, which are based on SONET mechanisms for the most part. Ring configurations work well for metro networks, as the fiber is mostly laid in rings. Laying fiber in ring configurations is economical, compared to using other configurations, such as a star (also called a hub and spoke) configuration. A star configuration requires two disjoint fiber routes to be laid between each access node and the central office. In contrast, multiple access nodes can be combined on a single fiber ring, and additional nodes can be added to the ring as needed, without having to lay new fiber routes each time a new node needs to be added. Some MSPs also include built-in WDM interfaces with optical add/drop (OADM) capabilities. Passive optical networks (PONs) are also emerging as potential candidates to deliver services to small and medium users of bandwidth. In order to prove in, they need to be cheaper and more flexible than SONET/SDH platforms or MSPs. We studied PONs in Chapter 11. However, as of this writing, WDM is just beginning to be used in metro networks, as its economics are not as compelling as in long-haul networks. More on this in Section 13.2.8. 13.2 Designing the Transmission Layer We will next look at the choices that service providers have to make in choosing the right tranmission layer. The historical trend has been to increase capacity in the network and at the same time drive down the cost per bit of bandwidth. Service providers generally look for at least a fourfold increase in capacity when planning their networks. As a rule of thumb, they expect to get this fourfold increase in capacity at about 2-2.5 times the cost of current equipment. There are fundamentally three ways of increasing transmission capacity. 1. The first approach is to light up additional fibers or to deploy additional fibers as needed. We can think of this as the space division multiplexing (SDM) approach: keep the bit rate the same but use more fibers. 2. The other traditional approach is to increase the transmission bit rate on the fiber. This is the TDM approach. 3. The third approach is to add additional wavelengths over the same fiber. This is the WDM approach. Note that the three techniques are complementary to each other and are all needed in the network for a variety of reasons. For instance, using SDM, particularly when existing fibers are close to being exhausted, can be viewed as a long-term way of 682 DEPLOYMENT CONSIDERATIONS 13.2.1 building up infrastructure; WDM and TDM can be viewed as providing the ability to turn up services rapidly over existing fiber infrastructure. Electronic TDM is required for grooming traffic at lower speeds in the network, where optics is not cost-effective. WDM provides the ability to scale the capacity of the infrastructure in a different dimension. Therefore, the network almost always employs a combination of these techniques in practice. The interesting question is not whether to use SDM or TDM or WDM all of these will be usedmbut to determine the right combination of these. For instance, let us look only at WDM and TDM. To get a total capacity of 80 Gb/s, should we deploy a network with 32 wavelengths at 2.5 Gb/s each, or a network with 8 wavelengths at 10 Gb/s each? This is a complicated question with many parameters affecting the right choice. When should we deploy more fibers, instead of investing in higher-capacity TDM or WDM systems? Several factors influence this decision-making process: 9 Is this a new network build or an upgrade of an existing network? If it is an upgrade, we need to consider the cost of adding channels to existing systems in lieu of deploying new systems. 9 The availability and cost of additional fiber. 9 The type of fiber available. 9 The cost of lighting up a new fiber versus adding additional capacity to an already-lit fiber. 9 The relative cost of TDM and WDM equipment. We will attempt to address some of these questions next. The problems at the end of the chapter also provide a partial insight into some of the issues. Using SDM Using additional fibers is a straightforward upgrade alternative. The viability of this approach depends on a few factors. First, are additional fibers available on the route? If so, then the next consideration is the route length. If the route length is short (typically a few tens of kilometers) and no regenerators or amplifiers are required along the route, then this is a good alternative. However, if amplifiers or regenerators are required, then this becomes an expensive proposition because each fiber requires a separate set of amplifiers or regenerators. However, it may be worth paying the price to light up a new fiber if the new equipment to be deployed over that fiber provides significantly reduced transmission costs compared to existing equipment on the already-lit fiber. If no fibers are available on the route, then we need to look at the cost associated with laying new fiber. This varies widely. If there is space in existing conduits, fiber 13.2 Designing the Transmission Layer 683 can be pulled through relatively inexpensively and quickly. However, if new conduits must be laid, the cost can be very expensive, even over short distances if the route is in a dense metropolitan area. If new conduits are to be laid, then the link can be populated with a large-count fiber cable. Today's fiber bundles come with hundreds of fibers. The other aspect of this problem is the time it takes to lay new fiber. Construct- ing new fiber links takes months to years and requires right-of-way permits from municipalities where the new link is laid. These permits may not be easy to obtain in dense metropolitan areas, due to the widespread impact caused by digging up the streets. In contrast, upgrading an existing fiber link using either TDM or WDM can be done within days to weeks. While it is necessary in some circumstances to lay new fibers, this is not a good mechanism for rapid response to service requests. Note that carriers are not likely to wait until the last fiber is exhausted before they consider an upgrade process. For example, an upgrade process may be triggered when it is time to light up the last few fibers on a route. This might result in installing additional fibers along the router. Alternatively, the carrier may deploy a higher-capacity TDM or WDM system on the last few fibers, and transfer the traffic from the lower-capacity fibers onto the new system deployed to free up existing fibers along the route. 13.2.2 Using TDM Clearly, TDM is required for grooming traffic at the lower bit rates where optics is not cost-effective. The question is to what bit rate should traffic be time division multiplexed before it is transmitted over the fiber (perhaps on a wavelength over the fiber). Today's long-haul links operate mostly at rates of 2.5 Gb/s or 10 Gb/s. We will see in Section 13.2.5 that the choice of bit rate here is dictated primarily by the type of fiber available. Metropolitan interoffice links operate mostly at 2.5 Gb/s, and access links operate at even lower speeds. Here the situation is somewhat more complicated, as we will explore in Section 13.2.8. Electronic TDM technology is already delivering the capability to reach 40 Gb/s transmission rates and may well push this out to 80 Gb/s in the future. Beyond these rates, it is likely that we will need some form of optical TDM. At the higher bit rates, we have to deal with more severe transmission impair- ments over the fiber, specifically chromatic dispersion, polarization-mode dispersion (PMD), and fiber nonlinearities. With standard single-mode fiber, from Figure 5.19, the chromatic dispersion limit is about 60 km at 10 Gb/s and about 1000 km at 2.5 Gb/s, assuming transmission around 1550 nm. With practical transmitters, the distances are even smaller. The 10 Gb/s limit can be further reduced in the presence of self-phase modulation. Beyond these distances, the signal must be electronically 684 DEPLOYMENT CONSIDERATIONS 13.2.3 regenerated, or some form of chromatic dispersion compensation must be employed. Practical 10 Gb/s systems being deployed today commonly use some form of chro- matic dispersion compensation. This is usually cheaper than using regeneration, particularly when combined with WDM. As we saw in Section 5.7.4, the distance limit due to PMD at 10 Gb/s is 16 times less than that at 2.5 Gb/s. On old fiber links, the PMD value can be as high as 2 ps/kvnk~. For this value, assuming a 1 dB penalty requirement, the distance limit calculated from (5.23) is about 25 km at 10 Gb/s. Electronic regeneration or PMD compensation is required for longer distances. The PMD-induced distance limit may be even lower because of additional PMD caused by splices, connectors, and other components along the transmission path. PMD does not pose a problem in newly constructed links where the PMD value can be kept as low as 0.1 ps/k~/-k-m. Finally, nonlinear effects such as self-phase modulation limit the maximum trans- mission power per channel, resulting in a need for closer amplifier spacing, and thus more amplifiers in the link, leading to somewhat higher costs. At 10 Gb/s, transmis- sion powers are usually limited to under 5 dBm per channel. Today 10 Gb/s TDM systems are widely deployed in long-haul networks, mostly in conjunction with WDM, and 40 Gb/s TDM systems will soon become commer- cially available. Using WDM It may be preferable to maintain a modest transmission bit rate, say, 10 Gb/s, and have multiple wavelengths over the fiber, than to go to a higher bit rate and have fewer wavelengths. Keeping the bit rate low makes the system less vulnerable to chromatic dispersion, polarization-mode dispersion, and some types of nonlinearities, such as self-phase modulation. On the other hand, WDM systems are generally not suitable for deployment over dispersion-shifted fiber because of the limitations imposed by four-wave mixing (see Chapter 5). WDM systems can be designed to be transparent systems. This allows different wavelengths to carry data at different bit rates and protocol formats. This can be a major advantage in some cases. Finally, WDM provides great flexibility in building networks. For example, if there is a network node at which most of the traffic is to be passed through and a small fraction is to be dropped and added, it may be more cost-effective to use a WDM optical add/drop element than terminating all the traffic and doing the add/drop in the electrical domain. There has been a relentless push in expanding the capacity of WDM systems over the past few years, as shown in Figure 13.9. We are now seeing systems with over 100 wavelengths becoming available. At the same time, channel spacings are being 13.2 Designing the Transmission Layer 685 10,000 1000 ,-0 100 10 1994 Long-haul 10 Gb/s/AT , . , S~ ~ Long-haul 2.5 Gb/s ~~'~/ ~~ ~'''~~O ~ ~~~-~-" Metro 1996 1998 2000 Year Long-haul Year Capacity 1995 1996 1997 1998 1998 1999 1999 2000 2001 8 • 2.5 Gb/s 16 • 2.5 Gb/s 8 • 10 Gb/s 40 x 2.5 Gb/s 16 x 10 Gb/s 80 x 2.5 Gb/s 32 x 10 Gb/s 80 • 10 Gb/s 160 x 10 Gb/s Ultra-long-haul Year 2000 Capacity 160 x 2.5 Gb/s 56 x 10 Gb/s Metro Year Capacity 1995 1998 2000 2001 20 x 1 Gb/s 24 x 2.5 Gb/s 32 x 2.5 Gb/s 64 x 2.5 Gb/s Figure 13.9 Trends in WDM system capacity for commercially available systems. The capacity indicated is the total capacity (bit rate x number of channels) on a fiber, between regenerators. Bit rates are usually 2.5 Gb/s or 10 Gb/s on each channel. Multiple data points for a given year indicate systems from different vendors we can have more channels at 2.5 Gb/s or less at 10 Gb/s. Metro systems typically have regenerator spacings of about 50-75 km. Long-haul systems have regenerator spacings of about 400-600 km. Ultra-long-haul systems stretch this distance to about 2500-4000 kin. Amplifier spacings in long-haul and ultra-long-haul systems are between 80 and 120 km. 686 DEPLOYMENT CONSIDERATIONS Figure 13.10 (a) Unidirectional and (b) bidirectional transmission systems. reduced, with 50 GHz channel spacings now common and 25 GHz channel spacings being achieved in some systems. With the opening up of the new L-band window in the 1565-1625 nm range, we can expect to see further increases in the number of wavelengths. Today's state-of-the-art long-haul systems carry about 100 channels at 10 Gb/s each and have regenerator spacings of 400 to 600 km. The ultra-long-haul systems expand spacing between regenerators to about 4000 km but have somewhat lower capacities than the long-haul systems. 13.2.4 Unidirectional versus Bidirectional WDM Systems A unidirectional WDM system uses two fibers, one for each direction of traffic, as shown in Figure 13.10(a). A bidirectional system, on the other hand, requires only one fiber and typically uses half the wavelengths for transmitting data in one direction and the other half for transmitting data in the opposite direction on the same fiber. Both types of systems are being deployed and have their pros and cons. We will compare the two types of systems, assuming that technology limits us to having a fixed number of wavelengths, say, W, per fiber in both cases. 1. A unidirectional system is capable of handling W full-duplex channels over two fibers. A bidirectional system handles W/2 full-duplex channels over one fiber. The bidirectional system, therefore, has half the total capacity, but allows a user to build capacity more gradually than a unidirectional system. Thus it may have a slightly lower initial cost. However, to go beyond W/2 channels, the user must buy a second bidirectional system and pay for this additional equipment at that time. 13.2 Designing the Transmission Layer 687 Figure 13.11 Implementing 1 + 1 protected configurations using unidirectional and bidirectional transmission systems: (a) two unidirectional systems using four fibers, (b) two bidirectional systems using two fibers. 2. If only one fiber (not two) is available, then there is no alternative but to deploy bidirectional systems. Implementing 1 + 1 or 1:1 configurations with unidirec- tional WDM systems requires a minimum of two pairs of fibers routed separately, but only requires two fibers with bidirectional systems, as shown in Figure 13.11. Note, however, that as mentioned above, the bidirectional systems provide half as much capacity. 3. Systems using distributed Raman amplification tend to be unidirectional. 4. As we saw in Chapter 10, if optical layer protection is required, unlike unidirectional systems, bidirectional systems do not require an automatic protection-switching (APS) protocol between the two ends of the link, since both ends detect a fiber cut simultaneously. 5. Consider two equivalent all-optical networks in terms of capacity. One network uses a bidirectional link between nodes with a total of W wavelengths per link. Another network uses two unidirectional links between nodes, with a total of W/2 wavelengths on each unidirectional link. Problem 8.10 shows that the bidi- rectional network is less efficient at utilizing the available capacity than the unidirectional network due to inefficiencies in wavelength assignment. 6. Bidirectional systems can potentially be configured to handle asymmetric traffic. Given a total number of wavelengths in the fiber, more wavelengths could be used in one direction compared to the other. While this may be easy to do for unamplified systems, it is more difficult to do in amplified systems because these systems typically use separate amplifiers for each direction. 688 DEPLOYMENT CONSIDERATIONS 7. In general, it is slightly more difficult to design the transmission system in bidirec- tional systems since more impairments must be taken into account, in particular, reflections, as discussed in Section 5.6.4. There are more components in the path, such as filters for separating the wavelengths in different directions, leading to higher losses. However, at high channel counts, even unidirectional systems may require these filters. 8. Although amplifiers for bidirectional systems may employ more complicated structures than unidirectional systems, they need to handle only half as many channels as unidirectional systems, which means that they can produce higher output powers per channel and provide more gain flatness. This of course as- sumes the use of a different amplifier for each direction, which is typically the case. However, for a given total capacity, twice as many amplifiers are required in a bidirectional system compared to a unidirectional system. 9. Bidirectional systems usually require a guard band between the two sets of wave- lengths traveling in opposite directions to avoid crosstalk penalties. However, high-channel-count unidirectional systems may also require guard bands due to the hierarchical nature of the multiplexing and demultiplexing in these systems. (We studied this in Section 3.3.10.) The guard band can be eliminated by in- terleaving the wavelengths in opposite directions, that is, by having adjacent wavelengths travel in opposite directions on the fiber. This also has the added advantage of effectively doubling the channel spacing. For instance, if we trans- mit 100 channels spaced 50 GHz apart over a fiber, then we have 50 channels spaced 100 GHz apart in each direction. 13.2.5 Long-Haul Networks The long-haul carriers in North America have links spanning several hundred to a few thousand kilometers. In Europe the links are somewhat shorter but still several hundred kilometers in length. The economics for deploying WDM on these links is quite compelling, based on the enormous savings in regenerator costs enabled by the use of optical amplifiers, as well as the time to market to deploy new services. Thus most long-haul carriers have deployed WDM extensively in their networks. The specific combination of WDM and TDM depends very much on the carrier's installed base of fiber and the type of services delivered. Among the major established carriers, AT&T and Sprint have primarily installed standard single-mode fiber. Thus WDM is an attractive option for them, and they have actively deployed WDM systems on many of their routes. Most of their links operate at 2.5 Gb/s (OC-48) rather than 10 Gb/s (OC-192). This is because of the older fiber base, with potential PMD problems as well as because of the need for a large amount of chromatic dispersion 13.2 Designing the Transmission Layer 689 13.2.6 compensation on standard single-mode fiber at 10 Gb/s. In addition, these carriers for the most part provide services at relatively low bit rates, such as DS3 (45 Mb/s). The OC-192 terminals initially provided low-speed interfaces down to OC-48 rates but now provide lower-speed interfaces down to OC-3/12 rates. Thus carriers providing DS3 services need to buy additional equipment to multiplex DS3s to OC-12s or OC-48s, which adds to their equipment cost. Another major carrier, Worldcom, has a mix of dispersion-shifted fiber and stan- dard single-mode fiber. This made them an early adopter of 10 Gb/s systems for those routes with dispersion-shifted fiber. At the same time, they have deployed WDM sys- tems on other routes that use standard single-mode fiber. Meanwhile, some of the new links being installed use nonzero-dispersion fiber, which allows both types of systems to be considered for deployment. Over the past few years there have been a number of new carriers building long-haul networks worldwide. In the United States, these include Qwest and Level 3 Communications, among others. These carriers have laid new fiber routes, and many have decided to install nonzero-dispersion fiber or the large effective area fiber (LEAF). In some cases, they have hedged their bets with respect to fiber type by leaving space in the conduits to pull additional fiber through later as needed. These carriers are for the most part delivering bulk bandwidth at OC-12/48/192 rates to their customers. Thus it makes sense for them to deploy WDM at OC-192 rates, and that is what they have done. As we have mentioned earlier, systems operating in the C-band as well as in the L-band are now available. The L-band requires a separate amplifier and is relatively more expensive than the C-band to deploy, due to the higher cost of the L-band ampli- fiers, compared to the C-band amplifiers (this is partially because L-band amplifiers require higher pump powers than their C-band counterparts). While most long-haul carriers have deployed C-band WDM systems, they have been slow to adopt L-band systems. This is because it is usually cheaper to deploy another C-band system over a new pair of fibers rather than add the L-band to an existing C-band system. Some of the new carriers who have recently built new fiber networks particularly have a large number of excess fibers and use this approach. Carriers who have deployed dispersion-shifted fibers are likely to be early adopters of the L-band for WDM (and other fiber bands besides the C-band) due to the difficulties associated with four-wave mixing and other nonlinearities in the C-band on this type of fiber. Long-Haul Network Case Study In this section, we look at a fairly realistic example of designing a North Amer- ican long-haul backbone network. We use the network topology shown in Fig- ure 13.8(a). We look at using conventional-reach long-haul (LH) systems as well . rates, and that is what they have done. As we have mentioned earlier, systems operating in the C-band as well as in the L-band are now available. The L-band requires a separate amplifier and. increases in the number of wavelengths. Today's state-of-the-art long-haul systems carry about 100 channels at 10 Gb/s each and have regenerator spacings of 400 to 600 km. The ultra-long-haul. added advantage of effectively doubling the channel spacing. For instance, if we trans- mit 100 channels spaced 50 GHz apart over a fiber, then we have 50 channels spaced 100 GHz apart in each