Gigabit, 10-Gb, and Future Ethernet 349 to expect that its evolution will cease. The higher speeds and greater transmission dis- tances that are making Ethernet both a LAN and a MAN protocol are not the only additions to the Ethernet standard that we are likely to see. Because fiber is being used as the transmission medium, the likelihood that an error in the data might occur during the passage of the Ethernet packet across the network is very low, much lower than in the original Ethernet. On a network with a very low error rate, it makes sense to trans- mit larger packets of data. The upper limit on the amount of data that can be carried in an Ethernet packet (a frame) is 1500 bytes. Sending more data than that in a frame would make it an invalid Ethernet frame and cause the network to discard it. This has been the standard since Ethernet was created. Given a low likelihood of errors on a network, large files could be moved over the network more efficiently if a larger amount of data could be carried in each frame. The reason for this is that it takes time for computers to generate and to process Ethernet headers and trailers. Each Ethernet frame must have a header and a trailer. For example, if six times as much data could be sent per frame, there would be fewer frames (only one sixth as many) needed to carry all the data in a file. This means that fewer headers and trailers would have to be generated by the transmitter and pro- cessed by the receiver. The result is a shorter amount of time needed to move a large file over a network between two computers. WANs that use fiber as their transmission medium routinely transmit large data packets. For this reason, especially when multigigabit LANs are connected to WANs, it is likely that we will see the use of Jumbo Ethernet frames. A Jumbo frame is any Ethernet frame that is carrying more than 1500 bytes of data. The proposed upper limit for the amount of data carried in a Jumbo frame is about 9,000 bytes. Jumbo frames are not currently a part of the new IEEE 802.3ae standard. However, it is very likely that some vendors of Ethernet networking equipment will allow Jumbo frames to be carried on Ethernet networks built using only their equipment. This might force the IEEE 802.3 committee to make support for larger frame sizes an option in new multigigabit standards. Table 6-13 shows the parameters for 10-Gb Ethernet operation. Table 6-13 Parameters for 10-Gbps Ethernet Operation Parameter Value Bit-time 0.1 nsec Slot time —* Interframe spacing 96 bits** continues 1102.book Page 349 Tuesday, May 20, 2003 2:53 PM 350 Chapter 6: Ethernet Technologies and Ethernet Switching Amazingly, 10GbE uses the same frame format (with a few special case exceptions) as 10-, 100-, and 1000-Mbps Ethernet. 10GbE Media, Connections, and Architecture 10-Gb Ethernet is a tenfold increase in speed over Gigabit Ethernet. Just as with Gigabit Ethernet, with this increase in speed comes extra requirements—the bits being sent get shorter in duration (1 ns), occur more frequently, and require more careful timing. In addition, their transmission requires frequencies closer to medium bandwidth limitations and they become more susceptible to noise. In response to these issues of synchroniza- tion, bandwidth, and SNR, two separate encoding steps are used by 10-Gb Ethernet. The basic idea is to use codes—which can be engineered to have desirable properties— to represent the user data in a way that is efficient to transmit, including synchronization, efficient usage of bandwidth, and improved SNR characteristics. Bit patterns from the MAC sublayer are converted into symbols, with symbols some- times controlling information (such as start frame, end frame, and medium idle condi- tions). The entire frame is broken up into control symbols and data symbols (data code groups). All of this extra complexity is necessary to achieve the tenfold increase in net- work speed over Gigabit Ethernet. 8B/10B encoding (similar to the 4B/5B concept) is used, followed by several different types of line encoding on the optical fiber. Collision attempt limit —* Collision backoff limit —* Collision jam size —* Maximum untagged frame size 1518 octets Minimum frame size 512 bits (64 octets) Burst limit —* Interframe spacing stretch ratio 104 bits*** * 10-Gbps Ethernet does not permit half-duplex operation, so parameters related to slot timing and collision handling do not apply. ** The value listed is the official interframe spacing. *** The interframe spacing stretch ratio applies exclusively to 10GBASE-W definitions. Table 6-13 Parameters for 10-Gbps Ethernet Operation (Continued) Parameter Value 1102.book Page 350 Tuesday, May 20, 2003 2:53 PM Gigabit, 10-Gb, and Future Ethernet 351 Figure 6-23 represents what happens to the 8B-10B before it is line-encoded. 10-Gb Ethernet uses a variety of complex encodings before line encoding, including 8B/10B and 64B/66B. Bits from these codes then are converted to line signals: low power light for binary 0 and higher power light for binary 1. Complex serial bit streams are used for all versions of 10GbE except for 10GBASE-LX4, which uses wide wavelength- division multiplexing (WWDM) to multiplex 4-bit simultaneous bit streams as four wavelengths of light launched into the fiber at one time. Figure 6-23 How 10GbE Converts MAC Frames to Four Lanes of Bits Figure 6-23 shows how 10GbE converts MAC frames to four lanes of bits for parallel transmission on four wire pairs of UTP or as a bit stream that is then serialized for laser transmission on single-mode fiber. Figure 6-24 represents the particular case of using four slightly different-colored laser sources. Upon receipt from the medium, the optical signal stream is demultiplexed into four separate optical signal streams. The four optical signal streams then are converted back into four electronic bit streams as they travel in approximately the reverse pro- cess back up through the sublayers to the MAC sublayer. Currently, most 10GbE products are in the form of modules (line cards) for addition to high-end switches and routers. As the 10GbE technologies evolve, an increasing diver- sity of signaling components can be expected. As optical technologies involve, improved transmitters and receivers will be incorporated into these products, further taking advantage of modularity. All 10GbE varieties use optical-fiber media. Fiber types include 10µm single-mode fiber, and 50µm and 62.5 µm multimode fibers. A range of fiber attenuation and dispersion characteristics are supported, but they limit operating distances. 1102.book Page 351 Tuesday, May 20, 2003 2:53 PM 352 Chapter 6: Ethernet Technologies and Ethernet Switching Figure 6-24 10GBASE-LX4 Signal Multiplexing SC fiber optic connectors most commonly are used. Because optical fiber is the medium used by 10GbE, typically a fiber pair connects Tx for device 1 to Rx for device 2, and vice versa. The primary devices connecting currently via 10GbE are high-end modular switches and routers. Table 6-14 lists the pinout options for 10GbE. 10-Gb Ethernet is available in full-duplex mode only and runs only over optical fiber. Hence, collisions are nonexistent and CSMA/CD is unnecessary. As 10GbE standards and products evolve, a wide range of architectures and applica- tion guidelines is becoming possible. Most important to consider is that the addition of 10GbE, with its LAN, SAN, MAN, and WAN capabilities, enables network engineers to consider very sophisticated end-to-end Ethernet networks. LAN, SAN, MAN, and WAN topologies using Gigabit Ethernet all are being implemented. 10-Gb Ethernet is supported only over fiber-optic media. Support is available for 62.5 µm and 50 µm multimode fiber, as well as 10 µm single-mode fiber. Even though support is limited to fiber-optic media, some of the maximum cable lengths are surprisingly short. No repeater is defined for 10-Gb Ethernet because half duplex explicitly is not supported. Table 6-14 10GbE Pinout Fiber Signal 1 Tx (laser transmitters) 2 Rx (high-speed photodiode detectors) 1102.book Page 352 Tuesday, May 20, 2003 2:53 PM Gigabit, 10-Gb, and Future Ethernet 353 As with 10-Mbps, 100,-Mbps and 1000-Mbps versions, it is possible to modify some of the architecture rules slightly. Possible architecture adjustments are related to signal loss and distortion along the medium. Because of dispersion of the signal and other issues, the light pulse becomes undecipherable beyond certain distances. Refer to the technical timing and spectral requirements in the current 802.3 standard, as well as the technical information about your hardware performance, before attempting any adjustments to the architecture rules. Table 6-15 shows the 10-Gb Ethernet implementations. Both R and W specifications are covered by each appropriate entry (for example, 10GBASE-E covers both 10GBASE-ER and 10GBASE-EW). Note the versatility of 10GbE. A diverse set of fiber types and laser sources can be used to achieve not only LAN, but also MAN and WAN distances. Table 6-15 10-Gb Ethernet Implementations Implementation Wavelength Medium Minimum Modal Bandwidth Operating Distance 10GBASE-LX4 1310 nm 62.5 µm MMF 500 MHz/km 2m to 300m 10GBASE-LX4 1310 nm 50 µm MMF 400 MHz/km 2m to 240m 10GBASE-LX4 1310 nm 50 µm MMF 500 MHz/km 2m to 300m 10GBASE-LX4 1310 nm 10 µm SMF — 2 km to 10 km 10GBASE-S 850 nm 62.5 µm MMF 160 MHz/km 2m to 26m 10GBASE-S 850 nm 62.5 µm MMF 200 MHz/km 2m to 33m 10GBASE-S 850 nm 50 µm MMF 400 MHz/km 2m to 66m 10GBASE-S 850 nm 50 µm MMF 500 MHz/km 2m to 82 m 10GBASE-S 850 nm 50 µm MMF 2000 MHz/km 2m to 300 m 10GBASE-L 1310 nm 10 µm SMF — 2 km to 10 km 10GBASE-E 1550 nm 10 µm SMF — 2 km to 30 km* *The standard permits 40-km lengths if link attenuation is low enough. 1102.book Page 353 Tuesday, May 20, 2003 2:53 PM 354 Chapter 6: Ethernet Technologies and Ethernet Switching The Future of Ethernet As the last several sections have documented, Ethernet has gone through an evolution from legacy to Fast to Gigabit to multigigabit technologies. Although other LAN tech- nologies are still in place (legacy installations), Ethernet dominates new LAN installa- tions—so much so that some have referred to Ethernet as the LAN “dial tone.” Ethernet is now the standard for horizontal, vertical, and interbuilding connections. In fact, recently developing versions of Ethernet are blurring the distinction between LANs, MANs, and WANs in terms of geographic distance covered as part of one network. Figure 6-25 illustrates the expanding scope of Ethernet. Figure 6-25 Ethernet’s Expanding Scope Although Gigabit Ethernet is now widely available and 10-Gb products becoming more available, the IEEE and the 10-Gb Ethernet Alliance currently have released 40-Gbps, 100-Gbps, and even 160-Gbps standards. Which technologies actually are adopted will depend on a number of factors, including the rate of maturation of the technologies and standards, the rate of adoption in the market, and cost. Proposals for Ethernet arbitration schemes other than CSMA/CD have been made. But the problem of collisions, so fundamental to physical bus topologies of 10BASE5, 10BASE2, 10BASE-T, and 100BASE-TX hubs, are no longer so common. Use of UTP and optical fiber, both of which have separate Tx and Rx paths, and the decreasing costs of switched instead of hubbed connections, make single shared-media, half-duplex media connections much less important. 1102.book Page 354 Tuesday, May 20, 2003 2:53 PM Ethernet Switching 355 The future of networking media is threefold: ■ Copper (up to 1000 Mbps, perhaps more) ■ Wireless (approaching 100 Mbps, perhaps more) ■ Optical fiber (currently at 10,000 Mbps and soon to be more) Unlike copper and wireless media, in which certain physical and practical limitations on the highest-frequency signals that can be transmitted are being approached, the bandwidth limitation on optical fiber is extremely large and is not a limiting factor for the foreseeable future. In fiber systems, the electronics technology (such as emitters and detectors) and the fiber-manufacturing processes most limit the speed. Therefore, upcoming developments in Ethernet likely will be heavily weighted toward laser light sources and single-mode optical fiber. When Ethernet was slower, half duplex (subject to collisions and a “democratic” process for prioritization) was not considered to have the quality of service (QoS) capabilities required to handle certain types of traffic. This included such things as IP telephony and video multicast. However, the full-duplex, high-speed Ethernet technologies that now dominate the market are proving to be sufficient at supporting even QoS-intensive applications. This makes the potential applications of Ethernet even wider. Ironically, end-to-end QoS capability helped drive a push for ATM to the desktop and to the WAN in the mid- 1990s, but now Ethernet, not ATM, approaching this goal. At 30 years old, Ethernet technologies continue to grow and have a very bright future. Ethernet Switching As more nodes are added to an Ethernet physical segment, the contention for the medium increases. The addition of more nodes increases the demands on the available bandwidth and places additional loads on the medium. With the additional traffic, the probability of collisions increases, resulting in more retransmissions. A solution to the problem is to break the large segment into parts and separate it using Catalyst switches. This isolates these newly segmented sections into isolated collision domains. This reduces the number of collisions and increases the reliability of the network. Bridging and switching are technologies that decrease congestion in LANs by reducing traffic and increasing bandwidth. LAN switches and bridges, operating at Layer 2 of the OSI reference model, forward frames based on the MAC addresses to perform the switching function. If the Layer 2 MAC address is unknown, the device floods the frame in an attempt to reach the desired destination. LAN switches and bridges also 1102.book Page 355 Tuesday, May 20, 2003 2:53 PM 356 Chapter 6: Ethernet Technologies and Ethernet Switching forward all broadcast frames. The result could be storms of traffic being looped end- lessly through the network. The Spanning Tree Protocol (STP) is a loop-prevention protocol; it is a technology that enables switches to communicate with each other to discover physical loops in the network. The sections that follow introduce Layer 2 bridging, Layer 2 switching, switching modes, and the Spanning Tree Protocol (STP). Layer 2 Bridging A bridge is a Layer 2 device designed to create two or more LAN segments, each of which is a separate collision domain. In other words, bridges were designed to create more usable bandwidth. The purpose of a bridge is to filter traffic on a LAN to keep local traffic local, yet allow connectivity to other parts (segments) of the LAN for traf- fic that is directed there. To filter or selectively deliver network traffic, bridges build tables of all MAC addresses located on a network segment and other networks, and map them to associated ports. The process is as follows: ■ If data comes along the network medium, a bridge compares the destination MAC address carried by the data to MAC addresses contained in its tables. ■ If the bridge determines that the destination MAC address of the data is from the same network segment as the source, it does not forward the data to other seg- ments of the network. This process is known as filtering. By performing this process, bridges significantly can reduce the amount of traffic between network segments by eliminating unnecessary traffic. ■ If the bridge determines that the destination MAC address of the data is not from the same network segment as the source, it forwards the data to the appropriate segment. ■ If the destination MAC address is unknown to the bridge, the bridge broadcasts the data to all devices on a network except the one on which it was received. The process is known as flooding. Generally, a bridge has only two ports and divides a collision domain into two parts. All decisions made by a bridge are based on MAC addresses or Layer 2 addressing, and do not affect the logical or Layer 3 addressing. Thus, a bridge divides a collision domain but not a logical or broadcast domain. No matter how many bridges are in a network, unless a device such as a router works on Layer 3 addressing, all of the net- work will share the same logical (broadcast) address space. A bridge will create more (and smaller) collision domains but will not add broadcast domains. Because every device on the network must pay attention to broadcasts, bridges always forward them. Therefore, all segments in a bridged environment are considered to be in the same broadcast domain. 1102.book Page 356 Tuesday, May 20, 2003 2:53 PM Ethernet Switching 357 Layer 2 Switching LAN switches are essentially multiport bridges that use microsegmentation to reduce the number of collisions in a LAN and increase the bandwidth. LAN switches also support features such as full-duplex communication and multiple simultaneous con- versations. Figure 6-30 shows a LAN with three workstations, a LAN switch, and the LAN switch’s address table. The LAN switch has four interfaces (or network connec- tions). Stations A and C are connected to the switch’s Interface 3, and Station B is on Interface 4. As indicated in Figure 6-26, Station A needs to transmit data to Station B. Figure 6-26 LAN Switch Operation Remember that as this traffic goes through the network, the switch operates at Layer 2, meaning that the switch can look at the MAC layer address. When Station A transmits and the switch receives the frames, the switch assesses the traffic as it goes through to dis- cover the source MAC address and store it in the address table, as shown in Figure 6-27. Figure 6-27 Building an Address Table AC B 1 2 3 4 10 Mbps 10 Mbps Interface Stations 12 3 4 Data from A to B AC B 1 2 3 4 10 Mbps 10 Mbps Interface Stations 12 3 4 A X 1102.book Page 357 Tuesday, May 20, 2003 2:53 PM 358 Chapter 6: Ethernet Technologies and Ethernet Switching As the traffic goes through the switch, an entry is made in the address table identifying the source station and the interface that it is connected to on the switch. The switch now knows where Station A is connected. When that frame of data is in the switch, it floods to all ports because the destination station is unknown, as shown in Figure 6-28. Figure 6-28 Flooding Data to All Switch Ports After the address entry is made in the table, however, a response comes back from Station B to Station A. The switch now knows that Station B is connected to Inter- face 4, as shown in Figure 6-29. Figure 6-29 Responding to the Flooding Message The data is transmitted into the switch, but notice that the switch does not flood the traffic this time. The switch sends the data out of only Interface 3 because it knows where Station A is on the network, as shown in Figure 6-30. 1102.book Page 358 Tuesday, May 20, 2003 2:53 PM . 300m 10 GBASE-LX4 13 10 nm 50 µm MMF 400 MHz/km 2m to 24 0m 10 GBASE-LX4 13 10 nm 50 µm MMF 500 MHz/km 2m to 300m 10 GBASE-LX4 13 10 nm 10 µm SMF — 2 km to 10 km 10 GBASE-S 850 nm 62. 5 µm MMF 16 0 MHz/km 2m. 6 -27 . Figure 6 -27 Building an Address Table AC B 1 2 3 4 10 Mbps 10 Mbps Interface Stations 12 3 4 Data from A to B AC B 1 2 3 4 10 Mbps 10 Mbps Interface Stations 12 3 4 A X 11 02. book Page 357. MHz/km 2m to 300 m 10 GBASE-L 13 10 nm 10 µm SMF — 2 km to 10 km 10 GBASE-E 15 50 nm 10 µm SMF — 2 km to 30 km* *The standard permits 40-km lengths if link attenuation is low enough. 11 02. book Page 353