Before fully explaining how subnetting is performed, it is necessary to define subnet masking Recall that the network portion of the address cannot be changed For a router to decide what part of the address is the network and what part is the host, a 32-bit number is used to mask out each portion This mask performs a logical AND operation with the IP address Wherever the mask is binary one, it is considered the network portion; when the mask is zero, it is considered the host A zero in the network portion tells the router not to consider this part of the network during routing decisions (see Figure 2-5) Figure 2-5 Subnetting Using the Logical AND Operation Subnetting is performed by borrowing bits from the host portion of the address, and then using them to indicate different segments on the network A network administrator can borrow any number of bits from the host portion, as long as a few bits are left available to assign host addresses NOTE Subnetting extends the network portion of the assigned address to accommodate all physical segments of a network Subnetting Example In network 131.108.0.0, suppose that you want to perform eight-bit masking of the class B network Recall that the first two bytes cannot be altered in a class B network Therefore, to perform subnet masking, you must borrow bits from the third byte and use it completely As mentioned earlier, subnet masking is performed by borrowing bits from the host portion of the assigned network address, which assigns more routable network addresses within the assigned network address For subnet masking, you would perform a logical AND operation between the network number and the mask assigned, as shown here: ANDed with = 1 ANDed with = 51 For eight-bit masking, you can further divide the network into smaller segments to produce the following: ^8 - = 256 - = 254 subnets All zeros and all ones signify the broadcast address so that they usually cannot be assigned as the subnet, but Cisco Systems does allow subnet zero to be used as a subnet address To enable the subnet zero as an IP subnet, you must use the ip subnet zero command: ip subnet zero This should be done carefully to ensure that there are no old hosts that not understand subnet zero as the broadcast address Cisco leaves the choice to the network administrator If the administrator is sure that all hosts in the network not treat subnet zero as the broadcast address, this additional subnet can be used on the network Subnetting is completed between the network number and the subnet mask by a logical AND operation: 131.108.0.0 255.255.255.0 When a logical AND is performed between the network numbers, the third byte is advertised as the network portion Recall that anything ANDed with one yields the same number If you assign an IP address of 131.108.1.1 to an interface, the result, after performing the logical AND with the mask, is 131.108.1.0—anything ANDed with zero yields zero Subnetting is performed to assign addresses to different segments of networks, as well as to isolate the broadcast domains It also provides more flexibility to a network On the host portion of each subnet, there is a maximum of 254 hosts Remember, however, that there are eight bits left for the host portion because the network was subnetted after borrowing eight bits from the assigned class B network address Because eight bits are left for the host portion, there are 256–2 addresses left for each host per subnet You cannot use all zeros and all ones in an address, which disqualifies two addresses out of the 256 host addresses Variable-Length Subnet Masking With eight-bit masking, there are 254 subnets and 254 hosts per subnet This model works well on transit broadcast networks, in which a large number of hosts share a common media As shown in Figure 2-6, the serial line needs only two addresses to assign an address with a subnet mask of /24 Therefore, leaving space for 254 hosts is a waste of address space 52 Figure 2-6 VLSM for Point-to-Point Links Address waste is a serious problem in today's networks Obtaining an IP address is difficult because of constant growth and increasing numbers of users Aggressive effort is required to spare address space from being used inappropriately This issue will be discussed further in the next section, "Classless Interdomain Routing." For this reason, you should perform variable-length subnet masking (VLSM) with point-to-point networks VLSM grants transit broadcast networks a large number of bits for the host portion, and only allows the point -to-point network to use two bits for the host portion NOTE Using a different mask for several types of media within the same major network is called variable-length subnet masking You can subnet further for the serial link so that each link has only two addresses to assign to it— one for each end of the link's connection For example, suppose you wanted to further subnet the 131.108.10.0 subnet You know that the original subnet mask was the eighth bit in the third octet For the serial point-to-point connection, you can perform additional masking in the fourth octet, to essentially create a subnet of a subnet As shown in Figure 2-7 depicting serial links, you can use the same third octet value of two and further subnet the fourth octet Figure 2-7 shows the new subnets The original subnet of 131.108.2.0/24 is now further divided into additional subnets of 131.108.2.0/30– 131.108.2.252/30 Figure 2-7 Introduction of CIDR and Route Aggregation 53 If you borrow six bits from the fourth octet and leave two bits for the host portion, the result is as follows: ^6 - = 64 In this case, the serial line addresses are 131.108.2.0 and 255.255.255.252, and the host addresses are 131.108.2.1 and 131.108.2.2 You cannot assign addresses of 131.108.2.0 and 131.108.2.3 as the host address because they become the broadcast address for this subnet This way, you then can reach 131.108.2.252.255.255.252.0 with the host addresses of 131.108.2.253 and 131.108.2.254 Similarly, you cannot assign host addresses of 131.108.2.252 and 131.108.2.255 because they are the broadcast address for this subnet Classless Interdomain Routing As the popularity of the Internet has grown, it has become the global media for the transfer of information However, as popularity increased, new problems continued to appear Small organizations applied for IP addresses, but providing them all with a class A or class B address was not feasible Instead, these organizations were assigned class C addresses, or, in a large number of cases, multiple class Cs With such a large distribution of IP addresses, the routing table on the Internet began to grow exponentially This is the reason CIDR entered the arena The following issues led to CIDR: • • • Lack of midsize address space and exhaustion of the class B network address space Class C is quite small (with 254 hosts), and class B is relatively large (with 65,534 addresses) Growth of Internet routing tables Eventual exhaustion of the 32-bit IP address space 54 It became evident that the first two problems needed to be addressed immediately This led to the proposal of RFC 1519, which prompted slower growth of the Internet routing table by condensing groups of network addresses that fell within close range (called route aggregation) Route aggregation is performed similar to masking, which led to its other name, supernetting With CIDR, the masks in the assigned address are grouped into one update If an ISP holds an address range for several class C networks, it does not need to advertise all the specific networks The ISP simply can send one update by supernetting them NOTE Route aggregation is the grouping of contiguous class C or class B networks into one update As an example of route aggregation, assume that ISP A owns class C networks from 201.1.0.0 to 201.1.127.0 Instead of the ISP advertising all the class C networks, it can group them into one update and advertise a single supernet network Supernetting helped significantly slow the growth of routing tables on the Internet routers As shown in Figure 2-7, ISP A does not need to advertise all the specific routes from its customer to the neighboring ISP B Instead, ISP A can send only a single route to all its neighboring ISPs because it can target specific customers The neighboring ISPs only need to forward traffic to ISP A for the range of networks IP Routing In the section on subnetting, you learned how a network is divided into smaller groups known as subnets Each subnet is given an individual identity All subnets need to be advertised by an algorithm within the autonomous system, and the network as a whole must be advertised outside the system To propagate the network within the autonomous system and beyond, routing protocolsare used Routing protocols are divided into two types: First, Interior Gateway Protocols (IGPs) are used to propagate routing information within an autonomous system Second, Exterior Gateway Protocols (EGPs) are used to pass routing information between autonomous systems Routing protocols running on the Internet include Routing Information Protocol (RIP), Open Shortest Path First (OSPF), and Intermediate System-to-Intermediate System (IS -IS) All these protocols are standards-based In addition, both the Interior Gateway Routing Protocol (IGRP) and the Enhanced IGRP are Cisco-proprietary protocols The only EGP presently on the Internet is BGP, and the current version is four Each of these protocols is briefly introduced in the following sections RIP RIP is a distance-vector protocol that uses the Balman Ford algorithm to compute the shortest route to the destination It is based on hop count, and does not have the capability to detect realtime parameters for making proper decisions to reach a destination This protocol also has a hard limit of 15 hops, which a network cannot exceed For more information about RIP, see Chapter 6, "Routing Information Protocol," and Chapter 7, "Routing Information Protocol Version 2." 55 IGRP IGRP is also based on distance-vector routing Cisco developed this protocol in response to RIP's shortcomings—for example, routers ignore a better bandwidth route in favor of a shorter hop path IGRP has more intelligence to make routing decisions than RIP It relies on composite metrics of load, reliability, delay, bandwidth, and MTU It does not have the 15-hop limit—an IGRP network can use up to 254 hops, thereby increasing the dimension of the network Enhanced IGRP Enhanced IGRP is an advanced distance vector Unlike RIP and IGRP, this protocol ensures that updates are not propagated beyond the affected nodes, ensuring that the entire network is unaffected This protocol uses diffuse update algorithm (DUAL) to achieve rapid, loop-free convergence Every router maintains a neighbor table, as well as the relevant information received from the neighbor For more information on Enhanced IGRP, see Chapter 8, "Enhanced Interior Gateway Routing Protocol." OSPF and IS-IS OSPF and IS-IS are both link-state protocols; each router within an area maintains an identical database Every router advertises all its connected functional links after the information is received in the database Then, the SPF algorithm is executed to find the shortest path to the destination BGP BGP exchanges routing information between autonomous systems It is called a path-vector protocol because it carries path information from the source, and attaches all the systems that the route has traversed This path information is used to detect routing loops As networks grow and large organizations merge, BGP is increasingly moving into enterprise backbones because of its scalability BGP was designed to handle routing updates and route processing for the dynamic Internet environment Summary In this chapter, you learned the fundamentals of IP and its addressing structure IP can communicate across any set of interconnected networks, but is not a reliable datagram service; it is a best-effort delivery An IP address is 32 bits, which includes a network address and a host address You also learned about subnets, subnet masking, variable-length masking, and why they are necessary Subnets provide flexibility to a network by dividing a large network into smaller units, so that the entire network is not restricted by one network address Subnet masking is performed by borrowing bits from the host portion of the assigned network address, so that more routable network addresses may be assigned within the network address Variable-length masking is crucial for preserving valuable address space and to allow continued growth of a network 56 Another area covered in this chapter is Classless Interdomain Routing CIDR controls the size of Internet routing tables It assists the ISP environment by limiting the number of routes advertised, which is done by condensing the number of contiguous prefixes that ISP must advertise Finally, you were introduced to the two types of major routing processes currently used on the Internet: Interior Gateway Protocols and Exterior Gateway Protocols IGPs include RIP, OSPF, IS-IS and EIGRP; EGPs include only BGP You will read about each of these in more depth in the following chapters Review Questions 1: How long is an IP address? 2: When is the best time to perform subnet masking? When is not good practice to use it? 3: What is route aggregation? When should it be utilized? Answers: 1: How long is an IP address? A: An IP address is 32 bits long and is divided into four octets Each octet is separated by a dotted decimal, as in 131.108.1.1 2: When is the best time to perform subnet masking? When is not good practice to use it? A: Subnet masking is necessary for any IP network, even when you have a single interface and cannot attach thousands of hosts If you have a large number of hosts in your network, you should subnet to separate broadcast domains 3: What is route aggregation? When should it be utilized? A: Route aggregation deals with groupings of contiguous addresses You should perform it as a regular practice whenever you have a contiguous block of addresses, or major nets that are all behind a certain router You should remove unnecessary routing information, so that it is not sent where it is not required For Further Reading… RFC 791 Stevens, W Richard TCP/IP Illustrated, Volume Reading, MA: Addison-Wesley, 1994 57 Chapter Network Technologies This chapter provides an overview of current local-area network (LAN), wide-area network (WAN), and metropolitan-area network (MAN) technologies, emphasizing their use in deploying IP networks In particular, you will learn about the following: Packet, circuit, and message switching This section introduces these three switching paradigms and discusses how they relate to IP networks Local-area networks and technologies You will read about the difference between token passing and collision-detection technologies Then, we describe why Ethernet has become a ubiquitous form of LAN technology Finally, we introduce the basic operation of IP over Ethernet Wide-area networks and technologies This section contrasts serial Time Division Multiplexing (TDM) for leased lines, Frame Relay, ATM, and Packet over SONET We describe the main benefits and drawbacks of these technologies, as well as their interaction with the Internet Protocol Metropolitan-area networks and technologies In this section, we briefly introduce various MAN technologies, along with our thoughts on the future direction of MANs Packet, Circuit, and Message Switching The three switching paradigms that you are likely to encounter with IP networks (packet switching, circuit switching, and message switching) each have their own characteristics and requirements that you should consider before deciding which one best suits your network The sections that follow define these switching paradigm characteristics and requirements Packet-Switched Networks The Internet, and IP networks in general, are packet-switching networks.This means that all data is segmented into variable-length IP packets, which are then routed across the network as discrete entities, as shown in Figure 3-1 Each IP packet contains a source and a destination, as well as mechanisms to detect packet corruption Routing in IP networks is usually based on an IP destination address Figure 3-1 Packet Switching 58 Packet routers in IP networks are able to detect IP packet errors, but they not perform error correction or provide substantial congestion control (Internet Control Message Protocol [ICMP] source-quench messages are typically ignored by routers and host) These functions are left to the Transport Control Protocol (TCP) stack that is implemented on the hosts that connect to the network While certain WAN technologies may implement error correction and congestion control in Layer 2, this process is transparent to the IP router Many experts argue that performing such functions in Layer can interfere with the performance of TCP, which causes TCP to degrade For large IP networks, therefore, it is not advisable to configure any Layer error correction or congestion control algorithms Sequential IP packets not necessarily follow the same path through the network, although in stable routed environments they generally should For example, the situation depicted in Figure 3-1, in which IP packets 1, 2, and 3, take different routes over the network is undesirable This is important because performance of the TCP error correction/congestion control is degraded by the rapid changes in round trip times when packets take multiple routes—it will look like congestion Note that load sharing traffic over multiple parallel WAN links is usually not problematic, if the propagation delay over each link is similar IP packets may be fragmented by IP routers to fit inside the maximum transmission unit (MTU) associated with particular Layer technologies The packets are re-assembled by the IP host that ultimately receives packets, rather than being re-assembled by routers Fragmentation normally reduces the efficiency of routers and IP hosts alike For this reason, it is important to avoid fragmentation within your network in most cases Note that most modern TCP applications also set the Don't Fragment-Bit in the header and are using the Path-MTU-Discovery mechanism (described in RFC 1191) to automatically detect the maximum possible path MTU size Because most host IP implementations usually source IP packets that require routing with a length of 512 bytes, fragmentation is generally not an issue in networks employing common WAN or LAN technologies that support much larger frame sizes It is worth noting that the ATM Adaptation Layer 5, usually via hardware assisted code, segments the IP packets into cells It then re-assembles them in the full IP packet prior to routing to other media Therefore, IP fragmentation is not an issue in ATM networks, providing the reassembly buffer at the remote end of the ATM cloud matches (or is at least smaller than) the MTU sizes used by other WAN or LAN technologies in the network Packet over SONET technologies are even more desirable, because the segmentation function and associated cell tax is completely avoided 59 NOTE Cell tax refers to the relatively low ratio of data payload (48 bytes) to header size (5 bytes) in an ATM cell Compare this with Ethernet frames, in which the ratio can be as high as 1500:26 While cell tax may not be in issue for applications that generated data in small discrete quantities, for applications involving bulk data transfer (such as downloading images), cell tax leads to a significant decrease in useful data throughput compared with other technologies operating at the same wire speed A packet-switched IP network, in conjunction with careful provisioning and congestion control techniques that are cognizant of TCP, offers extremely scalable technology for supporting a wide range of both non-real and real-time applications This scalability and flexibility is causing the communications world to focus on the use of IP networks to provide the traditional "Internet" applications, as well as applications that were traditionally carried by circuit-switched telephone networks IP packet switching is necessary for many large corporations and progressive carriers as the underlying technology for large networks of the future Circuit-Switched Networks Packet-switched networks fundamentally differ from circuit-switched networks As shown in Figure 3-2, a connection must first be established between two end hosts in order for them to communicate in a circuit-switched network This can be achieved by i n-band signaling (call_setup) within a circuit—in other words, the end host transmits a set of signals that allows the circuit to be extended, hop-by-hop, through the network Alternatively, as in the case of the Integrated Services Digital Network (ISDN), the circuits can be established with the assistance of a second "control-plane" network, which is usually a lower-bandwidth, packet-switched network, and carries only the call setup packets This requirement for a pre-established circuit prior to communication is in contrast to IP's "connectionless" paradigm, in which a host can begin transmitting to any other host on the network at any time Figure 3-2 Circuit Switching Also, unlike packet-switched networks, once the circuit is established, all data flows over the same path through the network In Figure 3-1, all data associated with the call passes through nodes A, B, and C; and follows the symetrical return path Therefore, the parameters of the session, such as delay and bandwidth, are fixed—this is both an advantage and a limitation to 60 Angeles as, say, router core1.lax, but it does indicate that core1.sfo understands that core1.lax and core2.lax, rather than core1.stl, are the gateways to all destinations in the Los Angeles region Backbone routers contain reachability intelligence for all destinations within the network They possess the capability to distinguish between the gateway information and the information that explains how to reach the outside world, which is through other peer networks or the Internet Distribution Routers Distribution routers consolidate connections from access routers They are often arranged in a configuration that is resilient to failure of a single core router Distribution routers usually contain topological information about their own region, but they forward packets to a backbone router for inter-region routing NOTE In smaller regions, distribution and backbone routers may be one and the same In larger regions, distribution routers themselves may form a hierarchy High-performance customers on permanent WAN links often may connect directly to distribution routers, whereas dial-on-demand customers typically not because this would impose the need to run dial-authentication software images of distribution routers Access Routers Access routers connect the customer or enterprise site to the distribution network In the ISP case, the router at the remote end of an access link is typically the customer premises equipment, and may be owned and operated by the customer For large enterprise networks, in which the LANs and WANs are managed by different divisions or contractors, the access router typically is managed by either the WAN or the LAN operator— usually this is the latter if the LAN is very large You now may wonder: Why is it important to distinguish between the backbone, access, and distribution routers? The reason is that they are increasingly becoming very distinct hardware/software combinations In access routers, for example, you already have seen the need for support of dial-on-demand and authentication, as well as route filtering and packet filtering and classification In distribution routers, the emphasis is on economical aggregation of traffic and the support of varied media WAN types and protocols In backbone routers, the emphasis is on supporting extremely high speeds, and aggregation of a very limited set of media types and routing protocols These differences are summarized in Table 4-1 Table 4-1 Characteristics of Backbone, Distribution, and Access Routers Router Type Characteristics Backbone router Scalable: packet forwarding, WAN links, QoS, routing Expensive 85 Redundant WAN links Distribution router National infrastructure Scalable: WAN aggregation, LAN speeds Redundant LAN links Access router Less expensive Scalable: WAN aggregation Cheap Complex routing/QoS policy setting, access security, and monitoring capabilities This discussion focused attention on the WAN environment and has avoided any issues of LAN design, other than the use of specific LAN technology within the distribution or access networks In particular, at the individual user or network host level, access technologies include ATM, FDDI, Token Ring, or the ubiquitous Ethernet; rather than such technologies as Frame Relay, T1, SMDS, and SONET Scaling LANs through the use of hierarchy is itself the subject of much literature To study this area further, interested readers should refer to the references listed at the end of this chapter The origins of the three-tiered, backbone-distribution-access hierarchy can be traced to the evolution of the Internet (refer to Chapter 1) However, hierarchical design is certainly nothing new and has been used in telephone networks and other systems for many years In the case of IP data networking, there are several reasons for adding hierarchy Not only does hierarchy allow the various elements of routing, QoS, accounting, and packet switching to scale; but it also presents the opportunity for operational segmentation of the network, simpler troubleshooting, less complicated individual router configurations, and a logical basis for distance-based packet accounting These issues are examined in great depth in Part II of this book, "Core and Distributing Networks." For the moment, we will examine the topologies used within the backbone, distribution, and access layers of the network architecture Backbone Core Network Design In early data networking, the topology for the network backbone was relatively simple: Operations were centralized, so a star topology made the most sense—and, in some cases, this was the only topology the technology would support This did cause the center of the star to become a single point of failure, but because no real traffic flows existed between spokes on the star, this was not a major cause for concern With the move toward multiple client -server and peer-t o-peer relationships, the choice of core network topology is not as clear The purpose of the backbone is to connect regional distribution networks and, in some instances, to provide connectivity to other peer networks A national infrastructure usually forms a significant part of the operational cost of the network Given its position at the top of the network hierarchy, two requirements of the backbone topology are clear: it must be reliable and it must scale 86 Making the Backbone Reliable Reliability can be acquired by employing two methods First, you can create more reliable routers through the use of "carrier-class" characteristics, such as multiple CPUs, power supplies, and generators; and even redundant routers Ultimately, however, any backbone will include WAN links that rely on a great deal of equipment and environmental stability for their operation, which represents a real risk of ultimate failure If the carrier's up-time guarantees are not sufficient, you have no choice but to design a backbone that is resilient to link failure The second option is to simply connect all distribution networks with a full mesh However, in terms of minimizing hop count within the network, the full mesh approach has several drawbacks: • • • First, given N regional distribution networks, you must have N(N-1)/2 backbone links in the core This creates expense in WAN circuitry, as well as in router and WAN switch hardware (channelized or ATM technology can reduce these issues) Moreover, PVC sizing requires that the traffic levels between any two distribution networks should be well understood, or that the network has the capability to circumvent congestion Although traffic engineering calculations and circumventing congestion are common in the telephone network, common IP networks and their associated routing protocols not provide this capability as readily One good reason is that the resources required by any TCP/IP session are not known a priori, and IP networks are traditionally engineered as best-effort Chapter 14 explores how to bypass best-effort by providing differentiated service in IP networks A full PVC mesh can also obviate one of the benefits of multiplexing, or trunking, in a best-effort network Round-trip time and TCP window size permitting, any user can burst traffic up to the full line rate of the trunk Furthermore, the routing complexity in a full mesh can consume bandwidth, computational, and operational management resources Most backbone topologies are, therefore, initially designed based on financial constraints, such as user population density, or application requirements; and WAN service availability This initial design can be subsequently refined quite effectively by statistical analysis of traffic levels after the backbone is operational, and the availability of new WAN technologies is known Data network requirements analysis is a relatively new art See [McCabe, 1998] for thorough coverage of this area Building the Backbone Topology Because you have a basic need for resilience in the backbone, a good starting point for the backbone topology is a ring connecting all distribution networks This ring could represent the minimum cost of WAN circuits, compromised by an initial estimate of major traffic flows, and possibly some very particular delay requirements (although this is rare, with notable exceptions being high-performance networks) Next, existing links can be fattened, or direct connections between backbone routers can be added as required or as is cost-effective This incremental approach should be considered when selecting WAN technologies, routing nodes, and interface types Backbone routing protocols, such as IBGP, properly coupled with OSPF, IS -IS, and Enhanced IGRP, can rapidly circumvent failures by simple link-costing mechanisms However, the bandwidth allocations with the core topology should consider failure modes What happens when the ring is broken due to WAN or node failure? Is the re-routed path sufficient to carry the additional traffic load? Although TCP performs extremely well in congested environments 87 compared with other protocols, it is still possible to render the network useless for most practical applications Analysis of historical traffic levels, captured by SNMP, for example, provides for a relatively accurate estimation of the consolidated load on the remaining links during various failure modes Traditionally, the use of a ring topology made it difficult to estimate the traffic levels between individual distribution networks SNMP statistics, for example, provided only input and output byte counts for WAN interfaces, making it difficult to determine the appropriate sizing for new direct links between distribution networks Typically, this had to be accomplished using a cumbersome approach, such as "sniffers" on WAN links, or through accounting capabilities within routers that scaled rather poorly However, IP accounting facilities, such as Netflow, now provide a scalable way for network managers to collect and analyze traffic flows, based on source and destination addresses, as well as many other flow parameters This significantly eases traffic engineering and accounting activities It is now possible to permanently collect and archive flow data for network design or billing purposes NOTE Netflow is a high-performance switching algorithm that collects comprehensive IP accounting information and exports it to a collection agent Load sharing is possible on the backbone network With Cisco routers, this can be either on a per-packet or a per-flow basis The latter usually is recommended because it avoids possible packet re-ordering, is efficiently implemented, and avoids the potential for widely varying roundtrip times, which interfere with the operation of TCP This is not a problem for per-packet load sharing over parallel WAN circuits, but it can be a problem when each alternate path is one or more routed hops It is possible to connect regional networks directly, avoiding the backbone altogether and possibly providing more optimal routing For example, in Figure 4-2, the DCs in SFO and LAX could be connected by a direct link Traffic between the SFO and LAX regional networks could then travel over this link rather than over the backbone However, this exercise should be viewed as the effective consolidation of two regional distribution networks, and the overall routing architecture for the newly combined regions should be reengineered to reflect this On an operational note, the backbone network and routers may be under different operational management teams to the regional networks One historical example is the arrangement between the NSFNET backbone and the regional networks described in Chapter Today, many smaller ISPs use the NSPs for WAN connectivity In this situation, the routing relationship between the backbone and the distribution networks is likely to be slightly different because an Exterior Gateway Protocol such as BGP will be used In this book, the operators of the backbone and regional networks are generally considered to be the same, which makes it possible for the two to share a hierarchical IGP In Chapter 16, "Design and Configuration Case Studies," you will examine a case study for scaling very large enterprise networks in which this is not the case 88 Distribution/Regional Network Design The role of the regional network is to route intra- and inter-regional traffic The regional network generally is comprised of a DC as the hub and a number of access POPs as the spokes Usually, two redundant routers in each regional network will connect to the backbone DCs may also provide services such as Web-caching, DNS, network management, and e-mail hosting In some cases, the latter functionality may be extended into major POPs Placement of DCs is generally an economical choice based on the geographical proximity to a number of access sites However, this does not mean that an access POP cannot be a minidistribution center or transit for another access POP, but this is the exception rather than the rule When an access POP site provides such transit, and when that transit is the responsibility of the service provider, it should be considered part of the distribution network functionality Although the DC may be the center of a star topology from a network or IP perspective, this does not limit the choice of data-link or WAN connectivity to point-to-point links Frame Relay or other cloud technologies can be—and often are—used to provide the connectivity from the customers, or from other distribution and access sites to the DC Even within the DC, a provider may utilize Layer aggregation equipment, such as a Frame Relay or ATM switch, or even an add/drop multiplexor A major DC typically consists of many routers, carrying either intra-regional or backbone-transit traffic As more customers receive service from the DC, the higher the stakes become Therefore, the backbone and intra-distribution network infrastructure must become more reliable A common option at major DCs is to provide dual aggregation LANs, dual backbone routers, and dual backbone WAN connections, as shown in Figure 4-3 This approach also can provide an element of load sharing between backbone routers Of course, a single aggregation LAN and single backbone router will also serve this purpose It is important to weigh the cost-versus reliability issues, and bear in mind that most simple MTBF calculations consider hardware, but often ignore both software bugs and human error FDDI rings are a logical choice for the aggregation LAN because of their inherent fail-over mechanisms However, with the development of low-cost/high-reliability LAN switches based on FDDI, Ethernet, or ATM technology—not to mention the ever-increasing intra-DC traffic levels—it is not uncommon to implement the dual aggregation LANs using switched media IP routing circumvents LAN failure at either the single line card or the single switch level, as discussed in upcoming chapters Of course, many other critical reliability issues have not yet been considered These include facilities, such as power supply and the choice of router and switching equipment NOTE The distribution network is hierarchical Router dist3 is located as an access POP, which services fewer customers, and therefore is not a resilient design The backbone/distribution/access hierarchy can be bypassed to achieve lower delays at the expense of reliability Customer may connect directly to router core2.sfo However, if core2.sfo 89 fails—albeit a rare event—customer is effectively cut off from the network Alternatively, customer may have a backup connection via dist3.sfo This arrangement is satisfactory, provided that it does not confuse the role of each router For example, directly connecting customer routers to the core router indicates that they may have to perform dial-up authentication, packet and router filtering, and packet classification Not only will this occupy precious switching cycles on the core router, but it also could mean running a larger and possibly less reliable software image Other possible failure modes include the following: • Core1 All intra-network traffic is routed through core2 All traffic to other ISPs is also routed through core2, presumably to another NAP connected to a backbone router elsewhere in the network • Ds1 Traffic destined for a remote distribution network is switched through ds2, as is traffic destined for other locations in the local distribution network • Dist1 Customer is re-routed through Dist2 • Dist3 Customer is cut off It is worth noting that any resilience at Layer results in routing complexity This is examined in detail in Part II As a matter of policy, the network service provider may choose not to allow customers to connect to core routers or even to dual distribution routers However, in the enterprise environment, reliability affects user satisfaction In the commercial environment, this may affect their choice of provider Policy that simplifies engineering must be carefully balanced against customer requirements Policy also must be balanced against the risk of human error A resilient routing environment might be more reliable in theory, but in practice it might have a greater risk of human configuration error, and possibly algorithmic or vendor implementation flaws Access Design In most cases, an access router serves a large number of customers With modern access technology, this number can reach the thousands As a result, resilient connectivity to the distribution routers is recommended This may be accomplished using a self-healing LAN technology, such as FDDI Alternatively, as with the connectivity between distribution and backbone routes, this may involve the use of redundant LAN switches If the access router is the only node in a small POP, redundant WAN connections to the nearest DC are an option 90 The design of the access topology is generally a choice of WAN technology between the CPE and the access router For redundancy or load-sharing purposes, two or more links may be homed into the same access router or possibly onto different access routers This is an issue of provider policy and capabilities Although the topology of the access network is relatively simple, it is here that the "policing" of customer connections, in terms of traffic rates and accounting, QoS, and routing policy, occurs The configuration and maintenance must be executed carefully The consequences of a router misconfiguration can be severe Summary Network design involves the art and science of meeting requirements while dealing with economic, technological, physical, and political constraints Scalability and extensibility are the hallmarks of a successful large-scale network design, and are encouraged through layering, modularization, and hierarchy Randomization, soft state, dampening, separation of the control plane, regionalization, and optimizing the common case are also important considerations for routing protocols and the overall routing topology Although requirement analysis is an important aspect of design, it should be viewed as an ongoing task and should be ratified by the collection of traffic statistics that describe actual network usage By categorizing routers into the roles of backbone, distribution, and access, you will simplify the hardware/software combinations and configuration complexity required for any particular router This consequently simplifies the operational support of the network Within the various tiers of the hierarchy, the topologies of ring, star, bus, and mesh may be employed The choice depends on reliability, traffic, and delay requirements In the case of WAN topologies, carrier service pricing also could be a determining factor Review Questions 1: If you need to support protocols other than IP in a large network, what would you do? 2: When would you consider breaking the hierarchy of a network design by linking distribution networks directly? 3: ATM is an ideal technology to grow a ring backbone to a partial mesh, and then to a full mesh Does this make it a better choice for a backbone technology than point-to-point links? Why or why not? 4: Could you use different routers in your access, distribution, and core networks? Answers: 1: If you need to support protocols other than IP in a large network, what would you do? A: If at all possible, try to tunnel the protocol in IP The current trend among 91 vendors of routers for large networks is to support only IP At some point, native support of other protocols simply may not be an option 2: When would you consider breaking the hierarchy of a network design by linking distribution networks directly? A: Break network hierarchy only when you have a very solid business case for doing so You should consider the expense of the additional operational complexity in adding the link In most cases, you may find that the same result can be achieved by adding more backbone capacity 3: ATM is an ideal technology to grow a ring backbone to a partial mesh, and then to a full mesh Does this make it a better choice for a backbone technology than point-t o-point links? Why or why not? A: It all comes down to cost, and this varies greatly from country to country Determine the cost of the two approaches over 1, 3, 5, and 10 years; and compare Of course, you can only estimate your future backbone requirements, which means that any approach will be a carefully calculated risk 4: Could you use different routers in your access, distribution, and core networks? A: If you are using standardized protocols, yes However, a multi-vendor environment increases operational complexity and vulnerability to interoperability issues Certainly, within the access network, you could run a mix of routers As you go up the hierarchy into the distribution and core networks, mixing products from different vendors becomes more risky For Further Reading… The available literature on network design (other than an abstract mathematical treatment) is surprisingly small If you have well-known requirements, McCabe's book is unique in its treatment of network design through requirements and flow analysis Bennett, G Designing TCP/IP Internetworks New York, NY: John Wiley & Sons, 1997 Galvin, P B and A Silberschatz Operating System Concepts Reading, MA: Addison-Wesley, 1997 Keshav, S An Engineering Approach to Computer Networking Reading, MA: Addison-Wesley, 1997 McCabe, J Practical Computer Network Analysis and Design San Francisco, CA: Morgan Kaufmann Publishers, 1998 Pressman, R Software Engineering: A Practitioners Approach, Fourth Edition New York, NY: McGraw-Hill, 1996 92 Chapter Routers The fundamental role of the router is route computation, packet scheduling, and forwarding This role has become confused as vendors bundle more functionality into operating systems and platforms that traditionally focused on simple routing and packet forwarding Accounting, security filtering, encapsulation, tunneling, address translation, packet classification, and proxying are just a few of the capabilities being squeezed into what is more accurately termed a general-purpose network appliance.Routing is merely a subset of this appliance's capabilities This chapter provides an overview of modern IP routers We focus on router functionality that is central to building a large-scale network Specifically, the following issues are covered: Router architecture Router hardware architectures have undergone three generations of development This section traces that development, summarizing the improvements that occurred within each generation The evolution of switching paradigms Packet-switching algorithms also have evolved with each generation of router architecture The evolution of the Cisco product line from process switching to Cisco Express Forwarding is described Routing and forwarding The functions of routing and forwarding are often confused Routing protocols and routecomputation are contrasted with forwarding algorithms and route-lookup Switching with QoS and Policy A router's job is complicated by the need to provide differentiated service This section summarizes queuing algorithms and introduces the role of a packet scheduler Router Architecture In Chapter 1, "Evolution of Data Networks," you learned about the evolution of router technology within the early Internet Essentially two approaches were presented: • • A single computer/central-processor approach, such as the original NSFNET "fuzzball" routers The parallel switching/multi-processor nodal switching subsystems (the successor) Each approach is discussed in more detail in the following sections NOTE A router isolates the link-layer broadcast domains of subnetworks, forwards IP packets between the domains, decrements the TTL, and sometimes performs media translation and fragmentation 93 To make a forwarding decision, the router exchanges topological information about the network with other routers Single CPU Designs The first approach utilizes a single CPU-controlled shared bus that connects a number of slave interface cards This arrangement can be based on a general-purpose computer, such as a PC running UNIX or Windows NT Various bus communication strategies (such as shared memory, DMA, and bus mastering), together with high-performance RISC CPUs, can result in a router of significant forwarding capabilities A large number of dedicated-purpose centralized CPU router platforms also are available on the low-end market, such as the ubiquitous Cisco 2500 series Over time, the forwarding performance and cost of such architectures have improved through the introduction of ASICs The advantage of the single CPU approach is the simplicity of software: The majority of the packet-switching intelligence, and certainly all of the route calculation intelligence, is in the single CPU Little danger exists of synchronization problems, such as inconsistent forwarding behavior between line cards In addition, if the CPU is a general-purpose computer, the interaction between the CPU motherboard and the line cards usually conforms to an open bus/operating-system standard, enabling the administrator to choose among multiple vendors for both the line cards and the CPU Single CPU designs also can be very cost-effective because the majority of the complex hardware is focused on the CPU itself The clear disadvantage of single CPU designs, however, is scalability and reliability Involving a single CPU and shared bus in all forwarding decisions is problematic when the number of interfaces or the traffic level becomes very high Moreover, even if using shared memory for packet storage minimizes bus transactions, shared memory access times can become a limiting factor Parallel Switching Designs An obvious improvement to the single CPU design is the introduction of more independent processing power One way to achieve this is to connect clusters of routers by using conventional LAN technology and routing protocols Each router in the cluster is required to perform only switching and route-calculations within the bounds of a single-CPU router design This approach is similar to that used in parallel processing supercomputers Unfortunately, even with modern routing protocols, RISC processors, and high-speed LAns, the use of generic networking components to connect routers does not provide the level of integration needed by most network managers Administrational overhead of such router clusters is high, and the protocols used for general-purpose network connectivity are inefficient and inflexible The necessary refinement may involve a generic high-speed switching fabric, connecting line cards with peer computational capabilities However, such multiprocessor designs are inherently complex to design and debug, as well as expensive to build It has taken some time for vendors to develop such packet switches, and it has taken time for network providers to accumulate the traffic levels that require these sophisticated systems 94 Three Generations of Routers The development of routers can be characterized through three generations The first generation consists of a single CPU controlling relatively unsophisticated line cards through a generalpurpose shared I/O bus Packet queues between line cards are maintained in central or shared memory by the CPU, which coordinates all packet forwarding In the worst case, packets may traverse the I/O bus twice to complete the forwarding process A second-generation switch supplements central switching functions with forwarding intelligence on the line cards The CPU usually places this forwarding information in the line card upon reception of the first packet in a flow Line cards communicate with the CPU and forward packets between one another using a passive shared bus (a bus with no electronics that provides switching intelligence) Third-generation switches replace the shared bus with an active switching fabric, containing specialized packet-switching electronics, which supports the simultaneous transfer of multiple packets This simultaneous transfer of packets circumvents the electrical problems inherent in extremely high-speed shared buses Both the switching fabric and the line cards are controlled by a central processor Figure 5-1 shows the first-, second-, and third-generation packet switches Figure 5-1 First -, Second-, and Third-Generation Packet Switches 95 96 Routers may be input-queued, or both input- and output-queued If the switching fabric is slower than the sum of the interface card speeds, both input and output queuing can occur Otherwise, queuing tends to occur at the outputs only, due to contention for the output interface Queues traditionally were first-in, first-out; with the introduction of service differentiation, however, per class-of-service queuing is increasingly common In other words, routers may have to maintain queues for packets, depending on information in the packet header that describes the priority of the packet, its source or destination, or the user application generating the packet Evolution of the Cisco Switching Algorithms The Cisco core product line has evolved through the generations, from the AGS of 1986 to the GSR of 1997 The next sections examine this evolution in more detail Process Switching The original Cisco AGS was a central CPU packet switch that was similar to host-based routers, except that the range of protocols and interfaces supported was greater, and the operating system was optimized for packet-forwarding functions Interface cards were connected to the CPU motherboard through the Motorola 16 Mbps Multibus, and interface cards maintained the simple packet buffers necessary when there was contention on the internal bus and external media (see Figure 5-2) Figure 5-2 Cisco AGS Architecture All packets were passed over the multibus to the CPU, which performed a routing table lookup, recalculated CRCs, and passed the packet again over the multibus to the appropriate line card Processing switching performance of 2000+ packets per second (pps) was possible with 68000based CPUs 97 Fast Switching Route lookup is an expensive computation; in Cisco routers, a hash-table mechanism is used A more efficient mechanism, called a trie, has since become the method of choice NOTE A trie is simply a method of arranging IP addresses that assists in locating a route with a minimal number of steps Because IP network transactions usually result in a stream of packets, it is a reasonable assumption that after a packet to a particular destination has been switched, another is likely to arrive in the near future By building a cache of recently switched destinations, there are considerable savings in full route table lookups for subsequent packets to the same destinations Moreover, other information that is required for the MAC header rewrite can be stored in the cache, rather than being recalculated This arrangement is called fast switching; it is the default switching mechanism on all Cisco router platforms Fast switching reduced the CPU utilization associated with packet switching and boosted the performance of the AGS to 20,000 pps NOTE Fast switching uses a cache prepopulated by the process switch engine, and operates at the CPU interrupt level Naturally, entries in the fast-switching route cache must be periodically timed-out; otherwise, the cache will grow boundlessly In addition, changes to the IP routing table must invalidate the cache Unfortunately, in an environment with high route churn, such as the Internet, the benefits of route caches are fairly limited This problem spurred the development of Cisco Express Forwarding (see the section, "Cisco Express Forwarding," later in this chapter) NOTE Cisco Express Forwarding (CEF) combines the benefits of caching MAC rewrite information and trie lookup algorithms Fast switching represents an ideal cost/performance compromise for low-end Cisco router architectures, such as those illustrated in Figure 5-3 Both the CPU and the line cards share memory for packet queuing and switching functions, whereas the CPU has dedicated memory for generic processing purposes Fast switching performance of low-end platforms was typically 6,000 pps (2500), 14,000 pps (4000), and 45,000 pps (4700) Figure 5-3 Cisco Low-End Router Architecture 98 TIP Note that security features are not bypassed by fast switching, or by using any of the autonomous or distributed schemes that follow If traffic is administratively forbidden, the fast-switching cache does not become populated This can entail a performance hit for complicated access lists, such as those involving TCP-level conditions However, Netflow switching (covered later in this chapter) addresses these issues Autonomous Switching One advantage of the single CPU architecture is that performance improvements could be obtained merely by increasing the speed of the CPU (which occurred through the CSC and CSC3 motherboards, respectively) However, as the demand for greater throughput increased, it became necessary to increase the bus speed and offload some of the switching from the CPU A new series of bit-slice-processor interface cards, coupled with a 533 Mbps cbus and associated controller, did just that In effect, the route-cache functionality was moved from the CPU to an auxiliary switching processor, so the CPU is interrupted only when a route-cache lookup fails (see Figure 5-4) Figure 5-4 Cisco AGS+ Architecture 99 ... can reach 131.108 .2. 2 52. 255 .25 5 .25 2.0 with the host addresses of 131.108 .2. 253 and 131.108 .2. 254 Similarly, you cannot assign host addresses of 131.108 .2. 2 52 and 131.108 .2. 255 because they are... serial line addresses are 131.108 .2. 0 and 25 5 .25 5 .25 5 .25 2, and the host addresses are 131.108 .2. 1 and 131.108 .2. 2 You cannot assign addresses of 131.108 .2. 0 and 131.108 .2. 3 as the host address because... the network number and the subnet mask by a logical AND operation: 131.108.0.0 25 5 .25 5 .25 5.0 When a logical AND is performed between the network numbers, the third byte is advertised as the network