Chapter 2 Interconnection Networks This chapter is the first of three chapters devoted to the study of network theory. The basic concepts of the interconnection networks are briefly outlined here. The aim is to introduce the terminology and define the properties that characterize an interconnection network. These networks will be described independently from the context in which they could be used, that is either a circuit switch or a packet switch. The classes of rearrangeable networks investigated in Chapter 3 and that of non-blocking networks studied in Chapter 4 will complete the net- work theory. The basic classification of interconnection network with respect to the blocking property is given in Section 2.1 where the basic crossbar network and EGS pattern are introduced before defining classes of equivalences between networks. Networks with full interstage connection patterns are briefly described in Section 2.2, whereas partially connected networks are investi- gated in Section 2.3. In this last section a detailed description is given for two classes of networks, namely banyan networks and sorting networks, that will play a very important role in the building of multistage networks having specific properties in terms of blocking. Section 2.4 reports the proofs of some properties of sorting networks exploited in Section 2.3. 2.1. Basic Network Concepts The study of networks has been pursued in the last decades by researchers operating in two different fields: communication scientists and computer scientists. The former have been studying structures initially referred to as connecting networks for use in switching systems and thus characterized in general by a very large size, say with thousands of inlets and outlets. The latter have been considering structures called interconnection networks for use in multiprocessor systems for the mutual connection of memory and processing units and so characterized by a reasonably small number of inlets and outlets, say at most a few tens. In principle we could say This document was created with FrameMaker 4.0.4 net_th_fund Page 53 Tuesday, November 18, 1997 4:43 pm Switching Theory: Architecture and Performance in Broadband ATM Networks Achille Pattavina Copyright © 1998 John Wiley & Sons Ltd ISBNs: 0-471-96338-0 (Hardback); 0-470-84191-5 (Electronic) 54 Interconnection Networks that connecting networks are characterized by a centralized control that sets up the permuta- tion required, whereas the interconnection networks have been conceived as based on a distributed processing capability enabling the set-up of the permutation in a distributed fash- ion. Interestingly enough the expertise of these two streams of studies have converged into a unique objective: the development of large interconnection networks for switching systems in which a distributed processing capability is available to set up the required permutations. The two main driving forces for this scenario have been the request for switching fabrics capable of carrying aggregate traffic on the order of hundreds of Gbit/s, as typical of a medium-size broadband packet switch, and the tremendous progress achieved in CMOS VLSI technology that makes the distributed processing of interconnection networks feasible also for very large networks. The connection capability of a network is usually expressed by two indices referring to the absence or presence of traffic carried by the network: accessibility and blocking . A network has full accessibility when each inlet can be connected to each outlet when no other I/O connec- tion is established in the network, whereas it has limited accessibility when such property does not hold. Full accessibility is a feature usually required today in all interconnection networks since electronic technology, unlike the old mechanical and electromechanical technology, makes it very easy to be accomplished. On the other hand, the blocking property refers to the network connection capability between idle inlets and outlets in a network with an arbitrary current permutation, that is when the other inlets and outlets are either busy or idle and arbi- trarily connected to each other. An interconnection network, whose taxonomy is shown in Table 2.1, is said to be: • Non-blocking , if an I/O connection between an arbitrary idle inlet and an arbitrary idle out- let can be always established by the network independent of the network state at set-up time. • Blocking , if at least one I/O connection between an arbitrary idle inlet and an arbitrary idle outlet cannot be established by the network owing to internal congestion due to the already established I/O connections. Depending on the technique used by the network to set up connections, non-blocking net- works can be of three different types: • Strict-sense non-blocking (SNB), if the network can always connect each idle inlet to an arbi- trary idle outlet independent of the current network permutation, that is independent of the already established set of I/O connections and of the policy of connection allocation. • Wide-sense non-blocking (WNB), if the network can always connect each idle inlet to an arbitrary idle outlet by preventing blocking network states through a proper policy of allo- cating the connections. • Rearrangeable non-blocking (RNB), if the network can always connect each idle inlet to an arbitrary idle outlet by applying, if necessary, a suitable internal rearrangement of the I/O connections already established. Therefore, only SNB networks are free from blocking states, whereas WNB and RNB net- works are not (see Table 2.1). Blocking states are never entered in WNB networks due to a suitable policy at connection set-up time. Blocking states can be encountered in RNB net- net_th_fund Page 54 Tuesday, November 18, 1997 4:43 pm Basic Network Concepts 55 works during the dynamic network evolution but new connections can always be established by possibly rearranging the connections already set up. It is intuitively clear that a SNB network satisfies at the same time the definition of WNB and RNB networks, but not vice versa. We will not develop here the subject of WNB net- works and will only focus on SNB networks, simply denoted in the following as non-blocking networks , and on RNB networks, referred to as rearrangeable networks . Note that an RNB net- work is also SNB if all the connections are set up and torn down at the same time. We can intuitively assume that the above three non-blocking network types are character- ized by a decreasing cost index, starting from strict-sense non-blocking and ending with rearrangeable non-blocking. Traditionally the cost index of the network has always assumed to be the number of crosspoints in the network, as reasonable in the space-division switching sys- tems of the sixties and seventies. Nowadays such a performance index alone does not characterize the cost of an interconnection network for broadband applications, owing to the extreme degree of integration of the electronic components in a single chip enabled by VLSI technologies. Consider for example the other cost indices: the gates per chip, the chips per board, the crosspoints per board, etc. Nevertheless, since it is very hard to find a unique, even composite, cost index for a network, we will continue to refer to the number of crosspoints in the network as the cost index for the network, by always bearing in mind its limited significance. The reference network is necessarily the well-known crossbar network (Figure 2.1) with N inlets, M outlets.The cost index of such a network, that is the number of its cross- points, referred to a squared structure , is Since each crosspoint is dedicated to a specific I/O connection, the crossbar network is implicitly non-blocking. A squared crossbar network is able to set up an arbitrary network permutation , that is an arbitrary set of N I/O connections; if P denotes the set of all the permutations set up by a generic network, in general , whereas in a crossbar network. Research activities have been undertaken for decades to identify network structures falling into one of the non-blocking classes, but cheaper than the crossbar network. The guideline for Table 2.1. Network taxonomy Network class Network type Network states Non-blocking Strict-sense non-blocking Without blocking states Wide-sense non-blocking With blocking states Rearrangeable non-blocking Blocking Others NM× NM=() CN 2 = NM=() PN!≤ PN!= net_th_fund Page 55 Tuesday, November 18, 1997 4:43 pm 56 Interconnection Networks this research is building multistage networks , with each stage including switching matrices each being a (non-blocking) crossbar network. The general model of an multistage network includes s stages with matrices at stage i , so that , . The matrix of the generic stage i , which is the basic building block of a multistage network, is assumed to be non-blocking (i.e. a crossbar network). The key feature that enables us to classify multistage networks is the type of interconnec- tion pattern between (adjacent) stages. The apparent condition always applies, that is the number of outlets of stage i equals the number of inlets of stage . As we will see later, a different type of interconnection pattern will be considered that cannot be classified according to a single taxonomy. Nevertheless, a specific class of connection pattern can be defined, the extended generalized shuffle (EGS) [Ric93], which includes as subcases a significant number of the patterns we will use in the following. Let the couple represent the generic inlet (outlet) j of the matrix k of the generic stage i , with and . The EGS pattern is such that the outlet of matrix , that is outlet , is connected to inlet with In other words, we connect the outlets of stage i starting from outlet (0,1) sequentially to the inlets of stage as An example is represented in Figure 2.2 for . A network built out of stages intercon- nected by means of EGS patterns is said to be an EGS network . Figure 2.1. Crossbar network 0 1 N-2 N-1 0 1 M-1 NM× r i n i m i × i 1 … s,,=() Nn 1 r 1 = Mm s r s = n i m i × m i r i n i 1+ r i 1+ = 0 ˜ is1–≤≤() i 1+ j i k i ,() j i 0 … n i 1–,,= j i 0 … m i 1–,,=() k i 1 … r i ,,= j i k i j i k i ,() j i 1+ k i 1+ ,() j i 1+ int m i k i 1–()j i + r i 1+ = k i 1+ m i k i 1–()j i +[] modr i 1+ 1+= r i m i i 1+ 01,()…0 r i 1+ ,()11,()…1 r i 1+ ,()…n i 1+ 1– 1,()…n i 1+ 1– r i 1+ ,(),, , ,, ,, ,, m i r i 1+ < net_th_fund Page 56 Tuesday, November 18, 1997 4:43 pm Basic Network Concepts 57 Multistage networks including s stages will now be described according to the following type of interstage pattern configuration: • full connection (FC), if each matrix in stage is connected to all the matrices in stages and ; • partial connection (PC), if each matrix in stage is not connected to all the matrices in stages and . It is worth noting that an EGS network with is a FC network, whereas it is a PC network if . 2.1.1. Equivalence between networks We refer now to a squared network and number the network inlets and outlets 0 through sequentially from the top to the bottom. If the network includes more than one stage and the generic matrix includes b outlets, each outlet matrix being labelled from 0 to , an underlying graph can be identified for the network: each matrix is mapped onto a graph node and each interstage link onto a graph edge. Graph nodes and edges are not labelled, so that network inlets and outlets need not be mapped onto edges, since they carry no information. Therefore the most external graph elements are the nodes representing the matrices of the first and last network stage. Figure 2.2. EGS interconnection pattern 1 n i -1 0 r i 2 r i -1 r i+1 r i+1 -1 m i m i +1 1 2 n i -1 0 m i+1 -1 0 m i+1 -1 0 ii+1 i i 2 … s 1–,,=() i 1– i 1+ i i 2 … s 1–,,=() i 1– i 1+ m i r i 1+ ≥ i 1 … s 1–,,=() m i r i 1+ < i 1 … s 1–,,=() NN× N 1– b 1– net_th_fund Page 57 Tuesday, November 18, 1997 4:43 pm 58 Interconnection Networks Two graphs A and B are said to be isomorphic if, after relabelling the nodes of graph A with the node labels of graph B, graph A can be made identical to graph B by moving its nodes and hence the attached edges. The mapping so established between nodes in the same position in the two original graphs expresses the “graph isomorphism”. A network is a more complex structure than a graph, since an I/O path is in general described not only by a sequence of nodes (matrices) but also by means of a series of labels each identifying the outlet of a matrix (it will be clear in the following the importance of such a more complete path description for the routing of messages within the network). Therefore an isomorphism between networks can also be defined that now takes into account the output matrix labelling. Two networks A ad B are said to be isomorphic if, after relabelling the inlets, outlets and the matrices of network A with the respective labels of network B, network A can be made identical to network B by moving its matrices, and correspondingly its attached links. It is worth noting that relabelling the inlets and outlets of network A means adding a proper inlet and outlet permutation to network A. Note that the network isomorphism requires the modi- fied network A topology to have the same matrix output labels as network B for matrices in the same position and is therefore a label-preserving isomorphism. The mapping so established between inlets, outlets and matrices in these two networks expresses the “network isomor- phism”. In practice, since the external permutations to be added are arbitrary, network isomorphism can be proven by just moving the matrices, together with the attached links, so that the topologies of the two networks between the first and last stage are made identical. By relying on the network properties defined in [Kru86], three kinds of relations between networks can now be stated: • Isomorphism: two networks are isomorphic if a label-preserving isomorphism holds between them. • Topological equivalence: two networks are topologically equivalent if an isomorphism holds between the underlying graphs of the two networks. • Functional equivalence: two networks A and B are functionally equivalent if they perform the same permutations, that is if . Two isomorphic networks are also topologically equivalent, whereas the converse is not always true. In general two isomorphic or topologically equivalent networks are not functionally equivalent. Nevertheless, if the two networks (isomorphic or not, topologically equivalent or not) are also non-blocking, they must also be functionally equivalent since both of them per- form the same permutations (all the permutations). Note that the same number of network components are required in two isomorphic networks, not in two functionally equiv- alent networks. For example, consider the two networks A and B of Figure 2.3: they are topologically equivalent since their underlying graph is the same (it is shown in the same Figure 2.3). Nev- ertheless, they are not isomorphic since the above-defined mapping showing a label- preserving isomorphism between A and B cannot be found. In fact, if matrices in network A are moved, the two networks A' and A" of Figure 2.3 can be obtained, which are close to B but not the same. A' has nodes with the same label but the links outgoing from H exchanged compared to the analogous outgoing from Y, whereas A" has the same topology as B but with P A P B = N! net_th_fund Page 58 Tuesday, November 18, 1997 4:43 pm Basic Network Concepts 59 the labels in H exchanged compared to Y. On the other hand both networks A' and A" are iso- morphic to network B in Figure 2.4 and the required mappings are also given in the figure. Note that the two networks A and B of both examples above are not functionally equiva- lent, since, e.g., the two connections (0,0) and (1,1) can belong to the same permutation in network A, whereas this is not true for network B (remember that inlets and outlets are num- bered 0 through 3 top to bottom of the network). The example of Figure 2.5 shows two isomorphic and functionally equivalent networks (it will be shown in Section 3.2.1.1 that both networks are rearrangeable). A useful tool for the analysis of multistage networks is the channel graph. A channel graph is associated with each inlet/outlet pair and is given by the sequence of network elements that are crossed to reach the selected outlet from the selected input. In the channel graph, matrices Figure 2.3. Example of topologically equivalent non-isomorphic networks Figure 2.4. Example of isomorphic networks Graph of A and B H a b c d I J e f g h 0 1 0 1 0 1 A s t u v w x y z X Y 0 1 0 1 Z 0 1 B c d a b f g h e I H 0 1 0 1 J 0 1 A' c d a b f g h e I H 0 1 1 0 J 0 1 A" H a b c d I J e f g h 0 1 0 1 0 1 A' s t u v w x y z X Y 0 1 0 1 Z 0 1 B a → u b → v c → s d → t H → Y I → X J → Z e → z f → w g → x h → y H a b c d I J e f g h 1 0 0 1 0 1 A'' net_th_fund Page 59 Tuesday, November 18, 1997 4:43 pm 60 Interconnection Networks are represented as nodes and interstage links as edges. Therefore the number of I/O paths in the channel graph represents the number of different modes in which the network outlet can be reached from the network inlet. Two I/O paths in the channel graph represent two I/O network connections differing in at least one of the crossed matrices. A network in which a single channel graph is associated with all the inlet/outlet pairs is a regular network. In a regular channel graph all the nodes belonging to the same stage have the same number of incoming edges and the same number of outgoing edges. Two isomorphic networks have the same chan- nel graph. The channel graphs associated with the two isomorphic networks of Figure 2.4 are shown in Figure 2.6. In particular, the graph of Figure 2.6a is associated with the inlet/outlet pairs terminating on outlets e or f in network A (y or z in network B), whereas the graph of Figure 2.6b represents the I/O path leading to the outlets g or h in network A (w or x in net- work B). In fact three matrices are crossed in the former case engaging either of the two middle-stage matrices, while a single path connects the inlet to the outlet, crossing only two matrices in the latter case. 2.1.2. Crossbar network based on splitters and combiners In general it is worth examining how a crossbar network can be built by means of smaller building blocks relying on the use of special asymmetric connection elements called splitters and combiners, whose size is respectively and with a cost index . Note that a splitter, as well as a combiner, is able to set up one connection at a time. The crossbar tree network (Figure 2.7) is an interconnection network functionally Figure 2.5. Example of isomorphic and functionally equivalent networks Figure 2.6. Channel graphs of the networks in Figure 2.4 a b c d e f g h H I 0 1 0 1 J K 0 1 0 1 L 0 1 A a' b' c' d' w x y z H' I' 0 1 0 1 J' K' 1 0 1 0 L' 0 1 B a → a' b → b' c → c' d → d' e → y f → z g → w h → x H → H' I → I' J → J' K → K' L → L' (a) (b) 1 K× K 1× 1 K⋅ K 1⋅ K == net_th_fund Page 60 Tuesday, November 18, 1997 4:43 pm Basic Network Concepts 61 equivalent to the crossbar network: it includes N splitters and N combiners interconnected by means of an EGS pattern and its cost is The crossbar tree can be built also by using multistage splitting and combining structures based on the use of elementary splitters and combiners, as shown in Figure 2.8 for . The cost index of such structure, referred to as a crossbar binary tree network, is It is interesting to note how the basic crossbar tree network of Figure 2.7 can be built using smaller splitters and combiners by means of a central switching stage. Our aim is a central stage with the smallest cost that still guarantees full input/output accessibility, thus suggesting a set of crossbar matrices each with the smallest possible size. In general if we have an expansion stage with size with , each inlet has access to K switching matrices from which all the N outlets must be reached, so that each switching matrix must have a number of outlets (and inlets) at least equal to (each matrix in the switching stage is connected to splitters and combiners with at most one link). By adopting for the two inter- stage connections the EGS pattern, which provides a cyclic connection to the elements of the following stage, it follows that such matrix size is sufficient to give full accessibility, as is shown in Figure 2.9 for and . Since the number of these matrices is Figure 2.7. Crossbar tree 1 N× N 1× C 2N 2 = 1 0 N-1 1 0 N-1 1 x NN x 1 12× 21× N 8= C 4N 2 i i 0= N 2 log 1– ∑ 4NN 1–()4N 2 4 N –=== 1 K× K 2 k k 1 … N 2 log 1–,,=()= NK⁄ N 8= K 24,= KN N K K 2 = net_th_fund Page 61 Tuesday, November 18, 1997 4:43 pm 62 Interconnection Networks the cost function of such a network is given by Figure 2.8. Crossbar binary tree Figure 2.9. Crossbar tree with one switching stage 1 0 7 1 0 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 C K 2 N K   2 2NK+ N 2 2N K +== net_th_fund Page 62 Tuesday, November 18, 1997 4:43 pm