1 1 Network Traffic Analysis Using tcpdump Judy Novak Judy Novak Johns Hopkins University Applied Physics Laboratory jhnovak@ix.netcom.com Writing tcpdump Filters All material Copyright  Novak, 2000, 2001. All rights reserved. 2 2 Writing tcpdump Filters • Introduction to tcpdump • Writing tcpdump Filters • Examination of Datagram Fields • Beginning Analysis • Real World Examples • Step by Step Analysis This page intentionally left blank. 3 3 Objectives • Review the foundations to understand and create tcpdump filters including: • tcpdump filter format • Review of bit/byte theory • Review of binary/hexadecimal numbering systems • Review of bit masking • Learning to formulate tcpdump filters • Review of tcpdump output tcpdump filters are necessary to selectively gather/read records of network traffic. While this section may be somewhat difficult to understand especially if you haven’t been exposed to this theory before, it is more than just an academic exercise. In order to comprehend network traffic at its most visceral level, you will have to understand tcpdump filters. Also, familiarity with tcpdump filters is necessary if you want to process tcpdump files for some trait. For instance, if you wanted to identify the beginning of a TCP connection, you would search for traffic with the SYN bit alone set. 4 4 Foundations For Understanding tcpdump Filters • Specify item of interest for record selection • Any field in the IP datagram • Examples: header length or TCP flags • Variables for more commonly used fields: • Examples: “port” or “host” • Less common fields: • Identify protocol • Identify byte displacement • Examples: ip[0], tcp[13] tcpdump filters need to specify an item of interest, a field in the IP datagram for record selection. Such items can be part of the IP header such as the IP header length, the TCP header such as TCP flags, the UDP header such as the destination port, or the ICMP message such as the message type. tcpdump provides a special name for each type of header. Much as you would expect, ip is used to denote a field in the IP header or data portion of the IP datagram, tcp for a field in the TCP header or segment, udp for the UDP header or UDP datagram, and ICMP for the ICMP message. For instance, ip[0] would indicate the first byte offset of the IP datagram which happens to be part of the IP header (remember counting starts at 0). tcp[13] would be the 13th byte offset into the TCP segment which is also part of the TCP header, and icmp[0] would be the first byte offset of the ICMP message which is the ICMP message type. Sample filters and reference material are found in: • tcpdump man pages 5 5 Specifying Fields 0 15 16 31 20 bytes 4-bit 4-bit IP 8-bit TOS 16-bit total length (in bytes) version header length 16-bit IP identification number 3-bit flags 13-bit fragment offset 8-bit time to live (TTL) 8-bit protocol 16-bit header checksum 32-bit source IP address 32-bit destination IP address ip[1] src host protocol[displacement] macro Looking at the IP header as an example, we learn two ways to specify different fields. The easier way to specify a field of interest is by using a tcpdump macro. Not all fields have these macros. The source IP can be specified by combining two macros “src” and “host” to identify the field. But, if we want to look at the type of service field, we have to identify a protocol in which the field is found (IP because this is in the IP header) and a displacement in bytes (1) offset in the protocol. What are some of the more common macros used in filters? host select the record if either the source or destination host matches this IP net select the record if either the source or destination subnet matches This is useful if there are several IP’s from the same subnet of interest to you port select the record if either the source or destination port matches src host select the record if the source host matches dst host select the record if the destination host matches src net select the record if the source subnet matches dst net select the record if the destination subnet matches src port select the record if the source port matches dst port select the record if the destination port matches icmp select the record if the protocol field ip[9] has a value of 1 tcp select the record if the protocol field ip[9] has a value of 6 udp select the record if the protocol field ip[9] has a decimal value of 17 6 6 The tcpdump Filter Format • The two different formats for a tcpdump filter are: • <protocol header> [offset: length] <relation> <value> ip[9] = 1 tcp[2:2] < 20 udp[4:2] != 0 icmp[0] = 8 • <variable> <value> port 23 dst host 1.2.3.4 src net 0 The first filter ip[9] = 1 selects any record with the IP protocol of 1 (ICMP). The second filter tcp[2:2] selects any record with a TCP destination port less than 20. The third filter udp[4:2] selects any UDP record with a non-zero UDP length. The fourth filter selects any record with an ICMP message type of 8, an ICMP echo request. The first variable filter selects any record with source/destination port of 23 (telnet). The second variable filter selects any record with destination host 1.2.3.4. The third variable filter selects any record with a source subnet of 0.x.x.x. 7 7 Bit/Byte Fundamentals • A byte is an 8 bit field • It is possible to denote a span of bytes, i.e. udp[0:2] • Smallest precision that the tcpdump “language” offers is a byte • How do you reference bits within a byte? • Bit masking First 4 bytes (bytes 0 - 3) of the IP header: BYTE 0 1 2 3 4 bit 4 bit 8 bit TOS 16 bit IP total length version length The bit is the smallest unit that can be represented by a computer - it can have a value of either 0 or 1. A byte is composed of 8 bits. Byte counting begins at byte 0; all successive bytes fall on these 8 bit boundaries. udp[0:2] specifies the byte in the UDP datagram beginning at byte 0 for a length of two bytes. Bit masking or using a combination of boolean arithmetic and binary/hexadecimal values will help “isolate” bits. 8 8 Decimal/Binary Representations Base 10 Arithmetic - Decimal 2 6 5 10 2 10 1 10 0 Base 2 Arithmetic - Binary 2 7 2 6 2 5 2 4 2 3 2 2 2 1 2 0 1 0 0 0 0 0 0 1 128 64 32 16 8 4 2 1 = 2x100 + 6x10 + 5x1 = 265 = 1x128 + 1x1 = 129 Because decimal is our native number system, we really don’t have to do any conversions to understand the value of a number. But, if you examine the number, you realize that a digit has value based on its placement in the number. The digits that are least significant (to the right) have less value and those that are most significant (to the left) have the most value. Each digit is represented by an increasing power of the native base or base 10. The same theory applies when we are dealing with binary or base 2. Instead of using exponents of 10, we use exponents of 2 to figure out the decimal representation of the number. Also, because we are talking in terms of a byte, we use 8 bits or binary digits to represent a byte. So, we see above how we convert the binary number of 10000001 to a decimal 129. 9 9 Binary/Hex Conversion Base 2 Arithmetic - Binary 2 7 2 6 2 5 2 4 2 3 2 2 2 1 2 0 1 0 0 0 0 0 0 1 128 64 32 16 8 4 2 1 = 1x128 + 1x1 = 129 . Base 16 Arithmetic - Hexadecimal 2 3 2 2 2 1 2 0 2 3 2 2 2 1 2 0 1 0 0 0 0 0 0 1 4 binary bits represent one hex character. 1000 0001 binary is 81 hex. To denote hex we use the 0x prefix - 0x81. 81 hex = 8x16 1 + 1x16 0 = 129 If you consider a byte as two hexadecimal characters, each character will be 4 bits long. So 16 different hex values can be represented - if all bits of a 4-bit chunk (nibble) are turned on or set to 1 the maximum value will be 15 (8 + 4 + 2 + 1). Counting in hex goes from 0 to 9, 10 = a, 11 = b, 12 = c, 13 = d, 14 =e, 15 = f. The leftmost bits are called the high-order bits - they have the most value, whereas the rightmost bits are referred to as the low-order bits. The same holds true for bytes; the left most are known as high-order bytes and right most are known as low-order bytes. Remember from arithmetic that any number with an exponent of 0 is 1. Terminology: Byte = 8 bits Nibble = 4 bits Hex char = 4 bits Word = 32 bits 10 10 Hexadecimal Representation 2 3 2 2 2 1 2 0 2 3 2 2 2 1 2 0 (Hex) 0 0 0 0 = 0 1 0 0 0 = 8 0 0 0 1 = 1 1 0 0 1 = 9 0 0 1 0 = 2 1 0 1 0 = 10 (a) 0 0 1 1 = 3 1 0 1 1 = 11 (b) 0 1 0 0 = 4 1 1 0 0 = 12 (c) 0 1 0 1 = 5 1 1 0 1 = 13 (d) 0 1 1 0 = 6 1 1 1 0 = 14 (e) 0 1 1 1 = 7 1 1 1 1 = 15 (f) When representing hexadecimal, we have a numbering system that goes from 1 to 15. The problem comes in representing values above 9 in a different scheme so that we can differentiate decimal and hexadecimal. A value of 10 decimal is a different value than 10 hexadecimal. A value of 10 hexadecimal has a value of 16 in decimal. So, when we get to values above 9, we use letters to represent 10 – 15 as you can see in the second column above. The letters in parentheses are the hexadecimal representations of the numbers in decimal. [...]... More Advanced Filters • More versatile by combining individual filters with the and, or, not operators (tcp and (tcp[13] & 0x0f != 0) and not port 25 and not port 20) or (ip and not (tcp or igrp or dst port 520)) 27 You can combine individual filters to make a more complex filter using the and, or and not operators 27 Wrap-up of Writing tcpdump Filters • Creating and understanding tcpdump filters: • The... filters: • The filter syntax • Concepts of bits and bytes • Binary/hex arithmetic • Bit masking to select bits that don’t fall on byte boundaries • Synthesizing simple filters to create more advanced filters 28 In conclusion, tcpdump filters are very straightforward when you are examining a field or fields that fall on byte boundaries In fact, there are even some macros that can be used to identify... 34 Section Questions (cont.) 11) A tcpdump filter of tcp[13] = 0x12 looks for what flags to be set: a) ACK and SYN b) RESET and SYN c) PUSH and SYN d) URG and SYN 12) A tcpdump filter of tcp[13] & 0x08 != 0 must have what flag set: URG ACK RESET PUSH a) b) c) d) 35 This page intentionally left blank 35 Section Questions (cont.) 13) An ICMP echo request can be found using which filter: a) icmp[0] = 0... When we examine tcpdump data, we may be interested in TCP data that has a particular flag set For instance, we may be interested in initial connections only, in which case the SYN bit alone is set So, in this case we need to be able to mask the other bits so that we check if the SYN bit only is highlighted Once this mask is superimposed over the tcp[13] byte, we will select corresponding tcpdump records... the protocol header layouts in the Reference section at the end of this course notebook to answer the following questions 1) tcpdump filter to see if the destination address is of the format x.x.x.0 (ie 192.168.5.0): a) ip[15] = 0 b) ip[19] = 0 c) tcp[19] = 0 d) udp[15] = 0 2) tcpdump filter to see if the destination address if of the format x.x.x.255 (ie 192.168.5.255): a) ip[15] = 0x255 b) ip[19]... ip[19] = 0x255 c) ip[19] = 0xff d) ip[15] = 0xff 30 This page intentionally left blank 30 Section Questions (cont.) 3) a) b) c) d) tcpdump filter to see if the UDP source port number is less than 20: udp[0] < 20 udp[0:2] < 20 udp[2:2] < 20 udp[2:1] < 20 4) a) b) c) d) tcpdump filter to find records to traceroute ports (UDP destination ports 33000-33999): udp[2:1] >= 33000 and udp[2:2] = 33000 and udp[2:2] 5 If this filter were included in the tcpdump statement with the proper notation or in a file and pointed to by the tcpdump option -F, all records read that had an IP header length of greater than 5 would be selected What would the mask be to preserve the high order 4 bits (the . tcpdump Filters All material Copyright  Novak, 2000, 2001. All rights reserved. 2 2 Writing tcpdump Filters • Introduction to tcpdump • Writing tcpdump Filters • Examination. 1 1 Network Traffic Analysis Using tcpdump Judy Novak Judy Novak Johns Hopkins University Applied Physics Laboratory jhnovak@ix.netcom.com Writing tcpdump