8 Network Programming The need for computers and devices to communicate across a network is one of the key ingredients of enterprise programming. In its relentless goal to simplify programming, the .NET Framework includes a slew of new networking classes that are logical, efficient, and consistent. The only drawback to networking with .NET is that no single dominant model exists. In this chapter, you’ll learn how to manage network interaction using sockets (recipes 8.8 to 8.11), but you won’t learn about two higher-level distributed programming frameworks— Web Services and .NET Remoting—which have their own dedicated chapters later in this book. Typically, socket-based network programming is ideal for closed systems that don’t require interoperability, where developers want to have complete flexibility to tailor communication and control the data before it hits the wire. Of course, this chapter doesn’t concentrate exclusively on socket programming. You’ll also learn about Web interaction, such as downloading a Web page from the Internet (recipe 8.5) or a single piece of information (recipe 8.6). You’ll also learn how to retrieve Web connectivity information for the current computer, look up Internet Protocol (IP) addresses and domain names, and ping another computer to gauge its response time. At the end of this chapter, two recipes (8.13 and 8.14) show how you can build on the Transmission Control Protocol (TCP) classes included with .NET to work with higher- level protocols such as Post Office Protocol 3 (POP3) for e-mail and File Transfer Protocol (FTP) for transferring files. 8.1 Get Web Connectivity Information for the Current Computer Problem You need to determine programmatically if the current computer can connect to the Internet. Solution Use the Microsoft Windows API function InternetGetConnectedState. Discussion The InternetGetConnectedState function returns True if the current computer is configured to access the Internet. It also returns a dwFlags parameter that specifies the type of connection using one (or more) of a series of constants. The following Console application defines the InternetGetConnectedState function and uses it to test the current computer’s connectivity: Public Module GetInternetState ’ Declare the API function. Private Declare Function InternetGetConnectedState Lib "wininet" _ (ByRef dwFlags As Long, ByVal dwReserved As Long) As Long ’ Define the possible types of connections. Private Enum ConnectStates LAN = &H2 Modem = &H1 Proxy = &H4 Offline = &H20 Configured = &H40 RasInstalled = &H10 End Enum Public Sub Main() ’ Get the connected status. Dim dwFlags As Long Dim Connected As Boolean = _ (InternetGetConnectedState(dwFlags, 0&) <> 0) If Connected Then Console.WriteLine("This computer is connected to the Interne t.") ’ Display all connection flags. Console.Write("Connection flags:") Dim ConnectionType As ConnectStates For Each ConnectionType In _ System.Enum.GetValues(GetType(ConnectStates)) If (ConnectionType And dwFlags) = ConnectionType Then Console.Write(" " & ConnectionType.ToString()) End If Next End If Console.ReadLine() End Sub End Module A sample output is shown here: This computer is connected to the Internet. Connection flags: LAN Notice that the InternetGetConnectedState reflects how the computer is configured. It doesn’t reflect whether the computer is configured correctly (in other words, whether the Internet connection is actually working). 8.2 Get the IP Address of the Current Computer Problem You want to retrieve the IP address of the current computer, perhaps to use later in networking code. Solution Use the System.Net.Dns class, which provides shared GetHostName and GetHostByName methods. Discussion The Dns class provides domain name resolution services. You can invoke its GetHostName to retrieve the host name for the current computer. You can then translate the host name into an IP address using GetHostByName. Dim HostName As String Dim IPAddress As String ‘ Look up the host name and IP address. HostName = System.Net.Dns.GetHostName() IPAddress = System.Net.Dns.GetHostByName(HostName).AddressList(0).ToStri ng() Console.WriteLine("Host name:" & HostName) Console.WriteLine("IP address:" & IPAddress) Be aware that the GetHostByName method returns a list of usable IP addresses. In most cases, this address list will contain only one entry. If you run this code, you’ll see something like this: Host name: fariamat IP address: 24.114.131.70 8.3 Look Up a Host Name for an IP Address Problem You want to determine the IP address for a computer based on its domain name by performing a Domain Name System (DNS) query. Solution Use the System.Net.Dns class, which wraps this functionality in the GetHostByName method. Discussion On the Web, publicly accessible IP addresses are often mapped to host names that are easier to remember using a network of DNS servers, which are a fundamental part of the Internet backbone. To perform a DNS lookup, the computer might contact its cache or a DNS sever (which might in turn forward the request to a DNS root server). This entire process is transparent if you use the System.Net.Dns class, which allows you to retrieve the IP address for a host name by calling GetHostByName. Here’s how you might retrieve the list of IP addresses mapped to http://www.yahoo.com: Dim IP As System.Net.IPAddress For Each IP In System.Net.Dns.GetHostByName("www.yahoo.com").AddressList Console.WriteLine(IP.AddressFamily.ToString()) Console.WriteLine(IP.ToString()) Next 8.4 Ping an IP Address Problem You want to check if a computer is online and gauge its response time. Solution Send a ping message. Discussion A ping message contacts a device at a specific IP address, sends a test message, and requests that the remote device respond by echoing back the packet. You can measure the time taken for a ping response to be received to gauge the connection latency between two computers. Despite the simplicity of ping messages compared to other types of network communication, implementing a ping utility in .NET requires a significant amount of complex low-level networking code. The .NET class library doesn’t have a prebuilt solution—instead, you must use raw sockets. However, at least one developer has solved the ping problem. Lance Olson, a developer at Microsoft, has provided C# code that allows you to ping a host by name or IP address and measure the milliseconds taken for a response. This code has been adapted into a PingUtility component, which is available with the code in this book’s sample files. To use the ping utility, you must first add a reference to the PingUtility.dll assembly. You can then use the shared Pinger.GetPingTime method with an IP address or domain name. The GetPingTime method returns the number of milliseconds that elapse before a response is received. Console.WriteLine("Milliseconds to contact www.yahoo.com: " & _ PingUtility.Pinger.GetPingTime("www.yahoo.com")) Console.WriteLine("Milliseconds to contact www.seti.org: " & _ PingUtility.Pinger.GetPingTime("www.seti.org")) Console.WriteLine("Milliseconds to contact the local computer: " & _ PingUtility.Pinger.GetPingTime("127.0.0.1")) The ping test allows you to verify that other computers are online. It can also be useful if your application needs to evaluate several different remote computers that provide the same content and to determine which one will offer the lowest network latency for communication. NOTE A ping attempt might not succeed if a firewall forbids it. For example, many heavily trafficked sites ignore ping requests because they’re wary of being swamped by a flood of simultaneous pings that will tie up the server (in essence, a denial of service attack). 8.5 Download a File Using HTTP Problem You want to retrieve a file from the Web. Solution Use the HttpWebRequest class to create your request, the WebResponse class to retrieve the response from the Web server, and some form of reader (typically a StreamReader for HTML or text data or a BinaryReader for a binary file) to parse the response data. Discussion Downloading a file from the Web takes the following four basic steps: 1. Use the shared Create method of the System.Net.WebRequest class to specify the page you want. This method returns a WebRequest-derived object, depending on the type of Uniform Resource Identifier (URI) you use. For example, if you use an HTTP URI (with the scheme http://), it will create an HttpWebRequest instance. If you use a file system URI (with the scheme file://), it will create a FileWebRequest instance. 2. Use the GetResponse method of the HttpWebRequest object to return a WebResponse object for the page. 3. Create a StreamReader or BinaryReader for the WebResponse stream. 4. Perform any steps you need to with the stream, such as writing it to a file. The following code is a test application that retrieves and displays the HTML of a Web page. For it to work, you must import both the System.Net and the System.IO namespaces. Public Module DownloadTest Public Sub Main() Dim Url As String = "http://www.prosetech.com/index.html" ’ Create the request. Dim PageRequest As HttpWebRequest = _ CType(WebRequest.Create(Url), HttpWebRequest) ’ Get the response. ’ This takes the most significant amount of time, particularly ’ if the file is large, because the whole response is retrieved. Dim PageResponse As WebResponse = PageRequest.GetResponse() Console.WriteLine("Response received.") ’ Read the response stream. Dim r As New StreamReader(PageResponse.GetResponseStream()) Dim Page As String = r.ReadToEnd() r.Close() ’ Display the retrieved data. Console.Write(Page) Console.ReadLine() End Sub End Module To deal efficiently with large files that need to be downloaded from the Web, you might want to use asynchronous techniques, as described in Chapter 7. You can also use the WebRequest.BeginGetResponse, which doesn’t block your code and calls a callback procedure when the response has been retrieved. 8.6 Retrieve a Single Piece of Information from a Web Page Problem You want to extract a single piece of information from a Web page. Solution Use the WebResponse class to retrieve the stream, copy it to a string, and use a regular expression. Discussion You can extract information from a Web stream in several ways. You could read through the stream, use methods of the String class such as IndexOf, or apply a regular expression. The latter of these—using a regular expression—is the most flexible and powerful. The first step is to create a regular expression that filters out the information you need. Recipe 1.17 provides several examples and a reference to basic regular expression syntax. For example, most Web pages include a text title that is stored in a <title></title> tag. To retrieve this piece of information, you use the following regular expression: <title>(?<match>.*?)</title> This expression retrieves all the text between the opening and closing <title> tag and places it in a named group called match. The following sample code uses this regular expression to retrieve the title from a URL the user enters. It requires three namespace imports: System.Net, System.IO, and System.Text. RegularExpressions. Public Module ExtractTitleTest Public Sub Main() Console.WriteLine("Enter a URL, and press Enter.") Console.Write(">") Dim Url As String = Console.ReadLine() Dim Page As String Try ’ Create the request. Dim PageRequest As HttpWebRequest = _ CType(WebRequest.Create(Url), HttpWebRequest) ’ Get the response. ’ This takes the most significant amount of time, particular ly ’ if the file is large, because the whole response is retrie ved. Dim PageResponse As WebResponse = PageRequest.GetResponse() Console.WriteLine("Response received.") ’ Read the response stream. Dim r As New StreamReader(PageResponse.GetResponseStream()) Page = r.ReadToEnd() r.Close() Catch Err As Exception Console.WriteLine(Err.ToString()) Return End Try ’ Define the regular expression. Dim TitlePattern As String = "<title>(?<match>.*?)</title>" Dim TitleRegex As New Regex(TitlePattern, _ RegexOptions.IgnoreCase Or RegexOptions.Singleline) ’ Find the title. Dim TitleMatch As Match = TitleRegex.Match(Page) ’ Display the title. If TitleMatch.Success Then Console.WriteLine("Found title: " & _ TitleMatch.Groups("match").Value) End If Console.ReadLine() End Sub End Module Here’s the output for a test run that retrieves the title from the Yahoo! search engine: Enter a URL, and press Enter. >http://yahoo.com Response received. Found title: Yahoo! If the Web page is extremely large, this approach might not be efficient because the entire stream is copied to a string in memory. Another option is to read through the stream character-by-character and try to build up a match to a search pattern. This approach requires more custom code and is demonstrated in detail with text searching in a file in recipe 5.8. NOTE Screen scraping solutions such as this one can be quite brittle. If the user interface for the Web site changes and the expected pattern is altered, you’ll no longer be able to extract the information you need. If you have control over the Web site, you can implement a much more robust approach using a Web service to return the desired information. Web services also support the full set of basic data types, which prevents another possible source of errors. 8.7 Find All Links in a Web Page Problem You want to retrieve all the hyperlinks in a Web page (perhaps because you want to download those pages also). Solution Retrieve the page using WebResponse, and use a regular expression to search for URIs. Discussion Retrieving links in a Web page is conceptually quite easy but often more difficult in practice. The problem is that Web pages follow a semi-standardized format and tolerate a great deal of variance. For example, a hyperlink can be added in the href attribute of an anchor, the onclick attribute of a JavaScript element such as a button, and so on. The URI itself could be relative (in which case it needs to be interpreted relative to the current page), fully qualified (in which case it can have one of countless schemes, including http:// or file:// or mailto://), or it might just be a bookmark (an anchor tag with an href that starts with the # character). Dealing with these myriad possibilities isn’t easy. The first step is to craft a suitable regular expression. In this case, we’ll consider only the links that are provided in the href attribute of an anchor tag. Here’s one regular expression that retrieves all href values from a Web page: href\s*=\s*(?:"(?<match>[^"]*)"|(?<match>\S+)) Another option is to retrieve absolute paths only. The following line of code is a slightly less complicated regular expression that matches href values that start with http://. href\s*=\s*"(?<match>http://.*?)" The following sample application uses the first option. It then manually checks the retrieved URIs to see if they are bookmarks (in which case they are discarded) and to determine if they’re relative or absolute. If the bookmarks are relative paths, the System.Uri class is used with the current page Uri to transform them into fully qualified paths. Public Module ExtractURITest Public Sub Main() Console.WriteLine("Enter a URL, and press Enter.") Console.Write(">") Dim Url As String = Console.ReadLine() Dim BaseUri As Uri Dim Page As String Try BaseUri = New Uri(Url) ’ Create the request. Dim PageRequest As HttpWebRequest = _ CType(WebRequest.Create(Url), HttpWebRequest) ’ Get the response. ’ This takes the most significant amount of time, particular ly ’ if the file is large, because the whole response is retrie ved. Dim PageResponse As WebResponse = PageRequest.GetResponse() Console.WriteLine("Response received.") ’ Read the response stream. Dim r As New StreamReader(PageResponse.GetResponseStream()) Page = r.ReadToEnd() r.Close() Catch Err As Exception Console.WriteLine(Err.ToString()) Console.ReadLine() Return End Try ’ Define the regular expression. Dim HrefPattern As String HrefPattern = "href\s*=\s*(?:""(?<match>[^""]*)""|(? <match>\S+))" Dim HrefRegex As New Regex(HrefPattern, _ RegexOptions.IgnoreCase Or RegexOptions.Compiled) ’ Find and display all the href matches. Dim HrefMatch As Match = HrefRegex.Match(Page) Do While HrefMatch.Success Dim Link As String = HrefMatch.Groups(1).Value If Link.Substring(0, 1) = "#" Then ’ Ignore this match, it was just a bookmark. Else ’ Attempt to determine if this is a fully-qualified link ’ by comparing it against some known schemes. Dim Absolute As Boolean = False If Link.Length > 8 Then Dim Scheme As String Scheme = Uri.UriSchemeHttp & "://" If Link.Substring(0, Scheme.Length) = Scheme Then _ Absolute = True Scheme = Uri.UriSchemeHttps & "://" If Link.Substring(0, Scheme.Length) = Scheme Then _ Absolute = True Scheme = Uri.UriSchemeFile & "://" If Link.Substring(0, Scheme.Length) = Scheme Then _ Absolute = True End If ’ (You could compare it against additional schemes here. ) If Absolute Then Console.WriteLine(Link) Else Console.WriteLine(New Uri(BaseUri, Link).ToString()) End If End If HrefMatch = HrefMatch.NextMatch() Loop Console.ReadLine() End Sub End Module This code investigates each URI by comparing it against a few common schemes. Another approach would be to try to instantiate a new System.Uri instance using the [...]... Message Number: 1 Size: 1 380 Return-Path: Delivered-To: somewhere.com%someone@somewhere.com Received: (cpmta 15300 invoked from network) ; 5 Dec 2002 06:57:13 - 080 0 Received: from 66. 185 .86 .71 (HELO fep01mail.bloor.is.net.cable.rogers.com) by smtp.c000.snv.cp.net (209.2 28. 32 .87 ) with SMTP; 5 Dec 2002 06:57:13 - 080 0 X-Received: 5 Dec 2002 14:57:13 GMT Received:... MTP id for ; Thu, 5 Dec 2002 09:57:11 -0500 Message-ID: From: To: Subject: Test Message Date: Thu, 5 Dec 2002 10:00: 48 -0500 MIME-Version: 1.0 Content-Type: text/plain; charset="iso -88 59-1" Content-Transfer-Encoding:... routers block all broadcast messages Otherwise, the network could be swamped in traffic To send a broadcast message, you use a broadcast IP address, which is the IP address that identifies the network and has all the host bits set to 1 For example, if the network is identified by the first three bytes (140 .80 .0), the broadcast address would be 140 .80 .0.255 Alternatively, you can set all bits to 1 (the... Define endpoint where messages are sent Console.WriteLine("Connect to IP: ") Dim IP As String = Console.ReadLine() Dim Port As Integer = 88 00 Dim RemoteEndPoint As New IPEndPoint(IPAddress.Parse(IP), _ Port) ’ Define local endpoint (where messages are received) LocalPort = 88 00 ’ Create a new thread for receiving incoming messages Dim ReceiveThread As New System.Threading.Thread( _ AddressOf ReceiveData)... http://www.nytimes.com/pages/technology/index.html http://www.nytimes.com/pages/science/index.html 8. 8 Communicate Using TCP Problem You need to send data between two computers on a network using a TCP/IP connection Solution Use the TcpClient and TcpListener classes Discussion TCP is a reliable, connection-based protocol that allows two computers to communicate over a network It provides built-in flow-control, sequencing, and error... 255.255.255.255), which specifies the entire network In this case, the broadcast message will still travel only inside the local subnet because routers won’t allow it to pass Dim IP As String = "255.255.255.255" Dim Port As Integer = 88 00 Dim RemoteEndPoint As New IPEndPoint(IPAddress.Parse(IP), _ Port) Dim Client As New UdpClient() Dim Data() As Byte = System.Text.Encoding.UTF8.GetBytes("Broadcast Messa ge")... on port 80 00 Dim Listener As New TcpListener (80 00) Console.WriteLine("About to initialize port.") Listener.Start() Console.WriteLine("Listening for a connection ") Try ’ Wait for a connection request, ’ and return a TcpClient initialized for communication Dim Client As TcpClient = Listener.AcceptTcpClient() Console.WriteLine("Connection accepted.") ’ Retrieve the network stream Dim Stream As NetworkStream... TcpClientTest Public Sub Main() Dim Client As New TcpClient() Try Console.WriteLine("Attempting to connect to the server " & _ "on port 80 00.") Client.Connect(IPAddress.Parse("127.0.0.1"), 80 00) Console.WriteLine("Connection established.") ’ Retrieve the network stream Dim Stream As NetworkStream = Client.GetStream() ’ Create a BinaryWriter for writing to the stream Dim w As New BinaryWriter(Stream) ’ Create... convert all data to a stream of bytes using an encoding class, as described in recipe 1.15 and recipe 2. 18 8.11 Send a Broadcast Message Problem You want to send a message to every user on the local subnet Solution Use the UdpClient class with the appropriate broadcast address Discussion Broadcasts are network messages that are forwarded to all the devices on a local subnet When a broadcast message is... recipe 8. 8 You can convert the server into a multithreaded server that supports multiple simultaneous connections quite easily First create a class that will interact with an individual client: Public Class ClientHandler Private Client As TcpClient Private ID As String Public Sub New(ByVal client As TcpClient, ByVal ID As String) Me.Client = client Me.ID = ID End Sub Public Sub Start() ’ Retrieve the network . 8 Network Programming The need for computers and devices to communicate across a network is one of the key ingredients of enterprise programming. . learn how to manage network interaction using sockets (recipes 8. 8 to 8. 11), but you won’t learn about two higher-level distributed programming frameworks— Web