Chapter Network Applications Introduction The previous chapter looked at how you connect a Linux box to a network and provide some basic services That is not enough information to produce a useful Linux machine You need to know how to configure and manage the higher level network services which are expected today, including print/file sharing, electronic mail, File Transfer Protocol, World-Wide Web and others That's where this chapter comes in It aims to provide an overview of how network applications work, how they operate and how they are configured There is no way a single chapter can provide information about all the available network applications There are hundreds of them and each one can be quite complex Instead this chapter focuses on the fundamentals, the concepts which are common to all these applications If you are comfortable with this knowledge, then learning how to configure a new application is quite simple The chapter closes with a detailed look at some specific network services including file/print sharing, messaging (email) and the World-Wide Web Other resources Other available resources which examine similar material include: · · · · · · HOW-TOs Firewall, IPCHAINS, Intranet Server (though it is a little dated), Mail, Mail User, NFS, NIS, Networking Overview (gives a very good overview of topics related to both networking chapters), SMB, VPN, Virtual Services, WWW, Apache SSL PHP/FI Mini HOW-TOs Apache SSL PHP/FI, Automount, Cipe+Masquerading, ISP Connectivity, NFSRoot, NFS-Root-Client, Qmail+MH, Remove Boot, Remote X Apps, Sendmail Address Rewrite, Sendmail+UUCP, Secure POP via SSH LAME Sections on DNS Configuration, sections on Windows and Mac file and print sharing, NFS section, configuring the Apache Web server, configuring the Squid HTTP caching proxy, Configuring sendmail Apache website http://www.apache.org/ Samba website http://www.samba.org The Red Hat reference and getting started guide has additional information about many of these topics Page 406 An expanded and constantly updated list of resources can be found on the course website How it all works So what are the common details about all the network applications? How they work? This section provides a general answer to these questions The provision of network services like FTP, telnet, e-mail and others relies on these following components: · · · · network ports Network ports are the logical (that means that ports are an imaginary construct which exist only in software) connections through which the information flows into and out of a machine A single machine can have thousands of programs all sending and receiving information via the network at the same time The delivery of this information to the right programs is achieved through ports Generally each program must have its own port network daemons Network daemons are the programs running on the network server machines that sit listening at pre-defined ports waiting for connections from other hosts These daemons wait for a request, perform some action and send a response back to the program that requested the action The program which requested the action is a network client network clients Users access network services using client programs Example network client programs include Netscape, Eudora and the ftp command on a UNIX machine The client programs turn user requests (for example typing in the URL http://www.linux.org/) into a request which is sent to a network daemon The requests and responses which flow between network daemons and network clients must take part in some agreed upon format, a network protocol network protocols Network protocols specify how the network clients and servers communicate They define the small "language" which both use for communication The following sections of this chapter go into more detail about each of these components Ports All network protocols, including http ftp SMTP, use either TCP or UDP to deliver information TCP and UDP are referred to as transport protocols Each transport protocol has its own characteristics and which one is used depends on the type of communication which occurs However, one thing is common between both transport protocols: the addresses they use to identify the source (where they are coming from) and the destination (where they are going to) Obviously, the first component of the source/destination address is the IP address, this identifies the computer The next component is the port number on that computer Every TCP or UDP header contains two 16 bit numbers that are Page 407 used to identify the source port (the port through which the information was sent) and the destination port (the port through which the information must be delivered) The IP address is stored in the IP header Since port numbers are 16 bit numbers, there can be approximately 64,000 (216 is about 64,000) different ports Some of these ports are used for predefined purposes The ports 0-256 are used by the network servers for well known Internet services (for example telnet, FTP, SMTP) Ports in the range from 256-1024 are used for network services that were originally UNIX specific Network client programs and other programs should use ports above 1024 Table 17.1 lists some of the port numbers for well known services Port number 20 21 23 25 80 119 Purpose ftp-data ftp telnet (mail) (WWW) nntp (network news) SMTP http Table 17.1 Reserved Ports This means that when you look at a TCP/UDP packet and see that it is addressed to port 25, then you can be sure that it is part of an email message being sent to a SMTP server A packet destined for port 80 is likely to be a request to a Web server Reserved ports So how does the computer know which ports are reserved for special services? On a UNIX computer, this is specified by the file /etc/services Each line in the services file is of the format: service-name port/protocol aliases Where service-name is the official name for the service, port is the port number that it listens on, protocol is the transport protocol it uses and aliases is a list of alternate names The following is an extract from an example /etc/services file Most /etc/services files will be the same, or at least very similar echo 7/tcp echo 7/udp discard 9/tcp sink null discard 9/udp sink null systat 11/tcp users daytime 13/tcp daytime 13/udp ftp-data 20/tcp ftp 21/tcp telnet 23/tcp smtp 25/tcp mail nntp 119/tcp usenet # Network News Transfer ntp 123/tcp # Network Time Protocol You should be able to match some of the entries in the above example, or in the /etc/services file on your computer, with the entries in Table 17.1 Page 408 Exercises 17.1 Examine your /etc/services file and discover the port on which the following protocols are used http ssh pop3 Look at ports, netstat The netstat command can be used for a number of purposes including looking at all of the current active network connections The following is an example of the output that netstat can produce (it's been edited to reduce the size): [david@cq-pan:~]$ netstat -a Active Internet connections (including servers) Proto Recv-Q Send-Q Local Address Foreign Address root tcp 7246 cq-pan.cqu.edu.au:www lore.cs.purdue.e:42468 tcp 0 cq-pan.cqu.edu.au:www sdlab142.syd.cqu.:1449 tcp 0 cq-pan.cqu.edu.au:www dialup102-4-9.swi:1498 tcp 22528 cq-pan.cqu.edu.au:www 205.216.78.103:3058 tcp 22528 cq-pan.cqu.edu.au:www barney.poly.edu:47547 tcp 0 cq-pan.cqu.edu.au:www eda.mdc.net:2395 tcp 22528 cq-pan.cqu.edu.au:www eda.mdc.net:2397 tcp 0 cq-pan.cqu.edu.au:www cphppp134.cyberne:1657 tcp 22528 cq-pan.cqu.edu.au:www port3.southwind.c:1080 tcp cq-pan.cqu.edu.:telnet dinbig.cqu.edu.au:1107 tcp 0 cq-pan.cqu.edu.au:ftp ppp2-24.INRE.ASU.:1718 (State) User CLOSING root CLOSE root FIN_WAIT2 root CLOSE root CLOSE root CLOSE root CLOSE root FIN_WAIT2 root CLOSE root ESTABLISHED root FIN_WAIT2 root Explanation Table 17.2 explains each column of the output Taking the column descriptions from the table, it is possible to make some observations: · · · · All of the entries except the last two, are for people accessing this machine's (cqpan.cqu.edu.au) World-Wide Web server You can say this because of cq-pan.cqu.edu.au:www This tells us that the port on the local machine is the www port (port 80) In the second last entry, I am telneting to cq-pan from my machine at home At that stage, my machine at home was called dinbig.cqu.edu.au The telnet client is using port 1107 on dinbig to talk to the telnet daemon The last entry is someone connecting to CQ-PAN's ftp server The connection for the first entry is shut down but not all the data has been sent (this is what the CLOSING state means) This entry, from a machine from Purdue University in the United States, still has 7246 bytes still to be acknowledged Page 409 Column name Proto Recv-Q Send-Q Local Address Foreign Address State User Explanation The name of the transport protocol (TCP or UDP) being used The number of bytes not copied to the receiving process The number of bytes not yet acknowledged by the remote host The local hostname (or IP address) and port of the connection The remote hostname (or IP address) and remote port The state of the connection (only used for TCP because UDP doesn't establish a connection), the values are described in the man page Some systems display the user that owns the local program serving the connection Table 17.2 Columns for net stat Network daemons The /etc/services file specifies which port a particular protocol will listen on For example SMTP (Simple Mail Transfer Protocol, the protocol used to transfer mail between different machines on a TCP/IP network) uses port 25 This means that there should be a network daemon that listens for SMTP connections on port 25 and knows what to with those connections This begs some questions: · · How we know which program acts as the network daemon for which protocol? How is that program started? How network daemons start There are two methods by which network daemons are started: · · by startup scripts Daemons started in this manner will show up in a ps list of all the current running processes These daemons are always running, waiting for a connection on the specified port This means that the daemon is using up system resources (RAM etc) because it is always in existence but it also means that it is very quick to respond when requests arrive for their services by the xinetd daemon (older systems use the inetd daemon) The xinetd daemon listens at a number of ports, and when information arrives, it starts the appropriate network daemon for that port if security and permissions permit Which daemon, for which port, is specified in the configuration file /etc/xinetd.conf Starting a network daemon via xinetd is usually done when there aren't many connections for that daemon If a network daemon is likely to get a large number of connections (a busy mail or WWW daemon for example), the daemon for that service should be started in the system startup files and always listen on the port The reason for this is overhead Using xinetd takes longer because for every connection, it needs to first create a new process (and we've seen already that creating Page 410 new processes can be a relatively expensive process) When the daemon is already running and listening to the port, it simply starts handling the request Of course the drawback with starting daemons in the startup scripts is that they are always there consuming RAM and other resources even if they aren't being used xinetd also provides access control and logging for network services xinetd and inetd is the extended Internet services daemon and has replaced inetd in modern Linux distributions They both perform the same task of starting network services as they are requested, but xinetd provides integrated logging and access control, whereas inetd used other services and programs to this inetd is still very common in commercial UNIX operating systems xinetd The /etc/xinetd.conf file specifies the network services provided by xinetd Actually, the xinetd.conf file contains little more than a few default values, but it ends with the line includedir /etc/xinetd.d This instructs xinetd to parse every file inside the /etc/xinetd.d directory as an xinetd configuration file This modularity allows you to specify services one per file within a directory, and simplifies the management and configuration of many services The following is an example of the contents of the xinetd.d directory: [root@linuxbox root]# ls /etc/xinetd.d chargen daytime-udp rsync sgi_fam chargen-udp echo servers swat daytime echo-udp services telnet time time-udp Every aspect of the access control and logging xinetd provides is configurable, and the configuration files can become very complex The basic format of one of these service configuration files is: service { } Configuration options are explained in detail in the xinetd.conf manual page The following is the xinetd configuration file for telnet A description of the attributes follows [root@linuxbox xinetd.d]# cat telnet # default: on # description: The telnet server serves telnet sessions; it uses \ # unencrypted username/password pairs for authentication service telnet { flags = REUSE socket_type = stream wait = no user = root server = /usr/sbin/in.telnetd log_on_failure += USERID disable = no } Page 411 flags - how xinetd is to behave with regards to this service There are many options and combinations of options available, dependent on the protocol and service in use socket_type – the type of data delivery service used (we don’t cover this) Values are generally stream for TCP, dgram for UDP and raw for direct IP packets - this attribute determines if the service is single threaded or multithreaded If its value is yes the service is singlethreaded; this means that xinetd will start the server and then it will stop handling requests for the service until the server dies If the attribute value is no, the service is multithreaded and xinetd will keep handling new service requests wait – the username to run the service as Usually this is root but there are some exceptions, generally for security reasons user server telnet – determines the program to execute for this service In this case it is the daemon – one of the many logging configuration options available Determines what information is logged when a server cannot be started (either because of a lack of resources or because of access control restrictions) The service id is always included in the log entry along with the reason for failure The options for this attribute include combinations of USERID, HOST and ATTEMPT In this case, the assignment operator in use is += which means “add the option USERID to existing options for this attribute” If you look in /etc/xinetd.conf you’ll see the default value for log_on_failure is HOST So, this makes the value of log_on_failure for telnet HOST USER log_on_failure - this is boolean yes or no This will result in the service being disabled and not starting Services can also be disabled with the flag DISABLE appearing in the flags option Note: If you find that a service is not starting when you try to access it, this option is worth checking, as many default to yes, which means it is disabled disable For a full list and description of the xinetd configuration options, check the xinetd and xinetd.conf manual pages How it works Whenever the machine receives a request on a port (on which the xinetd daemon is listening on), the xinetd daemon decides which program to execute on the basis of the security and program settings in the /etc/xinetd.conf file Page 412 Exercises 17.2 is a UNIX command which will give you a progressive display of the current running processes Use top to observe what happens when a network daemon is started For example, start top and then try to telnet or ftp to your machine Can you see the appropriate daemon start? (Remember you should be able to use the hostname localhost for your own machine even if you are not on a network.) 17.3 What happens if you change the /etc/xinetd.conf file or one of its included files? Does the xinetd daemon pick up the change automatically? How would you notify xinetd of the change? Note: you WILL have to experiment to find out the answer to this question It isn't included in the study material A suggested experiment is the following: try the command telnet localhost, this should cause inetd to some work; if it works, comment out the entry in the config file for the telnet service and try the first command again Does it work? If it does then xinetd hasn't seen the change How you tell it? top Network clients All of you will have used a number of network client programs If you are reading this online, you may well be using a web browser It's a network client program Checking your mail makes use of a network client A network client is simply a program (whether it is text based or a GUI program) that knows how to connect to a network daemon, pass requests to the daemon and then receive replies The telnet client By default, when you use the command telnet jasper, the telnet client program will attempt to connect to port 23 of the host jasper (23 is the telnet port as listed in /etc/services) It is possible to use the telnet client program to connect to other ports For example the command telnet jasper 25 will connect to port 25 of the machine jasper The usefulness and problem with this will be discussed on the next couple of pages Network protocols Each network service generally uses its own network protocol that specifies the services it offers, how those services are requested and how they are supplied For example, the ftp protocol defines the commands that can be used to move files from machine to machine When you use a command line ftp client, the commands you use are part of the ftp protocol Request for comment (RFCs) For protocols to be useful, both the client and daemon must agree on using the same protocol If they talk different protocols then no communication can occur The Page 413 standards used on the Internet, including those for protocols, are commonly specified in documents called Request for Comments (RFCs) (Not all RFCs are standards.) Someone proposing a new Internet standard will write and submit an RFC The RFC will be distributed to the Internet community who will comment on it and may suggest changes The standard proposed by the RFC will be adopted as a standard if the community is happy with it Protocol RFC FTP 959 Telnet 854 SMTP 821 DNS 1035 TCP 793 UDP 768 Table 17.4 RFCs for Protocols Table 17.4 lists some of the RFC numbers which describe particular protocols RFCs can and often are very technical and hard to understand unless you are familiar with the area (the RFC for ftp is about 80 pages long) Exercises 17.4 Take a look at http://www.faqs.org/ They maintain a collection of FAQs from Usenet news and also provide access to the RFCs Use this site to view the RFC for SMTP Take a look through it to get an idea of what is there The direct URL you want is http://www.faqs.org/rfcs/rfc821.html (at least at the time of writing) Text based protocols Some of these protocols smtp ftp nntp http are text-based They make use of simple text-based commands to perform their duty Table 17.5 contains a list of the commands that smtp understands smtp (simple mail transfer protocol) is used to transport mail messages across a TCP/IP network Command HELO hostname MAIL FROM: sender-address TO: recipient-address VRFY address EXPN address DATA RSET NOOP DEBUG [level] HELP QUIT Purpose Startup and give your hostname Mail is coming from this address Please send it to this address Does this address actually exist (verify) Expand this address I'm about to start giving you the body of the mail message Oops, reset the state and drop the current mail message Do nothing Set debugging level Give me some help please Close this connection Table 17.5 SMTP commands Page 414 How it works When transferring a mail message, a client (such as Eudora) will connect to the SMTP daemon (on port 25) The client will then carry out a conversation with the daemon using the commands from Table 17.5 Since these commands are just straight text you can use telnet to simulate the actions of an email client Doing this actually has some real use I often use this ability to check on a mail address or to expand a mail alias The following shows an example of how I might this The text in bold is what I've typed in The text in italics are comments I've added after the fact beldin:~$ telnet localhost 25 Trying 127.0.0.1 Connected to localhost Escape character is '^]' 220-beldin.cqu.edu.au Sendmail 8.6.12/8.6.9 ready at Wed, May 1996 13:20:10 +1 000 220 ESMTP spoken here vrfy david check the address david 250 David Jones