2Apr il 2003, 17:00:47 The Complete FreeBSD (upgrading.mm), page 585 32 Updating the system software In this chapter: • Upgrading ker nel and user land • Upgrading the ker nel • Upgrading the boot files • Upgrading the configuration files • Merging /etc/group • Mergemaster, second time around In this chapter: • Upgrading ker nel and user land • Upgrading the ker nel • Upgrading the boot files • Upgrading the configuration files • Merging /etc/group • Mergemaster, second time around In the previous chapter,welooked at howtoget an up-to-date FreeBSD source tree. Once you have the sources, you can build various components of the system. The main tool we use for this purpose is make,which we looked at on page 167. The best way to think of upgrading the system is that everything is a matter of changing files. Forthe purposes of this discussion, you can divide the files on your system into the following categories: • The userland,that part of the system software that is not the kernel. Unlikesome other operating systems, FreeBSD expects to keep userland and kernel at the same release level. We’ll look at the interaction between kernel and userland below. • The kernel. You may build a newkernel without updating the sources, of course, if you want to add functionality to the kernel. In this chapter we’ll look at upgrading the kernel in the context of a complete system upgrade. We’llconsider building a custom kernel in the next chapter,Chapter 33, Custom kernels. • Support for booting the machine, which is currently performed as a separate step. • Configuration files relating to your system. Some of them, such as /etc/fstab and /etc/rc.conf,overlap with the previous category. • The Ports Collection. This doesn’thav e to be done at the same time as userland and kernel, though if you upgrade to a significant newversion of FreeBSD, it’sagood idea to upgrade the ports as well. We looked at upgrading ports on page 178. upgrading.mm,v v4.5 (2003/04/02 04:49:48) 585 The Complete FreeBSD 586 2April 2003, 17:00:47 The Complete FreeBSD ( /tools/tmac.Mn), page 586 • Your own files. Theyhav e nothing to do with a software upgrade. Youcan makeupgrading less onerous by planning in advance. Here are some suggestions: • Keep system files and user files on different file systems. • Keep careful records of which configuration files you change, for example with RCS, the Revision Control System.This provestobethe most complicated part of the entire upgrade process. The only files that are upgraded are on the traditional root file system and /usr.Noothers are affected by an upgrade. Table 32-1, an abridged version of Table 10-2 on page 188, givesanoverviewofwhere the system files come from. Table 32-1: FreeBSD directory hierarchy directory name Usage Populated by /bin Executable programs of general use. makeworld /boot Files used when booting the system. makeinstall in /usr/src/sys. /dev Directory of device nodes. System startup (devfs) /etc Configuration files used at system startup. Install from CD-ROM only, mergemaster,administrator /sbin System executables needed at system startup time. makeworld /usr/X11R6 The X11 windowing system. X-based programs in the Ports Collection /usr/bin Standard executable programs that are not needed at system start. makeworld /usr/compat Adirectory containing code for emulated systems, such as Linux. Ports Collection /usr/games Games. makeworld /usr/include Header files for programmers. makeworld /usr/lib Library files. makeworld /usr/libexec Executable files that are not started direct- ly by the user. makeworld /usr/libdata Miscellaneous files used by system utili- ties. makeworld /usr/local Additional programs that are not part of the operating system. Ports collection /usr/obj Temporary object files created when building the system. makeworld /usr/ports The Ports Collection. sysinstall, cvs /usr/sbin System administration programs that are not needed at system startup. makeworld upgrading.mm,v v4.5 (2003/04/02 04:49:48) 587 Chapter 32: Updating the system software 2April 2003, 17:00:47 The Complete FreeBSD (upgrading.mm), page 587 directory name Usage Populated by /usr/share Miscellaneous read-only files, mainly in- formative. makeworld /usr/src System source files. sysinstall, cvs Upgrading kernel and userland The core part of a system upgrade consists of a synchronized update of both kernel and userland. It’srelatively simple to do, but depending on the speed of the machine, it may keep the computer busy for several hours. In general, you build and install the userland first, then you build and install the kernel. The traditional way to build the userland is: # cd /usr/src # make world This operation performs a number of functions, which can be influenced by variables you pass to make.Without anyvariables, makeworld performs the following steps: • It removesthe old build directories and creates newones. You can skip this step by setting the NOCLEAN variable. Don’tset NOCLEAN unless you knowexactly whyyou are doing so, since it can cause inconsistencies that come back to bite you later.In particular,ifyou do have problems after building the world in this manner,you should first go back and perform a complete rebuild without NOCLEAN. • It rebuilds and installs build tools, including make,the C compiler and the libraries. • It builds the rest of the system, with the exception of the kernel and the boot tools. • It installs everything. You can omit this stage by building the buildworld target instead of world. It does this by building a number of subtargets. Occasionally,you might find it useful to build them individually: makeworld can pose a chicken-and-egg problem. It creates the userland, and makekernel makes the kernel. Userland and kernel belong together,and if you upgrade the userland first, you may find that the newuserland takes advantage of differences in the newer version of the kernel. A typical situation is when a newsystem call is added to the kernel. In this case, you may find processes exiting with a signal 12 (invalid system call). If this happens, you may have toperform the upgrade with the sequence: # make buildworld # make kernel (reboot) # make installworld upgrading.mm,v v4.5 (2003/04/02 04:49:48) Upgrading ker nel and userland 588 2April 2003, 17:00:47 The Complete FreeBSD (upgrading.mm), page 588 You’ll find information about such requirements in the file /usr/src/UPDATING.Table 32-2 givesanoverviewofthe more useful targets to the top-level Makefile. Table 32-2: Targets for top-level Makefile Target Purpose buildworld Rebuild everything, including glue to help do upgrades. installworld Install everything built by buildworld. world Perform buildworld and installworld. update Update your source tree. most Build user commands, no libraries or include files. installmost Install user commands, but not libraries or include files. reinstall If you have a build server,you can NFS mount the source and object directories and do a make reinstall on the client to install new binaries from the most recent build on the server. buildkernel Build a kernel for your architecture. By default, use the GENERIC kernel configuration file. Youcan select a different configuration file, say MYKERNEL,with: # make buildkernel KERNCONF=MYKERNEL By default, this target builds all the KLDs (Kernel Loadable Modules), which significantly increases the time it takes. If you knowthat your KLDs will not change, or that you won’tbeusing any, you can skip building them by specifying the -DNO_MODULES flag. installkernel Install a kernel you have built with buildkernel. reinstallkernel Install a kernel you have built with buildkernel.Don’trename the previous kernel directory to kernel.old.Use this target when the previous kernel is not worth keeping. kernel Build and install a kernel. Another issue is that the system configuration might have changed. For example, in early 2002 the default configuration for sendmail changed. The process added a daemon user and group, both called smmsp.Toinstall the userland, this user already needed to be present. upgrading.mm,v v4.5 (2003/04/02 04:49:48) 589 Chapter 32: Updating the system software 2April 2003, 17:00:47 The Complete FreeBSD (upgrading.mm), page 589 The solution to this issue is called mergemaster,ascript that helps you to upgrade the configuration files. We’lllook at it in more detail below, but at this point you should knowthat you need to run it with the -p (pre-build)option: # mergemaster -p As we’ve seen in table 32-1, the installworld target changes a number of directories. Sometimes, though, it leavesold binaries behind: it doesn’tremove anything that it doesn’treplace. The result can be that you end up using old programs that have long passed their use-by date. One solution to this problem is to look at the last modification date of each program in the directories. Forexample, if you see: $ ls -lrt /usr/sbin -r-xr-xr-x 1 root wheel 397 Jul 14 11:36 svr4 -r-xr-xr-x 1 root wheel 422 Jul 14 11:29 linux -r-xr-xr-x 1 root wheel 142080 Jul 13 17:20 sshd . -r-xr-xr-x 1 root wheel 68148 Jul 13 17:16 uuchk -r-xr-xr-x 1 root wheel 6840 Jan 52002 ispppcontrol -r-xr-xr-x 1 root wheel 27996 Apr 21 2001 k5stash -r-xr-xr-x 1 root wheel 45356 Apr 21 2001 ktutil -r-xr-xr-x 1 root wheel 11124 Apr 21 2001 kdb_util -r-xr-xr-x 1 root wheel 6768 Apr 21 2001 kdb_init It’sfairly clear that the files dated April 2001 have not just been installed, so theymust be out of date. Youcan use a number of techniques to delete them; one might be: # find . -mtime +10 | xargs rm This command removesall files in the current directory (.)that are older than 10 days (+10). Of course, this method will only work if you haven’tinstalled anything in these directories yourself. Youshouldn’thav e done so; that’sthe purpose of the directory hierarchy /usr/local,toensure that you keep system files apart from ports and private files. Be careful with /usr/lib:anumber of ports refer to libraries in this directory hierarchy, and if you delete them, the ports will no longer work. In general there’snoproblem with old libraries in /usr/lib,unless theytakeuptoo much space, so you’re safer if you don’t clean out this directory hierarchy. Note that you need to specify the KERNCONF parameter to all the targets relating to kernel builds. Upgrading the kernel There are tworeasons for building a newkernel: it might be part of the upgrade process, which is what we’ll look at here, or you may build a kernel from your current sources to add functionality to the system. We’lllook at this aspect in Chapter 33. One point to notice is that if you’re upgrading from an older custom configuration file, upgrading.mm,v v4.5 (2003/04/02 04:49:48) Upgrading the ker nel 590 2April 2003, 17:00:47 The Complete FreeBSD (upgrading.mm), page 590 you could have a lot of trouble. We’llsee a strategy for minimizing the pain on page 607. In addition, when upgrading to FreeBSD Release 5 from an older release of FreeBSD, you need to install a file /boot/device.hints,which you can typically copyfrom /usr/src/sys/i386/conf/GENERIC.hints: # cp /usr/src/sys/i386/conf/GENERIC.hints /boot/device.hints See page 600 for more details. When upgrading the kernel, you might get error messages likethis one: # config GENERIC config: GENERIC:71: devices with zero units are not likely to be correct Alternatively,you might get a clearer message: # config GENERIC / /conf/files: coda/coda_fbsd.c must be optional, mandatory or standard Your version of config(8) is out of sync with your kernel source. Apart from that, you might find that the kernel fails to link with lots of undefined references. This, too, could mean that the config program is out of synchronization with the kernel modules. In each case, build and install the newversion of config: # cd /usr/src/usr.sbin/config # make depend all install clean Youneed to makeclean at the end since this method will store the object files in non- standard locations. Upgrading the boot files At the time of writing, it’sstill necessary to install the files in /boot separately.Dothis with: # cd /usr/src/sys # make install It’spossible that this requirement will go awayinthe future. Upgrading the configuration files Currently,the system build procedure does not install the configuration files in /etc.You need to do that separately.There are twopossible methods: • Do it manually: upgrading.mm,v v4.5 (2003/04/02 04:49:48) 591 Chapter 32: Updating the system software 2April 2003, 17:00:47 The Complete FreeBSD (upgrading.mm), page 591 1. Backup the old configuration files. They’re not very big, so you can probably makeacopyondisk somewhere. 2. Install pristine newconfiguration files: # cd /usr/src/etc/ # make install 3. Compare the files and update the newones with information from your configuration. • Use mergemaster,asemi-automatic method of doing effectively the same thing. The simple method is: run mergemaster with the options -i and -a,which tell it to run automatically (in other words, not to stop and ask questions), and to install newfiles automatically.That doesn’tmean intelligently: you may run into problems anyway. mergemaster produces a lot of output, and some of it in the middle is important, so you should save the output to disk with the tee command. The first time you try,you might see: # mergemaster -ia 2>&1 | tee -a /var/tmp/merge *** Creating the temporary root environment in /var/tmp/temproot *** /var/tmp/temproot ready for use *** Creating and populating directory structure in /var/tmp/temproot set - ‘grep "ˆ[a-zA-Z]" /usr/src/etc/locale.deprecated‘; while [ $# -gt 0 ] ; do for dir in /usr/share/locale /usr/share/nls /usr/local/share/nls; do test -d /va r/tmp/temproot/${dir} && cd /var/tmp/temproot/${dir}; test -L "$2" && rm -rf "$2"; test -L "$1" && test -d "$1" && mv "$1" "$2"; done; shift; shift; done mtree -deU -f /usr/src/etc/mtree/BSD.root.dist -p /var/tmp/temproot/ ./bin missing (created) ./boot missing (created) . ./vm missing (created) mtree -deU -f /usr/src/etc/mtree/BSD.sendmail.dist -p /var/tmp/temproot/ mtree: line 10: unknown user smmsp *** Error code 1 Stop in /usr/src/etc. *** FATAL ERROR: Cannot ’cd’ to /usr/src/etc and install files to the temproot environment These messages are somewhat misleading. First, the files that are created are all in /var/tmp/temproot.Inaddition, the message Cannot ’cd’ to /usr/src/etc does not refer to anyproblem with that directory; it’sjust an indication that it can’tcontinue with the installation due to the previous errors. The real issue here is that the user smmsp doesn’texist. As we sawabove,this user was added some time in 2002 to address some mail security problems. It’sinthe new /etc/master.passwd file, but it’snot in the one on the system. But howdoyou merge the twofiles? One waywould to be to use mergemaster with the -p option, but then mergemaster prompts you for every single file that it finds to be different, usually about 300 of them. In addition, the editing facilities are relatively basic. It’sbetter to edit the file in advance with an editor. upgrading.mm,v v4.5 (2003/04/02 04:49:48) Upgrading the configuration files 592 2April 2003, 17:00:47 The Complete FreeBSD (upgrading.mm), page 592 Merging the password file As we sawonpage 145, the password file is quite complicated. Depending on howmuch work you want to do, you have a couple of possibilities: • Youcan choose to completely replace the old /etc/master.passwd with the newone. This will cause all added user names and passwords to disappear,sounless this is just atest machine, it’sunlikely you’ll want to followthis path. • Youcan takeadvantage of the fact that, with the exception of root,the distribution /etc/master.passwd contains no ‘‘real’’users. You can merge the entries for real users with the entries in the distribution /etc/master.passwd.This works relatively well, but it removesthe passwords of the system users, so you have toset them again. We’lllook at howtodothat below. The distribution version of /etc/master.passwd looks something likethis: #$FreeBSD: src/etc/master.passwd,v 1.33 2002/06/23 20:46:44 des Exp $ # root::0:0::0:0:Charlie &:/root:/bin/csh toor:*:0:0::0:0:Bourne-again Superuser:/root: .etc The individual fields are separated by colons (:). We’ll look at only the fields that interest us in the following expansion. It’seasier to look at if they’re separated by spaces; numerically,they’re the first, second, eighth, ninth and tenth fields. Foradescription of the other fields, see the man page master.passwd(4). User passwordGECOS home directory shell root Charlie &/root /bin/csh toor * Bourne-again Superuser /root daemon * Owner of many processes /root /sbin/nologin operator * System & //sbin/nologin bin * Binaries Commands //sbin/nologin tty * Tty Sandbox //sbin/nologin kmem * KMem Sandbox //sbin/nologin games * Games pseudo-user /usr/games /sbin/nologin news * News Subsystem //sbin/nologin man * Mister Man Pages /usr/share/man /sbin/nologin sshd * Secure Shell Daemon /var/empty /sbin/nologin smmsp * Sendmail Submission /var/spool/clientmqueue /sbin/nologin mailnull * Sendmail Default User /var/spool/mqueue /sbin/nologin bind * Bind Sandbox //sbin/nologin xten * X-10 daemon /usr/local/xten /sbin/nologin pop * Post Office Owner /nonexistent /sbin/nologin www * World Wide Web Owner /nonexistent /sbin/nologin nobody * Unprivileged user /nonexistent /sbin/nologin The first field is the name of the user.Inthe course of time, a number of pseudo-users have been added to reduce exposure to security issues. The main issue in merging the files is to add these users. If you don’thav e the user in your current /etc/master.passwd, you can add the line from the distribution file. The second field contains the password.Inthe distribution file, it’susually *,which means it needs to be set before you can log in at all. Only root has no password; you need to be able to log in as root to set passwords. By contrast, in your installed upgrading.mm,v v4.5 (2003/04/02 04:49:48) 593 Chapter 32: Updating the system software 2April 2003, 17:00:47 The Complete FreeBSD (upgrading.mm), page 593 /etc/master.passwd,you will almost certainly have a password, and in general you will want to keep it. The home directory entry has not changed much. You’ll notice directory names like /nonexistent and /var/empty.The former is a fake, the latter a directory that can’tbe changed. It’spossible that this entry will change from one release to another,and it’s important to get it correct. Formanyaccounts, the shell field contains the name /sbin/nologin,which prints the text ‘‘This account is currently not available’’and exits. Currently only root has a real shell, butthat could change. To update the /etc/master.passwd,you can use the following method: • Makeacopy of your old /etc/master.passwd! • Maintain a strict separation of the original lines from the distribution file and your ownentries. This will help you with the next update. • Copythe entire distribution /etc/master.passwd to the top of your /etc/master.passwd file. At this point you will have a number of duplicates. • Check the entries for root.You can probably remove the distribution entry and leave your entry in the file, preserving the password and shell. In this case, you should makeanexception to the separation between distribution and local additions: due to the way the name lookups work, if you put user root belowuser toor (‘‘root’’spelt backwards, and the same user with possibly a different shell), all files will appear to belong to toor instead of to root. • Check what other entries you have for user ids under 1000. Youcan probably remove them all, but if you have installed ports that require their own user ID, you will need to keep them. • Youshould be able to keep all the entries for users with IDs above and including 1000, with the exception of user nobody (ID 65534). Use the entry from the distribution file for nobody. Once you have merged the files, you need to run pwd_mkdb to rebuild the password files /etc/passwd, /etc/pwd.db and /etc/spwd.db. /etc/passwd is gradually going out of use, butyou probably have one on your system, and some ports use it, so it’spreferable to recreate it. Do this with the -p option to pwd_mkdb: # pwd_mkdb -p /etc/master.passwd upgrading.mm,v v4.5 (2003/04/02 04:49:48) Upgrading the configuration files 594 2April 2003, 17:00:47 The Complete FreeBSD (upgrading.mm), page 594 Merging /etc/group In addition to /etc/master.passwd,you will probably need to upgrade /etc/group.Inthis case, the main issue is to add users to the wheel group. The distribution /etc/group looks likethis: #$FreeBSD: src/etc/group,v 1.27 2002/10/14 20:55:49 rwatson Exp $ # wheel:*:0:root daemon:*:1: kmem:*:2: sys:*:3: tty:*:4: operator:*:5:root mail:*:6: bin:*:7: news:*:8: man:*:9: games:*:13: staff:*:20: sshd:*:22: smmsp:*:25: mailnull:*:26: guest:*:31: bind:*:53: uucp:*:66: xten:*:67: dialer:*:68: network:*:69: www:*:80: nogroup:*:65533: nobody:*:65534: Again, newgroups have appeared for security reasons. Use a similar method to the one you used for /etc/master.passwd: • Makeacopy of your old /etc/group! • Maintain a strict separation of the original lines from the distribution file and your ownentries. This will help you with the next update. • Copythe entire distribution /etc/group to the top of your /etc/group file. At this point you will have a number of duplicates. • Check the entries for wheel.You can probably remove the distribution entry and leave your entry in the file, preserving the users. • In addition, you may have some users in other groups. Forexample, installing postfix adds the user postfix to group mail.You need to preservethese users. Youdon’tneed to do anything special after updating /etc/group.You can nowcontinue with mergemaster. upgrading.mm,v v4.5 (2003/04/02 04:49:48) [...]... meantime, though, you have added other tasks (the bottom four lines), and you have also commented out the periodic invocation of adjkerntz These are the changes you need to make to the new /etc/crontab before you install it There’s a simpler possibility here, though: the only real change that would then be left in /etc/crontab is the change in the starting times for the daily and weekly housekeeping Does... >/dev/null The lines starting with - show lines only in the old file, which is still in /etc/crontab The lines starting with + show lines only in the new file, which is in /var/tmp/temproot.1102.15.01.14/etc/crontab There are a number of changes here: the CVS ID ($FreeBSD$) has changed from 1.21 to 1.31, and the times of the periodic maintenance have changed In the meantime, though, you have added other tasks...2 April 2003, 17:00:47 The Complete FreeBSD (upgrading.mm), page 595 595 Chapter 32: Updating the system software Mergemaster, second time around Before running mergemaster again, you should delete the contents of /var/tmp/temproot Otherwise you might see something like: *** The directory specified for the temporary root environment, /var/tmp/temproot, exists This... them If you know for a fact that you have never changed them, for example if you have religiously kept track of your changes with RCS, you don’t need to bother: mergemaster errs on the side of safety You may have changed others, though The most obvious one above is /etc/crontab, which contains system- wide commands to be executed by cron To compare them, use diff : $ diff -wu /etc/crontab /var/tmp/temproot.1102.15.01.14/etc/crontab... empty The problem here is that the directory /var/empty has been set immutable Change that with the chflags command and try again: # find /var/tmp/temproot|xargs chflags noschg # rm -rf /var/tmp/temproot Run mergemaster in the same way as before, saving the output If you haven’t deleted the old /var/tmp/temproot directory, you might see: # mergemaster -ia 2>&1 | tee -a /var/tmp/merge *** Creating the. .. /var/tmp/temproot.1102.15.01.14/etc/syslog.conf *** You chose the automatic install option for files that did not exist on your system The following were installed for you: /etc/periodic/security/510.ipfdenied /etc/periodic/security/security.functions /etc/mac.conf You’re not done yet: there are 21 files above that need looking at There’s a good chance that you’ve never heard of some of them, let alone changed them If you know for a fact... risk if untrusted users have access to the system mergemaster does not delete the old directories: you should do so yourself If this file already exists, mergemaster ignores it and creates a new directory with a name like /var/tmp/temproot.0917.02.18.06 The numbers are a representation of the date and time of creation mergemaster doesn’t make it easy to remove the /var/tmp/temproot directory You may... anything: the old /etc/crontab is fine the way it is There’s a whole list of files that you’re likely to change from the defaults Here are some more likely candidates: • You may find it necessary to change /etc/syslog.conf If so, you may have to merge by hand, but it shouldn’t be too difficult • You will almost certainly change /etc/fstab About the only reason why you might need to merge changes would be if the. .. /etc/motd contains the login greeting There’s never a reason to take the new version • /etc/inetd.conf can be a problem: as new services are introduced, it changes At the same time, you may have added services via ports, or enabled services in the manner we will see on page 446 You definitely need to merge this one yourself upgrading.mm,v v4.5 (2003/04/02 04:49:48) 2 April 2003, 17:00:47 The Complete FreeBSD... 2003, 17:00:47 The Complete FreeBSD (upgrading.mm), page 596 Mergemaster, second time around 596 *** Beginning comparison *** Temp /etc/defaults/rc.conf and installed have the same CVS Id, deleting *** Temp /etc/defaults/pccard.conf and installed have the same CVS Id, deleting *** /etc/defaults/periodic.conf will remain for your consideration *** Temp /etc/gnats/freefall and installed have the same CVS . tools, including make ,the C compiler and the libraries. • It builds the rest of the system, with the exception of the kernel and the boot tools. • It installs. the following categories: • The userland,that part of the system software that is not the kernel. Unlikesome other operating systems, FreeBSD expects to