2Apr il 2003, 17:00:47 The Complete FreeBSD (building.mm), page 599 33 Custom kernels In this chapter: • Building a newker nel • Configur ing I/O devices • The ker nel build director y • The configuration file • Prepar ing for upgrades • Building and installing the new kernel • Making device nodes • Kernel loadable modules • sysctl • Living with FreeBSD- CURRENT • Analyzing ker nel crash dumps In this chapter: • Building a newker nel • Configur ing I/O devices • The ker nel build director y • The configuration file • Prepar ing for upgrades • Building and installing the new kernel • Making device nodes • Kernel loadable modules • sysctl • Living with FreeBSD- CURRENT • Analyzing ker nel crash dumps So far,everything we’ve done has been with the standard GENERIC kernel distributed with FreeBSD. Youmay find it an advantage to install a custom kernel: • As we sawinChapter 2, GENERIC doesn’tsupport everything that FreeBSD knows about. For example, if you want to install a Yoyodyne frobulator,you’ll need to install special support for it. 1 • It will takeless time to boot because it does not have tospend time probing for hardware that you do not have. • Acustom kernel often uses less memory.The kernel is the one system component that must always be present in memory,sounused code ties up memory that would otherwise be available to the virtual memory system. On a system with limited RAM, you can save some memory by building a custom kernel, but don’t overestimate the savings: a minimal kernel might save 500 kB overthe GENERIC kernel supplied with the system. • In addition, there are several kernel options that you can tune to fit your needs. • Finally,onpage 613 we’ll look at things to think about if you want to run the -CURRENT version of FreeBSD. In older releases of BSD, you needed to build a newkernel for just about anything you wanted to change, eventhings as simple as a different IRQ for a device. FreeBSD has ev olved quite a bit since then, and it’sbecoming increasingly less necessary to build a custom kernel. You will certainly need to do so if you want to enable kernel-wide 1. In fact, the developer working on the Yoyodyne has defected to the Free Software Foundation. See the GNU General Public License for further details. building.mm,v v4.15 (2003/04/02 06:30:12) 599 The Complete FreeBSD 600 2April 2003, 17:00:47 The Complete FreeBSD ( /tools/tmac.Mn), page 600 options, such as extra consistencychecking, but in manycases you have more flexible alternatives: • If you just need to add device support, you may be able to load a Kernel Loadable Module,orkld.See page 611 for more information on klds. • If you want to change ISA parameters such as I/O address, IRQ or DMA settings, you no longer need to build a newkernel: the kernel configuration file no longer knows about such parameters. Instead, they’re in the file /boot/device.hints,that we’ll look at below. • Anumber of kernel options have been replaced by the sysctl interface. For example, the GENERIC kernel does not perform packet routing by default. In older releases of FreeBSD, you had to build a newkernel with the option GATEWAY.Now adays you can turn this feature on and offatwill with a sysctl command. We’ll look at sysctls on page 611. Configuring a kernel has changed a lot since the early days of FreeBSD, but it’snot done yet. The information in this chapter represents a snapshot in the evolution of building kernels. The goal of these changes is to makeitunnecessary to build a kernel at all except to upgrade to a newrelease. Building a newkernel FreeBSD is distributed in source, and building a kernel primarily involves compiling the source files needed for the kernel. Tobuild a kernel, you perform the following steps: • Install the system source, if you haven’talready done so. We looked at that in Chapter 31. • Define your kernel configuration in a kernel configuration file.This file defines parameters to use during the build process. We’lllook at howtodothis starting on page 602. • Change to the directory /usr/src and run makekernel.This builds and installs the kernel and all modules. We’lllook at it in more detail on page 608, where we’ll also see alternativesthat give more control overthe process. Configuring I/O devices Alot of the configuration file relates to the I/O devices that you may connect to your machine. In older releases of FreeBSD, it was often necessary to specify some of the IRQ, DMA channel, board memory,and I/O addresses for the devices you configure, particularly for ISA boards. Since Release 5 of FreeBSD, this is no longer the case. Instead, you modify the file /boot/device.hints,which we looked at on page 568. building.mm,v v4.15 (2003/04/02 06:30:12) 601 Chapter 33: Custom ker nels 2April 2003, 17:00:47 The Complete FreeBSD ( /tools/tmac.Mn), page 601 The kernel installation does not install /boot/device.hints automatically.Ifitdoesn’t exist, copyitfrom the configuration directory: # cp -p /usr/src/sys/i386/conf/GENERIC.hints /boot/device.hints The kernel build directory The kernel sources are kept in the directory /usr/src/sys.The symbolic link /sys also points to this directory.There are a number of subdirectories of /usr/src/sys that represent different parts of the kernel, but for our purposes, the most important of them are the architecture dependent directories such as /usr/src/sys/i386/conf (for the i386 architecture), /usr/src/sys/alpha/conf (for the Alpha architecture), or /usr/src/sys/sparc64/conf (for the SPARC64 architecture) where you edit your custom kernel configuration. In addition, the old style kernel build described belowbuilds the kernel in the directory /usr/src/sys/i386/compile, /usr/src/sys/alpha/compile or /usr/src/sys/sparc64/compile respectively.Notice the logical organization of the directo- ry tree: each supported device, file system, and option has its own subdirectory.Inthe rest of this chapter,we’ll look at the i386 architecture. Most of this applies to other architectures as well. If your system doesn’thav e the directory /usr/src/sys,the kernel source has not been installed. If you have a CD-ROM, the sources are on the first CD-ROM in the directory /src.Toinstall from the CD-ROM, perform the following steps: # mkdir -p /usr/src/sys # ln -s /usr/src/sys /sys # cd / # cat /cdrom/src/ssys.[a-h]* | tar xzvf - The symbolic link /sys for /usr/src/sys is not strictly necessary,but it’sagood idea: some software uses it, and otherwise you may end up with twodifferent copies of the sources. By definition, the files on CD-ROM are out of date. See Chapter 31 for details of howto get the current, up-to-date sources. Next, move tothe directory i386/conf and copythe GENERIC configuration file to the name you want to give your kernel. For example: # cd /usr/src/sys/i386/conf # cp GENERIC FREEBIE Traditionally,this name is in all capital letters and, if you are maintaining multiple FreeBSD machines with different hardware, it’sagood idea to name it after your machine’shostname. In this example we call it FREEBIE. building.mm,v v4.15 (2003/04/02 06:30:12) The ker nel build directory602 2April 2003, 17:00:47 The Complete FreeBSD ( /tools/tmac.Mn), page 602 Now, edit FREEBIE with your favourite text editor.Change the comment lines at the top to reflect your configuration or the changes you’ve made to differentiate it from GENERIC: # # FREEBIE -- My personal configuration file # #For more information on this file, please read the handbook section on #Kernel Configuration Files: # #http://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook/kernelconfig-config.html # #The handbook is also available locally in /usr/share/doc/handbook #ifyou’ve installed the doc distribution, otherwise always see the #FreeBSD World Wide Web server (http://www.FreeBSD.org/) for the #latest information. # #Anexhaustive list of options and more detailed explanations of the #device lines is also present in the / /conf/NOTES and NOTES files. #Ifyou are in doubt as to the purpose or necessity of a line, check first #inNOTES. # #$FreeBSD: src/sys/i386/conf/FREEBIE,v 1.369 2002/10/19 16:54:07 rwatson Exp $ machine "i386" cpu "I486_CPU" cpu "I586_CPU" cpu "I686_CPU" ident FREEBIE maxusers 0 The configuration file The directory /sys/i386/conf contains a number of configuration files: GENERIC General-purpose configuration file LINT This file used to be a ‘‘complete’’configuration file with comments, used for testing and documentation. Since FreeBSD Release 5, it no longer exists. Youcan create it from the files NOTES and /usr/src/sys/conf/NOTES with the command: $ make LINT NOTES Acomplete pseudo-configuration file with copious comments. This file is descended from LINT,but it also includes device hints. Youcan’tuse it for building kernels. Instead, create the file LINT as described above. NOTES contains only platform-specific information. Most of the informa- tion is in the platform-independent file /usr/src/sys/conf/NOTES. building.mm,v v4.15 (2003/04/02 06:30:12) 603 Chapter 33: Custom ker nels 2April 2003, 17:00:47 The Complete FreeBSD (building.mm), page 603 OLDCARD Aconfiguration file for laptops that use PCCARD controllers. At the time of writing, PCCARD support has largely been rewritten, but the newcode does not support some of the devices that the old code supports. This configuration file uses the old PCCARD code instead of the newcode. When the newcode is complete, it will go away. The general format of a configuration file is quite simple. Each line contains a keyword and one or more arguments. Anything following a # is considered a comment and ignored. Keywords that contain numbers used as text must be enclosed in quotation marks. One of the results of this simplicity is that you can put in options that have absolutely no effect. For example, you could add a line likethis: options APPLE_MAC_COMPATIBILITY Youcan build a kernel with this option. It will makenodifference whatsoever. Now it’s unlikely that you’ll think up a non-existent option likethis, but it’smuch more possible that you’ll misspell a valid option, especially finger-twisters like SYSVSHM,with the result that you don’tcompile in the option you wanted. The config program warns if you use unknown options, so takethese warnings seriously. Kernel options change from release to release, so there’snopoint in describing them all here. In the following sections we’ll look at some of the more interesting ones; for a complete list, read LINT or the online handbook. See above for details of howtocreate LINT. Naming the kernel Every kernel you build requires the keywords machine, cpu,and ident.For example, machine "i386" Fori386 architecture machine "alpha" Foralpha architecture machine "sparc64" ForSPARC 64 architecture cpu "I486_CPU" cpu "I586_CPU" cpu "I686_CPU" ident FREEBIE machine The keyword machine describes the machine architecture for which the kernel is to be built. Currently it should be i386 for the Intel architecture, and alpha for the AXP architecture. Don’tconfuse this with the processor: for example, the i386 architecture refers to the Intel 80386 and all its successors, including lookalikes made by AMD, Cyrix and IBM. building.mm,v v4.15 (2003/04/02 06:30:12) The configuration file 604 2April 2003, 17:00:47 The Complete FreeBSD (building.mm), page 604 cpu cpu_type cpu describes which CPU chip or chips this kernel should support. Forthe i386 architecture, the possible values are I386_CPU, I486_CPU, I586_CPU and I686_CPU, and you can specify anycombination of these values. For a custom kernel, it is best to specify only the CPU you have.If, for example, you have anIntel Pentium, use I586_CPU for cpu_type. If you’re not sure what processor type you have,look at the output from the dmesg command when running the GENERIC kernel. For example: CPU: AMD Athlon(tm) XP processor 1700+ (1462.51-MHz 686-class CPU) Origin = "AuthenticAMD" Id = 0x662 Stepping = 2 Features=0x383f9ff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,SEP,MTRR,PGE,MCA,CMOV,PAT,PS E36,MMX,FXSR,SSE> AMD Features=0xc0480000<<b19>,AMIE,DSP,3DNow!> This shows that the processor,anAMD Athlon XP,isa‘‘686 class’’CPU, so to run a kernel on this processor,you must set I686_CPU in the config file. Since Release 5 of FreeBSD, it is no longer possible to build a single kernel with support for both the 80386 processor and later processors: the code for the later processors is optimized to use instructions that the 80386 processor does not have.Choose either I386_CPU or anycombination of the others. ident machine_name ident specifies a name used to identify the kernel. In the file GENERIC it is GENERIC. Change this to whateveryou named your kernel, in this example, FREEBIE.The value you put in ident will print when you boot up the kernel, so it’suseful to give a kernel a different name if you want to keep it separate from your usual kernel (if you want to build an experimental kernel, for example). As with machine and cpu,enclose your kernel’s name in quotation marks if it contains anynumbers. This name is passed to the C compiler as a variable, so don’tuse names like DEBUG,or something that could be confused with another machine or CPU name, like vax. Kernel options There are a number of global kernel options, most of which you don’tneed to change. In the following section, we’ll look at some of the fewexceptions. Configuring specific I/O devices There are some devices that the GENERIC kernel does not support. In older releases of FreeBSD, you needed to build a newkernel to support them. This is seldom the case any more: most devices can be supported by klds. Work is under way to support the remainder.Incase of doubt, look at the file HARDWARE.TXT on the installation CD- ROM. building.mm,v v4.15 (2003/04/02 06:30:12) 605 Chapter 33: Custom ker nels 2April 2003, 17:00:47 The Complete FreeBSD (building.mm), page 605 maxusersnumber This value sets the size of a number of important system tables. It is still included in the kernel configuration file, but you no longer need build a newkernel to change it. Instead, you can set it at boot time. Forexample, you might add the following line to your /boot/loader.conf file: maxusers="64" See 527 for more details of /boot/loader.conf. maxusers is intended to be roughly equal to the number of simultaneous users you expect to have onyour machine, but it is only used to determine the size of some system tables. The default value 0 tells the kernel to autosize the tables depending on the amount of memory on the system. If the autosizing doesn’twork to your satisfaction, change this value. Even if you are the only person to use the machine, you shouldn’tset maxusers lower than the default value 32, especially if you’re using X or compiling software. The reason is that the most important table set by maxusers is the maximum number of processes, which is set to 20 + 16 * maxusers,soifyou set maxusers to one, you can only have 36simultaneous processes, including the 18 or so that the system starts up at boot time, and the 15 or so you will probably create when you start X. Even a simple task likereading a man page can start up nine processes to filter,decompress, and viewit. Setting maxusers to 32 will allowyou to have upto532 simultaneous processes, which is normally ample. If, however, you see the dreaded proc table full error when trying to start another program, or are running a server with a large number of simultaneous users, you can always increase this number and reboot. maxusers does not limit the number of users who can log into the machine. It simply sets various table sizes to reasonable values considering the maximum number of users you will likely have on your system and howmanyprocesses each of them will be running. It’sprobable that this parameter will go awayinfuture. Multiple processors FreeBSD 5.0 supports most modern multiprocessor systems. The GENERIC kernel does not support them by default. For i386, set the following options: #Tomake an SMP kernel, the next two are needed options SMP #Symmetric MultiProcessor Kernel options APIC_IO #Symmetric (APIC) I/O ForAlpha, only the first line is necessary: options SMP #Symmetric MultiProcessor Kernel An SMP kernel will not run on a single processor Intel machine without an IOAPIC (I/O Advanced Programmable Interrupt Controller)chip. Be sure to disable the cpu "I386_CPU" and cpu "I486_CPU" options for SMP kernels. building.mm,v v4.15 (2003/04/02 06:30:12) The configuration file 606 2April 2003, 17:00:47 The Complete FreeBSD (building.mm), page 606 Debug options FreeBSD is a very stable operating system. No software is perfect, however, and sometimes it crashes. When it does, it provides a number of facilities to help fix the problem. Some of these are dependent on kernel build options. Even if you have nointention of debugging a kernel problem yourself, you should set debug symbols when you build a kernel. Theycost nothing except disk space, and if you are short on disk space, you can remove most of the files after the build. To set the debug symbols, remove the leading # mark from this line in the configuration file: makeoptions DEBUG=-g #Build kernel with gdb(1) debug symbols Under normal circumstances this makes no difference: the build process still installs the kernel without the debug symbols, and it has no effect on performance. If, however, you have a crash, the kernel with debug symbols is available in the kernel build directory,in this case /usr/src/sys/i386/compile/FREEBIE/kernel.debug,toassist analysis of the problem. Without this file it will be very difficult to find it. So whyisitcommented out? Without debug symbols, your build directory will take about 50 MB of disk space. With debug symbols, it will be about 250 MB. The FreeBSD Project couldn’tagree to change it. Other debugging options If you run into trouble with your system, there are a number of other debugging options that you can use. The following are the more important ones: options DDB options BREAK_TO_DEBUGGER options DDB_UNATTENDED #Don’t drop into DDB for a panic options GDB_REMOTE_CHAT #Use gdb remote debugging protocol options KTRACE options DIAGNOSTIC options INVARIANTS options INVARIANT_SUPPORT options WITNESS #Enable checks to detect deadlocks and cycles options WITNESS_SKIPSPIN #Don’t run witness on spinlocks for speed These options provide support for various debugging features. DDB Specify DDB to include the kernel debugger, ddb.Ifyou set this option, you might also want to set the BREAK_TO_DEBUGGER option, building.mm,v v4.15 (2003/04/02 06:30:12) 607 Chapter 33: Custom ker nels 2April 2003, 17:00:47 The Complete FreeBSD (building.mm), page 607 BREAK_TO_DEBUGGER Use the option BREAK_TO_DEBUGGER if you have installed the kernel debugger and you have the system console on a serial line. DDB_UNATTENDED If you have a panic on a system with ddb,itwill not reboot automatically.Instead, it will enter ddb and give you a chance to examine the remains of the system before rebooting. This can be a disadvantage on systems that run unattended: after a panic, theywould wait until somebody comes past before rebooting. Use the DDB_UNATTENDED option to cause asystem with ddb to reboot automatically on panic. GDB_REMOTE_CHAT ddb supports remote debugging from another FreeBSD machine via a serial connection. See the online handbook for more details. To use this feature, set the option GDB_REMOTE_CHAT. KTRACE Set KTRACE if you want to use the system call trace program ktrace. DIAGNOSTIC Anumber of source files use the DIAGNOSTIC option to enable extra sanity checking of internal structures. This support is not enabled by default because of the extra time it would taketocheck for these conditions, which can only occur as a result of programming errors. INVARIANTS and INVARIANT_SUPPORT INVARIANTS is used in a number of source files to enable extra sanity checking of internal structures. This support is not enabled by default because of the extra time it would taketocheck for these conditions, which can only occur as a result of programming errors. INVARIANT_SUPPORT option compiles in support for verifying some of the internal structures. It is a prerequisite for INVARIANTS.The intent is that you can set INVARIANTS for single source files (by changing the source file or specifying it on the command line) if you have INVARIANT_SUPPORT enabled. WITNESS and WITNESS_SKIPSPIN One of the big changes in FreeBSD Release 5 relates to the manner in which the kernel performs resource locking. As a result, the danger exists of deadlocks,locks that can’tbe undone without rebooting the machine. WITNESS checks for the danger of deadlocks and warns if it finds a potential deadlock (‘‘lock order reversal’’). This is a very expensive debugging option: it can slowthe machine down by an order of magnitude, so don’tuse it unless you have to. Acompromise that doesn’tuse quite so much processor power is the combination of building.mm,v v4.15 (2003/04/02 06:30:12) The configuration file 608 2April 2003, 17:00:47 The Complete FreeBSD (building.mm), page 608 WITNESS with WITNESS_SKIPSPIN,which avoids spin locks. It can still catch most problems. Preparing for upgrades When changing the configuration file, consider that it probably won’tbethe only time you makethese changes. At some time in the future, you’ll upgrade the system, and you’ll have tobuild your custom kernel all overagain. But the GENERIC kernel configuration file will have changed as well. Youhav e twochoices: incorporate the modifications to GENERIC into your configuration file, or incorporate your modifications to the old GENERIC file into the new GENERIC configuration file. It turns out that the latter path is easier. To prepare for this approach, try to change as little as possible in the body of the configuration file. Instead, add all your changes to the end, along with a comment so that you can readily recognize the changes. Forexample, you might append: #Added by grog, 24 October 2002 #Comment out WITNESS, WITNESS_SKIPSPIN and SCSI_DELAY above options SMP #Symmetric MultiProcessor Kernel options APIC_IO #Symmetric (APIC) I/O options BREAK_TO_DEBUGGER options SCSI_DELAY=3000 #Delay (in ms) before probing SCSI options CAMDEBUG options MSGBUF_SIZE=81920 #options TIMEREQUESTS #watch for delays device snp #Snoop device - to look at pty/vty/etc That won’tbeall, of course. Look at that option SCSI_DELAY.That option already exists in the configuration file (with a value of 15 seconds instead of 3). If you leave both, config will issue a warning. You need to comment out the first occurrence, as the comment at the top indicates. Building and installing the newkernel The traditional way to build a BSD kernel was, assuming a configuration file called FREEBIE: # cd /usr/src/sys/i386/conf # config FREEBIE # cd /compile/FREEBIE # make depend all install At the time of writing, this still works, but it will go awayatsome time in the future. It has the disadvantage that you need to knowyour architecture, and the kernel is built in building.mm,v v4.15 (2003/04/02 06:30:12) [...]... FreeBSD-current@FreeBSD.org mailing list, which you can join via majordomo See page 17 for details of how to use majordomo Build kernels with debug symbols FreeBSD-CURRENT is not as stable as the released releases To prepare yourself for possible problems, you should build kernels that include debug symbols The resultant kernel is about 30 MB in size, but it will make debugging with ddb (the kernel... the most recent kernel messages from the current boot After some time building.mm,v v4.15 (2003/04/02 06:30:12) 2 April 2003, 17:00:47 The Complete FreeBSD (building.mm), page 611 611 Chapter 33: Custom kernels the original messages are overwritten, so the system startup procedure saves the messages at boot time in the file /var/run/dmesg.boot Note that most laptops maintain the previous contents of the... kernel message buffer, which is also listed by the dmesg program building.mm,v v4.15 (2003/04/02 06:30:12) 2 April 2003, 17:00:47 The Complete FreeBSD (building.mm), page 613 613 • Chapter 33: Custom kernels kern.corefile specifies a template for the name of the core dump file generated when a process fails By default the core file ends up in the current working directory—whatever that might be By specifying...2 April 2003, 17:00:47 The Complete FreeBSD (building.mm), page 609 609 Chapter 33: Custom kernels the source tree, which is not a good idea The new kernel build method starts from /usr/src, the same directory as all other build operations, and it builds the kernel in the /usr/obj hierarchy... one you need The easiest way to access it is to use a symbolic link: building.mm,v v4.15 (2003/04/02 06:30:12) 2 April 2003, 17:00:47 The Complete FreeBSD (building.mm), page 615 615 Chapter 33: Custom kernels # ln -s /usr/src/sys/i386/conf/FREEBIE/kernel.debug # ls -lL -rw-r r 1 root wheel 3 Dec 29 10:09 bounds -rwxr-xr-x 1 grog lemis 16796546 Dec 18 14:21 kernel.debug -rw-r r 1 root wheel 4333000... system prepares to dump and writes to disk They’re no longer of interest building.mm,v v4.15 (2003/04/02 06:30:12) 2 April 2003, 17:00:47 The Complete FreeBSD (building.mm), page 617 617 Chapter 33: Custom kernels Finding out what really happened In general, you start analyzing a panic dump in the stack frame that called panic, but in the case of the fatal trap that we have here, the most important stack . Build kernels with debug symbols FreeBSD-CURRENT is not as stable as the released releases. To prepare yourself for possible problems, you should build kernels. information in this chapter represents a snapshot in the evolution of building kernels. The goal of these changes is to makeitunnecessary to build a kernel