Example 14-9. Adding the New Logical Logs curtis$ oninit -s curtis$ for I in 1 2 3 4 5 6 ; do > onparams -a -d llogdbs > done Logical log successfully added. Logical log successfully added. Logical log successfully added. Logical log successfully added. Logical log successfully added. Logical log successfully added. You now must perform a level-0 archive to make these logs available for use. (An example level-0 archive is shown in Example 14-10.) Example 14-10. A Level-0 Archive curtis$ ontape -s Please enter the level of archive to be performed (0, 1, or 2) 0 Please mount tape 1 on /informix/logical/crash/crash.level.0 and press Return to continue 100 percent done. Please label this tape as number 1 in the arc tape sequence. This tape contains the following logical logs: 35 Program over. As shown in Example 14-11, we now switch logs three times, which will cause the current log to be one of the new logs on the separate dbspace. Then we will force a checkpoint and perform a logical log backup. This will free the logs that are on the old dbspace, so that we may drop them. Example 14-11. Performing a Log Switch and Starting Continuous Backups curtis$ onmode -l ; onmode -l ; onmode -l curtis$ onmode -c curtis$ ontape -c Page 397 Example 14-11. Performing a Log Switch and Starting Continuous Backups (continued) Performing continuous backup of logical logs. Please mount tape 1 on /informix/logical/crash.log and press Return to continue In another window, we run an onstat -l that shows us that the logical log backup is done. We now can stop the ontape -c with a Ctrl-C. We also can drop the original three logical logs. See Example 14-12 for an example of these steps. Example 14-12. Example onstat Output and onparams Command curtis$ onstat -l # (output abbreviated) address number flags uniqid begin size used a04b424 1 U-B 34 100233 250 1 a04b440 2 U-B 35 10032d 250 15 a04b45c 3 U-B 36 100427 250 0 a04b478 4 U C-L 0 300035 250 0 a04b494 5 F 0 30012f 250 0 a04b4b0 6 F 0 300229 250 0 a04b4cc 7 F 0 300323 250 0 a04b4e8 8 F 0 30041d 250 0 a04b504 9 F 0 300517 250 0 curtis$ onparams -d -l 1 WARNING: Dropping a logical log file. Do you really want to continue? (y/n)y Logical log 1 successfuly dropped. curtis$ onparams -d -l 2 WARNING: Dropping a logical log file. Do you really want to continue? (y/n)y Logical log 2 successfully dropped. curtis$ onparams -d -l 3 WARNING: Dropping a logical log file. Do you really want to continue? (y/n)y Logical log 3 successfully dropped. curtis$ onstat -l # (output abbreviated) address number flags uniqid begin size used a04b478 4 U C-L 37 300035 250 7 a04b4 94 5 F 0 30012f 250 0 a04b4 94 5 F 0 30012f 250 0 a04b4b0 6 F 0 300229 250 0 a04b4cc 7 F 0 300323 250 0 a04b4e8 8 F 0 30041d 250 0 a04b504 9 F 0 300517 250 0 At this point, we have successfully moved all logical logs into the new dbspace. The following steps are purely a matter of personal preference. I like my logs to be numbered starting with one. In order to do that, I have to add three more and drop the last three, as demonstrated in Example 14-13. The new logs will be given Page 398 log numbers 1-3. Then we drop logs 7-9 in reverse order. (It's less confusing to delete them in reverse order.) Example 14-13. Dropping and Adding Logical Logs curtis$ onparams -a -d llogdbs Logical log successfully added. curtis$ onparams -a -d llogdbs Logical log successfully added. curtis$ onparams -a -d llogdbs Logical log successfully added. curtis$ onparams -d -l 9 WARNING: Dropping a logical log file. Do you really want to continue? (y/n)y Logical log 9 successfully dropped. curtis$ onparams -d -l 8 WARNING: Dropping a logical log file. Do you really want to continue? (y/n)y Logical log 8 successfully dropped. curtis$ onparams -d -l 7 WARNING: Dropping a logical log file. Do you really want to continue? (y/n)y Logical log 7 successfully dropped. curtis$ onstat -l # (output abbreviated) address number flags uniqid begin size used a04b424 1 A 0 300611 250 0 a04b424 1 A 0 300611 250 0 a04b440 2 A 0 30070b 250 0 a04b45c 3 A 0 300805 250 0 a04b478 4 U C-L 37 300035 250 16 a04b494 5 F 0 30012f 250 0 a04b4b0 6 F 0 300229 250 0 Recommendation: Mirror Essential dbspaces If you have the space, it is best to mirror the entire database. However, many places do not have that luxury. It doesn't take up much disk space to mirror the essential dbspaces, though. The essential dbspaces are those that contain the sys-master database and the physical and logical logs. Of these, the most essential dbspace would be the one that contains the logical logs. That is because all other dbspaces can be recovered up to the point of failure-if the logical logs are intact. When creating dbspaces, they can be set up as mirrored dbspaces. (This is demonstrated in the previous example about moving your logs to dedicated dbspaces.) However, it also can be done after the dbspace is created. Since we've already covered creating a mirrored dbspace, this section covers turning on mirroring for an existing dbspace. Assume for this example that we have a dbspace called Page 399 plogdbs, and that we wish to mirror it to /informix/physlog_mirror.dbf. We would run the commands shown in Example 14-14. Example 14-14. Adding Mirroring to an Existing dbspace $ onspaces -m plogdbs -f /informix/physlog_mirror.dbf -y $ onstat -d # (output abbreviated) a12a728 3 2 3 1 M informix plogdbs (The M flag above says that the plogdbs is mirrored.) address chk/dbs offset size free bpages flags pathname (It lists two chunks below. One is primary and the other secondary.) a12a358 2 2 0 5000 2447 PO- /informix/physlog.dbf a12a508 2 2 0 5000 0 MO- /informix/physlog_mirror.dbf Recommendation: Leave Room for the Logs Remember that the logical logs are absolutely essential for maintaining the integrity of your database. A common problem with many installations is that the logs become full. This is caused by either not backing up the logs or a long transaction that keeps too many logs open. Back up your logical logs Informix instances must have their logical logs backed up. If you do not back them up, all logical logs eventually become full. Once this happens, all activity on the instance stops, and you are forced to do a logical log backup. The problem is that backing up the logical logs requires creating more logical log entries. If all logs are full, you can't back up the logs. The only option that remains is a special tool that can be run only by Informix support. They have to be able to dial or telnet into your system. They then transfer the file to your system, run it, and remove it. They do not allow you to keep the tool, and there is no way for you to get it without Informix accessing your system. In short, this is not a situation you want to be in. Informix recently introduced a new parameter that prevents logical log entries from becoming full. Unfortunately, it is turned off by default.* The LBU_PRESERVE parameter (found in the onconfig file) needs to be set to 1 to reserve the last logical log for administrative activity, such as logical log backups. Long transactions Another "log full" problem is caused by a long transaction. A long transaction occurs when a user or application starts a transaction and does not commit it * I really don't understand this one. This is such a bad situation to get into, I can't imagine anyone who wouldn't want to turn on this new feature. Page 400 within a reasonable time. The problem with this is that a log cannot be freed if it contains an uncommitted transaction. Suppose there are 10 logical logs, and a long transaction is started in the first log. Once the first log is full, Informix moves on to the next log, but it cannot free the first log for reuse until the long transaction is completed and that log file has been backed up. If the transaction were allowed to remain open until the 10 th log was full, there would be no more free logs. One transaction has managed to span the entire set of logical logs. This is why it is called a long transaction. One might think that the transaction could be rolled back. However, rolling back a transaction requires creating more logical log entries. No more logical log entries can be created because there are no more free logical logs. Does that sound familiar? The long-transaction problem is why they introduced the high-water-mark parameters that are found in the onconfig file. The first is the long-transaction high-water mark (LTXHWM), and its default value is 50. Once the percentage of full logical logs reaches this value, the transaction is automatically aborted and rolled back. However, since other transactions are allowed to continue, the logical logs are still in danger of filling up. The second parameter, the long-transaction exclusive-access high-water mark (LTXEHWM), is designed to protect the instance from this; its default value is 60. If the percentage of full logs reaches this value, most database activity is suspended and the transaction is given "exclusive access" to the remaining logical logs so that it can complete its rollback. The values of 50 and 60 are the new default values for these parameters, and oninit complains if you attempt to initialize an instance with values higher than these. In summary, leave the LTXHWM and LTXEHWM values where they are and turn on LBU_PRESERVE by changing its value to 1 in the onconfig file. Fore more details about these values, consult the IDS Administration Guide. Which Backup Utility Should I Use? Before examining how to perform Informix backups, we need to understand the various backup options that are available to an Informix DBA. There are three backup utilities available with Informix: onbar, onarchive, and ontape. A Quick History of onarchive, onbar, and ontape Around 1993 and 1994, very large Informix databases started to appear. (Back then, "large" meant anything greater than 100 GB.) The only utility that Informix customers had was ontape. ontape's one big flaw is that it is single-threaded. How do you back up 100 GB to a single device, especially when the devices available could back up only at rates ranging from 500 KB/s to 3 MB/s? Informix customers started screaming for a multithreaded backup system. (I was one of them.) Page 401 Informix, in an attempt to meet these demands, purchased the rights to a thirdparty program that became known as onarchive. Its syntax was arcane, complex, and completely unlike any other command on Unix. (It wasn't as complex as sendmail, but it was close!) The forward-slash method of passing options to a command seemed bizarre to most Unix system administrators, and it came with a manual that is almost the size of this book. I was there By the time in my career that I met onarchive, I had learned dozens of products just by browsing the manuals. I read the onarchive manual cover to cover four times and attended the Informix onarchive class. I was still confused. The most disconcerting part of the class was how the instructor kept talking about this thing called onbar that would soon replace onarchive. I remember having 15 open bug reports for onarchive at one time. I remember calling support and getting the feeling that they didn't want to support the product any more than I wanted to use it. I remember spending months configuring and testing our onarchive setup. Informix came and saw what we did and thought it was one of the most impressive onarchive installations they had ever seen. Then we tested the restore. It didn't work. No matter what we did, we couldn't restore the instance that we had backed up with onarchive. It just wouldn't work. Suddenly we received approval to buy a much faster tape drive and go back to ontape. As a result of the input that they received from the field, Informix began work on a replacement for onarchive almost immediately. onbar was released with Version 7.21. Unfortunately, this created a problem. Users who need the multithreaded capabilities of onbar were required to purchase an interface to their storage manager-at a cost of more than $5000 per system. To avoid that cost, people who needed a multithreaded backup product were still being driven toward onarchive; this product, despite its incredible complexity and horrible reputation, remained the only choice available for some customers. Informix wanted to fix this problem, so they negotiated an arrangement with Legato Systems that allowed them to bundle a scaled-down version of Legato NetWorker with IDS 7.3. This product is known as Informix Storage Manager, or ISM. It allows you to send up to four data streams simultaneously to four storage devices (i.e., one data stream per drive). This is enough parallelism to back up a relatively large instance. Assuming that each data stream can "stream" each of four DLT 7000s, that is a total potential throughput of 40 MB/s, 144 GB/hr, or 1.1 TB in an eight-hour period. (Unfortunately, on many systems, you'll need more than one thread to stream a device; this reduces the overall effective throughput of an ISM/onbar combination, but it is still going to be faster than ontape.) Page 402 Don't use onarchive If you downloaded both the Archive and Backup Guide and the Backup and Restore Guide from http://www.informix.com/answers, you might get the impression that there are three viable options for backing up Informix databases: ontape, onbar, and onarchive. The manuals do explain that onbar is designed to work with storage managers, and ontape and onarchive are not. What the documentation does not explain is why two products (ontape and onarchive) that appear to perform the same function exist. Further, you might get the impression after reading the manual that onarchive is the way to go. It's not. The release of ISM means that there remains no valid reason for configuring a new backup system using onarchive. If you can live without multithreaded backup and recovery, use ontape and infback.sh (or something similar). If you need multithreaded capability or want some other advanced feature, use onbar and ISM. If you can afford an Informix server that needs more than four backup threads, you can afford a full-featured storage manager and its interface to onbar. If you're having trouble convincing your management to spend that money, show them this chapter. Informix plans to phase out onarchive over time. You can already see references that say things like "onbar is the preferred solution for 7.3," and "ontape and onarchive are supported for now." The only reason that they don't drop onarchive today is there are still people using it. I wouldn't be surprised if a future release of Informix configures onarchive in such a way that you would no longer be able to make backups with it. The restore functionality will be left for those who still have onarchive backup tapes. (I doubt that Informix will drop ontape, though, because it's so easy to use and support. They aren't adding any new features to it, but that could be considered a feature in itself.) Some people reading this section may disagree with my assessment of onarchive's value. They like the flexibility that such a complex product offers them. They don't care that they need to read almost 500 pages of documentation just to figure out how to back up their Informix database. They've figured it out now-and they don't want to have to figure out another product. If that is how you feel, please know that you are in the minority. The whole reason that onbar exists is that the Informix user community was in an uproar about onarchive. Remember also that a product that is difficult to use is difficult to support. Informix is probably doing everything it can to put this utility behind them. If you're using onarchive today, you should seriously consider looking at onbar as soon as possible or going back to ontape if you can. Trust me: onbar is as easy as it gets. Page 403 Pick a Utility, but Not Any Utility Here is a quick summary of the three utilities available for Informix backups: onbar onbar is the latest in Informix backup technology and is used only for interfacing with storage managers like the Informix Storage Manager (ISM). (onbar is covered in "Physical Backups with a Storage Manager: onbar.") If you are running Informix 7.3 or greater, onbar offers enhanced functionality and flexibility. onarchive onarchive was Informix's first attempt at an enhanced product, but its incredible complexity caused it to be just a stopgap measure until onbar was available. onarchive will not be supported in the future and therefore should not be used as the basis for any new backup system. (See "Don't use onarchive," earlier in this chapter, for more information.) ontape ontape is Informix's oldest backup utility, but it's still going strong. It does not interface with storage managers, therefore it is the easiest and best way to perform physical backups without a storage manager. In summary, if you do not have a storage manager, you really should be using ontape if you can. If your database is really large, and you need the multithreaded capabilities of onbar, you should either purchase a third-party storage manager or upgrade to a recent version of Informix that includes a free (albeit scaled-down) storage manager. Once you have done that, you can use the advanced functionality of onbar. I would not advise designing any new backup systems around onarchive. It is too difficult to learn, and it is being phased out. Physical Backups Without a Storage Manager: ontape If you do not have access to a storage manager, ontape might be the perfect tool for you. This section covers ontape's basic usage, then shows you how to automate it using shell scripts. Many environments without storage managers use ontape to automate their backups. Despite its age, many people continue to use ontape for several reasons: Informix version ontape does not require a storage manager. If you are not running Informix 7.3 or greater, you do not have ISM. That means that you will need to purchase a Page 404 storage manager and its interface to onbar. If you are running a version older than Informix 7.21, you don't even have access to onbar. Cost ontape is free. Although onbar itself is free, it will cost as much as $7000 per host* to purchase the interface to onbar, even if you already have one of the commercial storage management applications (e.g., those discussed in Chapter 5, Commercial Backup Utilities). Ease of use ontape is a breeze to learn. It has four required options, -s for archive,** -c for continuous backups, -a for automatic backups, and -r for restore. That's it. In contrast, onarchive is a nightmare of complexity. Its manual spans 436 pages, compared to ontape's 49 pages or onbar's 125 pages. onbar is much simpler to use than onarchive, but adding a storage manager into the picture still makes it more complicated than ontape. (The slight increase in complexity does come with greatly increased functionality, though.) ontape is easy to use. Even more importantly, ontape recoveries work! ontape was developed many years ago, before the advent of backup automation in the Unix world. It was ahead of its time with multilevel backups, but it does have one design flaw: it was designed to be run by an operator on a console. It prompts the operator for information even during normal operations. It was not designed for automated backups, but that doesn't mean you can't use it that way. Many scripts have been written that answer its prompts, providing a level of functionality never envisioned by the original programmers. Before looking at how to automate the use of ontape, though, we cover its basic usage. Configuring ontape Prior to running ontape, you must configure some parameters using the onmonitor utility. This utility changes these parameters in the file $INFORMIXDIR/etc/$ONCONFIG, where $INFORMIXDIR is a required environment variable specifying the Informix directory, and $ONCONFIG is a required environment variable specifying the name of the Informix configuration file. It is an editable text file, but not all changes will go into effect until you restart the instance if you edit it manually. Specifically, the LTAPE parameters must be changed using onmonitor if you want them to go into effect immediately. To configure ontape, run the onmonitor utility, choosing menu option A (for Archive) and T (for Tape Parameters). (An * Every commercial backup application charges per host for its interface to the database backup applications. In a large Informix environment, this can therefore reach $100,000 very quickly. It does provide significantly better functionality, but it is difficult to justify such an expense for a little 1-GB instance. ** I finally figured out why archive is -s! It stands for ''save"! Page 405 ontape backup of an instance is referred to as an archive.) This takes you into a screen that looks like Figure 14-1. Press ESC to change tape parameters. Press Interrupt to return to Archive menu. Press F2 or CTRL-F for field level help. Press F2 or CTRL-F for field level help. MODIFYING TAPE PARAMETERS Tape Device [/dev/rmt/0c ] Block Size [ 32] Kbytes Tape Size [ 102400] Kbytes Log Tape Device [/dev/rmt/0c ] Block Size [ 32] Kbytes Tape Size [ 10240] Kbytes Figure 14-1. onmonitor tape parameters menu The values in Figure 14-1 should be changed to ones appropriate for your environment. The appropriate values are also listed in Table 14-1. Table 14-1. Tape Configuration Values Variable in the $ONCONFIG file Value in onmonitor Screen Purpose TAPEDEV Tape Device Device to which archives (backups) are sent TAPEBLK Block Size Block size of tape device in kilobytes TAPESIZE Tape Size Size of tape in kilobytes LTAPEDEV Log Tape Device Device to which logical log backups are sent LTAPEBLK Block Size Block size of log device in kilobytes LTAPESIZE Tape Size Specifies the tape device for logical log backups. These values usually are set once and should be changed only if your configuration changes. One of the best ways to do this is to make the device name to which you are backing up a symbolic link to the real device. For example, you can specify that the tape device is /informix/tapedev and make that a symbolic link to /dev/rmt/Ocn. That would allow you to change the actual tape device without accessing onmonitor. Although optimum block size settings are determined by your tape drive manufacturer, a value of 64 or 128 is quite common. You should experiment with different values to see how they affect backup performance.* * It is usually true that the bigger the block size, the better the backup performance. However, in recent tests I found that a block size of 64 KB was actually faster than a block size of 128 KB. Your mileage may vary. Page 406 I like to set the TAPESIZE parameter to something slightly larger than the instance size but less than the actual tape size. For example, suppose you have a 16-GB tape but a 4-GB instance. One option would be to set the tape size to 15 GB and not worry about changing it again for a long time. You also could set it at 5 GB, and then watch the logs to make sure that [...]... details on how to do this.) Since no actual backup occurs, this is not an option to use if you will need point-in-time recovery Using this option allows you to recover only from your latest archive You will not be able to redo any transactions since the time of that archive This option typically is used only for development instances or instances in which data is loaded only via batch loads If you are considering... of ontape When continuous backups are running, logical logs are automatically backed up to the specified device as soon as they are full The logs are appended to the backup device, which is held open by Informix as long as continuous backups are running To perform a continuous backup of the logical logs, run the following command: $ ontape -c This command prompts you to hit Enter once you have mounted... 1: Does oninit Work? The obvious first step in determining if an instance is in need of recovery is to try and start the instance Do this by issuing the oninit command with no options If it works, it just returns the prompt to you You also could see one of two errors curtis $ oninit WARNING: Cannot access configuration file $INFORMIXDIR/etc/$ONCONFIG This error is pretty obvious If you see it, you are... basic configuration Once properly configured, you should be able to back up your Informix database from either side of the system That is, you should be able to run a backup from your storage manager and have it automatically activate onbar, causing it to begin sending an Informix backup to the storage manager You also should be able to execute onbar commands from the Informix command line and have... Use a storage manager and onbar The second option, and one you should seriously consider, is to use Informix Storage Manager (or some other storage manager) and onbar This combination would work around the 2-GB limit, still allow you to back up to disk if you wish, while providing you with the features that are available only in onbar Backing Up the Instance The Informix term for a backup of an instance... manager This section of this chapter, then, serves only as an overview of what lies before you if you choose to use onbar Configuring onbar The first thing that you need to do is make sure that you have recent releases and patches of both onbar and your storage manager Then, look for the following values in your onconfig file: # Backup/ Restore variables BAR_ACT_LOG /tmp/bar_act.log BAR_MAX _BACKUP 0 BAR_RETRY... logs You can use the ALARMPROGRAM parameter to automatically run a shell script for you when a log is full or is manually closed Elect not to back up the logs You could decide not to back up the logical logs This is done by setting the backup device to /dev/null Backing up logical logs continuously The historically preferred way to back up your logical logs is to use the continuous backup option of... functionality is not available with ontape Backing up the logical logs The following command is somewhat equivalent to the ontape -a command, since it backs up all full logs However, unlike the ontape -a command, it is the preferred command to back up the logical logs with onbar In fact, the following log_full.sh script uses this option to do continuous backups One main difference is that it does not... level-0 and/ or level-1 backups, as well as the location of all logical log backups since your last physical backup If you are using onbar, this procedure assumes you know how to use your storage manager well enough that you know how to respond to its prompts for any media that may not be in an autochanger In short, all media management is up to you The following examples use ontape The example hostname... location It doesn't do any good to back up your logical logs to a tape or disk and then leave that backup sitting right on the server You need to move the backup to another location When backing up to tape, this must be done manually of course When backing up to disk, this can be automated by copying the file to an NFS-mounted filesystem or running another backup program to back it up to a tape somewhere . prompt you for a second tape. You then could use expect to move the first file from its original location, create another file, and respond to ontape's prompt for a second tape. This method. ontape -c This command prompts you to hit Enter once you have mounted the tape; from then on, it performs a continuous backup of the logical logs to the storage media until you enter Ctrl-C or kill. this is not an option to use if you will need point-in-time recovery. Using this option allows you to recover only from your latest archive. You will not be able to redo any transactions since