Testing of Trips and Other Protective Systems 273 [a) A high-temperature trip on a furnace failed to operate. The furnace was seriously damaged. The trip did not operate because the pointer touched the plastic front of the instrument case, and this prevented it from moving to the trip level. The instrument had been tested regu- larly-by injecting a current from a potentiometer-but to do this the iizstriiment was removed porn its case and taken to the workshop. (b) A reactor was fitted with a high-temperature trip, which closed a valve in the feed line. When a high temperature occurred. the trip valve failed to close although it had been tested regularly. Investigation showed that the pressure drop through the trip valve-a globe valve-was so high that the valve could not close against it. There was a flow control valve in series with the trip valve (Figure 14- 1). and the trip normally closed this valve as well. However. this valve failed in the open position-this was the rea- son for the high temperature in the reactor-and the full upstream pressure was applied to the trip valve. Emergency valves should be tested against the maximum pres- sure or flow they may experience and, whenever possible, should be installed so that the flow assists closing. (c) If the response time of protective equipment is important. it should always be measured during testing. For example. machinery is often interlocked with guards so that if the guard is opened, the machinery stops. Brakes are often fitted so that the machinery stops quickly. The actual stopping time should be measured at regular intervals and compared with the design target. Another example: a mixture of a solid and water had to be heat- ed to 300°C at a gauge pressure of 1.000 psig (70 bar) before the To Reactor * Flow Control Valve Trip Valve Usually closes when trip operates but had failed in open position. Kept open by line pressure when flow control valve is fully open. Figure 11-1. When the control valve was open, the pressure prevented the trip valve from closing. 274 What Went Wrong? solid would dissolve. The mixture was passed through the tubes of a heat exchanger while hot oil, at low pressure, was passed over the outside of the tubes. It was realized that if a tube burst, the water would come into direct contact with the hot oil and would turn to steam with explosive violence. An automatic system was therefore designed to measure any rise in the oil pressure and to close four valves, in the water and oil inlet and exit lines. The heat exchanger was also fitted with a rupture disc, which discharged into a catch- pot. The system was tested regularly, but nevertheless, when a tube actually burst most of the oil was blown out of the system and caught fire, as the valves had taken too long to close. They had been designed to close quickly but had gotten sluggish; the time of response was not measured during the test, so no one knew that they were not responding quickly enough. Procedures. like equipment, also take time to operate. For exam- ple, how long does it take to empty your building when the fire alarm sounds? Is this quick enough? (d)A large factory could be supplied with emerency power from a diesel-driven generator. It was tested regularly to ensure that the diesel engine started up when required. When the power supply actually failed, the diesel generator started up, but the relay that connected it to the distribution system failed to operate. The emergency supply was tested when the distribution system was live. No one understood how the emergency circuits worked and did not realize that they were not being thoroughly tested [2]. (e) An example from another industry: for many years railway carriage doors in the United Kingdom opened unexpectedly from time to time. and passengers fell out. Afterward the locks were removed from the doors and sent for examination. No faults were found, and it was concluded that passengers had opened the doors. However. it was not the locks that were faulty but the alignment between the locks and the recesses in the doors. This was faulty and allowed them to open [3]. (f) A plant was pressure-tested before startup. but the check valves (nonreturn valves, NRV) in the feed lines to each unit (Figure 14-2) made it impossible to test the equipment to the left of them. A leak of liquefied petroleum gas (LPG) occurred during startup at the Testing of Trips and Other Protective Systems 275 To No. 1 Unit To No. 2 Unit To No. 3 Unit Figure 14-2. The check valves (nonreturn valves NRV) prevented a leak test of the equipment to the left of them. During startup a leak occurred at the point indicated. point indicated. The three check valves were then replaced by a sin- gle one in the common feed line at the extreme left of the diagram. (g) Before testing an interlock or isolation to make sure it is effective. ask what will happen if it is not. For example, if a pump or other item oE equipment has been electrically isolated by removing the fuses. it should be switched on to check that the correct fuses have been withdrawn. Suppose they have not; will the pump be dam- aged by starting it dry? A radioactive source was transferred from one container to another by remote operation in a shielded cell. A radiation detector, interlocked with the cell door, prevented anyone from opening the cell door when radiation could be detected inside it. To make sure the interlock was working, an operator tried to open the cell door. by remote control. during a transfer. He found he could open it. He then found that the closing mechanism would not work. Fortunate- ly he had not opened the door very far. (h) Do not test a trip or interlock by altering the set-point. The trip or interlock may operate at the altered set-point, but that does not prove it will operate at the original set-point. h PROTECTIVE EQUIPMENT SHOULD BE TESTED This section lists some protective equipment that has often been over- looked and not included in testing schedules. 276 What Went Wrong? 14.2.1 Leased Equipment After a low-temperature trip on a nitrogen vaporizer failed to operate, it was found that the trip was never tested. The equipment was rented, and the user assumed-wrongly-that the owner would test it. 14.2.2 Emergency Valves A pump leaked and caught fire. It was impossible to reach the suction and delivery valves. But there was a second valve in the suction line between the pump and the tank from which it was taking suction, situat- ed in the tank dike. Unfortunately this valve was rarely used and was too stiff to operate. All valves-whether manual or automatic-that may have to be oper- ated in an emergency should be tested regularly (weekly or monthly). If completely closing a valve will upset production, it should be closed halfway during testing and closed fully during shutdowns. Emergency blowdown valves are among those that should be tested regularly. Reference 5 describes in detail the measures necessary to test emergency isolation valves when very high reliability is needed. 14.2.3 Steam Tracing A furnace feed pump tripped out. The flowmeter was frozen, so the low-flow trip did not operate. Two tubes burst, causing a long and fierce fire. The structure and the other tubes were damaged, and the stack col- lapsed. In cold weather, the trace heating on instruments that form part of trip and alarm systems should be inspected regularly. This can be part of the test routine, but more frequent testing may be necessary. 14.2.4 Relief Valves, Vents, Flame Arrestors, Etc. Section 10.4.2 lists some items that should be registered for inspection as part of the relief valve register. Section 2.2 (a) described an accident that killed two men. A vent was choked, and the end of the vessel was blown off by compressed air. Open vents, especially those on storage tanks, are often fitted with flame arrestors. If the vents, and in particular the flame arrestors, are not Testing of Trips and Other Protective Systems 277 kept clean, they are liable to choke. and the tanks maybe sucked in (see Section 5.3 a). If the flame arrestors are ineffective, a lightning strike or other external source of ignition may ignite the flammable mixture often present inside the tank, above the liquid level, and produce an explosion. According to a 1989 report. in the Province of Alberta, Canada, alone, failures of flame arrestors were responsible for 10-20 tank explosions every year. Some of the failures were due to damage not detected during inspection, others to unsuitable design [4]. 14.2.5 Other Equipment Other equipment, in addition to that already mentioned, that should be tested regularly includes the following: e Check valves and other reverse-flow prevention devices. if their fail- 0 Drain holes in relief valve tailpipes. If they choke, rainwater will * Drain valves in tank dikes. If they are left open. the dike is useless. e Emergency equipment, such as diesel-driven fire water pumps and 0 Filters for both gases and liquids, including air filters. Their perfor- e Fire and smoke detectors and fire-fighting equipment. * Grounding connections. especially the movable ones used for grounding trucks. e Labels (see Chapter 4) are a sort of protective equipment. They van- ish with remarkable speed, and regular checks should be made to make sure. they are still there. ure can affect the safety of the plant. accumulate in the tailpipe (see Section 10.4). generators. mance should be checked. * Mechanical protective equipment, such as overspeed trips. 0 Nitrogen blanketing (on tanks, stacks, and centrifuges). a Passive protective equipment, such as insulation. If lQ% of the fire insulation on a vessel is missing, the rest is useless. * Spare pumps, especially those fitted with auto-starts. Steam traps. Trace heating (steam or electrical). 278 What Went Wrong? Trips, interlocks, and alarms. Valves, remotely operated and hand-operated, that have to be used in Ventilation equipment (see Section 17.6). Water sprays and steam curtains. an emergency. Finally, equipment used for carrying out tests should itself be tested. If equipment is not worth testing, then you don’t need it. Trips and interlocks should be tested after a major shutdown, especial- ly if any work has been done on them. The following incidents demon- strate the need to test all protective equipment: (a) A compressor was started up with the barring gear engaged. The barring gear was damaged. The compressor was fitted with a protective system that should have made it impossible to start the machine with the barring gear engaged. But the protective system was out of order. It was not tested regularly. (b) In an automatic fire-fighting system, a small explosive charge cut a rupture disc and released the fire-fighting agent, halon. The manu- facturers said it was not necessary to test the system. To do so, a charge of halon, which is expensive, would have to be discharged. The client insisted on a test. The smoke detectors worked, and the explosive charge operated, but the cutter did not cut the rupture disc. The explosive charge could not develop enough pressure because the volume between it and the rupture disc was too great. The volume had been increased as the result of a change in design: installation of a device for discharging the halon manually. (c) A glove box on a unit that handled radioactive materials was sup- posed to be blanketed with nitrogen, as some of the materials han- dled were combustible. While preparing to carry out a new opera- tion, an operator discovered that the nitrogen supply was disconnected and that there was no oxygen monitor. The supply was disconnected several years before when nitrogen was no longer needed for process use, and the fact that it was still needed for blanketing was overlooked. Disconnecting a service was not seen as a modification and was not treated as such. The oxygen analyzer had apparently never been fitted [6]. Testing of Trips and Other Protective Systems 279 One sometimes comes across a piece of protective equipment that is impossible to test. All protective equipment should be designed so that it can be tested easily. 14.3 TESTING CAN BE OVERDONE An explosion occurred in a vapor-phase hydrocarbon oxidation plant, injuring ten people and seriously damaging the plant, despite the fact that it was fitted with a protective system that measured the oxygen content and isolated the oxygen supply if the concentration approached the flam- mable limit. It is usual to install several oxygen analyzers, but this plant was fitted with only one. The management therefore decided to make up for the deficiency in numbers by testing it daily instead of weekly or monthly. The test took more than an hour. The protective system was therefore out of action for about 5% of the time. There was a chance of 1 in 20 that it would not prevent an explosion because it was being tested. It was, in fact. under test when the oxygen content rose. 14.4 PROTECTIVE SYSTEMS SHOULD NOT RESET THEMSELVES (a) A gas leak occurred at a plant and caught fire. The operator saw the fire through the window of the control room and operated a switch, which should have isolated the feed and opened a blowdawn valve. Nothing happened. He operated the switch several times, but still nothing happened. He then went outside and closed the feed valve and opened the blowdown valve by hand. The switch operated a solenoid valve, which vented the com- pressed air line leading to valves in the feed and blowdown lines (Figure 14-3). The feed valve then closed, and the blowdown valve opened. This did not happen instantly because it took a minute or so for the air pressure to fall in the relatively long lines between the solenoid valve and the other valves. The operator expected the system to function as soon as he oper- ated the switch. When it did not, he assumed it was faulty. Unfortu- nately, after operating the switch several times, he left it in its nor- mal position. 280 What Went Wrong? Feed Closed Switch in control room Blow- ’ down Open Vent ?-Air Solenoid supply Valve The operator had tested the system on several occasions, as it was used at every shutdown. However, it was tested in conditions of no stress, and he did not notice that it took a minute or so to operate. The solenoid valve should have been fitted with a latch so that once the switch had been operated, the solenoid valve could not return to its normal position until it was reset by hand. (b) A liquid-phase hydrocarbon oxidation plant was fitted with a high- temperature trip, which shut off the air and opened a drain valve that dumped the contents of the reactor in a safe place (Figure 14-4). If the air valve reopened after a dump. a flammable mixture could form in the reactor. One day the temperature-measuring device gave a false indica- tion of high temperature. The air valve closed, and the drain valve opened. The temperature indication fell, perhaps because the reac- tor was now empty. The drain valve stayed open. but the air valve reopened, and a flammable mixture was formed in the reactor. For- tunately it did not ignite. The air valve reopened because the solenoid valve in the instru- ment air line leading to the air valve would not stay in the tripped position. It should have been fitted with a latch. Testing of Trips and Other Protective Systems 281 u This valve closed 81 then reopened, filling the reactor with air. This valve opened & stayed open. The reactor emptied. Figure 14-4. When the air valve reopened after a dump, a flammable mixtwe formed in the reactor. 14.5 TRIPS SHOULD NOT BE DISARMED WITHOUT AUTHORIZATION Many accidents have occurred because operators made trips inopera- tive (that is. disarmed, blocked, or deactivated). The following incidents are typical: (a) Experience shows that when autoclaves or other batch rz L actors are fitted with drain valves. the valves may be opened at the wrong time and the contents tipped onto the floor, often inside a building. TO prevent this, the drain valves on a set of reactors were fitted with interlocks so that they could not be opened when the pressure was above a preset value. Nevertheless, a drain valve was opened when a reactor was up to pressure, and a batch emptied onto the floor. The inquiry disclosed that the pressure-measuring instru- ments were not very reliable. So the operators had developed the practice of defeating the interlocks either by altering the indicated pressure with the zero adjustment screw or by isolating the instm- ment air supply. One day the inevitable happened. Having defeated the interlock, an operator opened a drain valve in error instead of a transfer valve. Protective equipment may have to be defeated from time to time, but this should only be done after authorization in writing by a responsible person. And the fact that the equipment is out of action should be clearly signaled-for example, by a light on the panel. 282 What Went Wrong? (b) Soon after a startup, part of a unit was found to be too hot. Flanged joints were fuming. It was then found that the combined tempera- ture controller and high-temperature trip had been unplugged from the power supply. Trips should normally be designed so that they operate if the power supply is lost. If this will cause a dangerous upset in plant operation, then an alarm should sound when power is lost. Trips should be tested at startup if they have been worked on during a shutdown. Particularly important trips, such as those on furnaces and compressors and high-oxygen concentration trips, should always be tested after a major shutdown. The most common cause of a high temperature (or pressure, flow, level, etc.) is a fault in the temperature measuring or control sy s tem . (c) Trips and interlocks may have to be disarmed (that is, made inoper- ative) so that equipment can be maintained. The operators or main- tenance workers may then forget to re-arm the trip or interlock. For example, to maintain an emergency diesel generator, the auto-start mechanism was blocked. According to the procedure, when work is complete, one electrician should remove the block, and another should verify that it has been removed. Both signed the procedure to indicate that the block was removed. Nevertheless, a week later a routine test found that the block was still in position [7]. As stated in Sections 1.2.7 (e) and 3.2.7 (b), checking procedures often break down, as the first person assumes the checker will spot anything missed; after a while the checker, having never found anything wrong, stops checking. When safety equipment has to be blocked or disarmed, this should be clearly signaled by a light or prominent notice on the panel. (d) On computer-controlled plants, it may be possible to override an interlock by means of a software block. On one plant passwords and codes were needed for access to the program. They were kept under lock and key and issued only to electricians and engineering staff, but nevertheless 40 people had access to them. When an interlock was found, by routine tests, to be blocked, all 40 denied any knowledge. A secret shared by 40 people is no secret. [...]... had to be carried out during startup, temporary nitrile rubber gaskets were used during this period You can guess what happened One of them was left in position and corroded, 299 300 What Went Wrong? causing an acid leak Subsequent checks showed that many more gaskets were made of the wrong material (b) A carbon steel valve painted with aluminum paint was used instead of a stainless steel valve It... 15-1 The bucket was not grounded and acquired a charge 292 What Went Wrong? ever, will not prevent ignition by static electricity, as the following incidents show (b) A man held a 10-L metal container while it was being filled with acetone When he tried to close the valve in the acetone line, the acetone ignited, and the fire spread to other parts of the building The man was wearing insulating (crepe... presented at AIChE Loss Prevention Symposium, Minneapolis Aug 1987 5 Health and Safely Executive, Electrosiatic Ignition, Her Majesty’s Stationery Office, London 1982 298 What Went Wrong? 6 N Gibson and D J Harper, Joiirnal o Electrostatics, Vol 11 1981, f p 27 7 R W Johnson, Loss Preverztion, Vol 14, 1981, p 29 8 B D Berkey, T H Pratt, and G M Williams, "Review of Literature Related to Human Spark Scenarios,"... industries, many incidents have shown the need to tell contractors precisely what they should do and then check that they have done it, It is easy to forget this at a time of recession and economies Another incident occurred on a British submarine At the time small drain valves were used to check that the torpedo outer doors were 288 What Went Wrong? closed; if water came out of the drain valve, then the outer... didn’t, we were quite likely to apply the welding procedures of carbon steel to 2%% Cr steel with unfortunate results”[l] 302 What Went Wrong? As the result of incidents such as those described in (c) and (g) through (j) above, many companies now insist that if the use of the wrong grade of steel can affect the integrity of the plant, all steel (flanges, bolts, welding rods, etc., as well as pipes)... to the filling arms The extension pieces included ungrounded metal parts; charge accumulated on them and then discharged, igniting the vapor in the tank trucks [13] Several fires have occurred when powders were added manually to vessels containing flammable atmospheres and the use of mechanical methods of addition is recommended [ 5 , 111 It is better to prevent the formation of explosive mixtures by... atmosphere-it was an aluminum vessel that had been cleaned with sodium hydroxide solution so that hydrogen was produced-an ignition occurred Electrical equipment for use in flammable atmospheres should 296 What Went Wrong? have a surface resistance of less than 1 G ohm at 50% relative humidity The vessel should not, of course, have been inspected until it had been gas-freed A gasoline spillage ignited when sonieone... a handle had to be turned 100 times This gave ample time for him to consider the wisdom of his action [ 101 Many of these incidents show the value of routine testing 284 What Wenf Wrong? 14.6 INSTRUMENTS SHOULD MEASURE DIRECTLY WHAT WE NEED TO KNOW An ethylene oxide plant tripped, and a light on the panel told the operator that the oxygen valve had closed Because the plant was going to be restarted... tested without upsetting production On one occasion the trip failed to operate, and the furnace coils were overheated The operator was busy elsewhere on the unit and was not watching the furnace 286 What Went Wrong? All trips fail occasionally So if we are deliberately going to wait for a trip to operate, we should watch the readings and leave ourselves time to intervene if the trip fails to work 14.8... instead of the alloy specified (i)After some new pipes were found to be made of the wrong alloy, further investigation showed that many of the pipes, clips and valves in store were also made of the wrong alloys The investigation was extended to the rest of the plant, and the following are some examples of the findings: 1 The wrong electrodes had been used for 72 welds on the tubes of a fired heater 2 Carbon . control valve is fully open. Figure 11- 1. When the control valve was open, the pressure prevented the trip valve from closing. 274 What Went Wrong? solid would dissolve. The mixture. action should be clearly signaled-for example, by a light on the panel. 282 What Went Wrong? (b) Soon after a startup, part of a unit was found to be too hot. Flanged joints were fuming. It was. especially those fitted with auto-starts. Steam traps. Trace heating (steam or electrical). 278 What Went Wrong? Trips, interlocks, and alarms. Valves, remotely operated and hand-operated, that have