No Need For Alarm

An on-going An on-going program to reduce unwanted alarms and monitor, prioritize, and respond appropriately to the rest can keep a plant operating safely and effectively.

Technology is good, so more technologymust be better. Right?

Wrong. More is not always better. The advent of the microprocessor and the proliferation of the modern distributed control system (DCS) made it easy to alarm something…everything, in fact…at little or no cost. As a result, many facilities today have an overwhelming number of notifications daily, leading to frustrating, sometimes confusing,and occasionally tragic situations.

“Everyone knows alarm management is important, but somehow it rarely seems to be important enough to justify projects in the plant,” observes Todd Stauffer, PCS7 Marketing Manager, Siemens Energy & Automation. Recent events exposing the consequences of bad alarm management – among them the explosion at the BP refinery in Texas City in March 2005 that killed 15 and injured 170 – may be changing that attitude, however. (View a report about the event, attributed in part to alarms that were not fully functional, by visiting the video room on the U.S. Chemical Safety and Hazard Investigation Board Website, www.csb.gov.)

This and other such incidents have prompted personnel at many plants to re-think alarm management programs and look at what has led to the overwhelming number of alarms, learn about and adopt best practices, and promote standards development. The renewed interest has led companies to consider and incorporate benchmarks such as the Engineering Equipment and Materials Users’ Association’s (EEMUA) Publication 191;“Alarm Systems: A Guide to Design, Managementand Procurement,” called by many the de factoalarm management standard. Notes TimDonaldson, Director of Marketing at Iconics, "Alarmdistribution, tag frequency/chattering, crosscorrelations,operator response time and operatorchanges by interval are among reporting metrics thatare part of EEMUA and provide valuable informationto improve plant operations.” In addition, end-usersand vendors alike are supporting such standardsdevelopment as ISA’s SP-18.02, Managementof Alarm Systems for the Process Industries.

Getting started
Among the more obvious questions raised about alarm management is why so many? Stauffer explains it this way: “In the analog days, alarms were hardwired. They had to be deliberately designed and installed. There was a real cost to each one – about $1,000 an alarm – so they were done carefully. With the modern DCS, alarms are essentially free, so plants tended to enable every alarm theypossibly could.”

Indeed, today’s operator too often faces nearconstant alarm flurries. The average, recommends EEMUA’s Publication 191, should be 1 every 10 minutes, or not more than 144 a day. Most industries report significantly higher alarm levels, ranging from 5-9 per 10 min. interval (see comparison chart). David Gaertner, Director of Alarm Management Services for Invensys Process Systems, recalls one facility where five operators experienced some 5 million alarms in a six-month start-up period. “One device alarmed 550,000 times. It ran for months and still didn’t drive someone crazy enough to go and shut it down.”

Past practice has been to put in an alarm whether or not you were sure it was needed. The latest paradigm in alarm design, however, is to configure an alarm only when operator action is required. This philosophy, which reflects a fundamental change in system design practices and operator interaction, is included in the draft of ISA SP18. It defines an alarm as “an audible and/or visible means of indicating to the operator an equipment malfunction, process deviation, or abnormal condition requiring a response.” Following a practice that says ‘configure only when action is required’ lets the operator know that whenan alarm sounds or flashes, he needs to act.

Measure for measure
If one piece of advice for managing alarms recurs more than any other, it is: “Don’t do anything unless you have a tool (typically software) to measure with.” The words, from Nick Sands, co-chair of ISA’s SP-18.00.02 alarm management standards committee, and Process Control Technology Manager for the chemical solutions enterprise at DuPont, emphasize the necessity for monitoring. “A monitoring system must tell us what state alarms are in,” Sands explains. “How many are in maintenance? How many are highest priority? How many are related to safety? It also needs to tell you how your system is performing. Is it meetingyour goals, following your philosophy?”

Two basic parts of every alarm management program, says Invensys’ Gaertner, should be: “a good analysis tool that identifies which devices are creating the most alarms; and a good work process to bring personnel and resources together to eradicate the problems. The analysis tool helps you understand where the problems are. It can help determine the most frequent alarms, the chattering alarms, the nuisance alarms. It helps us see where and when alarms occur, so that we can do a root cause analysis to see why there are floods and reprioritize the alarms. A lot of plants have all their alarms set at high priority. That's not acceptable. The distribution typically should be to limit 5% of all alarms to priority #1, 15% to priority #2, and 80% to priority #3. Then the operator can respond to those that are really important.”

Nonetheless, cautions Mark McTavish, Director of Alarm Management Solutions at Matrikon, “Remember that software is a tool you use, not a solution in itself. Alarms should be the exception, highlighting things that are going out of bounds. Successful alarm management programs help plants reach that point. They help engineers manage their plants on a daily basis, achieving tighter quality control and higher productivity because they have fewer unplanned shutdowns.”

An ‘operator-centric’ function
Even having a good alarm system and a mechanism to monitor and analyze its performance is not enough, however. They need a philosophy, a guiding document that forms the foundation for the overall alarm system, stresses ISA SP18 co-chair Sands. In developing the standard, “We are focusing not only on the rationalization of alarms,” says Sands, “but on the whole lifecycle of alarms, including training, modifications, benchmarking, periodic monitoring against the processes that are in place. We want to take a holistic approach to alarm management, modeled in some ways after ISA 84.00.01, Functional Safety: Safety Instrumented Systems for the Process Industry Sector. (See alarm managementlife cycle model graphic.)

That approach includes the need to consider the operator. Most people underestimate the importance of operator participation, notes Matrikon’s McTavish.“Alarm management is ‘operator-centric.’Engineers find it hard to understandoperator problems unless they have actuallysat in the operator’s chair and experiencedalarm management. They think they knowwhat an operator needs, but often theydon’t.”

Presenting information to the operator properly through the HMI is a critical aspect of alarm management. Says Wonderware’s Jones, “Alarms need to be filtered so that only the right ones reach the operators. Software provides the tools they need to configure those parameters easily, but consistency and acknowledgement of response are important, too.”

The message that notifies an operator of an alarm must make clear what’s needed.

For example, says Siemens’ Stauffer, “When a process control engineer configures a system, he may label the physical device according to the ISA tag ID or loop ID: LIC-120 might be an alarm. But that’s not the way an operator typically refers to that piece of equipment. He sees it as ‘the level controller for tank XYZ’. If the message communicates the wrong information to the operator, it will create a problem. The operator is the target audience, not the process engineer. The operator is the one who responds. The message needs to be onehe will understand – immediately!”

Adds Eddie Habibi, Founder and CEO of PAS, "Operator effectiveness, which significantly impacts plant reliability and profitability, goes far beyond improving the alarm management system. Investing in operators is as important as investing in advanced process controls. One can’t have effective operators without proper human factors. A competent operator knows the process; has good interpersonal and communication skills; and stays alert on the job by practicing good life habits. Before the DCS,” he goes on, “the operator had a physical layout of the process, showing at a glance all piping and instrumentation. With the introduction of computer-based monitoring, hundreds of P&IDs were replicated into computer systems with little thought going into the design of the operator interface. When we went from analog systems and physical layouts of the control board to digital systems with screen interfaces, the operator lost the big picture.”

The operator also needs to be educated about the process, stresses Habibi. “Too often, we overlook training. What are the operating principles behind a pump or a compressor? An airline pilot receives countless hours of training. He must be experienced before he is allowed to take responsibility for so many lives. A chemical plant operator might have as many, or more, lives in his hands, yet he typically receives a couple of months of training, then learns on the job. We need to pay more attention to upgrading the competency of the plant operator.”

Paying the price
Good alarm management costs time and money. But bad alarm management does as well, leading to lost production and jeopardizing human life. Although initiating an alarm management program, or reviewing and revamping one, can be daunting, volumes of information are available to help establish and achievealarm management goals.

The most important factors are to set a goal and take action. Matrikon’s McTavish says a system should provide timely, unrepetitive alarms relevant to the situation to assist the operator in diagnosing the problem and determining a successful course of action. "The objective is to maintain the plant in a safe, reliable working condition so that it can make a quality product. In the end, the goal is to be financially lucrative. If a plant fails to accomplish that, then itsexistence is in question.”

Alarm management is a process, not a project, summarizes Invensys’ Gaertner.“It is like safety in the plant. It is ongoing.You are never done. We’ve learned the highcost of low performance, and plants are notwilling to pay anymore. The price is waytoo high.”

Alarms Down, Performance Up Proper management of alarms can be a significant source of hidden capacity in a plant, as this chemicals manufacturer recently discovered. Air Products and Chemicals Inc (APCI) was looking to make three performance improvements at its Calvert City, Kentucky, ethylene vinyl acetate plant: reduction in off-spec product; increase in capacity by reducing cycle time; and, due to reassignment of duties of the control-room staff, a better way to present and prioritize alarms. Complicating matters was a 20- year old DCS with no built-in alarm management mechanism. It would log alarm information directly to printers and text files, but users could not electronically access data such as alarm summaries, and were not able to perform more complex alarm analyses. A multimillion- dollar upgrade or replacement of the system was not an option. Hence, APCI needed a relatively low cost solution that would work with the legacy DCS to turn under utilized masses of alarm data into useful industrial intelligence and unlock the operation’s hidden capacity. Enter Matrikon and its ProcessGuard alarm management solution. One product, three solutions Matrikon’s ProcessGuard solution provided a grip on alarm data in three very different ways in order to meet APCI’s requirements. In the first part of the solution, metering alarm data is used to reduce off-spec product. Alarm messages with metering information were captured by the software, which then exported it into Excel reports. That prepared data is then used to identify meters which are not performing correctly, so APCI can pick out problem meters before a batch is made. By catching metering problems early, it was expected that APCI could reduce waste and faulty product. For the second aspect of the solution, to increase overall capacity by reducing process cycle time, sequencing data is captured by ProcessGuard and stored in batch records containing information on cycle time, phases, steps and parameters associated with the quality of the phase. This data had previously been transferred manually, with resulting inaccurate and missed data. With complete, accurate data available electronically, discrepancies in cycle times for batches, phases or steps can be easily identified and the root causes of these discrepancies pinpointed and rectified. Finally, an overall alarm rationalization was performed, and a state-of-the-art alarm management program implemented. Air Products used the ProcessGuard software to identify alarm problems, reduce frequency of alarms, and rank alarm data by frequency. Plant management and DCS technicians met daily and reviewed the previous day’s top 20 alarms, identified obvious nuisance alarms and addressed them by changing alarm limits, deadbands, etc. as needed. Once the majority of these chattering alarms were corrected, the software was used to identify parent/child relationships between alarms. Each parent/child group was then analyzed with an eye to long-term process improvement. Plant-wide benefits Following commissioning and implementation of these three solutions, each of the targeted areas saw substantial improvements with direct operational and bottom-line benefits. • The meter-alarm monitoring and intervention project saved engineering and equipment time due to early detection of metering problems. Inventory was reduced due to first-time material out of the reactor maximizing capacity and reducing cost. Overall, the solution enabled a 20 percent increase in meeting product specification. • The attempt to give production a bump by analyzing and correcting timing discrepancies to reduce the cycle time of reaction was a success. Overall capacity at Calvert City was increased by three percent, without capital expenditure – hidden capacity that was just lying inside the process, waiting to be tapped. • In the first two months of the project, an 80 per cent reduction in the number of alarms was achieved by maintaining a sustained engineering effort on alarm management activities – an ongoing process. Where there were five people in the control room watching alarms, the job is now handled easily by one using ProcessGuard’s Web viewer to display, filter, sort and prioritize alarms. What used to take up to one week is now accomplished in a matter of minutes. Overall, Air Products realized a total return on effort and investment in just three months. With a single software solution, a serviceable plant with a legacy DCS was revitalized – without replacing or upgrading physical assets. Based on infor mation from Matrikon Inc (www.matrikon.com)

Attributes of a ‘good’ alarm message Among the best practices encouraged in the EEMUA benchmarks document is the clear, consistent presentation of information in an alarm message. Each display screen should: • Clearly identify the condition that occurred; • Use terms familiar to the operator; • Use consistent abbreviations from a standard site dictionary of abbreviations; • Have a consistent message structure; • Not rely on learning tags names or numbers; and • Have been checked for usability during plant operations Information drawn from EEMUA Publication 191 (1999), “Alarm Systems: A Guide to Design, Management and Procurement.”