Introduction
Many power plants have a distributed control system (DCS) nearing the end of its useful life. Here's how to judge if and when to replace it and how to plan the upgrade project.
Power plants were among the leaders in implementing DCS technology, starting in the 1980s with systems based on mini-computers and continuing through the turn of the century. Although many of these older plants are mechanically sound, their control systems are now reaching the end of their useful lives, necessitating migration to a new automation system.
In the power industry, reliability is paramount, and it's critical to minimize the cost, risk, and downtime of any upgrade project. While many industries are driven to upgrade by the promise of operational improvements, power plant upgrade projects are typically guided by the goal of eliminating the risk of failure that accompanies older control systems.
Meeting these conditions requires deep domain knowledge and buy-in from all stakeholders, including power plant personnel and their upgrade partners. A key step toward this end is the development of an execution plan that minimizes risk and reduces downtime.
To reduce costs and downtime, power plants tend to shy away from changes to the automation system, preferring to keep things as is and opting for low-risk solutions. Yes, they need to either migrate or upgrade their automation system, but in a way that will cause the least disruption in both the short term and long term.
Another unique characteristic of the power industry is the wide range of auxiliary systems, e.g., burner management, turbines, cooling towers, water treatment, etc. These systems must all be integrated into the new automation system in some way, sometimes by connecting existing control systems to the new automation system, other times by using the new automation system to control these auxiliaries. Unfortunately, stakeholders often struggle to make a business case for replacing something that's working, albeit poorly, as is often the case with an older DCS.
Another issue that's germane to automation system upgrades is perception. It's easy to see that a railcar unloading station is in desperate need of replacement just by looking at it, even with an untrained eye. But an existing DCS will usually be tucked away in a climate-controlled room, leaving many to assume it is doing its duty. It takes an experienced automation engineer to know when it is time to migrate to a new automation system. The challenge is to prove to others that benefits like improved efficiency and productivity with less downtime will justify the cost and risk of a migration.
This article details the best approach for DCS migration, starting with justification and ending with commissioning and startup.
Do You Need a New Automation System?
Every older DCS must be replaced with a newer automation system at some point. The challenge is determining exactly when the replacement should be made. There's no single, quantitative formula for it. Essentially, it comes down to a judgment call based on a number of factors.
Factors driving the decision to replace a DCS include: (Those most important to the power industry are ranked at the top of the list.)
- Reliability issues
- Discontinued technical support and spare parts service
- Hard-to-procure replacement parts and accompanying services
- Sharp increases in vendor support costs
- Difficult-to-provide internal and third-party support
- Major plant expansion that can't be accommodated by the existing DCS
- Difficulty connecting DCS with new or existing third-party applications
- Desire to standardize automation systems among plants
- Better plant performance needed
- Improved cyber security required
The first five factors are all related to obsolescence of the current DCS and ultimately to reliability, the key driver for power plant control system upgrades. Vendors tend to encourage their customers to buy new automation systems by reducing support for older DCS technology. Older systems also face their own internal issues when trying to support decades-old designs.
As a result, vendor support will sharply decline at some point — in terms of services, parts and patches. This will result in significant price increases to keep the DCS running and will push many issues to internal support personnel. With each passing year, both DCS vendors and power plants will have a more difficult time finding personnel familiar with older DCS hardware, operating systems, and application software.
In the case of plant expansion or a major upgrade to the facility's mechanical infrastructure, often in response to a regulatory issue, it can be very expensive to expand or modify the DCS. It might even be impossible. Replacement becomes the lowest-cost option, particularly when other advantages of a new automation system are taken into account.
Nearly every power plant has auxiliary processes with their own controls, such as water treatment, fuel delivery, and ash removal systems. Interfacing these control systems with an old DCS can be problematic. Regulatory and other issues may drive the addition of continuous emissions monitoring or other systems, which, ideally would also interface with the DCS. But this is a difficult task if the new control system is PC-based and the old DCS is driven by a mini-computer, for example.
These subsystems often have critical inputs and outputs into the primary process, giving rise to not only compatibility issues but also maintainability concerns. The new automation system must also connect to existing high-level computing platforms such as plant historians.
Power companies are continually optimizing their mix of generation sources, often through acquisitions and divestitures, which can create a hodgepodge of automation systems and make support difficult. Replacing a DCS with a modern automation system can promote standardization, thereby easing internal support.
In almost all cases, a new automation system will provide superior performance, resulting in improvements that create big returns on the investment. For example, improving a plant's heat rate by just one or two percentage points can result in fuel savings sufficient enough to pay for the DCS replacement in short order. An improved operator interface can enable quicker response to incidents, preventing a small problem from escalating into expensive downtime. Finally, a new automation system will have superior cyber security. (See Sidebar Below)
So that's how you determine if and when it's time to replace your plant's DCS. The next step is implementation, which requires careful planning.
Proper Planning Predicts Success
When the decision is made to replace a DCS, the new automation system will be a modern DCS, a PLC-based system or a PAC-based system. But no matter which option is chosen, it's critical to follow a project plan, hopefully one that starts with a frontend loading (FEL) evaluation or similar type of study.
The FEL evaluation is a multistep process that includes a comprehensive evaluation of the facility. It will yield a comprehensive plan for success, a list of project tasks with budget data durations and an overall project schedule.
If a plant owner typically undertakes major capital projects internally, then the FEL may be performed by staff, assuming there's sufficient automation expertise. But because major automation system upgrades aren't performed regularly by most power companies, it's often more cost-effective to engage an outside service provider for the FEL evaluation. The provider should be independent of any particular vendor's automation system, ensuring that they are unbiased and positioned to recommend the best new automation system for a particular power plant application.
The right provider will have produced many FEL evaluations for automation system upgrade projects and will have extensive power industry experience. This enables the provider to work with internal staff to produce a thorough study. The FEL evaluation will produce a budget, show potential problems and provide a clear route to project success.
A DCS replacement is a major undertaking, requiring significant expenditures and staff training. There are risks involved, and in most cases there will be some downtime, although it can be minimized through careful planning. The FEL might show that a DCS replacement should be postponed, possibly until the end of the plant's useful life. In such a case, no replacement is required.
In other cases, where replacement is justified, the effort invested in the FEL evaluation will often be directly proportional to overall project success. Skipping the FEL evaluation or performing it in a perfunctory manner will increase risks and can lead to missed opportunities for plant improvements.
- The main items that should be addressed in a DCS replacement FEL are:
- Server-level applications such as HMIs, historians and the like
- Controllers
- I/O
- Networks among servers, HMIs, controllers and I/O
- Field wiring, networks and components
- Interfaces to ancillary systems
- Space requirements for new automation system components
- Demolition, installation and commissioning plans
- Compliance with safety standards and regulatory requirements
- Alarm management
You will need to make decisions about each of these devices including when, how, and with what they should be replaced. For example, it may be best to replace the HMIs first, often by installing the new HMIs in parallel with the old ones to ease operator anxiety and reduce risk (see figure 1).
Fig. 1: A fresh look. A new automation system provides power plant operators with a better view into the process via PC-based HMIs and should also include an improved alarm management regime.
New automation system hardware will need to be tied together with a variety of digital interfaces such as multiple variants of Ethernet and one or more permutations of fieldbus networks. Interfaces to ancillary systems will be required, some via networks and others via simple hardwired I/O. All of these networks and interfaces must be evaluated for compatibility with existing systems.
New automation systems may require different types of climate-controlled spaces, which will affect demolition, installation and commissioning plans. Compliance with regulatory requirements will probably include enhanced cyber security and, in many cases, a new alarm management system. When a proper FEL evaluation is conducted, the result is a successful DCS replacement, as illustrated in the following examples.
Ripon Finds Reasons to Upgrade
The MAVERICK company recently completed a DCS replacement for Ripon Cogeneration, a firm that owns and operates gas-fired cogeneration plants in California (see figure 2). The heat recovery steam generator, deaerator and feedwater system, gas compressor system, compressed air system and water treatment plant were the main equipment that needed to be migrated to a new automation system.
Fig. 2: Upgrade time, the existing DCS at this cogeneration plant needed to be replaced to improve operations, cut downtime, and minimize maintenance.
In the initial phases of the project, goals were determined from a technical and business perspective, a process that allowed the client to objectively look at the technology available to best achieve its objectives.
Ripon's goals were to take advantage of its existing HART smart instrumentation and to standardize on a new automation platform that would allow for increased maintenance team productivity. Ripon also wanted to include a central and actionable alarm system that would give the operators a means to quickly react and provide a solution to any process upset. Another area of concern was compliance reporting and ease of access to plant operating and performance data.
As with most power generators undertaking a major automation upgrade, Ripon was concerned with the impact of possible operational downtime on operations, and managers wanted to take advantage of existing field wiring and infrastructure. In addition, they wanted an open architecture automation system that would allow seamless interfacing with critical subsystems. Finally, they wanted a system that they could support primarily with internal resources.
These were the reasons Ripon needed a new automation system. These reasons framed the case for the DCS replacement and indicated what type of new system should be selected, in this case, a Rockwell Automation PlantPAx.
As part of the FEL evaluation, MAVERICK worked with Ripon to develop a cutover plan that would minimize downtime and risk. Existing processor and I/O drops were replaced, but all existing field terminations were left intact, a course of action that greatly reduced the new wiring required as well as associated cost and downtime issues (see figure 3).
Fig. 3: In the interest of time management, upfront planning can reveal many ways to minimize downtime when migrating from an old DCS to a new automation system. This new system allowed existing field terminations to remain intact.
The plant experienced no unscheduled outages during the migration process, and the facility started up on schedule with output ramped up to full power on the first day of operation. As confirmation that an upgrade was the right decision, the plant has experienced fewer outages than before, due in large part to the improved automation system. Performance has improved, and all other previously outlined goals above were attained.
Upgrading From The 1960s To The 21st Century
A municipal utility in Louisiana needed to replace existing pneumatic and relay-controlled combustion, steam, and burner management systems on two units. Each of these units used a natural gas-fired boiler to supply steam to a turbine generator. The capacity of each unit was between 50 MW and 100 MW, and the two units were commissioned decades ago.
Although the existing automation system was not a DCS, the challenges were similar. First, the reasons for the upgrade had to be determined, then the goals were identified, and finally a plan was made to perform the upgrade and meet those goals.
The initial challenges were to develop conceptual designs with basically no data from the original configuration and with minimal information on any subsequent upgrades and/or changes to the facility. The first steps were to identify the customer's goals and develop a baseline for the project.
The reasons for replacing the existing automation systems were to stabilize critical control variables such as steam pressure, temperature, and drum level in order to produce power based on varying dynamic load changes. The utility also needed to ensure it could maintain load during peak operating periods.
Plant operators needed an efficient operational management system that would enable them to quickly address system upsets, and the facility needed to generate regulatory reports from historical trend data. To achieve these goals, it was important to design a system offering seamless control while providing centrally located operator consoles. As with most upgrades, it was also critical to minimize downtime during system cutover and to provide the operations staff with an automation system that they could maintain with existing internal resources.
The evaluation study showed that the best way to satisfy operator demands was with a centralized HMI console that would include a number of PC-based operator interface terminals. This was a big step for the plant. The console replaced antiquated operator interface panel boards so extensive training would be required prior to commissioning and startup (see figure 4).
Fig. 4: When old-school interface panel boards are replaced with modern PC-based HMIs, extensive training is needed to bring plant personnel up to speed.
Study and planning also revealed that maintaining unit uptime was essential, as these units provide baseload power to the city. To address this issue, a redundant automation system was selected. New coordinated combustion control and burner management systems were provided for each unit, and each of these systems was provided with triple-redundant controllers and dual-redundant power supplies.
The new automation system is a GE Mark VI integrated control system with Cimplicity HMI; the optimal combination of hardware and software for the particular application. To meet the customer's reporting and compliance goals, an OSIsoft PI data historian was purchased, installed, and integrated with the new automation system.
The data historian allows plant personnel to quickly generate reports to ensure regulatory compliance and identify operational performance trends. Identification and analysis of these trends enables performance improvements and can also pinpoint problem issues before they escalate.
Conclusion
Ultimately, you're going to have to replace your DCS. Determining whether that should happen months or years down the road is the challenge. Before deciding whether to migrate or wait, you have to objectively assess the state of your current automation system. Is impending obsolescence making it difficult to secure support services or source replacement parts? Is expansion in other areas causing compatibility issues? Would standardization improve efficiency across your plants?
Though replacement may cost more upfront, the far-reaching benefits often make it more economical than attempting to upgrade or retrofit an aging system. That said, a front-end loading evaluation is key. And it may even indicate a preference for postponing DCS replacement indefinitely.
It adds a few extra steps, but evaluation will also help minimize risk and downtime. The plants that see the greatest success with DCS migration are the ones that take the time to pre-plan.
Sidebar: Cyber Security Concerns Encourage Upgrades
For regulatory and practical reasons, compliance with North American Electric Reliability Corporation Critical Infrastructure Protection (NERC-CIP) and other cyber security guidelines is more critical than ever. In addition to complying with current NERC-CIP requirements, a new automation system must also be flexible enough to take advantage of new security measures and patches.
An older DCS will have little or no built-in cyber security, making compliance with current standards extremely difficult. In some cases, the only way to make an older DCS secure is to completely isolate it from other systems. But this kind of setup greatly interferes with optimal power plant operation, making isolation an unattractive option. By contrast, a modern automation system will have a host of built-in functions that ease implementation and ongoing maintenance of cyber security measures, including a high level of application security, access control, and auditing.
Application security ensures that critical software that is executing on the underlying operating system, e.g., human machine interface (HMI) programs, will have priority access to key computing resources such as processing time, memory, and bandwidth. It also prevents interference among applications by limiting communications to non-critical times.
Access control features help plant personnel enforce policies concerning who is permitted to interact with the system and what they are permitted to do. For example, an operator may have full access to the HMI parameters, including the ability to change set points. Maintenance personnel, however might be limited to view-only privileges.
The new automation system should have sophisticated user account management controls that can assign a unique password to each user, restricting access only to what one needs to perform his or her job.
Auditing provides a centralized log to track all interactions with the automation system. Because plant personnel are aware that all activities are tracked, auditing helps detect security breaches and deter inappropriate actions. In addition, auditing can be used to track root causes of incidents, providing benefits over and above increased security.
Another cyber security feature to look for in a new automation system is a network security perimeter system, used to restrict access points where foreign software can enter the automation system. Similarly, built-in hardening of automation system workstations cuts off another access point before malicious software can infiltrate it.
No matter how well the automation system is designed in terms of cyber security, there will be an ongoing need for updates and patches as new threats emerge. The right automation system supplier will provide a means to test these changes before they are implemented. This is a necessity, as some updates and patches can cause problems with the operating system, the application software, or between the two.
To ease the complexity of implementing and maintaining cyber security compliance, select an automation system that contains built-in security features and a supplier that will work with you to provide short- and long-term cyber security support.
About the Author
Matt Sigmon, directs MAVERICK Technologies' DCS Next initiative, which helps manufacturers leverage DCS migration as an opportunity for significant advancement. In this role, Matt manages operations as well as client relationships. He holds a Bachelor of Science, Chemical Engineering (BSChE) from Clemson University. Matt can be contacted via email.
Related Stories
Emerson enhances process acquisition controller
TI Releases Energy Harvesting Circuits
The Proper Use of Guided-Wave Radar in Steam Loops
Successful Operation of China-Made, Large-Scale DCS