Thursday, February 05, 2009

SCADA Systems: Unknown Connections Could Spell Trouble

SCADA Systems: Unknown Connections Could Spell Trouble
By Michael Markulec, COO, Lumeta Corp.

The ease with which TCP/IP enables one to connect networks has been a huge boon for the power generation industry. Far-flung operations can be hooked up to the network and controlled remotely. And devices that once required manual adjustment can now be tweaked with the flick of a mouse. The easy flow of information throughout the organization has made nearly every aspect of business more efficient and effective.

Click here to enlarge image

Next generation SCADA networks present new challenges for information assurance teams tasked with protecting critical energy infrastructure.

But easy connectivity is not always desirable, especially the supervisory control and data acquisition (SCADA) systems that control the vital infrastructure of industrial operations in electric power and other infrastructure systems. Many of the SCADA and process control systems in use today were developed years ago, long before TCP/IP networking and distributed computing were widely available. As a result, the need for comprehensive network security measures within these legacy systems was not anticipated.

The accepted best practice security programs for protecting SCADA systems at the time were focused purely on restricting physical access to the network and the management consoles that controlled the systems. SCADA system operators rationalized that if the systems were physically isolated from entranceways and if access was strictly limited to authorized personnel, the systems would be secure and unlikely to be compromised.

Other organizations have worked to connect their TCP/IP networks to their SCADA systems to gain better accessibility and to lower costs. In so doing, however, they have also potentially subjected these critical industrial controls to higher security risks. And even though most organizations have worked to keep their SCADA networks separate from the main business TCP/IP network, connectivity becomes ever more ubiquitous. As a result, these networks have become inadvertently linked.

As utility companies deploy new SCADA applications, expand remote access points and link control systems together, they are also exposing these next generation systems to new risks and vulnerabilities that cannot be addressed solely through physical control policies. Often, these risks are underestimated due to complex network designs, lack of enforceable network security guidelines and assumptions about SCADA system privacy. Organizations are beginning to realize that the security of SCADA networks means more than the physical protection of next generation systems. In fact, new vulnerabilities that inexorably come with the use of TCP/IP networking and distributing computing technologies pose as much risk for potentially significant failures within the national critical energy infrastructure as do physical threats.

One thing is certain: SCADA security incidents will occur and, given how much of the world’s infrastructure they control, could potentially have serious repercussions.

In fact, there have already been some serious accidents and safety breaches as a result of linking SCADA systems to the main TCP/IP network. In March, for instance, the 883 MW Unit 2 at the Hatch Nuclear Power Plant in Georgia went through an emergency shutdown as a result of a software update that was made on the plant’s business network. The system update synchronized information on both systems, wiping out much of the data on the SCADA side. After everything was reset following a reboot, the SCADA safety system detected a lack of data, which it interpreted to mean that cooling system water levels for the nuclear fuel rods had dropped. It suggested a dangerous situation, indeed, if, in fact, the event had actually occurred. Unfortunately, the safety system instigated an automatic shutdown due to the software update.

Engineers were aware of the two-way communication link, but they did not know that the update would synchronize data between the two systems. Luckily, in this case no one was hurt. But as with any unplanned shutdown, it was expensive, as the plant was offline for three days. The Hatch incident was only the latest in a string of accidents and unnecessary shutdowns caused by some problem on the network. The Browns Ferry nuclear plant in Alabama, for example, shut down in 2006 when a network traffic overload locked up pump controls. And in 1999, a steel gas pipeline ruptured near Bellingham, Wash. An investigation found that a computer failure just before the accident locked out the central control room operating the pipeline, preventing technicians from relieving pressure, which caused the explosion.

SCADA systems need to be absolutely secure, given that they control some of the country’s most vulnerable infrastructure, from pipelines to nuclear facilities to water treatment plants. Network management needs to know whether and how their SCADA systems connect to the larger corporate network so that these connections can be locked down. As we’ve seen with recent incidents, it does not take a cyber attack to take out vital infrastructure; simple computer error will do the trick if connections do not comply with policy.

Some engineers believe the best protection is to sever all ties between the business and SCADA networks. But they would be mistaken in thinking that the SCADA network is safe without a regular assessment of connectivity to ensure that no connections between the SCADA network and the corporate network appear. TCP/IP networks are designed to make connectivity easy, and the ubiquity of today’s corporate networks open up the possibility of someone inadvertently connecting SCADA to the larger network, with potentially disastrous consequences.

In truth, the industry does not necessarily need to give up on the cost and management advantages of connecting its SCADA networks to the larger network. As long as the safety systems are strong and frequent and regularly scheduled network scans are conducted, both to understand the full scope of connectivity and to guarantee that all connections conform to security policy, then critical infrastructure should not fall prey to unforeseen security risks.

Network Vulnerabilities

Next generation SCADA networks present new challenges for information assurance teams tasked with protecting critical energy infrastructure.

Understanding the network risk profile of such systems requires new insights into the nature of security threats. The following are examples of common network vulnerabilities that today’s utility companies face when implementing next generation networks. These network vulnerabilities include, but are not limited to:

  • Remote Access Vulnerabilities— SCADA and other process control system networks often consist of a primary network linking SCADA-related facilities, with additional connections to the corporate network of the utility company. Because network connectivity is often permitted to the Internet, business partners, regulators and outsourcer networks, there is an increased potential for unauthorized access to supposedly “separate” SCADA networks as well. Use of remote access services such as modems, cable or DSL connections for emergency maintenance also increases the likelihood of security breaches. These types of connections are prone to unauthorized exploitation and represent potentially dangerous access points to networks.
  • Network Leak Vulnerabilities— TCP/IP networks by their very nature promote open communications between systems and networks, unless network security measures are implemented. Improper network configuration often leads to inbound and outbound network\leaks—between SCADA networks, corporate networks, business partners, regulators and outsourcers and even the Internet—which pose a significant threat to network reliability. Network leaks can allow worms, viruses or hackers direct visibility to vulnerable SCADA systems.
  • Network Security Design—The network infrastructure layer that supports SCADA and other process control systems is often developed and modified based on business and operational requirements, with little consideration for the potential security impact of network changes. Over time, security gaps may be inadvertently introduced within the network infrastructure. These gaps may represent a back door, or even a front door, into networks. Validating the effectiveness of network security defenses is of particular concern to the on-going reliability of SCADA systems.
  • Lack of Formal and Documented SCADA Network Policies, Processes and Procedures—Due to the highly proprietary and legacy nature of these systems, owners, administrators and vendors often do not follow strict configuration change management procedures. This may result in a lack of appropriate reviews, auditing and due diligence in day-to-day operational and maintenance procedures. This condition may lead to security oversights, which may again lead to serious network exposures and risks.
  • Improperl Configured and/or Unauthorized Network Services
    Use of improperly configured or unauthorized network services running on systems such as SendMail, Finger, Telnet, FTP and NFS, can create network exposures that could leave systems vulnerable to attackers. Finding and eliminating exposed network services is a critical step in minimizing network risks.

    This list represents just a few of the potential network-based risks associated with SCADA and other process control systems. Network operators and administrators should include these and other criteria in developing comprehensive security risk management programs for SCADA networks and the organization’s broader security posture.

    To protect their SCADA networks, utility companies need to develop comprehensive security risk management programs that adopt a proactive approach to isolating and closing network exposures. These exposures are often the “first point of attack” for intruders and as weaknesses that are vulnerable to internal errors like the incident at the Hatch Nuclear Power Plant. Solutions now exist that can identify network vulnerabilities while also conveying a deep understanding of how network defenses are deployed in relation to SCADA systems. These solutions can show how an IP-based SCADA network is wired together, including all the sub-networks, systems, devices and routes that IP traffic can traverse through the use of data analysis techniques.

    One of the more difficult challenges facing network operators charged with protecting the national critical energy infrastructure is identifying where one SCADA network ends and other networks begin. This dynamic network perimeter of next-generation networks can include hundreds or even thousands of “doors” or routes that can change with every new business relationship or network configuration change. Network visualization software can quickly reveal issues on the network. Through the use of regular scans, organizations can understand how their network is changing over time—even identifying devices that were previously unknown to administrators—so as to ensure that their SCADA networks remain separate and secure.

    Unknown network connections aren’t just security risks. When you’re talking about the controls systems for nuclear power plants, hydroelectric dams, chemical factories and other vital infrastructure, the stakes could quite literally involve hundreds or thousands of lives. The complex designs, interconnected nature and extreme sensitivity of SCADA and other process control systems mandate that utility organizations implement comprehensive plans for assessing and mitigating potential network vulnerabilities and threats. To do this successfully requires development of comprehensive security risk management programs that start with gaining control over network risk.

    Author: Michael Markulec is chief operating officer of Lumeta Corp.

    Power Engineering November, 2008