Danger isn’t always obvious. Often the worst threats are the ones that go undetected— until they strike.
Identifying and managing hidden threats before they become a problem is a growing challenge for water utilities utilizing SCADA (supervisory control and data acquisition) systems. Some cyber threats trickle in slowly in pieces, assembling only seconds before they attack. Others are designed to strike when a specific vulnerability occurs and will wait for months or even years to do any damage.
“SCADA security threats are becoming much harder to detect,” said Doug Johnson, director of water automation solutions and business development at Emerson Process Management. “Fifteen years ago, cybersecurity breaches of SCADA systems were not that big of an issue because every SCADA manufacture used different technology. Now there is more similarity across the systems, more people are experts, and wireless technology and the Internet have given hackers the ability to connect with computers halfway around the world.”
A SCADA system security breach can have devastating public health and environmental impacts and can damage critical, expensive equipment. It can also harm a utility’s reputation — even if the threat is contained.
Most water utilities have some SCADA cybersecurity programs in place, but many are not where they should be to adequately prevent attacks, said Johnson.
“The biggest problem is most people don’t know where to begin,” explained Johnson. “To have a good cybersecurity program takes a lot of thought.”
Create A Plan
The first step to preventing SCADA system attacks is basic, but often overlooked.
“Water utilities need to sit down and create a comprehensive SCADA cybersecurity plan,” suggested Johnson.
To do this, it is important to understand exactly what your SCADA system is connected to.
“Evaluate and manage the elements connected to your SCADA system on a regular basis,” said Johnson. “Things get added on all the time, and even just connecting a thumb drive can cause a huge security risk.”
Identifying the gaps and vulnerabilities in the system, and creating a defense plan that specifies exactly what to do if a new threat is identified, is also a must.
“You need to have a plan in place about what to do when there is an incident” suggested Johnson. “How do you recover, contain it, collect information, and find out what happened?”
For guidance on creating a cybersecurity plan, check out the American Water Works Association (AWWA) Cyber Security Guide, the National Institute of Standards and Technology (NIST) Framework for Improving Critical Infrastructure Cyber Security, and the NIST Guide to Industrial Control Systems (ICS) Security.
Harden The System
All security plans should include a series of measures that “harden the system” — secure it by reducing its vulnerability.
One of the most important steps required to reduce a SCADA system’s vulnerability is to encrypt all network communications, said water cybersecurity expert Darian Slywka.
“A secure, redundant data connectivity path should also be added for the event of a critical failure, and the network should be segmented logically and securely,” said Slywka. “Engineered controls should also be incorporated to allow for autonomous local automatic control in the event of a complete catastrophic SCADA system failure.”
Determining and limiting where data goes is also an effective way to harden a SCADA system.
“It is important to control physical access and communications from one part of the system to another, so you are not letting data just go wherever the data wants to go,” Johnson said. “Put up an electric perimeter limiting who can access the system and what the system can send out.”
A common mistake many utilities make is connecting SCADA operating systems to other systems in a way that isn’t secure.
“Sharing information can help make life easier, and that is OK, but you have to make sure you are not compromising security to do that,” said Johnson. “Any connection to the outside word could compromise the system and should be limited.”
Making sure security updates, antivirus programs, and operating systems installed throughout the utility network are all up to date will also reduce the chances of a SCADA security breach. Many utilities are unaware of or overlook the risks of working with out-of-date software.
“It is very common for embedded systems to be using outdated operating systems,” said Slywka. “I have personally witnessed computers with outdated antivirus and operating systems, no application whitelisting, no firewalls, no endpoint security, no backup systems, and no external drive scanning or control, in addition to inadequate alarming, reporting, notifications, or even centralized management.”
For more ways to harden SCADA systems, check out 21 Steps To Improve Cyber Security of SCADA Networks from the U.S. Department of Energy.
Not all threats come from the outside — often the biggest cybersecurity vulnerabilities are within the walls of a water utility.
“We think of security breaches as a bad guy with a truck full of explosives driving through the front gate, but security problems can also come from a disgruntled employee, an untrained employee, or a contractor that has access to the system. It can happen pretty close to home,” said Johnson.
Limiting who has access to SCADA systems, and properly training and vetting those who do, can reduce cyber threat risk. Background checks should be performed on anyone who will have access to a SCADA system, even if they’ve been working at the water utility for 30 years, Johnson suggested.
“To make the system itself less vulnerable, make sure it cannot be operated or impacted by someone who is not authorized. Look at where access to the system is, and limit who can access it locally and remotely,” he said.
Sometimes cyber vulnerabilities are created by employees accidentally. To avoid this, initial training programs for managing SCADA systems need to be rigorous, and refresher training should be frequent.
Transition planning, especially as many water utilities experience a flood of retirements, is also vitally important to cybersecurity. As new employees replace those with years of experience, it is important to train them thoroughly on all SCADA security protocols, so that lack of knowledge doesn’t increase cyber threat risk.
Security should be viewed as the responsibility of every single employee or contractor with access to a wastewater SCADA network or other utility system, not just the IT department.
“Utilities are much more diligent about protecting and defending external access to their networks, with less regard for internal network control measures, detection methods, internal policies, employee accountability, education, and training,” said Slywka. “It is simple to purchase a piece of hardware to defend a network. It becomes much more challenging to change a utility’s culture.”
Get Outside Help
Understanding and taking the necessary steps to protect a SCADA system from cyber threats can be a large — and sometimes daunting — task for a water utility.
“The ones who do it best recognize that it is an ongoing effort, and it takes people making it a big part of their responsibly,” explained Johnson. “A water utility really needs someone whose job is to be responsible for cybersecurity, someone who understands that SCADA has its own security concerns.”
If there isn’t someone qualified or available at a utility to take responsibility for SCADA cybersecurity, utilities should consider turning to outside experts or consultants. More often SCADA security is becoming an outsourced job function, explained Johnson.
As the ways hackers can disrupt a SCADA network evolve, protections measures must evolve, too.
“How to manage SCADA security is constantly changing, and SCADA systems are constantly changing,” said Johnson. “A good cybersecurity program needs to always be evolving. Utilities need to make sure they are paying enough attention to cybersecurity, because it can really impact health and safety if it isn’t considered a priority.”
Image credit: "code bug," © 2007 gui.tavares, used under an Attribution 2.0 Generic license: https://creativecommons.org/licenses/by/2.0/deed.en