Cybersecurity, The New Variable In The Water Equation

By Sielen Namdar, Industry Solutions Executive, Cisco

Water is a resource that is easily taken for granted. We see rain, we see lakes, we see rivers, we see creeks; we turn on our faucets and there it is. Rarely do we think about what it takes to get that water to our showers or sinks. We pay our bills and don’t give it much of a second thought.

That’s because there is an entire industry of folks who are thinking about it for you. And there is a lot that goes into it. Are the chemical compositions within U.S. EPA regulations? Is the water pressure at safe levels? Are the water supply lines free of sickness-inducing fungus or bacteria? Is there a leak in any of those supply lines?

Now, utilities have to add a new variable to the equation: Is the system safe from cyber hacks?

The Rude Awakening

In February 2021, the water utility industry got a big wakeup call by way of Oldsmar, FL. Hackers gained access, for just a few minutes, to the Oldsmar water utility’s operations network through a remote-access system. A few minutes might not sound too bad, but in that quick time the hackers changed the quantity of lye added to the water to poisonous levels. Fortunately, an employee noticed the change right away and was able to fix it before any real damage was done.

Water utilities across the globe took note. Were their systems safe from such hacks? There was good reason for concern; a 2020 report from the FBI showed a 69 percent year-over-year increase in cybercrime complaints. Perhaps most importantly is that, on average, hackers had unauthorized access to networks not for mere minutes like in Oldsmar — but for 56 days before detection.

Unfortunately, many water utilities are operating with outdated systems and have little visibility into what is happening across their networks. This makes them easy targets.

Some utilities invested in their digital infrastructures early, preparing them for situations like this. One of them is the Albuquerque Bernalillo Water Utility Authority (Water Utility) in New Mexico.

Albuquerque Water

The largest water utility in New Mexico, the Albuquerque Bernalillo County Water Utility Authority (Water Authority) serves just under 680,000 customers through more than 3,000 miles of pipeline. Their digital infrastructure allows them to have visibility across their entire network, successfully bridging their OT (operational) and IT (informational) environments. As Kristen Sanders, the utility’s Chief Information Security Officer, put it, “No one takes water more seriously than a desert community.”

While it’s easy to quip about water in the desert, Sanders wasn’t making a joke. Bernalillo County sees less than 12 inches of rain per year, compared to the 30 inches that most U.S. states get. It needs systems in place to make sure that every drop makes it to their customers.

Investing In The Future

There is a bit of a mantra in cybersecurity: You can’t protect what you can’t see. Visibility is the key to detecting anomalies across networks. The Water Authority didn’t have that visibility.

“There were gaps in our physical security and cybersecurity,” says Cody Stinson, the Water Authority’s Chief Information Officer. “Even if we knew there was a problem, we couldn’t necessarily identify where it was to stop it. We certainly didn’t have the predictive analytics in place to proactively address issues before they occurred.”

Stinson and his team worked with Cisco to completely rebuild the Water Authority’s network and add state-of-the-art solutions including Cisco Cyber Vision, designed to bring full visibility into industrial control systems, giving operators a view into everything that is happening across their entire operation.

“Having those eyes on the network all the time really helped us become more proactive, instead of reactive,” Stinson says. “We can fix problems as they happen or even before they happen, not a month later.”

They also invested in Cisco Duo Security, a multi-factor authentication tool, allowing them to verify the identity of anyone who tried to access the network, with their exact geolocation.

Worth The Cost

Taking this kind of a digital leap is always going to come at an expense. And, right now, unnecessary expense isn’t something that utilities can take on. Non-revenue water continues to be a constant challenge for water utilities across the U.S. A 2019 report from the American Society of Civil Engineers found that the U.S. loses an estimated 2.1 trillion gallons of treated water per day. Couple this with the pandemic, where the American Water Works Association suggests that, on average, drinking water utilities could lose as much as 17 percent of revenue this year, and you understandably have utilities clutching their wallets a little tighter.

But, as Sanders told us, there is no cost for peace of mind.

“Having Duo helped us navigate the pandemic easily, sending our employees to work remotely, without having to worry about how they would access our network. Having that visibility — knowing who is on the network and how our water supply lines are functioning — gave us a great peace of mind in an otherwise nonconventional time.”