By Dee Kimata And Jim Lemanowicz
As digitalization continues to grow in the water and wastewater industry, cybersecurity becomes an increasingly important responsibility.
The development of industrial control systems (ICS) over the past two decades has changed the face of many industries. Operational technology (OT) — largely industrial equipment — has become increasingly connected, and the integration of information technology (IT) components allows such devices to leverage software that drives data collection and analysis, resulting in enhanced performance and ultimately “smarter” machines.
With these benefits came vulnerabilities, including the possibility of malicious actors gaining access to critical assets through networks. The growing recognition of cybersecurity threats to critical infrastructure (e.g., energy, water, transportation) has brought the topic into the spotlight. Further, regulatory requirements on these industries have increased. Standards and policies have been created in an attempt to address the rapid technological changes; however, it is still challenging for companies to implement needed processes and keep personnel up to date and aligned, given the pace of change.
Meanwhile, cyberthreats continue to increase. According to IBM, the number of attacks aimed at ICS increased by 110 percent in 2016 compared to 2015.1 To add to this, leveraging third-party vendors and new cloud-based services results in additional areas of risk previously nonexistent in ICS.
Designing products secure from cyberattack became a topic of concern only about a decade ago, and the prevailing sense at that time was that isolation (“air gap”) and limited availability of technical knowledge (“security by obscurity”) protected ICS products. This false belief was quickly dismissed as wishful thinking after Stuxnet, and vendors began to respond to customers’ demands for more secure products. However, with often heterogeneous equipment and lifecycles counted in decades, it will take time for secure components to become the norm.
Meeting Cybersecurity Challenges
In an effort to address cybersecurity risks, the number of regulations and standards that have been created by governments, industry groups, and private organizations has grown considerably over the past 10 years. Organizations must go through the effort of understanding the regulations, determine which requirements are applicable, and then continuously monitor updates and changes to regulation to confirm compliance with the latest versions. Additionally, there is a very real threat that even when an organization attempts to faithfully comply, a lapse in proper execution can expose them to potential fines.
Although necessary, meeting regulatory requirements — and the endless focus on compliance, plus the reporting and documentation that this entails — can be both daunting and taxing. Nevertheless, this is necessary because, in many cases, compliance is a precursor to doing business with customers. It’s considered a way to show that the minimum cybersecurity requirements have been met.
In reality, compliance is a byproduct of security. Organizations need to look at security from a holistic standpoint, not a check-the-box or bare-minimum compliance standpoint.
Recommendations for how to approach security more comprehensively follow in the sections below.
The three foundations of cybersecurity are people, process, and technology. While many organizations’ policies focus on the latter two factors, it should be noted that people are just as critical to maintain a robust security posture.
The tremendous changes in technology are now resulting in increased demand for new skills and skill combinations; the current demand for cyber-professionals is not being met. Cost pressures and workforce reductions only compound this situation and can result in documentation slipping through the cracks and, ultimately, in a lack of compliance with regulatory requirements, as mentioned above.
Recommendations: Many companies address this shortfall by building collaborative teams drawn from both IT and OT staff within the organization. Other organizations turn to third-party providers to deliver IT/ OT expertise that is shared among multiple customers through managed services. Automation of routine security maintenance tasks and reporting can significantly reduce this burden as well.
A positive effect is that retraining programs and a greater interest in cybersecurity from a professional education perspective are becoming increasingly common.
Some of the major cybersecurity training programs and certifications are:
- SANS Institute — largest provider of cybersecurity training, with a side focus of preparing people for cybersecurity certifications and other widely recognized programs in the industry
- CISSP — Certified Information System Security Professional, considered a rite of passage for CISO (chief information security officer) professionals
- GICSP — Global Industrial Cyber Security Professional, the equivalent certification recognized within industry.
It is not always a given that organizations have a full inventory of or visibility into all the components across their operational enterprise, in their ICS, or those of third-party service providers. This can have a negative effect in the case of a vulnerability as an organization tries to understand the impact and react accordingly. Where a cyber-asset management system is not already in place, manual effort is required, resulting in increased costs and lengthy reaction times.
Recommendations: When installing new equipment or systems, organizations should also install programs that report their asset inventory (i.e., number of servers, human machine interfaces [HMIs], etc.) in real time. Such a system also allows users to look up multiple versions of products to determine their susceptibility to vulnerabilities.
The greatest challenge during incident response is the triage process. Asset inventory solutions take the triage process from being a manual effort to being automated, thereby shortening reaction times and reducing costs.
Lifecycle Of Products
As mentioned, ICS were not historically designed with cybersecurity as a first priority. While organizations may have more opportunity to implement cybersecurity standards in new products and systems, for older ICS it can be more difficult. This difficulty notwithstanding, organizations are still expected to address the cybersecurity needs of these previously installed systems, which are likely to have many fewer support options.
This means that remediation needs for older ICS are at times unknown to the organization and when known can be challenging and costly. In addition, many product lifecycles are counted in decades, rather than years, and it is not always straightforward to find capital to replace or upgrade products quickly.
Recommendations: Together with ICS providers, organizations should evaluate their existing operations base and prioritize remediation. A risk assessment will highlight what is worth fixing immediately. Organizations can prioritize and still greatly improve their risk posture.
Moving forward, organizations need to ensure that their programs and systems are secure by design and secure by default so that they do not have the same challenges in the next generation of products.
How To Implement Baseline Security Measures For Every ICS Organization
In addition to addressing these challenges, here are some recommendations for ways to address cybersecurity at each of the six stages of the cybersecurity lifecycle.
Ability To Identify
- Executive Support — An organization should establish a comprehensive security program with the support of the executive team. Executive leadership determines the budget for the overall company based on its level of risk tolerance; a strong cybersecurity program will require a significant investment, and large budgets are awarded aligned to organizational priorities. Executive leadership also has the authority to encourage and enforce that employees follow new cybersecurity procedures, as opposed to ad hoc uncoordinated security. Before beginning to put measures in place, it is therefore essential to align the program with corporate risk appetites and to obtain executive support of the program.
In 2015, approximately 30 substations in the Ukraine were shut down as cyberattackers infiltrated the SCADA networks of three power companies.
- Cybersecurity Audit — Very few companies have complete, up-to-date, and documented records of their entire networked systems and assets. The concept of a cybersecurity audit is not new, but it has been uncommon for ICS. However, due to challenges companies have had in keeping documentation current, they are an appropriate place to start. External or third-party audits can be a useful tool to drive companies to do a better job at maintaining an up-to-date inventory of all hardware and software. This includes documenting configurations, mapping networks, and identifying vulnerabilities and exposures. This information is essential to risk management.
Ability To Protect
- Harden All Hardware And Software Configurations — Systems and devices usually do not arrive configured for maximum security but for ease of use and access. Ports and services that may not be called for in the workplace may be left open by default. Hardening these assets, for example turning off software features and functions or enabling key access requirements on devices, reduces risk by decreasing the number of ways a malicious actor can attack them.
- User Accounts And Least-Privilege — As ICS environments rely more on connected computers for operational purposes, it is important to manage and apply user permissions and security policies across the entire ICS. Using a domain server with active directory or LDAP (lightweight directory access protocol) can help push consistent security policies to all your user machines. User permissions can be assigned to specific roles to ensure that every user has the least number of privileges allowable for executing their job. Security policies and user permissions are powerful tools to enforce best practices for things such as password policy, file access, removable media, etc.
- Integrated Update And Patch Management Program — Applying software updates and patches often receives low priority until an incident occurs. There are undoubtedly challenges to patching, including compatibility questions, uptime requirements, and manufacturer warranty constraints, and in some cases these may prohibit updates. An update and patch management program allows an organization to evaluate the risks of installing, delaying, or not installing patches and to determine its best plan of action, which may involve adding layers of other security controls around unpatched systems. Tools to automate the backup and patching of systems can significantly reduce the labor burden and cost of applying patches.
- Network Segmentation — Due to the growth of digitalization, many companies have embraced new connectivity at a rapid pace, unintentionally leaving networks with unprotected or inadequately protected points of access. Segmentation is key to reducing the impact of security breaches by adding control points and inhibiting the spread of malware. While companies have made efforts to segment their network, this has not always been achieved with optimally secure results. The outcome is that companies think they have segmented networks, but in reality they have a flat architecture, which opens up more risk. Organizations are encouraged to review network diagrams periodically to ensure the network matches what has been documented.
Ability To Detect
- Continuous Vulnerability Assessment And Remediation — Vulnerabilities in networked systems are discovered frequently, and it is the responsibility of the organization to become aware of these as soon as possible. Actively monitoring sources such as ICS-CERT, vendor websites, and industry journals is a best practice for an organization to increase awareness. A more proactive approach is to subscribe to receive push notifications, which are specifically related to a system’s installed components. Product and system suppliers must provide options for customers or any affiliate to confidentially report a security concern to promote timely remediation.
- Intrusion Detection And Prevention — It is becoming increasingly important to monitor ICS-specific protocols and define the anomalies to normal operations. With this monitoring comes the need for log collection, aggregation, and analysis. Because the ICS industry is production-focused, there are challenges with trusting IPS (intrusion prevention system) active blocking policies that may interrupt operations. There are, however, a number of passive monitoring technologies that can help identify a potential threat without adding to the risk of disruption. As the industry becomes comfortable with the analysis of these cybersecurity anomalies and more readily allows implementation of active prevention policies additional protection from cyberthreats to the ICS environment will be possible.
Ability To Respond
- Incident Response — It is highly likely that all organizations will eventually experience a security incident. The impact of that event is largely determined by the strength of its incident response program. Thoroughly planning and communicating the actions to be taken by each party ensure a coordinated response and greatly reduce the potential negative impact. Having a strong communication plan, with already drafted holding statements, helps customers and all those impacted feel more comfortable in the case of an incident. Holding incident response exercises allows companies to practice and gain familiarity with roles and responsibilities.
Ability To Recover
- Backup And Restoration Plans — Organizations must be able to back up and restore their systems to a near real-time position regardless of whether the event was caused by a cyberattack, human error, or physical failures. Unfortunately, some organizations find out that proper backup and recovery plans do not exist only after the event has occurred. This greatly reduces the speed of recovery, which increases the overall negative impact of the event. Ensuring that networking devices, HMIs, controller configurations, and PLC configurations are backed up regularly is imperative to a quick recovery. A planned and tested recovery strategy is key to reducing the impact that a cyberattack may have. Tools that can automate backup can also reduce the burden of performing routine backups to employees.
Ability To Comply
- Security Training Programs — Security is a product of people, process, and technology, and organizations often forget that these people include every individual with access to their networks and assets. Security awareness training of all personnel is necessary not only to educate everyone on their role but also to change corporate culture to one that prioritizes a robust security posture.
Effective Risk Management
Risk can be broken down into two categories: operational risk and cyber-risk. In operational risk, the effects tend to be tangible, including equipment failure, personnel safety, or environmental impact. In contrast, the goal for cyber-risk is to manage an organization’s exposure to vulnerabilities that may cause data loss, privacy concerns, or reduced network security. Ultimately, uptime, efficiency, revenue loss, and reputational damage are key focus areas regardless of type of risk.
The challenges outlined above, for example cyber-asset management, increasing industry standards, cost pressures, and staff reductions, make measuring cyber-risk difficult. This section will focus on best practices to achieve this goal.
Best Practices To Measure Cyber-Risk
Choose a consistent method to quantify cybersecurity risk. In theory this is simple, but in reality it is challenging. The method must be adjusted to align to a company’s unique use case.
The holistic organization (not only IT/OT/technology) should set the risk thresholds and obtain acknowledgement from the organization at large. The risk threshold must be easy to understand. For example, a threshold must explicitly define what is acceptable versus not acceptable as opposed to measuring risk on a scale of 1 to 5. It is critical to make measurement easy to comprehend for all.
Align the cyber-risk to enterprise risk. Risk has been traditionally presented to company executives and the board collectively. Boards are evaluated on measuring risk in an organization and are personally liable for decisions. A main reason cyber-risk management seems more complicated than it might be is because of the distinction in how risk rolls up to management.
Security is a continuous effort. Organizations should strive for increased cyber-risk management maturity levels each quarter, year, and period.
With the growing awareness of cybersecurity challenges, and how to surmount these as outlined above, come opportunities for organizations to be successful in a dynamic digital age.
About The Authors
Dee Kimata is the global product manager for cybersecurity and safety at ABB Energy Industries. In this role, she is responsible for driving and executing the product roadmap and go-to-market strategy aligned to cybersecurity and safety digital products. Dee has a B.S. in finance from Creighton University and is pursuing an MBA at Rice University with a concentration in entrepreneurship/ economics.
Jim Lemanowicz is the global cybersecurity manager for ABB’s power generation business. He has over 25 years of experience in the utility industry, serves on ABB’s Industrial Automation Division Cyber Security Council, and manages product security and compliance. Jim has a B.S. in electrical engineering from Cleveland State University and is a certified Global Industrial Cyber Security Professional.