Natalia: What’s the difference between OT, industrial control systems (ICS), and supervisory control and data acquisition (SCADA)?

Chris: OT, ICS, and SCADA are terms that describe non-IT digital systems. The main encompassing term is operational technology, or OT, which describes digital systems that interact with physical processes in the real world—such as turbines, mixing tanks, industrial robots, and automated warehouses. If you think about manufacturing, power grids, or oil and gas, OT encompasses the cyber-physical systems (CPS) that monitor and control production—how companies make their money producing things like food, water, pharmaceuticals, chemicals, or tractors.

Industrial control systems, or ICS, is under the umbrella of OT. A control system uses automation to take the human out of the equation. For instance, a car plant might have replaced an assembly line with robots, or a food processing plant replaced manual adjustments of ingredients with specific logic code. Industrial control systems are everywhere—manufacturing, retail distribution centers, water treatment, oil and gas, transportation and mining, as well as building automation (like HVAC, elevators, access control, and CCTV) in hospitals, smart buildings, and datacenters.

Supervisory control and data acquisition, or SCADA, is a specific type of industrial control system that enables organizations to monitor and control OT equipment across a wide geographic area. Power companies, oil and gas pipelines, and water facilities have SCADA systems because they cover a large area.

Natalia: What makes securing OT uniquely challenging?

Chris: Security for IT systems has been around for a long time. In the 1980s, control systems didn’t look like normal computers. They were designed for a specific purpose—to last long and to withstand heat and very cold temperatures in wet or caustic environments. These control systems were not connected to any other networks. IT had security, but it didn’t exist in control systems.

Over the years, control systems have become more connected to IT networks—and sometimes to the internet as well—because upper management wants to get a real-time view of the next day’s production or what the projections are for next week or next month based on historical output. The only way to get that information in real-time is to connect the two systems—IT and OT. If you connect control systems to something that’s eventually connected to the internet—it might have firewalls or it might not. That’s a problem.

If you take an IT security network sensor and put it in a control system, it will only understand what it knows—standard IT protocols like HTTP and FTP. It won’t understand the Siemens S7 protocol or the GE SRTP protocol that are not used in IT systems. You also can’t put antivirus or endpoint detection and response (EDR) agents on most of these systems because they’re not Windows or Linux. They’re often real-time embedded operating systems that may be completely custom, plus they also require fast response times that could be affected by antivirus and EDR operations.

Natalia: What threats are prevalent in OT environments?

Chris: We have seen five publicly known cyberattacks against control systems, including Stuxnet, the power grid cyberattacks on Ukraine in 2015 and 2016, and the 2017 Triton attack on safety control systems in a petrochemical facility.

Insider threats are also something to pay attention to. The first publicly known attack on a control system was in the late 1990s in Australia. A fired employee still had access to the equipment and caused a sewage spill. Several years ago, someone was fired at a paper mill in Louisiana, but no one removed his remote access. He logged in and shut down the plant. They knew exactly who it was so the FBI got him, but it cost them about three days of downtime, which likely cost them millions of dollars.

Besides security threats, there’s the risk of an honest mistake. Someone is making a change at 5 PM on a Friday that they didn’t test out, and it causes a network outage, and people have to work over the weekend to fix it. Not having a good change management procedure, standard operating procedures, or rollback plan can cost millions of dollars.

Natalia: What do you think about the incident on February 5, 2021, when a hacker gained access to the water treatment system of Oldsmar, Florida?

Chris: Many water and wastewater companies are just beginning their security journey. They don’t have a large budget and may have only one or two IT folks—notice I didn’t say IT security folks—and they have to wear multiple hats. In the case of the Florida attack, I’m not surprised because most don’t have security standards like active monitoring and ensuring secure access via VPN and multifactor authentication for employees and contractors. They’re not regulated to have strong cybersecurity controls and don’t experience many attacks.

Just because someone can change something on a screen to be 100 times the original value doesn’t mean it physically can change. When you change a chemical in a water system, it is not going to instantaneously change, and it may not even be physically possible to change to that amount. Water and wastewater facilities manually take multiple samples every day so they would have caught any changes before it affected water utility customers.

Natalia: Are contractors a potential attack vector for OT?

Chris: In this case too, it’s usually a byproduct of shadow IT, where OT personnel provide remote access to contractors without going through IT to do it in a secure way using VPN, multifactor authentication, and rotating passwords. You need to provide contractors with visibility and access to the OT network for ongoing maintenance and monitoring, and there are not too many of you. Your contractors are also probably not required to have security training.

In the early 2000s, we had remote access to substations. If you knew something was wrong, you could dial in and look, and then go back to what you were doing. But if something is on the internet, opportunistic threat groups and malicious cyber criminals are going to poke around and be able to do stuff. Organizations should be concerned and look at their security, including who has remote access.

Natalia: Are you seeing more ransomware attacks impacting OT?

Chris: We are. Ransomware is terrible, and it’s affecting hospitals, which have control systems, power plants, and water facilities because they can’t rely on the city water if it goes out. They also have life support systems, imaging, and surgery support. Ransomware has also affected oil and gas companies and power companies on the IT side.

A lot of the attacks were more effective because the organizations didn’t have any segmentation between control systems and the IT network. If you’re using the open platform communications (OPC) protocol, the old version requires 64,000 TCP ports to be open, which includes ports 3389 and VNC 5900. As a result, you don’t have a firewall between IT and OT.

There must be intentionally engineered design to help protect control systems because if you don’t, you leave yourself open to something that doesn’t care what you are.

Learn more

To learn more about IoT and Microsoft Security read our IoT security blog series.

To learn more about Microsoft Security solutions, visit our website. Bookmark the Security blog to keep up with our expert coverage on security matters. Also, follow us at @MSFTSecurity for the latest news and updates on cybersecurity.

The post Understanding the threat landscape and risks of OT environments appeared first on Microsoft Security Blog.

Natalia: What tools do you use to monitor and govern your OT environment?

Chris: First, you can use the control system itself, which already offers some level of visibility into what’s happening. It looks like NASA control. Operators sit and watch the process all day. You can see what looks normal and what doesn’t look normal.

What’s new is not just looking at the system itself but at OT network security. Especially in the last five or six years, the focus has been on getting network visibility sensors into the control network. There are several vendors out there that understand the protocols, like MODBUS, Siemens S7, and DNP3, and have developed sensors that are purpose-built to analyze OT network traffic rather than IT traffic.

With a newer control system, it’s much easier. Many times, they’ll use virtual machines to manage OT, so you can put agents in those areas. If it’s a Windows 10 or Windows 7 environment, you can even use Microsoft Defender Antivirus and collect the Windows event logs and switch logs. If you don’t look at the logs, you’re not going to know what’s there, so you need to monitor behavior at the network layer using technologies like deep packet inspection (DPI) to identify compromised devices.

Natalia: What are some best practices for securing remote access to the OT network?

Chris: Number one, if you don’t need it at all, don’t have it. That’s the most secure option.

Number two, if you have to have it, make sure it’s engineered for why it’s needed and tightly control who can use it. It’s also important to make sure it’s monitored and protected with multifactor authentication (MFA) unless it’s just for read-only access to the control network, in which case it’s less of a risk. A lot of times, these OT equipment vendors require in their warranty contracts that they have remote access with full control and the ability to change configurations, which means you’ve given someone a high level of privileged access to your control systems.

Number three, have a process and procedure for when that remote access is used and when it’s turned off. You should at least know who was there and for how long, and who did what, using audit logs, for example.

I want to highlight that the Water ISAC, the international security network created for the water and wastewater sector, published a free document called 15 Cybersecurity Fundamentals for Water and Wastewater Utilities. It’s a reminder to consider where remote access is coming from.

Natalia: What percentage of organizations are continuously monitoring their OT networks?

Chris: Today, it’s the exception, not the rule. The only ones monitoring are the ones that have to do it, such as nuclear companies, and the 3,000 or so largest electric utilities that are under North American Electric Reliability Corporation Critical Infrastructure Protection Standards (NERC CIP) regulation, as well as any companies that might have been attacked in the past. But even NERC CIP doesn’t require continuous network security monitoring, just monitoring event logs in a SIEM, for example, which means you can still miss stuff.

So percentage-wise, it’s not very many, especially in non-regulated sectors like manufacturing, pharmaceuticals, chemicals, oil and gas, mining, and warehousing and logistics.

Companies don’t like to spend money on security if they don’t have to. Unfortunately, it’s going to take an attack. We didn’t have electric reliability standards until we had two Northeast blackouts that affected millions of people in 1965 and in August 2003. After that, they said, “Oh, we should probably have some electric reliability standards.” When I started at the power company, one of the lineman safety instructors said, “Safety rules are written in blood.” The only reason why we have reliability rules is because we’ve had darkness.

Natalia: How can teams break down IT and OT silos?

Chris: Communication. It’s the only thing you can do. If you’re in IT, go take a box of doughnuts down to the operators and ask, “What are the pain points here? How can I learn more about what you do so I can understand and so you won’t slap my hand every time I say, ‘Please patch.’” They will be overjoyed that someone came and visited them to learn about what they do.

Generally, if an IT guy with a white hard hat that has never had a scratch on it comes in, operators think, “Don’t touch anything.” But if you build that trust and communication, that strengthens an organization, and you can start training and knowledge sharing.

Natalia: What should roles and responsibilities look like?

Chris: Now, anything that’s on a network, even in the control system environment, can report up through the chief information officer (CIO) or chief information security officer (CISO). Even in power companies, they’re putting everyone, even the folks who do SCADA for the power grid, under the CIO or CISO instead of under operations. At smaller companies, like water and wastewater, it’s still the old situation, where you have an IT guy and an OT engineer or operator. At larger companies, OT is coming through the IT organization under the CIO or IT is under the CIO and operations is still under operations, and the link is under the CISO. You might have security people in IT and security people in OT.

If you’re wondering whether the CISO should be responsible for both IT and OT security, it’s a simple answer. You can’t have enterprise-wide security unless you include OT. Security needs to be applied to it all, but go to a provider that says they provide enterprise-wide security and ask, “Do you know anything about OT networks in power plants?” “Nope.” OK, then, you don’t do enterprise-wide security. You’re not protecting what makes money.

Natalia: Should companies unify IT and OT security in the security operations center (SOC)?

Chris: I’ve seen it implemented as one unified SOC, but I’ve also seen two separate ones because if they have physically separate systems, they have to have physically separate SIEMs. For instance, a nuclear plant will have its own SOC, and corporate will have its own SOC. If a power company has a nuclear power plant, that plant will have its own SOC because it’s air-gapped and not connected to the outside world or the IT network. But if you have an oil and gas environment, it may have both combined into one.

There are pros and cons. If you have the money and the budget and the people, you can do it either way. Just put your people in a room, give them a lunch of pizza, and let them come up with the best solution. There are advantages of having a unified SOC. You don’t even need an OT-specific SOC analyst. Just have a good IT security person learn from the control engineers or operators, and then create those alerts, and do hunting, tool tuning, and rule tuning.

Natalia: What would you say to a board of directors to get them to prioritize OT security?

Chris: I’d keep it short and sweet: “What would happen if you couldn’t make hammers anymore?” If the CISO can’t answer that question, you know the person needs to gain that awareness. Do we have visibility of the network? Do we have offsite backups for our control systems? Do we have security awareness training?

Board members are not concerned with the latest and greatest advanced persistent threat (ATP), but they do care about risk to the business. They’ll say, “We don’t have any security because we don’t have enough people. If we don’t have security implemented, we have a small risk of having downtime.” If you talk to any manager, they’ll know exactly how much money they lose per day if production goes down. We look at business risk in terms of the equation: risk equals impact times probability. Since we don’t have enough data about cyberattacks in OT to have a probability, we tie cybersecurity to the risk register and substitute probability with exploitability. How easy is it to exploit? Can a script kiddie do it? Could my 13-year-old son do it?

If you’ve got an operating system exposed to the Internet, discoverable via Shodan, it is exploitable within minutes. What is the impact of that? If it’s in a chemical, pharmaceutical, food factory, or refinery, that’s a problem not just for downtime but more importantly because it could cause a safety or environmental incident. If it’s a temperature gauge, that’s much less risk. Companies will have a risk register for everything else, including natural disasters. They should have one for OT cybersecurity risk too.

Learn more

To learn more about Microsoft Security solutions, visit our website. Bookmark the Security blog to keep up with our expert coverage on security matters. Also, follow us at @MSFTSecurity for the latest news and updates on cybersecurity.

The post Mitigate OT security threats with these best practices appeared first on Microsoft Security Blog.