Details on ‘Horizontal’ connectivity considerations
Telecommunication infrastructure
The main considerations for telecommunications design and performance when deploying horizontal connectivity are briefly described below. These factors are interdependent, and together influence the overall performance of the communication network.
Shared vs Dedicated Telecom Infrastructure
Available telecommunication media (e.g., radio link, optical fiber cable) to connect the local site(s) with the remote site is a fundamental aspect of the solution. Without a suitable bandwidth and a reliable telecom media, most of the aforementioned remote functions may not be feasible.
Based on a high-level classification, the network between the remote and local site can be:
- Dedicated IACS (Industrial Automation and Control System) networks, that are not shared with any other networks, e.g., dedicated fiber optics between the two locations, which is owned and managed by the organization.
- Dedicated media for IACS, but reduced segregation between internal IACS networks (e.g., logical separation only).
- Shared Network, where IACS is extended using logical separation on a shared network, e.g., shared with a corporate IT network. The network infrastructure, in this case, is still owned and managed by the organization.
- Third-Party Network, where the IACS network is tunneled on a network shared between more than one organization and managed by a third-party network service provider.
The design selection between a fully dedicated or shared infrastructure will be driven by practicality, viability, and the performance requirement of remote functions.
Dedicated, operator owned infrastructure is likely to provide the highest level of control of risks. Physical geographies and available existing infrastructure may often require the use of third party providers and it may be cost-effective to share infrastructure with other networks. Using a dedicated wired network with limited and secure network management access can provide greater reliability from both security and performance perspectives.
IACS network communications using shared or third-party networks should be tunneled and encrypted to protect the authenticity and the accuracy of the data between the local site and the remote site or support center.
Network Bandwidth and Latency
Bandwidth and latency need to meet the operational requirements for remote functions. Certain technologies may not be conducive to such operational requirements.
Further, certain encryption techniques can introduce unacceptable delays or require excessive bandwidth.
Availability and Reliable Performance
Ensuring availability requires adequate redundancy and the use of diverse technologies to avoid common-mode failure.
As an illustration, while it may seem expensive to achieve 99.99% availability, this percentage corresponds to an outage of around 53 minutes in a year or 4.3 minutes every month, which may be unacceptable for certain remote functions.
To achieve an appropriate and effective solution, it is important to determine and identify acceptable performance requirements, quality of service, and service levels and include this in the functional specification and network service contracts.
The remote site itself requires adequate utilities to be provisioned, i.e., power management, emergency power supply, UPS, and appropriate environmental conditions (heating, ventilation, and air conditioning). Depending on remote functions being incorporated, the utility requirements at the remote site may need to be matched to that of the local site.
Access Control
‘Horizontal’ extension is meant to connect to a remote control room. Once extended horizontally to a remote location, the devices in the remote location remain in the same security zone as the local IACS. Therefore, the access protection in the remote location needs to meet that of the local IACS zone and to address any additional threats of unauthorized access due to the remote connectivity.
Manned local sites usually have robust physical security due to the hazardous and critical nature of their operations. Remote sites need to be risk assessed to ensure that their physical security matches similar levels of access control as that of local sites.
Any extension of network introduces interconnections over and above that of a purely local IACS network. Hence, it is important to assess the overall access privileges available to a single user and limit user access to the resources necessary to perform their job responsibilities.
To prevent malicious activities without collusion, additional considerations may be given, such that no single individual has control over more than one high privilege activity (e.g., separating duties between network administration support and IACS engineering support).
Multiple Sites
In situations where a single remote location operates or controls more than one site, further considerations may be required to ensure the integrity of each site is protected.
These may include:
- Appropriate network zoning to prevent the propagation of threats between sites.
- Access and privilege control to limit personnel to their authorized actions only
- Robust change management to prevent software changes to one site automatically being rolled out to the others.
- Identifying and eliminating any common or single-point failure that can affect multiple sites.
Interdependencies
Any interdependencies between the local and remote sites need to be also clearly identified, specified and tested. These may include factors such as time synchronization, shared printing, domain servers, backup/restore servers, engineering stations, process data server, location of master safety controllers for deterministic safety communication, etc.
The impact of a disconnection between the local site and remote site (for example, telecom link failure, black start, cyberattack, disaster recovery, performance issues, or system upgrades) is to be studied to ensure that all necessary functions are still available locally.
Response and recovery strategy
Network response and recovery procedures need to meet the performance requirements and to work in conjunction with the site black start procedure.
In the event of a communication failure, the network design should enable the intended response to take place, e.g., graceful shutdown of the local site, or transfer of control to the local site (island mode) with no impact on local performance. This includes the necessary alarms to be determined and incorporated to both the Operators and support personnel.
Similarly, recovery procedures need to be defined and tested to ensure correct restoration of services and transfer of control authority to enable a safe return to normal operations without compromising the integrity of the IACS.
Adequate planning and testing against potential failures along with appropriate support service agreements are indispensable in ensuring an effective response, troubleshooting, and recovery.
Details on ‘Vertical’ remote connectivity considerations
Secure network architecture
Deploying a secure baseline architecture is a prerequisite for all ‘vertical’ remote connections. The key elements are network separation, Demilitarized Zone (DMZ), and control of dataflow between networks.
Complete and accurate vendor documentation of the solution for vertical connectivity in all phases of the project is important to fully assess the security of the architecture. All components in the vertical chain, including third-party components, need to be secured, patched and updated.
Network Separation and Segmentation
Network separation and segmentation involves separating the IACS networks from all other networks and partitioning the IACS networks into smaller security domains. The IEC 62443 reference architecture for network segmentation forms the basis of this document.
IACS are generally separated using a firewall that can establish a DMZ or intermediate network between the enterprise network and the IACS networks. A DMZ makes it possible to avoid any direct communication between the IACS network and any other networks.
Firewalls are only effective when properly configured, monitored, and maintained. The default policy should be to ‘Deny All’ with unnecessary ports and services disabled or removed.
For more granular access control, or to contain network-based breaches from spreading across the IACS, the IACS may be further segmented or partitioned into separate security zones based on factors such as system criticality, interconnections or communication requirements, type of communication protocol, and access requirements.
It should be noted that any network barriers used within IACS will be required to meet the availability and reliability requirements as per the IACS system requirements. Often, firewalls capable of inspecting IACS specific protocols are used to monitor and control communications in IACS networks.
Encryption
Encryption protects against man-in-the-middle attacks for network traffic passing through public networks or shared network infrastructure.
Encrypting control system data exchanged with an external network with a strong algorithm provides protection against network-based attacks and network spoofing. This is mainly used when network infrastructure is shared with other networks.
Strong user Authentication and Authorisation
Authentication involves verifying the claimed identity of the user. Generally, authentication requires a user to provide a user ID and a password. Multifactor authentication adds an additional layer of security by requiring the user to provide additional verification data, such as a one-time-password or biometric information.
Authorization, on the other hand, requires that once the user is authenticated their privileges are restricted to only those required to perform their roles. Role-based access control technologies can be used to restrict IACS user privileges.
A centralized authentication and authorization system for the IACS zone allows scaling as well as effective maintenance of user accounts but requires a high degree of security as it is a valuable target to malicious actors.
Authentication and authorization can be further enhanced by applying appropriate division of responsibility to prevent a single user from having the authority to modify critical networks and systems or elevate their own privileges without having to collude with other users.
The identification, authentication, authorization and auditing system (such as Domain Controller and DNS) should be separated between the corporate system and IACS systems in order to avoid strong interdependencies and avoid creating exposure of the IACS system.
Remote user authentication
Remote external users are generally required to be authenticated at the enterprise network perimeter. Connecting to the enterprise network from an external network will normally require two-factor authentication.
On successful authentication at the enterprise network perimeter, the user is connected to a remote access server in the DMZ. To gain access to the IACS, the user is again verified using a two-factor authentication mechanism at the DMZ.
The connection between the remote user and the remote access server is encrypted to prevent communicating sensitive data such as user credentials in clear text and mitigate against man-in-the-middle threats.
If authentication is successful, the user is limited to access authorized systems only by using a granular access control at the DMZ. Further, privileges are locked by user-specific roles, e.g., view only, engineering, etc.
Figure: Remote User Authentication
Real-time remote and privileged session monitoring for remote users and accounts with higher privileges can help detect suspicious activities. Recorded session logs can be stored securely to prevent unauthorized alteration.
Enabling session termination allows a remote user session to be terminated by a system administrator, on-site personnel, or automatically after a set length of time. Accounts can also be set to lock after a set number of unsuccessful login attempts to reduce the potential for forced entries. In addition, accounts should be managed and removed after a defined period of inactivity.
To avoid system conflicts, concurrent remote and local sessions can be restricted to allow only one privileged user session at any time on critical systems.
Physical access to remote location access devices
Gaining physical access to a remote device connected to the IACS network is the same as gaining local access to the control system.
A combination of physical security controls such as locked doors with smart card access, access monitoring technology, and housing servers and workstations in secure racks can help prevent unauthorized access to capable of remote access to the IACS. It is important to implement session monitoring to prevent and detect malicious activities.
Where continuous operator sessions are not required, session locks can be used to prevent access to idle remote consoles. Additional controls include deploying tools that identify and prevent unauthorized devices to be connected to the remote device or remote access network.
Remote Access Server security
The IACS network can be attacked using the permitted application traffic from the DMZ. This risk is usually reduced by actively patching the servers in the DMZ and enforcing firewall rule-sets that only permit connections initiated by the devices in the IACS network.
Securing the remote access servers in the DMZ is vital as it can be the single point of entry into the IACS zone and can become the target for an attack.
The general requirements include:
- Maintaining full OS and application patches on the servers, auto-updating the latest anti-virus signatures, and running anti-malware tools full-time.
- Specifically hardening the servers to remove or disable unnecessary services, applications, and network protocols.
- Allowing management of the server to be performed only from trusted hosts on an internal network by authorized administrators.
- Periodically testing and refining security controls and removing vulnerabilities (e.g., using a combination of vulnerability scanning and penetration tests).
- Enabling system logs and deploying procedures and tools to process and analyze the log files for attempted and successful intrusions on the server.
The Remote host or device security
The remote device is the device used by the remote user to perform the remote connection. Unless permanently located in a controlled physical environment, it should be assumed that remote devices will get connected to other untrusted networks. If the remote device is used for multiple purposes, it can be exposed to threats before relevant patches and updates are applied.
Common mitigations can include:
- Configuring the remote devices to automatically check and install relevant software patches and anti-malware updates.
- Deploying technology to quarantine and check that all relevant patches and anti-virus updates are installed before permitting remote connection.
- Installing the remote device to be always behind a firewall with intrusion detection and prevention capability.
The company may require remote devices and users to be audited and qualified before allowing remote access. Organizations may not always have the authority to enforce security controls on the remote device and the device remains untrusted. In this case, applying security controls on the IACS environment with ‘vertical’ extensions is highly important due to the increased exposure.
Security Monitoring
Detection of suspicious or malicious activity can be done by monitoring all sessions through a single entry point.
Monitoring, logging, and auditing provide effective mechanisms to understand the state of the system, validate its integrity, troubleshoot performance issues and identify any violations of security settings.
The detection, monitoring, and alerting mechanisms include:
- Anti-virus tools that can detect and remove known malware, preventing it from infecting other devices
- Intrusion detection systems that monitor and alert security personnel on unusual patterns of network traffic and systems logs or changes to critical operating system files
- Security monitoring and network management tools at the boundaries and intermediate network nodes can provide early detection and response capability to network issues and potential intrusions
Effective deployment of Intrusion detection and prevention can require both network-based IDPS (Intrusion Detection and Protection System) that monitor and generate alarms on abnormal network traffic and Host-based IDPS that monitor and alert on unusual access patterns, system events, and system configuration changes.
Source: International Association of Oil & Gas Producers
Acknowledgments: IOGP Instrumentation and Automation Standards Subcommittee (IASSC) Remote Operating Centres Task Force.
Read Next:
- Industrial Automation System
- Control System Architecture
- Functions of PID Controller
- Basic Process Control System (BPCS)
- Design Guide of Control Room