This is a series of blog posts about X-Road® and containers. The first part provides an introduction to containers and container technologies in general. The second part concentrates on the challenges in containerizing the Security Server. The Security Server Sidecar – a containerized version of the Security Server – is discussed in the third part.
Security Server Sidecar is a containerized version of the Security Server that supports production use. The Sidecar is a Docker container that runs in the same virtual context (virtual host, Kubernetes Pod, etc.) with an information system. The containerized approach makes running the Security Server more cost-effective since no separate host server needs to be allocated for each Security Server. Besides, more and more information systems exchanging data using X-Road are running in containers too, so it’s beneficial to be able to run the Security Server on the same platform with the information systems that are connected to it.
The Sidecar solves the challenges related to running the Security Server in a container, and it uses the standard release versions of the Security Server software. In other words, the Sidecar is built from pre-built packages of the official X-Road releases, and it is a separate project that builds on the X-Road core.
The Sidecar project
From an administrative perspective, Security Server Sidecar is a project of the Finnish Digital Agency (DVV) that is implemented in collaboration with NIIS. The DVV owns the project, and NIIS is responsible for coordinating the daily development activities. All the deliverables are released on NIIS’s GitHub and Docker Hub accounts. The project is currently ongoing, and it will be completed by the end of 2020.
The project will produce a Security Server Sidecar Docker image with a couple of alternative configurations. The Sidecar slim is a lightweight version of the Security Server, and it does not include message log, operational monitoring, and environmental monitoring modules. It means that the slim version does not log messages or provide any monitoring capabilities. However, technically it can be used for both consuming and providing services if the capabilities mentioned before are not required.
Instead, the regular Sidecar includes message log, operational monitoring, and environmental monitoring modules too. Similarly, the regular Sidecar can be used for both consuming and producing services. The Sidecar slim is a lightweight version of the Security Server while the regular Sidecar provides all the features of a full-blown Security Server installation. Besides, also versions with country-specific meta-packages are available. Currently, the only country-specific configuration available is the Finnish meta-package.
In addition to the Security Server Sidecar Docker image, the project also produces documentation to support the use of the image. The documentation will cover best practices and examples of how to run the image on a Kubernetes cluster using Elastic Kubernetes Service (EKS) on the Amazon Web Services (AWS) cloud platform.
What is a sidecar?
In general, the sidecar is a design pattern commonly used in a microservices architecture. A sidecar is an additional component that is attached to a parent application to extend its functionalities. The pattern aims to divide the functionalities of an application into separate processes. The approach allows adding new capabilities to an application without changing the application itself. In this way, a sidecar is loosely coupled with the application. For example, logging and monitoring are functionalities that are often implemented using a sidecar.
Applying the sidecar pattern to the Security Server
When using the regular Security Server version on a Linux host, it’s strongly recommended that the Security Server is running on its own host and not on the same host with an information system that is connected to it. It means that at least two separate hosts are required. Instead, the idea of the sidecar architecture pattern is that an application and a sidecar run on the same host or context, close to each other. With the containerized Security Server, the goal can be achieved since the Security Server is packaged in a container that runs in its own isolated process.
Image 1. The Security Server Sidecar and an application in the same virtual context.
The original idea of the sidecar pattern is that multiple copies of the same sidecar are attached to the application so that each instance of the application has its own sidecar. In case different applications use the same sidecar, the same approach applies to all the applications and their instances.
Image 2. A single Security Server Sidecar instance is shared between multiple instances of an application, and between different applications.
Despite its name, the original sidecar pattern does not work very well with the Security Server Sidecar since the Sidecar requires the same configuration and registration process as the regular Security Server. Also, even if the Security Server is containerized, the footprint of the Sidecar container is still relatively massive compared to the footprint of average containers. Therefore, it’s recommended that a single Sidecar container is shared between multiple instances of the application, and it may also be shared between different applications too. For high availability and scalability, a Sidecar cluster consisting of a primary node and multiple secondary nodes can be considered. Let’s take a better look at the different deployment alternatives next.
Running the Sidecar on Kubernetes
The Sidecar can be deployed to different container management systems, thanks to standardization. One of the most popular container management systems is Kubernetes, which is available as a service on multiple cloud platforms, such as Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform. Kubernetes is open-source, which means that it can be used in on-premise and private cloud environments too. In this case, we’re going to concentrate on running the Sidecar on AWS Elastic Kubernetes Service (EKS). During the development project, the Sidecar has been tested using the Docker Engine and EKS.
Since the Security Server is a stateful application, it is required that an external database and persistent file storage are used in all the deployment alternatives. In this case, Amazon Relational Database Service (RDS) is used for the Security Server databases, and a Kubernetes persistent volume is used to store the configuration files.
Before going into the deployment models, a few words about Kubernetes and Pods since Pods play an essential role in the deployment models. In Kubernetes, a Pod is a group of one or more containers that run in a shared context, share the same storage and network resources, and share a specification of how the containers are run. Each Pod runs a single instance of an application, and scaling the application horizontally means using multiple Pods, one for each instance of the application.
Deployment is another Kubernetes concept that’s essential to the Sidecar deployment models. A Kubernetes deployment represents an application that consists of a set of identical Pods. The deployment specification defines the configuration of the Pods and the number of replicas to run. The deployment maintains the Pods and monitors that there’s a correct number of Pods running. Also, it’s possible to create a horizontal autoscaler for a deployment that automatically scales the number of running Pods based on the selected metrics.
Security Server as a sidecar
Image 3. The Security Server Sidecar as a real sidecar inside the same Pod with an information system.
The first alternative is to deploy the Sidecar as a real sidecar, which means deploying it in the same Pod with an information system. It is a feasible approach if there’s always only one Pod running, and other information systems do not need to access the Security Server. The information system can be a service consumer, service producer, or both.
In case the information system must be scaled horizontally, this approach does not work very well. The reason is that adding a new Pod means that the new Security Server running in the Pod must always be configured and registered before it can be used. Since the onboarding process of the Security Server may take from days to weeks, the approach is not feasible. Also, deploying a Security Server for each Pod would generate quite much overhead from the resource consumption perspective.
Single Security Server
Image 4. The Security Server Sidecar in its own Pod and shared by multiple information systems.
When multiple information systems or several instances of the same information system need to access the Security Server, it’s better to deploy the Security Server in a separate deployment using a single Pod. In this way, the information systems can be scaled independently from the Security Server, and some of them might even be running outside of the AWS EKS cluster. However, the number of Security Server instances is limited to one. Since the external database and persistent volume are used, Kubernetes can automatically recover the Security Server Pod in case of failures. Also, in this case, the information systems can be service consumers, service producers, or both.
In case multiple Security Servers are required because of high availability and/or scalability, there are two alternatives: multiple independent Security Servers or a Security Server cluster. Multiple independent Security Servers mean deploying several Security Servers, with each of them having a unique identity. This approach provides high availability. Instead, a Security Server cluster means deploying a group of Security Servers that share the same identity and that are accessed through an external load balancer. The cluster provides both high availability and scalability. More information about X-Road’s load balancing alternatives can be found here.
Multiple Security Servers
Image 5. Multiple instances of the Security Server Sidecar shared by several information systems.
Deploying multiple independent Security Servers provides high availability but not scalability from a performance point of view. In this setup, multiple Security Servers with unique identities are deployed as separate, independent applications. In practice, Security Servers are deployed using separate deployments, which means that they have their own run specifications. Besides, the number of the Security Server Pods within a deployment is limited to one for each Security Server. Adding a new Security Server to the setup means creating a new deployment plus configuring and registering the newly created Security Server.
The information systems can be service consumers, service producers, or both. Service consumers may connect to the Security Servers directly, or there may be another load balancer between the consumer information systems and the Security Servers (omitted in the diagram). In the case of service producers, Security Server’s internal load balancing enables publishing services on multiple Security Servers and routing service requests to all of them. However, the configuration (e.g., available services, access rights) must be manually synchronized between Security Servers providing the service. Only the Security Server cluster provides automatic synchronization between Security Servers.
Security Server cluster
Image 6. Security Server Sidecar cluster with an external load balancer shared by multiple information systems.
A Security Server cluster provides both high availability and scalability. It consists of a primary node and one or more secondary nodes that all share the same configuration and identity. In this setup, the primary node is used to manage the cluster, and it does not process messages. In practice, configuration changes are done on the primary node, and they’re automatically replicated to the secondary nodes. Replication covers the configuration database and configuration files. Changing the configuration on the secondary nodes is blocked. The secondary nodes are connected to a load balancer that distributes incoming traffic between them. Further implementation details of the Security Server cluster on Kubernetes are studied in more detail in the Sidecar project.
The information systems can be service consumers, service producers, or both. Service consumers may connect to the secondary nodes directly, or there may be another load balancer between the consumer information systems and the secondary nodes (omitted in the diagram).
Multiple Security Servers or a Security Server cluster?
The key difference between multiple independent Security Servers and a Security Server cluster is that the cluster provides both high availability and scalability. In contrast, independent Security Servers provide only high availability. In the cluster, secondary nodes can be scaled with less effort while setting up a new independent Security Server is a manual operation. Also, in the cluster setup, all Security Servers share the same identity and configuration that is synchronized automatically. Instead, multiple independent Security Servers each have their own unique identity and configuration, and there’s no synchronization between them. Which one is the best alternative depends on the use case and its requirements.
Containerized future?
The four deployment models described before give an overview of what kind of models can be considered for the Sidecar. The same models can be applied regardless of the underlying platform or environment where the Security Server is deployed. Of course, the implementation details vary between different platforms and environments, but the high-level architecture patterns remain the same. However, the models do not provide an exhaustive list of available alternatives since different models can be combined and new elements, such as load balancers, can be added to the described ones.
Adding support for containers does not mean dropping support for Linux – running the Security Server on Ubuntu and Red Hat will remain supported in the future too. Containers are an alternative way to run the Security Server, and X-Road members are free to choose between the available alternatives. Containers are a convenient way to run the Security Server when an organization already has the required capabilities to operate and manage containers in production environments. However, in case an organization is not quite there yet, using virtual machines might be a better alternative since mastering Security Server containers on a production level requires time, effort, and experience.
It must also be noted that the Security Server configuration process – registering and onboarding a fresh Security Server to an X-Road ecosystem – is always the same regardless of the Security Server packaging. From a process perspective, the containerized version of the Security Server is not different from the Linux packaged version. In this way, X-Road members can be sure that the same level of trust is always guaranteed in data exchange between X-Road members.