In today’s digital world information is stored across multiple information systems owned and maintained by different organisations. In addition to information spreading across multiple organisations, every organisation has internally numerous information systems that store information. Most of the digital services and processes require accessing multiple information systems and combining data from different sources – both inside an organisation and across multiple organisations. Without connections between different information systems building digital services would be extremely challenging if not impossible.
The ability of information systems to exchange and utilize information is known as interoperability. Unlike it may first sound like, interoperability is not only about technology and technical connectivity. On the contrary, interoperability consists of different layers that include also technology. The European Interoperability Framework (EIF) defines four layers of interoperability:
legal – aligned legislation
organisational – coordinated processes
semantical – precise meaning of exchanged information
technical – connecting information systems and services.
Image 1. EIF conceptual model. (source)
All the four layers are equally important when building digital services and processes. In addition, challenges on one layer are often reflected to other layers too. Therefore, it is important to be aware of all the layers and not to neglect any of them. That being said, in this blog post I’m going to concentrate on the technical layer and its dimensions because covering all the layers at once would be too big a bite to chew.
Data Exchange Scenarios
When it comes to a public sector organisation exchanging information, three top level data exchange scenarios can be recognized:
Internal – data exchange within an organisation
National – data exchange on national level
Cross-border – international data exchange.
The same rules, laws and regulations don’t apply to national and cross-border data exchange which is why they are two separate scenarios instead of a single “external” scenario. Cross-border data exchange between authorities usually requires both state level agreements and data exchange agreements between the data exchange parties. The two scenarios could probably be combined as a single scenario making the total number of different scenarios two: internal and external.
The common factor between the scenarios is that all three require certain technical basic elements including, but not limited to connectivity, secure communication protocols, interfaces and integration services. The more standardized these elements are, the less work is required to build new connections between information systems and services. For example, if there’s no commonly agreed solution to securely connect information systems to each other and to how the connections are managed, the result is probably a jungle of point-to-point connections which means agreeing on the connection details and then building the connections every time when a new connection is needed – this is repeated again, again and again.
However, even if the technical basic elements in all the scenarios are the same, they are usually implemented using different technical solutions and technologies. Implementing a standardized connectivity layer within an organisation is usually based on different technology than a standardized connectivity layer with external parties. Let’s take a look at an example of an organisation that has a microservice-based information system with REST APIs published to external consumers.
Image 2. A microservice-based information system with REST APIs published to external consumers.
Internal Communications
Internally the information system uses a service mesh to facilitate service-to-service communications between microservices. A service mesh is a dedicated infrastructure layer that provides features such as standardized and secure connections, service discovery, and centralized logging and monitoring capabilities. Microservices communicate with each other through a service mesh proxy that is usually responsible for microservice level authentication, message routing, service discovery, automatic retries, timeouts, logging etc. As these features are provided by the proxy, they do not need to be implemented in the application code of each microservice separately. In addition, a service mesh usually has a centralized control plane that can be used to configure the proxies, and access logging and monitoring information etc.
Requests originating outside of the mesh typically enter the mesh through a service mesh gateway component. Available capabilities vary between different solutions, but in general, a service mesh is designed to manage traffic internal to the service mesh. In this case the example was very simple, but in real life a service mesh could serve multiple information systems and span multiple networks and data centers.
Exposing Services Externally
When it comes to accepting traffic from outside of an organisation, an API gateway comes into the picture. An API gateway exposes backend services as managed APIs and distributes traffic internally – in and out of the service mesh. An API gateway provides a single entry point to all clients, and hides the details of individual microservices. An API gateway also typically provides capabilities such as logging, monitoring, metrics, access control, request limiting, message transformations, orchestration etc. In addition, an API gateway is usually well connected to other components of the API management ecosystem, e.g. API marketplace and API publishing portal.
Even though API gateways and service meshes are complementary solutions, they have many overlapping functionalities and features. They are often deployed together, but they can be deployed separately as well. In addition, an API gateway can be used for internal purposes too – not only for publishing services to external clients. Similarly, a service mesh could be used to publish services to external clients.
What X-Road Brings to the Puzzle?
As so far I have been writing about internal and external data exchange, but I haven’t written a word about X-Road yet. At this point you may be wondering what is X-Road needed for if internal and external data exchange can be implemented using other technologies.
First of all, X-Road is best suited for external data exchange over the public Internet. The most common use case is data exchange between two organisations, but a single organisation may have information systems that are hosted in different locations and communicate with each other over the Internet too. In this case X-Road is a good fit for internal data exchange as well.
At first sight X-Road may seem like a service mesh as the architecture and feature sets have many similarities – both provide secure and standardized connections, service-to-service authentication, logging, reporting etc. In addition, both are based on an architecture model that implements service level communication through a proxy component. However, X-Road is not a service mesh as service mesh is the connection layer between different services in microservices architecture. In other words, service mesh is used as an internal connection layer within an application or between multiple applications of a single organisation whereas X-Road is used as a connection layer between different organisations and information systems.
How about X-Road and an API gateway then – are they mutually exclusive or can they be used side by side? X-Road and an API gateway are both used to publish services to external clients. Their architecture and feature sets are different even though they have features in common too, e.g. publish APIs to external clients, service-to-service authentication, authorization, logging, metrics. The major difference between X-Road and API gateway is that X-Road requires that the Security Server is used on both service consumer and provider side whereas API gateway enables client connections directly without any additional components on the client side.
Image 3. Point-to-point connections, an API gateway and X-Road in comparison.
Overall, an API gateway provides more flexibility and API management related features compared to X-Road, but when the same client communicates with multiple API gateways the client must adapt to different requirements and configurations of multiple service providers. Instead, X-Road provides a single communication channel between multiple service providers and services that all share the same configuration that is automatically distributed and applied by X-Road. In addition, X-Road guarantees that both service consumer and service provider meet the same security requirements, and non-repudiation of all the processed messages by signing, time-stamping and logging every processed message on the consumer and provider side. The logs can be used in a court proceeding as evidence. These features make X-Road ideal solution for secure, reliable and auditable data exchange.
One Happy Family
X-Road, an API gateway and a service mesh all have their place in the interoperability puzzle, and they can be used together side-by-side. They all have their own strengths and they can be used to complement each other.
X-Road is an ideal solution for secure data exchange that requires strong authentication of data exchange parties and non-repudiation with recorded eIDAS compliant evidence. X-Road can connect to backend services directly or through an API gateway. X-Road does not support message transformations, orchestration, rate limiting, quotas etc. which can be implemented in the API gateway layer if they are required.
Some APIs may not require strict security controls or they should be accessible without an additional access point on the client side, e.g. APIs providing open data. There’s no reason why an API could not be published through multiple channels, for example an API providing open data can be published through both X-Road and an API gateway. The benefit of this approach is that organisations that are not using X-Road can access it directly through an API gateway and organisations using X-Road can access it using the same channel they use to access other services and APIs too.
Image 4. The example application with X-Road.
Let’s go back to the different data exchange scenarios mentioned earlier – internal, national and cross-border. X-Road is a good fit for national and cross-border data exchange, and it can be used for certain internal data exchange use cases too. An API gateway can basically be used for all the scenarios, but depending on the use cases and their requirements X-Road might be a better choice for external data exchange and a service mesh for internal data exchange. Last but not least, a service mesh is best suited for the internal scenario for microservice-based applications.
Disclaimer
Finally, it must be said that there’s one major difference between X-Road, an API gateway and service mesh that has not been brought up yet. API gateway and service mesh are architecture patterns which have multiple implementations that all have their own set of features and functionalities. In this blog post I have compared API gateway and service mesh to X-Road on a general level without referring to any specific implementation, solution or product. Instead, X-Road is a product with a specific set of features and functionalities. This means that conceptually X-Road, API gateway and service mesh are not the same thing.