X-Road® is open-source software and ecosystem solution that provides unified and secure data exchange between organisations. X-Road is based on a distributed model, and it enables decentralised data management and data sovereignty within the ecosystem. Every organization is in full control of its data and services, and data is always exchanged directly between two trusted members without third parties having access to it.
X-Road operator is the owner of the X-Road ecosystem and is responsible for all the aspects of the operations. The responsibilities include defining regulations and practices, accepting new members, providing support for members, and operating the central components of the X-Road software. X-Road members are organizations that have joined the ecosystem and produce and/or consume services with other members. A member organization can be a service provider, a service consumer, or both. Also, a functioning X-Road ecosystem requires two types of trust services: 1) time-stamping authority (TSA) and 2) certification authority (CA). Trust service providers are organizations providing these services.
Technically, the X-Road core consists of a Central Server and Security Server that are the foundational building blocks of the ecosystem. These components are required together with the trust services to establish a trusted network of organisations and enable secure data exchange between its members.
Besides the core and trust services, building a functional and scalable ecosystem requires some additional building blocks that support the operations and use of the ecosystem. These building blocks provide member management and onboarding capabilities, service discovery, metrics collection and reporting, and technical monitoring. The X-Road core doesn't currently offer the capabilities, and therefore, additional building blocks are required. In general, implementing and maintaining these building blocks is the responsibility of the X-Road operator. Next, let's take a closer look at these building blocks.
Service management portal
A service management portal or a self-service portal is a web portal for managing the administrative details of the ecosystem membership. The administrative tasks related to the membership and its management are usually completed using the portal. For example, new members must first send a membership application and, once it has been approved, sign a membership agreement where they agree to follow the terms and conditions of the ecosystem. Also, the portal doesn't have to be limited to the administrative level, and it may cover some technical configuration steps, e.g., requesting certificates, and provide ecosystem-specific documentation and instructions. Depending on the implementation, some parts of the process may be automated, while others require manual input. Also, the portal may provide separate interfaces for the representatives of the member organisations and the X-Road operator.
The Central Server contains a registry of X-Road member organisations and their Security Servers. The information managed by the Central Server is technical and doesn't overlap with the data managed by the service management portal. Also, the service management portal may support the technical onboarding process by automating parts that are not directly covered by the Central Server, e.g., requesting certificates. Since there's a strong connection between the technical and administrative information, the portal and Central Server may be connected. However, a management REST API for the Central Server that enables a seamless integration is a work in progress currently.
An alternative to the service management portal is to manage the membership information in a system that members cannot access directly and use email (or some other channel) for communications. For example, maintain the member information in an Excel file or an internal wiki page, and receive service requests by email. It is a quick and easy way to get started with the ecosystem, but it doesn't scale very well, provides little support for automation, and is not very user-friendly. Therefore, implementing a service management portal is highly recommended.
Currently, there’s no off-the-shelf open-source component available that could be used as a service management portal for X-Road. In general, a service management portal is a custom component, and it may also support a broader range of digital services. Also, it is often connected to ecosystem-specific backend services and registries, such as business registry, service catalog, authorisation service, etc. The Estonian and Finnish service management portals are good examples of how the portal can be implemented.
Service catalog
A service catalog is a web portal that contains descriptions of all the services available in the ecosystem. The primary purpose of the service catalog is to provide a user-friendly channel to search and discover available services. Also, the catalog may provide additional features to support the use of the services, e.g., request access to a service, sign a service agreement, etc. The service catalog is targeted at both business and technical users.
When services are connected to X-Road, their service descriptions are published on the Security Server by the Security Server administrator. The service descriptions can then be accessed using a service discovery mechanism provided by X-Road. However, the mechanism is very technical and requires direct access to the Security Server's messaging interface. Also, getting a list of all services available in the ecosystem would require querying each Security Server separately. Therefore, a more user-friendly service catalog is needed.
When implementing the service catalog, collecting the service descriptions from the Security Servers can be automated. In that way, the descriptions need to be maintained in a single place only, and all the changes in the source are automatically updated to the catalog. Nevertheless, additional metadata must be manually added and maintained on the catalog by a service administrator representing the organisation owning the service. The metadata may include any information related to the service and its use, e.g., a more detailed description of the data set, terms and conditions, contact information, pricing information, SLAs, etc.
The Estonian, Finnish and Icelandic (only in Icelandic) service catalogs serve as examples of how the catalog can be implemented. The source code of the Finnish catalog is freely available on GitHub, and it consists of two separate components: a service catalog portal and a collector to read the service descriptions from Security Servers and store them centrally. Currently, NIIS doesn’t provide a service catalog component for X-Road.
Reporting and metrics
Reporting and metrics mean collecting usage statistics and metrics from an X-Road ecosystem. The metrics include service usage statistics, response times, request sizes, service health data, etc. The metrics can be used to measure the size and activity of the ecosystem, and they also provide interesting information about the relationships between different member organisations and their services. The information enables the X-Road operator to overview the ecosystem's state and measure its growth. Thanks to the data, the X-Road operator can make informed decisions on the development and governance of the ecosystem.
To get an overview of the whole ecosystem, the raw metrics must first be read from all Security Servers and then stored and analysed centrally. However, it’s important to remember that the metrics do not contain data that is exchanged by the members, only metadata about the use of the services. Collecting the information requires installing the operational monitoring add-on on the Security Servers. The add-on collects the raw metrics data locally and makes it available through a query interface. Nevertheless, access to the interface is restricted so that only the X-Road operator can access the data of all member organisations. Regular members can access their data only.
X-Road Metrics is an open-source component maintained by NIIS to centrally collect, store, process, and publish the data provided by the operational monitoring add-on. X-Road Metrics consists of multiple modules, and its features include but are not limited to publishing the data as open data (from the Estonian ecosystem), generating a dependency graph of member organisations (from the Estonian ecosystem), and providing statistical reports to members. However, making other implementations that utilise the operational monitoring data is also possible since all the documentation and source code are publicly available. Of course, member organisations are free to use the data in their reporting and monitoring systems.
Technical monitoring
Technical monitoring means collecting technical monitoring and health data from an X-Road ecosystem. The data can be used to monitor the Security Server's health. The data includes system metrics (CPU load, free memory, available disk space, etc.), running processes list, X-Road version information, certificated details, etc. Also, the data can be used to recognise potential future problems and maintenance needs before they affect the operations of the Security Server, e.g., detect certificates that are about to expire, detect Security Server versions that are no longer supported. The information enables the X-Road operator to get an overview of the ecosystem’s health and monitor the maintenance of individual Security Servers.
At first, technical and operational monitoring may sound like the same thing or very similar. The difference is that technical monitoring concentrates on the Security Server while operational monitoring is about monitoring services connected to X-Road. However, the way how data is recorded on the Security Server and then collected and analyzed centrally is very similar.
To technically monitor the whole ecosystem, the raw monitoring data must first be read from all Security Servers and then stored and analysed centrally. The technical monitoring data doesn’t include sensitive information, only technical monitoring data. Also, the Security Server administrator can configure the data set that can be collected centrally. Collecting the information requires installing the environmental monitoring add-on on the Security Servers. The add-on collects the raw data locally and makes it available through a query interface. Like with operational monitoring, access to the interface is restricted so that only the X-Road operator can access the data of all Security Servers. Regular members can access their own Security Servers’ data only.
A common approach is to use existing monitoring tools and platforms to centrally store, analyse and visualize the technical monitoring data, e.g., Elasticsearch and Kibana. However, an X-Road-specific component is needed to read raw data from Security Servers. The Finnish Digital Agency has implemented such a component, and they've published it on GitHub under the MIT license. Currently, NIIS doesn’t provide a central environmental monitoring component for X-Road that could be used to monitor the ecosystem's health.
Where do I find the specifications?
Service management portal, service catalog, reporting and metrics, and technical monitoring building blocks offer capabilities that aren’t currently provided by the X-Road core. Those capabilities are not required when setting up a new X-Road ecosystem, but they certainly make operating and developing the ecosystem easier. First and foremost, they provide the X-Road operator with additional tools for informed decision-making and automating management processes.
The building blocks mentioned in this blog post are described on a conceptual level, and there's no formal specification for them. Therefore, every implementation of the building blocks is different and may not always provide the same set of features. However, how the building blocks are technically connected to X-Road must be based on the X-Road protocols and interfaces. Therefore, replacing one implementation with another should be possible if connections to other backend systems are ignored.
Currently, NIIS provides the implementation of the reporting and metrics building block in the form of X-Road Metrics. Also, implementations of the service catalog and technical monitoring building blocks are available as open-source. The service management portal is the only building block that doesn’t have open-source implementations available. More implementations are likely to become available in the future, and the technical specifications of the concepts will be defined in more detail.