X-Road 7.5.0 Is Here

During the first half of 2024, the focus on X-Road development shifted towards X-Road 8 “Spaceship”, and we've put much effort into X-Road 8-related development activities. Nevertheless, X-Road 7 hasn't been forgotten either, and we've worked on various improvements included in X-Road 7.5.0, released at the beginning of July. Let’s see what the highlights of the new version are! The full release notes are available here.

Support for automated authentication and sign certificate management

Until now, managing the Security Server authentication and sign certificates has been an entirely manual process. The Security Server administrator must create a new key pair, generate certificate signing requests (CSRs), send the CSRs to a Certificate Authority (CA) for signing, import and configure the issued certificates. Depending on the X-Road ecosystem and the CA, completing the whole process may take several days.

Starting from version 7.5.0, the Security Server authentication and sign certificate management can be automated using the Automatic Certificate Management Environment (ACME) protocol. At this point, the automation covers the most time-consuming part of the process – sending the CSR to the CA for signing, issuing the certificate and importing it to the Security Server. In other words, the Security Server administrator still needs to create a new key pair manually, provide the information included in the CSR, start the ACME process and activate the imported certificate. However, the whole process is now completed in minutes instead of days.

Using ACME on the Security Server requires at least one available CA that supports the ACME protocol. The X-Road Operator defines the CAs supporting ACME on the Central Server. In addition, depending on the CA, using ACME may require additional configuration on the Security Server, e.g., using ACME requires authentication, and the authentication credentials must be added to the Security Server’s configuration. The configuration is member-specific. The Security Server documentation provides more detailed information about the required configuration. Please note that the configuration may vary between CAs and X-Road ecosystems. Therefore, always consult your own X-Road Operator in ACME-related questions.

The ACME HTTP challenge communication port is one configuration detail shared between different CAs and X-Road ecosystems. According to the ACME protocol, the ACME server communicates with the Security Server using port 80 when verifying the ownership of the domain included in the certificate using the HTTP challenge. Therefore, the Security Server must allow incoming connections to port 80 from the ACME server. Similarly, outgoing connections from the Security Server to the ACME server must also be allowed. All the required firewall configurations can be found in the Security Server installation guide.

In version 7.4.0, the default client information system inbound communication ports on Ubuntu-based security servers were changed from 80 to 8080 and 443 to 8443. Instead, the port must be manually changed before using ACME on older Ubuntu-based Security Servers, where port 80 is still used as a client information system inbound communication port.

Version 7.5.0 doesn’t yet support automatic renewal of authentication and sign certificates using ACME, but this support will be added in version 7.6.0. In version 7.5.0, the renewal must be initiated manually by the Security Server administrator, but starting from version 7.6.0, the Security Server will try to renew the certificates automatically before their expiration date.

Support for Ubuntu 24.04 LTS

Starting from version 7.5.0, the Central Server, Security Server, and Configuration Proxy support the latest Ubuntu 24.04 LTS version. The support includes installation packages and instructions for fresh installation and migration from Ubuntu 22.04 LTS. In addition to version 24.04 LTS, versions 22.04 LTS and 20.04 LTS are also supported. However, Ubuntu 20.04 LTS is only supported to provide an easier upgrade path to Ubuntu 24.04 LTS. It’s strongly recommended that production environments use Ubuntu 22.04 LTS or Ubuntu 24.04 LTS.

Support for Red Hat Enterprise Linux 9 (RHEL9)

Starting from version 7.5.0, the Security Server supports the latest RHEL9 version. The support includes installation packages and instructions for fresh installation and migration from RHEL7 and RHEL8. In addition to RHEL9, RHEL8 and RHEL7 are also supported. However, RHEL7 is only supported to provide an easier upgrade path to RHEL9. It’s strongly recommended that production environments use RHEL8 or RHEL9.

Adding/removing Central Server cluster nodes

The Central Server cluster node connection details are included in the configuration anchor imported to the Security Server. Using the information in the anchor, the Security Server can connect to the nodes. Suppose cluster nodes are added/removed. In that case, a new configuration anchor must be generated on the Central Server, distributed to all the Security Server administrators through an external channel (e.g., email), and imported to each Security Server. Only after that are the changes applied by each Security Server.

Starting from version 7.5.0, the Central Server cluster node connection details are included in the global configuration. Thanks to this, adding/removing cluster nodes is possible without importing a new version of the configuration anchor to each Security Server. Instead, the Security Server gets the updated configuration information from the global configuration directly and applies it immediately.

However, the changed cluster node details are not updated to the configuration anchor on the Security Server. Instead, the information is only present in the global configuration. Suppose the Security Server's local copy of the global configuration expires. In that case, the Security Server returns using the connection details from the local configuration anchor – potentially the one uploaded to the Security Server when it was originally initialised. Therefore, if the Security Server's local copy of the global configuration is expired, importing a new version of the configuration anchor is required if the current connection details don't include any Central Server node from the local configuration anchor (=the address of every Central Server node has changed since the Security Server was initialised).

The same logic applies to rotating global configuration sign keys included in the global configuration since version 7.4.0. The new sign keys are not updated to the configuration anchor stored on the Security Server. Instead, they're only present in the global configuration.

Security Server Sidecar instructions for Microsoft Azure

The Kubernetes Security Server Sidecar User Guide and Kubernetes Security Server Sidecar Security User Guide have been updated to provide instructions on running the Security Server Sidecar using Azure Kubernetes Service (AKS). The user guides now cover AKS and Amazon Elastic Kubernetes Service (Amazon EKS).

Other improvements and updates

Besides the already-mentioned features, version 7.5.0 includes many other minor improvements and updates. For example, improve warning messages when adding new OpenAPI services to the Security Server, improve dialogue navigation in the Central Server UI, allow defining additional options for PostgreSQL CLI tools used in the Security Server and Central Server processes, bump the Security Server sidecar Docker images to Ubuntu 24.04, and many more. To fully understand all the changes, please review the release notes document.

What’s next?

Version 7.6.0 will support the automatic renewal of authentication and sign certificates using ACME. Other changes that will be introduced in version 7.6.0 include, but are not limited to, support for defining a full name for subsystems, support for getting operational metrics on a REST service endpoint level (not just on a service level), support for Elliptic Curve (EC) cryptography. Version 7.6.0 will be released during Q4 in 2024.

At the same time, when developing X-Road 7, NIIS continues to work on the next X-Road major version – X-Road 8 “Spaceship”. Updates regarding X-Road 8 development are published regularly. Stay tuned!

Making of X-Road 8 – PoC June Status Update

At the beginning of January 2024, NIIS kicked off a proof of concept (PoC) to develop a new major version, X-Road 8 “Spaceship”, which nurtures the proven ecosystem model and security while it takes X-Road to the next level by providing a solid data space infrastructure.

With the proof of concept, NIIS aims to validate the feasibility of replacing X-Road's custom protocol stack with the Dataspace Protocol (DSP) and align X-Road's trust framework with the Gaia-X trust framework. The project also tries to ensure smooth integration with previous X-Road versions for backwards compatibility, estimate the changes required for information systems when transitioning to X-Road 8, and assess potential changes to existing X-Road components.

The PoC was completed at the end of May. In this blog post, we explore the outcome of the PoC project.

Getting started

The project was kicked off in January 2024, and it aimed to:

• Validate the feasibility of replacing X-Road's custom protocol stack with the data space protocol stack.

• Look into making X-Road's trust framework technically compatible with the Gaia-X trust framework.

One of the main drivers for the PoC was to better understand whether it's possible to implement the planned changes in a backwards-compatible manner. Since X-Road has a broad existing user base around the world, it's vital to understand the number of changes that the X-Road user organisations must implement for their information systems when migrating to X-Road 8. At the same time, new users starting with X-Road 8 shouldn’t have to deal with the details of the previous X-Road versions.

Since the previous blog posts provide detailed information about the monthly progress, I won't repeat their content here. Instead, I will give a high-level overview of the PoC project and its outcome. The previous blog posts are available here:

• January

• February

• March

• April

• May

Support for the Dataspace Protocol

The PoC implemented support for the dataspace protocol. It was achieved by integrating the EDC Connector and Identity Hub into the X-Road 7 Security Server. In practice, the Security Server’s existing functionalities were extended by the EDC. This approach allowed us a working implementation from day one that was also backwards compatible with X-Road 7.

The first iteration completed in February already supported the dataspace protocol and X-Road’s custom protocol stack. The interfaces between connected information systems and the Security Server were fully backwards compatible. In other words, no changes to the information systems were required to move from X-Road 7 Security Server to X-Road 8 Security Server. However, many implementation details were still missing, e.g., authentication of the data exchange parties, access rights management, logging, and secure connections between the Security Servers.

During the following months, we worked individually on the missing features. The aim was to validate that the features could be implemented and recognise potential issues that needed to be investigated in more detail. For example, we looked into defining access rights using ODRL and implementing message logging in the context of the dataspace protocol. Several questions that required further investigation were raised, but fortunately, none of them turned out to be a show-stopper.

The dataspace protocol is divided into two layers: 1) the control plane and 2) the data plane. We decided to use the EDC's control plane implementation and concentrate on the X-Road-specific data plane(s). Also, our goal was to implement all the required changes by extending the EDC since we wanted to avoid forking and creating an X-Road-specific version. Implementing some changes took a lot of work, but eventually, we reached our goal without forking.

During the PoC, the following features were implemented:

• Support for SOAP and REST services.

• Support for signing, timestamping and logging messages.

• Support for defining access permissions using ODRL (including migrating current access permissions to ODRL).

• Support for TLS between the Security Servers (control + data plane).

• Caching and reusing JWT tokens generated as a result of a successful contract negotiation phase.

• Support for “light context” - consuming services without a Security Server on the consumer side.

• Publishing DCAT service descriptions and collecting them centrally using the EDC Federated Catalog component.

• Storing credentials in the EDC Identity Hub and using them in authentication and access control during the data exchange process.

Since the project aimed to validate if the selected approach could be implemented in the first place, the main focus was to test if and how different features could be implemented without paying too much attention to quality. Therefore, the quality of the PoC implementation is far from the production level. Also, many features require additional investigation since several details were left open during the PoC.

Support for Gaia-X trust framework

The PoC looked into making X-Road's trust framework technically compatible with the Gaia-X trust framework. In practice, X-Road 8 should be aligned with the technical specifications and meet the technical requirements of the Gaia-X trust framework.

However, making X-Road’s trust framework technically compatible and interoperable with the Gaia-X trust framework doesn’t automatically make all X-Road ecosystems and user organisations Gaia-X compliant. Instead, it makes X-Road user organisations technically compatible with Gaia-X, but being Gaia-X compliant requires that the user organisations meet the requirements of the Gaia-X Compliance rules.

The first step was to set up an instance of the Gaia-X Digital Clearing House (GXDCH) services for more flexibility. We also configured an instance of the X-Road Test CA as a trust anchor, enabling us to issue our own certificates trusted by the GXDCH services. These steps helped us issue Gaia-X credentials that we could use for testing purposes.

Once able to issue Gaia-X credentials using X-Road members' sign keys and certificates, the main challenge was utilising them in the data exchange process between two X-Road 8 Security Servers. The EDC Identity Hub was used to store the credentials, and the EDC Identity and Trust Protocol (IATP) was used to exchange them. However, the EDC components didn't provide off-the-shelf support for Gaia-X credentials, so the implementation was done by trial and error. Despite the challenges, the project implemented support for using Gaia-X credentials for authentication and access rights management in the data exchange process. However, it doesn’t mean full support for the whole Gaia-X trust framework yet.

In addition, an X-Road-specific credential was implemented in the project. The so-called "X-Road Compliance Credential" includes the X-Road member organisation's X-Road identifier and can be used for authentication and access rights management. However, the credential was just a quick technical experiment at the very end of the project, and therefore, a lot further work regarding the update of the X-Road trust framework is required.

Summary

At the beginning of the PoC project, the main question was: can X-Road be transformed into a data space technology in a backwards-compatible manner? Based on the outcome, the short answer is yes.

In practice, the answer means that the project didn't discover any significant roadblocks that would make reaching the goal impossible. Instead, the project discovered many questions and areas that require further investigation. In other words, it seems that the goal can be reached, but at this point, we don't know all the steps required to get there. Also, there are several areas that we haven’t looked into yet, e.g., federation, operational, and environmental monitoring.

Image 1. The data space protocol stack.

The aim is to make X-Road 8 compatible with the data space protocol stack. One of the challenges is that the stack is not stable yet, and many of the key specifications are currently being defined. It means waiting for the first stable version or using a draft version. Both alternatives come with some challenges. However, the positive thing is that it's still possible to contribute to developing the specifications by collaborating with the data spaces community. For example, NIIS is a Supporter Member of the Eclipse Dataspace Working Group that’s currently working on the following specifications:

• Dataspace Decentralized Claims Protocol (DCP)

• Data Rights Policies Profile (DRP)

• Dataspace Protocol (DSP)

Interoperability within one data space and between different data spaces and data space technologies requires that the specifications be detailed enough and cover the key areas. Missing or vague specifications can lead to a situation where interoperability is not achieved. Also, having multiple competing and non-interoperable specifications leads to the same problem. Therefore, it is important to work together on common specifications and aim for technological convergence between various data space initiatives.

What’s next?

The first X-Road 8 PoC project was completed at the end of May, but the hard work is just beginning. We will continue to work with the topics discovered during the PoC and make more detailed plans for the production-level implementation. The X-Road JIRA has a more detailed list of ongoing tasks.

Also, another PoC within Estonia’s X-Road ecosystem is scheduled for the second half of 2024. The first beta version is expected in 2025, and the first production version is planned for 2026.

More blog posts about X-Road 8 will follow. Stay tuned!

Making of X-Road 8 – PoC May Status Update

At the beginning of January 2024, NIIS kicked off a proof of concept (PoC) to develop a new major version, X-Road 8 “Spaceship”, which nurtures the proven ecosystem model and security while it takes X-Road to the next level by providing a solid data space infrastructure.

With the proof of concept, NIIS aims to validate the feasibility of replacing X-Road's custom protocol stack with the Dataspace Protocol (DSP) and align X-Road's trust framework with the Gaia-X Trust Framework. The project also tries to ensure smooth integration with previous X-Road versions for backwards compatibility, estimate the changes required for information systems when transitioning to X-Road 8, and assess potential changes to existing X-Road components.

In this blog post, we explore the current status of the PoC project.

Support for the Dataspace Protocol

Since the April update, we have continued to work with message logging.

Message logging

In X-Road 7, message logging is implemented as a Security Server Proxy add-on. The Proxy is the Security Server component responsible for processing all the incoming and outgoing messages using the X-Road Message Transport Protocol. Instead, X-Road 8 will support the Dataspace Protocol, and the support will be implemented in a separate EDC-based (Eclipse Dataspace Components) component. In other words, X-Road 8 will have two components for processing messages: the Proxy component for the X-Road Message Transport Protocol and the EDC component for the Dataspace Protocol. In this way, X-Road 8 will be backwards compatible with X-Road 7.

A couple of architectural approaches were considered when implementing support for message logging for messages sent using the Dataspace Protocol. One alternative is to use the existing proxy add-on and integrate the EDC over gRPC, which is used for internal communication between different Security Server components. With this approach, the EDC sends all the messages to the add-on over gRPC, and the add-on logs them according to the Security Server's configuration. In this way, a single component is responsible for all the logging. However, the downside is that the messages to be logged may be huge, which might cause performance issues since they’re transferred over gRPC. Also, the whole message is always sent to the add-on, even if message body logging is disabled.

The second alternative is to implement message logging directly in the EDC component so that it's responsible for logging the messages that it processes. In this way, messages are not transferred over gRPC, which improves performance, and data that’s not going to be logged does not get transferred between different components. However, the downside is that message logging is done by two different components. Nevertheless, there’s a separate Proxy add-on that’s responsible for archiving all the messages.

The second alternative was implemented in the PoC, even if it meant having two components responsible for message logging. The main reason for the choice was better performance compared to the other option. However, ideally, only one component should be responsible for all the logging, without potential performance bottlenecks and data transfers between components when they are not required. Therefore, the selected approach will be refined in future development phases.

Nevertheless, the X-Road 8 PoC has a working message logging implementation now. Compared to X-Road 7, the main difference is that each message is signed separately instead of batch signing. Instead, batch timestamping is used, just like in X-Road 7. However, adding support for batch signing and updating the batch timestamping algorithm are both topics that will be revisited after the PoC. Also, the performance impact of signing each message separately requires further attention.

Support for Gaia-X Trust Framework

When joining Gaia-X and becoming a Gaia-X Participant, every organisation needs to get a Gaia-X Compliance Credential. Getting the credential requires submitting three credentials to the Gaia-X Compliance Service:

• Registration Number Credential issued by the Gaia-X Notary Service

• Legal Participant Credential self-issued by the Participant

• Gaia-X Terms & Conditions Credential self-issued by the Participant.

Organisations must submit these three credentials to the Gaia-X Compliance Service, which does a minimum validation for them, e.g., the Registration Number Credential is issued by a Gaia-X approved Notary Service. If all three credentials satisfy the Gaia-X Compliance Rules, the Gaia-X Compliance Service issues a Gaia-X Compliance Credential. The organisation may then use the Compliance Credential to prove being a Gaia-X Participant and Gaia-X compliant.

The self-issued credentials must be signed using a certificate issued by a Gaia-X-approved trust anchor, e.g., an eIDAS-compliant Certificate Authority (CA). For X-Road Member organisations, this means that they can use their existing X-Road sign key and certificate to sign the credentials as long as the CA that issued the certificate is trusted by the Gaia-X Compliance Service.

In the PoC, we have set up our instance of the Gaia-X Digital Clearing House (GXDCH) services so that we’re free to define the trusted CAs. This enables us to use certificates issued by our internal X-Road Test CA - the same CA that we use to issue authentication and sign certificates for X-Road.

In the PoC, creating the credentials requires manual steps since the tools used to create them (e.g., the Gaia-X Wizard) are not connected to the X-Road Signer component that holds the sign key and certificate. In practice, the sign key and certificate must be exported from Signer, made available to the tools used in creating the credentials and stored in the EDC Identity Hub together with the credentials to enable their use during data exchange. In the future, the creation of the credentials will be a built-in feature of X-Road, and no external tools or exporting/importing keys and certificates will be required. During the PoC, utilising available 3rd party tools instead of building new X-Road-specific ones has been the fastest way to gain progress, and it has allowed us to concentrate on X-Road-specific questions.

After creating the Gaia-X credentials, they are stored in the EDC Identity Hub together with the X-Road member-specific sign key and certificate. This means the sign key and certificate are stored in the Signer and the Identity Hub. However, this is a temporary solution, and the long-term plan is to store the sign key and certificate in one place only.

Using VCs in data exchange

The Dataspace Protocol does not address how trust is established between data exchange parties. Therefore, a separate trust protocol is required to guarantee interoperability between data spaces on the trust plane level. The Eclipse Dataspace Working Group is working on the Eclipse Dataspace Decentralized Claims Protocol (DCP) that defines an interoperable overlay to the Dataspace Protocol for establishing trust. NIIS is a Supporter Member of the working group.

Image 1. The dataspace protocol layers.

The group’s work is based on the protocol developed as part of the Eclipse Tractus-X Open Source Project. In addition to Tractus-X, the EDC Identity Hub implements the protocol too. The Eclipse Dataspace Working Group aims to develop the protocol specifications further and make them independent from individual data space implementation projects. In the PoC, the current version of the DCP implemented by the Identity Hub is used.

The credentials are used to authenticate the data exchange parties and can be used in defining access and usage policies. In other words, the data exchange parties exchange their credentials so that they can verify them and get access to their attributes. The Gaia-X Compliance Credential proves that an organisation is a Gaia-X Participant, and the other credentials contain additional information about the organisation. For example, in the PoC, the aim is to implement support for defining access policies using the Registration Number Credential that contains the organisation's registration number, e.g., VAT ID. In practice, the registration number can be used to define access rights to services just like the subsystem code is used now.

Using the registration number to define access rights to services is just a simple example. The credentials contain other information, and additional credentials can be supported, too. The benefit of using the information stored in different credentials over the subsystem code is that the credentials contain a more comprehensive set of information about the data exchange parties, and therefore, they provide a much more fine-grained way to define access and usage policies. 

The aim is not to rely only on Gaia-X credentials but to look into implementing X-Road-specific credentials. For example, the information currently stored in the sign certificate could be stored in an X-Road-specific credential. There are already some initial plans for how the X-Road-specific credentials could look and what information they could contain. Also, the first tests with an X-Road-specific credential containing the X-Road identifier of an X-Road member organisation have already been implemented. However, before starting to work on more considerable changes to the current X-Road trust framework, we want to understand the Gaia-X Trust Framework better and have more practical experience in it and the technologies it's based on. The aim is to use the same technologies in X-Road’s trust framework too.

What’s next?

The main challenge continues to be integrating the Gaia-X trust framework with the EDC-based X-Road 8 Security Server. During the last month, we have gained progress, but we've also discovered some new issues. The good news is that despite the issues, we're only a minor step away from being able to utilise Gaia-X credentials for authentication and authorisation during the data exchange process between two Security Servers. The ongoing PoC will end at the end of May, but we're confident that the remaining issues will be resolved and we can reach the goal by then.

More blog posts about X-Road 8 and the proof of concept will follow. Stay tuned!

Making of X-Road 8 – PoC April Status Update

At the beginning of January 2024, NIIS kicked off a proof of concept (PoC) to develop a new major version, X-Road 8 “Spaceship”, which nurtures the proven ecosystem model and security while it takes X-Road to the next level by providing a solid data space infrastructure.

With the proof of concept, NIIS aims to validate the feasibility of replacing X-Road's custom protocol stack with standard data space protocols and align X-Road's trust framework with the Gaia-X trust framework. The project also tries to ensure smooth integration with previous X-Road versions for backwards compatibility, estimate the changes required for information systems when transitioning to X-Road 8, and assess potential changes to existing X-Road components.

In this blog post, we explore the current status of the PoC project.

Support for the standard dataspace protocol

Since the March update, we have continued to work with message signing, logging and service catalog.

Message signing

In March, support for message signatures was already implemented in the PoC. The implementation utilises the Digital Signature Services (DSS) Java library provided by the European Commission. Initially, the PoC implementation used the JSON-based JAdES-B-LT signatures instead of the XML-based XADeS signatures used by X-Road 7.

In April, the signing implementation was reverted to the XML-based XADeS signatures. The change was because the Security Server produces ASiC-E containers that contain archived message log records. Unfortunately, according to AsiC-E specifications, only XadES and CAdES signatures are supported. Instead, according to the DSS library documentation, the CAdES signatures do not support signing multiple documents, which the Security Server requires when signing REST messages since the message body and HTTP headers are signed separately. Therefore, the only remaining option was to use the XAdES signatures already used in X-Road 7.

The downside of the XAdES signatures compared to the JAdES-B-LT signatures is that they’re more complicated to implement and require more space, affecting performance. However, the good thing is that keeping the message log functionality backwards compatible is easier since X-Road 7 also uses the XAdES signatures.

Another factor that affects the performance is the lack of support for batch signatures by the DSS library. However, that doesn’t directly depend on the used signature algorhithm. Instead, X-Road 7 supports batch signatures, but the implementation includes some non-standard X-Road-specific steps that DSS does not cover. And why would it cover them since the customisations are X-Road-specific?

The aim is to get rid of non-standard X-Road-specific details in X-Road 8. Therefore, in the PoC, each message is signed separately, negatively impacting the Security Server performance. However, updating the signer implementation may resolve the performance issues or at least reduce the impact. Therefore, evaluating and redesigning the signer architecture and implementation is a high priority in the X-Road 8 backlog.

Message logging

Once the signature algorithm has been reverted to XAdES, the PoC project can support message logging. This means messages sent using the standard data space protocol can be logged just like those sent using X-Road 7. However, there's at least one remaining issue with logging that's related to batch timestamping. 

Batch timestamping works so that messages processed by the Security Server are timestamped in groups according to the interval defined in the Security Server’s configuration (by default, once in a minute). When batch timestamping is executed, all the messages that have been processed since the previous timestamping transaction and haven't been timestamped yet are timestamped as a single group. Batch timestamping works asynchronously in the background and doesn't directly affect the message throughput.

In X-Road 8, the aim is to replace the current batch timestamping implementation with the DSS library. The issue with the current implementation is that it includes some non-standard X-Road-specific steps. However, it should be possible to make the batch timestamping implementation conform to the ASiC-E specifications using DSS, but some changes to the current implementation are required. Nevertheless, those changes are outside the PoC's scope and will be implemented after the PoC. Therefore, the current batch timestamping implementation is used in the PoC.

Service catalog 

The standard dataspace protocol defines a catalog protocol that can be used to query metadata about available assets from a service provider. The metadata includes information about datasets and usage control policies. The protocol uses Data Catalog Vocabulary (DCAT) v3 to define datasets while usage control is expressed as Open Digital Rights Language (ODRL) policies.

Federated Catalog is an EDC component that utilises the catalog protocol to collect metadata from different service providers and store it centrally. In that way, the metadata of all the services provided by various service providers is available from a single place. However, the Federated Catalog doesn’t provide a web UI to access the data. Instead, the data can only be accessed over an API, enabling alternative presentation layers to be built.

The Federated Catalog is very similar to the X-Road Catalog component. The X-Road Catalog is an X-Road extension that collects information on members, subsystems and services from an X-Road ecosystem and provides REST and SOAP interfaces to access the data. The X-Road Catalog uses the X-Road service metadata protocol to access the information stored on Security Servers. Currently, X-Road services are described using WSDL and OpenAPI specification.

In April, we deployed the Federated Catalog component to understand better how describing services could work in X-Road 8 and the relationship between DCAT, WSDL and OpenAPI specification. The catalog protocol provides metadata about services using DCAT, while the X-Road service metadata protocol provides detailed API specifications using WSDL and OpenAPI specification. Therefore, DCAT is not a replacement for WSDL and OpenAPI specification. Instead, they rather complement each other. Also, DCAT supports a property for endpoint descriptions that can be used to reference external service descriptions. The property can be used to create a link from DCAT to existing WSDL and OpenAPI service descriptions.

By default, the Federated Catalog supports only a limited amount of DCAT properties. Fortunately, enriching the catalog response with additional DCAT properties is supported. However, implementing a custom data mapper is required, which could affect interoperability with other connectors that do not support the customisations. Therefore, extending the supported DCAT properties and how they affect interoperability requires further studying during the subsequent phases of X-Road 8 development.

Also, having separate components for collecting DCAT, WSDL and OpenAPI specification documents from the ecosystem is not optimal. Instead, combining the Federated Catalog and X-Road Catalog into one component is a better approach.

Support for Gaia-X trust framework

Already in March, we had set up an internal instance of the Gaia-X Digital Clearing House (GXDCH) services to have more flexibility. The instance includes Gaia-X Registry, Gaia-X Compliance, Gaia-X Notary and Gaia-X Wizard. Also, we had configured an instance of the X-Road Test CA as a trust anchor, enabling us to issue our own certificates trusted by the GXDCH services. Since then, we’ve been able to issue Gaia-X credentials that we can use for testing purposes.

Support for the Identity Hub

Supporting the Gaia-X trust framework requires that X-Road member organisations have a credential repository (such as a digital wallet) for storing Verified Credentials (VCs) and sharing Verifiable Presentations (VPs). When an organisation becomes a Gaia-X participant, a Gaia-X Compliance Credential is issued to the organisation by the Gaia-X Compliance Service. The organisation stores the credentials in its credential repository. Instead, when the organisation needs to prove that it's a Gaia-X Participant (e.g., authentication during a data transaction between participants), it generates a VP that includes the Gaia-X Compliance Credential and shares it with a verifier (e.g., a data exchange partner). Identity Hub is an EDC component that stores VCs and composes VCs into VPs.

In the EDC sample implementation, Minimum Viable Dataspace (MVD), Identity Hub and connector are embedded in a single component. In the case of X-Road, it meant that the Identity Hub was embedded in the Security Server. To make the implementation more modular, we split the Identity Hub into a separate component that communicates with the Security Server over interfaces.

In the PoC, each Security Server must have its own Identity Hub instance. However, the plan is to support sharing the same Identity Hub instance between multiple Security Servers in the future. In this way, a member organisation may have one Identity Hub instance or an Identity Hub cluster that’s shared by all the Security Servers used by the organisation.

The Identity Hub does not currently support participant onboarding, the protocol to issue VCs. According to the information in the EDC’s Discord channel, planning and designing the implementation should begin during the second half of 2024. Therefore, in the PoC, VCs are created using other tools and stored manually in the Identity Hub.

Using VCs in data exchange

The PoC aims to use VCs and VPs in data exchange to authenticate the message exchange parties and authorise access to services. The MVD provides a sample implementation, but it's currently based on older EDC and Gaia-X trust framework versions. Therefore, changes were required to support the latest EDC and Gaia-X trust framework versions.

Making the latest EDC version (0.6.2) work with the latest Gaia-X trust framework version has been more challenging than expected. This is because many of the technologies used in the implementation are relatively new to the PoC team and because the aim is to integrate the technologies into X-Road. In addition, we have discovered some bugs that have slowed the progress.

For example, we experienced an issue with VC verification – the EDC failed to verify VCs issued by the Gaia-X Compliance Service. It turned out that a transitive dependency used to normalise JSON-LD documents in the EDC had a bug in the normalisation algorithm that caused the normalisation to produce incorrect results. Consequently, the verification always failed for VCs created using another library. Fortunately, the solution was as simple as bumping the transitive dependency to the latest version, where the problem is already fixed. However, debugging the issue and discovering the root cause was time-consuming.

What’s next?

The main challenge continues to be integrating the Gaia-X trust framework with the EDC-based X-Road 8 Security Server. The integration includes looking into supporting X-Road's current trust framework and Gaia-X trust framework in parallel since backwards compatibility is required.

Even if the main challenge continues to be the same as in March, the PoC has gained a lot of progress in April. Currently, the PoC is not far from utilising Gaia-X credentials for authentication and authorisation during data exchange. However, we're talking about an initial implementation that is not production-ready. In the PoC, the aim is to have an implementation that works, but several things may still be missing, e.g., proper verification and validation of credentials. Once a working implementation is available, the focus will shift to the backward compatibility of the trust framework. Regardless of minor delays, we’re confident that the main goals will be achieved within the project timeline - by the end of May.

More blog posts about X-Road 8 and the proof of concept will follow. Stay tuned!

Making of X-Road 8 – PoC March Status Update

At the beginning of January 2024, NIIS kicked off a proof of concept (PoC) to develop a new major version, X-Road 8 “Spaceship”, which nurtures the proven ecosystem model and security while it takes X-Road to the next level by providing a solid data space infrastructure.

With the proof of concept, NIIS aims to validate the feasibility of replacing X-Road's custom protocol stack with standard data space protocols and align X-Road's trust framework with the Gaia-X trust framework. The project also tries to ensure smooth integration with previous X-Road versions for backwards compatibility, estimate the changes required for information systems when transitioning to X-Road 8, and assess potential changes to existing X-Road components.

In this blog post, we explore the current status of the PoC project.

Support for the standard dataspace protocol

Since the February update, we have continued to work with supporting SOAP services, caching contract negotiation results, defining access permissions using ODRL and message signing and logging.

Caching contract negotiation results

The PoC implementation has been improved by caching and reusing the control plane contract negotiation results. Before, every service request included a complete contract negotiation process that caused several requests back and forth between the data exchange parties. Now, contract negotiation is required only if the service consumer still needs to get a valid access token. In that case, the contract negotiation process is completed, and the consumer receives an access token that can be used for unlimited requests until the token expires. Once the token has expired, the contract has to be renegotiated, and a new access token is generated. Currently, the token is valid for 10 minutes, but the value will be configurable.

Defining access permissions using ODRL

The EDC defines access policy definitions using the Open Digital Rights Language (ODRL). In X-Road, access policies are defined using client identifiers, service codes, HTTP verbs and request paths. An X-Road-specific ODRL profile that extends the ODRL core was implemented to support the current X-Road functionalities. The current PoC implementation supports individual clients and global groups. Instead, local groups still need to be supported. Also, the access rights are maintained in the old Security Server configuration database and periodically exported to ODRL. However, the current implementation already demonstrates that ODRL can support X-Road's access policy definitions and that migration from the current storage format to ODRL can be automated.

Supporting SOAP services

After the latest changes, the PoC implementation supports X-Road's current REST and SOAP interfaces. However, not all the details have been implemented in the PoC, e.g., support for the request hash response header. Nevertheless, existing REST and SOAP service consumer and provider information systems can be connected to the PoC without any additional changes. In other words, it seems that it’s possible to keep the REST and SOAP interfaces fully backwards compatible.

Support for TLS

Initially, connections between two X-Road 8 Security Servers used plain text HTTP connection. The implementation has been updated to use TLS with some of the features already used in X-Road 7.

A provider Security Server must have an authentication certificate issued by an approved Certificate Authority (CA) and registered on the Central Server. A consumer Security Server verifies the certificate and checks its association with the provider Security Server in the global configuration. Instead, in the PoC implementation, the provider Security Server doesn’t verify the consumer Security Server’s authentication certificate. In other words, the PoC implementation doesn’t support mTLS between X-Road 8 Security Servers even though it’s enforced between X-Road 7 Security Servers.

How trust in TLS certificates is currently implemented in X-Road is very X-Road-specific. The issuer of a TLS certificate must be defined as trusted by the X-Road operator, the issuer must be included in the global configuration's trusted list, the TLS certificate must be registered on the Central Server and associated with the Security Server that's using it, the association must be included in the global configuration, and the certificate must have a valid OCSP response. The current implementation adds an extra layer of trust and security, but the downside is that it's not interoperable with other data space technologies. The available data space specifications do not say anything about TLS certificates. The default assumption is that a commonly trusted CA should issue them, but data space-specific exceptions are possible.

Nevertheless, in X-Road 8, it's necessary to reconsider the X-Road-specific additional layers. At least they should be limited on the data plane level so that the trust and control planes remain interoperable with other data space technologies. Otherwise, X-Road is not interoperable with other data space technologies, even if they're based on the same standards and specifications.

Message signing

Another X-Road-specific feature is the signing, logging and timestamping of messages. In March, support for message signatures was implemented in the PoC. The implementation uses the Digital Signature Services (DSS) Java library provided by the European Commission. In practice, signatures and sign certificate OCSP responses are passed in HTTP headers, meaning a separate transfer or wrapper message between the Security Servers is no longer required. The implementation uses the JAdES-B-LT signature, while currently, X-Road 7 uses XADeS signatures. However, the PoC implementation has some limitations compared to the signatures in X-Road 7. In the PoC, the message body and most of the HTTP headers are signed, while in X-Road 7, the signature covers HTTP verb, request path and URL parameters too. Also, batch signing is not supported, and therefore, each message is signed separately.

Message logging

The PoC implementation doesn't support message logging yet because the current timestamping implementation is incompatible with JAdES signatures. However, adding support for timestamping is possible, but it just requires additional work that may not be included in this PoC.

Before proceeding with the implementation, a more thorough analysis of different logging options is needed. For example, the International Data Space Reference Architecture Model version 4 (IDS-RAM 4) by the International Data Spaces Association (IDSA) defines a different approach to logging. According to IDS-RAM 4, the IDS Clearing House logs information relevant to clearing, billing, and usage control. The Clearing House is an intermediary that provides these services to all data space participants. In X-Road, the Clearing House would be the Central Server or a separate component operated by the X-Road Operator.

The data logged by the Clearing House should not include a complete message with payload but only message hashes, signatures and other non-sensitive information needed to identify the data exchange parties and verify the data's integrity. Instead, it's the responsibility of the data exchange participants to store the entire message and other related information required to verify the hashes and signatures. However, a common standard or technical specification defining implementation details for capabilities provided by the Clearing House does not exist.

Instead, in eDelivery, the evidence of data exchange is implemented using AS4 Receipt – an AS4 message that provides non-repudiation of origin and receipt at the level of message exchange. However, the AS4 Receipt doesn't include sensitive data, e.g., the message payload. Instead, the data must be stored separately by the data exchange parties.

X-Road 8 is expected to provide a solid data space infrastructure, so the Clearing House approach to logging should be followed. Since a common standard or technical specification defining the implementation details is currently missing, there's a lot of room for interpretation. From the X-Road members' point of view, the main difference to the current message log implementation is that the message log currently stores all the details required to resolve a dispute. In contrast, the Clearing House approach requires storing part of the evidence separately in an external system.

However, the current message log implementation and the Clearing House approach aren't mutually exclusive. X-Road could follow the Clearing House approach and continue to support the current logging method. In other words, logging could be configurable so that X-Road members can decide which logging alternative to use based on the data exchange use case - or both simultaneously. Service usage policies could be used to set requirements on the logging level that must be used when accessing a specific service. In this way, it would be up to the service provider to define the required logging level for both data exchange parties.

However, logging-related questions are more than just technical since multiple legal and organisational aspects define requirements for the technical implementation. Therefore, the topic needs to be studied more detail to understand the relevant constraints better.

Support for the Gaia-X trust framework

We have set up an internal instance of the Gaia-X Digital Clearing House (GXDCH) services to have more flexibility. The instance includes Gaia-X Registry, Gaia-X Compliance, Gaia-X Notary and Gaia-X Wizard. Also, we have configured an instance of the X-Road Test CA as a trust anchor, enabling us to issue our own certificates trusted by the GXDCH services. Now, we can issue Gaia-X credentials that we can use for testing purposes.

What’s next?

Next, the main challenge is integrating the Gaia-X trust framework with the EDC based X-Road 8 Security Server. The integration includes looking into supporting X-Road's current trust framework and Gaia-X trust framework in parallel since backwards compatibility is required.

In practice, the next step is to set up the EDC Minimum Viable Dataspace (MVD) example that uses credentials issued by the latest version of the GXDCH services. It includes using the EDC Identity Hub component to store Gaia-X credentials and using them in contract negotiation and defining access control policies. Once the MVD setup has been successfully completed, the same changes can be applied to the X-Road 8 Security Server. Finally, the Gaia-X trust framework must be connected with the EDC control and data planes so that all the planes work seamlessly together.

More blog posts about X-Road 8 and the proof of concept will follow. Stay tuned!

Making of X-Road 8 – PoC February Status Update

At the beginning of January 2024, NIIS kicked off a proof of concept (PoC) to develop a new major version, X-Road 8 “Spaceship”, which nurtures the proven ecosystem model and security while it takes X-Road to the next level by providing a solid data space infrastructure.

With the proof of concept, NIIS aims to validate the feasibility of replacing X-Road’s custom protocol stack with standard data space protocols and align X-Road’s trust framework with the Gaia-X trust framework. The project also tries to ensure smooth integration with previous X-Road versions for backwards compatibility, estimate the changes required for information systems when transitioning to X-Road 8, and assess potential changes to existing X-Road components.

In this blog post, we explore the current status of the PoC project.

Getting started

The PoC was kicked off at the beginning of January with an onboarding sprint whose aim was to look into existing data space initiatives and available open-source components and building blocks. Since the main challenge is to replace X-Road’s custom protocol stack with the data space protocol stack, we decided to start with integrating the standard dataspace protocol into X-Road and concentrate on the trust framework later.

First, we investigated available data space connectors and decided to focus on the Eclipse Dataspace Components (EDC) and the Tractus-X ConnectorKit. We were not looking for an off-the-shelf connector product but a customisable open-source framework that can be extended with additional features. Also, our focus was on Java-based building blocks since that’s the technology we’re the most familiar with. The EDC met these requirements well, while the Tractus-X ConnectorKit provided practical examples of implementing extensions on top of the EDC. Therefore, we decided to go with the EDC.

Let’s get working!

We decided to use the X-Road 7 Security Server as a starting point for the PoC and extend its current functionality by integrating the EDC into it. In this way, supporting existing functionalities together with the standard dataspace protocol is easier. The aim is to keep the interface between information systems and the Security Server as-is so that no changes to information systems are required. The interface between information systems and the Security Server is defined by the X-Road Message Protocol for REST and X-Road Message Protocol for SOAP.

In practice, the very first iteration of the X-Road 8 Security Server supports the standard dataspace protocol and X-Road's custom protocol stack. When two X-Road 8 Security Servers communicate, the standard dataspace protocol is used. Instead, when X-Road 8 and X-Road 7 Security Servers communicate, X-Road's custom protocol stack is used. Either way, information systems that exchange data over X-Road don’t know which protocol is used between Security Servers since the protocol between information systems and the Security Server has stayed the same.

Information about the protocols supported by a Security Server was added to the global configuration to enable the selection of the communication protocol between Security Servers. That way, the Security Server knows what protocols other Security Servers support. Therefore, the consumer Security Server can select a protocol the target Security Server supports. Whether the global configuration in its current form will be used in X-Road 8 remains to be seen. Still, a registry containing information about member organisations and Security Servers will be required. In the data space architecture, the component is called the Dataspace Registry.

The control plane and data plane

The standard dataspace protocol is divided into two different layers:

• The control plane manages contract negotiation and controls the data transfer process. It is generic and sharable between different data spaces. The standard data space protocol defines it.

• The data plane performs the actual data transfer. It reuses existing protocols and technologies and doesn't require defining a new data transfer protocol. Also, the data plane is typically use-case and/or data space-specific.

Since the control plane is standardised, the EDC provides the control plane implementation out of the box. However, the standard dataspace protocol specification is still a work in progress and has yet to reach a stable version. Nevertheless, we’re not going to touch the control plane implementation during the PoC but instead rely on the updates provided by the EDC. This approach enables the PoC to concentrate on implementing an X-Road-specific data plane.

X-Road data plane

The control plane discovery, contract negotiation and data transfer management processes are asynchronous and require multiple requests/responses between the data exchange parties. In the PoC, we have chosen to hide this complexity from the connected information systems, which makes it possible to keep the interface between them and the Security Server backwards compatible. Despite the contract negotiation process, a client information system needs to send a single request to the Security Server, and it will receive the requested data as a response. Diagram 1 provides a simplified version (despite the complexity, it's a streamlined version and doesn't include all the steps) of a data exchange process (e.g., a synchronous REST API call) using this approach.

Diagram 1. The first version of the X-Road data plane. Open the diagram in a new tab.

So far, we have implemented the data exchange flow illustrated in Diagram 1 in a backwards-compatible way. However, the first version of the X-Road 8 data plane implementation still needs many critical features required for a production-level implementation. Despite a good start, it's still too early to say whether a fully backwards-compatible implementation is feasible. For example, the following features are missing:

• Caching and reuse of contract negotiation results.

• Define access permissions using the Open Digital Rights Language (ODRL).

• Support for SOAP services.

• Signing of messages.

• Logging and timestamping.

• Authentication and authorisation.

• Support for operational and environmental monitoring.

• Support for federation. 

Light context

Also, we aim to look into providing support for the so-called "light context", which refers to a setup where the Security Server is required only on the service provider side. In other words, a service consumer could invoke services by connecting to the provider's Security Server directly without having a Security Server on the consumer side. Diagram 2 provides a simplified version of the scenario.

Diagram 2. Light context vs. normal request flow.

The idea is that the control plane phase would generate an API key or access token that could be utilised by a consumer information system operating in the light context. The API key or access token could be obtained from the service catalogue separately in advance so that the consumer information system could completely ignore the contract negotiation phase during a data transaction. This would make consuming X-Road services easier compared to the current situation being aligned with the standard dataspace protocol at the same time. The API key generation in the light context is ignored in Diagram 2. However, the light context requires compromises regarding the security guarantees X-Road provides.

What about the Gaia-X trust framework?

One goal of the PoC is to align X-Road’s trust framework with the Gaia-X trust framework. For now, we have used X-Road’s existing trust framework in the PoC since it enabled us to get started faster with the standard dataspace protocol-related changes. However, it doesn’t mean the Gaia-X trust framework has been forgotten.

So far, we have set up our instance of the Gaia-X Digital Clearing House (GXDCH) services. For clarification, our instance is for our internal testing and development purposes only – NIIS is not planning to become an official Gaia-X Digital Clearing House. The reason for setting up our instance instead of using the publicly available GXDCH instances is that it enables us to experiment more freely. For example, we are free to define our own trust anchors. Also, we have set up our instance of the Gaia-X Wizard.

The next step is building technical compatibility and interoperability with the Gaia-X trust framework. It means that X-Road should be aligned with the technical specifications and meet the technical requirements of the Gaia-X trust framework. This is an enormous change for X-Road since the current trust framework is based on X.509 certificates. In contrast, the Gaia-X trust framework is based on the Self-Sovereign Identity (SSI) principles and technologies like W3C Verified Credentials, Linked Data (RDF) and Decentralised Identifiers (DID).

Support for the Gaia-X trust framework can be achieved in different ways. For example, X-Road could continue to use its current trust framework and support the Gaia-X trust framework. Alternatively, X-Road's trust framework could be based on Gaia-X's technical specifications, making it technically interoperable with Gaia-X by default. A final decision hasn’t been taken yet, but at this point, making X-Road’s trust framework aligned with Gaia-X’s technical specifications is the more likely option. That’s the approach that will be investigated in the PoC.

However, making X-Road’s trust framework technically compatible and interoperable with the Gaia-X trust framework doesn’t automatically make all X-Road ecosystems and user organisations Gaia-X compliant. Instead, it makes X-Road user organisations technically compatible with Gaia-X, but being Gaia-X compliant requires that the user organisations meet the requirements of the Gaia-X Compliance rules. In other words, using X-Road 8 doesn't automatically make an organisation a Gaia-X participant. Instead, X-Road 8 will provide the technical means to become a Gaia-X participant when an organisation meets the requirements of the Gaia-X Compliance rules.

What’s the relationship between X-Road 7 and X-Road 8?

The decision to use the X-Road 7 Security Server as a basis for the X-Road 8 PoC doesn’t mean that X-Road 8 will be based on X-Road 7. It might be that some X-Road 7 components will initially be included in X-Road 8 to make the upgrade from X-Road 7 to X-Road 8 smoother for existing users. Once the upgrade has been completed, the X-Road 7 components can be dropped from X-Road 8.

I also want to highlight that new X-Road ecosystems that will be set up using X-Road 8 don't have to deal with X-Road 7. If backwards compatibility with X-Road 7 is not required, X-Road 8 can be deployed without X-Road 7-related dependencies.

What’s next?

Currently, the PoC is divided into two work streams:

• Support for the Standard Dataspace Protocol.

• Support for Gaia-X trust framework.

Support for the Standard Dataspace Protocol work stream continues to work on the X-Road data plane and extend its functionalities. Next, we will look into supporting SOAP services, caching contract negotiation results, defining access permissions using ODRL and message signing and logging.  

Instead, support for the Gaia-X trust framework work stream continues experimenting with the GXDCH services and the Gaia-X Wizard. Next, we plan to utilise existing sign keys and certificates of X-Road member organisations to create and sign Gaia-X Verifiable Credentials and Verifiable Presentations.

More blog posts about X-Road 8 and the proof of concept will follow. Stay tuned!

Unveiling the Open Data Product Specification (ODPS): A Holistic Approach to Data Products

In the ever-evolving landscape of data management, the concept of data products has gained prominence, offering a standardized approach to packaging and consuming relevant data resources. At the forefront of this movement is the Open Data Product Specification (ODPS), a vendor-neutral, open-source, and machine-readable data product metadata model. Let's delve into the unique features that set ODPS apart in the realm of data product standards.

Technical Emphasis and Current Version. ODPS places a strong emphasis on technical aspects, defining objects and attributes related to data products such as data pipelines, access, data models, and deployment. The current latest production version, 2.1, reflects continuous refinement, with the next release scheduled for early 2024. Naturally, ODPS has a governance model which provides needed structure and guidance for long-term support. 

Holistic Metadata Model: ODPS distinguishes itself by offering a full-stack metadata model that includes attributes for pricing plans, built-in support for internationalization (i18n), contract elements, Service Level Agreements (SLA), DataOps, licensing, and provider details. This comprehensive approach sets ODPS apart from other standards and enables the complete decoupling of data products from vendor systems, fostering decentralization.

Business-Focused Standards: The shift towards data product-focused standards signifies the growing need for a refreshed approach to treating data as a business-related commodity rather than a purely technical asset. ODPS addresses this need by providing an easy-to-adopt, developer-friendly standard with a distinct business focus, challenging the limitations faced by alternatives like DCAT.

Community Recognition: In Europe, the Data Spaces community acknowledges the importance of standards in creating data products and product offerings. Three standard templates stand out: DCAT, ODPS, and the TM Forum product model. ODPS, with its unique attributes, has gained recognition as a key player in this space.

Comparison with Alternatives: While ODPS differs significantly from the popular DCAT (Data Catalog Vocabulary), it's important to note that various standards, such as Data Product Descriptor Specification and Data Product Specification, share some similarities. However, ODPS stands out due to its holistic approach and inclusion of critical business-related attributes.

ODPS vs. DCAT: Understanding the Differences: A common question arises regarding the differences between ODPS and DCAT. A brief comparison reveals key distinctions:

• ODPS has a flat structure, while DCAT follows a linked structure.

• ODPS focuses solely on a single data product, whereas DCAT includes cataloging features.

• ODPS is lightweight and fast to understand, in contrast to the exhaustive nature of DCAT.

• Both share numerous attributes, but ODPS is tailored for data marketplace and sales, including data exchange.

• DCAT, maintained by W3C, leans towards cataloging and sharing, while ODPS is community-maintained and oriented towards data marketplace dynamics.

The value of ODPS in the eyes of new emerging companies is big. They consider ODPS to provide same value as DCAT, but with faster adoption as we can see from the testimonial of one architect responsible for a trusted data exchange platform: “For me as a software architect/developer the ODPS standard is a lot more approachable. I think the usage of JSON is also a lot more flexible than the RDF structure proposed by DCAT. Furthermore, the current document structure of ODPS is more clear and easier to understand with almost the same amount of properties.”

Extensibility with OpenAPI Specification: ODPS offers flexibility by easily integrating with the OpenAPI Specification (OAS), including business, legal, and SLA attributes that may be missing from the standard package. A proof-of-concept test conducted in December 2023 showcased the seamless extension of ODPS in the API description of X-Road.

Service Discovery in X-Road

When services are connected to X-Road, their technical service descriptions are published on the Security Server by the Security Server administrator. The supported formats are Open API Specification (OAS) for REST services and WSDL for SOAP services. The service descriptions can then be accessed using a service discovery mechanism provided by X-Road. However, the mechanism is quite technical and requires direct access to the Security Server's messaging interface. Also, getting a list of all services available in the ecosystem would require querying each Security Server separately. Therefore, a more user-friendly service catalogue is needed.

A service catalogue is a web portal that contains descriptions of all the services available in the ecosystem. The primary purpose of the service catalogue is to provide a user-friendly channel to search and discover available services.

Image 1. Collecting service descriptions from Security Servers to Service Catalogue.

When implementing the service catalogue, collecting the service descriptions from the Security Servers can be automated. In that way, the descriptions need to be maintained in a single place only, and all the changes in the source are automatically updated to the catalogue. Nevertheless, additional organisation and data product metadata must be manually added and maintained on the catalogue by an administrator representing the organisation owning the service. The metadata may include any information related to the service and its use, e.g., a more detailed description of the data set, terms and conditions, contact information, pricing information, SLAs, etc.

Using ODPS in X-Road

 Instead of maintaining the data product related metadata manually on the service catalogue, the metadata can be published in a machine readable format on the Security Server using ODPS. In practice, the OAS service descriptions can be extended with ODPS by including the selected parts of ODPS in them.

Image 2. Including ODS attributes in an OAS service description on the Security Server.

Starting to use the ODPS extension in the X-Road OAS service descriptions doesn’t require any code or configuration changes to the Security Server. It’s enough to update the OAS service description and the changes are immediately available over X-Road’s service discovery mechanism. This makes them accessible to all the service consumers and the X-Road Catalog extension (and other metadata harvesters) that’s used to centrally harvest service descriptions from all the service providers within an X-Road ecosystem. The approach was validated in a small PoC implemented in December 2023.

Image 3. An OAS service description with ODPS attributes. A full example is available here.

The attributes provided by ODPS can be utilised by the service catalogue which reduces the number of metadata that the X-Road Member Administrator must add to the catalogue manually. Using ODPS makes the information available in the service catalogue more unified, improves its quality, makes it machine readable, and eases its maintenance. However, utilising ODPS attributes in service catalogues provided by X-Road Operators may require changes to the catalogues so that they’re able to process and visualize the attributes. Nevertheless, at least some of the catalogues support visualizing and downloading raw OAS service descriptions which makes ODPS available to the catalogue users without additional changes. Since the Security Server already supports adding ODPS attributes to the OAS service descriptions, service providers may start utilising the extension immediately.

Conclusions

In conclusion, the Open Data Product Specification emerges as a unique and impactful player in the data product standards landscape, providing a holistic model that caters to the evolving needs of the data economy. Its focus on business-related attributes, coupled with technical precision, positions ODPS as a valuable asset for organizations navigating the complexities of data management in the modern era.

Extending X-Road OAS service descriptions with ODPS makes data product-related metadata available in the service catalogue more unified, improves its quality and enables more automation in the maintenance process. Service providers may start to use ODPS right away by updating their OAS service descriptions. Depending on the service catalogue solution, the ODPS attributes may be accessible in the catalogue without any additional changes to the catalogue software. However, visualising the ODPS-related information in a more user friendly way probably requires some changes. Overall, implementing ODPS is a low hanging fruit that improves the quality of the service metadata in X-Road ecosystems.

---

Jarkko Moilanen is the founder and strategy group chair of the Open Data Product Initiative.

Jarkko is an experienced senior data management professional with strong expertise on data monetization, platform and API economy. Currently Jarkko spearheads the Open Data Products track within the Data Enablement Program's data exchange platform under the aegis of the Abu Dhabi government. Jarkko also had a role in early phases of taking X-Road into national use in Finland and introducing REST APIs to the platform.

Open Data Product Specification (ODPS) which has matured into governed standard was ignited 2019 by Jarkko. ODPS was initially a sidetrack initiated as part of Data Product Toolkit development, but later was discovered valuable enough to be independent project as it gained traction from companies around the world.

On the international front, Jarkko was the first MIT CDOIQ Country CDO Ambassador for Finland for a few years. On top of practicing the data value creation, Jarkko has also written several business focused books on AI ("AI-Powered Data Products", 2023), API economy ("API Economy 101", 2018) and Data Economy (Deliver Value in the Data Economy, 2022).

NIIS Announces Proof of Concept for Revolutionary X-Road 8 “Spaceship” 🚀

NIIS is thrilled to announce the start of a proof of concept to develop a new major version, X-Road 8 “Spaceship”, which nurtures the proven ecosystem model and security while it takes X-Road to the next level by providing a solid data space infrastructure.

With the proof of concept, NIIS aims to validate the feasibility of replacing X-Road’s custom protocol stack with standard data space protocols and align X-Road’s trust framework with the Gaia-X trust framework.

In this blog post, we explore the X-Road 8 “Spaceship” plans and the scope of the proof of concept. Are you ready for the launch?

The current state of X-Road

X-Road provides a secure and unified way to exchange data between X-Road member organisations. X-Road enables data exchange within one X-Road ecosystem and between two independent X-Road ecosystems through federation. Members of the federated ecosystems can publish and consume services with each other as if they were members of the same ecosystem.

The interoperability within one X-Road ecosystem and between two independent X-Road ecosystems is based on the X-Road protocols. X-Road defines and implements a set of protocols that guarantee the interoperability and security of the data exchange. Multiple standards are used in X-Road's implementation, but no X-Road protocols have been standardised.

X-Road’s source code and all X-Road protocols are open, and documentation is publicly available, so anyone is free to implement X-Road’s protocol stack. However, no other open-source products implement X-Road’s protocol specifications. Instead, one commercial data exchange solution implements X-Road protocols to some extent, but estimating how compatible the implementation is with X-Road isn't easy.

X-Road is being developed as a product, and the protocol specifications are adapted according to the needs coming through the product development. Also, the specifications include some product-specific details like component version numbers. Therefore, developing and maintaining another product implementing X-Road’s protocol specifications would be challenging since the specifications include product-specific details. In practice, it’s unlikely that alternative implementations of X-Road’s protocol specifications will be seen, and the specifications will remain specific to the X-Road product.

Image 1. The current state of X-Road.

eDelivery for cross-border data exchange in Europe

eDelivery is a building block of the European Commission and is the default technology for cross-border data exchange in Europe. eDelivery is based on the AS4 specification, and multiple open-source and commercial solutions that conform to the specification are available. Therefore, users of eDelivery are free to choose any of the available solutions; alternatively, they can also decide to implement their own. Either way, interoperability between different solutions is based on the common AS4 specifications. The European Commission is coordinating the development of the specifications independently from any of the implementing products.

Since X-Road and eDelivery are based on different specifications, they’re not directly compatible with each other. Making X-Road and eDelivery compatible requires implementing a custom gateway component responsible for conversions between X-Road and eDelivery protocols. However, implementing one common gateway is not feasible because of the differences between various eDelivery ecosystems. Therefore, multiple gateway implementations are required. Even though eDelivery is based on the common AS4 specifications, the specifications leave room for ecosystem-specific differences in the implementation.

Data spaces enable data sharing and exchange within and across data ecosystems

Data space is a distributed system defined by a governance framework that enables secure and trustworthy data transactions between participants while supporting trust and data sovereignty. Data spaces enable data sharing and exchange within and across different data ecosystems. Just like eDelivery, data spaces are based on a set of common specifications and standards. Interoperability within and between data spaces requires following the same specifications and standards. Participants of different data spaces can choose the software products they use as long as the products implement the required standards.

Multiple organisations are developing specifications and standards for data spaces, e.g., the International Data Spaces Association (IDSA), Gaia-X, Fiware, the Data Spaces Support Centre (DSSC) and the Data Spaces Business Alliance (DSBA). The organisations work together to keep the specifications and standards aligned and not create multiple overlapping or competing specifications.

Currently, connecting X-Road to a data space requires implementing a custom gateway component responsible for conversions between X-Road and data space protocols. The same approach applies to eDelivery and other data exchange ecosystems, too.

The target state with X-Road

Implementing custom gateway components that enable X-Road to communicate with other data exchange ecosystems is not a feasible solution in the long term. The number of gateway components that need to be developed and maintained grows over time, and achieving full compatibility through them is highly challenging. Therefore, a better approach for X-Road is to move from X-Road-specific protocols to the data space protocol stack.

In practice, the protocols used in data spaces can be divided into three different layers:

• Trust plane manages participant identities and authentication. It can be sharable between different data spaces or data space-specific.

• Control plane manages contract negotiation and controls the data transfer process. It is generic and sharable between different data spaces. It’s defined by the standard data space protocol.

• Data plane performs the actual data transfer. It reuses existing protocols and technologies, and therefore, it doesn’t require defining a new data transfer protocol. Also, the data plane is typically use-case and/or data space specific.

Replacing the current X-Road-specific protocols with the data space protocol stack would mean that the same protocols are used within one X-Road ecosystem, between two federated X-Road ecosystems and between X-Road and other data exchange ecosystems. That way, the same set of protocols is used for internal and external data exchange.

Image 2. Target state with X-Road.

However, being interoperable with another data exchange ecosystem requires that the other ecosystem implements the same data space protocols, too. At least right now, some data exchange ecosystems call themselves data spaces even if they’re not using the data space protocol stack. Also, not all the data exchange ecosystems will start using the stack in the future either. In other words, the change doesn’t guarantee full interoperability with all different data exchange ecosystems. Still, at least it improves the current situation significantly and increases the level of standardisation in X-Road’s implementation.

The data space protocol stack covers various aspects of data exchange, e.g., usage conditions and policies, contract negotiation, and data transfer process. They do not set restrictions to the actual data transfer protocol or wire protocol – the protocol that is used to transfer the data. In other words, any existing transfer protocol can be used, which enables the use of existing solutions and supports many different transfer protocols. This is a big difference compared to X-Road and eDelivery, which both have their own transfer protocol.

Another difference is related to expressing and enforcing usage conditions and policies. In data spaces, they’re expressed in human-readable and machine-readable ways, which can be enforced in organisational or technical manners. Instead, in X-Road, they’re expressed in a human-readable way, which can be enforced only in an organisational manner.

Trust framework is another crucial factor in interoperability within and between data spaces. Different data space initiatives have their own trust frameworks, but the initiatives are currently aligning their trust frameworks to avoid those becoming a blocker for interoperability between various data space initiatives.

Many data space initiatives have trust frameworks based on technologies different from X-Road’s trust framework, e.g., Gaia-X. X-Road's current trust framework is compatible with the Gaia-X trust framework on some level. However, in practice, many additional steps are required for an X-Road member to become a member of a Gaia-X-compliant data space utilising the X-Road trust framework. Therefore, replacing X-Road’s trust framework with the trust framework of a selected data space initiative is required to maximise interoperability.

Image 3. The data space protocol stack.

Implementing the change doesn’t mean having to build everything from scratch. Several well-established and conformant open-source solutions are already available that can be taken as a basis for the implementation, e.g., Eclipse Dataspace Components (EDC). Many of the solutions support or are going to support the data space protocol stack, which means that it would be enough to implement changes that enable current X-Road users an easy migration path to the new version. The goal would be minimising the changes the current X-Road users must implement.

NIIS is currently responsible for developing and maintaining the X-Road core and extensions. Despite the worldwide X-Road community, external contributions have remained low. Migrating to the data space protocol stack by taking an existing open-source solution as a basis would change the current situation.

It is fair to assume that the chosen open-source solution already has a well-established developer community contributing to the core solution's maintenance and development. For example, the EDC project is hosted and facilitated by the Eclipse Foundation AISBL. The foundation is a Europe-based not-for-profit organisation that acts as a steward of the Eclipse open-source software development community. When NIIS isn’t alone responsible for the core development and maintenance, it enables NIIS to concentrate on areas that bring the most value to the NIIS members.

The downside of the approach is that NIIS would lose some control over the direction of X-Road’s future development because various data space initiatives lead the development of different layers of the data space protocol stack. NIIS could contribute to the development and suggest changes as needed, but it wouldn’t have the power to make decisions alone as it has now. However, it is unlikely that NIIS members’ needs would differ significantly from the needs of others. Also, the development of the X-Road core would be partly in the hands of another open-source project, and NIIS would be one of the contributors instead of being the project owner.

Data Space Infrastructure for X-Road

Implementing X-Road on top of a well-known open-source project conforming to the data space protocol stack would make X-Road interoperable with other data exchange ecosystems using the same protocols. However, connecting X-Road to data exchange ecosystems following other specifications would still require custom gateway solutions. Connecting X-Road to any other data exchange ecosystem today requires a custom gateway solution, so being interoperable with many other ecosystems would significantly improve X-Road.

Image 4. Data space infrastructure for X-Road.

Another benefit of the new approach is more flexibility in supporting different data transfer protocols. Currently, adding support for new data transfer protocols requires a lot of effort and is challenging to implement in a backwards-compatible manner. Instead, the standard data space protocols enable using various data transfer protocols which allows X-Road to support new transfer protocols more easily, e.g., data streaming, asynchronous data exchange.

Choosing the right trust framework for X-Road is important. To maximise the benefits, the best alternative is to utilise and extend the trust framework of a selected data space initiative, e.g., Gaia-X. In that way, there's no need to maintain a fully X-Road-specific trust framework and make it interoperable with the trust frameworks of different data space initiatives. Also, available data space connectors that can be used as a basis for the X-Road implementation are designed to support different trust frameworks.

Overall, the new approach makes it easier for X-Road users to exchange data with other data exchange ecosystems. Also, it helps to keep X-Road aligned with European data exchange initiatives and policies and relevant for many years to come. Most likely, it would also encourage other countries to select X-Road as a solution compatible with the European infrastructure.

The X-Road 8 Proof of Concept in 2024

The changes to X-Road presented in this blog post are significant and the plan is to implement them in the next major version, X-Road 8 “Spaceship”. The very first step is to implement a proof of concept that is divided into two phases taking place in 2024.

The first phase takes place in H1/2024 and its aim is to validate the feasibility of the selected approach – replacing X-Road's custom protocol stack with the data space protocol stack. The project also tries to ensure smooth integration with previous X-Road versions for backwards compatibility, estimate the changes required for information systems when transitioning to X-Road 8, and assess potential changes to existing X-Road components.

The proof of concept results are expected for review in May 2024. The second half of the year will witness another project to test the results within Estonia’s X-Road ecosystem.

More blog posts about X-Road 8 and the proof of concept will follow. Stay tuned!