There’s No Distributed Ledger Technology (DLT) in X-Road

In 2018 I wrote a blog post about blockchain and X-Road. The aim of the post was to explain that there's no blockchain technology in X-Road, unlike multiple writings had been claiming. Despite the facts provided in the post, still today, X-Road is often called a blockchain or distributed ledger technology (DLT) based solution. X-Road has experienced several major changes since 2018, but its relationship to blockchain hasn’t changed – there’s still no blockchain technology in X-Road.

My earlier blog post already explained the difference between X-Road and blockchain. But what is DLT, and how does it differ from blockchain? And most importantly, what’s the relationship between DLT and X-Road? Let’s find out!

Distributed ledgers

Essentially, DLT or a distributed ledger is a distributed database spread across multiple nodes that all hold an identical copy of the data stored in the ledger. The data is replicated, shared, and synchronized across all the nodes, and the nodes update themselves independently. When the ledger is updated, the nodes use a consensus algorithm to agree on which copy is correct. Once the nodes reach a consensus, they all update themselves with the new copy. In other words, the consensus algorithm enables the nodes to agree on the state of the ledger.

Typically, distributed ledgers are decentralized since they are based on a distributed architecture and don't have centralized governance. However, there are distributed ledgers that have some level of central administration. Therefore, it can be argued not all the distributed ledgers are entirely decentralized.

A distributed ledger may be permissionless, permissioned, or hybrid. Permissionless ledgers are public, and everyone can participate in the network and access the data stored in the ledger. Instead, permissioned ledgers are private, and they only allow trusted nodes to join the network. In private ledgers, the information is available to a specific group of permitted participants, and a network operator decides who is allowed to join the network. In addition, there are also hybrid ledgers that are a combination of permissionless and permissioned networks, e.g., anyone can read data from the ledger, but only authorized nodes are allowed to write data to the ledger.

Distributed ledgers typically have several properties, including but not limited to immutability, transparency, and trust. All the transactions in the ledger are immutable since once a transaction exists in the ledger, nobody can change it. Distributed ledgers are transparent because all the nodes have the same copy of the data. Also, distributed ledgers enable direct trust between parties that don't necessarily trust each other. Immutability and trust are accomplished through cryptography and digital signatures.

There are various types of distributed ledgers that implement the properties differently and have their characteristics. Different kinds of ledgers include blockchain, Directed Acyclic Graph (DAG), Hashgraph, and Holochain.

Sometimes distributed ledger or DLT and blockchain are used as synonyms. However, as we have just seen, there are different types of distributed ledgers, and blockchain is just one type of distributed ledger. Blockchain is a data structure of blocks linked to one another via cryptographic hashes, but distributed ledgers don’t require such a structure or chain.

In general, blockchains are distributed ledgers, but not all distributed ledgers are blockchains. Therefore, distributed ledger and blockchain shouldn’t be used as synonyms.

Is X-Road a distributed ledger?

X-Road and distributed ledgers are both based on decentralized architecture. Distributed ledgers don't typically have central administration, which means that they're entirely decentralized. Instead, X-Road is technically decentralized, but an X-Road ecosystem is always built around a centralized governance model managed by the X-Road operator.

A fundamental difference between X-Road and distributed ledgers is that distributed ledgers are decentralized databases for storing data, while X-Road is a decentralized solution to exchange data. All the distributed ledger nodes hold the same copy of the data, and they agree on the latest version using a consensus algorithm. Instead, in X-Road, data is exchanged between two data exchange parties, and other parties do not have access to it. Also, X-Road nodes do not permanently store data - they only act as secure proxies. All the nodes of an X-Road network have their own sequence of transactions that are not shared with any other nodes – not even in a Security Server cluster. Therefore, there’s no need for a consensus algorithm in X-Road.

Based on the above reasoning, it's evident that X-Road is not a distributed ledger. Comparing X-Road and distributed ledgers is like comparing apples to oranges. However, under the hood, both distributed ledgers and X-Road use the same mechanisms, cryptography and digital signatures, to secure their operations – they’re just applied differently.

Is distributed ledger technology used in X-Road?

Distributed ledger technology is not currently used in any areas of X-Road. The data exchange always happens between two parties, and the data is not shared with other parties of the X-Road ecosystem. Even in a Security Server cluster with an external load balancer where all the nodes share the same identity, the exchanged messages are Security Server specific. Only configuration is shared between the nodes. Consequently, the logs produced by different nodes of the cluster are unique since each message is processed by one node in the cluster only. Also, the logs are stored on the nodes only temporarily - once they’ve been archived, they are removed. Therefore, there aren't currently features that would directly benefit from distributed ledgers when it comes to data exchange. Although, that doesn't mean that some additional features that aren't directly related to data exchange couldn't be implemented using distributed ledgers.

However, some other aspects of X-Road could potentially benefit from distributed ledger technology in the future. For example, how configuration details are managed on the Central Server and distributed to the X-Road ecosystem through global configuration is a potential topic to study further. Also, when thinking about X-Road’s trust model and how organizational identities of X-Road members are managed, self-sovereign identity (SSI) together with decentralized identifiers (DIDs) and verifiable credentials could provide some exciting alternatives in the future. But that’s a topic for another blog post.

All in all, X-Road’s isn’t currently based on distributed ledger technology nor blockchain. When it comes to using distributed ledger technology in X-Road, even if something can be done using distributed ledger technology, it doesn't automatically mean that it should be used. A common principle in the X-Road development is to choose the technology that’s the best fit for the job instead of selecting the most popular alternative.

Protecting Data at Rest in X-Road 7

One of the core features of X-Road is to protect data in transit, and X-Road includes several mechanisms to protect data when it travels between the data exchange parties. Even if X-Road is not a data storage solution, it doesn’t mean that the Security Server wouldn’t store any sensitive data, at least temporarily. Therefore, X-Road 7.0.0 will introduce new security features to protect data at rest on the Security Server. The new features are:

• encrypt backup files (opt-in)

• verify the integrity of backup files before running a restore

• encrypted message payload in message log database (opt-in)

• encrypted message log archives (opt-in)

• group message log archives by member or subsystem (opt-in).

 This blog post gives an overview of the new security features and their key characteristics.

Common design principles

The new data protection features provide the possibility to encrypt and sign specific data stored on the Security Server. By default, encryption is turned off, and the Security Server administrator can manually enable it. All the encryption-related configuration operations require shell access to the Security Server.

The new data protection features have been implemented using the GNU Privacy Guard (GnuPG) – an open-source implementation of the OpenPGP standard as defined by RFC4880. All operations related to backup files and message log archive files have been implemented in an external GnuPG process. Instead, the message log database encryption has been implemented in Java code.

Image 1. Encryption keys that are used to protect data at rest on the Security Server.

The required keys are stored in GPG keyrings, and Security Server's internal key (1) used by default is generated automatically. The Security Server administrators can manually configure additional and member-specific keys (2, 3, 4). All the new encryption keys are illustrated in image 1.

Configuration backup encryption

Security Server configuration backups contain sensitive information, such as database connection credentials, private keys, etc., and therefore, they should be protected from unauthorized access. Starting from version 7.0.0, the Security Server supports signing and encrypting the backup files.

The backups are always signed, but backup encryption is initially disabled. When encryption is enabled, backup files are encrypted when they are created and stored on disk encrypted. It applies to both automatically created and user-created backups. When it comes to user-created backups, the configuration applies to Security Server UI and REST management API.

When backup encryption is enabled, the backups are signed and encrypted using the Security Server’s automatically generated internal key pair (1). In addition, the Security Server administrator can upload additional keys (2) to the Security Server that are used to encrypt the backups together with the internal key. In practice, the backups are encrypted with the internal key and all additional keys. The backups are always signed with the internal key.

During restore, the Security Server verifies the consistency of backup files automatically. However, if the Security Server’s internal key pair is lost, automatic verification is no longer possible. In that case, the Security Server administrator can restore the decrypted backups from the command line with appropriate options that enable skipping integrity verification.

Configuring additional encryption keys is strongly recommended since the Security Server administrator can use them to decrypt configuration backup files in a situation where the internal key pair is lost. Without additional keys, the encrypted configuration backups are unusable in such a situation.

Message log archive grouping and encryption

Message log archive files may contain sensitive information that different organizations can own if multiple members share a single Security Server. Therefore, in version 7, message log archive files can be encrypted and grouped by member or subsystem.

Message log archive encryption and grouping can be configured separately. For example, the archives can be encrypted but not grouped (or vice versa). By default, both features are disabled. Grouping and encryption are enabled/disabled on a Security Server level - they are either enabled or disabled for all the members and subsystems. It's not possible to enable/disable neither of them for selected members nor subsystems only. When encryption and grouping are combined, each member's messages are stored in separate files encrypted with member-specific keys (4). In addition, when message log archive encryption is enabled, the signed document download service returns log records encrypted.

Message grouping can be configured by a member or subsystem. By default, grouping is disabled, and all archive files go to the same default group. Message log archive files that belong to the same group are chained using cryptographic hashes. The implementation is not based on blockchain. However, there may be holes in the chains if grouping is turned on and off or the grouping level (none/member/subsystem) changes. Every time grouping is turned on/off or the grouping level changes, the chains start from zero.

When message log archive encryption is enabled, the archive files are encrypted and signed using the Security Server’s internal key (1) by default. Instead, when archive encryption and grouping are combined, the Security Server administrator must configure one or more member-specific encryption keys (4) for each member. The member-specific keys are used to encrypt the archive files belonging to the member. If the Security Server administrator configures no member-specific key, the Security Server's internal key is used as a fallback.

Also, the format of the message log archives' filenames depends on the configuration of the Security Server. The format varies between encrypted and plain text archives and between different grouping levels. If the archive files are automatically processed based on the file name, the matching rules may have to be adjusted. More information about the filename format is available in the Security Server user guide.

Message log database encryption

Just like the message log archive files, the message log database may contain sensitive information. Starting from version 7.0.0, the message body stored in the message log database can be optionally encrypted, but the default is no encryption. The database encryption is fully transparent to all the external interfaces, e.g., the signed document download service. The Security Server administrator is responsible for configuring the database encryption key (3) before enabling database encryption.

In the message log database, there are two separate columns for plaintext ("message") and encrypted ("ciphermessage") message body. The message body is always stored in one of the two columns depending on the configuration. Instead, the other column that is not used is left empty. When message log database encryption is enabled/disabled, the change doesn't affect already existing records in the database. For example, when message log database encryption is enabled, all the records created after the configuration change will be encrypted and stored in the "ciphermessage" column. Instead, all the records stored before the change will remain in plaintext in the "message" column.

Thanks to the way how the message body encryption in the message log database has been implemented, the change doesn't affect any external integrations. For example, suppose message logs are exported directly from the message log database to some centralized logging system. In that case, the integration doesn't need to be changed if the message log database encryption continues to be disabled.

During message log archiving, the message body is decrypted and written to the archive file in plaintext. Message log archive encryption is configured separately, and when it's enabled, the whole archive file is signed and encrypted. In practice, the message log database encryption may be enabled while the message log archive encryption is disabled (or vice versa). However, if the message log contains sensitive information, it's recommended to enable both the message log database encryption and the message log archive encryption.

Future enhancements

The new data protection features introduced in version 7.0.0 will be developed further in the upcoming releases. For example, one of the potential enhancements is to provide information on the statutes of different encryption configuration options in the Security Server UI to detect the current state and possible configuration errors. Also, making various encryption-related configuration options available through the Security Server UI and REST management API is part of the future development plans.

We’re glad to receive feedback about the new features and contributions regarding further enhancements.

Message Logging in X-Road 7

One of X-Road’s key features is to provide undeniable and tamper-proof evidence of data exchange. X-Road does this by digitally signing, timestamping, and logging all the messages processed by the Security Server. The logs can be used in a court proceeding as evidence. This logging feature is known as the message log. These previous blog posts provide additional information about the message log:

• X-Road Logs Explained - Part 1

• X-Road Logs Explained - Part 2

• X-Road Logs Explained - Part 3

Depending on the data exchange use case, indisputable proof that can be used as legal evidence may be required. On the other hand, regular application logs provide enough information in many cases, and no additional guarantees are needed. In the latter case, X-Road message logs may not be required, or it's enough to log only the metadata. To better support different kinds of use cases and logging requirements, message logging has been made more configurable in X-Road 7. This blog post provides more information about different configuration alternatives available in X-Road 7.

Supported alternatives in X-Road 7

Starting from X-Road 7.0.0, the Security Server supports three options for configuring message logging:

• Full logging

  • The whole message, including both message body and metadata, is logged.

  • The log records can be verified afterward, and they can be used as evidence.

  • Timestamping service is required.

• Metadata logging

  • Only metadata which includes X-Road-specific message headers is logged while the message body is not logged.

  • Verifying the log records afterward is not possible, and they cannot be used as evidence.

  • Timestamping service is required.

• No message logging

  • Message logging is fully disabled. Neither message body nor metadata is logged.

  • No log records are generated.

  • Timestamping service is not required.

Regardless of how logging is configured, messages exchanged between Security Servers are always signed and encrypted. Also, when full logging or metadata logging is enabled, the Security Server produces a signed and timestamped document (Associated Signature Container - ASiC) for each request and response. 

Is it possible to combine different alternatives? 

Full logging and metadata logging can be configured on Security Server and subsystem level. When the Security Server level configuration is used, the same configuration is applied to all the subsystems. Instead, when the subsystem level configuration is used, the configuration is applied to specific subsystems only. In addition, combining the Security Server and subsystem level configurations is also possible, e.g., set metadata logging on the Security Server level and enable full logging for specific subsystems only (or vice versa). Instead, message logging is fully disabled on a Security Server level and it cannot be combined with the other alternatives. Therefore, a subsystem that requires full or metadata logging should not be registered on the Same Security server with a subsystem that requires entirely disabling message logging.

Which alternative should I use?

Multiple factors should be considered when selecting the logging configuration for a data exchange use case. First, there may be legal requirements regarding logging and retention period of the logs. Second,  some logging-related requirements that all the member organisations must follow may have been set by the X-Road operator. Third, there may be some organization-specific requirements for different kinds of use cases. Fourth, the other data exchange party of a specific data exchange use case may have requirements for logging. It's essential to be aware of the different requirements and configure logging accordingly.

To make the decision easier for member organisations, X-Road operators may define a default value for logging-related configuration in a country-specific meta-package. Without any country-specific configuration, the Security Server uses full logging by default. For example, suppose the operator requires using metadata logging by default. In that case, the operator can set the configuration in the country-specific meta-package so that all member organisations use metadata logging by default. The configuration must be updated manually only if some other logging alternative should be used. However, fully disabling message logging is not possible using country-specific meta-packages. Instead, it must be configured separately by the Security Server administrator.

How does the logging configuration affect the Security Server in general?

The logging configuration has some direct and indirect effects on the other features of the Security Server. Fully disabling logging increases the overall performance, reduces disk space consumption, and doesn't require a timestamping service to be configured. Also, moving the message log archives to external long-term storage is not needed since, without the message logging, there are no archive files either. Overall, fully disabling message logging improves the performance and decreases the environmental footprint of the Security Server. If logging is disabled in the whole X-Road ecosystem, running the ecosystem without a timestamping service is possible. However, even if performance and environmental reasons are essential, legal and administrative requirements must not be forgotten.

All the changes related to the message log encryption and archive grouping will be covered in a separate blog post together with other new encryption features.

Configuration Changes in X-Road 7

X-Road 7 will be backward compatible on the interface level with X-Road 6 since there are no changes in the Message Protocol for SOAP and the Message Protocol for REST. Upgrading the Security Server software is enough, and no changes to the connected information systems are required. However, the Service Metadata Protocol for REST and the Security Server REST management API have some changes that might not be fully backward compatible depending on the client application. Also, some Security Server's configuration changes are not backward compatible, and manual actions may be necessary depending on the current configuration. This blog post provides more information about the backward-incompatible changes in the configuration.

A new configuration file for overrides

X-Road 7 will change how some configuration overrides are handled by X-Road system services, e.g., proxy, signer, etc. The changes make the configuration more secure and more consistent. Besides the Security Server, the changes affect the Central Server and Configuration Proxy too.

In version 6, the "/etc/xroad/services/local.conf" file is used to override the environment variables for X-Road system services, e.g., memory allocation for proxy and signer. The problem with the implementation is that the configuration file is an executable that may also contain scripts and code in addition to the configuration overrides. Since the file is executed during restore, it enables code execution as a part of the restore process.

In X-Road 7, the configuration override mechanism will be changed to define the overrides in a non-executable properties file. Starting from X-Road 7.0.0., the properties file "/etc/xroad/services/local.properties" is used to override X-Road's environment variables. The file may contain configuration overrides only to predefined environment variables, and everything else is ignored.

Changes in environment variable names

Also, the format and the names of X-Road's environment variables will be different. Starting from version 7.0.0, all the variables are prefixed with "XROAD_". For example, the difference in customising memory allocation for proxy in X-Road 6 and X-Road 7:

X-Road 6

File "/etc/xroad/services/local.conf":

PROXY_PARAMS="$PROXY_PARAMS -Xms200m -Xmx2000m "

X-Road 7

File "/etc/xroad/services/local.properties":

XROAD_PROXY_PARAMS=-Xms200m -Xmx2000m

Here's a list of the X-Road-related environment variables and their names in different X-Road versions.

X-Road 6	X-Road 7
XROAD_PARAMS	XROAD_PARAMS
SIGNER_PARAMS	XROAD_SIGNER_PARAMS
ADDON_PARAMS	XROAD_ADDON_PARAMS
CONFCLIENT_PARAMS	XROAD_CONFCLIENT_PARAMS
CONFPROXY_PARAMS	XROAD_CONFPROXY_PARAMS
JETTY_PARAMS	XROAD_JETTY_PARAMS
MONITOR_PARAMS	XROAD_MONITOR_PARAMS
OPMON_PARAMS	XROAD_OPMON_PARAMS
PROXY_PARAMS	XROAD_PROXY_PARAMS
PROXY_UI_API_PARAMS	XROAD_PROXY_UI_API_PARAMS
SIGNER_CONSOLE_PARAMS	XROAD_SIGNER_CONSOLE_PARAMS

Other changes that affect configuration

In addition, there are some changes in the backup and restore features that affect the configuration too. Permissions for all the restored files are set to safe, constant values during a restore which means that permissions in the backup file are ignored. Also, the Security Server database backup format has changed, and the new format prevents the execution of arbitrary commands during restore. All the changes related to the Security Server backup encryption will be covered in a separate blog post together with other new encryption features.

What happens when I upgrade to X-Road 7?

Unfortunately, the changes on the Security Server are backward incompatible. It means that backup files generated on X-Road 6 Security Server cannot be restored on X-Road 7 Security Server. However, the changes on the Central Server are backward compatible.

Despite the changes, when upgrading from version 6 to 7, all overrides defined in "/etc/xroad/services/local.conf" will continue to work as before, including the old environmental variable names. However, starting from X-Road 7.0.0, the backups no longer contain the "/etc/xroad/services/local.conf" file, but instead the "/etc/xroad/services/local.properties" file will be included. It means that in case a Central Server or Security Server is restored from a backup, all local configuration overrides defined in "/etc/xroad/services/local.conf" are lost.

Also, in version 7, the backups no longer contain other ".conf" files located in the "/etc/xroad/services" directory. Those files are system configuration files that should never be manually modified since X-Road version upgrades always overwrite them. In case they're lost or corrupted, it's enough to re-install the X-Road packages. However, if the files have been manually modified, all the modifications are lost in case of restore.

It is strongly recommended to migrate all configuration overrides to the new "/etc/xroad/services/local.properties" file when upgrading from version 6 to 7. There's no automation in the migration, and therefore, migrating the configuration overrides must be done manually by the administrator. However, if the "/etc/xroad/services/local.conf" configuration file isn't used in X-Road 6, no actions are required.The migration includes changing the variable names to the new "XROAD_" prefixed format and updating the values according to the new properties format. In case the old "/etc/xroad/services/local.conf" configuration file contains something other than modifications to X-Road-related environmental variables, the changes must be handled in some other way. The new configuration override mechanism doesn't support additional configuration changes.

When both old and new configuration files exist

Also, the new and old configuration files can coexist side by side. When both "local.conf" and "local.properties" files exist, "local.properties" is applied first, and "local.conf" is applied second. However, "local.properties" can only append to the new variables ("XROAD_*"), while "local.conf" can change any parts of the configuration. As a result, values defined in "local.conf" overwrite values defined in "local.properties" if the same property is defined in both files. However, non-overlapping values defined in "local.properties" are still applied. Also, in case both old and new variable names (e.g., "SIGNER_PARAMS", "XROAD_SIGNER_PARAMS") are used in "local.conf", the new variable is applied. For example:

When "/etc/xroad/services/local.properties" contains:

XROAD_SIGNER_PARAMS=-Dprop1=newValue1 -Dprop2=newValue2

And "/etc/xroad/services/local.conf" contains:

SIGNER_PARAMS="$SIGNER_PARAMS -Dlegacy=true"

XROAD_SIGNER_PARAMS="$XROAD_SIGNER_PARAMS -Dprop1=oldValue1   -Dlegacy=false"

The resulting values for different "XROAD_SIGNER_PARAMS" related properties are:

• prop1=oldValue1

• prop2=newValue2

• legacy=false

More detailed instructions for the migration are provided in the X-Road 7 Migration Guide.

Get Ready for X-Road 7

The long-awaited X-Road 7 “Unicorn” is almost here. Building the new version started in October 2020, so it has been rather a marathon than a sprint. The first public beta version of the Security Server was released in December 2020, and it provided the first view on the new major version. The first public release introduced a new visual style, but it didn't offer any functional changes yet. Since the initial release, many things have changed under the hood, and the new visual style has been polished. Now it's an excellent moment to look into the mouth of the Unicorn and have a comprehensive overview of X-Road 7.

X-Road 6 was initially released in 2015, and it has experienced several major changes during its lifecycle, but the core principles have remained the same. Over the years, X-Road 6 has proven to be secure, reliable, and scalable. Therefore, it was decided to use it as a basis for the next major version. Thanks to the solid foundation, there’s no need to reinvent the wheel. The aim is to keep all the good in X-Road 6 and get rid of the bad and the ugly. In other words, X-Road 7 will have all the strengths of X-Road 6 with numerous improvements in various areas. All in all, X-Road 7 is an evolutionary version of X-Road 6.

What happens to X-Road 6 when X-Road 7 is released?

X-Road’s current lifecycle policy is to support the latest version plus two previous versions. The supported versions are defined on MAJOR.MINOR level so the release of patch versions (MAJOR.MINOR.PATCH) does not effect on the support. The supported versions receive patches in case of bugs and vulnerabilities. Also, version upgrades are supported and tested between the supported versions. However, with X-Road 7.0.0, there's one exception – upgrade to version 7.0.0 is supported from version 6.26.0 only. If you're running an older version of X-Road, upgrade to version 6.26.0 is required first before upgrading to version 7.0.0.

Version 6.26.0 is the last planned release for X-Road 6. According to X-Road’s lifecycle policy, it will be supported until version 7.2.0 is released. Currently, the planned release date for version 7.2.0 is November 2022. However, version 6.26.0 or any other older version doesn't stop working when a new X-Road version is released. Instead, older versions continue to work without any immediate effects.

Nevertheless, it’s strongly recommended to use a supported version of X-Road to be safe if bugs or security vulnerabilities are detected. In addition, X-Road member organisations should follow X-Road ecosystem-specific guidelines and policies regarding supported versions set by the X-Road operator. For example, a common practice is that the operator first upgrades the Central Server to the new version, and the member organisations are allowed to upgrade their Security Servers only after that. 

Is X-Road 7 backward compatible with X-Road 6?

The last major version upgrade from X-Road 5 to X-Road 6 required a lot of work from X-Road member organisations because the message protocol between information systems and the Security Server changed. It meant that upgrading the Security Server wasn't enough, but the information systems required changes too. The good news is that X-Road 7 is backward compatible on the interface level with X-Road 6 since there are no changes in the Message Protocol for SOAP and the Message Protocol for REST. Upgrading the Security Server software is enough, and no changes to the connected information systems are required. However, the Service Metadata Protocol for REST and the Security Server REST management API have some changes that might not be fully backward compatible depending on the client application.

However, some changes in the Security Server's configuration are not backward compatible, and manual actions may be required depending on the current configuration. Overall, the estimated effort for the activities is low, and the version upgrade from version 6 to version 7 is more like a version upgrade between two X-Road 6 minor versions, e.g., version upgrade from version 6.23.0 to version 6.24.0.

How is X-Road 7 developed?

X-Road 7 will be implemented iteratively using agile software development methods. All the changes will not be included in the first production version, but they will be introduced one by one over time in various X-Road 7 minor versions.

The development of X-Road 7 is divided into multiple high-level focus areas. Each focus area consists of several topics that will be turned into actual features. The high-level focus areas and their main topics are:

• messaging patterns

• message logs

• onboarding process

• architecture

• operational insights

• sustainability.

More information on the focus areas is available on the X-Road website.

What’s new in X-Road 7.0.0? 

In version 7.0.0, the focus is on the Security Server, and there are only a few minor changes on the Central Server side. The Security Server will have a new visual style that implements the X-Road 7 visual style guide, while the Central Server still has the version 6 visual style. Here’s a summary of changes included in version 7.0.0:

• New X-Road 7 look and feel for the Security Server UI.

• Security improvements on the Security Server:

  • Encrypt backup files (opt-in)

  • Verify integrity of backup files on restore.

• Improvements in Security Server message logging:

  • Encrypt message payload in message log database (opt-in)

  • Encrypt message log archives (opt-in)

  • Group message log archives by member or subsystem (opt-in)

  • Support for fully disabling message logging.

• Change PIN code on the Security Server.

• Return REST API type (OPENAPI3 / REST) and API endpoints in REST metaservice responses.

• Run the Security Server on Java 11 by default.

• Make Security Server more modular by enabling installation without a local Postgres server.

• Version compatibility check for version upgrades - it is no longer possible to update from an unsupported version.

• Official Docker support for the Security Server with the Security Server Sidecar images.

• Other enhancements and bug fixes.

The development focus will shift to the Central Server in 2022 when it gets a new user interface and a management REST API. Also, the new Central Server will include several architectural changes, which will make X-Road more extensible and easier to operate.

What is the release schedule?

Version 7.0.0 will be released in Q4 / 2021. Before the final release version, there will be a public beta version available in October 2021. The beta version will include all the main features available in the final release, and it's targeted at anyone interested in testing the new version in advance. The official release notes that provide detailed information about all the changes included in the release will be published together with the beta. In addition, a separate migration guide that offers detailed information about any manual actions required in the upgrade process will be published.

This writing is the first part in a series of X-Road 7 related blog posts. More writings providing insights on X-Road 7 will be published on the NIIS blog during the following months. Stay tuned!

Reducing the environmental impact of X-Road

Introduction 

Nordic Institute for Interoperability Solutions (NIIS) has set a long-term goal to make X-Road the most sustainable data exchange solution in the world. To better understand the direct and tangible environmental impacts of the use of X-Road software, NIIS commissioned research to assess the current emissions profile across X-Road’s operations and services. In doing so, it delivers targeted recommendations for emissions reduction and sustainable business practices that may be integrated into future decision making.

The study was performed by Gofore and Stockholm Environment Institute Tallinn (SEI) in close collaboration with NIIS and the X-Road Governing Authorities in Estonia and Finland. Together, experts built an emissions calculator to assess the emissions profile of the X-Road instance, subject to the emissions boundary defined in our first blog post. The calculations used in this process have been derived from peer-reviewed literature and tailored to X-Road use cases in Estonia and Finland, which is well described in our previous blog post. However, the emission calculator is comprehensive enough to perform calculations for any instance residing in any region.

Results

The results of the calculator give a clear picture of the total carbon footprint of X-Road, as well as its individual components. The key findings demonstrate: 

1. Around 96 % of total emissions are related to the operations of X-Road Security Servers. 

2. Data transmission and storage provide marginal contributions to the total carbon footprint (around 1 % and 3 %, respectively).

3. The annual carbon footprint for Estonia and Finland was approximated as 45,685 KgCO2e and 22,593 KgCO2e, respectively.

4. The discrepancy between the two countries relates mostly to the difference in average electricity grid emission factor – this reflects the larger percentage of low carbon electricity sources in the Finnish grid and the relative reliance in Estonia on shale oil.

Based on these results, we have derived recommendations to manage or mitigate emissions relevant to different stakeholder groups. These groups have been determined based on their different levels of access and influence over the way X-Road is used; including X-Road user organisations (X-Road members), the X-Road governing authorities overseeing regulations guiding the service application, NIIS as the product owner and developer of X-Road, and the broader public. The recommendations can briefly be summarised as follows:

1. The future development of X-Road could ensure flexibility for stakeholders to disable or reduce the emissions of components, such as message logging and timestamping, subject to potential performance and security requirements.

2. Wherever possible, NIIS can recommend an energy tracking application to enable emissions modelling of X-Road in real-time. This will integrate transparent monitoring of emissions in reporting and encourage users to evaluate ways to reduce emissions as they will have a reference (Business As Usual) data set to experiment with and understand their emissions footprint.

3. X-Road governing authorities can provide clear information and support emissions reduction strategies, such as granting permission to host servers on the public cloud and reducing mandatory requirements for message logs and/or timestamping.

4. X-Road members should implement X-Road services efficiently and in a manner optimised for emissions reduction from the outset. This could include ensuring that equipment is efficient and server utilisation is maximised. Alternatively, servers could be hosted on the public cloud – as it is beginning to be allowed for organisations requiring the highest level of security, such as UK Defence – if permitted by local regulations and the governing authority.

5. Where the option exists, infrastructure should be powered using renewable electricity. Moreover, best practice should be followed, such as using power-efficient hardware devices and optimised data compression. 

Process of the study

As discussed in previous blog posts, this study was divided into three phases, with results and methods published for feedback by a steering committee and technical experts after each phase. The phases were as follows:

1. Determining an emissions boundary and mapping the main causes of environmental impacts of the X-Road instance.

2. Building a Carbon Footprint calculator for X-Road, based on best practice and X-Road use cases.

3. Defining recommendations for improving the sustainability of X-Road.

The calculator was designed to ensure a high degree of flexibility while obtaining directional results that could be accepted with high confidence. This ensured validity across the instances in Finland and Estonia and across the varying circumstances and technical literacy of different X-Road users. 

Outcomes

The report is available here and the simplified X-Road Emissions Calculator is available here.

The X-Road Carbon Emissions Calculator – Methodology and Results

Introduction 

To support Nordic Institute for Interoperability Solution’s (NIIS) mission on making X-Road the ‘most sustainable data exchange solution in the world’, a study is being carried out by Gofore and Stockholm Environment Institute Tallinn to assess the carbon impacts of the solution. In the first stage we defined the scope of the project and described what would and wouldn’t be included. We have proceeded a step further and developed a unique emissions calculator that measures the carbon footprint of X-Road’s operations.  The calculator was built using data from instances in Estonia and Finland but is comprehensive enough to perform calculations for any instance residing in any region.

From energy to emissions

The methodology proceeded in two stages. First, the team developed a model to understand the electricity burden of operating an X-Road Security Server. Once the electricity consumption of one Security Server was determined, the results were multiplied with the total number of Security Servers in an entire X-Road ecosystem. Next, this was converted into an estimation of released emissions by multiplying this value by an ‘emission factor’ for grid electricity. The emission factor describes the emissions for each unit of electricity and considers all the different sources of electricity generation within a country or region. This is important as the content of this ‘electricity mix’ can be wildly different between different countries, leading to very different emissions outcomes for the same energy use. Results are reported in units of CO2 equivalent (CO2e), which includes other important greenhouse gases, such as methane. For these calculations, emission factors published by Association of Issuing Bodies (AIB) for 2019 are used. 

The model

This model was refined in subsequent steps based on new literature findings, updated assumptions testing among the internal team, and interviews with end-users and industry experts. As it is not possible to investigate every single source of emissions within a single project, it is important to focus on the largest that will define the size of the carbon footprint. As such, three main sources of emission were identified:  infrastructure, data transaction and data storage. In this context, the ‘infrastructure’ component only included the Security Server which is required to process data and enable secure data exchange. The team hypothesized this would be the main source of emissions, since multiple processors can collectively contribute vast amounts of heat emission and energy consumption. To compensate, data storage was considered as a separate component to account for the recording of all the transactions occurring over X-Road servers.  

Infrastructure

The main physical infrastructure that enables secure data exchange through X-Road is the Security Server. For simplification, only CPU and RAM are considered as these components are responsible for almost all the energy consumption in a security server.  A Fujitsu server with a model “FUJITSU Server PRIMERGY RX1330 M4 is assumed to be a standard server throughout the calculations. This serves as a good representation of a sample server employed across the two instances based on surveys conducted with the Finnish and Estonian authorities. The server employs an Intel® Xeon® E-2288G Processor with 16M Cache, 3.70 GHz and 8 cores, and was published in the database in 2019. Values for the energy use of this server were obtained from a widely used published database, SPECpower_ssj2008.

The processor’s energy consumption can be modelled as a directly proportional relationship between CPU utilization and Average Active Power. In turn, the RAM energy consumption model is based on a study conducted by Pedro H. P. Castro et al. The RAM’s power consumption is divided into background power (depends only on memory states and on the frequency of the operation) and operational power (product of memory bandwidth and the power required to run a particular command). The total power consumption was taken as the sum of these two individual components.

A final factor to consider is the energy consumption of the supporting infrastructure. This is widely considered in the industry through a factor known as the power use effectiveness (PUE). PUE describes the amount of energy used by the IT devices compared to the amount used by the supporting infrastructure (in this project a PUE of 1.58 was used). Thus, the total energy consumption of a Security Server is calculated as a product of total electricity consumption and PUE.

Transaction

Work by Aslan et al pointed to an average data transmission efficiency of 0.06 kWh /GB for fixed line transmission, a value that was also seen empirically to half every 2 years. For 2021, including the expected efficiency gains, this value is found to be 0.0075 kWh of electricity consumed for 1 GB of data transferred over the local internet. The methodology simply involves taking the product of the total amount of data exchanged over X-Road and the electricity consumption per GB of data transferred. 

Storage

According to bilateral exchange of information with different X-Road members, data is stored in hard disk drives (HDDs). A study conducted by Adam Lewis et al from Athens state university outlines a comprehensive approach that enables accurate energy consumption calculations of data storage in an HDD. For calculating the energy consumption of data storage, an HDD’s start up power, power consumption during data writing, power consumption in idle mode and finally, power consumption in standby mode is all calculated and summed.  

Carbon footprint results

The following table summarizes the total emissions of the Estonian and Finnish instances.

Table 1. Estimated carbon footprint summary of X-Road operations in Estonia and Finland.

The total carbon dioxide equivalent (CO2e) emissions for Estonia amount up to 45,685 kgs and for Finland up to 22,593 kgs. The results include emissions from all three main operations (i.e.,  infrastructure,  transaction and storage). Although the Finnish instance has a lot more Security Servers than the Estonian instance (173 in Estonia and 290 in Finland), the disparity in the electricity emission factor (0.723 kg of CO2e/kWh for Estonia and 0.136 kg of CO2e/kWh for Finland) is the key reason for the contrast between the aggregated emission amounts of both regions. A comparison between the emissions of the three main operations allows us to identify and analyse areas which contribute most significantly to the total emissions. Resulting emissions from the electricity consumption are depicted in figure below.

Figure 1. Emission by main operations in the Finnish and Estonian instances

The electricity consumption (and therefore emissions) by the Security Servers in comparison to data transaction and storage is substantially high and dominates the total share. Emissions from servers contribute to 96% of the total emissions which amount to 21,617 kgs of CO2e for the Finnish instance and 43,999 kgs of CO2e for the Estonian instance. This directly points to the primary place of focus in order to make X-Road’s operations more sustainable. Data storage contributes to a mere 3% that amount to 1,507 kgs of CO2e for Estonia and 950 kgs of CO2e for Finland. The remaining are from data transaction (less than 1%) for both instances. 

In conclusion, the calculator depicts a holistic view of the total emissions due to X-Road’s operations. For the scope of this project, calculations are done for Estonia and Finland though the calculator is configurable for any instance in any region. This methodology aims to cater emission calculations to the key areas which emerge to have the greatest emissions, and to be flexible to the different circumstances used by X-road members. As there are hundreds of different Security Servers and storage devices spread all across Finland and Estonia, it is impossible to track down exact models of the components with their corresponding power consumption specifications and data exchange numbers. We therefore hope our approach strikes the ideal balance between accuracy and flexibility.

We welcome your feedback on the methodology and results and will shortly publish more details in a formal report. The next stage of the project will develop a series of recommendations for NIIS and X-Road members to ensure carbon emissions are minimized without compromising the effectiveness of the X-Road service. 

---

Adil Aslam is a Junior Expert At SEI Tallinn and joined the Climate and Energy programme in November 2020.

Adil has diverse experience in manufacturing, energy consultancy, modelling and project management. He is an expert in conventional and renewable energy systems while being proficient in a whole range of energy simulation software. He is adept in programming and has used Python and Matlab to develop various energy models. His interests lie in energy systems, sector coupling, green finance, sustainable business models, electricity markets and simulations.

He is currently working on simulating hydrogen potential in Estonia, climate neutral scenarios for 2050 and providing his expertise in various energy related projects. He actively takes part in the company’s business development operations and aims to explore different business opportunties.

He graduated from Technische Universität Berlin, Germany in 2020 with a Master’s in Business Engineering Energy. He did his Master’s thesis at Forschungszentrum Julich (one of the largest interdisciplinary research centres in Europe) and the Institut für Stromrichtertechnik und Elektrische Antriebe (ISEA) RWTH, where he worked on modelling an energy management system for residential PV home systems to sustain blackouts.

Introducing X-Road Emissions Calculator Project

Introduction

NIIS has set an ambitious goal to make X-Road the 'most sustainable data exchange layer solution in the world' and thereby align its operational model with the climate and sustainable development goals articulated under the Paris Agreement and the 2030 agenda for sustainable development. That is why X-Road's environmental impact is being identified by NIIS, Gofore and the Stockholm Environment Institute (SEI).

The study is being carried out by a team, which is a combination of the sectoral experience provided by Gofore and the academic competence provided by SEI Tallinn. Gofore's experts have experience with X-Road core development processes, X-Road implementations, and X-Road service developments. SEI Tallinn is a leading sustainability think tank ranked #2 globally in the environmental field by Uni Penn ranking. SEI Tallinn has long-standing experience in climate and energy policy scenarios, carbon footprint calculations and life-cycle impact assessments.

The current project measures the carbon footprint of X-Road operations within Estonia and Finland. It will give recommendations for all related parties, including X-Road governing authorities and user organisations around the world.

Project is in its early stages. The partners have scoped the most important environmental impacts of X-Road instance and outlined the initial approach for X-Road carbon footprint calculation that is expected to be carried out from February to March 2021. Based on this, additional feedback from all interested parties is now requested to improve the project outcomes.

Defining the scope

Figure 1: The architecture of X-Road

A critical first stage in determining environmental impacts is defining the boundary beyond which further emission sources shall not be included. In the case of X-Road, this is based on a rigorous understanding of the relevant operational and infrastructural components involved in the service, as defined in purple in the figure above. The essential operation is exchanging messages and data between two X-Road members over a pair of Security Servers. It is thought that the impact of the Central Server is likely to be quite minor and shall not be considered within the calculation.

In this context, the potential sources of emissions associated with X-Road have therefore been mapped, and can broadly be associated with the following categories:

• Life-cycle of the components used by infrastructures that allow X-Road's operations

  • Infrastructure for running a Security Server that can be on-premises or in the cloud.

  • Physical device that might be necessary for holding the keys and certificates that assure the identification of the members.

• Energy consumption of the Security Server infrastructure used by X-Road members or governing authorities.

• Energy consumption, in addition, for exchanging messages / using X-Road services.

However, a decision was reached not to include the life-cycle of different components within the footprint calculation. This is because similar studies have indicated that these areas provide only a small contribution to the total emissions relative to the use phase. This notwithstanding, a detailed analysis of X-Road's infrastructure will lead to a series of effective recommendations to minimise the life-cycle emissions. This means the calculation of emissions consists of two aspects: the operational emissions from hardware and those associated with the transmission of data.

The quantification and subsequent recommendations shall further proceed via a relative comparison between different 'use cases'. These provide a range of current and prospective parameters by which service can be defined, and thereby also a complete range of potential emission impacts within the study. Building on the experiences of Estonia and Finland, the project plans to showcase the changes to carbon footprint when changes in the following parameters are considered:

• Infrastructure level

  • The server used: On-premise infrastructure vs on cloud

  • Trust level: with and without a physical signing device

• Service level:

  • Message size across different services

  • Message log requirements (whole message, metadata only, or no message logging).
    

Call for interested parties

There have only been a limited number of studies to assess the climate impacts of specific software. The continued growth in digital services and infrastructure also means that the importance of such emissions is only likely to increase further in the future.

Beyond showcasing the proposed methodology, the project partners are further open to making use of all relevant expertise to maximise the project's success. This can be both in terms of changes to project scope, or suggestions for improving the calculation methodology. Interested parties should contact NIIS.

Report: study of the environmental impact of X-Road and the possibilities of reducing it

---

The author Peter Robert Walke joined SEI Tallinn in September 2020. He works in the sustainable development programme on a project related to developing quantitative tools for assessing the role of spatial planning policies at different scales on greenhouse gas emissions.

Peter’s background is in the physical sciences and he holds a PhD in chemistry from Katholieke Universiteit Leuven in Belgium, where he worked in nanoscience. Before that he obtained an MSci in Natural Sciences from University College London, and is currently also a postdoctoral researcher in the department of materials and environmental technology at Tallinn University of Technology. He hopes to now successfully apply his skills in research and data analysis to a new field.