This is a series of blog posts about the Message Room concept. The first part provides an introduction to synchronous and asynchronous data exchange. The second part concentrates on life-event-based services and potential implementation alternatives. Three alternative implementation approaches are discussed in more detail in the third part.
The Message Room concept was first introduced by Kristo Vaher, the government CTO of Estonia, in his paper Next Generation Digital Government Architecture in 2020. However, in the paper, the name X-Room was also used about the concept. In the latter design work conducted by NIIS, the name Message Room has been used, and therefore, I'm going to use it from here on.
According to the Next Generation Digital Government Architecture paper, the Message Room concept provides different administration sectors with an asynchronous communication channel that is the technical enabler for implementing life event-based services. In practice, the implementation would follow the publish-subscribe pattern, and it would be based on X-Road, including all X-Road's existing security guarantees. In that way, the existing infrastructure and data exchange ecosystem are utilized instead of reinventing the wheel and building everything from scratch.
What are life event-based services?
The idea behind life event-based services is that an event happens in a person's life and is registered by an information system. Then the information system notifies other information systems about the event that have registered their interest to receive updates on that specific event type. Based on the event, various processes are then triggered to provide the citizen with services related to the event. In this way, it is possible to offer or suggest services to citizens automatically instead of the citizens having to apply for them separately. For example, social benefits could be automatically offered to parents when a child is born.
What is a Message Room?
Generally, Message Room is an asynchronous messaging concept that decouples message producers and consumers and enables publishing messages and events to multiple consumers. Instead of sending messages directly between two information systems, message producers publish their messages to a Message Room with any number of consumers. The number of message publishers is not limited either, and a single Message Room can have one or more publishers.
Image 1. The Message Room concept.
A Message Room can be public or private. Anyone can publish and/or consume messages from public rooms, but private rooms can be accessed by authorized parties only. It is also possible that only authorized parties can publish messages, but consuming is allowed for anyone or vice versa.
Technically, a Message Room is an implementation of the publish-subscribe communication pattern. In other words, a Message Room can be considered a topic with multiple subscribers. A Message Room enables one-to-many and many-to-many communication.
Implementation alternatives
From a technical perspective, there are different ways to implement the Message Room concept. Every alternative has its pros and cons, and also the constraints and supported features vary between the alternatives. The implementation alternatives can be divided into five high-level categories:
built-in
integrated
connected
standalone
standardization.
It must be noted that the categories are not mutually exclusive, and they're partly overlapping. A potential outcome might very well be a combination of multiple categories.
Built-in
The idea of the built-in approach is that required features are implemented around the existing X-Road concepts, components, and protocols. In practice, it means extending the current protocols, introducing new entities, and expanding the functionalities of different components. Everything is designed and implemented using the "X-Road way," which means that the implementation provides all the same security guarantees that are provided to synchronous messages. At the same time, the implementation also has to deal with the constraints caused by the same security guarantees.
Integrated
Integrated means taking an existing open-source messaging solution (e.g., Apache Kafka, RabbitMQ) and integrating it into X-Road. In this way, the existing solution provides most of the messaging features, and there's no need to implement them separately. Some changes/additions to the X-Road protocols are likely to be required, but the data exchange inside a message room is based on the protocol(s) supported by the selected solution. Also, the integration covers most of the management-related tasks so that they can be completed using X-Road provided components, e.g., the Security Server UI and management API.
However, integrating the solution into X-Road requires potentially a significant effort, and there's a tight coupling between X-Road and the selected solution. Another downside is that X-Road becomes highly dependent on an external solution which future development and roadmap are out of NIIS's control. Also, it's very likely that providing all the same security guarantees provided by synchronous messages isn't possible, e.g., signing and timestamping all the data processed by a Message Room.
Connected
The connected approach uses X-Road to establish a connection between the data exchange parties and implement the actual data exchange through an external channel outside of X-Road. In this case, X-Road provides a secure channel for the initial handshake that includes exchanging the details of the external channel.
The handshake is a regular X-Road message in which headers and/or body include the required details. The external channel doesn't directly connect with X-Road, so it may use any data exchange protocol. In this way, the approach is not coupled with any specific solution or technology, and it can be used to support multiple different solutions and use cases. For example, different messaging solutions like Apache Kafka, Rabbit MQ, and Apache ActiveMQ could all be supported. However, the external solution is managed independent of X-Road, and all configuration tasks must be done using the native interfaces and tools provided by the solution. Also, the X-Road security guarantees are provided for the handshake but not for the data exchange.
Standalone
Standalone means implementing Message Rooms as a standalone, fully independent solution that can be connected to X-Road. In this way, the solution can be used together with X-Road, but also without it. Connecting the solution to X-Road could be done using the integrated or connected approach.
An existing open-source messaging platform, such as Apache Kafka, could be taken as a basis and developed further if needed. No additional development is required in the best case, and an existing solution can be used as-is. Also, the solution could be implemented using an existing standard or a set of standards that would support interoperability on a broader scale.
Standardization
Standardization is about creating a protocol stack based on a standard or a set of standards that define all the aspects of the communications required by a Message Room. In practice, it means leveraging existing standards and creating new ones in case they are needed. This approach would make Message Rooms both technology and solution agnostic. The first step would be to study whether suitable standards that meet the requirements already exist and participate in their development. For example, the Message Room concept could be connected to data spaces, e.g., an asynchronous communication method within a data space and across different data spaces.
Requirements for the implementation
In theory, all the described implementation alternatives are technically feasible, but there are various differences when looking into them in more detail. Eventually, the choice between the alternatives boils down to the expectations, requirements, and constraints for the Message Room implementation.
One of the recognized requirements for the implementation is a reuse of existing infrastructure and X-Road principles. In practice, it means utilizing X-Road's existing security mechanisms, including authentication, identity management, message logging, signing, and timestamping. Also, the Message Room concept implementation should be based on a decentralized architecture, and therefore, it must not include any centralized components.
All the Message Room-related research and development activities conducted by NIIS have concentrated on categories 1-3 (built-in, integrated, connected) since they directly include X-Road and, therefore, provide the most logical starting point for the implementation. The different activities will be covered in more detail in my next blog post.