In the ever-evolving landscape of data management, the concept of data products has gained prominence, offering a standardized approach to packaging and consuming relevant data resources. At the forefront of this movement is the Open Data Product Specification (ODPS), a vendor-neutral, open-source, and machine-readable data product metadata model. Let's delve into the unique features that set ODPS apart in the realm of data product standards.
Technical Emphasis and Current Version. ODPS places a strong emphasis on technical aspects, defining objects and attributes related to data products such as data pipelines, access, data models, and deployment. The current latest production version, 2.1, reflects continuous refinement, with the next release scheduled for early 2024. Naturally, ODPS has a governance model which provides needed structure and guidance for long-term support.
Holistic Metadata Model: ODPS distinguishes itself by offering a full-stack metadata model that includes attributes for pricing plans, built-in support for internationalization (i18n), contract elements, Service Level Agreements (SLA), DataOps, licensing, and provider details. This comprehensive approach sets ODPS apart from other standards and enables the complete decoupling of data products from vendor systems, fostering decentralization.
Business-Focused Standards: The shift towards data product-focused standards signifies the growing need for a refreshed approach to treating data as a business-related commodity rather than a purely technical asset. ODPS addresses this need by providing an easy-to-adopt, developer-friendly standard with a distinct business focus, challenging the limitations faced by alternatives like DCAT.
Community Recognition: In Europe, the Data Spaces community acknowledges the importance of standards in creating data products and product offerings. Three standard templates stand out: DCAT, ODPS, and the TM Forum product model. ODPS, with its unique attributes, has gained recognition as a key player in this space.
Comparison with Alternatives: While ODPS differs significantly from the popular DCAT (Data Catalog Vocabulary), it's important to note that various standards, such as Data Product Descriptor Specification and Data Product Specification, share some similarities. However, ODPS stands out due to its holistic approach and inclusion of critical business-related attributes.
ODPS vs. DCAT: Understanding the Differences: A common question arises regarding the differences between ODPS and DCAT. A brief comparison reveals key distinctions:
ODPS has a flat structure, while DCAT follows a linked structure.
ODPS focuses solely on a single data product, whereas DCAT includes cataloging features.
ODPS is lightweight and fast to understand, in contrast to the exhaustive nature of DCAT.
Both share numerous attributes, but ODPS is tailored for data marketplace and sales, including data exchange.
DCAT, maintained by W3C, leans towards cataloging and sharing, while ODPS is community-maintained and oriented towards data marketplace dynamics.
The value of ODPS in the eyes of new emerging companies is big. They consider ODPS to provide same value as DCAT, but with faster adoption as we can see from the testimonial of one architect responsible for a trusted data exchange platform: “For me as a software architect/developer the ODPS standard is a lot more approachable. I think the usage of JSON is also a lot more flexible than the RDF structure proposed by DCAT. Furthermore, the current document structure of ODPS is more clear and easier to understand with almost the same amount of properties.”
Extensibility with OpenAPI Specification: ODPS offers flexibility by easily integrating with the OpenAPI Specification (OAS), including business, legal, and SLA attributes that may be missing from the standard package. A proof-of-concept test conducted in December 2023 showcased the seamless extension of ODPS in the API description of X-Road.
Service Discovery in X-Road
When services are connected to X-Road, their technical service descriptions are published on the Security Server by the Security Server administrator. The supported formats are Open API Specification (OAS) for REST services and WSDL for SOAP services. The service descriptions can then be accessed using a service discovery mechanism provided by X-Road. However, the mechanism is quite technical and requires direct access to the Security Server's messaging interface. Also, getting a list of all services available in the ecosystem would require querying each Security Server separately. Therefore, a more user-friendly service catalogue is needed.
A service catalogue is a web portal that contains descriptions of all the services available in the ecosystem. The primary purpose of the service catalogue is to provide a user-friendly channel to search and discover available services.
Image 1. Collecting service descriptions from Security Servers to Service Catalogue.
When implementing the service catalogue, collecting the service descriptions from the Security Servers can be automated. In that way, the descriptions need to be maintained in a single place only, and all the changes in the source are automatically updated to the catalogue. Nevertheless, additional organisation and data product metadata must be manually added and maintained on the catalogue by an administrator representing the organisation owning the service. The metadata may include any information related to the service and its use, e.g., a more detailed description of the data set, terms and conditions, contact information, pricing information, SLAs, etc.
Using ODPS in X-Road
Instead of maintaining the data product related metadata manually on the service catalogue, the metadata can be published in a machine readable format on the Security Server using ODPS. In practice, the OAS service descriptions can be extended with ODPS by including the selected parts of ODPS in them.
Image 2. Including ODS attributes in an OAS service description on the Security Server.
Starting to use the ODPS extension in the X-Road OAS service descriptions doesn’t require any code or configuration changes to the Security Server. It’s enough to update the OAS service description and the changes are immediately available over X-Road’s service discovery mechanism. This makes them accessible to all the service consumers and the X-Road Catalog extension (and other metadata harvesters) that’s used to centrally harvest service descriptions from all the service providers within an X-Road ecosystem. The approach was validated in a small PoC implemented in December 2023.
Image 3. An OAS service description with ODPS attributes. A full example is available here.
The attributes provided by ODPS can be utilised by the service catalogue which reduces the number of metadata that the X-Road Member Administrator must add to the catalogue manually. Using ODPS makes the information available in the service catalogue more unified, improves its quality, makes it machine readable, and eases its maintenance. However, utilising ODPS attributes in service catalogues provided by X-Road Operators may require changes to the catalogues so that they’re able to process and visualize the attributes. Nevertheless, at least some of the catalogues support visualizing and downloading raw OAS service descriptions which makes ODPS available to the catalogue users without additional changes. Since the Security Server already supports adding ODPS attributes to the OAS service descriptions, service providers may start utilising the extension immediately.
Conclusions
In conclusion, the Open Data Product Specification emerges as a unique and impactful player in the data product standards landscape, providing a holistic model that caters to the evolving needs of the data economy. Its focus on business-related attributes, coupled with technical precision, positions ODPS as a valuable asset for organizations navigating the complexities of data management in the modern era.
Extending X-Road OAS service descriptions with ODPS makes data product-related metadata available in the service catalogue more unified, improves its quality and enables more automation in the maintenance process. Service providers may start to use ODPS right away by updating their OAS service descriptions. Depending on the service catalogue solution, the ODPS attributes may be accessible in the catalogue without any additional changes to the catalogue software. However, visualising the ODPS-related information in a more user friendly way probably requires some changes. Overall, implementing ODPS is a low hanging fruit that improves the quality of the service metadata in X-Road ecosystems.
Jarkko Moilanen is the founder and strategy group chair of the Open Data Product Initiative.
Jarkko is an experienced senior data management professional with strong expertise on data monetization, platform and API economy. Currently Jarkko spearheads the Open Data Products track within the Data Enablement Program's data exchange platform under the aegis of the Abu Dhabi government. Jarkko also had a role in early phases of taking X-Road into national use in Finland and introducing REST APIs to the platform.
Open Data Product Specification (ODPS) which has matured into governed standard was ignited 2019 by Jarkko. ODPS was initially a sidetrack initiated as part of Data Product Toolkit development, but later was discovered valuable enough to be independent project as it gained traction from companies around the world.
On the international front, Jarkko was the first MIT CDOIQ Country CDO Ambassador for Finland for a few years. On top of practicing the data value creation, Jarkko has also written several business focused books on AI ("AI-Powered Data Products", 2023), API economy ("API Economy 101", 2018) and Data Economy (Deliver Value in the Data Economy, 2022).