Skip to content

Guidance for Data Providers

NOTE: This document is primarily intended for technical and operations teams within organisations wishing to participate in the IB1 Trust Framework as Data Providers.

NOTE: The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in RFC 2119.

Data Provider Role and Responsibilities

Data Providers are IB1 Trust Framework members which make data sets available to:

  • Data Consumers within the Trust Framework

    • Applies to shared data sets
    • Data APIs secured using Financial-grade API security (FAPI)
  • Any third party, in the case of open data

    • Open data sets are made available through publicly accessible APIs
    • Data are made available under an appropriate open license agreement

In this context, Data APIs can be anything from a simple link to a CSV or other file (for open data) to a complex query system returning custom output on demand.

To support this role, Data Providers have certain responsibilities:

Responsibility - Data classification

Data Providers are responsible for classification of their data sets by data sensitivity class, and for ensuring that data are shared, or not shared, as appropriate.

  • Any data classed as IB1-C (Closed) or IB1-SP (Personal) MUST NOT be shared within the IB1 Trust Framework at present.
  • Data in IB1-O, IB1-SA, and IB1-SB MAY be shared within the IB1 Trust Framework as described in the following sections

Responsibility - Access and licensing specification

Data Providers are responsible for creating access rules for each dataset shared via the IB1 Trust Framework. For each data set, the Data Provider must create a set of one or more access rules as defined in Data Set Metadata. These rules specify the conditions under which access will be granted, along with the capabilities forming the license grant should those conditions be satisfied.

  • Data Providers MUST ensure that access conditions and capability grants are non-discriminatory and fair to Data Consumers

  • Data Providers SHOULD ensure that the validity time bounds of any published capability grants are sufficient for reasonable commercial operation of a data consumer

Responsibility - Data provision

Data Providers are responsible for providing a data API for each data set, and for configuring and securing each API in accordance with the kinds of data it exposes (along with any other access conditions deemed necessary and appropriate by the Data Provider on a per-data-set basis)

  • Open data MUST be made available with no authentication or authorization requirements

  • Shared data MUST made available through a FAPI compliant resource server connected to the TFGS’s authorization service (see Common Security Requirements for more details)

Data formats

The IB1 Trust Framework does not define or mandate individual data formats to be returned from data APIs - the diversity of kinds of information in the sector renders this impractical, and the additional overhead imposed on data providers would be undesirable.

With this said, we would encourage Data Providers to consider likely automated consumption of data when designing their data APIs - this implies using machine readable formats such as CSV, JSON and similar, with a preference for those formats compatible with existing software tools and libraries.

Responsibility - Data integrity and correctness

Data Providers are responsible for the correctness of data returned by their published data APIs. Open Net Zero provides a mechanism (a flag at the top-right of the page for the dataset) by which Data Consumers can report any data quality issues to Icebreaker One, which will be forwarded to a nominated contact within the Data Provider.

Responsibility - Metadata provision

Data Providers are responsible for creating, and publishing, metadata for each exposed data set. This provides visibility of each data set within the Trust Framework Governance Service (TFGS) Registry.

The metadata file covers, for each provided data set, whether shared or open:

  • Semantic content
  • Access rules and licensing of the data set
  • Transport mechanism specification
  • Syntactic content

The content and format of the metadata file can be found at Data Set Metadata.

The Data Provider is responsible for the accuracy and completeness of this metadata.

Responsibility - API availability

Data Providers are responsible for availability of their published data APIs. Availability will be monitored automatically by the TFGS, availability information will form part of the metadata for each data set record in the TFGS Registry.

The TFGS will provide a mechanism for Data Providers to announce scheduled downtime when planned, and to report unscheduled downtime when necessary.

Responsibility - API stability and change management

Data Providers are responsible for managing any change to the data APIs such that disruption to Data Consumers is minimised. This is handled through:

  • Semantic versioning of all API methods
  • Publication of anticipated changes and retirement of API versions through the TFGS
  • Changeover periods where new and prior API versions are run in parallel

Data API Requirements

Unlike Open Banking, the IB1 Trust Framework does not mandate particular APIs for data provision - it is expected that there will be a variety of mechanisms to expose data, with varying inputs (from a single data set ID to a complex query object) and varying kinds of output dependent on the information exposed.

With that said, there are certain properties that all data APIs must satisfy to interoperate successfully with other parties within the IB1 Trust Framework.

We refer to endpoints used to retrieve data as data endpoints, and others as infrastructure endpoints.

Date and time formats

Whenever date or time quantities are accepted or returned from a data API, these values MUST conform to |RFC| 3339. This is referenced elsewhere in this document as date/time

Endpoint security

Open data endpoints

Data endpoints which provide access to open data (in class IB1-O) MUST NOT require any form of authentication prior to access. This includes allowing access to entities which are not members of the IB1 Trust Framework.

Shared data endpoints

Data endpoints which provide access to shared data (in classes IB1-SA and IB1-SB) MUST implement the subset of the resource server part of the FAPI specification used within the IB1 Trust Framework as described in Common Security Requirements, and authorize against the TFGS authorization service.

Protected data endpoints MAY use the token introspection mechanism provided by FAPI to gather additional information about the client making the request prior to any access decision.

Heartbeat and monitoring endpoint

All data APIs SHOULD include a FAPI protected heartbeat infrastructure endpoint. This serves two purposes:

  • It allows the TFGS to monitor the liveness of the data API

  • It provides some level of verification that the resource server has been correctly configured

The heartbeat endpoint location is defined within the data set metadata file, if specified it MUST respond to GET requests from the TFGS monitoring system with a 200 status code.

If the heartbeat response contains a body, the body will be interpreted by the TFGS monitoring system as a JSON dictionary containing statistics for the period between the most recent two successful calls to the heartbeat endpoint (including the call to which this is a response). This response is RECOMMENDED as it provides oversight of API usage to the TFGS.

Legal keys, and the semantics of their associated values, are as follows:

Name

Content

period_start

date/time of the start of the period for which statistics are reported

period_end

date/time of the end of the period for which statistics are reported, this will typically be the date and time of the heartbeat request

api_call_response_[CODE]_count

integer number of requests to non-heartbeat endpoints within this data API which resulted in a response of type CODE. A distinct key:value pair is sent in the response for each distinct HTTP status code returned.

API documentation

Data APIs MUST be described within the transport section of the data set metadata file.

Versioning

All data APIs MUST include a version number in the path of each data endpoint. This version SHOULD use semantic versioning to differentiate between breaking and backwards compatible changes. This MUST be documented within the OpenAPI transport section of the metadata file.

Problem Resolution (Data Providers)

Effective resolution of problems (Data Providers)

A Data Provider MUST create documentation to clearly outline the policies, processes and systems it has in place for problem resolution and its respective service level objectives. This framework should enable the effective resolution of Data Consumer issues and therefore cater for problems that relate specifically to a Data Consumer’s use of a Data Provider’s data APIs. In the event that a Data Consumer is unable to resolve an issue with a Data Provider, the issue MAY be flagged to the TFGS Dispute Resolution function for independent support.

Online support

Data Providers SHOULD provide Frequently Asked Questions, which address areas that may be specific to Data Consumers such as technical advice or test facility guidance. They should also consider a means of identifying recurring questions or user-error issues so these can be collated into Frequently Asked Questions to support the early resolution of problems.

Problem resolution documentation, Frequently Asked Questions, contact details, opening times and out of hours support SHOULD be published and easily accessible in one collective area on the Data Provider’s website.

Ticket management process

Data Providers MUST ensure they have a functioning ticket management system which enables them to respond to issues and problems raised within clearly defined service level targets. A successful problem resolution framework will enable the efficient identification and resolution of problems which specifically impact Data Consumers. The system for raising and reporting on tickets should be transparent in order to fully inform users and engender trust across the ecosystem.

Dispute resolution for access control and capability grant policies (Data Providers)

In cases where a Data Consumer believes that access conditions or capability grant rules are being applied unfairly, the TFGS will act as a mediating party.

TFGS support (Data Providers)

The TFGS Service Desk will provide participants with a supplementary ticket management process and supports Data Providers in communicating problems to ecosystem participants via the noticeboard.

Change Management

This section outlines various change scenarios that may impact Data Consumers, and provides guidance for a Data Provider to consider when implementing a change and communicating to Data Consumers.

Downtime

Planned downtime, by definition, is something that a Data Provider anticipates and therefore is able to give advance notice to Data Consumers. It is not generally possible to give advance notice of unplanned downtime, but Data Providers should give notice as soon as they are aware of the downtime.

When providing notifications, Data Providers SHOULD provide a specific time period, so Data Consumers are aware that the data API will be unavailable for that time, or upon a subsequent notification to confirm that the service has been reinstated sooner than anticipated.

In future, TFGS Support Services will be able assist Data Providers with the dissemination of downtime information to the wider Icebreaker One ecosystem via its central noticeboard facility.

Planned downtime should be given at least five business days before the event. Apart from cancelling the planned downtime, no changes should be made to the planned downtime notification within the five business day period. Where practical, Data Providers should give advance notice via their own website, developer portal or TFGS of any planned downtime one calendar month in advance.

Licensing and access control changes

Data Providers MUST provide advance notice of any changes to access control and capability grant policies. This is normally covered by the time-bounded nature of these grants, but Data Providers MAY also use any notification services provided by the TFGS to publish additional information about changes.

Data API changes

Data Providers may periodically wish to upgrade and / or to deprecate versions of their data APIs. The following mitigation measures will reduce the impact on service providers of these changes.

Dual running and deprecation

Data Providers SHOULD support a minimum of two non-compatible API versions in a production context, providing both versions were previously supported by the Data Provider, for a period of time long enough to ensure that Data Consumers have had sufficient time to successfully test the new version and migrate their applications and customers. TFGS recommends dual running for at least six months for a major version, and three months for a minor version. Where a Data Provider implements a data API for the first time, they will only need to support this one version to start with.

The deprecation of unsupported versions is at the Data Provider’s discretion – based on usage metrics. However, the TFGS may recommend that any specific version (major, minor, or patch) should be deprecated at any time, and this should be implemented within three months of notification by the TFGS. This is to cater for critical defects, especially those relating to security. In exceptional circumstances it may be agreed by TFGS that support for a specific version is terminated earlier.

Data Providers must not apply any measures to induce Data Consumers to adopt a new version of the APIs (e.g. rate limiting the older version while providing better performance on a newer version).

API Credentials associated with an API should be version agnostic. Therefore, a Data Consumer accessing v1.0, v1.1 or v2.0 should be able to use the same API Credentials across all available API endpoints.