Guidance for Data Providers
NOTE: This document is primarily intended for technical and operations teams within organisations wishing to participate in the IB1 Trust Framework as Data Providers.
NOTE: The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in RFC 2119.
Data Provider Role and Responsibilities
Data Providers are IB1 Trust Framework members which make data sets available to:
Data Consumers within the Trust Framework
Any third party, in the case of open data
- Open data sets are made available through publicly accessible APIs
- Data are made available under an appropriate open license agreement
To support this role, Data Providers have certain responsibilities:
Responsibility - Data classification
Data Providers are responsible for classification of their data sets by data sensitivity class, and for ensuring that data are shared, or not shared, as appropriate.
- Any data classed as IB1-C (Closed) or IB1-SP (Personal) MUST NOT be shared within the IB1 Trust Framework at present.
- Data in IB1-O, IB1-SA, and IB1-SB MAY be shared within the IB1 Trust Framework as described in the following sections
Responsibility - Access and licensing specification
Data Providers are responsible for creating access rules for each dataset shared via the IB1 Trust Framework. For each data set, the Data Provider must create a set of one or more access rules as defined in Data Set Metadata. These rules specify the conditions under which access will be granted, along with the capabilities forming the license grant should those conditions be satisfied.
Data Providers MUST ensure that access conditions and capability grants are non-discriminatory and fair to Data Consumers
Data Providers SHOULD ensure that the validity time bounds of any published capability grants are sufficient for reasonable commercial operation of a data consumer
Responsibility - Data provision
Data Providers are responsible for providing a data API for each data set, and for configuring and securing each API in accordance with the kinds of data it exposes (along with any other access conditions deemed necessary and appropriate by the Data Provider on a per-data-set basis)
Open data MUST be made available with no authentication or authorization requirements
The IB1 Trust Framework does not define or mandate individual data formats to be returned from data APIs - the diversity of kinds of information in the sector renders this impractical, and the additional overhead imposed on data providers would be undesirable.
With this said, we would encourage Data Providers to consider likely automated consumption of data when designing their data APIs - this implies using machine readable formats such as CSV, JSON and similar, with a preference for those formats compatible with existing software tools and libraries.
Responsibility - Data integrity and correctness
Data Providers are responsible for the correctness of data returned by their published data APIs. Open Net Zero provides a mechanism (a flag at the top-right of the page for the dataset) by which Data Consumers can report any data quality issues to Icebreaker One, which will be forwarded to a nominated contact within the Data Provider.
Responsibility - Metadata provision
Data Providers are responsible for creating, and publishing, metadata for each exposed data set. This provides visibility of each data set within the Trust Framework Governance Service (TFGS) Registry.
The metadata file covers, for each provided data set, whether shared or open:
- Semantic content
- Access rules and licensing of the data set
- Transport mechanism specification
- Syntactic content
The content and format of the metadata file can be found at Data Set Metadata.
The Data Provider is responsible for the accuracy and completeness of this metadata.
Responsibility - API availability
Data Providers are responsible for availability of their published data APIs. Availability will be monitored automatically by the TFGS, availability information will form part of the metadata for each data set record in the TFGS Registry.
Responsibility - API stability and change management
- Semantic versioning of all API methods
- Publication of anticipated changes and retirement of API versions through the TFGS
- Changeover periods where new and prior API versions are run in parallel
Data API Requirements
Unlike Open Banking, the IB1 Trust Framework does not mandate particular APIs for data provision - it is expected that there will be a variety of mechanisms to expose data, with varying inputs (from a single data set ID to a complex query object) and varying kinds of output dependent on the information exposed.
With that said, there are certain properties that all data APIs must satisfy to interoperate successfully with other parties within the IB1 Trust Framework.
We refer to endpoints used to retrieve data as data endpoints, and others as infrastructure endpoints.
Date and time formats
Open data endpoints
Data endpoints which provide access to open data (in class IB1-O) MUST NOT require any form of authentication prior to access. This includes allowing access to entities which are not members of the IB1 Trust Framework.
Shared data endpoints
Data endpoints which provide access to shared data (in classes IB1-SA and IB1-SB) MUST implement the subset of the resource server part of the FAPI specification used within the IB1 Trust Framework as described in Common Security Requirements, and authorize against the TFGS authorization service.
Protected data endpoints MAY use the token introspection mechanism provided by FAPI to gather additional information about the client making the request prior to any access decision.
Heartbeat and monitoring endpoint
It provides some level of verification that the resource server has been correctly configured
The heartbeat endpoint location is defined within the data set metadata file, if specified it MUST respond to
GET requests from the TFGS monitoring system with a
200 status code.
If the heartbeat response contains a body, the body will be interpreted by the TFGS monitoring system as a JSON dictionary containing statistics for the period between the most recent two successful calls to the heartbeat endpoint (including the call to which this is a response). This response is RECOMMENDED as it provides oversight of API usage to the TFGS.
Legal keys, and the semantics of their associated values, are as follows:
date/time of the start of the period for which statistics are reported
date/time of the end of the period for which statistics are reported, this will typically be the date and time of the heartbeat request
integer number of requests to non-heartbeat endpoints within this data API which resulted in a response of type CODE. A distinct key:value pair is sent in the response for each distinct HTTP status code returned.
Data APIs MUST be described within the transport section of the data set metadata file.
All data APIs MUST include a version number in the path of each data endpoint. This version SHOULD use semantic versioning to differentiate between breaking and backwards compatible changes. This MUST be documented within the OpenAPI transport section of the metadata file.
Problem Resolution (Data Providers)
Effective resolution of problems (Data Providers)
A Data Provider MUST create documentation to clearly outline the policies, processes and systems it has in place for problem resolution and its respective service level objectives. This framework should enable the effective resolution of Data Consumer issues and therefore cater for problems that relate specifically to a Data Consumer’s use of a Data Provider’s data APIs. In the event that a Data Consumer is unable to resolve an issue with a Data Provider, the issue MAY be flagged to the TFGS Dispute Resolution function for independent support.
Data Providers SHOULD provide Frequently Asked Questions, which address areas that may be specific to Data Consumers such as technical advice or test facility guidance. They should also consider a means of identifying recurring questions or user-error issues so these can be collated into Frequently Asked Questions to support the early resolution of problems.
Problem resolution documentation, Frequently Asked Questions, contact details, opening times and out of hours support SHOULD be published and easily accessible in one collective area on the Data Provider’s website.
Ticket management process
Data Providers MUST ensure they have a functioning ticket management system which enables them to respond to issues and problems raised within clearly defined service level targets. A successful problem resolution framework will enable the efficient identification and resolution of problems which specifically impact Data Consumers. The system for raising and reporting on tickets should be transparent in order to fully inform users and engender trust across the ecosystem.
Dispute resolution for access control and capability grant policies (Data Providers)
TFGS support (Data Providers)
Planned downtime, by definition, is something that a Data Provider anticipates and therefore is able to give advance notice to Data Consumers. It is not generally possible to give advance notice of unplanned downtime, but Data Providers should give notice as soon as they are aware of the downtime.
When providing notifications, Data Providers SHOULD provide a specific time period, so Data Consumers are aware that the data API will be unavailable for that time, or upon a subsequent notification to confirm that the service has been reinstated sooner than anticipated.
Planned downtime should be given at least five business days before the event. Apart from cancelling the planned downtime, no changes should be made to the planned downtime notification within the five business day period. Where practical, Data Providers should give advance notice via their own website, developer portal or TFGS of any planned downtime one calendar month in advance.
Licensing and access control changes
Data Providers MUST provide advance notice of any changes to access control and capability grant policies. This is normally covered by the time-bounded nature of these grants, but Data Providers MAY also use any notification services provided by the TFGS to publish additional information about changes.
Data API changes
Dual running and deprecation
Data Providers SHOULD support a minimum of two non-compatible API versions in a production context, providing both versions were previously supported by the Data Provider, for a period of time long enough to ensure that Data Consumers have had sufficient time to successfully test the new version and migrate their applications and customers. TFGS recommends dual running for at least six months for a major version, and three months for a minor version. Where a Data Provider implements a data API for the first time, they will only need to support this one version to start with.
The deprecation of unsupported versions is at the Data Provider’s discretion – based on usage metrics. However, the TFGS may recommend that any specific version (major, minor, or patch) should be deprecated at any time, and this should be implemented within three months of notification by the TFGS. This is to cater for critical defects, especially those relating to security. In exceptional circumstances it may be agreed by TFGS that support for a specific version is terminated earlier.
API credentials, consent and authorisation
API Credentials associated with an API should be version agnostic. Therefore, a Data Consumer accessing v1.0, v1.1 or v2.0 should be able to use the same API Credentials across all available API endpoints.