Discussion

Supporting A Network of Data: Validator & API Updates

Rob Redpath (IATI Secretariat)
Rob Redpath (IATI Secretariat) • 3 November 2023

We’re looking into how we can improve tooling around activity identifier references in the standard and we’d like your input on how we implement this. 

Our intention is to update the IATI Validator to check all the activity identifier references in a file being validated, and identify times that an identifier doesn’t appear in either the datastore or the file being validated. This can then form the basis of a notification in the web interface, and another data point in the Validation API. If no activity ID references are provided, then no message is displayed. 

In order to support this, we’re intending to create an API endpoint that allows anyone to quickly check for the existence of activity identifiers in the datastore in bulk; this is already technically possible, but for very large numbers of activity identifiers this can quickly become burdensome. This is, of course, a completely new thing for us - so please bear with us as we work out the details. We’re aware that there will still be some gaps in coverage, such as when the activity identifier is in another very recently-published file or is in a file that has critical errors, but we’re expecting it will still be a helpful improvement. 

We will then use this API to allow the IATI Validator to check all the activity identifiers in a file being validated, and warn if an activity identifier doesn’t appear in either the datastore or the file being validated. 

We’d like feedback on the API design: 

  • Is this an API endpoint that you think would be useful for other applications - those developed by you, other community members, or other tools maintained by the Secretariat? 
  • Are there other, similar endpoints that you think would be useful to implement in the future? 

This will help shape our implementation and ensure that we’re building useful tools.


 

Comments (1)

Herman van Loon
Herman van Loon

Thank you very much for this initiative. Regarding you two questions:

1 - this API endpoint would be very useful for me. Currently I am maintaining my own corpus of valid IATI identifiers making use of the IATI datastore API. A new API endpoint would enable me to skip the building and maintaining of this corpus and directly validate a set of identifiers where I would expect an API response for each identifier <Exists>   or <Not exists>.

2 - a similar API endpoint could be made for references to organization identifiers. There might be some challenges though to build a corpus of valid organization identifiers.


Please log in or sign up to comment.