Discussion
1 Mar - 15 Mar 2021

IATI Validator Public API: user requirements feedback by 15 March

IATI Technical Team • 1 March 2021

We are creating a public API for IATI’s Validator tool and are seeking feedback from IATI’s Community on the draft API Contract by 15 March. An API contract is a technical document that sets out how the API will behave and acts as a contract between us (the API provider) and the developers that will use it.

Should I provide feedback?

Do you have an understanding of how APIs work? Are you a developer supporting an organisation in creating their IATI XML files? Have you built an in-house tool for your organisation that creates the IATI XML file? Are you an IATI  publishing tool provider? If so, then we’d like to hear from you!  We want to understand if the draft Validator API meets your user requirements!

Background information 

The IATI Validator is a public tool for anyone to use. It checks if data complies with the rules and guidance of the IATI Standard, and it aims to improve the quality of IATI data. This is to ensure the data is accessible and useful to anyone working with data on development and humanitarian resources and results. Since the launch of the IATI Validator, the IATI Technical Team has proactively engaged with IATI members to support them to fix data errors (as identified by the Validator) and improve their data quality.

Currently users can only interact with the IATI Validator via the web interface as the IATI Validator doesn’t provide a public API. In line with the IATI technical stocktake recommendations, over the last few months the IATI Technical Team directly engaged with Validator users to understand  their requirements for developing a public validator API. Based on that initial engagement and user requirement gathering we have drafted a Public Validator API contract and we’d like your feedback.

Post your feedback by 15 March

Please review the draft Validator API Contract  and share any feedback by responding to this thread by Monday 15 March.

If you require clarification or have questions about the draft Validator API contract, the IATI Technical Team is happy to arrange a call with you. You can email the Technical Team: support@iatistandard.org directly or tag us in the thread below.

After the deadline the IATI technical team will review your feedback and will begin work  on creating a public Validator API. 

 

Comments (8)

Maaike Blom

Dear technical team, as the company who has supplied the IATI secretariat with the IATI validator I like to point out that in the original ToR the request has been to come up with a web interface for validation. We would by the way appreciate the credentials for our build being mentioned on the current IATI validator page as it was included before. We welcome the discussion on building an API that was initiated after the technical stocktake and will provide feedback on the draft Validator API contract. 

Mark Brough

Hi IATI Technical Team thanks a lot for sharing this and for the opportunity to engage! It's very welcome to have this kind of document.

So that API users can have further certainty and expectations in terms of service, perhaps it would be worth setting some other expectations in the contract, e.g.:

  • what is the target level of service uptime, resolution of major outages/bugs, etc.
  • how long will particular API routes remain available for maintained or unmaintained / "as is" (e.g. what is the deprecation notice period)

This could help potential users make decisions around potentially integrating with this tool.

I think the document itself looks great. Though I can appreciate you might want to keep the scope small to start with, here are a few thoughts on a some potential additions that I think would be very useful:

  • Should there also be a set of additional endpoints to get:
    • a list of all files (including e.g. registry ID, hash, last validation date) - presumably paginated
    • a list of files currently in the queue awaiting processing
    • a list of files by publisher - again, presumably paginated
    • a list of files by various types of critical error^: e.g. couldn't be processed at all (e.g. they wouldn't download); failed schema validation
  • I think it's really important to be able to access files by the Registry package ID (e.g. worldbank-bd rather than a hash).

(^) maybe eventually it would be nice to display files by all the various types of errors in the validator but I think this would be a good and important place to start. It's also in line with one of the findings in the feedback mechanisms report:

https://iatistandard.org/en/news/improving-iati-data-quality-feedback-l…

Andy Lulham

Echoing Mark’s comments, this sort of document is really helpful – thanks for providing it.

The report route seems okay to me. I assume `id` in the request parameters refers to the registry package ID, e.g. fcdo-af.

I have some questions about the validate route. Specifically, about the fact it is synchronous:

1. The validator’s web interface is currently asynchronous – I click “validate” and then arrive at a loading page, where I wait for my uploaded file to be processed. (The previous public validator processed files synchronously.) Is the plan to change the web interface too, so that it also processes files synchronously? (That would allow the web interface to use this public facing API.)

2. Are there potentially problems with making this route synchronous, given the validator may receive simultaneous requests to process large XML files? E.g. is there a risk that operations may time out, or the server will be overloaded? I suppose I’m wondering about the justification for making this asynchronous in the current web interface, and why those reasons might not still hold true.

3. Could making this route synchronous limit future additions to the validation report? I.e. it might be desirable to add extra validation checks in the future, but this could be problematic to do synchronously if those extra checks are computationally expensive.

4. Will an XML file size limit be imposed? If so, what will that limit be? I have a feeling there used to be a recommended maximum file size for IATI XML, but I’m not able to find it in current documentation.

Thanks in advance,

Herman van Loon

In addition to the remarks of Mark and Andy, I would suggest adding response times as a non-functional requirement in the contract. E.g. reporting API requests return the validation report with maximum 10 seconds after the request is being made, Validating requests return the results within 30 seconds after the request is being made for XML files smaller than 10Mb, within 60 seconds for files smaller than 20Mb and for larger files on best effort basis. Of course these requirement need to be realistically achievable given the current performance of the IATI data validator.

Also something should be said i.m.o. on the number of concurrent users the API should be able to handle without exceeding the response time targets.

IATI Technical Team

Thanks to everyone for engaging in the request for feedback on the  draft Validator API. This  consultation was specifically about the API contract and whether it meets user requirements, rather than the implementation of the work.  We are pleased to see that based on the comments there are no objections and agreement that the draft API contract meets current user needs. We will proceed now with the implementation of the work.

In response to some of your specific comments:

[~469] 

  • what is the target level of service uptime, resolution of major outages/bugs, etc.

These are sensible questions and we will have that level of detail at a later stage, when we develop the non-functional requirements and SLA for this. Once we get to that stage we will update the community.

  • how long will particular API routes remain available for maintained or unmaintained / "as is" (e.g. what is the deprecation notice period)

Similar to the point above,  the routes will be available and as standard practice any deprecation period will be specified in advance.

  • Though I can appreciate you might want to keep the scope small to start with, here are a few thoughts on a some potential additions that I think would be very useful

Thanks for these. There are some useful suggestions and we will be happy to engage at a later date once the initial API contract has been delivered. One point about some of the suggestions is that some endpoints are available by the Registry API, so we want to ensure there is no duplication. The Validator does pull the list of organisations/publishers by using the IATI Registry publisher id. 

  • The report route seems okay to me. I assume `id` in the request parameters refers to the registry package ID, e.g. fcdo-af.

Yes, though we will make provision for the file to be accessed by either the package ID or the hash. 

Andy Lulham

Thanks for your questions above.  They will be really helpful for the next phase of this work, which is implementation. Our Developers will be working on delivering and ensuring that it works as expected.

To your specific point about IATI file size recommendation, when organisations add their datasets to the Registry,  they can  see the below message.

“All files should be in XML following the IATI standard. Note that datasets are not hosted on this site, but by the publisher of the data. Please ensure files are less than 40MB.”

Herman van Loon

Thanks for the points above-- they will be really useful when we move towards implementation and develop non-functional requirements. We will be load testing during the implementation phase to understand and ensure adequate performance. Also, do note that this API will be delivered in the next version of the Validator, and the infrastructure and architecture will be designed to allow the API contract to be adequately implemented.  

Andy Lulham

Petya Kangalova - I raised it here because I think it relates to the sync/async API issue that I raised in my consultation response. I suspect these dashboard links will sometimes end up pointing to a page that says “the validation report is not yet available” (since the url parameter is only available on the asynchronous route, that will sometimes give a 204 response).

Personally I’d prefer either synchronous validation (with comprehensive caching) or an asynchronous route that responds with a URL to poll. Of course, it’s a technical detail, but then agreeing an API contract is a technical discussion.


Please log in or sign up to comment.