Discussion

DQI Consultation: 1.2.2 Timeliness - Timelag (spend transactions)

IATI Secretariat
IATI Secretariat • 15 February 2022
6 comments

Data Quality Index - sub-section 1.2.2 Timeliness - Timelag (spend transactions)

Instructions for submitting your feedback 

1. Read through the proposed methodology for this measure and / or download the PDF at the bottom of this page;  

2. Share your feedback through the comment box below, consider the guiding questions in your comments and include the question number in your response;

3. And finally you can suggest track-changes or add comments directly on the specifics of the methodology of the Timeliness and Validation measures - go to this DQI Live-Editing-Page

Proposed Measures - 1.2.2 Timeliness - TIMELAG - SPEND TRANSACTIONS

Please find below the proposed methodology for this measure. Only active activities* will be assessed in the Timeliness measures.  

*Active activities refer to activities which have an actual-start-date in the past and an actual-end-date in the future. If actual start and end dates are not present, the planned start and end dates will be used

DEFINITION: assess how recent a publisher's spend transaction data is at the point it is published

For instance, a publisher may refresh their data monthly, but the refreshed data is in fact three months old. Alternatively a publisher may refresh their data only once a year, but when they do it contains current data that is less than one month out of date.

OBJECTIVE / EXPLANATION

  • Data quality objective: When data is updated, it contains recent spend data.

  • Based on the methodology, this is measuring whether the data a publisher has published contains spend transactions (either disbursements or expenditures) that have occurred recently.

  • As such, this will motivate publishers to ensure that their updated data contains spend transactions with recent transaction dates.

  • Bigger picture, the goal is to motivate publishers to ensure that they when they update their data, that they are including data that is as recent as possible.

OUTPUT

Categorisation of how many months in arrears the data is:

  • One month in arrears;
  • One quarter in arrears;
  • Six months in arrears;
  • One year in arrears;
  • More than one year in arrears;

Note: visualisation to include the month the data was last updated in.

 

METHODOLOGY

Count the number of spend transactions that took place in the last 12 months.

Collect the last update date.

Calculate arrears assessment by comparing last update date to transaction dates:

  • One month in arrears = spend transactions with transaction dates for at least 2 of the last 3 months;
  • Quarter in arrears = spend transactions with transaction dates for at least 1 of the last 3 months;
  • Six months in arrears = spend transactions with transaction dates for at least 1 of the last 6 months;
  • One year in arrears = spend transactions with transaction dates for at least 1 of the last 12 months;
  • More than one year in arrears = no spend transactions with transaction dates in the last 12 months.

Note: future transaction dates will be discounted from the measure as they contradict IATI rules.

Please find below a visualisation for this proposed measure. Do note that this has been created to help participants picture what the DQI could look like. It is not final, nor part of the proposed methodology:

Guiding questions - please refer to the question number when you respond via the comment box below!
1. Do you have any suggestions for a more intuitive name for this measure instead of Timelag? Perhaps 'Recency'?
2. How many and which arrears categories should be used?
  • Would it be useful to capture arrears of more than 1 year, e.g., more than two years in arrears?
3. How many updates should be required to qualify for each category?
  • Do you agree with the number of updates needed per category? If not, how would you change them?
  • Should a publisher get credit for having data one month in arrears when they have achieved this in only 2 of the past 3 months or should 3 months be required?
4. Do you have any suggestions on how this measure could be visualised?
5. Should publishers get credit for publishing transactions with a transaction value of 0?
Webinar

For each discussion, the IATI Secretariat will organise a webinar to explain the proposed methodology, answer questions and further explain how to engage.

  • Please find an overview of the most frequently asked questions of the Timeliness and Validation webinar here.
  • Missed the DQI Webinar on Data Completeness held on March 30? Watch the recordings here or read the summaryhere!

BACK TO MAIN DQI-PAGE

Files

Comments (6)

Anna Whitson
Anna Whitson • 2 years ago Moderator

Dear members of the IATI community,

As moderator of this consultation on the forthcoming IATI Data Quality Index, a warm welcome to you all! Thank you in advance for your inputs, which will no doubt provide invaluable as we work toward a DQI that supports our publishers to better understand and improve the quality of the data they publish. On behalf of the Secretariat, again, welcome!

-Anna Whitson; Outreach, Partnerships and Engagement Specialist, IATI Secretariat

Yohanna Loucheur
Yohanna Loucheur • 2 years ago

I provided comments in the paper itself, as there is a lot here - flagging for others who may wish to respond. 

The proposed Timelag measures are based on an implicit business model of activities (programs, projects) with multiple transactions. That is not always the case: some publishers may have only a few major grants, with 1 payment per year. It's not the majority model, but it does exist, so we should be careful that the DQI does not penalize legitimate models. 

The proposed indicators are quite duplicative: timelag in all transactions, then timelag in spend transactions, then spend updates. If we must keep a timelag measure (the value of which is unclear to us, but we're willing to be convinced), we suggest to focus on the timelag in spend transactions as a proxy for overall timeless. The spend updates measure is quite problematic and should definitely be removed. 

Anna Whitson
Anna Whitson • 2 years ago Moderator

Thanks very much for your comments, Yohanna, and again - a sincere apology for the delay in reponse here due to a technical error on my part. I am flagging for my colleagues Amy Silcock and Sarah McDuff your comments so that they can provide an appropriate response. Thank you!

leo stolk
leo stolk • 2 years ago

Agree with Yohanna, more mature partnerships can reflect in less and larger transactions. Same for local humanitarian leadership objective, instead of multiple expenditures due to own implementation, this is characterized by larger / less transfers to local partners. 

Sarah Scholz
Sarah Scholz • 2 years ago

Agree that these seem unnecessarily duplicative. This could be adjusted for in the weighting portion of this exercise, but I'm not sure if there's sufficient added benefit in considering these separately to justify the time/confusion costs, though there may be something I'm not thinking of.

Amy Silcock
Amy Silcock • 2 years ago

Thanks to everyone for their contributions here. With my data use hat on:  the availability of more real time data in IATI (compared to other sources like OECD) is where considerable value is derived by data users. However, data users continually face challenges in knowing how complete or current the data published is. In addition, spend data, based on considerable user research and our engagement with data users, is essential for many of IATI’s data users. As such, this measure (and the Spend Updates measure (1.3)) are intended to help assess the timeliness and predictability of the provision of spend data. It's well understood that organizations may not have spend every month and the objective is not to penalize organizations for their disbursement schedules but rather to enable data users to be able to understand how current the spend data is (at any point in time – to build understanding/trust/confident in the data they are analyzing) and whether this information is provided in a timely fashion (e.g. how long after the end of the calendar year has spend been provided for the last quarter). The measure simply provides this information; how this measure is weighted will determine how publishers are actually assessed on this measure and this will be determined in phase 3 of this consultation where we very much welcome your inputs on how we can ensure a useful measure that is fairly assessing publishers timeliness in the provision of spend data.

Please log in or sign up to comment.