Data Quality Index Consultation-

DATA COMPLETENESS

Sub-sections 2.1.1 and 2.1.2 Mandatory & Recommended


Instructions for submitting your feedback

1. Read through the proposed methodology for this measure and / or download the attached PDF at the bottom of this page; 

2. Share your feedback through the comment box below, consider the guiding questions in your comments and include the question number in your response;


Proposed Measures - 2.1.1 MANDATORY & RECOMMENDED

Please find below the proposed methodology for this measure. 

DEFINITION assess the proportion of activities that include all mandatory elements with valid data

OUTPUT

  • Percentage
  • Display percentage per mandatory element

 

METHODOLOGY 

Count of active activities containing all mandatory elements with valid data, divided by, count of all active activities

Mandatory elements include: Reporting-Organisation, IATI-identifier, Participating Organisation, Title, Description, Activity Status, Activity Date, Sector.

To assess if data is valid:

  • Reporting-Organisation: reporting-org/@ref must match publisher's reporting-org on the IATI Registry and reporting-org/narrative must have textual content
  • IATI-identifier: Activity must contain a nonempty iati-identifier (use Validator check)
  • Participating Organisation: Activity must contain at least 1 participating-org, with either reporting-org/@ref or reporting-org/narrative with textual content
  • Title: Activity must contain a non-empty title element (use Validator check)
  • Description: Activity must contain a non-empty description element (use Validator check)
  • Activity Status: Activity must contain an activity-status element with code from the activity-status codelist (use Validator codelist check)
  • Activity Date: Activity must contain at least 1 activity-date element. At least 1 activity-date element in the activity must have @type 1 or 2
  • Sector: At least one sector element present at activity level OR in all transactions. If activity level AND more than 1 per vocabulary, a percentage must be present and the combined percentages per vocabulary must add up to 100 (use Validator check)

Proposed Measures - 2.1.2 MANDATORY & RECOMMENDED

Please find below the proposed methodology for this measure. 

DEFINITION Assess the proportion of activities that include all recommended elements with valid data

OUTPUT

  • Percentage
  • Display percentage per recommended element

 

METHODOLOGY 

Count of active activities containing all recommended elements with valid data, divided by count of all active activities

Recommended elements include: Country or Region, Budget and Transaction.

To assess if data is valid:

  • Country and/or regions: Each activity must have at least country and/or region published only at activity level, or one country/region per transaction. If multiple countries/regions are published at activity level, percentages must add up to 100. If countries/regions are published at transaction level, one must be published for every transaction.
  • Budget: Activity must have at least 1 budget element with a value greater than 0 OR the activity must have the budget-not-provided attribute AND no budget elements.
  • Transaction: Activity must have at least 1 transaction element with a value greater than 0.

Guiding question - please refer to the index number when you respond via the comment box below!
  • Should activity date checks include the logic that end dates must not be in the past, and start dates must not be in the future?

GO BACK TO MAIN DQI-PAGE 

Webinar

For each discussion, the IATI Secretariat will organise a webinar to explain the proposed methodology, answer questions and further explain how to engage.

  • Please find an overview of the most frequently asked questions of the Timeliness and Validation webinar here.
  • Missed the DQI Webinar on Data Completeness held on March 30? Watch the recordings here or read the summaryhere!

Files

Comments (16)

Michelle Levesque
Michelle Levesque

In response to the question 

  • Should activity date checks include the logic that end dates must not be in the past, and start dates must not be in the future?

I don't see why this would need to be a rule.  if IATI is about providing information about future spend for planning why wouldn't we want to include project with start dates in the future?  I'm happy to hear alternative thoughts but so far, I'm not seeing what the benefit of this parameter would be.

Sarah Lotfi
Sarah Lotfi

Agree with Michelle, we wouldn't see a need to make it a rule which may create more issues, for instance, if the funding is reported as received in year 0 for activities in year 1. 

Elma Jenkins
Elma Jenkins

We have some general comments on this phase which can be found in the numbered points in the document below :  

https://docs.google.com/document/d/1BpEjcypxpHJ_SNCnscy2HZe7otEZZIYcjE7HPFdDPfI/edit?usp=sharing

As well as some indicator specific comments :

2.1.1: Reporting-Organisation, IATI-identifier, Participating Organisation:  

  • This could include anonymised data and/or non-specific names. 

2.1.1: Title, Descriptions:  

  • Will there be a minimum character count, what about titles made up of internal acronyms/codes, and nonsensical data? Or descriptions that simply repeat the title or provide no descriptive information about the activity? 

2.1.1: Activity Status, Date & Sector 

  • What about closed activities? If activities have been recently closed shouldn’t they be included in the tests, particularly for results and documents (evaluations for example)? 

2.1.2: Budgets  

  • Will the dates of the budget be checked i.e. if they are forward looking? And will this be total organisational budget or disaggregated by countries? If this is not specified they will become less standardised and less comparable.  

 

Amy Silcock
Amy Silcock

For the budgets question, under the mandatory and recommended check we're just looking for the presence of a budget element within an activity. A more detail check on budgets is included in the 2.7 financial section.

Evgenia Tyurina
Evgenia Tyurina

2.1.1: ILO agrees that for active activities the end date must not be in the past. The start date may be in the future for activities with the status "1- Pipeline/identification".

General comment: you may want to add a definition of “active activities” to the methodology for more clarity.

Yohanna  Loucheur
Yohanna Loucheur

2.1.1 We agree with the list of elements to be included. 

Regarding the specific question:

  • Should activity date checks include the logic that end dates must not be in the past, and start dates must not be in the future?

We propose this check to be broader (and to be more specific about what will be checked, to avoid ambiguity):

  • planned-start-date must be in the future
  • actual-start-date must be in the past
    • at least one of the above must be provided
  • planned-end-date must be in the future

 

Agree with Elma Jenkins' question about the quality of titles and descriptions. Simply having something in this field doesn't really help data users.  However, on budgets, given that this is about *activity budgets*, the country/region coding applies. 

Anna Whitson
Anna Whitson Moderator

It would be useful to hear from others what they think about Yohanna's proposal!:

We propose this check to be broader (and to be more specific about what will be checked, to avoid ambiguity):

  • planned-start-date must be in the future
  • actual-start-date must be in the past
    • at least one of the above must be provided
  • planned-end-date must be in the future
Michelle Levesque
Michelle Levesque

I don't know that the rules can be that strict.  For projects which have started or ended, the "planned start" date would always be in the past.  And for ended projects even the planned end date would be in the past.  I think ILO's comment would make sense.  Project which are active or in the furture have to have a future end date.  Projects can't be active if the start date is in the future.  

Amy Silcock
Amy Silcock

For the activity date discussion, the standard already includes rules about actual start and end dates not being in the future. Planned start and end can be in the future. As Yohanna Loucheur  has mentioned, I agree this should be made clearer.

Michelle Levesque  and Sarah Lotfi , given these are rules in the standard I think we should add them in to the DQI. The previous IATI Dashboard doesn't include the activity date rules, so it will hopefully drive publishers to increase the quality of their activity date reporting.

Amy Silcock
Amy Silcock

For the title and description question, if we were to check the number of characters or words in a description what would be a suitable measure for this? It could be within scope to assess that there is 'content' in the narratives, more than just non white-space. Assessing whether a narrative is meaningful depends on the data use context and is out of scope of IATI's DQI.

Michelle Levesque
Michelle Levesque

The challenge is that there is no mechanical way to systematically check for quality here, only quantity.  I thought the current dashboard already counts if the description is at least 80 characters before a publisher gets credit for having a description.    

Amy Silcock
Amy Silcock

Michelle Levesque  that's not part of the IATI Dashboard's assessment. Currently it just looks to check that the element is not empty. It is something that the DQI could do.

 

Dashboard_title_descriptionImage removed.

Sarah Scholz
Sarah Scholz

Q1: US agrees with other comments that this needs specifying whether it assesses active or planned activities, and the requirements adjusted accordingly.  As long as there are actual and planned activity start/end dates, then a rule should differentiate between the two. A rule stating that actual dates cannot be in the future makes sense.

Other: Regarding the methodology note for assessing whether country/region data is valid, “If countries/regions are published at transaction level, one must be published for every transaction.” Since this indicator counts activities, not transactions, should we assume 100% of the transactions within an activity must have country/region data to be considered a valid activity in this category? Can that be specified?

Amy Silcock
Amy Silcock

For your second point Sarah Scholz  it makes sense to me to specify more clearly that 100% of the transactions within an activity must have country/region data. This aligns with the standard rules.


Please log in or sign up to comment.