Data Quality Index Consultation
DATA COMPLETENESS
Sub-section 2.8.1 Humanitarian
Instructions for submitting your feedback
1. Read through the proposed methodology for this measure and / or download the attached PDF at the bottom of this page;
2. Share your feedback through the comment box below, consider the guiding questions in your comments and include the question number in your response;
Proposed Measures - 2.8.1 HUMANITARIAN
Please find below the proposed methodology for this measure.
DEFINITION Assess the proportion of humanitarian activities or activities that contain at least one humanitarian transaction within an organisation's publishing. Only activities published at v2.02+ will be included
REASONING This measure assesses the importance of identifying humanitarian data for our humanitarian data users. It aims to assess the extent to which IATI publishers are reporting on humanitarian activities. This is important because IATI data is recognised as one of the key platforms for accessing timely and open data on humanitarian assistance
OUTPUT |
|
METHODOLOGY |
Count number of active activities which have either:
Divided by total number of active activities. Note
|
Webinar
For each discussion, the IATI Secretariat will organise a webinar to explain the proposed methodology, answer questions and further explain how to engage.
In the ILO we think that this measure should not use percentage as OUTPUT because some of the activities will not have humanitarian flag at the activity and/or transaction levels because the relevant transactions and/or activities are NOT “humanitarian” but “development”. In our view this measure should simply assess whether the publisher uses the humanitarian flag or not to mark humanitarian activities/transactions.
We agree with Evgenia. The logic of this indicator is really not clear if the goal is to assess completeness. Not all activities are humanitarian, so why should we expect 100% (as is implicit in this measure)? It would make more sense, in fact, to compare the activities flagged as humanitarian with those that have the DAC humanitarian sector code, to check whether publishers are flagging their activities properly.
Should I read this in such a way that it says 'the more humanitarian activities, the better'? That doesn't seem right.
If the activity is already flagged as humanitarian, it shouldn't need a humanitarian flag for each transaction as well. You'd expect the activity level flag to work as a default, similar to flow type etc.
The intention in this to work the same way as the current Dashboard, to display the proportion of humanitarian activities within an organisation's data. The DQI can't derive a 'better' or 'worse' from this, but a data user/publisher may be able to and will find the information useful.
From this assessment, we can then further drill down into what humanitarian fields the humanitarian activities are using and from this we can make a statement on an organisation's humanitarian reporting.
The US agrees with existing comments from others. The current methodology seems to 'reward' publishers who have a greater percentage of humanitarian activities rather than 'rewarding' publishers with all their humanitarian activities (however many) tagged as such.
Amy - Thanks for the clarification. If this is the intention, it would certainly have to be framed very clearly as such. Still, there's a big risk here that any Index results will likely be seen or used as a better/worse score rather than a simple survey result. In some cases, that implied best practice makes sense, but in this and a few others, it may be problematic.
Please log in or sign up to comment.