Discussion

Data Quality Index - Section: Data Availability

IATI Secretariat • 1 September 2021
 Data Quality Index Consultation - sub-section Data Availability

Instructions for submitting your feedback
  1. Access each section by clicking on the buttons below, or download the attached full report.
  2. Share your feedback on the Data Availability sub-sections of your choice through the comment-box below.
  3. Consider the guiding questions for each part, and please refer to the indicated numbers in your comments.

3.2 Comprehensiveness - change to Data Availability

  • Do you agree with the proposed additions and changes to the measure of what comprehensive data looks like? 
  • Do you agree with the proposed change to call this measure Data Availability? Can you think of a better naming if you disagree?
  • Do you agree with the categorisations? If not, what would you propose? Have we missed a measure of data availability that is crucial for measuring data quality?

3.3 Data Availability - Mandatory

  • Mandatory: Do you agree that mandatory and recommended elements should be assessed under this category?

3.4 Data Availability - Location

  • Location: Do you agree with the proposed assessment of country/region/ recipient language?

3.5 Data Availability - Classification

  • Classification: Do you agree with all the proposed measures and elements under this category? Should they all remain or are there some you think are not relevant for all users? Do you agree that under policy markers only gender should be assessed?

3.6 Data Availability - SDGs

  • Do you agree that SDGs should be included as a separate measure?

3.7 Data Availability - identifiers and traceability

  • Do you think that we have captured the relevant traceability measures? Are there any that are missing?

3.8 Data Availability - Financial

  • Do you agree that both transactions and budgets should fall under financial? Previously budgets were assessed separately under forward looking?
  • Do you agree with the transaction types we want to look at and assess? 
  • Do you agree we should also include total budgets in the organisation file? Is assessing total budgets for next three years a realistic measure?

3.9 Data Availability - Humanitarian Data

  • Do you agree with the assessment areas included under humanitarian?

3.10 Data Availability - Results


3.11 Data Availability - Documents

  • Do you agree that a separate assessment  for results and documents should be included?

3.13 Data Availability - Coverage

  • Do you agree that total expenditure should be used?

BACK TO THE MAIN DQI-PAGE

Files

Comments (22)

Herman van Loon

3.2 Comprehensiviness

  • Do you agree with the proposed additions and changes to the measure of what comprehensive data looks like?

    Partly. I.m.o. the role the publisher has in the network should be taken into account. E.g. for locations, a bilateral donor channeling funds through iNGO's and multilaterals, is not likely to know where the activity will take place and so will be unable to publish a lot of location data. For an NGO, it is unlikely that the aid-type or flow-type is very relevant, since these are DAC elements primarily focused on bilateral donors. They should not be penalized for not publishing those elements.
     
  • Do you agree with the proposed change to call this measure Data Availability? Can you think of a better naming if you disagree?

    No, it is a bit confusing because it can also be read as coverage. Basically it is about the 'Completeness of published fields'. Coverage is a separate item. 
     
  • Do you agree with the categorisations? If not, what would you propose? Have we missed a measure of data availability that is crucial for measuring data quality?

    Yes
Herman van Loon

3.3 Data availability: mandatory

  • Mandatory: Do you agree that mandatory and recommended elements should be assessed under this category?

    Maybe. Shouldn't a distinction be made between mandatory and recommended elements? Otherwise you could call the recommended elements also mandatory, which is not the case. Of course we could also decide that a recommended element should be mandatory. That would require an integer update to the standard though.
Herman van Loon

3.4 Data availability - location

  • Location: Do you agree with the proposed assessment of country/region/ recipient language?

    No, since there is no distinction being made between types of publishers. Especially for location this is problematic. E.g. most (aprox. 80%) of the Netherlands aid, as a bilateral donor, is channeled through multilaterals and iNGO's or donor country  based NGO's. No specific locations where the aid will flow to. That is up to decide by the multilaterals and NGO's. So location information (geo-points) can not be published.

    On the other hand, if you are a local NGO you almost certainly will know where the activity takes place and than you can publish location information (provided there are no security concerns of course).
Herman van Loon

3.5 Data availability - classifications

  • Classification: Do you agree with all the proposed measures and elements under this category? Should they all remain or are there some you think are not relevant for all users? Do you agree that under policy markers only gender should be assessed?

    Partly.

    Again the type of publisher you are, should be taken into consideration. E.g. Aid-type, Finance-type, Flow-type and collaboration type are very OECD/DAC centric classifications, mostly relevant for bilateral donors and multilaterals, but nor relevant at all for NGO's, recipient governments and private sector. 

    Sector and policy markers can be relevant for all publishers.

    The disbursement channel also looks only relevant for bilateral donor's.
Herman van Loon

3.6 Data availability - SDG's

  • Do you agree that SDGs should be included as a separate measure?

    Yes
Herman van Loon

3.7 - Identifiers and traceability 

  • Do you think that we have captured the relevant traceability measures? Are there any that are missing?

    Partly.

    It is important when referring to the funding activity-id in an incoming fund or incoming commitment transaction, that also the validity of that activity id is checked (it must be an existing activity-id).

    When referring to an recipient organization in an outgoing commitment or disbursement transaction, it is often not possible to publish the activity-id of the recipient, because that activity might not yet exist in the activity administration of the recipient. 

    Also the type of publisher should be taken into account: for a bilateral donor there are no incoming funds or disbursements (because it is tax payers money being spend), with the exception of delegated cooperation.

    For all other publishers incoming funds or disbursements are relevant to publish, with the exception of the organizations responsible for actually doing the work on the ground. They will only have expenditures.
Herman van Loon

3.8 Data availability - Financial

  • Do you agree that both transactions and budgets should fall under financial? Previously budgets were assessed separately under forward looking?

    No, budgets are typically forward looking and it is unclear for me what problem is being solved by lumping budget together with financial.
     
  • Do you agree with the transaction types we want to look at and assess?
    In the text I see that you want to distinguish between incoming and outgoing transactions. That is ok. I do dot see any further references to specific transaction types.
     
  • Do you agree we should also include total budgets in the organization file? Is assessing total budgets for next three years a realistic measure?

    I am not sure on the organization level since multi year budgets are only indicative. Usually reliable budgets are only defined one year ahead.
Herman van Loon

3.9 Data availability - Humanitarian

  • Do you agree with the assessment areas included under humanitarian?

    Yes, with the exception of localization. When channeling humanitarian funds through multilaterals, iNGO's or donor country based NGO's, often the local actors are not known yet and so it is impossible to publish this information. The solution here is to make use of the traceability of activities between publishers: further down the funding chain, it is far more likely that local actors will be known. 

 

Herman van Loon

3.11. Data Availability - Documents

  • Do you agree that a separate assessment  for results and documents should be included?

    Yes
Herman van Loon

3.13 Data Availability - Coverage

  • Do you agree that total expenditure should be used?

    Yes
leo stolk

3.2 Comprehensiveness - change to Data availability

  • Do you agree with the proposed additions and changes to the measure of what comprehensive data looks like? 

in general I agree, scoring/weight could be adjusted per role of publisher in a trail or network. Risk is to make this too complex, and for instance distinguish between two type of roles, predominantly funding, or predominantly implementing/extending

  • Do you agree with the proposed change to call this measure Data Availability? Can you think of a better naming if you disagree?

Suggest Data Completeness rather than availability. Completeness of content per activity/record enables optimalisation of its data use. 

  • Do you agree with the categorisations? If not, what would you propose? Have we missed a measure of data availability that is crucial for measuring data quality?

Suggest to make the inclusion of accurate granular information of implementing actor(s) (national/local etc) as participating organization, an area of assessment. The last mile, is often missing or hidden, even in no-risk contexts. Excluding those who actually achieve the change. 

leo stolk

3.3 Mandatory: Do you agree that mandatory and recommended elements should be assessed under this category?

- Yes, but using the element is not sufficient to be complete and compliant to mandatory use. Check should be how those mandatory elements are used. For instance a publisher can declare itself as accountable participating org, so used the element, but not disclosing details of funding or implementing participating org. 

leo stolk

3.4 Location: Do you agree with the proposed assessment of country/region/ recipient language?

i agree with country region and sub national location details. Have serious doubts on the check of recipient country language,
Which larger publishers will be able to publish quality information in all official languages of the countries they work in?  For instance Oxfam would need to be able to publish in Spanish, French, Pashto, Arabic, Dari, Bahasa indonesia, Portugueese, Norwegian, Japanese etc etc. I expect that this will likely affect data quality and thus data use. 
Suggest instead to check each publisher's consistent use of its chosen working languages in its data set, that will probably contribute to data quality. 
Also this proposal is not clear will it be checked at each activity language <-> recipient country?    

leo stolk

3.5 Classification: Do you agree with all the proposed measures and elements under this category? Should they all remain or are there some you think are not relevant for all users?
Yes they should remain, but main weight should be given to SECTOR, additional points for the use of other classifications

Do you agree that under policy markers only gender should be assessed?

Not really,  i think it is important to check the use of gender policy marker, but again other policy marker like climate change mitigation and adaptation are growingly important too. 

leo stolk

3.6 Do you agree that SDGs should be included as a separate measure?

Yes at least at goal level.  below goal level, targets and indicators is way more problematic and poorly fitting the NGO sphere. 

leo stolk

3.7 Do you think that we have captured the relevant traceability measures? Are there any that are missing?

I disagree with the receiver org activity ID, as very often they are not available when starting publishing upstream. Also often receiver organisations of INGOs do not publish

leo stolk

3.8 data availability financial 

  • Do you agree that both transactions and budgets should fall under financial? Previously budgets were assessed separately under forward looking?

No problem, not all organization can declare next years budgets that easy. Longer term funding mechanisms are often the exception. 

  • Do you agree with the transaction types we want to look at and assess? 

yes

  • Do you agree we should also include total budgets in the organisation file? Is assessing total budgets for next three years a realistic measure?

yes, most organisations have a multiyear planning and budget allbeit indicative and not tight to recipient countries and regions. 

leo stolk

3.9  Do you agree with the assessment areas included under humanitarian?

yes fully. 

leo stolk

3.10 results
agree with proposed simple measure

3.11 Documents
 

  • Do you agree that a separate assessment  for results and documents should be included?

Yes.  

3.13 Coverage
Suggest to use total expenditure on mission goals.  INGOs also spend on fund raising and domestic campaigns, suggest to exclude when defining total expenditure.

Yohanna Loucheur

This is a loooong list of questions - perhaps it should have been further divided into sections? 

Agree with others re Data Completeness rather than Data Availability.

Agree that list of elements under Classification is too long and not adapted/reflective of all publishers. Index should use principle of parsimony and focus on core elements. Suggest removing from *Index* (not Standard!!): Finance Type, Flow Type, Collaboration type, Disbursement Channel. 

Also suggest to remove "Localisation" under Humanitarian Data, as it is duplicative with the Identifiers and Traceability section. 

Agree to include Policy Markers (especially GE) and SDGs (as long as use of SDG Indicator is not mandatory).

Yohanna Loucheur

3.4 Location

Agree with others that bilateral donors often lack details regarding the specific geographic location of projects. At the very least, this assessment should exclude certain types of aid/receiving organisations (e.g. multilaterals). 

Regarding sub-national locations, we should not be expecting both geographic coordinates AND location details - it would be duplicative, especially where a gazeteer ID is used (which should be the preferred option).

Recipient language: putting this under Location may cause confusion. Suggest rather to move it to the "Mandatory" section (which already contains non-mandatory elements), as it relates to titles and descriptions. 

3.7 Identifiers and Traceability

Agree with the importance of using correct organisation identifiers. However, the IATI Registry is definitely *not* the right reference list for participating organisations - many implementing organisations are not on the Registry. We've been talking about this for years and, as if I'm not mistaken, the best approach we've come to so far is to check that Org Identifiers are well formed, using org-id.guide as a reference. 

Agree with others that an activity identifier can be provided for incoming funds but rarely for outgoing funds - this should be reflected in the index.

Want to flag that Participating Organisation is already in the Mandatory section of the Index; perhaps it would make sense to have the role and type there as well? 

 

3.8 Financial

Unsure... I don't quite understand what is proposed by splitting transaction types and assessing them, and assessing whether organisations have reported spend. 

Also, the *number* of activities with budgets for the next tow years compared with the *number* the activities at the start of each year - what does that tell us? How does it relate to the total budget for the next three years? 

Marie-Line Simon

On location, financial, comprehensiveness

Publishing data concerning the organizations’ actions, and especially on IATI, should not give rise to risks for the people contributing to these actions or for these organizations’ leaders and workers. As we support human-rights-based interventions, we are particularly keen to ensure the protection of data based on the concrete experience of partner organizations struggling to defend socio-economic and human rights in their countries and at international level. These are thus regularly considered as obstacles by the economic and political actors, and are therefore exposed, in their lives and work, to risks of being persecuted, threatened or blocked in their functioning. Above all, the publication of specific budgets in relation to specific organizations represents a risk. This
information could allow the authorities to financially dispossess these social movements by blocking their bank accounts or resources (see the situation in Romania several years ago) or by completely preventing them from functioning (a real risk, for example, for trade unions in Cambodia).
Location is part of what we consider sensitive information. So I would recommend taking this into account in your reflection: anonymised data is not necessarily out of lack of accuracy.


Please log in or sign up to comment.