Dear IATI Community-members,

We are pleased to share the Background Paper for Discussion #1 of the Data Quality Index Consultation!

Following the structure of the Background Paper we have proposed specific questions under each proposed measure that we would like to get your feedback on. Questions have been split into four specific subsections, including an overall broader section giving people an opportunity to comment and add ideas for any questions we might not have asked. 

Click on the buttons to access each of the four subsections, check out the guiding questions and share your views!

Timeliness

Data Availability

New Measures

Overall Questions

We would like to give all our users the opportunity to feedback on the proposed measures by responding directly on Connect by Wednesday 6 October (extended from Thursday 30 September 2021). You don't have to respond to all questions to get involved in the Consultation and we do welcome feedback from all our stakeholders. General comments you can share via the comment box below.

Following the deadline we will be providing a summary of your comments that we can all discuss at the Virtual Community Exchange (12 & 13 October) and during a webinar on Wednesday 20 October 2pm UK time (register here).

In case you have any additional questions related to the Consultation, please contact us via support@iatistandard.org

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

We want to thank everyone who participated in the IATI Data Quality Index consultation by commenting on the Background Paper over September and October 2021:

We have listened to the community input and have provided a follow-up summary from Discussion 1  with a way forward for each proposed measure below.

Next steps: We want to allow enough time for the development of the methodology, incorporating feedback received in discussion 1. The IATI Data Quality Index technical methodology consultation will launch in mid-January 2022, including a number of focused group sessions for discussion in February 2022 (dates tbc). Please take a look at the revised timeline.



If you have any questions feel free to get in touch by responding to this thread or emailing support@iatistandard.org

Files

Comments (6)

Amy Silcock
Amy Silcock

Hi all, echoing Petya’s comments thank you for engaging. Below are some key points coming out of the feedback so far, we will update this again on the 6th October.

SUMMARY

- Support that the DQI metrics focus on active activities, not closed ones. For example, timeliness calculations only look at activities which updates are expected in, publishers won’t be penalised for having ‘closed’ activities.

- Overall support for the metrics contained with ‘Data availability’, suggestion to change the name to ‘Data completeness’.

ONGOING DISCUSSIONS

Further methodology considerations are needed on many metrics to:

- Ensure that the same metric doesn’t appear twice in the index

- Ensure data can be anonymised/sanitised without publishers being penalised

- Decide which metrics are suitable for which groups of publishers e.g. DAC publishers, humanitarian, NGOs

- Decide how to capture context dependent data e.g. including partner organisation’s iati-identifiers as this depends on whether or not that partner is publishing.

- Discussion on how/if a measure of ‘trust’ should be developed.

Further details have been added to each DQI sub-page.

Melinda Cuzner
Melinda Cuzner

Bringing in some comments from me and my team:

We look forward to having an Index/dashboard that is inclusive for all publishers. The quality measure look reasonable, but are they still orientated to OECD publish norms? I was hoping to learn here how other types of organizations would answer here. I would expect that the methodology would be adapted to the type of organization/role in the aid chain. As an example, it is reasonable that a local NGO reports incoming funds, but not bilateral donors. This might be found in the more detailed methodology with primary/secondary source. But still, what is the initial reaction of other types of publishers?

More specific comments:
Data quality: Using the term Data Quality Index may be misleading as it mostly is a measure of data availability. Availability does not guarantee quality. It can look great but be nonsense information.

The purpose as it is described in the paper ensures availability rather than the content holding good quality.

Trust: The measure needs to be a balance between transparency, sensitive data and data quality. Currently, the definition of trust is very narrow. As it stands redacting sensitive data would give negative points.

The word Trust points once again to the impression that the data is of good quality, not just available.

Also, the time frame needs to be considered here, is it changes withing the last year? It will be difficult for "good publishers" to get points here, and if it is time it takes to correct data it requires open feedback loops and the requests for improvements have to be reasonable, and somehow objectively measured.

Melinda Cuzner
Melinda Cuzner

And for those who. like me, missed to read the instructions properly, you can see more comments under each subsection by pressing the yellow buttons up top :-)

Petya Kangalova
Petya Kangalova

Hi everyone! Thank you very much to everyone who have engaged and fed back already above and in each specific section of the consultation! We will be providing a summary and response to your comments tomorrow in this thread.

We have decided to extend the deadline for written comments to COB Wednesday 6 October to give an opportunity to more organisations to get involved! We will then all discuss your feedback at the Virtual Community Exchange (12 & 13 October) and during a webinar on Wednesday 20 October 2pm UK time .

Petya Kangalova
Petya Kangalova

We want to thank everyone who participated in the IATI Data Quality Index consultation by commenting on the Background Paper over September and October 2021:
- IATI Connect - online written consultation
-VCE 2 session on 13 October
- Data Quality Index webinar on 20 October

We have listened to the community input and have provided a follow-up summary from Discussion 1 with a way forward for each proposed measure: https://drive.google.com/file/d/1HdFOly5HwgUAT9qJ3pZ7qI47BDcQ06fM/view

Next steps: Consultation on the technical methodology will be launched in mid-January: https://iaticonnect.org/Consultation-Data-Quality-Index#comment-10064

If you have any questions feel free to get in touch by responding to this thread or emailing support@iatistandard.org


Please log in or sign up to comment.