Data Quality Index Consultation - sub-section 3.2 Validation - Progress


Instructions for submitting your feedback 

1. Read through the proposed methodology for this measure and / or download the PDF at the bottom of this page;

2. Share your feedback through the comment box below, consider the guiding questions in your comments and include the question number in your response;

3. And finally you can suggest track-changes or add comments directly on the specifics of the methodology of the Timeliness and Validation measures - go to this DQI Live-Editing-Page


Proposed Measure - 3.2 Validation - PROGRESS: critical error, warning types

Please find below the proposed methodology for this measure.

DEFINITION show the changes in each publisher's validation report over time.

OBJECTIVE AND EXPLANATION 

  • Data quality objective: reduce the number of critical, error and warning types.
  • Show trend over time to motivate publishers to want to show improvement.

OUTPUT

  • Line graph.

 

METHODOLOGY

  • Use the Validator API to pull the count of critical, error and warning types per publisher.
  • Record the per week sum of critical, error and warning types per publisher.
  • Plot these on a graph for the past 12 months.

Please find below a visualisation for this proposed measure. Do note that this has been created to help participants picture what the DQI could look like. It is not final, nor part of the proposed methodology. 


Guiding questions - please refer to the question number when you respond via the comment box below!

1. What frequency should the graph show: per week, per month or other time period?

2. Should the graph show count of critical, error and warning types, or percentage change?

3. There are a set numbers of errors and warning types in the IATI Validator. As such, should this measure show the number of 'types' of errors / warnings; the occurrences of messages; or both?

  • If number of types, the number of occurrences is ignored.
  • If count of messages, the number of activities and occurrences is captured.

Webinar

For each discussion, the IATI Secretariat will organise a webinar to explain the proposed methodology, answer questions and further explain how to engage.

  • Please find an overview of the most frequently asked questions of the Timeliness and Validation webinar here.
  • Missed the DQI Webinar on Data Completeness held on March 30? Watch the recordings here or read the summary here!

BACK TO MAIN DQI-PAGE

Files

Comments (2)

Anna Whitson
Anna Whitson Moderator

Dear members of the IATI community,

As moderator of this consultation on the forthcoming IATI Data Quality Index, a warm welcome to you all! Thank you in advance for your inputs, which will no doubt provide invaluable as we work toward a DQI that supports our publishers to better understand and improve the quality of the data they publish. On behalf of the Secretariat, again, welcome!

-Anna Whitson; Outreach, Partnerships and Engagement Specialist, IATI Secretariat

Sarah Scholz
Sarah Scholz

Q1: The sample graph chart seems sufficient - organized by month but weekly detail is included in the line - seems the team found a good way to provide detail without over burdening the visualization. 

Q2: Percent change would allow for greater visual consistency across publishers. The temptation will be to compare publishers’ charts, and different scales would be misleading.


Please log in or sign up to comment.