Some of us (OK, Andy Lulham !) noticed that the DAC CRS codelists were updated by the OECD on May 9th.
It’s great that we have something from the community (also Andy Lulham !) in place to try and track this.
However, it would be great to know when, what and how these codelists will be moved by the OECD-DAC to a platform that ensures scalability, version control and machine legibility. I think there’s an upcoming WP-STAT meeting - can anyone inform us (maybe Ole Jacob (OJ) Hjøllund or Herman van Loon ?)?
A while back, I posted something about characteristics we might want to see from an external codelist. Perhaps we could start to list and detail what we’d expect from CRS DAC codelists in the IATI community, given that we are so reliant on their effective management?
It’s worth explicitly noting that this is not limited to the past, but in fact also applies to the present! The date-last-modified for the XML version says Jan 2016, and a quick eyeball suggests this is probably accurate. I believe Steven Flower has previously called for the link to the XML version to be removed from the OECD site, at least until it points to something up-to-date.
That said… It seems to me there’s a bit of complexity here, but the machine readable bit (i.e. XLS->XML) is the least of the problems. We know from experience that there’s XML… And then there’s “XML”. If IATI were to work with the XLS codelists that the DAC publishes right now, and help iron out issues that occur with those, it could help better inform the DAC about the requirements users (e.g. IATI) have.
Of course it would be useful if the DAC produced and maintained machine-readable codelists… But IATI would still need to (at least for a period) maintain some infrastructure to e.g. track changes (as Dale Potter mentions). Most of that can be set up right now (in fact, it’s exactly what https://andylolz.github.io/dac-crs-codes/ does). I don’t believe that would undermine efforts to get the DAC producing machine-readable codelists. IATI non-embedded codelists being months out-of-date is (in my opinion) largely independent of the DAC not publishing machine-readable codelists.
Thanks for these comments Andy Lulham
Following our face-to-face discussions with the OECD DAC in May, we have an open channel for dialogue and are next due to speak with our contact there this coming Friday. We will seek to find out about the status of the current XML version of their codelists that you mentioned. We will also be finding out more about how they are progressing with plans to automate the release of fully machine-readable XML, and will report back here. Aside from the issues that I mentioned in the above post and raised by Rory Scott here , do let us know if there are other usability issues that you would like us to highlight in our ongoing dialogue?
We accept that some recent changes to non-embedded codelists have taken some time to push through (example), often where there have been queries about recycled codes that may impact the meaning of published data. We have been discussing our protocols for this amongst the IATI Technical Team and have been looking to find better ways to handle these.
Pending full agreement within the team (and corresponding clarification on the IATI Codelist Management page), we are moving towards a situation where replicated non-embedded codelists (i.e. those from the OECD DAC, ISO, IANA and the US National Geospatial-Intelligence Agency) are agreed to be replicated ‘as is’, after at least 7 days notice has been given on the Non-embedded codelists category. We will also be seeking to include withdrawn codes as part of the source codelists, using the status, activation-date and withdrawal-date attributes.
Forgive me, but this strikes as creating further complexity. It’s no doubt appreciated that the IATI tech team are talking to the OECD DAC team, but surely the OECD DAC representatives can signal their intentions and plans in an open and public channel, for us all to engage around? It’s 2017 - we have the internet.
Apologies - I do not wish to sound flippant, but I do think this issue could be dealt with if the party controlling and managing these codes could interact with the community that (in)directly use them.
We were indeed planning to encourage the OECD DAC to do this in our meeting tomorrow, with the person responsible for managing their codelists. Unless we plan to find an alternative source for the codelists currently provided by the OECD DAC, and given that we are a separate entity, ultimately all we can do share our experiences and offer encouragement.