The essential goal of critical evaluation is to convey to data users, whatever their level of chemical sophistication, a well-considered and supported estimate of the consensus value based on experimental results for the quantity under consideration and of the level of uncertainty surrounding that value. Three aspects are involved in reaching this goal: compilation of experimental data from the chemical literature, evaluation of the data to establish consensus values and their uncertainty, and dissemination of the evaluated data along with information about the sources compiled and the evaluation procedures employed.
Advances in computer-based handling of scientific data are leading to new possibilities for data manipulation and interpretation. These advances also present new challenges of providing information about the data (meta-data) in formats that are assessable to both humans and computers. The delivery of data to users is presently in flux because of continuing rapid changes in electronic methods of data aggregation analysis and presentation. It is essential to make evaluated data available through channels that potential users prefer or at least will actually use.
This project will work with all Divisions, CPCDS, and any other interested parties within IUPAC toward developing the IUPAC guidance for the production and use of high-visibility evaluated data recognized widely as “Evaluated by IUPAC”.
> see also the Interdivisional Subcommittee on Critical Evaluation of Data
Critical evaluation of chemical data has long been an important activity within IUPAC. Thus the current ‘Guidelines for IUPAC Projects’ states, “The core activity of IUPAC is to provide critical evaluations of methods and data and to make recommendations for nomenclature, terminology, metrology, and measurement standards”. Members of the Analytical Chemistry Division’s Subcommittee on Solubility and Equilibrium Data, wanting to learn more about activities related to critical evaluation in other divisions and committees, organized an ad hoc interdivisional discussion of critical evaluation during the 2017 General Assembly (project 2016-043-1-500). This half-day meeting was attended by interested individuals from Divisions II, IV, and V and the Committee on Publications and Cheminformatics Data Standards. From this meeting and subsequent conversations (which have included Divisions I and VI as well as those listed above) two ideas emerged–that the exchange of information and experience among groups and individuals in IUPAC concerned with critical evaluation should continue and that an important first project would be to develop of guidance or best practices for critical evaluation of chemical data. This projecct addresses that second idea.
Currently active critical evaluation projects within IUPAC are highly varied treating both equilibrium and kinetic data, properties measured with either high and low relative uncertainty, and outputs ranging from monographs to databases. Given this range in the data sets evaluated and the rapidly evolving needs for cheminformatic documentation, guidance is needed to focus on best practices and provide flexibility while at the same time embodying high scientific standards. The Task Group that has been assembled for this project reflects the breadth of IUPAC’s experience in critical evaluation.
The Task Group will consider the three aspects of the critical evaluation process (compilation, evaluation, and dissemination) and develop best practice recommendations for each. Because of the wide variation in the data types that are evaluated, detailed protocols for the work are not considered appropriate. Rather the guidance will consider the qualities that are inherent in a good evaluation (completeness, clarity, traceability, etc.) and how those qualities can be achieved.
New project announced in Chem. Int. Jan 2019, p. 40
April 2020 update – First meeting of the TG was held during the Paris GA, July 12019. The scope of the work was discussed and writing assignments were distributed among TGMs with a target date of November 2019 for assembly of a first draft. The target was met and a draft is being reviewed by the task group. Anyone interested in the details of this project is welcome to review the current draft by contacting D Shaw for access details. A teleconference will be scheduled in the spring to discuss gaps in the draft, plans for completion, etc. Progress is satisfactory and on schedule for publication of a Technical Report in PAC.
April 2021 update – The scope of the project has expanded during the last 12 months. The group originally proposed to prepare a single Technical Report but it soon became evident the adequate treatment of subject will be best served by a multi-paper approach. The present approach envisions six TRs of which three are in active preparation. The latter include 1) a general overview of the principles of data evaluation, 2) a treatment of estimation of measurement uncertainty for data compiled from the chemical literature and 3) a glossary of terms used in critical evaluation of chemical data. The work is not proceeding rapidly because various groups have approached critical evaluation in different ways and this has lead extensive exchanges of views. Because a major subtext of this project has been to facilitate exchange of perspectives among evaluation projects, this is not considered a problem at the present time. However, it could become a problem, if progress slows to a standstill and members become discouraged.
Page last updated 17 April 2021