The essential goal of critical evaluation is to convey to data users, whatever their level of chemical sophistication, a well-considered and supported estimate of the consensus value based on experimental results for the quantity under consideration and of the level of uncertainty surrounding that value. Three aspects are involved in reaching this goal: compilation of experimental data from the chemical literature, evaluation of the data to establish consensus values and their uncertainty, and dissemination of the evaluated data along with information about the sources compiled and the evaluation procedures employed.
Advances in computer-based handling of scientific data are leading to new possibilities for data manipulation and interpretation. These advances also present new challenges of providing information about the data (meta-data) in formats that are assessable to both humans and computers. The delivery of data to users is presently in flux because of continuing rapid changes in electronic methods of data aggregation analysis and presentation. It is essential to make evaluated data available through channels that potential users prefer or at least will actually use.
This project will work with all Divisions, CPCDS, and any other interested parties within IUPAC toward developing the IUPAC guidance for the production and use of high-visibility evaluated data recognized widely as “Evaluated by IUPAC”.
> see also the Interdivisional Subcommittee on Critical Evaluation of Data
Critical evaluation of chemical data has long been an important activity within IUPAC. Thus the current ‘Guidelines for IUPAC Projects’ states, “The core activity of IUPAC is to provide critical evaluations of methods and data and to make recommendations for nomenclature, terminology, metrology, and measurement standards”. Members of the Analytical Chemistry Division’s Subcommittee on Solubility and Equilibrium Data, wanting to learn more about activities related to critical evaluation in other divisions and committees, organized an ad hoc interdivisional discussion of critical evaluation during the 2017 General Assembly (project 2016-043-1-500). This half-day meeting was attended by interested individuals from Divisions II, IV, and V and the Committee on Publications and Cheminformatics Data Standards. From this meeting and subsequent conversations (which have included Divisions I and VI as well as those listed above) two ideas emerged–that the exchange of information and experience among groups and individuals in IUPAC concerned with critical evaluation should continue and that an important first project would be to develop of guidance or best practices for critical evaluation of chemical data. This projecct addresses that second idea.
Currently active critical evaluation projects within IUPAC are highly varied treating both equilibrium and kinetic data, properties measured with either high and low relative uncertainty, and outputs ranging from monographs to databases. Given this range in the data sets evaluated and the rapidly evolving needs for cheminformatic documentation, guidance is needed to focus on best practices and provide flexibility while at the same time embodying high scientific standards. The Task Group that has been assembled for this project reflects the breadth of IUPAC’s experience in critical evaluation.
The Task Group will consider the three aspects of the critical evaluation process (compilation, evaluation, and dissemination) and develop best practice recommendations for each. Because of the wide variation in the data types that are evaluated, detailed protocols for the work are not considered appropriate. Rather the guidance will consider the qualities that are inherent in a good evaluation (completeness, clarity, traceability, etc.) and how those qualities can be achieved.
page last updated 18 Sep 2018