Establishment of an internationally harmonized protocol (guidelines) for the organisation and interpretation of collaborative trials for the validation of qualitative methods.
Background and problem statement
The organization of collaborative trials for the validation of analytical methods requires many important aspects to be taken into account, including the preparation of suitable test materials and the selection of an appropriate protocol for the organisation and interpretation of data from suitable collaborative trials. The protocol should be available as internationally agreed Guidelines. The Guidelines need to provide information as to be required number of participating laboratories, characteristics of the test materials to be used in the study and details on the statistical treatment of the results. It is now important for an objective assessment to be made on whether a method, once validated, is fit-for purpose and information on how this to be achieved should also be given in the Guidelines. Most quantitative analytical methods, when adopted and published as an International Standard, are now required to be validated according to the international harmonized protocol: W. Horwitz (1995), Protocol for the design, conduct and interpretation of method performance studies (Pure and Applied Chemistry 67:331-343 > access PAC content) or the ISO 5725 series of Standards. The Horwitz protocol is, however, only suitable for quantitative methods. As there is current and increasing demand and now availability of so called screening methods (qualitative or semi-quantitative), the existing protocol is not entirely suitable for inter-laboratory validation of such methods. Many problems in food control are only be handled by qualitative or semi-quantitative methods. Examples are PCR based methods for GMO detection and the ELISA based methods for food allergens. Lateral flow devices (dip sticks) have a great application for a rapid screening of food products or industrial production lines. For this reason, a bespoke protocol for screening methods, e.g. those based on immunological or DNA based technologies, is urgently needed. This project will result in suitable guidelines.
> project announcement published in Chem. Int. Mar-Apr 2006, p. 29
The first project group meeting took place in the EC JRC Headquarters in Brussels, Belgium on 20-21 February 2006. All task group members attended the meeting and after some initial presentations related to the state of the art of the qualitative method validation and the work done in this area by some of the participants, the structure of the proposed guidelines was agreed upon and the tasks distributed between the project members. A first draft is expected for 15 March 2006, the co-ordinator will then streamline the draft and completion will be done by electronic exchange.
> May 2006 report update (pdf file – 11KB)
> Since 2007 this project is working together with a corresponding working group of the European project “MoniQA” (www.moniqa.org)
> March 2009 report update (pdf file – 15KB). Some results are presented in a workshop (https://www.moniqa.org/node/263, IAM/MoniQA FST – “Method Performance and the Criteria Approach: Truth and Consequences) organised by the EU project (MoniQA) and the slides are attached(pdf file – 589KB).
> March 2011 – update status by Roy Macarthur (Fera) & Christoph von Holst (IRMM)
A protocol for analyzing the results of validation studies for qualitative methods of analysis has been produced by considering what information is needed from validation studies for qualitative methods; the features that a standard protocol for achieving this needs to have; and the recent work published in this area.The draft protocol for the validation of qualitative analytical methods can be applied to the validation of methods by collaborative trial or single-laboratory validation. The analysis of results using the draft protocol is based on the estimation of the average probability of a positive response and the observed reproducibility standard deviation. Because a single estimate of the probability of detection is produced in each laboratory (or in each group in a single laboratory study) when replicate analyses are undertaken under repeatability conditions, an analysis of variance such as that used in the IUPAC/ISO/AOAC harmonized protocol for the validation of quantitative methods is not useful. [see full text report update; pdf file – 21KB]
A full text of the study is being finalised; this report will trigger discussions/comments that will allow to finetune the validation protocol that is to appear in Pure and Applied Chemistry.
> Aug 2013: Main deliverables are (a) a publication in a peer reviewed journal, (b) the presentation of the results in an international conference organised by the MoniQa and (c) a contribution to a relevant AOAC document:
(a) Macarthur R., von Holst C.: A protocol for the validation of qualitative methods of detection. Anal. Methods, 2012, 4, 2744-2754; https://dx.doi.org/10.1039/c2ay05719k
(b) Presentation of the results at the 4th MoniQA International Conference in Budapest, Hungary, 26 February – 1 March 2013 <https://budapest2013.moniqa.org/>
(c) Contribution to the AOAC document: Guidelines for Validation of Qualitative Binary Chemistry Methods, Version 12.3 Approved March 14, 2013. AOAC International Stakeholder Panel on Alternative Methods, available from AOAC website www.aoac.org
The paper published in Anal. Methods presents a draft protocol for analyzing the results of validation studies for qualitative methods of detection which is designed to meet three competing goals: (1) to give correct answers, (2) have a broad scope of application, and (3) be accessible to a wide range of users. The draft protocol can be applied to the validation of methods by collaborative trial or to single-laboratory studies. The protocol produces an estimate of the probability of a positive response with a prediction interval within which 95% of laboratories (or analytical runs) are expected to fall when the method is applied in practice. The interval is calculated from the observed reproducibility (or within-laboratory reproducibility) standard deviation. Then a simple plot of prediction intervals for the probability of detection against the concentration of analyte, where the concentration is known, is used to provide an estimate of the range of limits of detection and false positive probability that we can expect to see when the validated method is applied in practice. The use of the draft protocol is demonstrated using results produced by three collaborative trials. A simulation study showed that a conclusion that a method is fit for purpose that is generated by the draft protocol is likely to be safe.
Last updated 6 Sep 2013