What is "reliability" in the context of a "quantitative analysis"?
What does "reliability" mean in measurement science?
We have a definition of INSTRUMENTAL "reliability" from Hodgson (Hodgson, Brian T., "The Validity of Evidential Breath testing", Can. Soc. Forensic Sci. J. Vol. 41 No 2 (2008) pp. 83-96) in the paper cited by the SCC in R. v. St‑Onge Lamoureux, 2012 SCC 57 (CanLII), [2012] 3 SCR 187, <https://canlii.ca/t/ftl1g>, but:
Are there additional components to instrumental "reliability" other than drift in accuracy and precision over time?
Are there additional components to "reliability" of a quantitative analysis measurement result other than instrumental reliability?
Are there, for example, human factors, that influence reliability of a measurement result, notwithstanding automation?
Is there, for example, a traceability factor, that influences reliability of a measurement result, notwithstanding automation?
Purpose of this sample cross-examination:
To obtain admissions from a CFS scientist respecting the factors that influence reliability in a laboratory
To suggest that there are multiple components to "reliability"
To lay the groundwork for an argument that reliability means a lot more than a formula of two tests 15 minutes apart with 02 agreement and control tests between 90 and 110
To define scientific "reliability" using ISO 17025
Q. Can I just ask you please to move ahead please to page 10 under paragraph 5? Certainly from – in a laboratory context, the description there at 5.1.1 indicates that there are many factors that determine the correctness and reliability of the tests and or calibrations performed by a laboratory. They include human factors. Is that consistent
with your experience? A. Yes.
Q. Accommodation and environmental conditions. I guess that can include temperature. A. Yes. So, human can – human condition – human factors would be such as an individual’s accidentally switching the vials, or the order or the vials that the analysis is done in. Q. Test and calibration methods and method validation. A. Yes. Q. Equipment, again, that’s consistent with your experience and the work that you do in a laboratory. A. Certainly is, yes. Q. Measurement traceability. A. Yes. Q. Right? And – and again, what’s measurement traceability? Can you help us with that please? A. So, in that case, the analysis that’s performed and the accuracy of that analysis is related to a specific parameter. So, you have an alcohol standard solution that is certified by the manufacturer to have this concentration in it, and that is something that you can then use to relate to the concentration of the unknown in a particular sample. So, you produce a calibration curve that we described earlier, right, well, varying – increasing concentration and those standards are certified to contain a certain amount of alcohol that can then produce a response, in relation to the concentration and then you compare your unknown to that to get the actual concentration. Q. But measurement traceability, the traceability is between the measuring device and the
standards, the reference standards that were used in the calibration of that device, right? A. That's correct. So, we have standards that are certified. We also have pipets that are calibrated and checked on a very regular basis to make sure that they haven’t drifted at all.
[Note to defence practitioners. Where does the "traceability" component of reliability come from? The traceability of the measurement result to alcohol standards, acknowledged by the CFS expert here, relates to the alcohol standards USED AT THE FACTORY TO CALIBRATE THE INSTRUMENT, NOT to the alcohol standard used at TIME OF INSTRUMENT USE with your client.}