Purpose:
To suggest that ATC/CFS scientists cannot provide any scientific opinion of the reliability of aging instruments out in the field without conducting an empirical study of instruments sampled from the field.
To suggest that ATC/CFS scientists have no idea whether or not instruments in the field are performing in accordance with manufacturer's specifications.
To identify the lack of any system of "verification" in Canada that instruments in the field are performing in a manner similar to the instruments originally evaluated and given type approval.
To contrast the evaluated accuracy and precision of instruments evaluated by the ATC with actual accuracy and precision of instruments in the field.
To suggest that unlike gas pumps and retail meat scales, in Canada, we have no system in place to control "verification" that an individual instrument conforms to "type approval" of an approved instrument.
To suggest that in Canada we have no system of mandatory periodic verification of individual instruments in the field to ensure that they conform to type approval.
Excerpts from VIML on sampling, type approval, and verification:
Q. Now, can I ask you something about the question of sampling? What – what is sampling? I’m not talking about breath samples. I’m talking about sampling, generally, for you as a scientist. What’s sampling? A. An example would be in blood testing, that you take a sample of blood that’s been collected from an individual, either from a hospital or through a blood demand, and that I use a pipet to sample that blood sample to do my analysis. And I’ll take two aliquots, or two samples of that blood sample, after having mixed it previously, to ensure that it’s uniform, and then doing the analysis on it. Q. Right. A. So, I’ve taken two samples, or two aliquots of that blood sample for the purposes of analysis. Q. Well, can I just ask this? When it comes to the work that’s done by the Centre of Forensic Sciences in relationship to Intoxilyzer 8000Cs, is there any sampling that’s being done by scientists at the Centre of Forensic Sciences, of Intoxilyzer 8000Cs that are out in the field and that are growing older? A. No. I know that our instruments are – date from 2009 and we still use them for breath testing courses, for teaching police officers around the province. We have 18 of them.
Q. And – and how....A. And occasionally they have to go in for service when it’s – when they’re not useful, or not able to be used in the – in the classroom for the purposes of teaching.service?
Q. Well, how often do you send them out for
A. As necessary.
Q. You don’t – you don’t have a particular service interval with respect to them?
A. We do not, no.
Q. And....
A. It’s as – it’s as needed.
Q. It’s as needed, only if they break down?
A. Or they’re deemed to be unacceptable as far that they’re producing, which would be evidentas the results from all the testing that’s done in the classroom.
Q. And when you – when you send them out – when you send them out, do you – you send them out, you don’t service them in-house, you don’t do any kind of inspection in- house within the Centre of Forensic Sciences, you send them out to the Canadian authorised distributor.
A. That's correct. The – no one at the Centre of Forensic Sciences as far as the scientists go, is qualified or has taken the training to do maintenance, internal maintenance of the instrument.
Q. But I–I guess just go back to the point of nobody at the Centre of Forensic Sciences has published any kind of scientific paper or done any empirical study of the instruments that are out in the field, in Ontario, that are growing old to determine whether or not they meet the same
accuracy and precision as evaluated by Ms. Martin at the time of the evaluation of the Intoxilyzer 8000C for the Alcohol Test Committee. A. Well, again, there’s two different sets of parameters. There’s the evaluation parameters and then there’s the operational parameters. They’re two different set of different requirements for both those applications. Q. Right, but it.... [The cross-examiner should have separated the concepts "type approval evaluation" v. "initial inspection/verification" v. "inspection/verification after repair" v. "mandatory periodic inspection/verification" v. use in the field.]
A. And so, you can’t, in the field, meet the – you can’t meet the evaluation criteria because it’s being used in the field, and as I think I said on Friday, that variability in the field is always going to be greater than it is going to be in a laboratory. Again, when it’s being conducted by police officers in the field or being done by scientists in a laboratory where the environment and the situation is completely controlled compared to doing it in the field.
ATC Best Practices 2014 recommendation [only a recommendation, nobody supervises]:
"INSPECTION, MAINTENANCE AND MODIFICATION Proper calibration and/or calibration check procedures are the primary means of assuring accuracy of the Approved Instrument, Approved Screening Device and accessory equipment at the time of use. Calibration of Approved Instruments and Approved Screening Devices shall be done with an aqueous alcohol standard. In addition to these calibrations and/or calibration checks, formal maintenance procedures are essential to the integrity of the breath test program. Records relating to periodic maintenance or inspections cannot address the working status of an Approved Instrument at the time of a breath test. The required quality control information which must be reviewed to assess the working order of an Approved Instrument is produced during the subject breath testing procedure. A. Inspection All Approved Instruments, Approved Screening Devices and accessory equipment intended for active use in the program shall be individually inspected before being placed into service, and periodically thereafter, to ensure that they initially meet, and continue to meet the manufacturer’s specifications. The recommended interval between inspections is one year. All inspections shall be performed by persons deemed by the Program Director to meet the qualifications described in paragraph III.C. below. Accessory equipment includes simulators or other equipment required for the use or calibration of Approved Instruments and Approved Screening Devices."
[Quaere: So why don't we have any "metrological authority", in Canada or in the provinces, that provides "metrological supervision", by conducting "verification by sampling"? Alternatively, why don't we do formal "verification" of each individual approved instrument as is done by Measurement Canada (the metrological authority) with gas pumps? ]
[The "sampling" should be done of "approved instruments" taken from out in the field. Such sampling would lead to empirical evidence that could form the basis for scientific opinion of the reliability of aging instruments out in the field. That verification would be in a calibration laboratory, under controlled conditions, not out in the field.]
[We don't do "verification by sampling" of approved instruments in Canada. Any opinion by a scientist as to the reliability of aging instruments out in the field, without empirical study, is without foundation, and should not be accepted as expert evidence.]
[We also don't do "verification" of individual approved instruments as "initial verification", "verification after repair", or "mandatory periodic verification".]