Original article / research
Year :
2024 |
Month :
July
|
Volume :
13 |
Issue :
3 |
Page :
BO04 - BO07 |
Full Version
|
|
Comparison of Sample Size and Rejection Limits for Lot-to-Lot Verification using Two Different Protocols: A Cross-sectional Study
|
R Arun Krishnan, S Jaya Kumari, Saranya Raj 1. Postgraduate Student, Department of Biochemistry, St. John’s Medical College, Bengaluru, Karnataka, India.
2. Professor and Head, Department of Biochemistry, St. John’s Medical College, Bengaluru, Karnataka, India.
3. Postgraduate Resident, Department of Biochemistry, St. John’s Medical College, Bengaluru, Karnataka, India.
|
|
Correspondence
Address :
R Arun Krishnan, S Jaya Kumari, Saranya Raj, S Jaya Kumari,
E-604, Raheja Residency, 3rd Block, Koramangala, Bengaluru-560034, Karnataka, India.
E-mail: jaya_kripal@yahoo.com
|
| ABSTRACT |  | : Introduction: Good laboratory practice necessitates verifying each new lot of reagents before it is put into service to ensure that the results of patient samples are consistent across different lots of reagents. Current laboratory protocols use uniform criteria such as a 10% difference, measurement uncertainty, etc., for the acceptance of Lot-to-Lot Verification (LTLV) across all parameters. Although a detailed guideline was introduced by the Clinical and Laboratory Standards Institute (CLSI), there is limited literature on laboratories using this.
Aim: To compare the sample size and rejection limits for LTLV for common chemistry analytes using two different protocols.
Materials and Methods: The present cross-sectional analytical study was carried out in the Department of Biochemistry, St. John’s Medical College, Bengaluru, Karnataka, India, from January 2017 to January 2019. The LTLV for reagents was done using patient and Internal Quality Control (IQC) data for common analytes such as glucose, creatinine, albumin and calcium on Siemens Dimension EXL 200 chemistry analyser. Sample size and rejection limits obtained using CLSI EP26-A guidelines were compared with one of the current protocols, which uses one sample at each concentration, usually a maximum of two at two different concentrations, and defines less than a 10% difference in value between the two lots as acceptable criteria. The quality control data was analysed, and descriptive statistics such as the coefficient of variation were used to arrive at precision. Rejection limit and sample size were directly read from the EP26-A guideline using the tables given for the same, based on the two ratios at 90% power.
Results: The sample size needed was found to be the same for creatinine and albumin using both protocols, while it was higher for glucose and calcium based on EP26-A guideline. The rejection limit obtained using EP26-A guideline was different for each parameter between the two protocols. The rejection limit obtained using EP26-A was lower for all analytes as compared to the first protocol.
Conclusion: Lot-to-Lot Verification (LTLV) using the CLSI guideline reinforces the fact that the sample size needed and the rejection limit for each parameter varies based on its performance and the critical difference to be detected. Hence, the practice of using a fixed sample size and fixed criteria, such as a 10% acceptable difference across all parameters, may not be appropriate.
|
 |
Keywords
: Accuracy, Critical difference, Internal quality control, Precision, Validation |
|
DOI and Others
: DOI: 10.7860/NJLM/2024/65536.2855
Date of Submission: May 23, 2023
Date of Peer Review: Aug 09, 2023
Date of Acceptance: Jan 19, 2024
Date of Publishing: Jul 01, 2024
AUTHOR DECLARATION:
• Financial or Other Competing Interests: None
• Was Ethics Committee Approval obtained for this study? Yes
• Was informed consent obtained from the subjects involved in the study? NA
• For any images presented appropriate consent has been obtained from the subjects. NA
PLAGIARISM CHECKING METHODS:
• Plagiarism X-checker: May 24, 2023
• Manual Googling: Dec 28, 2023
• iThenticate Software: Jan 18, 2024 (5%)
Etymology: Author Origin
Emendations: 7 |
|
|
INTRODUCTION |
 |
The Lot-to-Lot Verification (LTLV) is an important validation process to ensure that results of patient samples are consistent when changing lots of reagents. The validation of the new reagent lot can be done by running them in parallel with the old lot and analysing whether the results of IQC and patient samples obtained across the two lots are within the defined acceptability criteria (1).
Laboratory reagents are exposed to many variables during transportation and storage environments in different laboratory settings. The validation of new reagent lot is performed to ensure that there are no clinically significant differences in the results obtained when different lot numbers of reagents are used (2),(3),(4). This also allows for the detection of differences between lots, prediction of a change in the control value, determination of a new target value, and an acceptable range of IQC with the new reagent lot, if necessary. In addition, LTLV is also a requirement by accreditation bodies (5). Some of the variables that affect reagents and thus the LTLV are temperature and thermal shocks, reconstitution errors, status of calibrator, practical variations in reagent preparation, reagent manufacturing variability and warehousing issues (2),(6),(7).
Currently, laboratories use various rejection limits/acceptance criteria for LTLV such as a difference of less than 10%, measurement uncertainty, etc., across all parameters (5). A detailed protocol for LTLV has been published by the CLSI, i.e., EP26-A-user evaluation of between-reagent lot variation approved guideline (8). It is based on an elaborate and statistically sound protocol to evaluate the consistency of results when a new analytical reagent lot replaces a reagent lot currently in use based on the performance of each analyte. The CLSI EP26-A guideline for LTLV takes into consideration criteria such as method performance, critical difference to be detected, medical decision limit, etc., for arriving at the adequate number of patient samples (sample size) to be used and the rejection limit for each parameter for acceptance (2).
Although this guideline was established in 2013, very few laboratories are reviewing their LTLV adequately and establishing the fitness of the new reagent lot for use as per this protocol (2). Hence, the present study was conducted to analyse and compare the sample size and rejection limits used for LTLV by one of the current laboratory protocols and compare the same obtained using CLSI EP26-A guidelines. LTLV of common chemistry parameters like albumin, glucose, creatinine and calcium were used for the present study.
|
|
|
Material and Methods |
 |
The present cross-sectional analytical study was conducted in the Department of Biochemistry, St. John’s Medical College, Bengaluru, Karnataka, India, from January 2017 to January 2019 after obtaining Institutional Ethics Committee clearance (IEC Study Ref No.350/2017).
Inclusion criteria: The paired sample values of QC and patient samples, conducted as part of reagent LTLV in the laboratory after new lot verification for glucose, creatinine, albumin and calcium, were included in the study.
Exclusion criteria: Values obtained during faulty or unacceptable calibrations were excluded from the study.
Sample size calculation: It was a pilot study, and the sample size was selected as per the guidelines of the two protocols. According to the CLSI guideline, the rejection limit and sample size to be used for LTLV depend on the performance (precision) of the analyte. The sample size used to calculate the precision of each analyte for the study was 180 IQC values at level 1 and level 2 (2). This was obtained from IQC performance across six months for each parameter from a single reagent lot, as specified by the guidelines.
The analysis of a single medically relevant concentration of patient sample for each analyte was considered for both protocols. The sample size and acceptance criteria in the first protocol were compared with the sample size and rejection limit obtained using the second protocol for the same single medically relevant concentrations.
Study Procedure
Protocol 1 (Rejection limit/acceptable bias <10%)
Sample size: A fixed sample size and rejection limit were used. One patient sample at one medical decision concentration for each of the parameters was selected for each analyte.
Rejection limit/acceptance criteria: A difference of less than 10% of the concentration between the two lots was considered the acceptance criteria for all parameters studied (5).
Protocol 2 (Rejection limit/acceptable bias based on CLSI guideline)
Sample size: The sample size needed and rejection limit were not predetermined. They were calculated according to the guideline based on the analytical performance of each parameter using the following steps for the same medical decision concentration used in protocol 1 (2).
a) Defining the Critical Difference (CD): This is the maximum acceptable difference between lots. The total allowable error from the Clinical Laboratory Improvement Amendments (CLIA) guidelines was used to define the CD for each parameter.
b) Choosing the medical decision concentration(s): Only one decision limit was selected in the present study, as mentioned in protocol 1.
c) Historical precision performance: This was obtained from IQC data. SR-Repeatability (also known as within-run) was obtained from validation studies using repeated measures of IQC within a single run, and SWRL-within lot total precision was calculated from cumulative precision data of IQC on the existing lot of reagent using repeated measures of IQC across different analytical runs over a period of time on a single reagent lot. The IQC level chosen was close to the medically relevant concentration of the patient sample used in the study.
d) Desired statistical power: This was set at 90% to detect the critical difference between lots at 5% precision.
e) SR/WRL and CD/SWRL were calculated for each analyte.
f) Rejection limit and the number of samples were obtained using Table A1 in the appendix from the EP26-A guideline (2).
Rejection limit: The rejection limit is obtained from (Table/Fig 1) in EP26-A using the following procedure (2).
1. Locate the test measurement procedure’s CD to within-reagent lot imprecision (CD/SWRL) ratio in the first column.
2. Locate the test measurement procedure’s ratio of repeatability to within-reagent lot imprecision (SR/SWRL) from the rows in the second column that correspond to the ratio from step 1.
3. Move across the row from the cell located in step 2 until the second number in parentheses in the row’s cell is greater than or equal to the intended power (0.9).
4. The first number in parentheses is the type 1 error rate.
5. The number outside the parentheses is the necessary number of samples to test at each of the two concentration intervals with each reagent lot to detect the CD.
6. The third row of this column gives the CD multiplier. The CD multiplied by this factor gives the required rejection limit.
Data collection:
• A total of five new reagent LTLV verifications were included in the study. Each new lot of reagents was calibrated. The acceptability of the calibration data was evaluated by verifying that the slope is within 0.9 to 1.10 and the intercept is less than half of the lowest reportable value, with a correlation coefficient >0.95. IQC samples and patient samples were run as part of LTLV (5).
• Reagent LTLV was carried for most commonly done parameters such as: glucose, creatinine, albumin and calcium on fully automated Siemens Dimension EXL 200 chemistry analyser. Glucose was estimated using Hexokinase method, creatinine using Modified kinetic Jaffe technique, albumin using Bromocresol purple method and calcium using Modified o-cresolphthalein complexone method (9).
Statistical Analysis
The quality control data was analysed using a Microsoft Excel spreadsheet. Descriptive statistics, such as the coefficient of variation, were used to describe precision. The rejection limit and sample size were directly obtained from the EP26-A guideline using the provided tables based on the two ratios at 90% power.
|
|
|
Results |
 |
The sample size and acceptance criteria used in LTLV of patient samples in protocol 1 as shown in (Table/Fig 1). According to this protocol, up to 13 mg/dL, 0.125 mg/dL, 0.88 mg/dL, and 0.36 mg/dL variation were considered acceptable between reagent lots for glucose, creatinine, calcium, and albumin, respectively.
According to protocol 2, up to 7.80 mg/dL, 0.21 mg/dL, 0.60 mg/dL and 0.25 mg/dL variation were considered acceptable between reagent lots for glucose, creatinine, calcium, and albumin, respectively (Table/Fig 2). (Table/Fig 3) shows a comparison of the sample size between the two protocols. The sample size calculated using the CLSI guideline was higher for glucose and creatinine, while it was the same for albumin and calcium for the two protocols. (Table/Fig 4) shows a comparison of the rejection limit between the two protocols. The rejection limit calculated using the CLSI guideline was lower for all parameters for the medical decision limit chosen, except for creatinine, which was higher as per the CLSI protocol.
|
|
|
Discussion |
 |
The LTLV is a regulatory requirement, and the evaluation of the performance of quality control and patient samples on new reagent lots is especially crucial for analytes that are used for long-term patient follow-up or those that guide critical clinical decisions. LTLV evaluation is carried out by different protocols in laboratories. CLSI EP26-A is an elaborate procedure that describes reagent LTLV. A critical component of the EP26-A protocol involves a set of parameters and calculations made in advance of executing the reagent lot verification. This involves the use of quality control data to obtain precision (within lot and between reagent lot) and obtaining the critical limit. These parameters are used to determine the number of patient samples to be tested and rejection limits (2).
The present study showed that the sample size needed for creatinine and albumin were similar in both protocols, while it was higher for glucose (sample size=4) and calcium (sample size=6) as per protocol 2 using EP26-A guideline. Similarly, the critical limit calculated based on EP26-A shows that the acceptable difference between reagent lots was much lower than in protocol 1. The rejection limit was found to be lower for all parameters except creatinine. This shows that laboratories may be accepting LTLV with a higher difference if they use protocol 1, which may impact patient values, especially during serial monitoring. Katzman BM et al., also reported similar observations, indicating that sample sizes required as per CLSI EP26-A guidelines were higher in some cases with respect to immunoassay analytes (10).
Since the CLSI guideline uses historical analytical performance for the calculation of sample size, the number of samples required for LTLV as per this guideline largely depends on the assay imprecision. Another important consideration when following the EP26-A protocol is the target analyte concentrations for verification. Another variable critical in the calculation is CD, which can also vary based on the source of total allowable error used, such as CLIA guidelines, biological variation database, etc. Kim S et al., observed that sample sizes and rejection limits varied based on the protocol used to determine the CD (11). Additionally, a retrospective analysis by Algeciras-Schimnich A et al., observed that sample sizes based on their existing validation methods may be inadequate in the case of some reagents like Insulin-like Growth Factor-1 (IGF-1), indicating the need for standardised guidelines such as EP26-A (12). Tao R et al., in a study of 16 chemiluminescence analytes, observed that EP26-A suggested larger sample sizes similar to our study. However, they also observed higher rejection limits for most of the analytes, unlike our study. This may be attributed to the fact that the comparison was done against bias <1/3 total allowable error (13).
One of the most common observations among the various studies has been that CLSI EP26-A often suggests very large sample sizes, making it impractical to use [6,10-12,14]. However, it was also observed that the more stringent requirements by EP26-A helped to identify lots where the changes were left undetected by existing protocols (10).
Limitation(s)
Different medical decision limits should be used to arrive at the sample size for each of the limits. Since this was a pilot study, only one medical decision limit has been used and compared. Also, not all laboratories may be using 10% as an acceptable criterion. The present study has not evaluated other acceptance criteria such as measurement uncertainty.
|
|
|
Conclusion |
 |
The application of the CLSI EP26-A guideline shows that a uniform sample size of one or two patient samples currently used for reagent LTLV may not be appropriate and sufficient to detect variations. The rejection limit or the acceptability criteria also need to be established for each analyte based on the analytical performance and critical difference to be detected, rather than having a uniform criterion across all parameters. A robust LTLV protocol plays a crucial role in the appropriate acceptance of new reagents, which may significantly impact the serial monitoring of patient values. Although scientifically more sound, the LTLV using the CLSI guideline can be a substantial burden for clinical laboratories due to the technologist time involved in the selection of a large number of samples at various critical decision limits and the cost of reagents involved.
|
|
| 1. | Jo W. CLIA Requirements for Analytical Quality-Westgard. Available from: https://www.westgard.com/clia.htm. (Accessed 15 March 2023).
[ Google Scholar] | 2. | CLSI. User Evaluation of Between-Reagent Lot Variation; Approved Guideline. CLSI document EP26-A. Wayne, PA: Clinical and Laboratory Standards Institute; 2013, www.clsi.org.
[ Google Scholar] | 3. | M. Lehman C. EP31AIRE: Patient results within one health care system. Available from: https://clsi.org/standards/products/method-evaluation/documents/ep31/. (Accessed 15 March 2023).
[ Google Scholar] | 4. | Dufour DR. Lot-to-lot variation in anti-hepatitis C signal-to-cutoff ratio. Clin Chem. 2004;50(5):958-60. ?doi?https://doi.org/10.1373/clinchem.2003.031039#doi#
[ Google Scholar] [ CrossRef] | 5. | NABL 112. Specific Criteria for Accreditation of Medical Laboratories. 2019.
[ Google Scholar] | 6. | Thompson S, Chesher D. Lot-to-Lot Variation. Clin Biochem Rev. 2018;39(2):51-60.
[ Google Scholar] | 7. | Loh TP, Sandberg S, Horvath AR. Lot-to-lot reagent verification: Challenges and possible solutions. Clin Chem Lab Med. 2022;60(5):675-80. ?doi?https://doi.org/10.1515/cclm-2022-0092#doi#
[ Google Scholar] [ CrossRef] | 8. | Kim HS, Kang HJ, Whang DH, Lee SG, Park MJ, Park JY. Analysis of reagent lot-to-lot comparability tests in five immunoassay items. Ann Clin Lab Sci. 2012;42(2):165-73. ?doi?https://doi.org/10.3343/lmo.2012.2.4.235#doi#
[ Google Scholar] [ CrossRef] | 9. | SIEMENS Dimension clinical chemistry System. Flex reagent cartridge. 2016.
[ Google Scholar] | 10. | Katzman BM, Ness KM, Algeciras-Schimnich A. Evaluation of the CLSI EP26-A protocol for detection of reagent lot-to-lot differences. Clin Biochem. 2017;50(13-14):768-71. ?doi?https://doi.org/10.1016/j.clinbiochem.2017.03.012#doi#
[ Google Scholar] [ CrossRef] | 11. | Kim S, Chang J, Kim SK. Sample size and rejection limits for detecting reagent lot variability: Analysis of the applicability of the Clinical and Laboratory Standards Institute (CLSI) EP26-A protocol to real-world clinical chemistry data. Clin Chem Lab Med. 2020;59(1):127-38. ?doi?https://doi.org/10.1515/cclm-2020-0454#doi#
[ Google Scholar] [ CrossRef] | 12. | Algeciras-Schimnich A, Bruns DE, Boyd JC. Failure of current laboratory protocols to detect lot-to-lot reagent differences: Findings and possible solutions. Clin Chem. 2013;59(8):1187-94.?doi?https://doi.org/10.1373/clinchem.2013.205070#doi#
[ Google Scholar] [ CrossRef] | 13. | Tao R, Zhang C , Liu M, Yang M, Gao W, Chen J, et al. Research and discussion on the evaluation scheme of reagent lot-to-lot differences in 16 chemiluminescence analytes, established by the EP26-A guidelines of the CLSI. J Clin Lab Anal. 2021;35(3):e23675. Doi: 10.1002/jcla.23675. ?doi?https://doi.org/10.1002/jcla.23675#doi#
[ Google Scholar] [ CrossRef] | 14. | Don-Wauchope AC. Lot change for reagents and calibrators. Clin Biochem. 2016;49(16-17):1211-12. ?doi?https://doi.org/10.1016/j.clinbiochem.2016.04.003#doi# [ Google Scholar] [ CrossRef]
|
|
|
TABLES AND FIGURES |  |
|
|
|
 |
|