Original article / research
| Year :
2023 |
Month :
April
|
Volume :
12 |
Issue :
2 |
Page :
PO14 - PO18 |
Full Version
|
|
Sigma Metrics as a Valuable Tool for Effective Analytical Performance and Quality Control Planning in Clinical Laboratory: A Retrospective Study
|
Anupapama Raj Karaattuthazhathu, PP Sathi, Lathi Nair, PM Ramees, Reenu Manayani, Elizabeth Benny, Mary Benny
1. Associate Professor, Department of Pathology, KMCT Medical College, Kozhikode, Kerala, India.
2. Professor, Department of Pathology, KMCT Medical College, Kozhikode, Kerala, India.
3. Professor, Department of Microbiology, KMCT Medical College, Kozhikode, Kerala, India.
4. Chief Operations Officer, Department of Operations, KMCT Medical College, Kozhikode, Kerala, India.
5. Operations Manager, Department of Operations, KMCT Medical College, Kozhikode, Kerala, India.
6. Quality Manager, Department of Operations, KMCT Medical College, Kozhikode, Kerala, India.
7. Technical Manager, Department of Central Laboratory, KMCT Medical College, Kozhikode, Kerala, India.
|
| |
Correspondence
Address :
Anupapama Raj Karaattuthazhathu, PP Sathi, Lathi Nair, PM Ramees, Reenu Manayani, Elizabeth Benny, Mary Benny,
Dr. Anupama Raj Karaattuthazhathu,
Associate Professor, KMCT Medical College Hospital, P.O. Manassery, Kozhikode, Kerala, India.
E-mail: anupamarojith@gmail.com
|
| | | ABSTRACT |  | : Introduction: For the release of precise and accurate reports of routine tests, its necessary to follow a proper quality management system in the clinical laboratory. One of the most popular quality management system tools, for process improvement, six sigma has been accepted widely in the laboratory testing process. It gives an objective assessment of analytical methods and instrumentation. Six sigma measures the outcome of a process, on a scale of 0 to 6. The poor outcomes are measured in terms of defects per million opportunities (DPMO).
Aim: To do the performance assessment of each analyte by six sigma analysis and to plan and chart out a better, customised, quality control plan for each analyte, according to its own sigma value.
Materials and Methods: This was a retrospective observational study, conducted from January 2022 to June 2022, in the Department of Central Laboratory, KMCT Medical College, Kozhikode, Kerala, India. The precision and accuracy of 26 parameters in both haematology and biochemistry were assessed via Internal Quality Control (IQC) and External Quality Assurance (EQAS) Programme, analysis, and their performance was assessed by sigma analysis.
Results: Clinical chemistry parameters showed an average percentage of Coefficient of Variation (CV%) of 2.65% and 2.3% for all the parameters in L2 (normal level) and L3 (abnormal levels) respectively. In haematology, the average CV% came out as, 1.3% (high level),1.82% (low level), and 1.35% (normal level). These values indicate excellent precision for all parameters in both clinical chemistry and haematology; with CV% below 3%. It was observed in the month of May, due to reconstitution errors, bias% showed a setback in a few chemistry parameters, due to which the sigma was lowered. Parameters with <3 sigma metrics (poor performance) occupy 37%, 3-6 sigma metrics (good performance) occupy 29% and >6 sigma metrics (world-class performance) occupy 34% of all the 26 parameters of clinical chemistry and haematology. Mean, Standard Deviation (SD) for biochemistry parameters were calculated using the daily IQC data.
Conclusion: With the present study, sigma metric analysis provides a benchmark for the laboratory to design a protocol for IQC, address poor assay performance, and assess the efficiency of the existing laboratory processes. It is on the basis of strict quality control measures and sigma analysis, the present Institute, was able to achieve world-class performance in many analytes of clinical chemistry and haematology disciplines. However, a few analytes like alkaline phosphatase, alanine transaminase, aspartate transaminase, and total protein needed more stringent external quality assurance monitoring and modified quality control measures. |
 |
| Keywords
: Six sigma, Total analytical error, Westgards rules |
|
| DOI and Others
: DOI: 10.7860/NJLM/2023/60440.2714
Date of Submission: Oct 01, 2022
Date of Peer Review: Nov 23, 2022
Date of Acceptance: Dec 19, 2022
Date of Publishing: Apr 01, 2023
AUTHOR DECLARATION:
• Financial or Other Competing Interests: None
• Was Ethics Committee Approval obtained for this study? Yes
• Was informed consent obtained from the subjects involved in the study? Yes
• For any images presented appropriate consent has been obtained from the subjects. NA
PLAGIARISM CHECKING METHODS:
• Plagiarism X-checker: Oct 02, 2022
• Manual Googling: Dec 08, 2022
• iThenticate Software: Dec 10, 2022 (17%)
ETYMOLOGY: Author Origin |
|
| |
|
INTRODUCTION |
|
In a healthcare system, laboratory diagnostics play a very important role in accurate diagnosis and prompt disease management. The total testing process in a laboratory includes preanalytical, analytical, and post-analytical phases. Measures to improve quality can be applied successfully in any of the three phases. In the analytical phase, quality improvisation can be done by internal and external quality control measures, to ensure precision and accuracy of reporting. IQC involves running a control sample with an identitical matrix to patients’ samples and the sample has an established concentration range. The range should be available in high, low, or normal levels of the analyte, covering the medical decision points. EQAS or peer group programs involve, reporting periodically, proficiency testing samples,which are supplied by an external agency at a predefined time interval. The values obtained are compared with those obtained in other laboratories, participating in the same program and are interpreted as SD Index or Z-score. While IQC determines the precision (expressed as coefficient of variation, CV) of the testing process, EQAS measures its accuracy (expressed as Bias) (1). Sigma metrics is about non conformities or errors; a technique to quantify, and then minimise those defects (2). It is the benchmarking scale where all the process defects in all three phases of a total testing process are measured and judged.
Six sigma started decades ago, in Motorola by Sir Bill Smith, the father of “Six Sigma” in 1986 (3). The first paper to express its application inclinical laboratory processes was published in the year 2000 (4).
Six sigma measures the outcome of a process, on a scale of 0 to 6. The poor outcomes are measured in terms of Defects Per Million Opportunities (DPMO). The number of defects or errors of the laboratory in any area of the total testing process can be counted or estimated and converted to the DPMO ratio (5). The level of sigma metrics and corresponding DPMO is shown in (Table/Fig 1) (6).
Study Objectives
The objectives of the present study include:
• To determine, the precision of 36 parameters included in the haematology and clinical chemistry sections, quantitated by percentage Coefficient Variation (CV%)
• To assess the accuracy of the analytes, quantitated by the Bias% of each analyte after EQAS analysis.
• To get the performance assessment of each analyte by six sigma analysis
• To plan and chart out a better, customised, quality control plan for each analyte, according to its own sigma value.
Using the sigma metrics protocol, the authors were able to effectively evaluate the analytical process control procedures in the laboratory and could verify any shortcomings in the procedures. Thus, accuracy, precision, and error detection rate of the analytical phase of the total testing process may be detected and rectified.
|
| |
| |
|
Material and Methods |
|
This was a retrospective observational study, conducted for a period of six months (January 2022 to June 2022), in the Department of Central Laboratory, KMCT Medical College, Kozhikode, Kerala, India. The biochemical analytes included in the study are albumin, bilirubin (total and direct), total protein, calcium, glucose, urea, creatinine, uric acid, HDL, triglycerides, cholesterol, phosphorus, aspartate transaminase, alanine transaminase. Haematology parameters included, Haemoglobin (Hb), Red Blood Cell (RBC) count, haematocrit, Mean Corpuscular Haemoglobin Concentration (MCHC), Mean Corpuscular Volume (MCV), Mean Corpuscular Haemoglobin (MCH), and coefficient of variation of Red Cell Distribution Width (RDW-cv), Standard Deviation of Red Cell Distribution Width (RDW-SD), total White Blood Cell (WBC) count, platelet count, Mean Platelet Volume (MPV).
Study Procedure
The biochemistry internal quality controls were performed daily using L2 (normal) and L3 (pathological) levels, thrice a day. Haematological QC was performed thrice a day using three levels (High, Normal, Low).
The chemistry parameters’ QC samples were run in Randox Imola and sigma were calculated on a six monthly cumulative basis. The haematology parameters’ QC samples were run in Horiba ABX pentra XL80 and the CV% of each was calculated on a monthly basis, sigma on a cumulative six monthly basis. In the laboratory, the IQC data of all the disciplines were interpreted daily by Levy-Jenning’s charts and Westgard’s rules (7). Daily IQC outliers are detected and appropriate prompt corrective and preventive actions are taken. The patients’ samples were analysed, only when the IQC results were within control limits. The authors adopted the set of Westgard’s multi rules (7) in the laboratory as,
1-2s - warning rule
2-2s, 1-3s, R4s- rejection rules.
The QC practices, such as control material storage, reconstitution, and analysis were done as per the manufacturer’s instructions. EQC data was obtained by running monthly Randox International Quality Assurance control Samples (RIQAS).
Statistical Analysis
Statistical analysis was performed using Microsoft Office Excel 2010. Mean, Standard Deviation (SD) for biochemistry parameters were calculated using the daily IQC data.The coefficient of variation (CV%) was calculated using the formula, CV%= (SDX100)/Laboratory mean
Bias% and standard deviation index were calculated using RIQAS (external quality assurance scheme) data, for each analyte. Bias%=(Peer group mean-Laboratory mean)/Peer group mean Peer group mean is the mean of all QC values of laboratories enrolled in the RIQAS program. The allowable Total Analytical Error (TEa%) for each analyte was referred from the CLIA ‘24 guidelines and the consolidated analytical performance requirements (8). Sigma metrics for analytes were calculated using the formula, Sigma value = (TEa%-Bias%)/CV%).
|
| |
| |
|
Results |
|
Monthly CV% of level 2 (L2) controls for the chemistry analytes, from January to June 2022, and six monthly cumulative CV% for each are tabulated and summarised in (Table/Fig 2).
Monthly CV% of Level 3 (L3) controls for the chemistry analytes, from January to June 2022, and six monthly cumulative CV% for each are tabulated and summarised in (Table/Fig 3). Monthly CV%, from January 2022 to June 2022, of haematological parameters for all three levels (High, Low and Normal) were calculated, and six monthly cumulative CV% were calculated for each parameter, the same summarised in (Table/Fig 4),(Table/Fig 5),(Table/Fig 6).
Monthly biasfrom January 2022 to June 2022 and 6 monthly cumulative bias% of chemistry and haematological parameters were calculated, summarised in (Table/Fig 7),(Table/Fig 8).
The cumulative CV% and cumulative Bias% of each analyte in both chemistry and haematology are clubbed together and the allowable Total Analytical error of all the chemistry and haematological parameters according to the CLIA ’24 guidelines are summarised in (Table/Fig 8),(Table/Fig 9),(Table/Fig 10).
Average CV% was calculated and with the above-mentioned formula, Sigma Metrics was calculated for each parameter. The analytes were broadly categorised into sigma <3 (unacceptable), sigma 3-6 (good), and sigma >6 (excellent), using a pie diagram (Table/Fig 11),(Table/Fig 12). Upon analysis, Parameters with <3 sigma metrics (poor performance) occupy 37%, 3-6 sigma metrics (good performance) occupy 29% and >6 sigma metrics (world-class performance) occupy 34% of all the 26 parameters of clinical chemistry and haematology (Table/Fig 13).
|
| |
| |
|
Discussion |
|
The QC protocol implemented in most of the laboratories, the number of times and number of levels is scheduled based on national accreditation bodies. However, as per Good Laboratory Practices, each and every laboratory should design a customised Individualised quality control plan (IQCP) protocol, based on sigma analysis (9). By maintaining six standard deviations between the parameter average and its upper and lower limits, the incorporation of sigma metrics results in the reduction of laboratory errors (10).
According to the analysis, the CV% of both the chemistry and haematology parameters were well within the target set-up by our lab policy, according to the westgard desirable specifications (11).CV% ranged from 1.3% (albumin) to 4.35%(direct bilirubin) for level 2 and 0.91% (albumin) to 4.63% (direct bilirubin) for level 3 chemistry controls. Average CV% being, 2.65% and 2.3% for all the parameters in L2 and L3 levels, respectively. In haematology, the average CV% came out as, 1.3% (high level),1.82% (low level), and 1.35% (normal level). These values indicate excellent precision for all parameters in both clinical chemistry and haematology, in that the average CV% in both disciplines is below 3%.
It was found that the Bias% which denotes the accuracy of the analysis was found to be increased, especially for all the chemistry parameters in the month of May 2022. This contributed to the reduced sigma metrics (less than 3) of many parameters, which include ALP, ALT, AST,Total protein, glucose, urea, creatinine, uric acid, and phosphorus. Good and excellent performances, as per the sigma metrics were exhibited by, albumin, total cholesterol, bilirubin (total and direct), calcium and HDL. All the haematological parameters except RDW and low-level haematocrit showed good and excellent performance as per sigma metrics.
The reduced sigma in major chemistry analytes was identified and categorised as a major incident in the lab, studied, and root cause analysis was done, and was found due to the reconstitution and mixing error of the RIQAS proficiency testing sample in May 2022.
Similar studies were conducted by Kashyap A et al.,(12) and Zhou B et al., (13), including 15 biochemistry parameters and 16 parameters from both biochemistry and haematology, respectively. The variations in sigma values for a few analytes between our study and others can be due to the difference in the methodology of different parameters, Traceability of calibrators used, instruments used, quality control material used, reconstitution protocols followed by different laboratories, other preanalytical and analytical conditions and the analytical performance requirements of each parameter, followed by different laboratories. Parameters whose sigma showed a shift between levels, for example, albumin, total bilirubin, and haematocrit should be evaluated with discretion. This indicates, that Westgards multi-rules have to be implemented more stringently in them.
Westgard recently described statistical quality control procedures based on Sigma Metrics-Run Size Matrix and Westgard Sigma Rules with Run Size which includes three parameters: 1) selection of appropriate Westgard Sigma Rules; 2) the total number of control measurements per statistical quality control event; and 3) frequency ofevents (Run size) of patient samples between SQC events (14).
The frequency of IQC and the criteria for rejection of each QC run for each of the categories mentioned earlier were designed as follows (9): Tests >6 sigma value (excellent tests)- evaluate with two-level QC once a day and 1-3 SD rule. Tests with sigma values between 3 and 6-evaluate with two-level QC once a day (1-2.5 SD rule) Tests with sigma values <3 sigma-evaluate with two or three-level QC two times a day plus a combination of Westgard rules (1-3S/2-2S/R4S/4-1S).
While analysing the IQC and EQAS outliers, the quality of test results is dependent onvarious factors such as reagents quality, type and quality of QC materials, types of analysers, the methodology followed, environmental conditions, training, and personal competency of laboratory staff performing the tests. Hence, during the root causes analysis and implementation of various corrective and preventive measures, various aspects associated with methodology, materials, personnel, equipment, and working conditions were investigated. Laboratory staff training (for reagent preparation and control material reconstitution, instrument maintenance, reagent handling, and storage), periodical competency assessment program was introduced to improve their attitude and knowledge, in order to improve precision for parameters/analytes having low sigma values. Further, SOPs were also reviewed for those analytes having low sigma values and rewritten in a simpler and user-friendly manner.
Limitation(s)
Even though, authors had calculated the sigma values of each analyte, on a six-monthly basis, the effectiveness of quality control planning, using the new quality control strategy, for those with low sigma values, as mentioned above, is not included in the present paper.
|
| |
| |
|
Conclusion |
|
|
Being an easy and effective tool for implementing quality assurance in the analytical phase of the laboratory, Six Sigma metrics help us, to design the QC and formulate the most ideal methodology for a particular analyte. It is, also preferable to keep in mind the probability of false rejection and error detection, while validating the tests against westgards rules. However, sigma metrics can be utilised to plan the QC frequency accordingly thereby upgrading the quality management system of a clinical laboratory, which does approximately 80% of tests in-house and thus, deliver test results accurately with reliability in stipulated time. On the basis of sigma metrics analysis, it may be concluded that the Department of Central Laboratory, KMCT Medical College, Kozhikode was able to achieve satisfactory results, with world-class performance of many analytes; as a roadmap towards preparation for National Accreditation Board of Laboratories (NABL).
|
| |
| |
|
Acknowledgement |
|
|
The authors acknowledge the laboratory technical staffs for their cooperation and assistance.
|
| |
| | 1. | Westgard JO, Westgard SA. Quality control review: implementing a scientifically based quality control system. Ann Clin Biochem. 2016;53:32-50. https://doi. org/10.1177/0004563215597248. ?doi?https://doi.org/10.1177/0004563215597248#doi# ?pmid?26150675#pmid#
[ Google Scholar] [ CrossRef] [ PubMed] | | 2. | Westgard JO, Westgard SA. Measuring Analytical Quality: Total Analytical Error Versus Measurement Uncertainty. Clin Lab Med. 2017;37:1-13. ?doi?https://doi.org/10.1016/j.cll.2016.09.001#doi# ?pmid?28153359#pmid#
[ Google Scholar] [ CrossRef] [ PubMed] | | 3. | Angmo D, Kant S. Six sigma implementation in healthcare industry: Past, present and future. Int J Eng Res Technol. 2015;4:1078-82. ?doi?https://doi.org/10.17577/IJERTV4IS060950#doi#
[ Google Scholar] [ CrossRef] | | 4. | Nevalainen D, Berte L, Kraft C, Leigh E, Picaso L, Morgan T, et al. Evaluating laboratory performance on quality indicators with the six sigma scale. Arch Pathol Lab Med. 2000;124:516-19. ?doi?https://doi.org/10.5858/2000-124-0516-ELPOQI#doi# ?pmid?10747306#pmid#
[ Google Scholar] [ CrossRef] [ PubMed] | | 5. | Oosterhuis WP, Theodorsson E. Total error vs. measurement uncertainty: revolution or evolution? Clin Chem Lab Med. 2016;54:235-39. ?doi?https://doi.org/10.1515/cclm-2015-0997#doi#
[ Google Scholar] [ CrossRef] | | 6. | Kumar BV , Mohan T. Sigma metrics in evaluating the performance of clinical chemistry laboratory. Journal of Laboratory Physicians 2018;10:2:194-99 ?doi?https://doi.org/10.4103/JLP.JLP_102_17#doi# ?pmid?29692587#pmid#
[ Google Scholar] [ CrossRef] [ PubMed] | | 7. | Westgard.com [internet] Consolidated comparison of chemistry and toxicology performance specifications. Sten Westgard M.S Updated July 24, 2022 with CLIA 2024 proposed proficiency testing criteria Updated June 12, 2022, with EFLM minimum quality recommendations Updated February 17, 2022 with 2021 RIQASstate-of-the-artt goals Updated January 21, 2022. Available from https://www.westgard.com/ consolidated-goals-chemistry.htm
[ Google Scholar] | | 8. | International Organisation for Standardisation. Medical Laboratories - Particular Requirements for Quality and Competence. ISO 15189. Geneva: International Organisation for Standardisation (ISO); 2007.
[ Google Scholar] | | 9. | Westgard JO. Six Sigma Quality Design and Control. 2nd ed. Madison, WI: Westgard QC Inc.; 2006.
[ Google Scholar] | | 10. | Westgard.com. Desirable Specifications for Total Error, Imprecision, and Bias, derived from intra- and inter-individual biologic variation [updated 2014]. Available from https://www.westgard.com/biodatabase1.htm
[ Google Scholar] | | 11. | Adiga US, Preethika A, Swathi K. Sigma metrics in clinical chemistry laboratory—a guide to quality control. Al Ameen J Med Sci. 2015;8(4):281
[ Google Scholar] | | 12. | Kashyap A, Sampath S, Tripathi P ,Sen A. Sigma Metrics for Evaluating Laboratory Quality Control Journal of Laboratory Physicians. 2021;13(4) © 2021. The Indian Association of Laboratory Physicians ?doi?https://doi.org/10.1055/s-0041-1731145#doi# ?pmid?34975251#pmid#
[ Google Scholar] [ CrossRef] [ PubMed] | | 13. | Zhou B, Wu Y, He H, Li C, Tan L, Cao Y, et al. Practical application of Six Sigma management in analytical biochemistry processes in clinical settings. J Clin Lab Anal. 2020;34:e23126 ?doi?https://doi.org/10.1002/jcla.23126#doi#
[ Google Scholar] [ CrossRef] | | 14. | Westgard JO, Westgard SA. Establishing evidence-based statistical quality control practices. Am J Clin Pathol. 2019;151:364-70. ?doi?https://doi.org/10.1093/ajcp/aqy158#doi# ?pmid?30517600#pmid# [ Google Scholar] [ CrossRef] [ PubMed]
|
|
| |
| TABLES AND FIGURES | |
|
| |
|
|
|