Original article / research
|
|||||||||||||||||||||||||||||||||||||||
Sigma Metrics-Tool for Quality Assurance in Clinical Biochemistry Laboratory |
|||||||||||||||||||||||||||||||||||||||
Richa Singh, MN Vanitha Gowda 1. Resident, Department of Biochemistry, M.S. Ramaiah Medical College and Hospitals, Bengaluru, Karnataka, India. 2. Professor and Head, Department of Biochemistry, M.S. Ramaiah Medical College and Hospitals, Bengaluru, Karnataka, India. |
|||||||||||||||||||||||||||||||||||||||
Correspondence
Address : Vanitha Gowda MN, MSR Nagar, Bengaluru, Karnataka, India. E-mail: vanithasukesh@hotmail.com |
|||||||||||||||||||||||||||||||||||||||
ABSTRACT | |||||||||||||||||||||||||||||||||||||||
: Sigma metrics is a quality management tool used for process improvement which usually comes into application when there is a measurable outcome in the process. It can play an important role in health care laboratory services as Quality Assurance (QA) of the same, is the need of the hour. Aim: To gauge the performance of a few biochemical parameters by calculating their sigma metrics on a sigma scale. Materials and Methods: This retrospective study was undertaken using Quality Control (QC) and External QA Scheme (EQAS) data for 17 biochemical parameters from Biochemistry section, diagnostic laboratory of an M.S. Ramaiah Medical College and Hospital, Bengaluru. Sigma values for these parameters were determined and sigma metrics was evaluated for duration of 13 months. Results: In level 1 coefficient of variation percentage (CV%), five parameters (ALP, calcium, magnesium, triglycerides and HDL-cholesterol) showed an ideal performance of ≥6 sigma level and in level 2 CV%, eight parameters (total bilirubin, urea, creatinine, albumin, AST, total cholesterol, total protein and phosphorus) showed a sigma of ≥6. Quality Goal Index (QGI) for 11 analytes in level 1 and seven analytes in level 2 was <0.8, indicating imprecision. QGI was in the range of 0.8-1.2 for one analyte in level 1 and two analytes in level 2, indicating a problem of both imprecision and inaccuracy. Conclusion: Sigma metric analysis can serve as a tool to identify the poor assay performance and to assess the efficiency of processes that are in existence. The health care sector can be immensely benefited by implementation of sigma metrics for QA. | |||||||||||||||||||||||||||||||||||||||
Keywords : Bias, Laboratory proficiency testing, Quality control, Quality goal index | |||||||||||||||||||||||||||||||||||||||
INTRODUCTION | |||||||||||||||||||||||||||||||||||||||
Clinical laboratories play a major role in healthcare system (1). In a Clinical Biochemistry Laboratory (CBL), the measures used to assess the QC are Internal QC (IQC) and EQAS (2). Proper documentation, stability and reliability are some of the key features of laboratory QC (3). IQC is an important part of laboratory quality management whose products can determine the reliability of test results (4). Selection of a proper IQC procedure for implementation is the first essential practice in setting up IQC (5). IQC is interpreted using the standard Westgard rules and is run daily, as per National Accreditation Board for Testing and Calibration Laboratories (NABL) guidelines. IQC keeps an eye continuously on the analytical system to check whether the results are reliable enough to be released or not. Contrarily, EQAS sample, which is supplied by an outside agency, is run once every month and is interpreted using the Z-score (5). Approximately, two-thirds of important clinical decisions on patient management are based on laboratory test results (6). The entire process of testing in a CBL involves pre-analytical, analytical, and post-analytical phases. As there exists a chance of occurrence of error at any step of testing in a clinical laboratory, reduction of errors and an ongoing improvement in testing is required. To provide accurate and reliable reports within the agreed time, it is important to maintain and follow a proper Quality Management System (QMS) (7). This is what led to the evolution of sigma metrics methodology by Bill Smith back in 1986 (8). Sigma metrics can quantify the exact number of errors done in the analytical phase by the laboratory that cannot be gauged by running the internal and external QCs (3). By using sigma metrics in the laboratory, the number of errors or defects could be exactly quantified; 1-sigma parallels to 6,90,000 errors or defects per million reports, 2-sigma relates to 3,08,000 defects per million reports, 3-sigma equals 66,800 defects per million reports, 4-sigma to 6,210 defects per million reports, 5-sigma to 230 defects per million reports and 6-sigma parallels to 3.4 defects per million reports (9). Sigma is used in statistics to represent the SD, which indicates the degree of variation in a process. A 5% error rate corresponds to a 3.15 sigma performance and a 1% error rate corresponds to 3.85 sigma (10). Three-sigma is taken as the minimum allowable sigma for routine performance and six-sigma as the goal for world-class quality (11). The present study is among the few that have taken into consideration more than 15 biochemical parameters for studying the effectivity of sigma matrices. This study was conducted to gauge the performance of individual biochemical parameters in MS Ramaiah Hospital Clinical Laboratory and find out the errors associated with each parameter. | |||||||||||||||||||||||||||||||||||||||
MATERIAL AND METHODS | |||||||||||||||||||||||||||||||||||||||
This cross-sectional study was conducted in the Biochemistry Section of the Diagnostic Laboratory of M.S. Ramaiah Medical College and Hospital, Bengaluru from July 2017 to July 2018, using IQC and EQAS data of 17 biochemical parameters selected randomly (urea, creatinine, total bilirubin, aspartate aminotransferase, alanine aminotransferase, albumin, alkaline phosphatase, total cholesterol, total protein, phosphorus, calcium, magnesium, triglycerides, high density lipoproteins, glucose, sodium and potassium). Data analysis was done from July 2017 to July 2018 and the study was conducted from 1-10-18 to 20-10-18 (20 days). The data was obtained for IQC in terms of Coefficient of Variation percent (CV%) and for EQAS in terms of bias% using COBAS 6000 autoanalyser, in the diagnostic laboratory. Sigma values for these parameters were determined and sigma metrics was evaluated. Sigma (s) value is calculated with the following formula (10). Sigma metrics (s) = TEa%-Bias%/CV% Where, TEa is total allowable error percentage and CV is coefficient of variation. TEa= Allowable Bias + 1.65 × Allowable Imprecision Where, the allowable bias = 0.25 × (CVw2 + CVb2)1/2, the allowable imprecision = 0.5 × CVw, CVb is the inter individual imprecision, and CVw is the intra individual imprecision (12). Bias% = Our EQAS result- peer group mean (using the same instrument and method) X 100 Peer group mean (using the same instrument and method) CV was measured based on below formula, using Bio-Rad internal QC1 for all the parameters. CV% = SD X (100)/Mean The QGI ratio denotes the relative extent to which both bias and precision meet their respective quality goals (14). The QGI ratio was calculated using the following formula (7): QGI = Bias X CV%/1.5 TEa values were obtained from the Clinical Laboratory Improvement Amendment (CLIA) (13). The bias percentage for each parameter was calculated using Bio-Rad-EQAS (1). QGI can be used to assess the reason for lower sigma (due to imprecision or inaccuracy or both) in some analytes (Table/Fig 1). QGI ratio of <0.8 indicated imprecision, ratio of 0.8-1.2 indicated imprecision and inaccuracy and a ratio >1.2 indicated inaccuracy and was used in case test parameters fell short of six-sigma quality. STATISTICAL ANALYSIS Microsoft Excel spreadsheet version 2010 was used for statistical analysis. Bias, CV, QGI and sigma metrics were calculated using the above formulae. Bias and CV were presented as percentages. | |||||||||||||||||||||||||||||||||||||||
RESULTS | |||||||||||||||||||||||||||||||||||||||
The sigma metrics and QGI ratio for 13 months (July 2017- July 2018) were calculated using TEa, CV% (level-1 & level-2) and bias% for 17 biochemical parameters as given in [Tables/Fig-2], [Tables/Fig-3],[Tables/Fig-4]. Among the 17 analytes observed in level -1 IQC (Table/Fig 5), five analytes showed an ideal performance of ≥6 sigma level, eight analytes showed sigma level <6, whereas, four analytes showed a level <3, indicating poor performance. For level-2 IQC in (Table/Fig 5), seven analytes showed a performance of >6 sigma level, six analytes showed a sigma value of ≤6, whereas, four analytes showed a value of <3, indicating poor performance. From (Table/Fig 5), it was found that QGI for 11 analytes in level-1 and seven analytes in level-2 was <0.8, indicating imprecision. QGI was in the range of 0.8-1.2 for one analyte in level-1 and two analytes in level-2, indicating a problem of both imprecision and inaccuracy. | |||||||||||||||||||||||||||||||||||||||
DISCUSSION | |||||||||||||||||||||||||||||||||||||||
Laboratory performance has been measured as rate of errors or defects that occur per million tests or products. Utilisation of sigma matrices can be useful for measuring the performance of testing processes and service provision. Designing own QC procedures is considered as a good laboratory practice that a laboratory can engage in, to ensure a good quality of reports for the patient (15). Sigma metric analysis can be used to minimise the errors in the laboratory as it is a widely accepted universal benchmark of quality measurement. In this study, six sigma values were obtained for magnesium, ALP, triglyceride, and HDL cholesterol, in both IQC levels, and were in accordance with previously conducted studies (5),(10),(16). The results of the current study indicated that the methodologies adopted by in-study laboratory for estimation of the above analytes were appropriate and following very stringent QC protocols were not needed for these analytes. Following 13S Westgard rule alone would suffice in these cases. The sigma values for total protein, creatinine and phosphorus obtained in this study were between 3-6 for both the IQC levels. The values for total protein were found to be 4.36 and 4.67 in levels 1 and 2, respectively. Whereas, in the study done by Kumar BV and Mohan T, the sigma values obtained for total proteins were 3.00 and 3.27 in levels 1 and 2, respectively (5). For the sigma values between 4-6, indicating an acceptable performance a laboratory should follow Westgard Multirules, and QCs should be run twice daily (17). But, for sigma values of 3-4 that indicate poor laboratory performance, Westgard Multirules need to be followed stringently and two levels of QCs should be run twice daily (5). Authors found a sigma value of <3 and QGI ratio of <0.8 for sodium and potassium for both levels of IQC, indicating a problematic analyte and an imprecision issue, thereby pointing that the daily workflow needs to be revised. For such analytes, the frequency of QC runs has to be increased as per Westgard rules and the root cause of the problem should be found. Additionally, 2 or 4 IQC measurement of both IQC levels is needed with each daily run of 4 or 2. By incorporating the required changes in the workflow, as the sigma increases, consistency, steadiness, reliability, and overall performance of the test improves, followed by a decrease in the operating costs (18). The application of six sigma is hence proved to be helpful in assessing the laboratory testing processes in the study laboratory. However, there seem to be limitations in the clinical application of sigma metrics for some analytes like hormones. For such analytes, CV percentage and bias percentage were found to be better than sigma matrices (12). Limitation(s) Reference methods were not used for certain analytes, and external quality control system was used due to financial constraints. However, this study provided certain strong points that helped refine the previously followed QC protocol. The results of this study helped to improve the overall quality of the test procedure and also minimise the excessive QC monitoring, thereby reducing the overall costs for analysis with higher sigma metrices. Every clinical diagnostic laboratory should design their QC procedure and use six-sigma matrices for QC monitoring to improve the performance of the laboratory. This would make it easy for implementing changes in laboratory consumption of QC materials along with calibrators and reagents. This would also aid in interpretation of control rules and frequency of running QC materials for respective analyte. In addition to sigma metrics, other visual tools such as method decision charts could be used, which transform the sigma metrics into a simple in-built dashboard. It allows a laboratory to get a view of the working of the instruments. In addition, the Operational Specifications chart can be used, which is a graph like the method decision chart, that tells about various rules and controls required to provide the necessary error detection. | |||||||||||||||||||||||||||||||||||||||
CONCLUSION | |||||||||||||||||||||||||||||||||||||||
Sigma metric analysis can serve as a tool to identify the poor assay performance and to assess the efficiency of processes that are in existence. In this study, the sigma values observed for magnesium, ALP, triglyceride, and HDL cholesterol, in both the levels were >6. A satisfactory sigma (>3) was obtained for creatinine, total protein, and phosphorus, in both the IQC levels. Whereas a sigma value of <3 was observed for sodium and potassium in both levels. The six-sigma purpose is to lessen both variance and QC processes to assure compliance with the critical specifications. The health care sector can be immensely benefited by implementation of sigma metrics for QA. | |||||||||||||||||||||||||||||||||||||||
|
|||||||||||||||||||||||||||||||||||||||
TABLES AND FIGURES | |||||||||||||||||||||||||||||||||||||||