Original article / research
|
|||||||||||||||||||||||||||||||||||||
Sigma Metrics in Thyroid Testing: Striving for Perfection- An Observational, Cross-sectional Study |
|||||||||||||||||||||||||||||||||||||
|
Sekh Rajib, Kasturi Mukherjee, Bithika Ghosh, Raghunath Bhattacharyya, Mousumi Mukhopadhyay 1. Senior Resident, Department of Biochemistry, IPGMER, Kolkata, West Bengal, India. 2. Assistant Professor, Department of Biochemistry, IPGMER, Kolkata, West Bengal, India. 3. Assistant Professor, Department of Biochemistry, Burdwan Medical College, Kolkata, West Bengal, India. 4. Associate Professor, Department of Obstetrics and Gynaecology, IPGMER, Kolkata, West Bengal, India. 5. Professor, Department of Biochemistry, IPGMER, Kolkata, West Bengal, India. |
|||||||||||||||||||||||||||||||||||||
| Correspondence
Address : Kasturi Mukherjee, 244, A.J.C. Bose Road City, Kolkata-700020, West Bengal, India. E-mail: kbdrkm@gmail.com |
|||||||||||||||||||||||||||||||||||||
| ABSTRACT |  | ||||||||||||||||||||||||||||||||||||
| : Six Sigma is a potent tool for evaluating the quality of the analytical phase by combining bias, imprecision, and Total Allowable Error (TEa). Considering the variation in TEa values from different sources, analysis on the sigma scale needs to be carefully monitored. Aim: To assess the performance of Thyroid Stimulating Hormone (TSH) and free Thyroxine (fT4) on the Sigma scale. Materials and Methods: The present observational, cross-sectional study Immunoassay laboratory at the Department of Biochemistry, IPGMER and SSKM Hospital, Kolkata, India, from February 2021 to January 2022. The study involves 12 months of tri-level (L1, L2, L3) Internal Quality (IQ) control data and External Quality Assessment (EQAS) data. The bias percentage was obtained from EQAS, and the Coefficient of Variation (CV%) was obtained from IQ Control (IQC) data run on Advia Centaur CP (CLIA) each month. Sigma (σ) was calculated applying TEa from the desirable biological variation database. Sigma values of L1, L2, L3 of TSH and fT4 have been calculated using Microsoft spreadsheet software version 2010, applying the formula σ=(TEa-bias)/CV. Results: The CV% and bias% were found to be within an acceptable range, always less than the cut-off percentage of imprecision (I%) and inaccuracy (B%) for TSH and fT4 in the desirable specifications for imprecision and inaccuracy (updated 2014). However, sigma levels are near the satisfactory mark, found to be <5σ over 17 months for TSH and 23 months for fT4, considering all sigma values of L1, L2, and L3 IQ levels. Better sigma values (>5σ) in tri-level IQ for TSH with a higher numeric TEa% value (23.7%) were obtained in more months, whereas for fT4 with a lower TEa% value (8%), better sigma values (>5σ) were obtained in a lesser number of months. Conclusion: The present study establishes that Sigma values are affected by the numeric values of TEa% of a particular parameter taken from the source. Sigma values showed average performance despite satisfactory CV% and bias% for fT4 and TSH, creating chaos in the laboratory’s operational routine. The Sigma matrix is a good indicator, but it is difficult to maintain a good sigma value for parameters that have low TEa%. It becomes crucial to choose appropriate TEa to plan a quality control strategy for thyroid hormones. | |||||||||||||||||||||||||||||||||||||
 |
|||||||||||||||||||||||||||||||||||||
| Keywords : Coefficient of variation, Free thyroxine, Thyroid stimulating hormone, Total allowable error | |||||||||||||||||||||||||||||||||||||
| INTRODUCTION | | ||||||||||||||||||||||||||||||||||||
| As laboratory analysis plays a pivotal role in diagnosis, prognosis, treatment protocol decisions and recurrence assessment, there should be minimal scope for error. The analytical phase contains certain incidences of errors, accounting for approximately 10% of the total error generation (1). When conducting periodic examination of a measurement procedure to verify that it is performing according to pre-established specifications like accuracy (bias%) and precision (CV%), it is necessary to have an indicator that reflects the overall system of activities. Considering possible sources of variation in thyroid levels and analysis, the Sigma scale will guide laboratory personnel to obtain a holistic view of the results and help them release accurate and precise reports. Total Error (TEa) combines the effects of systematic and random errors, including inaccuracy and imprecision in its calculation. The Six Sigma methodology should be used to evaluate the quality of the analytical phase by combining bias, imprecision, and TEa. The present study was aimed to calculate sigma from imprecision and inaccuracy, monitor monthly sigma over 12 months, and then assess the impact of TEa numerical data on the calculation of sigma for TSH and fT4. An immunoassay laboratory commonly encounters samples related to endocrine abnormalities of the thyroid gland. The sigma value in thyroid profile parameters indicates the occurrence of errors; the higher the sigma value, the less likely the laboratory report contains defects. However, many laboratories fail to implement sigma to plan a quality strategy, as it depends on variables from the source to use in the formula. Sigma calculation creates confusion and jeopardises quality control planning. The present study was aimed to explore facts related to calculation and evaluate the effectiveness of sigma as a tool to design quality control frequency planning for TSH and fT4. | |||||||||||||||||||||||||||||||||||||
| MATERIAL AND METHODS | | ||||||||||||||||||||||||||||||||||||
| This observational, cross-sectional study was conducted in the Immunoassay laboratory at the Department of Biochemistry, IPGMER and SSKM Hospital, Kolkata, India, from February 2021 to July 2022. After obtaining proper consent from the laboratory director, the quality and integrity of reference materials were ensured. Ethical clearance was obtained from the Institutional Ethics Committee, IPGMER (IPGMER/IEC/2022/098). Inclusion criteria: Randox Immunoassay Premium plus Level-I, II, III Quality Control materials, and Calibrators for fT4 Siemens ADVIA Centaur CAL Calibrator A (CAL A), Siemens ADVIA Centaur Calibrator for TSH were included in the study. Exclusion criteria: If IQC and External Quality Control (EQC) material was received in inadequate quantities, or if the homogeneity of reference materials was affected, materials transported in improper means were excluded from the study. Study Procedure Internal quality and EQ materials were stored at 2-8 degrees celsius before reconstitution and at -20 degrees celsius after reconstitution in the refrigerator. Proper prerequisites for the study, such as air temperature, humidity and pressure, were taken into consideration, and strict biomedical waste management guidelines were followed (Bio-Medical Waste Management Rules, 2016, by Pollution Control Board, Dept. of Environment, West Bengal) (2). To calculate a sigma metric, one needs to determine the number of defects produced per million opportunities. The number of defects per million in a particular standard deviation, as per the normal Gaussian distribution with a perfectly centered mean, is known as the Sigma category. Sigma metrics of 1σ show 317,310 defects per million, 2σ show 45,500 defects per million, 3σ show 2699 defects per million, 4σ show 63 defects per million, and 5σ depict 0.573 defects per million, and 6σ 0.002 defects per million (3). This can be converted to a sigma metric by comparing it with the normal Gaussian distribution. Instruments: Twelve months tri-level (L1, L2, L3) Quality Control Data (February 2021 to January 2022) has been obtained. Serum Free Thyroxine (FT4) and TSH levels have been measured by the chemiluminescence method using the Advia Centaur CP immunoassay system (Siemens Healthcare Diagnostics Inc. Tarrytown, NY, 10591-5097, USA) with system pack (Siemens Advia Centaur TSH3UL Reagent, Siemens Advia Centaur FT4 Reagent). Proper AMC (Annual Maintenance Contract) and procurement of machines are being done as per standard guidelines. Methods: The Immunoassay laboratory has participated in the CMC (Christian Medical College) Vellore EQAS or Proficiency Testing (PT) scheme (EQAS program) and calculated bias% using PT samples as our reference material. No additional financial resources are required as study parameters are measured by reagents and machines procured within departmental resources, as per strict standard guidelines and standard operating procedure (4). Statistical Analysis The Sigma IQC and EQC data are tabulated in a Microsoft spreadsheet, software version 2010, by applying the TEa from the desirable biological variation database specifications as stated by J.O. Westgard (5). The TEa for TSH is taken as 23.7% and for fT4 it is 8% (5). The bias percentage is calculated using the formula {The present study lab result-Peer group mean (designated value)/Peer group mean (designated value)}×100. The CV% is calculated from the tri-level IQC data of a 10-day run in each month using the formula, CV%=(SD/Mean)×100 and σ is calculated as TE-bias/CV. | |||||||||||||||||||||||||||||||||||||
| RESULTS | | ||||||||||||||||||||||||||||||||||||
| Sigma values at three levels derived from tri-level control (L1, L2 and L3) for CV%, Bias% (from EQAS values), and TEa of TSH and fT4 ranged from 3 to 9 on the sigma scale from February 2021 to January 2022 [Table/Fig-1,2]. According to desirable biological variation specifications, the CV% and bias% for fT4 are 2.9% and 3.3%, respectively, and for TSH, 9.7% and 7.8%, respectively (4). Despite obtaining satisfactory CV% and bias% in all months for both parameters [Table/Fig-1,2], TSH and fT4 showed moderate performance on the sigma scale, <5σ in a total of 17 months for TSH and over 23 months for fT4, considering all sigma values of L1, L2 and L3 IQ levels. Better sigma values (>5σ) were achieved in the tri-level IQ for TSH, with a higher numeric TEa% value (23.7%) obtained in more months, whereas for fT4, with a lower TEa% value (8%), better sigma values (>5σ) were obtained in fewer months. Thyroid stimulating hormone, having a higher numerical value of TEa, demonstrated better performance on the higher side of the Sigma scale, achieving >6σ (excellent performance) in two months (L1), five months (L2), and two months (L3) for TSH, whereas only in one month for fT4, irrespective of L1, L2 and L3 data. However, the opposite was observed on the lower side of the Sigma scale, with <4σ (poor performance) in September in L1, November in L2, and June, November and December in L3, and 3σ in September in L3 for TSH. For fT4, considering all sigma values of L1, L2 and L3 IQ levels, only August showed <4σ (Table/Fig 3),(Table/Fig 4). | |||||||||||||||||||||||||||||||||||||
| DISCUSSION | | ||||||||||||||||||||||||||||||||||||
| Sigma metrics is a management strategy that helps improve the quality of process outputs by identifying and removing the causes of defects (errors). It provides a more quantitative framework for evaluating process performance, offering evidence for process improvement, and describes how many sigmas fit within the tolerance limits (6). Sigma, along with TEa of that parameter, acts as a determinant for assessing the quality of laboratory functioning. Quality is assessed on the “σ” scale, with a criterion of 3 “σ” as the minimum allowable sigma for routine performance and a sigma of 6 being the goal for world-class quality (7). It may act as a self-assessment tool for evaluating the functioning of a clinical laboratory (6). The stability and homogeneity of reagents have been regularly checked. There is a check of the machine for chemiluminescence signal generation and paramagnetic separation. Proper monitoring of work (regular preventive maintenance of the CLIA machine), in-house algorithm of laboratory workflows, and a special job list for lab technologists to maintain quality have all been ensured. Despite all these efforts, our immunoassay laboratory performance shows unfavorable sigma levels (around 3) achieved for both parameters, simply due to analytical instability. The manufacturer has provided documentation indicating no analytical issues with the instrument. All performance parameters are showing satisfactory results. These poor sigma values are obtained despite good bias% and acceptable CV% in all months. Isolated bias or CV% is considered a potent tool for evaluating the analytical performance of a laboratory. A detailed assessment of the analytical procedures has shown that TEa values taken from desirable biological variation database specifications and Westgard results in unsatisfactory Sigma values. TEa refers to the allowable difference from the true value, that is, the degree of change that needs to be detected in an analyte for a clinically important decision to be made. The Sigma matrix is a good indicator, but obtaining and maintaining a good sigma value for parameters with low TEa% can be quite challenging. Symptoms of hypothyroidism are non specific and highly prevalent in the population. The long-standing reliance on TSH and fT4 has come under increased scrutiny in the public domain, leading medicine providers to question the reliability of standard biochemical testing of thyroid function (8). Clinical labs have a responsibility to assess the testing process on a sigma scale. If the difference between the true concentration of an analyte and the reported concentration in a patient’s specimen exceeds TEa, the result is considered unreliable. The sigma metric expresses the number of analytical standard deviations of the test system process that fit within the specified allowable total error limits. Sigma values indicate the laboratory’s quality goal in thyroid testing, instilling confidence in releasing reports. In the IPGMER immunoassay laboratory, sigma matrix calculation helped us implement the correct quality control strategy and reduce the cost of the test procedure, improving patient compliance. A new QC strategy has been designed to rectify errors since the sigma was not satisfactory. However, good CV% and bias% of each parameter (as per desired biological variation specification) created confusion when choosing a stringent QC plan for both TSH and fT4 (Table/Fig 5) (9),(10),(11),(12),(13),(14),(15),(16),(17). TSH, having a higher numerical value of TEa, showed better performance on the higher side of the Sigma scale, whereas fT4, with lower numerical values of TEa, showed poor performance on all levels of the sigma scale (Table/Fig 1),(Table/Fig 2). The sigma, as determined through the said equation, is influenced by TEa values, corroborating with other studies (10),(11). The present study encountered trouble in obtaining suitable TEa required for sigma calculation, as the TEa picked up from the biological variation database showed unsatisfactory results. The International Federation of Clinical Chemistry (IFCC) official task force has started to develop sigma metrics to be a meaningful technique for the assessment of laboratory performance. Unfavourable sigma found in the present study laboratory demands suitable TEa values. The lack of appropriate TEa targets for the thyroid profile is a major variable in the interpretation and application of the sigma metrics in designing a QC plan. Limitation(s) The limitation of the present study is the use of a single EQ scheme and commercially available controls for the calculation of bias and imprecision, respectively, as well as, the inability to reveal the matrix effect on sigma. Financial constraints could explain the inability to use different materials to calculate the bias and imprecision. It is pertinent to mention that much attention is needed when choosing TEa sources, as there are multiple different literatures available. | |||||||||||||||||||||||||||||||||||||
| CONCLUSION | | ||||||||||||||||||||||||||||||||||||
| The present study realises that the selection of TEa is crucial and has a greater impact on the sigma metrics. Sigma metrics calculation depends on a certain formula, and numerical values of TEa can lead to variability. It is a challenge for every laboratory to select the right TEa to assess the quality requirements for the analytical process. This can either overestimate or underestimate the Sigma metrics and thus affect the performance. A proper TEa goal is needed to avoid chaos in the laboratory’s operational routine. The present study concludes with a demand to develop an appropriate TEa goal for thyroid hormones. | |||||||||||||||||||||||||||||||||||||
| ACKNOWLEDGEMENT | | ||||||||||||||||||||||||||||||||||||
| The authors would like to express their thanks to Professor, Dr. Indranil Chakroborty, Head of the Department of Biochemistry, West Bengal Medical educational service for his valuable suggestions. The authors are thankful to technologists of Immunoasay laboratory as they meticulously generated data set to calculate sigma values and help us to design the quality control policy. The views expressed are the sole responsibility of the authors. | |||||||||||||||||||||||||||||||||||||
|
|||||||||||||||||||||||||||||||||||||
| TABLES AND FIGURES | | ||||||||||||||||||||||||||||||||||||
|
|||||||||||||||||||||||||||||||||||||
|
|||||||||||||||||||||||||||||||||||||
