Quantifying operator subjectivity within flow cytometry data analysis as a source of measurement uncertainty and the impact of experience on results

Flow cytometry is a complex measurement characterization technique, utilized within the manufacture, measurement, and release of cell and gene therapy products for rapid, high-content, and multiplexed discriminatory cell analysis. A number of factors influence the variability in the measurement reported including, but not limited to, biological variation, reagent variation, laser and optical configurations, and data analysis methods. This research focused on understanding the contribution of manual operator variability within the data analysis phase. Thirty-eight participants completed a questionnaire, providing information about experience and motivational factors, before completing a simple gating study. The results were analyzed using gauge repeatability and reproducibility techniques to quantify participant uncertainty. The various stages of the gating sequence were combined through summation in quadrature and expanded to give each participant a representative uncertainty value. Of the participants surveyed, 85% preferred manual gating to automated data analysis, with the primary reasons being legacy (“it’s always been done that way”) and accuracy, not in the metrological sense but in the clear definition of the correct target population. The median expanded uncertainty was calculated as 3.6% for the population studied, with no significant difference among more or less experienced users. Operator subjectivity can be quantified to include within measurement uncertainty budgets, required for various standards and qualifications. An emphasis on biomanufacturing measurement terminology is needed to help understand future and potential solutions, possibly looking at translational clinical models to engage and enhance better training and protocols within industrial and research settings.