Ruggedness and reproducibility of the MBEC biofilm disinfectant efficacy test Authors: Albert E. Parker, Diane K. Walkera, Darla M. Goeres, N. Allan, M.E. Olson, & A. Omar NOTICE: this is the author’s version of a work that was accepted for publication in Journal of Microbiological Methods. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. A definitive version was subsequently published in Journal of Microbiological Methods, 102, July 2014, DOI#10.1016/j.mimet.2014.04.013. Parker AE, Walker DK, Goeres DM, Allan N, Olson ME, Omar A, "Ruggedness and reproducibility of the MBEC biofilm disinfectant efficacy test," Journal of Microbiological Methods, July 2014 102:55–64 Made available through Montana State University’s ScholarWorks scholarworks.montana.edu Ruggedness and reproducibility of the MBEC biofilm disinfectant efficacy test A.E. Parker a,b,⁎, D.K. Walker a, D.M. Goeres a, N a Center for Biofilm Engineering, Montana State University, Bozeman, MT 59715, USA b Department of Mathematical Sciences, Montana State University, Bozeman, MT 59715 c Innovotech Inc., Edmonton AB T6N 1H1, Canada A B S T R A C T ds: lm ess ility The ly stand eth resu om rugg ha tive) to bact te dura 0 repe st repr stu concentrations of each of 3 disinfectants (a non-chlorine oxidizer, a phenolic, and a quat method. The repeatability and rep acceptable, as indicated by s deviations (SD) (0.33 and 0.67 SDs uctio disin nged rang addi the sign conc I N T M nt effic ed by t an and 7; Behnke et al., 2011 hospital disinfec- tant cy agai to subs B year avaiclaims in the US (Tomasino et al., 2012)) are not appropriate to test disinfectant effica nst biofilms. Although regulatory agencies in the US still have not adopted any method Costerton, 2002; Davies, 2003; Fux et al., 2004; Buckingham-Meyer et al., 200 ). Thus, suspension tests and dried surface tests (currently used to support embers of academia, industry and regulatory agencies have come to recognize that disinfecta acy against biofilms (i.e. microorgan-isms attached to surfaces) cannot be adequately studi raditional microbiological methods that were developed for planktonic cells (e.g., see Donltantiate biofilm efficacy claim iofilm methods for research s, however, have some of th lableR O D U C T I O N of the biofilm log red fectant combinations ra ed from 0.27 to 1.70. In assay was statistically entrations.roducibility of the untreated control biofilms were mall repeatability and reproducibility standard log10(CFU/mm 2), respectively). The repeatability ns after application of the 24 concentration and from 0.22 to 1.61, and the reproducibility SDs tion, for each of the 3 disinfectant types considered, ifi-cantly responsive to the increasing treatment ernary ammonium compound) were applied to biofilms using the MBEC MBEC™ Physiology & Genetics Assay recent ardized biofilm disinfec-tant efficacy test m lts of the standardization process using Pseud edness testing of the MBEC method suggests t to small changes to the protocol with respect eria (when varied from 16 to 18 h), treatment tion (25–35 min), and sonication power (13 atability of MBEC results across multiple te oducibility across multiple labs, an 8-lab s, this is expected purposes have bee ese research meth. Allan c, M.E. Olson c, A. Omar c , USA Keywor Biofi Ruggedn Reproducib became the first approved ASTM od. This report summarizes the onas aeruginosa biofilms. Initial t the assay is rugged (i.e., insensi- 4 factors: incubation time of the mperature (20–24 °C), sonication –480 W). In order to assess the s in the same laboratory and the dy was conducted in which 8 in the next few years. n available for decades. Only in the past few ods completed the arduous process to become concentration (MBEC) device, a low shear reactor system where the biofilm is grown under batch conditions (i.e., there is no continuous supply of media to the biofilm), complements this suite of available biofilm growth methods. The MBEC device (originally referred to as the “Calgary device”), depicted in Fig. 1, is a widely used, 96-well plate system for growing biofilm that allows for high throughput screening (Ceri et al., 1999; Harrison et al., 2010). Biofilm is grown on pegs on a lid that fits over the 96 wells. With such a large number of wells, the pegs with the established biofilm may be immersed in multiple treatments and controls simultaneously. The layout of the 96wells used in the tests de- scribed in this paper is depicted in Fig. 2. Forty wells were used to study the effect of the application of 8 concentrations of disinfectant on the biofilm bacteria, while the remaining wells contained various media or reagents as controls. The MBEC assay was the first protocol approved by ASTM that is both a biofilm growth and disinfectant protocol (ASTM E2799). It is also the first ASTM biofilm method substantiated by a multi- laboratory (collaborative) study (ASTM RR:E35-1006, 2011). The purpose of the collaborative study was to quantitatively assess: the re- peatability across multiple tests and the reproducibility across multiple labs of the biofilm log reductions after application of the disinfectant treatments; the resemblance of the control biofilms across labs; and the responsiveness of the assay to detect increased log reductions in the biofilm as the concentration of the chemical was increased. Before the collaborative study began, theMBEC assay was subjected to a ruggedness test in order to assess the sensitivity of the assay to small changes of four important operational factors. Ruggedness test results provide regression equations that quantify how changes in the operational factors affect the responses of the MBEC assay. The magni- tudes of these changes were chosen to reflect the range of factor values that the method may be subjected to by multiple users at different labs. Ruggedness testing is rarely used in microbiology, which is unfortunate as there aremany benefits of this predictive test. Two standardized bio- film growth methods have previously been subjected to ruggedness testing: the CDC reactor (Goeres et al., 2005) and the DFR (Goeres et al., 2009). One purpose of this paper is to report the performance of the MBEC assay in the ruggedness test and in the subsequent collaborative study. We predict that this is the first of an infusion of new standardized methods needed to assess anti-biofilm efficacy over the next decade. Thus, the second purpose of this paper is to help inform this future work by clearly elucidating the steps that we took to standardize the first disinfectant test against biofilms.Fig. 1. A photograph of the MBEC plate.2. Methods 2.1. The MBEC method 2.1.1. Growth and treatment To initiate the standardized MBEC method (ASTM E2799), the wells of an MBEC device (Innovotech Inc., Edmonton, Ab. Canada), excluding those for sterility control (Fig. 2, “SC”), were filled with 150 μL of tryptic soy broth inoculated with 105 cells/mL of P. aeruginosa ATCC 15442, and then incubated for 16–18 h at 35 ± 2 °C on an orbital shaker at 110 ± 10 rpm to establish a biofilm on the pegs located on the lid of the device. Once the biofilm was grown, 5 pegs were detached from the lid to determine the initial density of bacteria (Fig. 2, “PC”) by immersing each peg into 0.2 mL buffered water, and then using sonica- tion to remove the biofilm from the pegs and to disaggregate the biofilm into the 0.2 mL volume. A 10 μL volume was removed from this 0.2 mL volume for a 10-fold dilution series. A “challenge plate” was prepared according to the layout in Fig. 2 with each well containing 200 μL/well of treatment(s) or control media/reagents. The lid with the remaining biofilm-covered pegs was placed on top of the challenge plate for the specified contact time for the disinfectant treatment. The lid was then transferred to a 96 well plate with each well containing 200 μL of neu- tralizer, and then the biofilm on the remaining 91 pegs was sonicated for 30 min to remove and disaggregate the biofilm. The contents of each rowwere then transferred to a new 96well plate for serial dilution followed by spot plating 10 μL on tryptic soy agar (TSA) in order to determine viable cell concentrations. 2.1.2. Device layout As depicted in Fig. 2, for each treatment, the first 5 wells of row A contained the highest concentration of disinfectant; row B contained half the concentration (50%) compared to row A; row C contained half the concentration from row B (25%); and so on until the most dilute concentration in row H contained half the concentration from row G (0.78%). Column 6 contained a volume of the disinfectant at the 100% concentration mixed with an equal volume of neutralizer (“50:N”). The wells of column 7 contained neutralizer only (“N”). Column 8 contained buffered water and was the untreated control (“UC”). Columns 9-11 are not used in the standard method and remain empty as do thewells in rows D–H of column 12 since the pegs are used for preliminary biofilm counts (“PC”). RowsA–C of column 12 contained sterility (negative) controls (“SC”) of the growthmedia, buffered water, and neutralizer. 2.2. Quantification of biofilm bacteria At each dilution d in the serial dilution series, the number of colony forming units (CFU) was enumerated, scaled up by the dilution factor 10d and then reported as 10d × CFU/10 μL. These bacterial counts were then converted to a log density (LD) permm2 of surface area using the following equation, LD ¼ log10 CFU=mm2   ¼ log10 10dCFU=10 μL   V=SAð Þ h i where V is the volume sonicated into (0.2mL); and SA is the surface area of the peg (46.63 mm2). Whenever no CFUs were observed in all dilu- tions, a ½ CFU was substituted in at the lowest dilution plated, and then scaled up (Hamilton et al., 2013). Since the 0th dilutionwas always plated, then after applying the CFU substitution rule, LD ¼ log10 0:5CFU=0:01mLð Þ 0:2 mL=46:63 mm2  h i ¼ log10 0:2145 CFU=mm2 h i ¼−0:67 was the substituted value for the LD. Note that changing units to log10 2(CFU/cm ) would increase all LDs by +2, and leave the standard 2.4. Ruggedness testing 2.4.1. Ruggedness test experimental design studeviation (SD) of the LDs unchanged. In otherwords, the SDof the LDs is unitless. Two separatemicrobial responseswere considered in all subsequent statistical analyses. First, the mean LD of organisms attached to the 8 untreated control pegs (in column 8, labeled “UC”) of each MBEC plate was calculated. Second, on each plate, the response of the biofilm grown on pegs and then subjected to disinfectants at the 8 different concentrations was quantified by the log reduction (LR), which is the mean LD of biofilm on the eight untreated control pegs minus the mean LD of biofilm that survived on the five pegs treated with a disin- fectant in one of rows A–H in columns 1–5. Thus, the LR is unitless. For each treatment concentration on a plate, denoted by rows A, B, C, D, E, F, G, and H in the MBEC plate, one LR is calculated. 2.3. Neutralization For all disinfectants and concentrations considered in this paper, D/E neutralizing broth was used (Neogen Corporation, Lansing, MI). The MBEC challenge plate contained two columns, each with 8 wells that allow for the evaluation of the neutralizer in two ways (Fig. 2): (1) was the disinfectant neutralized? and (2) does the neutralizer itself impact the biofilm? To answer these questions, the eight wells in column 6, labeled “50:N”, contain a neutralized disinfectant: 100 μL of the 100% concentration of the disinfectant and 100 μL of the neutralizer. The biofilms grown on pegs and then subjected to these wells were compared to the controls to determinewhether the neutralizer satisfac- torily neutralized the disinfectant. This 1:1 disinfectant to neutralizer ratio in the “50:N” column is a conservative check of the neutralizer's ability to inactivate the active compound. In practice, the biofilm- Fig. 2. Layout of the MBEC plate used in the ruggedness testing and multi-labladen lid that was exposed to the disinfectant is transferred to a fresh microtiter plate containing only neutralizer, in which case the disinfec- tant to neutralizer ratio is much larger (about 1:200 to 3:200, cf. Section 2.1.1). The eightwells in column7, labeled “N,” contain just neu- tralizer. Comparing the biofilm subjected to thesewells to the untreated control biofilms determined whether the neutralizer itself inhibits or otherwise affects biofilm bacteria. A mixed effects statistical model was used to test the efficacy of the neutralizer for each disinfectant sep- arately, with nested random effects due to lab, experimental daywithin a lab, and multiple wells within a plate per day. Treatment (with three levels, neutralized disinfectant (in column “50:N”), Neutralizer (in col- umn “N”), and the controls (in column “UC”)) was the fixed effect. As is commonly done for planktonic assays using suspended bacteria, all pairwise comparisons of the mean LDs amongst these levels were per- formed (ASTM E1054). These comparisons were made using Tukey's multiple comparison procedure. However, since we believe that it is more informative to assess statistical equivalence as opposed to failing to find statistical differences when assessing neutralizers, equivalence tests were performed by calculating 3 simultaneous 2-sided 90%In order to estimate both main effects and two-way interactions amongst the four factors, experiments were run at low, medium and high levels of the factors of interest (Table 1). These low and high levels were chosen in order to span a realistic range of deviations to the proto- col when conducted in different labs. Experiments at specific combina- tions of the factors were chosen according to a fractional factorialA ruggedness test determines the sensitivity of a protocol to small changes in operational factors. For the ruggedness test of the MBEC assay, the factors tested were treatment temperature (Temp), sonica- tion duration (SonDur), sonication power (Power), and incubation time (Inc). The non-chlorine oxidizer disinfectant Virkon (DuPont, Wilmington, DE) was used in all ruggedness experiments. For each MBEC test, Virkon was applied in 8 different concentrations (Fig. 2), starting at 535 mg/L (100%) in the five wells in row A of the plate, and then with half concentrations applied in each row thereafter (thus, 4.2 mg/L was tested in row H).Tukey confidence intervals for the truemean differences. If the 90% con- fidence intervals were contained in the interval [−0.5, 0.5] (Nelson et al., 2013), then statistical equivalence was concluded at 95% confi- dence (Richter and Richter, 2002; ASTM E2935). In other words, we as- sumed that mean LD differences less than½ logwere negligible and not of practical importance. dy. Section 2.1.2 describes the acronyms (50:N, N, UC, SC) used in the figure.experimental design and the run order was randomized as outlined in Table 2. Two of the ten pairs of experiments (runs 1 and 8) were run at standard operating procedure (SOP) values for all of the factors. These repeats (called “center points” in the statistical literature) allowed preliminary estimation of the variability of the control LDs and LRs when the method was run according to the SOP. Two techni- cians each performed the ten experiments in Table 2 side-by-side in order to minimize technician-to-technician variability in the experi- mental outcomes, thereby increasing the statistical power to detect Table 1 The four factors to be analyzed in the ruggedness testing, the short name used in subsequent tables, and the low, medium and high levels of each factor. Factor Short name Low Medium (SOP) High Incubation time Inc 16 h 17 h 18 h Treatment temperature Temp 20 °C 22 °C 24 °C Sonication power Power 130 W 250 W 480 W Sonication duration SonDur 25 min 30 min 35 min Caution in interpretation should be taken, however, since each coeffi- cient scales by the units of the factor, i.e., incubation time (h), sonication 2.5. Collaborative study 2.5.1. Study design An 8-lab study was conducted in 2011 as the last step of the process towards standardization of the MBEC assay with ASTM. Specific details of this study have been documented as a research report (ASTM RR: E35-1006, 2011). The main goal of this collaborative study was to determine the resemblance of the control LDs and of the repeatability and reproducibility of the LRs generated by the MBEC protocol. The responsiveness of the method to increasing concentrations of each treatment was also assessed (Hamilton et al., 2013). All 8 laboratories participated in an on-line training session prior to the study. During the collaborative study, 3 days of testing were performed at each lab. On each day, 3 disinfectants were tested, each on a separate plate. This experimental design satisfies ASTM's minimal requirements of 6 labs, 3 products, and 3 repeated tests of each product at each lab (ASTM E691). The 3 disinfectants used in this study were chosen to represent different classes of products: Virkon (DuPont, Wilmington, DE), a non-chlorine oxidizer (also used in the ruggedness test); Multi-Phenolic Disinfectant (Bio Agri Mix LP, Mitchell, Ontario, CA) (a phenol); and Biosentry 904 (Neogen Corporation, Lansing, MI), a quaternary ammonium compound (quat). The ranges of concentra- tions used in rows A–Hof theMBEC plates for each of the 3 disinfectants were 535–4.2 mg/L for the oxidizer; 1640–13 mg/L for the phenol; and Table 2 Fractional factorial schedule of ruggedness tests for one disinfectant and one technician. Run Inc Temp Power SonDur 1 17 h 22 °C 250 W 30 2 18 h 20 °C 130 W 25 3 16 h 24 °C 480 W 35 4 18 h 24 °C 480 W 25 5 18 h 24 °C 130 W 35 6 18 h 20 °C 480 W 35 7 16 h 20 °C 480 W 25 8 17 h 22 °C 250 W 30 9 16 h 20 °C 130 W 35 10 16 h 24 °C 130 W 25duration (min), sonication power (W) and treatment temperature (°C). The effect of each factor was further assessed by significance tests, mainthe effects of the four factors. Thus, the ruggedness test results are based on 20 MBEC runs, all performed in the same lab. 2.4.2. Assessing factor effects To assess the effect of each of the four factors of interest, a nested mixed effects statistical model was fit separately to the control LDs and the LRs. The three nested random effects in themodel were techni- cian, experiments (or plates) nestedwithin technician, andwells within an experiment. The fixed effects in the models were the four factors given in Tables 1, and 3 of the 6 two-way interactions. A covariate for “Biofilm Growth” was also included, defined as the mean LD of the “PC” pegs in the MBEC plate column 12 (Fig. 2). Interactions were assessed via interaction plots and significance tests. The statistical model for the LRs was the same as just described ex- cept that, instead of a covariate for Biofilm Growth, it included a covar- iate for disinfectant concentration. From the LRs calculated across the 8 different disinfectant concentrations in rows A–H on the MBEC chal- lenge plate, a dose–response curve was plotted. The covariate for disin- fectant concentration (DisConc) simply characterizes this curve with a line. For ease of interpretability, the covariate for disinfectant concentra- tion in MBEC plate rows A, B, C,…, H was encoded as the whole num- bers 7, 6, 5, 4,…, 0 respectively. These values are equivalent to taking the transform DisConc = log2(nominal concentration in mg/L) − log2(4.2 mg/L) where 4.2 mg/L is the lowest concentration of Virkon used (in row H of the MBEC challenge plate). The output of the statistical analyses is a regression equation, with a coefficient estimate and significance test for each factor and two-way interaction. The regression equations always contain all themain effects regardless of statistical significance, but only contain statistically signif- icant two-way interactions. A small coefficient in the regression equa- tion suggests that a response (i.e., either the mean control LD or the LR) is ruggedwith respect to small changes in the corresponding factor.effect plots, and interaction plots. Table 3 Neutralization results from the 8-lab collaborative study. Satisfactory neutralization is concluded if all 90% confidence intervals (CI) are contained in the equivalence interval of [−0.5, 0.5]. Disinfectant Comparison Mean LR SE(LR) 90% CI Non-chlorine oxidizer Neutralized disinfectant −0.09 0.0405 [−0.17,−0.01] Neutralizer toxicity −0.04 0.0405 [−0.12, 0.04] Phenol Neutralized disinfectant 0.67 0.0526 [0.56, 0.77] Neutralizer toxicity −0.09 0.0526 [−0.19, 0.02] Quat Neutralized disinfectant 0.21 0.0424 [0.13, 0.30] Neutralizer toxicity −0.06 0.0424 [−0.14, 0.03]3680–29mg/L for the quat. Since the goal of the studywas to assess the MBEC method and not these products, these products are referred to throughout the manuscript by their generic names. There were a few missing data, apparently due to the spot plating techniques. One lab reported a missing tip on the low-volume multi- channel-pipetter, and another lab reported that when spot plating “sometimes it happened that certain drops came into contact with the direct drop nearby.” At Lab 5, the phenol treatment on Day 2 had measurements from only 4 out of 5 pegs; the quat on Day 2 had only 4 measurements; and the quat on Day 3 had only 4 measurements. At Lab 6, there was no row H data for the quat on Day 1 for either the disinfected or control biofilms. All questions regarding the data were brought to the attention of the study director. All data were validated, and since no deviations from the study protocol were identified, all data collected from all 8 labs were in- cluded in the statistical analyses (consistent practicewith the guidelines in section 19 of ASTM E691). SonDur Power Inc Temp 353025 480130250480130 181618161718161816 202424202224202024 7.0 6.5 6.0 5.5 5.0 4.5 4.0 3.5 LD Fig. 3. Pooled across 2 technicians and 20 plates used in the ruggedness tests, the log densities (LD = log10(CFU/mm2)) of the untreated control biofilms grown using the MBEC are plotted. Each point corresponds to the LD of biofilm bacteria on a single control peg. The data in the rectangle are from plates 1 and 8 per technician that were run at the SOP values of the four operational factors. 2.5.2. Assessing resemblance, repeatability and reproducibility of the MBEC method To assess the resemblance of the LDs of untreated control biofilms across labs, a statistical model was fit with nested random effects due to lab, experimental day within a lab, plates within an experimental day, and wells within a plate. It was possible to estimate the plate vari- ability within each day because each lab ran three plates on each test LR for each disinfectant and concentration were also assessed using a 2.6. Other statistical details The analysis of the collaborative study data followed the guidelines for disinfectant tests described in Hamilton et al. (2013). Mixed effects models (Pinheiro and Bates, 2000) specifically for disinfectant efficacy data are described in detail by Parker and Hamilton (2011). Themethod of restricted maximum likelihood method was used to fit the nested mixedmodels using the software R v.2.11.0 (R Core Team, 2012), pack- age nlme (Pinheiro et al., 2012); the R code is provided in Parker and Hamilton (2013). Three types of plots were used to identify outliers and to check the fit of the statistical models: individual value, residual and normal probability plots. An additional check for outliers was to perform the hypothesis tests recommended by ASTM (cf. sections 16 and 17 in ASTM E691). All graphs were rendered by Minitab v.16. Minitab was also used to generate the fractional factorial experimental design for the ruggedness testing. All statistical tests, including the equivalence tests, were conducted at a significance level of 5%. 3. Results 3.1. Neutralization Table 4 Summary statistics for the regression equation fit to the log densities (LD = log10(CFU/ mm2)) of untreated control biofilms from the ruggedness tests. Units for the factors are provided in Table 1. Since therewere no significant 2-way interactions, themodel contains only main effects. Factors Estimate SE p-value Intercept 5.027 0.2445 b0.00005 Inc—17 0.1111 0.1146 0.3500 SonDur—30 −0.0042 0.0224 0.8534 Temp—22 −0.1178 0.0572 0.0601 Power—250 0.00036 0.0006 0.5616 BiofilmGrowth—5.87 0.3893 0.2217 0.1026mixed model with a random effect for lab. 2.5.3. Assessing responsiveness of the MBEC method To assess responsiveness of themethod to the 8 increasing disinfec- tant concentrations in rows A–H in the challenge plate, a dose–response curve with respect to the LRwas constructed. One approach to quantify responsiveness is with the slope of a regression line. Such a line was fit to the LRs for each disinfectant using a mixed effects model with random effects due to lab and tests within a lab, and a covariate for disinfectant concentration (see Section 2.4.2). 6 5day, one for each of the three disinfectants considered. The controls from all three of these plates were considered simultaneously in the same statistical analysis since they were processed similarly, including the use of the same D/E neutralizer. The model just described gives a pooled estimate of the repeatability SD of the method at a single lab by pooling the repeatability variances across all labs. The repeatability SD was also calculated at each lab separately using a model with ran- dom effects for experimental day, plates within an experimental day, and wells within a plate. The repeatability and reproducibility for theplate row 4321 & 8 EFGHABCDEFGHABCDEFGHABCDEFGHABCDEFGH 4 3 2 1 0 LR Fig. 4. From ruggedness tests, the log reductions (LR) as a function of disinfectant concentra concentration for one of two technicians. The plate numbers refer to the sets of experimental cThe results of the neutralization tests conducted during the collaborative study are presented in Table 3. For all 3 disinfectants tested, the mean LD of biofilm bacteria subjected to the control or neutralizer-only wells were statistically equivalent. For the non- chlorine oxidizer and the quat, the mean LD of biofilm bacteria subject- ed to the control or neutralized disinfectant wells were also statistically equivalent. Unfortunately, for the phenol, the control and neutralized disinfectant wells were not statistically equivalent. In fact, there was a statistically significant difference between the mean LD of the biofilm subjected to the “neutralized phenol” wells compared to the controls (p-value b 0.00005). Thus, the phenol in row A of the MBEC device did not appear to be adequately neutralized in the 8-lab study. 3.2. Ruggedness testing 3.2.1. Controls Fig. 3 depicts the LDs of the control biofilms for both technicians across the 9 experimental conditions outlined in Table 2. The tech-to- tech variability for the 4 SOP runs was estimated to be zero and was not statistically significant (p-value = 0.751). As the four factors were 109765 ABCDEFGHABCDEFGHABCDEFGHABCDEFGHABCD tion (labeled by MBEC rows A, B, C, D, E, F, G, and H). Each point is a LR at a disinfectant onditions in Table 2. varied from the SOP values (according to Table 2), there were no signif- icant two-way interaction effects on the control LDs (p-value≥ 0.2350). The model fit to the mean control LD with only main effects is LD ¼ 5:03þ 0:1111 Inc–17ð Þ−0:0042 SonDur−30ð Þ−0:1178 Temp–22ð Þ þ 0:0004 Power–250ð Þ þ 0:3893 BiofilmGrowth–5:87ð Þ: (p-value≥0.1795).However, two-way interactions betweenDisinfectant Concentration and each of treatment temperature and incubation were significant (p-value ≤0.0467). The model that included the significant two-way interactions and all of the main effects for the four factors is LR ¼ 0:2157–0:457 Inc–17ð Þ þ 0:0015 SonDur−30ð Þ–0:0696 Temp–22ð Þ þ 0:0003 Power–250ð Þ þ 0:7866DisConc −0:0305 Temp–22ð Þ  DisConcþ 0:0832 Inc–17ð ÞDisConc This equation has been centered at the middle values of the factors. Thus, the constant 0.2157 is the predicted LR when DisConc = 0 (corresponding a disinfectant concentration of 4.2 mg/L in row H). When DisConc = 7 (corresponding to 535 mg/L of disinfectant as in row A), the equation predicts a LR = 5.72. Table 5 gives a summary of the significance tests performed for each factor in the model. Just as for the control data, sonication duration (p-value = 0.9240) and sonication power (p-value = 0.5389) do not significantly affect the LR. The coefficient for DisConc is 0.7866 and is statistically significantly positive (p-value b 0.00005). Thus, when all of the factors are at their SOP values, as the disinfectant is increased Table 5 Summary statistics for the regression equation fit to the log reductions (LR) from the ruggedness tests. Units for the factors are provided in Table 1. The model depicted here contains all main effects and only statistically significant 2-way interactions. Factors Estimate SE p-value Intercept 0.2157 0.9206 0.8151 Inc—17 −0.4570 0.1308 0.0036 SonDur—30 0.0015 0.0152 0.9240 Temp—22 0.0696 0.0654 0.3052 Power—250 0.0003 0.0004 0.5389 DisConc 0.7866 0.0272 b0.00005 (Inc—17) × DisConc 0.0832 0.0304 0.0071 (Temp—22) × DisConc −0.0305 0.0152 0.0467The equation has been centered at the middle values of each of the factors. Hence, 5.03 log10(CFU/mm2) is the predicted mean LD of the control biofilms when all of the operational factors are set to their SOP values, and when the mean LD of the five biofilm growth check pegs (“PC”) is 5.87 log10(CFU/mm2). Table 4 summarizes the statistical analysis of the control biofilm LDs. None of the factors had a significant effect on the mean control LD (p-value≥ 0.0601). The small coefficients for the factors in the equation for the mean control LD explicitly show the ruggedness of the MBEC assay to changes in the four factors. For example, the coefficient of −0.0042 for SonDur estimates that for every increase of sonication duration by 1min (in the range 25–35 min), themean LD of the control wells will decrease on average by 0.004 log10(CFU/mm2). Keep in mind that calculations based on this equation are valid only for values of the factors in the ranges outlined in Table 1. 3.2.2. Treated data The LRs are given as a function of the 8 concentrations of disinfectant (in rows A−H of theMBEC challenge plate) in Fig. 4. The LRs calculated from the4 SOP runs (plates 1 and8 per technician) are plotted in the left panel of Fig. 4. None of the two-way interactions amongst the 4 factors had a significant effect on the LRs (p-value≥0.3522). None of the 3-way interactions with Disinfectant Concentration were significant either 8 7Lab Day plate 4321 321321321321 123123123123123123123123123123123123 6 5 4 LD Fig. 5. Control data from the 8-lab study. Each point in the graph is the log density (LD= log10( control well in an MBEC plate. Along the horizontal axis are listed the eight labs, the three expup a single row of the MBEC device (e.g., going from row E to row D, corresponding to DisConc increasing by 1), then it is predicted that the LR will increase by 0.7866. This demonstrates that the MBEC assay was statistically significantly responsive to increasing disinfectant concentrations, which is a required attribute of an efficacy test. Again, we emphasize that the regression equation is statistically valid only within the ranges outlined in Table 1. It is not prudent to extrapolate beyond these values. 3.3. Collaborative study The favorable ruggedness testing results indicated that the MBEC assay was ready for the next step in standardization, a collaborative study. The untreated control LDs from the collaborative study are plotted in Fig. 5. Table 6 summarizes the means and variance components attrib- utable to different sources of variability. The repeatability SD is given for each lab individually in order to identify labs with unusual variabil- ity. All of the repeatability and reproducibility SDs given in Table 6 are acceptable compared to results from historical data published by the Center for Biofilm Engineering (Table 1 in Parker and Hamilton, 2011). This indicates acceptable resemblance of the control data across multi- ple experiments in a single lab, and also across multiple laboratories. 8765 321321321321 123123123123123123123123123123123123 CFU/mm2)) of biofilm bacteria grown on a singleMBEC peg then subjected to an untreated erimental days within each lab, and the three plates within each experimental day. For each disinfectant type considered in the collaborative study, Figs. 6–8 present the LRs as a function of increasing disinfectant efficacy. Figs. 9–10 provide a graphical summary of the repeatability and reproducibility SDs respectively for each of the 24 disinfectant and concentration combinations. The increasing LR curves in Figs. 6–8 suggest that, for each disinfec- shown that sonication can affect antimicrobial efficacy measures such as the LR. Monsen et al. (2009) showed that increasing sonication dura- tion reduced gram-negative bacterial biofilms, including P. aeruginosa. The effect of sonication is even more pronounced when used in conjunc- tion with antimicrobials or at increased temperatures (Piyasena et al., 2003; Fux et al., 2004; Monsen et al., 2009). Piyasena et al. suggest that Table 6 Repeatability and reproducibility of the log densities (LD = log10(CFU/mm2)) of untreated control biofilms from the2011 collaborative study. Themean LDhas units log10(CFU/mm2); the SDs are unitless. Lab No. exp Mean LD Sources of variability Repeatability SD Reproducibility SD Within plate % Among plate % Among exp day % Among lab % 1 3 5.50 40% 34% 25% 0.1369 2 3 5.58 20% 27% 53% 0.4206 3 3 4.27 39% 12% 49% 0.1696 4 3 5.92 17% 0% 83% 0.2315 5 3 5.80 64% 0% 36% 0.1624 6 3 5.72 8% 7% 85% 0.5301 7 3 6.13 76% 24% 0% 0.1438 8 3 6.16 51% 0% 49% 0.2706 All 8 24 5.48 4% 11% 9% 76% 0.3252 0.6669 xidtant, the MBEC assay is responsive to increasing disinfectant efficacy. The model fit to the LRs for each disinfectant provides a quantitative linear measure of the responsiveness of the method to increasing disinfectant efficacy. Table 7 shows that the MBEC assay is statistically significantly responsive to the increasing disinfectant efficacy for all three disinfectants. 4. Discussion A laboratory method is considered ready for standardization if it has a successful publication record by different researchers in different labs, provides quantitative results, and uses typical equipment requiring typical laboratory expertise (Hamilton et al., 2013). The MBEC assay met these criteria with more than a decade of referenced use (Ceri et al., 1999). The next step in the standardization process for the MBEC assay was to conduct a ruggedness test with respect to important opera- tional factors in the protocol. Since biofilm bacteria are attached to a surface, the bacteria must be removed and disaggregated in order to provide unbiased estimates of the density of biofilm bacteria (Hamilton et al., 2009). Given the use of soni- cation for this purpose in the MBEC assay (and in many microbiological tests), it was imperative to assess the ruggedness of the MBEC assay with respect to factors that control sonication. Previous research has 8 6 OLab row 4321 ABCDEFGHABCDEFGHABCDEFGHABCDEFGH 4 2 0 LR Fig. 6. Log reductions (LR) for the oxidizer treatment in the 8-lab study. The horizontal axis lists single concentration of disinfectant in a single experiment.sonication kills by “thinning cell membranes,” which could explain the results by Monsen et al. that gram-positive bacteria with thicker cell walls are less susceptible to sonication. Fux et al. suggest that synergistic kill occurs because sonication effectively disaggregates biofilms, thereby increasing the efficacy of an antimicrobial. Furthermore, and of great im- port for this study, Monsen et al. reported significant among-experiment and among-apparatus variability in their sonication experiments. The two sonication factors considered in the ruggedness tests for the MBEC were sonication duration and power. In addition, incubation time for growing the biofilms, and temperature during the disinfectant contact timewith the biofilm,were also assessed. Based on the previous research, two-way interactions between the sonication factors and each of treat- ment temperature and disinfectant concentration were of particular interest. Overall, the ruggedness testing showed that the results from the MBEC assay are not influenced by small changes of the operational fac- tors. Interestingly, the mean control LDs were not significantly affected by any of the four factors (Table 4). When a disinfectant was applied, only incubation time and treatment temperature had statistically signif- icant effects on the LRs, and these effects depended on the disinfectant concentration. Importantly, none of the two-way interactions among the four factors had a statistically significant effect on the LR (Table 5). Explicit equations (in Tables 4–5) quantify the simultaneous effect of izer8765 ABCDEFGHABCDEFGHABCDEFGHABCDEFGH all 8 labs, and the 8 concentrations tested at each lab. Each point in the figure is the LR for a Lab row 87654321 ABCDEFGHABCDEFGHABCDEFGHABCDEFGHABCDEFGHABCDEFGHABCDEFGHABCDEFGH 8 7 6 5 4 3 2 1 0 -1 LR Phenol Fig. 7. Log reductions (LR) for the phenol treatment in the 8-lab study. The horizontal axis lists all 8 labs, and the 8 concentrations tested at each lab. Each point in the figure is the LR for a single concentration of disinfectant in a single experiment.all 4 of these factors on themean LDof untreated control biofilms grown in the MBEC; and also on the LR of the biofilm after application of mul- tiple concentrations of a non-chlorine oxidizer. The small coefficients in these equations demonstrate precisely how rugged the MBEC results are to changes in these factors. For example, the coefficient of 0.0015 for SonDur (Table 5) predicts that as sonication duration increases from 25 to 35 min, the LR will increase on average by 0.015, provided that disinfectant concentration and the other factors stay constant. Similarly, as sonication power is increased by 100 W (in the range 130–480 W), it is predicted that the LR will increase on average by 0.03. Due to statistically significant interactions with disinfectant concentration, the effects of treatment temperature and incubation time do not have this same simple interpretation. Instead, the effects of these 2 factors depend on the disinfectant concentration. Upon completion of the ruggedness tests, the estimates of MBEC variability were crucial since theywere our first quantitative assessment of the repeatability of the MBEC across multiple experiments. These estimates are consistent with what was found with more confidence later in the 8-lab study, and so are not reported in this paper. One assess- ment of interest was that the technician-to-technician variability for the four ruggedness test SOP runs was estimated to be zero (i.e., between Lab row 4321 ABCDEFGHABCDEFGHABCDEFGHABCDEFGH 7 6 5 4 3 2 1 0 -1 LR Qu Fig. 8. Log reductions (LR) for the quat treatment in the 8-lab study. The horizontal axis lists al single concentration of disinfectant in a single experiment.5.78 × 10−12 and 1.26 × 10−11) and not statistically significant for either the control LDs (p-value≥0.751), or the LRs attained by the high concen- trations of the oxidizer (in MBEC rows A–D, p-value ≥0.553). For the less-efficacious treatments (in rows E–H), the technician-to-technician variancewas estimated to be larger than 0, but not statistically significant (p-value ≥0.156). These results were expected since the experimental designpurposely sought tominimize technician-to-techniciandifferences by requiring that the technicians independently conduct experiments side-by-side on the same day so that the effect of the factors could be studied with more statistical power. In general, a method with large technician-to-technician variabilitywouldnot be suitable for standardiza- tion and would likely have a large reproducibility standard deviation. The advantage of beginning the standardization process with ruggedness testing is that it can predict how well (or poorly) a method will perform across multiple labs while only requiring experiments from a single lab. For example, the initial MBEC protocol specified that incubation time could be set in the range 16–24 h (Table 1). Based on the ruggedness test results, the incubation time range in the standard- ized MBEC method was tightened to 16–18 h. Other than this change, the ruggedness test results indicated that the MBEC assay was well suited for the next step required for standardization, the collaborative 8765 ABCDEFGHABCDEFGHABCDEFGHABCDEFGH at l 8 labs, and the 8 concentrations tested at each lab. Each point in the figure is the LR for a study. The ruggedness test results also made it clear that in the multi- laboratory study, in addition to incubation time, treatment temperature would need to be tightly monitored and controlled. In preparation of the collaborative study, much effort was expended to identify a neutralizer that satisfactorily neutralized all 3 disinfectants at all 8 concentrations. As described earlier, while the oxidizer and quat were effectively neutralized (on the average) by D/E broth during the collaborative study, the high concentration of the phenol (i.e., at least the 100% concentration) was not. This may explain why the 100% concentration of phenol attained the highest mean LR (=5.64) in the 8-lab study. More generally, these results underscore that, compared to assays of planktonic bacteria, it is more challenging to neutralize the highly concentrated chemicals formulated to be effective against biofilms. One benefit of the MBEC is that disinfectant neutralization is monitored in every plate. This is an advantage over other disinfectant test methods, where neutralization tests are usually performed ahead of time in a separate experiment, and then neutralization is assumed to be adequate for all later experimental runs. The collaborative study design for the MBEC assay satisfied ASTM's minimum requirements of 6 labs, 3 products, and 3 repeats of each product at each lab. As advocated by ASTM (E691), based on the multi-lab study data, we assessed the repeatability of the biofilm LRs time correctly failing ineffective ones, it is imperative to know the repeatability and reproducibility of the method. 5. Conclusion In vitro studies remain a cornerstone for research, development, and registration of a variety of disinfectant andmedical technologies. Under- standing the performance of the underlying method in the controlled environment of the laboratory is a crucial step in order to provide con- vincing data that is repeatable across experiments, reproducible across labs, responsive to varying treatment efficacies, and rugged to changes in factors that may have been changed from the protocol. Extensive testing is required to assess these attributes for any method. Based on 1.8 1.6 1.4 1.2 1.0 du ci bi lit y SD Oxidizer Phenol. Quat. Dis. Table 7 From the 2011 collaborative study, responsiveness of the MBEC to increasing concentra- tions of disinfectant. The trend is interpreted as the mean increase in the log reduction (LR) as the concentration of disinfectant increases two-fold (as occurs when moving up one row in the MBEC plate). Disinfectant Trend SE p-value Non-chlorine oxidizer 0.87 0.0379 b0.00005 Phenol 0.87 0.0376 b0.00005 Quat 0.50 0.0275 b0.00005across experiments, and the reproducibility across labs. In addition, since methods designed to test antimicrobials are based on an underly- ing biological system, we also assessed 3 other criteria that are not advocated by ASTM (E691): disinfectant neutralization, resemblance of the controls across labs, and responsiveness of the assay to increasing disinfectant efficacy (Hamilton et al., 2013). TheMBEC assay's performance in the collaborative study is depicted in Figs. 9–10. The frown-shaped quadratic relationship of the SDs as a function of disinfectant efficacy is to be expected for any disinfectant test method (Springthorpe and Sattar, 2005). This is because, for disin- fectantswith large LRs,most of the viable biofilm bacteria on the treated pegs are killed, which results in small variability of the treated LDs, so the observed variability for the LRs is close to the variability for the untreated control LDs. For low efficacy disinfectants with small LRs, the viable bacteria on the treated pegs resemble the viable bacteria on the untreated pegs, so, again, the observed variability for the LRs is close to the variability for the untreated control LDs. For moderately effective treatments, only some of the biofilm bacteria are killed, which results in a more variable LR. Since the three disinfectant types used in the collaborative study are considered representatives of classes of disinfectants, the actual concentrations of the disinfectants tested in 6543210 1.6 1.4 1.2 1.0 0.8 0.6 0.4 0.2 0.0 Mean LR R ep ea ta bi lit y SD Oxidizer Phenol. Quat. Dis. Fig. 9. For the collaborative study, the repeatability SDs calculated as a function of the mean log reduction (LR) for all 24 combinations of the 3 disinfectants and the 8 concentra- tions tested.the collaborative study are not as important as the mean LR achieved for each concentration of each disinfectant type. Thus, the interested reader can, for example, predict the variability of a quat in a future MBEC study in a single lab by looking up the quat's mean LR in Fig. 9 and then reading off the attained repeatability SD for the quat in this study. Or one can predict the quat's variability in a future MBEC study across multiple labs by looking up the mean LR in Fig. 10 and then reading off the attained reproducibility SD for the quat in this study. Either of these scenarios could occur, for example, when performing tests for registration of a product with a regulatory agency. From a regulator's point of view, the repeatability and reproducibil- ity SDs reported in this paper can be used to craft performance stan- dards for the MBEC method in order to determine the number of tests (i.e., independent plates), and the number of replicate wells in each plate, in order to achieve a target LR with a desired level of confidence (Parker et al., 2014). Based on another collaborative study (Tomasino et al., 2012), this approachwas recently used to update the performance standard for the use-dilution method (UDM), the current method used by EPA to register products for hospital disinfectant claims (Tomasino et al., 2014). In order to have confidence that a performance standard for a method correctly passes effective products, while at the same 6543210 0.8 0.6 0.4 0.2 Mean LR R ep ro Fig. 10. For the collaborative study, the reproducibility SDs calculated as a function of the mean log reduction (LR) for all 24 combinations of the 3 disinfectants and the 8 concentra- tions tested. ruggedness tests and the 8-lab study performed in 2011, these attri- butes for the MBEC assay were assessed and found to be acceptable by ASTM's subcommittee on antimicrobial agents (E35.11). Thus, the MBEC method is a low shear, high throughput, batch system that is rugged, responsive, repeatable and reproducible; and complements the suite of reactor systems available for biofilm growth and disinfection. Acknowledgments The authors are grateful to the 8 labs that volunteered their time and resources to participate in the inter-laboratory study. We also would like to thank ASTM for providing the technical expertise in the experi- mental design and for supporting the statistical analysis of the multi- lab study. Innovotech conducted the ruggedness tests, supported the statistical analysis of the ruggedness tests, and provided the treatment chemicals to the labs in the collaborative study. Innovotech also partial- ly supported the writing of this manuscript. References ASTM research report RR:E35-1006, 2011. Interlaboratory Study for ASTM Method Davies, D., 2003. Understanding biofilm resistance to antibacterial agents. Nat. Rev. Drug Discov. 2, 114–122. Donlan, R.M., Costerton, J.W., 2002. Biofilms: survival mechanisms of clinically relevant microorganisms. Clin. Microbiol. Rev. 15, 167–193. Fux, C.A., Wilson, S., Stoodley, P., 2004. Detachment characteristics and oxacillin resistance of Staphylococcus aureus biofilm emboli in an in vitro catheter infection model. J. Bacteriol. 186, 4486–4491. Goeres, D., Loetterle, L., Hamilton, M., Murga, R., Kirby, D., Donlan, R., 2005. Statistical assessment of a laboratory method for growing biofilms. Microbiology 151, 757–762. Goeres, D.M., Hamilton, M.A., Beck, N.A., Buckingham-Meyer, K., Hilyard, J.D., Loetterle, L.R., Lorenz, L.A., Walker, D.K., Stewart, P.S., 2009. A method for grow- ing a biofilm under low shear at the air–liquid interface using the drip flow bio- film reactor. Nat. Protoc. 4 (5), 783–788. Hamilton, M.A., Buckingham-Meyer, K., Goeres, D.M., 2009. Checking the validity of the harvesting and disaggregating steps in laboratory tests of surface disinfectants. JAOAC 92, 1755–1762. Hamilton, M., Hamilton, G., Goeres, D., Parker, A., 2013. Guidelines for the statistical analysis of a collaborative study of a laboratory method for testing disinfectant product performance. JAOAC 96 (5). Harrison, J., Stremick, C., Turner, R., Allan, N., Olson, M., Ceri, H., 2010. Microtiter suscep- tibility testing of microbes growing on peg lids: a miniaturized biofilm model for high-throughput screening. Nat. Protoc. 5, 1236–1254. Monsen, T., Lövgren, E., Widerström, M., Wallinder, L., 2009. In vitro effect of ultrasound on bacteria and suggested protocol for sonication and diagnosis of prosthetic infec- tions. J. Clin. Microbiol. 47 (8), 2496–2501. Nelson, M., LaBudde, R., Tomasin, S., Pines, R., 2013. Comparison of 3M PetrifilmTM aero- bic count plates to standard plating methodology for use with AOAC antimicrobial efficacy methods 955.14, 955.15, 964.02, and 966.04 as an alternative enumerationaeruginosa Biofilm Using the MBEC Assay. ASTM International, West Conshohocken, PA. ASTM standard E1054, 2013. Standard Test Methods for Evaluation of Inactivators of Antimicrobial Agents. ASTM International, West Conshohocken, PA. ASTM standard E2196, 2012. Standard Test Method for Quantification of Pseudomonas aeruginosa Biofilm Grown with Medium Shear and Continuous Flow Using Rotating Disk Reactor. ASTM International, West Conshohocken, PA. ASTM standard E2562, 2012. Standard Test Method for Quantification of Pseudomonas aeruginosa Biofilm Grown with High Shear and Continuous Flow using CDC Biofilm Reactor. ASTM International, West Conshohocken, PA. ASTM standard E2647, 2013. Standard Test Method for Quantification of Pseudomonas aeruginosa Biofilm Grown Using Drip Flow Biofilm Reactor with Low Shear and Continuous Flow. ASTM International, West Conshohocken, PA. ASTM standard E2799, 2012. Standard Test Method for Testing Disinfectant Efficacy against Pseudomonas aeruginosa Biofilm using the MBEC Assay. ASTM International, West Conshohocken, PA. ASTM standard E2935, 2013. Standard Practice for Conducting Equivalence Testing in Laboratory Applications. ASTM International, West Conshohocken, PA. ASTM standard E691, 2013. Standard Practice for Conducting an Interlaboratory Study to Determine the Precision of a Test Method. ASTM International, West Conshohocken, PA. Behnke, S., Parker, A., Woodall, D., Camper, A., 2011. Comparing the chlorine disinfection of detached biofilm clusters with sessile biofilms and planktonic cells in single and dual species cultures. Appl. Environ. Microbiol. 77 (20), 7176–7184. Buckingham-Meyer, K., Goeres, D., Hamilton, M., 2007. Comparative evaluation of biofilm disinfectant efficacy tests. J. Microbiol. Methods 70, 236–244. Ceri, H., Olson, M.E., Stremick, C., Read, R.R., Morck, D., Buret, A., 1999. The Calgary Biofilm Device: new technology for rapid determination of antibiotic susceptibilities of bacterial biofilms. J. Clin. Microbiol. 37, 1771–1776.procedure: collaborative study. JAOAC 96 (4). Parker, A., Hamilton, M., 2011. Assessing resemblance, repeatability and reproducibility for quantitative methods. Center for Biofilm, Engineering, KSA-SM-10 (http://www. biofilm.montana.edu/content/ksa-sm-10). Parker, A., Hamilton, M., 2013. Using R to assess resemblance, repeatability, and reproduc- ibility for quantitative and semi-quantitative disinfectant methods. Center for Biofilm, Engineering, KSA-SM-13 (http://www.biofilm.montana.edu/content/ksa-sm-13). Parker, A., Tomasino, S., Hamilton, M.A., 2014. Statistical model for assessing performance standards for quantitative and semi-quantitative disinfectant test methods. JAOAC 97 (1), 58–67. Pinheiro, J., Bates, D., 2000. Mixed Effects Models in S and S-PLUS. Springer. Pinheiro, J., Bates, D., DebRoy, S., Sarkar, D., R Development Core Team, 2012. nlme: linear and nonlinear mixed effects models. R package version 3.1-104. Piyasena, P., Mohareb, E., McKellar, R.C., 2003. Inactivation of microbes using ultrasound: a review. Int. J. Food Microbiol. 87, 207–216. R Core Team, 2012. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria 3-900051-07-0 (URL http://www.R-project. org/). Richter, S.J., Richter, C., 2002. A method for determining equivalence in industrial applica- tions. Qual. Eng. 14 (3), 375–380. Springthorpe, S., Sattar, S., 2005. Carrier tests to assess microbicidal activities of chemical disinfectants for use on medical devices and environmental surfaces. JAOAC 88 (1), 182–201. Tomasino, S., Parker, A., Hamilton, M., Hamilton, G., 2012. Performance of the AOAC use-dilution method with targeted modifications: collaborative study. JAOAC 95 (6), 1618–1628. Tomasino, S., Parker, A., Hamilton, M., 2014. Use of statistical modeling to reassess the performance standard for the AOAC use-dilution methods (955.15 and 964.02). JAOAC 97 (1), 68–77.E2799-11 Standard TestMethod for TestingDisinfectant Efficacy against Pseudomonas