Ruggedness and reproducibility of the 
MBEC biofilm disinfectant efficacy test
Authors: Albert E. Parker, Diane K. Walkera, Darla M. 
Goeres, N. Allan, M.E. Olson, & A. Omar
NOTICE: this is the author’s version of a work that was accepted for publication in Journal of 
Microbiological Methods. Changes resulting from the publishing process, such as peer review, 
editing, corrections, structural formatting, and other quality control mechanisms may not be 
reflected in this document. Changes may have been made to this work since it was submitted for 
publication. A definitive version was subsequently published in Journal of Microbiological 
Methods, 102, July 2014, DOI#10.1016/j.mimet.2014.04.013. 
Parker AE, Walker DK, Goeres DM, Allan N, Olson ME, Omar A, "Ruggedness and reproducibility 
of the MBEC biofilm disinfectant efficacy test," Journal of Microbiological Methods, July 2014 
102:55–64
Made available through Montana State University’s ScholarWorks 
scholarworks.montana.edu 
Ruggedness and reproducibility of the MBEC biofilm 
disinfectant efficacy test
A.E. Parker a,b,⁎, D.K. Walker a, D.M. Goeres a, N
a Center for Biofilm Engineering, Montana State University, Bozeman, MT 59715, USA
b Department of Mathematical Sciences, Montana State University, Bozeman, MT 59715
c Innovotech Inc., Edmonton AB T6N 1H1, Canada
A B S T R A C T
ds:
lm
ess
ility
The ly
stand eth
resu om
rugg ha
tive)  to
bact te
dura 0
repe st
repr stu
concentrations of each of 3 disinfectants (a non-chlorine oxidizer, a phenolic, and a 
quat
method. The repeatability and rep
acceptable, as indicated by s
deviations (SD) (0.33 and 0.67 
SDs uctio
disin nged
rang addi
the sign
conc
I N T
M nt 
effic ed 
by t an 
and 7; Behnke et al., 
2011 hospital disinfec-
tant cy 
agai to 
subs
B
year
avaiclaims in the US (Tomasino et al., 2012)) are not appropriate to test disinfectant effica
nst biofilms. Although regulatory agencies in the US still have not adopted any method Costerton, 2002; Davies, 2003; Fux et al., 2004; Buckingham-Meyer et al., 200
). Thus, suspension tests and dried surface tests (currently used to support embers of academia, industry and regulatory agencies have come to recognize that disinfecta
acy against biofilms (i.e. microorgan-isms attached to surfaces) cannot be adequately studi
raditional microbiological methods that were developed for planktonic cells (e.g., see Donltantiate biofilm efficacy claim
iofilm methods for research 
s, however, have some of th
lableR O D U C T I O N of the biofilm log red
fectant combinations ra
ed from 0.27 to 1.70. In 
assay was statistically 
entrations.roducibility of the untreated control biofilms were 
mall repeatability and reproducibility standard 
log10(CFU/mm
2), respectively). The repeatability 
ns after application of the 24 concentration and 
 from 0.22 to 1.61, and the reproducibility SDs 
tion, for each of the 3 disinfectant types considered, 
ifi-cantly responsive to the increasing treatment ernary ammonium compound) were applied to biofilms using the MBEC MBEC™ Physiology & Genetics Assay recent
ardized biofilm disinfec-tant efficacy test m
lts of the standardization process using Pseud
edness testing of the MBEC method suggests t
 to small changes to the protocol with respect
eria (when varied from 16 to 18 h), treatment 
tion (25–35 min), and sonication power (13
atability of MBEC results across multiple te
oducibility across multiple labs, an 8-lab s, this is expected
purposes have bee
ese research meth. Allan c, M.E. Olson c, A. Omar c
, USA
Keywor
Biofi
Ruggedn
Reproducib
 became the first approved ASTM 
od. This report summarizes the 
onas aeruginosa biofilms. Initial 
t the  assay  is rugged (i.e., insensi-
 4 factors: incubation time of the 
mperature (20–24 °C), sonication 
–480 W). In order to assess the 
s in the same laboratory and the 
dy was conducted in which 8  in the next few years.
n available for decades. Only in the past few 
ods completed the arduous process to become 
concentration (MBEC) device, a low shear reactor system where the
biofilm is grown under batch conditions (i.e., there is no continuous
supply of media to the biofilm), complements this suite of available
biofilm growth methods.
The MBEC device (originally referred to as the “Calgary device”),
depicted in Fig. 1, is a widely used, 96-well plate system for growing
biofilm that allows for high throughput screening (Ceri et al., 1999;
Harrison et al., 2010). Biofilm is grown on pegs on a lid that fits over
the 96 wells. With such a large number of wells, the pegs with the
established biofilm may be immersed in multiple treatments and
controls simultaneously. The layout of the 96wells used in the tests de-
scribed in this paper is depicted in Fig. 2. Forty wells were used to study
the effect of the application of 8 concentrations of disinfectant on the
biofilm bacteria, while the remaining wells contained various media
or reagents as controls.
The MBEC assay was the first protocol approved by ASTM that
is both a biofilm growth and disinfectant protocol (ASTM E2799).
It is also the first ASTM biofilm method substantiated by a multi-
laboratory (collaborative) study (ASTM RR:E35-1006, 2011). The
purpose of the collaborative study was to quantitatively assess: the re-
peatability across multiple tests and the reproducibility across multiple
labs of the biofilm log reductions after application of the disinfectant
treatments; the resemblance of the control biofilms across labs; and
the responsiveness of the assay to detect increased log reductions in
the biofilm as the concentration of the chemical was increased.
Before the collaborative study began, theMBEC assay was subjected
to a ruggedness test in order to assess the sensitivity of the assay to
small changes of four important operational factors. Ruggedness test
results provide regression equations that quantify how changes in the
operational factors affect the responses of the MBEC assay. The magni-
tudes of these changes were chosen to reflect the range of factor values
that the method may be subjected to by multiple users at different labs.
Ruggedness testing is rarely used in microbiology, which is unfortunate
as there aremany benefits of this predictive test. Two standardized bio-
film growth methods have previously been subjected to ruggedness
testing: the CDC reactor (Goeres et al., 2005) and the DFR (Goeres
et al., 2009).
One purpose of this paper is to report the performance of the MBEC
assay in the ruggedness test and in the subsequent collaborative study.
We predict that this is the first of an infusion of new standardized
methods needed to assess anti-biofilm efficacy over the next decade.
Thus, the second purpose of this paper is to help inform this future
work by clearly elucidating the steps that we took to standardize the
first disinfectant test against biofilms.Fig. 1. A photograph of the MBEC plate.2. Methods
2.1. The MBEC method
2.1.1. Growth and treatment
To initiate the standardized MBEC method (ASTM E2799), the wells
of an MBEC device (Innovotech Inc., Edmonton, Ab. Canada), excluding
those for sterility control (Fig. 2, “SC”), were filled with 150 μL of tryptic
soy broth inoculated with 105 cells/mL of P. aeruginosa ATCC 15442,
and then incubated for 16–18 h at 35 ± 2 °C on an orbital shaker at
110 ± 10 rpm to establish a biofilm on the pegs located on the lid of
the device. Once the biofilm was grown, 5 pegs were detached from
the lid to determine the initial density of bacteria (Fig. 2, “PC”) by
immersing each peg into 0.2 mL buffered water, and then using sonica-
tion to remove the biofilm from the pegs and to disaggregate the biofilm
into the 0.2 mL volume. A 10 μL volume was removed from this 0.2 mL
volume for a 10-fold dilution series. A “challenge plate” was prepared
according to the layout in Fig. 2 with each well containing 200 μL/well
of treatment(s) or control media/reagents. The lid with the remaining
biofilm-covered pegs was placed on top of the challenge plate for the
specified contact time for the disinfectant treatment. The lid was then
transferred to a 96 well plate with each well containing 200 μL of neu-
tralizer, and then the biofilm on the remaining 91 pegs was sonicated
for 30 min to remove and disaggregate the biofilm. The contents of
each rowwere then transferred to a new 96well plate for serial dilution
followed by spot plating 10 μL on tryptic soy agar (TSA) in order to
determine viable cell concentrations.
2.1.2. Device layout
As depicted in Fig. 2, for each treatment, the first 5 wells of row A
contained the highest concentration of disinfectant; row B contained
half the concentration (50%) compared to row A; row C contained half
the concentration from row B (25%); and so on until the most dilute
concentration in row H contained half the concentration from row
G (0.78%). Column 6 contained a volume of the disinfectant at the
100% concentration mixed with an equal volume of neutralizer
(“50:N”). The wells of column 7 contained neutralizer only (“N”).
Column 8 contained buffered water and was the untreated control
(“UC”). Columns 9-11 are not used in the standard method and remain
empty as do thewells in rows D–H of column 12 since the pegs are used
for preliminary biofilm counts (“PC”). RowsA–C of column 12 contained
sterility (negative) controls (“SC”) of the growthmedia, buffered water,
and neutralizer.
2.2. Quantification of biofilm bacteria
At each dilution d in the serial dilution series, the number of
colony forming units (CFU) was enumerated, scaled up by the dilution
factor 10d and then reported as 10d × CFU/10 μL. These bacterial counts
were then converted to a log density (LD) permm2 of surface area using
the following equation,
LD ¼ log10 CFU=mm2

 
¼ log10 10dCFU=10 μL

 
V=SAð Þ
h i
where V is the volume sonicated into (0.2mL); and SA is the surface area
of the peg (46.63 mm2). Whenever no CFUs were observed in all dilu-
tions, a ½ CFU was substituted in at the lowest dilution plated, and
then scaled up (Hamilton et al., 2013). Since the 0th dilutionwas always
plated, then after applying the CFU substitution rule,
LD ¼ log10 0:5CFU=0:01mLð Þ 0:2 mL=46:63 mm2

 h i
¼ log10 0:2145 CFU=mm2
h i
¼−0:67
was the substituted value for the LD. Note that changing units to log10
2(CFU/cm ) would increase all LDs by +2, and leave the standard
2.4. Ruggedness testing
2.4.1. Ruggedness test experimental design
studeviation (SD) of the LDs unchanged. In otherwords, the SDof the LDs is
unitless.
Two separatemicrobial responseswere considered in all subsequent
statistical analyses. First, the mean LD of organisms attached to the 8
untreated control pegs (in column 8, labeled “UC”) of each MBEC plate
was calculated. Second, on each plate, the response of the biofilm
grown on pegs and then subjected to disinfectants at the 8 different
concentrations was quantified by the log reduction (LR), which is the
mean LD of biofilm on the eight untreated control pegs minus the
mean LD of biofilm that survived on the five pegs treated with a disin-
fectant in one of rows A–H in columns 1–5. Thus, the LR is unitless.
For each treatment concentration on a plate, denoted by rows A, B, C,
D, E, F, G, and H in the MBEC plate, one LR is calculated.
2.3. Neutralization
For all disinfectants and concentrations considered in this paper, D/E
neutralizing broth was used (Neogen Corporation, Lansing, MI). The
MBEC challenge plate contained two columns, each with 8 wells that
allow for the evaluation of the neutralizer in two ways (Fig. 2):
(1) was the disinfectant neutralized? and (2) does the neutralizer itself
impact the biofilm? To answer these questions, the eight wells in
column 6, labeled “50:N”, contain a neutralized disinfectant: 100 μL of
the 100% concentration of the disinfectant and 100 μL of the neutralizer.
The biofilms grown on pegs and then subjected to these wells were
compared to the controls to determinewhether the neutralizer satisfac-
torily neutralized the disinfectant. This 1:1 disinfectant to neutralizer
ratio in the “50:N” column is a conservative check of the neutralizer's
ability to inactivate the active compound. In practice, the biofilm-
Fig. 2. Layout of the MBEC plate used in the ruggedness testing and multi-labladen lid that was exposed to the disinfectant is transferred to a fresh
microtiter plate containing only neutralizer, in which case the disinfec-
tant to neutralizer ratio is much larger (about 1:200 to 3:200, cf.
Section 2.1.1). The eightwells in column7, labeled “N,” contain just neu-
tralizer. Comparing the biofilm subjected to thesewells to the untreated
control biofilms determined whether the neutralizer itself inhibits or
otherwise affects biofilm bacteria. A mixed effects statistical model
was used to test the efficacy of the neutralizer for each disinfectant sep-
arately, with nested random effects due to lab, experimental daywithin
a lab, and multiple wells within a plate per day. Treatment (with three
levels, neutralized disinfectant (in column “50:N”), Neutralizer (in col-
umn “N”), and the controls (in column “UC”)) was the fixed effect. As
is commonly done for planktonic assays using suspended bacteria, all
pairwise comparisons of the mean LDs amongst these levels were per-
formed (ASTM E1054). These comparisons were made using Tukey's
multiple comparison procedure. However, since we believe that it is
more informative to assess statistical equivalence as opposed to failing
to find statistical differences when assessing neutralizers, equivalence
tests were performed by calculating 3 simultaneous 2-sided 90%In order to estimate both main effects and two-way interactions
amongst the four factors, experiments were run at low, medium and
high levels of the factors of interest (Table 1). These low and high levels
were chosen in order to span a realistic range of deviations to the proto-
col when conducted in different labs. Experiments at specific combina-
tions of the factors were chosen according to a fractional factorialA ruggedness test determines the sensitivity of a protocol to small
changes in operational factors. For the ruggedness test of the MBEC
assay, the factors tested were treatment temperature (Temp), sonica-
tion duration (SonDur), sonication power (Power), and incubation
time (Inc). The non-chlorine oxidizer disinfectant Virkon (DuPont,
Wilmington, DE) was used in all ruggedness experiments. For each
MBEC test, Virkon was applied in 8 different concentrations (Fig. 2),
starting at 535 mg/L (100%) in the five wells in row A of the plate, and
then with half concentrations applied in each row thereafter (thus,
4.2 mg/L was tested in row H).Tukey confidence intervals for the truemean differences. If the 90% con-
fidence intervals were contained in the interval [−0.5, 0.5] (Nelson
et al., 2013), then statistical equivalence was concluded at 95% confi-
dence (Richter and Richter, 2002; ASTM E2935). In other words, we as-
sumed that mean LD differences less than½ logwere negligible and not
of practical importance.
dy. Section 2.1.2 describes the acronyms (50:N, N, UC, SC) used in the figure.experimental design and the run order was randomized as outlined in
Table 2. Two of the ten pairs of experiments (runs 1 and 8) were run
at standard operating procedure (SOP) values for all of the factors.
These repeats (called “center points” in the statistical literature)
allowed preliminary estimation of the variability of the control LDs
and LRs when the method was run according to the SOP. Two techni-
cians each performed the ten experiments in Table 2 side-by-side in
order to minimize technician-to-technician variability in the experi-
mental outcomes, thereby increasing the statistical power to detect
Table 1
The four factors to be analyzed in the ruggedness testing, the short name used in
subsequent tables, and the low, medium and high levels of each factor.
Factor Short name Low Medium (SOP) High
Incubation time Inc 16 h 17 h 18 h
Treatment temperature Temp 20 °C 22 °C 24 °C
Sonication power Power 130 W 250 W 480 W
Sonication duration SonDur 25 min 30 min 35 min
Caution in interpretation should be taken, however, since each coeffi-
cient scales by the units of the factor, i.e., incubation time (h), sonication
2.5. Collaborative study
2.5.1. Study design
An 8-lab study was conducted in 2011 as the last step of the process
towards standardization of the MBEC assay with ASTM. Specific details
of this study have been documented as a research report (ASTM RR:
E35-1006, 2011). The main goal of this collaborative study was to
determine the resemblance of the control LDs and of the repeatability
and reproducibility of the LRs generated by the MBEC protocol. The
responsiveness of the method to increasing concentrations of each
treatment was also assessed (Hamilton et al., 2013).
All 8 laboratories participated in an on-line training session prior
to the study. During the collaborative study, 3 days of testing were
performed at each lab. On each day, 3 disinfectants were tested, each
on a separate plate. This experimental design satisfies ASTM's minimal
requirements of 6 labs, 3 products, and 3 repeated tests of each product
at each lab (ASTM E691). The 3 disinfectants used in this study were
chosen to represent different classes of products: Virkon (DuPont,
Wilmington, DE), a non-chlorine oxidizer (also used in the ruggedness
test); Multi-Phenolic Disinfectant (Bio Agri Mix LP, Mitchell, Ontario,
CA) (a phenol); and Biosentry 904 (Neogen Corporation, Lansing, MI),
a quaternary ammonium compound (quat). The ranges of concentra-
tions used in rows A–Hof theMBEC plates for each of the 3 disinfectants
were 535–4.2 mg/L for the oxidizer; 1640–13 mg/L for the phenol; and
Table 2
Fractional factorial schedule of ruggedness tests for one disinfectant and one technician.
Run Inc Temp Power SonDur
1 17 h 22 °C 250 W 30
2 18 h 20 °C 130 W 25
3 16 h 24 °C 480 W 35
4 18 h 24 °C 480 W 25
5 18 h 24 °C 130 W 35
6 18 h 20 °C 480 W 35
7 16 h 20 °C 480 W 25
8 17 h 22 °C 250 W 30
9 16 h 20 °C 130 W 35
10 16 h 24 °C 130 W 25duration (min), sonication power (W) and treatment temperature (°C).
The effect of each factor was further assessed by significance tests, mainthe effects of the four factors. Thus, the ruggedness test results are based
on 20 MBEC runs, all performed in the same lab.
2.4.2. Assessing factor effects
To assess the effect of each of the four factors of interest, a nested
mixed effects statistical model was fit separately to the control LDs
and the LRs. The three nested random effects in themodel were techni-
cian, experiments (or plates) nestedwithin technician, andwells within
an experiment. The fixed effects in the models were the four factors
given in Tables 1, and 3 of the 6 two-way interactions. A covariate for
“Biofilm Growth” was also included, defined as the mean LD of the
“PC” pegs in the MBEC plate column 12 (Fig. 2). Interactions were
assessed via interaction plots and significance tests.
The statistical model for the LRs was the same as just described ex-
cept that, instead of a covariate for Biofilm Growth, it included a covar-
iate for disinfectant concentration. From the LRs calculated across the 8
different disinfectant concentrations in rows A–H on the MBEC chal-
lenge plate, a dose–response curve was plotted. The covariate for disin-
fectant concentration (DisConc) simply characterizes this curve with a
line. For ease of interpretability, the covariate for disinfectant concentra-
tion in MBEC plate rows A, B, C,…, H was encoded as the whole num-
bers 7, 6, 5, 4,…, 0 respectively. These values are equivalent to taking
the transform DisConc = log2(nominal concentration in mg/L) −
log2(4.2 mg/L) where 4.2 mg/L is the lowest concentration of Virkon
used (in row H of the MBEC challenge plate).
The output of the statistical analyses is a regression equation, with a
coefficient estimate and significance test for each factor and two-way
interaction. The regression equations always contain all themain effects
regardless of statistical significance, but only contain statistically signif-
icant two-way interactions. A small coefficient in the regression equa-
tion suggests that a response (i.e., either the mean control LD or the
LR) is ruggedwith respect to small changes in the corresponding factor.effect plots, and interaction plots.
Table 3
Neutralization results from the 8-lab collaborative study. Satisfactory neutralization is
concluded if all 90% confidence intervals (CI) are contained in the equivalence interval
of [−0.5, 0.5].
Disinfectant Comparison Mean LR SE(LR) 90% CI
Non-chlorine oxidizer Neutralized disinfectant −0.09 0.0405 [−0.17,−0.01]
Neutralizer toxicity −0.04 0.0405 [−0.12, 0.04]
Phenol Neutralized disinfectant 0.67 0.0526 [0.56, 0.77]
Neutralizer toxicity −0.09 0.0526 [−0.19, 0.02]
Quat Neutralized disinfectant 0.21 0.0424 [0.13, 0.30]
Neutralizer toxicity −0.06 0.0424 [−0.14, 0.03]3680–29mg/L for the quat. Since the goal of the studywas to assess the
MBEC method and not these products, these products are referred to
throughout the manuscript by their generic names.
There were a few missing data, apparently due to the spot plating
techniques. One lab reported a missing tip on the low-volume multi-
channel-pipetter, and another lab reported that when spot plating
“sometimes it happened that certain drops came into contact with the
direct drop nearby.” At Lab 5, the phenol treatment on Day 2 had
measurements from only 4 out of 5 pegs; the quat on Day 2 had only
4 measurements; and the quat on Day 3 had only 4 measurements. At
Lab 6, there was no row H data for the quat on Day 1 for either the
disinfected or control biofilms.
All questions regarding the data were brought to the attention of the
study director. All data were validated, and since no deviations from the
study protocol were identified, all data collected from all 8 labs were in-
cluded in the statistical analyses (consistent practicewith the guidelines
in section 19 of ASTM E691).
SonDur
Power
Inc
Temp
353025
480130250480130
181618161718161816
202424202224202024
7.0
6.5
6.0
5.5
5.0
4.5
4.0
3.5
LD
Fig. 3. Pooled across 2 technicians and 20 plates used in the ruggedness tests, the log
densities (LD = log10(CFU/mm2)) of the untreated control biofilms grown using the
MBEC are plotted. Each point corresponds to the LD of biofilm bacteria on a single control
peg. The data in the rectangle are from plates 1 and 8 per technician that were run at the
SOP values of the four operational factors.
2.5.2. Assessing resemblance, repeatability and reproducibility of the
MBEC method
To assess the resemblance of the LDs of untreated control biofilms
across labs, a statistical model was fit with nested random effects due
to lab, experimental day within a lab, plates within an experimental
day, and wells within a plate. It was possible to estimate the plate vari-
ability within each day because each lab ran three plates on each test
LR for each disinfectant and concentration were also assessed using a
2.6. Other statistical details
The analysis of the collaborative study data followed the guidelines
for disinfectant tests described in Hamilton et al. (2013). Mixed effects
models (Pinheiro and Bates, 2000) specifically for disinfectant efficacy
data are described in detail by Parker and Hamilton (2011). Themethod
of restricted maximum likelihood method was used to fit the nested
mixedmodels using the software R v.2.11.0 (R Core Team, 2012), pack-
age nlme (Pinheiro et al., 2012); the R code is provided in Parker and
Hamilton (2013). Three types of plots were used to identify outliers
and to check the fit of the statistical models: individual value, residual
and normal probability plots. An additional check for outliers was to
perform the hypothesis tests recommended by ASTM (cf. sections 16
and 17 in ASTM E691). All graphs were rendered by Minitab v.16.
Minitab was also used to generate the fractional factorial experimental
design for the ruggedness testing. All statistical tests, including the
equivalence tests, were conducted at a significance level of 5%.
3. Results
3.1. Neutralization
Table 4
Summary statistics for the regression equation fit to the log densities (LD = log10(CFU/
mm2)) of untreated control biofilms from the ruggedness tests. Units for the factors are
provided in Table 1. Since therewere no significant 2-way interactions, themodel contains
only main effects.
Factors Estimate SE p-value
Intercept 5.027 0.2445 b0.00005
Inc—17 0.1111 0.1146 0.3500
SonDur—30 −0.0042 0.0224 0.8534
Temp—22 −0.1178 0.0572 0.0601
Power—250 0.00036 0.0006 0.5616
BiofilmGrowth—5.87 0.3893 0.2217 0.1026mixed model with a random effect for lab.
2.5.3. Assessing responsiveness of the MBEC method
To assess responsiveness of themethod to the 8 increasing disinfec-
tant concentrations in rows A–H in the challenge plate, a dose–response
curve with respect to the LRwas constructed. One approach to quantify
responsiveness is with the slope of a regression line. Such a line was fit
to the LRs for each disinfectant using a mixed effects model with
random effects due to lab and tests within a lab, and a covariate for
disinfectant concentration (see Section 2.4.2).
6
5day, one for each of the three disinfectants considered. The controls
from all three of these plates were considered simultaneously in the
same statistical analysis since they were processed similarly, including
the use of the same D/E neutralizer. The model just described gives a
pooled estimate of the repeatability SD of the method at a single lab
by pooling the repeatability variances across all labs. The repeatability
SD was also calculated at each lab separately using a model with ran-
dom effects for experimental day, plates within an experimental day,
and wells within a plate. The repeatability and reproducibility for theplate
row
4321 & 8
EFGHABCDEFGHABCDEFGHABCDEFGHABCDEFGH
4
3
2
1
0
LR
Fig. 4. From ruggedness tests, the log reductions (LR) as a function of disinfectant concentra
concentration for one of two technicians. The plate numbers refer to the sets of experimental cThe results of the neutralization tests conducted during the
collaborative study are presented in Table 3. For all 3 disinfectants
tested, the mean LD of biofilm bacteria subjected to the control or
neutralizer-only wells were statistically equivalent. For the non-
chlorine oxidizer and the quat, the mean LD of biofilm bacteria subject-
ed to the control or neutralized disinfectant wells were also statistically
equivalent. Unfortunately, for the phenol, the control and neutralized
disinfectant wells were not statistically equivalent. In fact, there was a
statistically significant difference between the mean LD of the biofilm
subjected to the “neutralized phenol” wells compared to the controls
(p-value b 0.00005). Thus, the phenol in row A of the MBEC device
did not appear to be adequately neutralized in the 8-lab study.
3.2. Ruggedness testing
3.2.1. Controls
Fig. 3 depicts the LDs of the control biofilms for both technicians
across the 9 experimental conditions outlined in Table 2. The tech-to-
tech variability for the 4 SOP runs was estimated to be zero and was
not statistically significant (p-value = 0.751). As the four factors were
109765
ABCDEFGHABCDEFGHABCDEFGHABCDEFGHABCD
tion (labeled by MBEC rows A, B, C, D, E, F, G, and H). Each point is a LR at a disinfectant
onditions in Table 2.
varied from the SOP values (according to Table 2), there were no signif-
icant two-way interaction effects on the control LDs (p-value≥ 0.2350).
The model fit to the mean control LD with only main effects is
LD ¼ 5:03þ 0:1111 Inc–17ð Þ−0:0042 SonDur−30ð Þ−0:1178 Temp–22ð Þ
þ 0:0004 Power–250ð Þ þ 0:3893 BiofilmGrowth–5:87ð Þ:
(p-value≥0.1795).However, two-way interactions betweenDisinfectant
Concentration and each of treatment temperature and incubation were
significant (p-value ≤0.0467). The model that included the significant
two-way interactions and all of the main effects for the four factors is
LR ¼ 0:2157–0:457 Inc–17ð Þ þ 0:0015 SonDur−30ð Þ–0:0696 Temp–22ð Þ
þ 0:0003 Power–250ð Þ þ 0:7866DisConc
−0:0305 Temp–22ð Þ  DisConcþ 0:0832 Inc–17ð ÞDisConc
This equation has been centered at the middle values of the factors.
Thus, the constant 0.2157 is the predicted LR when DisConc = 0
(corresponding a disinfectant concentration of 4.2 mg/L in row H).
When DisConc = 7 (corresponding to 535 mg/L of disinfectant as in
row A), the equation predicts a LR = 5.72.
Table 5 gives a summary of the significance tests performed for each
factor in the model. Just as for the control data, sonication duration
(p-value = 0.9240) and sonication power (p-value = 0.5389) do not
significantly affect the LR. The coefficient for DisConc is 0.7866 and is
statistically significantly positive (p-value b 0.00005). Thus, when all
of the factors are at their SOP values, as the disinfectant is increased
Table 5
Summary statistics for the regression equation fit to the log reductions (LR) from the
ruggedness tests. Units for the factors are provided in Table 1. The model depicted here
contains all main effects and only statistically significant 2-way interactions.
Factors Estimate SE p-value
Intercept 0.2157 0.9206 0.8151
Inc—17 −0.4570 0.1308 0.0036
SonDur—30 0.0015 0.0152 0.9240
Temp—22 0.0696 0.0654 0.3052
Power—250 0.0003 0.0004 0.5389
DisConc 0.7866 0.0272 b0.00005
(Inc—17) × DisConc 0.0832 0.0304 0.0071
(Temp—22) × DisConc −0.0305 0.0152 0.0467The equation has been centered at the middle values of each of the
factors. Hence, 5.03 log10(CFU/mm2) is the predicted mean LD of the
control biofilms when all of the operational factors are set to their SOP
values, and when the mean LD of the five biofilm growth check pegs
(“PC”) is 5.87 log10(CFU/mm2).
Table 4 summarizes the statistical analysis of the control biofilm LDs.
None of the factors had a significant effect on the mean control LD
(p-value≥ 0.0601). The small coefficients for the factors in the equation
for the mean control LD explicitly show the ruggedness of the MBEC
assay to changes in the four factors. For example, the coefficient of
−0.0042 for SonDur estimates that for every increase of sonication
duration by 1min (in the range 25–35 min), themean LD of the control
wells will decrease on average by 0.004 log10(CFU/mm2). Keep in mind
that calculations based on this equation are valid only for values of the
factors in the ranges outlined in Table 1.
3.2.2. Treated data
The LRs are given as a function of the 8 concentrations of disinfectant
(in rows A−H of theMBEC challenge plate) in Fig. 4. The LRs calculated
from the4 SOP runs (plates 1 and8 per technician) are plotted in the left
panel of Fig. 4. None of the two-way interactions amongst the 4 factors
had a significant effect on the LRs (p-value≥0.3522). None of the 3-way
interactions with Disinfectant Concentration were significant either
8
7Lab
Day
plate
4321
321321321321
123123123123123123123123123123123123
6
5
4
LD
Fig. 5. Control data from the 8-lab study. Each point in the graph is the log density (LD= log10(
control well in an MBEC plate. Along the horizontal axis are listed the eight labs, the three expup a single row of the MBEC device (e.g., going from row E to row D,
corresponding to DisConc increasing by 1), then it is predicted that
the LR will increase by 0.7866. This demonstrates that the MBEC assay
was statistically significantly responsive to increasing disinfectant
concentrations, which is a required attribute of an efficacy test. Again,
we emphasize that the regression equation is statistically valid only
within the ranges outlined in Table 1. It is not prudent to extrapolate
beyond these values.
3.3. Collaborative study
The favorable ruggedness testing results indicated that the MBEC
assay was ready for the next step in standardization, a collaborative
study.
The untreated control LDs from the collaborative study are plotted in
Fig. 5. Table 6 summarizes the means and variance components attrib-
utable to different sources of variability. The repeatability SD is given
for each lab individually in order to identify labs with unusual variabil-
ity. All of the repeatability and reproducibility SDs given in Table 6 are
acceptable compared to results from historical data published by the
Center for Biofilm Engineering (Table 1 in Parker and Hamilton, 2011).
This indicates acceptable resemblance of the control data across multi-
ple experiments in a single lab, and also across multiple laboratories.
8765
321321321321
123123123123123123123123123123123123
CFU/mm2)) of biofilm bacteria grown on a singleMBEC peg then subjected to an untreated
erimental days within each lab, and the three plates within each experimental day.
For each disinfectant type considered in the collaborative study,
Figs. 6–8 present the LRs as a function of increasing disinfectant efficacy.
Figs. 9–10 provide a graphical summary of the repeatability and
reproducibility SDs respectively for each of the 24 disinfectant and
concentration combinations.
The increasing LR curves in Figs. 6–8 suggest that, for each disinfec-
shown that sonication can affect antimicrobial efficacy measures such
as the LR. Monsen et al. (2009) showed that increasing sonication dura-
tion reduced gram-negative bacterial biofilms, including P. aeruginosa.
The effect of sonication is even more pronounced when used in conjunc-
tion with antimicrobials or at increased temperatures (Piyasena et al.,
2003; Fux et al., 2004; Monsen et al., 2009). Piyasena et al. suggest that
Table 6
Repeatability and reproducibility of the log densities (LD = log10(CFU/mm2)) of untreated control biofilms from the2011 collaborative study. Themean LDhas units log10(CFU/mm2); the
SDs are unitless.
Lab No. exp Mean LD Sources of variability Repeatability SD Reproducibility SD
Within plate % Among plate % Among exp day % Among lab %
1 3 5.50 40% 34% 25% 0.1369
2 3 5.58 20% 27% 53% 0.4206
3 3 4.27 39% 12% 49% 0.1696
4 3 5.92 17% 0% 83% 0.2315
5 3 5.80 64% 0% 36% 0.1624
6 3 5.72 8% 7% 85% 0.5301
7 3 6.13 76% 24% 0% 0.1438
8 3 6.16 51% 0% 49% 0.2706
All 8 24 5.48 4% 11% 9% 76% 0.3252 0.6669
xidtant, the MBEC assay is responsive to increasing disinfectant efficacy.
The model fit to the LRs for each disinfectant provides a quantitative
linear measure of the responsiveness of the method to increasing
disinfectant efficacy. Table 7 shows that the MBEC assay is statistically
significantly responsive to the increasing disinfectant efficacy for all
three disinfectants.
4. Discussion
A laboratory method is considered ready for standardization if it has
a successful publication record by different researchers in different labs,
provides quantitative results, and uses typical equipment requiring
typical laboratory expertise (Hamilton et al., 2013). The MBEC assay
met these criteria with more than a decade of referenced use (Ceri
et al., 1999). The next step in the standardization process for the MBEC
assay was to conduct a ruggedness test with respect to important opera-
tional factors in the protocol.
Since biofilm bacteria are attached to a surface, the bacteria must be
removed and disaggregated in order to provide unbiased estimates of the
density of biofilm bacteria (Hamilton et al., 2009). Given the use of soni-
cation for this purpose in the MBEC assay (and in many microbiological
tests), it was imperative to assess the ruggedness of the MBEC assay
with respect to factors that control sonication. Previous research has
8
6
OLab
row
4321
ABCDEFGHABCDEFGHABCDEFGHABCDEFGH
4
2
0
LR
Fig. 6. Log reductions (LR) for the oxidizer treatment in the 8-lab study. The horizontal axis lists
single concentration of disinfectant in a single experiment.sonication kills by “thinning cell membranes,” which could explain the
results by Monsen et al. that gram-positive bacteria with thicker cell
walls are less susceptible to sonication. Fux et al. suggest that synergistic
kill occurs because sonication effectively disaggregates biofilms, thereby
increasing the efficacy of an antimicrobial. Furthermore, and of great im-
port for this study, Monsen et al. reported significant among-experiment
and among-apparatus variability in their sonication experiments. The
two sonication factors considered in the ruggedness tests for the MBEC
were sonication duration and power. In addition, incubation time for
growing the biofilms, and temperature during the disinfectant contact
timewith the biofilm,were also assessed. Based on the previous research,
two-way interactions between the sonication factors and each of treat-
ment temperature and disinfectant concentration were of particular
interest.
Overall, the ruggedness testing showed that the results from the
MBEC assay are not influenced by small changes of the operational fac-
tors. Interestingly, the mean control LDs were not significantly affected
by any of the four factors (Table 4). When a disinfectant was applied,
only incubation time and treatment temperature had statistically signif-
icant effects on the LRs, and these effects depended on the disinfectant
concentration. Importantly, none of the two-way interactions among
the four factors had a statistically significant effect on the LR (Table 5).
Explicit equations (in Tables 4–5) quantify the simultaneous effect of
izer8765
ABCDEFGHABCDEFGHABCDEFGHABCDEFGH
all 8 labs, and the 8 concentrations tested at each lab. Each point in the figure is the LR for a
Lab
row
87654321
ABCDEFGHABCDEFGHABCDEFGHABCDEFGHABCDEFGHABCDEFGHABCDEFGHABCDEFGH
8
7
6
5
4
3
2
1
0
-1
LR
Phenol
Fig. 7. Log reductions (LR) for the phenol treatment in the 8-lab study. The horizontal axis lists all 8 labs, and the 8 concentrations tested at each lab. Each point in the figure is the LR for a
single concentration of disinfectant in a single experiment.all 4 of these factors on themean LDof untreated control biofilms grown
in the MBEC; and also on the LR of the biofilm after application of mul-
tiple concentrations of a non-chlorine oxidizer. The small coefficients in
these equations demonstrate precisely how rugged the MBEC results
are to changes in these factors. For example, the coefficient of 0.0015
for SonDur (Table 5) predicts that as sonication duration increases
from 25 to 35 min, the LR will increase on average by 0.015, provided
that disinfectant concentration and the other factors stay constant.
Similarly, as sonication power is increased by 100 W (in the range
130–480 W), it is predicted that the LR will increase on average by
0.03. Due to statistically significant interactions with disinfectant
concentration, the effects of treatment temperature and incubation
time do not have this same simple interpretation. Instead, the effects
of these 2 factors depend on the disinfectant concentration.
Upon completion of the ruggedness tests, the estimates of MBEC
variability were crucial since theywere our first quantitative assessment
of the repeatability of the MBEC across multiple experiments. These
estimates are consistent with what was found with more confidence
later in the 8-lab study, and so are not reported in this paper. One assess-
ment of interest was that the technician-to-technician variability for the
four ruggedness test SOP runs was estimated to be zero (i.e., between
Lab
row
4321
ABCDEFGHABCDEFGHABCDEFGHABCDEFGH
7
6
5
4
3
2
1
0
-1
LR
Qu
Fig. 8. Log reductions (LR) for the quat treatment in the 8-lab study. The horizontal axis lists al
single concentration of disinfectant in a single experiment.5.78 × 10−12 and 1.26 × 10−11) and not statistically significant for either
the control LDs (p-value≥0.751), or the LRs attained by the high concen-
trations of the oxidizer (in MBEC rows A–D, p-value ≥0.553). For the
less-efficacious treatments (in rows E–H), the technician-to-technician
variancewas estimated to be larger than 0, but not statistically significant
(p-value ≥0.156). These results were expected since the experimental
designpurposely sought tominimize technician-to-techniciandifferences
by requiring that the technicians independently conduct experiments
side-by-side on the same day so that the effect of the factors could be
studied with more statistical power. In general, a method with large
technician-to-technician variabilitywouldnot be suitable for standardiza-
tion and would likely have a large reproducibility standard deviation.
The advantage of beginning the standardization process with
ruggedness testing is that it can predict how well (or poorly) a method
will perform across multiple labs while only requiring experiments
from a single lab. For example, the initial MBEC protocol specified that
incubation time could be set in the range 16–24 h (Table 1). Based on
the ruggedness test results, the incubation time range in the standard-
ized MBEC method was tightened to 16–18 h. Other than this change,
the ruggedness test results indicated that the MBEC assay was well
suited for the next step required for standardization, the collaborative
8765
ABCDEFGHABCDEFGHABCDEFGHABCDEFGH
at
l 8 labs, and the 8 concentrations tested at each lab. Each point in the figure is the LR for a
study. The ruggedness test results also made it clear that in the multi-
laboratory study, in addition to incubation time, treatment temperature
would need to be tightly monitored and controlled.
In preparation of the collaborative study, much effort was expended
to identify a neutralizer that satisfactorily neutralized all 3 disinfectants
at all 8 concentrations. As described earlier, while the oxidizer and quat
were effectively neutralized (on the average) by D/E broth during the
collaborative study, the high concentration of the phenol (i.e., at least
the 100% concentration) was not. This may explain why the 100%
concentration of phenol attained the highest mean LR (=5.64) in the
8-lab study. More generally, these results underscore that, compared
to assays of planktonic bacteria, it is more challenging to neutralize
the highly concentrated chemicals formulated to be effective against
biofilms. One benefit of the MBEC is that disinfectant neutralization is
monitored in every plate. This is an advantage over other disinfectant
test methods, where neutralization tests are usually performed ahead
of time in a separate experiment, and then neutralization is assumed
to be adequate for all later experimental runs.
The collaborative study design for the MBEC assay satisfied ASTM's
minimum requirements of 6 labs, 3 products, and 3 repeats of each
product at each lab. As advocated by ASTM (E691), based on the
multi-lab study data, we assessed the repeatability of the biofilm LRs
time correctly failing ineffective ones, it is imperative to know the
repeatability and reproducibility of the method.
5. Conclusion
In vitro studies remain a cornerstone for research, development, and
registration of a variety of disinfectant andmedical technologies. Under-
standing the performance of the underlying method in the controlled
environment of the laboratory is a crucial step in order to provide con-
vincing data that is repeatable across experiments, reproducible across
labs, responsive to varying treatment efficacies, and rugged to changes
in factors that may have been changed from the protocol. Extensive
testing is required to assess these attributes for any method. Based on
1.8
1.6
1.4
1.2
1.0
du
ci
bi
lit
y 
SD
Oxidizer
Phenol.
Quat.
Dis.
Table 7
From the 2011 collaborative study, responsiveness of the MBEC to increasing concentra-
tions of disinfectant. The trend is interpreted as the mean increase in the log reduction
(LR) as the concentration of disinfectant increases two-fold (as occurs when moving up
one row in the MBEC plate).
Disinfectant Trend SE p-value
Non-chlorine oxidizer 0.87 0.0379 b0.00005
Phenol 0.87 0.0376 b0.00005
Quat 0.50 0.0275 b0.00005across experiments, and the reproducibility across labs. In addition,
since methods designed to test antimicrobials are based on an underly-
ing biological system, we also assessed 3 other criteria that are not
advocated by ASTM (E691): disinfectant neutralization, resemblance
of the controls across labs, and responsiveness of the assay to increasing
disinfectant efficacy (Hamilton et al., 2013).
TheMBEC assay's performance in the collaborative study is depicted
in Figs. 9–10. The frown-shaped quadratic relationship of the SDs as a
function of disinfectant efficacy is to be expected for any disinfectant
test method (Springthorpe and Sattar, 2005). This is because, for disin-
fectantswith large LRs,most of the viable biofilm bacteria on the treated
pegs are killed, which results in small variability of the treated LDs, so
the observed variability for the LRs is close to the variability for the
untreated control LDs. For low efficacy disinfectants with small LRs,
the viable bacteria on the treated pegs resemble the viable bacteria on
the untreated pegs, so, again, the observed variability for the LRs is
close to the variability for the untreated control LDs. For moderately
effective treatments, only some of the biofilm bacteria are killed,
which results in a more variable LR. Since the three disinfectant types
used in the collaborative study are considered representatives of classes
of disinfectants, the actual concentrations of the disinfectants tested in
6543210
1.6
1.4
1.2
1.0
0.8
0.6
0.4
0.2
0.0
Mean LR
R
ep
ea
ta
bi
lit
y 
SD
Oxidizer
Phenol.
Quat.
Dis.
Fig. 9. For the collaborative study, the repeatability SDs calculated as a function of the
mean log reduction (LR) for all 24 combinations of the 3 disinfectants and the 8 concentra-
tions tested.the collaborative study are not as important as the mean LR achieved
for each concentration of each disinfectant type.
Thus, the interested reader can, for example, predict the variability
of a quat in a future MBEC study in a single lab by looking up the quat's
mean LR in Fig. 9 and then reading off the attained repeatability SD for
the quat in this study. Or one can predict the quat's variability in a future
MBEC study across multiple labs by looking up the mean LR in Fig. 10
and then reading off the attained reproducibility SD for the quat in
this study. Either of these scenarios could occur, for example, when
performing tests for registration of a product with a regulatory agency.
From a regulator's point of view, the repeatability and reproducibil-
ity SDs reported in this paper can be used to craft performance stan-
dards for the MBEC method in order to determine the number of tests
(i.e., independent plates), and the number of replicate wells in each
plate, in order to achieve a target LR with a desired level of confidence
(Parker et al., 2014). Based on another collaborative study (Tomasino
et al., 2012), this approachwas recently used to update the performance
standard for the use-dilution method (UDM), the current method used
by EPA to register products for hospital disinfectant claims (Tomasino
et al., 2014). In order to have confidence that a performance standard
for a method correctly passes effective products, while at the same
6543210
0.8
0.6
0.4
0.2
Mean LR
R
ep
ro
Fig. 10. For the collaborative study, the reproducibility SDs calculated as a function of the
mean log reduction (LR) for all 24 combinations of the 3 disinfectants and the 8 concentra-
tions tested.
ruggedness tests and the 8-lab study performed in 2011, these attri-
butes for the MBEC assay were assessed and found to be acceptable by
ASTM's subcommittee on antimicrobial agents (E35.11). Thus, the
MBEC method is a low shear, high throughput, batch system that is
rugged, responsive, repeatable and reproducible; and complements the
suite of reactor systems available for biofilm growth and disinfection.
Acknowledgments
The authors are grateful to the 8 labs that volunteered their time and
resources to participate in the inter-laboratory study. We also would
like to thank ASTM for providing the technical expertise in the experi-
mental design and for supporting the statistical analysis of the multi-
lab study. Innovotech conducted the ruggedness tests, supported the
statistical analysis of the ruggedness tests, and provided the treatment
chemicals to the labs in the collaborative study. Innovotech also partial-
ly supported the writing of this manuscript.
References
ASTM research report RR:E35-1006, 2011. Interlaboratory Study for ASTM Method
Davies, D., 2003. Understanding biofilm resistance to antibacterial agents. Nat. Rev. Drug
Discov. 2, 114–122.
Donlan, R.M., Costerton, J.W., 2002. Biofilms: survival mechanisms of clinically relevant
microorganisms. Clin. Microbiol. Rev. 15, 167–193.
Fux, C.A., Wilson, S., Stoodley, P., 2004. Detachment characteristics and oxacillin
resistance of Staphylococcus aureus biofilm emboli in an in vitro catheter infection
model. J. Bacteriol. 186, 4486–4491.
Goeres, D., Loetterle, L., Hamilton, M., Murga, R., Kirby, D., Donlan, R., 2005. Statistical
assessment of a laboratory method for growing biofilms. Microbiology 151, 757–762.
Goeres, D.M., Hamilton, M.A., Beck, N.A., Buckingham-Meyer, K., Hilyard, J.D.,
Loetterle, L.R., Lorenz, L.A., Walker, D.K., Stewart, P.S., 2009. A method for grow-
ing a biofilm under low shear at the air–liquid interface using the drip flow bio-
film reactor. Nat. Protoc. 4 (5), 783–788.
Hamilton, M.A., Buckingham-Meyer, K., Goeres, D.M., 2009. Checking the validity of the
harvesting and disaggregating steps in laboratory tests of surface disinfectants.
JAOAC 92, 1755–1762.
Hamilton, M., Hamilton, G., Goeres, D., Parker, A., 2013. Guidelines for the statistical
analysis of a collaborative study of a laboratory method for testing disinfectant
product performance. JAOAC 96 (5).
Harrison, J., Stremick, C., Turner, R., Allan, N., Olson, M., Ceri, H., 2010. Microtiter suscep-
tibility testing of microbes growing on peg lids: a miniaturized biofilm model for
high-throughput screening. Nat. Protoc. 5, 1236–1254.
Monsen, T., Lövgren, E., Widerström, M., Wallinder, L., 2009. In vitro effect of ultrasound
on bacteria and suggested protocol for sonication and diagnosis of prosthetic infec-
tions. J. Clin. Microbiol. 47 (8), 2496–2501.
Nelson, M., LaBudde, R., Tomasin, S., Pines, R., 2013. Comparison of 3M PetrifilmTM aero-
bic count plates to standard plating methodology for use with AOAC antimicrobial
efficacy methods 955.14, 955.15, 964.02, and 966.04 as an alternative enumerationaeruginosa Biofilm Using the MBEC Assay. ASTM International, West Conshohocken,
PA.
ASTM standard E1054, 2013. Standard Test Methods for Evaluation of Inactivators of
Antimicrobial Agents. ASTM International, West Conshohocken, PA.
ASTM standard E2196, 2012. Standard Test Method for Quantification of Pseudomonas
aeruginosa Biofilm Grown with Medium Shear and Continuous Flow Using Rotating
Disk Reactor. ASTM International, West Conshohocken, PA.
ASTM standard E2562, 2012. Standard Test Method for Quantification of Pseudomonas
aeruginosa Biofilm Grown with High Shear and Continuous Flow using CDC Biofilm
Reactor. ASTM International, West Conshohocken, PA.
ASTM standard E2647, 2013. Standard Test Method for Quantification of Pseudomonas
aeruginosa Biofilm Grown Using Drip Flow Biofilm Reactor with Low Shear and
Continuous Flow. ASTM International, West Conshohocken, PA.
ASTM standard E2799, 2012. Standard Test Method for Testing Disinfectant Efficacy
against Pseudomonas aeruginosa Biofilm using the MBEC Assay. ASTM International,
West Conshohocken, PA.
ASTM standard E2935, 2013. Standard Practice for Conducting Equivalence Testing in
Laboratory Applications. ASTM International, West Conshohocken, PA.
ASTM standard E691, 2013. Standard Practice for Conducting an Interlaboratory Study
to Determine the Precision of a Test Method. ASTM International, West Conshohocken,
PA.
Behnke, S., Parker, A., Woodall, D., Camper, A., 2011. Comparing the chlorine disinfection
of detached biofilm clusters with sessile biofilms and planktonic cells in single and
dual species cultures. Appl. Environ. Microbiol. 77 (20), 7176–7184.
Buckingham-Meyer, K., Goeres, D., Hamilton, M., 2007. Comparative evaluation of biofilm
disinfectant efficacy tests. J. Microbiol. Methods 70, 236–244.
Ceri, H., Olson, M.E., Stremick, C., Read, R.R., Morck, D., Buret, A., 1999. The Calgary Biofilm
Device: new technology for rapid determination of antibiotic susceptibilities of
bacterial biofilms. J. Clin. Microbiol. 37, 1771–1776.procedure: collaborative study. JAOAC 96 (4).
Parker, A., Hamilton, M., 2011. Assessing resemblance, repeatability and reproducibility
for quantitative methods. Center for Biofilm, Engineering, KSA-SM-10 (http://www.
biofilm.montana.edu/content/ksa-sm-10).
Parker, A., Hamilton, M., 2013. Using R to assess resemblance, repeatability, and reproduc-
ibility for quantitative and semi-quantitative disinfectant methods. Center for Biofilm,
Engineering, KSA-SM-13 (http://www.biofilm.montana.edu/content/ksa-sm-13).
Parker, A., Tomasino, S., Hamilton, M.A., 2014. Statistical model for assessing performance
standards for quantitative and semi-quantitative disinfectant test methods. JAOAC 97
(1), 58–67.
Pinheiro, J., Bates, D., 2000. Mixed Effects Models in S and S-PLUS. Springer.
Pinheiro, J., Bates, D., DebRoy, S., Sarkar, D., R Development Core Team, 2012. nlme: linear
and nonlinear mixed effects models. R package version 3.1-104.
Piyasena, P., Mohareb, E., McKellar, R.C., 2003. Inactivation of microbes using ultrasound:
a review. Int. J. Food Microbiol. 87, 207–216.
R Core Team, 2012. R: A Language and Environment for Statistical Computing. R Foundation
for Statistical Computing, Vienna, Austria 3-900051-07-0 (URL http://www.R-project.
org/).
Richter, S.J., Richter, C., 2002. A method for determining equivalence in industrial applica-
tions. Qual. Eng. 14 (3), 375–380.
Springthorpe, S., Sattar, S., 2005. Carrier tests to assess microbicidal activities of chemical
disinfectants for use on medical devices and environmental surfaces. JAOAC 88 (1),
182–201.
Tomasino, S., Parker, A., Hamilton, M., Hamilton, G., 2012. Performance of the AOAC
use-dilution method with targeted modifications: collaborative study. JAOAC 95
(6), 1618–1628.
Tomasino, S., Parker, A., Hamilton, M., 2014. Use of statistical modeling to reassess the
performance standard for the AOAC use-dilution methods (955.15 and 964.02).
JAOAC 97 (1), 68–77.E2799-11 Standard TestMethod for TestingDisinfectant Efficacy against Pseudomonas