Development and applications of particle swarm optimization for constructing optimal experimental designs

Abstract

The primary objective motivating this dissertation was to illustrate the efficacy of particle swarm optimization (PSO) as the engine of an algorithm to generate optimal design of experiments (DoE). PSO is a wildly popular and successful metaheuristic and machine learning algorithm which makes no assumptions regarding the behavior of the function being optimized. Optimal DoE, in part thanks to modern computing, has become the current dominant paradigm for practitioners to generate a DoE with some desirable property. We bring together these concepts first by extending the PSO to optimizing functions that take matrix inputs. Julia software was developed for this purpose and validated against published results. A detailed benchmarking study of three PSO variants was conducted and a preferred version of the algorithm was identified for further research and application. Next, we implemented the approach to generating G-optimal designs--a difficult mini-max optimization problem. New heretofore unknown G-optimal designs have been produced and the efficacy of PSO in generating efficiently (w.r.t. computing time) highly G-optimal designs is compared to current published results. Next, a new algorithm for generating optimal designs with specified replication structure, and thereby guaranteed a degrees-of-freedom for estimating the pure error variance, is proposed, illustrated, benchmarked and validated. Last, we propose a new algorithm for generating optimal mixture experiment designs which implements a PSO type search using a non-Euclidean geometry (specifically the Aitchison geometry). In this setting the space of candidate matrices is the Cartesian product of standard (K - 1)-simplices. The algorithm is extended to mixture experiments with lower and upper constraints on the mixture proportions; in this setting, the space of candidate matrices is the Cartesian product of high-dimensional irregular convex polytopes. The algorithm is validated against very recent published results. In total, the work presented in this dissertation speaks very favorably to PSO as a tool for generating optimal DoEs. We believe this approach should become part of the standard machine learning and statistical tool box for generating optimal experimental designs.