Browsing by Author "Sheppard, John W."

Now showing 1 - 11 of 11

Agent-Based Modeling of Retail Electrical Energy Markets with Demand Response
(2018-08-01) Nehrir, Hashem; Dehghanpour, Kaveh; Sheppard, John W.; Kelly, Nathan
In this paper, we study the behavior of a Day-Ahead (DA) retail electrical energy market with price-based Demand Response (DR) from Air Conditioning (AC) loads through a hierarchical multiagent framework, employing a machine learning approach. At the top level of the hierarchy, a retailer agent buys energy from the DA wholesale market and sells it to the consumers. The goal of the retailer agent is to maximize its profit by setting the optimal retail prices, considering the response of the price-sensitive loads. Upon receiving the retail prices, at the lower level of the hierarchy, the AC agents employ a Q-learning algorithm to optimize their consumption patterns through modifying the temperature set-points of the devices, considering both consumption costs and users' comfort preferences. Since the retailer agent does not have direct access to the AC loads' underlying dynamics and decision process (i.e., incomplete information) the data privacy of the consumers becomes a source of uncertainty in the retailer's decision model. The retailer relies on techniques from the field of machine learning to develop a reliable model of the aggregate behavior of the price-sensitive loads to reduce the uncertainty of the decision-making process. Hence, a multiagent framework based on machine learning enables us to address issues such as interoperability and decision-making under incomplete information in a system that maintains the data privacy of the consumers. We will show that using the proposed model, all the agents are able to optimize their behavior simultaneously. Simulation results show that the proposed approach leads to a reduction in overall power consumption cost as the system converges to its equilibrium. This also coincides with maximization in the retailer's profit. We will also show that the same decision architecture can be used to reduce peak load to defer/avoid distribution system upgrades under high penetration of Photo-Voltaic (PV) power in the distribution feeder.
Factored performance functions and decision making in continuous time Bayesian networks
(2017-01) Sturlaugson, Liessman E.; Perreault, Logan J.; Sheppard, John W.
The continuous time Bayesian network (CTBN) is a probabilistic graphical model that enables reasoning about complex, interdependent, and continuous-time subsystems. The model uses nodes to denote subsystems and arcs to denote conditional dependence. This dependence manifests in how the dynamics of a subsystem changes based on the current states of its parents in the network. While the original CTBN definition allows users to specify the dynamics of how the system evolves, users might also want to place value expressions over the dynamics of the model in the form of performance functions. We formalize these performance functions for the CTBN and show how they can be factored in the same way as the network, allowing what we argue is a more intuitive and explicit representation. For cases in which a performance function must involve multiple nodes, we show how to augment the structure of the CTBN to account for the performance interaction while maintaining the factorization of a single performance function for each node. We introduce the notion of optimization for CTBNs, and show how a family of performance functions can be used as the evaluation criteria for a multi-objective optimization procedure.
Hyperspectral Band Selection for Multispectral Image Classification with Convolutional Networks
(2021) Morales, Giorgio; Sheppard, John W.; Logan, Riley D.; Shaw, Joseph A.
In recent years, Hyperspectral Imaging (HSI) has become a powerful source for reliable data in applications such as remote sensing, agriculture, and biomedicine. However, hyperspectral images are highly data-dense and often benefit from methods to reduce the number of spectral bands while retaining the most useful information for a specific application. We propose a novel band selection method to select a reduced set of wavelengths, obtained from an HSI system in the context of image classification. Our approach consists of two main steps: the first utilizes a filter-based approach to find relevant spectral bands based on a collinearity analysis between a band and its neighbors. This analysis helps to remove redundant bands and dramatically reduces the search space. The second step applies a wrapper-based approach to select bands from the reduced set based on their information entropy values, and trains a compact Convolutional Neural Network (CNN) to evaluate the performance of the current selection. We present classification results obtained from our method and compare them to other feature selection methods on two hyperspectral image datasets. Additionally, we use the original hyperspectral data cube to simulate the process of using actual filters in a multispectral imager. We show that our method produces more suitable results for a multispectral sensor design.
Hyperspectral Dimensionality Reduction Based on Inter-Band Redundancy Analysis and Greedy Spectral Selection
(2021-09) Morales, Giorgio; Sheppard, John W.; Logan, Riley D.; Shaw, Joseph A.
Hyperspectral imaging systems are becoming widely used due to their increasing accessibility and their ability to provide detailed spectral responses based on hundreds of spectral bands. However, the resulting hyperspectral images (HSIs) come at the cost of increased storage requirements, increased computational time to process, and highly redundant data. Thus, dimensionality reduction techniques are necessary to decrease the number of spectral bands while retaining the most useful information. Our contribution is two-fold: First, we propose a filter-based method called interband redundancy analysis (IBRA) based on a collinearity analysis between a band and its neighbors. This analysis helps to remove redundant bands and dramatically reduces the search space. Second, we apply a wrapper-based approach called greedy spectral selection (GSS) to the results of IBRA to select bands based on their information entropy values and train a compact convolutional neural network to evaluate the performance of the current selection. We also propose a feature extraction framework that consists of two main steps: first, it reduces the total number of bands using IBRA; then, it can use any feature extraction method to obtain the desired number of feature channels. We present classification results obtained from our methods and compare them to other dimensionality reduction methods on three hyperspectral image datasets. Additionally, we used the original hyperspectral data cube to simulate the process of using actual filters in a multispectral imager.
Hyperspectral imaging and machine learning for monitoring produce ripeness
(2020-04) Logan, Riley D.; Scherrer, Bryan; Senecal, Jacob; Walton, Neil S.; Peerlinck, Amy; Sheppard, John W.; Shaw, Joseph A.
Hyperspectral imaging is a powerful remote sensing tool capable of capturing rich spectral and spatial information. Although the origins of hyperspectral imaging are in terrestrial remote sensing, new applications are emerging rapidly. Owing to its non-destructive nature, hyperspectral imaging has become a useful tool for monitoring produce ripeness. This paper describes the process that uses a visible near-infrared (VNIR) hyperspectral imager from Resonon, Inc., coupled with machine learning algorithms to assess the ripeness of various pieces of produce. The images were converted to reflectance across a spectral range of 387.12 nm to 1023.5 nm, with a spectral resolution of 2.12 nm. A convolutional neural network was used to perform age classification for potatoes, bananas, and green peppers. Additionally, a genetic algorithm was used to determine the wavelengths carrying the most useful information for age classification. Experiments were run using RGB images, full spectrum hyperspectral images, and the genetic algorithm feature selection method. Results showed that the genetic algorithm-based feature selection method outperforms RGB images for all tested produce, outperforms hyperspectral imagery for bananas, and matches hyperspectral imagery performance for green peppers. This feature selection method is being used to develop a low-cost multi-spectral imager for use in monitoring produce in grocery stores.
Improved Yield Prediction of Winter Wheat Using a Novel Two-Dimensional Deep Regression Neural Network Trained via Remote Sensing
(MDPI AG, 2023-01) Morales, Giorgio; Sheppard, John W.; Hedgedus, Paul B.; Maxwell, Bruce D.
In recent years, the use of remotely sensed and on-ground observations of crop fields, in conjunction with machine learning techniques, has led to highly accurate crop yield estimations. In this work, we propose to further improve the yield prediction task by using Convolutional Neural Networks (CNNs) given their unique ability to exploit the spatial information of small regions of the field. We present a novel CNN architecture called Hyper3DNetReg that takes in a multi-channel input raster and, unlike previous approaches, outputs a two-dimensional raster, where each output pixel represents the predicted yield value of the corresponding input pixel. Our proposed method then generates a yield prediction map by aggregating the overlapping yield prediction patches obtained throughout the field. Our data consist of a set of eight rasterized remotely-sensed features: nitrogen rate applied, precipitation, slope, elevation, topographic position index (TPI), aspect, and two radar backscatter coefficients acquired from the Sentinel-1 satellites. We use data collected during the early stage of the winter wheat growing season (March) to predict yield values during the harvest season (August). We present leave-one-out cross-validation experiments for rain-fed winter wheat over four fields and show that our proposed methodology produces better predictions than five compared methods, including Bayesian multiple linear regression, standard multiple linear regression, random forest, an ensemble of feedforward networks using AdaBoost, a stacked autoencoder, and two other CNN architectures.
A Load Profile Management Integrated Power Dispatch Using a Newton-Like Particle Swarm Optimization Method
(2014-10) Wang, Caisheng; Miller, Carol J.; Nehrir, M. Hashem; Sheppard, John W.; McElmurry, Shawn P.
Load profile management (LPM) is an effective demand side management (DSM) tool for power system operation and management. This paper introduces an LPM integrated electric power dispatch algorithm to minimize the overall production cost over a given period under study by considering both fuel cost and emission factors. A Newton-like particle swarm optimization (PSO) algorithm has been developed to implement the LPM integrated optimal power dispatch. The proposed Newton-like method is embedded into the PSO algorithm to help handle equality constraints while penalty/fitness functions are used to deal with inequality constraints. In addition to illustrative example applications of the proposed Newton-like PSO technique, the optimization method has been used to realize the LPM integrated optimal power dispatch for the IEEE RTS 96 system. Simulation studies have been carried out for different scenarios with different levels of load management. The simulation results show that the LPM can help reduce generation costs and emissions. The results also verify the effectiveness of the proposed Newton-like PSO method.
Reduced-cost hyperspectral convolutional neural networks
(2020-09) Morales, Giorgio; Sheppard, John W.; Scherrer, Bryan; Shaw, Joseph A.
Hyperspectral imaging provides a useful tool for extracting complex information when visual spectral bands are not enough to solve certain tasks. However, processing hyperspectral images (HSIs) is usually computationally expensive due to the great amount of both spatial and spectral data they incorporate. We present a low-cost convolutional neural network designed for HSI classification. Its architecture consists of two parts: a series of densely connected three-dimensional (3-D) convolutions used as a feature extractor, and a series of two-dimensional (2-D) separable convolutions used as a spatial encoder. We show that this design involves fewer trainable parameters compared to other approaches, yet without detriment to its performance. What is more, we achieve comparable state-of-the-art results testing our architecture on four public remote sensing datasets: Indian Pines, Pavia University, Salinas, and EuroSAT; and a dataset of Kochia leaves [Bassia scoparia] with three different levels of herbicide resistance. The source code and datasets are available online.
Two-dimensional deep regression for early yield prediction of winter wheat
(2021-11) Morales, Giorgio; Sheppard, John W.
Crop yield prediction is one of the tasks of Precision Agriculture that can be automated based on multi-source periodic observations of the fields. We tackle the yield prediction problem using a Convolutional Neural Network (CNN) trained on data that combines radar satellite imagery and on-ground information. We present a CNN architecture called Hyper3DNetReg that takes in a multi-channel input image and outputs a two-dimensional raster, where each pixel represents the predicted yield value of the corresponding input pixel. We utilize radar data acquired from the Sentinel-1 satellites, while the on-ground data correspond to a set of six raster features: nitrogen rate applied, precipitation, slope, elevation, topographic position index (TPI), and aspect. We use data collected during the early stage of the winter wheat growing season (March) to predict yield values during the harvest season (August). We present experiments over four fields of winter wheat and show that our proposed methodology yields better results than five compared methods, including multiple linear regression, an ensemble of feedforward networks using AdaBoost, a stacked autoencoder, and two other CNN architectures.
Univariate Skeleton Prediction in Multivariate Systems Using Transformers
(Springer Nature, 2024-08) Morales, Giorgio; Sheppard, John W.
Symbolic regression (SR) methods attempt to learn mathematical expressions that approximate the behavior of an observed system. However, when dealing with multivariate systems, they often fail to identify the functional form that explains the relationship between each variable and the system’s response. To begin to address this, we propose an explainable neural SR method that generates univariate symbolic skeletons that aim to explain how each variable influences the system’s response. By analyzing multiple sets of data generated artificially, where one input variable varies while others are fixed, relationships are modeled separately for each input variable. The response of such artificial data sets is estimated using a regression neural network (NN). Finally, the multiple sets of input–response pairs are processed by a pre-trained Multi-Set Transformer that solves a problem we termed Multi-Set Skeleton Prediction and outputs a univariate symbolic skeleton. Thus, such skeletons represent explanations of the function approximated by the regression NN. Experimental results demonstrate that this method learns skeleton expressions matching the underlying functions and outperforms two GP-based and two neural SR methods.
Using machine learning to predict catastrophes in dynamical systems
(2012-03) Berwald, Jesse; Gedeon, Tomas; Sheppard, John W.
Nonlinear dynamical systems, which include models of the Earth’s climate, financial markets and complex ecosystems, often undergo abrupt transitions that lead to radically different behavior. The ability to predict such qualitative and potentially disruptive changes is an important problem with far-reaching implications. Even with robust mathematical models, predicting such critical transitions prior to their occurrence is extremely difficult. In this work, we propose a machine learning method to study the parameter space of a complex system, where the dynamics is coarsely characterized using topological invariants. We show that by using a nearest neighbor algorithm to sample the parameter space in a specific manner, we are able to predict with high accuracy the locations of critical transitions in parameter space.