Browsing by Author "Dimitrov, Alexander G."

Now showing 1 - 4 of 4

The mathematical structure of information bottleneck methods
(2012-03) Gedeon, Tomas; Parker, Albert E.; Dimitrov, Alexander G.
Information Bottleneck-based methods use mutual information as a distortion function in order to extract relevant details about the structure of a complex system by compression. One of the approaches used to generate optimal compressed representations is by annealing a parameter. In this manuscript we present a common framework for the study of annealing in information distortion problems. We identify features that should be common to any annealing optimization problem. The main mathematical tools that we use come from the analysis of dynamical systems in the presence of symmetry (equivariant bifurcation theory). Through the compression problem, we make connections to the world of combinatorial optimization and pattern recognition. The two approaches use very different vocabularies and consider different problems to be â€œinterestingâ€ . We provide an initial link, through the Normalized Cut Problem, where the two disciplines can exchange tools and ideas.
Symmetry breaking clusters in soft clustering decoding of neural codes
(2010-02) Parker, Albert E.; Dimitrov, Alexander G.; Gedeon, Tomas
Information-based distortion methods have been used successfully in the analysis of neural coding problems. These approaches allow the discovery of neural symbols and the corresponding stimulus space of a neuron or neural ensemble quantitatively, while making few assumptions about the nature of either the code or of relevant stimulus features. The neural codebook is derived by quantizing sensory stimuli and neural responses into a small set of clusters, and optimizing the quantization to minimize an information distortion function. The method of annealing has been used to solve the corresponding high-dimensional nonlinear optimization problem. The annealing solutions undergo a series of bifurcations, which we study using bifurcation theory in the presence of symmetries. In this contribution we describe these symmetry breaking bifurcations in detail, and indicate some of the consequences of the form of the bifurcations. In particular, we show that the annealing solutions break symmetry at pitchfork bifurcations, and that subcritical branches can exist. Thus, at a subcritical bifurcation, there are local information distortion solutions which are not found by the method of annealing. Since the annealing procedure is guaranteed to converge to a local solution eventually, the subcritical branch must turn and become optimal at some later saddle-node bifurcation, which we have shown occur generically for this class of problems. This implies that the rate distortion curve, while convex for noninformation-based distortion measures, is not convex for information-based distortion methods.
Symmetry-Breaking Bifurcations of the Information Bottleneck and Related Problems
(MDPI AG, 2022-09) Parker, Albert E.; Dimitrov, Alexander G.
In this paper, we investigate the bifurcations of solutions to a class of degenerate constrained optimization problems. This study was motivated by the Information Bottleneck and Information Distortion problems, which have been used to successfully cluster data in many different applications. In the problems we discuss in this paper, the distortion function is not a linear function of the quantizer. This leads to a challenging annealing optimization problem, which we recast as a fixed-point dynamics problem of a gradient flow of a related dynamical system. The gradient system possesses an 𝑆𝑁 symmetry due to its invariance in relabeling representative classes. Its flow hence passes through a series of bifurcations with specific symmetry breaks. Here, we show that the dynamical system related to the Information Bottleneck problem has an additional spurious symmetry that requires more-challenging analysis of the symmetry-breaking bifurcation. For the Information Bottleneck, we determine that when bifurcations occur, they are only of pitchfork type, and we give conditions that determine the stability of the bifurcating branches. We relate the existence of subcritical bifurcations to the existence of first-order phase transitions in the corresponding distortion function as a function of the annealing parameter, and provide criteria with which to detect such transitions
Temporal coding in a nervous system
(2011-05) Aldworth, Zane N.; Dimitrov, Alexander G.; Cummins, G.; Gedeon, Tomas; Miller, J. P.
We examined the extent to which temporal encoding may be implemented by single neurons in the cercal sensory system of the house cricket Acheta domesticus. We found that these neurons exhibit a greater-than-expected coding capacity, due in part to an increased precision in brief patterns of action potentials. We developed linear and non-linear models for decoding the activity of these neurons. We found that the stimuli associated with short-interval patterns of spikes (ISIs of 8 ms or less) could be predicted better by second-order models as compared to linear models. Finally, we characterized the difference between these linear and second-order models in a low-dimensional subspace, and showed that modification of the linear models along only a few dimensions improved their predictive power to parity with the second order models. Together these results show that single neurons are capable of using temporal patterns of spikes as fundamental symbols in their neural code, and that they communicate specific stimulus distributions to subsequent neural structures.