Three-Part Gaussian Kernel

Jeff Heaton
2014-07-19

For my project I created a Shiny application that allows you to experiment with a three-part Gaussian kernel. Compound Gaussian kernels are used for Continuous Ant Colony Optimization (ACO). I am currently researching an algorithm based on ACO documented in this journal paper. Blum, C. An ant colony optimization algorithm for continuous optimization: application to feed-forward neural network training. Neural Computing and Applications, 235-247.

Normal Distribution

Normal distributes are defined by their mean (mu) and standard deviation (sigma) values. Here are several normal (Gaussian) distributions.

plot of chunk unnamed-chunk-1

Multi-Gaussian Kernel

Gaussians are not effective at encoding a multi-peaked function. However, several Gaussians can be combined to produce a multi-peaked function. Fitting the individual mean and standard deviations allows complex multi-peaked functions to be approximated. Here you see a system of three Gaussian functions used to approximate a multi-peaked function.

plot of chunk unnamed-chunk-2

Combining Multiple Gaussians

A regular Gaussian function (the normal distribution) can be expressed as the following function, if we include the mean(\( \mu \)) and standard deviation(\( \sigma \)).

\[ g(x, \mu, \sigma) = \frac{1}{\sigma\sqrt{2\pi}} e^{ -\frac{(x-\mu)^2}{2\sigma^2} } \]

Summing the above function produces the compound Gaussian kernel shown on the previous slide. Any number of Gaussians (k) can be used.

\[ G(x)=\sum^k_{l=1} \omega_l g(x, \mu_l, \sigma_l) \]

My Shiny Experiment

Shiny looked like a great way to create a simple three-part Gaussian kernel for me to experiment with. You can change the mean and standard deviation for the three Gaussian functions that make up the kernel. This allows you to see how your changes affect the final kernel (dashed line). https://jeffheaton.shinyapps.io/shiny/

Here is the GitHub repo for this app: https://github.com/jeffheaton/jh-shinyapp