CONTINUOUS BERNOULI DISTRIBUTION
Abstract
What is a Probability Distribution?
The possible values a variable can take and how frequently they occur is called Distribution.
Let Y=The actual outcome of an event and y=One of the possible outcomes one of the ways to reach like hood of reaching 'y' is denoted as "P(Y=y)" or "p(y)".
Example: Let Y->The number of red marbles we draw out of bad and y->5 then we expresses the probability of getting exactly 5 red marbles as P(Y=5) or p(5).
Since we p(y) represents or calls as "the probability function".
The probability distributions or simply probabilities, measure the likelihood of an outcome depending on how often if features in the sample space.
We define distributions in two categories
1.Mean denoted as "mu" (a Greek word)
2.Variance denoted as "sigma squared"
In simple terms we define as follows
Mean->Average value
Variance->how spread-out the data is, we measure this "spread" by how far away from the mean all the values are. The more data is dispersed higher the variance will be.
Types of
probability distributions
Consider like rolling a die or picking a card, have a finite number of outcomes they follow "discreet distributions". Other types like recording time and distance in track and field, have infinitely many outcomes, they follow "continuous distributions".
Discrete
probability distributions
A discrete distribution describes the probability of occurrence of each value of a discrete random variable. The number of spoiled apples out of 6 in your refrigerator can be an example of a discrete probability distribution.
Each possible value of the discrete random variable can be associated with a non-zero probability in a discrete probability distribution.
Binomial Distribution
The binomial distribution is a discrete distribution with a finite number of possibilities. When observing a series of what are known as Bernoulli trials, the binomial distribution emerges. A Bernoulli trial is a scientific experiment with only two outcomes: success or failure.
Consider a random experiment in which you toss a biased coin six times with a 0.4 chance of getting head. If ‘getting a head’ is considered a 'success', the binomial distribution will show the probability of r successes for each value of r.
The binomial random variable represents the number of successes (r) in n consecutive independent Bernoulli trials.
Uses of this binomial distribution is that many complex problems in business can be solves by using this distribution.
Ex: what is the oil price?
What is the probability that stock market will crash..etc
The Binomial distribution computes the probabilities of events where only two possible outcomes can occur (success or failure), e.g. when you look at the when stock market will crash, the outcome of interest is whether the stock market is crashed or not.
Bernoulli’s Distribution
The Bernoulli distribution is a variant of the Binomial distribution in which only one experiment is conducted, resulting in a single observation. As a result, the Bernoulli distribution describes events that have exactly two outcomes.
The Bernoulli random variable’s expected value is p, which is also known as the Bernoulli distribution’s parameter.
The experiment’s outcome can be a value of 0 or 1. Bernoulli random variables can have values of 0 or 1.
The Bernoulli distribution is the basis of the extremely widely used Binomial distribution. The binomial has the parameters N and p, the Bernoulli is the same but with N=1, so it covers, e.g. one toss of a coin.
So in every situation of the Binomial with N=1, we can say it's the Bernoulli. But in real life applications N > 1 (I'm sure there must be cases of N =1, but I can''t think of any right now. I consider one flip of a coin to be so basic, the I don't use the term "distribution" in such a case.) this is the one of the use of Bernoulli.
Poisson Distribution
A Poisson distribution is a probability distribution used in statistics to show how many times an event is likely to happen over a given period of time. To put it another way, it’s a count distribution. Poisson distributions are frequently used to comprehend independent events at a constant rate over a given time interval. Simeon Denis Poisson, a French mathematician, was the inspiration for the name.
It has two parameters:
1. Lam: Known number of occurrences
2. Size: The shape of the returned array
The main uses of this distribution is that
1.To determine the how much variation there will likely be from that average number of occurrences.
2.And the probable maximum and minimum number of times the event will occur in limit of time.
Companies can utilize the Poisson Distribution to examine how they may be able to take steps to improve their operational efficiency
Continuous Probability Distributions
A continuous distribution describes the probabilities of a continuous random variable’s possible values. A continuous random variable has an infinite and uncountable set of possible values (known as the range). The mapping of time can be considered as an example of the continuous probability distribution. It can be from 1 second to 1 billion seconds, and so on.
The area under the curve of a continuous random variable’s PDF is used to calculate its probability. As a result, only value ranges can have a non-zero probability. A continuous random variable’s probability of equaling some value is always zero.
Now, look at some varieties of the continuous probability distribution.
Normal Distribution
Normal Distribution is one of the most basic continuous distribution types. Gaussian distribution is another name for it. Around its mean value, this probability distribution is symmetrical. It also demonstrates that data close to the mean occurs more frequently than data far from it. Here, the mean is 0, and the variance is a finite value.
Uses
The main use of this distribution is that continuous data in nature and psychology display this bell-shaped curve when graphed and one more main use is "to find prob of observatons in a distribution for both higher and lower limit values".
The graph shown is the sample of Normal distribution.
Continuous Uniform Distribution
In continuous uniform distribution, all outcomes are equally possible. Each variable has the same chance of being hit as a result. Random variables are spaced evenly in this symmetric probabilistic distribution, with a 1/ (b-a) probability.
Log-Normal Distribution
The random variables whose logarithm values follow a normal distribution are plotted using this distribution. Take a look at the random variables X and Y. The variable represented in this distribution is Y = ln(X), where ln denotes the natural logarithm of X values.
The size distribution of rain droplets can be plotted using log normal distribution.
Uses
most common applications where log-normal distributions are used in finance is in the analysis of stock prices. he prices of the stock, however, can be graphed in a log-normal distribution. The log-normal distribution curve can therefore be used to help better identify .
Lognormal distribution can be done in Excel.
Excel defines it as the following:
LOGNORM.DIST (x,mean,standard_dev, cumulative)
Exponential Distribution
In a Poisson process, an exponential distribution is a continuous probability distribution that describes the time between events (success, failure, arrival, etc.).
You can see in the below example how to get random samples of exponential distribution and return NumPy array samples by using the NumPy. random. Exponential() method.
Uses
The exponential distribution assumes that small values occur more frequently than large values. Consequently, it can model things like wait times, transaction times, and failure times. It can also model other variables, such as the size of orders at convenience stores.
This is used to calculate the amount of time until a certain event occurs.
And some other applications are
1.Time takes for your next call
2.Time until default in reduces form credit risk modelling.
Usage of probability
distributions
Probability distributions can be of type discrete or of type continuous, depending on whether you want to define probabilities for discrete variables or for continuous variables.
You can use probability distributions in either of the following ways:
By using a discrete distribution, you can define the probability of occurrence of each value of a discrete random variable. A discrete random variable is a random variable that has only countable values, for example, a list of non-negative integers. by using a continuous distribution, you can define the probabilities of the possible values of a continuous random variable. A continuous random variable is a random variable that has a set of possible infinite and uncountable values.
Introduction
**Continuous
Bernoulli distribution**
The Bernoulli distribution Is the simplest discret distribution and it the building block for the other more complicated discret distributions.
The continuous Bernoulli distribution arises in deep learning and computer vision, specifically in the context of variational autoencoders, for modelling the pixel intensities of natural images. As such, it defines a proper probabilistic counterpart for the commonly used binary cross entropy loss, which is often applied to continuous, [0,1]{} [-valued data. This practice amounts to ignoring the normalizing constant of the continuous Bernoulli distribution, since the binary cross entropy loss only defines a true log-likelihood for discrete, {0,1}{ {0,1}}{{-valued data. {p(x|) {x}(1-){1-x}.}
Here we introduce and fully characterize the continuous Bernoulli distribution (§3), both as a means to study the impact of this widespread modeling error, and to provide a proper VAE for [0, 1]-valued data. Before these details, let us ask the central question: who cares?{ p(x|)(x)}
In order to analyse the implications of this modelling error, we introduce the continuous Bernoulli, a novel distribution on [0, 1], which is parameterized by λ ∈ (0, 1) and defined by: X ∼ CB(λ) ⇐⇒ p(x|λ) ∝ p˜(x|λ) = λ ^v(1 − λ)^ 1−x
Formulas:
Here we are discussing about the formulas of the continuous Bernoulli's distribution. We have the probability distribution function formula, mean and variance. Through variance we can find the standard deviation.
Real life
application of distribution variational autoencoders (VAE)
Variational
autoencoders (VAE)
VAE has become the one of the most used tool in Machine learning which used to a broad range of data types and variable modes. While designing the VAE we consider the continuous Bernoulli distribution as per design technique.
Though doing so will not throw an obvious type error, the implied object is no longer a coherent probabilistic model, due to a neglected normalizing constant. This practice is extremely pervasive in the VAE literature, including the seminal work of Kingma and Welling [20] (who, while aware of it, set it aside as an inconvenience), highly-cited follow up work (for example [25, 37, 17, 6] to name but a few), VAE tutorials [7, 1], including those in hugely popular deep learning frameworks such as PyTorch [32] and Keras [3], and more.
Second, one might suppose this error can be interpreted or fixed via data augmentation, binarizing data (which is also a common practice), stipulating a different lower bound, or as a nonprobabilistic model with a "negative binary cross-entropy" objective. §4 explores these possibilities and finds them wanting. Also, one might be tempted to call the Bernoulli VAE a toy model or a minor point. Let us avoid that trap: MNIST is likely the single most widely used dataset in machine learning, and VAE is quickly becoming one of our most popular probabilistic models.
Third, and most importantly, empiricism; shows three key results:
(i) as a result of this error, we show that the Bernoulli VAE significantly underperforms the continuous Bernoulli VAE across a range of evaluation metrics, models, and datasets;
(ii) a further unexpected finding is that this performance loss is significant even when the data is close to binary, a result that becomes clear by consideration of continuous Bernoulli limits; and
(iii) we further compare the continuous Bernoulli to beta likelihood and Gaussian likelihood VAE, again finding the continuous Bernoulli performant. All together this work suggests that careful treatment of data type – neither ignoring normalizing constants nor defaulting immediately to a Gaussian likelihood – can produce optimal results when modelling some of the most core datasets in machine learning.
Mostly we are using this for fixing a pervasive error in variational autoencoders.
Usually this will explains when there are two outcomes in the field like winning a championship, outcome of tossing a coin. Etc.
Through the continuous Bernoulli's distribution we can find the probabilities of some other distributions like bernoulis,beta and exponential. Etc.
Analysis:
We can use the continuous Bernoulli distribution as some of the distributions.
Here we are using the beta distribution.
Problems
Conclusion:
Cbd is a linked to many distributions which are based and used to solve the many machine learning problems and shows the easy way to find the solution.As is the like the Bernoulli distribution these are can we conclude the required results from these distribution.As these values are between the [0,1] we can retrive the probabilities.
Here through this we have studied how can we fix the passive errors in the “variational auto encoders” which works on deep learning.aAnd we can use this by for huge datasets,range of matrices and sharpen images.
references:
https://en.wikipedia.org/wiki/Continuous_Bernoulli_distribution