Probability
Experiments in which the results are not known in advance are called random experiments, although we can predict what is likely to happen. A set of outcomes from a random experiment is called an event. A sample space is a set of all possible outcomes in a random experiment.
- Axiom 1 : Probability is always greater than or equal to 0 (
). - Axiom 2 : The probability of the sample space is 1 (
). - Axiom 3 : In the case of mutually exclusive events A and B, the relationship
holds. Mutually exclusive events mean that , and , , and denote union, intersection, and the empty set, respectively.
Probability is defined by a given case. Therefore, from the three axioms above, we know that
Random Variable
A random variable
For example,
The domain of a random variable is the sample space, and the range is
Since an event is a set of elements
Additionally, if a random variable (
Probability Distribution Function, Probability Density Function and Probability Mass Function
Probability Distribution Function
Since
The probability distribution function is expressed as follows:
According to the definition,
Probability Density Function
The probability density function
According to the definition above, if the probability density function is differentiable, it can be expressed as follows:
The probability that the random variable
According to the definition of the probability density function,
Probability Mass Function
A discrete random variable
where
By using the Dirac delta function (
Note that the Dirac delta function is defined by the following two properties:
Joint Probability Function
The joint probability distribution function
The joint probability density function
If
Since
Conditional Probability
The probability that an event A occurs given that event B has occurred is called the conditional probability of event A, and it is defined as equation [11].
The conditional probability density function
If event A is
If the probability of event A occurring given that
Conversely, the conditional probability density function of
Independent Random Variable
If the joint probability of events A and B equals the product of the probabilities of A and B, then events A and B are called independent events :
If the joint probability of the N events
Likewise, if the probability density function of random variables satisfies equation [18], the N random variables are independent.
If two random variables
Function of Random Variables
If the random variable
For example, let’s assume the functional relationship between two random variables
The probability density function of
It is also possible to calculate the probability density function of two random variables. Let’s assume that
Thus, the probability distribution function of
Next, to solve equation [24], we use Leibniz integral Rule[28] to become equation [24].
Likewise, equation [25] can be written like equation [26].
If
Leibniz Integral Rule
Sampling
A sample extracted from the random variable
Assume that
As shown in equation [30], if each sample is independently and equitably extracted from a population with some probabilistic features, the extracted sample is called an IID (independent and identically distributed) sample. By using equation [6], we can approximate
Then we can calculate the probability
Therefore, the histogram that shows the number of samples belonging to an arbitrary bin has the same shape as the approximation of the probability density function
For example, let’s approximate the probability density function of
As you can see in the picture above, it merely corresponds to the result of the example.
Example
Question.
Assume that
Find the probability density function of
Answer.
Therefore,