Expectation and Variance

Introduction

In the world of probability and statistics, it is important to understand how random events behave. Two fundamental concepts that help us understand these behaviors are "expectation" or "expected value" and "variance." These concepts allow us to summarize the characteristics of random variables, which are variables whose possible values are the numerical outcomes of a random event.

Expectation: expected value

The expected value, often denoted as E(X), is a central concept in probability. It represents the average outcome one might expect from a random experiment if it were repeated many times. Think of it as a "weighted average" of all possible values of the random variable, where each possible value is weighted according to its probability of occurring.

To calculate the expected value of a discrete random variable X, you use the formula:

E(X) = Σ [x * P(x)]

Here, x represents each possible value of X, and P(x) is the probability that X equals x. For continuous random variables, the formula involves an integration rather than a summation.

Example 1: Throwing fair dice

Let us consider a simple example of rolling a fair six-sided dice. The random variable X represents the outcome of the dice roll, and its possible values are 1, 2, 3, 4, 5, and 6, each with a probability of 1/6. The expected value of this random variable is calculated as follows:

E(X) = 1 * (1/6) + 2 * (1/6) + 3 * (1/6) + 4 * (1/6) + 5 * (1/6) + 6 * (1/6)

Solving this, we get the expected value:

E(X) = 3.5

Although you can't expect a die to roll exactly 3.5, this value represents the "average" result you would expect when rolling a number of dice.

Example 2: Expectation of a continuous random variable

For continuous random variables, the expected value is calculated using an integral. Let's say we have a random variable X with a probability density function (PDF) f(x). The expected value is given by the formula:

E(X) = ∫ x * f(x) dx

Assume that X follows a uniform distribution on the interval [a, b]. The PDF is f(x) = 1 / (b - a). The expected value is:

E(X) = ∫ (x / (ba)) dx, from a to b

this results in:

E(X) = (b + a) / 2

Variance: Measuring variability

While the expected value gives us a point estimate, it does not tell us how much variability there is around this average. The variance is a measure of this spread or variability in the possible values of a random variable.

The variance of a random variable X, denoted as Var(X) or σ², is defined as the expectation of the squared deviation of X from its expected value. The formula for variance is:

Var(X) = E[(X - E(X))²]

Let's break down this formula:

• First, calculate the expected value E(X).
• Then, find the deviation of each value x_i from E(X), which is x_i - E(X).
• Square these deviations to make sure they are nonnegative.
• Multiply each squared deviation by its probability P(x_i).
• Sum these weighted squared deviations to obtain Var(X).

Example 3: Variance of throwing a dice

Using the previous example of throwing a six-sided dice, let's find the variance.

Recall E(X) = 3.5. The variance would be calculated as follows:

Var(X) = (1 - 3.5)² * (1/6) + (2 - 3.5)² * (1/6) + ... + (6 - 3.5)² * (1/6)

Its evaluation is as follows:

Var(X) = 17.5 / 6 ≈ 2.92

This variance value gives us a measure of how “spread” the dice rolls are around the expected value of 3.5.

Example 4: Variance of a continuous random variable

For a continuous random variable, the variance is given by:

Var(X) = ∫ (x - E(X))² * f(x) dx

Assuming X is uniformly distributed as in the previous example, we have already found E(X) = (b+a)/2. The variance is:

Var(X) = ∫ [(x - (b+a)/2)² * f(x)] dx, from a to b

Properties of expectation and variance

Properties of expectation

• Linearity: The expected operator is linear, that is, if a and b are constants, then E(aX + bY) = aE(X) + bE(Y), where X and Y are random variables.
• Expectation of a constant: For a constant c, E(c) = c.
• Sum of random variables: If X and Y are random variables, then E(X+Y) = E(X) + E(Y).

Properties of variance

• Variance of a constant: The variance of a constant is zero because it does not change.
• Additive property: If X and Y are independent random variables, Var(X+Y) = Var(X) + Var(Y)
• Scaling: Var(aX) = a²Var(X) where a is a constant.

Conclusion

Expectation and variance are fundamental concepts in probability and statistics. They provide a way to summarize and understand random variables. While expectation tells us what the "average" outcome is, variance gives us an idea of how much variability should be expected from this average. Mastering these concepts is important for anyone who wants to delve deeper into fields related to probability and statistics.

Comments