Probability space

Introduction

The notion "probability space" provides a basis of the formal structure of probability theory. It may puzzle a non-mathematician, since

it is called "space" but is far from geometry;

it is said to provide a basis, but many people applying probability theory in practice neither understand nor need this quite technical notion;

not every set of sample points is treated as event and assigned probability.

These puzzling facts are explained below. First, a mathematical definition is given; it is quite technical, but the reader may skip it. Second, an elementary case (finite probability space) is presented. Third, the puzzling facts are explained. Next topics are countably infinite probability spaces, and general probability spaces.

Definition

A probability space is a measure space such that the measure of the whole space is equal to 1.

In other words: a probability space is a triple Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \textstyle (\Omega, \mathcal F, P)} consisting of a set Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \textstyle \Omega} (called the sample space), a σ-algebra (called also σ-field) Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \textstyle \mathcal F } of subsets of Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \textstyle \Omega} (these subsets are called events), and a measure Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \textstyle P} on Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \textstyle (\Omega, \mathcal F)} such that Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \textstyle P(\Omega)=1} (called the probability measure).

Elementary level: finite probability space

On the elementary level, a probability space consists of a finite number Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle n} of sample points Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \omega_1, \dots, \omega_n } and their probabilities Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle p_1, \dots, p_n } — positive numbers satisfying Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle p_1 + \dots + p_n = 1. } The set Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \Omega = \{ \omega_1, \dots, \omega_n \} } of all sample points is called the sample space. Every subset Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle A \subset \Omega } of the sample space is called an event; its probability is the sum of probabilities of its sample points. For example, if Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle A = \{ \omega_1, \omega_8, \omega_9 \} } then Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbb{P} (A) = p_1 + p_8 + p_9 } .

The case of equal probabilities is especially important: Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle p_1 = p_2 = \dots = p_n = 1/n ; } Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbb{P} (A) = |A|/n = |A|/|\Omega| ; } here Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle |A| } is the number of elements in Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle A.} This case is called the uniform distribution (on a finite set Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \Omega} ), or a symmetric probability space. The uniform distribution is invariant under all permutations, that is, one-to-one maps Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \Omega \to \Omega . }

A random variable Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle X } is described by real numbers Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle x_1, \dots, x_n } (not necessarily different) corresponding to the sample points Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \omega_1, \dots, \omega_n. } Its expectation is Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbb{E} (X) = x_1 p_1 + \dots + x_n p_n. }

On a symmetric probability space, the expectation is the arithmetic mean, Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbb{E} (X) = (x_1+\dots+x_n)/n. } Still, the values of Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle X} are not necessarily of equal probabilities, since the numbers Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle x_1,\dots,x_n } are not necessarily different.

The puzzling facts explained

Why "space"?

Fact: it is called "space" but is far from geometry.

Explanation: the modern mathematics treats "space" quite differently from the classical mathematics; see Space (mathematics).

What is it good for?

Fact: it is said to provide a basis, but many people applying probability theory in practice do not need this notion. For them, formulas (such as the addition rule, the multiplication rule, the inclusion-exclusion rule, the law of total probability, Bayes' rule etc.^[1]) are instrumental; probability spaces are not, they reign but do not rule.

Explanation 1. Likewise, one may say that points are of no use in geometry. Formulas connecting lengths and angles (such as Pythagorean theorem, law of sines etc.) are instrumental; points are not.

However, these useful formulas follow from the axioms of geometry formulated in terms of points (and some other notions). It would be very cumbersome and unnatural, if at all possible, to reformulate geometry avoiding points.

Similarly, the formulas of probability follow from the axioms of probability formulated in terms of probability spaces. It would be very cumbersome and unnatural, if at all possible, to reformulate probability theory avoiding probability spaces.

Explanation 2. One of the most useful formulas is linearity of expectation: Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbb{E} (aX+bY) = a \mathbb{E} (X) + b \mathbb{E} (Y) } whenever Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle X, Y } are random variables and Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle a, b } are (non-random) coefficients. One may derive this formula avoiding probability spaces, by transforming the sum

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \sum_{x,y} (ax+by) \mathbb{P} (X=x, Y=y ) }

into the linear combination

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle a \sum_x x \mathbb{P} (X=x) + b \sum_y y \mathbb{P} (Y=y). }

However, much better insight is provided by probability spaces: the expectation Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbb{E} (X) = x_1 p_1 + \dots + x_n p_n } is a linear function of the variables Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle x_1, \dots, x_n. } Moreover, a helpful connection to linear algebra appears: random variables form an Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle n} -dimensional linear space, and the expectation is a linear functional on this space.

Why some sets are better than others?

Fact: not every set of sample points is treated as event and assigned probability.

The explanation is postponed to the end of the article, since this is not an easy matter.

Two approaches to infinity

Everything is finite in applications, but mathematical theories often benefit by using infinity. In mathematical analysis, infinity appears only indirectly, via limiting procedure, when one says that something "tends to infinity". In the set theory, infinity appears directly; for instance, one say that "the set of prime numbers is infinite". Both approaches to infinity can be used in probability theory.

Example 1. "A randomly chosen positive integer is even with probability 0.5." This phrase is interpreted via limiting procedure: the fraction of even numbers among Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle 1,\dots,n } converges to 0.5 as Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle n } tends to infinity. This approach introduces an infinite sequence of finite probability spaces; the Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle n} -th space consists of sample points Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle 1,\dots,n } endowed with equal probabilities Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle 1/n. }

Example 2. "Flipping a fair coin repeatedly one must get "heads" sooner or later." Also this phrase may be interpreted via an infinite sequence of finite probability spaces: flipping the coin Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle n} times one gets "heads" at least once with the probability Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle 1 - 2^{-n} } that converges to 1 as Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle n} tends to infinity. Another interpretation is possible, via a single infinite probability space consisting of the sequences H, TH, TTH, TTTH and so on ("TTH" means: "tails" twice, then "heads"; the coin is tossed until "heads") having the probabilities

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbb{P} (H) = 1/2; \quad \mathbb{P} (TH) = 1/4; \quad \mathbb{P} (TTH) = 1/8; \quad \dots }

whose sum is Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle 2^{-1} + 2^{-2} + 2^{-3} + \dots = 1. } One may insert also the infinite sequence "TTT..." ("tails forever") to the sample space; but then necessarily

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbb{P} (TTT\dots) = 0 }

since the sum of probabilities cannot exceed 1.

It is tempting to extend this approach (a single infinite probability space) to the case of Example 1, defining

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbb{P} (A) = \lim_{n\to\infty} \frac{ | A \cap \{1,\dots,n\} | }{ n } }

for Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle A \subset \Omega = \{ 1,2,\dots \}; } here Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle | A \cap \{1,\dots,n\} | } is the number of elements of $A$ among $1,\dots ,n.$ This limit, called the density of $A,$ is a useful mathematical device. However, treating it as probability one gets numerous paradoxes. One paradox: a positive integer chosen at random must have more than one decimal digit, since $\mathbb {P} (\{1,\dots ,9\})=0.$ Similarly, it must have more than two digits; and so on. Thus, it must have infinitely many digits, which cannot happen to an integer. Another paradox: let two positive integers $X,Y$ be chosen at random, independently. Then $\mathbb {P} (X\leq Y)=0,$ since $\mathbb {P} (X\leq 1)=0,$ Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbb{P} ( X \le 2 ) = 0 } and so on. Similarly, Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbb{P} ( Y \le X ) = 0. } Thus, it must be Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle X > Y > X } .

By default (unless explicitly stated otherwise), probability theory deals with a single probability space. When solving a specific problem, the probability space is usually (but not always) chosen according to the given problem; when developing general theory, it is arbitrary.

The notions "negligible" and "almost sure"

A sample point of zero probability can be added to a probability space or removed from it at will, since it cannot contribute to any probability (or expectation). Such point is called negligible.

In Example 2 (above) the case "tails forever" is negligible.

An event of probability 1 is said to happen almost surely.

In Example 2 (above), "heads" appears (sooner or later) almost surely.

The following anecdote follows a real event.

Professor (dealing with a random variable Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle X} ): ...here we use the evident fact that Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle -1 \le \sin X \le 1 } almost surely.

Student: Why "almost surely"? It holds surely.

Professor (laughing): You see, I am a probabilist. We probabilists do not say "sure"; "almost sure" is our strongest expression.

Countable additivity

As was noted above, paradoxes prevent treating the density of a set Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle A \subset \Omega = \{ 1,2,\dots \} } as its probability. These paradoxes are caused by violation of countable additivity. Namely, single-point sets Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle A_1 = \{1\}, \, A_2 = \{2\}, \, A_3 = \{3\}, \dots } are of density 0 (each), but their union Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle A_1 \cup A_2 \cup \dots = \Omega } is of density 1.

The countable additivity requires

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbb{P} ( A_1 \cup A_2 \cup \dots ) = \mathbb{P} (A_1) + \mathbb{P} (A_2) + \dots }

whenever events Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle A_1, A_2, \dots } are mutually excluding (in other words, disjoint sets).

Countable additivity is an axiom of probability theory.

For a random choice of an integer, the countable additivity implies that the probability of a set is the sum of probabilities of its elements,

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbb{P} (A) = \sum_{n\in A} p_n . }

This is a finite sum for a finite Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle A,} but an infinite series for an infinite Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle A.} The order of terms does not matter, since all terms are nonnegative. The series converges, since its partial sums cannot exceed 1. For example, the probability of being even:

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \mathbb{P} (A) = p_2 + p_4 + p_6 + \dots \qquad \text{for } A = \{ 2,4,6,\dots \} . }

The numbers Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle p_n } must satisfy

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle p_n \ge 0 ; \qquad p_1 + p_2 + p_3 + \dots = 1 . }

Otherwise, they are arbitrary; every sequence of numbers satisfying these conditions leads to a probability space.

The case of equal probabilities, Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle p_1 = p_2 = p_3 = \dots, } is impossible, since the series Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle p+p+p+\dots } never converges to 1; it converges to 0 if Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle p = 0 } and diverges (to infinity) if Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle p > 0. } Thus, the phrase "an integer chosen at random" is meaningless if a probability distribution on the integers is not specified. "The uniform distribution on the integers" does not exist.

The need for uncountable probability spaces

Endless tossing of a fair coin is a classical object of probability theory. The weak law of large numbers, the strong law of large numbers, the central limit theorem, — they all were developed first for this special case, and later generalized.

Many textbooks in probability explain only (finite and) countable probability spaces, but do not hesitate to write "Consider an infinite sequence Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle A_1,A_2,\dots } of independent events of probability Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle 1/2 } ". The problem is that existence of Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle n} such events Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle A_1,\dots,A_n } implies that each sample point is of probability Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \le 2^{-n}; } thus, existence of the infinite sequence Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle A_1,A_2,\dots } implies that each sample point is of probability zero! In a (finite or) countable probability space this situation is impossible by countable additivity.

Another classical object of probability theory is the normal distribution. In the discrete framework one may speak about a sequence of discrete distributions converging to the normal shape. However, continuous distributions (normal, uniform etc.) of random variables are not possible on (finite or) countable probability spaces.

The two problems mentioned above are closely related:

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle U = \sum_{n=1}^\infty 2^{-n} I_{A_n} , }

where Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle U} is a random variable distributed uniformly on Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle (0,1);} Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle A_1,A_2,\dots } are independent events of probability 0.5 each; and Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle I_{A_n} } is the indicator of Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle A_n } (1 if Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle A_n } occurs and 0 otherwise). Thus, Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle I_{A_n} } is equal to the Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle n} -th binary digit of the number Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle U.}

A problem with uncountable probability spaces

In an uncountable probability space it is quite possible (and usual) that each point is of zero probability. Then, by the countable additivity, every countable set is of zero probability. However, the whole space must be of probability 1.

In general, the probability of an event is not the sum of probabilities of its sample points.

This is the problem with uncountable probability spaces. Several implications follow.

When choosing at random, uniformly, a number of the interval Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle (0,1),} the point 0.5 has no chance to be chosen; it is negligible. Moreover, the set of all rational numbers is (countable, therefore) negligible; the random number is irrational almost surely. In terms of its binary digits, they are (almost surely) a non-periodic sequence. However, the irrational numbers Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle 1/\sqrt2} and Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle 1/\pi} are also negligible. A puzzle: no matter which point is chosen, it had no chance to be chosen! An explanation: tossing a fair coin 1000 times one gets a sequence of 1000 characters H and T; no matter which sequence is obtained, it was practically impossible, since its probability was Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle 2^{-1000} \approx 10^{-301}. } For endless coin tossing the small probability becomes zero.

In contrast to the discrete probability, the property "all sample points are of equal probability" does not characterize the uniform distribution on the interval Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle (0,1).} This property holds trivially for each continuous distribution; all points are of (equal) zero probability! Also invariance under all one-to-one maps of Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle (0,1)} onto itself does not characterize the uniform distribution on the interval Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle (0,1),} for another reason: every distribution violates this property. Especially, the uniform distribution violates it: if $U$ is distributed uniformly (on the interval $(0,1)$ ) then $U^{2}$ is not; for instance, $\mathbb {P} (U^{2}<0.25)=\mathbb {P} (U<0.5)=0.5\neq 0.25$ in spite of the fact that $x\mapsto x^{2}$ is a one-to-one map of $(0,1)$ onto itself.

In fact, the uniform distribution on $(0,1)$ is characterized by the following property: intervals of equal length (within $(0,1)$ ) have equal probabilities. Intervals are the building blocks instead of points. This is a special case of the following approach.

Probabilities are initially assigned to some "simple" sets, and then extended to more "complicated" sets by countable additivity.

On the real line, some sets are related to intervals in such a way that their probabilities can be derived from probabilities of intervals. These sets are called measurable. A measurable set can be quite complicated. An example is the set of all numbers $x=\sum 2^{-n}\beta _{n}$ whose binary numbers $\beta _{n}$ satisfy the strong law of large numbers: $(\beta _{1}+\dots +\beta _{n})/n\to 1/2$ as $n\to \infty .$ This is a dense set, and its complement is also dense. Moreover, this set contains uncountably many points within every interval; and its complement does. Nevertheless, it is measurable.

The situation is similar in all uncountable probability spaces. For some sets, their probabilities can be derived from the probabilities of "simple" sets; such sets are called measurable and treated as events.

Fortunately, all sets that appear in practical problems belong to this class. Accordingly, applied mathematicians, physicists, engineers etc. usually need not bother about measurability. In contrast, probability theory as a rigorous mathematical theory always stipulates measurability, sometimes by assumption, sometimes by construction.

Notes

↑ See [1], [2]

[1] See [1], [2]

[1]

Probability space

Contents

Introduction

Definition

Elementary level: finite probability space