Undergraduate
  1. Algebra  1.1. Linear Algebra  1.1.1. Matrices  1.1.2. Determinants  1.1.3. Systems of Linear Equations  1.1.4. Vector Spaces  1.1.5. Linear Transformations  1.1.6. Eigenvalues and Eigenvectors  1.1.7. Diagonalization  1.1.8. Inner Product Spaces  1.1.9. Orthogonality and Orthonormal Bases  1.1.10. Singular Value Decomposition  1.2. Abstract Algebra  1.2.1. Groups  1.2.2. Subgroups  1.2.3. Cyclic Groups  1.2.4. Permutation Groups  1.2.5. Homomorphisms  1.2.6. Normal Subgroups  1.2.7. Rings  1.2.8. Fields  1.2.9. Ideals and Quotient Rings  1.2.10. Polynomial Rings  1.2.11. Vector Spaces over Fields  1.2.12. Modules  1.2.13. Galois Theory
  2. Calculus  2.1. Differential Calculus  2.1.1. Limits  2.1.2. Continuity  2.1.3. Derivatives  2.1.4. Chain Rule  2.1.5. Implicit Differentiation  2.1.6. Applications of Derivatives  2.1.7. Optimization Problems  2.1.8. Mean Value Theorem  2.2. Integral Calculus  2.2.1. Definite Integrals  2.2.2. Indefinite Integrals  2.2.3. Techniques of Integration  2.2.4. Improper Integrals  2.2.5. Applications of Integration  2.2.6. Arc Length and Surface Area  2.3. Multivariable Calculus  2.3.1. Partial Derivatives  2.3.2. Gradient  2.3.3. Divergence and Curl  2.3.4. Multiple Integrals  2.3.5. Line Integrals  2.3.6. Surface Integrals  2.3.7. Green's Theorem  2.3.8. Stokes' Theorem  2.3.9. Divergence Theorem  2.3.10. Jacobians
  3. Differential Equations  3.1. Ordinary Differential Equations  3.1.1. First Order Differential Equations  3.1.2. Higher Order Differential Equations  3.1.3. Systems of Differential Equations  3.1.4. Series Solutions  3.1.5. Numerical Methods for ODEs  3.2. Partial Differential Equations  3.2.1. Heat Equation  3.2.2. Wave Equation  3.2.3. Laplace Equation  3.2.4. Separation of Variables  3.2.5. Fourier Transform Methods
  4. Real Analysis  4.1. Sequences and Series  4.1.1. Convergence of Sequences  4.1.2. Series Tests  4.1.3. Power Series  4.1.4. Uniform Convergence  4.2. Functions of Real Variables  4.2.1. Limits and Continuity  4.2.2. Uniform Continuity  4.2.3. Differentiation in Functions of Real Variables  4.2.4. Riemann Integration  4.2.5. Lebesgue Integration
  5. Complex Analysis  5.1. Complex Numbers  5.1.1. Polar Form  5.1.2. Exponential Form  5.2. Functions of a Complex Variable  5.2.1. Analytic Functions  5.2.2. Cauchy-Riemann Equations  5.2.3. Contour Integrals  5.2.4. Cauchy's Integral Theorem  5.2.5. Residue Theorem  5.2.6. Conformal Mapping
  6. Probability and Statistics  6.1. Probability Theory  6.1.1. Basic Probability Concepts  6.1.2. Random Variables  6.1.3. Probability Distributions  6.1.4. Expectation and Variance  6.1.5. Conditional Probability and Bayes' Theorem  6.2. Statistics  6.2.1. Descriptive Statistics  6.2.2. Inferential Statistics  6.2.3. Hypothesis Testing  6.2.4. Regression Analysis  6.2.5. ANOVA
  7. Numerical Methods  7.1. Root-Finding Algorithms  7.2. Numerical Integration  7.3. Numerical Differentiation  7.4. Numerical Solutions to Differential Equations  7.5. Matrix Computations  7.6. Interpolation and Extrapolation
  8. Discrete Mathematics  8.1. Logic and Proof  8.2. Combinatorics  8.3. Graph Theory  8.4. Boolean Algebra  8.5. Recurrence Relations
  9. Linear Programming and Optimization  9.1. Simplex Method  9.2. Duality  9.3. Integer Programming  9.4. Nonlinear Optimization
  10. Topology  10.1. Metric Spaces  10.2. Topological Spaces  10.3. Continuity and Homeomorphisms  10.4. Compactness  10.5. Connectedness
  11. Geometry  11.1. Euclidean Geometry  11.2. Differential Geometry  11.3. Non-Euclidean Geometry  11.4. Projective Geometry
  12. Mathematical Physics  12.1. Fourier Series  12.2. Laplace Transforms  12.3. Applications of Differential Equations  12.4. Special Functions
  13. Set Theory and Logic  13.1. Sets and Subsets  13.2. Relations and Functions  13.3. Cardinality  13.4. Formal Logic  13.5. Zermelo-Fraenkel Set Theory
  14. Functional Analysis  14.1. Normed Spaces  14.2. Banach Spaces  14.3. Hilbert Spaces  14.4. Linear Operators

Undergraduate → Algebra → Linear Algebra ↓


Orthogonality and Orthonormal Bases


In linear algebra, "orthogonality" is a fundamental concept that deals with the perpendicularity of vectors. It carries the idea of perpendicular lines from geometry to vector spaces. The dot product of orthogonal vectors is zero, meaning that they are 'independent' directions. Understanding orthogonality is important in a variety of mathematical tasks, including simplifying vector calculations, and has applications in computer science, physics, and engineering.

Orthogonality

To understand orthogonality, we first need to explore the dot product (also called the inner product). For vectors in R n, the dot product is a way to create a scalar (a single number) by multiplying two vectors. If you have two vectors u and v in R n, the dot product is defined as

    u • v = u 1 v 1 + u 2 v 2 + ... + u n v n

where u i and v i are the components of the vectors u and v, respectively. Two vectors are called orthogonal if their dot product is zero:

    u • v = 0

Visual example:

In the visualization above, the red and blue lines represent two orthogonal vectors. They meet at right angles at the origin, which is represented by the green dot.

Properties of orthogonal vectors

  • Two non-zero vectors in a two-dimensional plane are orthogonal if they are at a 90 degree angle to each other.
  • Orthogonal vectors in higher dimensions still have the property that their dot product is zero.
  • If a set of vectors is mutually perpendicular (each pair of vectors in the set is perpendicular), then this set is called a perpendicular set.

Orthonormal vectors and bases

While orthogonal vectors are vectors that are at right angles to each other, orthonormal vectors add an additional condition: each vector must have the same length (magnitude). If all vectors in an orthogonal set also have the same length, then it becomes an orthonormal set. The key points are as follows:

  • Orthonormal vectors: A group of vectors is orthonormal if the magnitude of each vector is unity and any pair of vectors in the group are orthogonal.
  • Orthonormal basis: An orthonormal basis for a vector space is a basis that is also an orthonormal set.

Mathematically, for a set of vectors { e 1, e 2, ..., e n } to be orthonormal, the following must hold:

    e i • e j = 
    ,
        1 if i = j (same vectors)
        0 if i ≠ j (different vectors)
    ,

Visual example:

The red and blue lines above represent vectors that form an orthonormal basis for the 2D plane. They are at right angles and each has a magnitude of one.

Why orthonormal bases are useful

Having an orthonormal basis for a vector space is incredibly useful because it simplifies many calculations, such as finding the components of a vector in that space. Let b be the orthonormal basis for a vector space V Then, for any vector v in V, we can express v as follows:

    V = (V • E 1 )E 1 + (V • E 2 )E 2 + ... + (V • E n )E n

This is a powerful result because to find the coordinate of v along the direction of e i, one only needs the dot product v • e i. This simplicity arises because the dot product of v with itself will give one due to normalization.

Construction of an orthonormal basis: the Gram–Schmidt process

One of the most famous procedures for converting any basis into an orthonormal basis is the Gram-Schmidt procedure. It works like this:

  1. Start with an initial set of linearly independent vectors {v 1, v 2, ..., v n}.
  2. To convert the first vector to a unit vector, set e 1 = v 1 / |v 1|
  3. For each subsequent vector v k, make it orthogonal to all previously obtained orthogonal vectors.
  4. Normalize each orthogonal vector to turn it into a unit vector.

Illustration:

This illustration shows how initial vectors (red and blue) are transformed into orthogonal vectors using the Gram–Schmidt process, and ultimately, when all vectors are normalized, an orthonormal basis is formed.

Applications and examples

Orthonormal bases are used extensively in computer graphics, signal processing, and solving differential equations, as well as many other fields. They simplify calculations by allowing vector decomposition and projection in a straightforward manner.

Example calculation:

Consider the vectors u = (1, 0, -1) and v = (1, 2, 1). We want to see if they are orthogonal:

The dot product is calculated as follows:

    u • v = (1 * 1) + (0 * 2) + (-1 * 1) = 0

Since their dot product is zero, the vectors u and v are orthogonal.

Orthonormal bases allow efficient representation and manipulation of vector spaces. By understanding orthogonality and how to construct orthonormal sets, we can discover simple solutions to complex problems.


Undergraduate → 1.1.9


U
username
0%
completed in Undergraduate

← Prev (1.1.8)
Inner Product Spaces
Next (1.1.10) →
Singular Value Decomposition

Comments 



Orthogonality and Orthonormal Bases

https://www.buddymath.com/ug/1.1.9?bmal=en