A Gentle Introduction to Linear Algebra

Linear algebra is the language machines speak when they learn. Almost every model, from a simple regression to a transformer, is a long chain of matrix products underneath. The good news is that the core ideas are small and visual.

Vectors are arrows

A vector is an ordered list of numbers, but it helps to picture it as an arrow from the origin. In two dimensions, the vector $\vec{v} = \begin{pmatrix} 3 \\ 2 \end{pmatrix}$ points to the coordinate $(3, 2)$.

Its length is the Euclidean norm:

$$\lVert \vec{v} \rVert = \sqrt{v_1^2 + v_2^2 + \dots + v_n^2}$$

The dot product measures alignment

The dot product of two vectors $\vec{a}$ and $\vec{b}$ is a single number:

$$\vec{a} \cdot \vec{b} = \lVert \vec{a} \rVert \lVert \vec{b} \rVert \cos \theta$$

When the vectors point the same way, the dot product is large and positive. When they are perpendicular, it is zero. When they oppose each other, it is negative. A remarkable amount of "similarity" in software — search ranking, recommendations, embedding distance — is just this one idea wearing different clothes.

Matrices transform space

A matrix is a function that takes a vector and returns another vector. Multiplying a vector by a matrix rotates, scales, or shears the space it lives in. A $2 \times 2$ matrix looks like:

$$A = \begin{bmatrix} a_{11} & a_{12} \\ a_{21} & a_{22} \end{bmatrix}$$

Applying it to a vector $\vec{x}$ gives a new vector:

$$A\vec{x} = \begin{bmatrix} a_{11} & a_{12} \\ a_{21} & a_{22} \end{bmatrix} \begin{pmatrix} x_1 \\ x_2 \end{pmatrix} = \begin{pmatrix} a_{11}x_1 + a_{12}x_2 \\ a_{21}x_1 + a_{22}x_2 \end{pmatrix}$$

Eigenvectors are the directions that don't bend

For most vectors, applying a matrix changes their direction. Eigenvectors are the exceptions: they only get stretched. The stretch factor is the eigenvalue $\lambda$:

$$A\vec{v} = \lambda \vec{v}$$

This is more useful than it sounds. Principal component analysis, PageRank, and the spectral clustering behind a lot of data tooling all reduce to finding eigenvectors.

A worked example

Project a vector $\vec{u}$ onto a direction $\vec{d}$. The projection is:

$$\mathrm{proj}_{\vec{d}} \vec{u} = \frac{\vec{u} \cdot \vec{d}}{\vec{d} \cdot \vec{d}} \vec{d}$$

That single line is the foundation of least-squares regression, which is the foundation of a remarkable amount of statistics.

Where to go next

• Work through the visual series first — intuition before proofs.
• Implement a dot product, then a matrix multiply, in the language you already use.
• Pick one idea above and use it on real data this week.

You do not need to memorize proofs. You need to know what a matrix does to a vector, and the rest follows.

Mathematics rewards patience. Spend an afternoon with a pencil and a $2 \times 2$ matrix, and the abstraction stops being abstract.