Machine Learning Mathematics Syllabus

Mathematics syllabus — full chapter and topic list

Expand any chapter to see its topics and sub-topics. This is the whole examinable outline for Mathematics in Machine Learning, not a summary of it.

1. 1

   Linear Algebra

   4 topics

   • Vectors
     • Definition and Properties
     • Vector Operations
     • Dot Product
     • Cross Product
     • Norms
   • Matrices
     • Definition and Properties
     • Matrix Operations
     • Matrix Transpose
     • Matrix Inverse
     • Determinants
   • Eigenvalues and Eigenvectors
     • Definition and Properties
     • Characteristic Equation
     • Diagonalization
     • Applications in Machine Learning
   • Linear Transformations
     • Definition and Properties
     • Matrix Representation
     • Change of Basis
     • Singular Value Decomposition
2. 2

   Calculus

   3 topics

   • Differential Calculus
     • Limits and Continuity
     • Derivatives
     • Partial Derivatives
     • Gradient
     • Chain Rule
   • Integral Calculus
     • Definite and Indefinite Integrals
     • Techniques of Integration
     • Multivariable Integration
     • Applications in Probability and Statistics
   • Optimization
     • Unconstrained Optimization
     • Constrained Optimization
     • Lagrange Multipliers
     • Gradient Descent
     • Stochastic Gradient Descent
3. 3

   Probability and Statistics

   3 topics

   • Probability Theory
     • Probability Spaces
     • Conditional Probability
     • Bayes' Theorem
     • Random Variables
     • Probability Distributions
   • Descriptive Statistics
     • Measures of Central Tendency
     • Measures of Dispersion
     • Skewness and Kurtosis
     • Data Visualization
   • Inferential Statistics
     • Hypothesis Testing
     • Confidence Intervals
     • p-values
     • ANOVA
     • Regression Analysis
4. 4

   Discrete Mathematics

   3 topics

   • Set Theory
     • Sets and Subsets
     • Operations on Sets
     • Venn Diagrams
     • Cartesian Products
   • Graph Theory
     • Graphs and Digraphs
     • Graph Representations
     • Graph Traversal Algorithms
     • Shortest Path Algorithms
     • Applications in Machine Learning
   • Combinatorics
     • Permutations and Combinations
     • Binomial Theorem
     • Pigeonhole Principle
     • Inclusion-Exclusion Principle
5. 5

   Numerical Methods

   4 topics

   • Root Finding Algorithms
     • Bisection Method
     • Newton-Raphson Method
     • Secant Method
   • Linear System Solvers
     • Gaussian Elimination
     • LU Decomposition
     • Jacobi Method
     • Gauss-Seidel Method
   • Numerical Integration
     • Trapezoidal Rule
     • Simpson's Rule
     • Monte Carlo Integration
   • Optimization Algorithms
     • Gradient Descent
     • Newton's Method
     • Conjugate Gradient Method

Mathematics flashcards for Machine Learning

21 of 59 cards from the Mathematics deck — real questions with worked answers.

1. What is the dot product of two vectors $\vec{a}$ and $\vec{b}$ in $\mathbb{R}^n$, and what does it equal geometrically?

   $\vec{a} \cdot \vec{b} = \sum_{i=1}^{n} a_i b_i = \|\vec{a}\|\,\|\vec{b}\|\cos\theta$, where $\theta$ is the angle between the vectors.

2. How is the Euclidean ($L^2$) norm of a vector $\vec{v} = (v_1, \dots, v_n)$ defined?

   $\|\vec{v}\|_2 = \sqrt{\sum_{i=1}^{n} v_i^{2}} = \sqrt{\vec{v}\cdot\vec{v}}$.

3. When are two nonzero vectors orthogonal?

   When their dot product is zero: $\vec{a}\cdot\vec{b} = 0$, which corresponds to an angle of $90^\circ$ between them.

4. What is the cosine similarity between two vectors $\vec{a}$ and $\vec{b}$?

   $\cos\theta = \dfrac{\vec{a}\cdot\vec{b}}{\|\vec{a}\|\,\|\vec{b}\|}$, ranging from $-1$ to $1$.

5. How do you compute the projection of vector $\vec{a}$ onto vector $\vec{b}$?

   $\text{proj}_{\vec{b}}\vec{a} = \dfrac{\vec{a}\cdot\vec{b}}{\|\vec{b}\|^{2}}\,\vec{b}$.

6. For a $2\times 2$ matrix $A = \begin{pmatrix} a & b \\ c & d \end{pmatrix}$, what is its determinant?

   $\det(A) = ad - bc$.

7. What is the inverse of a $2\times 2$ matrix $A = \begin{pmatrix} a & b \\ c & d \end{pmatrix}$ when $\det(A) \neq 0$?

   $A^{-1} = \dfrac{1}{ad-bc}\begin{pmatrix} d & -b \\ -c & a \end{pmatrix}$.

8. State the rule for transposing a product of matrices: $(AB)^{T} = ?$

   $(AB)^{T} = B^{T}A^{T}$ (the order reverses).

9. What condition must hold for matrices $A$ and $B$ to be multipliable as $AB$, and what is the resulting shape?

   If $A$ is $m\times n$ and $B$ is $n\times p$ (inner dimensions match), then $AB$ is $m\times p$.

10. What does it mean for a square matrix to be singular?

    It is non-invertible; equivalently $\det(A) = 0$, its columns are linearly dependent, and its rank is less than full.

11. Define the rank of a matrix.

    The dimension of the column space (equal to the dimension of the row space) — the maximum number of linearly independent columns (or rows).

12. What is an identity matrix and its defining property?

    $I_n$ is the $n\times n$ matrix with $1$s on the diagonal and $0$s elsewhere, satisfying $AI = IA = A$ for any compatible $A$.

13. What equation defines an eigenvalue $\lambda$ and eigenvector $\vec{v}$ of matrix $A$?

    $A\vec{v} = \lambda\vec{v}$ with $\vec{v} \neq \vec{0}$.

14. How do you find the eigenvalues of a square matrix $A$?

    Solve the characteristic equation $\det(A - \lambda I) = 0$ for $\lambda$.

15. What is the relationship between the trace of a matrix and its eigenvalues?

    The trace (sum of diagonal entries) equals the sum of the eigenvalues: $\operatorname{tr}(A) = \sum_i \lambda_i$.

16. What is the relationship between the determinant of a matrix and its eigenvalues?

    The determinant equals the product of the eigenvalues: $\det(A) = \prod_i \lambda_i$.

17. When is a real symmetric matrix guaranteed to have real eigenvalues and orthogonal eigenvectors?

    Always — by the spectral theorem, every real symmetric matrix has real eigenvalues and a complete set of orthogonal (orthonormal) eigenvectors.

18. What does a linear transformation $T$ preserve, by definition?

    Additivity and scalar homogeneity: $T(\vec{u}+\vec{v}) = T(\vec{u}) + T(\vec{v})$ and $T(c\vec{v}) = cT(\vec{v})$.

19. How is every linear transformation $T: \mathbb{R}^n \to \mathbb{R}^m$ represented?

    By matrix multiplication $T(\vec{x}) = A\vec{x}$ for a unique $m\times n$ matrix $A$ whose columns are $T$ applied to the standard basis vectors.

20. Geometrically, what does $|\det(A)|$ represent for a linear transformation $\vec{x}\mapsto A\vec{x}$?

    The factor by which the transformation scales area (2D) or volume (3D). A negative determinant also indicates orientation reversal.

21. What is the power rule for derivatives: $\frac{d}{dx}x^{n} = ?$

    $\dfrac{d}{dx}x^{n} = n\,x^{n-1}$.

See more Mathematics flashcards →

Planning Mathematics for Machine Learning

Mathematics (Machine Learning) FAQ