Data Science & Machine Learning Mathematics and Statistics for Machine Learning Syllabus

Mathematics and Statistics for Machine Learning syllabus — full chapter and topic list

Expand any chapter to see its topics and sub-topics. This is the whole examinable outline for Mathematics and Statistics for Machine Learning in Data Science & Machine Learning, not a summary of it.

1. 1

   Linear Algebra Essentials

   3 topics

   • Vectors and Matrices
     • Vector operations and norms
     • Matrix multiplication
     • Transpose and inverse
   • Eigenvalues and Eigenvectors
     • Geometric interpretation
     • Application in dimensionality reduction
   • Matrix Decomposition
     • Singular Value Decomposition
2. 2

   Calculus for Optimization

   3 topics

   • Derivatives and Gradients
     • Partial derivatives
     • Gradient vector
   • Chain Rule
     • Composite function differentiation
   • Optimization Basics
     • Maxima and minima
     • Convexity
3. 3

   Descriptive Statistics

   4 topics

   • Measures of Central Tendency
     • Mean, median and mode
   • Measures of Dispersion
     • Variance and standard deviation
     • Range and IQR
   • Distribution Shape
     • Skewness and kurtosis
   • Correlation and Covariance
     • Pearson and Spearman correlation
4. 4

   Probability Theory

   4 topics

   • Probability Fundamentals
     • Sample space and events
     • Conditional probability
   • Bayes Theorem
     • Prior and posterior
   • Probability Distributions
     • Normal distribution
     • Binomial and Poisson
     • Uniform distribution
   • Random Variables
     • Expectation and variance
5. 5

   Inferential Statistics

   4 topics

   • Sampling and Estimation
     • Population vs sample
     • Central Limit Theorem
   • Hypothesis Testing
     • Null and alternative hypotheses
     • p-value and significance level
     • Type I and Type II errors
   • Statistical Tests
     • t-test and z-test
     • Chi-square test
     • ANOVA
   • Confidence Intervals
     • Margin of error

Mathematics and Statistics for Machine Learning flashcards for Data Science & Machine Learning

20 of 52 cards from the Mathematics and Statistics for Machine Learning deck — real questions with worked answers.

1. What is the difference between a scalar, a vector, and a matrix?

   A scalar is a single number (a 0-D quantity). A vector is an ordered 1-D array of numbers (magnitude and direction). A matrix is a 2-D rectangular array of numbers arranged in rows and columns.

2. What is the dot (inner) product of two vectors a and b, and what does it equal geometrically?

   a·b = Σ aᵢbᵢ (sum of element-wise products). Geometrically, a·b = ‖a‖‖b‖cosθ, where θ is the angle between them. It is zero when the vectors are orthogonal.

3. For matrix multiplication AB to be defined, what dimension condition must hold, and what is the result's shape?

   The number of columns of A must equal the number of rows of B. If A is m×n and B is n×p, then AB is m×p. Matrix multiplication is not commutative: AB ≠ BA in general.

4. What is the transpose of a matrix and a key property involving products?

   The transpose Aᵀ swaps rows and columns: (Aᵀ)ᵢⱼ = Aⱼᵢ. A key property is (AB)ᵀ = BᵀAᵀ (the transpose of a product reverses the order).

5. What is the identity matrix and the inverse of a matrix?

   The identity matrix I has 1s on the diagonal and 0s elsewhere; AI = IA = A. The inverse A⁻¹ satisfies AA⁻¹ = A⁻¹A = I. A square matrix is invertible only if its determinant is nonzero (non-singular).

6. What does the determinant of a square matrix tell you?

   The determinant is a scalar measuring how the matrix scales volume. A determinant of 0 means the matrix is singular (non-invertible) and its columns are linearly dependent.

7. What does it mean for vectors to be linearly independent?

   A set of vectors is linearly independent if no vector can be written as a linear combination of the others — i.e., the only solution to c₁v₁ + c₂v₂ + ... = 0 is all cᵢ = 0.

8. What defines an eigenvalue λ and eigenvector v of a matrix A?

   They satisfy Av = λv (v ≠ 0). The eigenvector v keeps its direction under transformation by A and is only scaled by the eigenvalue λ.

9. What is the characteristic equation used to find eigenvalues?

   det(A − λI) = 0. Solving this polynomial in λ gives the eigenvalues; substituting each λ back into (A − λI)v = 0 gives the corresponding eigenvectors.

10. How do the trace and determinant relate to eigenvalues?

    The trace (sum of diagonal entries) equals the sum of the eigenvalues, and the determinant equals the product of the eigenvalues.

11. What are the eigenvalues of a symmetric real matrix, and what is special about its eigenvectors?

    A real symmetric matrix has all real eigenvalues, and its eigenvectors corresponding to distinct eigenvalues are orthogonal. Such a matrix is always diagonalizable by an orthogonal matrix.

12. What is eigendecomposition of a matrix A?

    A = PDP⁻¹, where D is a diagonal matrix of eigenvalues and P is a matrix whose columns are the corresponding eigenvectors. It requires A to have a full set of linearly independent eigenvectors.

13. What is Singular Value Decomposition (SVD)?

    Any m×n matrix can be factored as A = UΣVᵀ, where U and V are orthogonal matrices and Σ is a diagonal matrix of non-negative singular values. SVD works for any matrix, not just square ones.

14. How does SVD relate to PCA (Principal Component Analysis)?

    PCA finds directions of maximum variance via the eigenvectors of the covariance matrix; equivalently, applying SVD to the mean-centered data matrix yields these principal components (right singular vectors), with singular values encoding variance.

15. What is the geometric meaning of a derivative f'(x)?

    The derivative is the instantaneous rate of change of f with respect to x — the slope of the tangent line to the curve at that point.

16. What is a gradient ∇f of a multivariable function?

    The gradient is the vector of all partial derivatives, ∇f = [∂f/∂x₁, ∂f/∂x₂, ...]. It points in the direction of steepest ascent, and its magnitude is the rate of that increase.

17. State the chain rule for a single-variable composite function f(g(x)).

    d/dx f(g(x)) = f'(g(x))·g'(x). You multiply the derivative of the outer function (evaluated at the inner) by the derivative of the inner function.

18. Why is the chain rule fundamental to training neural networks?

    Backpropagation applies the chain rule repeatedly to compute gradients of the loss with respect to each weight by propagating derivatives backward through the layers of nested functions.

19. What is a partial derivative?

    A partial derivative ∂f/∂xᵢ measures the rate of change of a multivariable function with respect to one variable while holding all other variables constant.

20. What is the update rule for gradient descent?

    θ_new = θ_old − η·∇f(θ), where η is the learning rate. Parameters move in the direction opposite the gradient to minimize the function.

See more Mathematics and Statistics for Machine Learning flashcards →

Planning Mathematics and Statistics for Machine Learning for Data Science & Machine Learning

Mathematics and Statistics for Machine Learning (Data Science & Machine Learning) FAQ