Data Science Deep Learning Syllabus

Deep Learning syllabus — full chapter and topic list

Expand any chapter to see its topics and sub-topics. This is the whole examinable outline for Deep Learning in Data Science, not a summary of it.

1. 1

   Introduction to Deep Learning

   2 topics

   • What is Deep Learning?
     • Definition and Overview
     • History and Evolution
     • Applications in Data Science
   • Neural Networks Basics
     • Perceptron
     • Activation Functions
     • Feedforward Neural Networks
2. 2

   Mathematical Foundations

   4 topics

   • Linear Algebra
     • Vectors and Matrices
     • Matrix Multiplication
     • Eigenvalues and Eigenvectors
   • Calculus
     • Derivatives and Integrals
     • Partial Derivatives
     • Chain Rule
   • Probability and Statistics
     • Probability Distributions
     • Bayes' Theorem
     • Statistical Measures
   • Optimization Techniques
     • Gradient Descent
     • Stochastic Gradient Descent
     • Advanced Optimization Algorithms
3. 3

   Deep Learning Frameworks

   3 topics

   • TensorFlow
     • Introduction to TensorFlow
     • Building Neural Networks with TensorFlow
     • TensorFlow APIs
   • PyTorch
     • Introduction to PyTorch
     • Building Neural Networks with PyTorch
     • PyTorch APIs
   • Keras
     • Introduction to Keras
     • Building Neural Networks with Keras
     • Keras APIs
4. 4

   Types of Neural Networks

   4 topics

   • Convolutional Neural Networks (CNNs)
     • Convolution Operation
     • Pooling Layers
     • CNN Architectures
   • Recurrent Neural Networks (RNNs)
     • Recurrent Layers
     • LSTM and GRU
     • Applications of RNNs
   • Generative Adversarial Networks (GANs)
     • GAN Architecture
     • Training GANs
     • Applications of GANs
   • Autoencoders
     • Encoder-Decoder Architecture
     • Variational Autoencoders
     • Applications of Autoencoders
5. 5

   Advanced sub-topics in Deep Learning

   4 topics

   • Transfer Learning
     • Concept of Transfer Learning
     • Pre-trained Models
     • Fine-Tuning
   • Deep Reinforcement Learning
     • Introduction to Reinforcement Learning
     • Deep Q-Learning
     • Policy Gradient Methods
   • Natural Language Processing (NLP)
     • Word Embeddings
     • Sequence-to-Sequence Models
     • Attention Mechanisms
   • Computer Vision
     • Image Classification
     • Object Detection
     • Image Segmentation
6. 6

   Practical Implementation

   4 topics

   • Data Preprocessing
     • Data Cleaning
     • Data Augmentation
     • Feature Scaling
   • Model Evaluation
     • Loss Functions
     • Evaluation Metrics
     • Cross-Validation
   • Hyperparameter Tuning
     • Grid Search
     • Random Search
     • Bayesian Optimization
   • Deployment
     • Model Serialization
     • Serving Models
     • Monitoring and Maintenance

Deep Learning flashcards for Data Science

18 of 52 cards from the Deep Learning deck — real questions with worked answers.

1. What is Deep Learning, and how does it relate to machine learning?

   Deep Learning is a subfield of machine learning that uses artificial neural networks with multiple hidden layers (deep architectures) to automatically learn hierarchical feature representations from raw data, removing the need for manual feature engineering.

2. What distinguishes a "deep" neural network from a "shallow" one?

   A deep network has two or more hidden layers between input and output, allowing it to learn hierarchical, increasingly abstract features; a shallow network typically has zero or one hidden layer.

3. What is an artificial neuron (perceptron), and what computation does it perform?

   A unit that computes a weighted sum of its inputs plus a bias, then applies an activation function: $y = f\left(\sum_{i} w_i x_i + b\right)$.

4. State the ReLU activation function and one advantage it has over sigmoid.

   $\mathrm{ReLU}(x) = \max(0, x)$. It mitigates the vanishing-gradient problem and is computationally cheap because its derivative is 0 or 1.

5. Write the sigmoid activation function and its output range.

   $\sigma(x) = \dfrac{1}{1 + e^{-x}}$, with output range $(0, 1)$.

6. Write the softmax function for a vector of logits $z$ and state its purpose.

   $\text{softmax}(z)_i = \dfrac{e^{z_i}}{\sum_{j} e^{z_j}}$. It converts logits into a probability distribution over classes that sums to 1, used in multi-class output layers.

7. What is the tanh activation function and its range?

   $\tanh(x) = \dfrac{e^{x} - e^{-x}}{e^{x} + e^{-x}}$, with output range $(-1, 1)$; it is zero-centered, unlike sigmoid.

8. What is backpropagation?

   An algorithm that computes gradients of the loss with respect to every network weight by applying the chain rule of calculus backward through the layers, enabling gradient-based optimization.

9. What is a dot product of two vectors $\vec{a}$ and $\vec{b}$, and why is it central to neural networks?

   $\vec{a} \cdot \vec{b} = \sum_{i} a_i b_i$. Each neuron's pre-activation is the dot product of its weight vector and the input vector, making it the core operation in forward propagation.

10. What are the dimensions of the product of an $m \times n$ matrix and an $n \times p$ matrix?

    $m \times p$. Matrix multiplication requires the inner dimensions ($n$) to match.

11. What is the identity matrix and what property does it have?

    A square matrix $I$ with 1s on the main diagonal and 0s elsewhere; for any compatible matrix $A$, $AI = IA = A$.

12. Define the transpose of a matrix $A$.

    $A^{T}$ is the matrix obtained by swapping rows and columns, so $(A^{T})_{ij} = A_{ji}$.

13. What are eigenvalues and eigenvectors of a matrix $A$?

    A nonzero vector $\vec{v}$ is an eigenvector with eigenvalue $\lambda$ if $A\vec{v} = \lambda \vec{v}$; the matrix scales the vector without changing its direction.

14. What is the derivative, and what does it represent geometrically?

    $f'(x) = \lim_{h \to 0} \dfrac{f(x+h) - f(x)}{h}$; it represents the instantaneous rate of change, i.e., the slope of the tangent line to the curve at $x$.

15. State the chain rule for $\frac{d}{dx} f(g(x))$.

    $\dfrac{d}{dx} f(g(x)) = f'(g(x)) \cdot g'(x)$. It is the foundation of backpropagation.

16. What is a gradient $\nabla f$, and in which direction does it point?

    The vector of partial derivatives $\nabla f = \left(\dfrac{\partial f}{\partial x_1}, \dots, \dfrac{\partial f}{\partial x_n}\right)$; it points in the direction of steepest increase of $f$.

17. What is a partial derivative $\frac{\partial f}{\partial x}$?

    The derivative of a multivariable function with respect to one variable, treating all other variables as constants.

18. State Bayes' theorem.

    $P(A \mid B) = \dfrac{P(B \mid A)\, P(A)}{P(B)}$.

See more Deep Learning flashcards →

Planning Deep Learning for Data Science

Deep Learning (Data Science) FAQ