Data Science & Machine Learning Data Engineering, Deployment and Projects Syllabus

What does the MapReduce programming model consist of?

A Map phase that processes and emits key-value pairs in parallel, and a Reduce phase that aggregates values by key.

What does ETL stand for, and what is its order?

Extract, Transform, Load: extract data from sources, transform/clean it, then load it into a target system.

How does ELT differ from ETL?

In ELT, data is Extracted and Loaded into the target (often a data warehouse/lake) first, then Transformed in place using the target's compute.

What is idempotency in the context of ETL/data pipelines?

The property that running a pipeline step multiple times produces the same result, preventing duplicate or inconsistent data on re-runs.

What is a DAG in workflow orchestration tools like Apache Airflow?

A Directed Acyclic Graph defining tasks and their dependencies, with no cycles, that determines execution order.

What is the purpose of the Transform step in ETL?

To clean, validate, deduplicate, reformat, aggregate, and enrich raw data so it conforms to the target schema and quality requirements.

What is web scraping?

The automated extraction of data from websites by programmatically requesting pages and parsing their HTML content.

Which Python libraries are commonly used together for web scraping?

requests (to fetch pages) and BeautifulSoup (to parse HTML); Scrapy and Selenium are used for larger/dynamic scraping.

What does a REST API HTTP GET request do versus a POST request?

GET retrieves data from a resource without modifying it; POST submits data to create or process a resource on the server.

What does HTTP status code 200 versus 404 indicate?

200 means OK/success; 404 means the requested resource was Not Found.

What is the role of a robots.txt file in web scraping ethics?

It specifies which parts of a site crawlers are allowed or disallowed from accessing; respecting it is part of ethical/legal scraping.

What data format do most modern web APIs return?

JSON (JavaScript Object Notation), a lightweight, human-readable key-value/text format.

What is model serialization in machine learning?

The process of saving a trained model's parameters/structure to disk (a file) so it can be stored, shared, and reloaded for inference later.

What Python module is commonly used to serialize ML models, and what is a key caution?

pickle (or joblib for large numpy arrays). Caution: never unpickle data from untrusted sources, as it can execute arbitrary code.

Why is joblib often preferred over pickle for scikit-learn models?

joblib is more efficient at serializing objects that carry large NumPy arrays, producing faster, smaller dumps.

What is a common cross-language/portable format for serializing ML models?

ONNX (Open Neural Network Exchange), which lets models be exported and run across different frameworks and runtimes.

What is the typical role of a web framework like Flask or FastAPI in serving an ML model?

It exposes the model's predict function via an HTTP endpoint so clients can send input data and receive predictions over the network.

What is a key advantage of FastAPI for building ML APIs?

It is high-performance (async) and provides automatic data validation (via Pydantic) and auto-generated interactive API docs (Swagger/OpenAPI).

What is the typical request/response flow for an ML prediction API endpoint?

Client sends input features (usually JSON) → API validates/preprocesses → model.predict() runs → API returns the prediction as JSON.

Why is containerization (e.g., Docker) commonly used when deploying ML APIs?

It packages the model, code, and dependencies into a portable, reproducible image that runs consistently across environments.

What does MLOps stand for and aim to do?

Machine Learning Operations: applying DevOps-style practices (CI/CD, automation, monitoring) to reliably deploy and maintain ML models in production.

What is model drift (data/concept drift) in production ML?

Degradation of model performance over time because the live data distribution or input-output relationship changes from the training data.

What is a model registry in MLOps?

A central versioned store for trained models, tracking versions, metadata, stage (staging/production), and lineage.

What is the purpose of a feature store in MLOps?

To centrally store, manage, and serve consistent feature data for both model training and online inference, avoiding training-serving skew.

What does CI/CD enable in an MLOps pipeline?

Continuous Integration/Continuous Delivery automates testing, building, and deploying code and models, enabling fast, reliable, repeatable releases.

What are the main stages of an end-to-end ML project lifecycle?

Problem definition → data collection → data cleaning/EDA → feature engineering → model training → evaluation → deployment → monitoring/maintenance.

Why is monitoring a critical final stage of the ML project lifecycle?

Because deployed models degrade over time (drift, data changes); monitoring detects performance drops and triggers retraining.

What is reproducibility in a data science project?

The ability to obtain the same results by re-running the same code on the same data, typically achieved via version control, fixed random seeds, and pinned dependencies.

Name three practices that improve reproducibility in ML projects.

Version controlling code/data (Git, DVC), setting random seeds, and pinning dependency versions (e.g., requirements.txt or environment.yml).

What are key principles of ethical and responsible AI?

Fairness, accountability, transparency/explainability, privacy, and avoiding/mitigating bias in data and model outcomes.

What is algorithmic bias and a common source of it?

Systematic unfair outcomes produced by a model, often caused by biased or unrepresentative training data reflecting historical or sampling inequities.