Machine Learning Project: Predicting House Prices Syllabus

What is feature selection, and name one technique for it.

Choosing the most relevant subset of features to reduce noise and overfitting. Techniques include correlation filtering, recursive feature elimination (RFE), or regularization (Lasso) that shrinks weak coefficients to zero.

What is Data Transformation and how does it differ from data cleaning?

Transformation changes the scale, distribution, or representation of data (scaling, normalizing, encoding) to suit algorithms. Cleaning fixes errors/missing values; transformation reshapes already-clean data.

Give the formula for min-max normalization and its output range.

$$x' = \frac{x - x_{\min}}{x_{\max} - x_{\min}}$$ It rescales values to the range $[0, 1]$.

Give the formula for standardization (z-score scaling) and describe its result.

$$x' = \frac{x - \mu}{\sigma}$$ It produces features with mean $0$ and standard deviation $1$.

Why is a log transformation often applied to the house-price target variable?

Prices are right-skewed; applying $y' = \log(y)$ compresses large values, makes the distribution more normal, stabilizes variance, and helps linear models fit better.

Which types of algorithms require feature scaling and which generally do not?

Distance/gradient-based models (KNN, SVM, linear/logistic regression, neural networks) require scaling. Tree-based models (decision trees, random forests, gradient boosting) generally do not, since they split on thresholds.

In 'Choosing Algorithms', name three regression algorithms suitable for predicting house prices.

Linear regression, decision tree regression, and ensemble methods such as random forest or gradient boosting (e.g., XGBoost). Also K-nearest neighbors and support vector regression.

State the linear regression model equation for house-price prediction.

$$\hat{y} = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \cdots + \beta_n x_n$$ where $\hat{y}$ is predicted price, $x_i$ are features, and $\beta_i$ are learned coefficients.

Compare linear regression and random forest for house prices in terms of interpretability and nonlinearity.

Linear regression is highly interpretable but assumes linear relationships. Random forest captures nonlinear interactions and is more accurate on complex data, but is less interpretable (a 'black box').

What is the difference between bagging and boosting ensemble methods?

Bagging (e.g., random forest) trains models in parallel on bootstrap samples and averages them to reduce variance. Boosting (e.g., gradient boosting) trains models sequentially, each correcting the previous one's errors, reducing bias.

What happens during the 'Model Training' step?

The algorithm learns parameters by fitting the training data — minimizing a loss function so predictions approximate true prices, iteratively adjusting weights/coefficients.

What is a loss (cost) function, and give the Mean Squared Error formula used in regression training.

A function measuring prediction error that the model minimizes. MSE is $$\text{MSE} = \frac{1}{n}\sum_{i=1}^{n}(y_i - \hat{y}_i)^2.$$

Explain the purpose of splitting data into training, validation, and test sets.

Training set fits the model, validation set tunes hyperparameters and selects models, and the test set gives an unbiased final estimate of generalization on unseen data.

What is k-fold cross-validation and why is it used during training?

Data is split into $k$ folds; the model trains on $k-1$ folds and validates on the remaining one, rotating $k$ times and averaging results. It gives a robust performance estimate and reduces dependence on a single split.

Distinguish overfitting from underfitting in a house-price model.

Overfitting: model learns noise, low training error but high test error (high variance). Underfitting: model too simple, high error on both training and test sets (high bias).

What is the difference between a model parameter and a hyperparameter?

Parameters (e.g., regression coefficients $\beta_i$) are learned from data during training. Hyperparameters (e.g., tree depth, learning rate) are set before training and control how learning proceeds.

In 'Assessing Model Performance', give the Mean Absolute Error (MAE) formula.

$$\text{MAE} = \frac{1}{n}\sum_{i=1}^{n}|y_i - \hat{y}_i|$$

Give the Root Mean Squared Error (RMSE) formula and explain why it is popular for house prices.

$$\text{RMSE} = \sqrt{\frac{1}{n}\sum_{i=1}^{n}(y_i - \hat{y}_i)^2}$$ It is in the same units as price (dollars) and penalizes large errors more heavily than MAE.

How does RMSE differ from MAE in sensitivity to outliers?

RMSE squares errors, so large errors dominate, making it more sensitive to outliers. MAE weights all errors linearly and is more robust to outliers.

Give the formula and interpretation of the coefficient of determination $R^2$.

$$R^2 = 1 - \frac{\sum_i (y_i - \hat{y}_i)^2}{\sum_i (y_i - \bar{y})^2}$$ It is the proportion of variance in price explained by the model; $1$ is perfect, $0$ means no better than predicting the mean.

What does an $R^2$ value of 0.85 mean for a house-price model?

The model explains 85% of the variance in house prices; the remaining 15% is unexplained by the model's features.

What is Mean Absolute Percentage Error (MAPE) and why is it useful for prices?

$$\text{MAPE} = \frac{100\%}{n}\sum_{i=1}^{n}\left|\frac{y_i - \hat{y}_i}{y_i}\right|$$ It expresses error as a percentage, making accuracy interpretable across different price ranges.

In 'Comparing Models', what makes a fair comparison between two candidate models?

Evaluating all models on the same test/validation data with the same metric(s) (e.g., RMSE, $R^2$), ideally via cross-validation, so differences reflect model quality not data luck.

When two models have similar accuracy, what other factors guide model selection?

Interpretability, training/inference speed, scalability, robustness, maintenance cost, and complexity. The simplest model that meets requirements is usually preferred (parsimony / Occam's razor).

What is the bias-variance tradeoff and how does it inform model comparison?

Simple models have high bias, low variance (underfit); complex models have low bias, high variance (overfit). The best model balances both to minimize total expected error on unseen data.

What does 'Preparing for Deployment' involve before a model goes live?

Finalizing/retraining the model on full data, serializing it (saving weights), packaging dependencies, building an API or pipeline, validating on production-like data, and setting up the serving environment.

What is model serialization and name a common format/tool.

Saving a trained model to a file so it can be reloaded without retraining. Common methods include Python pickle, joblib, or framework formats like ONNX, TensorFlow SavedModel, or PMML.

Differentiate batch (offline) deployment from real-time (online) deployment.

Batch deployment scores many records on a schedule (e.g., nightly price estimates). Real-time deployment serves predictions on demand via an API as requests arrive (e.g., instant price quote).

What is model/data drift, and why must you 'Monitor Model Performance' after deployment?

Drift is the change in data distribution (data drift) or input-output relationship (concept drift) over time, e.g., a changing housing market. Monitoring detects degrading accuracy so the model can be corrected before it harms decisions.

When should you 'Retrain the Model', and what triggers it?

Retrain when monitoring shows performance has dropped below a threshold, when significant drift is detected, or on a regular schedule as new sale data accumulates — restoring accuracy with up-to-date data.