Associate of the Society of Actuaries (ASA/FSA) Exam PA — Predictive Analytics Syllabus

What is the purpose of combining (collapsing) factor levels, and when is it appropriate?

Combining sparse or similar levels reduces the number of parameters, improves stability, and avoids overfitting on rare categories. It is appropriate when levels have few observations or statistically/practically similar target behavior, often guided by EDA and domain knowledge.

What is target (mean) encoding and what is its main risk?

Target encoding replaces each category level with a summary statistic of the target for that level (e.g., its mean response). Its main risk is target leakage and overfitting, especially for rare levels; it should be computed using only training data, often with smoothing or cross-validation.

Why are numeric variables transformed, and name common transformations.

Transformations reduce skew, stabilize variance, linearize relationships, and reduce the influence of outliers. Common ones: logarithm, square root, Box-Cox, standardization (z-score), min-max scaling, binning/discretization, and polynomial terms.

Give the standardization (z-score) formula and state when standardization matters.

$$z = \frac{x - \mu}{\sigma}$$ It rescales a variable to mean $0$ and standard deviation $1$. It matters for distance-based methods (k-NN, clustering), regularized regression (lasso/ridge), and PCA, where variables on different scales would otherwise dominate.

When is a log transform $x \to \log(x)$ appropriate, and how is the zero problem handled?

A log transform is appropriate for strictly positive, right-skewed variables (e.g., income, claim size) to compress large values and linearize multiplicative relationships. For zeros, use $\log(x + c)$ with a small constant $c$ (commonly $\log(x+1)$).

What is binning (discretization) and what are its trade-offs?

Binning converts a continuous variable into discrete categories (e.g., age bands). It can capture nonlinearity and reduce outlier influence and is interpretable, but it discards within-bin information, depends on arbitrary cut points, and can reduce predictive power.

What is the purpose of dimensionality reduction in practice?

To reduce the number of predictors while retaining most relevant information — combating the curse of dimensionality, reducing collinearity, lowering computation, mitigating overfitting, and aiding visualization. Common methods include PCA, feature selection, and combining correlated features.

Explain Principal Component Analysis (PCA) and what the principal components represent.

PCA finds orthogonal linear combinations of the (standardized) predictors that successively capture the maximum remaining variance. The first principal component explains the most variance; each subsequent component is uncorrelated with the previous and explains the next-most variance.

How do you decide how many principal components to retain?

Use the cumulative proportion of variance explained (e.g., retain enough components for 80-90%), a scree plot looking for the 'elbow', or the Kaiser rule (keep components with eigenvalue $> 1$ on standardized data).

What is the proportion of variance explained by the $k$-th principal component, in terms of eigenvalues?

$$\text{PVE}_k = \frac{\lambda_k}{\sum_{j=1}^{p} \lambda_j}$$ where $\lambda_k$ is the eigenvalue (variance) of the $k$-th component and $p$ is the number of variables. Cumulative PVE sums these over the retained components.

What is a major interpretability drawback of using PCA components as model predictors?

Principal components are linear mixtures of all original variables, so coefficients on PCs do not correspond to any single business feature. This makes the model hard to explain to stakeholders, conflicting with interpretation-focused goals.

State the three components that define a Generalized Linear Model (GLM).

(1) A random component: the response distribution from the exponential family (e.g., Normal, Poisson, Gamma, Binomial). (2) A systematic component: the linear predictor $\eta = \beta_0 + \sum \beta_j x_j$. (3) A link function $g$ connecting them: $g(\mu) = \eta$, where $\mu = E[Y]$.

For common GLM targets, give the typical distribution and link function used in Exam PA.

Binary/classification: Binomial with logit link. Count data: Poisson with log link. Positive continuous / claim severity: Gamma with log link. Symmetric continuous: Normal with identity link (ordinary linear regression).

Write the logistic (logit) link and the implied probability for a binary GLM.

Link: $$\log\!\left(\frac{p}{1-p}\right) = \beta_0 + \sum_{j} \beta_j x_j = \eta.$$ Inverse gives the probability $$p = \frac{e^{\eta}}{1 + e^{\eta}} = \frac{1}{1 + e^{-\eta}}.$$

How is a coefficient interpreted in a logistic regression GLM?

A one-unit increase in predictor $x_j$ multiplies the odds of the event by $e^{\beta_j}$ (the odds ratio), holding other predictors fixed. $\beta_j$ itself is the additive change in the log-odds.

How is a coefficient interpreted in a GLM with a log link (e.g., Poisson or Gamma)?

With $\log(\mu) = \beta_0 + \sum \beta_j x_j$, a one-unit increase in $x_j$ multiplies the expected response $\mu$ by $e^{\beta_j}$, holding others fixed — i.e., the log link gives a multiplicative effect on the mean.

What is an offset in a GLM and when is it used?

An offset is a predictor with coefficient fixed at $1$ (e.g., $\log(\text{exposure})$) included in the linear predictor. It is used to model rates rather than counts — for example, $\log(\mu) = \log(\text{exposure}) + \beta_0 + \sum \beta_j x_j$ models claims per unit exposure.

Compare ridge (L2) and lasso (L1) regularization for GLMs.

Ridge adds penalty $\lambda \sum \beta_j^{2}$, shrinking coefficients toward zero but never exactly to zero. Lasso adds $\lambda \sum |\beta_j|$, which can shrink coefficients exactly to zero, performing variable selection. The elastic net combines both penalties.

What is an interaction term in a GLM and what does it capture?

An interaction is a product term (e.g., $x_1 x_2$) added to the linear predictor. It captures the effect of one predictor on the response depending on the level of another — i.e., the predictors' effects are not purely additive.

Describe how a decision tree makes predictions and how splits are chosen.

A decision tree recursively partitions the predictor space into regions via binary splits; a new observation follows the splits to a terminal leaf whose prediction is the leaf's mean (regression) or majority/probability (classification). Splits are chosen to maximize purity gain, e.g., minimizing RSS, Gini impurity, or entropy.

Define Gini impurity and entropy as node-impurity measures for classification trees.

For class proportions $p_k$ in a node: Gini $= \sum_{k} p_k(1 - p_k) = 1 - \sum_k p_k^{2}$; Entropy $= -\sum_{k} p_k \log_2 p_k$. Both are minimized (zero) when a node is pure. Splits are chosen to reduce the weighted impurity.

Contrast a single decision tree's strengths and weaknesses.

Strengths: highly interpretable, handles nonlinearity and interactions automatically, needs little preprocessing (no scaling, handles mixed types). Weaknesses: high variance/instability, prone to overfitting, and typically lower predictive accuracy than ensembles.

Compare bagging (random forests) with boosting.

Bagging/random forests build many deep trees in parallel on bootstrap samples (forests also subsample features per split) and average them to reduce variance. Boosting builds trees sequentially, each correcting the previous one's errors, reducing bias. Boosting often achieves higher accuracy but is more prone to overfitting and slower to tune.

What is a random forest's key mechanism for decorrelating its trees?

At each split, the random forest considers only a random subset of $m$ predictors (commonly $m \approx \sqrt{p}$ for classification). This prevents strong predictors from dominating every tree, decorrelating the trees so that averaging reduces variance more effectively than plain bagging.

Name the key hyperparameters of a gradient boosting machine and their effect.

Number of trees (more can overfit), learning rate/shrinkage $\eta$ (smaller is more robust but needs more trees), tree depth/interaction depth (controls complexity), and subsampling fraction. Learning rate and number of trees trade off and are tuned together.

Why is the train/test (and validation) split essential in model validation?

Evaluating a model on the same data used to train it gives an optimistically biased estimate of performance. A held-out test set estimates generalization to unseen data; a separate validation set (or CV) is used for tuning so the test set remains untouched until final assessment.

Explain k-fold cross-validation and its advantage.

Data are split into $k$ equal folds; the model is trained on $k-1$ folds and validated on the held-out fold, rotating so each fold serves once as validation. The $k$ errors are averaged. It uses data efficiently and gives a lower-variance estimate of out-of-sample performance than a single split.

Define the bias-variance trade-off and relate it to model complexity.

Expected test error decomposes into $\text{Bias}^{2} + \text{Variance} + \text{Irreducible error}$. Simple models have high bias, low variance (underfit); complex models have low bias, high variance (overfit). The goal is the complexity that minimizes total test error.

List common metrics for evaluating a classification model.

Accuracy, precision, recall (sensitivity), specificity, F1-score, confusion-matrix counts, the ROC curve with AUC, and log-loss. The choice depends on class balance and the relative cost of false positives vs false negatives.

Define precision and recall and give their formulas.

Precision $= \frac{TP}{TP + FP}$ — of predicted positives, the fraction truly positive. Recall (sensitivity) $= \frac{TP}{TP + FN}$ — of actual positives, the fraction correctly identified. There is generally a trade-off between them, summarized by the F1-score.

What does the ROC curve plot and what does AUC of 0.5 versus 1.0 mean?

The ROC curve plots the true positive rate (sensitivity) against the false positive rate ($1 - $ specificity) across classification thresholds. AUC (area under the curve) of $0.5$ indicates no better than random guessing; $1.0$ indicates perfect discrimination.

Name common metrics for evaluating a regression model.

Mean squared error (MSE), root MSE (RMSE), mean absolute error (MAE), and the coefficient of determination $R^{2}$. For GLMs, deviance and AIC/BIC are also used to compare fit while penalizing complexity.

Give the formulas for AIC and BIC and state their purpose.

$\text{AIC} = -2\ln(\hat{L}) + 2k$ and $\text{BIC} = -2\ln(\hat{L}) + k\ln(n)$, where $\hat{L}$ is the maximized likelihood, $k$ the number of parameters, and $n$ the sample size. Both penalize complexity to compare models; lower is better. BIC penalizes parameters more heavily for large $n$.

How is residual analysis used to validate a GLM?

Residuals (e.g., deviance or Pearson residuals) are plotted against fitted values and predictors to check for systematic patterns, non-constant variance, or outliers. Randomly scattered residuals around zero support the chosen distribution and link; patterns signal missing terms or a poor model form.

What is a variable importance measure and how is it obtained for tree ensembles?

Variable importance ranks predictors by their contribution to the model. For tree ensembles it is commonly the total reduction in impurity (or RSS) attributed to splits on that variable, summed over all trees, or a permutation-based drop in accuracy when the variable is shuffled.

What is a partial dependence plot (PDP) and what does it show?

A PDP shows the marginal effect of one (or two) predictor(s) on the predicted response, averaging out the effects of all other predictors. It helps interpret 'black-box' models like boosted trees by revealing the shape and direction of a predictor's relationship with the target.

Distinguish statistical significance from practical (business) significance when interpreting model output.

Statistical significance (e.g., a small p-value) indicates an effect is unlikely due to chance, but with large samples even tiny effects can be significant. Practical significance asks whether the effect's magnitude is large enough to matter for business decisions. Both should be considered.

How should an actuary tailor communication of model results to a non-technical stakeholder?

Lead with the business answer and recommendation, avoid jargon, use intuitive visuals (charts, examples), translate coefficients into concrete impacts (e.g., effect on premium), state key assumptions and limitations, and quantify uncertainty in plain terms. Reserve technical detail for an appendix or technical audience.

What elements should an effective predictive analytics report or executive summary contain?

The business problem and objective, data sources and key assumptions, the modeling approach and why it was chosen, main findings and their business implications, the recommendation, limitations/caveats, and next steps. Technical methodology supports but does not dominate the narrative.

Why is conveying model uncertainty and limitations important when communicating results?

Stakeholders make decisions based on the model; failing to disclose uncertainty (confidence intervals, validation error), data limitations, and assumptions can lead to overconfidence, poor decisions, and erosion of trust. Transparency about limitations is part of professional responsibility.

What ethical risks arise from using protected or proxy variables in predictive models?

Including protected attributes (race, gender, etc.) — or proxies highly correlated with them — can produce unfair discrimination and legal/regulatory violations, even unintentionally. Models must be checked for disparate impact and proxy discrimination, not just exclusion of the explicit attribute.

Define algorithmic bias and unfairness in a predictive model.

Algorithmic bias occurs when a model systematically produces outcomes that disadvantage particular groups, often because the training data reflects historical bias or unrepresentative sampling. Fairness assessment checks whether error rates or predictions differ unjustifiably across protected groups.

What professional and regulatory considerations apply to an actuary building predictive models?

Adherence to the Actuarial Standards of Practice (ASOPs) and the Code of Professional Conduct, data privacy and consent laws, documentation and reproducibility, transparency with stakeholders, avoiding unfair discrimination, and ensuring the model is appropriate, validated, and used within its intended scope.

Why must data privacy and consent be considered in an actuarial analytics project?

Personal data is often subject to privacy regulations and policyholder expectations. Actuaries must ensure data is collected and used lawfully and with appropriate consent, is secured and anonymized where required, and is not used beyond its intended purpose, to protect individuals and the firm's reputation.

What is overfitting, what are its symptoms, and how is it mitigated?

Overfitting is when a model learns noise specific to the training data, performing well in-sample but poorly out-of-sample. Symptoms: training error far below validation/test error. Mitigations: simpler models, regularization, pruning, cross-validation-guided tuning, more data, and early stopping.

Why is reproducibility and documentation emphasized in the Exam PA workflow?

Documenting data sources, cleaning steps, modeling choices, assumptions, and code enables others to review, audit, reproduce, and maintain the work. It supports professional standards, regulatory scrutiny, knowledge transfer, and defensible decision-making.