Government Operational Research Service (GORS) Assessment Data Analysis, Modelling and Analytical Tools Syllabus

Define unsupervised learning and name two common task types.

Unsupervised learning finds structure in unlabelled data without a target variable. Common tasks are clustering (grouping similar observations, e.g. k-means) and dimensionality reduction (e.g. PCA).

Describe the k-means clustering algorithm.

1) Choose $k$ and initialise $k$ centroids; 2) assign each point to its nearest centroid (usually by Euclidean distance); 3) recompute each centroid as the mean of its assigned points; 4) repeat steps 2-3 until assignments stop changing. It minimises within-cluster sum of squares.

What is the 'elbow method' for choosing the number of clusters?

Plot the within-cluster sum of squares (inertia) against $k$. As $k$ increases, inertia falls; the 'elbow' point, where the rate of improvement sharply flattens, suggests a good value of $k$.

What does Principal Component Analysis (PCA) do?

PCA is a dimensionality-reduction technique that transforms correlated variables into a smaller set of uncorrelated principal components, ordered so each successive component captures the maximum remaining variance. It is used to compress data and aid visualisation.

Distinguish overfitting from underfitting.

Overfitting: the model learns noise/idiosyncrasies of the training data, giving low training error but poor generalisation (high variance). Underfitting: the model is too simple to capture the underlying pattern, giving high error on both training and test data (high bias).

Explain the bias-variance tradeoff.

Expected prediction error decomposes into bias (error from overly simple assumptions), variance (sensitivity to training-data fluctuations), and irreducible error. Reducing bias tends to raise variance and vice versa; the goal is the model complexity that minimises total error.

What is k-fold cross-validation?

The data is split into $k$ equal folds. The model is trained on $k-1$ folds and validated on the held-out fold, repeated so each fold is used once for validation. The $k$ scores are averaged to give a more robust performance estimate than a single train/test split.

Define the confusion matrix terms TP, FP, TN, FN.

True Positive: correctly predicted positive. False Positive: predicted positive but actually negative (Type I error). True Negative: correctly predicted negative. False Negative: predicted negative but actually positive (Type II error).

Give the formulas for accuracy, precision, recall and F1 score.

$\text{Accuracy} = \frac{TP+TN}{TP+TN+FP+FN}$; $\text{Precision} = \frac{TP}{TP+FP}$; $\text{Recall} = \frac{TP}{TP+FN}$; $F_1 = 2\cdot\frac{\text{Precision}\times\text{Recall}}{\text{Precision}+\text{Recall}}$.

What does the ROC curve plot, and what does AUC measure?

The ROC curve plots true positive rate (recall) against false positive rate across all classification thresholds. AUC (area under the curve) summarises overall separability: 0.5 is no better than random, 1.0 is perfect.

Why is accuracy a poor metric for imbalanced classes?

With a rare positive class, a model predicting the majority class for everything can score very high accuracy while never detecting the cases of interest. Precision, recall, F1, or AUC give a truer picture of minority-class performance.

Compare R^2 and RMSE as regression evaluation metrics.

$R^{2} = 1 - \frac{\sum(y_i-\hat{y}_i)^{2}}{\sum(y_i-\bar{y})^{2}}$ is the proportion of variance explained (unitless, higher is better). RMSE $=\sqrt{\frac{1}{n}\sum(y_i-\hat{y}_i)^{2}}$ is the typical error magnitude in the target's own units (lower is better).

What is regularisation and how do L1 and L2 differ?

Regularisation adds a penalty on coefficient size to the loss to reduce overfitting. L1 (Lasso) penalises $\sum|\beta_j|$ and can shrink coefficients to exactly zero (feature selection). L2 (Ridge) penalises $\sum \beta_j^{2}$ and shrinks coefficients smoothly without eliminating them.

Distinguish interpretable models from black-box models, with the role of explainability.

Interpretable (glass-box) models, e.g. linear/logistic regression and shallow decision trees, expose how inputs drive predictions. Black-box models, e.g. deep neural networks and large ensembles, are accurate but opaque. Explainability techniques (e.g. SHAP, LIME) approximate why a black-box made a decision.

What is SHAP and what does it provide?

SHAP (SHapley Additive exPlanations) uses cooperative game theory to assign each feature a contribution value for an individual prediction, fairly distributing the difference from the baseline. It gives consistent local and global feature-importance explanations.

Name the UK government framework for ethical AI use and its core themes.

The Data Ethics Framework plus guidance such as 'Understanding artificial intelligence ethics and safety' (Alan Turing Institute) and the AI regulation principles: safety/security/robustness, appropriate transparency and explainability, fairness, accountability and governance, and contestability and redress.

What is algorithmic transparency in the UK public sector, and what tool supports it?

It means disclosing how algorithmic tools are used in decisions affecting the public. The Algorithmic Transparency Recording Standard (ATRS) provides a standardised template for departments to publish information about the algorithmic tools they use.

Define a Reproducible Analytical Pipeline (RAP).

A RAP is an approach that replaces manual, error-prone analytical processes with automated, code-based pipelines that are reproducible, auditable, well-documented and version-controlled, typically using open-source tools (R or Python) and Git.

List the core principles/components of RAP.

Version control (Git), open-source code (R/Python), modular functions, automated testing, peer review of code, dependency management, documentation, and end-to-end automation from raw data to output. These deliver reproducibility, transparency, auditability and efficiency.

What problems with traditional spreadsheet-based analysis does RAP address?

Manual copy-paste and point-and-click steps that are hard to reproduce, error-prone, undocumented, not version-controlled, and difficult to audit or quality-assure. RAP makes the process code-driven, repeatable and reviewable.

Compare R and Python for government analytical work.

R excels at statistics, official-statistics workflows, and visualisation (tidyverse, ggplot2, Shiny). Python is a general-purpose language strong in data wrangling (pandas), machine learning (scikit-learn), and production/automation. Both are open-source and endorsed for RAP.

In R's tidyverse, what do the core dplyr 'verbs' do?

filter() selects rows by condition; select() picks columns; mutate() creates/modifies columns; arrange() sorts rows; summarise() aggregates; and group_by() defines groups for grouped operations. The pipe (%>% or |>) chains them.

What is the equivalent of R's data.frame in Python's pandas, and a key wrangling operation?

The pandas DataFrame is the equivalent tabular structure. Common operations include groupby() for split-apply-combine aggregation, merge()/join for combining tables, and pivot_table() for reshaping.

Name three common Excel lookup/reference functions and what they do.

VLOOKUP/XLOOKUP retrieve a value from a table by matching a key; INDEX returns a value at a given row/column position; MATCH returns the position of a lookup value. INDEX-MATCH (or XLOOKUP) is more flexible than VLOOKUP.

What is the difference between a relative and an absolute cell reference in Excel?

A relative reference (e.g. A1) adjusts when the formula is copied to another cell. An absolute reference (e.g. $A$1) stays fixed. Mixed references (A$1 or $A1) lock only the row or only the column.

What is the UK government guidance on good spreadsheet practice?

The 'Good practice guidance for the development of spreadsheets' / spreadsheet QA guidance (linked to the Aqua Book) recommends separating inputs, calculations and outputs, documenting assumptions, avoiding hard-coded values in formulas, using version control, and proportionate testing to reduce model error.

Name specialist OR and simulation software/techniques used in government analysis.

Discrete-event simulation tools (e.g. Simul8, Arena, AnyLogic), system dynamics tools (e.g. Vensim, Stella), optimisation/LP solvers (e.g. CPLEX, Gurobi, open-source via Python PuLP/R), and Monte Carlo simulation. SimPy is a Python discrete-event library.

Distinguish discrete-event simulation from system dynamics.

Discrete-event simulation models individual entities flowing through a system as discrete events over time (e.g. patients through a clinic, queueing) and is bottom-up/stochastic. System dynamics models aggregate stocks and flows with feedback loops at a high level, and is typically continuous and deterministic.

What is Monte Carlo simulation?

A technique that repeatedly samples random values from input probability distributions, runs the model for each draw, and aggregates the outputs to estimate the distribution of results, capturing uncertainty and risk rather than a single point estimate.

State Edward Tufte's principles of graphical excellence / data-ink ratio.

Maximise the data-ink ratio: $\text{data-ink ratio} = \frac{\text{ink used to show data}}{\text{total ink}}$. Show the data, avoid 'chartjunk' and unnecessary decoration, maximise data density, and tell the truth about the data (avoid distortion).

How should you choose a chart type for the data relationship you want to show?

Comparison among categories: bar chart. Trend over time: line chart. Part-to-whole: stacked bar or (sparingly) pie. Distribution: histogram/boxplot. Relationship between two variables: scatter plot. Match the encoding to the message and the data type.

What is Tufte's 'lie factor' and how is it computed?

$\text{Lie factor} = \frac{\text{size of effect shown in graphic}}{\text{size of effect in the data}}$. A value of 1 is honest; values far from 1 indicate the graphic distorts the data (e.g. truncated axes, 3D effects, varying area to show a 1D quantity).

List good-practice rules for accessible, honest charts in government.

Start bar-chart axes at zero, label axes and units clearly, use colour-blind-safe palettes and not colour alone, provide alt text/data tables, keep it simple (high data-ink), order categories meaningfully, and follow the Government Statistical Service (GSS) data visualisation guidance.

What distinguishes a dashboard from a static report?

A dashboard presents key metrics and visuals in a single consolidated, often interactive and regularly refreshed view, letting users filter and drill down. A static report is a fixed snapshot. Dashboards suit ongoing monitoring; reports suit one-off communication of a finished analysis.

Name common tools for building dashboards and interactive reporting in government.

R Shiny, Python Dash/Streamlit, Power BI, Tableau, and Quarto/RMarkdown for interactive HTML. The GOV.UK style and accessibility requirements apply to public-facing dashboards.

What design principles make a dashboard effective?

Lead with the most important KPI, use a clear visual hierarchy and logical layout (e.g. top-left first), keep it uncluttered, ensure consistent and meaningful use of colour, provide context (targets, trends, comparisons), and make it accessible and fast to load.

What is a confidence interval and how is a 95% interval interpreted?

A confidence interval is a range, computed from data, that quantifies estimate uncertainty. A 95% CI means that if the procedure were repeated many times, about 95% of the intervals would contain the true parameter. For a mean: $\bar{x} \pm z_{0.025}\frac{s}{\sqrt{n}}$ (large samples).

What is a prediction interval and how does it differ from a confidence interval?

A confidence interval bounds the uncertainty in an estimated parameter (e.g. the mean). A prediction interval bounds where a single future observation is likely to fall, so it is wider because it also includes individual-level variability, not just estimation uncertainty.

What visual methods communicate uncertainty in charts?

Error bars, shaded confidence bands/fan charts (e.g. Bank of England inflation fan chart), violin/box plots showing distributions, gradient/blurred edges, and showing multiple plausible scenarios. The aim is to avoid presenting a single line as if it were certain.

How should analysts communicate uncertainty verbally to decision-makers?

Use a consistent, calibrated probability language (e.g. the IPCC/PHIA likelihood scale: 'highly likely' ~90-100%, 'likely' ~55-75%, 'unlikely', etc.), state key assumptions and caveats clearly without burying the headline, and quantify ranges rather than implying false precision.

What is the core idea of 'storytelling with data' for decision-makers (per Cole Nussbaumer Knaflic)?

Move from merely showing data to communicating a clear narrative: understand the context and audience, choose an effective visual, eliminate clutter, focus attention with preattentive attributes (e.g. colour, size), and tell a story with a clear 'so what' and call to action.

What are 'preattentive attributes' and why do they matter in data communication?

Preattentive attributes (colour, size, position, orientation, shape, intensity) are visual properties the brain processes almost instantly, before conscious attention. Strategic use of one or two of them directs the audience's eye to the most important point in a chart.

Outline a structure for presenting an analytical finding to senior decision-makers.

Lead with the bottom-line answer/recommendation (BLUF), give the essential supporting evidence and one clear visual, state key assumptions, caveats and uncertainty, then set out implications and options. Tailor depth to the audience and end with a clear call to action.

What is data minimisation and why is it important for government analysts?

Data minimisation (a UK GDPR principle) means collecting and processing only the personal data that is adequate, relevant and limited to what is necessary for the stated purpose. It reduces privacy risk, breach impact, and the chance of unlawful or unfair processing.