Government Operational Research Service (GORS) Assessment Quality Assurance, Standards and the Wider Analytical Context Syllabus

What is the difference between appraisal and evaluation in the Green/Magenta Book framework?

Appraisal (Green Book) is prospective — assessing options before a decision to decide what to do. Evaluation (Magenta Book) is retrospective/ongoing — assessing what actually happened and its impact after or during implementation. Together they form the ROAMEF policy cycle.

What does the ROAMEF policy cycle stand for?

Rationale, Objectives, Appraisal, Monitoring, Evaluation, Feedback — the continuous cycle linking Green Book appraisal and Magenta Book evaluation, where feedback informs future policy.

What is the 'counterfactual' in impact evaluation and why does it matter?

The counterfactual is what would have happened in the absence of the intervention. Impact is the difference between the observed outcome and the counterfactual; without a credible counterfactual you cannot attribute outcomes to the policy.

What is optimism bias in Green Book appraisal?

Optimism bias is the demonstrated, systematic tendency for appraisers to be over-optimistic about costs, durations and benefits of projects. The Green Book requires explicit adjustments to correct for it, based on empirical data from past projects.

How is optimism bias corrected in practice in HM Treasury appraisal?

By applying empirically-derived percentage uplifts to capital and operating costs (and adjusting timescales/benefits) at the start of appraisal, with the uplift reduced as more robust, project-specific evidence and risk management reduce the uncertainty.

What is the distinction between risk and uncertainty in appraisal?

Risk refers to outcomes where probabilities can be quantified (you can attach likelihoods). Uncertainty refers to outcomes where probabilities are unknown or unknowable. Appraisal handles risk via expected values/sensitivity analysis and optimism bias for deeper uncertainty.

What is sensitivity analysis and why is it required in appraisal?

Sensitivity analysis tests how the appraisal result (e.g. NPV) changes when key assumptions or inputs vary. It identifies which assumptions matter most (switching values) and shows the robustness of the conclusion to uncertainty.

What is a 'switching value' in sensitivity analysis?

A switching value is the value an input or assumption would have to take for the decision to change (e.g. for NPV to fall to zero or for the preferred option to change). It highlights how robust the recommendation is.

What is the difference between cost-effectiveness analysis (CEA) and cost-benefit analysis (CBA)?

CBA monetises both costs and benefits to give a net value (NPV/BCR). CEA compares cost per unit of a single non-monetised outcome (e.g. cost per life saved, cost per QALY) and is used when benefits are hard to value in money.

What does 'value for money' (VfM) mean in government analysis, often summarised as the '4 Es'?

VfM is securing the best mix of quality and effectiveness for the least outlay. The 4 Es are Economy (cheap inputs), Efficiency (output per input), Effectiveness (outputs achieving outcomes), and Equity (fair distribution of impacts).

What is a Quality Adjusted Life Year (QALY) and where is it used?

A QALY is a measure of health outcome combining length and quality of life (1 = one year in full health). Cost per QALY is used in health cost-effectiveness analysis (e.g. by NICE) to compare interventions and judge value for money.

What is the Government Analysis Function?

The Analysis Function is the cross-government network bringing together the analytical professions (statisticians, economists, social and operational researchers, etc.) to improve the use and quality of analysis in government. It is led by a Head of the Analysis Function.

Name the main analytical professions that make up the Government Analysis Function.

The Government Statistician Group (GSS), Government Economic Service (GES), Government Operational Research Service (GORS), Government Social Research (GSR), Government Geography, Government Actuary's profession, and Data Scientists — among others.

What is GORS and what do operational researchers do?

GORS is the Government Operational Research Service. Operational researchers use analytical and mathematical/modelling methods (e.g. optimisation, simulation, problem structuring) to help decision-makers understand complex problems and choose between options.

What is a Head of Profession (HoP) in the analytical context?

A Head of Profession is the senior leader responsible within a department (or across government) for a particular analytical profession — setting standards, assuring quality, managing recruitment and development, and championing that profession's analysis.

What is the role of analytical leadership in government?

Analytical leaders set the quality and ethical standards, build analytical capability, ensure analysis informs decisions, champion the value of analysis to senior officials and ministers, and maintain the professionalism and impartiality of the function.

What is the Code of Practice for Statistics and what three pillars does it have?

It is the standard ensuring official statistics serve the public good, regulated by the Office for Statistics Regulation (OSR). Its three pillars are: Trustworthiness, Quality, and Value (often summarised as TQV).

What does 'Trustworthiness' mean in the Code of Practice for Statistics?

Trustworthiness relates to the organisation and people producing statistics — their honesty, integrity, independence from political interference, transparency about methods, and sound governance. It is about confidence in the producers, not just the numbers.

What is the Office for Statistics Regulation (OSR)?

The OSR is the regulatory arm of the UK Statistics Authority. It sets the Code of Practice for Statistics, assesses whether statistics meet it (awarding 'National Statistics'/accredited status), and acts to uphold the public good of statistics.

Why must analysts collaborate with policy and digital/data functions?

Collaboration ensures analysis answers the real policy question, is grounded in operational and data realities, can be delivered using available digital/data infrastructure, and is actually used. Working in silos risks irrelevant, un-implementable or duplicated analysis.

What is meant by 'speaking truth to power' for a government analyst?

It is the duty to give decision-makers honest, objective, evidence-based advice — including uncomfortable findings — without distorting analysis to fit a preferred political conclusion. It underpins analytical integrity and impartiality.

What is analytical impartiality and how does it relate to the Civil Service Code?

Impartiality means analysis is objective, evidence-led and free from political bias, serving the government of the day equally well whatever its politics. It reflects the Civil Service Code values of integrity, honesty, objectivity and impartiality.

What are the four core values of the Civil Service Code?

Integrity, Honesty, Objectivity, and Impartiality. Government analysts must uphold all four — particularly objectivity (basing advice on evidence) and impartiality (serving the government of the day without political bias).

What ethical principles should govern the use of data and models in policymaking?

Transparency about methods and limitations, fairness and avoidance of bias/discrimination, accountability for outputs, respect for privacy and consent, proportionality, and ensuring analysis serves the public good — guided by frameworks like the Data Ethics Framework.

What is the Data Ethics Framework?

It is UK government guidance setting out principles for the appropriate and responsible use of data in the public sector. Its core principles are transparency, accountability, and fairness, supported by clear user need and public benefit.

What is the main UK legislation governing personal data protection, and what supplements it?

The UK GDPR (UK General Data Protection Regulation) is the main framework, supplemented and tailored by the Data Protection Act 2018. Together they govern lawful, fair and transparent processing of personal data.

Name several core data protection principles under UK GDPR relevant to analysts.

Lawfulness, fairness and transparency; purpose limitation; data minimisation; accuracy; storage limitation; integrity and confidentiality (security); and accountability. Analysts should also consider anonymisation/pseudonymisation to reduce risk.

What is information governance and a Data Protection Impact Assessment (DPIA)?

Information governance is the framework of policies and controls for handling information securely, legally and ethically. A DPIA is a required assessment of privacy risks before high-risk data processing, identifying and mitigating risks to individuals' rights.

What is the Public Sector Equality Duty (PSED) and which Act creates it?

The PSED, under the Equality Act 2010 (section 149), requires public bodies to have due regard to eliminating discrimination, advancing equality of opportunity, and fostering good relations between people who share a protected characteristic and those who do not.

How should equality, diversity and inclusion (EDI) shape analysis itself?

Analysts should disaggregate data by relevant groups (e.g. protected characteristics) where appropriate, check for and mitigate bias in data and models, assess differential/distributional impacts of policies, and ensure conclusions do not entrench inequality — supporting Equality Impact Assessments.

What is distributional analysis and why does the Green Book require it?

Distributional analysis examines how the costs and benefits of a policy fall across different groups (e.g. by income, region, age or protected characteristic). The Green Book requires it because aggregate value (NPV) can hide unequal or regressive impacts that matter for fair decisions.