UI/UX Design Testing, Metrics & Professional Practice Syllabus

What two dimensions are most commonly used to prioritize usability fixes?

Impact (severity of the problem and how many users it affects, especially on critical tasks) versus effort (development cost to fix). High-impact, low-effort fixes — 'quick wins' — are done first.

In an impact–effort prioritization matrix, how are the four quadrants typically treated?

High impact / low effort = quick wins, do first; high impact / high effort = major projects, plan deliberately; low impact / low effort = fill-ins, do when convenient; low impact / high effort = avoid or drop.

Why is iterative testing (test–fix–retest) recommended over one large usability test?

Fixes can introduce new problems, and each round of a small test (about 5 users) uncovers most remaining major issues cheaply; repeated small cycles progressively improve the design more reliably than a single large study.

According to Nielsen's formula, what proportion of usability problems is found by $n$ users, and why are 5 users often enough?

Problems found $= 1 - (1 - p)^{n}$, where $p \approx 0.31$ is the average probability that one user encounters a given problem. With $n = 5$, this yields roughly $85\%$ of problems, so 5 users per iteration is highly cost-effective.

What is a UX KPI?

A Key Performance Indicator for user experience — a quantifiable metric tracked over time to measure how well a design meets user and business goals (e.g., task success rate, time on task, error rate, SUS score, conversion rate, retention).

How is task success (completion) rate calculated?

$$\text{Task success rate} = \frac{\text{number of tasks completed successfully}}{\text{number of tasks attempted}} \times 100\%$$

How is user error rate calculated in usability testing?

$$\text{Error rate} = \frac{\text{number of errors made}}{\text{total number of attempts (or error opportunities)}} \times 100\%$$

Distinguish behavioral (performance) UX metrics from attitudinal (self-reported) metrics, with examples.

Behavioral metrics measure what users actually do: task success rate, time on task, error rate, click paths. Attitudinal metrics measure what users say or feel: SUS score, Net Promoter Score, satisfaction ratings, survey responses.

What do the five letters of Google's HEART framework stand for?

Happiness, Engagement, Adoption, Retention, and Task success.

In the HEART framework, what is the Goals–Signals–Metrics process?

For each HEART category you define Goals (what success means), Signals (observable user behaviors or attitudes that indicate progress toward the goal), and Metrics (quantifiable measures of those signals that can be tracked over time).

In the HEART framework, what is the difference between Adoption and Retention?

Adoption measures how many new users start using a product or feature (e.g., new sign-ups, first use of a feature); Retention measures how many existing users keep coming back over time (e.g., percentage of users active again after 30 days).

What is the System Usability Scale (SUS)?

A standardized 10-item questionnaire (created by John Brooke, 1986) that measures perceived usability. Each item is rated on a 5-point Likert scale from 'strongly disagree' (1) to 'strongly agree' (5), and the result is a score from $0$ to $100$.

How is a SUS score calculated from the 10 questionnaire responses?

For odd-numbered (positive) items the contribution is $x_i - 1$; for even-numbered (negative) items it is $5 - x_i$. Sum the ten contributions and multiply by $2.5$: $$SUS = 2.5\left[\sum_{i \in \text{odd}} (x_i - 1) + \sum_{i \in \text{even}} (5 - x_i)\right]$$ giving a score between $0$ and $100$.

What SUS score is considered average, and what is generally considered good?

The benchmark average is $68$ (the 50th percentile). Scores above $68$ are above average; roughly $80$ or higher is considered good-to-excellent (around an 'A' grade), while scores well below $68$ indicate usability problems.

Why do the SUS items alternate between positively and negatively worded statements?

The alternation forces respondents to read each statement carefully and counteracts acquiescence and response bias (mindlessly agreeing or ticking the same column for every item).

What is A/B testing?

A controlled experiment in which live traffic is randomly split between two versions of a design — A (control) and B (variant differing in one change) — and a defined success metric (e.g., conversion rate) is compared to determine which version performs better.

In A/B testing, what does statistical significance mean, and what threshold is commonly used?

It means the observed difference between variants is unlikely to be due to random chance. The common threshold is $p < 0.05$, i.e., a $95\%$ confidence level that the difference is real.

How is conversion rate calculated?

$$\text{Conversion rate} = \frac{\text{number of users who completed the desired action}}{\text{total number of visitors}} \times 100\%$$

Name three common pitfalls that invalidate A/B test results.

(1) Stopping the test early as soon as significance appears ('peeking'); (2) sample sizes too small or test duration too short to capture normal behavior cycles; (3) testing multiple changes at once (or many metrics), making it impossible to attribute the effect to one cause.

What is behavioral analytics in UX?

The collection and analysis of data about what users actually do in a live product — page views, click streams, funnels, drop-off points, session duration, retention — used to identify where and how often problems occur (the 'what', complementing qualitative research's 'why').

Name three types of heatmaps and what each shows.

Click (tap) maps show where users click or tap most; scroll maps show how far down a page users scroll before leaving; move (hover) maps show mouse-movement patterns, which loosely correlate with visual attention.

What is a funnel analysis and what does drop-off indicate?

Funnel analysis tracks the percentage of users who progress through each step of a multi-step flow (e.g., product page → cart → checkout → purchase). A large drop-off at a step signals a point of friction, confusion, or failure worth investigating.

What is microcopy and why does it matter?

The small pieces of interface text — button labels, form hints, error messages, empty states, tooltips, confirmations. It matters because it guides users at decision points, reduces errors and anxiety, and directly affects task completion and conversion.

What are the key qualities of good UI microcopy, especially for button labels?

Clear, concise, specific, and action-oriented: use verbs that describe the outcome ('Save changes', 'Create account') rather than vague labels ('OK', 'Submit'), use users' language instead of jargon, and stay consistent in terminology across the product.

What is the difference between voice and tone in UX writing?

Voice is the product's consistent personality across all content (e.g., friendly, professional); tone is how that voice flexes for context and the user's emotional state — celebratory in a success message, calm and empathetic in an error or billing failure. Voice stays constant; tone adapts.

What is content strategy in UX?

The practice of planning the creation, delivery, and governance of useful, usable content — defining what content users need, in what structure, voice, and format, who owns and maintains it, and how it stays accurate over its lifecycle. Often summarized as 'the right content, to the right people, at the right time.'

What three elements should an effective error message contain?

(1) A plain-language statement of what went wrong, (2) why or in what context it happened (without blaming the user or using error codes/jargon), and (3) constructive guidance on how to fix it or what to do next.

What is helper text in forms, and how does it relate to error prevention?

Short guidance shown with an input field (e.g., format hints like 'Password must be at least 8 characters') that tells users what is expected before they submit. It supports Nielsen's error-prevention heuristic by stopping mistakes from happening rather than only reporting them afterward.

What is Lean UX and what is its core build–measure–learn cycle?

A collaborative, hypothesis-driven approach (from Lean Startup thinking) that minimizes heavy deliverables and documentation. Teams state assumptions as testable hypotheses, build an MVP (minimum viable product), measure real user response, and learn — then iterate: Build $\to$ Measure $\to$ Learn.

How do UX designers typically integrate their work into Agile sprints?

By working 'one sprint ahead' (dual-track or staggered sprints): while developers build the current sprint's stories, designers research, design, and validate items for upcoming sprints, participate in ceremonies (planning, stand-ups, reviews), and continuously test with users so design is a parallel track rather than an upfront phase.