Book 10 · Cheat sheet

Testing & Quality

How teams check that code works and keeps working: kinds of tests, test doubles and the tools that catch mistakes early.

01A/B Testing: A/B testing is an experiment that shows two versions of a feature to random groups of real users and measures which one performs better on a chosen metric.
A/B testing compares a control (A) and a variant (B) with real users.
Users are randomly and consistently assigned to one version.
Choose the metric and sample size before the experiment starts.
02Acceptance Testing: Acceptance testing checks whether software meets the requirements agreed with its users or customers, so the team can decide if it is ready to release.
Acceptance testing checks software against agreed user or business requirements.
Automated acceptance tests usually map one test to one acceptance criterion.
User acceptance testing (UAT) is done by real users or customers before sign-off.
03Assertion: An assertion is a statement in code declaring that a condition must be true at that point, stopping the test or program with an error if it is false.
An assertion states a condition that must be true and fails loudly when it isn't.
In tests, assertions compare actual results with expected results.
In production code, assertions document invariants and catch bugs early.
04Behavior-Driven Development: Behavior-driven development is a practice where developers, testers, and business people agree on plain-language examples that become automated tests.
BDD describes features as concrete examples that everyone on the team can read.
Scenarios follow the Given-When-Then pattern, often written in Gherkin.
Step definitions turn scenarios into automated acceptance tests.
05Black-Box Testing: Black-box testing checks software only through its inputs and outputs, against what it is supposed to do, without looking at or relying on the code inside.
Black-box testing checks inputs and outputs against requirements.
Testers don't need to know or read the code.
Equivalence partitioning and boundary value analysis structure it.
06Contract Testing: Contract testing is a technique that checks whether two services agree on the requests and responses they exchange, without running both of them together.
A contract describes the requests a consumer sends and the responses it expects.
The consumer and the provider are each tested against the contract separately.
In consumer-driven contract testing, consumers define the contract and providers verify it.
07Cypress: Cypress is a JavaScript framework that runs end-to-end and component tests for web apps inside the browser, with automatic waiting and an interactive runner.
Cypress is a JavaScript framework for end-to-end and component tests.
It runs tests inside the browser alongside the app.
Commands retry automatically, so manual waits are rarely needed.
08End-to-End Test: An end-to-end test is an automated test that runs a complete user journey through the whole application, from the user interface to the database and back.
E2E tests simulate real user journeys through the full application.
Web E2E tests typically automate a real or headless browser.
They are the slowest and most expensive tests, so keep them for critical flows.
09Exploratory Testing: Exploratory testing is a hands-on approach in which a tester learns about the software, designs tests, and runs them at once, guided by what they discover.
The tester designs and runs tests at the same time, adapting as they learn.
Sessions are timeboxed and guided by a written charter.
Notes on bugs, questions, and risks are shared in a short debrief.
10Flaky Test: A flaky test is an automated test that sometimes passes and sometimes fails without any change to the code, which makes its results hard to trust.
A flaky test passes and fails randomly with no code change.
Common causes are timing issues, shared state, test order, network calls, and dates.
Flaky tests erode trust in the whole test suite.
11Fuzz Testing: Fuzz testing is an automated technique that feeds a program huge numbers of unexpected or malformed inputs to find crashes, hangs, and security vulnerabilities.
Fuzzing feeds a program massive amounts of unexpected or malformed input.
It looks for crashes, hangs, memory errors, and security vulnerabilities.
Coverage-guided fuzzers evolve inputs that reach new code paths.
12Integration Test: An integration test is an automated test that checks whether several parts of a system, such as code, a database, and an API, work correctly together.
Integration tests check that multiple real components work together.
They catch bugs at boundaries, such as database queries, API contracts, and configuration.
They are slower than unit tests because they use real dependencies.
13Jest: Jest is a popular JavaScript testing framework that bundles a test runner, assertions, mocking, snapshots and code coverage in one package with little setup.
Jest is an all-in-one JavaScript testing framework.
It includes a test runner, assertions, mocking, snapshots and coverage.
Tests use describe, test and expect with matchers such as toEqual.
14Linting: Linting is the automated analysis of source code, without running it, to flag likely bugs, style problems, and suspicious patterns before the code ships.
A linter analyzes code without running it.
It flags likely bugs, risky patterns, and style inconsistencies.
Rules are configurable, and many issues can be fixed automatically.
15Load Testing: Load testing is a type of performance testing that simulates many users or requests at once to measure how a system behaves under expected traffic.
Load testing simulates many concurrent users or requests against a system.
The key measurements are response time percentiles, throughput, and error rate.
Percentiles such as p95 and p99 reveal slow outliers that averages hide.
16Mocking: Mocking is a testing technique that replaces a real dependency, such as a database or API, with a fake you control so that code can be tested in isolation.
A mock replaces a real dependency with a controllable fake.
Mocks make tests fast, repeatable, and free of side effects.
Mocks can verify how they were called, not just return data.
17Mutation Testing: Mutation testing measures the quality of a test suite by inserting small deliberate bugs into the code and checking whether the tests fail and catch each one.
Mutation testing inserts small bugs, called mutants, to test the tests.
A mutant is killed when a test fails and survives when all tests pass.
The mutation score is the percentage of mutants the suite killed.
18Performance Testing: Performance testing measures how fast, stable and scalable a system is under expected and extreme load, from response times to its breaking point.
Performance testing measures speed, stability and scalability under load.
It includes load, stress, spike, soak and scalability tests.
Track percentiles such as p95 and p99, throughput and error rate.
19Playwright: Playwright is Microsoft's open-source framework for end-to-end testing and browser automation, driving Chromium, Firefox and WebKit through one API.
Playwright automates Chromium, Firefox and WebKit through one API.
Role-based locators find elements the way users see them.
Auto-waiting before every action reduces flaky tests.
20Property-Based Testing: Property-based testing checks that a rule holds for many automatically generated inputs, instead of only for a handful of examples written by hand.
You state a rule that must hold for all valid inputs, not a single expected output.
The framework generates many inputs, including tricky edge cases.
Failing inputs are shrunk to the smallest example that still fails.
21Quality AssuranceQA: Quality assurance (QA) is the set of practices that ensure software meets its requirements and works reliably, aiming to prevent defects, not just find them.
QA is the set of practices that keeps software meeting its requirements.
It aims to prevent defects, not only to find them.
Quality control checks the product; quality assurance improves the process.
22Regression Testing: Regression testing is the practice of re-running existing tests after a code change to make sure that features which used to work have not broken.
A regression is a bug in something that used to work.
Regression testing re-runs existing tests after every change.
Automated test suites in CI/CD make regression testing cheap and constant.
23Selenium: Selenium is an open-source suite of tools for automating web browsers, used mainly for end-to-end testing, with its WebDriver API available in many languages.
Selenium automates real web browsers, mainly for end-to-end tests.
It started in 2004; WebDriver became a W3C standard in 2018.
Tests can be written in Java, Python, C#, JavaScript, Ruby and more.
24Smoke Test: A smoke test is a quick, shallow check that the most important features of a build work at all, run before any time is spent on deeper testing.
A smoke test is a fast, shallow check that the critical parts of a build work.
A failed smoke test means the build is broken, so deeper testing is skipped.
Smoke tests commonly run after each build and after each deployment.
25Snapshot Testing: Snapshot testing is a technique that saves the output of code to a file on the first run and fails later runs if the output no longer matches that copy.
A snapshot test compares the current output with a saved, committed copy.
The first run creates the snapshot; later runs fail if the output differs.
Intentional changes are accepted by updating the snapshot and committing it.
26Static Analysis: Static analysis is the automated examination of source code without running it, to find bugs, security vulnerabilities, and quality problems early.
Static analysis examines code without executing it.
It finds bugs, security vulnerabilities, and quality issues early.
Techniques range from pattern matching to data-flow and taint analysis.
27Stress Testing: Stress testing pushes a system beyond its expected workload on purpose to find its breaking point and to check that it fails gracefully and recovers afterward.
Stress testing pushes a system past its expected load to find the breaking point.
It reveals which resource or component fails first.
A good result is graceful failure and quick recovery, not just a high limit.
28Test Automation: Test automation is using code to run tests and check their results, so the same checks repeat quickly and consistently on every change instead of by hand.
Test automation runs tests and checks results with code.
Unit, integration and end-to-end tests can all be automated.
Running the suite in CI catches regressions within minutes.
29Test Case: A test case is a check of one specific behavior: starting conditions, input or steps, and the expected result that shows if the software behaves correctly.
A test case checks one behavior with conditions, steps and an expected result.
Related test cases are grouped into test suites.
Arrange, Act, Assert keeps automated test cases readable.
30Test Coverage: Test coverage is a metric that measures how much of a program's source code is executed when its automated tests run, usually shown as a percentage.
Coverage shows which code ran during tests, usually as a percentage.
Common types are line, branch, and function coverage.
Branch coverage is stricter and catches untested conditions.
31Test Fixture: A test fixture is the known state a test needs before it runs, such as sample data or a configured object, plus the code that sets it up and tears it down.
A fixture is the known starting state that a test relies on.
Setup creates the state, and teardown cleans it up afterward.
Fixtures can be scoped per test, per file, or per test run.
32Test Pyramid: The test pyramid is a model for balancing automated tests: many fast unit tests, fewer integration tests, and only a few slow end-to-end tests at the top.
Write many unit tests, fewer integration tests, and only a few end-to-end tests.
Tests higher up are slower and more fragile but test more of the system at once.
Push each test to the lowest level that can catch the bug.
33Test Runner: A test runner is a tool that finds a project's automated tests, executes them, and reports which ones passed or failed, usually from a single command.
A test runner discovers, executes, and reports on automated tests.
A nonzero exit code on failure lets CI pipelines block broken changes.
Common features include parallel runs, watch mode, filtering, and timeouts.
34Test-Driven Development: Test-driven development is a coding practice in which you write a failing test first, then write just enough code to pass it, and then clean up the design.
Write a failing test before writing the code that makes it pass.
The cycle is red (fail), green (pass), refactor (clean up).
Each cycle is small and fast, often just a few minutes.
35Unit Test: A unit test is a small, automated check that verifies one function, method, or class behaves correctly in isolation from the rest of the program.
A unit test checks one small piece of code, such as a single function, in isolation.
Most unit tests follow the Arrange, Act, Assert pattern.
Unit tests are fast because they avoid real databases, networks, and file systems.
36White-Box Testing: White-box testing designs tests from knowledge of the code's internal structure, so that its statements, branches and paths are exercised and checked directly.
White-box testing uses knowledge of the code to design tests.
The goal is to exercise statements, branches and paths.
Coverage tools and mutation testing measure how thorough it is.