Throughput

Pronunciation: THROO-put

Updated October 3, 20262 min read

In short

Throughput is the amount of data or work a system actually handles per unit of time, like megabits per second on a network or requests per second on a server.

What is throughput?

Throughput is what you really get. A network link might offer 1 Gbps of bandwidth, but a file transfer over it may reach only 600 Mbps because of protocol overhead, congestion, packet loss or a slow server at the other end. For servers and databases, throughput is measured in work completed: requests, transactions or messages per second.

It is closely tied to latency. For TCP, the amount of unacknowledged data in flight is limited by the window size, so a single connection over a long distance can be slow even on a fast link: throughput is roughly the window size divided by the round-trip time. Parallel connections, larger windows and better congestion control raise it.

In system design, throughput is one of the main capacity measures, together with latency and error rate. Load tests increase traffic until throughput stops rising while latency climbs, which reveals the bottleneck, whether it is CPU, a database lock, a connection pool or the network. Batching, caching and horizontal scaling are typical ways to raise it.

A common misconception is that throughput, bandwidth and latency are the same thing. Bandwidth is the maximum capacity of a link, throughput is the rate actually achieved, and latency is how long one piece of data takes to arrive. A system can have high throughput and high latency at the same time, as a truck full of hard drives does.

Key takeaways

Throughput is the work or data actually handled per unit of time.
Networks measure it in bits per second; servers in requests per second.
It is usually lower than bandwidth because of overhead and congestion.
TCP throughput is limited by window size divided by round-trip time.
Bandwidth is capacity, throughput is achieved rate, latency is delay.

Example

Measuring throughput of a network link and a web serverbash

# Network: run iperf3 -s on the server, then from the client
iperf3 -c 10.0.0.5 -t 10
# [ ID] Interval      Transfer     Bitrate
# [  5] 0.00-10.00 sec 1.10 GBytes  944 Mbits/sec

# Web server: 50 concurrent connections for 30 seconds
wrk -t4 -c50 -d30s https://localhost:8443/api/health
# Requests/sec:  12873.41   ← throughput
# Latency avg:   3.85ms

Readers ask

What is the difference between bandwidth and throughput?

Bandwidth is how much data a link could carry at most. Throughput is how much actually gets through in practice, which is reduced by overhead, congestion, errors and the limits of the devices at each end.

What is the difference between latency and throughput?

Latency is the time a single request or packet takes. Throughput is how many requests or how much data are handled per second. Improving one does not automatically improve the other.

How do I increase a server's throughput?

Find the bottleneck first with load tests and profiling, then remove it: cache repeated work, batch database writes, tune connection pools, use asynchronous I/O or add more instances behind a load balancer.

Often compared

Latency vs ThroughputLatency is how long one request or piece of data takes to arrive, while throughput is how much data or how many requests a system handles per unit of time.