Background Job

In Turkish: Arka Plan İşi

Updated September 30, 20263 min read

In short

A background job is a task that a server runs outside the normal request-response cycle, so slow work like sending emails doesn't make users wait.

What is a background job?

A background job is work that an application hands off to be done later or elsewhere, instead of doing it while the user waits for a response. When a user signs up, for example, the web server saves the account and responds right away, while a background job sends the welcome email a moment later. This keeps pages and API responses fast even when the underlying work takes seconds or minutes.

The usual setup has three parts. The web application creates a job, a small record with a name and arguments such as a user ID, and puts it on a queue, which is often stored in Redis, a database table, or a message broker. Separate worker processes pick up jobs, run them, and mark them as done, retrying failed jobs with increasing delays and setting aside jobs that keep failing so a developer can inspect them.

It's like dropping off clothes at a dry cleaner: you get a ticket immediately and walk away, and the cleaning happens in the back while you do other things. Typical background jobs include sending emails and notifications, resizing uploaded images, generating PDF reports, importing large files, and calling slow third-party APIs. Common open-source libraries include Sidekiq for Ruby, Celery for Python, and BullMQ for Node.js.

Background jobs are often confused with cron jobs and message queues. A cron job is started by the clock on a fixed schedule, while a background job is usually triggered by an event such as a user action, and a message queue is the infrastructure many job systems are built on. Because retries mean a job can run more than once, job handlers should be idempotent, meaning running them twice has the same effect as running them once.

Key takeaways

Background jobs move slow work out of the request so users get fast responses.
The app enqueues a job; separate worker processes run it later.
Failed jobs are retried, often with exponential backoff.
Jobs should be idempotent, because retries can run them more than once.
Cron jobs are triggered by time; background jobs are usually triggered by events.

Example

Enqueuing and processing a job with BullMQjavascript

import { Queue, Worker } from "bullmq"; // stores jobs in Redis
const connection = { host: "localhost", port: 6379 };
const emailQueue = new Queue("emails", { connection });

// In the web app: enqueue the job and respond immediately
app.post("/signup", async (req, res) => {
  const user = await createUser(req.body);
  await emailQueue.add("welcome", { userId: user.id }, { attempts: 3 });
  res.status(201).json(user);
});

// In a separate worker process: run each job as it arrives
new Worker("emails", async (job) => {
  await sendWelcomeEmail(job.data.userId);
}, { connection });

Readers ask

What is the difference between a background job and a cron job?

A cron job runs on a fixed schedule, such as every night at 2 a.m. A background job is usually created on demand by an event, such as a user uploading a file, and runs as soon as a worker is free, although many job systems also support scheduled jobs.

What is a worker in background processing?

A worker is a separate process that waits for jobs on a queue, runs them, and reports whether they succeeded. You can run several workers in parallel to process jobs faster.

When should I use a background job?

Use one when work takes more than a fraction of a second, depends on a slow external service, or doesn't need to finish before the user gets a response. Sending emails, processing uploads, and generating reports are classic examples.