Design Job Scheduler

Understanding the Problem

A job scheduler is a system that executes predefined pieces of work at a particular time or according to a recurring schedule. It is commonly used for things like periodic data processing, automated reports, cleanup workflows, backups, and other work that should happen without a user manually triggering it.

Before designing the system, it helps to distinguish between two related concepts.

• Task: Describes the reusable type of work that can be performed. For example, generate monthly invoice might be a task. The same task definition can be executed many times with different inputs.

• Job: Represents one configured execution of a task. It combines the task with its execution schedule and any parameters required to run it. For example, a job might mean: run the generate monthly invoice task for customer ACME-42 at midnight on the first day of every month.

In other words, the Task defines what the system knows how to do, while the Job specifies when that work should run and with which inputs. The core responsibility of the scheduler is therefore straightforward: accept a collection of jobs, determine when each one becomes due, and make sure the corresponding task is executed at the appropriate time.

Functional Requirements

1. Schedule Jobs: Users should be able to create jobs that run immediately, execute once at a specified future time, or repeat according to a recurring schedule. For example, a job might run every weekday at 8:00 AM or once at midnight next Friday.

2. Track Job Status: Users should be able to inspect their jobs and see their current execution state, such as whether a job is waiting to run, currently executing, completed successfully, or failed.

Non-Functional Requirements

1. High Availability: The scheduler should remain available even when individual components fail. We will generally prefer availability over strong consistency, since temporarily delayed or duplicated scheduling decisions are easier to recover from than making the entire scheduling system unavailable.

2. Low Scheduling Delay: A job should begin execution close to its intended trigger time, ideally within 2 seconds of the configured schedule under normal operating conditions.

3. Horizontal Scalability: The architecture should scale to dispatch and process up to roughly 10,000 jobs per second without relying on a single scheduler or worker node.

4. At-Least-Once Execution: Once a job becomes due, the system should make sure it is attempted at least once. Failures, worker crashes, or message redelivery may occasionally cause duplicate executions, so downstream tasks should be designed to tolerate retries or use idempotency where possible.

Data Model

Even though a job scheduler is primarily an infrastructure system, it still helps to define the main domain objects before designing the execution pipeline. We do not need every column yet; the goal is simply to establish what each entity represents.

1. Task: Describes a reusable unit of work that the system knows how to execute. For example, generateReport or cleanupExpiredSessions could each be represented as a task definition.

2. Job: Represents one configured use of a task. A job links the task to the parameters required for execution and tracks its lifecycle, such as whether it is scheduled, running, completed, or failed.

3. Schedule: Defines when the job should run. It may represent a one-time timestamp or a recurring rule such as a CRON expression. Keeping scheduling information separate makes it easier to support different trigger types without changing the underlying task definition.

4. User: Represents the account that creates and manages jobs. A user should be able to schedule new work, inspect existing jobs, and view their execution status.

Together, these entities separate the system cleanly: the Task defines what work exists, the Job defines a configured execution of that work, and the Schedule determines when that execution should be triggered.

API Design

The API surface can be derived directly from the two functional requirements: users need a way to create scheduled work and a way to inspect the jobs they have already submitted.

Create a Job

To schedule new work, the client creates a Job and specifies which task should run, when it should execute, and the parameters required by that task.

For example:

POST /jobs

{
  "taskId": "generate_report",
  "schedule": {
    "type": "cron",
    "expression": "0 8 * * 1-5"
  },
  "parameters": {
    "reportType": "sales-summary",
    "region": "us-west"
  }
}

The same endpoint can also support a one-time execution by replacing the recurring CRON rule with a specific timestamp:

{
  "taskId": "cleanup_export",
  "schedule": {
    "type": "once",
    "executeAt": "2026-09-01T03:00:00Z"
  },
  "parameters": {
    "exportId": "exp_8421"
  }
}

An immediate job can either use an explicit immediate schedule type or simply omit a future execution time, depending on how we define the API contract. The authenticated user identity should come from the session or access token rather than from a client-supplied userId.

Query Jobs

Users also need to inspect their submitted jobs and filter them by execution state or time range.

A simple endpoint is:

GET /jobs?status={status}&startTime={startTime}&endTime={endTime}

which returns a paginated collection of jobs belonging to the authenticated user.

For example, the client might request all failed jobs created during a particular period:

GET /jobs?status=FAILED&startTime=2026-08-01T00:00:00Z&endTime=2026-08-31T23:59:59Z

The response can expose fields such as the task, schedule, current status, most recent execution time, and last failure information. Together, these endpoints cover the basic control plane of the scheduler: POST /jobs defines work that should happen in the future, while GET /jobs gives users visibility into its current state.

Job Scheduling Flow

At a high level, a job moves through four stages.

1. Job Submission: A user creates a job by specifying the task to run, its execution schedule, and any parameters required by that task.

2. Durable Persistence: The scheduler stores the job before acknowledging it. This ensures that a successfully submitted job is not lost if a server crashes immediately afterward.

3. Scheduled Execution: When the job becomes due, the system dispatches it to an available worker. The worker executes the underlying task using the parameters stored with the job. If execution fails because of a transient problem, the system retries the job using exponential backoff rather than retrying continuously.

4. Status Update: The execution result is written back to durable storage so the user can see whether the job is still waiting, currently running, completed successfully, or failed.

This basic lifecycle gives us the foundation for the high-level design. The harder questions come next: how to efficiently discover jobs that are due, how to distribute execution across many workers, and how to guarantee that jobs are not lost when components fail.

---

High-level Design

Users Should Be Able to Schedule Immediate, Future, and Recurring Jobs

When a user creates a job, they provide three pieces of information: the task to execute, the schedule that determines when it should run, and the parameters required by that task. The client sends those values to POST /jobs, and the Job Service persists the request before returning success. That durable write matters because once the API acknowledges the job, the scheduler should still be able to recover and execute it even if the service crashes immediately afterward.

A simple first version might store everything in one record: