# Python Scheduled Task Monitoring: APScheduler, Celery, Cron

> Monitor Python scheduled tasks: APScheduler, Celery Beat, system cron, and custom schedulers. Detect missed runs, silent failures, and backpressure early.

Source: https://crontinel.com/blog/python-scheduled-task-monitoring/

Python powers everything from data pipelines to background job processing. But unlike Laravel or Rails, Python doesn't have a single dominant scheduling framework — teams use APScheduler, Celery Beat, system cron, or custom scheduling logic. Each has different failure modes, and most teams discover problems when a pipeline stops producing reports or a data sync falls behind.

This guide covers how to monitor Python scheduled tasks across every common setup, what to watch for beyond "did it run," and how to catch failures before they become customer-facing issues.

> **Supported check-in today:** post one HTTP outcome receipt (`outcome_checkin.py` or the same JSON body). Do not treat `pip install crontinel` as parity with the Laravel package. Recipes: [docs.crontinel.com/check-in/recipes/](https://docs.crontinel.com/check-in/recipes/).

## The Python Scheduling Landscape

Unlike PHP frameworks with built-in schedulers, Python teams typically use one of these:

| Tool | Typical Use | Scheduling Model |
|------|-------------|-----------------|
| **APScheduler** | In-process scheduled jobs | Cron-like triggers, interval, date-based |
| **Celery Beat** | Distributed task scheduling | Crontab entries, periodic task registry |
| **System cron** | Shell scripts, management commands | Traditional crontab entries |
| **Custom schedulers** | Data pipelines, ETL | Database-driven or in-memory queues |
| **AsyncIO tasks** | FastAPI/async services | asyncio.sleep loops, task queues |

Each has different failure modes. A system cron job that silently fails produces no logs. A Celery Beat task that loses its schedule after a restart runs forever without anyone noticing. An APScheduler job stuck in a retry loop burns CPU without doing useful work.

## What to Monitor

### 1. Heartbeat — Did It Run?

The foundation. Your task sends a signal when it starts or finishes. If the signal doesn't arrive within the expected window, something is wrong.

**Without monitoring:** A Celery Beat task stops running at 2am. You discover it at 9am when the daily report doesn't arrive.

**With monitoring:** Alert fires at 2:05am. You fix it before anyone notices.

```python
# crontinel_http.py: standard library only
from datetime import datetime, timezone
from functools import wraps

ORIGIN = "https://app.crontinel.com/api/v1/ingest"

def now():
    return datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")

def send(path, body):
    request = urllib.request.Request(
        f"{ORIGIN}/{path}",
        data=json.dumps(body).encode(),
        headers={
            "Authorization": "Bearer " + os.environ["CRONTINEL_INGEST_KEY"],
            "Content-Type": "application/json",
        },
        method="POST",
    )
    try:
        urllib.request.urlopen(request, timeout=5)
    except Exception:
        pass  # Monitoring must not fail the business job.

def monitored(command):
    """Send a running receipt, then completed or failed with the same request_key."""
    def decorate(job):
        @wraps(job)
        def run(*args, **kwargs):
            started = now()
            key = f"{command}-{started}"
            send("cron", {"command": command, "request_key": key, "status": "running", "started_at": started})
            try:
                records = job(*args, **kwargs)
            except Exception as error:
                send("cron", {"command": command, "request_key": key, "status": "failed", "exit_code": 1,
                              "started_at": started, "finished_at": now(), "output": str(error)})
                raise
            body = {"command": command, "request_key": key, "status": "completed", "exit_code": 0,
                    "started_at": started, "finished_at": now()}
            if isinstance(records, (int, float)) and not isinstance(records, bool):
                body["outcomes"] = {"metrics": {"processed_records": records}}
            send("cron", body)
            return records
        return run
    return decorate
```

```python
# APScheduler
from apscheduler.schedulers.blocking import BlockingScheduler

scheduler = BlockingScheduler()

@scheduler.scheduled_job('cron', hour=2, minute=0)
@monitored("daily-report")
def daily_report():
    return generate_report()  # return the number of records produced
```

The `running` receipt lets Crontinel alert when a job starts and never finishes. Register the job with its cron expression ([docs](https://docs.crontinel.com/check-in/schedule/)) and a run that never starts alerts too.

### 2. Duration — Is It Slow?

A job that usually takes 30 seconds suddenly takes 10 minutes. It still "works," but something is wrong — database lock, API rate limit, memory pressure.

**Watch for:**
- Jobs taking 2x+ longer than their historical average
- Gradual duration increase over days (resource leak)
- Spike in duration correlating with traffic or data volume

### 3. Exit Code — Did It Succeed?

A job runs, finishes, but exits with code 1. In system cron, this produces no output unless you explicitly redirect stderr. In Celery, it raises an exception. In APScheduler, it might silently swallow the error.

```python
# Bad: exception is caught and forgotten
try:
    process_data()
except Exception as e:
    logger.error(f"Failed: {e}")  # Logged, but who's watching?

# Good: the decorator sends a failed receipt, then re-raises
@monitored("process-data")
def process():
    return process_data()
```

### 4. Queue Depth — Is Work Backing Up?

If your scheduled tasks push jobs to a queue (Redis, RabbitMQ, database), monitor the queue depth. A growing queue means workers can't keep up.

**Redis queue monitoring:**

```python

r = redis.Redis()

def check_queue_depth(queue_name="celery"):
    depth = r.llen(queue_name)
    if depth > 1000:
        send("event", {"type": "queue.backlog", "severity": "warning",
                       "payload": {"queue": queue_name, "depth": depth}})
    return depth
```

### 5. Schedule Drift — Is It Running on Time?

A task scheduled for 2:00am starts running at 2:15am because the previous run took too long. Or timezone changes cause jobs to fire at the wrong time.

**With APScheduler:**

Register the job with its cron expression and a grace period. Crontinel alerts when the start does not arrive in the window, so the job does not need to measure its own drift.

```python
send("schedule", {"source": "named_by_user", "tasks": [
    {"command": "daily-sync", "expression": "0 2 * * *", "timezone": "UTC", "grace_seconds": 300},
]})
```

## Monitoring APScheduler

APScheduler runs in-process, so if your application crashes, all scheduled jobs stop silently.

### Setup

```python
from apscheduler.schedulers.asyncio import AsyncIOScheduler
from apscheduler.events import EVENT_JOB_ERROR, EVENT_JOB_MISSED

scheduler = AsyncIOScheduler()

def on_job_error(event):
    send("event", {"type": "apscheduler.job_error", "severity": "critical",
                   "payload": {"job": event.job_id, "error": str(event.exception)}})

def on_job_missed(event):
    send("event", {"type": "apscheduler.job_missed", "severity": "warning",
                   "payload": {"job": event.job_id}})

scheduler.add_listener(on_job_error, EVENT_JOB_ERROR)
scheduler.add_listener(on_job_missed, EVENT_JOB_MISSED)

# Wrap each job so it sends receipts (monitored() is defined above)
@scheduler.scheduled_job('interval', minutes=5, id='health-check')
@monitored("health-check")
def health_check():
    check_services()

scheduler.start()
```

### Common Failures

1. **Scheduler stops after app restart** — No persistence configured. Jobs exist only in memory.
2. **Job throws exception** — Without EVENT_JOB_ERROR listener, the error is logged but nobody sees it.
3. **Job misses its window** — Without misfire_grace_time, APScheduler skips late jobs silently.

## Monitoring Celery Beat

Celery Beat is a separate process that enqueues tasks on schedule. If Beat stops, no new tasks are created. If workers stop, tasks pile up in the queue.

### Setup

```python
# tasks.py (monitored() and send() are defined above)
from celery import Celery

app = Celery('myapp')

@app.task(name='reports.daily')
@monitored("celery-reports-daily")
def daily_report():
    return generate_report()
```

Celery Beat only schedules the task. Register the same name with its cron expression so a skipped Beat run alerts.

### Watch for Beat Process Health

```bash
# Check if Beat is running
ps aux | grep celery beat

# Check Celery flower dashboard
curl http://localhost:5555/api/workers
```

### Common Failures

1. **Beat process dies** — No new tasks are scheduled. Everything else looks fine.
2. **Worker pool exhausted** — All workers busy, new tasks queue up.
3. **Task loses state after restart** — Database-backed scheduler needed for persistence.

## Monitoring System Cron

System cron is the simplest but least observable. Jobs run as shell commands. If they fail, you need explicit logging.

### Setup

```bash
# crontab entry with logging
0 2 * * * /usr/bin/python3 /app/manage.py daily_report >> /var/log/cron/daily_report.log 2>&1; \
  [ $? -eq 0 ] && echo "$(date): SUCCESS" >> /var/log/cron/monitor.log || echo "$(date): FAILED" >> /var/log/cron/monitor.log
```

### Better: Use a wrapper script

```bash
#!/bin/bash
# /usr/local/bin/cron-wrapper.sh
JOB_NAME=$1
shift

STARTED=$(date -u +%FT%TZ)
KEY="$JOB_NAME-$STARTED"
post() {
    curl -sS -m 5 -X POST "https://app.crontinel.com/api/v1/ingest/cron" \
        -H "Authorization: Bearer $CRONTINEL_INGEST_KEY" \
        -H "Content-Type: application/json" -d "$1" || true
}

post "{\"command\":\"$JOB_NAME\",\"request_key\":\"$KEY\",\"status\":\"running\",\"started_at\":\"$STARTED\"}"
"$@"
EXIT_CODE=$?

STATUS=completed
[ $EXIT_CODE -ne 0 ] && STATUS=failed
post "{\"command\":\"$JOB_NAME\",\"request_key\":\"$KEY\",\"status\":\"$STATUS\",\"exit_code\":$EXIT_CODE,\"started_at\":\"$STARTED\",\"finished_at\":\"$(date -u +%FT%TZ)\"}"

exit $EXIT_CODE
```

```bash
# crontab
0 2 * * * cron-wrapper.sh daily-report python3 /app/manage.py daily_report
```

## HTTP check-in (supported path)

Prefer the workspace script or the same JSON over PyPI:

```bash
python3 outcome_checkin.py \
  --command my-task \
  --request-key run-1 \
  --exit-code 0 \
  --started-at 2026-10-06T02:00:00Z \
  --finished-at 2026-10-06T02:00:04Z \
  --records 0 \
  --url https://app.crontinel.com \
  --api-key "$CRONTINEL_INGEST_KEY"
```

Full recipes: [docs.crontinel.com/check-in/recipes/](https://docs.crontinel.com/check-in/recipes/).

## Quick Comparison

| Feature | APScheduler | Celery Beat | System Cron | Crontinel |
|---------|------------|-------------|-------------|-----------|
| Missed run detection | ✅ (with listener) | ❌ | ❌ | ✅ |
| Duration tracking | ❌ | ❌ | ❌ | ✅ |
| Queue depth | N/A | ✅ (with Flower) | N/A | ✅ |
| Auto-alerts | ❌ | ❌ | ❌ | ✅ |
| Works with all schedulers | ✅ | ✅ | ✅ | ✅ |
