Skip to content

Background Worker

The BullMQ background worker handles batch quiz generation asynchronously.

Architecture

Teacher clicks "Start Generation"
  → POST /api/admin/quizzes/batches/{id}/run
  → Enqueues BullMQ job + returns 202 { jobId }

Worker (separate Deployment)
  → Picks up job from Redis queue
  → Runs AI generation with concurrency control
  → Updates DB per item (status, content, error)
  → Handles credit refund on completion

Frontend
  → Polls GET /api/.../status every 3 seconds
  → Renders BatchProgress UI
  → Teacher can leave and return — state is in DB

Queue Configuration

  • Queue name: batch-generation
  • Redis: Same cluster used by web and worker
  • Concurrency: 2-3 parallel AI calls per job

Job Data

typescript
interface BatchGenerationJobData {
  batchId: string;
  teacherId: string;
  chunkSize: number;
}

Development

bash
# Build the worker
pnpm build:worker

# Run locally (requires Redis)
REDIS_HOST=localhost REDIS_PORT=6379 node dist/worker/index.js

The worker logs [worker] Ready, listening on queue: batch-generation when connected.

Monitoring

bash
# Check worker pod status
kubectl get pods -n quizzz | grep worker

# View worker logs
kubectl logs -n quizzz deployment/dev-quizzz-worker --tail=50

# Check Redis queue
kubectl exec -n prod redis-cluster-master-0 -- \
  redis-cli -a redis123 LLEN bull:batch-generation:wait

Scaling

The worker deployment starts with 1 replica. Scale based on queue depth:

bash
kubectl scale deployment/dev-quizzz-worker -n quizzz --replicas=2

Each worker instance processes jobs independently with BullMQ's built-in concurrency control.

TechTrans Lab