Docker Installation Guide

This guide shows how to run TestPlanIt using Docker Compose with all required services included. The setup is self-contained and supports both development and production modes.

Services Included

The Docker Compose setup starts these containerized services:

1. TestPlanIt Application (dev/prod) - Next.js web application
2. Background Workers (workers-dev/workers) - Process asynchronous jobs:
   • Email sending and notifications
   • Scheduled tasks (daily digests at 8 AM, forecast updates at 3 AM)
   • Search indexing and file processing
3. PostgreSQL (postgres) - Main database with automatic schema initialization
4. Valkey (valkey) - Job queue and caching (Redis-compatible in-memory data store)
5. Elasticsearch (elasticsearch) - Search and indexing engine
6. MinIO (minio) - S3-compatible object storage for file attachments
7. Nginx (nginx) - Reverse proxy for internal routing and file access
8. Database Initialization (db-init/db-init-prod) - Automatic schema setup and seeding
9. MinIO Initialization (minio-init) - Automatic bucket creation and permissions

Prerequisites

• Docker with Compose plugin

• Git

• RAM Requirements:

  Phase	Minimum	Recommended	Notes
  Building	8GB	8GB	Peaks ~7GB, only during the build
  Running (Full Stack)	8GB	14GB	All services combined

  Memory-constrained systems: The build peaks around 7GB and the running stack needs 8–14GB, so a machine sized for the running total also covers builds.

  Per-service breakdown (running):

  Service	Minimum	Recommended
  TestPlanIt Application	3GB	4GB
  Background Workers	2GB	4GB
  PostgreSQL	1GB	2GB
  Elasticsearch	2GB	3GB
  MinIO	512MB	1GB
  Valkey (Redis)	32MB	64MB
  Total	~8GB	~14GB

  The Background Workers figure is steady-state for a typical single-tenant install. One always-on worker (SCIM access recompute) idles near 1.4GB on its own regardless of whether SCIM is configured, which is why the fleet no longer fits in the old 512MB floor. Individual workers also restart at higher ceilings under load (forecast and SCIM access recompute at 2GB, the webhook workers at 3GB each — whose steady-state RSS can approach 1.9GB each in busy multi-tenant clusters). Multi-tenant operators should size the worker fleet well above the recommended figure; see the worker memory tiers for the per-worker breakdown.

• 25GB+ disk space for data and images

Installation & Setup Steps

1. Clone the repository: Open your terminal and clone the TestPlanIt monorepo:

   git clone https://github.com/testplanit/testplanit.git
   cd testplanit

2. Navigate to the application directory:

   cd testplanit

3. Set up Environment Variables:

   For Development:

   cp .env.example .env.development

   For Production:

   cp .env.example .env.production

   Then open .env.production and update these values for your deployment:

   # REQUIRED CHANGES for production:
   
   # Application URL (change to your domain)
   NEXTAUTH_URL="https://yourdomain.com"
   
   # Generate a secure secret (run this command and paste the result)
   # openssl rand -base64 32
   NEXTAUTH_SECRET="your-generated-secret-here"
   
   # External file access domain (must match your public URL)
   AWS_PUBLIC_ENDPOINT_URL="https://yourdomain.com"
   
   # Change the default admin email and password!
   [email protected]
   ADMIN_NAME="Administrator Account"
   ADMIN_PASSWORD=your-secure-password
   
   # OPTIONAL: Email settings (required for Magic Link authentication and notifications)
   EMAIL_SERVER_HOST=smtp.your-provider.com
   EMAIL_SERVER_PORT=587
   [email protected]
   EMAIL_SERVER_PASSWORD=your-password
   [email protected]

   Important: The .env.production file contains many other variables (database, Valkey, Elasticsearch, MinIO connections) that are already configured correctly for Docker. Only modify the variables shown above unless you're using external services.

   Optional: Change External Docker Ports

   You can override host-exposed ports with DOCKER_*_PORT variables (for example DOCKER_POSTGRES_PORT=5732).

   Compose files use fallback syntax such as ${DOCKER_POSTGRES_PORT:-5432}:5432, which means:

   • use DOCKER_POSTGRES_PORT when set
   • otherwise fall back to 5432

   Important behavior:

   • env_file: in docker-compose*.yml sets runtime variables inside containers
   • it does not control Compose interpolation for published ports

   If you changed port values in .env.development or .env.production, run Compose with --env-file:

   # Development
   docker compose --env-file .env.development -f docker-compose.dev.yml up -d --build
   
   # Production
   docker compose --env-file .env.production -f docker-compose.prod.yml up -d --build

   Alternatively, place the DOCKER_*_PORT values in a local .env file (auto-loaded by Compose).

   Note: this only changes external host bindings. Internal service ports stay the same (for example PostgreSQL remains postgres:5432 inside the Docker network).

4. Choose Your Services:

   TestPlanIt uses Docker Compose profiles to make services optional:

   • dev / prod - Development/Production app+workers profiles (includes all dependencies)
   • with-postgres - PostgreSQL database
   • with-valkey - Valkey/Redis cache and queue
   • with-elasticsearch - Elasticsearch search (optional)
   • with-minio - MinIO file storage + Nginx

   Quick Start Options:

   All Services (Full Stack):

   # Development
   docker compose --profile dev up --build
   
   # Production
   docker compose --profile prod up --build

   Selective Services:

   # Development without search
   docker compose up dev workers-dev postgres valkey minio --build
   
   # Production with only essential services
   docker compose up prod workers postgres valkey --build

5. Start the Application:

   Development Mode (with hot-reload, all services):

   docker compose --profile dev up --build

   Production Mode (optimized builds, all services):

   docker compose --profile prod up --build

   Production with External Services:

   # Example: Using external database and S3, Docker for cache/search
   docker compose -f docker-compose.prod.yml \
     --profile with-valkey \
     --profile with-elasticsearch \
     up --build

   The --build flag ensures images are built from the latest code.

   What Happens During Startup:

   1. Selected services start based on profiles
   2. PostgreSQL (if included) starts and becomes healthy
   3. Other services (Valkey, Elasticsearch, MinIO) start
   4. Database initialization runs (if using PostgreSQL container)
   5. MinIO initialization creates buckets (if using MinIO container)
   6. Main application and workers start
   7. Nginx proxy becomes available (if using MinIO)

   First startup takes 2-5 minutes as images are built and services initialize.

6. Access TestPlanIt:

   • Development (dev): http://localhost:3000
     • Override with DOCKER_DEV_APP_PORT
   • Production (prod): http://localhost:30000
     • Override with DOCKER_PROD_APP_PORT
   • Default login: [email protected] / admin (change in production!)
   • Demo Project: A pre-populated Demo Project is created during initial setup with sample test cases, test runs, sessions, milestones, and issues. Use the Help menu > Start Demo Project Tour for a guided walkthrough.

7. Access Additional Services (if enabled):

   • MinIO Console (with-minio): http://localhost:9001
     • Override with DOCKER_MINIO_CONSOLE_PORT
   • Elasticsearch (with-elasticsearch): http://localhost:9200
     • Override with DOCKER_ELASTICSEARCH_HTTP_PORT
   • PostgreSQL (with-postgres): localhost:5432 (user: user / password: password)
     • Override with DOCKER_POSTGRES_PORT
   • Valkey (with-valkey): localhost:6379
     • Override with DOCKER_VALKEY_PORT

Environment Management

Starting & Stopping

# Start in foreground (see logs)
docker compose up prod workers

# Start in background (detached)
docker compose up prod workers -d

# Stop services (keeps data)
docker compose down

# Stop and remove containers/networks
docker compose down --remove-orphans

# Fresh start (removes all data!)
docker compose down
sudo rm -rf docker-data/
docker volume rm testplanit-postgres-data
docker compose up prod workers --build

Updates & Maintenance

# Update to latest version
git pull
docker compose build
docker compose up prod workers -d

# Rebuild specific service
docker compose build prod
docker compose up prod -d

# View resource usage
docker compose top
docker system df

File Storage Configuration

MinIO (Default): Provides S3-compatible file storage with automatic setup:

• Internal: http://minio:9000 (app ↔ MinIO)
• External: https://yourdomain.com/testplanit/... (browser access)
• Console: http://localhost:9001 (admin interface)
• Bucket: testplanit (auto-created with public read permissions)

Nginx Reverse Proxy: Automatically routes /testplanit/ requests to MinIO for external file access while preserving AWS signature validation.

Switching to AWS S3: To use AWS S3 instead of MinIO:

# In .env.production, change:
AWS_ENDPOINT_URL=""  # Empty = use AWS S3
AWS_PUBLIC_ENDPOINT_URL=""  # Empty = use AWS S3
AWS_BUCKET_NAME=your-s3-bucket-name
AWS_ACCESS_KEY_ID=your-aws-access-key
AWS_SECRET_ACCESS_KEY=your-aws-secret-key

Then restart without the with-minio profile:

# Production example without MinIO
docker compose -f docker-compose.prod.yml \
  --profile with-postgres \
  --profile with-valkey \
  --profile with-elasticsearch \
  up --build

Data Management

Data Persistence

Most service data persists in ./docker-data/:

docker-data/
├── valkey/        # Valkey job queue persistence
├── elasticsearch/ # Search indexes
└── minio/         # File attachments

Postgres is the exception: it uses the named Docker volume testplanit-postgres-data instead of a bind mount. Its data directory can't live inside ./docker-data/ (or anywhere else under the build context) — once Postgres has run, it locks the directory down to 0700, which makes docker compose build fail with a permission error on any later build.

Postgres was also bumped from 15 to 18 in the same change, so this isn't a straight file-copy upgrade — Postgres 18 cannot read a Postgres 15 data directory. Existing deployments need a pg_dump / pg_restore migration instead.

Upgrading an existing deployment

Deployments created before this change run Postgres 15 with data directly in docker-data/postgres/. Dump the database before switching to the updated compose files — once you do, postgres starts on the new image and can no longer read the old data directory:

# 1. On the OLD stack (still Postgres 15), dump the database.
docker compose exec -T postgres pg_dump -U user -Fc -d testplanit_prod > testplanit-backup.dump
docker compose down

Pull the updated compose files, then bring up a fresh Postgres 18 instance and restore into it:

# 2. Bring up the new (empty) Postgres 18 named volume and restore.
docker compose up postgres -d
# wait for it to report healthy, then:
docker compose exec -T postgres pg_restore -U user -d testplanit_prod --clean --if-exists < testplanit-backup.dump

# 3. Verify the app reads the migrated data correctly, then bring up the rest.
docker compose up prod workers -d

Once you've confirmed the migrated data is intact, the old docker-data/postgres/ directory is no longer used and can be removed. It's already locked down to 0700 by the old Postgres process, so removing it needs a root context — either sudo rm -rf docker-data/postgres, or from a throwaway container that doesn't need host-level sudo:

docker run --rm -v "$(pwd)/docker-data:/data" alpine rm -rf /data/postgres

Backup & Restore

Create Backup:

# Stop services
docker compose down

# Back up the bind-mounted service data
tar -czf testplanit-backup-$(date +%Y%m%d).tar.gz docker-data/

# Back up the Postgres named volume separately
docker run --rm \
  -v testplanit-postgres-data:/data \
  -v "$(pwd):/backup" \
  alpine tar -czf /backup/testplanit-postgres-backup-$(date +%Y%m%d).tar.gz -C /data .

# Restart services
docker compose up prod workers -d

Restore Backup:

# Stop and remove current data
docker compose down
sudo rm -rf docker-data/
docker volume rm testplanit-postgres-data

# Extract the bind-mounted service data
tar -xzf testplanit-backup-YYYYMMDD.tar.gz

# Restore the Postgres named volume
docker volume create testplanit-postgres-data
docker run --rm \
  -v testplanit-postgres-data:/data \
  -v "$(pwd):/backup" \
  alpine tar -xzf /backup/testplanit-postgres-backup-YYYYMMDD.tar.gz -C /data

# Restart services
docker compose up prod workers -d

Database Operations

# Access database directly
docker compose exec postgres psql -U user -d testplanit_prod

# Dump database
docker compose exec postgres pg_dump -U user testplanit_prod > backup.sql

# Restore database (with services stopped)
docker compose exec postgres psql -U user -d testplanit_prod < backup.sql

Service Management

Start specific service combinations:

# Full development stack
docker compose up dev workers-dev --build

# Full production stack
docker compose up prod workers --build

# Minimal development (no search/files)
docker compose up dev workers-dev postgres valkey --build

# Start in background (detached)
docker compose up prod workers --build -d

Service Profiles:

• dev - Development environment with all dependencies
• prod - Production environment with all dependencies
• with-postgres - PostgreSQL database container
• with-valkey - Valkey/Redis cache container
• with-elasticsearch - Elasticsearch search container
• with-minio - MinIO storage + Nginx proxy containers

Mix profiles to customize your deployment (e.g., --profile with-postgres --profile with-valkey for just database and cache).

Monitoring & Logs

# Check service status
docker compose ps

# View all logs (follow mode)
docker compose logs -f

# View specific service logs
docker compose logs -f prod        # Main application
docker compose logs -f workers     # Background jobs
docker compose logs -f postgres    # Database
docker compose logs -f elasticsearch # Search engine
docker compose logs -f minio       # File storage

# View initialization logs
docker compose logs db-init-prod    # Database setup
docker compose logs minio-init      # Storage setup

Troubleshooting

Common Issues

Services won't start:

# Check service status
docker compose ps

# Check specific service health
docker compose ps postgres
docker compose ps elasticsearch

# View startup logs
docker compose logs db-init-prod
docker compose logs minio-init

Database connection issues:

# Check PostgreSQL health
docker compose ps postgres
docker compose logs postgres

# Test database connection
docker compose exec postgres psql -U user -d testplanit_prod -c "SELECT 1;"

Search not working:

# Check Elasticsearch status
curl http://localhost:9200/_cluster/health

# Reindex data through admin interface
# Go to: Admin → System → Reindex Search Data

File upload/display issues:

Mixed content errors / uploads fail with "Failed to fetch":

• Cause: The app generates presigned URLs using the internal MinIO hostname (e.g., http://minio:9000), which the browser cannot reach. When the page is served over HTTPS, the browser blocks the HTTP request to the internal URL.

• Symptoms: Browser console shows Mixed Content: ... requested an insecure resource 'http://minio:9000/...' and file uploads fail.

• Fix: Set IS_HOSTED=true in your .env.production to enable proxy mode, which routes uploads through the Next.js server instead of directly to MinIO:

  IS_HOSTED=true

  Then recreate the container: docker compose up -d --force-recreate prod

• Alternative: Set AWS_PUBLIC_ENDPOINT_URL=https://yourdomain.com if MinIO is accessible through your reverse proxy. See File Storage docs for details.

403 Forbidden on file uploads:

• Cause: AWS signature mismatch between app and MinIO
• Fix: Ensure AWS_PUBLIC_ENDPOINT_URL matches your external domain exactly
• Check: MinIO console at http://localhost:9001 → bucket testplanit exists with public read

Files not displaying:

# Test MinIO direct access
curl -I http://localhost:9000/testplanit/

# Test nginx proxy
curl -I http://localhost:80/testplanit/

# Check MinIO health
curl http://localhost:9000/minio/health/live

Image optimization errors:

• Add your domain to next.config.mjs images.remotePatterns
• Verify files accessible at: https://yourdomain.com/testplanit/...

Background jobs not processing:

# Check worker status
docker compose logs workers

# Test Valkey connection
docker compose exec valkey valkey-cli ping

# Check worker processes inside container
docker exec testplanit-workers pm2 list

Reset Everything

# Nuclear option - fresh start
docker compose down
sudo rm -rf docker-data/
docker volume rm testplanit-postgres-data
docker compose up prod workers --build