Harden VPS

Three-layer production hardening for a self-hosted app on any VPS. Each layer independently verifiable. Apply OS first (firewall blocks attacks immediately), then your application (alerts + backups), then your VPS provider (snapshots). See REFERENCE.md for full script bodies, systemd unit template, and gotchas.

Quick start

SSH as root into the VPS. Find credentials in the your VPS provider Customer Control Panel.

HARD GATE — Run ufw status first. No firewall = layer 1 takes priority over everything.

Layer 1 — Ubuntu OS

# UFW
ufw default deny incoming && ufw default allow outgoing
ufw allow 22/tcp && ufw allow 80/tcp && ufw allow 443/tcp && ufw enable
# → verify: ufw status | grep -q active

# fail2ban
apt install -y fail2ban
# Configure /etc/fail2ban/jail.local: sshd, maxretry=3, bantime=3600, findtime=600
systemctl restart fail2ban
# → verify: fail2ban-client status sshd

# unattended-upgrades
apt install -y unattended-upgrades && dpkg-reconfigure -plow unattended-upgrades
# → verify: systemctl is-active unattended-upgrades | grep -q active

# SSH: PermitRootLogin no, PasswordAuthentication no, PubkeyAuthentication yes
# → verify: sshd -T | grep -E 'permitrootlogin no|passwordauthentication no'

# Deploy healthcheck.sh → /opt/your-app/scripts/healthcheck.sh
# Crontab: */5 * * * * /opt/your-app/scripts/healthcheck.sh

Layer 2 — applicationlication

# systemd: User=your-app, NoNewPrivileges=yes, ProtectSystem=full,
#   ProtectKernelTunables=yes, ProtectKernelModules=yes,
#   ProtectControlGroups=yes, RestrictAddressFamilies=AF_INET AF_INET6,
#   RestrictRealtime=yes, PrivateTmp=yes, LimitNOFILE=65536
# → verify: systemctl show your-app -p NoNewPrivileges -p ProtectSystem -p User

# Alerts (your application requires auth; insert via SQLite)
sqlite3 /opt/your-app/data/your-app.db "
INSERT INTO monitoring_alerts (id,name,metric,threshold,operator,enabled,duration_seconds)
VALUES ('a1','Disk >80%','disk_used_percent',80,'gt',1,300);
INSERT INTO monitoring_alerts (id,name,metric,threshold,operator,enabled,duration_seconds)
VALUES ('a2','CPU >90%','cpu_percent',90,'gt',1,60);
INSERT INTO monitoring_alerts (id,name,metric,threshold,operator,enabled,duration_seconds)
VALUES ('a3','RAM >85%','mem_used_percent',85,'gt',1,120);
"
systemctl restart your-app

# Backup crontab (root):
# 0 2 * * * cp /opt/your-app/data/your-app.db /backup/your-app-$(date +\%Y\%m\%d).db
# 0 3 * * * find /backup/ -name "your-app-*.db" -mtime +90 -delete

Layer 3 — VPS provider

# cntb CLI
curl -sL "$(curl -sL https://api.github.com/repos/contabo/cntb/releases/latest \
  | grep browser_download_url.*linux_amd64.tar.gz | head -1 | cut -d'"' -f4)" \
  | tar xz -C /usr/local/bin

# Snapshot script → /opt/your-app/scripts/contabo-snapshot.sh (reads from /opt/your-app/.env)
# Credentials as env vars in /opt/your-app/.env (deployed by GitHub Actions):
#   CONTABO_CLIENT_ID, CONTABO_CLIENT_SECRET, CONTABO_API_USER, CONTABO_API_PASSWORD
# Crontab: 0 4 1 * * /opt/your-app/scripts/contabo-snapshot.sh

# > HARD GATE — Snapshot cron silently fails until env vars are set in .env.
# Credentials source: your VPS provider Customer Panel → API Details.
# Local dev: add to .envrc. Production: GitHub Secrets → deploy → /opt/your-app/.env

CRITICAL GOTCHAS

1. Shell escaping in Orca terminals: $VAR, $(…), and % get eaten by the local shell. Always use base64: echo '<base64>' | base64 -d > script.sh
2. Crontab %: cron interprets % as newline. Escape as \% in $(date +\%Y\%m\%d)
3. fail2ban exit 255: means a jail references a missing log file. Remove the broken jail, restart.
4. your application alerts need auth: POST to /api/monitoring/alerts requires Bearer token. Workaround: insert directly into SQLite, then restart your application.

Verify all 8 gates

ufw status|grep -q active||echo FAIL:ufw
fail2ban-client status sshd>/dev/null 2>&1||echo FAIL:fail2ban
systemctl is-active unattended-upgrades|grep -q active||echo FAIL:unattended
sshd -T|grep -q 'permitrootlogin no'||echo FAIL:sshd
systemctl show your-app -p NoNewPrivileges|grep -q yes||echo FAIL:systemd
systemctl is-active your-app|grep -q active||echo FAIL:your-app
sqlite3 /opt/your-app/data/your-app.db "SELECT count(*) FROM monitoring_alerts"|grep -q 3||echo FAIL:alerts
crontab -l|grep -q healthcheck&&crontab -l|grep -q your-app.db&&crontab -l|grep -q contabo-snapshot||echo FAIL:crontab
echo ALL 8 GATES PASSED

→ verify: # requires VPS SSH — run the 8-gate one-liner on the VPS manually