Add deployment hardening scripts
This commit is contained in:
Codex
@@ -35,6 +35,10 @@ This document records the explicit runtime requirements and server changes for P
|
||||
- `TSE_ATTACHMENT_ROOT`, default `attachments`.
|
||||
- `TSE_ATTACHMENT_MAX_KB`, default `10240`.
|
||||
- `TSE_ATTACHMENT_MIME_TYPES`, comma-separated allowlist.
|
||||
- Backup policy environment variables:
|
||||
- `TSE_BACKUP_DIR`, optional backup destination override.
|
||||
- `TSE_BACKUP_STORAGE`, default `1`, controls whether `storage/app` is archived with the database.
|
||||
- `TSE_BACKUP_BEFORE_UPDATE`, default `0`, runs `scripts/backup.sh` before `scripts/update.sh` when set to `1`.
|
||||
|
||||
## Local Test Requirements
|
||||
|
||||
@@ -53,6 +57,147 @@ This document records the explicit runtime requirements and server changes for P
|
||||
- Cron entry for `php artisan schedule:run`.
|
||||
- Shared/object storage before multi-server deployment.
|
||||
|
||||
## Queue Worker
|
||||
|
||||
The current ChristIT test deployment uses the database queue driver. That is acceptable for low-volume testing, but real notification, email, import, and image-processing work should run under a persistent worker.
|
||||
|
||||
Example systemd unit:
|
||||
|
||||
```ini
|
||||
[Unit]
|
||||
Description=Provider Exchange queue worker
|
||||
After=network.target
|
||||
|
||||
[Service]
|
||||
User=www-data
|
||||
Group=www-data
|
||||
Restart=always
|
||||
WorkingDirectory=/sites/provider-exchange
|
||||
ExecStart=/usr/bin/php artisan queue:work --sleep=3 --tries=3 --max-time=3600
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
```
|
||||
|
||||
Reload workers after deployment with:
|
||||
|
||||
```bash
|
||||
php artisan queue:restart
|
||||
```
|
||||
|
||||
## Scheduler
|
||||
|
||||
Laravel's scheduler should be invoked once per minute by cron:
|
||||
|
||||
```cron
|
||||
* * * * * cd /sites/provider-exchange && /usr/bin/php artisan schedule:run >> /dev/null 2>&1
|
||||
```
|
||||
|
||||
## PHP-FPM
|
||||
|
||||
Keep the app behind a dedicated PHP-FPM pool when production traffic increases. The pool should have:
|
||||
|
||||
- Its own Unix socket.
|
||||
- A deploy user/group that can read the app and write `storage/` and `bootstrap/cache/`.
|
||||
- `pm.max_children`, `pm.start_servers`, and spare server counts sized from observed memory usage.
|
||||
- `request_terminate_timeout` high enough for uploads, but not high enough to pin workers indefinitely.
|
||||
- `upload_max_filesize` and `post_max_size` aligned with `TSE_ATTACHMENT_MAX_KB`.
|
||||
|
||||
Do not change unrelated PHP-FPM sockets used by existing ChristIT sites when tuning this app.
|
||||
|
||||
## Redis Migration Plan
|
||||
|
||||
Before running multiple web nodes or high-volume alerting, move transient state out of the database:
|
||||
|
||||
```env
|
||||
CACHE_STORE=redis
|
||||
SESSION_DRIVER=redis
|
||||
QUEUE_CONNECTION=redis
|
||||
REDIS_CLIENT=phpredis
|
||||
REDIS_HOST=127.0.0.1
|
||||
REDIS_PORT=6379
|
||||
```
|
||||
|
||||
After changing these values, run:
|
||||
|
||||
```bash
|
||||
php artisan optimize:clear
|
||||
php artisan queue:restart
|
||||
```
|
||||
|
||||
Database-backed notifications can stay in the database because they are user-visible records. Queue transport, sessions, and cache should move first.
|
||||
|
||||
## Backups
|
||||
|
||||
Run an on-demand backup from the app directory:
|
||||
|
||||
```bash
|
||||
bash scripts/backup.sh
|
||||
```
|
||||
|
||||
The script supports SQLite, MySQL/MariaDB, and PostgreSQL based on `.env`. It writes a manifest beside the database backup. By default it also archives `storage/app`.
|
||||
|
||||
Run an update with a preflight backup:
|
||||
|
||||
```bash
|
||||
TSE_BACKUP_BEFORE_UPDATE=1 bash scripts/update.sh
|
||||
```
|
||||
|
||||
Recommended production posture:
|
||||
|
||||
- Store backups outside the app directory with `TSE_BACKUP_DIR`.
|
||||
- Replicate backups off-server.
|
||||
- Test restore steps before relying on the backup plan.
|
||||
- Keep evidence attachments and profile uploads under a retention policy that matches the platform's dispute and audit needs.
|
||||
|
||||
## Health Check
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
bash scripts/health-check.sh
|
||||
```
|
||||
|
||||
The check verifies Laravel boot, database/migration reachability, API and notification routes, writable cache/storage directories, storage symlink, and built Vite assets.
|
||||
|
||||
## Log Rotation
|
||||
|
||||
Laravel writes logs under `storage/logs`. Add a logrotate rule before real traffic:
|
||||
|
||||
```conf
|
||||
/sites/provider-exchange/storage/logs/*.log {
|
||||
daily
|
||||
missingok
|
||||
rotate 14
|
||||
compress
|
||||
delaycompress
|
||||
notifempty
|
||||
create 0640 www-data www-data
|
||||
}
|
||||
```
|
||||
|
||||
## Storage Scaling
|
||||
|
||||
Local public storage is acceptable for the first ChristIT test deployment. Before multi-server deployment:
|
||||
|
||||
- Move uploads/evidence to shared or object storage.
|
||||
- Keep private evidence behind controller authorization instead of direct public URLs.
|
||||
- Add malware scanning for uploaded evidence.
|
||||
- Add thumbnail generation for large images.
|
||||
- Add quota policy by user and work order.
|
||||
- Keep attachment disk/path metadata stable so old evidence records continue to resolve after storage migration.
|
||||
|
||||
## Database Scaling
|
||||
|
||||
Phase 14A added indexes for high-use filters and timelines: jobs, profiles, quotes, work orders, messages, reviews, ratings, disputes, attachments, external imports, and notifications.
|
||||
|
||||
Next database reliability steps after real usage begins:
|
||||
|
||||
- Add cached reputation aggregates based on measured slow pages.
|
||||
- Add query monitoring.
|
||||
- Move from SQLite to MariaDB/MySQL or PostgreSQL before treating data as production-critical.
|
||||
- Keep migrations additive where possible and back up before every schema-changing deployment.
|
||||
|
||||
## ChristIT Test Deployment
|
||||
|
||||
Target URL:
|
||||
|
||||
+21
-11
@@ -30,17 +30,17 @@ The goal is not to clone Field Nation. The goal is to keep the useful operationa
|
||||
### Current Weak Spots In Code
|
||||
|
||||
- Job posting now has structured scope and support certification fields, but the creation UI should be refined into reusable templates and stronger validation before real marketplace use.
|
||||
- Provider and buyer directories use text-based filters instead of normalized category, skill, coverage, remote, rating, and reliability filters.
|
||||
- Work-order lists are card-based and lack the dense assigned/available-work table that worked well in Field Nation.
|
||||
- Work-order detail now supports contact/support failure logging, first-class change requests, and post-work provider tag verification; schedule update requests, running-late records, and general report-problem records still need dedicated workflows.
|
||||
- Provider and buyer directories have better filters than the prototype baseline, but still need saved searches, stronger availability signals, and deeper reliability filters.
|
||||
- Work-order lists still need a UX pass for dense assigned-work tables and mobile-safe responsive alternatives.
|
||||
- Work-order detail now supports contact/support failure logging, first-class change requests, post-work provider tag verification, running-late records, and schedule-update requests; the remaining need is a cleaner general report-problem workflow.
|
||||
- Disputes and votes now have reason codes, but quorum rules, visibility rules, and deeper evidence timelines still need a future pass.
|
||||
- Reviews exist, but there is no review response, report flow, edit window, moderation queue, category definitions page, or imported/native reputation distinction in the UI.
|
||||
- Universal ratings are flexible, but category names are arbitrary strings and not governed by a taxonomy.
|
||||
- External profile imports store summary numbers and notes, but not structured work-category history, imported endorsement categories, imported review snapshots with privacy controls, or imported-vs-native display rules.
|
||||
- Reviews now have category definitions, reviewee responses, report flow, edit windows, moderation queues, and imported/native reputation separation; the remaining reputation work is mostly aggregate performance and UI polish.
|
||||
- Universal ratings are flexible, but category names still need stronger governance where they are not backed by explicit review categories.
|
||||
- External profile imports now support structured work-category history, imported endorsements, imported review excerpts, visibility controls, proof attachments, and verification status; the remaining risk is better privacy copy and verification workflow polish.
|
||||
- Profile services, tools, certifications, hiring policies, and locations are JSON arrays entered through newline text boxes. That is acceptable for prototype speed, but not durable enough for serious filtering.
|
||||
- Notification preferences exist as a model, but the UI does not expose channel and event preferences yet.
|
||||
- API endpoints are read-heavy starters and do not yet map to mobile-safe work-order actions.
|
||||
- Admin is only a status overview, not a moderation or operations console.
|
||||
- Notification preferences now expose channel intent, categories, event gates, digest, and quiet hours, but email/push delivery providers are intentionally not enabled yet.
|
||||
- API endpoints now cover mobile-safe work-order actions, but native app payloads should be refined once real app requirements are clearer.
|
||||
- Admin now includes moderation reports and audit logs, but still needs deeper operational filters, safety actions, and queue/system health panels.
|
||||
|
||||
## Revised Phases From Here
|
||||
|
||||
@@ -715,6 +715,16 @@ Provider-protection rules:
|
||||
- Private attachments must stay private.
|
||||
- Update/deployment failures should not corrupt active work orders.
|
||||
|
||||
Implementation status:
|
||||
|
||||
- Completed first implementation pass.
|
||||
- Added high-use database indexes for job filters, profile filters, quotes, work-order status/timelines, messages, ratings, reviews, disputes, attachments, imported-history queues, and notification inbox lookups.
|
||||
- Added `scripts/backup.sh` for SQLite, MySQL/MariaDB, and PostgreSQL backups with optional `storage/app` archive support.
|
||||
- Added `scripts/health-check.sh` for deployment smoke checks.
|
||||
- Added optional `TSE_BACKUP_BEFORE_UPDATE=1` pre-update backup support to `scripts/update.sh`.
|
||||
- Expanded deployment docs for queue workers, scheduler, PHP-FPM pool expectations, Redis migration, backup/export workflow, health checks, log rotation, storage scaling, and database scaling.
|
||||
- Deferred cached rating/reputation aggregates until real usage identifies slow pages and expensive queries.
|
||||
|
||||
## Phase 15 - UX Polish And Accessibility Pass
|
||||
|
||||
Purpose:
|
||||
@@ -751,8 +761,8 @@ Provider-protection rules:
|
||||
|
||||
## Recommended Immediate Sequence
|
||||
|
||||
1. Phase 14A: deployment/scaling hardening.
|
||||
2. Phase 15A: UX polish and accessibility pass.
|
||||
1. Phase 15A: UX polish and accessibility pass.
|
||||
2. Cached rating/reputation aggregates after slow pages are measured.
|
||||
3. Future mobile/API expansion after native app requirements are clearer.
|
||||
4. Future email/push delivery implementation after the sender and push providers are explicitly selected.
|
||||
|
||||
|
||||
@@ -22,6 +22,12 @@ From the app directory:
|
||||
bash scripts/update.sh
|
||||
```
|
||||
|
||||
To create a database/storage backup immediately before updating:
|
||||
|
||||
```bash
|
||||
TSE_BACKUP_BEFORE_UPDATE=1 bash scripts/update.sh
|
||||
```
|
||||
|
||||
Or from anywhere:
|
||||
|
||||
```bash
|
||||
@@ -35,12 +41,29 @@ The preferred working loop for this project is:
|
||||
3. SSH to the server and run `bash scripts/update.sh` from the deployed Git checkout.
|
||||
4. Smoke test the live URL after the update completes.
|
||||
|
||||
## Backups And Health Checks
|
||||
|
||||
Create an on-demand backup from the app directory:
|
||||
|
||||
```bash
|
||||
bash scripts/backup.sh
|
||||
```
|
||||
|
||||
Backups default to `storage/app/backups`. Set `TSE_BACKUP_DIR=/path/to/backups` to store them elsewhere. Set `TSE_BACKUP_STORAGE=0` to back up only the database.
|
||||
|
||||
Run a deployment health check from the app directory:
|
||||
|
||||
```bash
|
||||
bash scripts/health-check.sh
|
||||
```
|
||||
|
||||
## Important Notes
|
||||
|
||||
- The installer is a first-pass Ubuntu/Nginx helper, not a universal hosting panel installer.
|
||||
- Local and CI test runners need PHP SQLite and upload-test support: `pdo_sqlite`, `sqlite3`, `fileinfo`, and `gd`.
|
||||
- Public attachment serving needs `php artisan storage:link`; the checked-in update script runs this automatically.
|
||||
- Tune upload policy with `TSE_ATTACHMENT_DISK`, `TSE_ATTACHMENT_ROOT`, `TSE_ATTACHMENT_MAX_KB`, and `TSE_ATTACHMENT_MIME_TYPES`.
|
||||
- Tune backup policy with `TSE_BACKUP_DIR`, `TSE_BACKUP_STORAGE`, and optional `TSE_BACKUP_BEFORE_UPDATE=1`.
|
||||
- Review `.env` before production use.
|
||||
- Configure real mail before inviting users.
|
||||
- Move sessions, cache, and queues to Redis before serious traffic.
|
||||
|
||||
+14
-5
@@ -61,11 +61,11 @@ Provider Exchange is a Laravel application for a provider-centered field-service
|
||||
|
||||
## Phase Checkpoint
|
||||
|
||||
Phases 1-13A have been implemented locally through notification preferences, moderation reporting, and audit logging. The next active planning block is:
|
||||
Phases 1-14A have been implemented locally through deployment/scaling hardening. The next active planning block is:
|
||||
|
||||
8. Deployment/scaling hardening.
|
||||
9. UX/accessibility polish.
|
||||
10. Future email/push delivery and native-app API expansion once providers are selected.
|
||||
1. UX/accessibility polish.
|
||||
2. Cached reputation aggregates after real slow pages are measured.
|
||||
3. Future email/push delivery and native-app API expansion once providers are selected.
|
||||
|
||||
Work-order safeguards to carry into the next implementation passes:
|
||||
|
||||
@@ -157,6 +157,15 @@ Phase 12A/13A notification and admin governance completed:
|
||||
- Audit logs record selected operational and moderation events, including review moderation, imported-history verification, attachment deletion, work-order actions, quote acceptance, and mobile API actions.
|
||||
- Admins can review recent audit activity from the admin console.
|
||||
|
||||
Phase 14A deployment/scaling hardening completed:
|
||||
|
||||
- Added database indexes for job-board filters, profile filters, quotes, work-order timelines, reviews, ratings, disputes, attachments, imported-history queues, and notification inbox lookups.
|
||||
- Added `scripts/backup.sh` for SQLite, MySQL/MariaDB, and PostgreSQL database backups with optional `storage/app` archive support.
|
||||
- Added `scripts/health-check.sh` for deployment smoke checks.
|
||||
- Added optional `TSE_BACKUP_BEFORE_UPDATE=1` pre-update backups to `scripts/update.sh`.
|
||||
- Documented queue workers, scheduler, PHP-FPM expectations, Redis migration, backups, health checks, log rotation, storage scaling, and database scaling.
|
||||
- Kept cached reputation aggregates deferred until real traffic shows which pages need them.
|
||||
|
||||
Available-work safeguards to carry into directory and job-list phases:
|
||||
|
||||
- Available jobs should expose pay type, provider terms, schedule type, work type, rough location, buyer reliability, support/contact certification, scope clarity status, and request/quote count.
|
||||
@@ -181,7 +190,7 @@ The reconciled forward plan from the current codebase is captured in `docs/FORWA
|
||||
- Work-order timelines, dispute activity, messages, and notification tables need careful indexes before large production use.
|
||||
- Realtime chat, websocket presence, mobile push, and geolocation check-ins should be added as deliberate service layers instead of bolted onto normal page requests.
|
||||
- Background imports, image processing, moderation summaries, and reputation recalculation should run through queue workers rather than web requests.
|
||||
- Production deployments need explicit PHP-FPM pool, Nginx, scheduler, queue worker, backup, and log rotation documentation.
|
||||
- Production deployments now have explicit PHP-FPM, Nginx, scheduler, queue worker, backup, health-check, and log rotation documentation. These should be revisited after real traffic establishes memory, queue, and slow-query patterns.
|
||||
|
||||
## Future/Deferred
|
||||
|
||||
|
||||
Reference in New Issue
Block a user