LiveSept 8 - Oct 28, 2026 · 128 commits
Fleet Sentinel
Uptime, security, updates and backups across your WordPress sites, in one panel.
I built Fleet Sentinel to look after a fleet of WordPress sites from one panel. It watches uptime, vulnerable plugins and themes, pending updates and the agent on each site. It also scans pages, finds broken links, tests whether a site's email arrives, records who changed what and takes encrypted backups. Alerts go to Google Chat or email, and the public demo runs the whole panel in your browser on sample data.
- passing Vitest tests
- 2,832
- recorded design decisions
- 77
- commits
- 128
- SQL migrations
- 34
- classified failure types
- 15
- scheduled background jobs
- 13
- permissions across 4 roles
- 37
- sample sites in the public demo
- 43
Architecture
How Fleet Sentinel is built
My role: Product owner and developer. I set the requirements, reviewed and tested the work, and deployed it, using AI-assisted development with Claude Code.
apps/panelThe Next.js 15 panel and its API: 105 route files, a Postgres schema of 48 tables and 34 SQL migrations.
apps/wp-pluginThe WordPress agent plugin: plain PHP, version 1.6.0, 30 classes, no Composer packages and one table of its own, for the activity log.
apps/checkerA Node worker with no dependencies that calls the panel's health sweep every 60 seconds.
apps/mail-workerA Cloudflare Email Worker that receives the email tester's messages and passes them to the panel with a signature.
packages/protocolShared TypeScript for request signing, key generation, AES-256-GCM encryption and the Zod schemas both sides validate against.
- Next.js 15
- React 19
- TypeScript
- Tailwind CSS
- Drizzle ORM
- Neon Postgres
- Auth.js v5
- Zod
- Recharts
- sharp
- Playwright
- axe-core
- Nodemailer
- libsodium
- PHP 7.4+
- Docker
- Cloudflare Workers
- Vercel Cron
- Vitest
The panel
Seven screens from the live demo
The demo is the production panel with its backend switched off. Every number in these pictures is sample data: 43 invented sites in three fleets.
(opens the full-size screenshot in a new tab)
(opens the full-size screenshot in a new tab)Site scans: one page, or a designed scan of many pages with its AI cost shown.
(opens the full-size screenshot in a new tab)Crawl: robots.txt and sitemap health, and a 404 scan that knows which pages used to work.
(opens the full-size screenshot in a new tab)Email test: one message sent through the site's own mail setup, scored, with a checklist of fixes.
(opens the full-size screenshot in a new tab)Backups: what the host allows, the pinned fleet key and an encrypted backup on demand.
(opens the full-size screenshot in a new tab)Activity: who changed what on every site, how it came in and which plugin made it.
(opens the full-size screenshot in a new tab)Diagnostics: the panel's hourly check of itself and the fleet, with each problem explained.
Product
Fleet monitoring dashboard features
Fleet dashboard
- Sites at risk, critical CVEs, pending updates, sites down and silent agents.
- Uptime and severity trend charts, and a prioritized “needs attention” list.
Site directory & detail
- Search, status filter, pagination, bulk actions, mute alerts, sync now, and add/delete with pairing keys.
- Twelve tabs per site, from plugins and findings to email tests, activity and backups.
- Notes and an audit trail of who changed which plugin.
Vulnerabilities
- Ingests the Wordfence Intelligence v3 feed (free or Pro key), streaming a catalog of over 117 MB.
- Matches it against every installed plugin and theme, and resolves or reopens findings as versions change.
- A rate-limit gate, error diagnosis, and key testing with auto-save.
Updates & findings
- Pending plugin/theme updates grouped across the fleet, labelled correctly.
- Automatic findings: vulnerabilities, updates, core updates, PHP end-of-life, TLS expiry, site down, agent lost.
- TLS expiry is read from the socket: a warning at 21 days and a critical finding at 7.
- Any finding can be assigned to a team member who can see the site.
Uptime monitoring
- A built-in prober that starts with a HEAD request and escalates to a browser-style GET.
- It recognises a WordPress failure page served with a 200, which a status-code check would pass.
- Two failed sweeps open an incident. Incidents carry durations, and uptime is rolled up per day for 30 days.
- UptimeRobot and BetterStack integrations with an inbound webhook.
Alerts in Google Chat and email
- Destinations are Google Chat spaces or email lists, with per-site or fleet-wide routing rules.
- 24 event types, a minimum severity per rule, quiet hours, per-site mute and a delivery log.
- Daily and weekly summaries at 8 AM in each destination's time zone.
- Email goes out through Resend or any SMTP server, set up by the superuser.
Multi-tenant fleets
- Fleets are separate tenants with per-fleet membership and roles; access can be restricted to chosen sites.
- A fleet switcher that stays in sync across browser tabs.
- Hardened fleet icon upload: content-sniffed type check, no SVG, decompression-bomb protection, re-encoded to WebP up to 512×512.
Access control & settings
- Viewer, editor, admin and superuser roles, sharing 37 permissions between them.
- Sign-in is by invitation, sent by email. A matching email domain grants nothing by itself.
- Removing someone, or a change of superuser, takes effect on their next request.
Background jobs
- 13 scheduled jobs. Health checks, inventory, crawls, scans, backups and mail tests tick every minute.
- Vulnerability evaluation every 15 minutes, a wordpress.org refresh every 6 hours and self-diagnosis every hour.
- No queue service: jobs lease rows in Postgres with SKIP LOCKED.
Beyond monitoring
Site scans, broken links, email, activity and backups
Six tools added in October 2026, each with its own tab or page and its own permissions.
Site scans
- Scan one page, or design a scan of up to 500 sitemap pages and repeat it daily or weekly.
- Eight focus areas, from SEO and accessibility to the security surface and WordPress hygiene.
- Rules and PageSpeed run first. Jev decides whether the copy needs a review, and only then does Claude read it.
- The dialog prices a scan before it runs, against each fleet's own AI keys and daily allowance.
Broken links & sitemaps
- Checks robots.txt and the sitemap, then the pages it lists and the internal links on them.
- A page that used to work and now fails is a regression and sends an alert. A link that never worked is recorded quietly.
- A failure counts only after a recheck at least ten minutes later.
- The plugin also reports the paths visitors get a 404 on, and the panel rechecks them itself.
Email delivery tests
- Sends one fixed message through the site's own wp_mail() to a single-use address.
- Checks SPF, DKIM, DMARC alignment, reverse DNS, three blocklists and an rspamd spam score.
- Gives a verdict, a score out of 100 and fixes to copy. Two failures in a row send an alert.
Activity log
- Sign-ins, users and roles, plugins, themes, settings, content and WooCommerce order statuses.
- Each event says who, how the request came in and which plugin or theme's code made the change.
- Search, filters, CSV or JSON export, and alerts for new administrators, installs and brute-force bursts.
- IP addresses are anonymised unless the fleet chooses otherwise.
Encrypted backups
- Opt-in: a site administrator enrols the site in wp-admin and compares the fleet key's fingerprint.
- The site encrypts each backup with libsodium before it leaves, so a destination holds data it cannot read.
- Copies go to panel storage, S3-compatible storage, SFTP, FTP/FTPS or Google Drive.
- A stepped job built for shared hosting resumes after PHP is killed mid-run.
Self-diagnosis
- Every hour, 11 platform checks and 15 fleet checks: late jobs, failing alert destinations, broken keys, backups and mail.
- Each problem says what it means and what to do, with AI explanations when the fleet has keys.
- Safe fixes, such as re-running a job, are offered only when the problem lists them, and are audited.
Reliability
Agent connectivity and data collection
Shared hosting is unreliable, so I redesigned the panel to pull data from the sites instead of waiting for heartbeats to arrive.
The first version had each site push a heartbeat to the panel. Many hosts disable, throttle or block wp-cron and outbound HTTP, so a healthy site would go quiet and the panel would raise a false “agent lost” alert.
Now the panel asks. It pings every site once a minute and fetches the full inventory only when the inventory hash or the event counter has moved, so a quiet site costs one small request. The only thing an agent sends on its own is a signed notice that says something changed, with no content in it. The design borrows from how ManageWP and MainWP connect to sites, from Stripe's thin events and from the φ accrual failure detector.
Transport
- The panel tries several transports in turn (a signed RPC endpoint, then admin-ajax, then REST variants) and remembers which one works.
- Every request is signed with HMAC-SHA256, with clock-skew correction and jittered retries.
- admin-ajax is always among the transports tried, so a site left in maintenance mode can still be reached.
Failure detection
- Failures are sorted into 15 types (blocked, timeout, TLS, clock skew, a different panel and more).
- A φ accrual failure detector (Hayashibara et al.) decides when an agent is unreachable, never sooner than 4 minutes and never later than 20.
- If most sites fail at once, the panel assumes its own network is the problem and holds alerts.
Pairing & moves
- Since plugin 1.6.0, a pairing is tied to the panel's address and the site's own.
- A plugin whose saved panel has gone moves to the new one by itself after 30 minutes, or with one click.
- Instant change notices, a goodbye notice when the plugin is deactivated and an “Agent reconnected” all-clear.
Security
API security and request signing
Request authenticity
- Requests between panel and agent are signed over the method, path, timestamp, nonce and body hash, carried in five headers.
- A 300-second clock window, single-use nonces, a timing-safe comparison and rate limits.
Secrets
- Site secrets, Chat webhook URLs and each fleet's AI keys are encrypted with AES-256-GCM on the panel.
- The agent encrypts its own secret with a key derived from the site's AUTH_KEY.
- Backup downloads and the recovery-key export need a sign-in within the last 15 minutes.
Sign-in and sessions
- Google sign-in only, with a verified email. The account and its fleets are read again on every request.
- The middleware denies by default, and every page carries a Content-Security-Policy.
Tenancy & roles
- Every tenant query is scoped to its fleet, and audit logs are scoped per fleet.
- A site in another fleet returns the same 404 as a site that does not exist.
- Platform-wide actions and log deletion are superuser-only, so fleet admins can't erase the audit trail.
Outbound requests
- Uptime probes, crawls, scans and agent pulls go through a guard that refuses private addresses and checks every redirect.
- The scan's browser runs in its own container, whose only way out is a proxy that applies the same rules.
Demo mode
A public demo without a backend
The public demo is the production panel built with one flag. Nothing you change there reaches a server.
In demo mode the page replaces the browser's fetch, so every call the panel makes to its own API is answered on the spot. The handler, about 8,400 lines, applies the same validation and role checks as the real API, which is why the forms reject what the real ones reject.
State lives in sessionStorage and resets when the tab closes. A clock keeps simulating health checks while you look around, and a “View as” control switches between the four roles, the quickest way to see what a viewer cannot do.
The server refuses everything else. Every /api path returns 404, any request that is not a GET or HEAD returns 405, and the database client throws if anything tries to use it.
Sample data
- 3 fleets and 43 sites on .example domains.
- 621 plugins and themes, 43 findings (26 open) and 12 incidents.
- Vulnerability ids are invented and prefixed DEMO-CVE.
Switched off
- Sign-in: the demo opens as a superuser.
- Webhook URLs are discarded, and API keys are kept only as a mask.
- Scans, crawls, email tests, backups and background jobs are simulated. The demo has no plugin download.
Quality
Automated tests and coverage
Automated
- 2,832 passing Vitest tests in 212 files (run on 28 October 2026), plus 17 end-to-end tests that run only when asked.
- Cross-language end-to-end tests run the real PHP plugin against the TypeScript panel.
- 21 PHP test scripts, including a backup job killed 50 times that must resume each time.
Reviews
- Three reviews wrote 315 edge-case tests. Each failing one proved a bug, and all 52 are fixed.
- Every page checked at 375, 768 and 1440 px: no sideways scroll on a phone, and no touch target under 24 px.
- An earlier pass tested every page as each role and found 8 bugs, including 3 security issues.
Bugs fixed along the way
- False “agent silent” alerts and missing recovery alerts.
- Buttons chosen from the platform role while the server checked the role in the active fleet.
- A removed member who stayed signed in, and redirects to private addresses that a probe followed.
Limits
Known limitations
These are the gaps a buyer would ask about first.
Not built
- It reports pending updates and cannot apply them.
- No client-facing or white-label reports.
- Backups can be verified and downloaded, but there is no one-click restore to the site.
- No two-factor sign-in of its own.
Operational
- Accessibility checks and phone screenshots need a separate browser service. Without it, those two scan areas are skipped.
- Email tests need a receiving subdomain on Cloudflare Email Routing.
- On Vercel's Hobby plan cron fires once a day, so minute-by-minute checks need the checker worker or the Docker scheduler.
DevOps
Production deployment
Vercel
- Monorepo deployed with a root vercel.json, cron configuration and maxDuration limits for the Hobby plan.
- Environment separation and secrets kept out of git; Google OAuth redirects configured.
- The public demo is the same build with demo mode on, served at fleetsentinel.hamzaahmadaslam.com.
Docker
- One image runs as the web server, the worker, the cron scheduler and the migration runner.
- The scan renderer, its egress proxy, rspamd and unbound run as separate containers. The renderer and rspamd have no route out.
Data & plugin
- Neon Postgres migrations applied in production through a migration script.
- The plugin zip is built from source on every deploy and offered in the panel. A build fails if its version numbers disagree.
Cost
- A pause switch takes the panel offline so an idle Neon database can sleep.
- Removed a scheduled GitHub Actions workflow that failed every run and would have burned billed minutes.
Growth
Technologies and skills used
- Full-stack TypeScript
- Next.js 15 App Router, React 19, server and client components, API route design.
- Databases
- PostgreSQL schema design, Drizzle ORM, migrations, SKIP LOCKED leasing, Neon serverless.
- Auth
- Auth.js, Google OAuth, JWT sessions, RBAC, multi-tenant isolation.
- Security
- HMAC signing, replay protection, AES-GCM, safe uploads, webhook hardening.
- Distributed systems
- φ accrual detection, pull vs. push, jittered retries, clock skew, caching.
- WordPress & PHP
- Plugin architecture, REST API, hooks and cron, signed endpoints, packaging.
- Integrations
- Google Chat, Wordfence Intelligence, UptimeRobot, BetterStack, wordpress.org.
- Testing
- Vitest, React Testing Library, cross-language E2E, PHP harnesses, role-based QA.
- DevOps
- Vercel deployment and cron, monorepo config, environments, production migrations.
- Engineering practice
- Research-led design, a decision log, documentation and AI-assisted development.