Fixplatform-runtimes-executorPlatform Runtimes Executor

0.25.3-rvnxx.3

Fork-only change on upstream 0.25.3.

Don't crash the executor when a reaped container is already gone

Maintenance.php called orchestration->remove() unguarded inside a Swoole\Coroutine\batch() job. When the Docker Engine answered 404 No such container, the exception escaped the coroutine — which is fatal in Swoole, not a caught request error. The whole executor process died with exit 255, taking every in-flight execution on that node with it.

Measured in production before the fix: six crashes on 2026-07-24 across worker-1/4/5, each costing 10.6–11.3s of downtime for that replica. OPR_EXECUTOR_INACTIVE_THRESHOLD=86400 with OPR_EXECUTOR_MAINTENANCE_INTERVAL=3600 made it roughly a daily event per replica.

The stale entry arises because the runtime record is process-local Swoole-table state that a restart wipes while the containers survive, so the reaper can chase a container that is no longer there.

Changed

  • OrchestrationErrors::isContainerNotFound() — deliberately narrow: only an Orchestration exception carrying No such container. A 409 "removal already in progress", a driver 500 or an unreachable daemon all remain errors.
  • Maintenance::removeContainer() treats that one case as success, logs anything else and returns false instead of aborting the tick.
  • The maintenance timer callback is wrapped, so no tick can take down the server.
  • Docker::deleteRuntime() made idempotent.

Not included

The executor crashes were refuted as the main cause of the intermittent gateway 5xx on payments/forms: ~66 replica-seconds of downtime out of 518,400 is ~0.013%, which cannot produce the observed failure rate. That turned out to be the gateway's per-attempt timeout amplifying cold starts, handled separately in revenexx/api#34.

Also observed but not fixed: revenexx-tag-* runtime names are 65 bytes against Swoole's 63-byte table-key cap (Runtimes.php:75). Correlated with the crashing runtimes, unproven as causal.