Async Dispatch Retries
3 min read
The automation engine dispatches all rule actions asynchronously through Action Scheduler with automatic retry on failure. This page covers the dispatch model, the retry policy, what happens when an action fails permanently, and how to monitor dispatcher health.
The Dispatch Model #
When a rule fires:
- The engine evaluates conditions in-process (fast, blocking)
- If conditions pass, each action becomes a queued Action Scheduler job
- The triggering event proceeds normally — no waiting
- Action Scheduler picks up the jobs on its next tick (usually within seconds)
- Each action’s handler runs the side effect (send email, fire webhook, etc.)
- Success or failure is logged
- Failed jobs are rescheduled per the retry policy
This separation means: a customer’s checkout completes immediately even if a downstream webhook receiver is slow. Actions don’t slow down customer-facing flows.
The Retry Policy #
| Attempt | Delay From Original Trigger |
|---|---|
| 1 (initial) | Immediate (within seconds of trigger) |
| 2 (retry 1) | + 60 seconds |
| 3 (retry 2) | + 120 seconds (180s from original) |
| 4 (retry 3) | + 240 seconds (420s from original) |
| Stop | After 4 total attempts, action is marked failed |
The backoff is exponential-ish (60 → 120 → 240). This catches most transient failures — a 2-minute Slack outage will resolve before retry 3.
What Counts as a Failure #
| Action | Failure Conditions |
|---|---|
| Block / Allowlist / Tag customer | Database error (very rare) |
| Hold / Cancel order | Order not found, status transition rejected by WooCommerce |
| Send email | wp_mail() returns false, or SMTP plugin reports delivery failure |
| Fire webhook | HTTP 4xx/5xx response, timeout (10s), DNS failure, connection refused |
Non-2xx HTTP responses are treated as failure. A 200 with an error body is treated as success — TrustLens only checks the status code.
Permanent Failure #
After 4 total attempts (initial + 3 retries), the action is marked failed and stops retrying. The failure is recorded in the Automation Log with the final attempt’s error.
Permanent failures don’t automatically alert. To get notified when actions fail repeatedly:
- Review the Automation Log weekly
- Build a meta-rule that watches for failed actions and alerts
- Pro: enable the “Action Failure Notification” setting under Reports
Action Scheduler Health #
The dispatcher depends on Action Scheduler (bundled with WooCommerce) running its scheduled actions reliably. If Action Scheduler stalls:
- Queued actions sit in the queue but never execute
- Retries never happen
- The Automation Log shows queued actions without completion events
To check Action Scheduler health: WooCommerce → Status → Scheduled Actions. Filter by status:
- Pending — waiting to run; should drain within minutes on a healthy site
- In progress — actively executing; should clear quickly
- Complete — finished successfully
- Failed — exception thrown during execution
If pending count is climbing without draining, WP-Cron is probably disabled or your host is rate-limiting. Set up a real cron job hitting wp-cron.php every minute.
Deduplication #
The dispatcher deduplicates jobs at queue time. If the same trigger fires multiple actions on the same target within a few seconds, the dispatcher may collapse duplicate jobs to prevent double-firing.
This dedup is action-aware — multiple distinct actions (block + email + webhook) on the same target each get one job, but a duplicate “send email” job for the exact same target/context within milliseconds is treated as one.
Cooldowns vs Retries #
Two related but distinct concepts:
- Cooldown — minimum time between firings of a rule on the same target. Set at the rule level. Default: none.
- Retry — automatic re-attempt of a failed action. Set at the engine level (60/120/240). Not configurable per rule.
A rule with cooldown = 1 hour fires at most once per hour per target. A rule with no cooldown fires every time the trigger event matches. Both behaviors are independent of the retry policy.
Webhook-Specific Retry Behavior #
For webhook actions, retry honors HTTP semantics:
- 5xx responses: retry (server probably temporary issue)
- 4xx responses: retry (might be a transient routing issue)
- Timeouts: retry
- 2xx response: success, no retry
Some webhook receivers signal “don’t retry” with specific status codes (some APIs use 410 Gone). TrustLens doesn’t special-case any 4xx — all retry the same way. If you have a receiver that genuinely shouldn’t retry, return 2xx and ignore the payload instead of erroring.
Monitoring Dispatch Health #
Periodically check:
- Action Scheduler queue depth — anything chronically large indicates the dispatcher is falling behind
- Automation Log failure rate — climbing failures suggest an upstream issue (slow webhook receiver, broken email config)
- Average action latency — visible in the log; should be sub-second for most actions, sub-10s for webhooks
If you’re running 100+ rule fires per day, build a habit of reviewing dispatch health weekly. It’s the kind of system that’s silent when working correctly and silent when broken — the failure mode is usually “nothing is happening anymore.”
Synchronous Execution (Not Supported) #
The engine doesn’t support synchronous action execution. All actions are async. If you have a use case that genuinely requires synchronous side effects (e.g. modifying an order before WooCommerce processes it further), don’t use the automation engine — write a custom WooCommerce action hook handler directly.
The trade-off: async dispatch is much safer (no risk of slowing checkout, no risk of taking down the site if a receiver is slow) but means actions are eventually consistent, not immediately consistent.
