Blog Automation health
Automation health
Why Does Automation Stop Working After Launch, and Who Notices?
The demo worked. Weeks later an inquiry sits untouched and nobody got an alert. An automation can stop working without failing loudly. It finishes without an error, or never starts at all, and the first person to notice is a client.
We read Google's Site Reliability Engineering guidance on monitoring and paging, Google's OAuth documentation on when access stops working, Amazon's Builders' Library on retries and duplicate-safe requests, Stripe's webhook documentation, n8n's error handling documentation and Google Cloud's guide to evaluating AI output. The common thread: a technical success is one layer of proof, and some failures produce no error to catch. What we offer is an operating contract, five checks for "working", ten prepared test cases and a plan for who gets the signal.
4
golden signals Google SRE names for monitoring a user-facing system: latency, traffic, errors and saturation
6 months
without use is one documented reason a Google OAuth refresh token stops working (Google Identity)
10
single-fault test cases to run in a test copy, one fault at a time (Teamulate method)
The thesis
Do not call an automation working because the last run was green. Write down the result it promises, check that result in the place it should appear, watch for runs that never start, and name who acts when it fails.
The scene
The automation is green, and the inquiry is still waiting
A teaching example of a green run that leaves an inquiry waiting.
An automation routes new inquiries into the CRM and assigns an owner. It worked in the demo and for weeks after. Then a prospect follows up to ask why nobody replied. The run history shows no errors. The inquiry is in the inbox, but not in the CRM.
Nothing "failed". The scheduled run had been switched off during a small change and never switched back on. There was no failed run to report, so no alert went out.
How we looked
What the engineering guidance says
What the documentation says.
We read Monitoring Distributed Systems from the Google SRE book; Google Identity's OAuth 2.0 documentation; the Amazon Builders' Library articles on timeouts, retries and backoff with jitter and on making retries safe with idempotent APIs; Stripe's webhook documentation; n8n's error handling documentation; and Google Cloud's overview of evaluating generative AI. n8n and Stripe are documented examples, not recommendations.
The method below is built from that documentation.
The evidence
Why a working automation stops working
Access, events and retries all change after launch.
Access does not last forever. Google documents that a refresh token can stop working when the user revokes the app's access, when it has not been used for six months, when time-based access expires, or when an admin restricts a service. "We connected the account once" is not proof of access today.
Events do not always arrive once, or in order. Stripe says webhook endpoints might occasionally receive the same event more than once, and that it does not guarantee events are delivered in the order they were generated. An automation that assumes one clean event per action can create a duplicate.
Retries are not free. Amazon's engineers note that if errors are caused by load, retries can be ineffective when every client retries at the same time; they add jitter, cap the backoff and limit the number of retries. And a timeout does not prove the outside action failed: Amazon's answer is a request identifier that lets a repeated request be recognized, so a retry does not cause a second side effect.
Error handling has a blind spot. n8n documents that an error workflow runs when an execution fails, and that some error details are missing when the error is in the trigger node, because the workflow does not execute. A run that never started can slip past error alerts entirely.
The checks
Five checks for "working"
A green run covers only the second one.
What "working" should mean
| Check | Question |
|---|---|
| Input received | Did the expected event or record reach the process? |
| Processing done | Did the needed technical steps run? |
| Result correct | Does the result meet the agreed criteria? |
| Result in place | Is it in the right system, with the right owner? |
| On time | Did it finish while it still mattered? |
A Teamulate working method, based on the documentation listed below.
Google SRE separates white-box monitoring, which looks inside the system, from black-box monitoring, which is symptom-oriented: "the system isn't working correctly, right now." For a small business, the black-box question is simple: did the inquiry end up with an owner in the CRM? If the last check is out of date, the honest status is Unknown, not Working.
Symptoms
Three kinds of problem, three kinds of check
One "workflow failed" alert does not cover them.
What it looks like, and how you catch it
| Problem | How you catch it |
|---|---|
| Visible failure: the run returns an error | Error alerts from the tool |
| Silent miss: no error, work missing | Compare expected work with work done |
| Wrong result: the record is wrong | Data and quality checks on the result |
A Teamulate working method.
For a scheduled automation, check when it should have started, whether there is a start record and a confirmed result, and whether the allowed window has passed. For one triggered by incoming work, compare accepted items in the source with records in the target. Zero items is not proof of a failure; there may simply have been no demand.
The contract
Write down what the automation promises
Eight fields, one page.
- Trigger: what starts it and when it should run.
- Input: what it accepts and what it must reject.
- Output: what it produces and where that result appears.
- Proof: how you confirm the result for one ID.
- Time limit: how long is acceptable from input to result.
- Approvals: which actions need a person, such as sending.
- On failure: what happens to the input, and who is told.
- Exceptions: who handles what the automation cannot.
If the automation prepares a draft, success is "draft ready for review", not "reply sent". Keeping that difference visible avoids the worst kind of surprise.
Recovery
Retry safely, and resume from the right step
Endless retrying is not recovery.
Retry only errors that a retry can fix, with a limit on attempts and total time, spread out over time. A retry does not fix wrong permissions, a missing field, a wrong rule or a poor AI result. Give each business task one ID and reuse it on every retry. If the result is unknown and the outside system offers no protection, hand the task to a person.
Partial success needs its own plan. A process can create the CRM record, add the task and then fail to send the notice. Starting again from the top can create a second record. Log each step as confirmed, failed or unknown, and resume from the step that failed. Rolling back the automation does not undo what it already did.
Who notices
Name who gets the signal, and what it says
No responsibility left unnamed.
Name a business owner, a technical owner, a backup and a decision owner. In a small team one person can hold several roles. A useful signal says which automation, which result is missing, since when, how many tasks, what was already checked, whether any outside action happened, and the next safe step. Never put keys or tokens in shared alerts.
Not every hiccup deserves an interruption. Google SRE says every page should be actionable, and if a page merits only a robotic response, it should not be a page. A simple split works: urgent, respond within an agreed window, and for the record. Group repeated errors from one incident into one case.
AI output
AI quality is a separate check
A valid format can carry the wrong meaning.
AI can return the right fields with the wrong category, a readable summary that drops a deadline, or a draft with an old price. Check structure, grounding in permitted sources, business fit and whether the action is allowed. Google Cloud describes both computation-based metrics against a reference answer and rubric-based evaluation by a model; either way, keep a set of realistic examples and rerun them whenever the model, prompt or source changes.
An AI judge can support review, but its agreement is not independent proof that an answer is true. Critical cases need checkable rules or a person.
Testing
Ten test cases, one fault at a time
Write down what you expect before each run.
In a separate test copy with test data, start with one normal control run, then introduce one fault at a time: access removed, the run never starts, a field renamed, the same event twice, a timeout, partial success, a wrong AI label, the owner gone, a volume spike and a changed business rule. For each, write down what you expect to see before you run it.
Ten single tests do not cover every combination of failures.
Support
What a marketing team with AI support can keep running, and what stays with you
Start with the automation you trust most.
Start small: one automation, one operating contract, one check for the run that never started, and one fault test in a test copy. If a weekly look at the CRM answers the question, that is a good answer too.
What stays with you: deciding what a correct result is, how late is too late, and who may restart or change the rules.
For B2B teams, watching results, handling exceptions and recovering safely is part of the marketing operations our AI marketing team runs alongside you, with every step visible. You can see how our workflows run. To talk it through, request a demo.
Checklist
Checklist: before you call it running
Run it on one automation.
- The automation has a written operating contract.
- "Working" means the five checks, not a green run.
- There is a check for the run that never started.
- Retries are limited, spread out and reuse one task ID.
- Partial success has a log and a resume point.
- Owners, a backup and a decision owner are named.
- At least one fault test has been run in a test copy.
Key terms and facts
The short version
- Silent miss
- No error is reported, but the expected work is missing, for example a scheduled run that never started.
- Duplicate-safe request
- A request the receiving system recognizes and handles without repeating a side effect.
- Operating contract
- A short written note of what an automation takes in, what it must produce, where, how fast, and what happens on failure.
- Partial success
- Some steps finished and others did not, so starting again from the top can create duplicates.
Key facts
- TEAMULATE REVIEWGoogle SRE book, Google Identity, Amazon Builders' Library, Stripe, n8n and Google Cloud documentation, checked October 29, 2026.
- DOCUMENTEDGoogle SRE describes black-box monitoring as symptom-oriented and says every page should be actionable.
- DOCUMENTEDa Google OAuth refresh token can stop working when access is revoked, after six months unused, or when time-based access expires.
- DOCUMENTEDStripe says webhook endpoints might receive the same event more than once and does not guarantee event order.
- DOCUMENTEDan n8n error workflow runs when an execution fails.
- TEAMULATE METHODten single-fault test cases in a test copy.
Takeaways
What to do with this
Green is not done
A run without errors is one layer of proof, not the delivered result.
Watch for silence
A run that never started may produce no error. Compare expected work with work done.
Retry with an ID
Reuse one task ID on every retry so a repeat does not create a duplicate.
Name who acts
Every signal needs an owner, a backup and a next safe step.
Questions and answers
Common questions
My automation shows no errors. Is it working?
Not necessarily. A run without errors is one check. Confirm the result exists in the right place, with an owner, on time. A run that never started may produce no error at all.
Why did a connection that worked for months suddenly stop?
Access can lapse. Google documents that a refresh token can stop working when access is revoked, after six months unused, or when time-based access expires.
Should the automation just retry until it works?
No. Retry only errors a retry can fix, with limits, spread out over time, and with one task ID reused so a repeat does not create a duplicate. Then hand it to a person.
Why do I get duplicate records?
Some providers send the same event more than once. Stripe documents this for webhooks. Use one ID per business task and skip events you have already processed.
Who should get the alert?
A named business owner and technical owner, with a backup. The alert should say what is missing, since when, and the next safe step.
Sources
Named sources
Teamulate review of official documentation and engineering guidance, checked October 29, 2026:
- Google SRE book: Monitoring Distributed Systems: https://sre.google/sre-book/monitoring-distributed-systems/
- Google Identity: Using OAuth 2.0 to Access Google APIs: https://developers.google.com/identity/protocols/oauth2
- Amazon Builders' Library: Timeouts, retries, and backoff with jitter: https://aws.amazon.com/builders-library/timeouts-retries-and-backoff-with-jitter/
- Amazon Builders' Library: Making retries safe with idempotent APIs: https://aws.amazon.com/builders-library/making-retries-safe-with-idempotent-APIs/
- Stripe documentation: Receive Stripe events in your webhook endpoint: https://docs.stripe.com/webhooks
- n8n documentation: Error handling: https://docs.n8n.io/flow-logic/error-handling/
- Google Cloud: Gen AI evaluation service overview: https://cloud.google.com/vertex-ai/generative-ai/docs/models/evaluation-overview
Download the kit
Write down what one automation promises to deliver, check the result and not just the run, plan for the run that never starts, make retries safe, and name who gets the signal and what they do next.
Download The Automation Health Check Kit (PDF)