How to tell whether an automation actually worked

You cannot measure what you did not baseline

The most common reason nobody can prove an automation worked is that nobody measured the process it replaced. Once it is gone, the old cost is a matter of recollection, and recollection is generous in whichever direction suits the speaker.

Baselining takes very little effort and has to happen before the build. How many times does this run a week, how long does it take, how often does it go wrong, and what does fixing it cost.

Rough numbers are fine. Precision matters far less than having recorded anything at all.

Hours saved is a weak metric on its own

Time returned is the usual headline and it is easy to inflate. Twenty minutes saved across ten people does not reliably become three hours of new output; frequently it becomes a slightly less pressured day, which is valuable but different.

Better measures are usually about outcomes. Errors caught before they reached a customer. Response time from request to first action. Jobs quoted per week. Percentage of accounts contacted within their threshold.

Those are harder to argue with because they connect to something the business already cares about.

Count what it costs to keep running

Automations are not free after launch. Integrations break, exceptions need handling, and somebody reviews the cases the system routes to a human.

An honest evaluation subtracts that. A system returning ten hours a week and consuming three in exception handling returns seven, and knowing the real figure is what lets you decide whether to invest in reducing the exception rate.

Systems evaluated only on their best case tend to get quietly abandoned when the maintenance becomes visible.

Set the review date at the start

Agree when you will look, before you build. Ninety days is usually right: long enough for novelty to wear off, short enough to act on.

Then actually look, including at the possibility that it did not work. Being able to switch something off is what makes it safe to try things, and a portfolio where nothing is ever retired is not a portfolio, it is an accumulation.