Why the pursuit of efficiency can sometimes backfire
Businesses are under constant pressure to move faster, lower costs and do more with less. It’s no surprise that business process automation is often positioned as the answer: remove manual steps, shorten turnaround times and let technology handle repetitive work.
But automation creates value only when the process being accelerated is sound. Automate a weak process, poor decision or faulty assumption and the result isn’t simply the same problem at greater speed – it can become a much bigger one.
That’s the hidden cost of over-automation. When the goal shifts from improving outcomes to removing people, organisations risk stripping away the judgement, context and accountability that keep complex systems safe.
The better principle is simple: automation should amplify human capability, not replace human judgement.
Why the best automation strategies augment people, not replace them
Successful automation removes repetitive, predictable work while keeping people involved where decisions require context, empathy, accountability or specialist expertise.
Human-centred automation asks a better question than “How many people can we take out of this process?” It asks, “Where can technology reduce effort, improve consistency and give people better information to make stronger decisions?”
That distinction matters across everything from customer service automation to financial approvals and software delivery. Machines can route, calculate, flag and recommend. People should still be able to interrogate the result, handle an exception and intervene when the situation falls outside the rules.
Why the ‘automate everything’ mindset creates unintended consequences
Automation programmes are often measured by manual steps removed or costs saved. Useful metrics – but incomplete ones.
Not all manual work is waste. A checkpoint may exist because somebody needs to notice that a case is unusual, ask a question or exercise judgement. Remove it without understanding its purpose and efficiency can come at the expense of decision quality, resilience and control.
This is particularly visible in customer journey automation. A journey may look efficient on a process map while feeling deeply inefficient to a customer who can’t correct a mistake, explain an unusual circumstance or reach a person when the workflow stops making sense.
The hidden costs of over-automation
Many high-profile automation failures show what happens when assumptions aren’t tested, controls are too weak or systems are trusted too quickly.
In 2012, Knight Capital incorrectly deployed software to a trading system. Automated activity then sent more than four million orders into the market in around 45 minutes, ultimately contributing to losses of more than US$460 million. The US Securities and Exchange Commission later found that Knight lacked adequate controls and procedures that could have limited the risks created by its automated systems.
GitLab’s 2017 database incident tells a different version of the same story. An engineer accidentally deleted production data while troubleshooting, but several backup and replication mechanisms also failed to provide the recovery capability the team expected. GitLab ultimately reported losing around six hours of production database data.
Investigations into the Boeing 737 MAX’s MCAS flight-control system likewise identified shortcomings in safety assessments and assumptions surrounding the automated system and its interaction with pilots. The FAA-commissioned Joint Authorities Technical Review found that assumptions about crew response influenced the design and certification of MCAS and that the operational environment faced by pilots may not have been adequately anticipated during certification.
The lesson isn’t that automation is inherently unsafe. It’s that automation becomes dangerous when its authority, failure modes and human interaction aren’t understood well enough.
Why human judgement still matters in the age of AI
Rules are relatively easy to automate. Judgement isn’t.
People remain essential when decisions depend on incomplete information, competing priorities, ethical or regulatory considerations, customer circumstances or unusual edge cases. They can recognise when the available inputs don’t tell the whole story.
This becomes even more important as AI enters automated workflows. AI-generated recommendations and code can arrive quickly and confidently, but confidence isn’t correctness. AI can’t fully understand an organisation’s business context, risk appetite or accountability obligations.
That makes governance, validation and human review more important, not less.
The power of human-in-the-loop automation
A human in the loop model creates a deliberate division of labour: machines handle high-volume, repeatable work; people handle exceptions and consequential decisions.
That might mean automating document classification but escalating low-confidence cases, using AI to recommend an action while requiring human approval, or allowing customer experience automation to resolve routine queries while making escalation to a person quick and obvious.
The point isn’t to add unnecessary friction. It’s to put human judgement where the consequences justify it.
What great automation looks like in practice
Resilient automation is designed with failure in mind.
SpaceX’s Falcon 9 provides a useful engineering example. NASA describes the integrated static-fire test performed ahead of launch as a “routine but critical” milestone. SpaceX’s Falcon documentation also describes safety measures including greater redundancy, rigorous fault mitigation and a fault-tolerant avionics architecture.
The underlying principle is important: automation earns trust through validation and safeguards, not simply because it performs correctly under normal conditions.
Netflix applies a similar idea through Chaos Monkey and its wider chaos-engineering practices. Rather than assuming infrastructure will recover when something fails, Netflix deliberately introduces failures in production environments to expose weaknesses and encourage engineers to build services that remain resilient when instances fail.
Great automation, then, isn’t just efficient. It’s observable, testable, recoverable and resilient.
How organisations can automate responsibly
Start with the process, not the technology. Repetitive, rules-based and lower-risk work – reporting, routine administration, standard routing or repeatable testing – is a natural candidate for automation.
Keep people involved where the consequences are higher: financial approvals, security changes, customer-sensitive actions and high-risk operational decisions. Then engineer safeguards into the workflow through validation checks, approval gates, monitoring, rollback mechanisms and tested recovery procedures.
BBD’s packaged testing approach reflects this principle by combining manual and automated testing according to what’s being built and the level of risk involved – helping teams catch problems earlier and release with greater confidence.
How BBD approaches intelligent automation
BBD’s approach to software development and artificial intelligence starts with the business problem and the outcome required.
Automation can remove repetitive load, improve throughput and support better decisions, while engineering controls, testing and human oversight protect quality and reliability. BBD’s AI approach similarly emphasises governance, controls and experienced human review so that increased velocity doesn’t come at the expense of correctness.
Whether the goal is internal business process automation, customer service automation or a sophisticated AI-enabled workflow, the objective should be better business and customer outcomes – not automation for its own sake.
The warning signs of over-automation are often easy to spot: customers struggle to reach a human, teams routinely correct automated mistakes, exceptions repeatedly break workflows, decision quality falls, or nobody fully understands the process anymore. These are signals to revisit the design, controls and role of human judgement.
Watch the full talk
This article is inspired by BBD Platform Engineer Gaurav Choudhary’s Escape presentation, The hidden cost of automation: When doing less does more harm.
Gaurav explores examples from Knight Capital, GitLab, Boeing, Netflix and modern AI systems to demonstrate why automation works best when paired with strong validation, governance and human oversight.
For engineering leaders, technology teams and business decision-makers, the message is simple: the goal isn’t to automate everything. It’s to automate the right things, in the right way, with the right safeguards.
