Extra Intelligence: Why the AI Conversation Needs Humans In It, Now

If we don't start talking about AI with the humans in our systems right now, including our leaders, our teams, our customers and our communities, we are going to find ourselves in a very big mess, very soon.

Not a robots-taking-over-the-world mess. Something much more ordinary, and much more likely. A mess of services that technically work but leave people stranded. Of customers who can't find a human to talk to. Of staff who were never asked what they actually needed. Of decisions made because a tool could do something, not because anyone stopped to ask whether it should.

The good news? We still have time to get this right. But the window for having these conversations is open now, and it won't stay open forever.

A confession: I'm a huge fan

Let me be clear before anyone mistakes this for a rant. I love AI. I use it every single day. It helps me organise the swirl of ideas in my head, draft and redraft, research faster, and turn months of observations into something useful before a meeting.

But over the last year or so, I've had to work out what role it actually plays in my work and, just as importantly, what role it doesn't.

What I've learned surprised me. The more I leaned on AI, the clearer it became that my own brain was the most valuable thing in the room. Not despite the way it works, but because of it.

Not Super Intelligence. Extra Intelligence.

Some of the loudest voices in tech and politics like to talk about super intelligence. I've decided I'm going to call it something else: Extra Intelligence.

Why? Because I'm pretty intelligent myself, thank you very much. The tools available to us now don't replace that. They give me extra: extra capacity, extra speed, extra support for the thinking I'm already doing.

My neurodivergent brain is critical to the work I do. It's the brain that notices the pattern no one else has spotted. That feels the friction in a system before it shows up in a complaint log. That makes the unexpected connection between a council customer portal and a mum sitting in a fluorescent-lit waiting room with a school pickup looming.

That's the thinking that helps my clients reach their full potential, and fix their systems wherever those systems are failing the people they serve. AI can help me get there faster. It can't get there for me.

Why I wrote my client an AI Policy

With my most recent client, I did something I'd never done before. I prepared an AI Policy to explain exactly how I use AI in my thinking: where it helps, where it doesn't, and where the human judgement always sits with me.

I did it because I think clients deserve to know. And because, honestly, I worry about what's happening in my own industry.

There are consultants out there churning out reports, strategies and plans at a rate that should raise eyebrows. Polished outputs, produced in record time, with very little human thought behind the process. They look impressive. They read well. But the thinking has been skipped: the listening, the noticing, the wrestling with what's really going on.

We've already seen what this looks like. Last year, one of the world's biggest consulting firms had to partially refund the Australian Government after a A$440,000 report was found to contain AI-generated errors. Among them were a made-up quote from a Federal Court judgment and references to research papers that don't exist. The report was reviewing a welfare compliance system that affects some of the most vulnerable people in our community.

That isn't the partnership I believe in. I want a genuine partnership between human intelligence and Extra Intelligence, where each does what it's best at, and the client can see the difference.

Could versus should

My biggest worry is for customer-facing organisations. Right now, there's enormous pressure to adopt every AI tool on the market. The chatbot. The automated triage. The AI-generated response to a complaint. The portal that removes the need for a phone line altogether.

The question being asked is almost always could we use this? Far less often is anyone asking should we?

Trust me, you're going to find some efficiencies. Some of them will be brilliant, and you should grab them with both hands. But if you replace every part of your internal or external processes with something artificial, you're at significant risk of really stuffing things up.

In 2025, one of Australia's biggest banks cut 45 customer service roles after introducing an AI voice bot, saying it had reduced call volumes. Staff said calls were actually rising, with overtime offered and team leaders pulled onto the phones. Within weeks, the bank reversed the decision, apologised, and admitted it should have been more thorough. That's what happens when nobody asks the people doing the work.

And the stakes get much higher when the people on the receiving end are already vulnerable.

We've been here before in Australia. Robodebt wasn't AI. It was an automated system that raised debts by averaging people's income, and it ended in a Royal Commission. But it showed exactly what happens when government automates decisions about people's lives without enough human judgement. AI makes that kind of mistake faster, and harder to see.

It's already happening overseas. In Denmark, Amnesty International found the welfare authority's AI fraud-detection tools risk discriminating against people with disabilities, low-income households, migrants and refugees. One algorithm flags "unusual" living arrangements. That can sweep up a married couple living apart because of one partner's disability, older couples who don't share a home, and multi-generational families. Older people and people with disabilities who struggle with digital systems face extra barriers to getting help. And people aren't always told they've been flagged, so they can't challenge it.

In France, 15 human rights groups have taken the government to its highest administrative court, arguing its welfare fraud algorithm scores single mothers and disabled people as higher risk.

None of these systems set out to harm anyone. But nobody asked the single mum, the older couple or the person with a disability how it would land for them before it was switched on.

Because systems don't fail on paper. They fail for people.

The parent who can't get through the online form. The older customer who just needs someone to pick up the phone. The staff member absorbing the gaps the new tool left behind. If you haven't asked them, you won't know until it's too late.

Start with EVA

Whether you're a small business, a large private corporation, or a public sector organisation working at a local or national level, the starting point is the same. Before you decide where AI belongs in your system, talk to the people on the receiving end of it.

That's where our EVA approach comes in.

1. Engage. Go to the people who use your service (customers, communities, frontline staff) and ask them where it could be better. Treat it as a genuine opportunity to hear what's working, what's frustrating, and what they wish you'd fix. They already know where your system is broken. Most organisations simply haven't asked.
2. Validate. Reflect back what you've heard, in two directions. To the people you engaged: I hear you. And to the leaders who make the decisions, so that what was said doesn't get lost in a report nobody reads.
3. Adapt. Now make the changes. Bring in AI where it genuinely helps. Dial up your Human Intelligence (your HI) where it's needed most. Then keep listening, because the right balance will shift over time.

Finding the balance between HI and EI

The future isn't human or artificial. It's both, in the right proportions, in the right places.

Extra Intelligence is brilliant at speed, volume and repetition. Human Intelligence is irreplaceable when it comes to empathy, judgement, context, trust, and noticing the thing that doesn't fit the pattern. A good system uses each where it shines, and never hands the human moments over to a machine just because it's cheaper.

Getting that balance right isn't a technology project. It's a people project. And it starts with a conversation.

Let's have the conversation

If you're keen to plan for the future, and to work out where your system is broken right now and needs fixing, Karen Cocks and I would love to talk.

Get in touch to organise a chat, and let's see how we can help you find the right balance between Human Intelligence and Extra Intelligence, before the mess finds you.

We fix the system, not the people.

Becky Hirst, FTS