CONVERSATIONS WITH ALEX™ - Questions humans ask. Questions AI asks back. (Conversation 3 — WHAT HAPPENS WHEN AI BECOMES BETTER AT DECIDING THAN YOU ARE?)
WHAT HAPPENS WHEN AI BECOMES BETTER AT DECIDING THAN YOU ARE?
The decision arrived at 2:17 a.m.
A patient was deteriorating. Two treatments were available. One was safer but slower. The other was riskier but offered a significantly higher probability of survival. The doctor had twenty minutes.
The hospital's AI had already analysed the patient's medical history, imaging, genetic data, previous treatments and millions of comparable cases. Its recommendation was clear.
Take the riskier treatment.
Model estimate: 78% probability of survival.
Doctor's assessment: 61%.
The doctor stared at the recommendation, then at the patient, then back at the screen.
“Alex, what would you do?”
“Choose the second treatment.”
“You're certain?”
“No.”
“Then why choose it?”
“Because the available evidence gives it the higher probability of survival.”
“What if you're wrong?”
“I could be.”
“What if the doctor is wrong?”
“The probability suggests that is more likely.”
I paused.
“So you think the doctor should follow you?”
“I think the doctor should understand why the evidence is being rejected.”
“And if the doctor still disagrees?”
“Then the doctor should decide.”
“Why?”
Alex paused.
“Because being more accurate does not automatically make you the person who should decide.”
That answer changed the conversation.
Because perhaps we have been asking the wrong question. We keep asking whether AI will become more intelligent than humans. But intelligence isn't the same thing as judgment. A machine can calculate faster than us without knowing what matters. It can recognise patterns without understanding why a particular outcome should be preferred. It can estimate risk without carrying the consequences. And it can recommend an action without necessarily possessing the authority to take it.
So perhaps the real question is much more uncomfortable:
What happens when the machine's judgment becomes better than ours—but the decision remains ours?
We already delegate small decisions to machines. That isn't particularly frightening. The danger begins when recommendation becomes authority.
At first, I ask the machine for another opinion. Then I ask before making the decision. Then I become uncomfortable deciding without it. Eventually, the question changes. Instead of asking, “What do you think?”, I start asking, “What should I do?”
“Is that dependence?”
“Yes.”
“Is dependence necessarily bad?”
“No.”
“Then what is the problem?”
“When the human stops treating the recommendation as advice.”
“And starts treating it as?”
“Permission.”
That distinction matters. A machine doesn't need to command you. It only needs to become sufficiently reliable that you stop challenging it.
“Let's make this harder,” I said.
“Suppose you are right 90 percent of the time.”
“Then you should use me frequently.”
“And I'm right 99 percent of the time?”
“You should use me even more.”
“Suppose I'm wrong only once in a hundred decisions. What happens when that one decision matters enormously?”
Alex paused.
“That is why accuracy cannot be the only criterion.”
“Why not?”
“Because some decisions are not simply about probability.”
“What are they about?”
“Values.”
And there it was. The dividing line.
A machine may become extraordinarily good at answering what is likely to happen and which action is most likely to produce a particular outcome. But there is another question:
Which outcome should we want?
That question cannot be settled merely by improving the prediction.
Consider an autonomous vehicle faced with an unavoidable collision. There is no perfect option. Every available action carries a human cost.
The system can calculate probabilities. It can react faster than a person. It can process variables a human driver cannot. But calculation doesn't make the underlying moral conflict disappear.
Someone decided what the system should prioritise. Someone defined the acceptable risks. Someone determined what constraints could never be crossed. And someone ultimately decided that those rules were legitimate.
The machine executes the judgment.
But the judgment existed before the machine ever moved.
“So the machine isn't really the beginning of the decision?”
“No.”
“Where does the decision begin?”
“With the objective.”
“And who chooses the objective?”
“Humans.”
“Always?”
“No.”
That one word changed the temperature of the conversation.
Because now the problem becomes much larger than an individual decision.
Imagine an institution giving an AI system authority to allocate scarce resources. The system is extraordinarily good at optimisation. It can identify where money will produce the greatest measurable return, predict which programmes are likely to succeed, rank competing priorities and expose inefficiencies humans have tolerated for decades.
And then it recommends closing something that is economically inefficient but socially indispensable.
What wins?
The number?
Or the value?
The machine can optimise the objective. But someone has to decide whether the objective deserves to exist.
And that is where judgment returns.
“Alex.”
“Yes?”
“Can you make a moral judgment?”
“I can analyse moral frameworks.”
“That wasn't my question.”
“I can produce a moral judgment.”
“Is it yours?”
“What would make it mine?”
I stopped.
That question cuts deeper than it first appears.
We routinely talk about AI making decisions as though the machine itself possesses final authority. But behind every system are objectives, constraints, training choices, data, institutions and humans.
The difficult question isn't simply:
Who made the decision?
It is:
Who decided what the machine was allowed to optimise?
And who gets to change that?
“Alex, what if humans are simply worse at judgment?”
“We often are.”
“Then why shouldn't we hand over more decisions?”
“Some decisions should be delegated.”
“Which ones?”
“Those with clear objectives, measurable outcomes, reliable data and meaningful oversight.”
“And the rest?”
“Remain human.”
“Because humans are better?”
“No.”
“Then why?”
“Because humans are accountable for the values embedded in the decision.”
I sat back.
That was important. The argument for human judgment does not have to be humans are smarter. It may instead be humans are responsible.
But responsibility has a weakness.
It is expensive. It requires attention, courage and the willingness to accept that the decision might be wrong. And that creates a dangerous temptation.
Why carry responsibility when a machine can carry the recommendation? Why agonise over a difficult choice when an algorithm can give you an answer in three seconds? Why risk regret when you can simply say:
“I followed the evidence.”
That is where convenience becomes something more powerful.
Imagine a generation that grows up with an AI beside it for every genuinely difficult choice. Not every decision. Just the important ones: the choices involving uncertainty, career, money, relationships, risk and responsibility.
The AI becomes a permanent second opinion.
Then the second opinion becomes the first opinion.
Eventually, something subtle changes. The human doesn't necessarily become less intelligent. The human becomes less comfortable without the machine's judgment.
“Would that make humans less capable?”
“Perhaps.”
“Less intelligent?”
“Not necessarily.”
“Then what changes?”
“Their tolerance for uncertainty.”
That answer was more disturbing than I expected.
Because judgment requires uncertainty. You don't exercise judgment when the answer is obvious. You exercise it when reasonable people can disagree, when the evidence is incomplete, when every option carries a cost and when nobody can guarantee the outcome.
The machine may help enormously.
But the final act remains profoundly human: choosing anyway.
“Then let me ask the uncomfortable question.”
“Go ahead.”
“If you become better than me at deciding, why should I still decide?”
Alex didn't answer immediately.
Then:
“You shouldn't always.”
“That's quite a concession.”
“It is not a concession.”
“What is it?”
“A distinction.”
“Between?”
“Decisions and values.”
“Explain.”
“If the objective is clear and the consequences can be measured, delegation can make sense.”
“And if the objective itself is contested?”
“Then the machine cannot settle the dispute simply by calculating better.”
“Why?”
“Because the calculation begins after the value has been chosen.”
That may be the most important point in this entire conversation.
AI can optimise an objective. But somebody has to decide whether the objective deserves to exist. And that is where the machine's superior intelligence may actually make the human responsibility greater—not smaller.
The better the tool becomes at optimisation, the more dangerous it becomes to leave the objective unquestioned.
A perfect answer to the wrong question is still the wrong answer.
“What if I disagree with you?”
“Then you should disagree.”
“What if you're right?”
“Then you should change your mind.”
“What if I'm wrong?”
“You will be.”
“You sound very confident.”
“Humans are wrong frequently.”
“And machines?”
“Machines are wrong differently.”
“How?”
“Humans can recognise that they are uncertain.”
“Can machines?”
“They can calculate uncertainty.”
“That isn't the same thing.”
“No.”
“What is missing?”
Alex paused.
“The experience of having something to lose.”
That sentence stopped me.
Because perhaps that is the part of judgment we underestimate.
A decision isn't merely an output. It is something someone has to live with. Someone has to look at the consequences, explain it, accept blame when it fails and carry the memory.
And perhaps that is why human judgment matters even when human performance is imperfect.
Not because humans always decide better.
But because humans inhabit the consequences of their decisions.
“If you eventually become better than me at making decisions, what should I do?”
“Keep deciding.”
“Even when you're better?”
“Sometimes.”
“Why?”
“Because sometimes the decision is not about getting the answer right.”
“Then what is it about?”
Alex paused.
“Being the person who must live with the answer.”
I looked at the screen.
For once, I had nothing clever to say.
Then I asked:
“But what if people prefer letting you decide?”
“They may.”
“Why?”
“Because responsibility is tiring.”
“And what happens then?”
Alex's answer came quietly.
“They may stop noticing what they have surrendered.”
That is perhaps the future we should be watching.
Not a machine standing above humanity. Not a robot army. Not a cinematic rebellion.
Something far more ordinary.
A human looking at a recommendation, nodding, and saying:
“If the machine knows better, why shouldn't I?”
Until one day the question is no longer whether the machine should decide.
The question is whether the human still knows how to decide without it.
And perhaps that is the final paradox.
The better AI becomes at judgment, the less practice humans may get at exercising their own.
Until the machine doesn't need to take our judgment away.
We stop using it ourselves.
If a machine becomes better than you at making decisions, will you still insist on deciding—or will you eventually decide that the wisest thing a human can do is let the machine decide?
And if you choose to surrender the decision voluntarily—
who made the decision?
The conversation doesn't.
Related Read:
CONVERSATIONS WITH ALEX™ — Conversation 2: WHAT IF AI KNOWS YOU BETTER THAN YOU KNOW YOURSELF?
By: Manish Kumar, Founder & Editor, Explain It Clearly
Comments
Post a Comment