In the first part of our series we explained what Human-in-the-Loop is and why the development of artificial intelligence does not mean that humans should disappear from the decision-making process.
We learned three basic collaboration models:
- Human-in-the-Loop - a human actively participates in the decision-making process,
- Human-on-the-Loop - AI acts autonomously, but a human supervises the system,
- Human-out-of-the-Loop - the system operates autonomously, without ongoing human intervention.
Now it's time for the most important question: How do you know which model to choose?
Because this is where the real challenge begins.
It's not enough to say: "This decision is simple, so let's give it to AI."
Nor: "This decision is important, so a human must always approve it."
Reality is more complex.
A good AI system architecture should take into account, among other things:
- level of risk,
- cost of error,
- reversibility of the decision,
- impact on people,
- data availability,
- explainability of recommendations,
- frequency of decisions,
- possibility of effectively supervising the system.
Only an analysis of these elements allows us to determine how much autonomy we can safely give to AI.
The most important rule: do not automate decisions you cannot control
AI implementation often starts with the question: "What can we automate?"
In a mature organization the question should be: "What can we automate and still effectively control?"
That's a fundamental difference.
After all, we can build a system that automatically makes thousands of decisions a day. Technically it may work perfectly.
But if no one knows:
- why it made a particular decision,
- what data it relied on,
- when it starts making mistakes,
- who can stop the system,
- who is responsible for the consequences,
then we are not dealing with mature automation. We are dealing with automatic risk generation.
Four questions to ask before automating a decision
Before handing a decision over to an AI system, it's worth conducting a simple analysis.
1. What happens if the AI makes a mistake?
This is the basic question.
Not every mistake has the same consequences.
If an AI misclassifies a newsletter, probably nothing serious will happen. If it misidentifies a fraud attempt, financial loss may occur. If it incorrectly assesses a patient's health, the consequences may be far more serious.
Therefore, the level of autonomy should be proportional to the potential effects of an error.
2. Can the decision be easily undone?
This is one of the most important and often overlooked parameters.
Imagine two cases.
AI automatically changes the order of product displays in an online store. The mistake can be detected and reversed. The risk is relatively small.
Now another example.
AI automatically deletes a user's account and erases all their data. Such a decision may be much harder to reverse.
Another situation: AI automatically sends a contract termination notice to a client. Here the consequences are even more serious.
Therefore decision reversibility should be one of the basic criteria in designing AI systems.
3. Does the decision directly affect a human?
If AI optimizes the route of a delivery van, the level of risk may be relatively low.
But if the system assesses a job candidate, grants credit, or determines access to a particular service, the situation looks completely different.
Decisions concerning people require special caution. Not only for ethical reasons, but also because of regulations, organizational responsibility, and risk of discrimination.
In such areas a human can be more than just the "last click." They should have a real ability to challenge the system's recommendation.
4. Can a human effectively supervise the system?
This question is more deceptive than it might seem.
Imagine a system that generates 50,000 recommendations a day. Formally a human could approve them all. Practically this is impossible. After a while the worker will start accepting decisions automatically.
This gives rise to a phenomenon called automation bias - the tendency to over-trust system recommendations and treat them as default correct.
The human is still "in the loop." But in reality they stop performing a control function. They become merely an operator of the "Approve" button.
That is not true Human-in-the-Loop.
Decision risk assessment model
In practice you can apply a simple assessment model.
Analyze each decision against five parameters:
1. Probability of error
How often can the system be wrong?
2. Scale of consequences
How serious will the error be?
3. Reversibility
Can the decision be quickly undone?
4. Impact on people
Does the decision directly affect rights, finances, health or safety of a person?
5. Supervision capability
Is a human able to effectively monitor the process?
The higher the level of risk, the greater the need for human involvement. This does not mean, however, that every high-risk decision must be made manually. It means the system should have properly designed control mechanisms.
Level 1 - direct automation
These are the simplest decisions.
Examples:
- sorting messages,
- classifying documents,
- generating summaries,
- recommending products,
- assigning tickets to categories.
AI can operate autonomously. A human does not need to approve every decision. The system should, however, be monitored.
This model is close to Human-on-the-Loop.
Level 2 - AI recommends, human approves
Here AI analyzes data and presents a proposal.
A human makes the decision.
Examples:
- recommendation of discount amount,
- suggested response to a complaint,
- recommendation of service actions,
- transaction risk assessment,
- suggested ticket priority.
This is classic Human-in-the-Loop.
The model works well where AI can greatly speed up analysis but the decision requires human experience.
Level 3 - AI acts autonomously, but with possibility of intervention
The system makes decisions automatically. However, a human can stop or change them.
An example might be automatic blocking of suspicious transactions. The system detects an unusual pattern and blocks the operation. If the client reports an issue, an employee can verify the decision.
In this model the keys are:
- monitoring,
- alerts,
- ability for manual intervention,
- decision logging,
- fallback mechanisms.
Level 4 - full autonomy
AI operates without ongoing human involvement.
This model should be reserved for processes that are:
- repetitive,
- well-defined,
- low-risk,
- easy to monitor,
- reversible.
An example might be automatic scaling of cloud infrastructure resources.
The system detects increased load. It launches additional resources. After the traffic drops it shuts them down. No one needs to approve each individual decision.
However, if the system begins to generate unforeseen costs, there should be a mechanism to limit its operation.
Autonomy should never mean lack of control.
Example 1 - e-commerce
Imagine an online store.
AI analyzes:
- purchase history,
- user behavior,
- seasonality,
- competitor prices,
- product availability.
It can recommend products to a customer. This is a relatively safe process.
The system can operate autonomously. But what if AI is to set prices independently? Then things get complicated.
Dynamic pricing can be automated, but it should operate within defined limits.
The system can have:
- a minimum price,
- a maximum price,
- a maximum price change per day,
- promotion rules.
If AI wants to exceed certain limits, the decision can be passed to a human.
This is an example of guardrails, i.e., constraints that secure system operation.
Example 2 - recruitment
AI analyzes candidates' CVs.
It can:
- extract experience,
- organize information,
- match competencies to requirements.
This can significantly speed up a recruiter's work.
But the decision: "We reject this candidate." is of a completely different kind.
If the system makes this decision automatically, it can perpetuate biases present in historical data.
Therefore, in HR processes human involvement can be fundamentally important. AI can support the recruiter. It should not automatically decide a person's value based on incomplete data.
Example 3 - logistics
An AI system plans deliveries.
It analyzes:
- distances,
- traffic jams,
- weather,
- driver availability,
- working hours,
- fuel costs.
It can independently optimize routes.
But what if the system chooses a cheaper route that doesn't account for local restrictions? Or if it optimizes delivery time at the cost of overworking drivers?
Therefore the system should operate within defined business rules.
AI optimizes. The organization defines boundaries. A human intervenes when the system goes beyond the safe range.
Example 4 - banking
AI can detect anomalies in transactions.
If the system detects an unusual operation, it can automatically flag it.
But what next?
Several scenarios are possible:
- just an alert for an employee,
- temporary suspension of the transaction,
- automatic block,
- a block requiring additional verification.
The choice depends on the risk level.
For a suspicious transaction of a small amount, automation can be applied. For more complex cases an expert analysis may be needed.
This is why AI systems should have escalation mechanisms, i.e., passing the case to a human when the level of uncertainty or risk exceeds a set threshold.
Confidence score - does AI know when it is uncertain?
One interesting mechanism is using a confidence level.
The system can assess: "My recommendation has a high probability of being correct."
Then the action can be executed automatically.
However, if the system assesses its recommendation as uncertain, the process can be passed to a human.
For example:
Confidence score > 95% Automatic execution.
Confidence score 70-95% Recommendation for a human.
Confidence score < 70% Mandatory expert verification.
One must be cautious.
A confidence score is not automatically proof of truth. A model can be very confident and still be wrong. Therefore, the confidence level should be one element of the assessment, not the sole decision criterion.
The greatest danger: a human who stopped checking the AI
You can design a system with Human-in-the-Loop.
You can add a button: "Approve AI decision."
You can hire a person for control.
And still create a dangerous system.
Why? Because a person may begin to trust AI more than their own experience.
If the system works correctly for 1,000 cases, a human stops analyzing the 1,001st in detail. It's natural.
Therefore a good system should not only require approval. It should also provide the human with the appropriate context.
For example:
- why the AI made that decision,
- what data it considered,
- what alternatives it evaluated,
- what the level of uncertainty is,
- what might happen after executing the decision.
Then the human is not only a "safety brake."
They become a genuine partner of the system.
Human-in-the-Loop cannot be a fiction
This is one of the most important principles of AI system design.
If a human is to control AI, they must have:
- sufficient knowledge,
- adequate time,
- access to data,
- ability to challenge the recommendation,
- ability to stop the process,
- clear responsibility.
If the worker can only click "Approve" or "Reject" but cannot check why the AI made a decision, their role is severely limited.
Even worse if the system works so quickly that the human cannot keep up with the analysis. Then formally we have Human-in-the-Loop. Practically - we have automation with a human playing a decorative role.
Practical checklist - can this decision be given to AI?
Before automating, answer the following questions:
☐ Is the decision repeatable?
☐ Do we have sufficiently good data?
☐ Can we determine when the AI is wrong?
☐ Is the error acceptable?
☐ Can the decision be reversed?
☐ Does the decision directly affect a person?
☐ Are there clear business rules?
☐ Can the system be monitored?
☐ Can a human stop the AI's operation?
☐ Is it clear who is responsible?
☐ Can the decision be explained?
☐ Does the human have enough time and knowledge to effectively supervise the system?
The more answers are "no," the more cautiously you should approach automation.
Glossary of terms
Automation Bias
The tendency of humans to over-trust recommendations from an automated system and treat them as correct without appropriate verification.
Guardrails
Constraints and protective mechanisms that define what an AI system can and cannot do.
Confidence Score
An indicator of the model's confidence in a result or recommendation. It should not be equated with a guarantee of correctness.
Escalation
A mechanism for passing a case to a human when the system detects a high level of risk, uncertainty, or a situation beyond established rules.
Human Override
The ability to manually replace a decision or action taken by an AI system.
Decision Reversibility
The reversibility of a decision, i.e., the ability to undo its effects without significant consequences.
High-impact decision
A decision that may have a significant impact on a person's life, rights, opportunities, or economic situation. In such cases requirements for supervision and responsibility are particularly important.
The most important conclusion
Not every decision should be automated, but not every decision requires a human sitting at a computer approving every single operation.
The true goal of designing AI systems is not to choose between: "a human decides" and "AI decides".
The goal is to create an appropriate collaboration model.
AI should act autonomously where the risk is low. It should recommend solutions where human experience is needed. It should stop and ask for help where the consequences of an error are too serious. And a human should always have the option to take control when the situation goes beyond the anticipated scenario.
A well-designed AI system is not the one that makes the most decisions independently. It is the one that can correctly recognize when it should make a decision, when it should present a recommendation, and when to say: "This time I need a human."
In the next part we will move from single-decision level to the organizational level.
Because even the best-designed AI system can become a problem if the company does not know who is responsible for it, who controls it, and what to do when it starts operating differently than expected.
In the next part we will discuss AI Governance - how to build rules, responsibilities and control mechanisms for artificial intelligence in an organization.
