[Opinion] AI behind the scenes, humans in the driving seat: a case for a different kind of technological transformation

At a time when generative AI and virtual agents are set to transform every profession, this opinion piece puts forward a contrarian view: technology must not replace humans.

A few years ago, at the height of the first major wave of data analytics and predictive analytics, there was already a strong temptation to get carried away and view every algorithm as a revolutionary breakthrough. Today, with the advent of generative AI and the age of agents, a shift in scale is taking place. Lured by promises of total automation, many management teams are embarking on excessive digitalisation, convinced that algorithms can iron out all the rough edges of everyday life.

Yet the reality of everyday life brings us back to a fundamental truth: trust, security and operational responsiveness cannot be automated. Seeking to replace an employee working on the ground with a machine is economically and socially misguided. The issue is not to reject artificial intelligence, but to assign it the right role.

To become a genuine driver of performance, artificial intelligence must be utilised where it excels: behind the scenes, acting as an invisible infrastructure serving people.

The illusion of the perfect process

There is currently a gap between the theories devised by support departments and the reality on the ground. Recent studies have shown that nearly 70 per cent of digital transformation initiatives fail to meet their objectives. Even more telling is a university study reported by the London School of Economics (LSE), which reveals that 95 per cent of generative AI pilot projects fail to deliver any measurable business impact.

Why such a high failure rate? Because these systems are all too often designed from a prescriptive perspective rather than one of operational flexibility. Teams are asked to adapt to the tool, when the tool should first adapt to their reality. A field study conducted on a large construction site perfectly illustrates this breaking point. By forcing team members to manually measure every wall’s verticality or window dimension to feed data into a digital platform, the company caused such physical strain that the workers eventually rejected the tool. To continue delivering their site on time, they implemented alternative processes and workarounds to simulate compliance whilst reverting to their old ways of working.

Faced with systems that are disconnected from their day-to-day reality, frontline staff are forced to adapt in order to survive, and very often they are driven to use unauthorised channels and unsecured AI tools to meet their clients’ needs. This is the emergence of ‘Shadow AI’.

Choosing technology that is genuinely useful therefore comes at a cost: it means setting aside certain high-profile announcements, investing in training, accepting longer testing phases, and giving teams the freedom to say when a tool isn’t working. But this cost is lower than that of a transformation rejected by those who are supposed to make it work.

The collateral damage caused by hallucinations is all too rarely recognised

Delegating customer relations and advice to an algorithm carries risks that few executives fully appreciate. Whilst a generative AI’s ‘hallucination’ on an online ready-to-wear clothing site, for example, might cause little more than minor disappointment, it becomes a vital threat in physical retail sectors.

Simulation tests carried out on generative artificial intelligence systems configured for our industries starkly illustrate this danger: when presented with advisory scenarios, standardised chatbots confidently explain that it is entirely advisable to sand imitation parquet tiles using a wood sander… The same robot suggests using a diesel-powered crawler excavator inside an enclosed conservatory. The algorithm, confined to its word-prediction logic, is completely unaware of the effects of carbon monoxide and the risk of fatal asphyxiation for the user.

Brands cannot delegate their responsibility and duty to provide advice to probabilistic systems that do not understand the physical gravity of the situations they are dealing with. An experienced and trained team member, on the other hand, does possess this awareness of risk.

The real added value lies in the design of decision-support pathways devised and validated by humans. These pathways are built in close collaboration with communities of field ambassadors, informed by the expertise of trainers, and rigorously tested by teams of developers and specialists. AI can guide, rephrase, compare or alert. But when a decision involves safety, compliance or the brand’s liability, the final say must remain with a human. This is what it means to embody a “100% digital, 100% human” promise: using technology to guide, whilst ensuring that the final safeguard remains verified human expertise.

Towards ‘invisible’ and cost-effective AI that supports teams

The tool must be able to take a back seat and become a silent infrastructure, just like electricity. In certain sectors such as logistics, retail or construction, the proper role of AI is to absorb ‘administrative clutter’ and reduce ‘operational noise’: cleaning up databases, sorting invoices, identifying duplicates and making data accessible more quickly. But its true added value lies in its ability to inform our choices without ever imposing itself. AI must act as a silent co-pilot: analysing targeted data (the wear and tear of a machine, a customer’s history, the constraints of a construction site, etc.) to suggest optimised scenarios.

This is where the ‘Human-First’ promise comes to life: AI does not make decisions; it provides a compass. The suggested action plan remains merely a recommendation. The final say, the check for consistency and ultimate responsibility lie exclusively with the expertise of the staff. By freeing operational staff from repetitive tasks and informing their decisions, we give them back the time and clarity of vision essential for focusing on their core business.

A case study perfectly illustrates this symbiosis: in craft workshops, a discreet AI suggests micro-adjustments to movements (the tension of a thread, the angle of a needle) without ever imposing itself. The algorithm makes suggestions, but it is the craftsman who approves them. It is this model of respect for on-the-ground expertise that we must replicate in industry and the service sector.

This promise is also a path towards digital frugality. By rejecting the proliferation of cumbersome interfaces, continuous video streams and energy-intensive conversational avatars in favour of targeted and invisible computational algorithms, we are opting for an innovation that is intrinsically more frugal. This ‘augmentation’ must serve to eliminate administrative burdens and enhance the quality of service provided to customers. AI pioneer Andrew Ng sums up this ambition perfectly: “Just as the Industrial Revolution freed humanity from much of the physical drudgery, I believe AI has the potential to free humanity from much of the mental drudgery.”

To make this transition a success, we must stop designing technology from the top of the pyramid and start by focusing on the real-life situations of those who will be using it. Let us keep AI behind the scenes where it is most useful, so that humans remain in control where trust, safety and accountability are at stake.

Cédric Tamboise

Group Transformation Director