Computer says no: unpicking the employment risks of AI

Last week several newspapers ran a story about three make-up artists who had been’dismissed by algorithm’ during a redundancy exercise. They sued their former employer and received an out of court settlement. I was intrigued. Had Estee Lauder effectively outsourced its decision making to a machine to determine which employees to retain or dismiss? Well yes, but mainly no. It had used AI (specifically facial recognition technology) to interview the women but, according to Estee Lauder, this only accounted for 1% of its decision making. The rest of the process was conducted by a human being.

We consider the legal implications for employers using AI to make significant decisions about their staff.

Background Background

The development of new technologies, including those powered by AI and machine learning, is transforming the world of work. Automation has started to replace people in some sectors (particularly in manufacturing and retail). But it’s not just about machines performing work that used to require physical labor –technology is also being use to make decisions about employees. Last year, the TUC published an important report into the use of AI –technology managing people: the worker experience. It concluded that there had been:

‘a quiet technological revolution taking place in how people are managed at work. AI-powered tools are now used at all stages of the employment relationship, from recruitment to line management to dismissal. This includes the use of algorithms to make decisions about people at work.

According to a report prepared for the TUC: Technology Managing People – the legal implications, by September 2020, over 40% of enterprises had adopted at least one AI-powered technology and a quarter had adopted at least two across the EU27, Norway, Iceland The pandemic has accelerated the use of new technology and it’s likely that these figures underestimate the current numbers of businesses using, or planning to adopt, AI.

These systems appeal because they can perform some tasks much quicker than humans. For example, in the employment context, employers can use AI to weed out unsuitable candidates. The programme scans CV’s and, via key words, identifies those candidates who have the essential qualities required for the job, creating a smaller pile of applications to be reviewed for shortlisting.

However, AI is not confined to recruitment and is increasingly being used in a wider HR context to allocate shifts and holiday dates, generate disciplinary or absence reviews and to train staff. their productivity.

Legal issues

The key components driving most AI / machine learning applications involve data input, data processing and predictive models generated by machine learning. The instructions to the machine come from algorithms. And the machine can be programmed to learn and make decisions based on new data and to Each individual component of an AI system (the data, the predictive models, the decision rules and the resulting action) has relevance for workers who are being managed by AI.

There’s a perception that a decision reached by a machine is untainted by human biases and is therefore completely logical and objective. But that’s not necessarily the case. These new forms of technology can create discrimination as well as replicate and embed existing discrimination through the machine learning process. Robin Allen QC and Dee Masters argue that:

‘It is obvious that the AI ​​system an employer uses will be the product of many different actors: those who wrote the initial code for the first algorithm, those who wrote the machine-learning instructions, those who supplied the initial data on which the system was trained, and so on, down to those who determined that the system should be bought and used and those who then applied its outcomes. So, we can see that discrimination might be perpetrated by an employer in circumstances where a different organisation had created the discrimination in the first place.’

A good example of this is the algorithm Amazon used to recruit staff. It used data submitted by applicants over a 10-year period to determine who was the ideal candidate. The problem was most of the data it relied on were from male applicants. And because the data was skewed to favorite men, the machine taught itself that male candidates were preferred.


An employer is legally responsible for the decisions it makes about its staff even if these are generated by AI. And its not a defense to say they didn’t know a particular programme was discriminatory or reached an unfair outcome.

Discrimination claims

In the context of discrimination claims, although the Equality Act 2010 was drafted without AI in mind, it does provide a framework for candidates and employees to bring claims.

It’s arguable that algorithms and the databases used to train machine learning and reach fully or semi-automated decisions can amount to a provision, criteria or practice required to support a claim of indirect discrimination. Even if an employer can establish that it had a legitimate reason for using AI (which probably won’t be too difficult) it has to show that using it in the way it did was both proportionate and that it couldn’t have achieved the same result in a less discriminatory way. That’s likely to be a It’s also possible that if the employer can’t explain how the decision has been reached, a tribunal might infer discrimination from the lack of transparency within the AI-powered process itself.

Unfair dismissal

There’s also a risk that employees who are dismissed as a result of computer generated decisions will be able to bring unfair dismissal claims. Ordinarily, an employee can ask questions or seek to challenge a human-made decision to understand whether it is rational and fair. That’s much harder where decisions have been made by AI. The TUC report found that managers often didn’t understand why certain decisions had been made and were, therefore, unable to properly explain them.

Employees have a right to be treated fairly both in terms of the process adopted and the outcome reached. That means that employers need to be able to explain their decisions, give the employee a right of reply and usually allow them to appeal. If the rationale behind the decision is not transparent, employees will be able to argue that their treatment is not fair or unlawful.

Trust and confidence

There’s also a wider point. The relationship between an employee and employer is based on trust and confidence. It’s difficult to have a personal relationship with a member of staff if most of the decisions made are automated. An extreme example is gig economy workers who may be managed almost entirely by an App and have little personal interaction with the company they work for.

We know from the TUC report that many workers are suspicious about AI and fear that it is being deployed in ways they haven’t agreed to and don’t properly understand. If they don’t trust the processes their employers are using to manage them And, if their employer can’t reassure them that the process is both rational and fair (by explaining how the computer reached the decision it did), they are likely to complain. In serious cases, it could also undermine the relationship to such an extent that the employee resigns and claims constructive unfair dismissal.

Privacy and data protection

Employees have a reasonable expectation of privacy and, while some degree of monitoring is a normal part of working life, there are legal limits on intrusive monitoring or surveillance. Employers must comply. with the Data Protection Act 1998 and GDPR, update privacy notices and carry out a data impact assessment.

The Information Commissioner has updated its guidance on Big data, artificial intelligence, machine learning and data protection which emphasises the need for businesses using AI to be transparent about why they are using it, ensure that the information being processed is accurate and not kept longer than is necessary.

Acas recommendations

In March 2020, Acas published a research paper: My boss the algorithm: an ethical look at algorithms in the workplace which makes a number of recommendations (some of which are directed at government, rather than employers):

  1. Algorithms should be used to advise and work alongside human line managers but not to replace them. A human manager should always have final responsibility for any workplace decisions..
  2. Employers should understand clearly the problem they are trying to solve and consider alternative options before adopting an algorithmic management approach.
  3. Line managers need to be trained in how to understand algorithms and how to use an ever increasing amount of data.
  4. Algorithms should never be used to mask intentional discrimination by managers.
  5. There needs to be greater transparency for employees (and prospective employees) about when algorithms are being used and how they can be challenged; particularly in recruitment, task allocation and performance management.
  6. We need agreed standards on the ethical use of algorithms around bias, fairness, surveillance and accuracy.
  7. Early communication and consultation between employers and employees are the best way to ensure new technology is well implemented and improves workplace outcomes.
  8. Company reporting on equality and diversity, such as around the gender pay gap, should include information on any use of relevant algorithms in recruitment or pay decisions and how they are programmed to minimise biases.
  9. The benefits of algorithms at work should be shared with workers as well as employers.
  10. We need a wider debate about the likely winners and losers in the use of all forms of technology at work.


Leave a Comment