10.6092/issn.1561-8048/14099
Artificial intelligence (AI) has growing impact on working life. Risks of job losses, discrimination, data protection violations, surveillance pressure and health hazards require legal regulation. But which legal framework is necessary to effectively protect the rights and interests of workers?Workplace co-determination can help to minimize risks and reap the benefits of AI systems. It is therefore a problem that collective, co-determinated solutions do not seem to be considered in the drafting of the AI Act at EU level. This does not live up to the aspirations currently expressed in the European Pillar of Social Rights, and some member states go decidedly further, too. Fair AI requires involvement and empowerment of the workforce. Otherwise, the intended "human-centered" approach to AI in the world of work remains just a phrase.
application/pdf
1561-8048
14099
2
en
Johanna Wenckebach
https://creativecommons.org/licenses/by/4.0
Italian Labour Law e-Journal
Democracy at workplace
Artificial intelligence, work, power imbalance and democracy – why co-determination is essential
Text.Serial.Journal
https://illej.unibo.it/
14
Miscellaneous
2021-12-22
2021-12-19
2021-12-22
2021-12-22
157-171
ILLeJ
Johanna Wenckebach
2021/12/22
10.6092/issn.1561-8048/14099
157
Hugo Sinzheimer Institute for Labour and Social Security Law
1561-8048
2
Democracy at workplace
en
171
Artificial intelligence, work, power imbalance and democracy – why co-determination is essential
https://illej.unibo.it/article/download/14099/13694
14
Artificial intelligence, work, power imbalance and democracy – why                     Italian Labour Law e-Journal
co-determination is essential                                                                 Issue 2, Vol. 14 (2021)
Ernesto Klengel, Johanna Wenckebach                                                                 ISSN 1561-8048
ernesto-klengel@boeckler.de                                          https://doi.org/10.6092/issn.1561-8048/14099
johanna-wenckebach@boeckler.de




       Artificial intelligence, work, power imbalance and
        democracy – why co-determination is essential
                          Ernesto Klengel, Johanna Wenckebach



 1. Introduction. 2. AI in the world of work. 3. AI: A collective labour law approach is
 necessary. 3.1. The legal framework of works council rights in AI in Germany. 3.2. AI in the
 practice of workplace co-determination. 3.3. The 2021 Law on the Modernisation of Works
 Councils. 4. The Commission’s draft of an AI Act. 5. Conclusions.



                                                        Abstract

     Artificial intelligence (AI) has growing impact on working life. Risks of job losses,
     discrimination, data protection violations, surveillance pressure and health hazards require legal
     regulation. But which legal framework is necessary to effectively protect the rights and interests
     of workers?
     Workplace co-determination can help to minimize risks and reap the benefits of AI systems. It
     is therefore a problem that collective, co-determinated solutions do not seem to be considered
     in the drafting of the AI Act at EU level. This does not live up to the aspirations currently
     expressed in the European Pillar of Social Rights, and some member states go decidedly further,
     too. Fair AI requires involvement and empowerment of the workforce. Otherwise, the intended
     "human-centered" approach to AI in the world of work remains just a phrase.

     Keyword: Artificial intelligence; Algorithmic management; Works council; Co-determination in
     the digital transformation; Collective regulation; Data protection; Democracy at workplace



 1. Introduction.

    Moral and social consequences of "intelligent" technical systems have been debated for
 some time – be it in art with the motif of technology taking on a life of its own, in social
 science research on technology and the future, or in the media. Now the point seems to be
 getting closer where further increases in computing power and digital networking will lead

 
  Ernesto Klengel is Scientific Associate for Labour Law at the Hugo Sinzheimer Institute for Labour and Social
 Security Law, Frankfurt a.M.. Johanna Wenckebach is Scientific Director at the Hugo Sinzheimer Institute for
 Labour and Social Security Law, Frankfurt a.M. This article has been submitted to a double-blind peer review
 process.
    Ernesto Klengel                                                                      Italian Labour Law e-Journal
    Johanna Wenckebach                                                                          Issue 2, Vol. 14 (2021)
                                                                                                Section: Miscellaneous




to profound changes in society as a whole and in the world of work in particular. As is so
often the case, light and shadow are being drawn. The light: machines take over unpleasant
work and enable creative development and more autonomy at the workplace. Efficiency
gains allow working hours to be reduced, new jobs are being created, access to the labour
market gets easier. The shadows: Artificial intelligence eliminates jobs and takes over
command, human needs count for little, precarious, monotone work is created.
Comprehensive personality profiles increase pressure on workers, working behaviour is
monitored around the clock and the boundaries between work and private life disappear.
There are signs in both directions. Which way the pendulum will swing also depends on
political decisions and the balance of power in society. Good work, through and with
digitalisation, remains a question of power, but under new conditions. Collectives – works
councils and trade unions – are the crucial factor to redress the balance of power in
companies in favour of the workers' side, they democratize work and AI.
    With reference to AI legislation in Germany and the EU, this text makes clear why
collectives must be considered in the European legislative processes and why they need new
competences in order to be able to exercise their important function of democratisation also
in the introduction and application of AI. In the first chapter, we point out which areas of
the labour environment are effected by AI and what changes this entails for employees. Next,
the need for regulatory action is discussed, noting that in some member states, collective
actors often play an important role in protecting workers' rights. This is illustrated by the
case of Germany, where AI was an important issue in the last revision of the Works
Constitution Act (BetrVG). In contrast, a look at the EU shows that workplace co-
determination has not played a significant role in the AI Act, as will become clear in Chapter
4, although co-determination is firmly anchored in the fundamental treaties. Finally, the
summary raises the question of a procedural approach specifically for the regulation of AI at
EU level.


2. AI in the world of work.

    Big Data is already making its way into the world of work, albeit to varying degrees in
different sectors and regions of the world. One of the areas in focus is logistics – courier
services, including digital giants such as Amazon, Uber or Deliveroo, as well as start-ups
already worth billions, such as the Berlin-based company Gorillas, rely on algorithmic
control. But AI-supported systems are also finding their way into offices and production1 –
here, the efficiency of employees is illuminated in detail by workplace analytics, humans and
machines are increasingly interacting in production, and algorithms are used in human
resources management for ranking, clustering and even decision-making.2 It goes hand in

1 De Stefano V., Taes S., Algorithmic management and collective bargaining in Foresight Brief, ETUI, #10 – May 2021,
2021, https://www.etui.org/publications/algorithmic-management-and-collective-bargaining, accessed 3 Oct.
2021.
2 Schubert C., Hütt M., Economy-on-demand and the fairness of algorithms, in European Labour Law Journal, 10, 1, 2019,

3 and ff.

https://doi.org/10.6092/issn.1561-8048/14099                                                                        158
    Ernesto Klengel                                                                       Italian Labour Law e-Journal
    Johanna Wenckebach                                                                           Issue 2, Vol. 14 (2021)
                                                                                                 Section: Miscellaneous




hand with permanent monitoring and evaluation of workers' behaviour. Employees become
transparent – for example, where every visit to the toilet by warehouse workers affects their
own score, or every interaction with colleagues is being evaluated. Obviously, the digitalised
production mode has consequences for labour relations.
   Although HR management based on automated decision-making (ADM) frequently leads
to staff cuts, its primary focus is on more efficient use of the workforce. With the aim of
optimally deploying the skills of employees in the interests of the company, it is by no means
only finding its way into the "new economy", but also into more traditional industries. It is
being used with different objectives:3 Human resource analytics working with descriptive ADM
make it possible to refine data that has already been collected, such as employee absences,
staff turnover and performance feedback. The software makes this data available for complex
evaluations, but also to observe communication behaviour within the company and uncover
internal social networks. Predictive ADM relies on such present-related data to describe future
developments. It is also used in the recruitment process to allow a forecast of which applicant
is best suitable for a specific position or with which emotionality an employee reacts to
challenges or who has intentions of changing jobs. Prescriptive ADM, on the other hand,
generates solutions for specific questions and thus provides automated decisions. An
example of this is the logistics software that controls the delivery of parcels.
   The use of such systems becomes a (supposed) competitive advantage that a company
must make use of at the threat of losing its existence. Work intensification is therefore the
logical goal of automatic personnel management, and the small-scale evaluation of
performance is the means, which in turn requires permanent data collection. Although
hierarchies often disappear and are less visible, workers increasingly become objects of the
algorithm. The character of work as a social process recedes into the background, processes
of alienation are fostered: The app appears as the boss,4 internal and external crowdsourcing
and contracting are on the rise, the protection of labour law is narrowing down.
   Algorithmic decisions have the appearance of an objective and unquestionable procedure.
This is also problematic in light of discrimination: since the data with which AI systems work
comes from a world of work in which categories such as gender, age and origin play a role,
automated decision-making processes can reinforce existing disadvantages under the guise
of a "neutral procedure".5 But anti-discrimination law is not made for automated decision-




3  Gießler S., Was ist automatisiertes Personalmanagement?, 2021, 8 and ff., https://algorithmwatch.org/de/wp-
content/uploads/2021/05/Was-ist-automatisiertes-Personalmanagement-Giesler-AlgorithmWatch-2021.pdf,
accessed 3 Oct. 2021.
4 Adams-Prassl J., What if your Boss was an Algorithm? Economic Incentives, Legal Challenges, and the Rise of Artificial

Intelligence at Work, in Comparative Labour Law and Policy Journal, 41, 1, 2019, 123 and ff; Ivanova M., Bronowicka
J., Kocher E., Degner A., Foodora and Deliveroo: The App as a Boss?, 2018, https://www.boeckler.de/de/faust-
detail.htm?sync_id=HBS-07048, accessed 3 Oct. 2021.
5 Federal Government of Germany, Digitalisierung geschlechtergerecht gestalten – Gutachten für den Dritten

Gleichstellungsbericht     der      Bundesregierung,    2020,        91       and      ff.,     https://www.dritter-
gleichstellungsbericht.de/de/topic/73.gutachten.html, accessed 3 Oct. 2021; Schubert C., Hütt, T., nt. (2), 6
and ff.

https://doi.org/10.6092/issn.1561-8048/14099                                                                         159
    Ernesto Klengel                                                                     Italian Labour Law e-Journal
    Johanna Wenckebach                                                                         Issue 2, Vol. 14 (2021)
                                                                                               Section: Miscellaneous




making.6 Here it becomes a problem that in many cases it is not transparent how AI tools
are working7 and on the basis of which data the algorithms deployed have come about.8
   AI in the world of work therefore entails considerable and, in this dimension, new dangers
for the personality and health of workers and does not necessarily lead to better working
conditions.


3. AI: A collective labour law approach is necessary.

   Freedom of association and consultations at plant level are structural elements of the
labour law across the EU. In the legal order of the union, collective elements have been
strengthened in the recent past. The Charter of Fundamental Rights includes not only the
freedom of association and the freedom to conclude collective agreements, but also the right
to information and consultation within the undertaking (Article 27). The Pillar of Social
Rights of 16 November 2017 refers to the Social dialogue and involvement of workers as a
principle of a strong social Europe.9 The Union has established minimum standards in the
area of informing and consulting through European Works Councils through the directive
2009/38/EC.10 The current Proposal for a Directive on adequate minimum wages in the EU 11 goes
further and focuses on collective rights in the EU. Every Member State is obliged to take
measures to strengthen “social partners”, even more if collective bargaining coverage is less
than 70%.12 Although industrial relations in the member states differ greatly in their
regulatory approach, there is a convergence in the essential concepts of collective labour law.
This must be reflected in the legal approach of new phenomena in the world of work life.
   The regulation of the use of AI is still in its early stages. Nevertheless, first regulatory
approaches of the states and the collective actors show that employee participation and self-
regulation in labour law could be decisive in ensuring the use of AI contributes to a humane
world of work and reduces dangers associated with the introduction of AI.



6 Bodie M. T., Cherry M. The Law and Policy of People Analytics, in University of Colorado Law Review, 88, 4, 2017,
961, 1020 and ff.; Wachter. S., Mittelstadt B., Russell C., Why Fairness cannot be automated: Bridging the gap between
EU non-discriminiation law and AI, in Computer Law & Security Review, 41, 2021.
7 De Stefano V., Taes S. nt. (1), 7.
8 Köchling A., Riazy S., Wehner M.C., Simbeck K., Highly Accurate, But Still Discriminatory, in Business &

Information Systems Engineering, 63, 2021, 39-53; Orwat C., Diskriminierungsrisiken durch Verwendung von Algorithmen,
2019,
https://www.antidiskriminierungsstelle.de/SharedDocs/Downloads/DE/publikationen/Expertisen/studie_
diskriminierungsrisiken_durch_verwendung_von_algorithmen.pdf?__blob=publicationFile&v=62019,
accessed 3 Oct. 2021.
9 Nr. 8 of the European Pillar of Social Rights of the European Parliament, the Council and the Commission.
10 Schulten T., Müller T., A paradigm shift towards Social Europe? The proposed Directive on adequate minimum wages in

the European Union, in Italian Labour Law e-Journal, 14, 1, 2021, speak of a paradigm shift towards a social Europe,
see for the discussion the contributions in the Italian Labour Law e-Journal, Vol. 14 No. 1 (2021).
11 European Commission, Proposal for a Directive of the European Parliament and of the Council on adequate minimum

wages in the European Union, Brussels, 28 October 2020, COM(2020) 682 final, https://eur-lex.europa.eu/legal-
content/EN/TXT/PDF/?uri=CELEX:52020PC0682&from=EN, accessed 3 Oct. 2021.
12 Art. 4 of the Proposal.



https://doi.org/10.6092/issn.1561-8048/14099                                                                       160
     Ernesto Klengel                                                                         Italian Labour Law e-Journal
     Johanna Wenckebach                                                                             Issue 2, Vol. 14 (2021)
                                                                                                    Section: Miscellaneous




    For example, in Spain, according to the new drafting of Article 64(4) of the Ley des
Estatuto de los Trabajadores,13 the works council has a right to information on the
functioning of AI tools, albeit limited to "people who work on delivery and distribution".14
A weak point of the new regulation seems to be the enforcement in case of conflict.
Transparency provisions are included in Spanish collective agreements, for example in the
banking sector: the XXIV Convenio colectivo del sector de la banca contains digital rights
in Art. 80, with paragraph 5 addressing the transparency of AI. It obliges employers to
disclose to workers' representatives what data is being processed, the logic according to
which it works and how the results are to be classified.15 This example shows that it is crucial
to give the collective actors room for maneuvers in terms of requirements and procedures
tailored to sectors and well-established systems above minimum standards.
    Another good example is the situation in Germany, which will be explained below by way
of a case in point.


3.1. The legal framework of works council rights in AI in Germany.

    Workplace co-determination is one of the pillars of employee participation in Germany,
alongside trade union rights (collective agreements) and company co-determination in
supervisory boards. There is discussion about the extent to which the far-reaching works
council rights are applicable to AI systems.
    German works councils are no trade union bodies (although trade unions are represented
in them in many cases) but elected by the whole workforce.16 They do not have the right to
strike, instead they have various legally defined rights of participation. These range from
information rights and monitoring compliance with labour law to rigid rights of co-
determination, especially on collective issues (so-called “social matters”, whereas matters as
working time and remuneration are reserved for collective agreements of labour unions).
    Albeit the use of AI systems is technologically new, it can already be subsumed under the
regulations of the Works Constitution Act (BetrVG) in various respects.17 For example, the
works council must be informed about plans of the employer concerning technical plants,
working procedures and about manpower planning (Sections 90, 92). The information given
by the employer must be comprehensive and in a good time so that the body can fulfil its
legally defined tasks.



13 Introduced by Real Decreto – ley 9/2021, de 11 de mayo, https://boe.es/diario_boe/txt.php?id=BOE-A-
2021-7840, accessed 3 Oct. 2021.
14 Villarroel Luque C., Workers vs Algorithms – What can the new Spanish provision on artificial intelligence and employment

achieve?, 2021, https://verfassungsblog.de/workers-vs-ai/, accessed 3 Oct. 2021; Todolí-Signes A., Spanish riders
law and the right to be informed about the algorithm, in Euopean Labour Law Journal, 12, 3, 2021, 399–402.
15 See https://www.boe.es/diario_boe/txt.php?id=BOE-A-2021-5003, accessed 3 Oct. 2021.
16 For more details see Weiss M., Schmidt M., Hlava D., Labour Law and Industrial Relations in Germany, 5th ed.,

2020, para. 577.
17 Klebe T.: Künstliche Intelligenz – eine Herausforderung für die Mitbestimmung, in Soziales Recht 2018, 128 and ff.; an

English version of the Act can be found at www.gesetze-im-internet.de/englisch_betrvg, accessed 3 Oct. 2021.

https://doi.org/10.6092/issn.1561-8048/14099                                                                            161
     Ernesto Klengel                                                                   Italian Labour Law e-Journal
     Johanna Wenckebach                                                                       Issue 2, Vol. 14 (2021)
                                                                                              Section: Miscellaneous




    In the case that AI-technologies lead to job losses, works councils have options to mitigate
the consequences for workers. Above all, further training and qualification are crucial to
prevent job losses due to digitalisation. Both, the fundamental change in requirement profiles
through AI and the elimination of jobs in the longer term, must be countered through
education. Qualification and training during ongoing employment for workers whose jobs
are likely to be lost due to future digitalisation and ecologic transformation is a main strategy
of the German labour market policy to prevent unemployment and social divide.18 Therefore,
the existing co-determination rights in qualification and further vocational training are all the
more important.
    The introduction of AI technologies may lead to increased intensity and monitoring
pressure. A mandatory co-determination right exists (section 87 (1) no. 7 of the Works
Constitution Act). Of particular note are the far-reaching rights of the works council in the
protection of personality, which also includes the protection of personal data. But the
relevant section 87 (1) no. 6 BetrVG was created in the 1970s when the technical facilities
were still rather rudimentary.19 The technical possibilities and with them the dangers of
monitoring and performance control have increased since then. Case law suggests that the
introduction and modification of all technical equipment (including software) that processes
employee data, which is common in AI applications, is covered by this co-determination
right.20 That means: The tool is not be used until the works council gives its consent or the
consent is replaced by a conciliation committee.21 What is missing is the statutory right of
the works council to take the initiative itself.22 Other co-determination rights relate to
guidelines on personnel selection.
    The overview shows that, regardless of reforms for co-determination of AI systems, in
theory, AI already is subject to co-determination in the German labour law system.


3.2. AI in the practice of workplace co-determination.

   Works councils have several functions in the introduction of AI: Firstly, they monitor
compliance with the individual rights of workers, which include health, privacy and
protection against discrimination. Works councils are thus guarantors for the implementation
of the legal framework for AI. In addition, they have a policy-shaping role: In the best case,


18 Federal Government, Entwurf eines Gesetzes zur Stärkung der Chancen für Qualifizierung und für mehr Schutz in der
Arbeitslosenversicherung (Qualifizierungschancengesetz) of 19 Sep. 2018, https://www.bmas.de/DE/Service/Gesetze-
und-
Gesetzesvorhaben/qualifizierungschancengesetz.html;jsessionid=90572B9411AFADB80713BB3DA44555E5
.delivery1-master, accessed 3 Oct. 2021.
19 For more details on the concept of co-determination, see Weiss M., Schmidt M., Hlava D., nt. (16), para 620

and ff.
20 Settled case law, cf. Federal Labour Court, order of 14 May 1974 – 1 ABR 45/73; see also Fitting, BetrVG

commentary, 30th ed., 2020, § 87 para. 226.
21 See Däubler W., Klebe T., Wedde P., BetrVG commentary, 17th ed. 2020, § 87 para. 5 and ff.
22 Federal Labour Court, order of 28 November 1989, 1 ABR 97/88 – a position that is currently increasingly

being challenged, see e.g. Regional Labour Court Hamm, order of 27 July 2021 – 7 TaBV 79/20; Klebe, T., nt.
(17), 135 and ff.

https://doi.org/10.6092/issn.1561-8048/14099                                                                      162
     Ernesto Klengel                                                                     Italian Labour Law e-Journal
     Johanna Wenckebach                                                                         Issue 2, Vol. 14 (2021)
                                                                                                Section: Miscellaneous




they help to structure the processes concerning the introduction, application, evaluation and
further development of new technical systems in the workplace. Collective actors relieve the
individual worker of the burden of litigating as an individual with her or his employer, on
whom she or he is dependent. This makes works councils a decisive factor for better
enforcement of individual rights in labour law.
    When exercising these rights, works councils in Germany generally take a constructive
approach. The reason for this is probably also to be found in their legal constitution and
their operational focus: The company agreements concluded by works councils have a lower
legal status than collective agreements reached by trade unions. Works councils are, as
mentioned above, prohibited from calling strikes and advertising for trade unions, and their
bodies are limited to the company, the enterprise and, at most, the group. Trade secrets must
be kept.
    Contrary to what many employers claim, works councils therefore prove in practice not
to be impeders, but often even drivers of digitalisation, or at least actors who put the use of
AI systems on a more rational basis.23 Comprehensive studies show that co-determination is
also a decisive factor for the willingness of the workforce to contribute with their potential
to change and see digitalisation as a positive, less frightening process.24 Employers often lack
a future concept for digitalisation, which recently became clear in a survey in the industrial
sector.25 Moreover, it is often not clear what an IT tool actually does and whether it can fulfil
the provider’s promise. AI systems turn out to be a black box, and some employers claim
that their development has taken place outside their sphere of influence and therefore neither
influence could be exerted, nor transparency established. Works councils that persistently
ask critical questions and demand information can play an important role here. A problem
in practice is often that works councils are informed about new technologies too late to be
able to assess the consequences and initiate their own checks. In general, works council rights
are often disregarded, especially in small and medium-sized companies. If not a workers
representation is missing at all:
    Co-determination practice in AI in Germany has other, not specifically AI-related deficits.
One is the permanently decreasing number of works councils. In the absence of a works
council, the aforementioned rights do not apply. Especially in the IT sector, works councils
are often only established when the employer runs into economic difficulties. In part, the
Works Council Modernisation Act has currently addressed this issue (see the next
subsection).26 Furthermore, many works councils are far from being able to exercise the
rights to which they are entitled. There is often a lack of resources to deal with the complex

23 See e.g. the case studies of Massolle J., Die Transformation von Arbeit mitgestalten, 2021, https://www.imu-
boeckler.de/fpdf/HBS-008086/p_mbf_praxis_2021_41.pdf, accessed 3 Oct. 2021; Falkenberg J., Haipeter T.,
Krzywdzinski M., Kuhlmann M., Schietinger M., Virgillito A., Digitalisierung in Industriebetrieben, 2020,
https://www.boeckler.de/fpdf/HBS-007638/p_fofoe_report_006_2020.pdf, accessed 3 Oct. 2021.
24 Kirchner S., Zeit für ein Update – Was die Menschen in Deutschland über Digitalisierung denken, 2019,

http://library.fes.de/pdf-files/fes/15549.pdf, accessed 3 Oct. 2021.
25 Allmendinger J., Schroeder W., Die Situation von Industriebeschäftigten während der Corona-Pandemie: Ergebnisse der

Beschäftigtenbefragung 2020 der IG Metall, 2021, https://bibliothek.wzb.eu/pdf/2021/p21-001.pdf, accessed 3
Oct. 2021.
26 Wenckebach J., Eine echte Modernisierung steht weiterhin aus, 2021, https://www.boeckler.de/newsletter-

rest/t/review/ANONYMOUS.2H3SZ.3C0B6F1843878586EC95A652D52FEA52/, accessed 3 Oct. 2021.

https://doi.org/10.6092/issn.1561-8048/14099                                                                        163
     Ernesto Klengel                                                               Italian Labour Law e-Journal
     Johanna Wenckebach                                                                   Issue 2, Vol. 14 (2021)
                                                                                          Section: Miscellaneous




and – in the view of a large number of systems in use – extensive issues. Works councils are
vulnerable to pressure, especially in difficult economic situations, such as when employers
threaten relocations.
    Trade union rights in the workplace are therefore crucial. Both for works councils and
trade unions, the isolation and individualisation of working people caused by digitalisation is
a problem. It is not only a question of how solidarity can be organised when, for example,
crowd workers meet less frequently, and more and more people work remote. Direct
communication between workers and unions – starting point for any collective movement
and solidarity – no longer takes place in all workplaces of the digitalized world of work. Here,
too, the applicable law reaches its limits. This is why in Germany, digital access rights for
trade unions and works councils are being discussed to expand the legal framework for this.
It is about access to apps, intranet sites, email addresses and chat groups.


3.3. The 2021 Law on the Modernisation of Works Councils.

    The challenges posed by AI to the pillar of collective rights in its German version have
already given rise to some legal reforms. The last major revision of the law on workplace co-
determination took place decades ago and coincided with a time when technological
developments of today seemed utopian. Now, digitalisation, but also globalisation, pose
major challenges to labour law as a whole. It is being questioned whether central labour law
concepts such as employee, employer and establishment will capture the reality of the
working world of the future.27 The changes do not only affect new forms of employment
such as crowd and gig work, but also traditional industrial companies. In addition to
digitalisation, the ecological transformation of the economy is a central social challenge. The
question is whether the "Green Deal" will also be a "social deal" – a European question.
Trade unions and academics are calling for a democratisation of these transformation
processes, which have the dimension of an industrial revolution.28
    Legislators in Germany have recognised the issue. In spring 2021, the coalition of
Conservatives and Social Democrats had set out to reform workplace co-determination –
not only about shaping digitalisation, but also about curbing union busting and counteracting
the trend that fewer and fewer employees in Germany have a representation of their interests
at their side.29 In addition to the slightly simplified and better-protected election of works
councils, new regulations for co-determination in the use of AI in the company were a focal
point: The rights of the works council in the design of the working environment and work
procedures and operations will now explicitly also cover the use of AI. The rights of the
works council in setting up guidelines for the selection of personnel shall also apply if they


27 See for the concept of the employer e.g. the contributions on the concept of “employer” in Vol. 13 No. 1 of
the Italian Labour Law e-Journal.
28 Boeckler Impuls 7/2021 https://www.boeckler.de/de/boeckler-impuls-no-green-deal-without-a-social-

deal-32649.htm; https://democratizingwork.org/, accessed 3 Oct. 2021.
29 Explanatory memorandum to the law, Bundestags-Drucksache (printed material of Bundestag) 19/28899 of

22 April 2021, p. 12 and ff.

https://doi.org/10.6092/issn.1561-8048/14099                                                                  164
     Ernesto Klengel                                                                       Italian Labour Law e-Journal
     Johanna Wenckebach                                                                           Issue 2, Vol. 14 (2021)
                                                                                                  Section: Miscellaneous




have been created by or with the help of algorithms. These regulations are important
clarifications of the legal situation as it is already widely shared today.30
   It was also clarified that in order to carry out its duties under the Works Constitution Act,
the works council must assess the introduction or application of AI and can call in an expert,
who must be paid by the employer. This is the legislator's response to practical needs. It
requires know-how to be able to assess algorithms. Democratic rights only make sense if
collectives have access to this expertise and the financing of technical expertise is ensured.
Even if there are certified AI applications at some point, this practical need for advice will
not disappear.


4. The Commission’s draft of an AI Act.

    The EU is tackling the issue of regulating AI, too. The Commission published a
"Regulatory Framework for AI" in April 2021, which resulted from a consultation process
which took several years.31 This regulatory framework lays down harmonized rules for
introduction of AI in the market. Its consequences for the labour relations are only one
aspect. The act implements a "risk-based" approach that classifies different AI applications
into different risk categories (Articles 5, 6 of the proposal): unacceptable risk, high risk, low
risk and minimal risk.
    The good news is: Work-related AI systems are to be classified as "high-risk", at least
according to recital 36 of the proposal. More concrete is No. 4 Annex III to the proposal, in
which, based on Article 6(2), Article 7(1)(b), (2), the Commission defines certain categories
of AI systems whose risks have already materialised or there is a tangible risk of them doing
so: According to No. 4 of Annex III, recruiting systems as well as AI used to decide on the
beginning and end of the contractual relationship, for task assignment and for monitoring,
performance and success control are qualified as high-risk. This by no means covers all
relevant risk areas of working life. And besides that, in practical application, it will be
questionable when an AI is "intended to be used (…) for monitoring and performance
control and behavior" and whether its suitability for monitoring and performance control
should not suffice. After all, the particular risk of AI lies precisely in the fact that personal
data, once collected, form a panopticon of monitoring so the suitability of the software
should be sufficient, its purpose is not what matters (on the German legal situation in the
context of the works constitution, see above under point 2 a).
    Providers of AI systems bear several obligations, of which only the most important can
be mentioned here. According to the proposal, a detailed risk management system is
required. This should be applied during the entire life cycle of the system and includes various
defined steps. AI-tools must be tested which ensures that high-risk AI systems perform

30   Klebe T., „Nur ein Schritt in die richtige Richtung“, 2021, https://www.bund-
verlag.de/betriebsrat/aktuellesbr~Nur-ein-Schritt-in-die-richtige-Richtung~.html, accessed 3 Oct. 2021.
31 European Commission: Proposal für a Regulation laying down harmonized rules on artificial intelligence, 21 April 2021,

COM(2021) 206 final, https://digital-strategy.ec.europa.eu/en/library/proposal-regulation-laying-down-
harmonised-rules-artificial-intelligence, accessed 3 Oct. 2021.

https://doi.org/10.6092/issn.1561-8048/14099                                                                          165
     Ernesto Klengel                                                                              Italian Labour Law e-Journal
     Johanna Wenckebach                                                                                  Issue 2, Vol. 14 (2021)
                                                                                                         Section: Miscellaneous




consistently for their intended purpose and follow the legal requirements, Article 9. It also
specifies a certain procedure for AI systems that are trained with user data whereby biases
are to be avoided, inter alia (Article 10). Furthermore, the system must be subject to ex ante
technical documentation (Article 11). The principles of transparency, human oversight,
verifiability, accuracy, reliability (Articles 12 – 15) are laid down for dealing with AI. These
principles must already be fulfilled in the design and development of the systems. For
example, events must be automatically documented (logs). Users must be able to interpret
the system’s output and use it appropriately and human oversight shall prevent risks for the
health, safety, or fundamental rights. Providers of AI systems are obliged to introduce a
quality management system (Article 17). Of particular importance is that providers must
subject the AI tool to a conformity assessment procedure (Article 19), which can, however,
take the form of a self-assessment (Article 43 (2) of the proposal and its Annex VI).32
Providers must carry out quality and risk management and are subject to ex-post control and
enforcement.
   With the transparency requirements, the Commission's regulatory proposal takes up an
important regulatory approach known from data protection, which should be consistently
pursued. Given the wide range of duties, however, the crux of the matter is: Under which
circumstances are employers AI providers? According to Article 3 (2), Article 28(1) they are
in the role of providers, if they put an AI system into service under its own name or
trademark, modify the intended purpose of a high-risk AI system already put into service or
make a substantial modification to the high-risk AI system. This may be the case but does
not seem to be the norm. Rather, they will often qualify as users, for whom most of the
duties do not apply.
   The draft’s lack of labour law is further exacerbated by the fact that employees cannot be
regarded as "users" of an AI system within the meaning of the Directive. According to Article
3 (4) of the draft, only those “using an AI system under its authority” are to be considered
as such. In an employment relationship, this is the employer. And in the important case of
platform work, it must be questioned whether the tools used by employers are to be regarded
as AI in the sense of the Directive.33 Thus, the draft does not provide for specific individual
workers' rights in the use of AI. It is hardly conceivable that this will be realised in the final
regulation.
   In addition, the regulatory approach lays down a minimum standard; stricter protection
by the member states is not envisaged.34 Against the background of collective bargaining
autonomy, it should be made explicitly clear that higher standards can always be set by the
parties to collective agreements. The mechanisms of the employment relationship – the
unequal power relationship and the higher personal attachment of workers to the employer
– may mean that this level will often be undercut without consequences. In particular,



32 Battista L., The European Framework Agreement on Digitalisation: a tough coexistence within the EU mosaic of actions, in
Italian Labour Law e-Journal, 1, 14, 2021, 105, 119: “disappointing”.
33 Purificato I., Behind the Scenes of Deliveroo’s Algorithm: in the Blindness of “Frank” its Discriminatory Potential, in Italian

Labour Law e-Journal, 1, 14, 2021, 169, 189 and ff.
34 De Stefano V., Taes S., nt. (1), 11.



https://doi.org/10.6092/issn.1561-8048/14099                                                                                  166
     Ernesto Klengel                                                                      Italian Labour Law e-Journal
     Johanna Wenckebach                                                                          Issue 2, Vol. 14 (2021)
                                                                                                 Section: Miscellaneous




employers – within the framework of the general explainability of the system35 – should not
be able to rely on the absence of knowledge how the AI tool works. To avoid but also
effectively prevent or defeat discrimination, data sets must not only be tested as foreseen in
Art. 10. It has to be made sure that works councils or other workers representatives have
access to those data sets. Training, testing and validation of data cannot be left to the
employer alone.36 The collective element is at the core of labour law – and, also seen
internationally – of data protection law.37
    The short summary of AI regulation is therefore: the EU Commission leaves labour law
out of the equation when regulating AI. The big step backwards compared to the European
Pillar of Social Rights as well as regulation at member state level is that there is no mention
of collective rights at all. This has to be made up for.


5. Conclusions.

   The EU's regulatory approach reveals an important trend in the regulation of the working
world of the future and especially in the use of AI: the strong role of data protection in a
broad understanding of the word. This is because the mass collection, gathering, processing
and transmission of personal data for the purposes of performance optimisation and
behavioural control lead to a new dimension in the control of employee behaviour. This is
linked to new threats to the mental and emotional health of employees. Just one example:
Germany has been increasing the number of sick leave notifications due to mental health
problems for years.38 The protection of data and the personality of workers is thus at the
same time a facet of health protection.
   Besides that, the wording of the proposal remains too vague at crucial points and needs
to be concretised in order to be applicable and helpful – therefore enforceable – in labour
law conflicts. The scope of Annex III is too narrow. Any AI system that processes employee
data should be categorised as high-risk technology. With the experience of union busting
strategies from the US of spying on workers activities for unionisation39 it has to be made
sure that any Big Data analytics on trade union activities should be recognised as high-risk
technology. Furthermore, individual rights for employees are needed to protect them from
discrimination and data misuse.

35 Käde L., v. Maltzan S., Die Erklärbarkeit von Künstlicher Intelligenz (KI), in Computer und Recht, 36, 1, 2020, p. 1.
36 Battista L., nt. (32), 105, 119; Purificato I., nt. (33), 169, 190.
37 De Stefano V., Negotiating the algorithm, in Comparative Labour Law and Policy Journal, 41, 1, 2019, 15, 41.
38 This is shown, for example, by the absenteeism report of the AOK, the largest German health insurance

association: Meyer M., Wiegand S., Schenkel A., Krankheitsbedingte Fehlzeiten in der deutschen Wirtschaft 2019, 2020,
403,
https://www.wido.de/fileadmin/Dateien/Dokumente/Publikationen_Produkte/Buchreihen/Fehlzeitenrepo
rt/wido_pra_fzr_2020_krankheitsbedingte_fehlzeiten.pdf, accessed 3 Oct. 2021. In the period from 2010 to
2019, absenteeism due to mental illness has successively increased by a total of 30%, in contrast to the other
types of illness.
39 Amazon uses social media to track environmental activism and social movements in Europe, see Gurley L.

K., Secret Amazon Reports Expose the Company’s Surveillance of Labor and Environmental Groups, in VICE,
https://www.vice.com/en/article/5dp3yn/amazon-leaked-reports-expose-spying-warehouse-workers-labor-
union-environmental-groups-social-movements, accessed 3 Oct. 2021.

https://doi.org/10.6092/issn.1561-8048/14099                                                                         167
     Ernesto Klengel                                                     Italian Labour Law e-Journal
     Johanna Wenckebach                                                         Issue 2, Vol. 14 (2021)
                                                                                Section: Miscellaneous




    Individuals are usually reluctant to take legal action against their employer, and often lack
crucial resources. The existing EU instruments on employee involvement are not adequate
to address the challenges associated with the introduction of AI, although it is hardly
doubtful that their scope can be opened beside the AI Act with the introduction of an AI-
tool. Article 4(2) Directive 2002/14/EC provides for informing and consulting employees
in the event that a company decision could have an impact on the employment situation in
the company or may affect the organisation of work. Both can occur with the introduction
of an AI tool, but does not have to. A clarification that AI tools meet these requirements could
help to prevent disputes at company level. However, based on the experience that AI tools
overwhelm works councils due to their scope and technical complexity, it would be more
important to give councils access to IT expertise and the necessary resources, as has been
regulated in Germany.
    For this very reason, it is a central shortcoming of the draft that the collective mechanisms
of labour law are not addressed – trade unions and employers' associations, but also
workplace co-determination, are not even mentioned. What is needed are procedural
regulations that give collective actors substantial competences in the compliance and humane
design of work with AI-supported work equipment.40 The management systems and
assessment procedures provided for in Art. 17 and Art. 43 must be democratic from the
beginning, i.e. they must provide for co-determination. In this context, it is important to
involve collective actors not only when the procedures to be created are not complied with,
but also when they are being designed. The AI Act, which may regulate the admission
procedures of such tools across the EU, is an excellent opportunity for this. What is more:
In the interest of effective enforcement of the regulation, class action should be considered.
In any case, it would be desirable to clarify that the implementation of the AI-Regulation
must not lag the application of EU-law on information and consultation and the enforcement
of national labour law, which includes collective means. The draft lacks a provision such as
Art. 88 GDPR to clearly state that the AI Act leaves room for more suitable and specific
measures by collective agreements, especially for those, which ensure the protection of
employee’s rights. This is the only way to ensure that Art. 28 CFR – the right of collective
bargaining and action – is adequately taken into account.
    At the same time, AI puts an instrument in the hands of employers that facilitates the
reorganisation of work and thus helps to circumvent labour standards. To counter this, it
would help to make the European legal acts protecting against precarious employment –
whether due to AI or not – more effective. The health risks outlined can be countered by
strengthening the rights of works councils in occupational health and safety, with staffing
measurements being highlighted. Qualification is another important area for the enabling of
employees, but also for securing jobs. The Law on Modernising Works Councils in Germany
recognizes the importance of this field of action but does not go far enough: it retains rights
of co-determination, but in cases of conflict the employer prevails if an attempt to reach
agreement – albeit a mandatory one – fails. The reform does not provide for procedural co-


40   De Stefano V., Taes S., nt. (1), 9.

https://doi.org/10.6092/issn.1561-8048/14099                                                        168
 Ernesto Klengel                                                            Italian Labour Law e-Journal
 Johanna Wenckebach                                                                Issue 2, Vol. 14 (2021)
                                                                                   Section: Miscellaneous




determination rights, which are also necessary for the use of AI. Neither does a digital right
of access to the digital workplace.
   The regulation of AI must also take into account at the European level that algorithms in
the world of work do not only raise technical or legal questions, but also questions of power.
Collective actors – i.e. trade unions and works councils – are a crucial counterweight here.
Fair AI requires their involvement and empowerment. Otherwise, the intended "human-
centered" approach to AI in the world of work remains just a phrase.




                                               Bibliography


Adams-Prassl J., What if your Boss was an Algorithm? Economic Incentives, Legal Challenges, and the
Rise of Artificial Intelligence at Work, in Comparative Labour Law and Policy Journal, 41, 1, 2019;
Allmendinger J., Schroeder W., Die Situation von Industriebeschäftigten während der Corona-
Pandemie: Ergebnisse der Beschäftigtenbefragung 2020 der IG Metall, 2021,
https://bibliothek.wzb.eu/pdf/2021/p21-001.pdf, accessed 3 Oct. 2021;
Battista L., The European Framework Agreement on Digitalisation: a tough coexistence within the EU
mosaic of actions, in Italian Labour Law e-Journal, 1, 14, 2021, 105-121.
Bodie M. T., Cherry M. The Law and Policy of People Analytics, in University of Colorado Law
Review, 88, 4, 2017, 961-1042;
Däubler W., Klebe T., Wedde P, BetrVG commentary, 17th ed. 2020, § 87 para. 5 and ff;
De Stefano V., Negotiating the algorithm, in Comparative Labour Law and Policy Journal, 41, 1, 2019;
De Stefano V., Taes S., Algorithmic management and collective bargaining, in Foresight Brief, ETUI,
#10 – May 2021, 2021, https://www.etui.org/publications/algorithmic-management-and-
collective-bargaining, accessed 3 Oct. 2021;
Falkenberg J., Haipeter T., Krzywdzinski M., Kuhlmann M., Schietinger M., Virgillito A.,
Digitalisierung      in       Industriebetrieben,   2020,       https://www.boeckler.de/fpdf/HBS-
007638/p_fofoe_report_006_2020.pdf, accessed 3 Oct. 2021;
Federal Government of Germany, Digitalisierung geschlechtergerecht gestalten – Gutachten für den
Dritten Gleichstellungsbericht der Bundesregierung, 2020, p. 91 and ff., https://www.dritter-
gleichstellungsbericht.de/de/topic/73.gutachten.html, accessed 3 Oct. 2021;
Fitting, BetrVG commentary, 30th ed., 2020, § 87, para. 226;
Gießler          S.,         Was         ist      automatisiertes      Personalmanagement?,      2021,
https://algorithmwatch.org/de/wp-content/uploads/2021/05/Was-ist-automatisiertes-
Personalmanagement-Giesler-AlgorithmWatch-2021.pdf, accessed 3 Oct. 2021;
Gurley L. K., Secret Amazon Reports Expose the Company’s Surveillance of Labor and Environmental
Groups, in VICE, https://www.vice.com/en/article/5dp3yn/amazon-leaked-reports-
expose-spying-warehouse-workers-labor-union-environmental-groups-social-movements,
accessed 3 Oct. 2021;


https://doi.org/10.6092/issn.1561-8048/14099                                                           169
 Ernesto Klengel                                                               Italian Labour Law e-Journal
 Johanna Wenckebach                                                                   Issue 2, Vol. 14 (2021)
                                                                                      Section: Miscellaneous




Ivanova M., Bronowicka J., Kocher E., Degner A., Foodora and Deliveroo: The App as a Boss?,
2018, https://www.boeckler.de/de/faust-detail.htm?sync_id=HBS-07048, accessed 3 Oct.
2021;
Käde L., v. Maltzan S., Die Erklärbarkeit von Künstlicher Intelligenz (KI), in Computer und Recht,
36, 1, 2020, 1;
Kirchner S., Zeit für ein Update – Was die Menschen in Deutschland über Digitalisierung denken, 2019,
http://library.fes.de/pdf-files/fes/15549.pdf, accessed 3 Oct. 2021;
Klebe T., Künstliche Intelligenz – eine Herausforderung für die Mitbestimmung, in Soziales Recht 2018,
128 and ff;
Klebe T., „Nur ein Schritt in die richtige Richtung“, 2021, https://www.bund-
verlag.de/betriebsrat/aktuellesbr~Nur-ein-Schritt-in-die-richtige-Richtung~.html, accessed
3 Oct. 2021;
Köchling A., Riazy S., Wehner M.C., Simbeck K., Highly Accurate, But Still Discriminatory, in
Business & Information Systems Engineering, 63, 2021, 39-53;
Massolle J., Die Transformation von Arbeit mitgestalten, 2021, https://www.imu-
boeckler.de/fpdf/HBS-008086/p_mbf_praxis_2021_41.pdf, accessed 3 Oct. 2021;
Meyer M., Wiegand S., Schenkel A., Krankheitsbedingte Fehlzeiten in der deutschen Wirtschaft 2019,
2020,                                                                                               403,
https://www.wido.de/fileadmin/Dateien/Dokumente/Publikationen_Produkte/Buchreih
en/Fehlzeitenreport/wido_pra_fzr_2020_krankheitsbedingte_fehlzeiten.pdf, accessed 3
Oct. 2021;
Orwat C., Diskriminierungsrisiken durch Verwendung von Algorithmen, 2019,
https://www.antidiskriminierungsstelle.de/SharedDocs/Downloads/DE/publikationen/E
xpertisen/studie_diskriminierungsrisiken_durch_verwendung_von_algorithmen.pdf?__blo
b=publicationFile&v=62019, accessed 3 Oct. 2021;
Purificato I., Behind the Scenes of Deliveroo’s Algorithm: in the Blindness of “Frank” its Discriminatory
Potential, in Italian Labour Law e-Journal, 1, 14, 2021, 169-194;
Schubert C., Hütt M., Economy-on-demand and the fairness of algorithms, in European Labour Law
Journal, 10, 1, 2019.
Schulten T., Müller T., A paradigm shift towards Social Europe? The proposed Directive on adequate
minimum wages in the European Union, in Italian Labour Law e-Journal, 14, 1, 2021;
Todolí-Signes A., Spanish riders law and the right to be informed about the algorithm, in Euopean
Labour Law Journal, 12, 3, 2021, 399–402;
Villarroel Luque C., Workers vs Algorithms – What can the new Spanish provision on artificial
intelligence and employment achieve?, 2021, https://verfassungsblog.de/workers-vs-ai/, accessed
3 Oct. 2021;
Wachter. S., Mittelstadt B., Russell C., Why Fairness cannot be automated: Bridging the gap between
EU non-discriminiation law and AI, in Computer Law & Security Review, 41, 2021;
Weiss M., Schmidt M., Hlava D., Labour Law and Industrial Relations in Germany, 5th ed., 2020,
para. 577;
Wenckebach           J.,   Eine       echte    Modernisierung      steht     weiterhin    aus,     2021,
https://www.boeckler.de/newsletter-


https://doi.org/10.6092/issn.1561-8048/14099                                                              170
 Ernesto Klengel                                                Italian Labour Law e-Journal
 Johanna Wenckebach                                                    Issue 2, Vol. 14 (2021)
                                                                       Section: Miscellaneous




rest/t/review/ANONYMOUS.2H3SZ.3C0B6F1843878586EC95A652D52FEA52/,
accessed 3 Oct. 2021.




Copyright © 2021 Ernesto Klengel, Johanna Wenckebach. This article is released under a
Creative Commons Attribution 4.0 International License




https://doi.org/10.6092/issn.1561-8048/14099                                               171
