Human oversight and risk-based approach to artificial intelligence: What does the Artificial Intelligence Act have in common with discussions about lethal autonomous weapon systems?
DOI:
https://doi.org/10.46941/2024.2.8Keywords:
Artificial Intelligence, Ethics, European Union, Lethal Autonomous Weapon Systems, Human Oversight, Risk-Based Approach.Abstract
The regulation of artificial intelligence (AI) is a pressing global concern, with various regulatory bodies aiming to foster innovation while safeguarding humanity’s interests. This article synthesises perspectives on AI regulation in civilian and military domains, highlighting common ethical foundations and legal proposals. Emphasising the European Union’s ethical community as delineated by fundamental rights, it explores the Artificial Intelligence Act and debates on lethal autonomous weapon systems within the Convention on Certain Conventional Weapons. By analysing the overlap between civilian and military ethics, the article argues for a shared objective: promoting innovation while upholding human dignity through robust regulations that ensure human oversight and a risk-based approach. The article contends that the consensus on substantive issues regarding military AI regulation is imminent, but its formalisation through legal means may lag behind.
References
Anand, A., Deng H. (2023) Towards Responsible AI in Defence: A Mapping and Comparative Analysis of AI Principles Adopted by States. [Online]. Available at https://unidir.org/publication/towards-responsible-ai-in-defence-a-mapping-and-comparative-analysis-of-ai-principles-adopted-by-states/ (Accessed: 1 July 2024).
Bo, M., Dorsey, J. (2024) Symposium on Military AI and the Law of Armed Conflict: The ‘Need’ for Speed – The Cost of Unregulated AI-Decision Support Systems to Civilians. Opinio Juris. [Online]. Available at: https://opiniojuris.org/2024/04/04/symposium-on-military-ai-and-the-law-of-armed-conflict-the-need-for-speed-the-cost-of-unregulated-ai-decision-support-systems-to-civilians/ (Accessed: 10 April 2024).
Bode, I. (2024) The problem of algorithmic bias and military applications of AI. Humanitarian Law & Policy Blog. [Online]. Available at: https://blogs.icrc.org/law-and-policy/2024/03/14/falling-under-the-radar-the-problem-of-algorithmic-bias-and-military-applications-of-ai/ (Accessed: 29 March 2024).
Díaz-Rodríguez, N., Del Ser, J., Coeckelbergh, M., López de Prado, M., Herrera-Viedma, H., Herrera, F. (2023) ‘Connecting the dots in trustworthy Artificial Intelligence: From AI principles, ethics, and key requirements to responsible AI systems and regulation’ Information Fusion 99, p. 101896; https://doi.org/10.1016/j.inffus.2023.101896.
Enqvist, L. (2023) ‘Human oversight’ in the EU artificial intelligence act: what, when and by whom?’ Law, Innovation and Technology 15(2), pp. 508–535; https://doi.org/10.1080/17579961.2023.2245683.
Kowalczewska, K. (2021a) Unia Europejska wobec autonomicznych systemów śmiercionośnych broni (LAWS) – znacząca ludzka kontrola jako fundament wiarygodnej sztucznej inteligencji in Fischer, B., Pązik, A., Świerczyński, M. (eds.) Prawo sztucznej inteligencji i nowych technologii, Wolters Kluwer Polska, pp. 465–486.
Kowalczewska, K. (2021b) Sztuczna inteligencja na wojnie. Perspektywa MPHKZ. Przypadek autonomicznych systemów śmiercionośnej broni. Wydawnictwo Naukowe Scholar.
Laux, J. (2023) ‘Institutionalised distrust and human oversight of artificial intelligence: towards a democratic design of AI governance under the European Union AI Act’, AI & SOCIETY; https://doi.org/10.2139/ssrn.4377481.
Michel, A. (2020) The Black Box, Unlocked: Predictability and Understandability in Military AI. [Online]. Available at: https://unidir.org/publication/the-black-box-unlocked/ (Accessed: 1 July 2024).
Michel, A. (2021) Known Unknowns: Data Issues and Military Autonomous Systems. United Nations Institute for Disarmament Research. [Online]. Available at: https://unidir.org/publication/known-unknowns (Accessed: 1 July 2024).
Puscas, I. (2022) Human-Machine Interfaces in Autonomous Weapon Systems. [Online]. Available at: https://unidir.org/publication/human-machine-interfaces-in-autonomous-weapon-systems/ (Accessed: 1 July 2024).
Puscas, I. (2023) AI and International Security: Understanding the Risks and Paving the Path for Confidence-Building Measures. [Online]. Available at: https://unidir.org/publication/ai-and-international-security-understanding-the-risks-and-paving-the-path-for-confidence-building-measures/ (Accessed: 1 July 2024).
Ramos, G., Squicciarini, M., Lamm, E. (2024) ‘Making AI Ethical by Design: The UNESCO Perspective’, Computer, 57(2), pp. 33–43; https://doi.org/10.1109/MC.2023.3325949.
Ruschemeier, H. (2023) ‘AI as a challenge for legal regulation – the scope of application of the artificial intelligence act proposal’, ERA Forum, 23(3), pp. 361–376; https://doi.org/10.1007/s12027-022-00725-6.
Stix, Ch. (2021) ‘Actionable Principles for Artificial Intelligence Policy: Three Pathways’, Science and Engineering Ethics 27(1), p. 15; https://doi.org/10.1007/s11948-020-00277-3.
Wouters, J. (2020) From an economic community to a union of values: The emergence of the EU’s commitment to human rights. in Wouters, J. et al. The European Union and Human Rights. Oxford University Press. pp. 11-38. https://doi.org/10.1093/oso/9780198814191.003.0002.
Załucki, M., Miraut, M. (2021). Artificial intelligence and human rights. Dykinson.
Article 36. (2023) Completely outside human control? [Online]. Available at: https://article36.org/wp-content/uploads/2023/03/Completely-outside-human-control.pdf (Accessed: 29 March 2024).
CCW (2022) Working paper submitted by Finland, France, Germany, the Netherlands, Norway, Spain, and Sweden to the 2022 Chair of the Group of Governmental Experts (GGE) on emerging technologies in the area of lethal autonomous weapons systems (LAWS). [Online]. Available at: https://documents.unoda.org/wp-content/uploads/2022/07/WP-LAWS_DE-ES-FI-FR-NL-NO-SE.pdf (Accessed: 1 July 2024).
CCW (2023a) Draft articles on autonomous weapon systems – prohibitions and other regulatory measures on the basis of international humanitarian law (“IHL”). [Online]. Available at: https://docs-library.unoda.org/Convention_on_Certain_Conventional_Weapons_-Group_of_Governmental_Experts_on_Lethal_Autonomous_Weapons_Systems_(2023)/CCW_GGE1_2023_WP.4_US_Rev2.pdf (Accessed: 1 July 2024).
CCW (2023b) Non-exhaustive compilation of definitions and characterizations. [Online]. Available at: https://docs-library.unoda.org/Convention_on_Certain_Conventional_Weapons_-Group_of_Governmental_Experts_on_Lethal_Autonomous_Weapons_Systems_(2023)/CCW_GGE1_2023_CRP.1_0.pdf (Accessed: 1 July 2024).
Convention for the Protection of Human Rights and Fundamental Freedoms (1950).
Charter of Fundamental Rights of the European Union (2000).
European Parliament (2024) Artificial Intelligence Act. European Parliament legislative resolution of 13 March 2024 on the proposal for a regulation of the European Parliament and of the Council on laying down harmonised rules on Artificial Intelligence (Artificial Intelligence Act) and amending certain Union Legislative Acts. [Online]. Available at: https://www.europarl.europa.eu/RegData/seance_pleniere/textes_adoptes/definitif/2024/03-13/0138/P9_TA(2024)0138_EN.pdf (Accessed: 29 March 2024).
Human Rights Watch. (2012) Losing humanity: the case against killer robots. Amsterdam Berlin: Human Rights Watch. [Online]. Available at: https://www.hrw.org/sites/default/files/reports/arms1112_ForUpload.pdf (Accessed: 29 March 2024).
Human Rights Watch. (2015) Mind the Gap: The Lack of Accountability for Killer Robots. [Online]. Available at: https://www.hrw.org/sites/default/files/reports/arms0415_ForUpload_0.pdf (Accessed: 1 July 2024).
OECD (n.d.) AI-Principles Overview. [Online]. Available at: https://oecd.ai/en/ai-principles (Accessed: 1 July 2024).
REAIM (2023) REAIM 2023 Call to Action. [Online]. Available at: https://www.government.nl/documents/publications/2023/02/16/reaim-2023-call-to-action (Accessed: 1 July 2024).
UNESCO (n.d.) Ethics of Artificial Intelligence. [Online]. Available at: https://www.unesco.org/en/artificial-intelligence/recommendation-ethics (Accessed: 28 August 2023).
UNIDIR (2015) The Weaponization of Increasingly Autonomous Technologies: Considering Ethics and Social Values. [Online]. Available at: https://unidir.org/publication/the-weaponization-of-increasingly-autonomous-technologies-considering-ethics-and-social-values/ (Accessed: 1 July 2024).
UNSG. (2023) Secretary-General’s remarks to the Security Council on Artificial Intelligence. [Online]. Available at: https://www.un.org/sg/en/content/sg/speeches/2023-07-18/secretary-generals-remarks-the-security-council-artificial-intelligence. (Accessed 1 July 2024).
(n.d.) Key Issue 3: Risk-Based Approach - EU AI Act. [Online]. Available at: https://www.euaiact.com/key-issue/3 (Accessed: 29 March 2024).
(n.d.) Key Issue 4: Human Oversight - EU AI Act. [Online]. Available at: https://www.euaiact.com/key-issue/4 (Accessed: 29 March 2024).
