Human oversight of artificial intelligence: An operations management perspective

Abstract

Purpose: This paper presents a theoretical framework for AI oversight and examines the key dimensions used by operational managers to define their oversight activities for AI applications.

Design/methodology/approach: The research combines a theoretical and qualitative approach. The theoretical part analyzes and proposes a framework for studying AI oversight from an operational perspective, drawing on cybernetic and control theory and recent literature on human oversight. This framework is then compared and categorized with the perceptions of managers regarding AI management and oversight. The operational perspective views oversight not only as a safety mechanism but also as a governance mechanism that encompasses safety, ethical, and compliance requirements, as well as technical and business goals. Importantly, oversight is necessary regardless of the application's risk level.

Findings: The paper offers a more operational definition and framework for oversight, combining theoretical concepts and practical insights from industry practitioners. The theoretical framework clarifies the recursive nature of oversight within organizational control loops. The practical categorization of oversight design dimensions identifies key factors influencing the selection of resources, methods, and tools for AI applications oversight.

Research limitations/implications: The theoretical proposal is grounded in specific cybernetic and control theories, but other theoretical frameworks could be explored. The qualitative study provides a categorization of oversight dimensions, but each AI application and organization should adapt this framework to its specific needs.

Practical implications: This paper aims to assist companies in designing effective AI oversight functions that align with legal, technical, and business requirements.

Social implications: A meaningful and effective oversight of AI applications will enhance the trustworthiness of AI integration within organizations for all stakeholders, including employees, customers, investors and society at large.

Originality/value: This paper contributes to the ongoing discussion on human oversight of AI, which has been heavily focused on legal aspects since the publication of the European Union’s AI Act. By adopting an operational perspective, the paper offers both conceptual and practical insights.