Whiteshield Advisory Case: Regulating Artificial Intelligence in the GCC

In an initial meeting with the Minister of Human Resources, you are expected to deliver a comprehensive overview of AI's societal impact and the diverse opportunities it offers across various industries. 

A strong answer should cover all relevant areas in the first level and include 3-5 substantial points per bucket in the second level. It's essential to recognize that multiple structures can be suitable for successfully completing the case. Typically, a good structure comprises 3-5 buckets in the first level to ensure a comprehensive approach and at least 3-5 sub-points in the second level to provide sufficient depth in the analysis.

As we are preparing to depart from the Minister's office, he commends, "Your presentation on the implications of AI for society and business was impressive. Now, I'm curious, how can I harness the potential of this new tool, ChatGPT, effectively within my ministry?" Caught off guard, we swiftly engage in a creative brainstorming session. 

A strong answer will have >8 ideas that are logically grouped. 

Amid the rising utilization of AI technology, which promises heightened productivity and resource efficiency in economies, the European Union's recent implementation of the EU Artificial Intelligence Act has set a precedent with risk-specific regulations. Now, Whiteshield faces the task of evaluating the extent to which AI use cases can be classified by risk across different enterprise functions. As the head of the workstream, you are tasked with presenting the following chart to the Minister. He seeks insights from the chart and, specifically, the hypotheses you can offer regarding the distribution of risks across enterprise functions.

A strong answer describes key observations from both charts, offers hypotheses (e.g., why certain functions have a higher proportion of risk), and implications of these observations (e.g. that the high proportion of unclassified/unclear cases requires clearer regulation). The following is an example of observations. 

Top Level: This level encompasses uses that pose an unacceptable risk to people's safety, livelihoods, and rights. These use cases are prohibited unless specifically authorized by law for national security purposes. Examples include social scoring AI systems, behavior manipulation causing harm, and mass surveillance.

High-Risk Level: Use cases falling into this category require a conformity assessment before deployment in the market. The assessment evaluates data quality to minimize risks and discriminatory outcomes, documentation and traceability, transparency, user information provision, human oversight, robustness, accuracy, and cybersecurity provisions. The EU has defined a list of high-risk AI uses, such as access to employment, education, public services, management of critical infrastructures, safety components of vehicles, law enforcement, and administration of justice.

Limited Risk Level: This level pertains to uses with limited risk, and their only requirement is transparency obligations. For instance, AI-based chatbots must inform users that they are interacting with a machine.

Minimal Risk Level: Use cases with minimal risk do not have any obligations, but the adoption of voluntary codes of conduct is recommended. This voluntary adoption can build trust in AI adoption and provide a competitive advantage for service providers.

Unsurprisingly, more than 75% of AI systems in human resources are classified in the high-risk category, with more than 25% each in customer service, accounting and finance, and IT and security. On the other hand, unclear classifications are found in all enterprise functions, but mostly in accounting and finance at over 70%.

Possible reasons for High Risk share

1. Sensitive Data Handling: Human resources deal with an array of sensitive data, including personal information, employee performance evaluations, and health records. The usage of AI in HR processes can amplify data privacy concerns and potential harm if not handled with utmost care. High-risk AI systems in HR might involve automated hiring decisions or performance evaluations, which can lead to discriminatory outcomes and infringement on individuals' rights.

2. Financial Impact and Decision-Making: AI's application in accounting and finance often involves tasks like financial forecasting, investment analysis, and risk assessment. Inaccurate AI predictions or financial anomalies can have severe financial repercussions for businesses, investors, and individuals. Therefore, the potential risks involved in these critical financial decisions warrant a high-risk classification.

3. Vulnerability to Cyberattacks: The finance sector deals with substantial financial transactions and sensitive data. AI systems in this domain can be attractive targets for cybercriminals seeking to exploit vulnerabilities. Malicious manipulation of AI models in financial processes can lead to significant financial losses, making robustness and cybersecurity crucial considerations for risk assessment.

The risk class of an AI system profoundly affects the likelihood that it will be developed, adapted, or funded. High-risk systems tend to have a lower chance of being implemented due to the increase in cost and complexity, driven by the additional requirements that raise the barrier to adaptation.

Most high-risk systems are expected to be in human resources, customer service, accounting and finance, and legal. Consequently, fewer companies are likely to benefit from AI in these areas. The implications of this pattern warrant consideration for resource allocation and strategic planning in adopting AI technologies.

Moreover, unclear risk classifications slow down investment and innovation. Examining the causes of uncertainty can result in concrete recommendations to policymakers and companies to promote responsible AI innovation. Some of the main causes of unclear risk classifications could include:

1. Ambiguity in regulatory guidelines: Unclear or evolving regulatory frameworks may lead to varying interpretations and evaluations of AI systems' risk levels.

2. Lack of standardized evaluation criteria: Absence of universally accepted criteria for assessing AI risks could result in inconsistencies across assessments.

3. Complexity of AI algorithms: Advanced AI algorithms and deep learning models may pose challenges in understanding their decision-making processes, leading to uncertainty in risk evaluation.

4. Limited historical data: In some cases, AI systems may be exploring new territories, and the absence of extensive historical data might hinder accurate risk assessments.

5. Ethical considerations: Ethical dilemmas arising from AI applications, such as bias and fairness issues, may create uncertainty in assigning risk levels.

Addressing these underlying causes of uncertainty could pave the way for a more robust and streamlined AI regulation and adoption process, facilitating responsible and beneficial AI deployment across various industries.

The Minister of Human Resources, our client, is exploring the implementation of a mandatory conformity assessment for AI systems classified as high-risk in HR functions across industries. This assessment aims to ensure responsible AI deployment, mitigate potential risks, and foster public trust. However, conducting these assessments incurs costs for both the government and companies. There are 650,000 companies affected, of which 42% currently deploy AI systems, with an average of 10 systems per company. Each conformity assessment will incur a cost of $200,000 for an external auditor. The ministry estimates that preventing AI risks through these assessments could save the economy from incurring damages amounting to more than 1 billion USD. The Minister seeks advice on whether the regulatory costs are appropriate considering these potential economic savings.

Step 1: Total Number of Companies Subject to Conformity Assessment:

Given that there are 650,000 companies affected, the total number of companies subject to the mandatory conformity assessment is 650,000.

Step 2: Total Cost of Conformity Assessments:

Out of the 650,000 companies, 42% currently deploy AI systems, which amounts to 0.42 * 650,000 = 273,000 companies deploying AI.

The average number of AI systems per company is 10. Therefore, the total number of AI systems is 273,000 * 10 = 2,730,000 AI systems.

Only high-risk systems will be required to undergo an assessment, from the chart before we can see this share to be ~90%, hence number of AI systems to be assessed is 2,730,000 * 90% = 2,457,000 high-risk AI systems.

The cost of each conformity assessment is $200,000. Thus, the total cost of conducting these assessments is 2,457,000 * $200,000 = $491,600,000.

Step 3: Projected Cost-Benefit Analysis:

The Ministry expects that by conducting these conformity assessments, they can prevent AI risks that could cost more than 1 billion USD to the economy. On the other hand, the conformity assessments are going to cost half a billion USD. 

Recommendation

After careful consideration of the cost-benefit analysis, I recommend against proceeding with the implementation of the mandatory conformity assessment for AI systems classified as high-risk in HR functions across industries. Instead, I propose exploring alternative ways to regulate and lower the costs of these conformity assessments. Here are the balanced arguments supporting this recommendation:

Arguments Against the Regulation:

High Costs per Company: The significant cost of conducting conformity assessments, especially for smaller businesses, poses a substantial financial burden. The average cost of ~1.8m USD per effected company with high-risk systems might hinder companies' competitiveness, liquidity, and capacity to invest in other essential areas, potentially limiting their growth and innovation.

Impact on AI Adoption: The mandatory assessments could lead to slower AI adoption in HR functions as companies may be deterred by the high costs and complex evaluation processes. Slower AI adoption may prevent organizations from benefiting from the potential efficiencies and advancements that AI technologies can offer.

Efficiency and Expertise Concerns: The effectiveness of external auditors in conducting standardized and robust evaluations across all companies might be challenging to ensure. Potential inconsistencies in risk evaluations could reduce the overall effectiveness of the conformity assessment process.

Alternative Approach:

Rather than enforcing mandatory conformity assessments, the government should consider exploring alternative approaches to regulate and ensure responsible AI deployment in HR functions. Some potential alternatives include:

Voluntary Best Practices: Encourage companies to voluntarily adopt best practices for AI deployment in HR functions. Provide guidelines and resources to promote ethical AI usage and responsible decision-making, without imposing substantial costs.

Industry Standards and Certification: Develop industry-wide standards for AI systems in HR functions and establish certification programs that companies can voluntarily participate in. This would provide a transparent framework for evaluating AI systems' risk without mandatory assessments.

Public-Private Partnerships: Foster partnerships between the government, private sector, and AI experts to collectively develop guidelines and assess high-risk AI systems. By sharing the costs and expertise, conformity assessments can be made more affordable and efficient.

In conclusion, while the aim of ensuring responsible AI deployment is essential, the current form of mandatory conformity assessment appears to be cost-prohibitive and might impede AI adoption in HR functions. It is advisable to explore alternative regulatory approaches that are more financially feasible and encourage responsible AI usage without imposing undue financial burdens on companies.