Designing an AI system

The design process helps AI system designers, implementers and decision-makers uncover unexpected and undesirable behaviour in an AI system

The process requires input from the agency’s leaders, policy experts, domain experts, and data scientists. The AI system needs to be gradually tested and scaled to full implementation.

The major stages of each iteration are as follows:

  1. Quantify what matters: identify as many of the considerations relevant to the AI system’s performance as possible, including potential unintended consequences of the system’s operation. These performance considerations and potential unintended consequences need to be quantified so that:
    1. AI system designers can verify and optimise the system’s behaviour
    2. Decision makers have a clear understanding of which metrics are visible to the system.
  2. Design Actions (Attachment B): work with domain experts to design the system’s possible actions, and design operating parameters that control how the system operates with respect to those actions. 
  3. Model Impact (Attachment C): model changes to the AI system’s operating parameters to see how it impacts the performance metrics.
  4. Balance objectives (Attachment D): choose the operating parameters that result in the best performance outcomes with respect to the AI system’s objective(s), constraints and risks.
  5. Test, Implement & Iterate (Attachment E): agencies need to continue to test the AI system throughout the design phase to:
    1. Ensure the AI system’s objective is correct
    2. Identify and respond to unintended consequences
    3. Balance of competing objectives
    4. Identify any performance improvements through design changes or data collection.

Change Management

Change management means working with each of the different stakeholders affected by the new AI-enabled project and the accompanying new processes to transition from the current state to the future state.

Prepare for the change: Understand your solution’s impact and outline how the change is going to be communicated

Involve change makers: From the very beginning, involve leaders (including middle managers) to help you implement the change

Address impact: Whether it’s people, organisational, technology, processes, culture or structure, it’s important to understand the breadth of impact and prepare for how that change is going to be implemented.

Build a plan: Work with change makers to build a plan of action. Include time to address feedback

Implement: Update those impacted in a timely manner with quality updates about the change. Manage the change with care and sensitivity for those impacted.

Communicating about deployment

Consider what communication or awareness raising/education may be required for your stakeholders, especially the people who will be using or be potentially impacted by your AI-enabled decision-making process.

Last updated