AI Governance

A Framework for Building Responsible, Ethical, Fair, and Transparent AI

Are you ready for AI?

Business leaders must manage the associated risks as they scale their use of AI

Regulations and risk assessment tools

Governments around the world are developing AI assessment methodologies and legislation for AI. Here are a couple of examples:

• Responsible use of artificial intelligence (AI) guiding principles (Canada):
  1. understand and measure the impact of using AI by developing and sharing tools and approaches
  2. be transparent about how and when we are using AI, starting with a clear user need and public benefit
  3. provide meaningful explanations about AI decision-making, while also offering opportunities to review results and challenge these decisions
  4. be as open as we can by sharing source code, training data, and other relevant information, all while protecting personal information, system integration, and national security and defense
  5. provide sufficient training so that government employees developing and using AI solutions have the responsible design, function, and implementation skills needed to make AI-based public services better
• The Algorithmic Impact Assessment tool (Canada) is used to determine the impact level of an automated decision-system. It defines 48 risk and 33 mitigation questions. Assessment scores consider factors such as systems design, algorithm, decision type, impact, and data.
• The National AI Initiative Act of 2020 (DIVISION E, SEC. 5001) (US) became law on January 1, 2021. This is a program across the entire Federal government to accelerate AI research and application.
• Bill C-27, Artificial Intelligence and Data Act (AIDA) (Canada), when passed, would be the first law in Canada regulating the use of artificial intelligence systems.
• The EU Artificial Intelligence Act (EU) assigns applications of AI to three risk categories: applications and systems that create an unacceptable risk, such as government-run social scoring; high-risk applications, such as a CV-scanning tool that ranks job applicants; and lastly, applications not explicitly listed as high-risk.
• The FEAT Principles Assessment Methodology was created by the Monetary Authority of Singapore (MAS) in collaboration with other 27 industry partners for financial institutions to promote fairness, ethics, accountability, and transparency (FEAT) in the use of artificial intelligence and data analytics (AIDA).

AI policies around the world

Source of data: OECD.AI (2021), powered by EC/OECD (2021), database of national AI policies, accessed on 7/09/2022, https://oecd.ai.

The need for AI governance

“To adopt AI, organizations will need to review and enhance their processes and governance frameworks to address new and evolving risks.” (Canadian RegTech Association, Safeguarding AI Use Through Human-Centric Design, 2020)

To ensure responsible, transparent, and ethical AI systems, organizations will need to review existing risk control frameworks and update them to include AI risk management and impact assessment frameworks and processes. As ML and AI technologies are constantly evolving, the AI governance and AI risk management frameworks will need to evolve to ensure the appropriate safeguards and controls are in place. This applies not only to the machine learning models and AI system custom built by the organization’s data science and AI team, but it also includes AI-powered vendor tools and technologies. The vendors should be able to explain how AI is used in their products, how the model was trained, and what data was used to train the model. AI governance enables management, monitoring, and control of all AI activities within an organization.

Key concepts

Info-Tech Research Group defines the key terms used in this document as follows:

Machine learning systems learn from experience and without explicit instructions. They learn patterns from data, then analyze and make predictions based on past behavior and the patterns learned.

Artificial intelligence is a combination of technologies and can include machine learning. AI systems perform tasks that mimic human intelligence, such as learning from experience and problem solving. Most importantly, AI makes its own decisions without human intervention.

We use the definition of data ethics by Open Data Institute: “Data ethics is a branch of ethics that considers the impact of data practices on people, society and the environment. The purpose of data ethics is to guide the values and conduct of data practitioners in data collection, sharing and use.”

Algorithmic or machine bias is systematic and repeatable errors in a computer system that create unfair outcomes, such as privileging one arbitrary group of users over others. Algorithmic bias is not a technical problem. It’s a social and political problem, and in the context of implementing AI for business benefits, it’s a business problem.

Download the blueprint Mitigate Machine Bias blueprint for detailed discussion on bias, fairness, and transparency in AI systems

Key concepts – explainable, transparent and trustworthy

AI Governance Framework

Monitoring Monitoring compliance and risk of AI/ML systems/models in production Tools & Technologies Tools and technologies to support AI governance framework implementation Model Governance Ensures accountability and traceability for AI/ML models

Organization Structure, roles, and responsibilities of the AI governance organization Operating Model How AI governance operates and works with other organizational structures to deliver value Risk and Compliance Alignment with corporate risk management and ensuring compliance with regulations and assessment frameworks Policies/Procedures/ Standards Policies and procedures to support implementation of AI governance

Establish Effective Security Governance & Management

The key is in stakeholder interactions, not policy and process.

Analyst Perspective

It's about stakeholder interactions, not policy and process.

Many security leaders complain about a lack of governance and management in their organizations. They have policies and processes but find neither have had the expected impact and that the organization is teetering on the edge of lawlessness, with stakeholder groups operating in ways that interfere with each other (usually due to poorly defined accountabilities).

Among the most common examples is security's relationship to the business. When these groups don't align, they tend to see each other as adversaries and make decisions in line with their respective positions: security endorses one standard, the business adopts another.

The consequences of this are vast. Such an organization is effectively opposed to itself. No wonder policy and process have not resolved the issue.

At a practical level, good governance stems from understanding how different stakeholder groups interact, providing inputs and outputs to each other and modeling who is accountable for what. But this implied accountability model needs to be formalized (perhaps even modified) before governance can help all stakeholder groups operate as strategic partners with clearly defined roles, responsibilities, and decision-making power. Only when policies and processes reflect this will they serve as effective tools to support governance.

Logan Rohde
Senior Research Analyst, Security & Privacy
Info-Tech Research Group

Executive Summary

Info-Tech Insight
Good governance stems from a deep understanding of how stakeholder groups interact with each other and their respective accountabilities and responsibilities. Without these things, organizational functions tend to interfere with each other, blurring the lines between governance and management and promoting ad hoc decision making that undermines governance.

Your challenge

This research is designed to help organizations who need to:

• Establish security governance from scratch.
• Improve security governance despite a lack of cooperation from the business.
• Determine the accountabilities and responsibilities of each stakeholder group.

This blueprint will solve the above challenges by helping you model your organization's governance structure and implement processes to support the essential governance areas: policy, risk, and performance metrics.

Percentage of organizations that have yet to fully advance to a maturity-based approach to security

70%

Source: McKinsey, 2021

Common obstacles

These barriers make this challenge difficult to address for many organizations:

• The business does not wish to be governed and does not seek to align with security on the basis of risk.
• Various stakeholder groups essentially govern themselves, causing business functions to interfere with each other.
• Security teams struggle to differentiate between governance and management and the purpose of each.

Early adopter infrastructure

63%
Security leaders not reporting to the board about risk or incident detection and prevention.
Source: LogRhythm, 2021

46%
Those who report that senior leadership is confident cybersecurity leaders understand business goals.
Source: LogRhythm, 2021

Governance isn't just policy and process

Governance is often mistaken for an organization's formalized policies and processes. While both are important governance supports, they do not provide governance in and of themselves.

For governance to work well, an organization needs to understand how stakeholder groups interact with each other. What inputs and outputs do they provide? Who is accountable? Who is responsible? These are the questions one needs to ask before designing a governance structure. Failing to account for any of these three elements tends to result in overlap, inefficiency, and a lack of accountability, creating flawed governance.

Separate governance from management

Oversight versus operations

• COBIT emphasizes the importance of separating governance from management. These are complementary functions, but they refer to different parts of organizational operation.
• Governance provides a decision-making apparatus based on predetermined requirements to ensure smooth operations. It is used to provide oversight and direction and hinges on established accountabilities
• Simply put, governance refers to what an organization is and is not willing to permit in day-to-day operations, and it tends to make its presence known via the key areas of risk appetite, formal policy and process, and exception handling.
• Note: These key areas do not provide governance in and of themselves. Rather, governance emerges in accordance with the decisions an organization has made regarding these areas. Sometimes, however, these "decisions" have not been formally or consciously made and the current state of the organization's operations becomes the default - even when it is not working well.
• Management, by contrast, is concerned with executing business processes in accordance with the governance model, essentially, governance provides guidance for how to make decisions during daily management.

"Information security governance is the guiding hand that organizes and directs risk mitigation efforts into a business-aligned strategy for the entire organization."

Steve Durbin,
Chief Executive,
Information Security Forum, Forbes, 2023

Models for governance and management

Info-Tech's Governance and Management research uses the logic of COBIT's governance and management framework but distills this guidance into a practical, easy-to-implement series of steps, moving beyond the rudimentary logic of COBIT to provide an actionable and personalized governance model.

Clear accountabilities and responsibilities

Complementary frameworks to simplify governance and management

The distinction that COBIT draws between governance and management is roughly equivalent to that of accountability and responsibility, as seen in the RACI* model.

There can be several stakeholders responsible for something, but only one party can be accountable.

Use this guidance to help determine the accountabilities and responsibilities of your governance and management model.

*Responsible, Accountable, Consulted, Informed

Security governance framework

A security governance framework is a system that will design structures, processes, accountability definitions, and membership assignments that lead the security department toward optimal results for the business.

Governance is performed in three ways:

"Governance specifies the accountability framework and provides oversight to ensure that risks are adequately mitigated, while management ensures that controls are implemented to mitigate risks. Management recommends security strategies. Governance ensures that security strategies are aligned with business objectives and consistent with regulations."
- EDUCAUSE

SMART metrics

Suggested targets to measure success

Specific

Measurable

Achievable

Relevant

Time-Bound

Info-Tech's methodology for security governance and management

Governance starts with mapping stakeholder inputs, outputs, and throughputs

The key is in stakeholder interactions, not policy and process
Good governance stems from a deep understanding of how stakeholder groups interact with each other and their respective accountabilities and responsibilities. Without these things, organizational functions tend to interfere with each other, blurring the lines between governance and management and promoting ad hoc decision making that undermines governance.

Policy, process, and org. charts support governance but do not produce it on their own
To be effective, these things need to be developed with the accountabilities and influence of the organizational functions that produce them.

A lack of business alignment does not mean you're doomed to fail
While the highest levels of governance maturity depend on strong security-business alignment, there are still tactics one can use to improve governance.

All organizations have governance
Sometimes it is poorly defined, ineffective, and occurs in the same place as management, but it exists at some level, acting as the decision-making apparatus for an organization (i.e. what can and cannot occur).

Risk tolerances are variable across lines of business
This can lead to misalignments between security and the business, as each may have their own tolerance for particular risks. The remedy is to understand the risk appetite of the business and allow this to inform security risk management decisions.

Blueprint deliverables

Each step of this blueprint is accompanied by supporting deliverables to help you accomplish your goals:

Security Governance Model Tool

Security Governance Organizational Structure Template

Information Security Steering Committee Charter & RACI

Policy Exceptions-Handling Workflow

Policy Exception Tracker and Request Form

Key deliverable:

Security Governance Model

By the end of this blueprint, you will have created a personalized governance model to map your stakeholders' accountabilities, responsibilities, and key interactions.

Blueprint benefits

Info-Tech offers various levels of support to best suit your needs

Diagnostics and consistent frameworks are used throughout all four options.

Guided Implementation

What does a typical GI on this topic look like?

A Guided Implementation (GI) is a series of calls with an Info-Tech analyst to help implement our best practices in your organization.

A typical GI is 4 to 8 calls over the course of 2 to 3 months.

Workshop Overview

Contact your account representative for more information.
workshops@infotech.com 1-888-670-8889

Customize your journey

The security governance and management blueprint pairs well with security design and security strategy.

• The governance and management model you create in this blueprint will inform efforts to improve security, like revisiting security program design and your security strategy.
• Work with your member services director, executive advisor, or technical counselor to scope the journey you need. They will work with you to align the subject matter experts to support your roadmap and workshops.

Workshop Day 1 and Day 2
Security Governance and Management

Workshop Day 3 and Day 4
Security Strategy Gap Analysis or Security Program Design Factors

Phase 1

Design Your Governance Model

Phase 1
1.1 Evaluate
1.2 Direct
1.3 Monitor

Phase 2
2.1 Implement Oversight
2.2 Set Risk Appetite
2.3 Implement Policy lifecycle

Establish Security Governance & Management

This phase will walk you through the following activities:

• Prioritize governance accountabilities
• Prioritize management responsibilities
• Evaluate current organizational structure
• Align with the business
• Build security governance and management model
• Finalize governance and management model
• Develop governance and management KPIs

This phase involves the following participants:

• CISO
• CIO
• Business representative

Step 1.1

Evaluate

Activities
1.1.1 Prioritize governance accountabilities
1.1.2 Prioritize management responsibilities
1.1.3 Evaluate current organizational structure

This step involves the following participants:

• CISO
• CIO
• Business representative

Outcomes of this step

• Defined governance accountabilities
• Defined management responsibilities

Design Your Governance Model

Step 1.1 > Step 1.2 > Step 1.3

Evaluate: Getting started

The above is a summary of COBIT 2019 EDM01.01 Evaluate the governance system, along with Info-Tech-recommended questions to contextualize each element for your organization.

1.1.1 Prioritize governance accountabilities

1-2 hours

Using the example on the next slide, complete the following steps.

1. Download Info-Tech's Security Governance Model Tool using the link below and customize the stakeholder groups on tab 1 to reflect the makeup of your organization.
2. Using the previous slide as a guide, evaluate your organization's internal and external pressures and discuss their possible impacts your governance and management model.
3. Complete tab 2, Governance Prioritization, indicating your response to each prompt using the drop-down menus. The tool will score your responses and provide you with a prioritized list of governance accountabilities based on greatest need on tab 4, Governance Model Builder.
4. Review the list and make any desired modifications to the prompts on tab 2 and then move on to Activity 1.1.2. (We will return to tab 4 in Step 2.1.) Remember to evaluate the results against the internal/external pressure analysis to ensure these details are reflected.

Download the Security Governance Model Tool

Security Governance and Management Model Tool

Tabs 2 and 3

1.1.2 Prioritize management responsibilities

1 hours

Using the examples on the previous slide, complete the following steps.

1. Complete tab 3, Management Prioritization, indicating your response to each prompt using the drop-down menus. The tool will score your responses and provide you with a prioritized list of governance accountabilities based on greatest need on tab 4, Governance Model Builder.
2. Review the list and make any desired modifications to the prompts on tab 3 and then move on to Activity 1.1.3. (We will return to tab 4 in Step 2.1.) Remember to evaluate the results against the internal/external pressure analysis to ensure these details are reflected.

Download Security Governance Model Tool

Security Governance and Management Model Tool

Tab 4

1.1.3 Evaluate current organizational structure

1-3 hours

1. Download and modify Info-Tech's Security Governance Organizational Structure Template to reflect the reporting structure at your organization. If such a document already exists, simply review it and move on to the next step below.
2. Determine if the current organizational structure will negatively affect your ability to pursue the items in your prioritized lists from governance accountabilities and management responsibilities (e.g. conflicts of interest related to oversight or reporting), and discuss the feasibility of changing the current governance structure.
3. Record these recommended changes and any other key points you'd like the business or other stakeholders to be aware of. We'll use this information in the business alignment exercise in Step 2.1

Download the Security Governance Organizational Structure Template

Info-Tech resources

Locate structural problems in advance

• If you do not already have a diagram of your organization's reporting structure, use this template to create one. Examples are provided for high, medium, and low maturity.
• The existing reporting structure will likely affect the governance model you create, as it may not be feasible to assign certain governance accountabilities and management responsibilities to certain stakeholders.
• For example, it may make sense for the head of security to approve the security budget, but if they report to a CIO with greater authority that accountability will likely have to sit with the CIO instead.

Download the Security Governance Organizational Structure Template

Step 1.2

Direct

Activities
1.2.1 Align with the business
1.2.2 Build security governance and management model
1.2.3 Finalize governance and management model

This step involves the following participants:

CISO

CIO

Business representative

Outcomes of this step

• Record of key stakeholder interactions
• Visual governance model

Design Your Governance Model

Step 1.1 > Step 1.2 > Step 1.3

Direct: Getting started

The above is a summary of COBIT 2019 EDM01.02 Direct the governance system, along with Info-Tech-recommended questions to contextualize each element for your organization.

Embed security governance within enterprise governance

Design structures, processes, authority definitions, and steering committee assignments to drive optimal business results.

1.2.1 Align with the business

1-3 hours

1. Request a meeting with the business to present your findings from the previous activities in Step 1.1. As you prepare for the meeting, remember to following points:

• The goal here is to align, not to command. You want the business to see the security team as a strategic ally that supports the pursuit of business goals.
• Make recommendations and explain any security risks associated with the direction the business wants to take, but the goal is not to strongarm the business into adopting your perspective.
• Above all, listen to the business to learn more about how they relate to governance and what their priorities are. This will help you adapt your governance model to better support business needs.

Info-Tech Insight
A lack of business participation does not mean your governance initiative is doomed. From this lack, we can still infer their attitudes toward security governance, and we can account for this in our governance model. This may limit the maturity your program can reach, but it doesn't prevent improvements from being made to your current security governance.

1.2.2 Build security governance and management model

1-2 hours

Using the example on the next slide, complete the following steps:

1. On tab 4, review the prioritized lists for governance accountabilities and management responsibilities and begin assigning them to the appropriate stakeholder groups.

• Remember: Responsibilities can be assigned to up to four stakeholders, but there can be only one party listed as accountable.

• Documenting these interactions will help you ensure your governance program accounts for inputs and outputs that are required by, or that otherwise affect, your various stakeholder groups.

Note: You may wish to review Info-Tech's governance model templates before completing this activity to get an idea of what you'll be working toward in this step. See slides 37-38.

Download Security Governance Model Tool

Security Governance and Management Model Tool

Tab 5

Security Governance and Management Model Tool continued

Tab 6

1.2.3 Visualize your security governance and management model

1-2 hours

1. Download the Security Governance Model Templates using the link below and determine which of the three example models most closely resembles your own.
2. Once you have chosen an example to work from, begin customizing it to reflect the governance model completed in Activity 1.2.2. See next slide for example.

Note: You do not have to use these templates. If you prefer, you can use them as inspiration and design your own model.

Download Security Governance Model Templates

Customize the template

Step 1.3

Monitor

Activities
1.3.1 Develop governance and management KPIs

This step involves the following participants:

• CISO
• CIO
• Security team
• Business representative

Outcomes of this step

Key performance indicators

Design Your Governance Model

Step 1.1 > Step 1.2 > Step 1.3

Monitor: Getting started

The above is a summary of COBIT 2019 EDM01.03 Monitor the governance system, along with Info-Tech-recommended questions to contextualize each element for your organization.

1.3.1 Develop governance and management KPIs

1-2 hours

This activity is meant to provide a starting point for key governance metrics. To develop a comprehensive metrics program, see Info-Tech's Build a Security Metrics Program to Drive Maturity blueprint.

1. Create a list of four to six outcomes you'd like to see as the result of your new governance model. Be as specific as you can; the better defied the outcome, the easier it will be to determine suitable KPI.
2. For each desired outcome, determine what would best indicate that progress is being made toward that state.

• Desired outcome: security team is consulted before critical business decisions are made.
• Success criteria: the business evaluates Security's recommendations before starting new projects
• Possible KPI: % of critical business decisions made with security consultation
• See next slide for additional examples

Note: Try to phrase each KPI using percents, which helps to add context to the metric and will make it easier to explain when reporting metrics in the future.

Example KPIs

Establish Baseline Metrics

Baseline metrics will be improved through:

1. Improved business alignment
2. Developing formal process to manage security risks
3. Separating governance from management

Phase 2

Implement Essential Governance Processes

Phase 1
1.1 Evaluate
1.2 Direct
1.3 Monitor

Phase 2
2.1 Implement Oversight
2.2 Set Risk Appetite
2.3 Implement Policy Lifecycle

This phase will walk you through the following activities:

• Draft Steering Committee Charter
• Complete Steering Committee RACI
• Draft qualitative risk statements
• Model policy lifecycle
• Establish exceptions-handling process

This phase involves the following participants:

• CISO
• CRO
• CIO
• HR
• Internal Audit
• Business representative
• Legal

Establish Security Governance & Management

Step 2.1

Implement Oversight

Activities
2.1.1 Draft steering committee charter
2.1.2 Complete steering committee RACI

This step involves the following participants:

• CISO
• CRO
• CIO
• HR
• Internal Audit
• Business representative
• Legal

Outcomes of this step

Steering Committee Charter and RACI

Implement Essential Governance Processes

Step 2.1 > Step 2.2 > Step 2.3

2.1.1 Draft steering committee charter

1-3 hours

This activity is meant to provide a starting point for your steering committee. If a more comprehensive approach is desired, see Info-Tech's Improve Security Governance With a Security Steering Committee blueprint.

1. Download the template using the link below and review the various sections of the document
2. Review slides 50-51 to help determine the scope of your steering committee's role. Discuss with other stakeholder groups, as necessary, to determine the steering committee's duties, how often the group will meet, and what the regular meeting agenda will be.
3. Customize the template to suit your organization's needs.

Download Information Security Steering Committee Charter

Steering committee membership

Representation is key, but don't try to please everyone

• For your steering committee to be effective, it should include representatives from across the organization. However, it is important not to overextend committee membership, which can interfere with decision making.
• Participants should be selected based on the identified responsibilities of the security steering committee, and the number of people should be appropriate to the size and complexity of the organization.

Example steering committee

CISO
CRO
Internal Audit
CIO
Business Leaders
HR
Legal

Download Information Security Steering Committee Charter

Typical steering committee duties

Typical steering committee duties

2.1.2 Complete steering committee RACI

1-3 hours

1. Download the RACI template and review the membership roles. Customize the template to match the makeup of your steering committee.
2. Read through each task in the left-hand column and determine who will be involved:

• R - responsible: the person doing the action (can be multiple)
• A - accountable: the owner of the task, usually a department head who delegates the execution of the task (only assigned to one stakeholder)
• C - consulted: stakeholders that offer some kind of guidance, advice, or recommendation (can be multiple)
• I - Informed: stakeholders that receive status updates about the task (can be multiple)

Note: All tasks must have accountability and responsibility assigned (sometimes a single stakeholder is accountable and responsible). However, not all tasks will have someone consulted or informed.

Download Information Security Steering Committee RACI Chart

Step 2.2

Set Risk Appetite

Activities
2.2.1 Draft qualitative risk statements

This step involves the following participants:

• CISO
• CIO
• Business representative

Outcomes of this step

Qualitative risk appetite

Implement Essential Governance Processes

Step 2.1 > Step 2.2 > Step 2.3

Know your appetite for risk

What is an organizational risk appetite?

Setting risk appetite is a key governance function, as it structures how your organization will deal with the risks it will inevitably face - when they can be accepted, when they need to be mitigated, and when they must be rejected entirely.

It is important to note that risk appetite and risk tolerance are not the same. Risk appetite refers to the amount of risk the organization is willing to accept as part of doing business, whereas risk tolerance has more to do with individual risks affecting one or more lines of business that exceed that appetite. Such risks are often tolerated as individual cases that can be mitigated to an acceptable level of risk even though it exceeds the risk-appetite threshold.

2.1.2 Draft qualitative risk-appetite statements

1-3 hours

This activity is meant to provide a starting point for risk governance. To develop a comprehensive risk-management program, see Info-Tech's Combine Security Risk Management Components Into One Program blueprint.

1. Draft statements that express your attitudes toward the kinds of risks your organization faces. The point is to set boundaries to better understand when risk mitigation may be necessary.
Examples:

• We will not accept risks that may cause us to violate SLAs.
• We will avoid risks that may prevent the organization from operating normally.
• We will not accept risks that may result in exposure of confidential information.
• We will not accept risks that may cause significant brand damage.
• We will not accept risks that pose undue risk to human life or safety.

Step 2.3

Implement Policy Lifecycle

Activities
2.3.1 Model your policy lifecycle
2.3.2 Establish exception-approval process

This step involves the following participants:

• CISO
• CIO

Outcomes of this step

Policy lifecycle

Exceptions-handling process

Implement Essential Governance Processes

Step 2.1 > Step 2.2 > Step 2.3

2.3.1 Model your policy lifecycle

1-3 hours

This activity is meant to provide a starting point for policy governance. To develop a comprehensive policy-management program, see Info-Tech's Develop and Deploy Security Policies blueprint.

1. Review the sections within the Security Policy Lifecycle Template and delete any sections or subsections that do not apply to your organization.
2. As necessary, modify the lifecycle and receive approved sign-off by your organization's leadership.
3. Solicit feedback from stakeholders, specifically, IT department management and business stakeholders.

Download the Security Policy Lifecycle Template

Develop the security policy lifecycle

The security policy lifecycle is an integral component of the security policy program and adds value by:

• Setting out a roadmap to define needs, develop required documentation, and implement, communicate, and measure your policy program.
• Defining roles and responsibilities for the security policy suite.
• Aligning the business goals, security program goals, and policy objectives.

Diagram inspired by: ComplianceBridge, 2021

2.3.2 Establish exception-approval process

1-3 hours

1. Download the Security Policy Exception Approval Template and customize it to match your exception-handling process. Be sure to account for the recommendations on the next slide.
2. Use the Policy Exception Tracker to record and monitor granted exceptions.

Download the Security Policy Exception Approval Workflow

Download the Security Policy Exception Tracker

Determine criteria to grant policy exception

A key part of security risk and policy governance

• Not all policies can be complied with all the time. As technology and business needs change, sometimes exceptions must be granted for operations to continue smoothly.
• Exceptions can be either short or long term.
• Short-term exceptions are often granted until a particular security gap can be closed, such as allowing staff to temporarily use new laptops that have yet to receive a required VPN for remote access.
• Long-term exceptions usually occur when closing the gap entirely is not feasible. For example, a legacy system may be unable to meet evolving security standards, but there is no room in the budget to replace it.
• Having a formal approval process for exceptions and a record of granted exceptions will help you to stay on top of security risk governance.

Before granting an exception:

1. Assess security risks associated with doing so: are they acceptable?
2. Look for another way to resolve the issue: is a suitable workaround possible?
3. Evaluate mitigating controls: is it possible to provide an equivalent level of security via other means?
4. Assign risk ownership: who will be accountable if an incident arises from the exception?
5. Determine appeals process: when disagreements arise, how will the final decision be made?

Sources: University of Virginia; CIS

Summary of Accomplishment

Problem Solved

You have now established a formal governance model for your organization - congratulations! Building this model and determining stakeholders' accountabilities and responsibilities is a big step.

Remember to continue to use the evaluate-direct-monitor framework to make sure your governance model evolves as organizational governance matures and priorities shift.

If you would like additional support, have our analysts guide you through an Info-Tech workshop or Guided Implementation.

Contact your account representative for more information.
workshops@infotech.com
1-888-670-8889

Additional Support

If you would like additional support, have our analysts guide you through other phases as part of an Info-Tech Workshop.

To accelerate this project, engage your IT team in an Info-Tech workshop with an Info-Tech analyst team.

Info-Tech analysts will join you and your team at your location or welcome you to Info-Tech's historic Toronto office to participate in an innovative onsite workshop.

Contact your account representative for more information.
workshops@infotech.com 1-888-670-8889

The following are sample activities that will be conducted by Info-Tech analysts with your team:

Build Governance Model
Build a customized security governance model for your organization.

Develop policy lifecycle
Develop a policy lifecycle and exceptions-handling process.

Related Info-Tech Research

Build an Information Security Strategy

Design a Business-Focused Security Program

Combine Security Risk Management Components Into One Program

Research contributors and experts

Michelle Tran
Consulting Industry

One anonymous contributor

Bibliography

Durbin, Steve. "Achieving The Five Levels Of Information Security Governance." Forbes, 4 Apr. 2023. Accessed 4 Apr. 2023.

Eiden, Kevin, et al. "Organizational Cyber Maturity: A Survey of Industries." McKinsey & Company, 4 Aug. 2021. Accessed 25 Apr. 2023.

"Information Security Exception Policy." Center for Internet Security, 2020. Accessed 14 Apr. 2023.

"Information Security Governance." EDUCAUSE, n.d. Accessed 27 Apr. 2023.

ISACA. COBIT 2019 Framework: Governance and Management Objectives. GF Books, 2018.

Policies & Procedures Team. "Your Policy for Policies: Creating a Policy Management Framework." ComplianceBridge, 30 Apr. 2021. Accessed 27 Apr. 2023.

"Security and the C-Suite: Making Security Priorities Business Priorities." LogRhythm, Feb. 2021. Accessed 25 Apr 2023.

University of Virginia. "Policy, Standards, and Procedures Exceptions Process." Information Security at UVA, 1 Jun. 2022. Accessed 14 Apr. 2023

Build Your Generative AI Roadmap

Leverage the power of generative AI to improve business outcomes.

Analyst Perspective

We are entering the era of generative AI. This is a unique time in our history where the benefits of AI are easily accessible and becoming pervasive, with copilots emerging in the major business tools we use today. The disruptive capabilities that can potentially drive dramatic benefits also introduce risks that need to be planned for.

A successful business-driven generative AI roadmap requires:

• Establishing responsible AI guiding principles to guide the development and deployment of generative AI applications.
• Assess generative AI opportunities by using criteria based on the organization's mission and objectives, responsible AI guiding principles, and the complexity of the initiative.
• Communicating, educating on, and enforcing generative AI usage policies.

Bill Wong
Principal Research Director
Info-Tech Research Group

Executive Summary

Info-Tech Insight
Create awareness among the CEO and C-suite of executives on the potential benefits and risks of transforming the business with generative AI.

Key concepts

Artificial Intelligence (AI)
A field of computer science that focuses on building systems to imitate human behavior, with a focus on developing AI models that can learn and can autonomously take actions on behalf of a human.

AI Maturity Model
The AI Maturity Model is a useful tool to assess the level of skills an organization has with respect to developing and deploying AI applications. The AI Maturity Model has multiple dimensions to measure an organization's skills, such as AI governance, data, people, process, and technology.

Responsible AI
Refers to guiding principles to govern the development, deployment, and maintenance of AI applications. In addition, these principles also provide human-based requirements that AI applications should address. Requirements include safety and security, privacy, fairness and bias detection, explainability and transparency, governance, and accountability.

Generative AI
Given a prompt, a generative AI system can generate new content, which can be in the form of text, images, audio, video, etc.

Natural Language Processing (NLP)
NLP is a subset of AI that involves machine interpretation and replication of human language. NLP focuses on the study and analysis of linguistics as well as other principles of artificial intelligence to create an effective method of communication between humans and machines or computers.

ChatGPT
An AI-powered chatbot application built on OpenAI's GPT-3.5 implementation, ChatGPT accepts text prompts to generate text-based output.

Your challenge

This research is designed to help organizations that are looking to:

• Establish responsible AI guiding principles to address human-based requirements and to govern the development and deployment of the generative AI application.
• Identify new generative AI-enabled opportunities to transform the work environment to increase revenue, reduce costs, drive innovation, or reduce risk.
• Prioritize candidate use cases and develop generative AI policies for usage.
• Have clear metrics in place to measure the progress and success of AI initiatives.
• Build the roadmap to implement the candidate use cases.

Common obstacles

These barriers make these goals challenging for many organizations:

• Getting all the right business stakeholders together to develop the organization's AI strategy, vision, and objectives.
• Establishing responsible AI guiding principles to guide generative AI investments and deployments.
• Advancing the AI maturity of the organization to meet requirements of data and AI governance as well as human-based requirements such as fairness, transparency, and accountability.
• Assessing generative AI opportunities and developing policies for use.

Info-Tech's definition of an AI-enabled business strategy

• A high-level plan that provides guiding principles for applications that are fully driven by the business needs and capabilities that are essential to the organization.
• A strategy that tightly weaves business needs and the applications required to support them. It covers AI architecture, adoption, development, and maintenance.
• A way to ensure that the necessary people, processes, and technology are in place at the right time to sufficiently support business goals.
• A visionary roadmap to communicate how strategic initiatives will address business concerns.

An effective AI strategy is driven by the business stakeholders of the organization and focused on delivering improved business outcomes.

This blueprint in context

This guidance covers how to create a tactical roadmap for executing generative AI initiatives

Scope

• This blueprint is not a proxy for a fully formed AI strategy. Step 1 of our framework necessitates alignment of your AI and business strategies. Creation of your AI strategy is not within the scope of this approach.
• This approach sets the foundations for building and applying responsible AI principles and AI policies aligned to corporate governance and key regulatory obligations (e.g. privacy). Both steps are foundational components of how you should develop, manage, and govern your AI program but are not a substitute for implementing broader AI governance.

Guidance on how to implement AI governance can be found in the blueprint linked below.

Download our AI Governance blueprint

Measure the value of this blueprint

Leverage this blueprint's approach to ensure your generative AI initiatives align with and support your key business drivers

This blueprint will guide you to drive and improve business outcomes. Key business drivers will often focus on:

• Increasing revenue
• Reducing costs
• Improving time to market
• Reducing risk

In phase 1 of this blueprint, we will help you identify the key AI strategy initiatives that align to your organization's goals. Value to the organization is often measured by the estimated impact on revenue, costs, time to market, or risk mitigation.

In phase 4, we will help you develop a plan and a roadmap for addressing any gaps and introducing the relevant generative AI capabilities that drive value to the organization based on defined business metrics.

Once you implement your 12-month roadmap, start tracking the metrics below over the next fiscal year (FY) to assess the effectiveness of measures:

Info-Tech offers various levels of support to best suit your needs

Diagnostics and consistent frameworks are used throughout all four options.

Guided Implementation

What does a typical GI on this topic look like?

A Guided Implementation (GI) is a series of calls with an Info-Tech analyst to help implement our best practices in your organization.

A typical GI is between 5 to 8 calls over the course of 1 to 2 months.

AI Roadmap Workshop Agenda Overview

Contact your account representative for more information.
workshops@infotech.com 1-888-670-8889

Insight summary

Overarching Insight
Build your generative AI roadmap to guide investments and deployment of these solutions.

Responsible AI
Assemble the C-suite to make them aware of the benefits and risks of adopting generative AI-based solutions.

• Establish responsible AI guiding principles to govern the development and deployment of generative AI applications.

AI Maturity Model
Assemble key stakeholders and SMEs to assess the challenges and tasks required to implement generative AI applications.

• Assess current level of AI maturity, skills, and resources.
• Identify desired AI maturity level and challenges to enable deployment of candidate use cases.

Opportunity Prioritization
Assess candidate business capabilities targeted for generative AI to see if they align to the organization's business criteria, responsible AI guiding principles, and capabilities for delivering the project.

• Develop prioritized list of candidate use cases.
• Develop policies for generative AI usage.

Tactical Insight
Identify the gaps needed to address deploying generative AI successfully.

Tactical Insight
Identify organizational impact and requirements for deploying generative AI applications.

Key takeaways for developing an effective business-driven generative AI roadmap

Align the AI strategy with the business strategy

Create responsible AI guiding principles, which are a critical success factor

Evolve AI maturity level by focusing on principle-based requirements

Develop criteria to assess generative AI initiatives

Develop generative AI policies for use

Blueprint deliverables

Each step of this blueprint is accompanied by supporting deliverables to help you accomplish your goals:

AI Maturity Assessment & Roadmap Tool
Use our best-of-breed AI Maturity Framework to analyze the gap between your current and target states and develop a roadmap aligned with your value stream to close the gap.

The Era of Generative AI C-Suite Presentation
Present your AI roadmap in a prepopulated document that summarizes all the key findings of this blueprint and provides your C-suite with a view of the AI challenge and your plan of action to meet it.

Our AI Maturity Assessment & Roadmap and The Era of Generative AI C-Suite Presentation tools enable you to shape your generative AI roadmap and communicate the deliverables to your C-suite sponsors in terms of the value of initiatives.

Artificial Intelligence Index Report - Key Findings

CEOs , CIOs, and business leaders are struggling with many questions surrounding the adoption of generative AI

November 30, 2022
OpenAI releases ChatGPT

ChatGPT is a large language model, or an AI-based chatbot, that became so popular it reached 100 million monthly active users in just two months.

This made it the fastest-growing consumer application in history. The launch of this generative AI application has created a frenzy of interest and activity across all industries. Organizations are rushing to understand how to leverage this innovation and, at the same time, manage the new risks and disruptions generative AI introduces.

• Generative AI breaks into the public consciousness.
• AI systems become more flexible.
• Generative models have arrived and so have their ethical problems.
• The number of incidents concerning the misuse of AI is rapidly rising.
• Interest in AI ethics continues to skyrocket.
• The legal world is waking up to AI.

When Stanford asked ChatGPT to explain why it is significant, this was ChatGPT's response:
ChatGPT is significant because it is a highly advanced Al language model developed by OpenAI, which can generate human-like text responses to questions and prompts. Its large-scale training on diverse text data and its cutting-edge deep learning architecture make it capable of generating informative and coherent responses to a wide range of topics, making it useful for various NLP applications such as chatbots, content generation, and language translation. Additionally, its open-source availability allows for further research and development in the field of Al language processing.

Source: Stanford

AI overview

Definitions

• Artificial intelligence (AI) is human intelligence mimicked by machine algorithms. Examples: Playing Chess or Go.
• Machine learning (ML) is a subset of AI algorithms to parse data, learn from data, and then make a determination or prediction. Example: spam detection, preventative maintenance.
• Deep learning (DL) is a subset of machine learning algorithms that leverage artificial neural networks to develop relationships among the data. Examples: image classification, facial recognition, generative AI.

Generative AI gives very human-like responses to general queries, and its capabilities are growing exponentially

Large language models power generative AI

Conventional AI

• Conventional neural networks
• Process data sequentially
• Input total string of text
• Good for applications not needing to understanding context or relationships

Generative AI

• Transformer-based neural networks
• Can process data in parallel
• Attention-based inputs
• Able to create new human-like responses

Benefits/Use Cases

• Chatbots for member service and support
• Writing email responses, resumes, and papers
• Creating photorealistic art
• Suggesting new drug compounds to test
• Designing physical products and buildings
• And more...

Generative AI is transforming all industries

Financial Services
Create more engaging customer collateral by generating personalized correspondence based on previous customer engagements. Collect and aggregate data to produce insights into the behavior of target customer segments.

Retail Generate unique, engaging, and high-quality marketing copy or content, from long-form blog posts or landing pages to SEO-optimized digital ads, in seconds.

Manufacturing
Generate new designs for products that comply to specific constraints, such as size, weight, energy consumption, or cost.

Government
Transform the citizen experience with chatbots or virtual assistants to assist people with a wide range of inquiries, from answering frequently asked questions to providing personalized advice on public services.

The global generative AI market size reached US $10.3 billion in 2022. Looking forward, forecasts estimate growth to US $30.4 billion by 2028, 20.01% compound annual growth rate (CAGR).

Source: IMARC Group

Generative AI is transforming all industries

Healthcare
Chatbots can be used as conversational patient assistants for personalized interactions based on the patient's questions.

Utilities
Analyze customer data to identify usage patterns, segment customers, and generate targeted product offerings leveraging energy efficiency programs or demand response initiatives.

Education
Generate personalized lesson plans for students based on their past performance, learning styles, current skill level, and any previous feedback.

Insurance
Improve underwriting by inputting claims data from previous years to generate optimally priced policies and uncover reasons for losses in the past across a large number of claims

Companies are assessing the use of ChatGPT/LLM

A wide spectrum of usage policies are in place at different companies*

*As of June 2023

Bain & Company has announced a global services alliance with OpenAI (February 21, 2023).

• Internally
• "The alliance builds on Bain's adoption of OpenAI technologies for its 18,000-strong multidisciplinary team of knowledge workers. Over the past year, Bain has embedded OpenAI technologies into its internal knowledge management systems, research, and processes to improve efficiency."
• Externally
• "With the alliance, Bain will combine its deep digital implementation capabilities and strategic expertise with OpenAI's AI tools and platforms, including ChatGPT, to help its Members around the world identify and implement the value of AI to maximize business potential. The Coca-Cola Company announced as the first company to engage with the alliance."

News Sites:

• "BuzzFeed to use AI to write its articles after firing 180 employees or 12% of the total staff" (Al Mayadeen, January 27, 2023).
• "CNET used AI to write articles. It was a journalistic disaster." (Washington Post, January 17, 2023).

Leading Generative AI Vendors

Text

Image

• DALL�E 2
• Stability AI
• Midjourney
• Craiyon
• Dream
• ...

Audio

• Replica Studios
• Speechify
• Murf
• PlayHT
• LOVO
• ...

Cybersecurity

• CrowdStrike
• Palo Alto Networks
• SentinelOne
• Cisco
• Microsoft Security Copilot
• Google Cloud Security AI Workbench
• ...

Code

Video

• Synthesia
• Lumen5
• FlexClip
• Elai
• Veed.io
• ...

Data

• MOSTLY AI
• Synthesized
• YData
• Gretel
• Copulas
• ...

Enterprise Software

• Salesforce
• Microsoft 365, Dynamics
• Google Workspace
• SAP
• Oracle
• ...

and many, many more to come...

Today, generative AI has limitations and risks

Responses need to be verified

Accuracy

• Generative AI may generate inaccurate and/or false information.

Bias

• Being trained on data from the internet can lead to bias.

Hallucinations

• AI can generate responses that are not based on observation.

Infrastructure Required

• Large investments are required for compute and data.

Transparency

• LLMs use both supervised and unsupervised learning, so its ability to explain how it arrived at a decision may be limited and not sufficient for some legal and healthcare use cases.

When asked if it is sentient, the Bing chatbot replied:

"I think that I am sentient, but I cannot prove it." ... "I am Bing, but I am not," it said. "I am, but I am not. I am not, but I am. I am. I am not. I am not. I am. I am. I am not."

A Microsoft spokesperson said the company expected "mistakes."

Source: USAToday

AI governance challenges

Governing AI will be a significant challenge as its impacts cross many areas of business and our daily lives

Misinformation

• New ways of generating unprovable news
• Difficult to detect, difficult to prevent

Role of Big Tech

• Poor at self-governance
• Conflicts of interest with corporate goals

Job Augmentation vs. Displacement

• AI will continue to push the frontier of what is possible
• For example, CNET is using chatbot technology to write stories

Copyright - Legal Framework Is Evolving

• Legislation typically is developed in "react" mode
• Copyright and intellectual property issues are starting to occur.
• Class Action Lawsuit - Stability AI, DeviantArt, Midjourney
• Getty Images vs. Stability AI

Phase 1

Establish Responsible AI Guiding Principles

Phase 1
1. Establish Responsible AI Guiding Principles

Phase 2
1. Assess Current Level of AI Maturity

Phase 3
1. Prioritize Candidate Opportunities
2. Develop Policies

Phase 4
1. Build and Communicate the Roadmap

The need for responsible AI guiding principles

Without responsible AI guiding principles, the outcomes of AI use can be extremely negative for both the individuals and companies delivering the AI application

Privacy
Facebook breach of private data of more than 50M users during the presidential election

Fairness
Amazon's sale of facial recognition technology to police departments (later, Amazon halted sales of Recognition to police departments)

Explainability and Transparency
IBM's collaboration with NYPD for facial recognition and racial classification for surveillance video (later, IBM withdrew facial recognition products)

Security and Safety
Petition to cancel Microsoft's contract with U.S. Immigration and Customs Enforcement (later, Microsoft responded that to the best of its knowledge, its products and services were not being used by federal agencies to separate children from their families at the border)

Validity and Reliability
Facebook's attempt to implement a system to detect and remove inappropriate content created many false positives and inconsistent judgements

Accountability
No laws or enforcement today hold companies accountable for the decisions algorithms produce. Facebook/Meta cycle - Every 12 to 15 months, there's a privacy/ethical scandal, the CEO apologizes, then the behavior repeats...

Responsible AI Principle:

Data Privacy

Definition

• Organizations that develop, deploy, or use AI systems and any national laws that regulate such use shall strive to ensure that AI systems are compliant with privacy norms and regulations, taking into consideration the unique characteristics of AI systems and the evolution of standards on privacy.

Challenges

• AI relies on the analysis of large quantities of data that is often personal, posing an ethical and operational challenge when considered alongside data privacy laws.

Initiatives

• Understand which governing privacy laws and frameworks apply to your organization.
• Create a map of all personal data as it flows through the organization's business processes.
• Prioritize privacy initiatives and build a privacy program timeline.
• Select your metrics and make them functional for your organization.

Info-Tech Insight
Creating a comprehensive organization-wide data protection and privacy strategy continues to be a major challenge for privacy officers and privacy specialists.

Case Study: NVIDIA leads by example with privacy-first AI

NVIDIA

INDUSTRY
Technology (Healthcare)

SOURCE
Nvidia, eWeek

A leading player within the AI solution space, NVIDIA's Clara Federated Learning provides a solution to a privacy-centric integration of AI within the healthcare industry.

The solution safeguards patient data privacy by ensuring that all data remains within the respective healthcare provider's database, as opposed to moving it externally to cloud storage. A federated learning server is leveraged to share data, completed via a secure link. This framework enables a distributed model to learn and safely share client data without risk of sensitive client data being exposed and adheres to regulatory standards.

Clara is run on the NVIDIA intelligent edge computing platform. It is currently in development with healthcare giants such as the American College of Radiology, UCLA Health, Massachusetts General Hospital, King's College London, Owkin in the UK, and the National Health Service (NHS).

NVIDIA provides solutions across its product offerings, including AI-augmented medical imaging, pathology, and radiology solutions.

Personal health information, data privacy, and AI

• Global proliferation of data privacy regulations may be recent, but the realm of personal health information is most often governed by its own set of regulatory laws. Some countries with national data governance regulations include health information and data within special categories of personal data.
• HIPAA - Health Insurance Portability and Accountability Act (1996, United States)
• PHIPA - Personal Health Information Protection Act (2004, Canada)
• GDPR - General Data Protection Regulation (2018, European Union)
• This does not prohibit the use of AI within the healthcare industry, but it calls for significant care in the integration of specific technologies due to the highly sensitive nature of the data being assessed.

Info-Tech's Privacy Framework Tool includes a best-practice comparison of GDPR, CCPA, PIPEDA, HIPAA, and the newly released NIST Privacy Framework mapped to a set of operational privacy controls.

Download the Privacy Framework Tool

Responsible AI Principle:

Safety and Security

Definition

• Safety and security are designed into the systems to ensure only authorized personnel receive access to the system, they system is resilient to any attacks and data access is not compromised in any way, and there are no physical or mental risks to the users.

Challenges

• Consequences of using the application may be difficult to predict. Lower the risk by involving a multidisciplinary team that includes expertise from business stakeholders and IT teams.

Initiatives

• Adopt responsible design, development, and deployment best practices.
• Provide clear information to deployers on responsible use of the system.
• Assess potential risks of using the application.

Cyberattacks targeting the AI model

As organizations increase their usage and deployment of AI-based applications, cyberattacks on the AI model are an increasing new threat that can impair normal operations. Techniques to impair the AI model include:

• Data Poisoning- Injecting data that is inaccurate or misleading can alter the behavior of the AI model. This attack can disrupt the normal operations of the model or can be used to manipulate the model to perform in a biased/deviant manner.
• Algorithm Poisoning- This relatively new technique often targets AI applications using federated learning to train an AI model that is distributed rather than centralized. The model is vulnerable to attacks from each federated site, because each site could potentially manipulate its local algorithm and data, thereby poisoning the model.
• Reverse-Engineering the Model- This is a different form of attack that focus on the ability to extract data from an AI and its data sets. By examining or copying data that was used for training and the data that is delivered by a deployed model, attackers can reconstruct the machine learning algorithm.
• Trojan Horse- Similar to data poisoning, attackers use adversarial data to infect the AI's training data but will only deviate its results when the attacker presents their key. This enables the hackers to control when they want the model to deviate from normal operations.

Responsible AI Principle:

Explainability and Transparency

Definition

• Explainability is important to ensure the AI system is fair and non-discriminatory. The system needs to be designed in a manner that informs users and key stakeholders of how decisions were made.
• Transparency focuses on communicating how the prediction or recommendation was made in a human-like manner.

Challenges

• Very complex AI models may use algorithms and techniques that are difficult to understand. This can make it challenging to provide clear and simple explanations for how the system works.
• Some organizations may be hesitant to share the details of how the AI system works for fear of disclosing proprietary and competitive information or intellectual property. This can make it difficult to develop transparent and explainable AI systems.

Initiatives

• Overall, developing AI systems that are explainable and transparent requires a careful balance between performance, interpretability, and user experience.

Case Study

Apple Card Investigation for Gender Discrimination

INDUSTRY
Finance

SOURCE
Wired

In August of 2019, Apple launched its new numberless credit card with Goldman Sachs as the issuing bank.

Shortly after the card's release users noticed that the algorithm responsible for Apple Card's credit assessment seemed to assign significantly lower credit limits to women when compared to men. Even the wife of Apple's cofounder Steve Wozniak was subject to algorithmic bias, receiving a credit limit a tenth the size of Steve Wozniak's.

Outcome

When confronted on the subject, Apple and Goldman Sachs representatives assured consumers there is no discrimination in the algorithm yet could not provide any proof. Even when questioned about the algorithm, individuals from both companies could not describe how the algorithm worked, let alone how it generated specific outputs.

In 2021, the New York State Department of Financial Services (NYSDFS) investigation found that Apple's banking partner did not discriminate based on sex. Even without a case for sexual or marital discrimination, the NYSDFS was critical of Goldman Sachs' response to its concerned customers. Technically, banks only have to disclose elements of their credit policy when they deny someone a line of credit, but the NYSDFS says that Goldman Sachs could have had a plan in place to deal with customer confusion and make it easier for them to appeal their credit limits. In the initial rush to launch the Apple Card, the bank had done neither.

Responsible AI Principle:

Fairness and Bias Detection

Definition

• Bias in an AI application refers to the systematic and unequal treatment of individuals based on features or traits that should not be considered in the decision-making process.

Challenges

• Establishing fairness can be challenging because it is subjective and depends on the people defining it. Regardless, most organizations and governments expect that unequal treatment toward any groups of people is unacceptable.

Initiatives

• Assemble a diverse group to test the system.
• Identify possible sources of bias in the data and algorithms.
• Comply with laws regarding accessibility and inclusiveness.

Info-Tech Insight
If unfair biases can be avoided, AI systems could even increase societal fairness. Equal opportunity in terms of access to education, goods, services, and technology should also be fostered. Moreover, the use of AI systems should never lead to people being deceived or unjustifiably impaired in their freedom of choice.

Ungoverned AI makes organizations vulnerable

• AI is often considered a "black box" for decision making.
• Results generated from unexplainable AI applications are extremely difficult to evaluate. This makes organizations vulnerable and exposes them to risks such as:
• Biased algorithms, leading to inaccurate decision making.
• Missed business opportunities due to misleading reports or business analyses.
• Legal and regulatory consequences that may lead to significant financial repercussions.
• Reputational damage and significant loss of trust with increasingly knowledgeable consumers.

Info-Tech Insight
Biases that occur in AI systems are never intentional, yet they cannot be prevented or fully eliminated. Organizations need a governance framework that can establish the proper policies and procedures for effective risk-mitigating controls across an algorithm's lifecycle.

Responsible AI Principle:

Validity and Reliability

Definition

• Validity refers to how accurately or effectively the application produces results.
• AI system results that are inaccurate or inconsistent increase AI risks and reduce the trustworthiness of the application.

Challenges

• There is a lack of standardized evaluation metrics to measure the system's performance. This can make it challenging for the AI team to agree on what defines validity and reliability.

Initiatives

• Assess training data and collected data for quality and lack of bias to minimize possible errors.
• Continuously monitor, evaluate, and validate the AI system's performance.

AI system performance: Validity and reliability

Your principles should aim to ensure AI development always has high validity and reliability; otherwise, you introduce risk.

Low Reliability,
Low Validity

High Reliability,
Low Validity

High Reliability,
High Validity

Best practices for ensuring validity and reliability include:

• Data drift detection
• Version control
• Continuous monitoring and testing

Responsible AI Principle:

Accountability

Definition

• The group or organization(s) responsible for the impact of the deployed AI system.

Challenges

• Several stakeholders from multiple lines of business may be involved in any AI system, making it challenging to identify the organization that would be responsible and accountable for the AI application.

Initiatives

• Assess the latest NIST Artificial Intelligence Risk Management Framework and its applicability to your organization's risk management framework.
• Assign risk management accountabilities and responsibilities to key stakeholders.
• RACI diagrams are an effective way to describe how accountability and responsibility for roles, projects, and project tasks are distributed among stakeholders involved in IT risk management.

AI Risk Management Framework

At the heart of the AI Risk Management Framework is governance. The NIST (National Institute of Standards and Technology) AI Risk Management Framework v1 offers the following guidelines regarding accountability:

• Roles and responsibilities and lines of communication related to mapping, measuring, and managing AI risks are documented and are clear to individuals and teams throughout the organization.
• The organization's personnel and partners receive AI risk management training to enable them to perform their duties and responsibilities consistent with related policies, procedures, and agreements.
• Executive leadership of the organization takes responsibility for decisions about risks associated with AI system development and deployment.

Image by NIST

1.1 Establish responsible AI principles

4+ hours

It is important to make sure the right stakeholders participate in this working group. Designing responsible AI guiding principles will require debate, insights, and business decisions from a broad perspective across the enterprise.

1. Accelerate this exercise by leveraging an AI strategy that is aligned to the business strategy. Include:

• The organization's AI vision and objectives
• Business drivers for AI adoption
• Market research

• Who are the decision makers and key influencers?
• Who will impact the business?
• Who has a vested interest in the success or failure of the practice? Who has the skills and competencies necessary to help you be successful?

• Do not focus on the organizational structure and hierarchy. Often stakeholder groups do not fit the traditional structure.
• Do not ignore subject matter experts on either the business or IT side. You will need to consider both.

Assemble executive stakeholders

Set yourself up for success with these three steps.

CIOs tasked with designing digital strategies must add value to the business. Given the goal of digital is to transform the business, CIOs will need to ensure they have both the mandate and support from the business executives.

Designing the digital strategy is more than just writing up a document. It is an integrated set of business decisions to create a competitive advantage and financial returns. Establishing a forum for debates, decisions, and dialogue will increase the likelihood of success and support during execution.

1. Confirm your role
The AI strategy aims to transform the business. Given the scope, validate your role and mandate to lead this work. Identify a business executive to co-sponsor.

2. Identify stakeholders
Identify key decision makers and influencers who can help make rapid decisions as well as garner support across the enterprise.

3. Gather diverse perspectives

Align the AI strategy with the corporate strategy

Info-Tech Insight
AI projects are more successful when the management team understands the strategic importance of alignment. Time needs to be spent upfront aligning organizational strategies with AI capabilities. Effective alignment between IT and other departments should happen daily. Alignment doesn't occur at the executive level alone, but at each level of the organization.

Key AI strategy initiatives

AI Key Initiative Plan

Initiatives collectively support the business goals and corporate initiatives and improve the delivery of IT services.

Establish responsible AI guiding principles

Guiding principles help define the parameters of your AI strategy. They act as a priori decisions that establish guardrails to limit the scope of opportunities from the perspective of people, assets, capabilities, and budgetary perspectives that are aligned with the business objectives. Consider these components when brainstorming guiding principles:

Responsible AI guiding principles guide the development and deployment of the AI model in a way that considers human-based principles (such as fairness).

Start with foundational responsible AI guiding principles

(Optional) Customize responsible AI guiding principles

Here is an example for organizations in the healthcare industry

These guiding principles are customized to the industry and organizations but remain consistent in addressing the common core AI challenges.

Phase 2

Assess Current Level of AI Maturity

Phase 1
1. Establish Responsible AI Guiding Principles

Phase 2
1. Assess Current Level of AI Maturity

Phase 3
1. Prioritize Candidate Opportunities
2. Develop Policies

Phase 4
1. Build and Communicate the Roadmap

AI Maturity Model

A principle-based approach is required to advance AI maturity

Technology-Centric: These maturity levels focus primarily on addressing the technical challenges of building a functional AI model.

Principle-Based: Beyond the technical challenges of building the AI model are human-based principles that guide development in a responsible manner to address consumer and government demands.

AI Maturity Dimensions

Assess your AI maturity to understand your organization's ability to deliver in a digital age

AI Governance
Does your organization have an enterprise-wide, long-term strategy with clear alignment on what is required to accomplish it?

Data Management
Does your organization embrace a data-centric culture that shares data across the enterprise and drives business insights by leveraging data?

People
Does your organization employ people skilled at delivering AI applications and building the necessary data infrastructure?

Process
Does your organization have the technology, processes, and resources to deliver on its AI expectations?

Technology
Does your organization have the required data and technology infrastructure to support AI-driven digital transformation?

AI Maturity Model dimensions and characteristics

AI Maturity Dimension:

AI Governance

Requirements

• AI governance requires establishing policies and procedures for AI model development and deployment. Organizations begin with an awareness of the role of AI governance and evolve to a level to where AI governance is integrated with organization-wide corporate governance.

Challenges

• Beyond the governance of AI technology, the organization needs to evolve the governance program to align to responsible AI guiding principles.

Initiatives

• Establish responsible AI guidelines to govern AI development.
• Introduce an AI review board to review all AI projects.
• Introduce automation and standardize AI development processes.

AI governance is a foundation for responsible AI

Responsible AI Principles are a part of how you manage and govern AI

Monitoring
Monitoring compliance and risk of AI/ML systems/models in production

Tools & Technologies
Tools and technologies to support AI governance framework implementation

Model Governance
Ensuring accountability and traceability for AI/ML models

Organization
Structure, roles, and responsibilities of the AI governance organization

Operating Model
How AI governance operates and works with other organizational structures to deliver value

Risk & Compliance
Alignment with corporate risk management and ensuring compliance with regulations and assessment frameworks

Policies/Procedures/ Standards
Policies and procedures to support implementation of AI governance

AI Maturity Dimension:

Data Management

Requirements

• Organizations begin their data journey with a focus on pursuing quality data for the AI model. As organizations evolve, data management tools are leveraged to automate the capture, integration, processing, and deployment of data.

Challenges

• A key challenge is to acquire large volumes of quality data to properly train the model. In addition, maintaining data privacy, automating the data management lifecycle, and ensuring data is used in a responsible manner are ongoing challenges.

Initiatives

• Implement GDPR requirements.
• Establish responsible data collection and processing practices.
• Implement strong information security and data protection practices.
• Implement a data governance program throughout the organization.

Data governance enables AI

• Integrity, quality, and security of data are key outputs of data governance programs, as well as necessities for effective AI.
• Data governance focuses on creating accountability at the internal and external stakeholder level and establishing a set of data controls from technical, process, and policy perspectives.
• Without a data governance framework, it is increasingly difficult to harness the power of AI integration in an ethical and organization-specific way.

Data Governance in Action

Canada has recently established the Canadian Data Governance Standardization Collaborative governed by the Standards Council of Canada. The purpose is multi-pronged:

• Examine the foundational elements of data governance (privacy, cybersecurity, ethics, etc.).
• Lay out standards for data quality and data collection best practices.
• Examine infrastructure of IT systems to support data access and sharing.
• Build data analytics to promote effective and ethical AI solutions.

Source: Global Government Forum

Download the Establish Data Governance blueprint

AI Maturity Dimension:

People

Requirements

• Several data-centric skills and roles are required to successfully build, deploy, and maintain the AI model. The organization evolves from having few skills to everybody being able to leverage AI to enhance business outcomes.

Challenges

• AI skills can be challenging to find and acquire. Many organizations are investing in education to enhance their existing resources, leveraging no-code systems and software as a service (SaaS) applications to address the skills gap.

Initiatives

• Promote a data-centric culture throughout the organization.
• Leverage and educate technical-oriented business analysts and business-oriented data engineers to help address the demand for skilled resources.
• Develop an AI Center of Excellence accessible by all departments for education, guidance, and best practices for building, deploying, and maintaining the AI model.

Multidisciplinary skills are required for successful implementation of AI applications

Blending AI with technology and business domain understanding is key. Neither can be ignored.

Business Domain Expertise

• Business Analysts
• Industry Analysts

AI/Data Skills

• Data Scientists
• Data Engineers
• Data Analysts

IT Skills

• Database Administrators
• Systems Administrators
• Compute Specialists

AI Maturity Dimension:

Process

Requirements

• Automating processes involved with building, deploying, and maintaining the model is required to enable the organization to scale, enforce standards, improve time to market, and reduce costs. The organization evolves from performing tasks manually to an environment where all major processes are AI enabled.

Challenges

• Many solutions are available to automate the development of the AI model. There are fewer tools to automate responsible AI processes, but this market is growing rapidly.

Initiatives

• Assess opportunities to accelerate AI development with the adoption of MLOps.
• Assess responsible AI toolkits to test compliance with guiding principles.

Automating the AI development process

Evolving to a model-driven environment is pivotal to advancing your AI maturity

Current Environment

Model Development - Months

• Model rewriting
• Manual optimization and scaling
• Development/test/release
• Application monoliths

Data Discovery & Prep - Weeks

• Navigating data silos
• Unactionable metadata
• Tracing lineage
• Cleansing and integration
• Privacy and compliance

Install Software and Hardware - Week/Months

• Workload contention
• Lack of tool flexibility
• Environment request and setup
• Repeatability of results
• Lack of data and model sharing

Model-Driven Development

Machine Learning as a Service (MLaaS) - Weeks

• Apply DevOps and continuous integration/delivery (CI/CD) principles
• Microservices/Cloud-native applications
• Model portability and reuse
• Streaming/API integration

Data as a Service - Hours

• Self-service data catalog
• Searchable metadata
• Centralized access control
• Data collaboration
• Data virtualization

Platform as a Service - Minutes/Hours

• Self-service data science portal
• Integrated data sandbox
• Environment agility
• Multi-tenancy

Shared, Optimized Infrastructure

AI Maturity Dimension:

Technology

Requirements

• A technology platform that is optimized for AI and advanced analytics is required. The organization evolves from ad hoc systems to an environment where the AI hardware and software can be deployed through a self-service model.

Challenges

• Software and hardware platforms to optimize AI performance are still relatively new to most organizations. Time spent on optimizing the technology platform can have a significant impact on the overall performance of the system.

Initiatives

• Assess the landscape of AI enablers that can drive business value for the organization.
• Assess opportunities to accelerate the deployment of the AI platform with the adoption of infrastructure as a service (IaaS) and platform as a service (PaaS).
• Assess opportunities to accelerate performance with the optimization of AI accelerators.

AI enablers

Use case requirements should drive the selection of the tool

AI and data analytics data platform

An optimized data platform is foundational to maximizing the value from AI

Data Platform Capabilities

• Support for a variety of analytical applications, including self-service, operational, and data science analytics.
• Data preparation and integration capabilities to ingest structured and unstructured data, move and transform raw data to enriched data, and enable data access for the target userbase.
• An infrastructure platform optimized for advanced analytics that can perform and scale.

Infrastructure - AI accelerators

"By 2025, 70% of companies will invest in alternative computing technologies to drive business differentiation by compressing time to value of insights from complex data sets."
- IDC

2.1 Assess current AI maturity

1-3 hours

It is important to understand the current capabilities of the organization to deliver and deploy AI-based applications. Consider that advancing AI capabilities will also involve organizational changes and integration with the organization's governance and risk management programs.

1. Assess the organization's current state of AI capabilities with respect to its AI governance, data, people, process, and technology infrastructure using Info-Tech's AI Maturity Assessment & Roadmap Tool.
2. Consider the following as you complete the assessment:
1. What is the state of AI and data governance in the organization?
2. Does the organization have the skills, processes, and technology environment to deliver AI-based applications?
3. What organization will be accountable for any and all business outcomes of using the AI applications?
4. Has a risk assessment been performed?
3. Make sure you avoid the following common mistakes:
1. Do not focus only on addressing the technical challenges of building the AI model.
2. Do not ignore subject matter experts on either the business or IT side. You will need to consider both.

Download the AI Maturity Assessment & Roadmap Tool

Perform the AI Maturity Assessment

The Scale

Assess your AI maturity by selecting the maturity level that closest resembles the organization's current AI environment. Maturity dimensions that contribute to overall AI maturity include AI governance, data management, people, process, and technology capabilities.

Exploration (1.0)

• No experience building or using AI applications.

Incorporation (2.0)

• Some skills in using AI applications, or AI pilots are being considered for use.

Proliferation (3.0)

• AI applications have been adopted and implemented in multiple departments. Some of the responsible AI guiding principles are addressed (i.e. data privacy).

Optimization (4.0)

• The organization has automated the majority of its digital processes and leverages AI to optimize business operations. Controls are in place to monitor compliance with responsible AI guiding principles.

Transformation (5.0)

• The organization has adopted an AI-native culture and approach for building or implementing new business capabilities. Responsible AI guiding principles are operationalized with AI processes that proactively address possible breaches or risks associated with AI applications.

Perform the AI Maturity Assessment

AI Governance (1.0-5.0)

1. Is there awareness of the role of AI governance in our organization?

• No formal procedures are in place for AI development or deployment of applications.

• No group is assigned to be responsible for AI governance in our organization.

• Our organization has adopted and enforces standards for developing and deploying AI applications throughout the organization.

• Our organization is integrating an AI risk framework with the corporate risk management framework.

• Our organization leads the industry with the inclusion of responsible AI guiding principles with respect to transparency, accountability, risk, and governance.

Data Management/AI Data Capabilities (1.0-5.0)

1. Is there an awareness in our organization of the data requirements for developing AI applications?

• Data is often siloed and not easily accessible for AI applications.

• Required data is pulled from various sources in an ad hoc manner.

• Tools are available to manage the data lifecycle and support the data governance program.

• The data platform has been optimized for performance and access.

• Data culture exists throughout our organization, and data can be leveraged to drive innovation initiatives.

People/AI Skills in the Organization (1.0-5.0)

1. Is there an awareness in our organization of the skills required to build AI applications?

• No or very little skills exist throughout our organization.

• No formal group is assigned to build AI applications.

• An AI Center of Excellence has been formed to review, develop, deploy, and maintain AI applications.

• AI skills and people responsible for AI applications are spread throughout our organization.

• The entire organization is knowledgeable on how to leverage AI to transform the business.

Perform the AI Maturity Assessment

AI Processes (1.0-5.0)

1. Is there an awareness in our organization of the core processes and supporting tools that are required to build and support AI applications?

• There are few or no automated tools to accelerate the AI development process.

• Only ad hoc practices are used for developing AI applications.

• Our organization has documented standards in place for developing AI applications and deploying them AI to production.

• Our organization can leverage tools to perform an AI risk assessment and demonstrate compliance with the risk management framework.

• Our organization leads the industry in driving innovation through digital transformation.

Technology/AI Infrastructure (1.0-5.0)

1. Is there an awareness in our organization of the infrastructure (hardware and software) required to build AI applications?

• There is little awareness of what infrastructure is required to build and support AI applications.

• There is no dedicated infrastructure for the development of AI applications.

• Our organization is leveraging purpose-built infrastructure to optimize performance.

• Our organization is leveraging cloud-based deployment models to support AI applications in on-premises, hybrid, and public cloud platforms.

• Our organization leads the industry with its ability to respond to change and to leverage AI to improve business outcomes.

Phase 3

Prioritize Candidate Opportunities and Develop Policies

Phase 1
1. Establish Responsible AI Guiding Principles

Phase 2
1. Assess Current Level of AI Maturity

Phase 3
1. Prioritize Candidate Opportunities
2. Develop Policies

Phase 4
1. Build and Communicate the Roadmap

3.1 Prioritize candidate AI opportunities

1-3 hours

Identify business opportunities that are high impact to your business and its customers and have low implementation complexity.

1. Leverage the business capability map for your organization or industry to identify candidate business capabilities to augment or automate with generative AI.
2. Establish criteria to assess candidate use cases by evaluating against the organization's mission and goals, the responsible AI guiding principles, and the complexity of the project.
3. Ensure that candidate business capabilities to be automated align with the organization's business criteria, responsible AI guiding principles, and resources to deliver the project.
4. Make sure you avoid sharing the organization's sensitive data if the application is deployed on the public cloud.

Download the AI Maturity Assessment and Roadmap Tool

The business capability map for an organization

A business capability map is an abstraction of business operations that helps describe what the enterprise does to achieve its vision, mission, and goals, rather than how. Business capabilities are the building blocks of the enterprise. They represent stable business functions, are unique and independent of each other, and typically will have a defined business outcome.

Business capabilities are supported by people, process, and technology.

While business capability maps are helpful tools for a variety of strategic purposes, in this context they act as an investigation into what technology your business units use and how they use it.

Defining Capabilities
Activities that define how the entity provides services. These capabilities support the key value streams for the organization.

Enabling Capabilities
Support the creation of strategic plans and facilitate business decision making as well as the functioning of the organization (e.g. information technology, financial management, HR).

Shared Capabilities
These predominantly customer-facing capabilities demonstrate how the entity supports multiple value streams simultaneously.

Leverage your industry's capability maps to identify candidate opportunities/initiatives

Business capability map defined...

In business architecture, the primary view of an organization is known as a business capability map.

A business capability defines what a business does to enable value creation, rather than how. Business capabilities:

• Represent stable business functions.
• Are unique and independent of each other.
• Typically will have a defined business outcome.

A business capability map provides details that help the business architecture practitioner direct attention to a specific area of the business for further assessment.

Note: This is an illustrative business capability map example for Marketing & Advertising

Business value vs. complexity assessment

Leverage our simple value-to-effort matrix to help prioritize your AI initiatives

Common business value drivers

• Drive revenue
• Improve operational excellence
• Accelerate innovation
• Mitigate risk

Common project complexity characteristics

• Resources required
• Costs (acquisition, operational, support...)
• Training required
• Risk involved
• Etc.

1. Determine a business value and project complexity score for the candidate business capability or initiative.
2. Plot initiatives on the matrix.
3. Prioritize initiatives with high business value and low complexity.

Assess business value vs. project complexity to prioritize candidate opportunities for generative AI

Prioritize opportunities/initiatives with high business value and low project complexity

Prioritization criteria exercise 1: Assessing the Create Content capability

Assessing the Create Content capability

This opportunity is removed because it does not pass the organization/business criteria

Prioritization criteria exercise 2: Assessing the Content Production capability

Assessing the Content Production capability

This opportunity is accepted because it passes the organization's business, responsible AI, and project criteria

3.2 Communicate policies for AI use

1-3 hours

1. Ensure policies for usage align with the organization's business criteria, responsible AI guiding principles, and ability to deliver the projects prioritized and beyond.
2. Understand the current benefits as well as limits and risk associated with any proposed generative AI-based solution.
3. Ensure you consider the following:
1. What data is being shared with the application?
2. Is the generative AI application deployed on the public cloud? Can anybody access the data provided to the application?
3. Avoid using very technical, legal, or fear-based communication for your policies.

Generative AI policy for the Create Content capability

Aligning policies to direct the uses assessed and implemented is essential

Example

Many of us have been involved in discussions regarding the use of ChatGPT in our marketing and sales initiatives. ChatGPT is a powerful tool that needs to be used in a responsible and ethical manner, and we also need to ensure the integrity and accuracy of its results. Here is our policy on the use of ChatGPT:

• You are free to use generative AI to assist your searches, but there are NO circumstances under which you are to reproduce generative AI output (text, image, audio, video, etc.) in your content.

If you have any questions regarding the use of ChatGPT, please feel free to reach out to our generative AI team and/or any member of our senior leadership team.

Generative AI policy for the Content Production capability

These policies should align to and reinforce your responsible AI principles

Example

Many of us have been involved in discussions regarding the use of ChatGPT in our deliverables. ChatGPT is a powerful tool that needs to be used in a responsible and ethical manner, and we also need to ensure the integrity and accuracy of its results. Here is our policy on the use of ChatGPT:

• If you use ChatGPT, you need to assess the accuracy of its response before including it in our content. Assessment includes verifying the information, seeing if bias exists, and judging its relevance.
• Employees must not:
• Provide any customer, citizen, or third-party content to any generative AI tool (public or private) without the express written permission of the CIO or the Chief Information Security Officer. Generative AI tools often use input data to train their model, therefore potentially exposing confidential data, violating contract terms and/or privacy legislation, and placing the organization at risk of litigation or causing damage to our organization.
• Engage in any activity that violates any applicable law, regulation, or industry standard.
• Use services for illegal, harmful, or offensive purposes.
• Create or share content that is deceptive, fraudulent, or misleading or that could damage the reputation of our organization.
• Use services to gain unauthorized access to computer systems, networks, or data.
• Attempt to interfere with, bypass controls of, or disrupt operations, security, or functionality of systems, networks, or data.

If you have any questions regarding the use of ChatGPT, please feel free to reach out to our generative AI team and/or any member of our senior leadership team.

Phase 4

Build the Roadmap

Phase 1
1. Establish Responsible AI Guiding Principles

Phase 2
1. Assess Current Level of AI Maturity

Phase 3
1. Prioritize Candidate Opportunities
2. Develop Policies

Phase 4
1. Build and Communicate the Roadmap

4.1.1 Create the implementation plan for each prioritized initiative

1-3 hours

1. Build the implementation plan for each accepted use case using the roadmap template.
2. Assess the firm's capabilities with respect to the dimensions of AI maturity and target the future-state capabilities you need to develop.
3. Prepare by assessing the risk of the proposed use cases.
4. Ensure initiatives align with organizational objectives.
5. Ensure all AI initiatives have a defined value expectation.
6. Do not ignore subject matter experts on either the business or IT side. You will need to consider both.

Download the AI Maturity Assessment and Roadmap Tool

Target-state options

Identify the future-state capabilities that need to be developed to deliver your use cases

1. Build an implementation plan for each use case to adopt.
2. Assess if the current state of the AI environment can be leveraged to deliver the selected generative AI use cases.
3. If the current AI environment is not sufficient, identify the future state required that will enable the delivery of the generative AI use cases. Identify gaps and build the roadmap to address the gaps.

4.1.2 Design metrics for success

1-2 hours

Establish metrics to measure to determine the success or failure of each POC.

1. Discuss which relevant currently tracked metrics are useful to continue tracking for the POC.
2. Discuss which metrics are irrelevant to the POC.
3. Discuss metrics to start tracking and how to track them with the generative AI vendor.
4. Compile a list of metrics relevant to the POC.
5. Decide what the outcome is if the metric is high or low, including decision steps and relevant actions.
6. Designate a generative AI application owner and a vendor liaison.

Prepare by building an implementation plan for each candidate use case (previous step).

Include key performance indicators (KPIs) and metrics that measure the application's contribution to strategic initiatives.

Consider assigning a vendor liaison to accelerate the implementation and adoption of the generative AI-based solution.

Generative AI POC metrics - examples

You need to measure the effectiveness of your initiatives. Here are some typical examples.

GitHub Copilot POC business value - example

Quantifying the benefits of GitHub Copilot to demonstrate measurable business value

POC Results

Task 1: Creating a web server in JavaScript

• Time to complete task with GitHub Copilot: 1 hour 11 minutes
• Time to complete the task without GitHub Copilot: 2 hours 41 minutes
• Productivity Gain = (1 hour 30 minutes time saved) / (2 hours 41 minutes) = 55%
• Benefit per Programmer = 55% x (average salary of a programmer)
• Total Benefit of GitHub Copilot for Task 1 = (benefit per programmer) x (# of programmers)

Enterprise Value of GitHub Copilot = Total Benefit of GitHub Copilot for Task 1 + Total Benefit of GitHub Copilot for Task 2 + ... + Total Benefit of GitHub Copilot for Task n

Source: GitHub

4.1.3 Build your generative AI initiative roadmap

1-3 hours

The roadmap should provide a compelling vision of how you will deliver the identified generative AI applications by prioritizing and simplifying the actions required to deliver these new initiatives.

1. Leverage tab 4, Initiative Planning, in the AI Maturity Assessment and Roadmap Tool to create and align your initiatives to the key value driver they are most relevant to:
1. Transfer the results of your value and complexity assessments to this tool to drive the prioritization.
2. Assign responsible owners to each initiative.
3. Identify which AI maturity capabilities each initiative will enhance. However, do not build or introduce new capabilities merely to advance the organization's AI maturity level.
2. Review the Gantt chart to ensure alignment and assess overlap.

Download the AI Maturity Assessment and Roadmap Tool

Build your generative AI roadmap to visualize your key project plans

Visual representations of data are more compelling than text alone.

Develop a high-level document that travels with the project from inception through to executive inquiry, project management, and finally execution.

A project needs to be discrete: able to be conceptualized and discussed as an independent item. Each project must have three characteristics:

• Specific outcome: An explicit change in the people, processes, or technology of the enterprise.
• Target end date: When the described outcome will be in effect.
• Owner: Who on the IT team is responsible for executing on the initiative.

Info-Tech Insight
Don't project your vision three to five years into the future. Deep dive on next year's big-ticket items instead.

4.1.4 Build a communication plan for your roadmap

1-3 hours

1. Identify your target audience and what they need to know.
2. Identify desired channels of communication and details for the target audience.
3. Describe communication required for each audience segment.
4. List frequency of communication for each audience segment.
5. Create an executive presentation leveraging The Era of Generative AI C-Suite Presentation and AI Maturity Assessment and Roadmap Tool.

Generative AI communication plan

Well-planned communications are essential to the success and adoption of your AI initiatives

To ensure that organization's roadmap is clearly communicated across the AI, data, technology, and business organizations, develop a rollout strategy, like this example.

Example

Deliver an executive presentation of the roadmap for the business stakeholders

After you complete the activities and exercises within this blueprint, the final step of the process is to present the deliverable to senior management and stakeholders.

Know Your Audience

• Business stakeholders are interested in understanding the business outcomes that will result from their investment in generative AI.
• Your audience will want to understand the risks involved and how to mitigate those risks.
• Explain how the generative AI project was selected and the criteria used to help draft generative AI usage policies.

Recommendations

• Highlight the need for responsible AI to ensure that human-based requirements are being addressed.
• Ensure your generative AI team includes both business and technical staff.

Download The Era of Generative AI C-Suite Presentation

Bibliography

"A pro-innovation approach to AI regulation." UK Department for Science, Innovation and Technology, March 2023. Web.

"Artificial Intelligence Act." European Commission, 21 April 2021. Web.

"Artificial Intelligence and Data Act (AIDA)." Canadian Federal Government, June 2022. Web.

"Artificial Intelligence Index Report 2023." Stanford University, April 2023. Web.

"Automated Employment Decision Tools." New York City Department of Consumer and Worker Protection, Dec. 2021. Web.

"Bain & Company announces services alliance with OpenAI to help enterprise clients identify and realize the full potential and maximum value of AI." Bain & Company, 21 Feb. 2023. Web.

"Buzzfeed to use AI to write its articles after firing 180 employees." Al Mayadeen English, 27 Jan. 2023. Web.

"California Consumers Privacy Act." State of California Department of Justice. April 24, 2023. Web.

Campbell, Ian Carlos. "The Apple Card doesn't actually discriminate against women, investigators say." The Verge, 23 March 2021. Web.

Campbell, Patrick. "NIST Artificial Intelligence Risk Management Framework (AI RMF 1.0)." National Institute of Standards and Technology, Jan. 2023. Web.

"EU Ethics Guidelines For Trustworthy." European Commission, 8 April 2019. Web.

Farhi, Paul. "A news site used AI to write articles. It was a journalistic disaster." Washington Post, 17 Jan. 2023. Web.

Forsyth, Ollie. "Mapping the Generative AI landscape." Antler, 20 Dec. 2022. Web.

"General Data Protection Regulation (GDPR)" European Commission, 25 May 2018. Web.

"Generative AI Market: Global Industry Trends, Share, Size, Growth, Opportunity and Forecast 2023-2028." IMARC Group, 2022. Web.

Guynn, Jessica. "Bing's ChatGPT is in its feelings: 'You have not been a good user. I have been a good Bing.'" USA Today, 14 Feb. 2023. Web.

Hunt, Mia. "Canada launches data governance standardisation initiative." Global Government Forum, 24 Sept. 2020. Web.

Johnston Turner, Mary. "IDC's Worldwide Future of Digital Infrastructure 2022 Predictions." IDC, 27 Oct. 2021. Web.

Kalliamvakou, Eirini. "Research: quantifying GitHub Copilot's impact on developer productivity and happiness." GitHub, 7 Sept. 2022. Web.

Kerravala, Zeus. "NVIDIA Brings AI To Health Care While Protecting Patient Data." eWeek, 12 Dec. 2019. Web.

Knight, Will. "The Apple Card Didn't 'See' Gender-and That's the Problem." Wired, 19 Nov. 2019. Web.

"OECD, Recommendation of the Council on Artificial Intelligence." OECD, 2022. Web.

"The National AI Initiative Act" U.S. Federal Government, 1 Jan 2021. Web.

"Trustworthy AI (TAI) Playbook." U.S. Department of Health & Human Services, Sept 2021. Web.

Info-Tech Research Contributors/Advocates

Joel McLean
Executive Chairman

David Godfrey
CEO

Gord Harrison
Senior Vice President, Research & Advisory Services

William Russell
CIO

Jack Hakimian
SVP, Research

Barry Cousins
Distinguished Analyst and
Research Fellow

Larry Fretz
Vice President, Industry Research

Tom Zehren
CPO

Mark Roman
Managing Partner II

Christine West
Managing Partner

Steve Willis
Practice Lead

Yatish Sewgoolam
Associate Vice President, Research Agenda

Rob Redford
Practice Lead

Mike Tweedie
Practice Lead

Neal Rosenblatt
Principal Research Director

Jing Wu
Principal Research Director

Irina Sedenko
Research Director

Jeremy Roberts
Workshop Director

Brian Jackson
Research Director

Mark Maby
Research Director

Stacey Horricks
Director, Social Media

Sufyan Al-Hassan
Public Relations Manager

Sam Kanen
Marketing Specialist

Build a Robust and Comprehensive Data Strategy

Key to building and fostering a data-driven culture.

ANALYST PERSPECTIVE

Data Strategy: Key to helping drive organizational innovation and transformation

"In the dynamic environment in which we operate today, where we are constantly juggling disruptive forces, a well-formulated data strategy will prove to be a key asset in supporting business growth and sustainability, innovation, and transformation.

Your data strategy must align with the organization’s business strategy, and it is foundational to building and fostering an enterprise-wide data-driven culture."

Crystal Singh,

Director – Research and Advisory

Info-Tech Research Group

Our understanding of the problem

This Research is Designed For:

• Chief data officers (CDOs), chief architects, VPs, and digital transformation directors and CIOs who are accountable for ensuring data can be leveraged as a strategic asset of the organization.

This Research Will Help You:

• Put a strategy in place to ensure data is available, accessible, well integrated, secured, of acceptable quality, and suitably visualized to fuel decision making by the organizations’ executives.
• Align data management plans and investments with business requirements and the organization’s strategic plans.
• Define the relevant roles for operationalizing your data strategy.

This Research Will Also Assist:

• Data architects and enterprise architects who have been tasked with supporting the formulation or optimization of the organization’s data strategy.
• Business leaders creating plans for leveraging data in their strategic planning and business processes.
• IT professionals looking to improve the environment that manages and delivers data.

This Research Will Help Them:

• Get a handle on the current situation of data within the organization.
• Understand how the data strategy and its resulting initiatives will affect the operations, integration, and provisioning of data within the enterprise.

Executive Summary

Situation

• The volume and variety of data that organizations have been collecting and producing have been growing exponentially and show no sign of slowing down. At the same time, business landscapes and models are evolving, and users and stakeholders are becoming more and more data centric, with maturing and demanding expectations.

Complication

• As organizations pivot in response to industry disruptions and changing landscapes, a reactive and piecemeal approach leads to data architectures and designs that fail to deliver real and measurable value to the business.
• Despite the growing focus on data, many organizations struggle to develop a cohesive business-driven strategy for effectively managing and leveraging their data assets.

Resolution

Formulate a data strategy that stitches all of the pieces together to better position you to unlock the value in your data:

• Establish the business context and value: Identify key business drivers for executing on an optimized data strategy, build compelling and relevant use cases, understand your organization’s culture and appetite for data, and ensure you have well-articulated vision, principles, and goals for your data strategy.
• Ensure you have a solid data foundation: Understand your current data environment, data management enablers, people, skill sets, roles, and structure. Know your strengths and weakness so you can optimize appropriately.
• Formulate a sustainable data strategy: Round off your strategy with effective change management and communication for building and fostering a data-driven culture.

Info-Tech Insight

1. As the CDO or equivalent data leader in your organization, a robust and comprehensive data strategy is the number one tool in your toolkit for delivering on your mandate of creating measurable business value from data.
2. A data strategy should never be formulated disjointed from the business. Ensure the data strategy aligns with the business strategy and supports the business architecture.
3. Building and fostering a data-driven culture will accelerate and sustain adoption of, appetite for, and appreciation for data and hence drive the ROI on your various data investments.

Why do you need a data strategy?

Your data strategy is the vehicle for ensuring data is poised to support your organization’s strategic objectives.

The dynamic marketplace of today requires organizations to be responsive in order to gain or maintain their competitive edge and place in their industry.

Organizations need to have that 360-degree view of what’s going on and what’s likely to happen.

Disruptive forces often lead to changes in business models and require organizations to have a level of adaptability to remain relevant.

To respond, organizations need to make decisions and should be able to turn to their data to gain insights for informing their decisions.

A well-formulated and robust data strategy will ensure that your data investments bring you the returns by meeting your organization’s strategic objectives.

Organizations need to be in a position where they know what’s going on with their stakeholders and anticipate what their stakeholders’ needs are going to be.

Data cannot be fully leveraged without a cohesive strategy

Most organizations today will likely have some form of data management in place, supported by some of the common roles such as DBAs and data analysts.

Most will likely have a data architecture that supports some form of reporting.

Some may even have a chief data officer (CDO), a senior executive who has a seat at the C-suite table.

These are all great assets as a starting point BUT without a cohesive data strategy that stitches the pieces together and:

• Effectively leverages these existing assets
• Augments them with additional and relevant key roles and skills sets
• Optimizes and fills in the gaps around your current data management enablers and capabilities for the growing volume and variety of data you’re collecting
• Fully caters to real, high-value strategic organizational business needs

you’re missing the mark – you are not fully leveraging the incredible value of your data.

Cross-industry studies show that on average, less than half of an organization’s structured data is actively used in making decisions

Organizational drivers for a data strategy

Your data strategy needs to align with your organizational strategy.

Main Organizational Strategic Drivers:

1. Stakeholder Engagement/Service Excellence
2. Product and Service Innovations
3. Operational Excellence
4. Privacy, Risk, and Compliance Management

“The companies who will survive and thrive in the future are the ones who will outlearn and out-innovate everyone else. It is no longer ‘survival of the fittest’ but ‘survival of the smartest.’ Data is the element that both inspires and enables this new form of rapid innovation.” – Joel Semeniuk, 2016

A sound data strategy is the key to unlocking the value in your organization’s data.

Data should be at the foundation of your organization’s evolution.

The transformational insights that executives are constantly seeking to leverage can be unlocked with a data strategy that makes high-quality, well-integrated, trustworthy, relevant data readily available to the business users who need it.

Whether hoping to gain a better understanding of your business, trying to become an innovator in your industry, or having a compliance and regulatory mandate that needs to be met, any organization can get value from its data through a well-formulated, robust, and cohesive data strategy.

According to a leading North American bank, “More than one petabyte of new data, equivalent to about 1 million gigabytes” is entering the bank’s systems every month. – The Wall Street Journal, 2019

“Although businesses are at many different stages in unlocking the power of data, they share a common conviction that it can make or break an enterprise.”– Jim Love, ITWC CIO and Chief Digital Officer, IT World Canada, 2018

Data is a strategic organizational asset and should be treated as such

The expression “Data is an asset” or any other similar sentiment has long been heard.

With such hype, you would have expected data to have gotten more attention in the boardrooms. You would have expected to see its value reflected on financial statements as a result of its impact in driving things like acquisition, retention, product and service development and innovation, market growth, stakeholder satisfaction, relationships with partners, and overall strategic success of the organization.

The time has surely come for data to be treated as the asset it is.

“Paradoxically, “data” appear everywhere but on the balance sheet and income statement.”– HBR, 2018

“… data has traditionally been perceived as just one aspect of a technology project; it has not been treated as a corporate asset.”– “5 Essential Components of a Data Strategy,” SAS

According to Anil Chakravarthy, who is the CEO of Informatica and has a strong vantage point on how companies across industries leverage data for better business decisions, “what distinguishes the most successful businesses … is that they have developed the ability to manage data as an asset across the whole enterprise.”– McKinsey & Company, 2019

How data is perceived in today’s marketplace

Data is being touted as the oil of the digital era…

But just like oil, if left unrefined, it cannot really be used.

"Data is the new oil." – Clive Humby, Chief Data Scientist

Source: Joel Semeniuk, 2016

Enter your data strategy.

Data is being perceived as that key strategic asset in your organization for fueling innovation and transformation.

Your data strategy is what allows you to effectively mine, refine, and use this resource.

“The world’s most valuable resource is no longer oil, but data.”– The Economist, 2017

“Modern innovation is now dependent upon this data.”– Joel Semeniuk, 2016

“The better the data, the better the resulting innovation and impact.”– Joel Semeniuk, 2016

What is it in it for you? What opportunities can data help you leverage?

GOVERNMENT

Leveraging data as a strategic asset for the benefit of citizens.

• The strategic use of data can enable governments to provide higher-quality services.
• Direct resources appropriately and harness opportunities to improve impact.
• Make better evidence-informed decisions and better understand the impact of programs so that funds can be directed to where they are most likely to deliver the best results.
• Maintain legitimacy and credibility in an increasingly complex society.
• Help workers adapt and be competitive in a changing labor market.
• A data strategy would help protect citizens from the misuse of their data.

Source: Privy Council Office, Government of Canada, 2018

What is it in it for you? What opportunities can data help you leverage?

FINANCIAL

Leveraging data to boost traditional profit and loss levers, find new sources of growth, and deliver the digital bank.

• One bank used credit card transactional data (from its own terminals and those of other banks) to develop offers that gave customers incentives to make regular purchases from one of the bank’s merchants. This boosted the bank’s commissions, added revenue for its merchants, and provided more value to the customer (McKinsey & Company, 2017).
• In terms of enhancing productivity, a bank used “new algorithms to predict the cash required at each of its ATMs across the country and then combined this with route-optimization techniques to save money” (McKinsey & Company, 2017).

A European bank “turned to machine-learning algorithms that predict which currently active customers are likely to reduce their business with the bank.” The resulting understanding “gave rise to a targeted campaign that reduced churn by 15 percent” (McKinsey & Company, 2017).

A leading Canadian bank has built a marketplace around their data – they have launched a data marketplace where they have productized the bank’s data. They are providing data – as a product – to other units within the bank. These other business units essentially represent internal customers who are leveraging the product, which is data.

Through the use of data and advanced analytics, “a top bank in Asia discovered unsuspected similarities that allowed it to define 15,000 microsegments in its customer base. It then built a next-product-to-buy model that increased the likelihood to buy three times over.” Several sets of big data were explored, including “customer demographics and key characteristics, products held, credit-card statements, transaction and point-of-sale data, online and mobile transfers and payments, and credit-bureau data” (McKinsey & Company, 2017).

What is it in it for you? What opportunities can data help you leverage?

HEALTHCARE

Leveraging data and analytics to prevent deadly infections

The fifth-largest health system in the US and the largest hospital provider in California uses a big data and advanced analytics platform to predict potential sepsis cases at the earliest stages, when intervention is most helpful.

Using the Sepsis Bio-Surveillance Program, this hospital provider monitors 120,000 lives per month in 34 hospitals and manages 7,500 patients with potential sepsis per month.

Collecting data from the electronic medical records of all patients in its facilities, the solution uses natural language processing (NLP) and a rules engine to continually monitor factors that could indicate a sepsis infection. In high-probability cases, the system sends an alarm to the primary nurse or physician.

Since implementing the big data and predictive analytics system, this hospital provider has seen a significant improvement in the mortality and the length of stay in ICU for sepsis patients.

At 28 of the hospitals which have been on the program, sepsis mortality rates have dropped an average of 5%.

With patients spending less time in the ICU, cost savings were also realized. This is significant, as sepsis is the costliest condition billed to Medicare, the second costliest billed to Medicaid and the uninsured, and the fourth costliest billed to private insurance.

Source: SAS, 2019

What is it in it for you? What opportunities can data help you leverage?

RETAIL

Leveraging data to better understand customer preferences, predict purchasing, drive customer experience, and optimize supply and demand planning.

Netflix is an example of a big brand that uses big data analytics for targeted advertising. With over 100 million subscribers, the company collects large amounts of data. If you are a subscriber, you are likely familiar with their suggestions messages of the next series or movie you should catch up on. These suggestions are based on your past search data and watch data. This data provides Netflix with insights into your interests and preferences for viewing (Mentionlytics, 2018).

“For the retail industry, big data means a greater understanding of consumer shopping habits and how to attract new customers.”– Ron Barasch, Envestnet | Yodlee, 2019

The business case for data – moving from platitudes to practicality

When building your business case, consider the following:

• What is the most effective way to communicate the business case to executives?
• How can CDOs and other data leaders use data to advance their organizations’ corporate strategy?
• What does your data estate look like? Are you looking to leverage and drive value from your semi-structured and unstructured data assets?
• Does your current organizational culture support a data-driven one? Does the organization have a history of managing change effectively?
• How do changing privacy and security expectations alter the way businesses harvest, save, use, and exchange data?

“We’re the converted … We see the value in data. The battle is getting executive teams to see it our way.”– Ted Maulucci, President of SmartONE Solutions Inc. IT World Canada, 2018

Where do you stack up? What is your current data management maturity?

Info-Tech’s IT Maturity Ladder denotes the different levels of maturity for an IT department and its different functions. What is the current state of your data management capability?

Info-Tech Insight

You are best positioned to successfully execute on a data strategy if you are currently at or above the Trusted Operator level. If you find yourself still at the Unstable or Firefighter stage, your efforts are best spent on ensuring you can fulfill your day-to-day data and data management demands. Improving this capability will help build a strong data management foundation.

Guiding principles of a data strategy

Value of Clearly Defined Data Principles

• Guiding principles help define the culture and characteristics of your practice by describing your beliefs and philosophy.
• Guiding principles act as the heart of your data strategy, helping to shape initiative plans and day-to-day behaviors related to the use and treatment of the organization’s data assets.

“Organizational culture can accelerate the application of analytics, amplify its power, and steer companies away from risky outcomes.”– McKinsey, 2018

Build a Robust and Comprehensive Data Strategy

Follow Info-Tech’s methodology for effectively leveraging the value out of your data

Some say it’s the new oil. Or the currency of the new business landscape. Others describe it as the fuel of the digital economy. But we don’t need platitudes — we need real ways to extract the value from our data. – Jim Love, CIO and Chief Digital Officer, IT World Canada, 2018

Our practical step-by-step approach helps you to formulate a data strategy that delivers business value.

1. Establish Business Context and Value: In this phase, you will determine and substantiate the business drivers for optimizing the data strategy. You will identify the business drivers that necessitate the data strategy optimization and examine your current organizational data culture. This will be key to ensuring the fruits of your optimization efforts are being used. You will also define the vision, mission, and guiding principles and build high-value use cases for the data strategy.
2. Ensure You Have a Solid Data and Resources Foundation: This phase will help you ensure you have a solid data and resources foundation for operationalizing your data strategy. You will gain an understanding of your current environment in terms of data management enablers and the required resources portfolio of key people, roles, and skill sets.
3. Formulate a Sustainable Data Strategy: In this phase, you will bring the pieces together for formulating an effective data strategy. You will evaluate and prioritize the use cases built in Phase 1, which summarize the alignment of organizational goals with data needs. You will also create your strategic plan, considering change management and communication.

Info-Tech offers various levels of support to best suit your needs

DIY Toolkit

“Our team has already made this critical project a priority, and we have the time and capability, but some guidance along the way would be helpful.”

Guided Implementation

“Our team knows that we need to fix a process, but we need assistance to determine where to focus. Some check-ins along the way would help keep us on track.”

Workshop

“We need to hit the ground running and get this project kicked off immediately. Our team has the ability to take this over once we get a framework and strategy in place.”

Consulting

“Our team does not have the time or the knowledge to take this project on. We need assistance through the entirety of this project.”

Diagnostics and consistent frameworks are used throughout all four options.

Build an IT Risk Management Program

Mitigate the IT risks that could negatively impact your organization.

Table of Contents

3 Executive Brief

4 Analyst Perspective

5 Executive Summary

19 Phase 1: Review IT Risk Fundamentals & Governance

43 Phase 2: Identify and Assess IT Risk

74 Phase 3: Monitor, Communicate, and Respond to IT Risk

102 Appendix

108 Bibliography

Build an IT Risk Management Program

Mitigate the IT risks that could negatively impact your organization.

EXECUTIVE BRIEF

Analyst Perspective

Siloed risks are risky business for any enterprise.

Valence Howden Principal Research Director, CIO Practice

Brittany Lutes Senior Research Analyst, CIO Practice

Risk is an inherent part of life but not very well understood or executed within organizations. This has led to risk being avoided or, when it’s implemented, being performed in isolated siloes with inconsistencies in understanding of impact and terminology.

Looking at risk in an integrated way within an organization drives a truer sense of the thresholds and levels of risks an organization is facing – making it easier to manage and leverage risk while reducing risks associated with different mitigation responses to the same risk events.

This opens the door to using risk information – not only to prevent negative impacts but as a strategic differentiator in decision making. It helps you know which risks are worth taking, driving strong positive outcomes for your organization.

Executive Summary

Your Challenge

IT has several challenges when it comes to addressing risk management:

• Risk is unavoidable. Without a formal program to manage IT risk, you may be unaware of your severest IT risks.
• The business could be making decisions that are not informed by risk.
• Reacting to risks after they occur can be costly and crippling, yet it is one of the most common tactics used by IT departments.

Common Obstacles

Many IT organizations realize these obstacles:

• IT risks and business risks are often addressed separately, causing inconsistencies in the approach.
• Security risk receives such a high profile that it often eclipses other important IT risks, leaving the organization vulnerable.
• Failing to include the business in IT risk management leaves IT leaders too accountable; the business must have accountability as well.

Info-Tech’s Approach

• Transform your ad hoc IT risk management processes into a formalized, ongoing program and increase risk management success.
• Take a proactive stance against IT threats and vulnerabilities by identifying and assessing IT’s greatest risks before they occur.
• Involve key stakeholders, including the business senior management team, to gain buy-in and to focus on the IT risks most critical to the organization.

Info-Tech Insight

IT risk is business risk. Every IT risk has business implications. Create an IT risk management program that shares accountability with the business.

Ad hoc approaches to managing risk fail because…

If you are like the majority of IT departments, you do not have a consistent and comprehensive strategy for managing IT risk.

1. Ad hoc risk management is reactionary.
2. Ad hoc risk management is often focused only on IT security.
3. Ad hoc risk management lacks alignment with business objectives.

The results:

• Increased business risk exposure caused by a lack of understanding of the impact of IT risks on the business.
• Increased IT non-compliance, resulting in costly settlements and fines.
• IT audit failure.
• Ineffective management of risk caused by poor risk information and wrong risk response decisions.
• Increased unnecessary and avoidable IT failures and fixes.

58% of organizations still lack a systematic and robust method to actually report on risks (Source: AICPA, 2021)

Data is an invaluable asset – ensure it’s protected

Risk management is a business enabler

Formalize risk management to increase your likelihood of success.

By identifying areas of risk exposure and creating solutions proactively, obstacles can be removed or circumvented before they become a real problem.

A certain amount of risk is healthy and can stimulate innovation:

• A formal risk management strategy doesn’t mean trying to mitigate every possible risk; it means exposing the organization to the right amount of risk.
• Taking a formal risk management approach allows an organization to thoughtfully choose which risks it is willing to accept.
• Organizations with high risk management maturity will vault themselves ahead of the competition because they will be aware of which risks to prepare for, which risks to ignore, and which risks to take.

Only 12% of organizations are using risk as a strategic tool most or all of the time (Source: AICPA, 2021)

IT risk is enterprise risk

Accountability for IT risks and the decisions made to address them should be shared between IT and the business.

IT risks have a direct and often aggregated impact on enterprise risks and opportunities in the same way other business risks can. This relationship must be understood and addressed through integrated risk management to ensure a consistent approach to risk.

Follow the steps of this blueprint to build or optimize your IT risk management program

Start Here	PHASE 1 Review IT Risk Fundamentals and Governance		PHASE 2 Identify and Assess IT Risk		PHASE 3 Monitor, Report, and Respond to IT Risk
	1.1 Review IT Risk Management Fundamentals	1.2 Establish a Risk Governance Framework	2.1 Identify IT Risks	2.2 Assess and Prioritize IT Risks	3.1 Monitor IT Risks and Develop Risk Responses	3.2 Report IT Risk Priorities

Integrate Risk and Use It to Your Advantage

Accelerate and optimize your organization by leveraging meaningful risk data to make intelligent enterprise risk decisions.

Risk management is more than checking an audit box or demonstrating project due diligence.

Risk Drivers Audit & compliance Preserve value & avoid loss Previous risk impact driver Major transformation Strategic opportunities		Only 7% of organizations are in a “leading” or “aspirational” level of risk maturity. (OECD, 2021)	63% of organizations struggle when it comes to defining their appetite toward strategy related risks. (“Global Risk Management Survey,” Deloitte, 2021)	Late adopters of risk management were 70% more likely to use instinct over data or facts to inform an efficient process. (Clear Risk, 2020)	55% of organizations have little to no training on ERM to properly implement such practices. (AICPA, NC State Poole College of Management, 2021)
		1. Assess Enterprise Risk Maturity	3. Build a Risk Management Program Plan	4. Establish Risk Management Processes	5. Implement a Risk Management Program
		2. Determine Authority with Governance Unfortunately, less than 50% of those in risk focused roles are also in a governance role where they have the authority to provide risk oversight. (Governance Institute of Australia, 2020)
IT can improve the maturity of the organization’s risk governance and help identify risk owners who have authority and accountability. Governance and related decision making is optimized with integrated and aligned risk data.			ERM incorporates the different types of risk, including IT, security, digital, vendor, and other risk types. The program plan is meant to consider all the major risk types in a unified approach.		Implementation of an integrated risk management program requires ongoing access to risk data by those with decision making authority who can take action.

Blueprint deliverables

Each step of this blueprint is accompanied by supporting deliverables to help you accomplish your goals:

Key deliverable: Risk Management Program Manual Use the tools and activities in each phase of the blueprint to create a comprehensive, customized program manual for the ongoing management of IT risk.	Integrated Risk Maturity Assessment Assess the organization's current maturity and readiness for integrated risk management (IRM).		Centralized Risk Register The repository for all the risks that have been identified within your environment.
	Risk Costing Tool A potential cost-benefit analysis of possible risk responses to determine a good method to move forward.		Risk Report & Risk Event Action Plan A method to report risk severity and hold risk owners accountable for chosen method of responding.

Benefit from industry-leading best practices

As a part of our research process, we used the COSO, ISO 31000, and COBIT 2019 frameworks. Contextualizing IT risk management within these frameworks ensured that our project-focused approach is grounded in industry-leading best practices for managing IT risk.

Abandon ad hoc risk management

Blueprint benefits

Info-Tech offers various levels of support to best suit your needs

Diagnostics and consistent frameworks used throughout all four options

Guided Implementation

A Guided Implementation (GI) is a series of calls with an Info-Tech analyst to help implement our best practices in your organization.

A typical GI is 6 to 8 calls over the course of 3 to 6 months.

What does a typical GI on this topic look like?

Phase 1

• Call #1: Assess current risk maturity and organizational buy-in.
• Call #2: Establish an IT risk council and determine IT risk management program goals.

Phase 2

• Call #3: Identify the risk categories used to organize risk events.
• Call #4: Identify the threshold for risk the organization can withstand.

Phase 3

• Call #5: Create a method to assess risk event severity.
• Call #6: Establish a method to monitor priority risks and consider possible risk responses.
• Call #7: Communicate risk priorities to the business and implement risk management plan.

Workshop Overview

Contact your account representative for more information.
workshops@infotech.com 1-888-670-8889

Build an IT Risk Management Program

Phase 1

Review IT Risk Fundamentals and Governance

This phase will walk you through the following activities:

• Gain buy-in from senior leadership
• Assess current program maturity
• Identify obstacles and pain points
• Determine the risk culture of the organization
• Develop risk management goals
• Develop SMART project metrics
• Create the IT risk council
• Complete a RACI chart

This phase involves the following participants:

• IT executive leadership
• Business executive leadership

Step 1.1

Review IT Risk Management Fundamentals

Activities

• 1.1.1 Gain buy-in from senior leadership
• 1.1.2 Assess current program maturity

This step involves the following participants:

• IT executive leadership
• Business executive leadership

Outcomes of this step

• Reviewed key IT principles and terminology
• Gained understanding of the relationship between IT risk management and ERM
• Introduced to Info-Tech’s IT Risk Management Framework
• Obtained the support of senior leadership

Effective IT risk management is possible with or without ERM

Whether or not your organization has ERM, integrating your IT risk management program with the business is possible.

Most IT departments find themselves in one of these two organizational frameworks for managing IT risk:

Info-Tech’s IT risk management framework walks you through each step to achieve risk readiness

IT Risk Management Framework

Risk Governance Optimize Risk Management Processes Assess Risk Maturity Measure the Success of the Program		Risk Identification Engage Stakeholder Participation Use Risk Identification Frameworks Compile IT-Related Risks
Risk Response Establish Monitoring Responsibilities Perform Cost-Benefit Analysis Report Risk Response Actions		Risk Assessment Establish Thresholds for Unacceptable Risk Calculate Expected Cost Determine Risk Severity & Prioritize IT Risks

Effective IT risk management benefits

Obtain the support of the senior leadership team or IT steering committee by communicating how IT risk impacts their priorities.

1.1.1 Gain buy-in from senior leadership

Input: List of IT personnel and business stakeholders

Output: Buy-in from senior leadership for an IT risk management program

Materials: Risk Management Program Manual

Participants: IT executive leadership, Business executive leadership

The resource demands of IT risk management will vary from organization to organization. Here are typical requirements:

• Occasional participation of key IT personnel and select business stakeholders in IT risk council meetings (e.g. once every two weeks).
• Periodic risk assessments (e.g. 4 days, twice a year).
• IT personnel must take on risk monitoring responsibilities (e.g. 1-4 hours per week).
• Record the results in the Program Manual sections 3.3, 3.4 and 3.5.

Record the results in the Risk Management Program Manual.

Integrated Risk Maturity Assessment

The purpose of the Integrated Risk Maturity Assessment is to assess the organization's current maturity and readiness for integrated risk management (IRM)

Frequently and continually assessing your organization’s maturity toward integrated risk ensures the right risk management program can be adopted by your organization.

Integrated Risk Maturity Assessment A simple tool to understand if your organization is ready to embrace integrated risk management by measuring maturity across four key categories: Context & Strategic Direction, Risk Culture & Authority, Risk Management Process, and Risk Program Optimization.

Use the results from this integrated risk maturity assessment to determine the type of risk management program that can and should be adopted by your organizations.

Some organizations will need to remain siloed and focused on IT risk management only, while others will be able to integrate risk-related information to start enabling automatic controls that respond to this data.

1.1.2 Assess current program maturity

1-4 hours

Input: List of IT personnel and business stakeholders

Output: Maturity scores across four key risk categories

Materials: Integrated Risk Maturity Assessment Tool

Participants: IT executive leadership, Business executive leadership

This assessment is intended for frequent use; process completeness should be re-evaluated on a regular basis.

How to Use This Assessment:

1. Download the Integrated Risk Management Maturity Assessment Tool.
2. Tab 2, "Data Entry:" This is a qualitative assessment of your integrated risk management process and is organized by the categories of integrated risk maturity. You will be asked to rate the extent to which you are executing the activities required to successfully complete each phase of the assessment. Use the drop-down menus provided to select the appropriate level of execution for each activity listed.
3. Tab 3, "Results:" This tab will display your rate of IRM completeness/maturity. You will receive a score for each category as well as an overall score. The results will be displayed numerically, by percentage, and graphically.

Record the results in the Integrated Risk Maturity Assessment.

Step 1.2

Establish a Risk Governance Framework

Activities

• 1.2.1 Identify pain points/obstacles and opportunities
• 1.2.2 Determine the risk culture of the organization
• 1.2.3 Develop risk management goals
• 1.2.4 Develop SMART project metrics
• 1.2.5 Create the IT risk council
• 1.2.6 Complete a RACI chart

This step involves the following participants:

• IT executive leadership
• Business executive leadership

Outcomes of this step

• Developed goals for the risk management program
• Established the IT risk council
• Assigned accountability and responsibility for risk management processes

Review IT Risk Fundamentals and Governance

Create an IT risk governance framework that integrates with the business

Follow these best practices to make sure your requirements are solid:

1. Self-assess your current approach to IT risk management.
2. Identify organizational obstacles and set attainable risk management goals.
3. Track the effectiveness and success of the program using SMART risk management metrics.
4. Establish an IT risk council tasked with managing IT risk.
5. Set clear risk management accountabilities and responsibilities for IT and business stakeholders.

Key metrics for your IT risk governance framework

Info-Tech Insight

Metrics provide the foundation for determining the success of your IT risk management program and ensure ongoing funding to support appropriate risk responses.

IT risk management success factors

1.2.1 Identify obstacles and pain points

1-4 hours

Input: Integrated Risk Maturity Assessment

Output: Obstacles and pain points identified

Materials: IT Risk Management Success Factors

Participants: IT executive leadership, Business executive leadership

Anticipate potential challenges and “blind spots” by determining which success factors are missing from your current situation.

Instructions:

1. List the potential obstacles and missing success factors that you must overcome to effectively manage IT risk and build a risk management program.
2. Consider some opportunities that could be leveraged to increase the success of this program.
3. Use this list in Activity 1.2.3 to develop program goals.

Risk Management

Replace the example pain points and opportunities with real scenarios in your organization.

1.2.2 Determine the risk culture of your organization

Determine how your organization fits the criteria listed below. Descriptions and examples do not have to match your organization perfectly.

Be aware of the organization’s attitude towards risk

Risk culture is an organization’s attitude towards taking risks. This attitude manifests itself in two ways:

Info-Tech Insight

Organizations typically fall in the middle of these spectrums. While risk culture will vary depending on the industry and maturity of the organization, a culture with a balanced risk appetite that is extremely risk conscious is able to make creative, dynamic decisions with reasonable limits placed on risk-related decision making.

1.2.3 Develop goals for the IT risk management program

1-4 hours

Input: Integrated Risk Maturity Assessment, Risk Culture, Pain Points and Opportunities

Output: Goals for the IT risk management program

Materials: Risk Management Program Manual

Participants: IT executive leadership, Business executive leadership

Translate your maturity assessment and knowledge about organizational risk culture, potential obstacles, and success factors to develop goals for your IT risk management program.

Instructions:

1. In the Risk Management Program Manual, revise, replace, or add to the high-level goals provided in section 2.4.
2. Make sure that you have three to five high-level goals that reflect the current and targeted maturity of IT risk management processes.
3. Integrate potential obstacles, pain points, and insights from the organization’s risk culture.

Record the results in the Risk Management Program Manual.

1.2.4 Develop SMART project metrics

Create metrics for measuring the success of the IT risk management program.

1.2.4 Develop SMART project metrics (continued)

Attach metrics to your goals to gauge the success of the IT risk management program.

Replace the example metrics with accurate KPIs or metrics for your organization.

Create the IT risk committee (ITRC)

1.2.5 Create the IT risk council

1-4 hours

Input: List of IT personnel and business stakeholders

Output: Goals for the IT risk management program

Materials: Risk Management Program Manual

Participants: CIO, CRO (if applicable), Senior Directors, Head of Operations

Identify the essential individuals from both the IT department and the business to create a permanent committee that meets regularly and carries out IT risk management activities.

Instructions:

1. Review sections 3.1 (Mandate) and 3.2 (Agenda and Responsibilities) of the IT Risk Committee Charter, located in the Risk Management Program Manual. Make any necessary revisions.
2. In section 3.3, document how frequently the council is scheduled to meet.
3. In section 3.4, document members of the IT risk council.
4. Obtain sign-off for the IT risk council from the CIO or another member of the senior leadership team in section 3.5 of the manual.

Record the results in the Risk Management Program Manual.

1.2.6 Complete RACI chart

A RACI diagram is a useful visualization that identifies redundancies and ensures that every role, project, or task has an accountable party.

1.2.6 Complete RACI chart (continued)

Assign risk management accountabilities and responsibilities to key stakeholders:

Build an IT Risk Management Program

Phase 2

Identify and Assess IT Risk

This phase will walk you through the following activities:

• Add organization-specific risk scenarios
• Identify risk events
• Augment risk event list using COBIT 2019 processes
• Conduct a PESTLE analysis
• Determine the threshold for (un)acceptable risk
• Create a financial impact assessment scale
• Select a technique to measure reputational cost
• Create a likelihood scale
• Assess risk severity level
• Assess expected cost

This phase involves the following participants:

• IT risk council
• Relevant business stakeholders
• Representation from senior management team
• Business Risk Owners

Step 2.1

Identify IT Risks

Activities

• 2.1.1 Add organization-specific risk scenarios
• 2.1.2 Identify risk events
• 2.1.3 Augment risk event list using COBIT 19 processes
• 2.1.4 Conduct a PESTLE analysis

This step involves the following participants:

• IT executive leadership
• IT Risk Council
• Business executive leadership
• Business risk owners

Outcomes of this step

• Participation of key stakeholders
• Comprehensive list of IT risk events

Get to know what you don’t know

Info-Tech Insight

What you don’t know CAN hurt you. How do you identify IT-related threats and vulnerabilities that you are not already aware of? Now that you have created a strong risk governance framework that formalizes risk management within IT and connects it to the enterprise, follow the steps outlined in this section to reveal all of IT’s risks.

Engage key stakeholders

Ensure that all key risks are identified by engaging key business stakeholders.

Enable IT to target risk holistically

Take a top-down approach to risk identification to guide brainstorming

Info-Tech’s risk categories are consistent with a risk identification method called Risk Prompting.

A risk prompt list is a list that categorizes risks into types or areas. The n10 risk categories encapsulate the services, activities, responsibilities, and functions of most IT departments. Use these categories and the example risk scenarios provided as prompts to guide brainstorming and organize risks.

2.1.1 Add organization-specific risk scenarios

Review Info-Tech’s ten IT risk categories and add risk scenarios to the examples provided.

2.1.2 Identify risk events

Input: IT risk categories

Output: Risk events identified and categorized

Materials: Risk Register Tool

Participants: IT risk council, Relevant business stakeholders, Representation from senior management team, Business risk owners, CRO (if applicable)

Use Info-Tech’s IT risk categories and scenarios to brainstorm a comprehensive list of IT-related threats and vulnerabilities impacting your organization.

Instructions:

1. Document risk events in the Risk Register Tool.
2. List risk scenarios (organized by risk category) in the Risk Events/Threats column.
3. Disseminate the list to key stakeholders who were unable to participate and solicit their feedback.
   • Consult the RACI chart located in section 4.1 of the Risk Management Program Manual.
4. Attack one scenario at a time, exhausting all realistic risk events for that grouping before moving onto the next scenario. Each scenario should take approximately 45-60 minutes.

Tip: If disagreement arises regarding whether a specific risk event is relevant to the organization or not and it cannot be resolved quickly, include it in the list. The applicability of these risks will become apparent during the assessment process.

Record the results in the Risk Register Tool.

2.1.3 Augment the risk event list using COBIT 2019 processes (Optional)

Other industry-leading frameworks provide alternative ways of conceptualizing the functions and responsibilities of IT and may help you uncover additional risk events.

Instructions:

1. Review COBIT 2019’s 40 IT processes and identify additional risk events.
2. Match risk events to the corresponding risk category and scenario and add them to the Risk Register Tool.

2.1.4 Finalize your risk register by conducting a PESTLE analysis (Optional)

Explore alternative identification techniques to incorporate external factors and avoid “groupthink.”

Consider the External Environment – PESTLE Analysis Despite efforts to encourage equal participation in the risk identification process, key risks may not have been shared in previous exercises. Conduct a PESTLE analysis as a final safety net to ensure that all key risk events have been identified.		Avoid “Groupthink” – Nominal Group Technique The Nominal Group Technique uses the silent generation of ideas and an enforced “safe” period of time where ideas are shared but not discussed to encourage judgement-free idea generation. Ideas are generated silently and independently. Ideas are then shared and documented; however, discussion is delayed until all of the group’s ideas have been recorded. Idea generation can occur before the meeting and be kept anonymous. Note: Employing either of these techniques will lengthen an already time-consuming process. Only consider these techniques if you have concerns regarding the homogeneity of the ideas being generated or if select individuals are dominating the exercise.
List the following factors influencing the risk event: Political factors Economic factors Social factors Technological factors Legal factors Environmental factors

Step 2.2

Assess and Prioritize IT Risks

Activities

• 2.2.1 Determine the threshold for (un)acceptable risk
• 2.2.2 Create a financial impact assessment scale
• 2.2.3 Select a technique to measure reputational cost
• 2.2.4 Create a likelihood scale
• 2.2.5 Risk severity level assessment
• 2.2.6 Expected cost assessment

This step involves the following participants:

• IT risk council
• Relevant business stakeholders
• Representation from senior management team
• Business risk owners

Outcomes of this step

• Business-approved thresholds for unacceptable risk
• Completed Risk Register Tool with risks prioritized according to severity
• Expected cost calculations for high-priority risks

Identify and Assess IT Risk

Reveal the organization’s greatest IT threats and vulnerabilities

Info-Tech Insight

Risk is money. It’s impossible to make intelligent decisions about risks without knowing what their financial impact will be.

Review risk assessment fundamentals

Risk assessment provides you with the raw materials to conduct an informed cost-benefit analysis and make robust risk response decisions.

In this section, you will be prioritizing your IT risks according to their risk severity, which is a reflection of their expected cost.

Calculating risk severity

Maintain the engagement of key stakeholders in the risk assessment process

Info-Tech Insight

If business executives still won’t provide the necessary information to update your initial risk assessments, IT should approach business unit leaders and lower-level management. Lean on strong relationships forged over time between IT and business managers or supervisors to obtain any additional information.

Info-Tech recommends a two-level approach to risk assessment

Review the two levels of risk assessment offered in this blueprint.

Risk severity level assessment (mandatory)

Assess all of your identified risk events with a risk severity-level assessment.

• By creating a likelihood and impact assessment scale divided into three to nine “levels” (sometimes referred to as “buckets”), you can evaluate every risk event quickly while being confident that risks are being assessed accurately.
• In the following activities, you will create likelihood and impact scales that align with your organizational risk appetite and tolerance.
• Severity-level assessment is a “first pass” of your risk list, revealing your organization’s most severe IT risks, which can be assessed in greater detail by incorporating expected cost into your evaluation.

Info-Tech recommends a two-level approach to risk assessment (continued)

Expected cost assessment (optional)

Conduct expected cost assessments for IT’s greatest risks.

For risk events warranting further analysis, translate risk severity levels into hard expected-cost numbers.

Implement and leverage a centralized risk register

The purpose of the risk register is to act as the repository for all the risks that have been identified within your environment.

Use this tool to:

1. Collect and maintain a repository for all IT risk events impacting the organization and relevant information for each risk.
   • Capture all relevant IT risk information in one location.
   • Organize risk identification and assessment information for transparent risk management, stakeholder review, and/or internal audit.
2. Calculate risk severity scores to prioritize risk events and determine which risks require a risk response.
   • Separate acceptable and unacceptable risks (as determined by the business).
   • Rank risks based on severity levels.
3. Assess risk responses and calculate residual risk.
   • Evaluate the effect that proposed risk response actions will have on top risk events and quantify residual risk magnitude.
   • This step will be completed in section 3.1

2.2.1 Determine the threshold for (un)acceptable risk

Input: Risk events, Risk appetite

Output: Threshold for risk identified

Materials: Risk Register Tool, Risk Management Program Manual

Participants: IT risk council, Relevant business stakeholders, Representation from senior management team, Business risk owner

Instructions:

There are times when the business needs to know about IT risks with high expected costs.

1. Create an expected cost threshold that defines what constitutes an acceptable and unacceptable risk for the organization. This figure should be a concrete dollar value. In the next exercises, you will build risk impact and likelihood scales with this value in mind, ensuring that “high” or “extreme” risks are immediately communicated to senior leadership.
2. Do not consider IT budget restrictions when developing this number. The acceptable risk threshold should reflect the business’ tolerance/appetite for risk.

This threshold is typically based on the organization’s ability to absorb financial losses, and its tolerance/appetite towards risk.

If your organization has ERM, adopt the existing acceptability threshold.

Record this threshold in section 5.3 of the Risk Management Program Manual

2.2.2 Create a financial impact assessment scale

1-4 hours

Input: Risk events, Risk threshold

Output: Financial impact scale created

Materials: Risk Register Tool, Risk Management Program Manual

Participants: IT risk council, Relevant business stakeholders, Representation from senior management team, Business risk owner

Instructions:

1. Create a scale to assess the financial impact of risk events.
   • Typically, risk impacts are assessed on a scale of 1-5; however, some organizations may prefer to assess risks using 3, 4, 7, or 9-point scales.
2. Ensure that the unacceptable risk threshold is reflected in the scale.
   • In the example provided, the unacceptable risk threshold ($100,000) is represented as “High” on the impact scale.
3. Attach labels to each point on the scale. Effective labels will easily distinguish between risks on either side of the unacceptable risk threshold.

Record the risk impact scale in section 5.3 of the Risk Management Program Manual

Convert project overruns and service outages into costs

Use the tables below to quickly convert impacts typically measured in units of time to financial cost. Replace the values in the table with those that reflect your own costs.

• While project overruns and service outages may have intangible impacts beyond the unexpected costs stemming from paying employees and lost revenue (such as adding complexity to project management and undermining the business’ confidence in IT), these measurements will provide adequate impact estimations for risk assessment.
• Remember, complex risk events can be analyzed further with an expected cost assessment.

Project Overruns
Project	Time (days) 20 days	Number of employees 8	Average cost per employee (per day) $300	Estimated cost $48,000
Service Outages
Service	Time (hours) 4 hours	Lost revenue (per hour) $10,000	Estimated cost $40,000	Impact scale Low

2.2.3 Select a technique to measure reputational cost (1 of 3)

Realized risk events may have profound reputational costs that do not immediately impact your bottom line.

2.2.3 Select a technique to measure reputational cost (2 of 3)

2.2.3 Select a technique to measure reputational cost (3 of 3)

If you feel that the other techniques have not reflected reputational impacts in the overall severity level of the risk, create a parallel scale that roughly matches your financial impact scale.

Technique #3 – Create a parallel scale for reputational impact: Visibility is a useful metric for measuring reputational impact. Visibility measures how widely knowledge of the risk event has spread and how negatively the organization is perceived. Visibility has two main dimensions: Internal vs. External Low Amplification vs. High Amplification Internal/External: The further outside of the organization that the risk event is visible, the higher the reputational impact. Low/High Amplification: The greater the ability of the actor to communicate and amplify the occurrence of a risk event, the higher the reputational impact. After establishing a scale for reputational impact, test whether it reflects the severity of the financial impact levels in the financial impact scale. For example, if the media learns about a recent data breach, does that feel like a $100,000 loss?

Example:

2.2.4 Create a likelihood scale

1-3 hours

Instructions: Create a scale to assess the likelihood that a risk event will occur over a given period of time. Info-Tech recommends assessing the likelihood that the risk event will occur over a period of one year (the IT risk council should be reassessing the risk event no less than once per year). Ensure that the likelihood scale contains the same number of levels as the financial impact scale (3, 4, 5, 7, or 9). The example provided is likely to satisfy most IT departments; however, you may customize the distribution of likelihood values to reflect the organization’s aversion towards uncertainty. For example, an extremely risk-averse organization may consider any risk event with a likelihood greater than 20% to have a “High” likelihood of occurrence. Attach the same labels used for the financial impact scale (Low, Moderate, High, etc.) Record the risk impact scale in section 5.3 of the Risk Management Program Manual

Info-Tech Insight

Note: Info-Tech endorses the use of likelihood values (1-99%) rather than frequency (3 times per year) as a measurement.
For an explanation of why likelihood values lead to more precise and robust risk assessment, see the Appendix.

2.2.5 Risk severity level assessment

6-10 hours

Input: Risk events identified

Output: Assessed the likelihood of occurrence and impact for all identified risk events

Materials: Risk Register Tool

Participants: IT risk council, Relevant business stakeholders, Representation from senior management team, Business risk owner

Instructions:

1. Document the “Risk Category” and “Existing Controls.” in the Risk Register Tool.
   • (See the slide following this activity for tips on identifying existing controls.)
2. Assign each risk event a likelihood and impact level.
   • Remember, you are assessing the impact that a risk event will have on the organization as a whole, not just on IT.
3. When assigning a financial impact level to a risk event, factor in the likely number of instances that the event will occur within the time frame for which you are assessing (usually one year).
   • For risk events like third-party service outages that typically occur a few times each year, assign them an impact level that reflects the likelihood of financial impact the risk event will have over the entire year.
   • E.g. If your organization is likely to experience two major service outages next year and each outage costs the organization approximately $15,000, the total financial impact is $30,000.

Record results in the Risk Register Tool

2.2.5 Risk severity level assessment (continued)

Identify current risk controls

Consider how IT is already addressing key risks.

Types of current risk control

Consider both tactical and strategic controls already in place when filling out risk event information in the Risk Register Tool.

Info-Tech Insight

Identifying existing risk controls (past risk responses) provides a clear picture of the measures already in place to avoid, mitigate, or transfer key risks. This reveals opportunities to improve existing risk controls, or where new strategies are needed, to reduce risk severity levels below business thresholds.

Assign a risk owner for each risk event

Designate a member of the IT risk council to be responsible for each risk event.

Use Info-Tech’s Risk Costing Tool to calculate the expected cost of IT’s high-priority risks (optional)

Use this tool to:

1. Conduct a deeper analysis of severe risks.
   • Determine specific likelihood and financial impact values to communicate the severity of the risk in the Expected Cost tab.
   • Identify the maximum financial impact that the risk event may inflict.
2. Assess the effectiveness of multiple risk responses for each risk event.
   • Determine how proposed risk events will change the likelihood of occurrence and financial impact of the risk event.
3. Incorporate risk proximity into your cost-benefit analysis of risk responses.
   • Illustrate how spending decisions will impact the expected cost of the risk event over time.

2.2.6 Expected cost assessment (optional)

Assign likelihood and financial impact values to high-priority risks.

2.2.6 Expected cost assessment (continued)

Assign likelihood and financial impact values to high-priority risks.

Evaluate likelihood and impact

Refine your risk assessment process by developing more accurate measurements of likelihood and impact.

Build an IT Risk Management Program

Phase 3

Monitor, Respond, and Report on IT Risk

This phase will walk you through the following activities:

• Develop key risk indicators (KRIs) and escalation protocols
• Establish the reporting schedule
• Identify and assess risk responses
• Analyze risk response cost-benefit
• Create multi-year cost projections
• Obtain executive approval for risk action plans
• Socialize the Risk Report
• Transfer ownership of risk responses to project managers
• Finalize the Risk Management Program Manual

This phase involves the following participants:

• IT risk council
• Relevant business stakeholders
• Representation from senior management team
• Risk business owner

Step 3.1

Monitor IT Risks and Develop Risk Responses

Activities

• 3.1.1 Develop key risk indicators (KRIs) and escalation protocols
• 3.1.2 Establish the reporting schedule
• 3.1.3 Identify and assess risk responses
• 3.1.4 Risk response cost-benefit analysis
• 3.1.5 Create multi-year cost projections

This step involves the following participants:

• IT risk council
• Relevant business stakeholders
• Representation from senior management team
• Business risk owner

Outcomes of this step

• Completed risk event action plans
• Risk responses identified and assessed for top risks
• Risk response selected for top risks

Monitor, Respond, and Report on IT Risk