Select a Security Outsourcing Partner

Outsource the right functions to secure your business.

Analyst Perspective

Understanding your security needs and remaining accountable is the key to selecting the right partner.

The need for specialized security services is fast becoming a necessity to most organizations. However, resource challenges will always mean that organizations will still have to take practical measures to ensure that the time, quality, and service that they require from outsourcing partners have been carefully crafted and packaged to elicit the right services that cover all their needs and requirements.

Organizations must ensure that security partners are aligned not only with their needs and requirements, but also with the corporate culture. Rather than introducing hindrances to daily operations, security partners must support business goals and protect the organization’s interests at all times.

And as always, outsource only your responsibilities and do not outsource your accountability, as that will cost you in the long run.

Danny Hammond
Research Analyst
Security, Risk, Privacy & Compliance Practice
Info-Tech Research Group

Executive Summary

Info-Tech Insight

Mitigate security risks by developing an end-to-end process that ensures you are outsourcing your responsibilities and not your accountability.

Your Challenge

This research is designed to help organizations select an effective security outsourcing partner.

• A security outsourcing partner is a third-party service provider that offers security services on a contractual basis depending on client needs and requirements.
• An effective outsourcing partner can help an organization improve its security posture by providing access to more specialized security experts, tools, and technologies.
• One of the main challenges with selecting a security outsourcing partner is finding a partner that is a good fit for the organization's unique security needs and requirements.
• Security outsourcing partners typically have access to sensitive information and systems, so proper controls and safeguards must be in place to protect all sensitive assets.
• Without careful evaluation and due diligence to ensure that the partner is a good fit for the organization's security needs and requirements, it can be challenging to select an outsourcing partner.

Outsourcing is effective, but only if done right

• 83% of decision makers with in-house cybersecurity teams are considering outsourcing to an MSP (Syntax, 2021).
• 77% of IT leaders said cyberattacks were more frequent (Syntax, 2021).
• 51% of businesses suffered a data breach caused by a third party (Ponemon, 2021).

Common Obstacles

The problem with selecting an outsourcing partner isn’t a lack of qualified partners, it’s the lack of clarity about an organization's specific security needs.

• Most organizations do not have a clear understanding of their current security posture, their security goals, and the specific security services they require. Without a clear understanding of their needs, organizations may struggle to identify a partner that can meet their requirements.
• Breakdowns and lack of communication can be a significant obstacle, especially when clear lines of communication with partners, including regular check-ins, reporting, and incident response protocols, have not been clearly established.
• Ensuring that security partner's systems and processes integrate seamlessly with existing systems can be a challenge for most organizations. This is in addition to making sure that security partners have the necessary access and permissions to perform their services effectively.
• Adhering to security policies is rarely a priority to users, as compliance often feels like an interference to daily workflow. For a lot of organizations, security policies are not having the desired effect.

Source: IBM, 2022 Cost of a Data Breach; N=537.

Info-Tech’s methodology for selecting a security outsourcing partner

Determine your responsibilities

Determine what responsibilities you can outsource to a service partner. Analyze which responsibilities you should outsource versus keep in-house? Do you require a service partner based on identified responsibilities?

Scope your requirements

Refine the list of role-based requirements, variables, and features you will require. Use a well-known list of critical security controls as a framework to determine these activities and send out RFPs to pick the best candidate for your organization.

Manage your outsourcing program

Adopt a program to manage your third-party service security outsourcing. Trust your managed security service providers (MSSP) but verify their results to ensure you get the service level you were promised.

Select a Security Outsourcing Partner

Blueprint benefits

Insight summary

Overarching insight: You can outsource your responsibilities but not your accountability.

Determine what to outsource: Assess your responsibilities to determine which ones you can outsource. It is vital that an understanding of how outsourcing will affect the organization, and what cost savings, if any, to expect from outsourcing is clear in order to generate a list of responsibilities that can/should be outsourced.

Select the right partner: Create a list of variables to evaluate the MSSPs and determine which features are important to you. Evaluate all potential MSSPs and determine which one is right for your organization

Manage your MSSP: Align the MSSP to your organization. Adopt a program to monitor the MSSP which includes a long-term strategy to manage the MSSP.

Identifying security needs and requirements = Effective outsourcing program: Understanding your own security needs and requirements is key. Ensure your RFP covers the entire scope of your requirements; work with your identified partner on updates and adaptation, where necessary; and always monitor alignment to business objectives.

Measure the value of this blueprint

After each Info-Tech experience, we ask our members to quantify the real-time savings, monetary impact, and project improvements our research helped them achieve.

Overall Impact: 8.9 /10

Overall Average Cost Saved: $22,950

Overall Average Days Saved: 9

Info-Tech offers various levels of support to best suit your needs

DIY Toolkit
"Our team has already made this critical project a priority, and we have the time and capability, but some guidance along the way would be helpful."

Guided Implementation
"Our team knows that we need to fix a process, but we need assistance to determine where to focus. Some check-ins along the way would help keep us on track."

Workshop
"We need to hit the ground running and get this project kicked off immediately. Our team has the ability to take this over once we get a framework and strategy in place."

Consulting
"Our team does not have the time or the knowledge to take this project on. We need assistance through the entirety of this project."

Diagnostics and consistent frameworks are used throughout all four options.

Reduce Shadow IT With a Service Request Catalog

Foster business partnerships with sourcing-as-a-service.

Analyst Perspective

Improve the request management process to reduce shadow IT.

In July 2022, Ivanti conducted a study on the state of the digital employee experience, surveying 10,000 office workers, IT professionals, and C-suite executives. Results of this study indicated that 49% of employees are frustrated by their tools, and 26% of employees were considering quitting their jobs due to unsuitable tech. 42% spent their own money to gain technology to improve their productivity. Despite this, only 21% of IT leaders prioritized user experience when selecting new tools.

Any organization’s workers are expected to be productive and contribute to operational improvements or customer experience. Yet those workers don’t always have the tools needed to do the job. One option is to give the business greater control, allowing them to choose and acquire the solutions that will make them more productive. Info-Tech's blueprint Embrace Business-Managed Applications takes you down this path.

However, if the business doesn’t want to manage applications, but just wants have access to better ones, IT is positioned to provide services for application and equipment sourcing that will improve the employee experience while ensuring applications and equipment are fully managed by the asset, service, and security teams.

Improving the request management and deployment practice can give the business what they need without forcing them to manage license agreements, renewals, and warranties.

Sandi Conrad
ITIL Managing Professional
Principal Research Director, IT Infrastructure & Operations,
Info-Tech Research Group

Your challenge

This research is designed to help organizations that are looking to improve request management processes and reduce shadow IT.

Shadow IT: The IT team is regularly surprised to discover new products within the organization, often when following up on help desk tickets or requests for renewals from business users or vendors.

Renewal management: The contracts and asset teams need to be aware of upcoming renewals and have adequate time to review renewals.

Over-purchasing and over-spending: Contracts may be renewed without a clear picture of utilization, potentially renewing unused applications. Applications or equipment may be purchased at retail price where corporate, government, or educational discounts exist.

Info-Tech Insight

To increase the visibility of the IT environment, IT needs to transform the request management process to create a service that makes it easier for the business to access the tools they need rather than seeking them outside of the organization.

— Source: Zylo, SaaS Trends for IT Leaders, 2022

Common obstacles

Too many layers of approvals and a lack of IT workers makes it difficult to rethink service request fulfillment.

Delays: The business may not be getting the applications they need from IT to do their jobs or must wait too long to get the applications approved.

Denials: Without IT’s support, the business is finding alternative options, including SaaS applications, as they can be bought and used without IT’s input or knowledge.

Threats: Applications that have not been vetted by security or installed without their knowledge may present additional threats to the organization.

Access: Self-serve isn’t mature enough to support an applications catalog.

8: average number of applications entering the organization every 30 days

— Source: Zylo, SaaS Trends for Procurement, 2022

Info-Tech’s approach

Improve the request management process to create sourcing-as-a-service for the business.

• Improve customer service
• Reduce shadow IT
• Gain control in a way that keeps the business happy

Blueprint deliverables

Each step of this blueprint is accompanied by supporting deliverables to help you accomplish your goals:

Create a request management process and service catalog to improve delivery of technology to the business

Reduce Risk With Rock-Solid Service-Level Agreements

Hold Service Providers more accountable to their contractual obligations with meaningful SLA components & remedies

EXECUTIVE BRIEF

Analyst Perspective

Reduce Risk With Rock-Solid Service-Level Agreements

Every year organizations outsource more and more IT infrastructure to the cloud, and IT operations to managed service providers. This increase in outsourcing presents an increase in risk to the CIO to save on IT spend through outsourcing while maintaining required and expected service levels to internal customers and the organization. Ensuring that the service provider constantly meets their obligations so that the CIO can meet their obligation to the organization can be a constant challenge. This brings forth the importance of the Service Level Agreement.

Research clearly indicates that there is a general lack of knowledge when comes to understanding the key elements of a Service Level Agreement (SLA). Even less understanding of the importance of the components of Service Levels and the Service Level Objectives (SLO) that service provider needs to meet so that the outsourced service consistently meets requirements of the organization. Most service providers are very good at providing the contracted service and they all are very good at presenting SLOs that are easy to meet with very few or no ramifications if they don’t meet their objectives. IT leaders need to be more resolute in only accepting SLOs that are meaningful to their requirements and have meaningful, proactive reporting and associated remedies to hold service providers accountable to their obligations.

Ted Walker

Principal Research Director, Vendor Practice

Info-Tech Research Group

Executive Brief

Vendors provide service level commitments to customers in contracts to show a level of trust, performance, availability, security, and responsiveness in an effort create a sense of confidence that their service or platform will meet your organization’s requirements and expectations. Sifting through these promises can be challenging for many IT Leaders. Customers struggle to understand and evaluate what’s in the SLA – are they meaningful and protect your investment? Not understanding the details of SLAs applicable to various types of Service (SaaS, MSP, Service Desk, DR, ISP) can lead to financial and compliance risk for the organization as well as poor customer satisfaction.

This project will provide IT leadership the knowledge & tools that will allow them to:

• Understand what SLAs are and why they need them.
• Develop standard SLAs that meet the organization’s requirements.
• Negotiate meaningful remedies aligned to Service Levels metrics or KPIs.
• Create SLA monitoring & reporting and remedies requirements to hold the provider accountable.

This research:

1. Is designed for:

• The CIO or CFO who needs to better understand their provider’s SLAs.
• The CIO or BU that could benefit from improved service levels.
• Vendor management who needs to standardize SLAs for the organization IT leadership that needs consistent service levels to the business
• The contract manager who needs a better understanding of contact SLAs

• Understand what a Service Level Agreement is and what it’s for
• Learn what the components are of an SLA and why you need them
• Create a checklist of required SLA elements for your organization
• Develop standard SLA template requirements for various service types
• Learn the importance of SLA management to hold providers accountable

Reduce Risk With Rock-Solid Service-Level Agreements (SLAs)

Hold service providers more accountable to their contractual obligations with meaningful SLA components and remedies

The Problem

IT Leadership doesn't know how to evaluate an SLA.

Misunderstanding of obligations given the type of service provided (SAAS, IAAS, DR/BCP, Service Desk)

Expectations not being met, leading to poor service from the provider.

No way to hold provider accountable.

Why it matters

SLAS are designed to ensure that outsourced IT services meet the requirements and expectations of the organization. Well-written SLAs with all the required elements, metrics, and remedies will allow IT departments to provide the service levels to their customer and avoid financial and contractual risk to the organization.

The Solution

1. Understand the key service elements within an SLA

• Develop a solid understanding of the key elements within an SLA and why they're important.

• Prioritize contractual services and establish concise SLA checklists and performance metrics.

Service types

• Availability/Uptime
• Response Times
• Resolution Time
• Accuracy
• First-Call Resolution

Agreement Types

• SaaS/IaaS
• Service Desk
• MSP
• Co-Location
• DR/BCP
• Security Ops

Performance Metrics

• Reporting
• Remedies & Credits
• Monitoring
• Exclusion

Example SaaS Provider

• Response Times ✓
• Availability/Uptime ✓
• Resolution Time ✓
• Update Times ✓
• Coverage Time ✓
• Monitoring ✓
• Reporting ✓
• Remedies/Credits ✓

SLA Management Framework

1. SLO Monitoring

• SLOs must be monitored by the provider, otherwise they can't be measured.

• This is the key element for the provider to validate their performance.

• Capturing SLO metric attainment provides performance trending for each provider.

• Tracking details provide input into overall vendor performance ratings.

Executive Summary

Your Challenge

To understand which SLAs are required for your organization and how they can differ depending on the service type. In addition, these other challenges can also cloud your knowledge of SLAs

• No standardized SLA documents, Service levels, or metrics
• Dealing with lost productivity & revenue due to persistent downtime
• Understanding SLA components and what service levels are requires for a particular service
• How to manage the SLA and hold the vendor accountable

Common Obstacles

There are several unknowns that SLA can present to different departments within the organization:

• Little knowledge of what service levels are required
• Not knowing SLO standards for a service type
• Lack of resources to manage vendor obligations
• Negotiating required metrics/KPIs with the provider
• Low understanding of the risk that poor SLAs can present to the organization

Info-Tech's Approach

Info-Tech has a three-step approach to effective SLAs

• Understand the elements of an SLA
• Create Requirements for your organization
• Manage the SLA obligations

There are some basic components that every SLA should have – most don’t have half of what is required

Info-Tech Insight

SLAs need to have clear, easy to measure objectives to meet your expectations and service level requirements, including meaningful reporting and remedies to hold the provider accountable to their obligations.

Your challenge

This research is designed to help organizations gain a better understanding of what an SLA is, understand the importance of SLAs in IT contracts, and ensure organizations are provided with rock-solid SLAs that meet their requirements and not just what the vendor wants to provide.

• Vendors can make SLAs weak and difficult to understand; sometimes the metrics are meaningless. Not fully understanding what makes up a good SLA can bring unknown risks to the organization.
• Managing vendor SLA obligations effectively is important. Are adequate resources available? Does the vendor provide manual vs. automated processes and which do you need? Is the process proactive from the vendor or reactive from the customer?

SLAs come in many variations and for many service types. Understanding what needs to be in them is one of the keys to reducing risk to your organization.

“One of the biggest mistakes an IT leader can make is ignoring the ‘A’ in SLA,” adds Wendy M. Pfeiffer, CIO at Nutanix. “

An agreement isn’t a one-sided declaration of IT capabilities, nor is it a one-sided demand of business requirements,” she says. “An agreement involves creating a shared understanding of desired service delivery and quality, calculating costs related to expectations, and then agreeing to outcomes in exchange for investment.” (15 SLA mistakes IT leaders still make | CIO)

Common obstacles

There are typically a lot of unknowns when it comes to SLAs and how to manage them.

Most organizations don’t have a full understanding of what SLAs they require and how to ensure they are met by the vendor. Other obstacles that SLAs can present are:

• Inadequate resources to create and manage SLAs
• Poor awareness of standard or required SLA metrics/KPIs
• Lack of knowledge about each provider’s commitment as well as your obligations
• Low vendor willingness to provide or negotiate meaningful SLAs and credits
• The know-how or resources to effectively monitor and manage the SLA’s performance

SLAs need to address your requirements

55% of businesses do not find all of their service desk metrics useful or valuable (Freshservice.com)

27% of businesses spend four to seven hours a month collating metric reports (Freshservice.com)

Executive Summary

Info-Tech’s Approach

• Understand the elements of an SLA
  • Availability
  • Monitoring
  • Response Times
  • SLO Calculation
  • Resolution Time
  • Reporting
  • Milestones
  • Exclusions
  • Accuracy
  • Remedies & Credits
• Create standard SLA requirements and criteria
  • SLA Element Checklist
  • Corporate Requirements and Standards
  • SLA Templates and Policy
• Effectively Manage the SLA Obligations
  • SLA Management Framework
    • SLO Monitoring
    • Concise Reporting
    • Attainment Tracking
    • Score Carding
    • Remedy Reconciliation

Info-Tech’s three phase approach

Reduce Risk With Rock-Solid Service-Level Agreements

Phase 1

Understand SLA Elements

Phase Content:

• 1.1 What are SLAs, types of SLAs, and why are they needed?
• 1.2 Elements of an SLA
• 1.3 Obligation management monitoring, Reporting requirements
• 1.4 Exclusions
• 1.5 SLAs vs. SLOs vs. SLIs

Outcome:

This phase will present you with an understanding of the elements of an SLA: What they are, why you need them, and how to validate them.

Phase 2

Create Requirements

Phase Content:

• 2.1 Create a list of your SLA criteria
• 2.2 Develop SLA policy & templates
• 2.3 Create a negotiation strategy
• 2.4 SLA Overachieving discussion

Outcome:

This phase will leverage knowledge gained in Phase 1 and guide you through the creation of SLA requirements, criteria, and templates to ensure that providers meet the service level obligations needed for various service types to meet your organization’s service expectations.

Phase 3

Manage Obligations

Phase Content:

• 3.1 SLA Monitoring, Tracking
• 3.2 Reporting
• 3.3 Vendor SLA Reviews & Optimizing
• 3.4 Performance management

Outcome:

This phase will provide you with an SLA management framework and the best practices that will allow you to effectively manage service providers and their SLA obligations.

Insight summary

Overarching insight

SLAs need to have clear, easy-to-measure objectives to meet your expectations and service level requirements, including meaningful reporting and remedies to hold the provider accountable to their obligations.

Phase 1 insight

Not understanding the required elements of an SLA and not having meaningful remedies to hold service providers accountable to their obligations can present several risk factors to your organization.

Phase 2 insight

Creating standard SLA criteria for your organization’s service providers will ensure consistent service levels for your business units and customers.

Phase 3 insight

SLAs can have appropriate SLOs and remedies but without effective management processes they could become meaningless.

Tactical insight

Be sure to set SLAs that are easily measurable from regularly accessible data and that are straight forward to interpret.

Tactical insight

Beware of low, easy to attain service levels and metrics/KPIs. Service levels need to meet your expectations and needs not the vendor’s.

Blueprint deliverables

Each step of this blueprint is accompanied by supporting deliverables to help you accomplish your goals:

SLA Tracker & Trending Tool

Track the provider’s SLO attainment and see how their performance is trending over time

SLA Evaluation Tool

Evaluate SLA service levels, metrics, credit values, reporting, and other elements

SLA Template & Metrics Reference Guide

Reference guide for typical SLA metrics with a generic SLA Template

Service-Level Agreement Checklist

Complete SLA component checklist for core SLA and contractual elements.

Key deliverable:

Service-Level Agreement Evaluation Tool

Evaluate each component of the SLA , including service levels, metrics, credit values, reporting, and processes to meet your requirements

Blueprint objectives

Understand the components of an SLA and effectively manage their obligations

• To provide an understanding of different types of SLAs, their required elements, and what they mean to your organization. How to identify meaningful service levels based on service types. We will break down the elements of the SLA such as service types and define service levels such as response times, availability, accuracy, and associated metrics or KPIs to ensure they are concise and easy to measure.
• To show how important it is that all metrics have remedies to hold the service provider accountable to their SLA obligations.

Once you have this knowledge you will be able to create and negotiate SLA requirements to meet your organization’s needs and then manage them effectively throughout the term of the agreement.

InfoTech Insight:

Right-size your requirements and create your SLO criteria based on risk mitigation and create measurements that motivate the desired behavior from the SLA.

Blueprint benefits

IT Benefits

• An understanding of standard SLA service levels and metrics
• Reduced financial risk through clear and concise easy-to-measure metrics and KPIs
• Improved SLA commitments from the service provider
• Meaningful reporting and remedies to hold the provider accountable
• Service levels and metrics that meet your requirements to support your customers

Business Benefits

• Better understanding of an SLA framework and required SLA elements
• Improved vendor performance
• Standardized service levels and metrics aligned to your organization’s requirements
• Reduced time in reviewing and comprehending vendor SLAs
• Consistent performance from your service providers

Measure the value of this blueprint

1. Dollars Saved

• Improved performance from your service provider
• Reduced financial risk through meaningful service levels & remedies
• Dollars gained through:
  • Reconciled credits from obligation tracking and management
  • Savings due to automated processes

• Reduced time in creating effective SLAs through requirement templates
• Time spent tracking and managing SLA obligations
• Reduced negotiation time
• Time spent tracking and reconciling credits

Info-Tech offers various levels of support to best suit your needs

DIY Toolkit

"Our team has already made this critical project a priority, and we have the time and capability, but some guidance along the way would be helpful."

Guided Implementation

"Our team knows that we need to fix a process, but we need assistance to determine where to focus. Some check-ins along the way wound help keep us on track."

Workshop

"We need to hit the ground running and get this project kicked off immediately. Our team has the ability to take this over once we get a framework and strategy in place."

Consulting

"Our team does not have the time or the knowledge to take this project on. We need assistance through the entirety of this project."

Diagnostics and consistent frameworks are used throughout all four options.

Guided Implementation

What does a typical GI on this topic look like?

A Guided Implementation (GI) is a series of calls with an Info-Tech analyst to help implement our best practices in your organization.

A typical GI is between three to six calls over the course of two to three months.

Phase 1 - Understand

• Call #1: Scope requirements, objectives, and your specific SLA challenges

Phase 2 - Create Requirements

• Call #2: Review key SLA and how to identify them
• Call #3: Deep dive into SLA elements and why you need them
• Call #4: Review your service types and SLA criteria
• Call #5: Create internal SLA requirements and templates

Phase 3 - Management

• Call #6: Review SLA Management Framework
• Call #7: Review and create SLA Reporting and Tracking

Workshop Overview

Contact your account representative for more information.

workshops@infotech.com 1-888-670-8889

Reduce Risk With Rock-Solid Service-Level Agreements

Phase 1

Phase 1

Understand SLA Elements

Phase Steps

• 1.1 What are SLAs, the types of SLAs, and why are they needed?
• 1.2 Elements of an SLA
• 1.3 Obligation management monitoring, Reporting requirements
• 1.4 Exclusions and exceptions
• 1.5 SLAs vs. SLOs vs. SLIs

Create Requirements

Manage Obligations

1.1 What are SLAs, the types of SLAs, and why are they needed?

SLA Overview

What is a Service Level Agreement?

An SLA is an overarching contractual agreement between a service provider and a customer (can be external or internal) that describes the services that will be delivered by the provider. It describes the service levels and associated performance metrics and expectations, how the provider will show it has attained the SLAs, and defines any remedies or credits that would apply if the provider fails to meet its commitments. Some SLAs also include a change or revision process.

SLAs come in a few forms. Some are unique, separate, standalone documents that define the service types and levels in more detail and is customized to your needs. Some are separate documents that apply to a service and are web posted or linked to an MSA or SSA. The most common is to have them embedded in, or as an appendix to an MSA or SSA. When negotiating an MSA it’s generally more effective to negotiate better service levels and metrics at the same time.

Objectives of an SLA

To be effective, SLAs need to have clearly described objectives that define the service type(s) that the service provider will perform, along with commitment to associated measurable metrics or KPIs that are sufficient to meet your expectations. The goal of these service levels and metrics is to ensure that the service provider is committed to providing the service that you require, and to allow you to maintain service levels to your customers whether internal or external.

1.1 What are SLAs, the types of SLAs, and why are they needed?

Key Elements of an SLA

Principle service elements of an SLA

There are several more common service-related elements of an SLA. These generally include:

• The Agreement – the document that defines service levels and commitments.
• The service types – the type of service being provided by the vendor. These can include SaaS, MSP, Service Desk, Telecom/network, PaaS, Co-Lo, BCP, etc.
• The service levels – these are the measurable performance objectives of the SLA. They include availability (uptime), response times, restore times, priority level, accuracy level, resolution times, event prevention, completion time, etc.
• Metrics/KPIs – These are the targets or commitments associated to the service level that the service provider is obligated to meet.
• Other elements – Reporting requirements, monitoring, remedies/credit values and process.

Contractual Construct Elements

These are construct components of an SLA that outline their roles and responsibilities, T&Cs, escalation process, etc.

In addition, there are several contractual-type elements including, but not limited to:

• A statement regarding the purpose of the SLA.
• A list of services being supplied (service types).
• An in-depth description of how services will be provided and when.
• Vendor and customer requirements.
• Vendor and customer obligations.
• Acknowledgment/acceptance of the SLA.
• They also list each party’s responsibilities and how issues will be escalated and resolved.

Common types of SLAs explained

Service-level SLA

• This service-level agreement construct is the Service-based SLA. This SLA covers an identified service for all customers in general (for example, if an IT service provider offers customer response times for a service to several customers). In a service-based agreement, the response times would be the same and apply to all customers using the service. Any customer using the service would be provided the same SLA – in this case the same defined response time.

Customer-based SLA

• A customer-based SLA is a unique agreement with one customer. The entire agreement is defined for one or all service levels provided to a particular customer (for example, you may use several services from one telecom vendor). The SLAs for these services would be covered in one contract between you and the vendor, creating a unique customer-based vendor agreement. Another scenario could be where a vendor offers general SLAs for its services but you negotiate a specific SLA for a particular service that is unique or exclusive to you. This would be a customer-based SLA as well.

Multi-level SLA

• This service-level agreement construct is the multi-level SLA. In a multi-level SLA, components are defined to the organizational levels of the customer with cascading coverage to sublevels of the organization. The SLA typically entails all services and is designed to the cover each sub-level or department within the organization. Sometimes the multi-level SLA is known as a master organization SLA as it cascades to several levels of the organization.

InfoTech Insight: Beware of low, easy to attain Service levels and metrics/KPIs. Service levels need to meet your requirements, expectations, and needs not the vendor’s.

1.2 Elements of SLA-objectives, service types, and service levels

Objectives of Service Levels

The objective of the service levels and service credits are to:

• Ensure that the services are of a consistently high quality and meet the requirements of the customer
• Provide a mechanism whereby the customer can attain meaningful recognition of the vendors failure to deliver the level of service for which it was contracted to deliver
• Incentivize the vendor or service provider to comply with and to expeditiously provide a remedy for any failure to attain the service levels committed to in the SLA
• To ensure that the service provider fulfills the defined objectives of the outsourced service

Service types

There are several service types that can be part of an SLA. Service types are the different nature of services associated with the SLA that the provider is performing and being measured against. These can include:

Service Desk, SaaS, PaaS, IaaS, ISP/Telecom/Network MSP, DR & BCP, Co-location security ops, SOW.

Each service type should have standard service level targets or obligations that can vary depending on your requirements and reliance on the service being provided.

Service levels

Service levels are measurable targets, metrics, or KPIs that the service provider has committed to for the particular service type. Service levels are the key element of SLAs – they are the performance expectations set between you and the provider. The service performance of the provider is measured against the service level commitments. The ability of the provider to consistently meet these metrics will allow your organization to fully benefit from the objectives of the service and associated SLAs. Most service levels are time related but not all are.

Common service levels are:

Response times, resolution times per percent, restore/recovery times, accuracy, availability/uptime, completion/milestones, updating/communication, latency.

Each service level has standard or minimum metrics for the provider. The metrics, or KPIs, should be relatively easy to measure and report against on a regular basis. Service levels are generally negotiable to meet your requirements.

1.2.1 Activity SLA Checklist Tool

1-2 hours

Input

• SLA content, Service elements
• Contract terms & exclusions
• Service metrices/KPIs

Output

• A concise list of SLA components
• A list of missing SLA elements
• Evaluation of the SLA

Materials

• Comprehensive checklist
• Service provider SLA
• Internal templates or policies

Participants

• Vendor or contract manager
• IT or business unit manager
• Legal
• Finance

Using this checklist will help you review a provider’s SLA to ensure it contains adequate service levels and remedies as well as contract-type elements.

Instructions:

Use the checklist to identify the principal service level elements as well as the contractual-type elements within the SLA.

Review the SLA and use the dropdowns in the checklist to verify if the element is in the SLA and whether it is within acceptable parameters as well the page or section for reference.

The checklist contains a list of service types that can be used for reference of what SLA elements you should expect to see in that service type SLA.

Download the SLA Checklist Tool

1.3 Monitoring, reporting requirements, remedies/credit process

Monitoring & Reporting

As mentioned, well-defined service levels are key to the success of the SLA. Validating that the metrics/KPIs are being met on a consistent basis requires regular monitoring and reporting. These elements of the SLA are how you hold the provider accountable to the SLA commitments and obligations. To achieve the service level, the service must be monitored to validate that timelines are met and accuracy is achieved.

• Data or details from monitoring must then be presented in a report and delivered to the customer in an agreed-upon format. These formats can be in a dashboard, portal, spreadsheet, or csv file, and they must have sufficient criteria to validate the service-level metric. Reports should be kept for future review and to create historical trending.
• Monitoring and reporting should be the responsibility of the service provider. This is the only way that they can validate to the customer that a service level has been achieved.
• Reporting criteria and delivery timelines should be defined in the SLA and can even have a service level associated with it, such as a scheduled report delivery on the fifth day of the following month.
• Reports need to be checked and balanced. When defining report criteria, be sure to define data source(s) that can be easily validated by both parties.
• Report criteria should include compliance requirements, target metric/KPIs, and whether they were attained.
• The report should identify any attainment shortfall or missed KPIs.

Too many SLAs do not have these elements as often the provider tries to put the onus on the customer to monitor their performance of the service levels. .

1.3.1 Monitoring, reporting requirements, remedies/credit process

Remedies and Credits

Service-level reports validate the performance of the service provider to the SLA metrics or KPIs. If the metrics are met, then by rights, the service provider is doing its job and performing up to expectations of the SLA and your organization.

• What if the metrics are not being met either periodically or consistently? Solving this is the goal of remedies. Remedies are typically monetary costs (in some form) to the provider that they must pay for not meeting a service-level commitment. Credits can vary significantly and should be aligned to the severity of the missed service level. Sometimes there no credits offered by the vendor. This is a red flag in an SLA.
• Typically expressed as a monetary credit, the SLA will have service levels and associated credits if the service-level metric/KPI is not met during the reporting period. Credits can be expressed in a dollar format, often defined as a percentage of a monthly fee or prorated annual fee. Although less common, some SLAs offer non-financial credits. These could include: an extension to service term, additional modules, training credits, access to a higher support level, etc.
• Regardless of how the credit is presented, this is typically the only way to hold your provider accountable to their commitments and to ensure they perform consistently to expectations. You must do a rough calculation to validate the potential monetary value and if the credit is meaningful enough to the provider.

Research shows that credit values that equate to just a few dollars, when you are paying the provider tens of thousands of dollars a month for a service or product, the credit is insignificant and therefore doesn’t incent the provider to achieve or maintain a service level.

1.3.2 Monitoring, reporting requirements, remedies/credit process

Credit Process

Along with meaningful credit values, there must be a defined credit calculation method and credit redemption process in the SLA.

Credit calculation. The credit calculation should be simple and straight forward. Many times, we see providers define complicated methods of calculating the credit value. In some cases complicated service levels require higher effort to monitor and report on, but this shouldn’t mean that the credit for missing the service level needs to require the same effort to calculate. Do a sample credit calculation to validate if the potential credit value is meaningful enough or meets your requirements.

Credit redemption process. The SLA should define the process of how a credit is provided to the customer. Ideally the process should be fairly automated by the service provider. If the report shows a missed service level, that should trigger a credit calculation and credit value posted to account followed by notification. In many SLAs that we review, the credit process is either poorly defined or not defined at all. When it is defined, the process typically requires the customer to follow an onerous process and submit a credit request that must then be validated by the provider and then, if approved, posted to your account to be applied at year end as long as you are in complete compliance with the agreement and up-to-date on your account etc. This is what we need to avoid in provider-written SLAs. You need a proactive process where the service provider takes responsibility for missing an SLA and automatically assigns an accurate credit to your account with an email notice.

Secondary level remedies. These are remedies for partial performance. For example, the platform is accessible but some major modules are not working (i.e.: the payroll platform is up and running and accessible but the tax table is not working properly so you can’t complete your payroll run on-time). Consider the requirement of a service level, metric, and remedy for critical components of a service and not just the platform availability.

Info-Tech Insight SLA’s without adequate remedies to hold the vendor accountable to their commitments make the SLAs essentially meaningless.

1.4 Exclusions indemnification, force majeure, scheduled maintenance

Contract-Related Exclusions

Attaining service-level commitments by the provider within an SLA can depend on other factors that could greatly influence their performance to service levels. Most of these other factors are common and should be defined in the SLA as exclusions or exceptions. Exceptions/exclusions can typically apply to credit calculations as well. Typical exceptions to attaining service levels are:

• Denial of Service (DoS) attacks
• Communication/ISP outage
• Outages of third-party hosting
• Actions or inactions of the client or third parties
• Scheduled maintenance but not emergency maintenance
• Force majeure events which can cover several different scenarios

Attention should be taken to review the exceptions to ensure they are in fact not within the reasonable control of the provider. Many times the provider will list several exclusions. Often these are not reasonable or can be avoided, and in most cases, they allow the service provider the opportunity to show unjustified service-level achievements. These should be negotiated out of the SLA.

1.5 Activity SLA Evaluation Tool

1-2 hours

Input

• SLA content
• SLA elements
• SLA objectives
• SLO calculation methods

Output

• Rating of the SLA service levels and objectives
• Overall rating of the SLA content
• Targeted list of required improvements

Materials

• SLA comprehensive checklist
• Service provider SLA

Participants

• Vendor or contract manager
• IT manager or leadership
• Application or business unit manager

The SLA Evaluation Tool will allow you evaluate an SLA for content. Enter details into the tool and evaluate the service levels and SLA elements and components to ensure the agreement contains adequate SLOs to meet your organization’s service requirements.

Instructions:

Review and identify SLA elements within the service provider’s SLA.

Enter service-level details into the tool and rate the SLOs.

Enter service elements details, validate that all required elements are in the SLA, and rate them accordingly.

Capture and evaluate service-level SLO calculations.

Review the overall rating for the SLA and create a targeted list for improvements with the service provider.

Download the SLA Evaluation Tool

1.5 Clarification: SLAs vs. SLOs vs. SLIs

SLA – Service-Level Agreement The promise or commitment

• This is the formal agreement between you and your service provider that contains their service levels and obligations with measurable metrics/KPIs and associated remedies. SLAs can be a separate or unique document, but are most commonly embedded within an MSA, SOW, SaaS, etc. as an addendum or exhibit.

SLO – Service-Level Objective The goals or targets

• This service-level agreement construct is the customer-based SLA. A Customer-based SLA is a unique agreement with one customer. The entire agreement is defined for one or all service levels provided to a particular customer. For example, you may use several services from one telecom vendor. The SLAs for these services would be covered in one contract between you and the Telco vendor, creating a unique customer-based to vendor agreement. Another scenario: a vendor offers general SLAs for its services and you negotiate a specific SLA for a particular service that is unique or exclusive to you. This would be a customer-based SLA as well.

Other common names are Metrics and Key Performance Indicators (KPIs )

SLI – Service-Level Indicator How did we do? Did we achieve the objectives?

• An SLI is the actual metric attained after the measurement period. SLI measures compliance with an SLO (service level objective). So, for example, if your SLA specifies that your systems will be available 99.95% of the time, your SLO is 99.95% uptime and your SLI is the actual measurement of your uptime. Maybe it’s 99.96%. maybe 99.99% or even 99.75% For the vendor to be compliant to the SLA, the SLI(s) must meet or exceed the SLOs within the SLA document.

Other common names: attainment, results, actual

Info-Tech Insight:

Web-posted SLAs that are not embedded within a signed MSA, can present uncertainty and risk as they can change at any time and typically without direct notice to the customer

Reduce Risk With Rock-Solid Service-Level Agreements

Phase 2

Understand SLA Elements

Phase 2

Create Requirements

Phase Steps

• 2.1 Create a list of your SLA criteria
• 2.2 Develop SLA policy & templates
• 2.3 Create a negotiation strategy
• 2.4 SLA overachieving discussion

Manage Obligations

2.1 Create a list of your SLA criteria

Principle Service Elements

With your understanding of the types of SLAs and the elements that comprise a well-written agreement

• The next step is to start to create a set of SLA criteria for service types that your organization outsources or may require in the future.
• This criteria should define the elements of the SLA with tolerance levels that will require the provider to meet your service expectations.
• Service levels, metrics/KPIs, associated remedies and reporting criteria. This criteria could be captured into table-like templates that can be referenced or inserted into service provider SLAs.
• Once you have defined minimum service-level criteria, we recommend that you do a deeper review of the various service provider types that your organization has in place. The goal of the review is to understand the objective of the service type and associated service levels and then compare them to your requirements for the service to meet your expectations. Service levels and KPIs should be no less than if your IT department was providing the service with its own resources and infrastructure.
• Most IT departments have service levels that they are required to meet with their infrastructure to the business units or organization, whether it’s App delivery, issue or problem resolution, availability etc. When any of these services are outsourced to an external service provider, you need to make all efforts to ensure that the service levels are equal to or better than the previous or existing internal expectations.
• Additionally, the goal is to identify service levels and metrics that don’t meet your requirements or expectations and/or service levels that are missing.

2.2 Develop SLA policies and templates

Contract-type Elements

After creating templates for minimum-service metrics & KPIs, reporting criteria templates, process, and timing, the next step should be to work on contract-type elements and additional service-level components. These elements should include:

• Reporting format, criteria, and timelines
• Monitoring requirements
• Minimum acceptable remedy or credits process; proactive by provider vs. reactive by customer
• Roles & responsibilities
• Acceptable exclusion details
• Termination language for persistent failure to meet SLOs

These templates or criteria minimums can be used as guidelines or policy when creating or negotiating SLAs with a service provider.

Start your initial element templates for your strategic vendors and most common service types: SaaS, IaaS, Service Desk, SecOps, etc. The goal of SLA templates is to create simple minimum guidelines for service levels that will allow you to meet your internal SLAs and expectations. Having SLA templates will show the service provider that you understand your requirements and may put you in a better negotiating position when reviewing with the provider.

When considering SLO metrics or KPIs consider the SMART guidance:

Simple: A KPI should be easy to measure. It should not be complicated, and the purpose behind recording it must be documented and communicated.

Measurable: A KPI that cannot be measured will not help in the decision-making process. The selected KPIs must be measurable, whether qualitatively or quantitatively. The procedure for measuring the KPIs must be consistent and well-defined.

Actionable: KPIs should contribute to the decision-making process of your organization. A KPI that does not make any such contributions serves no purpose.

Relevant: KPIs must be related to operations or functions that a security team seeks to assess.

Time-based: KPIs should be flexible enough to demonstrate changes over time. In a practical sense, an ideal KPI can be grouped together by different time intervals.

(Guide for Security Operations Metrics)

2.2.1 Activity: Review SLA Template & Metrics Reference Guide

1-2 hours

Input

• Service level metrics
• List of who is accountable for PPM decisions

Output

• SLO templates for service types
• SLA criteria that meets your organization’s requirements

Materials

• SLA Checklist
• SLA criteria list with SLO & credit values
• PPM Decision Review Workbook

Participants

• Vendor manager
• IT leadership
• Procurement or contract manager

1. Review the SLA Template and Metrics Reference Guide for common metrics & KPIs for the various service types. Each Service Type tab has SLA elements and SLO metrics typically associated with the type of service.
2. Some service levels have common or standard credits* that are typically associated with the service level or metric.
3. Use the SLA Template to enter service levels, metrics, and credits that meet your organization’s criteria or requirements for a given service type.

Download the SLA Template & Metrics Reference Guide

*Credit values are not standard values, rather general ranges that our research shows to be the typical ranges that credit values should be for a given missed service level

2.3 Create a negotiation strategy

Once you have created service-level element criteria templates for your organization’s requirements, it’s time to document a negotiation position or strategy to use when negotiating with service providers. Not all providers are flexible with their SLA commitments, in fact most are reluctant to change or create “unique” SLOs for individual customers. Particularly cloud vendors providing IaaS, SaaS, or PaaS, SLAs. ISP/Telcom, Co-Lo and DR/BU providers also have standard SLOs that they don’t like to stray far from. On the other hand, security ops (SIEM), service desk, hardware, and SOW/PS providers who are generally contracted to provide variable services are somewhat more flexible with their SLAs and more willing to meet your requirements.

• Service providers want to avoid being held accountable to SLOs, and their SLAs are typically written to reflect that.

The goal of creating internal SLA templates and policies is to set a minimum baseline of service levels that your organization is willing to accept, and that will meet their requirements and expectations for the outsourced service. Using these templated SLOs will set the basis for negotiating the entire SLA with the provider. You can set the SLA purpose, objectives, roles, and responsibilities and then achieve these from the service provider with solid SLOs and associated reporting and remedies.

Info-Tech Insight

Web-posted SLAs that are not embedded within a signed MSA can present uncertainty and risk as they can change at any time and typically without direct notice to the customer

2.3.1 Negotiating strategy guidance

• Be prepared. Create a negotiating plan and put together a team that understands your organization’s requirements for SLA.
• Stay informed. Request provider’s recent performance data and negotiate SLOs to the provider’s average performance.
• Know what you need. Corporate SLA templates or policies should be positioned to service providers as baseline minimums.
• Show some flexibility. Be willing to give up some ground on one SLO in exchange for acceptance of SLOs that may be more important to your organization.
• Re-group. Have a fallback position or Plan B. What if the provider can’t or won’t meet your key SLOs? Do you walk?
• Do your homework. Understand what the typical standard SLOs are for the type of service level.

2.4 SLO overachieving incentive discussion

Monitoring & Reporting

• SLO overachieving metrics are seen in some SLAs where there is a high priority for a service provider to meet and or exceed the SLOs within the SLA. These are not common terms but can be used to improve the overall service levels of a provider. In these scenarios the provider is sometimes rewarded for overachieving on the SLOs, either consistently or on a monthly or quarterly basis. In some cases, it can make financial sense to incent the service provider to overachieve on their commitments. Incentives can drive behaviors and improved performance by the provider that can intern improve the benefits to your organization and therefore justify an incent of some type.
• Example: You could have an SLO for invoice accuracy. If not achieved, it could cost the vendor if they don’t meet the accuracy metric, however if they were to consistently overachieve the metric it could save accounts payable hours of time in validation and therefore you could pass on some of these measurable savings to the provider.
• Overachieving incentives can add complexity to the SLA so they need to be easily measurable and simple to manage.
• Overachieving incentives can also be used in provider performance improvement plans, where a provider might have poor trending attainment and you need to have them improve their performance in a short period of time. Incentives typically will motivate provider improvement and generally will cost much less than replacing the provider.
• There is another school of thought that you shouldn’t have to pay a provider for doing their job; however, others are of the opinion that incentives or bonuses improve the overall performance of individuals or teams and are therefore worth consideration if both parties benefit from the over performance.

Reduce Risk With Rock-Solid Service-Level Agreements

Phase 3

Understand SLA Elements

Create Requirements

Phase 3

Manage Obligations

Phase Steps

• 3.1 SLA monitoring and tracking
• 3.2 Reporting
• 3.3 Vendor SLA reviews & optimizing
• 3.4 Performance management

3.1 SLA monitoring, tracking, and remedy reconciliation

The next step to effective SLAs is the management component. It could be fruitless if you were to spend your time and efforts negotiating your required service levels and metrics and don’t have some level of managing the SLA. In that situation you would have no way of knowing if the service provider is attaining their SLOs.

There are several key elements to effective SLA management:

• SLO monitoring
• Simple, concise reporting
• SLO attainment tracking
• Score carding & trending
• Remedy reconciliation

SLA Management framework

SLA Monitoring → Concise Reporting → Attainment Tracking → Score Carding →Remedy Reconciliation

“A shift we’re beginning to see is an increased use of data and process discovery tools to measure SLAs,” says Borowski of West Monroe. “While not pervasive yet, these tools represent an opportunity to identify the most meaningful metrics and objectively measure performance (e.g., cycle time, quality, compliance). When provided by the client, it also eliminates the dependency on provider tools as the source-of-truth for performance data.” – Stephanie Overby

3.1 SLA management framework

SLA Performance Management

• SLA monitoring provides data for SLO reports or dashboards. Reports provide attainment data for tacking over time. Attainment data feeds scorecards and allows for trending analysis. Missed attainment data triggers remedies.
• All service providers monitor their systems, platforms, tickets, agents, sensors etc. to be able to do their jobs. Therefore, monitoring is readily available from your service provider in some form.
• One of the key purposes of monitoring is to generate data into internal reports or dashboards that capture the performance metrics of the various services. Therefore, service-level and metric reports are readily available for all of the service levels that a service provider is contracted or engaged to provide.
• Monitoring and reporting are the key elements that validate how your service provider is meeting its SLA obligations and thus are very important elements of an SLA. SLO report data becomes attainment data once the metric or KPI has been captured.
• As a component of effective SLA management, this attainment data needs to be tracked/recorded in an easy-to-read format or table over a period of time. Attainment data can then be used to generate scorecards and trending reports for your review both internally and with the provider as required.
• If attainment data shows that the service provider is meeting their SLA obligations, then the SLA is meeting your requirements and expectations. If on the other hand, attainment data shows that obligations are not being met, then actions must be taken to hold the service provider accountable. The most common method is through remedies that are typically in the form of a credit through a defined process (see Sec. 1.3). Any credits due for missed SLOs should also be tracked and reported to stakeholders and accounting for validation, reconciliation, and collection.

3.2 Reporting

Monitoring & Reporting

• Many SLAs are silent on monitoring and reporting elements and require that the customer, if aware or able, to monitor the providers service levels and attainment and create their own KPI and reports. Then if SLOs are not met there is an arduous process that the customer must go through to request their rightful credit. This manual and reactive method creates all kinds of risk and cost to the customer and they should make all attempts to ensure that the service provider proactively provides SLO/KPI attainment reports on a regular basis.
• Automated monitoring and reporting is a common task for many IT departments. There is no reason that a service provider can’t send reports proactively in a format that can be easily interpreted by the customer. The ideal state would be to capture KPI report data into a customer’s internal service provider scorecard.
• Automated or automatic credit posting is another key element that service providers tend to ignore, primarily in hopes that the customer won’t request or go through the trouble of the process. This needs to change. Some large cloud vendors already have automated processes that automatically post a credit to your account if they miss an SLO. This proactive credit process should be at the top of your negotiation checklist. Service providers are avoiding thousands of credit dollars every year based on the design of their credit process. As more customers push back and negotiate more efficient credit processes, vendors will soon start to change and may use it as a differentiator with their service.

3.2.1 Performance tracking and trending

What gets measured gets done

SLO Attainment Tracking

A primary goal of proactive and automated reporting and credit process is to capture the provider’s attainment data into a tracker or vendor scorecard. These tracking scorecards can easily create status reports and performance trending of service providers, to IT leadership as well as feed QBR agenda content.

Remedy Reconciliation

Regardless of how a credit is processed it should be tracked and reconciled with internal stakeholders and accounting to ensure credits are duly applied or received from the provider and in a timely manner. Tracking and reconciliation must also align with your payment terms, whether monthly or annually.

“While the adage, ‘You can't manage what you don't measure,’ continues to be true, the downside for organizations using metrics is that the provider will change their behavior to maximize their scores on performance benchmarks.” – Rob Lemos

3.2.1 Activity SLA Tracker and Trending Tool

1-2 hours setup

Input

• SLO metrics/KPIs from the SLA
• Credit values associated with SLO

Output

• Monthly SLO attainment data
• Credit tracking
• SLO trending graphs

Materials

• Service provider SLO reports
• Service provider SLA
• SLO Tracker & Trending Tool

Participants

• Contract or vendor managers
• Application or service managers
• Service provider

An important activity in the SLA management framework is to track the provider’s SLO attainment on a monthly or quarterly basis. In addition, if an SLO is missed, an associated credit needs to be tracked and captured. This activity allows you to capture the SLOs from the SLA and track them continually and provide data for trending and review at vendor performance meetings and executive updates.

Instructions: Enter SLOs from the SLA as applicable.

Each month, from the provider’s reports or dashboards, enter the SLO metric attainment.

When an SLO is met, the cell will turn green. If the SLO is missed, the cell will turn red and a corresponding cell in the Credit Tracker will turn green, meaning that a credit needs to be reconciled.

Use the Trending tab to view trending graphs of key service levels and SLOs.

Download the SLO Tracker and Trending Tool

3.3 Vendor SLA reviews and optimizing

Regular reviews should be done with providers

Collecting attainment data with scorecards or tracking tools provides summary information on the performance of the service provider to their SLA obligations. This information should be used for regular reviews both internally and with the provider.

Regular attainment reviews should be used for:

• Performance trending upward or downward
• Identifying opportunities to revise or improve SLOs
• Optimizing SLO and processes
• Creating a Performance Improvement Plan (PIP) for the service provider

Some organizations choose to review SLA performance with providers at regular QBRs or at specific SLA review meetings

This should be determined based on the criticality, risk, and strategic importance of the provider’s service. Providers that provide essential services like ERP, payroll, CRM, HRIS, IaaS etc. should be reviewed much more regularly to ensure that any decline in service is identified early and addressed properly in accordance with the service provider. Negative trending performance should also be documented for consideration at renewal time.

3.4 Performance management

Dealing with persistent poor performance and termination

Service providers that consistently miss key service level metrics or KPIs present financial and security risk to the organization. Poor performance of a service provider reflects directly on the IT leadership and will affect many other business aspects of the organization including:

• Ability to conduct day-to-day business activities
• Meet internal obligations and expectations
• Employee productivity and satisfaction
• Maintain corporate policies or industry compliance
• Meet security requirements

Communication is key. Poor performance of a service provider needs to be dealt with in a timely manner in order to avoid more critical impact of the poor performance. Actions taken with the provider can also vary depending again on the criticality, risk, and strategic importance of the provider’s service.

Performance reviews should provide the actions required with the goal of:

• Making the performance problems into opportunities
• Working with the provider to create a PIP with aggressive timelines and ramifications if not attained
• Non-renewal or termination consideration, if feasible including provider replacement options, risk, costs, etc.
• SLA renegotiation or revisions
• Warning notifications to the service provider with concise issues and ramifications

To avoid the issues and challenges of dealing with chronic poor performance, consider a Persistent or Chronic Failure clause into the SLA contract language. These clauses can define chronic failure, scenarios, ramifications there of, and defined options for the client including increased credit values, non-monetary remedies, and termination options without liability.

Info-Tech Insight

It’s difficult to prevent chronic poor performance but you can certainly track it and deal with it in a way that reduces risk and cost to your organization.

SLA Hall of Shame

Crazy service provider SLA content collection

• Excessive list of unreasonable exclusions
• Subcontractors’ behavior could be excluded
• Downtime credit, equal to downtime percent x the MRC
• Controllable FM events (internal labor issues, health events)
• Difficult downtime or credit calculations that don’t make sense
• Credits are not valid if agreement is terminated early or not renewed
• Customer is not current on their account, SLA or credits do not count/apply
• Total downtime = to prorated credit value (down 3 hrs = 3/720hrs = 0.4% credit)
• SLOs don’t apply if customer fails to report the issue or request a trouble ticket
• Downtime during off hours (overnight) do not count towards availability metrics
• Different availability commitments based on different support-levels packages
• Extending the agreement term by the length of downtime as a form of a remedy

SLA Dos and Don’ts

Dos

• Do negotiate SLOs to vendor’s average performance
• Do strive for automated reporting and credit processes
• Do right-size and create your SLO criteria based on risk mitigation
• Do review SLA attainment results with strategic service providers on a regular basis
• Do ensure that all key elements and components of an SLA are present in the document or appendix

Don'ts

• Don’t accept the providers response that “we can’t change the SLOs for you because then we’d have to change them for everyone”
• Don’t leave SLA preparation to the last minute. Give it priority as you negotiate with the provider
• Don’t create complex SLAs with numerous service levels and SLOs that need to be reported and managed
• Don’t aim for absolute perfection. Rather, prioritize which service levels are most important to you for the service

Summary of Accomplishment

Problem Solved

Knowledge Gained

• Understanding of the elements and components of an SLA
• A list of SLO metrics aligned to service types that meet your organization’s criteria
• SLA metric/KPI templates
• SLA Management process for your provider’s service objectives
• Reporting and tracking process for performance trending

Deliverables Completed

• SLA component and contract element checklist
• Evaluation or service provider SLAs
• SLA templates for strategic service types
• SLA tracker for strategic service providers

If you would like additional support, have our analysts guide you through other phases as part of an Info-Tech workshop.

Contact your account representative for more information.

workshops@infotech.com

1-888-670-8889

Related Info-Tech Research

Improve IT-Business Alignment Through an Internal SLA

• Understand business requirements, clarify current capabilities, and enable strategies to close service-level gaps.

Data center Co-location SLA & Service Definition Template

• In essence, the SLA defines the “product” that is being purchased, permitting the provider to rationalize resources to best meet the needs of varied clients, and permits the buyer to ensure that business requirements are being met.

Ensure Cloud Security in IaaS, PaaS, and SaaS Environments

• Keep your information security risks manageable when leveraging the benefits of cloud computing.

Bibliography

Henderson, George. “3 Most Common Types of Service Level Agreement (SLA).” Master of Project Academy. N.d. Web.

“Guide to Security Operations Metrics.” Logsign. Oct 5, 2020. Web.

Lemos, Rob. “4 lessons from SOC metrics: What your SpecOps team needs to know.” TechBeacon. N.d. Web.

“Measuring and Making the Most of Service Desk Metrics.” Freshworks. N.d. Web.

Overby, Stephanie. “15 SLA Mistakes IT Leaders Still Make.” CIO. Jan 21, 2021.

Integrate IT Risk Into Enterprise Risk

Don’t fear IT risks, integrate them.

EXECUTIVE BRIEF

Analyst Perspective

Having siloed risks is risky business for any enterprise.

Valence Howden Principal Research Director, CIO Practice

Petar Hristov Research Director, Security, Privacy, Risk & Compliance

Ian Mulholland Research Director, Security, Risk & Compliance

Brittany Lutes Senior Research Analyst, CIO Practice

Ibrahim Abdel-Kader Research Analyst, CIO Practice

Every organization has a threshold for risk that should not be exceeded, whether that threshold is defined or not.

In the age of digital, information and technology will undoubtedly continue to expand beyond the confines of the IT department. As such, different areas of the organization cannot address these risks in silos. A siloed approach will produce different ways of identifying, assessing, responding to, and reporting on risk events. Integrated risk management is about embedding IT uncertainty to inform good decision making across the organization.

When risk is integrated into the organization's enterprise risk management program, it enables a single view of all risks and the potential impact of each risk event. More importantly, it provides a consistent view of the risk event in relation to uncertainty that might have once been seemingly unrelated to IT.

And all this can be achieved while remaining within the enterprise’s clearly defined risk appetite.

Executive Summary

Your Challenge

Most organizations fail to integrate IT risks into enterprise risks:

• IT risks, when considered, are identified and classified separately from the enterprise-wide perspective.
• IT is expected to own risks over which they have no authority or oversight.
• Poor behaviors, such as only considering IT risks when conducting compliance or project due diligence, have been normalized.

Common Obstacles

IT leaders have to overcome these obstacles when it comes to integrating risk:

• Making business leaders aware of, involved in, and able to respond to all enterprise risks.
• A lack of data or information being used to support a holistic risk management process.
• A low level of enterprise risk maturity.
• A lack of risk management capabilities.

Info-Tech’s Approach

By leveraging the Info-Tech Integrated Risk approach, your business can better address and embed risk by:

• Understanding gaps in the organization’s current approach to risk management practices.
• Establishing a standardized approach for how IT risks impact the enterprise as a whole.
• Driving a risk-aware organization toward innovation and considering alternative options for how to move forward.
• Helping integrate IT risks into the foundational risk practice.

Info-Tech Insight

Stop avoiding risk – integrate it. This provides a holistic view of uncertainty for the organization to drive innovative new approaches to optimize its ability to respond to risk.

What is integrated risk management?

• Integrated risk management is the process of ensuring all forms of risk information, including information and technology, are considered and included in the enterprise’s risk management strategy.
• It removes the siloed approach to classifying risks related to specific departments or areas of the organization, recognizing that each of those risks is a threat to the overarching enterprise.
• Aggregating the different threats or uncertainty that might exist within an organization allows for informed decisions to be made that align to strategic goals and continue to drive value back to the business.
• By holistically considering the different risks, the organization can make informed decisions on the best course of action that will reduce any negative impacts associated with the uncertainty and increase the overall value.

Enterprise Risk Management (ERM)

• IT
• Security
• Digital
• Vendor/Third Party
• Other

Enterprise risk management is the practice of identifying and addressing risks to your organization and using risk information to drive better decisions and better opportunities.

IT risk is enterprise risk

IT risks have a direct and often aggregated impact on enterprise risks and opportunities in the same way other business risks can. This relationship must be understood and addressed through integrated risk management to ensure a consistent approach to risk.

Your challenge

Embedding IT risks into the enterprise risk management program is challenging because:

• Most organizations classify risks based on the departments or areas of the business where the uncertainty is likely to happen.
• Unnecessary expectations are placed on the IT department to own risks over which they have no authority or oversight.
• Risks are often only identified when conducting due diligence for a project or ensuring compliance with regulations and standards.

Risk-mature organizations have a unique benefit in that they often have established an overarching governance framework and embedded risk awareness into the culture.

35% — Only 35% of organizations had embraced ERM in 2020. (Source: AICPA and NC State Poole College of Management)

12% — Only 12% of organizations are leveraging risk as a tool to their strategic advantage. (Source: AICPA and NC State Poole College of Management)

Common obstacles

These barriers make integrating IT risks difficult to address for many organizations:

• IT risks are not seen as enterprise risks.
• The organization’s culture toward risk is not defined.
• The organization’s appetite and threshold for risk are not defined.
• Each area of the organization has a different method of identifying, assessing, and responding to risk events.
• Access to reliable and informative data to support risk management is difficult to obtain.
• Leadership does not see the business value of integrating risk into a single management program.
• The organization’s attitudes and behaviors toward risk contradict the desired and defined risk culture.
• Skills, training, and resources to support risk management are lacking, let alone those to support integrated risk management.

Integrating risks has its challenges

62% — Accessing and disseminating information is the main challenge for 62% of organizations maturing their organizational risk management. (Source: OECD)

20-28% — Organizations with access to machine learning and analytics to address future risk events have 20 to 28% more satisfaction. (Source: Accenture)

Integrate Risk and Use It to Your Advantage

Accelerate and optimize your organization by leveraging meaningful risk data to make intelligent enterprise risk decisions.

Risk management is more than checking an audit box or demonstrating project due diligence.

Risk Drivers Audit & compliance Preserve value & avoid loss Previous risk impact driver Major transformation Strategic opportunities		Only 7% of organizations are in a “leading” or “aspirational” level of risk maturity. (OECD, 2021)	63% of organizations struggle when it comes to defining their appetite toward strategy related risks. (“Global Risk Management Survey,” Deloitte, 2021)	Late adopters of risk management were 70% more likely to use instinct over data or facts to inform an efficient process. (Clear Risk, 2020)	55% of organizations have little to no training on ERM to properly implement such practices. (AICPA, NC State Poole College of Management, 2021)
		1. Assess Enterprise Risk Maturity	3. Build a Risk Management Program Plan	4. Establish Risk Management Processes	5. Implement a Risk Management Program
		2. Determine Authority with Governance Unfortunately, less than 50% of those in risk focused roles are also in a governance role where they have the authority to provide risk oversight. (Governance Institute of Australia, 2020)
IT can improve the maturity of the organization’s risk governance and help identify risk owners who have authority and accountability. Governance and related decision making is optimized with integrated and aligned risk data.			ERM incorporates the different types of risk, including IT, security, digital, vendor, and other risk types. The program plan is meant to consider all the major risk types in a unified approach.		Implementation of an integrated risk management program requires ongoing access to risk data by those with decision making authority who can take action.

Integrated Risk Mapping — Downside Risk Focus

Integrated Risk Mapping — Downside and Upside Risk

Third-Party Risk Example

Insight Summary

Overarching insight

Stop fearing risk – integrate it. Integration leads to opportunities for organizations to embrace innovation and new digital technologies as well as reducing operational costs and simplifying reporting.

Govern risk strategically

Governance of risk management for information- and technology-related events is often misplaced. Just because it's classified as an IT risk does not mean it shouldn’t be owned by the board or business executive.

Assess risk maturity

Integrating risk requires a baseline of risk maturity at the enterprise level. IT can push integrating risks, but only if the enterprise is willing to adopt the attitudes and behaviors that will drive the integrated risk approach.

Manage risk

It is not a strategic decision to have different areas of the organization manage the risks perceived to be in their department. It’s the easy choice, but not the strategic one.

Implement risk management

Different areas of an enterprise apply risk management processes differently. Determining a single method for identification, assessment, response, and monitoring can ensure successful implementation of enterprise risk management.

Tactical insight

Good risk management will consider both the positives and negatives associated with a risk management program by recognizing both the upside and downside of risk event impact and likelihood.

Integrated risk benefits

Measure the value of integrating risk

• Reduce Operating Costs

  • Organizations can reduce their risk operating costs by 20 to 30% by adopting enterprise-wide digital risk initiatives (McKinsey & Company).

• Increase Cybersecurity Threat Preparedness

  • Increase the organization’s preparedness for cybersecurity threats. 79% of organizations that were impacted by email threats in 2020 were not prepared for the hit (Diligent)

• Increase Risk Management’s Impact to Drive Strategic Value

  • Currently, only 3% of organizations are extensively using risk management to drive their unique competitive advantage, compared to 35% of companies who do not use it at all (AICPA & NC State Poole College of Management).

• Reduce Lost Productivity for the Enterprise

  • Among small businesses, 76% are still not considering purchasing cyberinsurance in 2021, despite the fact that ransomware attacks alone cost Canadian businesses $5.1 billion in productivity in 2020 (Insurance Bureau of Canada, 2021).

“31% of CIO’s expected their role to expand and include risk management responsibilities.” (IDG “2021 State of the CIO,” 2021)

Make integrated risk management sustainable

Assessment

Assess your organization’s method of addressing risk management to determine if integrated risk is possible

Assessing the organization’s risk maturity

Mature or not, integrated risk management should be a consideration for all organizations The first step to integrating risk management within the enterprise is to understand the organization’s readiness to adopt practices that will enable it to successfully integrate information. In 2021, we saw enterprise risk management assessments become one of the most common trends, particularly as a method by which the organization can consolidate the potential impacts of uncertainties or threats (Lawton, 2021). A major driver for this initiative was the recognition that information and technology not only have enterprise-wide impacts on the organization’s risk management but that IT has a critical role in supporting processes that enable effective access to data/information. A maturity assessment has several benefits for an organization: It ensures there is alignment throughout the organization on why integrated risk is the right approach to take, it recognizes the organization’s current risk maturity, and it supports the organization in defining where it would like to go.

Maturity should inform your approach to risk management

The outcome of the risk maturity assessment should inform how risk management is approached within the organization.

For organizations with a low maturity, remaining superficial with risk will offer more benefits and align to the enterprise’s risk tolerance and appetite. This might mean no integrated risk is taking place.

However, organizations that have higher risk maturity should begin to integrate risk information. These organizations can identify the nuances that would affect the severity and impact of risk events.

Integrated Risk Maturity Assessment

The purpose of the Integrated Risk Maturity Assessment is to assess the organization's current maturity and readiness for integrated risk management (IRM).

Frequently and continually assessing your organization’s maturity toward integrated risk ensures the right risk management program can be adopted by your organization.

Integrated Risk Maturity Assessment A simple tool to understand if your organization is ready to embrace integrated risk management by measuring maturity across four key categories: Context & Strategic Direction, Risk Culture & Authority, Risk Management Process, and Risk Program Optimization

Use the results from this integrated risk maturity assessment to determine the type of risk management program that can and should be adopted by your organization.

Some organizations will need to remain siloed and focused on IT risk management only, while others will be able to integrate risk-related information to start enabling automatic controls that respond to this data.

Identify and Manage Security Risk Impacts on Your Organization

Know where the attacks are coming from so you know where to protect.

Analyst perspective

It is time to start looking at risk realistically and move away from “trust but verify” toward zero trust.

Frank Sewell,
Research Director, Vendor Management
Info-Tech Research Group

We are inundated with a barrage of news about security incidents on what seems like a daily basis. In such an environment, it is easy to forget that there are ways to help prevent such things from happening and that they have actual costs if we relax our diligence.

Most people are aware of defense strategies that help keep their organization safe from direct attack and inside threats. Likewise, they expect their trusted partners to perform the same diligence. Unfortunately, as more organizations use cloud service vendors, the risks with n-party vendors are increasing.

Over the last few years, we have learned the harsh lesson that downstream attacks affect more businesses than we ever expected as suppliers, manufacturers of base goods and materials, and rising transportation costs affect the global economy.

“Trust but verify” – while a good concept – should give way to the more effective zero-trust model in favor of knowing it’s not a matter of if an incident happens but when.

Executive Summary

Info-Tech Insight
Organizations must evolve their security risk assessments to be more adaptive to respond to global changes in the market. Ongoing monitoring of third-party vendor risks and holding those vendors accountable throughout the vendor lifecycle are critical to preventing disastrous impacts.

Info-Tech’s multi-blueprint series on vendor risk assessment

There are many individual components of vendor risk beyond cybersecurity.

This series will focus on the individual components of vendor risk and how vendor management practices can facilitate organizations’ understanding of those risks.

Out of Scope:
This series will not tackle risk governance, determining overall risk tolerance and appetite, or quantifying inherent risk.

Security risk impacts

Potential losses to the organization due to security incidents

• In this blueprint we’ll explore security risks, particularly from third-party vendors, and their impacts.
• Identify potentially disruptive events to assess the overall impact on organizations and implement adaptive measures to correct security plans.

The world is constantly changing

The IT market is constantly reacting to global influences. By anticipating changes, leaders can set expectations and work with their vendors to accommodate them.

When the unexpected happens, being able to adapt quickly to new priorities ensures continued long-term business success.

Below are some things no one expected to happen in the last few years:

Identify and manage security risk impacts on your organization

Due diligence will enable successful outcomes.

What is third-party risk?

Third-Party Vendor: Anyone who provides goods or services to a company or individual in exchange for payment transacted with electronic instructions (Law Insider).

Third-Party Risk: The potential threat presented to organizations’ employee and customer data, financial information, and operations from the organization’s supply chain and other outside parties that provide products and/or services and have access to privileged systems (Awake Security).

It is essential to know not only who your vendors are but also who their vendors are (n-party vendors). Organizations often overlook that their vendors rely on others to support their business, and those layers can add risk to your organization.

Identify and manage security risks

Global Pandemic

Very few people could have predicted that a global pandemic would interrupt business on the scale experienced today. Organizations should look at their lessons learned and incorporate adaptable preparations into their security planning and ongoing monitoring moving forward.

Vendor Breaches

The IT market is an ever-shifting environment; more organizations are relying on cloud service vendors, staff augmentation, and other outside resources. Organizations should hold these vendors (and their downstream vendors) to the same levels of security and standards of conduct that they hold their internal resources.

Resource Shortages

A lack of resources is often overlooked, but it’s easily recognized as a reason for a security incident. All too often, companies are unwilling to dedicate resources to their vendors’ security risk assessment and ongoing monitoring needs. Only once an incident occurs do companies decide it is time to reprioritize.

Create a Right-Sized Disaster Recovery Plan

Close the gap between your DR capabilities and service continuity requirements.

ANALYST PERSPECTIVE

An effective disaster recovery plan (DRP) is not just an insurance policy.

"An effective DRP addresses common outages such as hardware and software failures, as well as regional events, to provide day-to-day service continuity. It’s not just insurance you might never cash in. Customers are also demanding evidence of an effective DRP, so organizations without a DRP risk business impact not only from extended outages but also from lost sales. If you are fortunate enough to have executive buy-in, whether it’s due to customer pressure or concern over potential downtime, you still have the challenge of limited time to dedicate to disaster recovery (DR) planning. Organizations need a practical but structured approach that enables IT leaders to create a DRP without it becoming their full-time job."

Frank Trovato,

Research Director, Infrastructure

Info-Tech Research Group

Is this research for you?

This Research Is Designed For:

• Senior IT management responsible for executing DR.
• Organizations seeking to formalize, optimize, or validate an existing DRP.
• Business continuity management (BCM) professionals leading DRP development.

This Research Will Help You:

• Create a DRP that is aligned with business requirements.
• Prioritize technology enhancements based on DR requirements and risk-impact analysis.
• Identify and address process and technology gaps that impact DR capabilities and day-to-day service continuity.

This Research Will Also Assist:

• Executives who want to understand the time and resource commitment required for DRP.
• Members of BCM and crisis management teams who need to understand the key elements of an IT DRP.

This Research Will Help Them:

• Scope the time and effort required to develop a DRP.
• Align business continuity, DR, and crisis management plans.

Executive summary

Situation

• Any time a natural disaster or major IT outage occurs, it increases executive awareness and internal pressure to create a DRP.
• Industry standards and government regulations are driving external pressure to develop business continuity and IT DR plans.
• Customers are asking suppliers and partners to provide evidence that they have a workable DRP before agreeing to do business.

Complication

• Traditional DRP templates are onerous and result in a lengthy, dense plan that might satisfy auditors, but will not be effective in a crisis.
• The myth that a DRP is only for major disasters leaves organizations vulnerable to more common incidents.
• The growing use of outsourced infrastructure services has increased reliance on vendors to meet recovery timeline objectives.

Resolution

• Create an effective DRP by following a structured process to discover current capabilities and define business requirements for continuity:
  • Define appropriate objectives for service downtime and data loss based on business impact.
  • Document an incident response plan that captures all of the steps from event detection to data center recovery.
  • Create a DR roadmap to close gaps between current DR capabilities and recovery objectives.

Info-Tech Insight

1. At its core, DR is about ensuring service continuity. Create a plan that can be leveraged for both isolated and catastrophic events.
2. Remember Murphy’s Law. Failure happens. Focus on improving overall resiliency and recovery, rather than basing DR on risk probability analysis.
3. Cost-effective DR and service continuity starts with identifying what is truly mission critical so you can focus resources accordingly. Not all services require fast failover.

An effective DRP is critical to reducing the cost of downtime

If you don’t have an effective DRP when failure occurs, expect to face extended downtime and exponentially rising costs due to confusion and lack of documented processes.

Potential Lost Revenue

The impact of downtime tends to increase exponentially as systems remain unavailable (graph at left). A current, tested DRP will significantly improve your ability to execute systems recovery, minimizing downtime and business impact. Without a DRP, IT is gambling on its ability to define and implement a recovery strategy during a time of crisis. At the very least, this means extended downtime – potentially weeks or months – and substantial business impact.

Adapted from: Philip Jan Rothstein, 2007

Cost of Downtime for the Fortune 1000

Cost of unplanned apps downtime per year: $1.25B to $2.5B.

Cost of critical apps failure per hour: $500,000 to $1M.

Cost of infrastructure failure per hour: $100,000.

35% reported to have recovered within 12 hours.

17% of infrastructure failures took more than 24 hours to recover.

13% of application failures took more than 24 hours to recover.

Source: Stephen Elliot, 2015

Info-Tech Insight

The cost of downtime is rising across the board, and not just for organizations that traditionally depend on IT (e.g. e-commerce). Downtime cost increase since 2010:

Hospitality: 129% increase

Transportation: 108% increase

Media organizations: 104% increase

An effective DRP also sets clear recovery objectives that align with system criticality to optimize spend

Take a practical approach that creates a more concise and actionable DRP

DR planning is not your full-time job, so it can’t be a resource- and time-intensive process.

DR must be integrated with day-to-day incident management to ensure service continuity

When a tornado takes out your data center, it’s an obvious DR scenario and the escalation towards declaring a disaster is straightforward.

The challenge is to be just as decisive in less-obvious (and more common) DR scenarios such as a critical system hardware/software failure, and knowing when to move from incident management to DR. Don’t get stuck troubleshooting for days when you could have failed over in hours.

Bridge the gap with clearly-defined escalation rules and criteria for when to treat an incident as a disaster.

Source: Info-Tech Research Group; N=92

Myth busted: The DRP is separate from day-to-day ops and incident management.

The most common threats to service continuity are hardware and software failures, network outages, and power outages

Source: Info-Tech Research Group; N=87

Info-Tech Insight

Does this mean I don’t need to worry about natural disasters? No. It means DR planning needs to focus on overall service continuity, not just major disasters. If you ignore the more common but less dramatic causes of service interruptions, you are diminishing the business value of a DRP.

Myth busted: DRPs are just for destructive events – fires, floods, and natural disasters.

DR isn’t about identifying risks; it’s about ensuring service continuity

The traditional approach to DR starts with an in-depth exercise to identify risks to IT service continuity and the probability that those risks will occur.

Here’s why starting with a risk register is ineffective:

• Odds are, you won’t think of every incident that might occur. If you think of twenty risks, it’ll be the twenty-first that gets you. If you try to guard against that twenty-first risk, you can quickly get into cartoonish scenarios and much more costly solutions.
• The ability to failover to another site mitigates the risk of most (if not all) incidents (fire, flood, hardware failure, tornado, etc.). A risk and probability analysis doesn’t change the need for a plan that includes a failover procedure.

Where risk is incorporated in this methodology:

• Use known risks to further refine your strategy (e.g. if you are prone to hurricanes, plan for greater geographic separation between sites; ensure you have backups, in addition to replication, to mitigate the risk of ransomware).
• Identify risks to your ability to execute DR (e.g. lack of cross-training, backups that are not tested) and take steps to mitigate those risks.

Myth busted: A risk register is the critical first step to creating an effective DR plan.

You can’t outsource accountability and you can’t assume your vendor’s DR capabilities meet your needs

Outsourcing infrastructure services – to a cloud provider, co-location provider, or managed service provider (MSP) – can improve your DR and service continuity capabilities. For example, a large public cloud provider will generally have:

• Redundant telecoms service providers, network infrastructure, power feeds, and standby power.
• Round-the-clock infrastructure and security monitoring.
• Multiple data centers in a given region, and options to replicate data and services across regions.

Still, failure is inevitable – it’s been demonstrated multiple times1 through high-profile outages. When you surrender direct control of the systems themselves, it’s your responsibility to ensure the vendor can meet your DR requirements, including:

• A DR site and acceptable recovery times for systems at that site.
• An acceptable replication/backup schedule.

Sources: Kyle York, 2016; Shaun Nichols, 2017; Stephen Burke, 2017

Myth busted: I outsource infrastructure services so I don’t have to worry about DR. That’s my vendor’s responsibility.

Choose flowcharts over process guides, checklists over procedures, and diagrams over descriptions

IT DR is not an airplane disaster movie. You aren’t going to ask a business user to execute a system recovery, just like you wouldn’t really want a passenger with no flying experience to land a plane.

In reality, you write a DR plan for knowledgeable technical staff, which allows you to summarize key details your staff already know. Concise, visual documentation is:

• Quicker to create.
• Easier to use.
• Simpler to maintain.

"Without question, 300-page DRPs are not effective. I mean, auditors love them because of the detail, but give me a 10-page DRP with contact lists, process flows, diagrams, and recovery checklists that are easy to follow."

– Bernard Jones, MBCI, CBCP, CORP, Manager Disaster Recovery/BCP, ActiveHealth Management

Source: Info-Tech Research Group; N=95

*DR Success is based on stated ability to meet recovery time objectives (RTOs) and recovery point objectives (RPOs), and reported confidence in ability to consistently meet targets.

Myth busted: A DRP must include every detail so anyone can execute recovery.

A DRP is part of an overall business continuity plan

A DRP is the set of procedures and supporting documentation that enables an organization to restore its core IT services (i.e. applications and infrastructure) as part of an overall business continuity plan (BCP), as described below. Use the templates, tools, and activities in this blueprint to create your DRP.

Note: For DRP, we focus on business-facing IT services (as opposed to the underlying infrastructure), and then identify required infrastructure as dependencies (e.g. servers, databases, network).

Take a practical but structured approach to creating a concise and effective DRP

Info-Tech offers various levels of support to best suit your needs

DIY Toolkit

"Our team has already made this critical project a priority, and we have the time and capability, but some guidance along the way would be helpful."

Guided Implementation

“Our team knows that we need to fix a process, but we need assistance to determine where to focus. Some check-ins along the way would help keep us on track.”

Workshop

“We need to hit the ground running and get this project kicked off immediately. Our team has the ability to take this over once we get a framework and strategy in place.”

Consulting

“Our team does not have the time or the knowledge to take this project on. We need assistance through the entirety of this project.”

Diagnostics and consistent frameworks used throughout all four options

Info-Tech advisory services deliver measurable value

Info-Tech members save an average of $22,983 and 22 days by working with an Info-Tech analyst on DRP (based on client response data from Info-Tech Research Group’s Measured Value Survey, following analyst advisory on this blueprint).

Why do members report value from analyst engagement?

1. Expert advice on your specific situation to overcome obstacles and speed bumps.
2. Structured project and guidance to stay on track.
3. Project deliverables review to ensure the process is applied properly.

Guided implementation overview

Your trusted advisor is just a call away.

Define DRP scope (Call 1)

Scope requirements, objectives, and your specific challenges. Identify applications/ systems to focus on first.

Define current status and system dependencies (Calls 2-3)

Assess current DRP maturity. Identify system dependencies.

Conduct a BIA (Calls 4-6)

Create an impact scoring scale and conduct a BIA. Identify RTO and RPO for each system.

Recovery workflow (Calls 7-8)

Create a recovery workflow based on tabletop planning. Identify gaps in recovery capabilities.

Projects and action items (Calls 9-10)

Identify and prioritize improvements. Summarize results and plan next steps.

Your guided implementations will pair you with an advisor from our analyst team for the duration of your DRP project.

Workshop overview

Contact your account representative or email Workshops@InfoTech.com for more information.

End-user complaints distract from serious IT-based risks to business continuity

Case Study

Industry: Manufacturing
Source: Info-Tech Research Group Client Engagement

A global manufacturer with annual sales over $1B worked with Info-Tech to improve DR capabilities.

DRP BIA

Conversations with the IT team and business units identified the following impact of downtime over 24 hours:

• Email: Direct Cost: $100k; Goodwill Impact Score: 8.5/16
• ERP: Direct Cost: $1.35mm; Goodwill Impact Score: 12.5/16

Tabletop Testing and Recovery Capabilities

Reviewing the organization’s current systems recovery workflow identified the following capabilities:

• Email: RTO: minutes, RPO: minutes
• ERP: RTO: 14 hours, RPO: 24 hours

Findings

Because of end-user complaints, IT had invested heavily in email resiliency though email downtime had a relatively minimal impact on the business. After working through the methodology, it was clear that the business needed to provide additional support for critical systems.

Insights at each step:

Identify DR Maturity and System Dependencies

Conduct a BIA

Outline Incident Response and Recovery Workflow With Tabletop Exercises

Mitigate Gaps and Risks

Create a Right-Sized Disaster Recovery Plan

Phase 1

Define DRP Scope, Current Status, and Dependencies

Step 1.1: Set Scope, Kick-Off the DRP Project, and Create a Charter

This step will walk you through the following activities:

• Establish a team for DR planning.
• Retrieve and review existing, relevant documentation.
• Create a project charter.

This step involves the following participants:

• DRP Coordinator
• DRP Team (Key IT SMEs)
• IT Managers

Results and Insights

• Set scope for the first iteration of the DRP methodology.
• Don’t try to complete your DR and BCPs all at once.
• Don’t bite off too much at once.

Kick-off your DRP project

You’re ready to start your DR project.

This could be an annual review – but more likely, this is the first time you’ve reviewed the DR plan in years.* Maybe a failed audit might have provided a mandate for DR planning, or a real disaster might have highlighted gaps in DR capabilities. First, set appropriate expectations for what the project is and isn’t, in terms of scope, outputs, and resource commitments. Very few organizations can afford to hire a full-time DR planner, so it’s likely this won’t be your full-time job. Set objectives and timelines accordingly.

Gather a team

• Often, DR efforts are led by the infrastructure and operations leader. This person can act as the DRP coordinator or may delegate this role.
• Key infrastructure subject-matter experts (SMEs) are usually part of the team and involved through the project.

Find and review existing documentation

• An existing DRP may have information you can re-purpose rather than re-create.
• High-level architecture diagrams and network diagrams can help set scope (and will become part of your DR kit).
• Current business-centric continuity of operations plans (COOPs) or BCPs are important to understand.

Set specific, realistic objectives

• Create a project charter (see next slide) to record objectives, timelines, and assumptions.

*Only 20% of respondents to an Info-Tech Research Group survey (N=165) had a complete DRP; only 38% of respondents with a complete or mostly complete DRP felt it would be effective in a crisis.

List DRP drivers and challenges

1(a) Drivers and roadblocks

Estimated Time: 30 minutes

Identify the drivers and challenges to completing a functional DRP plan with the core DR team.

DRP Drivers

• Past outages (be specific):
  • Hardware and software failures
  • External network and power outages
  • Building damage
  • Natural disaster(s)
• Audit findings
• Events in the news
• Other?

DRP Challenges

• Lack of time
• Insufficient DR budget
• Lack of executive support
• No internal DRP expertise
• Challenges making the case for DRP
• Other?

Write down insights from the meeting on flip-chart paper or a whiteboard and use the findings to inform your DRP project (e.g. challenges to address).

Clarify expectations with a project charter

1(b) DRP Project Charter Template

DRP Project Charter Template components:

Define project parameters, roles, and objectives, and clarify expectations with the executive team. Specific subsections are listed below and described in more detail in the remainder of this phase.

• Project Overview: Includes objectives, deliverables, and scope. Leverage relevant notes from the “Project Drivers” brainstorming exercise (e.g. past outages and near misses which help make the case).
• Governance and Management: Includes roles, responsibilities, and resource requirements.
• Project Risks, Assumptions, and Constraints: Includes risks and mitigation strategies, as well as any assumptions and constraints.
• Project Sign-Off: Includes IT and executive sign-off (if required).

Note: Identify the initial team roles and responsibilities first so they can assist in defining the project charter.

Step 1.2: Assess Current State DRP Maturity

This step will walk you through the following activities:

• Complete Info-Tech’s DRP Maturity Scorecard.

This step involves the following participants:

• DRP Coordinator
• IT SMEs

Results and Insights

• Identify the current state of the organization’s DRP and continuity management. Set a baseline for improvement.
• Discover where improvement is most needed to create an effective plan.

Only 38% of IT departments believe their DRPs would be effective in a real crisis

Even organizations with documented DRPs struggle to make them actionable.

• Even when a DRP does become a priority (e.g. due to regulatory or customer drivers), the challenge is knowing where to start and having a methodical step-by-step process for doing the work. With no guide to plan and resource the project, it becomes work that you complete piecemeal when you aren’t working on other projects, or at night after the kids go to bed.
• Far too many organizations create a document to satisfy auditors rather than creating a usable plan. People in this group often just want a fill-in-the-blanks template. What they will typically find is a template for the traditional 300-page manual that goes in a binder that sits on a shelf, is difficult to maintain, and is not effective in a crisis.

Use the DRP Maturity Scorecard to assess the current state of your DRP and identify areas to improve

1(c) DRP Maturity Scorecard

Info-Tech’s DRP Maturity Scorecard evaluates completion status and process maturity for a comprehensive yet practical assessment across three aspects of an effective DRP program – Defining Requirements, Implementation, and Maintenance.

Completion Status: Reflects the progress made with each component of your DRP Program.

Process Maturity: Reflects the consistency and quality of the steps executed to achieve your completion status.

DRP Maturity Assessment: Each component (e.g. BIA) of your DRP Program is evaluated based on completion status and process maturity to provide an accurate holistic assessment. For example, if your BIA completion status is 4 out of 5, but process maturity is a 2, then requirements were not derived from a consistent defined process. The risk is inconsistent application prioritization and misalignment with actual business requirements.

Step 1.3: Identify Applications, Systems, and Dependencies

This step will walk you through the following activities:

• Identify systems, applications, and services, and the business units that use them.
• Document applications, systems, and their dependencies in the DRP Business Impact Analysis Tool.

This step involves the following participants:

• DRP Coordinator
• DRP Team

Results and Insights

• Identify core services and the applications that depend on them.
• Add applications and dependencies to the DRP Business Impact Analysis Tool.

Select 5-10 services to get started on the DRP methodology

1(d) High-level prioritization

Estimated Time: 30 minutes

Working through the planning process the first time can be challenging. If losing momentum is a concern, limit the BIA to a few critical systems to start.

Run this exercise if you need a structured exercise to decide where to focus first and identify the business users you should ask for input on the impact of system downtime.

1. On a whiteboard or flip-chart paper, list business units in a column on the left. List key applications/systems in a row at the top. Draw a grid.
2. At a high level, review how applications are used by each unit. Take notes to keep track of any assumptions you make.
   • Add a ✓ if members of the unit use the application or system.
   • Add an ✱ if members of the unit are heavy users of the application or system and/or use it for time sensitive tasks.
   • Leave the box blank if the app isn’t used by this unit.
3. Use the chart to prioritize systems to include in the BIA (e.g. systems marked with an *) but also include a few less-critical systems to illustrate DRP requirements for a range of systems.

Draw a high-level sketch of your environment

1(e) Sketch your environment

Estimated Time: 1-2 hours

A high-level topology or architectural diagram is an effective way to identify dependencies, application ownership, outsourced services, hardware redundancies, and more.

Note:

• Network diagrams or high-level architecture diagrams help to identify dependencies and redundancies. Even a rough sketch is a useful reference tool for participants, and will be valuable documentation in the final DR plan.
• Keep the drawings tidy. Visualize the final diagram before you start to draw on the whiteboard to help with spacing and placement.
• Collaborate with relevant SMEs to identify dependencies. Keep the drawing high-level.
• Illustrate connections between applications or components with lines. Use color coding to illustrate where applications are hosted (e.g. in-house, at a co-lo, in a cloud or MSP environment).

Document systems and dependencies

Collaborate with system SMEs to identify dependencies for each application or system. Document the dependencies in the DRP Business Impact Analysis Tool (see image below)

• When listing applications, focus on business-facing systems or services that business users will recognize and use terminology they’ll understand.
• Group infrastructure components that support all other services as a single core infrastructure service to simplify dependency mapping (e.g. core router, virtual hosts, ID management, and DNS).
• In general, each data center will have its own core infrastructure components. List each data center separately – especially if different services are hosted at each data center.
• Be specific when documenting dependencies. Use existing asset tracking tables, discovery tools, asset management records, or configuration management tools to identify specific server names.
• Core infrastructure dependencies, such as the network infrastructure, power supply, and centralized storage, will be a common set of dependencies for most applications, so group these into a separate category called “Core Infrastructure” to minimize repetition in your DR planning.
• Document production components in the BIA tool. Capture in-production, redundant components performing the same work on a single dependency line. List standby systems in the notes.

Info-Tech Best Practice

In general, visual documentation is easier to use in a crisis and easier to maintain over time. Use Info-Tech’s research to help build your own visual SOPs.

Document systems and dependencies

1(f) DRP Business Impact Analysis Tool – Record systems and dependencies

Stories from the field: Info-Tech clients find value in Phase 1 in the following ways

An organization uncovers a key dependency that needed to be treated as a Tier 1 system

Reviewing the entire ecosystem for applications identified key dependencies that were previously considered non-critical. For example, a system used to facilitate secure data transfers was identified as a key dependency for payroll and other critical business processes, and elevated to Tier 1.

A picture’s worth a thousand words (and 1600 servers)

Drawing a simple architectural diagram was an invaluable tool to identify key dependencies and critical systems, and to understand how systems and dependencies were interconnected. The drawing was an aha moment for IT and business stakeholders trying to make sense of their 1600-server environment.

Make the case for DRP

A member of the S&P 500 used Info-Tech’s DRP Maturity Scorecard to provide a reliable objective assessment and make the case for improvements to the board of directors.

State government agency initiates a DRP project to complement an existing COOP

Info-Tech's DRP Project Charter enabled the CIO to clarify their DRP project scope and where it fit into their overall COOP. The project charter example provided much of the standard copy – objectives, scope, project roles, methodology, etc. – required to outline the project.

Phase 1: Insights and accomplishments

Created a charter and identified current maturity

Identified systems and dependencies for the BIA

Summary of Accomplishments:

• Created a DRP project charter.
• Completed the DRP Maturity Scorecard and identified current DRP maturity.
• Prioritized applications/systems for a first pass through DR planning.
• Identified dependencies for each application and system.

Up Next: Conduct a BIA to establish recovery requirements

Create a Right-Sized Disaster Recovery Plan

Phase 2

Conduct a BIA to Determine Acceptable RTOs and RPOs

Step 2.1: Define an Objective Impact Scoring Scale

This step will walk you through the following activities:

• Create a scoring scale to measure the business impact of application and system downtime.

This step involves the following participants:

• DRP Coordinator
• DRP Team

Results and Insights

• Use a scoring scale tied to multiple categories of real business impact to develop a more objective assessment of application and system criticality.

Align capabilities to appropriate and acceptable RTOs and RPOs with a BIA

Too many organizations avoid a BIA because they perceive it as onerous or unneeded. A well-managed BIA is straightforward and the benefits are tangible.

A BIA enables you to identify appropriate spend levels, maintain executive support, and prioritize DR planning for a more successful outcome. Info-Tech has found that a BIA has a measurable impact on the organization’s ability to set appropriate objectives and investment goals.

Info-Tech Insight

Business input is important, but don’t let a lack of it delay a draft BIA. Complete a draft based on your knowledge of the business. Create a draft within IT, and use it to get input from business leaders. It’s easier to edit estimates than to start from scratch; even weak estimates are far better than a blank sheet.

Pick impact categories that are relevant to your business to develop a holistic view of business impact

Direct Cost Impact Categories

• Revenue: permanently lost revenue.
  • Example: one third of daily sales are lost due to a website failure.
• Productivity: lost productivity.
  • Example: finance staff can’t work without the accounting system.
• Operating costs: additional operating costs.
  • Example: temporary staff are needed to re-key data.
• Financial penalties: fines/penalties that could be incurred due to downtime.
  • Example: failure to meet contractual service-level agreements (SLAs) for uptime results in financial penalties.

Goodwill, Compliance, and Health and Safety Categories

• Stakeholder goodwill: lost customer, staff, or business partner goodwill due to harm, frustration, etc.
  • Example: customers can’t access needed services because the website is down.
  • Example: a payroll system outage delays paychecks for all staff.
  • Example: suppliers are paid late because the purchasing system is down.
• Compliance, health, and safety:
  • Example: financial system downtime results in a missed tax filing.
  • Example: network downtime disconnects security cameras.

Info-Tech Insight

You don’t have to include every impact category in your BIA. Include categories that could affect your business. Defer or exclude other categories. For example, the bulk of revenue for governmental organizations comes from taxes, which won’t be permanently lost if IT systems fail.

Modify scoring criteria to help you measure the impact of downtime

The scoring scales define different types of business impact (e.g. costs, lost goodwill) using a common four-point scale and 24-hour timeframe to simplify BIA exercises and documentation.

Use the suggestions below as a guide as you modify scoring criteria in the DRP Business Impact Analysis Tool:

• All the direct cost categories (revenue, productivity, operating costs, financial penalties) require the user to define only a maximum value; the tool will populate the rest of the criteria for that category. Use the suggestions below to find the maximum scores for each of the direct cost categories:
  • Revenue: Divide total revenue for the previous year by 365 to estimate daily revenue. Assume this is the most revenue you could lose in a day, and use this number as the top score.
  • Loss of Productivity: Divide fully-loaded labor costs for the organization by 365 to estimate daily productivity costs. Use this as a proxy measure for the work lost if all business stopped for one day.
  • Increased Operating Costs: Isolate this to known additional costs that result from a disruption (e.g. costs for overtime or temporary staff). Estimate the maximum cost for the organization.
  • Financial Penalties: Isolate this to known financial penalties (e.g. due to failure to meet SLAs or compliance requirements). Use the estimated maximum penalty as the highest value on the scale.
• Impact on Goodwill: Use an estimate of the percentage of all stakeholders impacted to assess goodwill impact.
• Impact on Compliance; Impact on Health and Safety: The BIA tool contains default scoring criteria that account for the severity of the impact, the likelihood of occurrence, and in the case of compliance, whether a grace period is available. Use this scale as-is, or adapt this scale to suit your needs.

Modify the default scoring scale in the DRP Business Impact Analysis Tool to reflect your organization

2(a) DRP Business Impact Analysis Tool – Scoring criteria

Step 2.2: Estimate the Impact of Downtime

This step will walk you through the following activities:

• Identify the business impact of service/system/application downtime.

This step involves the following participants:

• DRP Coordinator
• DRP Team
• IT Service SMEs
• Business-Side Technology Owners (optional)

Results and Insights

• Apply the scoring scale to develop a more objective assessment of the business impact of downtime.
• Create criticality tiers based on the business impact of downtime.

Estimate the impact of downtime for each system and application

2(b) Estimate the impact of systems downtime

Estimated Time: 3 hours

On tab 3 of the DRP Business Impact Analysis Tool indicate the costs of downtime, as described below:

1. Have a copy of the “Scoring Criteria” tab available to use as a reference (e.g. printed or on a second display). In tab 3 use the drop-down menu to assign a score of 0 to 4 based on levels of impact defined in the “Scoring Criteria” tab.
2. Work horizontally across all categories for a single system or application. This will familiarize you with your scoring scales for all impact categories, and allow you to modify the scoring scales if needed before you proceed much further.

For example, if a core call center phone system was down:

• Loss of Revenue would be the portion of sales revenue generated through the call center. This might score a 1 or 2 depending on the percent of sales that are processed by the call center.
• The Impact on Customers might be a 2 or 3 depending on the extent that some customers might be using the call center to receive support or purchase new products or services.
• The Legal/Regulatory Compliance and Health or Safety Risk might be a 0, as the call center has no impact in either area.

Add impact scores to the DRP Business Impact Analysis Tool

2(c) DRP Business Impact Analysis Tool

Record business reasons and assumptions that drive BIA scores

2(d) DRP BIA Scoring Context Example

Info-Tech suggests that IT leadership and staff identify the impact of downtime first to create a version that you can then validate with relevant business owners. As you work through the BIA as a team, have a notetaker record assumptions you make to help you explain the results and drive business engagement and feedback.

Some common assumptions:

• You can’t schedule a disaster, so Info-Tech suggests you assume the worst possible timing for downtime. Base the impact of downtime on the worst day for a disaster (e.g. year-end close, payroll run).
• Record assumptions made about who and what are impacted by system downtime.
• Record assumptions made about impact severity.
• If you deviate from the scoring scale, or if a particular impact doesn’t fit well into the defined scoring scale, document the exception.

Use Info-Tech’s DRP BIA Scoring Context Example as a note-taking template.

Info-Tech Insight

You can’t build a perfect scoring scale. It’s fine to make reasonable assumptions based on your judgment and knowledge of the business. Just write down your assumptions. If you don’t write them down, you’ll forget how you arrived at that conclusion.

Assign a criticality rating based on total direct and indirect costs of downtime

2(e) DRP Business Impact Analysis Tool – Assign criticality tiers

Once you’ve finished estimating the impact of downtime, use the following rough guideline to create an initial sort of applications into Tiers 1, 2, and 3.

1. In general, sort applications based on the Total Impact on Goodwill, Compliance, and Safety first.
   • An effective tactic for a quick sort: assign a Tier 1 rating where scores are 50% or more of the highest total score, Tier 2 where scores are between 25% and 50%, and Tier 3 where scores are below 25%. Some organizations will also include a Tier 0 for the highest-scoring systems.
   • Then review and validate these scores and assignments.
2. Next, consider the Total Cost of Downtime.
   • The Total Cost is calculated by the tool based on the Scoring Criteria in tab 2 and the impact scores on tab 3.
   • Decide if the total cost impact justifies increasing the criticality rating (e.g. from Tier 2 to Tier 1 due to high cost impact).
3. Review the assigned impact scores and tiers to check that they’re in alignment. If you need to make an exception, document why. Keep exceptions to a minimum.

Example: Highest total score is 12

Step 2.3: Determine Acceptable RTO/RPO Targets

This step will walk you through the following activities:

• Review the “Debate Space” approach to setting RTO and RPO (recovery targets).
• Set preliminary RTOs and RPOs by criticality tier.

This step involves the following participants:

• DRP Coordinator
• DRP Team

Results and Insights

• Align recovery targets with the business impact of downtime and data loss.

Use the “Debate Space” approach to align RTOs and RPOs with the impact of downtime

The business must validate acceptable and appropriate RTOs and RPOs, but IT can use the guidelines below to set an initial estimate.

Right-size recovery.

A shorter RTO typically requires higher investment. If a short period of downtime has minimal impact, setting a low RTO may not be justifiable. As downtime continues, impact begins to increase exponentially to a point where downtime is intolerable – an acceptable RTO must be shorter than this. Apply the same thinking to RPOs – how much data loss is unnoticeable? How much is intolerable?

The “Debate Space” is between minimal impact and maximum tolerance for downtime.

Estimate appropriate, acceptable RTOs and RPOs for each tier

2(f) Set recovery targets

Estimated Time: 30 minutes

RTO and RPO tiers simplify management by setting similar recovery goals for systems and applications with similar criticality.

Use the “Debate Space” approach to set appropriate and acceptable targets.

1. For RTO, establish a recovery time range that is appropriate based on impact.
   • Overall, the RTO tiers might be 0-4 hours for gold, 4-24 hours for silver, and 24-48 hours for bronze.
2. RPOs reflect target data protection measures.
   • Identify the lowest RPO within a tier and make that the standard.
   • For example, RPO for gold data might be five minutes, silver might be four hours, and bronze might be one day.
   • Use this as a guideline. RPO doesn’t always align perfectly with RTO tiers.
3. Review RTOs and RPOs and make sure they accurately reflect criticality.

Info-Tech Insight

In general, the more critical the system, the shorter the RPO. But that’s not always the case. For example, a service bus might be Tier 1, but if it doesn’t store any data, RPO might be longer than other Tier 1 systems. Some systems may have a different RPO than most other systems in that tier. As long as the targets are acceptable to the business and appropriate given the impact, that’s okay.

Add recovery targets to the DRP Business Impact Analysis Tool

2(g) DRP Business Impact Analysis Tool – Document recovery objectives

Stories from the field: Info-Tech clients find value in Phase 2 in the following ways

Most organizations discover something new about key applications, or the way stakeholders use them, when they work through the BIA and review the results with stakeholders. For example:

Why complete a BIA? There could be a million reasons

• A global manufacturer completed the DRP BIA exercise. When email went down, Service Desk phones lit up until it was resolved. That grief led to a high availability implementation for email. However, the BIA illustrated that ERP downtime was far more impactful.
• ERP downtime would stop production lines, delay customer orders, and ultimately cost the business a million dollars a day.
• The BIA results clearly showed that the ERP needed to be prioritized higher, and required business support for investment.

Move from airing grievances to making informed decisions

The DRP Business Impact Analysis Tool helped structure stakeholder consultations on DR requirements for a large university IT department. Past consultations had become an airing of grievances. Using objective impact scores helped stakeholders stay focused and make informed decisions around appropriate RTOs and RPOs.

Phase 2: Insights and accomplishments

Estimated the business impact of downtime

Set recovery targets

Summary of Accomplishments

• Created a scoring scale tied to different categories of business impact.
• Applied the scoring scale to estimate the business impact of system downtime.
• Identified appropriate, acceptable RTOs and RPOs.

Up Next:Conduct a tabletop planning exercise to establish current recovery capabilities

Create a Right-Sized Disaster Recovery Plan

Phase 3

Identify and Address Gaps in the Recovery Workflow

Step 3.1: Determine Current Recovery Workflow

This step will walk you through the following activities:

• Run a tabletop exercise.
• Outline the steps for the initial response (notification, assessment, disaster declaration) and systems recovery (i.e. document your recovery workflow).
• Identify any gaps and risks in your initial response and systems recovery.

This step involves the following participants:

• DRP Coordinator
• IT Infrastructure SMEs (for systems in scope)
• Application SMEs (for systems in scope)

Results and Insights

• Use a repeatable practical exercise to outline and document the steps you would use to recover systems in the event of a disaster, as well as identify gaps and risks to address.
• This is also a knowledge-sharing opportunity for your team, and a practical means to get their insights, suggestions, and recovery knowledge down on paper.

Tabletop planning: an effective way to test and document your recovery workflow

In a tabletop planning exercise, the DRP team walks through a disaster scenario to map out what should happen at each stage, and effectively defines a high-level incident response plan (i.e. recovery workflow).

Tabletop planning had the greatest impact on meeting recovery objectives (RTOs/RPOs) among survey respondents.

*Note: Relative importance indicates the contribution an individual testing methodology, conducted at least annually, had on predicting success meeting recovery objectives, when controlling for all other types of tests in a regression model. The relative-importance values have been standardized to sum to 100%.

Success was based on the following items:

• RTOs are consistently met.
• IT has confidence in the ongoing ability to meet RTOs.
• RPOs are consistently met.
• IT has confidence in the ongoing ability to meet RPOs.

Why is tabletop planning so effective?

• It enables you to play out a wider range of scenarios than technology-based testing (e.g. full-scale, parallel) due to cost and complexity factors.
• It is non-intrusive, so it can be executed more frequently than other testing methodologies.
• It easily translates into the backbone of your recovery documentation, as it allows you to review all aspects of your recovery plan.

Focus first on IT DR

Your DRP is IT contingency planning. It is not crisis management or BCP.

The goal is to define a plan to restore applications and systems following a disruption. For your first tabletop exercise, Info-Tech recommends you use a non-life-threatening scenario that requires at least a temporary relocation of your data center (i.e. failing over to a DR site/environment). Assume a gas leak or burst water pipe renders the data center inaccessible. Power is shut off and IT must failover systems to another location. Once you create the master procedure, review the plan to ensure it addresses other scenarios.

Info-Tech Insight

When systems fail, you are faced with two high-level options: failover or recover in place. If you document the plan to failover systems to another location, you’ll have documented the core of your DR procedures. This differs from traditional scenario planning where you define separate plans for different what-if scenarios. The goal is one plan that can be adapted to different scenarios, which reduces the effort to build and maintain your DRP.

Conduct a tabletop planning exercise to outline DR procedures in your current environment

3(a) Tabletop planning

Estimated Time: 2-3 hours

For each high-level recovery step, do the following:

1. On white cue cards:
   • Record the step.
   • Indicate the task owner (if required for clarity).
   • Note time required to complete the step. After the exercise, use this to build a running recovery time where 00:00 is when the incident occurred.
2. On yellow cue cards, document gaps in people, process, and technology requirements to complete the step.
3. On red cue cards, indicate risks (e.g. no backup person for a key staff member).

Do:

• Review the complete workflow from notification all the way to user acceptance testing.
• Keep focused; stay on task and on time.
• Revisit each step and record gaps and risks (and known solutions, but don’t dwell on this).
• Revise and improve the plan with task owners.

Don't:

• Get weighed down by tools.
• Document the details right away – stick to the high-level plan for the first exercise.
• Try to find solutions to every gap/risk as you go. Save in-depth research/discussion for later.

Flowchart the current-state incident response plan (i.e. document the recovery workflow)

3(b) DRP Recovery Workflow Template and Case Study: Practical, Right-Sized DRP

Why use flowcharts?

• Flowcharts provide an at-a-glance view, ideal for disaster scenarios where pressure is high and quick upward communication is necessary.
• For experienced staff, a high-level reminder of key steps is sufficient.

Use the completed tabletop planning exercise results to build this workflow.

"We use flowcharts for our declaration procedures. Flowcharts are more effective when you have to explain status and next steps to upper management." – Assistant Director, IT Operations, Healthcare Industry

Source: Info-Tech Research Group Interview

For a formatted template you can use to capture your plan, see Info-Tech’s DRP Recovery Workflow Template.

For a completed example of tabletop planning results, review Info-Tech’s Case Study: Practical, Right-Sized DRP.

Identify RPA

What’s my RPA? Consider the following case:

• Once a week, a full backup is taken of the complete ERP system and is transferred over the WAN to a secondary site 250 miles away, where it is stored on disk.
• Overnight, an incremental backup is taken of the day’s changes, and is transferred to the same secondary site, and also stored on disk.
• During office hours, the SAN takes a snapshot of changes which are kept on local storage (information on the accounting system usually only changes during office hours).
• So what’s the RPA? One hour (snapshots), one day (incrementals), or one week (full backups)?

When identifying RPA, remember the following:

You are planning for a disaster scenario, where on-site systems may be inaccessible and any copies of data taken during the disaster may fail, be corrupt, or never make it out of the data center (e.g. if the network fails before the backup file ships). In the scenario above, it seems likely that off-site incremental backups could be restored, leading to a 24-hour RPA. However, if there were serious concerns about the reliability of the daily incrementals, the RPA could arguably be based on the weekly full backups.

Info-Tech Best Practice

The RPA is a commitment to the maximum data you would lose in a DR scenario with current capabilities (people, process, and technology). Pick a number you can likely achieve. List any situations where you couldn’t meet this RPA, and identify those for a risk tolerance discussion. In the example above, complete loss of the primary SAN would also mean losing the snapshots, so the last good copy of the data could be up to 24-hours old.

Add recovery actuals (RTA/RPA) to your copy of the BIA

3(c) DRP Business Impact Analysis Tool– Recovery actuals

On the “Impact Analysis” tab in the DRP Business Impact Analysis Tool, enter the estimated maximum downtime and data loss in the RTA and RPA columns.

1. Estimate the RTA based on the required time for complete recovery. Review your recovery workflow to identify this timeline. For example, if the notification, assessment, and declaration process takes two hours, and systems recovery requires most of a day, the estimated RTA could be 24 hours.
2. Estimate the RPA based on the longest interval between copies of the data being shipped offsite. For example, if data on a particular system is backed up offsite once per day, and the onsite system was destroyed just before that backup began, the entire day’s data could be lost and estimated RPA could be 24 hours. Note: Enter 9999 to indicate that data is unrecoverable.

Info-Tech Best Practice

It’s okay to round numbers to the nearest shift, day, or week for simplicity (e.g. 24 hours rather than 22.5 hours, or 8 hours rather than 7.25 hours).

Test the recovery workflow against additional scenarios

3(d) Workflow review

Estimated Time: 1 hour

Review your recovery workflow with a different scenario in mind.

• Work from and update the soft copy of your recovery workflow.
• Would any steps be different if the scenario changes? If yes, capture the different flow with a decision diamond. Identify any new gaps or risks you encounter with red and yellow cards. Use as few decision diamonds as possible.

Info-Tech Best Practice

As you start to consider scenarios where injuries or loss of life are a possibility, remember that health and safety risks are the top priority in a crisis. If there’s a fire in the data center, evacuating the building is the first priority, even if that means foregoing a graceful shut down. For more details on emergency response and crisis management, see Implement Crisis Management Best Practices.

Consider additional IT disaster scenarios

3(e) Thought experiment – Review additional scenarios

Walk through your recovery workflow in the context of additional, different scenarios to ensure there are no gaps. Collaborate with your DR team to identify changes that might be required, and incorporate these changes in the plan.

Step 3.2: Identify and Prioritize Projects to Close Gaps

This step will walk you through the following activities:

• Analyze the gaps that were identified from the maturity scorecard, tabletop planning exercise, and the RTO/RPO gaps analysis.
• Brainstorm solutions to close gaps and mitigate risks.
• Determine a course of action to close these gaps. Prioritize each project. Create a project implementation timeline.

This step involves the following participants:

• DRP Coordinator
• IT Infrastructure SMEs

Results and Insights

• Prioritized list of projects and action items that can improve DR capabilities.
• Often low-cost, low-effort quick wins are identified to mitigate at least some gaps/risks. Higher-cost, higher-effort projects can be part of a longer-term IT strategy. Improving service continuity is an ongoing commitment.

Brainstorm solutions to address gaps and risk

3(f) Solutioning

Estimated Time: 1.5 hours

1. Review each of the risk and gap cards from the tabletop exercise.
2. As a group, brainstorm ideas to address gaps, mitigate risks, and improve resiliency. Write the list of ideas on a whiteboard or flip-chart paper. The solutions can range from quick-wins and action items to major capital investments.
3. Try to avoid debates about feasibility at this point – that should happen later. The goal is to get all ideas on the board.

Info-Tech Best Practice

It’s about finding ways to solve the problem, not about solving the problem. When you’re brainstorming solutions to problems, don’t stop with the first idea, even if the solution seems obvious. The first idea isn’t always the best or only solution; other ideas can expand on and improve that first idea.

Select an optimal DR deployment model from a world of choice

There are many options for a DR deployment. What makes sense for you?

• Sifting through the options for a DR site can be overwhelming. Simplify by eliminating deployment models that aren’t a good fit for your requirements or organization using Info-Tech’s research.
• Someone will ask you about DR in the cloud. Cut to the chase and evaluate cloud for fit with your organization’s current capabilities and requirements. Read about the 10 Secrets for Successful DR in the Cloud.
• Selecting and deploying a DR site is an exercise in risk mitigation. IT’s role is to advise the business on options to address the risk of not having a DR site, including cost and effort estimates. The business must then decide how to manage risk. Build total cost of ownership (TCO) estimates and evaluate possible challenges and risks for each option.

Is it practical to invest in greater geo-redundancy that meets RTOs and RPOs during a widespread event?

Info-Tech suggests you consider events that impact both sites, and your risk tolerance for that impact. Outline the impact of downtime at a high level if both the primary and secondary site were affected. Research how often events severe enough to have impacted both your primary and secondary sites have occurred in the past. What’s the business tolerance for this type of event?

A common strategy: have a primary and DR site that are close enough to support low RPO/RTO, but far enough away to mitigate the impact of known regional events. Back up data to a remote third location as protection against a catastrophic event.

Info-Tech Insight

Approach site selection as a project. Leverage Select an Optimal Disaster Recovery Deployment Model to structure your own site-selection project.

Set up the DRP Roadmap Tool

3(g) DRP Roadmap Tool – Set up tool

Use the DRP Roadmap Tool to create a high-level roadmap to plan and communicate DR action items and initiatives. Determine the data you’ll use to define roadmap items.

Plan next steps by estimating timeline, effort, priority, and more

3(h) DRP Roadmap Tool – Describe roadmap items

Review and communicate the DRP Roadmap Tool

3(i) DRP Roadmap Tool – View roadmap chart

Step 3.3: Review the Future State Recovery Process

This step will walk you through the following activities:

• Update the recovery workflow to outline your future recovery procedure.
• Summarize findings from DR exercises and present the results to the project sponsor and other interested executives.

This step involves the following participants:

• DRP Coordinator
• IT SMEs (Future State Recovery Flow)
• DR Project Sponsor

Results and Insights

• Summarize results from DR planning exercises to make the case for needed DR investment.

Outline your future state recovery flow

3(j) Update the recovery workflow to outline response and recovery in the future

Estimated Time: 30 minutes

Outline your expected future state recovery flow to demonstrate improvements once projects and action items have been completed.

1. Create a copy of your DRP recovery workflow in a new tab in Visio.
2. Delete gap and risk cards that are addressed by proposed projects. Consolidate or eliminate steps that would be simplified or streamlined in the future if projects are implemented.
3. Create a short-, medium-, and long-term review of changes to illustrate improvements over time to the project roadmap.
4. Update this workflow as you implement and improve DR capabilities.

Validate recovery targets and communicate actual recovery capabilities

3(k) Validate findings, present recommendations, secure budget

Estimated Time: time required will vary

1. Interview managers or process owners to validate RTO, RPO, and business impact scores.Use your assessment of “heavy users” of particular applications (picture at right) to remind you which business users you should include in the interview process.
2. Present an overview of your findings to the management team.Use Info-Tech’s DRP Recap and Results Template to summarize your findings.
3. Take projects into the budget process.With the management team aware of the rationale for investment in DRP, build the business case and secure budget where needed.

Present DRP findings and make the case for needed investment

3(I) DRP Recap and Results Template

Create a communication deck to recap key findings for stakeholders.

• Write a clear problem statement. Identify why you did this project (what problem you’re solving).
• Clearly state key findings, insights, and recommendations.
• Leverage the completed tools and templates to populate the deck. Callouts throughout the template presentation will direct you to take and populate screenshots throughout the document.
• Use the presentation to communicate key findings to, and gather feedback from, business unit managers, executives, and IT staff.

Stories from the field: Info-Tech clients find value in Phase 3 in the following ways

Tabletop planning is an effective way to discover gaps in recovery capabilities. Identify issues in the tabletop exercise so you can manage them before disaster strikes. For example:

Back up a second…

A client started to back up application data offsite. To minimize data transfer and storage costs, the systems themselves weren’t backed up. Working through the restore process at the DR site, the DBA realized 30 years of COBOL and SQR code – critical business functionality – wasn’t backed up offsite.

Net… work?

A 500-employee professional services firm realized its internet connection could be a significant roadblock to recovery. Without internet, no one at head office could access critical cloud systems. The tabletop exercise identified this recovery bottleneck and helped prioritize the fix on the roadmap.

Someone call a doctor!

Hospitals rely on their phone systems for system downtime procedures. A tabletop exercise with a hospital client highlighted that if the data center were damaged, the phone system would likely be damaged as well. Identifying this provided more urgency to the ongoing VOIP migration.

The test of time

A small municipality relied on a local MSP to perform systems restore, but realized it had never tested the restore procedure to identify RTA. Contacting the MSP to review capabilities became a roadmap item to address this risk.

Phase 3: Insights and accomplishments

Outlined the DRP response and risks to recovery

Brainstormed risk mitigation measures

Summary of Accomplishments

• Planned and documented your DR incident response and systems recovery workflow.
• Identified gaps and risks to recovery and incident management.
• Brainstormed and identified projects and action items to mitigate risks and close gaps.

Up Next: Leverage the core deliverables to complete, extend, and maintain your DRP

Create a Right-Sized Disaster Recovery Plan

Phase 4

Complete, Extend, and Maintain Your DRP

Phase 4: Complete, Extend, and Maintain Your DRP

This phase will walk you through the following activities:

• Identify progress made on your DRP by reassessing your DRP maturity.
• Prioritize the highest value major initiatives to complete, extend, and maintain your DRP.

This phase involves the following participants:

• DRP Coordinator
• Executive Sponsor

Results and Insights

• Communicate the value of your DRP by demonstrating progress against items in the DRP Maturity Scorecard.
• Identify and prioritize future major initiatives to support the DRP, and the larger BCP.

Celebrate accomplishments, plan for the future

Congratulations! You’ve completed the core DRP deliverables and made the case for investment in DR capabilities. Take a moment to celebrate your accomplishments.

This milestone is an opportunity to look back and look forward.

• Look back: measure your progress since you started to build your DRP. Revisit the assessments completed in phase 1, and assess the change in your overall DRP maturity.
• Look forward: prioritize future initiatives to complete, extend, and maintain your DRP. Prioritize initiatives that are the highest impact for the least requirement of effort and resources.

We have completed the core DRP methodology for key systems:

• BIA, recovery objectives, high-level recovery workflow, and recovery actuals.
• Identify key tasks to meet recovery objectives.

What could we do next?

• Repeat the core methodology for additional systems.
• Identify a DR site to meet recovery requirements, and review vendor DR capabilities.
• Create a summary DRP document including requirements, capabilities, and change procedures.
• Create a test plan and detailed recovery documentation.
• Coordinate the creation of BCPs.
• Integrate DR in other key operational processes.

Revisit the DRP Maturity Scorecard to measure progress and identify remaining areas to improve

4(a) DRP Maturity Scorecard – Reassess your DRP program maturity

1. Find the copy of the DRP Maturity Scorecard you completed previously. Save a second copy of the completed scorecard in the same folder.
2. Update scoring where you have improved your DRP documentation or capabilities.
3. Review the new scores on tab 3. Compare the new scores to the original scores.

Info-Tech Best Practice

Use the completed, updated DRP Maturity Scorecard to demonstrate the value of your continuity program, and to help you decide where to focus next.

Prioritize major initiatives to complete, extend, and maintain the DRP

4(b) Prioritize major initiatives

Estimated Time: 2 hours

Prioritize major initiatives that mitigate significant risk with the least cost and effort.

1. Use the scoring criteria below to evaluate risk, effort, and cost for potential initiatives. Modify the criteria if required for your organization. Write this out on a whiteboard or flip-chart paper.
2. Assign a score from 1 to 3. Multiply the scores for each initiative together for an aggregate score. In general, prioritize initiatives with higher scores.

Info-Tech Best Practice

You can use a similar scoring exercise to prioritize and schedule high-benefit, low-effort, low-cost items identified in the roadmap in phase 3.

Example: Prioritize major initiatives

4(b) Prioritize major initiatives continued

Write out the table on a whiteboard (record the results in a spreadsheet for reference). In the case below, IT might decide to work on repeating the core methodology first as they create the active testing plans, and tackle process changes later.

Info-Tech Best Practice

Find a pace that allows you to keep momentum going, but also leaves enough time to act on the initial findings, projects, and action items identified in the DRP Roadmap Tool. Include these initiatives in the Roadmap tool to visualize how identified initiatives fit with other tasks identified to improve your recovery capabilities.

Repeat the core DR methodology for additional systems and applications

You have created a DR plan for your most critical systems. Now, add the rest:

• Build on the work you’ve already done. Re-use the BIA scoring scale. Update your existing recovery workflows, rather than creating and formatting an entirely new document. A number of steps in the recovery will be shared with, or similar to, the recovery procedures for your Tier 1 systems.

Risks and Challenges Mitigated

• DR requirements and capabilities for less-critical systems have not been evaluated.
• Gaps in the recovery process for less critical systems have not been evaluated or addressed.
• DR capabilities for less critical systems may not meet business requirements.

Info-Tech Best Practice

Use this example of a complete, practical, right-size DR plan to drive and guide your efforts.

Extend your core DRP deliverables

You’ve completed the core DRP deliverables. Continue to create DRP documentation to support recovery procedures and governance processes:

• DR documentation efforts fail when organizations try to boil the ocean with an all-in-one plan aimed at auditors, business leaders, and IT. It’s long, hard to maintain, and ends up as shelfware.
• Create documentation in layers to keep it manageable. Build supporting documentation over time to support your high-level recovery workflow.

Risks and Challenges Mitigated

• Key contact information, escalation, and disaster declaration responsibilities are not identified or formalized.
• DRP requirements and capabilities aren’t centralized. Key DRP findings are in multiple documents, complicating governance and oversight by auditors, executives, and board members.
• Detailed recovery procedures and peripheral information (e.g. network diagrams) are not documented.

Info-Tech Best Practice

Use this example of a complete, practical, right-size DR plan to drive and guide your efforts.

Select an optimal DR deployment model and deployment site

Your DR site has been identified as inadequate:

• Begin with the end in mind. Commit to mastering the selected model and leverage your vendor relationship for effective DR.
• Cut to the chase and evaluate the feasibility of cloud first. Gauge your organization’s current capabilities for DR in the cloud before becoming infatuated with the idea.
• A mixed model gives you the best of both worlds. Diversify your strategy by identifying fit for purpose and balancing the work required to maintain various models.

Risks and Challenges Mitigated

• Without an identified DR site, you’ll be scrambling when a disaster hits to find and contract for a location to restore IT services.
• Without systems and application data backed up offsite, you stand to lose critical business data and logic if all copies of the data at your primary site were lost.

Info-Tech Best Practice

Use Info-Tech’s blueprint, Select the Optimal Disaster Recovery Deployment Model, to help you make sense of a world of choice for your DR site.

Extend DRP findings to business process resiliency with a BCP pilot

Integrate your findings from DRP into the overall BCP:

• As an IT leader you have the skillset and organizational knowledge to lead a BCP project, but ultimately business leaders need to own the BCP – they know their processes and requirements to resume business operations better than anyone else.
• The traditional approach to BCP is a massive project that most organizations can’t execute without hiring a consultant. To execute BCP in-house, carve up the task into manageable pieces.

Risks and Challenges Mitigated

• No formal plan exists to recover from a disruption to critical business processes.
• Business requirements for IT systems recovery may change following a comprehensive review of business continuity requirements.
• Outside of core systems recovery, IT could be involved in relocating staff, imaging and issuing new end-user equipment, etc. Identifying these requirements is part of BCP.

Info-Tech Best Practice

Use Info-Tech’s blueprint, Develop a Business Continuity Plan, to develop and deploy a repeatable BCP methodology.

Test the plan to validate capabilities and cross-train staff on recovery procedures

You don’t have a program to regularly test the DR plan:

• Most DR tests are focused solely on the technology and not the DR management process – which is where most plans fail.
• Be proactive – establish an annual test cycle and identify and coordinate resources well in advance.
• Update DRP documentation with findings from the plan, and track the changes you make over time.

Risks and Challenges Mitigated

• Gaps likely still exist in the plan that are hard to find without some form of testing.
• Customers and auditors may ask for some form of DR testing.
• Staff may not be familiar with DR documentation or how they can use it.
• No formal cycle to validate and update the DRP.

Info-Tech Best Practice

Uncover deficiencies in your recovery procedures by using Info-Tech’s blueprint Reduce Costly Downtime Through DR Testing.

“Operationalize” DRP management

Inject DR planning in key operational processes to support plan maintenance:

• Major changes, or multiple routine changes, can materially alter DR capabilities and requirements. It’s not feasible to update the DR plan after every routine change, so leverage criticality tiers in the BIA to focus your change management efforts. Critical systems require more rigorous change procedures.
• Likewise, you can build criticality tiers into more focused project management and performance measurement processes.
• Schedule regular tasks in your ticketing system to verify capabilities and cross-train staff on key recovery procedures (e.g. backup and restore).

Risks and Challenges Mitigated

• DRP is not updated “as needed” – as requirements and capabilities change due to business and technology changes.
• The DRP is disconnected from day-to-day operations.

Review infrastructure service provider DR capabilities

Insert DR planning in key operational processes to support plan maintenance:

• Reviewing vendor DR capabilities is a core IT vendor management competency.
• As your DR requirements change year-to-year, ensure your vendors’ service commitments still meet your DR requirements.
• Identify changes in the vendor’s service offerings and DR capabilities, e.g. higher costs for additional DR support, new offerings to reduce potential downtime, or conversely, a degradation in DR capabilities.

Risks and Challenges Mitigated

• Vendor capabilities haven’t been measured against business requirements.
• No internal capability exists currently to assess vendor ability to meet promised SLAs.
• No internal capability exists to track vendor performance on recoverability.

Phase 4: Insights and accomplishments

Identified progress against targets

Prioritized further initiatives

Added initiatives to the roadmap

Summary of Accomplishments

• Developed a list of high-priority initiatives that can support the extension and maintenance of the DR plan over the long term.
• Reviewed and update maturity assessments to establish progress and communicate the value of the DR program.

Summary of accomplishment

Knowledge Gained

• Conduct a BIA to determine appropriate targets for RTOs and RPOs.
• Identify DR projects required to close RTO/RPO gaps and mitigate risks.
• Use tabletop planning to create and validate an incident response plan.

Processes Optimized

• Your DRP process was optimized, from BIA to documenting an incident response plan.
• Your vendor evaluation process was optimized to identify and assess a vendor’s ability to meet your DR requirements, and to repeat this evaluation on an annual basis.

Deliverables Completed

• DRP Maturity Scorecard
• DRP Business Impact Analysis Tool
• DRP Roadmap Tool
• Incident response plan and systems recovery workflow
• Executive presentation

Info-Tech’s insights bust the most obstinate myths of DRP

Myth #1: DRPs need to focus on major events such as natural disasters and other highly destructive incidents such as fire and flood.

Reality: The most common threats to service continuity are hardware and software failures, network outages, and power outages.

Myth #2: Effective DRPs start with identifying and evaluating potential risks.

Reality: DR isn’t about identifying risks; it’s about ensuring service continuity.

Myth #3: DRPs are separate from day-to-day operations and incident management.

Reality: DR must be integrated with service management to ensure service continuity.

Myth #4: I use a co-lo or cloud services so I don’t have to worry about DR. That’s my vendor’s responsibility.

Reality: You can’t outsource accountability. You can’t just assume your vendor’s DR capabilities will meet your needs.

Myth #5: A DRP must include every detail so anyone can execute the recovery.

Reality: IT DR is not an airplane disaster movie. You aren’t going to ask a business user to execute a system recovery, just like you wouldn’t really want a passenger with no flying experience to land a plane.

Supplement the core documentation with these tools and templates

• An Excel workbook workbook to track key roles on DR, business continuity, and emergency response teams. Can also track DR documentation location and any hardware purchases required for DR.
• A questionnaire template and a response tracking tool to structure your investigation of vendor DR capabilities.
• Integrate escalation with your DR plan by defining incident severity and escalation rules . Use this example as a template or integrate ideas into your own severity definitions and escalation rules in your incident management procedures.
• A minute-by-minute time-tracking tool to capture progress in a DR or testing scenario. Monitor progress against objectives in real time as recovery tasks are started and completed.

Next steps: Related Info-Tech research

Select the Optimal Disaster Recovery Deployment Model Evaluate cloud, co-lo, and on-premises disaster recovery deployment models.

Develop a Business Continuity Plan Streamline the traditional approach to make BCP development manageable and repeatable.

Prepare for a DRP Audit Assess your current DRP maturity, identify required improvements, and complete an audit-ready DRP summary document.

Document and Maintain Your Disaster Recovery Plan Put your DRP on a diet: keep it fit, trim, and ready for action.

Reduce Costly Downtime Through DR Testing Improve your DR plan and your team’s ability to execute on it.

Implement Crisis Management Best Practices An effective crisis response minimizes the impact of a crisis on reputation, profitability, and continuity.

Research contributors and experts

• Alan Byrum, Director of Business Continuity, Intellitech
• Bernard Jones (MBCI, CBCP, CORP, ITILv3), Owner/Principal, B Jones BCP Consulting, LLC
• Paul Beaudry, Assistant Vice-President, Technical Services, MIS, Richardson International Limited
• Yogi Schulz, President, Corvelle Consulting

Glossary

• Business Continuity Management (BCM) Program: Ongoing management and governance process supported by top management and appropriately resourced to implement and maintain business continuity management. (Source: ISO 22301:2012)
• Business Continuity Plan (BCP): Documented procedures that guide organizations to respond, recover, resume, and restore to a pre-defined level of operation following disruption. The BCP is not necessarily one document, but a collection of procedures and information.
• Crisis: A situation with a high level of uncertainty that disrupts the core activities and/or credibility of an organization and requires urgent action. (Source: ISO 22300)
• Crisis Management Team (CMT): A group of individuals responsible for developing and implementing a comprehensive plan for responding to a disruptive incident. The team consists of a core group of decision makers trained in incident management and prepared to respond to any situation.
• Disaster Recovery Planning (DRP): The activities associated with the continuing availability and restoration of the IT infrastructure.
• Incident: An event that has the capacity to lead to loss of, or a disruption to, an organization’s operations, services, or functions – which, if not managed, can escalate into an emergency, crisis, or disaster.

BCI Editor’s Note: In most countries “incident” and “crisis” are used interchangeably, but in the UK the term “crisis” has been generally reserved for dealing with wide-area incidents involving Emergency Services. The BCI prefers the use of “incident” for normal BCM purposes. (Source: The Business Continuity Institute)

• Incident Management Plan: A clearly defined and documented plan of action for use at the time of an incident, typically covering the key personnel, resources, services, and actions needed to implement the incident management process.
• IT Disaster: A service interruption requiring IT to rebuild a service, restore from backups, or activate redundancy at the backup site.
• Recovery Point: Time elapsed between the last good copy of the data being taken and failure/corruption on the production environment; think of this as data loss.
• Recovery Point Actual (RPA): The currently achievable recovery point after a disaster event, given existing people, processes, and technology. This reflects expected maximum data loss that could actually occur in a disaster scenario.
• Recovery Point Objective (RPO): The target recovery point after a disaster event, usually calculated in hours, on a given system, application, or service. Think of this as acceptable and appropriate data loss. RPO should be based on a business impact analysis (BIA) to identify an acceptable and appropriate recovery target.
• Recovery Time: Time required to restore a system, application, or service to a functional state; think of this as downtime.
• Recovery Time Actual (RTA): The currently achievable recovery time after a disaster event, given existing people, processes, and technology. This reflects expected maximum downtime that could actually occur in a disaster scenario.
• Recovery Time Objective (RTO): The target recovery time after a disaster event for a given system, application, or service. RTO should be based on a business impact analysis (BIA) to identify acceptable and appropriate downtime.

Bibliography

BCMpedia. “Recovery Objectives: RTO, RPO, and MTPD.” BCMpedia, n.d. Web.

Burke, Stephen. “Public Cloud Pitfalls: Microsoft Azure Storage Cluster Loses Power, Puts Spotlight On Private, Hybrid Cloud Advantages.” CRN, 16 Mar. 2017. Web.

Elliot, Stephen. “DevOps and the Cost of Downtime: Fortune 1000 Best Practice Metrics Quantified.” IDC, 2015. Web.

FEMA. Planning & Templates. FEMA, 2015. Web.

FINRA. “Business Continuity Plans and Emergency Contact Information.” FINRA, 2015. Web.

FINRA. “FINRA, the SEC and CFTC Issue Joint Advisory on Business Continuity Planning.” FINRA, 2013. Web.

Gosling, Mel, and Andrew Hiles. “Business Continuity Statistics: Where Myth Meets Fact.” Continuity Central, 2009. Web.

Hanwacker, Linda. “COOP Templates for Success Workbook.” The LSH Group, n.d. Web.

Homeland Security. Federal Information Security Management Act (FISMA). Homeland Security, 2015. Web.

Nichols, Shaun. “AWS's S3 Outage Was So Bad Amazon Couldn't Get Into Its Own Dashboard to Warn the World.” The Register, 1 Mar. 2017. Web.

Potter, Patrick. “BCM Regulatory Alphabet Soup.” RSA Archer Organization, 2012. Web.

Rothstein, Philip Jan. “Disaster Recovery Testing: Exercising Your Contingency Plan.” Rothstein Associates Inc., 2007. Web.

The Business Continuity Institute. “The Good Practice Guidelines.” The Business Continuity Institute, 2013. Web.

The Disaster Recovery Journal. “Disaster Resource Guide.” The Disaster Recovery Journal, 2015. Web.

The Disaster Recovery Journal. “DR Rules & Regulations.” The Disaster Recovery Journal, 2015. Web.

The Federal Financial Institution Examination Council (FFIEC). Business Continuity Planning. IT Examination Handbook InfoBase, 2015. Web.

York, Kyle. “Read Dyn’s Statement on the 10/21/2016 DNS DDoS Attack.” Oracle, 22 Oct. 2016. Web.

Build a Data Integration Strategy

Integrate your data or disintegrate your business.

ANALYST PERSPECTIVE

Integrate your data or disintegrate your business.

"Point-to-point integration is an evil that builds up overtime due to ongoing business changes and a lack of integration strategy. At the same time most businesses are demanding consistent, timely, and high-quality data to fuel business processes and decision making.

A good recipe for successful data integration is to discover the common data elements to share across the business by establishing an integration platform and a canonical data model.

Place yourself in one of our use cases and see how you fit into a common framework to simplify your problem and build a data-centric integration environment to eliminate your data silos."

Rajesh Parab, Director, Research & Advisory Services

Info-Tech Research Group

Our understanding of the problem

This Research Is Designed For:

• Data engineers feeling the pains of poor integration from inaccuracies and inefficiencies during the data integration lifecycle.
• Business analysts communicating the need for improved integration of data.
• Data architects looking to design and facilitate improvements in the holistic data environment.
• Data architects putting high-level architectural design changes into action.

This Research Will Also Assist:

• CIOs concerned with the costs, benefits, and the overall structure of their organization’s data flow.
• Enterprise architects trying to understand how improved integration will affect overall organizational architecture.

This Research Will Help You:

• Understand what integration is, and how it fits into your organization.
• Identify opportunities for leveraging improved integration for data-driven insights.
• Design a loosely coupled integration architecture that is flexible to changing needs.
• Determine the needs of the business for integration and design solutions for the gaps that fit the requirements.

This Research Will Help Them:

• Get a handle on the current data situation and how data interacts within the organization.
• Understand how data architecture affects operations within the enterprise.

Executive summary

Situation

• As organizations process more information at faster rates, there is increased pressure for faster and more efficient data integration.
• Data integration is becoming more and more critical for downstream functions of data management and for business operations to be successful. Poor integration holds back these critical functions.

Complication

• Investments in integration can be a tough sell for the business, and it is difficult to get support for integration as a standalone project.
• Evolving business models and uses of data are growing rapidly at rates that often exceed the investment in data management and integration tools. As a result, there is often a gap between data availability and the business’ latency demands.

Resolution

• Create a data-centric integration solution that supports the flow of data through the organization and meets the organization’s requirements for data accuracy, relevance, availability, and timeliness.
• Build your data-centric integration practice with a firm foundation in governance and reference architecture; use best-fit reference architecture patterns and the related technology and resources to ensure that your process is scalable and sustainable.
• The business’ uses of data are constantly changing and evolving, and as a result the integration processes that ensure data availability must be frequently reviewed and repositioned to continue to grow with the business.

Info-Tech Insight

1. Every IT project requires data integration.Any change in the application and database ecosystem requires you to solve a data integration problem.
2. Integration problem solving needs to start with business activity. After understanding the business activity, move to application and system integration to drive optimal data integration activities.
3. Integration initiatives need to be backed by requirements that depend on use cases. Info-Tech’s use cases will help identify organizational requirements and the ideal data-centric integration solution.

Your data is the foundation of your organization’s knowledge and ability to make decisions

Integrate the Data, Not the Applications

Data is one of the most important assets in a modern organization. Contained within an organization’s data are the customers, the products, and the operational details that make an organization function. Every organization has data, and this data might serve the needs of the business today.

However, the only constant in the world is change. Changes in addresses, amounts, product details, partners, and more occur at a rapid rate. If your data is isolated, it will quickly become stale. Getting up-to-date data to the right place at the right time is where data-centric integration comes in.

"Data is the new oil." – Clive Humby, Chief Data Scientist Source: Medium, 2016

Organizations are having trouble keeping up with the rapid increases in data growth and complexity

To keep up with increasing business demands and profitability targets and decreasing cost targets, organizations are processing and exchanging more data than ever before.

To get more value from their information, organizations are relying on more and more complex data sources. These diverse data sources have to be properly integrated to unlock the full potential of your data:

The most difficult integration problems are caused by semantic heterogeneity (Database Research Technology Group, n.d.).

80% of business decisions are made using unstructured data (Concept Searching, 2015).

85% of businesses are struggling to implement the correct integration solution to accurately interpret their data (KPMG, 2014).

Break Down Your Silos

Integrating large volumes of data from the many varied sources in an organization has incredible potential to yield insights, but many organizations struggle with creating the right structure for that blending to take place, and data silos form.

Data-centric integration capabilities can break down organizational silos. Once data silos are removed and all the information that is relevant to a given problem is available, problems with operational and transactional efficiencies can be solved, and value from business intelligence (BI) and analytics can be fully realized.

Data-centric integration is the solution you need to bring data together to break down data silos

On one hand…

Data has massive potential to bring insight to an organization when combined and analyzed in creative ways.

On the other hand…

It is difficult to bring data together from different sources to generate insights and prevent stale data.

How can these two ideas be reconciled?

Answer: Info-Tech’s Data Integration Onion Framework summarizes an organization’s data environment at a conceptual level, and is used to design a common data-centric integration environment.

Info-Tech’s Data Integration Onion Framework

Poor integration will lead to problems felt by the business and IT

The following are pains reported by the business due to poor integration:

59% Of managers said they experience missing data every day due to poor distribution results in data sets that are valuable to their central work functions. (Experian, 2016)

42% Reported accidentally using the wrong information, at least once a week. (Computerworld, 2017)

37% Of the 85% of companies trying to be more data driven, only 37% achieved their goal. (Information Age, 2019)

"I never guess. It is a capital mistake to theorize before one has data. Insensibly one begins to twist facts to suit theories, instead of theories to suit facts." – Sir Arthur Conan Doyle, Sherlock Holmes

Poor integration can make IT less efficient as well:

90% Of all company generated data is “dark.” Getting value out of dark data is not difficult or costly. (Deloitte Insights, 2017)

5% As data sits in a database, up to 5% of customer data changes per month. (Data.com, 2016)

"Most traditional machine learning techniques are not inherently efficient or scalable enough to handle the data. Machine learning needs to reinvent itself for big data processing primarily in pre-processing of data." – J. Qiu et al., ‎2016

Understand the common challenges of integration to avoid the pains

There are three types of challenges that organizations face when integrating data:

1. Disconnect from the business

Poor understanding of the integration problem and requirements lead to integrations being built that are not effective for quality data.

50% of project rework is attributable to problems with requirements. (Info-Tech Research Group)

45% of IT professionals admit to being “fuzzy” about the details of a project’s business objectives. (Blueprint Software Systems Inc., 2012)

2. Lack of strategy

90% Of organizations will lack an integration strategy through to 2018. (Virtual Logistics, 2017)

Integrating data without a long-term plan is a recipe for point-to-point integration spaghettification:

3. Data complexity

Data architects and other data professionals are increasingly expected to be able to connect data using whatever interface is provided, at any volume, and in any format – all without affecting the quality of the data.

36% Of developers report problems integrating data due to different standards interpretations. (DZone, 2015)

These challenges lead to organizations building a data architecture and integration environment that is tightly coupled.

A loose coupling integration strategy helps mitigate the challenges and realize the benefits of well-connected data

Loose Coupling

Most organizations don’t have the foresight to design their architecture correctly the first time. In a perfect world, organizations would design their application and data architecture to be scalable, modular, and format-neutral – like building blocks.

Benefits of a loosely coupled architecture:

• Increased ability to support business needs by adapting easily to changes.
• Added ability to incorporate new vendors and new technology due to increased flexibility.
• Potential for automated, real-time integration.
• Elimination of re-keying/manual entry of data.
• Federation of data.

Vs. Tight Coupling

However, this is rarely the case. Most architectures are more like a brick wall – permanent, hard to add to and subtract from, and susceptible to weathering.

Problems with a tightly coupled architecture:

• Delays in combining data for analysis.
• Manual/Suboptimal DI in the face of changing business needs.
• Lack of federation.
• Lack of flexibility.
• Fragility of integrated platforms.
• Limited ability to explore new functionalities.

Create and Implement an IoT Strategy

Gain control of your IoT environment

Create and Implement an IoT Strategy

Gain control of your IoT environment

EXECUTIVE BRIEF

Table of Contents

Analyst perspective

IoT is an extremely efficient automated data collection system which produces millions of pieces of data. Many organizations will purchase point solutions to help with their primary business function to increase efficiency, increase profitability, and most importantly provide scalable services that cannot exist without automated data collection and analytical tools.

Most of the solutions available are designed to perform a specific function within the parameters of the devices and applications designed by vendors. As these specific use cases proliferate within any organization, the data collected can end up housed in many places, owned by each specific business unit and used only for the originally designed purpose. Imagine though, if you could take the health information of many patients, anonymize it, and compare overall health of specific regions, rather than focusing only on the patient record as a correlated point; or many data points within cities to look at pedestrian, bike, and vehicle traffic to better plan infrastructure changes, improve city plans, and monitor pollution, then compared to other cities for additional modeling.

In order to make these dramatic shifts to using many IoT solutions, it’s time to look at creating an IoT strategy that will ensure all systems meet strategic goals and will enable disparate data to be aggregated for greater insights. The act of aggregation of systems and data will require additional scrutiny to mitigate the potential perils for privacy, management, security, and auditability

The strategy identifies who stewards use of the data, who manages devices, and how IT enables broader use of this technology. But with the increased volume of devices and data, operational efficiency as part of the strategy will also be critical to success.

This project takes you through the process of defining vision and governance, creating a process for evaluating proposed solutions for proof of value, and implementing operational effectiveness.

Sandi Conrad Principal Research Director Info-Tech Research Group

Executive Summary

Early intervention will improve results. IoT is one of the biggest challenges for IT departments to manage today. The large volume of devices and lack of insight into vendor solutions is making it significantly harder to plan for upgrades and contract renewals, and to guarantee security protocols are being met. Create a multistep onboarding process, starting with an initial assessment process to increase success for the business, then look to derive additional benefits to the business and mitigate risks.

Your challenge

Point solutions may be installed and configured with support outsourced to vendors, where integrations may be light or non-existent. Each point solution will be owned by the business, with data used for a specific purpose, and may only require infrastructure support from the internal IT department. Operational needs must be met to protect the business’ investment, and without involving IT early, agreements may be signed that don’t meet long-term goals of high value at reasonable prices. To fully realize value from multiple disparate systems, a cohesive strategy to bring together data will be required, but with that comes a need to improve technology, determine data ownership, and improve oversight with strengthened security, privacy, and communications. Where IoT is becoming a major source of data, taking a piecemeal approach will no longer be enough to be successful. IoT solutions may be chosen by the business, but to be successful and meet their requirements, a partnership with IT will ensure better communications with the service provider for a less stressful implementation with governance over security needs and protection of the organization’s data, and it will ensure that continual value is enabled through effective operations.

(Source: Beecham Research qtd. in Software AG)

Common obstacles

These barriers make IoT challenging to implement for many organizations: Solutions managed outside of IT, whether through an operational technology team or an outsourced vender, will require a comprehensive approach that encourages collaboration, common understandings of risk, and the ability to embrace change. Technical expertise required will be broad and deep for a multi-solution implementation. Many types of devices, with varied connections and communications methods, will need to be architected with flexibility to accommodate changing technology and scalability needs. Understanding the myriad options available and where it makes sense to deploy cutting-edge vs. proven technologies, as well as edge computing and digital twins. External consultants specializing in IoT may need to be engaged to make these complex solutions successful, and they also need to be skilled in facilitating discussions within teams to bring them to a common understanding. Analysis skills and a data strategy will be key to successfully correlating data from multiple sources, and AI will be key to making sense of vast amounts of data available and be able to use it for predictive work. According to the Microsoft IoT Signals report of October 2020, “79% of organizations adopt AI as part of their IoT solution, and those who do perceive IoT to be more critical to their company’s success (95% vs. 82%) and are more satisfied with IoT (96% vs. 87%).“

(Source: Microsoft IoT Signals, Edition 2, October 2020 n=3,000)

Internet of Things Framework

GOVERNANCE

What should I build? What are my concerns?
Where should I build it? Why does it need to be built?

CONTROLS

How do I operate and maintain it?

BRIDGING THE PHYSICAL WORLD AND THE VIRTUAL WORLD

How should it be built?

Data Normalization' from physical to virtual and 'Instructions' from virtual to physical.">

Insight summary

Real value to the business will come from insights derived from data

Many point solutions will solve many business issues and produce many data sets. Ensure your strategy includes plans on how to leverage data to further your organizational goals. A data specialist will make a significant difference in helping you determine how best to aggregate and analyze data to meet those needs.

Provide the right level of oversight to help the business adopt IoT

Regardless of who is initiating the request or installing the solution, it’s critical to have a framework that protects the organization and their data and a plan for managing the devices.

The business doesn’t always know what questions to ask, so it’s important for IT to enable them if moving to a business-led innovation model, and it’s critical to helping them achieve business value early.

Do a pre-implementation assessment to engage early and at the right level

Many IoT solutions are business- and vendor-led and are hosted outside of the organization or managed inside the business unit.

Having IT engage early allows the business to determine what level of support is appropriate for them, allows IT to ensure data integrity, and allows IT to ensure that security, privacy, and long-term operational needs are managed appropriately.

Blueprint deliverables

Blueprint benefits

Evaluate digital transformation opportunities with these guiding principles for smart solutions

Info-Tech Insight

When looking for a quick win, consider customer journey mapping exercises to find out what it takes to do the work today, for example, map the journey to apply for a building permit, renew a license, or register a patient.

Measure the value of IoT

There is a broad range of solutions for IoT all designed to collect information and execute actions in a way designed to increase profitability and/or improve services. McKinsey estimates value created through interoperability will account for 40% to 60% of the potential value of IoT applications.

Info-Tech offers various levels of support to best suit your needs

Guided Implementation

A Guided Implementation (GI) is a series of calls with an Info-Tech analyst to help implement our best practices in your organization.

A typical GI is between 4 to 8 calls over the course of 2 to 4 months.

What does a typical GI on this topic look like?

Create and Implement an IoT Strategy

Phase 1

Define your governance process

This phase will provide the following activities

• Create the steering committee project charter

Assemble the right team to ensure the success of your IoT ecosystem

Business stakeholders will provide clarity for their strategy and provide input into how they envision IoT solutions furthering those goals and how they may gain relevant insights from secondary data. As IoT solutions move beyond their primary goals, it will be critical to evaluate the continually increasing data to mitigate risks of unintended consequences as new data sets converge. The security team will need to evaluate solutions and enforce standards. CDO and analysts will assess opportunities for data convergence to create new insights into how your services are used.

IT stakeholders will be driving these projects forward and ensuring all necessary resources are available and funded. Operational plans will include asset management, monitoring, and support to meet functional goals and manage throughout the asset lifecycle. Each solution added to the environment will need to be chosen and architected to meet primary functions and secondary data collection.

Identify IoT steering committee participants to ensure broad assessment capabilities are available

The committee should include team members experienced enough to provide an effective assessment of IoT projects, and to provide input and oversight regarding business value, privacy, security, operational support, infrastructure, and architectural support. A data specialist will be critical for evaluating opportunities to expand use of data and ensure data can be effectively validated and aggregated. Additional oversight will be needed to review aggregated data to protect against the unintended consequences of having data combined and creating personas that will identify individuals. Additional experts may be invited to committee meetings as appropriate, and ideas should be discussed and clarified with the business unit bringing the ideas forward or that may be impacted by solutions. Invite appropriate IT and business leaders to the initial meeting to gain agreement and form the governance model.

Determine responsibilities of the committee to gain consensus and universal understanding

	STRATEGIC ALIGNMENT	Define the IoT vision in alignment with the organizational strategy and mission. Define strategy, policies and communication requirements for IoT projects. Assess and bring forward proposals to utilize IoT to further organizational strategy.
	VALUE DELIVERY	Define criteria for evaluating and prioritizing proposals and projects. Validate the IoT proposals to ensure value drivers are understood and achievable. Identify opportunities to combine data sets for secondary analysis and insights.
	RISK OPTIMIZATION	Evaluate data and combined data sets to avoid unintended consequences. Ensure security standards are adhered to when integrating new solutions. Reinforce privacy regulations, policy, and communications requirements.
	RESOURCE OPTIMIZATION	Identify and validate investment and resource requirements. Evaluate technical requirements and capabilities. Align IoT management requirements to operations goals within IT.
	PERFORMANCE MANAGEMENT	Assess validity of pilot project plan, including success criteria. Identify corner cases to assess functionality and potential risks beyond core features. Monitor progress, evaluate results, and ensure organizational needs will be met. Evaluate pilot to determine if it will be moved into full production, reworked, or rejected.

1.1 Exercise:
Define the committee’s roles & responsibilities in the IoT steering committee charter

Input: Current policies and assessment tools for security and privacy, Current IT strategy for introducing new solutions and setting standards

Output: List of roles and responsibilities, High-level discussion points

Materials: Whiteboard/flip charts, Steering committee workbook

Participants: IT executive, Privacy & Security senior staff, Infrastructure & Operations senior staff, Senior data specialist, Senior business executive(s)

1. Identify and document core and auxiliary members of the committee, ensuring all important facets of the IoT environment can be assessed.
2. Identify and document the committee chair.
3. Gain consensus on responsibilities of the steering committee.

Download the IoT Steering Committee Charter

1.2 Exercise:
Define the IoT steering committee’s vision statement and mandate

1 hour

Input: Organizational vision and IT strategy

Output: Vision statement

Materials: Whiteboard/flip charts, Steering committee workbook

Participants: Steering committee, which may include: IT executive, Privacy & Security senior staff, Infrastructure & Operations senior staff, Senior data specialist, Senior business executive(s)

1. Starting with the organizational mission statement, brainstorm areas of focus with the steering committee and narrow down the statement.
2. Make sure it’s broad enough to encompass your goals, but succinct enough to allow you to identify projects that don’t meet the vision.
3. Test with a few existing ideas.
4. Document in your steering committee charter.

Download the IoT Steering Committee Charter

Use the COPIS methodology to define your project review process

Customer Executive leadership Business leaders	Outputs Risk assessment Approvals to proceed Pilot plan Assessment to approve for production or reject	Process Review proposals Ask questions and discuss with proposer & committee Review pilot & testing plan Engage with IT Team to define requirements	Inputs Request form including: New idea Business value defined Data collected Initial risk assessment Implementation plan Definition of success	Suppliers IT operations team Device and software vendors IT leaders Risk committee
Agenda & process flow			Determine where people will access request form	Ending point

Create a committee charter to ensure roles are clarified and mandates can be met

The purpose of the committee is to quickly assess and protect organizational interests while furthering the needs of the business The committee needs to be seen as an enabler to the business, not as a gatekeeper, so it must be thorough but responsive. The charter should include: The vision to ensure clarity of purpose. IoT mandates to focus the committee on assessment criteria. Roles, responsibilities, and assignments to engage the right people who will provide the kind of guidance needed to ensure success. Procedures to make the best use of each committee member’s time. Process flow to guide evaluations to avoid unnecessary delays while reducing organizational risks.

1.3 Exercise:
Define procedures for reviewing proposals and projects

2-3 hours

Input: Schedules of committee members, Process documentation for evaluating new technology

Output: Procedures for reviewing proposals, Reference documentation for evaluating proposals

Materials: Whiteboard/flip charts, Steering committee workbook

Participants: Steering committee, which may include: IT executive, Privacy & Security senior staff, Infrastructure & Operations senior staff, Senior data specialist, Senior business executive(s)

1. Discuss as a group how often you will meet for reviews and project updates. Which roles will have veto rights on project approvals?
2. Define the intake process and requirements for scheduling based on average lead time to get the group together and preview documentation.
3. Identify where process documentation already exists to use for evaluation of proposals and projects, and what needs to be created to quickly move from evaluation to action phases.
4. Define basic rules of engagement.
5. Define process flow using COPIS methodology as a framework. Note the different stages that may be part of the intake flow. Some business partners may bring solutions to IT, and others may just have an idea that needs to be solutioned.

Download the IoT Steering Committee Charter

Create and Implement an IoT Strategy

Phase 2

Define the intake and assessment process

This phase will provide the following activities

• Define requirements for requesting new IoT solutions
• Define procedures for review proposals and projects
• Define service objectives and evaluation process for reviewing proposals and projects

Determine what information is necessary to start the intake process

2.1 Exercise:
Define requirements for requesting new IoT solutions

Input: Business requirements for requesting IT solutions

Output: Request form for business users, Section 1 of the IoT Solution Playbook

Materials: Whiteboard/flip charts, IoT Solution Playbook

Participants: Steering committee, which may include: IT executive, Privacy & Security senior staff, Infrastructure & Operations senior staff, Senior data specialist, Senior business executive(s)

1. Review template for the IoT Solution Playbook to ensure it meets your needs; modify as necessary.
2. Determine requirements for initiating an assessment.
   1. Will a business case be necessary to start, or can the assessment feed into the business case?
   2. How can you best access the work already done by the requester to not start over?
   3. Determine the right questions to understand how they will define success to ensure this solution will do what they need.
   4. Do you need a breakdown of the way they do the job today?
   5. What level of authorization needs to be on the request to move forward?
3. Try to balance the effort of the requester against their role. Don’t expect them to investigate solutions beyond the business value.
4. Provide them with a means to provide you any information they have gathered, especially if they have already spoken to vendors.

Download the IoT Solution Playbook

Define what role the BA or BRM will play to support the request process

Clearly articulate the business benefits to secure funding and resources

If the business users need to build a business case, the information being collected will help to define the value, estimate costs, and evaluate risk

IoT point solutions can be straightforward to articulate the business benefits as they will have very specific benefits which will likely fit into one of these categories: Financial – to increase profitability or reduce costs through predictive maintenance and efficiency. Business Development – innovation for new products, services, and methodologies Improve specific outcomes – typically these will be industry specific, such as improved patient health care, reduced traffic congestion or use of city resources, improved billing, or fire prevention for utility companies. As you start to look at the bigger picture of how these different systems can bring together disparate data sets, the benefits will be harder to define, and the costs to implement this next level of data analysis can be daunting and expensive. This doesn’t necessitate a complete alignment of data collection purposes; there may be benefits to improving operations in secondary areas such as updating HVAC systems to reduce energy costs in a hospital, though the updated systems may also include sensors to monitor air quality and further improve patient outcomes. In these cases, there may be future opportunities to use this data in unexpected ways, but even where there aren’t, applying the same standards for security, privacy, and operations should apply.

(Microsoft IoT Signals Report 2020, n= 3,000 IT Professionals)

2.2 Exercise – BA/BRM: Define procedures for reviewing proposals and projects

1 hour

Input: Process documentation for evaluating new technology, Business case requirements

Output: Interview questions and assessment criteria for BA/BRM

Materials: Whiteboard/flip charts, IoT Solution Playbook

Participants: Steering committee, which may include: Business analyst or business relationship manager, IT executive(s), Senior data specialist, Senior business executive(s)

1. Review template for the IoT Solution Playbook to ensure it meets your needs; modify as necessary.
2. Identify the questions that will need to be asked of the business to determine whether the request will be fit for purpose.
3. Additional questions may help to:
   1. Identify project sponsors to determine if requirements are defined or need to be, and who will champion this project through to implementation.
   2. Identify what additional work will be needed for you to shepherd the project through the various stage gates.
   3. Identify any prioritization criteria including business-specific milestones and outcomes.
4. Document when a formal business case needs to be created.

Download the IoT Solution Playbook

Assess the vendor’s solution for accessibility to ensure data will be available and useable

Determine how often you need to access and download data

Requirements will vary depending on whether sensors are collecting data for later analysis or if they are actuators that need to process data at the source.

Determine where the data will reside and how it will be structured. If it will be open and controlled within your own environment, confer with your data team to ensure the solution is integrated into your data systems. If, however, the solution is a point solution which will be hosted by the vendor, understand who will be normalizing the data and how frequently you can export or transfer it into your own data repository. If APIs will need to be installed to enable data transfer, work with the vendor to test them. Self-contained or closed solutions may be quick to install and configure and may require minimal technical support from within your own IT team, but they will not provide visibility to the inner workings of the solution. This may create issues around integration and interoperability which could limit the functionality and usability beyond the point solution. If the solution chosen is a closed system, determine how you will need to interact with the vendor to gain access to the data. Interoperability may not be an option, so work with the vendor to set up a regular cadence for accessing the data. Questions for the vendor could include: How often can we access the data? Will the vendor push it on a regular basis? Is it on demand? Or will we need to pull the data? Is there an API? Will the data be normalized? Will the data be transferred, or will the vendor keep a historical record? Are there additional fees for archiving or for data extraction?

Identify whether digital twins are needed

Create a virtual world to safely test and fail without impacting the real-world applications.

As actuators are processing information and executing actions, there may be a benefit to assess the effectiveness and impact of various scenarios in a safe environment. Digital twins enable the creation of a virtual world to test these new use cases using real world scenarios. These virtual replicas will not be necessary for every IoT application as many solutions will be very straightforward in their application. But for those complex systems, such as smart buildings, smart cities and mechanically complex projects, digital twins can be created to run multiple simulations to aid in business continuity planning, performance assessments, R&D and more. Due to the expense and complexity of creating a full digital twin, carefully weighing the benefits, and identifying how it will be used, can help to build the business case to invest in the technology. Without the skills in house, reliance on a vendor to create the model and test scenarios will likely be part of the overall solution. The assessment will also include understanding what data will be transferred into the model, how often it will be updated, how it will be protected and who will need to be involved in the modeling process. Download the blueprint: Double Your Organization’s Effectiveness With a Digital Twin. if you need more information on how to leverage digital twin technology.

To fully realize value in IoT, think beyond single use case solutions to leverage the data collected

• A single IoT solution can add hundreds of sensors, collecting a wide variety of data for specific purposes. If multiple solutions are in place, there may be divergent data sets that may never be seen by anyone other than their specific data stewards.
• Many organizations have started out with one or two solutions that support their primary business and may include some more mature offerings such as HVAC systems, which have used sensors for years. However, not all data is used today. In many cases, data is used for anomaly detection to improve operations, and only the non-standard information is used for alerting. McKinsey estimates less than 1% of data is used in these applications, with the remaining data stored or deleted, rather than used for optimization and predictive analysis.
• Thinking beyond the initial use cases, there may be opportunities to create new services, improve services for existing products, or improve insights through analysis of juxtaposed data.
• McKinsey reports up to $11.1 trillion a year in economic value may be possible by 2025 through the linking of the physical and digital worlds. Personal devices and all industries are potential growth areas – though factories and anywhere that could use predictive maintenance, cities, retail, and transportation will see the largest probable increases. Interoperability was identified as being required to maximize value, accounting for 40% to 60% of the potential value of IT applications.
• Where data is used to correct and control anomalies, very little data is retained and used for optimization or predictive analysis. By taking a deliberate approach to normalize, correlate, and analyze data, organizations can gain insight into the way their products are used, benefit from predictive maintenance, improve health care, reduce costs, and more.

By 2025 an estimated data volume of 79.4 zettabytes will be attributed to connected IoT devices. (Statistia)

Build data governance and analysis into your strategy to find new insights from correlating new and existing data

• Some industries, such as governments looking to build smart cities, will have a very broad range of opportunities for IoT devices, as well as high levels of difficulty managing very disparate systems; other industries, such as healthcare, will have very focused prospects for data collection and analysis.
• In any case, the introduction of new IoT solutions can create very large amounts of data quickly, and if used only for a single purpose, there may be lost opportunity for expanding use of data to better understand your product, customers, or environment.
• Don’t limit analysis to only IoT-collected data, as this can be consolidated with other sources for validation, enhancement, and insights. For example, fleet transponders can be connected to travel logs and dispatch records for validation and evaluation of fuel and resource consumption.
• Determine the best time and methods for consolidation and normalization; consider using data consolidation vendors if the expertise is not available in-house.
• As data combines, there may be unintended consequences of unique anonymous identifiers combining to identify employees or customers, and the potential for privacy breeches will need to be evaluated as all new systems come on-line.

“We find very little IoT data in real life flows through analytics solutions, regardless of customer size. Even in the large organizations, they tend to build at-purpose applications, rather than creating those analytical scenarios or think of consolidating the IoT data in a data lake like environment.” (Rajesh Parab, Info-Tech Research Group)

2.3 Exercise – data specialists: Define criteria for assessing proposals and projects

1-2 hours

Input: Process documentation for evaluating new technology, Data governance documents

Output: Interview questions and assessment criteria for data specialists

Materials: Whiteboard/flip charts, IoT Solution Playbook

Participants: Steering committee, which may include: Business analyst or business relationship manager, IT executive, Senior data specialist, Senior business executive(s), Privacy & Security senior staff, Infrastructure & Operations senior staff

1. Review template for the IoT Solution Playbook to ensure it meets your needs; modify as necessary.
2. Identify the questions that will need to be asked of the solution to ensure data governance and accessibility needs will be met.
3. Additional questions may help to:
   1. Identify data owners or stewards to determine who will have authority over data and ensure their needs will be met.
   2. Identify what additional work will be needed for the data team to access, validate, normalize, and centralize data.
   3. Identify any concerns that will identify the solution as unviable.
   4. Identify any risks to data accessibility which will require mitigation.

This initial review is designed to identify risks to data ownership or integrity and ensure data is available for additional uses as deemed appropriate to the organizational goals. This assessment is designed to find major flaws and to mitigate and integrate should the project be approved as viable.

Download the IoT Solution Playbook

Security assessments will need to include risk reviews specific to IoT

Align risk assessments to your existing risk registry, to quickly approve low-risk solutions and mitigate high risk

Physical security: Will these systems be accessible to the public, and can they be secured in a way to minimize theft and vandalism? Will they require additional housing or waterproofing? Could access be completely secured? For example, could anyone access and install malware on a disconnected camera’s SD card?

Security settings: For ease of service and installation, a vendor may use default security settings and passwords. This can create easy access for hackers to access the network and access sensitive data. Is there a possibility of IP theft though access by sensors? Determine who will have remote access to the system, and if the vendor will be supporting the system, will they be using least privilege or zero trust models? Determine their adherence to your security policy.

Internet and network access and monitoring: Review connectivity and data transmission requirements and whether these can be accommodated in a way that balances security with operational needs. Will there be a need for air gapping, firewalls, or secure tunnelling, and will these solutions allow for discovery and monitoring? Can the vendor guarantee there are no back doors built into the code? Will the system be monitored for unauthorized access and activity, and what is the response process? Can it be integrated into your security operations center?

Failover state: IoT devices with actuators or that may impact health and safety will need to be examined. Can you ensure actions in event of a failure will not be negatively impactful? For example, a door that locks on failover and cannot be opened from the inside will create safety risks; however, a door that opens on failover could result in theft of property or IP. Who controls and can access these settings?

Firmware updates: Assess the history of updates released by the vendor and determine how these updates are sent to the devices and validated. Ensure the product has been developed using trusted platforms with security lifecycle models. Many devices will have embedded security solutions. Ensure these can be integrated into organizational security solutions and risk mitigation strategies.

Enterprise IoT strategy will require a focus on privacy and risk

Don’t make assumptions about the type of data gathered with devices – ask the vendor to clearly state how and what is collected

Carefully review how this information can be used by machine learning, in combination with other solutions, and if there is a possibility of unintended consequences that will create issues for your customers and therefore your own data sets. Look for ways of capturing information that will meet your business requirements while mitigating risk of capturing personally identifiable information. Examples would be LiDAR to capture movement instead of video, or AI to blur faces or license plate numbers at time of image capture. This chart identifies data collected by smartphone accelerometers which could be used to identify and profile an individual and understand their behaviors.

Mobile device accelerometer data Overview of sensitive inferences that can be drawn from accelerometer data. (Source: Association for Computing Machinery, 2019.)

2.4 Exercise – Privacy & Security specialists: Define criteria for assessing proposals and projects

1-2 hours

Input: Process documentation for evaluating new technology, Data governance documents

Output: Interview questions and assessment criteria for Privacy & Security specialists

Materials: Whiteboard/flip charts, IoT Solution Playbook

Participants: Steering committee, which may include: Business analyst or business relationship manager, IT executive, Senior data specialist, Senior business executive(s), Privacy & Security senior staff, Infrastructure & Operations senior staff

1. Review template for the IoT Solution Playbook to ensure it meets your needs; modify as necessary.
2. Identify the questions that will need to be asked of the solution to ensure security and privacy needs will be met.
3. Additional questions may help to:
   1. Identify biggest risks created by a large influx of sensors and additional vendors.
   2. Identify options for mitigating risks for privacy and regulatory requirements.

This initial review is designed to identify risks to data ownership or integrity and ensure data is available for additional uses as deemed appropriate to the organizational goals. This assessment is designed to find major flaws and to mitigate and integrate should the project be approved as viable.

Download the IoT Solution Playbook

Review infrastructure requirements to proactively engage with vendors

Determine operational readiness to support and secure IoT solutions

Identify what needs to happen to onboard these solutions into your support portfolio

Evaluate support options to determine the best way to support the business. Even if support is completely outsourced, a support plan will be critical for holding vendors to account, bringing support in-house if support doesn’t meet your needs, and understanding dependencies while navigating through incidents and problem- and change-enablement processes. Regular maintenance for your team may include battery swaps, troubleshooting camera outages or intermittent sensors, or deploying patches. Understand the support requirements for the product lifecycle and who will be responsible for that work. If the vendor will be applying patches and upgrading firmware, get clarity on how often and how they’ll be deployed and validated. Ask the vendor about support documentation and offerings. Determine the best ways of collecting inventory on the solution. Determine what the solution offers to help with this process; however, if the project plan requires specific location details to add sensors, the project list may be the best way to initially onboard the sensors into inventory. Determine if warranty offerings are an appropriate solution for devices in each project, to schedule and record appropriate maintenance details and plan replacements as sensors reach end of life. Document dependencies for future planning.

2.5 Exercise – Infrastructure & Operations specialists: Define criteria for assessing proposals and projects

1-2 hours

Input: Process documentation for evaluating new technology, Data governance documents

Output: Interview questions and assessment criteria for Infrastructure & Operations specialists

Materials: Whiteboard/flip charts, IoT Solution Playbook

Participants: Steering committee, which may include: Business analyst or business relationship manager, IT executive, Senior data specialist, Senior business executive(s), Privacy & Security senior staff, Infrastructure & Operations senior staff

1. Review template for the IoT Solution Playbook to ensure it meets your needs; modify as necessary.
2. Identify the questions that will need to be asked of the solutions to ensure the solutions can be integrated into the existing environment and operational processes.
3. Additional questions may help to:
   1. Reduce risks and project failures from solutions that will be difficult to integrate or secure.
   2. Improve project planning for projects that are often driven by the vendor and the business.
   3. Reduce operational risks due to lack of integration with asset and operational processes.

This initial review is designed to identify risks to data ownership or integrity and ensure data is available for additional uses as deemed appropriate to the organizational goals. This assessment is designed to find major flaws and to mitigate and integrate should the project be approved as viable.

Download the IoT Solution Playbook

2.6 Exercise: Define service objectives and evaluation process

1 hour

Input: List of criteria in the playbook, Understanding of resource availability of solution evaluators

Output: Steering committee criteria for progressing projects through the process

Materials: Whiteboard/flip charts, IoT Steering Committee Charter workbook

Participants: Steering committee, which may include: Business analyst or business relationship manager, IT executive, Senior data specialist, Senior business executive(s), Privacy & Security senior staff, Infrastructure & Operations senior staff

Now that you’ve defined the initial review requirements, meet as a group once more to finalize the process for reviewing requests. Look for ways to speed the process, including asynchronous communications and reviews. Consider meeting as a group for any solutions that may be deemed high risk or highly complex.

1. Agree on what can be identified as a reasonable SLA to respond to the business on these requests.
2. Agree on methods of communication between committee members and the business.
3. Determine the criteria for determining when a proof of value should be initiated, and who will lead the process.

Download the IoT Steering Committee Charter

Create and Implement an IoT Strategy

Phase 3

Prepare for a Proof of Value

This phase will provide the following activities

• Create proof of value criteria
• Create proof of value template

A proof of value can quickly help you prove value or fail fast

Investing a small amount of time and money up front will validate the possibility of your proposed solution.

A proof of value will require a vision and definition of your criteria for success, which will be necessary to determine if the project should go ahead. It should take no longer than three months and may be as short as a week. When should you run a proof of value? When it is difficult to confirm that the solution is fit for purpose. When the value of the solution is indeterminate. When the solution is early in its lifecycle and not widely proven in the marketplace. When scalability is questionable or unproven. When the solution requires customization or configuration. Info-Tech Insight Where a solution is well known in the market, requires minimal customization, and is proven to be fit for purpose, a shorter evaluation or conversations with reference clients or partners may be all that is necessary.

(Microsoft IoT Signals Report 2020, n= 3,000 IT Professionals)

3.1 Exercise: Define the criteria for running a proof of value

Input: Agreement of steering committee members to create a process to mitigate risk for complex solutions.

Output: Proof of value template for use as appropriate to evaluate IoT solutions.

Materials: IoT Solution Playbook

Participants: Steering committee, which may include: Business analyst or business relationship manager, IT executive, Senior data specialist, Senior business executive(s), Privacy & Security senior staff, Infrastructure & Operations senior staff

1. As a group, review the circumstances for when to run a proof of value.
2. Determine who will help to build the proof of value plan.
3. Determine requirements for participation in the proof of value process. Consider project size, complexity and risk and visibility.

Download IoT Solution Playbook

Design your proof of value to test the viability of the solution

Engage the right stakeholders early to gather feedback and analysis and determine suitability

Define your objectives for the proof of value

Referencing documents submitted to the committee, continue to refine the problem statement.

Define use cases to test against current methods

Outline the solution to the problem

Determine the appropriate project team

Estimate the resources required for the pilot

Time, money, technology, resources

Conduct a post-proof of value review to finalize the decision to move forward

• The core working group is responsible for producing a vision of the future and outlining new technology’s disruptive potential. The actual implementation of the proof of value (purchasing the hardware, negotiating the SLA with the vendor) is beyond the committee’s responsibilities.
• If the proof of value goes ahead, the facilitator should block some time to evaluate the completed project against the key performance indicators identified in the initial plan.
• Use the Proof of Value Template section of the IoT Solution Playbook to document POV requirements as well as finalizing the feedback loop.
• Determine ratings for the proof of value to identify which solutions are not viable and which levels of viability are worth moving forward. Some viable solutions may need a different vendor, and some may need customization or multiple integrations. This is important for the project team to move ahead with the implementation.
• Encourage everyone to provide enough feedback on the various processes to be confident in their declarations of worthiness and to confirm the proof of value was thorough.
• Communicate your working group’s findings and success to a wide audience to gain interest in IoT solutions as well as to encourage the business to work with the committee to integrate solutions into the governance and operational structure.

3.2 Exercise: Create a template for designing a proof of value

1-3 hours

Input: Agreement of steering committee members to create a process to mitigate risk for complex solutions

Output: Proof of value template for use as appropriate to evaluate IoT solutions

Materials: Whiteboard/flip charts, IoT Solution Playbook

Participants: Steering committee, which may include: Business analyst or business relationship manager, IT executive, Senior data specialist, Senior business executive(s), Privacy & Security senior staff, Infrastructure & Operations senior staff

1. As a group, review the Proof of Value Template section of the IoT Solution Playbook to determine if it will meet the needs of your business and technical groups.
2. Determine who will work with the business to create the proof of value plan.
3. Modify the template to suit your needs, keeping in mind a need for clarity of purpose, communications throughout the POV, and clearly stated goals and definitions of success.
4. Set a target timeframe to run the POV, preferably no longer than 90 days.
5. Determine appropriate steps to take for POVs that do not garner the expected participation to qualify a solution to move forward.
6. Determine appropriate reporting for the evaluation process.

Download IoT Solution Playbook

Communications

As with any new product, marketing and communications will be an important first step in letting the business know how to engage IT in its assessments of IoT innovations. As these solutions prove themselves, or even as you help the business to find better solutions, share your successes with the rest of the organization.

Business units are already being courted by the vendors, so it’s up to IT to insert themselves in the process in a way that helps improve the success of the business team while still meeting IT’s objectives.

Your customers will not willingly engage in highly bureaucratic processes and need to see a reason to engage.

1. Keep the intake process simple.
2. Provide support to answer the tough questions.
3. Be clear on the benefits to the organization and the business unit by engaging with your group, and be clear about how you will help within a reasonable time frame.
   • IT will help navigate the vendor prerequisites, contracts, and product setup.
   • IT will assume some of the responsibility for the solution, especially around security and privacy.
   • The business unit will reap the rewards of the solution with minimal operational effort.

Info-Tech Insight

Consider building your playbook into your service catalog to make it easy for business users to start the request process. From there, you can create workflows and notifications, track progress, set and meet SLAs, and enable efficient asynchronous communications.

Research Contributors and Experts

	John Burwash Senior Director, Executive Services Info-Tech Research Group			INFO~TECH RESEARCH GROUP Info-Tech Research Group is an IT research and advisory firm with over 23 years of experience helping enterprises around the world with managing and improving core IT processes. They write highly relevant and unbiased research to help leaders make strategic, timely, and well-informed decisions. External contributors 4 external contributors have asked to remain anonymous.
	Jennifer Jones Senior Research Advisor, Industry Info-Tech Research Group		Aaron Shum Vice President, Security, Privacy & Risk Info-Tech Research Group
	Rajesh Parab Research Director, Applications, Data & Analytics Info-Tech Research Group		Frank Sargent Senior Director Practice Lead, Security, Privacy & Risk Info-Tech Research Group
	Scott Young Principal Research Advisor, Infrastructure Info-Tech Research Group		Rocco Rao Director, Research Advisor, Industry Info-Tech Research Group

Related Info-Tech Research

Understand and apply Internet-of-Things Use Cases to Drive Organizational Success A concise guide to understanding how IoT applications will create value in your firm.

Industry Coverage

Bibliography

Ayyaswamy, Regu, et al. “IoT Is Enabling Enterprise Strategies for New Beginnings.” Tata Consulting Services, 2020. Web.

“Data Volume of Internet of Things (IoT) Connections Worldwide in 2019 and 2025.” Statistia, 2020.

Dos Santos, Daniel, et al. “Cybersecurity in Building Automation Systems (BAS).” Forescout, 2020. Web.

Earle, Nick. “Overcoming the Barriers to Global IoT Connectivity: How Regional Operators Can Reap Rewards From IoT.” IoTNow, 30 June 2021. Web.

Faludi, Rob. “How Do IoT Devices Communicate?” Digi, 26 Mar. 2021. Web.

Halper, Fern, and Philip Russom. “TDWI IoT Data Readiness Guide, Interpreting Your Assessment Score.” Cloudera, 2018. Web.

Horwitz, Lauren. “IoT Enterprise Deployments Continue Apace, Despite COVID-19.” IoT World Today, 22 Apr. 2021.

“How Does IoT Data Collection Work?” Digiteum, 13 Feb. 2020. Web.

“IoT Data: How to Collect, Process, and Analyze Them.” Spiceworks, 26 Mar. 2019. Web.

IoT Signals Report: Edition 2, Hypothesis Group for Microsoft, Oct. 2020. Web.

King, Stacey. “4 Key Considerations for Consistent IoT Manageability and Security.” Forescout, 22 Aug. 2019. Web.

Krämer, Jurgen. “Why IoT Projects Fail and How to Beat the Odds.” Software AG, 2020. Web.

Kröger, Jacob Leon, et al. “Privacy Implications of Accelerometer Data: A Review of Possible Inferences” ICCSP, Jan. 2019, pp. 81-7. Web.

Manyika, James, et al. “Unlocking the Potential of the Internet of Things.” McKinsey Global Institute, 1 June 2015. Web.

Ricco, Emily. “How To Run a Successful Proof of Concept – Lessons From Hubspot.” Filtered. Web.

Rodela, Jimmy. “The Blueprint, Your Complete Guide to Proof of Concept.” Motley Fool, 2 Jan 2021. Web.

Sánchez, Julia, et al. “An Integral Pedagogical Strategy for Teaching and Learning IoT Cybersecurity.” Sensors, vol. 20, no. 14, July 2020, p. 3970.

The IoT Generation of Vulnerabilities. SC Media, 2020. E-book.

Woods, James P., Jr. “How Consumer IoT Devices Can Break Your Security.” HPE, 2 Nov. 2021.