Implement Risk-Based Vulnerability Management

Get off the patching merry-go-round and start mitigating risk!

Table of Contents

Table of Contents

Analyst Perspective

Vulnerabilities will always be present. Know the unknowns!

In this age of discovery, technology changes at such a rapid pace. New things are discovered, both in new technology and in old. The pace of change can often be very confusing as to where to start and what to do.

The ever-changing nature of technology means that vulnerabilities will always be present. Taking measures to address these completely will consume all your department’s time and resources. That, and your efforts will quickly become stale as new vulnerabilities are uncovered. Besides, what about the systems that simply can’t be patched? The key is to understand the vulnerabilities and the levels of risk they pose to your organization, to prioritize effectively and to look beyond patching.

A risk-based approach to vulnerability management will ensure you are prioritizing appropriately and protecting the business. Reduce the risk surface!

Vulnerability management is more than just systems and application patching. It is a full process that includes patching, compensating controls, segmentation, segregation, and heightened diligence in security monitoring.

Jimmy Tom Research Advisor – Security, Privacy, Risk, and Compliance Info-Tech Research Group

Executive Summary

Info-Tech Insight

Vulnerability management does not always equal patch management. There is more than one way to tackle the problem, particularly if a system cannot be easily patched or replaced. If a vulnerability cannot be completely remediated, steps to reduce the risk to a tolerable level must be taken.

Common obstacles

These barriers make vulnerability management difficult to address for many organizations: The value of vulnerability management is not well articulated in many organizations. As a result, investment in vulnerability scanning technology is often insufficient. Many organizations feel that a “patch everything” approach is the most effective path. Vulnerability management is commonly misunderstood as being a process that only supports patch management. There is often misalignment between SecOps and ITOps in remediation action and priority, affecting the timeliness of remediation.

CVSS Score Distribution From the National Vulnerability Database: (Source: NIST National Vulnerability Database Dashboard)

Leverage risk to sort, triage, and prioritize vulnerabilities

Reduce your risk surface to avoid cost to your business; everything else is table stakes.

Reduce the critical and high vulnerabilities below the risk threshold and operationalize the remediation of medium/low vulnerabilities by following your effective vulnerability management program cycles.

Identify vulnerability sources

An inventory of your scanning tool and vulnerability threat intelligence data sources will help you determine a viable strategy for addressing vulnerabilities. Defining roles and responsibilities ahead of time will ensure you are not left scrambling when dealing with vulnerabilities.

Triage and prioritize

Bring the vulnerabilities into context by assessing vulnerabilities based on your security posture and mechanisms and not just what your data sources report. This will allow you to gauge the true urgency of the vulnerabilities based on risk and determine an effective mitigation plan.

Remediate vulnerabilities

Address the vulnerabilities based on their level of risk. Patching isn't the only risk mitigation action; some systems simply cannot be patched, but other options are available.

Reduce the risk down to medium/low levels and engage your regular operational processes to deal with the latter.

Measure and formalize

Upon implementation of the program, measure with metrics to ensure that the program is successful. Improve the program with each iteration of vulnerability mitigation to ensure continuous improvement.

Tactical Insight 1

All actions to address vulnerabilities should be based on risk and the organization’s established risk tolerance.

Tactical Insight 2

Reduce the risk surface down below the risk threshold.

The industry has shifted to a risk-based approach

Traditional vulnerability management is no longer viable.

“For those of us in the vulnerability management space, ensuring that money, resources, and time are strategically spent is both imperative and difficult. Resources are dwindling fast, but the vulnerability problem sure isn’t.” (Kenna Security)

“Using vulnerability scanners to identify unpatched software is no longer enough. Keeping devices, networks, and digital assets safe takes a much broader, risk-based vulnerability management strategy – one that includes vulnerability assessment and mitigation actions that touch the entire ecosystem.” (Balbix)

“Unlike legacy vulnerability management, risk-based vulnerability management goes beyond just discovering vulnerabilities. It helps you understand vulnerability risks with threat context and insight into potential business impact.” (Tenable)

“A common mistake when prioritizing patching is equating a vulnerability’s Common Vulnerability Scoring System (CVSS) score with risk. Although CVSS scores can provide useful insight into the anatomy of a vulnerability and how it might behave if weaponized, they are standardized and thus don’t reflect either of the highly situational variables — namely, weaponization likelihood and potential impact — that factor into the risk the vulnerability poses to an organization.” (SecurityWeek)

Why a take risk-based approach?

Vulnerabilities, by the numbers

60% — In 2019, 60% of breaches were due to unpatched vulnerabilities.

74% — In the same survey, 74% of survey responses said they cannot take down critical applications and systems to patch them quickly. (Source: SecurityBoulevard, 2019)

Info-Tech Insight

Taking a risk-based approach will allow you to focus on mitigating risk, rather than “just patching” your environment.

The average cost of a breach in 2020 is $3.86 million, and “…the price tag was much less for mature companies and industries and far higher for firms that had lackluster security automation and incident response processes.” (Dark Reading)

Info-Tech’s vulnerability management methodology

Focus on developing the most efficient processes.

Vulnerability management isn’t “old school.”

The vulnerability management market is relatively mature; however, vulnerability management remains a very relevant and challenging topic.

Security practitioners are inundated with the advice they need to prioritize their vulnerabilities. Every vulnerability scanning vendor will proclaim their ability to prioritize the identified vulnerabilities.

Third-party prioritization methodology can’t be effectively applied across all organizations. Each organization is too unique with different constraints. No tool or service can account for these variables.

When patching is not possible, other options exist: configuration changes (hardening), defense-in-depth, compensating controls, and even elevated security monitoring are possible options.

Info-Tech Insight

Vulnerability management is not only patch management. Patching is only one aspect.

Blueprint deliverables

Each step of this blueprint is accompanied by supporting deliverables to help you accomplish your goals:

Key deliverable: Vulnerability Management SOP The Standard operating procedure (SOP) will comprise the end-to-end description of the program: roles & responsibilities, data flow, and expected outcomes of the program.	Vulnerability Management Policy Template for your vulnerability management policy.		Vulnerability Tracking Tool This tool offers a template to track vulnerabilities and how they are remedied.
	Vulnerability Scanning RFP Template Request for proposal template for the selection of a vulnerability scanning tool.		Vulnerability Risk Assessment Tool Methodology to assess vulnerability risk by determining impact and likelihood.

Blueprint benefits

Info-Tech’s process can save significant financial resources

Guided Implementation

A Guided Implementation (GI) is a series of calls with an Info-Tech analyst to help implement our best practices in your organization.

A typical GI is between 8 to 12 calls over the course of 4 to 6 months.

What does a typical GI on this topic look like?

Workshop Overview

Contact your account representative for more information.
workshops@infotech.com 1-888-670-8889

Implement Risk-Based Vulnerability Management

Phase 1

Identify Vulnerability Sources

This phase will walk you through the following activities:

Establish a common understanding of vulnerability management, define the roles, scope, and information sources of vulnerability detection.

This phase involves the following participants:

• Security operations team
• IT Security Manager
• IT Director
• CISO

Step 1.1

Vulnerability Management Defined

Activities

None for this section

This step will walk you through the following activities:

Establish a common understanding of vulnerability management and its place in the IT organization.

This step involves the following participants:

• Security operations team
• IT Security Manager
• IT Director
• CISO

Outcomes of this step

Foundational knowledge of vulnerability management in your organization.

What is vulnerability management?

It’s more than just patching.

Effective vulnerability management

It’s not easy, but it’s much harder without a process in place. Effective vulnerability management requires a formal process for organizations to follow; without one, vulnerabilities are dealt with in an ad hoc fashion. Patching isn’t the only solution, but it’s the one that often draws focus. Responsibilities for the different aspects of vulnerability management are often unclear, such as for testing, remediation, and implementation. Identifying new threats without proper vulnerability scanning tools can be a near-impossible task. Determining which vulnerabilities are most urgent can be an inconsistent process, increasing the organizational risk. Measuring the effectiveness of your vulnerability remediation activities can help you better manage resources in SecOps and ITOps. Your staff will be spending the appropriate effort on vulnerabilities that warrant that level of attention.

You’re not just doing this for yourself. It’s also for your auditors. Many compliance and regulatory obligations require organizations to have thorough documentation of their vulnerability management practices.

Vulnerability management revolves around your asset security services

Vulnerabilities can be found primarily within your assets but also connect to your information risk management. These must be effectively managed as part of a holistic security program. Without management, vulnerabilities left unattended can be easy for attackers to exploit. It becomes difficult to identify the correct remediation option to mitigate against the vulnerabilities.

Vulnerability management works in tandem with SecOps and ITOps

Vulnerability Management Process Inputs/Outputs: Arrows denote direction of information feed

Vulnerability management serves as the input into a number of processes for remediation, including: Incident management, to deal with issues Release management, for patch management Change management, for change control IT asset management, to track version information, e.g. for patching Application security testing, for the verification of vulnerabilities A two-way data flow exists between vulnerability management and: Security risk management, for the overall risk posture of the organization Threat intelligence, as vulnerability management reveals only one of several threat vectors

For additional information please refer to Info-Tech’s research for each area:

Info-Tech’s Information Security Program Framework

Vulnerability management is a component of the Infrastructure Security section of Security Management

For more information, review our Build an Information Security Strategy blueprint, or speak to one of our analysts. Info-Tech Insight Vulnerability management is but one piece of the information security puzzle. Ensure that you have all the pieces!

Case Study

INDUSTRY: Manufacturing SOURCE: Cimpress, 2016

One organization is seeing immediate benefits by formalizing its vulnerability management program.

Step 1.2

Defining the scope and roles

Activities

• 1.2.1 Define the scope and boundary of your organization’s security program
• 1.2.2 Assign responsibility for vulnerability identification and remediation

This step will walk you through the following activities:

Define and understand the scope and boundary of the security program. For example, does it include OT? Define roles and responsibilities for vulnerability identification and remediation

This step involves the following participants:

• Security operations team
• IT Security Manager
• IT Director
• CISO

Outcomes of this step

Understand how far vulnerability management extends and what role each person in IT plays in the remediation of vulnerabilities

Determine the scope of your security program

This will help you adjust the depth and breadth of your vulnerability management program. Determining the scope will help you decide how much organizational risk the vulnerability management program will oversee. Scope can be defined along four aspects: Data Scope – What data elements in your organization does your security program cover? How is data classified? Physical Scope – What physical scope, such as geographies, does the security program cover? Organizational Scope – How are business units engaged with security initiatives? Does the scope cover all subsidiary organizations? IT Scope – What parts of the organization does IT cover? Does their coverage include operational technology (OT) and industrial control systems (ICS)?

1.2.1 Define the scope and boundary of your organization’s security program

Input: List of Data Scope, Physical Scope, Organization Scope, and IT Scope

Output: Defined scope and boundaries of the IT security program

Materials: Whiteboard/Flip Charts, Sticky Notes, Markers, Vulnerability Management SOP Template

Participants: Business stakeholders, IT leaders, Security team members

1. On a whiteboard, write the headers: Data Scope, Physical Scope, Organizational Scope, and IT Scope.
2. Give each group member a handful of sticky notes. Ask them to write down as many items as possible for the organization that could fall under one of the four scope buckets.
3. In a group, discuss the sticky notes and the rationale for including them. Discuss your security-related locations, data, people, and technologies, and define their scope and boundaries.

The goal is to identify what your vulnerability management program is responsible for and document it.

Consider the following:

How is data being categorized and classified? How are business units engaged with security initiatives? How are IT systems connected to each other? How are physical locations functioning in terms of information security management?

Download the Vulnerability Management SOP Template

Assets are part of the scope definition

An inventory of IT assets is necessary if there is to be effective vulnerability management.

Assign responsibility for vulnerability identification and remediation

Determine who is critical to effectively detecting and managing vulnerabilities. Some of the remediation steps will involve members of IT management to identify the true organizational risk of a vulnerability. Vulnerability remediation comes in different shapes and sizes. In addition to patching, this can include implementing compensating controls, server and application hardening, or the segregating of vulnerable systems. Who carries out each of these activities? Who coordinates the activities and tracks them to ensure completion? The people involved may be members outside of the security team, such as members from IT operations, infrastructure, and applications. The specific roles that each of these groups play should be clearly identified.

1.2.2 Assign responsibility for vulnerability identification and remediation

Input: Sample list of vulnerabilities and requisite actions from each group, High-level organizational chart with area functions

Output: Defined set of roles and responsibilities for member groups

Materials: Vulnerability Management SOP Template

Participants: CIO, CISO, IT Management representatives for each area of IT

1. Display the table of responsibilities that need to be assigned.
2. List all the positions within the IT security team.
3. Map these to the positions that require IT security team members.
4. List all positions that are part of the IT team.
5. Map these to the positions that require IT team members.

If your organization does not have a dedicated IT security team, you can perform this exercise by mapping the relevant IT staff to the different positions shown on the right.

Download the Vulnerability Management SOP Template

Step 1.3

Cloud considerations for vulnerability management

Activities

None for this section.

This step will walk you through the following activities:

Review cloud considerations for vulnerability management

This step involves the following participants:

• Security operations team
• IT Security Manager
• IT Director
• CISO

Outcomes of this step

Understand the various types of cloud offerings and the implications (and limitations) of vulnerability management in a cloud environment.

Cloud considerations

Cloud will change your approach to vulnerability management. There will be a heavy dependence on the cloud service provider to ensure that vulnerabilities in their foundational technologies have been addressed. Depending on the level of “as-a-Service,” customers will have varying degrees of control and visibility into the underlying operations. With vendor acquiescence, you can set your tool to scan a given cloud environment, depending on how much visibility you have into their environment based on the service you have purchased. Due to compliance obligations of their customers, there is a growing trend among cloud providers to allow more scanning of cloud environments. In the absence of customer scanning capability, vendors may offer attestation of vulnerability management and remediation.

For more information, see Info-Tech Research Group’s Document Your Cloud Strategy blueprint.

Cloud environment scanning

Cloud scanning is becoming a more common necessity but still requires special consideration.

An organization’s cloud environment is just an extension of its own environment. As such, cloud environments need to be scanned for vulnerabilities.

Step 1.4

Vulnerability detection

Activities

• 1.4.1 Develop a monitoring and review process of third-party vulnerability sources
• 1.4.2 Incident management and vulnerability management

This step will walk you through the following activities:

Create an inventory of your vulnerability monitoring capability and third-party vulnerability information sources.

Determine how incident management and vulnerability management interoperate.

This step involves the following participants:

• Security operations team
• IT Security Manager
• IT Director
• CISO

Outcomes of this step

Catalog of vulnerability information data sources. Understanding of the intersection of incident management and vulnerability management.

Vulnerability detection

Vulnerabilities can be identified through numerous mediums.

Info-Tech has determined the following to be the four most common ways to identify vulnerabilities.

Automate with a vulnerability scanning tool

Vulnerability scanning tool considerations

Select a vulnerability scanning tool with the features you need to be effective. Vulnerability scanning tool selection can be an exciting and confusing process. You will need to consider what features you desire in a tool and whether you want the tool to go beyond just scanning and reporting. In addition to vulnerability scanning, some tools will integrate with your IT service management (service desk ticketing system) tool and asset, configuration, and change management modules. This can facilitate the necessary workflow that the remediation process follows once a vulnerability is discovered. A number of vulnerability scanning tool vendors have started offering remediation as part of their software features. This includes the automation and orchestration functionality and configuration and asset management to track its remediation activities. A side benefit of the asset discovery feature in vulnerability scanning tools is that it can help enhance an organization’s asset inventory and license compliance, particularly in cases where end users are able to install software on their workstations.

For guidance on tool vendors Visit SoftwareReviews for information on vulnerability management tools and vendors.

Vulnerability scanning tool best practices

How often should scans be performed?

One-off scans provide snapshots in time. Repeated scans over time provide tracking for how systems are changing and how well patches are being applied and software is being updated.

The results of a scan (asset inventory, configuration data, and vulnerability data) are basic information needed to understand your security posture. This data needs to be as up to date as possible.

ANALYST PERSPECTIVE: Organizations should look for continuous scanning

Continuous scanning is the concept of providing continual scanning of your systems so any asset, configuration, or vulnerability information is up to date. Most vendors will advertise continuous scanning but you need to be skeptical of how this feature is met.

Continuous Scanning Methods

Vulnerability scanning tool best practices

Where to perform a scan.

Vulnerability scanning tool best practices

Pre-Scan Communication With Users

• It is always important to inform owners and users of systems that a scan will be happening.
• Although it is unlikely any performance issues will arise, it is important to notify end users of potential impact.
• Local admins or system owners may have controls in place that stop vulnerability scans and you need to inform the owners so that they can safelist the scanner you will be using.

Vulnerability Scanning Tool Tracking Metrics

• Vulnerability score by operating system, application, or organization division.
  • This provides a look at the widely accepted severity of the vulnerability as it relates across the organization’s systems.
• Most vulnerable applications and application version.
  • This provides insight into how outdated applications are creating risk exposure for an organization.
  • This will also provide metrics on the effectiveness of your patching program.
• Number of assets scanned within the last number of days.
  • This provides visibility into how often your assets are being scanned and thus protected.
• Number of unowned devices or unapproved applications.
  • This metric will track how many unowned devices or unapproved applications may be on your network. Unowned devices may be rogue devices or just consultant/contractor devices.

Third-party vulnerability information sources

IT security forums and mailing lists are another source of vulnerability information.

Proactively identify new vulnerabilities as they are announced.

By monitoring for vulnerabilities as they are announced through industry alerts and open-source mechanisms, it is possible to identify vulnerabilities beyond your scanning tool’s penetration tests.

Common sources: Vendor websites and mailing lists Vendors are the trusted sources for vulnerability and patch information on their products, particularly with new industry vulnerability disclosure requirements. Vendors are the most familiar with their products, downloads are most likely malware free, and additional information is often included. There are some issues: vendors won’t announce a vulnerability until a patch is created, which creates a potential unknown risk exposure; numerous vendor sites will have to be monitored continually. Third-party websites A non-vendor site providing information on vulnerabilities. They often will cover a specific technology or an industry section, becoming a potential “one-stop shop” for some. They will often provide vulnerability information that is augmented with different remediation recommendations faster than vendors. However, it’s more likely that malicious code could be downloaded and it will often not be comprehensive information on patching. Third-party mailing lists, newsgroups, live paid subscriptions, and live open-source feeds These are alerting and notification services for the detection and dissemination of vulnerability information. They provide information on the latest and most critical vulnerabilities, e.g. US-CERT Cybersecurity Alerts. Vulnerability databases These usually consist of dedicated databases on vulnerabilities. They perform the hard work of identifying and aggregating vulnerability and patch information into a central repository for end-user consumption. The commentary features on these databases provide excellent insight for practitioners, e.g. National Vulnerability Database (NVD).

Third-party vulnerability information sources

IT security forums and mailing lists are another source of vulnerability information.

Third-party sources for vulnerabilities

• Open Source Vulnerability Database (OSVDB)
  • An open-source database that is run independently of any vendors.
• Common Vulnerabilities and Exposures (CVE)
  • Free, international dictionary of publicly known information security vulnerabilities and exposures.
• National Vulnerability Database (NVD)
  • Through NIST, the NVD is the US government’s repository of vulnerabilities and includes product names, flaws, and any impact metrics.
  • The National Checklist Repository Program (NCRP), also provided by NIST, provides security checklists for configurations of operating systems and applications.
  • The Center for Internet Security, a separate entity unrelated to NIST, provides configuration benchmarks that are often referenced by the NCRP.
• Open Web Application Security Project (OWASP)
  • OWASP is another free project helping to expose vulnerabilities within software.
• US-CERT National Cyber Alert System (US-CERT Alerts)
  • Cybersecurity Alerts – Provide timely information about current security issues, vulnerabilities, and exploits.
  • Cybersecurity Tips – Provide advice about common security issues for the general public.
  • Cybersecurity Bulletins – Provide weekly summaries of new vulnerabilities. Patch information is provided when available.
• US-CERT Vulnerability Notes Database (US-CERT Vulnerability Notes)
  • Database of searchable security vulnerabilities that were deemed not critical enough to be covered under US-CERT Alerts. Note that the NVD covers both US-CERT Alerts and US-CERT Notes.
• Open Vulnerability Assessment Language (OVAL)
  • Coding language for security professionals to discuss vulnerability checking and configuration issues. Vulnerabilities are identified using tests that are disseminated in OVAL definitions (XML executables that can be used by end users).

1.4.1 Develop a monitoring and review process for third-party vulnerability sources

Input: Third-party resources list

Output: Process for review of third-party vulnerability sources

Materials: Whiteboard, Whiteboard markers, Vulnerability Management SOP Template

Participants: IT Security Manager, SecOps team members, ITOps team members, CISO

1. Identify what third-party resources are useful and relevant.
2. Shortlist your third-party sources.
3. Identify what is the best way to receive information from a third party.
4. Document the method to receive or check information from the third-party source.
5. Identify who is responsible for maintaining third-party vulnerability information sources
6. Capture this information in the Vulnerability Management SOP Template.

Download the Vulnerability Management SOP Template

Incidents and vulnerability management

Incidents can also be a sources of vulnerabilities.

When any incident occurs, for example:

• A security incident, such as malware detected on a machine
• An IT incident, such as an application becomes unresponsive
• A crisis occurs, like a worker accident

There can be underlying vulnerabilities that need to be processed.

1.4.2 Incident management and vulnerability management

Input: Existing incident response processes, Existing crisis communications plans

Output: Alignment of vulnerability management program with existing incident management processes

Materials: Whiteboard, Whiteboard markers, Vulnerability Management SOP Template

Participants: IT Security Manager, SecOps team members, ITOps team members, including tiers 1, 2, and 3, CISO, CIO

1. Inventory what incident response plans the organization has. These include:
   1. Information Security Incident Response Plan
   2. IT Incident Plan
   3. Problem Management Plan
   4. Crisis Management Plan
2. Identify what part of those plans contains the post-response recap or final analysis.
3. Formalize a communication process between the incident response plan and the vulnerability mitigation process.

Note: Most incident processes will cover some sort of root cause analysis and investigation of the incident. If a vulnerability of any kind is detected within this analysis it needs to be reported on and treated as a detected vulnerability, thus warranting the full vulnerability mitigation process.

Download the Vulnerability Management SOP Template

Implement Risk-Based Vulnerability Management

Phase 2

Triage & prioritize

This phase will walk you through the following activities:

Examine the elements that you will use to triage and analyze vulnerabilities, prioritizing using a risk-based approach, and prepare for remediation options.

This phase involves the following participants:

• IT Security Manager
• SecOps team members
• ITOps team members, including tiers 1, 2, and 3
• CISO
• CIO

Step 2.1

Triage vulnerabilities

Activities

• 2.1.1 Evaluate your identified vulnerabilities

This step will walk you through the following activities:

Review your vulnerability information sources and determine a methodology that will be used to consistently evaluate vulnerabilities as your scanning tool alerts you to them.

This step involves the following participants:

• IT Security Manager
• SecOps team members
• ITOps team members, including tiers 1, 2, and 3
• CISO
• CIO

Outcomes of this step

A consistent, documented process for the evaluation of vulnerabilities in your environment.

Triaging vulnerabilities

Use Info-Tech’s methodology to allocate urgencies to your vulnerabilities to assign the appropriate resources to each one.

When evaluating numerous vulnerabilities, use the following three factors to help determine the urgency of vulnerabilities:

• The intrinsic qualities of the vulnerability
• The business criticality of the affected asset
• The sensitivity of the data stored on the affected asset

Intrinsic qualities of the vulnerability — Vulnerabilities need to be examined for the inherent risk they pose specifically to the organization, which includes if an exploit has been identified or if the industry views this as a serious and likely threat.

Business criticality of the affected asset — Assets with vulnerabilities need to be assessed for their criticality to the business. Vulnerabilities on systems that are critical to business operations or customer interactions are usually top of mind.

Sensitivity of the data of the affected asset — Beyond just the criticality of the business, there must be consideration of the sensitivity of the data that may be compromised or modified as a result of any vulnerabilities.

Info-Tech Insight

This methodology allows you to determine urgency of vulnerabilities, but your remediation approach needs to be risk-based, within the context of your organization.

Triage your vulnerabilities, filter out the noise

Triaging enables your vulnerability management program to focus on what it should focus on. Use the Info-Tech Vulnerability Mitigation Process Template to define how to triage vulnerabilities as they first appear. Triaging is an important step in vulnerability management, whether you are facing ten to tens of thousands of vulnerability notifications. Many scanning tools already provide the capability to compare known vulnerabilities against existing assets through integration with the asset inventory. There are two major use cases for this process: For organizations that have identified vulnerabilities but do not know their own systems well enough. This can be due to a lack of a formal asset inventory. For proactive organizations that are regularly staying up to date with industry announcements regarding vulnerabilities. Once an alert has been made publicly, this process can assist in confirming if the vulnerability is relevant to the organization.

The Info-Tech methodology for initial triaging of vulnerabilities: Even if neither of these use cases apply to your organization, triaging still addresses the issues of false positives. Triaging provides a quick way to determine if vulnerabilities are relevant.

After eliminating the noise, evaluate your vulnerabilities to determine urgency

Consider the intrinsic risk to the organization.

• For a vulnerability to become a true threat to the organization, it must be exploited to cause damage. In today’s threat landscape, exploit kits are sold online that allow individuals with low technical knowledge to exploit a vulnerability.
• Not all vulnerabilities have an associated exploit, but this does not mean that these vulnerabilities can be left alone. In many cases, it is just a matter of time before an exploit is created.
• Another point to consider is that while exploits can exist theoretically, they may not be verified. Vulnerabilities always pose some level of risk, but if there are no known verified exploits, there is less risk attached.

• Common Vulnerability Scoring System (CVSS) is an open-source industry scoring method to assess the potential severity of vulnerabilities.
• CVSS takes into account: attack vector, complexity, privileges required, user interaction, scope, confidentiality impact, integrity impact, and availability impact.
• Vulnerabilities that have a score of 4.0 or lower are classified as low vulnerabilities, while scores between 4.0 and 6.9 are put in the medium category. Scores of 7 or higher are in the high and critical categories. As we will review in the Risk Assessment section, you will want to immediately deal with high and critical vulnerabilities.

• Even though a vulnerability may appear to be part of an inconsequential asset, it is important to consider whether it can be leveraged to gain access to other areas of the network or system by an attacker.
• Another consideration should be whether the vulnerability can be exploited by remote or local access. Remote exploits pose a greater risk as this can mean that attackers can perform an exploit from any location. Local exploits carry less risk, although the risk of insider threats should be considered here as well.

2.1.1 Evaluate your identified vulnerabilities

Input: Visio workflow of Info-Tech’s vulnerability management process

Output: Adjusted workflow to reflect your current processes, Vulnerability Tracking Tool

Materials: Whiteboard, Whiteboard markers, Vulnerability Management SOP Template

Participants: IT Security Manager, SecOps team members, ITOps team members, including tiers 1, 2, and 3, CISO, CIO

Using the criteria from the previous slide, Info-Tech has created a methodology to evaluate your vulnerabilities by examining their intrinsic qualities.

The methodology categorizes the vulnerabilities into high, medium, and low risk importance categorizations, before assigning final urgency scores in the later steps.

1. Review the evaluation process in the Vulnerability Management Workflow library.
2. Determine if this process makes sense for the organization; otherwise, change the flow to include any other considerations of process flows.
3. As this process is used to evaluate vulnerabilities, document vulnerabilities to an importance category. This can be done in the Vulnerability Tracking Tool or using a similar internal vulnerability tracking document, if one exists.

Download the Vulnerability Management SOP Template

Step 2.2

Determine high-level business criticality

Activities

• 2.2.1 Determine high-level business criticality
• 2.2.2 Determine your high-level data classifications

This step will walk you through the following activities:

Determining high-level business criticality and data classifications will help ensure that IT security is aligned with what is critical to the business. This will be very important when decisions are made around vulnerability risk and the urgency of remediation action.

This step involves the following participants:

• IT Security Manager
• SecOps team members
• CISO

Outcomes of this step

Understanding and consistency in how business criticality and business data is assessed by IT in the vulnerability management process.

Understanding business criticality is key to determining vulnerability urgency

Prioritize operations that are truly critical to the operation of the business, and understand how they would be impacted by an exploited vulnerability.

2.2.1 Determine high-level business criticality

Input: List of business operations, Insight into business operations impacts to the business

Output: List of business operations and their criticality and impact to the business

Materials: Vulnerability Management SOP Template

Participants: Participants from the business, IT Security Manager, CISO, CIO

1. List your core business operations at a high level.
2. Use a High, Medium, or Low ranking to prioritize the business operations based on mission-critical criteria and the impact of the vulnerability.
3. When using the process flow, consider if the vulnerability directly affects any of these business operations and move through the process flow based on the corresponding High, Medium, or Low ranking.

Example prioritization of business operations for a manufacturing company:

Questions to ask: Is there a hard-dollar impact from downtime? Is there impact on goodwill or customer trust? Is regulatory compliance a factor? Is there a health or safety risk?

Download the Vulnerability Management SOP Template

Determine vulnerability urgency by its data classification

Consider how to classify your data based on if the Confidentiality, Integrity, or Availability (CIA) is compromised.

To properly classify your data, consider how the confidentiality, integrity, and availability of that data would be affected if it were to be exploited by a vulnerability. Review the table below for an explanation for each objective. Confidentiality Preserving authorized restrictions on information access and disclosure, including means for protecting personal privacy and proprietary information. Integrity Guarding against improper information modification or destruction, and ensuring information non-repudiation and authenticity. Availability Ensuring timely and reliable access to and use of information. Each piece of data should be ranked as High, medium, or low across confidentiality, integrity, and availability based on adverse effect. Low — Limited adverse effect Moderate — Serious adverse effect High — Severe or catastrophic adverse effect If you wish to build a whole data classification methodology, refer to our Discover and Classify Your Data blueprint.

How to determine data classification when CIA differs: The overall ranking of the data will be impacted by the highest objective’s ranking. For example, if confidentiality and availability are low, but integrity is high, the overall impact is high. This process was developed in part by Federal Information Processing Standards Publication 199.

2.2.2 Determine your high-level data classifications

Input: Knowledge of data use and sensitivity

Output: Adjusted workflow to reflect your current processes, Vulnerability Tracking Tool

Materials: Whiteboard, Whiteboard markers, Vulnerability Management SOP Template

Participants: IT Security Manager, CISO, CIO

If your organization has formal data classification in place, it should be leveraged to determine the high, medium, and low rankings necessary for the process flows. However, if there is no formal data classification in place, the process below can be followed:

1. List common assets or applications that are prone to vulnerabilities.
2. Consider the data that is on these devices and provide a high (severe or catastrophic adverse effect), medium (serious adverse effect), or low (limited adverse effect) ranking based on confidentiality, availability, and integrity.
   1. Use the table on the previous slide to assist in providing the ranking.
   2. Remember that it is the highest ranking that dictates the overall ranking of the data.
3. Document which data belongs in each of the categories to provide contextual evidence.

Download the Vulnerability Management SOP Template

This process should be part of your larger data classification program. If you need assistance in building this out, review the Info-Tech research, Discover and Classify Your Data.

Step 2.3

Consider current security posture

Activities

• 2.3.1 Document your defense-in-depth controls

This step will walk you through the following activities:

Your defense-in-depth controls are the existing layers of security technology that protects your environment. These are relevant when considering the urgency and risk of vulnerabilities in your environment, as they will mitigate some of the risk.

This step involves the following participants:

• IT Security Manager
• SecOps team members
• ITOps team members, including tiers 1, 2, and 3
• CISO
• CIO

Outcomes of this step

Understanding and documentation of your current defense-in-depth controls.

Review your current security posture

What you have today matters. In most cases, your vulnerability scanning tool alone will not have the context of your security posture in the results of its scans. This can skew the true urgency of detected vulnerabilities in your environment. What you have in place today is what comprises your organization’s overall security posture. This bears high relevance to the determination of the risk that a vulnerability poses to your environment. Elements such as enterprise architecture and defense in depth mechanisms should be factored into determining the risk of a vulnerability and what kind of immediacy is warranted to address it. Details of your current security posture will also contribute to the assessment and selection of remediation options.

Enterprise architecture considerations

Defense-in-depth

Defense-in-depth provides extra layers of protection to the organization.

2.3.1 Document your defense-in-depth controls

Input: List of technologies within your environment, List of IT security controls that are in place

Output: List of defense-in-depth controls

Materials: Whiteboard/flip charts, Vulnerability Management SOP Template

Participants: IT Security Manager, Infrastructure Manager, IT Director, CISO

1. Document the existing defense-in-depth controls within your system.
2. Review the initial list that has been provided and see if these are controls that currently exist.
3. Indicate any other controls that are being used by the organization. This may already exist if you have a security services catalog.
4. Indicate who the owners of the different controls are.
5. Track the information in the Vulnerability Management SOP Template.

Download the Vulnerability Management SOP Template

Step 2.4

Risk assessment of vulnerabilities

Activities

• 2.4.1 Build a classification scheme to consistently assess impact
• 2.4.2 Build a classification scheme to consistently assess likelihood

This step will walk you through the following activities:

Assessing risk will be the cornerstone of how you evaluate vulnerabilities and what priority you place on remediation. This is actual risk to the organization and not simply what the tool reports without the context of your defense-in-depth controls.

This step involves the following participants:

• IT Security Manager
• IT Operations Management
• CISO
• CIO

Outcomes of this step

A risk matrix tailored to your organization, based on impact and likelihood. This will provide a consistent, unambiguous way to assess risk across the vulnerability types that is reported by your scanning tool.

Vulnerabilities and risk

Vulnerabilities must be addressed to mitigate risk to the business. Vulnerabilities are a concern because they are potential threats to the business. Vulnerabilities that are not addressed can turn from potential threats into actual threats; it is only a matter of time and opportunity. Your organization will already be familiar with risk management, as every decision carries a business risk component. There may even be a senior manager assigned as corporate risk officer to manage organizational risk. The organization likely has a risk tolerance level that defines the organization’s risk appetite. This may be measured in dollars, non-productivity time, or other units of inefficiency. The risk of a vulnerability can be calculated using impact and likelihood. Impact is the effect that the vulnerability will have if it is exploited by a malicious actor. Likelihood is the degree to which a vulnerability exploit can possibly occur.

Info-Tech Insight Risk to the organization is business language that everyone can understand. This is particularly true when the risk is to productivity or to the company’s bottom line.

A risk-based approach to vulnerability management

CVSS scores are just the starting point!

Prioritize remediation by levels of risk

Address critical and high risk with high immediacy.

Addressing the critical and high-risk vulnerabilities with urgency will ensure that you are addressing a more manageable number of vulnerabilities. An optimized vulnerability management process will address the medium and low risk vulnerabilities within the regular cycle. This may be very similar to what you do today in an ad hoc fashion: Zero-day vulnerabilities tend to warrant a stop in operations and are dealt with immediately (or as soon as a vendor has a fix). The standard remediation process (patching/updating, change of configuration, etc.) happens within a regular controlled time cycle. Formalizing this process will ensure that appropriate attention is given to vulnerabilities that warrant it and that the remaining vulnerabilities are dealt with as a regular, recurring activity.

Mitigate the risk surface by reducing the time across the phases

Risk matrix

Risk = Impact x Likelihood Info-Tech’s Vulnerability Management Risk Assessment Tool provides a method of calculating the risk of a vulnerability. The risk rating is assigned using the impact of the risk and the likelihood or probability that the event may occur. The tool puts the vulnerability into your organization’s context: How many people will be affected? What service types are vulnerable and how does that impact the business? Is there an anticipated update from the vendor of the system being affected? Urgency of remediation should be based on the business consequences if the vulnerability were to be exploited, relative to the business’ risk tolerance. Info-Tech Insight Risk determination should be done within the context of your current environment and not simply based on what your vulnerability tool is reporting.

A risk matrix is useful in calculating a risk rating for vulnerabilities.

2.4.1 Build a classification scheme to consistently assess impact

Input: Knowledge of IT environment, Knowledge of business impact for each IT component or service

Output: Vulnerability Management Risk Assessment Tool formatted to your organization

Materials: Vulnerability Management Risk Assessment Tool

Participants: Functional Area Managers, IT Security Manager, CISO

Risk always has a negative impact, but the size of the impact can vary considerably in terms of cost, number of people or sites affected, and the severity of the impact. Impact questions tend to be more objective and quantifiable than likelihood questions.

1. Define a set of questions to measure risk impact or edit existing questions in the tool.
2. For each question, assign a weight that should be placed on that factor.
3. Define criteria for each question that would categorize the risk. The drop-down box content can be modified in the hidden Labels tab.

Note that you are looking to baseline vulnerability types, rather than categorizing every single vulnerability your scanning tool reports. The volume of vulnerabilities will be high, but vulnerabilities can be categorized into types on a regular basis.

Download the Vulnerability Management Risk Assessment Tool

2.4.2 Build a classification scheme to consistently assess likelihood

Input: Knowledge of IT environment, Knowledge of business impact for each IT component or service

Output: Vulnerability Management Risk Assessment Tool formatted to your organization

Materials: Vulnerability Management Risk Assessment Tool

Participants: Functional Area Managers, IT Security Manager, CISO

Risk always has a negative impact, but the size of the impact can vary considerably in terms of cost, number of people or sites affected, and the severity of the impact. Impact questions tend to be more objective and quantifiable than likelihood questions.

1. Define a set of questions to measure risk impact or edit existing questions in the tool.
2. For each question, assign a weight that should be placed on that factor.
3. Define criteria for each question that would categorize the risk. The drop-down box content can be modified in the hidden Labels tab.

Note that you are looking to baseline vulnerability types, rather than categorizing every single vulnerability that your scanning tool reports. The volume of vulnerabilities will be high, but vulnerabilities can be categorized into types on a regular basis.

Download the Vulnerability Management Risk Assessment Tool

Prioritize based on risk

Implement Risk-Based Vulnerability Management

Phase 3

Remediate vulnerabilities

This phase will walk you through the following activities:

• Identifying potential remediation options.
• Developing criteria for each option with regards to when to use and when to avoid.
• Establishing exception procedure for testing and remediation.
• Documenting the implementation of remediations and verification.

This phase involves the following participants:

• CISO, or equivalent
• Security Manager/Analyst
• Network, Administrator, System, Database Manager
• Other members of the vulnerability management team
• Risk managers for the risk-related steps

Determining how to remediate

Patching is only one option.

Step 3.1

Assessing remediation options

Activities

• 3.1.1 Develop risk and remediation action

This step will walk you through the following activities:

With the risk assessment from the previous activity, we can now examine remediation options and make a decision. This activity will guide us through that.

This step involves the following participants:

• IT Security Manager
• SecOps team members
• ITOps team members, including tiers 1, 2, and 3
• CISO
• CIO

Outcomes of this step

List of remediation options and criteria on when to consider each.

Identify remediation options

There are four options when it comes to vulnerability remediation.

Patches and updates

Patches are often the easiest and most common method of remediation.

Compensating controls

Compensating controls can decrease the risk of vulnerabilities that cannot be (immediately) remediated.

Configuration changes

Accept the risk and do nothing

By choosing not to remediate vulnerabilities, you must accept the associated risk. This should be your very last option.

Every time that a vulnerability is not remediated, it continues to pose a risk to the organization. While it may seem that every vulnerability needs to be remediated, this is simply not possible due to limited resources. Further, it can take away resources from other security initiatives as opposed to low-priority vulnerabilities that are extremely unlikely to be exploited.

Risk considerations

When determining if risk acceptance is appropriate, consider the cost of not mitigating vulnerabilities.

Don’t accept the risk because it seems easy. Consider the financial impact of leaving vulnerabilities open.

3.1.1 Develop risk and remediation action

Input: List of remediation options

Output: List of remediation options sorted into “when to use” and “when to avoid” lists

Materials: Whiteboard/flip charts, Vulnerability Management SOP Template

Participants: IT Security Manager, IT Infrastructure Manager, IT Operations Manager, Corporate Risk Officer, CISO

It is important to define and document your organization-specific criteria for when a remediation option is appropriate and inappropriate.

1. List each remediation option on a flip chart and create two headings: “When to use” and “When to avoid.”
2. Each person will list “when to use” criteria on a green sticky note and “when to avoid” criteria on a red one for each option; these will be placed on the appropriate flip chart.
3. Discuss as a group which criteria are appropriate and which should be removed.
4. Move on to the next remediation option when completed.
   • Ensure to include when there are remediation options that will be connected. For example, the risk may be accepted until the next available change window, or a defense-in-depth control is used before a patch can be fully installed.
5. Once the criteria has been established, document this in the Vulnerability Management SOP Template.

Download the Vulnerability Management SOP Template

Step 3.2

Scheduling and executing remediation

Activities

None for this section.

This step will walk you through the following activities:

Although there are no specific activities for this section, it will walk you through your existing processes configuration and change management to ensure that you are leveraging those activities in your vulnerability remediation actions.

This step involves the following participants:

• IT Security Manager
• SecOps team members
• ITOps team members, including tiers 1, 2, and 3
• CISO
• CIO

Outcomes of this step

Gained understanding of how IT operations processes configuration and change management can be leveraged for the vulnerability remediation process. Don’t reinvent the wheel!

Implementing the remediation

Vulnerability management converges with your IT operations functions. Once a remediation strategy has been formulated, you can leverage your release and change management processes to orchestrate the testing, version tracking, scheduling, approval, and implementation activities. Each of these processes should exist in your environment in some form. Leveraging these will engage the IT operations team to carry out their tasks in the remediation process. There can be a partial or full handoff to these processes, however, the owner of the vulnerability management program is responsible for verifying the application of the remediation measure and that the overall risk has been reduced. Although full blueprints exist that cover each of these processes in great detail, the following slides provide an overview of each of these IT operations processes and how they intersect with vulnerability management.

Release Management

Control the quality of deployments and releases of software updates.

Change Management

Post-implementation activities

Step 3.3

Continuous improvement

Activities

None for this section.

This step will walk you through the following activities:

Although this section has no activities, it will review the process by which you may continually improve vulnerability management.

This step involves the following participants:

• IT Security Manager
• SecOps team members
• ITOps team members, including tiers 1, 2, and 3
• CISO
• CIO

Outcomes of this step

An understanding of the importance of ongoing improvements to the vulnerability management program.

Drive continuous improvement

Continuous Improvement

Continuously re-evaluate the vulnerability management process.

Implement Risk-Based Vulnerability Management

Phase 4

Measure and formalize

This phase will walk you through the following activities:

• You will determine what ought to be measured to track the success of your vulnerability management program.
• If you lack a scanning tool this phase will help you determine tool selection.
• Lastly, penetration testing is a good next step to consider once you have your vulnerability management program well underway.

This phase involves the following participants:

• IT Security Manager
• SecOps team members
• Procurement representatives
• CISO
• CIO

Step 4.1

Metrics, Key Performance Indicators (KPIs), and Critical Success Factors (CSFs)

Activities

• 4.1.1 Measure your program with metrics, KPIs, and CSFs

This step will walk you through the following activities:

After a review of the differences between raw metrics, key performance indicators (KPI), and critical success factors (CSF), compile a list of what metrics you will be tracking, why, and the business goals for each.

This step involves the following participants:

• IT Security Manager
• SecOps team members
• CISO
• CIO

Outcomes of this step

Outline of metrics you can configure your vulnerability scanning tool to report on.

You can’t manage what you can’t measure

Metrics provides visibility.

Management consultant Peter Drucker introduced the concept of metrics tied to key performance indicators (KPIs), and the concept holds true: without metrics, you lack the visibility to manage or improve a process. Metrics aren’t just a collection of statistics, they have to be meaningful, they have to tell the story, and most importantly, they have to answer the “so what?” question. What is the significance of a metric – do they illustrate a trend or an anomaly? What actions should be carried out when a metric hits a certain threshold? It would be prudent to track several metrics that can be combined to tell the full story. For example, tracking the number of critical vulnerabilities alone does not give a sense of the overall risk to the organization, nor does it offer any information on how quickly they have been remediated or what amount of effort was invested.

Metrics, KPIs, and CSFs

Tracking relevant information

Tell the story in the numbers.

4.1.1 Measure your program with metrics, KPIs, and CSFs

Input: List of metrics current being measured by the vulnerability management tool

Output: List of relevant metrics to track, and the KPIs, CSFs, and business goals related to the metric

Materials: Whiteboard/flip charts, Vulnerability Management SOP Template

Participants: IT Security Manager, IT operations management, CISO

Metrics can offer a way to view how the organization is dealing with vulnerabilities and if there is improvement.

1. Determine the high-level vulnerability management goals for the organization.
2. Even with a formal process in place, the organization should be considering ways it can improve.
3. Determine metrics that can help quantify those goals and how they can be measured.
4. Metrics should always be easy to measure. If it’s a complex process to find the information required, it means that it is not a metric that should be used.
5. Document your list of metrics in the Vulnerability Management SOP Template.

Download the Vulnerability Management SOP Template

Step 4.2

Vulnerability Management Policy

Activities

• 4.2.1 Update the vulnerability management program policy

This step will walk you through the following activities:

If you have a vulnerability management policy, this activity may help augment it. Otherwise, if you don’t have one, this would be a great starting point.

This step involves the following participants:

• IT Security Manager
• CISO
• CIO
• Human resources representative

Outcomes of this step

An inaugural policy covering vulnerability management

Vulnerability Management Program Policy

Policies provide governance and enforcement of processes. Policies offer formal guidance on the “rules” of a program, describing its purpose, scope, detailed program description, and consequences of non-compliance. Often they will have a employee sign-off acknowledging understanding. In many organizations, policies are endorsed by senior executives, which gives the policy its “teeth” across the company. The human resources department will always have input due to the implications of the non-compliance aspect. Policies are written to ensure an outcome of consistent expected behavior and are often written to protect the company from liability. Policies should be easy to understand and unambiguous, reflect the current state, and be enforceable. Enforceability can come in the form of audit, technology, or any other means of determining compliance and enforcing behavior.

4.2.1 Update the vulnerability management policy

Input: Vulnerability Management SOP, HR guidance on policy creation and approval

Output: Completed Vulnerability Management Policy

Materials: Vulnerability Management SOP, Vulnerability Management Policy Template

Participants: IT Security Manager, IT operations management, CISO, Human resources representative

After having built your entire process in this project, formalize it into a vulnerability management policy. This will set the standards and expectations for vulnerability management in the organization, while the process will be around the specific actions that need to be taken around vulnerability management.

This is separate and distinct from the Vulnerability Management SOP Template, which is a process and procedure document. Review Info-Tech’s Vulnerability Management Policy and customize it to your organization’s specifications. Use your Vulnerability Management SOP as a resource when specifying some of the details within the policy.

Download the Vulnerability Management Policy Template

Step 4.3

Select and implement a scanning tool

Activities

• 4.3.1 Create an RFP for vulnerability scanning tools

This step will walk you through the following activities:

If you need to select a new vulnerability scanning tool, or replace your existing one, this activity will help set up a request for proposal (RFP).

This step involves the following participants:

• IT Security Manager
• SecOps team members
• CISO

Outcomes of this step

The provisions needed for you to create and deploy an RFP for a vulnerability management tool.

Vulnerability management and penetration testing

Similar in nature, yet provide different security functions.

Vulnerability scanning software

Vulnerability scanning market

Vulnerability scanning tool features

Vulnerability scanning deployment options

Vulnerability scanning methods

IP Management and IPv6

4.3.1 Create an RFP for vulnerability scanning tools

Input: List of key feature requirements for the new tool, List of intersect points with current software, Network topology and layout of servers and applications

Output: Completed RFP document that can be distributed to vendor proponents

Materials: Whiteboard/flip charts, Vulnerability Scanning Tool RFP Template

Participants: IT Security Manager, IT operations managers, CISO, Procurement department representative

Use a request for proposal (RFP) template to convey your desired scanning tool requirements to vendors and outline the proposal and procurement steps set by your organization.

1. Determine what kind of requirements will be needed for your scanning tool RFP, based on people, process, and technology requirements.
2. Consider items such as the desired capabilities and the scope of the scanning.
3. Conduct interviews with relevant stakeholders to determine the exact requirements needed.
4. Use Info-Tech’s Vulnerability Scanning Tool RFP Template. It lists many requirements but can be customized to your organization’s specific needs.

Download the Vulnerability Scanning Tool RFP Template

4.3.1 Create an RFP for vulnerability scanning tools (continued)

Download the Vulnerability Scanning Tool RFP Template

Step 4.4

Penetration testing

Activities

• 4.1.1 Create an RFP for penetration tests

This step will walk you through the following activities:

We will review penetration testing, its distinction from vulnerability management, and why you may want to engage a penetration testing service.

We provide a request for proposal (RFP) template that we can review if this is an area of interest.

This step involves the following participants:

• IT Security Manager
• SecOps team members
• CISO
• CIO

Outcomes of this step

An understanding of penetration testing, and guidance on how to get started if there is interest to do so.

Penetration testing

Penetration tests are critical parts of any strong security program.

Start by considering the spectrum of penetration tests

Info-Tech Insight

Your pen test should use multiple methods. Demonstrating weakness in one area is good but easy to identify. When you blend techniques, you get better success at breaching and it becomes more life-like. Think about prevention, detection, and response testing to provide full insight into your security defenses.

Penetration testing types

Evaluate four variables to determine which type of penetration test is most appropriate for your organization.

Penetration testing scope

Establish the scope of your penetration test before engaging vendors.

Determining the scope of what is being tested is the most important part of a penetration test. Organizations need to be as specific as possible so the vendor can actually respond or ask questions.

Organizations need to define boundaries, objectives, and key success factors.

What would be out of scope?

• Are there systems, IP addresses, or other things you want out of scope? These are things you don’t explicitly want any penetration tester to touch.
• Are there third-party connections to your environment that you don’t want to be tested? These are instances such as cloud providers, supply chain connections, and various services.
• Are there things that would be awkward to test? For example, determine if you include high-level people in a social engineering test. Do you conduct social engineering for the CEO? If you get their credentials, it could be an awkward moment.

Ways to break up a penetration test:

• Location – This is the most common way to break up a penetration test.
• Division – Self-contained business units are often done as separate tests so you can see how each unit does.
• IT systems – For example, you put certain security controls in a firewall and want to test its effectiveness.
• Applications – For example, you are launching a new website or a new portal and you want to test it.

Penetration testing appropriateness

Determine your penetration testing appropriateness.

4.4.1 Create an RFP for penetration tests

Input: List of criteria and scope for the penetration test, Systems and application information if white box

Output: Completed RFP document that can be distributed to vendor proponents

Materials: Whiteboard/flip charts, Penetration Test RFP Template

Participants: IT Security Manager, IT operations managers, CISO, Procurement department representative

Use an RFP template to convey your desired penetration test requirements to vendors and outline the proposal and procurement steps set by your organization.

1. Determine what kind of requirements will be needed for your penetration test RFP based on people, process, and technology requirements.
   • Consider items such as your technology environment and the scope of the penetration tests.
2. Conduct an interview with relevant stakeholders to determine the exact requirements needed.
3. Use Info-Tech’s Penetration Test RFP Template, which lists many requirements but can be customized to your organization’s specific needs.

Download the Penetration Test RFP Template

4.4.1 Create an RFP for penetration tests (continued)

Download the Penetration Test RFP Template

Penetration testing considerations – service providers

Consider what type of penetration testing service provider is best for your organization

Professional Service Providers

Professional Services Firms. These firms will often provide a myriad of professional services across auditing, financial, and consulting services. If they offer security-related consulting services, they will most likely offer some level of penetration testing.

Security Service Firms. These are dedicated security consulting or advisory firms that will offer a wide spectrum of security-related services. Penetration testing may be one aspect of larger security assessments and strategy development services.

Dedicated Penetration Testing Firms. These are service providers that will often offer the full gamut of penetration testing services.

Integrators

Managed Security Service Providers. These providers will offer penetration testing. For example, Dell SecureWorks offers numerous services including penetration testing. For organizations like this, you need to be skeptical of ulterior motives. For example, expect recommendations around outsourcing from Dell SecureWorks.

Regional or Small Integrators. These are service providers that provide security services of some kind. For example, they would help in the implementation of a firewall and offer penetration testing services as well.

Info-Tech Recommends:

• Always be conscientious of who is conducting the testing and what else they offer. Even if you get another party to test rather than your technology provider, they will try to obtain you as a client. Remember that for larger technology vendors, security testing is a small revenue stream for them and it’s a way to find technology clients. They may offer penetration testing for free to obtain other business.
• Most of the penetration testers were systems administrators (for network testing) or application developers (for application testing) at some point before becoming penetration testers. Remember this when evaluating providers and evaluating remediation recommendations.
• Evaluate what kind of open-source tools, commercial tools, and proprietary tools are being used. In general, you don’t want to rely on an open-source scanner. For open source, they will have more outdated vulnerability databases, system identification can also be limited compared to commercial, and reporting is often lacking.
• Above all else, ensure your testers are legally capable, experienced, and abide by non-disclosure agreements.

Penetration testing best practices – communications

Penetration testing best practices – results and remediation

What to expect from penetration test results report:

A final results report will state all findings including what was done by the testers, what vulnerabilities or exploitations were detected, how they were compromised, the related risk, and related remediation recommendations.

• Introduction. An overview of the penetration test methodology including rating methodology of vulnerabilities.
• Executive Summary. A management-level description of the test, often including a summary of any recommendations.
• Technical Review. An overview of each item that was looked at and touched. This area breaks down what was done, how it was done, what was found, and any related remediation recommendations. Expect graphs and visuals in this section.
• Detailed Findings. An in-depth breakdown of all testing methods used and results. Each vulnerability will be explained regarding how it was detected, what the risk is, and what the remediation recommendation is.

Summary of Accomplishment

Problem Solved

At the conclusion of this blueprint, you will have created a full vulnerability management program that will allow you to take a risk-based approach to vulnerability remediation.

Assessing a vulnerability’s risk will enable you to properly determine the true urgency of a vulnerability within the context of your organization; this ensures you are not just blindly following what the tool is reporting.

The risk-based approach will allow you to prioritize your discovered vulnerabilities and take immediate action on critical and high vulnerabilities while allowing your standard remediation cycle to address the medium to low vulnerabilities.

With your program defined and developed, you now need to configure your vulnerability scanning tool or acquire one if you don’t already have a tool in place.

Lastly, while vulnerability management will help address your systems and applications, how do you know if you are secure from external malicious actors? Penetration testing will offer visibility, allowing you to plug those holes and attain an environment with a smaller risk surface.

If you would like additional support, have our analysts guide you through other phases as part of an Info-Tech workshop.

Contact your account representative for more information.

workshops@infotech.com 1-888-670-8889

Additional Support

If you would like additional support, have our analysts guide you through other phases as part of an Info-Tech workshop.

Contact your account representative for more information. workshops@infotech.com 1-888-670-8889

To accelerate this project, engage your IT team in an Info-Tech workshop with an Info-Tech analyst team.

Info-Tech analysts will join you and your team at your location or welcome you to Info-Tech’s historic Toronto office to participate in an innovative onsite workshop.

The following are sample activities that will be conducted by Info-Tech analysts with your team:

Review of the Implement Vulnerability Management storyboard

Build your vulnerability management SOP

Contributors

Contributors from 2016 version of this project:

• Morey Haber, Vice President of Technology, BeyondTrust
• Richard Barretto, Manager, Information Privacy and Security, Cimpress
• Joel Shapiro, Vice President Sales, Digital Boundary Group

Contributors from current version of this project:

• 2 anonymous contributors from the manufacturing sector
• 1 anonymous contributor from a US government agency
• 2 anonymous contributors from the financial sector
• 1 anonymous contributor from the medical technology industry
• 2 anonymous contributors from higher education
• 1 anonymous contributor from a Canadian government agency
• 7 anonymous others; information gathered from advisory calls

Bibliography

Arya. “COVID-19 Impact: Vulnerability Management Solution Market | Strategic Industry Evolutionary Analysis Focus on Leading Key Players and Revenue Growth Analysis by Forecast To 2028 – FireMon, Digital Shadows, AlienVault.” Bulletin Line, 6 Aug. 2020. Accessed 6 Aug. 2020.

Campagna, Rich. “The Lean, Mean Vulnerability Management Machine.” Security Boulevard, 31 Mar. 2020. Accessed 15 Aug. 2020.

Constantin, Lucian. “What are vulnerability scanners and how do they work?” CSO Online, 10 Apr. 2020. Accessed 1 Sept. 2020.

“CVE security vulnerabilities published in 2019.” CVE Details. Accessed 22 Sept. 2020.

Garden, Paul, et al. “2019 Year End Report – Vulnerability QuickView.” Risk Based Security, 2020. Accessed 22 Sept. 2020.

Keary, Eoin. “2019 Vulnerability Statistics Report.” Edgescan, Feb. 2019. Accessed 22 Sept. 2020.

Lefkowitz, Josh. ““Risk-Based Vulnerability Management is a Must for Security & Compliance.” SecurityWeek, 1 July 2019. Accessed 1 Nov. 2020.

Mell, Peter, Tiffany Bergeron, and David Henning. “Creating a Patch and Vulnerability Management Program.” Creating a Patch and Vulnerability Management Program. NIST, Nov. 2005. Web.

“National Vulnerability Database.” NIST. Accessed 18 Oct. 2020.

“OpenVAS – Open Vulnerability Assessment Scanner.” OpenVAS. Accessed 14 Sept. 2020.

“OVAL.” OVAL. Accessed 21 Oct. 2020.

Paganini, Pierluigi. “Exploiting and Verifying Shellshock: CVE-2014-6271.” INFOSEC, 27 Sept. 2014. Web.

Pritha. “Top 10 Metrics for your Vulnerability Management Program.” CISO Platform, 28 Nov. 2019. Accessed 25 Oct. 2020.

“Risk-Based Vulnerability Management: Understanding Vulnerability Risk With Threat Context And Business Impact.” Tenable. Accessed 21 Oct. 2020.

Stone, Mark. “Shellshock In-Depth: Why This Old Vulnerability Won’t Go Away.” SecurityIntelligence, 6 Aug. 2020. Web.

“The Role of Threat Intelligence in Vulnerability Management.” NOPSEC, 18 Sept. 2014. Accessed 18 Aug. 2020.

“Top 15 Paid and Free Vulnerability Scanner Tools in 2020.” DNSstuff, 6 Jan. 2020. Accessed 15 Sept. 2020.

Truta, Filip. “60% of Breaches in 2019 Involved Unpatched Vulnerabilities.” Security Boulevard, 31 Oct. 2019. Accessed 2 Nov. 2020.

“Vulnerability Management Program.” Core Security. Accessed 15 Sept. 2020.

“What is Risk-Based Vulnerability Management?” Balbix. Accessed 15 Sept. 2020.

White, Monica. “The Cost Savings of Effective Vulnerability Management (Part 1).” Kenna Security, 23 April 2020. Accessed 20 Sept. 2020.

Wilczek, Marc. “Average Cost of a Data Breach in 2020: $3.86M.” Dark Reading, 24 Aug. 2020. Accessed 5 Nov 2020.

Manage Requirements in an Agile Environment

Agile and requirements management are complementary, not competitors

Analyst's Perspective

The temptation when moving to Agile is to deemphasize good requirements practices in favor of perceived speed. If you're not delivering on the needs of the business then you have failed, regardless of how fast you've gone.

Delivery in Agile doesn't mean you stop needing solid business analysis. In fact, it's even more critical, to ensure your products and projects are adding value. With the rise of Agile, the role of the business analyst has been misunderstood.

As a result, we often throw out the analysis with the bathwater, thinking we'll be just fine without analysis, documentation, and deliberate action, as the speed and dexterity of Agile is enough.

Consequently, what we get is wasted time, money, and effort, with solutions that fail to deliver value, or need to be re-worked to get it right.

The best organizations find balance between these two forces, to align, and gain the benefits of both Agile and business analysis, working in tandem to manage requirements that bring solutions that are "just right".

Vincent Mirabelli
Principal Research Director, Applications Delivery and Management
Info-Tech Research Group

EXECUTIVE BRIEF

Executive Summary

Your Challenge

The process of navigating from waterfall to Agile can be incredibly challenging. And even more problematic; how do you operate your requirements management practices once there? Since there traditionally isn't a role for a business analyst; the traditional keeper of requirements. it isn't like switching on a light.

You likely find yourself struggling to deliver high quality solutions and requirements in Agile. This is a challenge for many organizations, regardless of how long they've leveraged Agile.

But you aren't here for assurances. You're here for answers and help.

Common Obstacles

many organizations and teams face is that there are so busy doing Agile that they fail to be Agile.

Agile was supposed to be the saving grace of project delivery but is misguided in taking the short-term view of "going quickly" at the expense of important elements, such as team formation and interaction, stakeholder engagement and communication, the timing and sequencing of analysis work, decision-making, documentation, and dealing with change.

The idea that good requirements just happen because you have user stories is wrong. So, requirements remain superficial, as you "can iterate later"…but sometimes later never comes, or doesn't come fast enough.

Organizations need to be very deliberate when aligning their Agile and requirements management practices. The work is the same. How the work is done is what changes.

Info-Tech's Approach

Infotech's advice? Why choose? Why have to pick between traditional waterfall and Agile delivery? If Agile without analysis is a recipe for disaster, Agile with analysis is the solution. And how can you leverage the Info-Tech approach to align your Agile and requirements management efforts into a powerful combination?

Manage Requirements in an Agile Environment is your guide.

Use the contents and exercises of this blueprint to gain a shared understanding of the two disciplines, to find your balance in your approach, to define your thresholds, and ultimately, to prepare for new ways of working.

Info-Tech Insight

Agile and requirements management are complementary, not competitors.

The temptation when moving to Agile is to deemphasize good requirements practices in favor of perceived speed. If you're not delivering on the needs of the business, then you have failed, regardless of how fast you've gone.

Insight summary

Overarching insight

Agile and requirements management are complementary, not competitors.

The temptation when moving to Agile is to deemphasize good requirements practices in favor of perceived speed. If you're not delivering on the needs of the business, then you have failed, regardless of how fast you've gone

Phase 1 insight

• The purpose of requirements in waterfall is for approval. The purpose in Agile is for knowledge management, as Agile has no memory.
• When it comes to the Agile manifesto, "over" does not mean "instead of".
• In Agile, the what of business analysis does doesn't change. What does change is the how and when that work happens.

Phase 2 insight

• Understand your uncertainties; it's a great way to decide what level of Agile (if any) is needed.
• Finding your "Goldilocks" zone will take time. Be patient.

Phase 3 insight

• Right-size your governance, based on team dynamics and project complexity. A good referee knows when to step in, and when to let the game flow.
• Agile creates a social contract amongst the team, and with their leaders and organization.
• Documentation needs to be valuable. Do what is acceptable and necessary to move work to future steps. Not documenting also comes with a cost, but one you pay in the future. And that bill will come due, with interest (aka, technical debt, operational inefficiencies, etc.).
• A lack of acceptable documentation makes it more difficult to have agility. You're constantly revalidating your current state (processes, practices and structure) and re-arguing decisions already made. This slows you down more than maintaining documentation ever would.

Phase 4 insight

• Making Agile predictable is hard, because people are not predictable; people are prone to chaos.

There have been many challenges with waterfall delivery

It turns out waterfall is not that great at reducing risk and ensuring value delivery after all

• Lack of flexibility
• Difficulty in measuring progress
• Difficulties with scope creep
• Limited stakeholder involvement
• Long feedback loops

48%
Had project deadlines more than double

85%
Exceeded their original budget by at least 20%

25%
At least doubled their original budget

Source: PPM Express.

Agile was meant to address the shortcomings of waterfall

The wait for solutions was too long for our business partners. The idea of investing significant time, money, and resources upfront, building an exhaustive and complete vision of the desired state, and then waiting months or even years to get that solution, became unpalatable for them. And rightfully so. Once we cast a light on the pains, it became difficult to stay with the status quo. Given that organizations evolve at a rapid pace, what was a pain at the beginning of an initiative may not be so even 6 months later.

Agile became the answer.

Since its' first appearance nearly 20 years ago, Agile has become the methodology of choice for a many of organizations. According to the 15th Annual State of Agile report, Agile adoption within software development teams increased from 37% in 2020 to 86% in 2021.

Adopting Agile led to challenges with requirements

Requirements analysis, design maturity, and management are critical for a successful Agile transformation.

"One of the largest sources of failure we have seen on large projects is an immature Agile implementation in the context of poorly defined requirements."
– "Large Scale IT Projects – From Nightmare to Value Creation"

"Requirements maturity is more important to project outcomes than methodology."
– "Business Analysis Benchmark: Full Report"

"Mature Agile practices spend 28% of their time on analysis and design."
– "Quantitative Analysis of Agile Methods Study (2017): Twelve Major Findings"

"There exists a Requirements Premium… organizations using poor practices spent 62% more on similarly sized projects than organizations using the best requirements practices."
– "The Business Case for Agile Business Analysis" - Requirements Engineering Magazine

Strong stakeholder satisfaction with requirements results in higher satisfaction in other areas

N= 324 small organizations from Info-Tech Research Group's CIO Business Vision diagnostic.

Note: High satisfaction was classified as organizations with a score greater or equal to eight and low satisfaction was every organization that scored below eight on the same questions.

Info-Tech's Agile requirements framework

Agile requirements are a balancing act

Collaboration

Many subject matter experts are necessary to create accurate requirements, but their time is limited too.

Communication

Stakeholders should be kept informed throughout the requirements gathering process, but you need to get the right information to the right people.

Documentation

Recording, organizing, and presenting requirements are essential, but excessive documentation will slow time to delivery.

Control

Establishing control points in your requirements gathering process can help confirm, verify, and approve requirements accurately, but stage gates limit delivery.

What changes for the business analyst?

In Agile, the what of business analysis does not change.

What does change is the how and when that work happens.

Business analysts need to focus on six key elements when managing requirements in Agile.

• Team formation and interaction
• Stakeholder engagement and communication
• The timing and sequencing of their work
• Decision-making
• Documentation
• Dealing with change

Where does the business analysis function fit on an Agile team?

Team formation is key, as Agile is a team sport

A business analyst in an Agile team typically interacts with several different roles, including:

• The product owner,
• The Sponsor or Executive
• The development team,
• Other stakeholders such as customers, end-users, and subject matter experts
• The Design team,
• Security,
• Testing,
• Deployment.

How we do our requirements work will change

• Team formation and interaction
• Stakeholder engagement and communication
• The timing and sequencing of their work
• Decision-making
• Documentation
• Dealing with change

As a result, you'll need to focus on;

• Emphasizing flexibility
• Enabling continuous delivery
• Enhancing collaboration and communication
• Developing a user-centered approach

Get stakeholders on board with Agile requirements

1. Stakeholder feedback and management support are key components of a successful Agile Requirements.
2. Stakeholders can see a project's progression and provide critical feedback about its success at critical milestones.
3. Management helps teams succeed by trusting them to complete projects with business value at top of mind and by removing impediments that are inhibiting their productivity.
4. Agile will bring a new mindset and significant numbers of people, process, and technology changes that stakeholders and management may not be accustomed to. Working through these issues in requirements management enables a smoother rollout.
5. Management will play a key role in ensuring long-term Agile requirements success and ultimately rolling it out to the rest of the organization.
6. The value of leadership involvement has not changed even though responsibilities will. The day-to-day involvement in projects will change but continual feedback will ultimately dictate the success or failure of a project.

Measuring your success

Tracking metrics and measuring your progress

As you implement the actions from this Blueprint, you should see measurable improvements in;

• Team and stakeholder satisfaction
• Requirements quality
• Documentation cost

Without sacrificing time to delivery

Info-Tech's methodology for Agile requirements

Blueprint tools and templates

Key deliverable:

Agile Requirements Playbook

A practical playbook for aligning your teams and articulating the guidelines for managing your requirements in Agile

Each step of this blueprint is accompanied by supporting deliverables to help you accomplish your goals:

Documentation Calculator

A tool to help you answer the question: What is the right level of Agile requirements documentation for my organization?

Agile Requirements Assessment

Establishes your current maturity level, defines your target state, and supports planning to get there.

Agile Requirements Workbook

Supporting tools and templates in advancing your Agile requirements practice, to be used with the Agile Requirements Blueprint and Playbook.

Info-Tech offers various levels of support to best suit your needs

DIY Toolkit

"Our team has already made this critical project a priority, and we have the time and capability, but some guidance along the way would be helpful."

Guided Implementation

"Our team knows that we need to fix a process, but we need assistance to determine where to focus. Some check-ins along the way would help keep us on track."

Workshop

"We need to hit the ground running and get this project kicked off immediately. Our team has the ability to take this over once we get a framework and strategy in place."

Consulting

"Our team does not have the time or the knowledge to take this project on. We need assistance through the entirety of this project."

Diagnostics and consistent frameworks used throughout all four options

Workshop Overview

Contact your account representative for more information.
workshops@infotech.com 1-888-670-8889

Guided Implementation

A Guided Implementation (GI) is a series of calls with an Info-Tech analyst to help implement our best practices in your organization.

A typical GI is 10 calls over the course of 4 to 6 months.

Framing Agile and Business Analysis

Phase 1

Framing Agile and Business Analysis

This phase will walk you through the following activities:

• EXERCISE: What do Agile and requirements mean in your organization?
• ASSESSMENT: Agile requirements assessment
• KEY DELIVERABLE: Agile Requirements Playbook

This phase involves the following participants:

• Business analyst and project team
• Stakeholders
• Sponsor/Executive

Managing Requirements in an Agile Environment

Step 1.1

Understand the benefits and limitations of Agile and business analysis

Activities

1.1.1 Define what Agile and business analysis mean in your organization

This step involves the following participants:

• Business analyst and project team
• Sponsor/Executive

Outcomes of this step

• Recognize the benefits and detriments of both Agile and business analysis

Framing Agile and Business Analysis

There have been many challenges with waterfall delivery

It turns out waterfall is not that great at reducing risk and ensuring value delivery after all

• Lack of flexibility
• Difficulty in measuring progress
• Difficulties with scope creep
• Limited stakeholder involvement
• Long feedback loops

48%
Had project deadlines more than double

85%
Exceeded their original budget by at least 20%

25%
At least doubled their original budget

Source: PPM Express.

Business analysis had a clear home in waterfall

Business analysts had historically been aligned to specific lines of business, in support of their partners in their respective domains. Somewhere along the way, the function was moved to IT. Conceptually this made sense, in that it allowed BAs to provide technical solutions to complex business problems. This had the unintended result of lost domain knowledge, and connection to the business.

It all starts with the business. IT enables business goals. The closer you can get to the business, the better.

Business analysts were the main drivers of helping to define the business requirements, or needs, and then decompose those into solution requirements, to develop the best option to solve those problems, or address those needs. And the case for good analysis was clear. The later a poor requirement was caught, the more expensive it was to fix. And if requirements were poor, there was no way to know until much later in the project lifecycle, when the cost to correct them was exponentially higher, to the tune of 10-100x the initial cost.

Adapted from PPM Express. "Why Projects Fail: Business Analysis is the Key".

Agile was meant to address the shortcomings of waterfall

The wait for solutions was too long for our business partners. The idea of investing significant time, money, and resources upfront, building an exhaustive and complete vision of the desired state, and then waiting months or even years to get that solution became unpalatable for them. And rightfully so. Once we cast a light on the pains, it became difficult to stand pat in the current state. And besides, organizations evolve at a rapid pace. What was a pain at the beginning of an initiative may not be so even six months later.

Agile became the answer.

Since its first appearance nearly 20 years ago, Agile has become the methodology of choice for a huge swathe of organizations. According to the 15th Annual State of Agile report, Agile adoption within software development teams increased from 37% in 2020 to 86% in 2021.

To say that's significant is an understatement.

The four core values of Agile helped shift focus

According to the Agile manifesto, "We value. . ."

"…while there is value in the items on the right, we value the items on the left more."

Source: Agilemanifesto, 2001

Agile has made significant inroads in IT and beyond

94% of respondents report using Agile practices in their organization

according to Digital.AI's "The 15th State of Agile Report"

That same report notes a steady expansion of Agile outside of IT, as other areas of the organization seek to benefit from increased agility and responsiveness, including Human Resources, Finance and Marketing.

While it addressed some problems…

"Agile projects are 37% faster to market than [the] industry average"

(Requirements Engineering Magazine, 2017)

• Business requirements documents are massive and unreadable
• Waterfall erects barriers and bottlenecks between the business and the development team
• It's hard to define the solution at the outset of a project
• There's a long turnaround between requirements work and solution delivery
• Locking in requirements dictates an often-inflexible solution. And the costs to make changes tend to add up.

…Implementing Agile led to other challenges

Adopting Agile led to challenges with requirements

Requirements analysis, design maturity, and management are critical for a successful Agile transformation.

"One of the largest sources of failure we have seen on large projects is an immature Agile implementation in the context of poorly defined requirements."
– BCG, 2015

"Requirements maturity is more important to project outcomes than methodology."
– IAG Consulting, 2009.

"Mature Agile practices spend 28% of their time on analysis and design."
– InfoQ, 2017."

"There exists a Requirements Premium… organizations using poor practices spent 62% more on similarly sized projects than organizations using the best requirements practices."
– Requirements Engineering Magazine, 2017

Strong stakeholder satisfaction with requirements results in higher satisfaction in other areas

N= 324 small organizations from Info-Tech Research Group's CIO Business Vision diagnostic.

Note: High satisfaction was classified as organizations with a score greater or equal to eight and low satisfaction was every organization that scored below eight on the same questions.

Agile is being misinterpreted as an opportunity to bypass planning and analysis activities

Agile is a highly effective tool.

This isn't about discarding Agile. It is being used for things completely outside of what was originally intended. When developing products or code, it is in its element. However, outside of that realm, its being used to bypass business analysis activities, which help define the true customer and business need.

Business analysts were forced to adapt and shift focus. Overnight they morphed into product owners, or no longer had a place on the team. Requirements and analysis took a backseat.

The result?

Increased rework, decreased stakeholder satisfaction, and a lot of wasted money and effort.

"Too often, the process of two-week sprints becomes the thing, and the team never gets the time and space to step back and obsess over what is truly needed to delight customers."
Harvard Business Review, 9 April 2021.

Info-Tech Insight

Requirements in Agile are the same, but the purpose of requirements changes.

• The purpose of requirements in waterfall is for stakeholder approval.
• The purpose of requirements in Agile is knowledge management; to maintain a record of the current state.

Many have misinterpreted the spirit of Agile and waterfall

The stated principles of waterfall say nothing of how work is to be linear.

Source: Royce, Dr. Winston W., 1970.

For more on Agile methodology, check out Info-Tech's Agile Research Centre

How did the pendulum swing so far?

Shorter cycles of work made requirements management more difficult. But the answer isn't to stop doing it.

Organizations went from engaging business stakeholders up front, and then not until solution delivery, to forcing those partners to give up their resources to the project. From taking years to deliver a massive solution (which may or may not even still fit the need) to delivering in rapid cycles called sprints.

This tug-of-war is costing organizations significant time, money, and effort.

Your approach to requirements management needs to be centered. We can start to make that shift by better aligning our Agile and business analysis practices. Outside of the product space, Agile needs to be combined with other disciplines (Harvard Business Review, 2021) to be effective.

Agility is important. Though it is not a replacement for approach or strategy (RCG Global Services, 2022). In Agile, team constraints are leveraged because of time. There is a failure to develop new capabilities to address the business needs Harvard Business Review, 2021).

Agility needs analysis.

Agile requirements are a balancing act

Collaboration

Many subject matter experts are necessary to create accurate requirements, but their time is limited too.

Communication

Stakeholders should be kept informed throughout the requirements gathering process, but you need to get the right information to the right people.

Documentation

Recording, organizing, and presenting requirements are essential, but excessive documentation will slow time to delivery.

Control

Establishing control points in your requirements gathering process can help confirm, verify, and approve requirements accurately, but stage gates limit delivery.

Start by defining what the terms mean in your organization

We do this because there isn't even agreement by the experts on what the terms "Agile" and "business analysis" mean, so let's establish a definition within the context of your organization.

1.1.1 What do Agile and business analysis mean in your organization?

Estimated time: 30 Minutes

1. Explore the motivations behind the need for aligning Agile with business analysis. Are there any current challenges related to outputs, outcomes, quality? How can the team and organization align the two more effectively for the purposes of requirements management?
2. Gather the appropriate stakeholders to discuss their definition of the terms "Agile" and "business analysis" It can be related to their experience, practice, or things they've read or heard.
3. Brainstorm and document all shared thoughts and perspectives.
4. Synthesize those thoughts and perspectives into a shared definition of each term, of a sentence or two.
5. Revisit this definition as needed, and as your Agile requirements efforts evolve.

Input

• Challenges and experiences/perspectives related to Agile and business requirements

Output

• A shared definition of Agile and business analysis, to help guide alignment on Agile requirements management

Materials

• Agile Requirements Workbook

Participants

• Business Analyst(s)
• Project Team
• Sponsor/Executive
• Relevant Stakeholders

Build your Agile Requirements Playbook

Keep the outcomes of this blueprint in a single document

Share at the beginning of a new project, as part of team member onboarding, and revisit as your practice matures.

Your Agile Requirements Playbook will include

• Your shared definition of Agile and business analysis for your organization
• The Agile Requirements Maturity Assessment
• A Methodology Selection Matrix
• Agile requirements RACI
• A defined Agile requirements process
• Documentation Calculator
• Your Requirements Repository Information
• Capability Gap List (from current to target state)
• Target State Improvement Roadmap and Action Plan

Step 1.2

Align Agile and Business Analysis Within Your Organization

Activities

1.2.1 Assess your Agile requirements maturity

This step involves the following participants:

• Business Analyst and Project Team
• Stakeholders
• Sponsor/Executive

Outcomes of this step

• Complete the Agile Requirements Maturity Assessment to establish your current and target states

Framing Agile and Business Analysis

Consider the question: "Why Agile?"

What is the driving force behind that decision?

There are many reasons to leverage the power of Agile within your organization, and specifically as part of your requirements management efforts. And it shouldn't just be to improve productivity. That's only one aspect.
Begin by asking, "Why Agile?" Are you looking to improve:

• Time to market
• Team engagement
• Product quality
• Customer satisfaction
• Stakeholder engagement
• Employee satisfaction
• Consistency in delivery of value
• Predictably of your releases

Or a combination of the above?

Info-Tech Insight

Project delivery methodologies aren't either/or. You don't have to be 100% waterfall or 100% Agile. Select the right approach for your project, product, or service.

In the end, your business partners don't want projects delivered faster, they want value faster!

For more on understanding Agile, check out the Implement Agile Practices That Work Blueprint

Responses to a 2019 KPMG survey:

13% said that their top management fully supports Agile transformation.

76% of organizations did not agree that their organization supports Agile culture.

62% of top management believe Agile has no implications for them.

What changes for the business analyst?

Business analysts need to focus on six key elements when managing requirements in Agile.

• Team formation and interaction
• Stakeholder engagement and communication
• The timing and sequencing of their work
• Decision-making
• Documentation
• Dealing with change

In Agile, the what of business analysis does not change.

What does change is the how and when that work happens.

1.2.1 Assess your Agile requirements maturity

1.2.1 Assess your Agile requirements maturity

Estimated time: 30 Minutes

1. Using the Agile Requirements Maturity Assessment, gather all appropriate stakeholders, and discuss and score the current state of your practice. Scoring can be done by:
1. Consensus: Generally better with a smaller group, where the group agrees the score and documents the result
2. Average: Have everyone score individually, and aggregate the results into an average, which is then entered.
3. Weighted Average: As above, but weight the individual scores by individual or line of business to get a weighted average.
2. When current state is complete, revisit to establish target state (or hold as a separate session) using the same scoring approach as in current state.
1. Recognize that there is a cost to maturity, so don't default to the highest score by default.
2. Resist the urge at this early stage to generate ideas to navigate from current to target state. We will re-visit this exercise in Phase 4, once we've defined other pieces of our process and practice.

Input

• Participant knowledge and experience

Output

• A current and target state assessment of your Agile requirements practice

Materials

• Agile Requirements Maturity Assessment

Participants

• Business Analyst(s)
• Project Team
• Sponsor/Executive
• Relevant Stakeholders

Tailoring Your Approach

Phase 2

This phase will walk you through the following activities:

• Selecting the appropriate delivery methodology
• Managing your requirements backlog
• Tracing from business need to user story

This phase involves the following participants:

• Business Analyst(s)
• Project Team
• Sponsor/Executive
• Relevant Stakeholders

Managing Requirements in an Agile Environment

Step 2.1

Confirm the Best-fit Approach for Delivery

Activities

2.1.1 Confirm your methodology

This step involves the following participants:

• Business Analyst(s)
• Project Team
• Sponsor/Executive
• Relevant Stakeholders

Outcomes of this step

• A review of potential delivery methodologies to select the appropriate, best-fit approach to your projects

Confirming you're using the best approach doesn't have be tricky

Selecting the right approach (or confirming you're on the right track) is easier when you assess two key inputs to your project; your level of certainty about the solution, and the level of complexity among the different variables and inputs to your project, such as team experience and training, the number of impacted stakeholders or context. lines of business, and the organizational

Solution certainty refers to the level of understanding of the problem and the solution at the start of the project. In projects with high solution certainty, the requirements and solutions are well defined, and the project scope is clear. In contrast, projects with low solution certainty have vague or changing requirements, and the solutions are not well understood.

Project complexity refers to the level of complexity of the project, including the number of stakeholders, the number of deliverables, and the level of technical complexity. In projects with high complexity, there are many stakeholders with different priorities, many deliverables, and high technical complexity. In contrast, projects with low complexity have fewer stakeholders, fewer deliverables, and lower technical complexity.

"Agile is a fantastic approach when you have no clue how you're going to solve a problem"

• Ryan Folster, Consulting Services Manager, Business Analysis, Dimension Data

Use Info-Tech's methodology selection matrix

Waterfall methodology is best suited for projects with high solution certainty and high complexity. This is because the waterfall model follows a linear and sequential approach, where each phase of the project is completed before moving on to the next. This makes it ideal for projects where the requirements and solutions are well-defined, and the project scope is clear.

On the other hand, Agile methodology is best suited for projects with low solution certainty. Agile follows an iterative and incremental approach, where the requirements and solutions are detailed and refined throughout the project. This makes it ideal for projects where the requirements and solutions are vague or changing.

Note that there are other models that exist for determining which path to take, should this approach not fit within your organization.

Use info-tech's-methodology-selection-matrix

Adapted from The Chaos Report, 2015 (The Standish Group)

Download the Agile Requirements Workbook

2.1.1 Confirm your methodology

Estimated time: 30 Minutes

1. Using the Agile Requirements Workbook, find the tab labelled "Methodology Assessment" and answer the questions to establish your complexity and certainty scores, where;

1 = Strongly disagree
2 = Disagree
3 = Neutral
4 = Agree
5 = Strongly agree.

2. In the same workbook, plot the results in the grid on the tab labelled "Methodology Matrix".
3. Projects falling into Green are good fits for Agile. Yellow are viable. And Red may not be a great fit for Agile.
4. Note: Ultimately, the choice of methodology is yours. Recognize there may be additional challenges when a project is too complex, or uncertainty is high.

Input

• Current project complexity and solution certainty

Output

• A clear choice of delivery methodology

Materials

• Agile Requirements Workbook

Participants

• Business Analyst(s)
• Project Team
• Sponsor/Executive
• Relevant Stakeholders

Step 2.2

Manage Your Requirements Backlog

Activities

2.2.1 Create your user stories

This step involves the following participants:

• Business Analyst(s)
• Project Team
• Sponsor/Executive
• Relevant Stakeholders

Outcomes of this step

• Understand how to convert requirements into user stories, which populate the Requirements Backlog.

Tailoring Your Approach

There is a hierarchy to requirements

	Higher-level statements of the goals, objectives, or needs of the enterprise. Business requirements focus on the needs of the organization, and not the stakeholders within it.	Defines
		Intended benefits and outcomes
	Statements of the needs of a particular stakeholder or class of stakeholders, and how that stakeholder will interact with a solution.	Why it is needed, and by who
	Describes the characteristics of a solution that meets business requirements and stakeholder requirements. Functional describes the behavior and information that the solution will manage. They describe capabilities the system will be able to perform in terms of behaviors or operations. Non-functional represents constraints on the ultimate solution and tends to be less negotiable.	What is needed, and how its going to be achieved

Connect the dots with a traceability matrix

Business requirements describe what a company needs in order to achieve its goals and objectives. Solution requirements describe how those needs will be met. User stories are a way to express the functionality that a solution will provide from the perspective of an end user.

A traceability matrix helps clearly connect and maintain your requirements.

To connect business requirements to solution requirements, you can start by identifying the specific needs that the business has and then determining how those needs can be met through technology or other solutions; or what the solution needs to do to meet the business need. So, if the business requirement is to increase online sales, a solution requirement might include implementing a shopping cart feature on your company website.

Once you have identified the solution requirements, you can then use those to create user stories. A user story describes a specific piece of functionality that the solution will provide from the perspective of a user.

For example, "As a customer, I want to be able to add items to my shopping cart so that I can purchase them." This user story is directly tied to the solution requirement of implementing a shopping cart feature.

Tracing from User Story back up to Business Requirement is essential in ensuring your solutions support your organization's strategic vison and objectives.

Download the Info-Tech Requirements Traceability Matrix

Improve the quality of your solution requirements

A solution requirement is a statement that clearly outlines the functional capability that the business needs from a system or application.

For more on developing high quality requirements, check out the Improve Requirements Gathering Blueprint

Prioritize your requirements

When everything is a priority, nothing is a priority.

Prioritization is the process of ranking each requirement based on its importance to project success. Each requirement should be assigned a priority level. The delivery team will use these priority levels to ensure efforts are targeted toward the proper requirements as well as to plan features available on each release. Use the MoSCoW Model of Prioritization to effectively order your requirements.

The MoSCoW Model of Prioritization

The MoSCoW model was introduced by Dai Clegg of Oracle UK in 1994

(Source: ProductPlan).

Base your prioritization on the right set of criteria

Info-Tech Insight

It is easier to prioritize requirements if they have already been collapsed, resolved, and rewritten. There is no point in prioritizing every requirement that is elicited up front when some of them will eventually be eliminated.

Manage solution requirements in a Product backlog

What is a backlog?

Agile teams are familiar with the use of a Sprint Backlog, but in Requirements Management, a Product Backlog is a more appropriate choice.

A product backlog and a Sprint backlog are similar in that they are both lists of items that need to be completed in order to deliver a product or project, but there are some key differences between the two.

A product backlog is a list of all the features, user stories, and requirements that are needed for a product or project. It is typically created and maintained by the business analyst or product owner and is used to prioritize and guide the development of the product.

A Sprint backlog, on the other hand, is a list of items specifically for an upcoming sprint, which is an iteration of work in Scrum. The Sprint backlog is created by the development team and is used to plan and guide the work that will be done during the sprint. The items in the Sprint backlog are typically taken from the product backlog and are prioritized based on their importance and readiness.

For more on building effective product backlogs, visit Deliver on Your Digital Product Vision

A backlog stores and organizes requirements at various stages

Your backlog must give you a holistic understanding of demand for change in the product.

A well-formed backlog can be thought of as a DEEP backlog

Detailed appropriately: Requirements are broken down and refined as necessary

Emergent: The backlog grows and evolves over time as requirements are added and removed.

Estimated: The effort to deliver a requirement is estimated at each tier.

Prioritized: A requirement's value and priority are determined at each tier.

Adapted from Essential Scrum

Ensure requests and requirements are ready for development

Clearly define what it means for a requirement, change, or maintenance request to be ready for development.

This will help ensure the value and scope of each functionality and change are clear and well understood by both developers and stakeholders before the start of the sprint. The definition of ready should be two-fold: ready for the backlog, and ready for coding.

1. Create a checklist that indicates when a requirement or request is ready for the development backlog. Consider the following questions:
1. Is the requirement or request in the correct format?
2. Does the desired functionality or change have significant business value?
3. Can the requirement or request be reasonably completed within defined release timelines under the current context?
4. Does the development team agree with the budget and points estimates?
5. Is there an understanding of what the requirement or request means from the stakeholder or user perspective?
2. Create a checklist that indicates when a requirement or request is ready for development. Consider the following questions:
1. Have the requirements and requests been prioritized in the backlog?
2. Has the team sufficiently collaborated on how the desired functionality or change can be completed?
3. Do the tasks in each requirement or request contain sufficient detail and direction to begin development?
4. Can the requirement or request be broken down into smaller pieces?

Converting solution requirements into user stories

Define the user

Who will be interacting with the product or feature being developed? This will help to focus the user story on the user's needs and goals.

Create the story

Create the user story using the following template: "As a [user], I want [feature] so that [benefit]."
This helps articulate the user's need and the value that the requirement will provide.

Decompose

User stories are typically too large to be implemented in a single sprint, so they should be broken down into smaller, more manageable tasks.

Prioritize

User stories are typically too large to be implemented in a single sprint, so they should be broken down into smaller, more manageable tasks.

2.2.1 Create your user stories

Estimated time: 60 Minutes

1. Gather the project team and relevant stakeholders. Have access to your current list of solution requirements.
2. Leverage the approach on previous slide "Converting Solution Requirements into User Stories" to generate a collection of user stories.

NOTE: There is not a 1:1 relationship between requirements and user stories.
It is possible that a single requirement will have multiple user stories, and similarly, that a single user story will apply to multiple solution requirements.

Input

• Requirements
• Use Case Template

Output

• A collection of user stories

Materials

• Current Requirements

Participants

• Business Analyst(s)
• Project Team
• Relevant Stakeholders

Use the INVEST model to create good user stories

At this point your requirements should be high-level stories. The goal is to refine your backlog items, so they are . . .

Independent: Ideally your user stories can be built in any order (i.e. independent from each other). This allows you to prioritize based on value and not get caught up in sequencing and prerequisites. Negotiable: As per the Agile principle, collaboration over contracts. Your user stories are meant to facilitate collaboration between the developer and the business. Therefore, they should be built to allow negotiation between all parties. Valuable: A user story needs to state the value so it can be effectively prioritized, but also so developers know what they are building. Estimable: As opposed to higher-level approximation given to epics, user stories need more accuracy in their estimates in order to, again, be effectively prioritized, but also so teams can know what can fit into a sprint or release plans. Small: User stories should be small enough for a number of them to fit into a sprint. However, team size and velocity will impact how many can be completed. A general guideline is that your teams should be able to deliver multiple stories in a sprint. Testable: Your stories need to be testable, which means they must have defined acceptance criteria and any related test cases as defined in your product quality standards. Source: Agile For All

Defining Your Requirements Thresholds

Phase 3

Defining Your Requirements Thresholds

This phase will walk you through the following activities:

• Assigning roles and responsibilities optional (Tool: RACI)
• Define your Agile requirements process
• Calculate the cost of your documentation (Tool: Documentation Calculator)
• Define your backlog refinement plan

This phase involves the following participants:

• Business Analyst(s)
• Project Team
• Sponsor/Executive
• Relevant Stakeholders

Managing Requirements in an Agile Environment

Step 3.1

Define Project Roles and Responsibilities

Activities

3.1.1 Define your Agile requirements RACI (optional)

3.1.2 Define your Agile requirements process

Defining Your Requirements Thresholds

This step involves the following participants:

• Business Analyst(s)
• Project Team
• Sponsor/Executive
• Relevant Stakeholders

Outcomes of this step

• A defined register of roles and responsibilities, along with a defined process for how Agile requirements work is to be done.

Defining Your Requirements Thresholds

Where does the BA function fit on an Agile team?

Team formation is key, as Agile is a team sport

A business analyst in an Agile team typically interacts with several different roles, including the product owner, development team, and many other stakeholders throughout the organization.

• The product owner, to set the priorities and direction of the project, and to gather requirements and ensure they are being met. Often, but not always, the BA and product owner are the same individual.
• The development team, to provide clear and concise requirements that they can use to build and test the product.
• Other stakeholders, such as customers, end-users, and subject matter experts to gather their requirements, feedback and validate the solution.
  • Design, to ensure that the product meets user needs. They may provide feedback and ensure that the design is aligned with requirements.
  • Security, to ensure that the solution meets all necessary security requirements and to identify potential risks and appropriate use of controls.
  • Testing, to ensure that the solution is thoroughly tested before it is deployed. They may create test cases or user scenarios that validate that everything is working as intended.
  • Deployment, to ensure that the necessary preparations have been made, including testing, security, and user acceptance.

Additionally, during the sprint retrospectives, the team will review their performance and find ways to improve for the next sprint. As a team member, the business analyst helps to identify areas where the team could improve how they are working with requirements and understand how the team can improve communication with stakeholders.

3.1.1 (Optional) Define Your Agile Requirements RACI

Estimated Time: 60 Minutes

1. Identify the project deliverables: The first step is to understand the project deliverables and the tasks that are required to complete them. This will help you to identify the different roles and responsibilities that need to be assigned.
2. Define the roles and responsibilities: Identify the different roles that will be involved in the project and their associated responsibilities. These roles may include project manager, product owner, development team, stakeholders, and any other relevant parties.
3. Assign RACI roles: Assign a RACI role to each of the identified tasks. The RACI roles are:
1. Responsible: the person or team who is responsible for completing the task
2. Accountable: the person who is accountable for the task being completed on time and to the required standard
3. Consulted: the people or teams who need to be consulted to ensure the task is completed successfully
4. Informed: the people or teams who need to be informed of the task's progress and outcome
4. Create the RACI chart: Use the information gathered in the previous steps to create a matrix or chart that shows the tasks, the roles, and the RACI roles assigned to each task.
5. Review and refine: Review the RACI chart with the project team and stakeholders to ensure that it accurately reflects the roles and responsibilities of everyone involved. Make any necessary revisions and ensure that all parties understand their roles and responsibilities.
6. Communicate and implement: Communicate the RACI chart to all relevant parties and ensure that it is used as a reference throughout the project. This will help to ensure that everyone understands their role and that tasks are completed on time and to the required standard.

Input

• A list of required tasks and activities
• A list of stakeholders

Output

• A list of defined roles and responsibilities for your project

Materials

• Agile Requirements Workbook

Participants

• Business Analyst(s)
• Project Team
• Sponsor/Executive
• Relevant Stakeholders

A Case Study in Team Formation

Industry: Anonymous Organization in the Energy sector
Source: Interview

Challenge

Agile teams were struggling to deliver within a defined sprint, as there were consistent delays in requirements meeting the definition of ready for development. As such, sprints were often delayed, or key requirements were descoped and deferred to a future sprint.

During a given two-week sprint cycle, the business analyst assigned to the team would be working along multiple horizons, completing elicitation, analysis, and validation, while concurrently supporting the sprint and dealing with stakeholder changes.

Solution

As a part of addressing this ongoing pain, a pilot program was run to add a second business analyst to the team.

The intent was, as one is engaged preparing requirements through elicitation, analysis, and validation for a future sprint, the second is supporting the current sprint cycle, and gaining insights from stakeholders to refine the requirements backlog.

Essentially, these two were leap-frogging each other in time. At all times, one BA was focused on the present, and one on the future.

Result

A happier team, more satisfied stakeholders, and consistent delivery of features and functions by the Agile teams. The pilot team outperformed all other Agile teams in the organization, and the "2 BA" approach was made the new standard.

Understanding the Agile requirements process

Shorter cycles make effective requirements management more necessary, not less

Short development cycles can make requirements management more difficult because they often result in a higher rate of change to the requirements. In a shorter timeframe, there is less time to gather and verify requirements, leading to a higher likelihood of poor or incomplete requirements. Additionally, there may be more pressure to make decisions quickly, which can lead to less thorough analysis and validation of requirements. This can make it more challenging to ensure that the final solution meets the needs of the stakeholders.
When planning your requirements cycles, it's important to consider;

• Your sprint logistics (how long?)
• Your release plan (at the end of every sprint, monthly, quarterly?)
• How the backlog will be managed (as tickets, on a visual medium, such as a Kanban board?)
• How will you manage communication?
• How will you monitor progress?
• How will future sprint planning happen?

Info-Tech's Agile requirements framework

	Sprint N(-1)	Sprint N	Sprint N(+1)
Changes from waterfall to Agile	Gathering and documenting requirements: Requirements are discovered and refined throughout the project, rather than being gathered and documented up front. This can be difficult for business analysts who are used to working in a waterfall environment where all requirements are gathered and documented before development begins. Prioritization of requirements: Requirements are prioritized based on their value to the customer and the team's ability to deliver them. This can be difficult for business analysts who are used to prioritizing requirements based on the client's needs or their own understanding of what is important.	Defining acceptance criteria: Acceptance criteria are defined for each user story to ensure that the team understands what needs to be delivered. Business analysts need to understand how to write effective acceptance criteria and how to use them to ensure that the team delivers what the customer needs. Supporting Testing and QA: The business analyst plays a role in ensuring that testing (and test cases) are completed and of proper quality, as defined in the requirements.	Managing changing requirements: It is expected that requirements will change throughout the project. Business analysts need to be able to adapt quickly to changing requirements and ensure that the team is aware of the changes and how they will impact the project. Collaboration with stakeholders: Requirements are gathered from a variety of stakeholders, including customers, users, and team members. Business analysts need to be able to work effectively with all stakeholders to gather and refine requirements and ensure that the team is building the right product.

3.1.2 Define your Agile requirements process

Estimated time: 60 Minutes

1. Gather all relevant stakeholders to discuss and define your process for requirements management.
2. Have a team member facilitate the session to define the process. The sample in the Agile Requirements Workbook can be used optionally as a starting point. You can also use any existing processes and procedures as a baseline.
3. Gain agreement on the process from all involved stakeholders.
4. Revisit the process periodically to review its performance and make adjustments as needed.

NOTE: The process is intended to be at a high enough level to leave space and flexibility for team members to adapt and adjust, but at a sufficient depth that everyone understands the process and workflows. In other words, the process will be both flexible and rigid, and the two are not mutually exclusive.

Input

• Project team and RACI
• Existing Process (if available)

Output

• A process for Agile requirements that is flexible yet rigid

Materials

• Agile Requirements Workbook

Participants

• Business Analyst(s)
• Project Team
• Sponsor/Executive
• Relevant Stakeholders

Establish the right level of governance and decision-making

Establishing the right level of governance and decision making is important in Agile requirements because there is a cost to decision making, as time plays an important factor. Even the failure to decide can have significant impacts.

Good governance and decision-making practices can help to minimize risks, ensure that requirements are well understood and managed, and that project progress is tracked and reported effectively.

In Agile environments, this often involves establishing clear roles and responsibilities, implementing effective communication and collaboration practices, and ensuring that decision-making processes are efficient and effective.

Good requirements management practices can help to ensure that projects are aligned with organizational goals and strategy, that stakeholders' needs are understood and addressed, and that deliverables are of high quality and meet the needs of the business.

By ensuring that governance and decision-making is effective, organizations can improve the chances of project success, and deliver value to the business. Risks and costs can be mitigated by staying small and nimble.

Check out Make Your IT Governance Adaptable

Develop an adaptive governance process

Maturing governance is a journey

Organizations should look to progress in their governance stages. Ad-hoc and controlled governance tends to be slow, expensive, and a poor fit for modern practices.

The goal as you progress through your stages is to delegate governance and empower teams to make optimal decisions in real-time, knowing that they are aligned with the understood best interests of the organization.

Automate governance for optimal velocity, while mitigating risks and driving value.

This puts your organization in the best position to be adaptive and able to react effectively to volatility and uncertainty.

Five key principles for building an adaptive governance framework

Delegate and empower

Decision making must be delegated down within the organization, and all resources must be empowered and supported to make effective decisions.

Define outcomes

Outcomes and goals must be clearly articulated and understood across the organization to ensure decisions are in line and stay within reasonable boundaries.

Make risk- informed decisions

Integrated risk information must be available with sufficient data to support decision making and design approaches at all levels of the organization.

Embed / automate

Governance standards and activities need to be embedded in processes and practices. Optimal governance reduces its manual footprint while remaining viable. This also allows for more dynamic adaptation.

Establish standards and behavior

Standards and policies need to be defined as the foundation for embedding governance practices organizationally. These guardrails will create boundaries to reinforce delegated decision making.

Sufficient decision-making power should be given to your Agile teams

Push the decision-making process down to your pilot teams.

• Bring your business stakeholders and subject matter experts together to identify the potential high-level risks.
• Bring your business stakeholders and subject matter experts together to identify the potential high-level risks.
• Discuss with the business the level of risk they are willing to accept.
• Define the level of authority project teams have in making critical decisions.

"Push the decision making down as far as possible, down to the point where sprint teams completely coordinate all the integration, development, and design. What I push up the management chain is risk taking. [Management] decides what level of risk they are willing to take and [they] demonstrate that by the amount of decision making you push down."
– Senior Manager, Canadian P&C Insurance Company, Info-Tech Interview

Step 3.2

Define Your Level of Acceptable Documentation

Activities

3.2.1 Calculate the cost of documentation

This step involves the following participants:

• Business Analyst(s)
• Project Team
• Relevant Stakeholders

Outcomes of this step

• Quantified cost of documentation produced for your Agile project.

Defining Your Requirements Thresholds

Right-size Your Documentation

Why do we need it, and what purpose does it serve?

Before creating any documentation, consider why; why are you creating documentation, and what purpose is it expected to serve?
Is it:

• … to gain approval?
• … to facilitate decision-making?
• .. to allow the team to think through a challenge or compare solution options?

Next, consider what level of documentation would be acceptable and 'enough' for your stakeholders. Recognize that 'enough' will depend on your stakeholder's personal definition and perspective.
There may also be considerations for maintaining documentation for the purposes of compliance, and auditability in some contexts and industries.
The point is not to eliminate all documentation, but rather, to question why we're producing it, so that we can create just enough to deliver value.

"What does the next person need to do their work well, to gain or create a shared understanding?"
- Filip Hendrickx, Innovating BA and Founder, altershape

Documentation comes at a cost

We need to quantify the cost of documentation, against the expected benefit

All things take time, and that would imply that all things have an inherent cost. We often don't think in these terms, as it's just the work we do, and costs are only associated with activities requiring additional capital expenditure. Documentation of requirements can come at a cost in terms of time and resources. Creating and maintaining detailed documentation requires effort from project team members, which could be spent on other aspects of the project such as development or testing. Additionally, there may be costs associated with storing and distributing the documentation.

When creating documentation, we are making a decision. There is an opportunity cost of investing time to create, and concurrently, not working on other activities. Documentation of requirements can come at a cost in terms of time and resources. Creating and maintaining detailed documentation requires effort from project team members, which could be spent on other aspects of the project such as development or testing. Additionally, there may be costs associated with storing and distributing the documentation.

In order to make better informed decisions about the types, quantity and even quality of the documentation we are producing, we need to capture that data. To ensure we are receiving good value for our documentation, we should compare the expected costs to the expected benefits of a sprint or project.

3.2.1 Calculate the cost of documentation

Estimated time: as needed

1. Use this tool to quantify the cost of creating and maintaining current state documentation for your Agile requirements team. It provides an indication, via the Documentation Cost Index, of when your project is documenting excessively, relative to the expected benefits of the sprint or project.
2. In Step 1, enter the hourly rate for the person (or persons) completing the business analysis function for your Agile team. NB: This does not have to be a person with the title of business analyst. If there are multiple people fulfilling this role, enter the average rate (if their rates are same or similar) or a weighted average (if there is a significant range in the hourly rate)
3. In Step 2, enter the expected benefit (in $) for the sprint or project.
4. In Step 3, enter the total number of hours spent on each task/activity during the sprint or project. Use blank spaces as needed to add tasks and activities not listed.
5. In Step 4, you'll find the Documentation Cost Index, which compares your total documentation cost to the expected benefits. The cell will show green when the value is < 0.8, yellow between 0.8 and 1, and red when >1.
6. Use the information to plan future sprints and documentation needs, identify opportunities for improvement in your requirements practice, and find balance in "just enough" documentation.

Input

• Project team and RACI
• Existing Process (if available)

Output

• A process for Agile requirements that is flexible yet rigid

Materials

• Agile Requirements Workbook

Participants

• Business Analyst(s)
• Project Team
• Sponsor/Executive
• Relevant Stakeholders

Lack of documentation also comes at a cost

Lack of documentation can bring costs to Agile projects in a few different ways.

• Onboarding new team members
• Improving efficiency
• Knowledge management
• Auditing and compliance
• Project visibility
• Maintaining code

Info-Tech Insight

Re-using deliverables (documentation, process, product, etc.) is important in maintaining the velocity of work. If you find yourself constantly recreating your current state documentation at the start of a project, it's hard to deliver with agility.

Step 3.3

Manage Requirements as an Asset

Activities

3.3.1 Discuss your current perspectives on requirements as assets

This step involves the following participants:

• Business Analyst(s)
• Project Team
• Relevant Stakeholders

Outcomes of this step

• Awareness of the value in, and tactics for enabling effective management of requirements as assets

Defining Your Requirements Thresholds

What do we mean by "assets"?

And when do requirements become assets?

In order to delivery with agility, you need to maximize the re-usability of artifacts. These artifacts could take the form of current state documentation, user stories, test cases, and yes, even requirements for re-use.
Think of it like a library for understanding where your organization is today. Understanding the people, processes, and technology, in one convenient location. These artifacts become assets when we choose to retain them, rather than discard them at the end of a project, when we think they'll no longer be needed.
And just like finding a single book in a vast library, we need to ensure our assets can be found when we need them. And this means making them searchable.
We can do this by establishing criteria for requirements and artifact reuse;

• What business need and benefit is it aligned to?
• What metadata needs to be attached, related to source, status, subject, author, permissions, type, etc.?
• Where will it be stored for ease of retrieval?

Info-Tech Insight

When writing requirements for products or services, write them for the need first, and not simply for what is changing.

The benefits of managing requirements as assets

Retention of knowledge in a knowledge base that allows the team to retain current business requirements, process documentation, business rules, and any other relevant information.
A clearly defined scope to reduce stakeholder, business, and compliance conflicts.
Impact analysis of changes to the current organizational assets.

Source: Requirement Engineering Magazine, 2017.

A case study in creating an asset repository

Industry: Anonymous Organization in the Government sector
Source: Interview

Challenge

A large government organization faced a challenge with managing requirements, processes, and project artifacts with any consistency.

Historically, their documentation was lacking, with multiple versions existing in email sent folders and manila folders no one could find. Confirming the current state at any given time meant the heavy lift of re-documenting and validating, so that effort was avoided for an excessive period.

Then there was a request for audit and compliance, to review their existing documentation practices. With nothing concrete to show, drastic recommendations were made to ensure this practice would end.

Solution

A small but effective team was created to compile and (if not available) document all existing project and product documentation, including processes, requirements, artifacts, business cases, etc.

A single repository was built and demonstrated to key stakeholders to ensure it would satisfy the needs of the audit and compliance group.

Result

A single source of truth for the organization, which was;

• Accessible (view access to the entire organization).
• Transparent (anyone could see and understand the process and requirements as intended).
• A baseline for continuous improvement, as it was clear what the one defined "best way" was.
• Current, where no one retained current documentation outside of this library.

3.3.1 Discuss your current perspectives on requirements as assets

Estimated time: 30 Minutes

1. Gather all relevant stakeholder to share perspectives on the use of requirements as assets, historically in the organization.
2. Have a team member facilitate the session. It is optional to document the findings.
3. After looking at the historical use of requirements as assets, discuss the potential uses, benefits, and drawbacks of managing as assets in the target state.

Input

• Participant knowledge and experience

Output

• A shared perspective and history on requirements as assets

Materials

• A method for data capture (optional)

Participants

• Business Analyst(s)
• Project Team
• Sponsor/Executive
• Relevant Stakeholders

Apply changes to baseline documentation

Baseline + Release Changes = New Baseline

• Start from baseline documentation dramatically to reduce cost and risk
• Treat all scope as changes to baseline requirements
• Sum of changes in the release scope
• Sum of changes and original baseline becomes the new baseline
• May take additional time and effort to maintain accurate baseline

What is the right tool?

While an Excel spreadsheet is great to start off, its limitations will become apparent as your product delivery process becomes more complex. Look at these solutions to continue your journey in managing your Agile requirements:

Step 3.4

Define Your Requirements Change Management Plan

Activities

3.4.1 Triage your requirements

This step involves the following participants:

• Business Analyst(s)
• Project Team
• Relevant Stakeholders

Outcomes of this step

• An approach for determining the appropriate level of governance over changes to requirements.

Expect and embrace change

In Agile development, change is expected and embraced. Instead of trying to rigidly follow a plan that may become outdated, Agile teams focus on regularly reassessing their priorities and adapting their plans accordingly. This means that the requirements can change often, and it's important for the team to have a process in place for managing these changes.

A common approach to managing change in Agile is to use a technique called "backlog refinement." Where previously we populated our backlog with requirements to get them ready for development and deployment, this involves regularly reviewing and updating the list of work to be done. The team will prioritize the items on the evolving backlog, and the prioritized items will be worked on during the next sprint. This allows the team to quickly respond to changes in requirements and stay focused on the most important work.

Another key aspect of managing change in Agile is effective communication. The team should have regular meetings, such as daily stand-up meetings or weekly sprint planning meetings, to discuss any changes in requirements and ensure that everyone is on the same page.

Best practices in change and backlog refinement

Communicate

Clearly communicate your change process, criteria, and any techniques, tools, and templates that are part of your approach.

Understand impacts/risks

Maintain consistent control and communication and ensure that an impact assessment is completed. This is key to managing risks.

Leverage tools

Leverage tools when you have them available. This could be a Requirements Management system, a defect/change log, or even by turning on "track changes" in your documents.

Cross-reference

For every change, define the source of the change, the reason for the change, key dates for decisions, and any supporting documentation.

Communicate the reason, and stay on message throughout the change

Leaders of successful change spend considerable time developing a powerful change message: a compelling narrative that articulates the desired end state and makes the change concrete and meaningful to staff. They create the change vision with staff to build ownership and commitment.

• The change message should:
• Explain why the change is needed.
• Summarize the things that will stay the same.
• Highlight the things that will be left behind.
• Emphasize the things that are being changed.
• Explain how the change will be implemented.
• Address how the change will affect the various roles in the organization.
• Discuss staff's role in making the change successful.

The five elements of communicating the reason for the change:

How to make the management of changes more effective

Key decisions and considerations

How will changes to requirements be codified?
How will intake happen?

• What is the submission process?
• Who has approval to submit?
• What information is needed to submit a request?

How will potential changes be triaged and evaluated?

• What criteria will be used to assess the impact and urgency of the potential change?
• How will you treat material and non-material changes?

What is the review and approval process?

• How will acceptance or rejection status be communicated to the submitter?

3.4.1 Triage Your requirements

If there's no material impact, update and move on

Material changes require oversight and approval

Planning Your Next Steps

Phase 4

Planning Your Next Steps

This phase will walk you through the following activities:

• Completing Your Agile Requirements Playbook
• EXERCISE: Capability Gap List

This phase involves the following participants:

• Business Analyst(s)
• Project Team
• Sponsor/Executive
• Relevant Stakeholders

Managing Requirements in an Agile Environment

Step 4.1

Preparing New Ways of Working

Activities

4.1.1 Define your communication plan

Planning Your Next Steps

This step involves the following participants:

• Business Analyst(s)
• Project Team
• Sponsor/Executive
• Relevant Stakeholders

Outcomes of this step

• Recognize the changes required on the team and within the broader organization, to bring stakeholders on board.

How we do requirements work will change

Change within the team, and in the broader organization

How to build an effective blend Agile and requirements management

Within the team

• Meetings should happen as needed
• Handoffs should be clear and concise
• Interactions should add value
• Stand-ups should similarly add value, and shouldn't be for status updates

Within the organization

• PMO inclusion, to ensure alignment across the organization
• Business/Operating areas, to recognize what they are committing to for time, resources, etc.
• Finance, for how your project or product is funded
• Governance and oversight, to ensure velocity is maintained

"Whether in an Agile environment or not, collaboration and relationships are still required and important…how you collaborate, communicate, and how you build relationships are key."
- Paula Bell, CEO, Paula A. Bell Consulting

Get stakeholders on board with Agile requirements

1. Stakeholder feedback and management support are key components of successful Agile requirements.
2. Stakeholders can see a project's progression and provide critical feedback about its success at critical milestones.
3. Management helps teams succeed by trusting them to complete projects with business value at top of mind and by removing impediments that are inhibiting their productivity.
4. Agile will bring a new mindset and significant amounts of people, process, and technology changes that stakeholders and management may not be accustomed to. Working through these issues in requirements management enables a smoother rollout.
5. Management will play a key role in ensuring long-term Agile requirements success and ultimately rolling it out to the rest of the organization.
6. The value of leadership involvement has not changed even though responsibilities will. The day-to-day involvement in projects will change but continual feedback will ultimately dictate the success or failure of a project.

4.1.1 Define your communication plan

Estimated time: 60 Minutes

1. Gather all relevant stakeholder to create a communication plan for project or product stakeholders.
2. Have a team member facilitate the session.
3. Identify
4. ;
1. Each stakeholder
2. The nature of information they are interested in
3. The channel or medium best to communicate with them
4. The frequency of communication
5. (Optional) Consider validating the results with the stakeholders, if not present.
6. Document the results in the Agile Requirements Workbook and include in Agile Requirements Playbook.
7. Revisit as needed, whether at the beginning of a new initiative, or over time, to ensure the content is still valid.

Input

• Participant knowledge and experience

Output

• A plan for communicating with stakeholders

Materials

• Agile Requirements Workbook

Participants

• Business Analyst(s)
• Project Team

Step 4.2

Develop a Roadmap for Next Steps

Activities

4.2.1 Develop your Agile requirements action plan

4.2.2 Prioritize with now, next, later

This step involves the following participants:

• Business Analyst(s)
• Project Team
• Sponsor/Executive
• Relevant Stakeholders

Outcomes of this step

• A comprehensive and prioritized list of opportunities and improvements to be made to mature the Agile requirements practice.

Planning Your Next Steps

Identify opportunities to improve and close gaps

Maturing at multiple levels

With a mindset of continuous improvement, there is always some way we can get better.

As you mature your Agile requirements practice, recognize that those gaps for improvement can come from multiple levels, from the organizational down to the individual.

Each level will bring challenges and opportunities.

The organization

• Organizational culture
• Organizational behavior
• Political will
• Unsupportive stakeholders

The team

• Current ways of working
• Team standards, norms and values

The individual

• Practitioner skills
• Practitioner experience
• Level of training received

Make sure your organization is ready to transition to Agile requirements management

	Learning: Agile is a radical change in how people work and think. Structured, facilitated learning is required throughout the transformation to help leaders and practitioners go from doing Agile to being Agile.
	Automation: While Agile is tool-agnostic at its roots, Agile work management tools and DevOps inspired SDLC tools that have become a key part of Agile practices.
	Integrated Teams: While temporary project teams can get some benefits from Agile, standing, self-organizing teams that cross business, delivery, and operations are essential to gain the full benefits of Agile.
	Metrics and Governance: Successful Agile implementations require the disciplined use of delivery and operations metrics that support governance focused on developing better teams.
	Culture: Agile teams believe that value is best created by standing, self-organizing cross-functional teams who deliver sustainably in frequent, short increments supported by leaders who coach them through challenges.

Info-Tech Insight

Agile gaps may only have a short-term, perceived benefit. For example, coding without a team mindset can allow for maximum speed to market for a seasoned developer. Post-deployment maintenance initiatives, however, often lock the single developer as no one else understands the rationale for the decisions that were made.

4.2.1 Develop your Agile requirements action plan

Estimated time: 60 Minutes

1. Gather all relevant stakeholder to create a road map and action plan for requirements management.
2. Have a team member facilitate the session using the results of the Agile Requirements Maturity Assessment.
3. Identify gaps from current to future state and brainstorm possible actions that can be taken to address those gaps. Resist the urge to analyze or discuss the feasibility of each idea at this stage. The intent is idea generation.
4. When the group has exhausted all ideas, the facilitator should group like ideas together, with support from participants. Discuss any ideas that are unclear or ambiguous.
5. Document the results in the Agile Requirements Workbook.

Note: the feasibility and timing of the ideas will happen in the following "Now, Next, Later" exercise.

Prioritize your roadmap

Taking steps to mature your Agile requirements practice.

The "Now, Next, Later" technique is a method for prioritizing and planning improvements or tasks. This involves breaking down a list of tasks or improvements into three categories: "Now" tasks are those that must be completed immediately. These tasks are usually urgent or critical, and they must be completed to keep the project or organization running smoothly. "Next" tasks are those that should be completed soon. These tasks are not as critical as "now" tasks, but they are still important and should be tackled relatively soon. "Later" tasks are those that can be completed later. These tasks are less critical and can be deferred without causing major problems. By using this technique, you can prioritize and plan the most important tasks first, while also allowing for flexibility and the ability to adjust plans as necessary. This process also helps you get a clear picture on what needs to be done first and what can be done later. This way you can work on the most important things first, and keep track of what you need to do next, for keeping the development/improvement process smooth and efficient.

Monitor your progress

Monitoring progress is important in achieving your target state. Be deliberate with your actions, to continue to mature your Agile requirements practice.

As you navigate toward your target state, continue to monitor your progress, your successes, and your challenges. As your Agile requirements practice matures, you should see improvements in the stated metrics below.

Establish a cadence to review these metrics, as well as how you are progressing on your roadmap, against the plan.

This is not about adding work, but rather, about ensuring you're heading in the right direction; finding the balance in your Agile requirements practice.

Appendix

Research Contributors and Experts

Emal Bariali
Business Architect & Business Analyst
Bariali Consulting

Emal Bariali is a Senior Business Analyst and Business Architect with 17 years of experience, executing nearly 20 projects. He has experience in both waterfall and Agile methodologies and has delivered solutions in a variety of forms, including custom builds and turnkey projects. He holds a Master's degree in Information Systems from the University of Toronto, a Bachelor's degree in Information Technology from York University, and a post-diploma in Software & Database Development from Seneca College.

Paula Bell
Paula A. Bell Consulting, LLC

Paula Bell is the CEO of Paula A Bell Consulting, LLC. She is a Business Analyst, Leadership and Career Development coach, consultant, speaker, and author with 21+ years of experience in corporate America in project roles including business analyst, requirements manager, business initiatives manager, business process quality manager, technical writer, project manager, developer, test lead, and implementation lead. Paula has experience in a variety of industries including media, courts, manufacturing, and financial. Paula has led multiple highly-visible multi-million-dollar technology and business projects to create solutions to transform businesses as either a consultant, senior business analyst, or manager.

Currently she is Director of Operations for Bridging the Gap, where she oversees the entire operation and their main flagship certification program.

Ryan Folster
Consulting Services Manager, Business Analysis
Dimension Data

Ryan Folster is a Business Analyst Lead and Product Professional from Johannesburg, South Africa. His strong focus on innovation and his involvement in the business analysis community have seen Ryan develop professionally from a small company, serving a small number of users, to large multi-national organizations. Having merged into business analysis through the business domain, Ryan has developed a firm grounding and provides context to the methodologies applied to clients and projects he is working on. Ryan has gained exposure to the Human Resources, Asset Management, and Financial Services sectors, working on projects that span from Enterprise Line of Business Software to BI and Compliance.

Ryan is also heavily involved in the local chapter of IIBA®; having previously served as the chapter president, he currently serves as a non-executive board member. Ryan is passionate about the role a Business Analyst plays within an organization and is a firm believer that the role will develop further in the future and become a crucial aspect of any successful business.

Filip Hendrickx
Innovating BA, Visiting Professor @ VUB
altershape

Filip loves bridging business analysis and innovation and mixes both in his work as speaker, trainer, coach, and consultant.

As co-founder of the BA & Beyond Conference and IIBA Brussels Chapter president, Filip helps support the BA profession and grow the BA community in and around Belgium. For these activities, Filip received the 2022 IIBA® EMEA Region Volunteer of the Year Award.

Together with Ian Richards, Filip is the author ofBrainy Glue, a business novel on business analysis, innovation and change. Filip is also co-author of the BCS book Digital Product Management and Cycles, a book, method and toolkit enabling faster innovation.

Fabricio Laguna
Professional Speaker, Consultant, and Trainer
TheBrazilianBA.com

Fabrício Laguna, aka The Brazilian BA, is the main reference on business analysis in Brazil. Author and producer of videos, articles, classes, lectures, and playful content, he can explain complex things in a simple and easy-to-understand way. IIBA Brazil Chapter president between 2012-2022. CBAP, AAC, CPOA, PMP, MBA. Consultant and instructor for more than 25 years working with business analysis, methodology, solution development, systems analysis, project management, business architecture, and systems architecture. His online courses are approved by students from 65 countries.

Ryland Leyton
Business Analyst and Agile Coach
Independent Consultant

Ryland Leyton, CBAP, PMP, CSM, is an avid Agile advocate and coach, business analyst, author, speaker, and educator. He has worked in the technology sector since 1998, starting off with database and web programming, gradually moving through project management and finding his passion in the BA and Agile fields. He has been a core team member of the IIBA Extension to the BABOK and the IIBA Agile Analysis Certification. Ryland has written popular books on agility, business analysis, and career. He can be reached at www.RylandLeyton.com.

Steve Jones
Supervisor, Market Support Business Analysis
ISO New England

Steve is a passionate analyst and BA manager with more than 20 years of experience in improving processes, services and software, working across all areas of software development lifecycle, business change and business analysis. He rejoices in solving complex business problems and increasing process reproducibility and compliance through the application of business analysis tools and techniques.

Steve is currently serving as VP of Education for IIBA Hartford. He is a CBAP, certified SAFe Product Owner/Product Manager, Six Sigma Green Belt, and holds an MS in Information Management and Communications.

Angela Wick
Founder
BA-Squared and BA-Cube

Founder of BA-Squared and BA-Cube.com, Angela is passionate about teaching practical, modern product ownership and BA skills. With over 20 years' experience she takes BA skills to the next level and into the future!
Angela is also a LinkedIn Learning instructor on Agile product ownership and business analysis, an IC-Agile Authorized Trainer, Product Owner and BA highly-rated trainer, highly-rated speaker, sought-after workshop facilitator, and contributor to many industry publications, including:

• IIBA BABOK v3 Core Team, leading author on the BABOK v3
• Expert Reviewer, IIBA Agile Extension to the BABOK
• PMI BA Practice Guide – Expert Reviewer
• PMI Requirements Management Practice Guide – Expert Reviewer
• IIBA Competency Model – Lead Author and Team Lead, V1, V2, and V3.

Rachael Wilterdink
Principal Consultant
Infotech Enterprises

Rachael Wilterdink is a Principal Consultant with Infotech Enterprises. With over 25 years of IT experience, she holds multiple business analysis and Agile certifications. As a consultant, Rachael has served clients in the financial, retail, manufacturing, healthcare, government, non-profit, and insurance industries. Giving back to the professional community, Ms. Wilterdink served on the boards of her local IIBA® and PMI® chapters. As a passionate public speaker, Rachael presents various topics at conferences and user groups across the country and the world. Rachael is also the author of the popular eBook "40 Agile Transformation Pain Points (and how to avoid or manage them)."

Bibliography

"2021 Business Agility Report: Rising to the Challenge." Business Agility, 2021. Accessed 13 June 2022.
Axure. "The Pitfalls of Agile and How We Got Here". Axure. Accessed 14 November 2022.
Beck, Kent, et al. "Manifesto for Agile Software Development." Agilemanifesto. 2001.
Brock, Jon, et al. "Large-Scale IT Projects: From Nightmare to Value Creation." BCG, 25 May 2015.
Bryar, Colin and Bill Carr. "Have We Taken Agile Too Far?" Harvard Business Review, 9 April 2021. Accessed 11 November, 2022.
Clarke, Thomas. "When Agile Isn't Responsive to Business Goals" RCG Global Services, Accessed 14 November 2022.
Digital.ai "The 15th State of Agile Report". Digital.ai. Accessed 21 November 2022.
Hackshall, Robin. "Product Backlog Refinement." Scrum Alliance. 9 Oct. 2014.
Hartman, Bob. "New to Agile? INVEST in good user stories." Agile For All.
IAG Consulting. "Business Analysis Benchmark: Full Report." IAG Consulting, 2009.
Karlsson, Johan. "Backlog Grooming: Must-Know Tips for High-Value Products." Perforce. 18 May 2018
KPMG. Agile Transformation (2019 Survey on Agility). KPMG. Accessed November 29.
Laguna, Fabricio "REQM guidance matrix: A framework to drive requirements management", Requirements Engineering Magazine. 12 September 2017. Accessed 10 November 2022.
Miller, G. J. (2013). Agile problems, challenges, & failures. Paper presented at PMI® Global Congress 2013—North America, New Orleans, LA. Newtown Square, PA: Project Management Institute.
Product Management: MoSCoW Prioritization." ProductPlan, n.d. Web.
Podeswa, Howard "The Business Case for Agile Business Analysis" Requirements Engineering Magazine. 21 February 2017. Accessed 7 November 2022.
PPM Express. "Why Projects Fail: Business Analysis is the Key". PPM Express. Accessed 16 November 2022.
Reifer, Donald J. "Quantitative Analysis of Agile Methods Study: Twelve Major Findings." InfoQ, 6 February, 2017.
Royce, Dr. Winston W. "Managing the Development of Large Software Systems." Scf.usc.edu. 1970. (royce1970.pdf (usc.edu))
Rubin, Kenneth S. Essential Scrum: A Practical Guide to the Most Popular Agile Process. Pearson Education. 2012.
Singer, Michael. "15+ Surprising Agile Statistics: Everything You Need To Know About Agile Management". Enterprise Apps Today. 22 August 2022.
The Standish Group. The Chaos Report, 2015. The Standish Group.

Where do I go next?

Improve Requirements Gathering

Back to basics: great products are built on great requirements.

Make the Case for Product Delivery

Align your organization on the practices to deliver what matters most.

Requirements for Small and Medium Enterprises

Right-size the guidelines of your requirements gathering process.

Implement Agile Practices that Work

Improve collaboration and transparency with the business to minimize project failure.

Create an Agile-Friendly Gating and Governance Model

Use Info-Tech's Agile Gating Framework as a guide to gating your Agile projects following a "trust but verify" approach.

Make Your IT Governance Adaptable

Governance isn't optional, so keep it simple and make it flexible.

Deliver on Your Digital Product Vision

Build a product vision your organization can take from strategy through execution.

Engineer Your Event Management Process

Track monitored events purposefully and respond effectively.

EXECUTIVE BRIEF

Analyst Perspective

Event management is useless in isolation.

Event management creates no value when implemented in isolation. However, that does not mean event management is not valuable overall. It must simply be integrated properly in the service management environment to inform and drive the appropriate actions.

Every step of engineering event management, from choosing which events to monitor to actioning the events when they are detected, is a purposeful and explicit activity. Ensuring that event management has open lines of communication and actions tied to related practices (e.g. problem, incident, and change) allows efficient action when needed.

Catalog your monitored events using a standardized framework to allow you to know:

1. The value of tracking the event.
2. The impact when the event is detected.
3. The appropriate, right-sized reaction when the event is detected.
4. The tool(s) involved in tracking the event.

Properly engineering event management allows you to effectively monitor and understand your IT environment and bolster the proactivity of the related service management practices.

Benedict Chang
Research Analyst, Infrastructure & Operations
Info-Tech Research Group

Executive Summary

Your Challenge

Strive for proactivity. Implement event management to reduce response times of technical teams to solve (potential) incidents when system performance degrades.

Build an integrated event management practice where developers, service desk, and operations can all rely on event logs and metrics.

Define the scope of event management including the systems to track, their operational conditions, related configuration items (CIs), and associated actions of the tracked events.

Common Obstacles

Managed services, subscription services, and cloud services have reduced the traditional visibility of on- premises tools.

System(s) complexity and integration with the above services has increased, making true cause and effect difficult to ascertain.

Info-Tech’s Approach

Clearly define a limited number of operational objectives that may benefit from event management.

Focus only on the key systems whose value is worth the effort and expense of implementing event management.

Understand what event information is available from the CIs of those systems and map those against your operational objectives.

Write a data retention policy that balances operational, audit, and debugging needs against cost and data security needs.

Info-Tech Insight

More is NOT better. Even in an AI-enabled world, every event must be collected with a specific objective in mind. Defining the purpose of each tracked event will cut down on data clutter and response time when events are detected.

Your challenge

This research is designed to help organizations who are facing these challenges or looking to:

• Build an event management practice that is situated in the larger service management environment.
• Purposefully choose events and to track as well as their related actions based on business-critical systems, their conditions, and their related CIs.
• Cut down on the clutter of current events tracked.
• Create a framework to add new events when new systems are onboarded.

33%

In 2020, 33% of organizations listed network monitoring as their number one priority for network spending. 27% of organizations listed network monitoring infrastructure as their number two priority.
Source: EMA, 2020; n=350

Common obstacles

These barriers make this challenge difficult to address for many organizations:

• Many organizations have multiple tools across multiple teams and departments that track the current state of infrastructure, making it difficult to consolidate event management into a single practice.
• Managed services, subscription services, and cloud services have reduced the traditional visibility of on-premises tools
• System(s) complexity and integration with the above services has increased, making true cause and effect difficult to ascertain.

Build event management to bring value to the business

33%

33% of all IT organizations reported that end users detected and reported incidents before the network operations team was aware of them.
Source: EMA, 2020; n=350

64%

64% of enterprises use 4-10 monitoring tools to troubleshoot their network.
Source: EMA, 2020; n=350

Info-Tech’s approach

Choose your events purposefully to avoid drowning in data.

The Info-Tech difference:

1. Start with a list of your most business-critical systems instead of data points to measure.
2. Decompose your business-critical systems into their configuration items. This gives you a starting point for choosing what to measure.
3. Choose your events and label them as notifications, warnings, or exceptions. Choose the relevant thresholds for each CI.
4. Have a pre-defined action tied to each event. That action could be to log the datapoint for a report or to open an incident or problem ticket.
5. With your event catalog defined, choose how you will measure the events and where to store the data.

Event management is useless in isolation

Define how event management informs other management practices.

Logging, Archiving, and Metrics

Monitoring and event management can be used to establish and analyze your baseline. The more you know about your system baselines, the easier it will be to detect exceptions.

Change Management

Events can inform needed changes to stay compliant or to resolve incidents and problems. However, it doesn’t mean that changes can be implemented without the proper authorization.

Automatic Resolution

The best use case for event management is to detect and resolve incidents and problems before end users or IT are even aware.

Incident Management

Events sitting in isolation are useless if there isn’t an effective way to pass potential tickets off to incident management to mitigate and resolve.

Problem Management

Events can identify problems before they become incidents. However, you must establish proper data logging to inform problem prioritization and actioning.

Info-Tech’s methodology for Engineering Your Event Management Process

Insight summary

Event management is useless in isolation.

The goals come from the pain points of other ITSM practices. Build handoffs to other service management practices to drive the proper action when an event is detected.

Start with business intent.

Trying to organize a catalog of events is difficult when working from the bottom up. Start with the business drivers of event management to keep the scope manageable.

Keep your signal-to-noise ratio as high as possible.

Defining tracked events with their known conditions, root cause, and associated actions allows you to be proactive when events occur.

Improve slowly over time.

Start small if need be. It is better and easier to track a few items with proper actions than to try to analyze events as they occur.

More is NOT better. Avoid drowning in data.

Even in an AI-enabled world, every event must be collected with a specific objective in mind. Defining the purpose of each tracked event will cut down on data clutter and response time when events are detected.

Add correlations in event management to avoid false positives.

Supplement the predictive value of a single event by aggregating it with other events.

Blueprint deliverables

Each step of this blueprint is accompanied by supporting deliverables to help you accomplish your goals:

Key deliverable:

Event Management Cookbook
Use the framework in the Event Management Cookbook to populate your event catalog with properly tracked and actioned events.

Event Management RACI
Define the roles and responsibilities needed in event management.

Event Management Workflow
Define the lifecycle and handoffs for event management.

Event Catalog
Consolidate and organize your tracked events.

Event Roadmap
Roadmap your initiatives for future improvement.

Blueprint benefits

IT Benefits

• Provide a mechanism to compare operating performance against design standards and SLAs.
• Allow for early detection of incidents and escalations.
• Promote timely actions and ensure proper communications.
• Provide an entry point for the execution of service management activities.
• Enable automation activity to be monitored by exception
• Provide a basis for service assurance, reporting and service improvements.

Business Benefits

• Less overall downtime via earlier detection and resolution of incidents.
• Better visibility into SLA performance for supplied services.
• Better visibility and reporting between IT and the business.
• Better real-time and overall understanding of the IT environment.

Case Study

An event management script helped one company get in front of support calls.

INDUSTRY - Research and Advisory

SOURCE - Anonymous Interview

Challenge

One staff member’s workstation had been infected with a virus that was probing the network with a wide variety of usernames and passwords, trying to find an entry point. Along with the obvious security threat, there existed the more mundane concern that workers occasionally found themselves locked out of their machine and needed to contact the service desk to regain access.

Solution

The system administrator wrote a script that runs hourly to see if there is a problem with an individual’s workstation. The script records the computer's name, the user involved, the reason for the password lockout, and the number of bad login attempts. If the IT technician on duty notices a greater than normal volume of bad password attempts coming from a single account, they will reach out to the account holder and inquire about potential issues.

Results

The IT department has successfully proactively managed two distinct but related problems: first, they have prevented several instances of unplanned work by reaching out to potential lockouts before they receive an incident report. They have also successfully leveraged event management to probe for indicators of a security threat before there is a breach.

Info-Tech offers various levels of support to best suit your needs

DIY Toolkit

“Our team has already made this critical project a priority, and we have the time and capability, but some guidance along the way would be helpful.”

Guided Implementation

“Our team knows that we need to fix a process, but we need assistance to determine where to focus. Some check-ins along the way would help keep us on track.”

Workshop

“We need to hit the ground running and get this project kicked off immediately. Our team has the ability to take this over once we get a framework and strategy in place.”

Consulting

“Our team does not have the time or the knowledge to take this project on. We need assistance through the entirety of this project.”

Diagnostics and consistent frameworks used throughout all four options

Guided Implementation

What does a typical GI on this topic look like?

A Guided Implementation (GI) is a series of calls with an Info-Tech analyst to help implement our best practices in your organization.

A typical GI is between 6 to 12 calls over the course of 4 to 6 months.

Workshop Overview

Contact your account representative for more information.
workshops@infotech.com 1-888-670-8889

Phase 1

Situate Event Management in Your Service Management Environment

Engineer Your Event Management Process

This phase will walk you through the following activities:

1.1.1 List your goals and challenges

1.1.2 Build a RACI chart for event management

1.2.1 Set your scope using business impact

This phase involves the following participants:

Infrastructure management team

IT managers

Step 1.1

Set Operational and Informational Goals

Activities

1.1.1 List your goals and challenges

1.1.2 Build a RACI chart for event management

Situate Event Management in Your Service Management Environment

This step will walk you through the following activities:

Set the overall scope of event management by defining the governing goals. You will also define who is involved in event management as well as their responsibilities.

This step involves the following participants:

Infrastructure management team

IT managers

Outcomes of this step

Define the goals and challenges of event management as well as their data proxies.

Have a RACI matrix to define roles and responsibilities in event management.

Situate event management among related service management practices

Event management needs to interact with the following service management practices:

• Incident Management – Event management can provide early detection and/or prevention of incidents.
• Availability and Capacity Management – Event management helps detect issues with availability and capacity before they become an incident.
• Problem Management – The data captured in event management can aid in easier detection of root causes of problems.
• Change Management – Event management can function as the rationale behind needed changes to fix problems and incidents.

Consider both operational and informational goals for event management

Event management may log real-time data for operational goals and non-real time data for informational goals

• Knowing what the outcomes are expected to achieve helps with the design of that process.
• A process targeted to fewer outcomes will generally be less complex, easier to adhere to, and ultimately, more successful than one targeted to many goals.
• Iterate for improvement.

1.1.1 List your goals and challenges

Gather a diverse group of IT staff in a room with a whiteboard.

Have each participant write down their top five specific outcomes they want from improved event management.

Consolidate similar ideas.

Prioritize the goals.

Record these goals in your Event Management Cookbook.

Input

• Pain points

Output

• Prioritized list of goals and outcomes

Materials

• Whiteboard/flip charts
• Sticky notes

Participants

• Infrastructure management team
• IT managers

Download the Event Management Cookbook

Event management is a group effort

• Event management needs to involve multiple other service management practices and service management roles to be effective.
• Consider the roles to the right to see how event management can fit into your environment.

Infrastructure Team

The infrastructure team is accountable for deciding which events to track, how to track, and how to action the events when detected.

Service Desk

The service desk may respond to events that are indicative of incidents. Setting a root cause for events allows for quicker troubleshooting, diagnosis, and resolution of the incident.

Problem and Change Management

Problem and change management may be involved with certain event alerts as the resultant action could be to investigate the root cause of the alert (problem management) or build and approve a change to resolve the problem (change management).

1.1.2 Build a RACI chart for event management

1. As a group, complete the RACI chart using the template to the right. RACI stands for the following:
   • Responsible. The person doing the work.
   • Accountable. The person who ensures the work is done.
   • Consulted. Two-way communication.
   • Informed. One-way communication
   • There must be one and only one accountable person for each task. There must also be at least one responsible person. Depending on the use case, RACI letters may be combined (e.g. AR means the person who ensures the work is complete but also the person doing the work).
2. Start with defining the roles in the first row in your own environment.
3. Look at the tasks on the first column and modify/add/subtract tasks as necessary.
4. Populate the RACI chart as necessary.

Download the Event Management Cookbook