Identify and Manage Operational Risk Impacts on Your Organization

Understand internal and external vendor risks to avoid potential disaster.

Analyst perspective

Organizations need to be aware of the operational damage vendors may cause to plan around those impacts effectively.

Organizations must be mindful that operational risks come from internal and external vendor sources. Missing either component in the overall risk assessment can significantly impact day-to-day business processes that cost revenue, delay projects, and lead to customer dissatisfaction.

Frank Sewell,

Research Director, Vendor Management
Info-Tech Research Group

Executive Summary

Info-Tech Insight

Organizations must evolve their risk assessments to be more adaptive to respond to threats in the market. Ongoing monitoring of the vendors tied to company operations, and understanding where those vendors impact your operations, is imperative to avoiding disasters.

Info-Tech’s multi-blueprint series on vendor risk assessment

There are many individual components of vendor risk beyond cybersecurity.

This series will focus on the individual components of vendor risk and how vendor management practices can facilitate organizations’ understanding of those risks.

Out of Scope:
This series will not tackle risk governance, determining overall risk tolerance and appetite, or quantifying inherent risk.

Operational risk impacts

Potential losses to the organization due to incidents that affect operations.

• In this blueprint we’ll explore operational risks, particularly from third-party vendors, and their impacts.
• Identify potentially disruptive events to assess the overall impact on organizations and implement adaptive measures to identify, manage, and monitor vendor performance.

The world is constantly changing

The IT market is constantly reacting to global influences. By anticipating changes, leaders can set expectations and work with their vendors to accommodate them.

When the unexpected happens, being able to adapt quickly to new priorities ensures continued long-term business success.

Below are some things no one expected to happen in the last few years:

Assess internal and external operational risk impacts

Due diligence and consistent monitoring are the keys to safeguarding your organization.

Internal

• Poorly vetted supplemental staff
• Bad system configurations
• Lack of relevant skills
• Poor vendor performance
• Failure to follow established processes
• Weak contractual accountability
• Unsupportable or end-of-life system components

External

• Cyberattacks
• Supply Chain Issues
• Geopolitical Disruptions
• Vendor Acquisitions
• N-Party Non-Compliance
• Vendor Fraud

Operational risk is the risk of losses caused by flawed or failed processes, policies, systems, or events that disrupt business operations.

Internal operational risk

Vendors operating within your secure perimeter can open your organization to substantial risk.

Frequently monitor your internal process around vendor management to ensure safe operations.

• Poorly vetted supplemental staff
• Bad system configurations
• Lack of relevant skills
• Poor vendor performance
• Failure to follow established processes
• Weak contractual accountability
• Unsupportable or end-of-life system components

Info-Tech Insight

You may have solid policies, but if your employees and vendors are not following them, they will not protect the organization.

External operational risks

• Cyberattacks
• Supplier issues and geopolitical instability
• Vendor acquisitions
• N-party vendor non-compliance

Identify and manage operational risks

Poorly configured systems

Failing to ensure that your vendor-supported systems are properly configured and that your vendors are meeting your IT change control and configuration standards is more commonplace than expected. Proper oversight and management of your support vendors are crucial to ensure they are meeting expectations in this regard.

Failure to follow processes

Most companies have policies and procedures around IT change and configuration control, security standards, risk management, vendor performance standards, etc. While having these processes is a good start, failure to perform continuous monitoring and management of these leads to increased risks of incidents.

Supply chain disruptions

Awareness of the supply chain's complications, and each organization's dependencies, are increasing for everyone. However, most organizations still do not understand the chain of n-party vendors that support their specific vendors or how interruptions in their supply chains could affect them. The 2022 Toyota shutdown due to Kojima is a perfect example of how one essential parts vendor could shut down your operations.

What to look for

Identify operational risk impacts

• Does the vendor have a business continuity plan they will share for your review?
• Is the vendor operating on old hardware that may be out of warranty or at end of life?
• Is the vendor operating on older software or shareware that may lack the necessary patches?
• Does the vendor self-audit, or do they use a vetted third-party audit firm to issue a SOC report annually?
• Does the vendor have sufficient personnel in acceptable regions to support your operations?
• Is the vendor willing to make concessions on contractual protections, or are they only offering “one-sided” agreements with “as-is” warranties?

Operational risks

Not knowing where your risks come from creates additional risks to operations.

• Supply chain disruptions and global shortages.
  • Geopolitical disruptions and natural disasters have caused unprecedented interruptions to business. Do you know where your critical vendors are getting their supplies? Are you aware of their business continuity plans to accommodate for those interruptions?
• Poor vendor performance.
  • Organizations need to understand where vendors are acting in their operations and manage the impact of replacing that vendor and cutting their losses rather than continuing to throw good money away after a bad performance.
• Vendor acquisitions.
  • A lot of acquisition is going on in the market today. Large companies are buying competitors, imposing new terms on customers, or removing competing products from the market. Understand your options if a vendor is acquired by a company with which you do not wish to be in a relationship.

It is important to identify where potential risks to your operations may come from to manage and potentially eliminate them from impacting your organization.

Info-Tech Insight

Most organizations realize that their vendors could operationally affect them if an incident occurs. Still, they fail to follow the chain of events that might arise from those incidents to understand the impact fully.

Prepare your vendor risk management for success

Due diligence will enable successful outcomes.

1. Obtain top-level buy-in; it is critical to success.
2. Build enterprise risk management (ERM) through incremental improvement.
3. Focus initial efforts on the “big wins” to prove the process works.
4. Use existing resources.
5. Build on any risk management activities that already exist in the organization.
6. Socialize ERM throughout the organization to gain additional buy‑in.
7. Normalize the process long term with ongoing updates and continuing education for the organization.

How to assess third-party operational risk

1. Review Organizational Operations

   Understand the organization’s operational risks to prepare for the “what if” game exercise.

2. Identify and Understand Potential Operational Risks

   Play the “what if” game with the right people at the table.

3. Create a Risk Profile Packet for Leadership

   Pull all the information together in a presentation document.

4. Validate the Risks

   Work with leadership to ensure that the proposed risks are in line with their thoughts.

5. Plan to Manage the Risks

   Lower the overall risk potential by putting mitigations in place.

6. Communicate the Plan

   It is important not only to have a plan but also to socialize it in the organization for awareness.

7. Enact the Plan

   Once the plan is finalized and socialized, put it in place with continued monitoring for success.

Insight summary

Operational risk impacts often come from unexpected places and have unforeseen impacts. Knowing where your vendors place in critical business processes and those vendors' business continuity plans concerning your organization should be a priority for those who manage the vendors.

Insight 1

Organizations fail to plan for vendor acquisitions appropriately.

Vendors routinely get acquired in the IT space. Does your organization have appropriate safeguards from inadvertently entering a negative relationship? Do you have plans around replacing critical vendors purchased in such a manner?

Insight 2

Organizations often fail to understand how they factor into a vendor’s business continuity plan.

If one of your critical vendors goes down, do you know how they intend to re-establish business? Do you know how you factor into their priorities?

Insight 3

Organizations need to have a comprehensive understanding of how their vendor-managed systems integrate with Operations.

Do you understand where in the business processes vendor-supported systems lie? Do you have contingencies around disruptions that account for those pieces missing from the process?

Identifying operational vendor risk

Who should be included in the discussion

• While it is true that executive-level leadership defines the strategy for an organization, it is vital for those making decisions to make informed decisions.
• Getting input from operational experts at your organization will enhance your organization's long-term potential for success.
• Involving those who not only directly manage vendors but also understand your business processes will aid in determining the forward path for relationships with your current vendors and identifying new emerging potential partners.

See the blueprint Build an IT Risk Management Program

Review your operational plans for new risks on a regular basis.

Keep in mind Risk = Likelihood x Impact (R=L*I).

Impact (I) tends to remain the same, while Likelihood (L) is becoming closer to 100% as threat actors become more prevalent

Managing vendor operational risk impacts

What can we realistically do about the risks?

• Review vendors’ business continuity plans and disaster recovery testing.
  • Understand your priority in their plans.
• Institute proper contract lifecycle management.
  • Make sure to follow corporate due diligence and risk assessment policies and procedures.
  • Failure to do so consistently can be a recipe for disaster.
• Develop IT governance and change control.
• Introduce continual risk assessment to monitor the relevant vendor markets.
  • Regularly review your operational plans for new risks and evolving likelihoods.
  • Risk = Likelihood x Impact (R=L*I).
    • Impact (I) tends to remain the same and be well understood, while Likelihood (L) may often be considered 100%.
• Be adaptable and allow for innovations that arise from the current needs.
  • Capture lessons learned from prior incidents to improve over time and adjust your plans accordingly.

Organizations need to review their organizational risk plans, considering the placement of vendors in their operations.

Pandemics, extreme weather, and wars that affect global supply chains are current realities, not unlikely scenarios.

Ongoing improvement

Incorporating lessons learned

• Over time, despite everyone’s best observations and plans, incidents will catch us off guard.
• When it happens, follow your incident response plans and act accordingly.
• An essential step is to document what worked and what did not – collectively known as the “lessons learned.”
• Use the lessons learned document to devise, incorporate, and enact a better risk management process.

Sometimes disasters occur despite our best plans to manage them.

When this happens, it is important to document the lessons learned and improve our plans going forward.

The "what if" game

1-3 hours

Vendor management professionals are in an excellent position to help senior leadership identify and pull together resources across the organization to determine potential risks. By playing the "what if" game and asking probing questions to draw out – or eliminate – possible adverse outcomes, everyone involved adds their insight into parts of the organization to gather a comprehensive picture of potential impacts.

• Break into smaller groups (or if too small, continue as a single group).
• Use the Operational Risk Impact Tool to prompt discussion on potential risks. Keep this discussion flowing organically to explore all potentials but manage the overall process to keep the discussion pertinent and on track.
• Collect the outputs and ask the subject matter experts (SMEs) for management options for each one in order to present a comprehensive risk strategy. You will use this to educate senior leadership so that they can make an informed decision to accept or reject the solution.

Download the Operational Risk Impact Tool

High risk example from tool

Being overly reliant on a single talented individual can impose risk to your operations. Make sure you include resiliency in your skill sets for critical business practices.

Low risk example from tool

Summary

Seek to understand all aspects of your operations.

• Organizations need to understand and map out where vendors are critical to their operations.
• Those organizations that consistently follow their established risk assessment and due diligence processes will be better positioned to avoid disasters.
• Bring the right people to the table to outline potential risks in the market and your organization.
• Understand how your vendors prioritize your organization in their business continuity processes.
• Incorporate “lessons learned” from prior incidents into your risk management process to build better plans for future issues.

Organizations must evolve their operational risk assessments considering their vendor portfolio.

Ongoing monitoring of the market and the vendors tied to company operations is imperative to avoiding disaster.

Related Info-Tech Research

Identify and Manage Financial Risk Impacts on Your Organization

• Vendor management practices educate organizations on the different potential financial impacts that vendors may incur and suggest systems to help manage them.
• Standardize your processes for identifying and monitoring vendor risks to manage financial impacts with our Financial Risk Impact Tool.

Identify and Manage Reputational Risk Impacts on Your Organization

• Vendor management practices educate organizations on the different potential risks to vendors in your market and suggest creative and alternative ways to avoid and help manage them.
• Standardize your processes for identifying and monitoring vendor risks to manage potential impacts on your reputation and brand with our Reputational Risk Impact Tool.

Identify and Manage Strategic Risk Impacts on Your Organization

• Vendor management practices educate organizations on the different potential risks to vendors in your market and suggest creative and alternative ways to avoid and help manage them.
• Standardize your processes for identifying and monitoring vendor risks to manage potential impacts on your strategic plan with our Strategic Risk Impact Tool.

Bibliography

“Weak Cybersecurity is taking a toll on Small Businesses.” Tripwire. August 7, 2022.

SecureLink 2022 White Paper SL_Page_EA+PAM (rocketcdn.me)

Member Poll March 2021 "Guide: Evolving Work Environments Impact of Covid-19 on Profile and Management of Third Parties.“ Shared Assessments. March 2021.

“Operational Risk.” Wikipedia.

Tonello, Matteo. “Strategic Risk Management: A Primer for Directors.” Harvard Law School Forum on Corporate Governance, August 23, 2012.

Frigo, Mark L., and Richard J. Anderson. “Embracing Enterprise Risk Management: Practical Approaches for Getting Started.” COSO, 2011.

IT Talent Trends 2022

The last two years have been a great experiment … but it’s not over yet.

Incorporate new ways of working into your business to build and keep the best team.

Over the past two years, organizations have ventured into unprecedented ways of working and supporting their employees, as they tried to maintain productivity through the pandemic. This experiment has made lasting changes to both business models and employee expectations, and these effects will continue to be seen long after we return to a “new normal.”

While the pandemic forced us to work differently for the past two years, looking forward, successful organizations will incorporate new ways of working into their business models – beyond simply having a remote work policy.

How we work, source roles, and develop talent continue to evolve as we navigate a different world with employees being more vocal in their desires, and leaders continue to play a key role.

The IT talent market will never be the same, and organizations must reevaluate their employee experience from the bottom up to successfully weather the shift to the new normal.

IT Talent Trends 2022

The pandemic has clarified employees’ needs and amplified their voices

If 2021 was about beginning to act on employee needs, 2022 will be about strategically examining each trend to ensure that the actions taken by the organization are more than lip service.

Employees have always been able to see through disingenuous attempts to engage them, but in 2022 the stakes are higher due to increased talent mobility.

Trends that were just starting to come into focus last year have established themselves as critical determinants of the employee experience in 2022.

2021			DEI: A Top Talent Objective	Remote Work Is Here to Stay	Uncertainty Unlocks Performance	A Shift in Skills Priorities	A Greater Emphasis on Wellbeing
2022	Strategic Recruiting Finds Good Talent Finding talent in a strained talent market requires a marketing approach. Posting a job description isn’t enough.	The (Not So) Great Resignation IT is faring better than other functions; however, specific industries need to pay attention.	Grow Your DEI Practices Into Meaningful Actions Good intentions are not enough.	Remote Work Is Here – Can Your Culture Adapt? The Great Experiment is over. Are leaders equipped to capitalize on its promises?		Management Skills Drive Success in a Remote World Despite the need for remote team management training, it is still not happening.

What employees are looking for is changing

Superficial elements of traditional office culture were stripped away by the quick shift to a remote environment, giving employees the opportunity to reevaluate what truly matters to them in a job.

The biggest change from 2019 (pre-pandemic) to today is increases in the importance of culture, flexible/remote work, and work-life balance.

Organizations that fail to keep up with this shift in priorities will see the greatest difficulty in hiring and retaining staff.

As an employee, which of the following would be important to you when considering a potential employer?

	2019		2021
Flexible Work	n=275		n=206
Work-Life Balance	n=277		n=206
Culture	n=277		n=206
Source: Info-Tech Talent Trends Survey data collected in 2019 and 2021	Very Important Somewhat Important Not at All Important

IT’s top talent priorities in 2022

IT Talent Trends 2022

Look beneath the surface of the trends to navigate them successfully

Above Ground Focusing on what you see 'Above the line" won't solve the problem. Talent isn't a checklist.	Strategic Recruiting Finds Good Talent Finding talent in a strained talent market requires a marketing approach. Posting a job description isn't enough. The number of job openings increased to 11.4 million on the last business day of October, up from 10.6 million in September (US Bureau of Labor Statistics, Dec. 2021)	The (Not So) Great Resignation IT is faring better than other functions; however, specific industries need to pay attention. In September, in the US, 4.4 million people left their jobs. That number dropped to 4.2 million in October. (US Labor Stats, Dec. 2021) 30% of workers will likely switch jobs if they have to return to the office full time. (McKinsey, Dec. 2021)	Grow Your DEI Practices Into Meaningful Actions Good intentions are not enough. 95% of organizations are focusing on DEI. (2022 HR Trends Report) 48% of IT departments have delivered training on DEI over the past year.	Remote Work is Here. Can Your Culture Adapt? The Great Experiment is over. Are you equipped to capitalize on its promises? 85% of organizations saw the same or higher productivity during the pandemic. 91% of organizations are continuing remote work.	Management Skills Drive Success in a Remote World Despite the need for remote team management training, it is still not happening. 72% of IT departments report high effectiveness at managing remote staff. Learning and development is IT's top priority.
Beneath the Surface For each trend, a strategic approach to get "under the line" will help form your response. Talent needs a holistic approach, as under the line everything is connected. If you are experiencing challenges in one area, analyzing data (e.g. engagement, exit surveys, effectiveness of DEI program and leader training) can help drive overall experience.	100% of job seekers cite culture as somewhat to very important. Only 40% of employers advertise culture in job postings.	70% of IT departments state voluntary turnover is less than 10% Top reasons for resignation are salary, development, and opportunity for innovative work. Resignation rates were higher in fields that had experienced extreme stress due to the pandemic (HBR, Dec. 2021)	Senior leadership is overestimating their own commitment to DEI. Most IT departments are not driving their own DEI initiatives. Without effectively measuring DEI practices, organizations will see 1.6x more turnover. (2022 HR Trends Report)	Senior leadership is not open to remote work in 23% of organizations. Without leadership support, employees will not buy into remote work initiatives. A remote work policy will not bring organizational benefits without employee buy-in.	75% of senior managers believe remote team management is highly effective, but only 60% of frontline staff agree. Training focuses on technical skills, to the exclusion of soft skills, including management and leadership.
Solutions Recommendations depending on your department's maturity level.	Attention is required for candidate experience underpinned by a realistic employee value proposition.	Gather and review existing data (e.g. early retirements, demographics) to understand your turnover rate.	Use employee engagement tools to gauge employee sentiment among impacted groups and build out an engagement strategy to meet those needs.	Conduct a cultural assessment to reveal hidden biases that may stand in the way of remote work efficacy.	Provide management training on performance management and development coaching.

This report is based on organizations just like yours

Survey timeline = October 2021
Total respondents = 245 IT professionals

01	United States	45%	08	Middle East	2%
02	Canada	23%	09	Other (Asia)	2%
03	Africa	8%	10	Germany	1%
04	Great Britain	6%	11	India	1%
05	Latin America, South America or Caribbean	4%	12	Netherlands	1%
06	Other (Europe)	4%	13	New Zealand	1%
07	Australia	2%			(N-245)

(n=191)

This report is based on organizations just like yours

Industry

Info-Tech IT Maturity Model

(n=225)

Innovator – Transforms the Business Reliable Technology Innovation Business Partner – Expands the Business Effective Execution Projects, Strategic Use of Analytics and Customer Technology Trusted Operator – Optimizes Business Effective Fulfillment of Work Orders, Functional Business Applications, and Reliable Data Quality Firefighter – Supports the Business Reliable Infrastructure and IT Service Desk Unstable – Struggles to Support Inability to Provide Reliable Business Services

This report is based on people just like you

Which of the following ethnicities (ethnicity refers to a group with a shared or common identity, culture, and/or language) do you identify with? Select all that apply.	What gender do you identify most with?
(N=245)	(n=228)

This report is based on people just like you

What is your sub-department of IT?	Which title best describes your position?
(n=227)	(N=245)

IT Talent Trends 2022

Each trend is introduced with key questions you can ask yourself to see how your department fares in that area.

The report is based on statistics from a survey of 245 of your peers.

It includes recommendations of next steps and a key metric to track your success.

It lists Info-Tech resources that you, as a member, can leverage to begin your journey to improve talent management in your department.

The report is based on data gathered from Info-Tech Research Group’s 2022 IT Talent Trends Survey. The data was gathered in September and October of 2021.

Strategic Recruiting Finds Good Talent

Trend 1 | The Battle to Find and Keep Talent

As the economy has stabilized, more jobs have become available, creating a job seeker’s market. This is a clear sign of confidence in the economy, however fragile, as new waves of the pandemic continue.

Info-Tech Point of View

Recruiting tactics are an outcome of a well-defined candidate experience and employee value proposition.

Introduction

	During our interviews, members that focused on sharing their culture with a strong employee value proposition were more likely to be successful in hiring their first-choice candidates.
	Questions to ask yourself Do you have a well-articulated employee value proposition? Are you using your job postings to market your company culture? Have you explored multiple channels for posting jobs to increase your talent pool of candidates?

47% of respondents are hiring external talent to fill existing gaps, with 40% using external training programs to upgrade current employees. (Info-Tech IT Talent Trends 2022 Survey)

In October, the available jobs (in the USA) unexpectedly rose to 11 million, higher than the 10.4 million experts predicted. (CNN Business, 2021)

Where has all the talent gone?

IT faces multiple challenges when recruiting for specialized talent

Talent scarcity is focused in areas with specialized skill sets such as security and architecture that are dynamic and evolving faster than other skill sets.

“It depends on what field you work in,” said ADP chief economist Nela Richardson. “There were labor shortages in those fields pre-pandemic and two years forward, there is even more demand for people with those skills” (CNBC, 19 Nov. 2021).

37% of IT departments are outsourcing roles to fill internal skill shortages. (Info-Tech Talent Trends 2022 Survey)

Roles Difficult to Fill

(Info-Tech Talent Trends 2022 Survey)

Case Study: Using culture to drive your talent pool

This case study is happening in real time. Please check back to learn more as Goddard continues to recruit for the position.

Recruiting at NASA

Goddard Space Center is the largest of NASA’s space centers with approximately 11,000 employees. It is currently recruiting for a senior technical role for commercial launches. The position requires consulting and working with external partners and vendors.

NASA is a highly desirable employer due to its strong culture of inclusivity, belonging, teamwork, learning, and growth. Its culture is anchored by a compelling vision, “For the betterment of Humankind,” and amplified by a strong leadership team that actively lives their mission and vision daily.

Firsthand lists NASA as #1 on the 50 most prestigious internships for 2022.

Rural location and no flexible work options add to the complexity of recruiting

The position is in a rural area of Eastern Shore Virginia with a population of approximately 60,000 people, which translates to a small pool of candidates. Any hire from outside the area will be expected to relocate as the senior technician must be onsite to support launches twice a month. Financial relocation support is not offered and the position is a two-year assignment with the option of extension that could eventually become permanent.

“Looking for a Talent Unicorn; a qualified, experienced candidate with both leadership skills and deep technical expertise that can grow and learn with emerging technologies.”

Steve Thornton
Acting Division Chief, Solutions Division,
Goddard Space Flight Center, NASA

Case Study: Using culture to drive your talent pool

A good brand overcomes challenges

Culture takes the lead in NASA's job postings, which attract a high number of candidates. Postings begin with a link to a short video on working at NASA, its history, and how it lives its vision. The video highlights NASA's diversity of perspectives, career development, and learning opportunities.

NASA's company brand and employer brand are tightly intertwined, providing a consistent view of the organization.

The employer vision is presented in the best place to reach NASA's ideal candidate: usajobs.gov, the official website of the United States Government and the “go-to” for government job listings. NASA also extends its postings to other generic job sites as well as LinkedIn and professional associations.

Interview with Robert Leahy
Chief Information Officer
Goddard Space Flight Center, NASA

“Making sure we have the tools and mechanisms are two hiring challenges we are going to face in the future as how we work evolves and our work environment changes. What will we need to consider with our job announcements and the criteria for selecting employees?”

Liteshia Dennis,
Office Chief, Headquarter IT Office, Goddard Space Flight Center, NASA

The ability to attract and secure candidates requires a strategy

Despite prioritizing recruiting, IT departments see candidate experience as THE last Priority, either not focusing on it or relegating it to HR

Candidate experience is listed as one of the bottom IT challenges, but without a positive experience, securing the talent you want will be difficult.

Candidate experience starts with articulating your unique culture, benefits, and opportunities for development and innovative work as well as outlining flexible working options within an employer brand. Defining an employee value proposition is key to marketing your roles to potential employees.

81% of respondents' rate culture as very important when considering a potential employer. (Info-Tech IT Talent Trends 2022 Survey)

Tactics Used in Job Postings to Position the Organization Favorably as a Potential Employer

(Info-Tech IT Talent Trends 2022 Survey)

Case Study: Increasing talent pool at Info-Tech Research Group

Strong sales leads to growth in operation capacity

Info-Tech Research Group is an IT research & advisory firm helping IT leaders make strategic, timely, and well-informed decisions. Our actionable tools and analyst guidance ensure IT organizations achieve measurable results.

The business has grown rapidly over the last couple of years, creating a need to recruit additional talent who were highly skilled in technical applications and approaches.

In response, approval was given to expand headcount within Research for fiscal year 2022 and to establish a plan for continual expansion as revenue continues to grow.

Looking for deep technical expertise with a passion for helping our members

Hiring for our research department requires talent who are typically subject matter experts within their own respective IT domains and interested in and capable of developing research and advising clients through calls and workshops.

This combination of skills, experience, and interest can be challenging to find, especially in an IT labor market that is more competitive than ever.

Interview with Practice Lead Tracy-Lynn Reid

Focus on Candidate Experience increases successful hire rate

The senior leadership team established a project to focus on recruiting for net-new and open roles. A dedicated resource was assigned and used guidance from our research to enhance our hiring process to reduce time to hire and expand our candidate pool. Senior leaders stayed actively involved to provide feedback.

The hiring process was improved by including panel interviews with interview protocols and a rubric to evaluate all candidates equitably.

The initial screening conversation now includes a discussion on benefits, including remote and flexible work offerings, learning and development budget, support for post-secondary education, and our Buy-a-Book program.

As a result, about 70% of the approved net-new headcount was hired within 12 weeks, with recruitment ongoing.

Formalize Your Digital Business Strategy

Stay relevant in an evolving digital economy

Executive Summary

Info-Tech Insight

Turn your existing digital strategy into a compelling change story that will create a unified vision of how you want to transform your business.

Info-Tech’s Digital Transformation Journey

Your journey: An IT roadmap for your Digital Business Strategy

By now, you understand your current business context and capabilities

Since 2020, market dynamics have forced organizations to reassess their strategies

The unprecedented pace of global disruptions has become both a curse and a silver lining for many CIOs. The ability to maximize the value of digital will be vital to remain relevant in the new digital economy.

Formalize your digital strategy to address industry trends and market dynamics

The goal of this phase is to ensure the scope of the current digital strategy reflects the right opportunities to allocate capital to resources, assets, and capabilities to drive strategic growth and operational efficiency.

There are three key activities outlined in this deck that that can be undertaken by industry members to help evolve their current digital business strategy.

1. Identify New Digitally Enabled Growth Opportunities
   • Host an ideation session to identify new leapfrog ideas
   • Discuss assumptions, value drivers, and risks
   • Translate ideas into opportunities and consolidate
2. Evaluate New Digital Opportunities and Business Capabilities
   • Build an opportunity profile
   • Identify business capabilities for transformation
3. Transform Stakeholder Journeys
   • Understand the impact of opportunities on value-chains
   • Identify stakeholder personas
   • Build a stakeholder journey map
   • Compile your new list of digital opportunities

Info-Tech’s approach

1. Identify New Digital Opportunities
   • Conduct an ideation session
   • Identify leapfrog ideas from trends
   • Evaluate each leapfrog idea to define opportunity
2. Evaluate Opportunities and Business Capabilities
   • Build Opportunity Profile
   • Understand the impact of opportunities on business capabilities
3. Transform Stakeholder Journeys
   • Analyze value chains
   • Map your Stakeholder Journey
   • Breakdown opportunities into initiatives

Overview of Key Activities

Glossary of Terms

LEAPFROG IDEAS

The concept was originally developed in the area of industrial organizations and economic growth. Leapfrogging is the notion that organizations can identify opportunities to skip one or several stages ahead of their competitors.

DIGITAL OPPORTUNITIES

Opening of new possibilities to transform or change your business model and create operational efficiencies and customer experiences through the adoption of digital platforms, solutions, and capabilities.

INITIATIVES

Breakdown of opportunities into actionable initiatives that creates value for organizations through new or changes to business models, operational efficiencies, and customer experiences.

1. LEAPFROG IDEAS:
   • Precision medicine
2. DIGITAL OPPORTUNITY:
   • Machine Learning to sniff out pre-cancer cells
3. INITIATIVES:
   1. Define genomic analytics capabilities and recruit
   2. Data quality and cleansing review
   3. Implement Machine Learning SW

Identify Digitally Enabled Opportunities

Host an ideation session to turn trends into growth opportunities with new leapfrog ideas.

Phase 1

Host an Ideation Session to Identify New Digital Opportunities

Host an Ideation Session

1.1 Executive Stakeholder Engagement

Assemble Executive Stakeholders

Set yourself up for success with these three steps.

1.2 Guiding Principles

Set the Guiding Principles

Guiding principles help define the parameters of your digital strategy. They act as priori decisions that establish the guardrails to limit the scope of opportunities from the perspective of people, assets, capabilities, and budgets that are aligned with the business objectives. Consider these components when brainstorming guiding principles:

Consider these three components when brainstorming

1.2 Guiding Principles

Examples of Guiding Principles

1.3 Trend-Analysis

Leverage strategic foresight to identify growth opportunities

What is Strategic Foresight?

In times of increasing uncertainty, rapid change, market volatility, and complexity, the development of strategies can be difficult. Strategic foresight offers a solution.
Strategic foresight refers to an approach that uses a range of methodologies, such as scanning the horizon for emerging changes and signals, analyzing megatrends, and developing multiple scenarios to identify opportunities (source: OECD, 2022). However, it cannot predict the future and is distinct from:

• Forecasting tools
• Strategic planning
• Scenario planning (only)
• Predictive analyses of the future

Why is Strategic Foresight useful?

• Reduce uncertainties about the future
• Better anticipate changes
• Future-proof to stress test proposed strategies
• Explore innovation to reveal new products, services, and approaches

Explore Info-Tech’s Strategic Foresight Process Tool

“When situations lack analogies to the past, it’s hard to envision the future.”

- J. Peter Scoblic, HBR, 2020

1.3 Trend-Analysis

Leverage industry roundtables and trend reports to understand the art of the possible

1.3 Trend-Analysis

Scan the Horizon

Understand how the environment is evolving in your industry

Scan the horizon to detect early signs of future changes or threats.

Horizon scanning involves scanning, analyzing, and communicating changes in an organization’s environment to prepare for potential threats and opportunities. Much of what we know about the future is based around the interactions and trajectory of macro trends, trends, and drivers. These form the foundations for future intelligence.

Identify signals of change in the present and their potential future impacts.

1.3 Trend-Analysis

Identify macro trends

Macro trends capture a global shift that can change the market and the industry. Here are examples of macro-trends to consider when scanning the horizon for your own organization:

1.3 Trend-Analysis

Determine impact and relevance of trends

Understand which trends create opportunities or risks for your organization.

Key Concepts:

Once an organization has uncovered a set of trends that are of potential importance, a judgment must be made on which of the trends should be prioritized to understand their impact on your market and ultimately, the implications for your business or organization. Consider the following criteria to help you prioritize your trends.

Impact to Industry: The degree of impact the trend will have on your industry and market to create possibilities or risks for your business. Will this trend create opportunities for the business? Or does it pose a risk that we need to mitigate?

Relevance to Organization. The relevance of the trend to your organization. Does the trend align with the mission, vision, and business objectives of your organization?

Activity: 2-4hours

In order to determine which trends will have an impact on your industry and are relevant to your organization, you need to use a gating approach to short-list those that may create opportunities to capitalize on while you need to manage the ones that pose risk.

1.3 Trend-Analysis

Prioritize Trends for Exploration

Info-Tech Insight

While the scorecard may produce a ranking based on weighted metrics, you need to leverage the group discussion to help contextualize and challenge assumptions when validating the priority. The room for debate is important to truly understand whether a trend is a fad or a fact that needs to be addressed.

1.3 Trend-Analysis

Discuss the driver(s) behind the trend

Determining the root cause(s) of a trend is an important precursor to understanding the how, why, and to what extent a trend will impact your industry and market.

Trend analysis can be a valuable approach to reduce uncertainties about the future and an opportunity to understand the underlying drivers (forces) that may be contributing to a shift in pattern. Understanding the drivers is important to help determine implication on your organization and potential opportunities.

1.3 Trend-Analysis

Examples of driver(s)

INDUSTRY

Healthcare Exemplar

Understanding the drivers is not about predicting the future. Don’t get stuck in “analysis paralysis.” The key objective is to determine what opportunities and risks the trend and its underlying driver pose to your business. This will help elicit leapfrog opportunities that can be funneled into actionable initiatives.

Other examples…

1.3 Trend-Analysis

Case Study

Industry

Healthcare

Artificial Intelligence (AI) in Precision Medicine (Genomics)

Precision Medicine has become very popular over the recent years fueled by research but also political and patient demands to focus more on better outcomes vs. profits. A cancer care center in Canada wanted to look at what was driving this popularity but more importantly, what this potentially meant to their current service delivery model and operations and what opportunities and risks they needed to address in the foreseeable future. They determined the following drivers:

• Improve patient outcomes
• Earlier detection of cancer
• Better patient experience
• Ability to compute vast amounts of data to reduce manual effort and errors
• Accelerate from research to clinical trials to delivery

1.3 Trend-Analysis

INDUSTRY

Healthcare Exemplar

Sample Output From Trend Analysis

1.3 Elicit New Opportunities

Leapfrog into the future

Turn trends into growth opportunities.

To thrive in the digital age, organizations must innovate big, leverage internal creativity, and prepare for flexibility.

In this digital era, organizations are often playing catch up to a rapidly evolving technological landscape and following a strict linear approach to innovation. However, this linear catch-up approach does not help companies get ahead of competitors. Instead, organizations must identify avenues to skip one or several stages of technological development to leapfrog ahead of their competitors.

“The best way to predict the future is to invent it.”

– Alan Kay

Leapfrogging takes place when an organization introduces disruptive innovation into the market and sidesteps competitors, who are unable to mobilize to respond to the opportunities.

1.3 Elicit New Opportunities

Funnel trends into leapfrog ideas

Go from trend insights into ideas for opportunities

Brainstorm ways to generate leapfrog ideas from trend insights.

Dealing with trends is one of the most important tasks for innovation. It provides the basis of developing the future orientation of the organization. However, being aware of a trend is one thing, to develop strategies for response is another.

To identify the impact the trend has on the organization, consider the four areas of growth for the organization:

1. New Customers: Leverage the trend to target new customers for existing products or services.
2. New Business Models: Adjust the business model to capture a change in how the organization delivers value.
3. New Markets: Enter or create new markets by applying existing products or services to different problems.
4. New Product or Service Offerings: Introduce new products or services to the existing market.

1.3 Elicit New Opportunities

INDUSTRY: Healthcare

SOURCE: Memorial Sloan Kettering Cancer Center

Case Study

Machine Learning Sensor to Sniff Out Cancer

From trends to leapfrog ideas

Additional Examples in the Appendix

Example of leapfrog ideas that can generate opportunities for consideration

Phase 1: Deliverable

Phase 1 Deliverable

Example of output from phase 1 ideation session

Phase 2

Evaluate Opportunities and Business Capabilities

Build a better understanding of the opportunities and their impact on your business.

Phase 2

Evaluate Opportunities and Business Capabilities

2.1 Build an opportunity profile

Evaluate each opportunity

Discussion Framework:

In your discussion, evaluate each opportunity to assess assumptions, value drivers, and benefits.

Ideas matter, but not all ideas are created equal. Now that you have elicited opportunities, discuss the assumptions, risks, and benefits associated with each new digital opportunity.

Design Thinking

Leverage the guiding principles as the guardrails to limit the scope of your new digital opportunities. You may want to consider taking a design-thinking approach to innovation by discussing the merits of each opportunity based on:

• DesirabilityDesirability: People want it. Does the solution enable the organization to meet the expectations of stakeholders?
• Feasibility

Feasibility: Able to Execute. Do we have the capabilities to deliver e.g., the right skills, partners, technology, and leadership?

• Viability

Viability: Delivers Value. Will this idea meet business goals e.g., cost, revenue, and benefits?

Source: Adapted from IDEO

Source: Adapted from MoSCoW prioritization model

Exemplar: Opportunity Profile

Example:

An example of a template to capture the output of discussion.

Automate the Registration Process Around Admission, Discharge, and Transfer (ADT)

2.2 Business capabilities impact

Understand the impact on your business capabilities

Each opportunity has the potential to impact multiple areas of your business. Prioritize where to start acting on new opportunities based on your business objectives and capabilities.

You will need: Industry Reference Architecture	Activity: 1-2 hours Using your industry reference architecture, highlight the business capabilities that may be impacted by the opportunity. Use a value chain analysis approach to help with this exercise. Referring to your Prioritized Opportunities for Transformation, prioritize areas to transform. Priority should be given to low maturity areas that are highly or urgently relevant to your overall strategic goals.
+
Prioritized Opportunities for Transformation	⇒

2.2 Business capabilities impact

Start with a value chain analysis

This will help identify the impact on your business capabilities.

As we identify and prioritize the opportunities available to us, we need to assess impacts on value chains. Does the opportunity directly impact an existing value chain? Or does it open us to the creation of a new value chain?

The value chain perspective allows an organization to identify how to best minimize or enhance impacts and generate value.

As we move from opportunity to impact, it is important to break down opportunities into the relevant pieces so we can see a holistic picture of the sources of differentiation.

Exemplar: Prioritized Business Capability Map

In this example, intelligent automation for referral and admission would create opportunity to virtualize repeatable tasks.

Phase 3

ETransform Stakeholder Journeys

Understand the impact of opportunities across the value chain and possibilities of new or better stakeholder experiences.

Phase 3

Identify opportunities to transform stakeholder experiences

Map Stakeholder Journey

Conduct a journey mapping exercise to further refine and identify value streams to transform.

Download the Define Your Digital Business Strategy blueprint for Customer Journey Mapping Activities

3.1 Persona identification

Identify a stakeholder persona and journey scenario

From value chain to journey scenario.

Stakeholder personas and scenarios help us build empathy towards our customers. It helps put us into the shoes of a stakeholder and relate to their experience to solve problems or understand how they experience the steps or processes required to accomplish a goal. A user persona is a valuable basis for stakeholder journey mapping.

A stakeholder persona is a fictitious profile to represent a customer or a user segment. Creating this persona helps us understand who your customers really are and why they are using your service or product.

A stakeholder scenario describes the situation the journey map addresses. Scenarios can be real (for existing products and services) or anticipated.

Learn more about applying design thinking methodologies

3.1 Persona identification

Identify a stakeholder persona

Who are you transforming for?

To define a stakeholder scenario, we need to understand who we are mapping for. In each value chain, we identified a stakeholder who gains value from that value chain. We now need to develop a stakeholder persona: a representation of the end user to gain a strong understanding of who they are, what they need, and their pains and gains.

One of the best ways to flesh out your stakeholder persona is to engage with the stakeholders directly or to gather the input of those who may engage with them within the organization.

For example, if we want to define a journey map for a student, we might want to gather the input of students or teaching faculty that have firsthand encounters with different student types and are able to define a common student type.

Info-Tech Insight

Run a survey to understand your end users and develop a stronger picture of who they are and what they are seeking to gain from your organization.

3.1 Persona identification

Identify stakeholder scenarios to map

For your digital strategy, leverage the existing and opportunity value chains identified in phases 1 and 2 for journey mapping.

Identify two existing value chains to be transformed. In section 1, we identified existing value chains to be transformed. For example, your stakeholder persona is a registration clerk who is part of the Health Information Management team responsible for registering and adjudicating patient identity.
Identify one new value chain. In section 2, we identified a new value chain. However, for a new opportunity, the scenario is more complex as it may capture many different areas of a value chain. Subsequently, a journey map for a new opportunity may require mapping all parts of the value chain.

3.1 Persona identification

Example Stakeholder Persona

3.1 Persona identification

Define a journey statement for mapping

Now that we understand who we are mapping for, we need to define a journey statement to capture the stakeholder journey.

Leverage the following format to define the journey statement.

“As a [stakeholder], I need to [prioritized value chain task], so that I can [desired result or overall goal].”

3.2 Stakeholder Journey-Map

Leverage customer journey mapping to capture value chains to be transformed

Conduct a journey mapping exercise to identify opportunities for innovation or automation.

A journey-based approach helps an organization understand how a stakeholder moves through a process and interacts with the organization in the form of touch points, channels, and supporting characters. By identifying pain points in the journey and the activity types, we can identify opportunities for innovation and automation along the journey.

Embrace design-thinking methodologies to elevate the stakeholder journey and build a competitive advantage for your organization.

3.2 Stakeholder Journey-Map

Key Concepts

0. Name: Annie Smith

Age: 35

Occupation: HIM Registration Clerk for Unity Hospital System

0.Stakeholder Persona A fictitious profile of a representative stakeholder group that shares a common yet discrete set of characteristics that embodies how they think, feel, and act. 1. Journey (Value Chain) Describes the end-to-end steps or processes that a customer takes across the value chain that groups a set of activities, interactions, touch-points, and experiences. 2. Persona’s Goals Exemplifies what the persona is thinking and wanting across each specific step of their journey. 3. Nature of Activity (see detailed definition in this section) This section captures two key components: 1) the description of the action or interaction between the personas to achieve their goals, and 2) the classification of the activity to determine the feasibility for automation. The type is based on four main characteristics: 1) routine cognitive, 2) non-routine cognitive , 3) routine manual, and 4) non-routine manual. 4. Type of Touch-Point The channel by which a persona interacts or touches products, services, the organization, or information. 5. Key Moments & Pain Points Captures the emotional experience and value of the persona across each step and interaction. 6. Metrics This section captures the KPIs used to measure the experience, process or activity today. Future KPIs will need to be developed to measure the opportunities. 7. Opportunities refer to both the possible initiatives to address the persona’s pain points, and the ability to enable business goals.

3.2 Stakeholder Journey-Map

Opportunities for Automation: Nature of Activity

Example
We identified opportunities for automation

Categorize the activity type to identify opportunities for automation. While there is no perfect framework for automation, this 4x4 matrix provides a general guide to identifying automation opportunities for consideration.

Info-Tech Insight

Automation is more than a 1:1 relationship between the defined task or job and automation. When considering automation, look for opportunities to: 1) streamline across multiple processes, 2) utilize artificial intelligence to augment or virtualize manual tasks, and 3) create more structured data to allow for improved data quality over the long-term.

3.2 Stakeholder Journey-Map

Example of stakeholder journey output: Healthcare

Stakeholder: HIM Clerks

Journey: Follow-up visit of 80-year-old diabetes patient at diabetic clinic outpatient

Break opportunities into a series of initiatives aligned to business objectives

FOI = Freedom of Information

Info-Tech Insight

Evaluate if an opportunity will require a series of discrete activities to execute and/or if they can be a stand-alone initiative.

Now you are ready to select and prioritize digital initiatives for business case development

Appendix: Additional Examples

From trend to leapfrog ideas

Every idea is a good one, unless you need one that works.

Additional Examples
Examples of leapfrog ideas that can generate opportunities for consideration

From trend to leapfrog ideas

Every idea is a good one, unless you need one that works.

Additional Exemplars in Appendix

Examples of leapfrog ideas that can generate opportunities for consideration

Case Study: International Airport

Persona Journey Map: International/Domestic Departure

Persona: Super Traveler

Name: Annie Smith

Age: 35

Occupation: Engineer, Global Consultant

Journey Activity Name: Inspired to Travel

Related Info-Tech Research

Tech Trends and Priorities Research Center

• Access Info-Tech’s Tech Trends reports and research center to learn about current industry trends, shifts in markets, and disruptions that are impacting your industry and sector. This is a great starting place to gain insights into how the ecosystem is changing your business and the impact of these changes on IT.

Digital Business Strategy

• Leverage Info-Tech’s Digital Business Strategy to identify opportunities to transform the customer experience.

Industry Reference Architecture

• Access Info-Tech’s Industry coverage to accelerate your understanding of your business capabilities and opportunities for automation.

Contact Your Account Manager

Research Contributors and Experts

Joanne Lee Principal, Research Director, CIO Strategy Info-Tech Research Group

Kim Osborne-Rodgriguez Research Director, CIO Strategy Info-Tech Research Group

Joanne is an executive with over 25 years of in digital technology and management consulting across both public and private entities from solution delivery to organizational redesign across Canada and globally. Prior to joining Info-Tech Research Group, Joanne was a management consultant within KPMG’s CIO management consulting services and the Western Canada Digital Health Practice lead. She has held several executive roles in the industry with the most recent position as Chief Program Officer for a large $450M EHR implementation. Her expertise spans cloud strategy, organizational design, data and analytics, governance, process redesign, transformation, and PPM. She is passionate about connecting people, concepts, and capital. Joanne holds a Master’s in Business and Health Policy from the University of Toronto and a Bachelor of Science (Nursing) from the University of British Columbia.

Kim is a professional engineer and Registered Communications Distribution Designer (RCDD) with over a decade of experience in management and engineering consulting spanning healthcare, higher education, and commercial sectors. She has worked on some of the largest hospital construction projects in Canada, from early visioning and IT strategy through to design, specifications, and construction administration. She brings a practical and evidence-based approach to digital transformation, with a track record of supporting successful implementations. Kim holds a Bachelor’s degree in Mechatronics Engineering from University of Waterloo.

Research Contributors and Experts

Jack Hakimian Vice President, Research Info-Tech Research Group

Charl Lombard President, Digital Transformation Consulting Info-Tech Research Group

Jack has more than 25 years of technology and management consulting experience. He has served multi-billion dollar organizations in multiple industries including Financial Services and Telecommunications. Jack also served a number of large public sector institutions. Prior to joining the Info-Tech Research Group, he worked for leading consulting players such as Accenture, Deloitte, EY, and IBM. Jack led digital business strategy engagements as well as corporate strategy and M&A advisory services for clients across North America, Europe, the Middle East, and Africa. He is a seasoned technology consultant who has developed IT strategies and technology roadmaps, led large business transformations, established data governance programs, and managed the deployment of mission-critical CRM and ERP applications. He is a frequent speaker and panelist at technology and innovation conferences and events and holds a Master’s degree in Computer Engineering as well as an MBA from the ESCP-EAP European School of Management.

Charl has more than 20 years of professional services experience, “majoring” in digital transformation and strategic topics. He has led multiple successful Digital Transformation programs across a range of industries like Information technology, hospitality, Advanced Industries, High Tech, Entertainment, Travel and Transport, Insurance & Financial Services, Metals & Mining, Electric Power, Renewable Energy, Telecoms, Manufacturing) across different geographics (i.e., North America, EU, Africa) in both private and public sectors. Prior to joining Info-Tech Research Group, Charl was the Vice President of Global Product Management and Strategy (Saber Hospitality Solution), Associate President, McKinsey Transformation Practice, e-Business Practice for PwC, and tech start-up founder and investor. Charl is a frequent speaker at innovation and digital transformation conferences and holds an MBA from the University of Cape Town Graduate School of Business, and a bachelor’s degree from the University of Pretoria, South Africa.

Research Contributors and Experts

Mike Tweedie Practice Lead, CIO Strategy Info-Tech Research Group

Michael Alemany Vice President, Digital Transformation Consulting Info-Tech Research Group

Mike Tweedie brings over 25 years of experience as a technology executive. He’s led several large transformation projects across core infrastructure, application, and IT services as the head of Technology at ADP Canada. He was also the Head of Engineering and Service Offerings for a large French IT services firm, focused on cloud adoption and complex ERP deployment and management. Mike holds a Bachelor’s degree in Architecture from Ryerson University.

Michael is a leader in Info-Tech’s digital transformation consulting practice. He brings over 10 years of experience working with companies across a range of industries. His work experience includes ~4.5 years at McKinsey & Company where he led large-scale transformations for fortune 500 companies. Prior to joining Info-Tech, he worked for Sabre Corp., an SaaS platform provider for the travel and hospitality sector, leading Product Strategy & Operations. Michael holds an MBA from the Tuck School of Business at Dartmouth and a B.S in Business Strategy from Brigham Young University.

Research Contributors and Experts

Duane Cooney Executive Counselor, Healthcare Info-Tech Research Group	Denis Goulet Senior Workshop Director Info-Tech Research Group
Duane brings over 30 years of experiences a healthcare IT leader with a passion for the transformation of people, processes, and technology. He has led large-scale health technology transformation and operations across the enterprise. Before joining Info-Tech, Duane served as the Deputy CIO, Senior Information Technology Director, and Enterprise Architect for both public not-for-profit and private sectors. He has a Bachelors in Computer Science and is a graduate of EDS Operations. He holds certifications in EHR, LEAN/Agile, ITIL, and PMP.	Denis is an IAF Certified Professional Facilitator who has helped organizations and technology executives develop IT strategies for small to large global enterprises. He firmly believes in a collaborative value-driven approach. Prior to joining Info-Tech Research Group, Denis held several industry positions as CIO, Chief Administrative Office (City Manager), General Manager, and Vice President of Engineering. Denis holds an MBA from Queen’s University and a Diploma in Technology Engineering and Executive Municipal Management.
Jay Cappis Executive Advisor, Real-Estate Info-Tech Research Group	Christine Brick Executive Advisor, Financial Services Info-Tech Research Group
Jay brings over 30 years of experience in management and technology across small and medium enterprises to large global enterprises including Exxon and Xerox. His cross-industry experience includes professional services, commercial real estate, oil and gas, digital start-ups, insurance, and aerospace. Jay has led business process improvements and change management and has expertise in software development lifecycle management and DevOps practices.	Christine brings over 20 years in IT transformation across DevOps, infrastructure, operations, supply chain, IT Strategy, modernization, cost optimization, data management, and operational risk. She brings expertise in business transformation, mergers and acquisitions, vendor selection, and contract management.

Bibliography

Bhatia, AD. “Transforming through disruptions: A conversation with Dan Antonelli. Transformation Insights.” McKinsey & Company. January 31, 2022. Web
Bertoletti, Antonella and Peter Eeles. “Use an IT Maturity Model.” IBM Garage Methodology. Web. accessed May 30, 2022.
Catlin, Tanguy, Jay Scanlan, and Paul Willmott. “Raising your Digital Quotient.” McKinsey Quarterly. June 1, 2015. Article
Custers, Heidi. “Digital Blueprint. Reference Architecture.” Deloitte Digital.Accessed May 15, 2022.
Coundouris, Anthony. “Reviewed: The Top 5 Digital Transformation Frameworks in 2020.” Run-frictionless Blog. Accessed May 15, 2022. Web.
Daub, Matthias and Anna Wiesinger. “Acquiring the Capabilities you need to go digital.” Business Technology Office – McKinsey and Company. March 2015. Web.
De La Boutetiere, Alberto Montagner and Angelika Reich. “Unlocking success in digital transformations.” McKinsey and Company. October 2018. Web.
“Design Thinking Defined.” IDEO.com. November 21, 2022. Web.
Dorner, Karle and David Edelman. “What ‘Digital’ really means.” McKinsey Digital. July 2015. Web
“Everything Changed. Or Did it? Harvey Nash KPMG CIO Survey 2020.” KPMG, 2020
Kane, Gerald C., Doug Palmer, Ahn Nguyen Phillips, David Kiron, Natasha Buckley. “Aligning the organization for its digital future.” Findings from the 2016 Digital Business Global Executive Study and Research Project. MIT Sloan Management Review. July 26, 2016. Web
LaBerge, Laura, et al. “How COVID-19 has pushed companies over the technology tipping point—and transformed business forever.” McKinsey, 5 Oct. 2020. Accessed 14 June 2021
Mindtools Content Team. “Cause and Effect Analysis.” Mindtools.com. November 21, 2022. Web.
“Strategic Foresight.” OECD.org. November 21, 2022, Web
Sall, Sherman, Dan Lichtenfeld. “The Digital ME Method. Turning digital opportunities into customer engagement and business growth.” Sygnific. 2017. Web.
Scoblic, J. Peter. “Learning from the Future. How to make robust strategy in times of deep uncertainty.” Harvard Business Review, August 2020.
Silva, Bernardo and Schoenwaelder, Tom. ‘Why Good Strategies fail. Addressing the three critical strategic tensions.” Deloitte Monitor Group. 2019.

Tell Your Story With Data Visualization

Build trust with your stakeholders.

Analyst Perspective

Build trust with your stakeholders.

Data visualization refers to graphical representations of data which help an audience understand. Without good storytelling, however, these representations can distract an audience with enormous amounts of data or even lead them to incorrect conclusions.

Good storytelling with data visualization involves identifying the business problem, exploring potential drivers, formulating a hypothesis, and creating meaningful narratives and powerful visuals that resonate with all audiences and ultimately lead to clear actionable insights.

Follow Info-Tech's step-by-step approach to address the business bias of tacit experience over data facts, improve analysts' effectiveness and support better decision making.

Ibrahim Abdel-Kader
Research Analyst,
Data, Analytics, and Enterprise Architecture

Nikitha Patel
Research Specialist,
Data, Analytics, and Enterprise Architecture

Ruyi Sun
Research Specialist,
Data, Analytics, and Enterprise Architecture

Our understanding of the problem

This research is designed for

• Business analysts, data analysts, or their equivalent who (in either a centralized or federated operating model) look to solve challenging business problems more effectively and improve communication with audiences by demonstrating significant insights through visual data storytelling.

This research will also assist

• A CIO or business unit (BU) leader looking to improve reporting and analytics, reduce time to information, and embrace decision making.

This research will help you

• Identify the business problem and root causes that you are looking to address for key stakeholders.
• Improve business decision making through effective data storytelling.
• Focus on insight generation rather than report production.
• Apply design thinking principles to support the collection of different perspectives.

This research will help them

• Understand the report quickly and efficiently, regardless of their data literacy level.
• Grasp the current situation of data within the organization.

Executive Summary

Info-Tech Insight
As organizations strive to become more data-driven, good storytelling with data visualization supports growing corporate data literacy and helps analysts provide insights that improve organizational decision-making and value-driving processes, which ultimately boosts business performance.

Glossary

• Data: Facts or figures, especially those stored in a computer, that can be used for calculating, reasoning, or planning. When data is processed, organized, structured, or presented in a given context to make it useful, it is called information. Data leaders are accountable for certain data domains and sets.
• Data storytelling: The ability to create a narrative powered by data and analytics that supports the hypothesis and intent of the story. Narrators of the story should deliver a significant view of the message in a way easily understood by the target audience. Data visualization can be used as a tactic to enhance storytelling.
• Data visualization: The ability to visually represent a complete story to the target audience powered by data & analytics, using data storytelling as an enabling mechanism to convey narratives. Typically, there are two types of visuals used as part of data visualization: explanatory/informative visuals (the entire story or specific aspects delivered to the audience) and exploratory visuals (the collected data used to clarify what questions must be answered).
• Data literacy: The ability to read, work with, analyze, and argue with data. Easy access to data is essential to exercising these skills. All organizational employees involved with data-driven decisions should learn to think critically about the data they use for analytics and how they assess and interpret the results of their work.
• Data quality: A measure of the condition of data based on factors such as accuracy, completeness, consistency, reliability, and being up-to-date. This is about how well-suited a data set is to serve its intended purpose, therefore business users and stakeholders set the standards for what is good enough. The governance function along with IT ensures that data quality measures are applied, and corrective actions taken.
• Analytics/Business intelligence (BI): A technology-driven process for analyzing data and delivering actionable information that helps executives, managers, and workers make informed business decisions. As part of the BI process, organizations collect data from internal IT systems and external sources, prepare it for analysis, run queries against the data, and create data visualizations.
  Note: In some frameworks, analytics and BI refer to different types of analyses (i.e. analytics predict future outcomes, BI describes what is or has been).

Getting value out of effective storytelling with data visualization

Data storytelling is gaining wide recognition as a tool for supporting businesses in driving data insights and making better strategic decisions.

92% of respondents agreed that data storytelling is an effective way of communicating or delivering data and analytics results.

87% of respondents agreed that if insights were presented in a simpler/clearer manner, their organization's leadership team would make more data-driven decisions.

93% of respondents agreed that decisions made based on successful data storytelling could potentially help increase revenue.

Source: Exasol, 2021

Despite organizations recognizing the value of data storytelling, issues remain which cannot be remedied solely with better technology.

61% Top challenges of conveying important insights through dashboards are lack of context (61%), over-communication (54%), and inability to customize contents for intended audiences (46%).

49% of respondents feel their organizations lack storytelling skills, regardless of whether employees are data literate.

Source: Exasol, 2021

Info-Tech Insight
Storytelling is a key component of data literacy. Although enterprises are increasingly investing in data analytics software, only 21% of employees are confident with their data literacy skills. (Accenture, 2020)

Prerequisite Checklist

Before applying Info-Tech's storytelling methodology, you should have addressed the following criteria:

• Select the right data visualization tools.
• Have the necessary training in statistical analysis and data visualization technology.
• Have competent levels of data literacy.
• Good quality data founded on data governance and data architecture best practices.

To get a complete view of the field you want to explore, please refer to the following Info-Tech resources:

Select and Implement a Reporting and Analytics Solution

Build a Data Architecture Roadmap

Establish Data Governance

Build Your Data Quality Program

Foster Data-Driven Culture With Data Literacy

Info-Tech's Storytelling With Data Visualization Framework

Info-Tech Insight
As organizations strive to become more data-driven, good storytelling with data visualization supports growing corporate data literacy and helps analysts provide insights that improve organizational decision-making and value-driving processes, which ultimately boosts business performance.

Research Benefits

Info-Tech offers various levels of support to best suit your needs

Diagnostics and consistent frameworks are used throughout all four options.

Adopt Generative AI in Solution Delivery

Drive solution quality and team productivity with the right generative AI capabilities.

Analyst Perspective

Build the case for Gen AI with the right opportunities.

Generative AI (Gen AI) presents unique opportunities to address many solution delivery challenges. Code generation can increase productivity, synthetic data generation can produce usable test data, and scanning tools can identify issues before they occur. To be successful, teams must be prepared to embrace the changes that Gen AI brings. Stakeholders must also give teams the opportunity to optimize their own processes and gauge the fit of Gen AI.

Start small with the intent to learn. The right pilot initiative helps you learn the new technology and how it benefits your team without the headache of complex setups and lengthy training and onboarding. Look at your existing solution delivery tools to see what Gen AI capabilities are available and prioritize the use cases where Gen AI can be used out of the box.

Andrew Kum-Seun
Research Director,
Application Delivery and Management
Info-Tech Research Group

Executive Summary

Your Challenge

Delivery teams are under continuous pressure to deliver high-value, high-quality solutions with limited capacity in complex business and technical environments. Common challenges experienced by these teams include:

• Attracting and retaining talent
• Maximizing the return on technology
• Confidently shifting to digital
• Addressing competing priorities
• Fostering a collaborative culture
• Creating high-throughput teams

Generative AI (Gen AI) offers a unique opportunity to address many of these challenges.

Common Obstacles

• Your stakeholders' understanding of what is Gen AI, its value and its application, can be driven by hype and misinterpretation. This confusion can lead to unrealistic expectations and set the wrong precedent for the role Gen AI is intended to play.
• Your solution delivery process is not well documented and is often executed inconsistently. An immature practice will not yield the benefits stakeholders expect.
• The Gen AI marketplace is very broad and diverse. Selecting the appropriate tools and partners is confusing and overwhelming.
• There is a skills gap for what is needed to configure, adopt, and operate Gen AI.

Info-Tech's Approach

• Ground your Gen AI expectations. Set realistic and achievable goals centered on driving business value and efficiency across the entire solution delivery process by enabling Gen AI in key tasks and activities. Propose this process as the ideal pilot for Gen AI.
• Select the right Gen AI opportunities. Discuss how proven Gen AI capabilities can be applied to your solution delivery practice and achieve the outcomes and priorities stakeholders expect. Lessons learned sow the foundation for future Gen AI scaling.
• Assess your Gen AI readiness in your solution delivery teams. Clarify the roles, processes, and tools needed for the implementation, use, and maintenance of Gen AI.

Info-Tech Insight

Position Gen AI as a tooling opportunity to enhance the productivity and depth of your solution delivery practice. Current Gen AI tools are unable to address the various technical and human complexities that commonly occur in solution delivery. Assess the fit of Gen AI by augmenting low-risk, out-of-the-box tools in key areas of your solution delivery process and teams.

Insight Summary

Overarching Info-Tech Insight

Position Gen AI is a tooling opportunity to enhance the productivity and depth of your solution delivery practice. However, current Gen AI tools are unable to address the various technical and human complexities that commonly occur in solution delivery. Assess the fit of Gen AI by augmenting low-risk, out-of-the-box tools in key areas of your solution delivery process and teams.

Understand and optimize first, automate with Gen AI later.
Gen AI magnifies solution delivery inefficiencies and constraints. Adopt a user-centric perspective to understand your solution delivery teams' interactions with solution delivery tools and technologies to better replicate how they complete their tasks and overcome challenges.

Enable before buy. Buy before build.
Your solution delivery vendors see AI as a strategic priority in their product and service offering. Look into your existing toolset and see if you already have the capabilities. Otherwise, prioritize using off-the-shelf solutions with pre-trained Gen AI capabilities and templates.

Innovate but don't experiment.
Do not reinvent the wheel and lower your risk of success. Stick to the proven use cases to understand the value and fit of Gen AI tools and how your teams can transform the way they work. Use your lessons learned to discover scaling opportunities.

Blueprint benefits

What is Gen AI?

Generative AI (Gen AI)
A form of ML whereby, in response to prompts, a Gen AI platform can generate new output based on the data it has been trained on. Depending on its foundational model, a Gen AI platform will provide different modalities and use case applications.

Machine Learning (ML)
The AI system is instructed to search for patterns in a data set and then make predictions based on that set. In this way, the system learns to provide accurate content over time. This requires a supervised intervention if the data is inaccurate. Deep learning is self-supervised and does not require intervention.

Artificial Intelligence (AI)
A field of computer science that focuses on building systems to imitate human behavior. Not all AI systems have learning behavior; many systems (such as customer service chatbots) operate on preset rules.

Info-Tech Insight

Many vendors have jumped on Gen AI as the latest marketing buzzword. When vendors claim to offer Gen AI functionality, pin down what exactly is generative about it. The solution must be able to induce new outputs from inputted data via self-supervision – not trained to produce certain outputs based on certain inputs.

Augment your solution delivery teams with Gen AI

Position Gen AI as a tooling opportunity to enhance the productivity and depth of your solution delivery practice. Current Gen AI tools are unable to address the various technical and human complexities that commonly occur in solution delivery; assess the fit of Gen AI by augmenting low-risk, out-of-the-box tools in key areas of your solution delivery process and teams.

Research deliverable

This research is accompanied by a supporting deliverable to help you accomplish your goals.

Gen AI Solution Delivery Readiness Assessment Tool

Assess the readiness of your solution delivery team for Gen AI. This tool will ask several questions relating to your people, process, and technology, and recommend whether the team is ready to adopt Gen AI practices.

Step 1.1

Set the context

Activities

1.1.1 Understand the challenges of your solution delivery teams.

1.1.2 Outline the value you expect to gain from Gen AI.

This step involves the following participants:

• Applications VP
• Applications Director
• Solution Delivery Manager
• Solution Delivery Team

Outcomes of this step

• SWOT Analysis to help articulate the challenges facing your teams.
• A Gen AI Canvas that will articulate the value you expect to gain.

IT struggles to deliver solutions effectively

• Lack of skills and resources
  Forty-six percent of respondents stated that it was very or somewhat difficult to attract, hire, and retain developers (GitLab, 2023; N=5,010).
• Delayed software delivery
  Code development (37%), monitoring/observability (30%), deploying to non-production environments (30%), and testing (28%) were the top areas where software delivery teams or organizations encountered the most delays (GitLab, 2023, N=5,010).
• Low solution quality and satisfaction
  Only 64% of applications were identified as effective by end users. Effective applications are identified as at least highly important and have high feature and usability satisfaction (Application Portfolio Assessment, August 2021 to July 2022; N=315).
• Burnt out teams
  While workplace flexibility comes with many benefits, longer work hours jeopardize wellbeing. Sixty-two percent of organizations reported increased working hours, while 80% reported an increase in flexibility ("2022 HR Trends Report," McLean & Company, 2022; N=394) .

Creating high-throughput teams is an organizational priority.

CXOs ranked "optimize IT service delivery" as the second highest priority. "Achieve IT business" was ranked first.

(CEO-CIO Alignment Diagnostics, August 2021 to July 2022; n=568)

1.1.1 Understand the challenges of your solution delivery teams

1-3 hours

1. Complete a SWOT analysis of your solution delivery team to discover areas where Gen AI can be applied.
2. Record this information in the Gen AI Solution Delivery Readiness Assessment Tool.

Record the results in the Gen AI Solution Delivery Readiness Assessment Tool

Output

• SWOT analysis of current state of solution delivery practice

Participants

• Applications VP
• Applications Director
• Solution Delivery Manager
• Solution Delivery Team

Gen AI can help solve your solution delivery challenges

Why is software delivery an ideal pilot candidate for Gen AI?

• Many software delivery practices are repeatable and standardized.
• Software delivery roles that are using and implementing Gen AI are technically savvy.
• Automation is a staple in many commonly used tools.
• Change will likely not impact business operations.

Improved productivity

Gen AI jumpstarts the most laborious and mundane parts of software delivery. Delivery teams saved 22 hours (avg) per software use case when using AI in 2022, compared to last year when AI was not used ("Generative AI Speeds Up Software Development," PRNewswire, 2023).

Fungible resources

Teams are transferrable across different frameworks, platforms, and products. Gen AI provides the structure and guidance needed to work across a wider range of projects ("Game changer: The startling power generative AI is bringing to software development," KPMG, 2023).

Improved solution quality

Solution delivery artifacts (e.g. code) are automatically scanned to quickly identify bugs and defects based on recent activities and trends and validate against current system performance and capacity.

Business empowerment

AI enhances the application functionalities workers can build with low- and no-code platforms. In fact, "AI high performers are 1.6 times more likely than other organizations to engage non-technical employees in creating AI applications" ("The state of AI in 2022 — and a half decade in review." McKinsey, 2022, N=1,492).

However, various fears, uncertainties, and doubts challenge Gen AI adoption

Black Box

Little transparency is provided on the tool's rationale behind content creation, decision making, and the use and storage of training data, creating risks for legal, security, intellectual property, and other areas.

Role Replacement

Some workers have job security concerns despite Gen AI being bound to their rule-based logic framework, the quality of their training data, and patterns of consistent behavior.

Skills Gaps

Teams need to gain expertise in AI/ML techniques, training data preparation, and continuous tooling improvements to support effective Gen AI adoption across the delivery practice and ensure reliable operations.

Data Inaccuracy

Significant good quality data is needed to build trust in the applicability and reliability of Gen AI recommendations and outputs. Teams must be able to combine Gen AI insights with human judgment to generate the right outcome.

Slow Delivery of AI Solution

Timelines are sensitive to organizational maturity, experience with Gen AI, and investments in good data management practices. 65% of organizations said it took more than three months to deploy an enterprise-ready AIOps solution (OpsRamp, 2022).

Define the value you want Gen AI to deliver

Well-optimized Gen AI instills stakeholder confidence in ongoing business value delivery and ensures stakeholder buy-in, provided proper expectations are set and met. However, business value is not interpreted or prioritized the same across the organization. Come to a common business value definition to drive change in the right direction by balancing the needs of the individual, team, and organization.

Business value cannot always be represented by revenue or reduced expenses. Dissecting value by the benefit type and the value source's orientation allows you to see the many ways in which Gen AI brings value to the organization.

Financial benefits vs. intrinsic needs

• Financial benefits refers to the degree to which the value source can be measured through monetary metrics, such as revenue generation and cost saving.
• Intrinsic needs refers to how a product, service, or business capability enhanced with Gen AI meets functional, user experience, and existential needs.

Inward vs. outward orientation

• Inward refers to value sources that are internally impacted by Gen AI and improve your employees' and teams' effectiveness in performing their responsibilities.
• Outward refers to value sources that come from your interaction with external stakeholders and customers and were improved from using Gen AI.

See our Build a Value Measurement Framework blueprint for more information about business value definition.

Measure success with the right metrics

Establishing and monitoring metrics are powerful ways to drive behavior and strategic changes in your organization. Determine the right measures that demonstrate the value of your Gen AI implementation by aligning them with your Gen AI objectives, business value drivers, and non-functional requirements.

Select metrics with different views

1. Solution delivery practice effectiveness
   The ability of your practice to deliver, support, and operate solutions with Gen AI
   Examples: Solution quality and throughput, delivery and operational costs, number of defects and issues, and system quality
2. Solution quality and value
   The outcome of your solutions delivered with Gen AI tools
   Examples: Time and money saved, utilization of products and services, speed of process execution, number of errors, and compliance with standards
3. Gen AI journey goals and milestones
   Your organization's position in your Gen AI journey
   Examples: Maturity score, scope of Gen AI adoption, comfort and
   confidence with Gen AI capabilities, and complexity of Gen AI use cases

Leverage Info-Tech's Diagnostics

IT Management & Governance

• Improvement to application development quality and throughput effectiveness
• Increased importance of application delivery and maintenance capabilities across the IT organization
• Delegation of delivery accountability across more IT roles

CIO Business Vision

• Improvements to IT satisfaction and value from delivered solutions
• Changes to the value and importance of IT core services enabled with Gen AI
• The state of business and IT relationships
• Capability to deliver and support Gen AI effectively

1.1.2 Outline the value you expect to gain from Gen AI

1-3 hours

1. Complete the following fields to build your Gen AI canvas:
   1. Problem that Gen AI is intending to solve
   2. List of stakeholders
   3. Desired business and IT outcomes
   4. In-scope solution delivery teams, systems, and capabilities.
2. Record this information in the Gen AI Solution Delivery Readiness Assessment Tool.

Output

• Gen AI Canvas

Participants

• Applications VP
• Applications Director
• Solution Delivery Manager
• Solution Delivery Team

Record the results in the Gen AI Solution Delivery Readiness Assessment Tool

1.1.2 Example

Example of an outline of the value you expect to gain from Gen AI

Problem statements

• Manual testing procedures hinder pace and quality of delivery.
• Inaccurate requirement documentation leads to constant redesigning.

Business and IT outcomes

• Improve code quality and performance.
• Expedite solution delivery cycle.
• Improve collaboration between teams and reduce friction.

List of stakeholders

• Testing team
• Application director
• CIO
• Design team
• Project manager
• Business analysts

In-scope solution delivery teams, system, and capabilities

• Web
• Development
• App development
• Testing
• Quality assurance
• Business analysts
• UI/UX design

Align your objectives to the broader AI strategy

Why is an organizational AI strategy important for Gen AI?

• All Gen AI tactics and capabilities are designed, delivered, and managed to support a consistent interpretation of the broader AI vision and goals.
• An organizational strategy gives clear understanding of the sprawl, criticality, and risks of Gen AI solutions and applications to other IT capabilities dependent on AI.
• Gen AI initiatives are planned, prioritized, and coordinated alongside other software delivery practice optimizations and technology modernization initiatives.
• Resources, skills, and capacities are strategically allocated to meet the needs of Gen AI considering other commitments in the software delivery optimization backlog and roadmap.
• Gen AI expectations and practices uphold the persona, values, and principles of the software delivery team.

What is an AI strategy?

An AI strategy details the direction, activities, and tactics to deliver on the promise of your AI portfolio. It often includes:

• AI vision and goals
• Application, automation, and process portfolio involved or impacted by AI
• Values and principles
• Health of your AI portfolio
• Risks and constraints
• Strategic roadmap

Step 1.2

Evaluate opportunities for Gen AI

Activities

1.2.1 Align Gen AI opportunities with teams and capabilities.

This step involves the following participants:

• Applications VP
• Applications Director
• Solution Delivery Manager
• Solution Delivery Team

Outcomes of this step

• Understand the Gen AI opportunities for your solution delivery practice.

Learn how Gen AI is employed in solution delivery

Other common AI opportunities solutions include test case generation, code translation, use case creation, document generation, and automated testing.

Opportunity 1: Synthetic data generation

Create artificial data that mimics the structure of real-life data.

What are the expected benefits?

• Availability of test data: Creation of large volumes of data compatible for testing multiple systems within the organization.
• Improved privacy: Substituting real data with artificial leads to reduced data leaks.
• Quicker data provisioning: Automated generation of workable datasets aligned to company policies.

What are the notable risks and challenges?

• Generalization and misrepresentations: Data models used in synthetic data generation may not be an accurate representation of production data because of potentially conflicting definitions, omission of dependencies, and multiple sources of truth.
• Lack of accurate representation: It is difficult for synthetic data to fully capture real-world data nuances.
• Legal complexities: Data to build and train the Gen AI tool does not comply with data residency and management standards and regulations.

How should teams prepare for synthetic data generation?

• Understand and align requirements with objectives and involve all stakeholders. Refer to Create a Data Management Roadmap.
• Evaluate quality and validate effectiveness. Refer to Build Your Data Quality Program.
• Build your test data management practices. Refer to Provision and Mask Your Test Data With the Right Tool.

It can be used:

• To train machine learning models when there is not enough real data, or the existing data does not meet specific needs.
• To improve quality of test by using data that closely resembles production without the risk of leveraging sensitive and private information.

"We can simply say that the total addressable market of synthetic data and the total addressable market of data will converge,"
Ofir Zuk, CEO, Datagen (Forbes, 2022)

Opportunity 2: Code generation

Learn patterns and automatically generate code.

What are the expected benefits?

• Increased productivity: It allows developers to generate more code quickly.
• Improved code consistency: Code is generated using a standardized model and lessons learnt from successful projects.
• Rapid prototyping: Expedite development of a working prototype to be verified and validated.

What are the notable risks and challenges?

• Limited contextual understanding: AI may lack domain-specific knowledge or understanding of requirements.
• Dependency: Overreliance on AI generated codes can affect developers' creativity.
• Quality concerns: Generated code is untested and its alignment to coding and quality standards is unclear.

How should teams prepare for code generation?

• Improve the development roles and processes. Refer to Improve Application Development Throughput.
• Build quality into the solution delivery process. Refer to Build a Software Quality Assurance Program.
• Build policies and governance around code generation. Refer to Satisfy Digital End Users With Low- and No-Code.

It can be used to:

• Build solutions without the technical expertise of traditional development.
• Discover different solutions to address coding challenges.
• Kickstart new development projects with prebuilt code.

According to a survey conducted by Microsoft's GitHub, a staggering 92% of programmers were reported as using AI tools in their workflow (GitHub, 2023).

Opportunity 3: Defect forecasting & debugging

Predict and proactively address defects before they occur.

What are the expected benefits?

• Reduced maintenance cost: Find defects earlier in the delivery process, when it's cheaper to fix them.
• Increased efficiency: Testing efforts can remain focused on critical and complex areas of solution.
• Reduced risk: Find critical defects before the product is deployed to production.

What are the notable risks and challenges?

• False positives and negatives: Incorrect interpretation and scope of defect due to inadequate training of the Gen AI model.
• Inadequate training: Training data does not reflect the complexity of the solutions code.
• Not incorporating feedback: Gen AI models are not retrained in concert with solution changes.

How should teams prepare for defect forecasting and debugging?

• Augment Gen AI with software delivery team. Refer to Improve Application Development Throughput and Embed Security into the DevOps Pipeline.
• Gain understanding of past solution performances. Refer to IT Service Management Selection Guide.
• Learn or discover tool sets. Refer to Automate Testing to Get More Doneand Software Quality Assurance.

It can be used to:

• Perform static and dynamic code analysis to find vulnerabilities in the solution source code.
• Forecast potential issues of a solution based on previous projects and industry trends.
• Find root cause and suggest solutions to address found defects.

Using AI technologies, developers can reduce the time taken to debug and test code by up to 70%, allowing them to finish projects faster and with greater accuracy (Aloa, 2023).

Opportunity 4: Requirements documentation & elicitation

Capturing, documenting, and analyzing function and nonfunctional requirements.

What are the expected benefits?

• Improve quality of requirements: Obtain different perspectives and contexts for the problem at hand and help identify ambiguities and misinterpretation of risks and stakeholder expectation.
• Increased savings: Fewer resources are consumed in requirements elicitation activities.
• Increased delivery confidence: Provide sufficient information for the solution delivery team to confidently estimate and commit to the delivery of the requirement.

What are the notable risks and challenges?

• Conflicting bias: Gen AI models may interpret the problem differently than how the stakeholders perceive it.
• Organization-specific interpretation: Inability of the Gen AI models to accommodate unique interpretation of terminologies, standards, trends and scenarios.
• Validation and review: Interpreting extracted insights requires human validation.

How should teams prepare for requirements documentation & elicitation?

• Feed high quality, customizable data for domain-specific Use. Refer to Build Your Data Quality Program.
• Continuously train the model. Refer to Improve Requirements Gathering.
• Refine organization policies, processes, and roles. Refer to Requirements Gathering SOP and BA Playbook.

It can be used to:

• Document requirements in a clear and concise manner that is usable to the solution delivery team.
• Analyze and test requirements against various user, business, and technical scenarios.

91% of top businesses surveyed report having an ongoing investment in AI (NewVantage Partners, 2021).

Opportunity 5: UI design and prototyping

Analyze existing patterns and principles to generate design, layouts, and working solutions.

What are the expected benefits?

• Increased experimentation: Explore different approaches and tactics to solve a solution delivery problem.
• Improved collaboration: Provide quick design layouts that can be reshaped based on stakeholder feedback.
• Ensure design consistency: Enforce a UI/UX design standard for all solutions.

What are the notable risks and challenges?

• Misinterpretation of UX Requirements: Gen AI model incorrectly assumes a specific interpretation of user needs, behaviors, and problem.
• Incorrect or missing requirements: Lead to extensive redesigns and iterations, adding to costs while hampering user experience.
• Design creativity: May lack originality and specific brand aesthetics if not augmented well with human customizability and creativity.

How should teams prepare for UI design and prototyping?

• Understand the solution's end users. Refer to Apply Design Thinking to Build Empathy With the Business.
• Enhance your practice. Refer to Satisfy Digital End Users With Low- and No-Code.
• Implement an iterative delivery process. Refer to Agile & DevOps Resource Center.

It can be used to:

• Visualize the solution through different views and perspectives such as process flows and use-case diagrams.
• Create working prototypes that can be verified and validated by stakeholders and end users.

A study by McKinsey & Company found that companies that invest in AI-driven design outperform their peers in revenue growth and customer experience metrics. They were found to achieve up to two times higher revenue growth than industry peers and up to 10% higher net promoter score (McKinsey & Company, 2018).

Determine the importance of your opportunities by answering these questions

Realizing the complete potential of Gen AI relies on effectively fostering its adoption and resulting changes throughout the entire solution delivery process.

What are the challenges faced by your delivery teams that could be addressed by Gen AI?

• Recognize the precise pain points, bottlenecks, or inefficiencies faced by delivery teams.
• Include all stakeholders' perspectives during problem discovery and root cause analysis.

What's holding back Gen AI adoption in the organization?

• Apart from technical barriers, address cultural and organizational challenges and discuss how organizational change management strategies can mitigate Gen AI adoption risk.

Are your objectives aligned with Gen AI capabilities?

• Identify areas where processes can be modernized and streamlined with automation.
• Evaluate the current capabilities and resources available within the organization to leverage Gen AI technologies effectively.

How can Gen AI improve the entire solution delivery process?

• Investigate and evaluate the improvements Gen AI can reasonably deliver, such as increased accuracy, quickened delivery cycles, improved code quality, or enhanced cross-functional collaboration.

1.2.1 Align Gen AI opportunities to teams and capabilities

1-3 hours

1. Associate the Gen AI opportunities that can be linked to your system capabilities. These opportunities refer to the potential applications of generative AI techniques, such as code generation or synthetic data, to address specific challenges.
   1. Start by analyzing your system's requirements, constraints, and areas where Gen AI techniques can bring value. Identify the potential benefits of integrating Gen AI, such as increased productivity, or enhanced creativity.
   2. Next, discern potential risks or challenges, such as dependency or quality concerns, associated with the opportunity implementation.
2. Record this information in the Gen AI Solution Delivery Readiness Assessment Tool.

Output

• Gen AI opportunity selection

Participants

• Applications VP
• Applications Director
• Solution Delivery Manager
• Solution Delivery Team

Record the results in the Gen AI Solution Delivery Readiness Assessment Tool

Keep an eye out for red flags

Not all Gen AI opportunities are delivered and adopted the same. Some present a bigger risk than others.

• Establishing vague targets and success criteria
• Defining Gen AI as substitution of human capital
• Open-source software not widely adopted or validated
• High level of dependency on automation
• Unadaptable cross-functional training across organization
• Overlooking privacy, security, legal, and ethical implications
• Lack of Gen AI expertise and understanding of good practices

Step 1.3

Assess your readiness for Gen AI

Activities

1.3.1 Assess your readiness for Gen AI.

This step involves the following participants:

• Applications VP
• Applications Director
• Solution Delivery Manager
• Solution Delivery Team

Outcomes of this step

• A completed Gen AI Readiness Assessment to confirm how prepared you are to embrace Gen AI in your solution delivery team.

Prepare your SDLC* to leverage Gen AI

As organizations evolve and adopt more tools and technology, their solution delivery processes become more complex. Process improvement is needed to simplify complex and undocumented software delivery activities and artifacts and prepare it for Gen AI. Gen AI scales process throughput and output quantity, but it multiplies the negative impact of problems the process already has.

When is your process ready for Gen AI?

• Solution value – Ensures the accuracy and alignment of the committed feature and change requests to what the stakeholder truly expects and receives.
• Throughput – Delivers new products, enhancements, and changes at a pace and frequency satisfactory to stakeholder expectations and meets delivery commitments.
• Process governance – Has clear ownership and appropriate standardization. The roles, activities, tasks, and technologies are documented and defined. At each stage of the process someone is responsible and accountable.
• Process management – Follows a set of development frameworks, good practices, and standards to ensure the solution and relevant artifacts are built, tested, and delivered consistently and repeatably.
• Technical quality assurance – Accommodates committed non-functional requirements within the stage's outputs to ensure products meet technical excellence expectations.

*software development lifecycle

To learn more, visit Info-Tech's Modernize Your SDLC blueprint.

To learn more, visit Info-Tech's Build a Winning Business Process Automation Playbook

Assess the impacts from Gen AI changes

Ensure that no stone is left unturned as you evaluate the fit of Gen AI and prepare your adoption and support plans.

By shining a light on considerations that might have otherwise escaped planners and decision makers, an impact analysis is an essential component to Gen AI success. This analysis should answer the following questions on the impact to your solution delivery teams.

1. Will the change impact how our clients/customers receive, consume, or engage with our products/services?
2. Will there be an increase in operational costs, and a change to compensation and/or rewards?
3. Will this change increase the workload and alter staffing levels?
4. Will the vision or mission of the team change?
5. Will a new or different set of skills be needed?
6. Will the change span multiple locations/time zones?
7. Are multiple products/services impacted by this change?
8. Will the workflow and approvals be changed, and will there be a substantial change to scheduling and logistics?
9. Will the tools of the team be substantially different?
10. Will there be a change in reporting relationships?

See our Master Organizational Change Management Practices blueprint for more information.

Brace for impact

A thorough analysis of change impacts will help your software delivery teams and change leaders:

• Bypass avoidable problems.
• Remove non-fixed barriers to success.
• Acknowledge and minimize the impact of unavoidable barriers.
• Identify and leverage potential benefits.
• Measure the success of the change.

Many key IT capabilities are required to successfully leverage Gen AI

Portfolio Management

An accurate and rationalized inventory of all Gen AI tools verifies they support the goals and abide to the usage policies of the broader delivery practice. This becomes critical when tooling is updated frequently and licenses and open- source community principles drastically change (e.g. after an acquisition).

Quality Assurance

Gen AI tools are routinely verified and validated to ensure outcomes are accurate, complete, and aligned to solution delivery quality standards. Models are retrained using lessons learned, new use cases, and updated training data.

Security & Access Management

Externally developed and trained Gen AI models may not include the measures, controls, and tactics you need to prevent vulnerabilities and protect against threats that are critical in your security frameworks, policies, and standards.

Data Management & Governance

All solution delivery data and artifacts can be transformed and consumed in various ways as they transit through solution delivery and Gen AI tools. Data integrations, structures, and definitions must be well-defined, governed, and monitored.

OPERATIONAL SUPPORT

Resources are available to support the ongoing operations of the Gen AI tool, including infrastructure, preparing training data, and managing integration with other tools. They are also prepared to recover backups, roll back, and execute recovery plans at a moment's notice.

Apply Gen AI good practices in your solution delivery practice

1. Keep the human in the loop.
   Gen AI models cannot produce high-quality content with 100% confidence. Keeping the human in the loop allows people to directly give feedback to the model to improve output quality.
2. Strengthen prompt and query engineering.
   The value of the outcome is dependent on what is being asked. Good prompts and queries focus on creating the optimal input by selecting and phrasing the appropriate words, sentence structures, and punctuation to illustrate the focus, scope, problem, and boundaries.
3. Thoughtfully prepare your training data.
   Externally hosted Gen AI tools may store your training data in their systems or use it to train their other models. Intellectual property and sensitive data can leak into third-party systems and AI models if it is not properly masked and sanitized.
4. Build guardrails into your Gen AI models.
   Guardrails can limit the variability of any misleading Gen AI responses by defining the scope and bounds of the response, enforcing the policies of its use, and clarifying the context of its response.
5. Monitor your operational costs.
   The cost breakdown will vary among the types of Gen AI solution and the vendor offerings. Cost per query, consultant fees, infrastructure hosting, and licensing costs are just a few cost factors. Open source can be an attractive cost-saving option, but you must be willing to invest in the roles to assume traditional vendor accountabilities.
6. Check the licenses of your Gen AI tool.
   Each platform has licenses and agreements on how their solution can or cannot be used. They limit your ability to use the tool for commercial purposes or reproductions or may require you to purchase and maintain a specific license to use their solution and materials.

See Build Your Generative AI Roadmap for more information.

Assess your Gen AI readiness

• Solution delivery team
  The team is educated on Gen AI, its use cases, and the tools that enable it. They have the skills and capacity to implement, create, and manage Gen AI.
• Solution delivery process and tools
  The solution delivery process is documented, repeatable, and optimized to use Gen AI effectively. Delivery tools are configured to enable, leverage and manage Gen AI assets to improve their performance and efficiency.
• Solution delivery artifacts
  Delivery artifacts (e.g. code, scripts, documents) that will be used to train and be leveraged by Gen AI tools are discoverable, accurate, complete, standardized, of sufficient quantity, optimized for Gen AI use, and stored in an accessible shared central repository.
• Governance
  Defined policies, role definitions, guidelines, and processes that guide the implementation, development, operations, and management of Gen AI.
• Vision and executive support
  Clear alignment of Gen AI direction, ambition, and objectives with broader business and IT priorities. Stakeholders support the Gen AI initiative and allocate human and financial resources for its implementation within the solution delivery team.
• Operational support
  The capabilities to manage the Gen AI tools and ensure they support the growing needs of the solution delivery practice, such as security management, hosting infrastructure, risk and change management, and data and application integration.

1.3.1 Assess your readiness for Gen AI

1-3 hours

1. Review the current state of your solution delivery teams including their capacity, skills and knowledge, delivery practices, and tools and technologies.
2. Determine the readiness of your team to adopt Gen AI.
3. Discuss the gaps that need to be filled to be successful with Gen AI.
4. Record this information in the Gen AI Solution Delivery Readiness Assessment Tool.

Record the results in the Gen AI Solution Delivery Readiness Assessment Tool

Output

• Gen AI Solution Delivery Readiness Assessment

Participants

• Applications VP
• Applications Director
• Solution Delivery Manager
• Solution Delivery Team

Recognize that Gen AI does not require a fully optimized solution delivery process

To learn more, visit Info-Tech's Develop Your Value-First Business Process Automation (BPA) Strategy.

Be prepared to take the next steps

Deliver Gen AI to your solution delivery teams

Modernize Your SDLC
Efficient and effective SDLC practices are vital, as products need to readily adjust to evolving and changing business needs and technologies.

Adopt Generative AI in Solution Delivery
Generative AI can drive productivity and solution quality gains to your solution delivery teams. Level set expectations with the right use case to demonstrate its value potential.

Select Your AI Vendor & Implementation Partner
The right vendor and partner are critical for success. Build the selection criteria to shortlist the products and services that best meets the current and future needs of your teams.

Drive Business Value With Off-the-Shelf AI
Build a framework that will guide your teams through the selection of an off-the-shelf AI tool with a clear definition of the business case and preparations for successful adoption.

Build Your Enterprise Application Implementation Playbook
Your Gen AI implementation doesn't start with technology, but with an effective plan that your team supports and is aligned to broader stakeholder and sponsor priorities and goals.

Build your Gen AI practice

• Get Started With AI
• AI Strategy & Generative AI Roadmap
• AI Governance

Related Info-Tech Research

Build a Winning Business Process Automation Playbook
Optimize and automate your business processes with a user-centric approach.

Embrace Business Managed Applications
Empower the business to implement their own applications with a trusted business-IT relationship.

Application Portfolio Management Foundations
Ensure your application portfolio delivers the best possible return on investment.

Maximize the Benefits from Enterprise Applications with a Center of Excellence
Optimize your organization's enterprise application capabilities with a refined and scalable methodology.

Create an Architecture for AI
Build your target state architecture from predefined best-practice building blocks.

Deliver on Your Digital Product Vision
Build a product vision your organization can take from strategy through execution.

Enhance Your Solution Architecture Practices
Ensure your software systems solution is architected to reflect stakeholders' short- and long-term needs.

Apply Design Thinking to Build Empathy With the Business
Use design thinking and journey mapping to make IT the business' go-to problem solver.

Modernize Your SDLC
Deliver quality software faster with new tools and practices.

Drive Business Value With Off-the-Shelf AI
A practical guide to ensure return on your off-the-shelf AI investment.

Bibliography

"Altran Helps Developers Write Better Code Faster with Azure AI." Microsoft, 2020.
"Apply Design Thinking to Complex Teams, Problems, and Organizations." IBM, 2021.
Bianca. "Unleashing the Power of AI in Code Generation: 10 Applications You Need to Know — AITechTrend." AITechTrend, 16 May 2023.
Biggs, John. "Deep Code Cleans Your Code with the Power of AI." TechCrunch, 26 Apr 2018.
"Chat GPT as a Tool for Business Analysis — the Brazilian BA." The Brazilian BA, 24 Jan 2023.
Davenport, Thomas, and Randy Bean. "Big Data and AI Executive Survey 2019." New Vantage Partners, 2019.
Davenport, Thomas, and Randy Bean. "Big Data and AI Executive Survey 2021." New Vantage Partners, 2021.
Das, Tamal. "9 Best AI-Powered Code Completion for Productive Development." Geek flare, 5 Apr 2023.
Gondrezick, Ilya. "Council Post: How AI Can Transform the Software Engineering Process." Forbes, 24 Apr 2020.
"Generative AI Speeds up Software Development: Compass UOL Study." PR Newswire, 29 Mar 2023.
"GitLab 2023 Global Develops Report Series." Gitlab, 2023.
"Game Changer: The Startling Power Generative AI Is Bringing to Software Development." KPMG, 30 Jan 2023.
"How AI Can Help with Requirements Analysis Tools." TechTarget, 28 July 2020.
Indra lingam, Ashanta. "How Spotify Is Upleveling Their Entire Design Team." Framer, 2019.
Ingle, Prathamesh. "Top Artificial Intelligence (AI) Tools That Can Generate Code to Help Programmers." Matchcoat, 1 Jan 2023.
Kaur, Jagreet . "AI in Requirements Management | Benefits and Its Processes." Xenon Stack, 13 June 2023.
Lange, Danny. "Game On: How Unity Is Extending the Power of Synthetic Data beyond the Gaming Industry." CIO, 17 Dec 2020.
Lin, Ying. "10 Artificial Intelligence Statistics You Need to Know in 2020." OBERLO, 17 Mar. 2023.
Mauran, Cecily. "Whoops, Samsung Workers Accidentally Leaked Trade Secrets via ChatGPT." Mashable, 6 Apr 2023.

INFO-TECH RESEARCH GROUP

Analyze Your Service Desk Ticket Data

Take a data-driven approach to service desk optimization.

EXECUTIVE BRIEF

Analyst Perspective

Benedict Chang Research Analyst, Infrastructure & Operations Info-Tech Research Group

Ken Weston ITIL MP, PMP, Cert.APM, SMC Research Director, Infrastructure & Operations Info-Tech Research Group

The perfect time to start analyzing your ticket data is now

Service desks improve their services by leveraging ticket data to inform their actions. However, many organizations don’t know where to start. It’s tempting to wait for perfect data, but there’s a lot of value in analyzing your ticket data as it exists today.

Start small. Track key tension metrics based on the out-of-the-box functionality in your tool. Review the metrics regularly to stay on track.

By reviewing your ticket data, you’re going to get better organically. You’re going to learn about the state of your environment, the health of your processes, and the quality of your services. Regularly analyze your data to drive improvements.

Make ticket analysis a weekly habit. Every week, you should be evaluating how the past week went. Every month, you should be looking for patterns and trends.

Executive Summary

Your Situation

Leverage your service desk ticket data to gain insights for improving your operations:

1. Use a data-based approach to allocate service desk resources.
2. Design appropriate SLOs and SLAs to better service end users.
3. Gain efficiencies for your shift-left strategy.
4. Communicate the current and future value of the service desk to the business.

Common Obstacles

Properly analyzing ticket data is challenging for the following reasons:

• Poor ticket hygiene and unclear ticket handling guidelines can lead to untrustworthy results.
• Undocumented tickets from various intake channels prevents you from seeing the whole picture.
• Service desk personnel are not sure where to start with analysis and are too busy to find time.
• Too many metrics are tracked to parse actionable insights from the noise.

Info-Tech’s Approach

Info-Tech’s approach to improvement:

• To reduce the noise, standardize your ticket data in a format that will ease analysis.
• Start with common analyses using the cleaned data set.
• Identify action items based on your ticket data.

Analyze your ticket data to help continually improve your service desk.

Slow down. Give yourself time.

Give yourself time to observe the new metrics and draw enough insights to make recommendations for improvement. Then, execute on those recommendations. Slow and steady improvement of the service desk only adds business value and will have a positive impact on customer satisfaction.

Your challenge

This research is designed to help service desk managers analyze their ticket data

Analyzing ticket data involves:

• Collecting ticket data and keeping it clean. Based on the metrics you’re analyzing, define ticket expectations and keep the data up to date.
• Showing the value of the service desk. SLAs are meaningless if they are not met consistently. The prerequisite to implementing proper SLAs is fully understanding the workload of the service desk.
• Understanding – and improving – the user experience. You cannot improve the user experience without meaningful metrics that allow you to understand the user experience. Different user groups will have different needs and different expectations of the level of service. Your metrics should reflect those needs and expectations.

36% of organizations are prioritizing ticket handling in IT for 2021 (Source: SDI, 2021)

12% of organizations are focusing directly on service desk improvement (Source: SDI, 2021)

Common obstacles

Many organizations face these barriers to analyzing their ticket data:

• Finding time to properly analyze ticket data is a challenge. Not knowing where to start can lead to not analyzing the proper data. Service desks end up either tracking too much data or not tracking the proper metrics.
• Data, even if clean, can be housed in various tools and databases. It’s difficult to aggregate data if the data is stored throughout various tools. Comparisons may also be difficult if the data sets aren’t consistent.
• Shifting left to move tickets toward self-service is difficult when there is no visibility into which tickets should be shifted left.

What your peers are saying about why they can’t start analyzing their ticket data:

• “My technicians do not consistently update and close tickets.”
• “My ITSM doesn’t have the capabilities I need to make informed decisions on shifting tickets left.”
• “My tickets are always missing data”
• “I’m constantly firefighting. I have no time for ticket data analysis.”
• “I have no idea where to start with the amount of data I have.”

Common obstacles that prevent effective ticket analysis

We asked IT service desk managers and teams about their biggest hurdles

Info-Tech’s approach

Repeat this analysis every business cycle:

• Gather Your Data
  Collect your ticket data OR start measuring the right metrics.
• Extract & Analyze
  Organize and visualize your data to extract insights
• Action the Results
  Implement low-effort improvements and celebrate quick successes.
• Implement Larger Changes
  Reference your ticket data while implementing process, tooling, and other changes.
• Communicate the Results
  Use your data to show the value of your effort.

Measure the value of this blueprint

Track these metrics as you improve

Use the data to tell you which aspects of IT need to be shifted left and which need to be automated

Your data will show you where you can improve.

As you act on your data, you should see:

• Lower costs per ticket
• Decreased average time to resolve
• Increased end-user satisfaction
• Fewer tickets escalated beyond Tier 1

See Info-Tech’s blueprint Optimize the Service Desk With a Shift-Left Strategy.

Info-Tech’s methodology for analyzing service desk tickets

Insight summary

Slow down. Give yourself time.

Give yourself time to observe the new metrics and draw enough insights to make recommendations for improvement. Then, execute on those recommendations. Slow and steady improvement of the service desk only adds business value and will have a positive impact on customer satisfaction.

Iterate on what to track rather than trying to get it right the first time.

Tracking the right data in your ticket can be challenging if you don’t know what you’re looking for. Start with standardized fields and iterate on your data analysis to figure out your gaps and needs.

If you don’t know where to go, ticket data can point you in the right direction.

If you have service desk challenges, you will need to allocate time to process improvement. However, prioritizing your initiatives is easier if you have the ticket data to point you in the right direction.

Start with data from one business cycle.

Service desks don’t need three years’ worth of data. Focus on gathering data for one business cycle (e.g. three months). That will give you enough information to start generating value.

Let the data do the talking.

Leverage the data to drive organizational and process change in your organization by tracking meaningful metrics. Choose those metrics using business-aligned goals.

Paint the whole picture.

Single metrics in isolation, even if measured over time, may not tell the whole story. Make sure you design tension metrics where necessary to get a holistic view of your service desk.

Blueprint deliverables

Blueprint benefits

IT Benefits

• Discover and implement the proper metrics to improve your service desk
• Use a data-based approach to improve your customer service and operational goals
• Increase visibility with the business and other IT departments using a structured presentation

Business Benefits

• Quicker resolutions to incidents and service requests
• Better expectations for the service desk and IT
• Better visibility into the current state, challenges, and goals of the service desk
• More effective support when contacting the service desk

Info-Tech offers various levels of support to best suit your needs

Diagnostics and consistent frameworks used throughout all four options

Guided Implementation

A Guided Implementation (GI) is a series of calls with an Info-Tech analyst to help implement our best practices in your organization.

A typical GI is 3-4 calls over the course of 2-3 months.

What does a typical GI on this topic look like?

Phase 1

• Call #1: Scope requirements, objectives, and your specific challenges. Enter your data into the tool.

Phase 2

• Call #2: Assess the current state across the different dashboards.

Phase 3

• Call #3: Identify improvements and insights to include in the communication report.
• Call #4: Review the service desk ticket analysis report.

PHASE 1

Import Your Ticket Data

This phase will walk you through the following activities:

• 1.1.1 Define your objectives for analyzing ticket data
• 1.1.2 Identify success metrics
• 1.1.3 Import your ticket data into the tool
• 1.1.4 Update your ticket fields for future analysis

This phase involves the following participants:

• Service Desk Manager
• ITSM Manager
• Service Desk Technician

1.1.1 Define your objectives for analyzing ticket data

Input: Understanding of current service desk process and ticket routing

Output: Defined objectives for the project

Materials: Whiteboard/flip charts, Ticket Analysis Report

Participants: Service Desk Staff, Service Desk Manager, IT Director, CIO

Use the discussion questions below as a guide

1. Identify your main objective for analyzing ticket data. Use these three sample objectives as a starting point:
   • Demonstrate value to the business by improving customer service.
   • Improve service desk operations.
   • Reduce the number of recurring incidents.
2. Answer the following questions as a group:
   • What challenges do you have getting accurate data for this objective?
   • What data is missing for supporting this objective?
   • What kind of issues must be solved for us to make progress on achieving this objective?
   • What decisions are held up from a lack of data?
   • How can better ticket data help us to more effectively manage our services and operations?

Document in the Ticket Analysis Report.

1.1.2 Identify success metrics

Select metrics that will track your progress on meeting the objective identified in Activity 1.1.1.

Input: Understanding of current service desk process and ticket routing

Output: Defined objectives for the project

Materials: Whiteboard/flip charts, Ticket Analysis Report

Participants: Service Desk Manager, IT Director, CIO

Use these sample metrics as a starting point:

Document in the Ticket Analysis Report.

Inefficient ticket-handling processes lead to SLA breaches and unplanned downtime

Analyze the ticket data to catch mismanaged or lost tickets that lead to unnecessary escalations and impact business profitability

• Ticket Category – Are your tickets categorized by type of asset? By service?
• Average Ticket Times – How long does it take to resolve or fulfill tickets?
• Ticket Priority – What is the impact and urgency of the ticket?
• SLA/OLA Violations – Did we meet our SLA objectives? If not, why?
• Ticket Channel – How was the issue reported or ticket received?
• Response and Fulfillment – Did we complete first contact resolution? How many times was it transferred?
• Associated Tasks and Tickets – Is this incident associated with any other tasks like change tickets or problem tickets?

Encourage proper ticket-handling procedures to enable data quality

Ensure everyone understands the expectations and the value created from having ticket data that follows these expectations

• Create and update tickets, but not at the expense of good customer service. Agents can start the ticket but shouldn’t spend five minutes creating the ticket when they should be troubleshooting the problem.
• Update the ticket when the issue is resolved or needs to be escalated. If agents are escalating, they should make sure all relevant information is passed along within the ticket to the next technician.
• Update user of ETA if issue cannot be resolved quickly.
• Ticket templates for common incidents can lead to fast creation, data input, and categorizations. Templates can reduce the time it takes to create tickets from two minutes to 30 seconds.
• Update categories to reflect the actual issue and resolution.
• Reference or link to the knowledgebase article as the documented steps taken to resolve the incident.
• Validate with the client that the incident is resolved; automate this process with ticket closure after a certain time.
• Close or resolve the ticket on time.

Info-Tech Insight

Ticket handling ensures clean handovers, whether it is to higher tiers or back to the customer. When filling the ticket out with information intended for another party, ensure the information is written for their benefit and from their point of view.

Service Desk Ticket Analysis Tool overview

The Service Desk Ticket Analysis Tool will help you standardize your ticket data in a meaningful format that will allow you to apply common analyses to identify the actions you need to take to improve service desk operations

TABS 1 & 2 INSTRUCTIONS & DATA ENTRY	TAB 3 : TICKET SUMMARY TICKET SUMMARY DASHBOARDS	TABS 4 to 8: DASHBOARDS INCIDENT SERVICE REQUEST CATEGORY
Input at least three months of your exported ticket data into the corresponding columns in the tool to feed into the common analysis graphs in the other tabs.	This tab contains multiple dashboards analyzing how tickets come in, who requests them, who resolves them, and how long it takes to resolve them.	These tabs each have dashboards outlining analysis on incidents and service requests. The category tab will allow you to dive deeper on commonly reported issues.

1.1.3 Import your data into our Service Desk Ticket Analysis Tool

You can still leverage your current data, but use this opportunity to improve your service desk ticket fields down the line

Input: ITSM data log

Output: Populated Service Desk Ticket Data Analysis Tool

Materials: Whiteboard/flip charts, Service Desk Ticket Analysis Tool

Participants: Service Desk Manager, Service Desk Technicians

Start here:

• Extract your ticket data from your ITSM tool in an Excel or text format.
• Look at the fields on the data entry tab of the Service Desk Ticket Analysis Tool.
• Fill the fields with your ticket data by copying and pasting relevant sections. It is okay if you don’t have all the fields, but take note of the fields you are missing.
• With the list of the fields you are missing, run through the following activity to decide if you will need to adopt or add fields to your own service desk ticket tool.

When entering your data, pay close attention to the following fields:

• Time to Resolve: This is automatically calculated using data in the Open Date, Open Time, Close Date, and Close Time fields. You have three options for entering your data in these fields:
  1. Enter your data as the fields describe. Ensure your data contain only the field description (e.g. Open Date separated from Open Time). If your data contain Open Date AND Open Time, Excel will not show both.
  2. Enter your data only in Open Date and Close Date. If your ITSM does not separate date and time, you can keep the data in a single cell and enter it in the column. The formula in Time to Resolve will still be accurate.
  3. If your ITSM outputs Time to Resolve, overwrite the formula in the Time to Resolve column.
• SLA: If your ITSM outputs SLA fulfilled: Y/N, enter that directly into the SLA Fulfilled column.
• Blank Columns: If you do not have data for all the columns, that is okay. Continue with the following activity. Note that some stock dashboards will be empty if that is the case.
• Incidents vs. Service Requests: If you separate incidents and service requests, be sure to capture that in the SR/Incident for Tabs 4 and 5. If you do not separate the two, then you will only need to analyze Tab 3.

Use Info-Tech’s tool instead of building your own. Download the Service Desk Ticket Analysis Tool.

1.1.4 Update your ticket fields for future analysis

Input: Populated Service Desk Ticket Data Analysis Tool

Output: New ticket fields to track

Materials: Whiteboard/flip charts, Service Desk Ticket Analysis Tool

Participants: Service Desk Manager, Service Desk Technicians

As a group, pay attention to the ticket fields populated in the tool as well as the ticket fields that you were not able to populate. Use the example “Fields Captured” table to the right, which lists all fields present in the ticket analysis tool.

Discuss the following questions:

1. Consider the fields not captured. Would it be valuable to start capturing that data for future analysis?
2. If so, does your ITSM support that field?
3. Can you make the change in-house or do you have to bring in an external ITSM administrator to make the change?
4. Capture the results in the Ticket Analysis Report.

Document in the Ticket Analysis Report.

Info-Tech Insight

Don’t wait for your ticket quality to be perfect. You can still draw actions from your ticket data. They will likely be process improvements initially, but the exercise of pulling the data is a necessary first step.

Common ticket fields tracked by your peers

Which of these metrics do you track and action?

• Remember you don’t have to track every metric. Only track metrics that are actionable.

For each metric that you end up tracking:

• Look for trends over time.
• Brainstorm reasons why the metric could rise or fall.

Associate a metric with each improvement you execute.

• Performing this step will allow you to better see the value from your team’s efforts.
• It will also give you a quicker response than waiting for spikes in your data.

(Source: Info-Tech survey, 2021; N=20)

PHASE 2

Analyze Your Ticket Data

This phase will walk you through the following activities:

• 2.1.1 Review high-level ticket dashboards
• 2.2.1 Review incident, service request, and ticket category dashboards

This phase involves the following participants:

• Service Desk Manager
• Service Desk Technicians
• IT Managers

Visualize your ticket data as a first step to analysis

Identifying trends is easier when looking at diagrams, graphs, and figures

Start your analysis with common visuals employed by other service desk professionals

• Phase 2 will walk you through visualizing your data to get a better understanding of your ticket intake, incident management, and service request management.
• Each step will walk you through:
  • Common visualizations used by service desks
  • Patterns to look for in your visualizations
  • Actions to take to address negative patterns and to continue positive trends
• Share diagrams that underscore both the value being provided by the service desk as well as the scope of the pain points. Use Info-Tech’s Ticket Analysis Report template as a starting point.

“Being able to tell stories with data is a skill that’s becoming ever more important in our world of increasing data and desire for data-driven decision making. An effective data visualization can mean the difference between success and failure when it comes to communicating the findings of your study, raising money for your nonprofit, presenting to your board, or simply getting your point across to your audience.” - Cole Knaflic, Founder and CEO, Storytelling with Data: A Data Visualization Guide for Business Professionals

Use the detailed dashboards to determine the next steps for improvement

A single number doesn’t tell the whole picture

Analyze trends over time:

• Analyze trends by day, by week, by month, and by year to determine:
  • When are the busy periods? (E.g. Do tickets tend to spike every morning, every Monday, or every September?)
  • When are the slow periods? (E.g. Do tickets drop at the end of the day, at midday, on Fridays, or over the summer?)
• Are spikes or drops in volume consistent trends or one-time anomalies?

Then build a plan to address them:

• How will you handle volume spikes, if they’re consistent?
• What can your resources work on during slow times, if they are consistent?
• If you assume no shrinkage, can you handle the peaks in volume if you make all FTEs available to work on tickets at a certain time of day?

Look for seasonal trends. In this example, we see high ticket volumes in May and January, with lower ticket volumes in June and July when many staff are taking holidays. However, also be careful to look at the big picture of how you pulled the data. August through October sees a high volume of open tickets because the data set is pulled in November, not because there’s a seasonal spike on tickets not closing at the end of the fiscal year.

Track ticket data over time

Make low-effort adjustments before major changes

Don’t rush to a decision based off the first numbers you see

Review ticket summary dashboard

Ideally, you should track ticket patterns over an entire year to get a full sense of trends within each month of the year. At minimum, track for 30 days, then 60, then 90, and see if anything changes. The longer you can track ticket patterns, the more accurate your picture will be.

Review additional dashboards

If you separate incidents and service requests, and you have accurate ticket categories, then you can use these dashboards to further break down the data to identify ticket trends.

The output of the ticket analysis will only be as accurate as its input.
To get the most accurate results, first ensure your data is accurate, then analyze it over as much time as possible. Aggregating with accurate data will give you a better picture of the trends in demand that your service desk sees.

Not separating incidents and service requests? Need to fix your ticket categories? Visit Standardize the Service Desk to get started.

Analyze incidents and requests separately

Each type has its own set of customer experiences and expectations

• Different ticket types are associated with radically different prioritization, routing, and service levels. For instance, most incidents are resolved within a business day, but requests take longer to implement.
• If you fail to distinguish between ticket types, your metrics will obscure service desk performance.
• From a ticket analysis standpoint, separating ticket types prior to analysis or, better yet, at intake allows for cleaner data. In turn, this means more structured analyses, better insights, and more meaningful actions. Not separating ticket types may still get you to the same conclusions, but it will be much more difficult to sift through the data.

Incident

An unanticipated interruption of a service.
The goal of incident management is to restore the service as soon as possible, even if the resolution involves a workaround.

Request

A generic description for a small change or service access.
Requests are small, frequent, and low risk. They are best handled by a process distinct from incident, change, and project management.

Not separating incidents and service requests? Need to fix your ticket categories? Visit Standardize the Service Desk to get started.

Step 2.1

Analyze Your High-Level Ticket Data

Dashboards

• Ticket Volume
• Ticket Intake
• Ticket Handling and Resolution
• Ticket Categorization

This step will walk you through the following activities:

Visualize the current state of your service desk.

This step involves the following participants:

• Service Desk Manager
• Service Desk Technicians
• IT Managers

Outcomes of this step

Build your metrics baseline to compare with future metric results.

Dashboards: Ticket Volume

Analyze your data for insights

• Analyze volume trends by day, by week, by month, and by year to determine:
  • When are the busy periods? (E.g. Do tickets tend to spike every morning, every Monday, or every September?)
  • When are slow periods? (E.g. Do tickets drop at the end of the day, at midday, on Fridays, or over the summer?)
• Are spikes or drops in volume consistent trends or one-time anomalies?
• What can your resources be working on during slow times? Are you able to address ticket backlog?

Dashboards: Ticket Intake

Analyze your data for insights

• Determine how to drive intake to the most appropriate solution for your organization:
  • A web portal is the most efficient intake method, but it must be user friendly to increase its adoption.
  • The phone should be available for urgent requests or incidents. Encourage those who call with a request to submit a ticket through the portal.
  • Discourage use of email if it is unstructured, as users don’t provide enough detail, and often two or three transactions are required for triage.
  • If walk-ups are encouraged, structure and formalize the support so it can be resourced and managed rather than interrupt-driven.

Dashboard: Ticket Handling and Resolution

Analyze your data for insights

• Look at your ticket load by technician and by tier. This is an essential step to set your baseline to measure your shift-left initiatives. If you are focusing on self-service or Tier 1 training, the ticket load from higher tiers should decrease over time.
• If Tiers 2 and 3 are handling the majority of the tickets, this could be a red flag indicating tickets are inappropriately escalated or Tier 1 could use more training and support.
• For average time to resolve and average time to resolve by tier, are you meeting your SLAs? If not, are your SLAs too aggressive? Are tickets left open and not properly closed?

Dashboard: Ticket Categorization

Analyze your data for insights

• Ticket categorization is critical to clean data. Having a categorization scheme with categories that are miscellaneous, too specific, or too general easily leads to inaccurate reporting or confusing workflows for technicians.
• When looking at your ticket categories, first look for duplicate categories that could be collapsed into one.
• Also look at your top five to seven categories and see if they make sense. Are these good candidates in your organization for automation or shift-left?
• Compare your Tier 1 categories. The level of specificity for these categories should be comparable to easily run reports. If they are not, assess the need for a category redesign.

Step 2.2

Analyze Incidents, Service Requests, and Ticket Categories

Dashboards

• Incidents
• Service Requests
• Volume by Ticket Category
• Resolution Times by Priority and/or Category
• Tabs for More Granular Investigation and Reporting

This step will walk you through the following activities:

Visualize your incident and service request ticket load and analyze trends. Use this information and cross reference data sets to gain a holistic view of how the service desk interacts with IT and the business.

This step involves the following participants:

• Service Desk Manager
• Service Desk Technicians
• IT Managers

Outcomes of this step

Gain actionable, data-driven improvements based on your incident and service request data. Show the value of the service desk and highlight improvements needed.

Incident and Service Requests Dashboard: Priority and SLA

Analyze your data for insights

• Your ticket priority distribution for overall load and time to resolve (TTR) should look something like above with low-priority tickets having higher load and TTR and high/critical-priority tickets having a lower load and lower TTR. If it is reversed, that is a good indication that the service desk is too reactive or isn’t properly prioritizing its work.
• If your SLA has a high failure rate, consider reassessing your targets with SLOs that you can meet before publishing them as achievable SLAs.

Incident and Service Requests Dashboard: Priority and SLA

Analyze your data for insights

• Examine your ticket handling by looking at ticket status and resolution codes.
  • If you have a lot of blanks, then tickets are not properly handled. Consider reinforcing your standards for close codes and statuses.
  • Alternatively, if tickets are left open, you may have to build follow-ups on stale tickets into your process or introduce proper auto-close processes.

Category, Resolution Time, and Resolution Code Dashboards

These PivotCharts allow you to dig deeper

Investigate whether there are trends in ticket volume and resolution times within specific categories and subcategories

Tab 6, Category Dashboard; tab 7, Resolution Time Dashboard; and tab 8, Resolution Code Dashboard are PivotCharts. Use these tabs to investigate whether there are trends in ticket volume, resolution times, and resolution codes within specific categories and subcategories.

Start with the charts that are available. The +/- buttons will allow you to show more granular information. By default, this granularity will be into the levels of the ticket categorization scheme.

For most categorization schemes, there will be too many categories to properly graph. You can apply a filter to investigate specific categories by clicking on the drop-down buttons.

Use these tabs for more granular investigation and reporting

TAB 6 CATEGORY DASHBOARD	TAB 7 RESOLUTION TIME DASHBOARD	TAB 8 RESOLUTION TIME DASHBOARD
Investigate ticket distributions in first, second, and third levels. Are certain categories overcrowded, suggesting they can be split? Are certain categories not being used?	Do average resolution times match your service level agreements? Do certain categories have significantly different resolution times? Are there areas that can benefit from shift-left?	Are resolution codes being accurately used? Are there trends in resolution codes? Are these codes providing sufficient information for problem management?

PHASE 3

Communicate Your Insights

This phase will walk you through the following activities:

• 3.1.1 Review common recommendations
• 3.2.1 Review ticket reports daily
• 3.2.2 Incorporate ticket data into retrospectives and team updates
• 3.2.3 Regularly review trends with business leaders
• 3.2.4 Tell a story with your data

This phase involves the following participants:

• Service Desk Manager
• Service Desk Technicians
• IT Managers

Step 3.1

Build Recommendations Based on Your Ticket Data

Activities

• 3.1.1 Review common recommendations

This step will walk you through the following activities:

Review common recommendations as a first step to extracting insights from your own data.

This step involves the following participants:

• Service Desk Manager
• Service Desk Technicians

Outcomes of this step

You will gain an understanding of the common challenges with service desks and ticket analysis in general. See which ones apply to you to inform your ticket data analysis moving forward.

Review these common recommendations

1. Fix your ticket categories
   Organize your ticket categorization scheme for proper routing and reporting.
2. Focus more on self-service
   Self-service is essential to enable shift-left strategies. Focus on knowledgebase processes and portal ease of use.
3. Update your service catalog
   Improve your service catalog, if necessary, to make it easy for end users to request services and for the service desk to provide those services.
4. Direct volume toward other channels
   Walk-ups make it more difficult to properly log tickets and assign service desk resources. Drive volume to other channels to improve your ticket quality.
5. Crosstrain Tier 1 on certain topics
   Tier 1 breadth of knowledge is essential to drive up first contact resolution.
6. Build more automation
   Identify bottlenecks and challenges with your ticket data to streamline ticket handling and resolution.
7. Revisit service level agreements
   Update your SLAs and/or SLOs to prioritize expectation management for your end users.
8. Improve your data quality
   You can only analyze data that exists. Revisit your ticket-handling guidelines and more regularly check tickets to ensure they comply with those standards.

Optimize your processes and look for opportunities for automation

Leverage Info-Tech research to improve service desk processes

Review your service desk processes and tools for optimization opportunities:

• Clearly establish ticket-handling guidelines.
• Use ticket templates to reduce time spent entering tickets.
• Document incident management and service request fulfillment workflows and eliminate any unnecessary steps.
• Automate manual tasks wherever possible.
• Build or improve a self-service portal with a knowledgebase to allow users to resolve their own issues, reducing incoming ticket volume to the service desk.
• Optimize your internal knowledgebase to reduce time spent troubleshooting recurring issues.
• Leverage AI capabilities to speed up ticket processing and resolution.

Standardize the Service Desk

This project will help you build and improve essential service desk processes, including incident management, request fulfillment, and knowledge management.

Optimize the Service Desk With a Shift-Left Strategy

This project will help you build a strategy to shift service support left to optimize your service desk operations and increase end-user satisfaction.

Step 3.2

Action and Communicate Your Ticket Data

Activities

• 3.2.1 Review your ticket queues daily
• 3.2.2 Incorporate ticket data into retrospectives and team status updates
• 3.2.3 Regularly review trends with business leaders
• 3.2.4 Tell a story with your data

This step will walk you through the following activities:

Organize your scrums to report on the metrics that will inform daily and monthly operations.

This step involves the following participants:

• Service Desk Manager
• Service Desk Technicians
• IT Managers

Outcomes of this step

Use the dashboards and data to inform your daily and monthly scrums.

3.2.1 Review your ticket queues daily

Clean data is still useless if not used properly

• The metrics you’ve chosen to measure and visualize in the previous step are useful for informing your day-to-day, week-to-week, and month-to-month strategies for the service desk and IT. Conduct scrums daily to action your dashboard data to help clear ticket queues.
• Reference your dashboards daily with each IT team.
• You need to have a dashboard of open tickets assigned to each team.

Review Daily

• Ticket volume over the last day (look for spikes)
• SLA breach risks/SLA breaches
• Recurring incidents
• Tickets open
• Tickets handed over (confirmation of handover)

3.2.2 Incorporate ticket data into retrospectives and team status updates

Explain your metric spikes and trends

• Hold weekly or monthly meetings to review the ticket trends selected during Phases 1 and 2 of this blueprint.
• Review ticket spikes, identify seasonal trends, and discuss root causes (e.g. projects/changes going live, onboarding blitz).
• Discuss any actions associated with spikes and seasonal trends (e.g. resource allocation, hiring, training).
• You can incorporate other IT leaders or departments in this meeting as needed to discuss action items for improvement, quality assurance concerns, customer service concerns, and/or operating level agreement concerns.

Review Weekly/Monthly

• Ticket volume
• Ticket category by priority level over time
• Tickets from different business groups, VIP groups, and different vertical levels
• Tickets escalated, tickets that didn’t need to be escalated, tickets that were incorrectly escalated
• Ticket priority levels over time
• Most requested services
• Tickets resolved by which group over time
• Ability to meet SLAs and OLAs over time by different groups

3.2.3 Regularly review trends with business leaders

Use your data to help improve business relationships

Review the following with business leaders:

• Volume of work done this past time cycle for the leader’s group
• Trends and spikes in the data and possible explanations for them (note: get their input on the potential causes of trends)
• Improvements you plan to execute within the service desk
• Action items you need from the business leader

Use your data to show the value you provide to the group. Schedule quarterly meetings with the heads of different business groups to discuss the work that the service desk does for each group.

Show trends in incidents and service requests: “I see you have a spike in CRM tickets. I’ve been working with the CRM team to address this issue.”

3.2.4 Tell a story with your data

Effectively communicate with the business and leadership

• With your visualized metrics, organize your story into a presentation for different stakeholder groups. You can use the Ticket Analysis Report as a starting point to provide data about:
  • Value provided by the service desk
  • Successes
  • Opportunities for Improvements
  • Current state of KPIs
• Include information about the causes of data trends and actions you will take in response to the data.
• For each of these themes, look at the metrics you’ve chosen to track and see which ones fit to tell the story. Let the data do the talking.
• Consider supplementing the ticket data with data from other systems. For example, you can include data on transactional customer satisfaction surveys, knowledgebase utilization, and self-service utilization.

Download the Ticket Analysis Report.

Ticket Analysis Report

Include the following information as you build your ticket analysis report:

• Value Provided by the Service Desk
  Start with the value provided by the service desk to different areas of the business. Include information about first contact resolution, average resolution times, ticket volume (e.g. by category, priority, location, requestor).
• Successes
  Successes is a general field that can include how process improvements have impacted the service desk or how initiatives have enhanced shift-left opportunities. Highlight any positive trends over time.
• Opportunities for Improvement
  Let the data guide the conversation to where improvements can be made. Day-to-day ops, self-service tools, shifting work left from Tier 2, Tier 3, standardizing a non-standard service, and staffing adjustments are possibilities for this section.
• Current State of KPIs
  Mean time to resolve, FCR, ticket volume, and end-user satisfaction are great KPIs to include as a starting point.

Download the Ticket Analysis Report.

Summary of Accomplishment

Problem Solved

You now have a better understanding of how to action your service desk ticket data, including improvements to your current ticket templates for incidents and service requests.

You also have the data to craft a story to different stakeholder groups to celebrate the successes of the service desk and highlight possible improvements. Continue this exercise iteratively to continue improving the service desk.

Remember, ticket analysis is not a single event but an ongoing initiative. As you track, analyze, and action more data, you will find more improvements.

If you would like additional support, have our analysts guide you through other phases as part of an Info-Tech workshop.

Contact your account representative for more information.

workshops@infotech.com 1-888-670-8889

Additional Support

If you would like additional support, have our analysts guide you through other phases as part of an Info-Tech workshop.

Contact your account representative for more information.

workshops@infotech.com 1-888-670-8889

To accelerate this project, engage your IT team in an Info-Tech workshop with an Info-Tech analyst team. Info-Tech analysts will join you and your team at your location or welcome you to Info-Tech’s historic Toronto office to participate in an innovative onsite workshop.

The following are sample activities that will be conducted by Info-Tech analysts with your team:

Analyze your dashboards An analyst will walk through the ticket data and dashboards with you and your team to help interpret the data and tailor improvements	Populate your ticket data report Given the action items from this solution set, an analyst will help you craft a report to celebrate the successes and highlight needed improvements in the service desk.

Related Info-Tech Research

Optimize the Service Desk With a Shift-Left Strategy

The best type of service desk ticket is the one that doesn’t exist.

Incident & Problem Management

Don’t let persistent problems govern your department.

Design & Build a User-Facing Service Catalog

Improve user satisfaction with IT with a convenient menu-like catalog.

Bibliography

Bayes, Scarlett. “ITSM: 2021 & Beyond.” Service Desk Institute, 2021. Web.

“Benchmarking Report v.9.” Service Desk Institute, 17 Jan. 2020. Web.

Bennett, Micah. “The 9 Help Desk Metrics That Should Guide Your Customer Support.” Zapier, 3 Dec. 2015. Web.

“Global State of Customer Service: The transformation of customer service from 2015 to present day.” Microsoft Dynamics 365, Microsoft, 2020. Web.

Goodey, Ben. “How to Manually Analyze Support Tickets.” SentiSum, 26 July 2021. Web.

Jadhav, Megha. “Four Metrics to Analyze When Using Ticketing Software.” Vision Helpdesk Blog, 21 Mar. 2016. Web.

Knaflic, Cole Nussbaumer. Storytelling with Data: A Data Visualization Guide for Business Professionals. Wiley, 2015.

Li, Ta Hsin, et al. “Incident Ticket Analytics for IT Application Management Services.” 2014 IEEE International Conference on Services Computing, 2014. Web.

Olson, Sarah. “10 Help Desk Metrics for Service Desks and Internal Help Desks.” Zendesk Blog, Sept. 2021. Web.

Paramesh, S.P., et al. “Classifying the Unstructured IT Service Desk Tickets Using Ensemble of Classifiers.” 2018 3rd International Conference on Computational Systems and Information Technology for Sustainable Solutions (CSITSS), 2018. Web.

Volini, Erica, et al. “2021 Global Human Capital Trends: Special Report.” Deloitte Insights, 21 July 2021. Web.

“What Kind of Analysis You Can Perform on a Ticket Management System.” Commence, 3 Dec. 2019. Web.

INFO-TECH RESEARCH GROUP