Foster Data-Driven Culture With Data Literacy

Data literacy is an essential part of a data-driven culture, bridging the data knowledge gaps across all levels of the organization.

Analyst Perspective

Data literacy is the missing link to becoming a data-driven organization.

“Digital transformation” and “data driven” are two terms that are inseparable. With organizations accelerating in their digital transformation roadmap implementation, organizations need to invest in developing data skills with their people. Talent is scarce and the demand for data skills is huge, with 70% of employees expected to work heavily with data by 2025. There is no time like the present to launch an organization-wide data literacy program to bridge the data knowledge gap and foster a data-driven culture.

Data literacy training is as important as your cybersecurity training. It impacts all levels of the organization. Data literacy is critical to success with digital transformation and AI analytics.

Annabel Lui

Principal Advisory Director, Data & Analytics Practice
Info-Tech Research Group

Executive Summary

Info-Tech Insight

By thoughtfully designing a data literacy training program for the audience's own experience, maturity level, and learning style, organizations build the data-driven and engaged culture that helps them to unlock their data's full potential and outperform other organizations.

Your Challenge

Data literacy is the missing link to drive business outcomes from data.

• Having a data-driven culture as an organization’s mission statement without implementing a data literacy program is like making an empty promise and leaving the value unrealized and unattainable.
• A study conducted by the Data Literacy Project clearly indicates that organizations with aggressive data literacy programs will outperform those who do not have such programs. By 2030, data literacy will be one of the most sought-after skill sets. All employees require data literacy skills.
• Everyone has a role in data. From employees who are actively involved in data collection to operational teams who create reports with analytics tools and finally to executives who use data to make business decisions – they all require continuous data literacy training in a data-driven organization. Because of differences in maturity, data literacy strategies cannot be one-size-fits-all.

“Data literacy is the ability to read, work with, analyze, and communicate with data. It's a skill that empowers all levels of workers to ask the right questions of data and machines, build knowledge, make decisions, and communicate meaning to others.” – Qlik, n.d.

75% of organizational employees have access to data tools – only 21% demonstrated confidence in their data skills.

Source: Accenture, 2020.

89% of C-level executives expect team members to explain how data has informed their decisions, but only 11% employees are fully confident in their ability to read, analyze, work with, and communicate with data

Source: Qlik, 2022.

Data debt or data asset?

Manage your data as strategic assets.

“[Data debt is] when you have undocumented, unused, incomplete, and inconsistent data,” according to Secoda (2023). “When … data debt is not solved, data teams could risk wasting time managing reports no one uses and producing data that no one understands.”

Signs of data debt when considering investing in data literacy:

• Lack of definition and understanding of data terms, therefore they don’t speak the same language. Without data literacy, an organization will not succeed in becoming a data-driven organization.
• Putting data literacy as a low priority. Organization sees this as “another” training to put on the list and keeps it on the back burner.
• Data literacy is not seen as the number one skill set needed in the organization. However, anyone who works with data requires data skills.
• End users are not trained on self-serve features and tools.
• Focusing on a minority group of people rather than everyone in the organization or seeing it as a one-off exercise.
• Delays or failure to deliver digital transformation projects due to lack of data skills and data access issues.

66%

of organizations say a backlog of data debt is impacting new data management initiatives.

40%

of organizations say individuals within the business do not trust data insights.

30%

of organizations are unable to become data-driven.

Source: Experian, 2020

Info-Tech’s Approach

Data literacy is critical to success with digital transformation and AI analytics.

The Info-Tech difference:

1. More than just technical training. Data literacy program isn’t just about data but rather encompasses aspects of business, IT, and data.
2. More than a one-off exercise. To keep literacy skills alive, the program must be routine and sustainable, tailored to different needs across all levels of the organization.
3. More than one delivery format. Different delivery methods need to be considered to suit various learning styles.

Data needs to be processed

Data – facts – are organized, processed, and given meaning to become insights.

Image source: Welocalize, 2020.

Data represents a discrete fact or event without relation to other things (e.g. it is raining). Data is unorganized and not useful on its own.

Information organizes and structures data so that it is meaningful and valuable for a specific purpose (i.e. it answers questions). Information is a refined form of data.

When information is combined with experience and intuition, it results in knowledge. It is our personal map/model of the world.

Knowledge set with context generates insight. We become knowledgeable as a result of reading, researching, and memorizing (i.e. accumulating information).

Wisdom means the ability to make sound judgments. Wisdom synthesizes knowledge and experiences into insights.

Investment in data literacy is a game changer.

Data literacy is the ability to collect, manage, evaluate, and apply data in a critical manner.

A data-driven culture is “an operating environment that seeks to leverage data whenever and wherever possible to enhance business efficiency and effectiveness” (Forbes).

Info-Tech Insight

Data-driven culture refers to a workplace where decisions are made based on data evidence, not on gut instinct.

Info-Tech’s methodology for building a data literacy program

Insight Summary

“In a world of more data, the companies with more data-literate people are the ones that are going to win.”

– Miro Kazakoff, senior lecturer, MIT Sloan, in MIT Sloan School of Management, 2021

Overarching insight

By thoughtfully designing a data literacy training program personalized to each audience's maturity level, learning style, and experience, organizations can develop and grow a data-driven culture that unlocks the data's full potential for competitive differentiation.

Module 1 insight

We can learn a lot from each other. Literacy works both ways – business data stewards learn to “speak data” while IT data custodians understand the business context and value. Everyone should strive to exchange knowledge.

Module 2 insight

Avoid traditional classroom teaching – create a data literacy program that is learner-centric to allow participants to learn and experiment with data.

Aligning program design to those learning styles will make participants more likely to be receptive to learning a new skill.

Module 3 insight

A data literacy program isn’t just about data but rather encompasses aspects of business, IT, and data. With executive support and partnership with business, running a data literacy program means that it won’t end up being just another technical training. The program needs to address why, what, how questions.

Tactical insight

A lot of programs don’t include the fundamentals. To get data concepts to stick, focus on socializing the data/information/knowledge/wisdom foundation.

Tactical insight

Many programs speak in abstract terms. We present case studies and tangible use cases to personalize training to the audience’s world and showcase opportunities enabled through data.

Key performance indicators (KPIs) for your data literacy program

How do you know if your data literacy program is successful? Here are some useful KPIs:

Program Adoption Metrics

• Percentage of employees attending data literacy training
• Percentage of participants who report gains in data management knowledge after training sessions
• Maturity assessment result
• Survey and diagnostic feedback before and after training
• Trend analysis of overall data literacy program

Operational Metrics

• Number of requests for analytics/reporting services
• Number of reports created by users
• Speed and quality of business decisions
• User satisfaction with reports and analytics services
• Improved business performance (customer satisfaction)
• Improved valuation of organization data

A data-driven culture builds tools and skills, builds users’ trust in the quality of data across sources, and raises the skills and understanding among the frontlines by encouraging everyone to leverage data for critical thinking and innovation.

Info-Tech offers various levels of support to best suit your needs

DIY Toolkit

"Our team has already made this critical project a priority, and we have the time and capability, but some guidance along the way would be helpful."

Guided Implementation

"Our team knows that we need to fix a process, but we need assistance to determine where to focus. Some check-ins along the way would help keep us on track."

Workshop

"We need to hit the ground running and get this project kicked off immediately. Our team has the ability to take this over once we get a framework and strategy in place."

Consulting

"Our team does not have the time or the knowledge to take this project on. We need assistance through the entirety of the project."

Diagnostics and consistent frameworks are used throughout all four options.

Workshop Overview

Contact your account representative for more information.
workshops@infotech.com 1-888-670-8889

Phase 1

Define Data Literacy Objectives

Foster Data-Driven Culture With Data Literacy

This phase will walk you through the following activities:

• Understand the organization’s needs.
• Create vision and objective for data literacy program.

This phase involves the following participants:

• Data governance sponsor
• Data owners
• Data stewards
• Data custodians

1.1 Gauge your organization’s current data culture

Conduct data culture survey or diagnostic.

1. Identify members of the data user base, data consumers, and other key stakeholders for surveying.
2. Conduct an information session to introduce Info-Tech’s Data Culture Diagnostic survey. Explain the objective and importance of the survey and its role in helping to understand the organization’s current data culture and inform the improvement of that culture.
3. Roll out the Info-Tech Data Culture Diagnostic survey to the identified users and stakeholders.
4. Debrief and document the results and scorecard in the Data Strategy Stakeholder Interview Guide and Findings document.

Contact your Info-Tech Account Representative for details on launching a Data Culture Diagnostic.

1.2 Define data literacy objectives

1. Understand the organization’s needs by identifying opportunities and challenges relating to data. Document the described real-life examples.
2. Categorize the list and identify areas where data literacy can address the business problem.
3. Create a vision statement for the data literacy program, ensuring that it covers all levels of the organization.
4. Articulate the intended targets and goals in planning for a data literacy program.

Quick wins for improving data literacy

Data collected through Info-Tech’s Data Culture Diagnostic suggests three ways to improve data literacy:

87%

think more can be done to define and document commonly used terms with methods such as a business data glossary.

68%

think they can have a better understanding of the meaning of all data elements that are being captured or managed.

86%

feel that they can have more training in terms of tools as well as on what data is available at the organization.

Source: Info-Tech Research Group's Data Culture Diagnostic, 2022; N=2,652

Quick Wins

• Create a business data glossary to document and define common terms.
• Provide easy access to the business data glossary and procedures on how data is captured and managed.
• Launch an organization-wide data literacy program.

Delivering value is a means and the goal

Start with real business problems in a hands-on format to demonstrate the value of data.

Identify business problem:

• Business decisions without facts are just guesses.
• Management spends a lot of time finding and fixing data.
• Unknown challenges on data assets and risk.
• Incomplete view of customer/client and industry.
• Not ready for modern data opportunities (e.g. artificial intelligence).

Create an objective

Treat data as a strategic asset to gain insight into our customers for all levels of organization.

The solution: Data-driven culture powered by people who speak data.

• Data dictionary
• Data literacy
• Trusted single source
• Access to analytics tools
• Decision making

"According to Forrester, 91% of organizations find it challenging to improve the use of data insights for decision-making – even though 90% see it as a priority. Why the disconnect? A lack of data literacy."

– Alation, 2020

Fundamental data literacy

Data literacy is more than just a technical training or a one-off exercise.

Info-Tech provides various topics suited for a data literacy program that can accommodate different data skill requirements and encompasses relevant aspects of business, IT, and data.

Info-Tech Research Group’s Data Literacy Program

Use discovery and diagnostics to understand users’ comfort level and maturity with data.

Data lunch 'n' learn

• The power and value of data
• Everyone is a data steward
• Becoming data literate
• Data 101
• The future is data

1 hour

Speak data

• What is data
• Meet the data team
• Day in the life of a steward
• How data impacts you
• Tools of the trade

1/2 day

Your data story

• Ask the right questions
• Find the top five data elements
• Understand your data
• Present your data story
• Lessons from COVID-19

1/2 day

Phase 2

Assess Learning Style and Align to Program Design

Foster Data-Driven Culture With Data Literacy

This phase will walk you through the following activities:

• Identify your audience.
• Assess learning styles and align them to the data program design.
• Determine the right delivery method.

This phase involves the following participants:

• Data governance sponsor
• Data owners
• Data stewards
• Data custodians

Avoid common pitfalls

75%

feel that training was too long to remember or to apply in their day-to-day work.

21%

find training had insufficient follow-up to help them apply on the job.

Source: Grovo, 2018.

1. Information Overload

   Trying to cover too much useful information results in overwhelm and does not deliver on key training objectives.

2. Limited Implementation

   Learning is only the beginning. The real results are obtained when learning is followed by practice, which turns new knowledge into reliable habits.

3. Lack of Organizational Alignment

   Implementing training without a clear link to organizational objectives leaves you unable to clearly communicate its value, undermines your ability to secure buy-in from attendees and executives, and leaves you unable to verify that the training is actually improving effectiveness.

2.1 Understand learning style

1. Create persona and identify the audiences and their roles in data across all levels of the organization.
2. Identify the data program initiatives and assign the best delivery method to each initiative.
3. Assign participants to each program initiative based on their skill gap and learning style.

You and data

Is data an integral part of your work?

Do you feel comfortable finding and using data in your organization?

• Many people feel intimidated by data and therefore miss out on what data can do for them.
• Often the obstacle is language. If you don’t understand the semantics around data, you will not feel confident to contribute to discussions around data.
• You use data every day but need additional vocabulary to understand how to handle it properly.
• Data literacy is the ability to “speak data” and to understand what data means (i.e. how to read charts and graphs, draw valid conclusions, and recognize when data is misinterpreted or used inappropriately to be misleading).
• The business often doesn’t understand its role in data governance and how it informs and assists IT in responsible data management.

Info-Tech Insight

IT and data professionals need to understand the business as much as business needs to talk about data. Bidirectional learning and feedback improves the synergy between business and IT.

Create personas

Persona creation is a way to brainstorm ideas for the data literacy program.

Choose a data role (e.g. data steward, data owner, data scientist).

Describe the persona based on goals, priorities, tenures, preferred learning style, type of work with data.

Identify data skill and level of skills required.

Consider these other ways to brainstorm:

• Review current in-flight projects.
• Analyze types of data requests.
• Understand needs by department.
• Share learnings in a community of practice.

Program design

Categorize into six data skill areas

Not everyone needs the same level of skill sets

Map the personas to the program

Bridging the data knowledge gap.

• Each component will promote the value of data to all levels of employees when demonstrating the right way for data to be understood, managed, and consumed in the organization.
• Categorizing the data literacy program into six areas and levels of skill sets will provide clarity into which areas to focus on.
• The program is intended to be implemented in stages, allowing the audience to learn and adopt the new skills. Leveraging in-flight projects for rolling out training will have a higher success because the need is already built into the project.

Align program design to learning styles

Info-Tech Insight

Tailor your data literacy program to meet your organization’s needs, filling your range of knowledge gaps and catering to different levels of users.

When it comes to rolling out a data literacy program, there is no one-size-fits-all solution. Your data literacy program is intended to spread knowledge throughout your organization. It should target everyone from executive leadership to management to subject matter experts across all functions of the business.

Discussion method

Delivery Method

• Interactive format between instructor and learner
• Instructor empowers and motivates learner through dialogues and exercises

The imaginative learner

The imaginative learner group likes to engage in feelings and spend time on reflection. This type of learner desires personal meaning and involvement. They focus on personal values for themselves and others and make connections quickly.

For this group of learners, their question is: why should I learn this?

Learning characteristics

• Seek meaning
• Need to be personally involved
• Learn by listening and sharing ideas
• Function through social interaction

Information method

Delivery Method

• Instructor does most of the talking in the training
• Instructor is teaching the content, delivering the training content, and demonstrating

Analytical learner

The analytical learner group likes to listen, to think about information, and to come up with ideas. They are interested in acquiring facts and delving into concepts and processes. They can learn effectively and enjoy doing independent research.

For this group of learners, their question is: what should I learn?

Learning characteristics

• Seek and examine the facts
• Need to know what experts think
• Interested in ideas and concepts
• Critique information and collect data
• Function by adapting to experts

Coaching method

Delivery Method

• Learning has on-the-job training or learning through role-play exercises
• Instructor is coaching and facilitating learner

Common sense learner

The common sense learner group likes thinking and doing. They are satisfied when they can carry out experiments, build and design, and create usability. They like tinkering and applying useful ideas.

For this group of learners, their question is: how should I learn?

Learning characteristics

• Seek usability
• Need to know how things work
• Learn by testing theories using practical methods
• Use factual data to build concepts
• Enjoy hands-on experience

Self-discovery method

Delivery Method

• Interactive format between instructor and learner
• Instructor provides evaluation and remedial instruction

Common sense learner

The dynamic learner group learns through doing and experiencing. They are continually looking for hidden possibilities and researching ideas to make original adjustments. They learn through trial and error and self-discovery.

For this group of learners, their question is: what if I learn this?

Learning characteristics

• Seek hidden possibilities
• Need to know what can be done with things
• Learn by trial and error
• Enjoy variety and excel in being flexible

Delivery method considerations

There are four common ways to learn a new skill: by watching, conceptualizing, doing, and experiencing. The following are some suggestions on ways to implement your data literacy program through different delivery methods.

Phase 3

Map Out Data Literacy Roadmap and Milestones

Foster Data-Driven Culture With Data Literacy

This phase will walk you through the following activities:

• Complete a roadmap exercise.
• Set key performance metrics and milestones.

This phase involves the following participants:

• Data governance sponsor
• Data owners
• Data stewards
• Data custodians

3.1 Build the data literacy roadmap and milestones

1-3 hours

1. Gather the data literacy objectives and list of program initiatives with their assigned groups.
2. Discuss each program initiative with the data literacy creation team, assigning content owners and estimating effort required to build the content.

For the Gantt chart:

• Input the roadmap start year.
• List each data literacy topic and delivery method.
• Populate the planned start and end dates for the prepopulated list of program initiatives.

Data literacy journey mapping

Making it sustainable

• Deliver the literacy program in stages to make it easier for the audience to consume the content.
• Allow opportunities to apply the learnings at work.
• Map out the data literacy trainings as they get delivered and identify gaps, if any. Continue to refine and adjust the program and delivery method for better outcome.
• Set clear goals and KPIs measurement up front.
• Conduct Info-Tech Research Group’s Data Culture Diagnostics to set the baseline and repeat the assessment in 12 to 18 months.
• Assign champions to lead change and influence end users to adopt better processes.

Research contributors

Info-Tech’s Data Literacy Program

Contact your account representative for more information.
workshops@infotech.com 1-888-670-8889

Related Info-Tech Research

Establish Data Governance

Deliver measurable business value.

Build a Robust and Comprehensive Data Strategy

Key to building and fostering a data-driven culture.

Create a Data Management Roadmap

Streamline your data management program with our simplified framework.

Bibliography

About Learning. “4MAT overview.” About Learning., 16 Aug. 2001. Web.

Accenture. “The Human Impact of Data Literacy,” Accenture, 2020. Web.

Anand, Shivani. “IDC Reveals India Data and Content Technologies Predictions for 2022 and onwards; Focus on Data Literacy for an Elevated data Culture.” IDC, 14 Mar. 2022. Web.

Belissent, Jennifer, and Aaron Kalb. “Data Literacy: The Key to Data-Driven Decision Making.” Alation, April 2020. Web.

Brown, Sara. “How to build data literacy in your company.” MIT Sloan School of Management, 9 Feb 2021. Web.

---. “How to build a data-driven company.” MIT Sloan School of Management, 24 Sept. 2020. Web.

Domo. “Data Never Sleeps 9.0.” Domo, 2021. Web.

Dykes, Brent. “Creating A Data-Driven Culture: Why Leading By Example Is Essential.” Forbes, 26 Oct. 2017. Web.

Experian. “10 signs you are sitting on a pile of data debt.” Experian, 2020. Accessed 25 June 2021. Web.

Experian. “2019 Global Data Management Research.” Experian, 2019. Web.

Knight, Michelle. “Data Literacy Trends in 2023: Formalizing Programs.” Dataversity, 3 Jan. 2023. Web.

Ghosh, Paramita. “Data Literacy Skills Every Organization Should Build.” Dataversity, 2 Nov. 2022. Web.

Johnson, A., et al., “How to Build a Strategy in a Digital World,” Compact, 2018, vol. 2. Web.

LifeTrain. “Learning Style Quiz.” EMTrain, Web.

Lambers, E., et al. “How to become data literate and support a data-drive culture.” Compact, 2018, vol. 4. Web.

Marr, Benard. “Why is data literacy important for any business?” Bernard Marr & Co., 16 Aug. 2022. Web.

Marr, Benard. “8 simple ways to enhance your data literacy skills.” Bernard Marr & Co., 16 Aug. 2022. Web/

Mendoza, N.F. “Data literacy: Time to cure data phobia” Tech Republic, 27 Sept. 2022. Web.

Mizrahi, Etai. “How to stay ahead of data debt and downtime?” Secoda, 17 April 2023. Web.

Needham, Mass., “IDC FutureScape: Top 10 Predictions for the Future of Intelligence.” IDC, 5 Dec. 2022. Web.

Paton, J., and M.A.P. op het Veld. “Trusted Analytics.” Compact, 2017, vol. 2. Web.

Qlik. “Data Literacy to be Most In-Demand Skill by 2030 as AI Transforms Global Workplaces.” Qlik., 16 Mar 2022. Web.

Qlik. “What is data literacy?” Qlik, n.d. Web.

Reed, David. Becoming Data Literate. Harriman House Publishing, 1 Sept. 2021. Print.

Salomonsen, Summer. “Grovo’s First-Time Manager Microlearning® Program Will Help Your New Managers Thrive in 2018.” Grovos Blog, 5 Dec. 2018. Web.

Webb, Ryan. “More Than Just Reporting: Uncovering Actionable Insights From Data.” Welocalize, 1 Sept. 2020. Web.

Manage Exponential Value Relationships

Are you ready to manage outcome-based agreements?

Analyst Perspective

Outcome-based agreements require a higher degree of mutual trust.

Exponential IT brings with it an exciting new world of cutting-edge technology and increasingly accelerated growth of business and IT. But adopting and driving change through this paradigm requires new capabilities to grow impactful and meaningful partnerships with external vendors who can help implement technologies like artificial intelligence and virtual reality. Building outcome-based partnerships involves working very closely with vendors who, in many cases, will have just as much to lose as the organizations implementing these new technologies. This requires a greater degree of trust between parties than a standard vendor relationship. It also drastically increases the risks to both organizations; as each loses some control over data and outcomes, they must trust that the other organization will follow through on commitments and obligations. Outcome-based partnerships build upon traditional vendor management practices and create the potential for organizations to embrace emerging technology in new ways. Kim Osborne Rodriguez Research Director, CIO Advisory Info-Tech Research Group

Executive Summary

Info-Tech Insight

Outcome-based partnerships require a higher degree of trust than traditional vendor relationships. Build trust by sharing risks and rewards.

Vendor relationships can be worth billions of dollars

Positive vendor relationships directly impact the bottom line, sometimes to the tune of billions of dollars annually.

• Organizations typically spend 40% to 80% of their total budget on external suppliers.
• Greater supplier trust translates directly to greater business profits, even in traditional vendor relationships.1
• Based on over a decade of data from vehicle manufacturers, greater supplier relationships nearly doubled the unit profit margin on vehicles, contributing over $20 billion to Toyota’s annual profits based on typical sales volume.2
• Having positive vendor relationships can be instrumental in times of crisis – when scarcity looms, vendors often choose to support their best customers.3,4 For example, Toyota protected itself from the losses many original equipment manufacturers (OEMs) faced in 2020 and showed improved profitability that year due to increased demand for vehicles which it was able to supply as a result of top-ranked vendor relationships.

1 PR Newswire, 2022.
2 Based on 10 years of data comparing Toyota and Nissan, every 1-point increase in the company’s Working Relations Index was correlated with a $15.77 net profit increase per unit. Impact on Toyota annual profits is based on 10.5 million units sold in 2021 and 2022.
3 Interview with Renee Stanley, University of Texas at Arlington. Conducted 17 May 2023.
4 Plante Moran, 2020.

Sources: Macrotrends, Plante Moran 2022, Nissan 2022 and 2023, and Toyota 2022. Profit per car is based on total annual profit divided by total annual sales volume.

Outcome-based relationships are a new paradigm

In a new model where organizations are procuring autonomous capabilities, outcomes will govern vendor relationships.

An outcome-based relationship requires a higher level of mutual trust than traditional vendor relationships. This requires shared reward and shared risk.

Don’t forget about traditional vendor management relationships! Not all vendor relationships can (or should) be outcome-based.

Case study

INDUSTRY: Technology

SOURCE: Press Release

Microsoft and OpenAI partner on Azure, Teams, and Microsoft Office suite

In January 2023, Microsoft announced a $10 billion investment in OpenAI, allowing OpenAI to continue scaling its flagship large language model, ChatGPT, and giving Microsoft first access to deploy OpenAI’s products in services like GitHub, Microsoft Office, and Microsoft Teams.

Shared risk

Issues with OpenAI’s platforms could have a debilitating effect on Microsoft’s own reputation – much like Google’s $100 billion stock loss following a blunder by its AI platform Bard – not to mention the financial loss if the platform does not live up to the hype.

Shared reward

This was a particularly important strategic move by Microsoft, as its main competitors develop their own AI models in a race to the top. This investment also gave OpenAI the resources to continue scaling and evolving its services much faster than it would be capable of on its own. If OpenAI’s products succeed, there is a significant upside for both companies.

Adapt your approach to vendor relationships

Both traditional vendors and exponential relationships are important.

Use Info-Tech’s research to Jump Start Your Vendor Management Initiative.

Common obstacles

Exponential relationships require new approaches to vendor management as businesses autonomize:

• Autonomization refers to the shift toward autonomous business capabilities which leverage technologies such as AI and quantum computing to operate independently of human interaction.
• The speed and complexity of technology advancement requires that businesses move quickly and confidently to develop strong relationships and deliver value.
• We are seeing businesses shift from procuring products and services to procuring autonomous business capabilities (sometimes called “as a service,” or aaS). This shift can drive exponential value but also increases complexity and risk.
• Exponential IT requires a shift in emphasis toward more mature relationship and risk management strategies, compared to traditional vendor management.

The shift from technology service agreements to business capability agreements needs a new approach

Eighty-seven percent of organizations are currently experiencing talent shortages or expect to within a few years.

Source: McKinsey, “Mind the [skills] gap”, 2021.

Sixty-three percent of IT leaders plan to implement AI in their organizations by the end of 2023.

Source: Info-Tech Research Group survey, 2022

Insight summary

Assess your readiness for exponential value relationships

Key deliverable:

Exponential Relationships Readiness Assessment

Determine your readiness to build exponential value relationships.

Measure the value of this blueprint

Save thousands of dollars by leveraging this research to assess your readiness, before you lose millions from a relationship gone bad.

Our research indicates that most organizations would take months to prepare this type of assessment without using our research. That’s over 80 person-hours spent researching and gathering data to support due diligence, for a total cost of thousands of dollars. Doesn’t your staff have better things to do?

Start by answering a few brief questions, then return to this slide at the end to see how much your answers have changed.

Use Info-Tech’s research to Exponential Relationships Readiness Assessment.

Establish a baseline

Gauge the effectiveness of this research by asking yourself the following questions before and after completing your readiness assessment:

How to use this research

Five tips to get the most out of your readiness assessment.

1. Each category consists of five competencies, with a maximum of five points each. The maximum score on this assessment is 100 points.
2. Effectiveness levels range from basic (level 1) to advanced (level 5). Level 1 is generally considered the baseline for most effectively operating organizations. If your organization is struggling with level 1 competencies, it is recommended to improve maturity in those areas before pursuing exponential relationships.
3. This assessment is qualitative; complete the assessment to the best of your ability, based on the scoring rubric provided. If you fall between levels, use the lower one in your assessment.
4. The scoring rubric may not perfectly fit the processes and practices within every organization. Consider the spirit of the description and score accordingly.
5. Other industry- and region-specific competencies may be required to succeed at exponential relationships. The competencies in this assessment are a starting point, and internal validation and assessments should be conducted to uncover additional competencies and skills.

Financial management

Manage your budget and spending to stay on track throughout your relationship.

“Most organizations underestimate the amount of time, money, and skill required to build and maintain a successful relationship with another organization. The investment in exponential relationships is exponential in itself – as are the returns.”

– Jennifer Perrier, Principal Research Director,
Info-Tech Research Group

This step involves the following participants:

• Executive leadership team, including CIO
• CFO
• Vendor management leader
• Other internal stakeholders of vendor relationships

Activities:

• Assess your ability to manage scope and budget in exponential IT relationships.

Successfully manage complex finances

Stay on track and keep your relationship running smoothly.

Why is this important?

• Finance is at the core of most business – it drives decision making, acts as a constraint for innovation and optimization, and plays a key role in assessing options (such as return on investment or payback period).
• Effectively managing finances is a critical success factor in developing strong relationships. Each organization must be able to manage their own budget and spending in order to balance the risk and reward in the relationship. Often, these risks and rewards will come in the form of profit and loss or revenue and spend.

Build it into your practice:

1. Ensure your financial decision-making practices are aligned with the organizational and relationship strategy. Do metrics and criteria reflect the organization’s goals?
2. Develop strong accounting and financial analysis practices – this includes the ability to conduct financial due diligence on potential vendors.
3. Develop consistent methodology to track and report on the desired outcomes on a regular basis.

Build your ability to manage finances

The five competencies needed to manage finances in exponential value relationships are:

Relationship management

Drive exponential value by becoming a customer of choice.

“The more complex the business environment becomes — for instance, as new technologies emerge or as innovation cycles get faster — the more such relationships make sense. And the better companies get at managing individual relationships, the more likely it is that they will become “partners of choice” and be able to build entire portfolios of practical and value-creating partnerships.”

(“Improving the management of complex business partnerships.” McKinsey, 2019)

This step involves the following participants:

• Executive leadership team, including CIO
• Vendor management leader
• Other internal stakeholders of vendor relationships

Activities:

• Assess your ability to manage relationships in exponential IT relationships.

Take your relationships to the next level

Maintaining positive relationships is key to building trust.

Why is this important?

• All relationships will experience challenges, and the ability to resolve these issues will rely heavily on the relationship management skills and soft skills of the leadership within each organization.
• Based on a 20-year study of vendor relationships in the automotive sector, business-to-business trust is a function of reasonable demands, follow-through, and information sharing.

(Source: Plante Moran, 2020)

Build it into your practice:

1. Develop the soft skills necessary to promote psychological safety, growth mindset, and strong and open communication channels.
2. Be smart about sharing information – you don’t need to share everything, but being open about relevant information will enhance trust.
3. Both parties need to work hard to develop trust necessary to build a true relationship. This will require increased access to decision-makers, clearly defined guardrails, and the ability for unsatisfied parties to leave.

Build your ability to manage relationships

The five competencies needed to manage relationships in exponential partnerships are:

Performance management

Outcomes management focuses on results, not methods.

According to Jennifer Robinson, senior editor at Gallup, “This approach focuses people and teams on a concrete result, not the process required to achieve it. Leaders define outcomes and, along with managers, set parameters and guidelines. Employees, then, have a high degree of autonomy to use their own unique talents to reach goals their own way.” (Forbes, 2023)

In the context of exponential relationships, vendors can be given a high degree of autonomy provided they meet their objectives.

This step involves the following participants:

• Executive leadership team, including CIO
• Vendor management leader
• Other internal stakeholders of vendor relationships

Activities:

• Assess your ability to manage outcomes in exponential IT relationships.

Manage outcomes to drive mutual success

Build trust by achieving shared objectives.

Why is this important?

• Relationships are based on shared risk and shared reward for all parties. In order to effectively communicate the shared rewards, you must first understand and communicate your objectives for the relationship, then measure outcomes to ensure all parties are benefiting.
• Effectively managing outcomes reduces the risk that one party will choose to leave based on a perception of benefits not being achieved. Parties may still leave the agreement, but decisions should be based on shared facts and issues should be communicated and addressed early.

Build it into your practice:

1. Clearly articulate what you hope to achieve by entering an outcome-based relationship. Each party should outline and agree to the goals, objectives, and desired outcomes from the relationship.
2. Document how rewards will be shared among parties. What type of rewards are anticipated? Who will benefit and how?
3. Develop consistent methodology to track and report on the desired outcomes on a regular basis. This might consist of a vendor scorecard or a monthly meeting.

Build your ability to manage outcomes

The five competencies needed to manage outcomes in exponential value relationships are:

Risk management

Exponential IT means exponential risk – and exponential rewards.

One of the key differentiators between traditional vendor relationships and exponential relationships is the degree to which risk is shared between parties. This is not possible in all industries, which may limit companies’ ability to participate in this type of exponential relationship.

This step involves the following participants:

• Executive leadership team, including CIO
• Vendor management leader
• Risk management leader
• Other internal stakeholders of vendor relationships

Activities:

• Assess your ability to manage risk in exponential IT relationships.

Relationships come with a lot of hidden risks

Successfully managing complex risks can be the difference between a spectacular success and company-ending failure.

Why is this important?

• Relationships inherently involve a loss of control. You are relying on another party to fulfill their part of the agreement, and you depend on the success of the outcome. Loss of control comes with significant risks.
• Sharing in risk is what differentiates an outcome-based relationship from a traditional vendor relationship; vendors must have skin in the game.
• Organizations must consider many different types of risk when considering a relationship with a vendor: fraud, security, human rights, labor relations, ESG, and operational risks. Remember that risk is not inherently bad; some risk is necessary.

Build it into your practice:

1. Build or hire the necessary risk expertise needed to properly assess and evaluate the risks of potential vendor relationships. This includes intellectual property, ESG, legal/regulatory, cybersecurity, data security, and more.
2. Develop processes and procedures which clearly communicate and report on risk on a regular basis.

Info-Tech Insight

Some highly regulated industries (such as finance) are prevented from transferring certain types of risk. In these industries, it may be much more difficult to form vendor relationships.

Don’t forget about third-party ESG risk

Customers care about ESG. You should too.

Protect yourself against third-party ESG risks by considering the environmental and social impacts of your vendors.

Third-party ESG risks can include the following:

• Environmental risk: Vendors with unsustainable practices such as carbon emissions or waste generation of natural resource depletion can negatively impact the organization’s environmental goals.
• Social risk: Unsafe or illegal labor practices, human rights violations, and supply chain management issues can reflect negatively on organizations that choose to work with vendors who engage in such practices.
• Governance risk: Vendors who engage in illegal or unethical behaviors, including bribery and corruption or data and privacy breaches can impact downstream customers.

Working with vendors that have a poor record of ESG carries a very real reputational risk for organizations who do not undertake appropriate due diligence.

A global survey of nearly 14,000 customers revealed that…

Source: EY Future Consumer Index, 2021

Seventy-seven percent of customers believe companies have a responsibility to manufacture sustainably.

Sixty-eight percent of customers believe businesses should ensure their suppliers meet high social and environmental standards.

Fifty-five percent of customers consider the environmental impact of production in their purchasing decisions.

Build your ability to manage risk

The five competencies needed to manage risk in exponential value relationships are:

Contract management

Contract management is a critical part of vendor management.

Well-managed contracts include clearly defined pricing, performance-based outcomes, clear roles and responsibilities, and appropriate remedies for failure to meet requirements. In outcome-based relationships, contracts are generally used as a secondary method of enforcing performance, with relationship management being the primary method of addressing challenges and ensuring performance.

This step involves the following participants:

• Executive leadership team, including CIO
• Vendor management leader
• Risk management leader
• Other internal stakeholders of vendor relationships

Activities:

• Assess your ability to manage risk in exponential IT relationships.

Build your ability to manage contracts

The five competencies needed to manage contracts in exponential value relationships are:

Activity 1: Assess your readiness for exponential relationships

1-3 hours

1. Gather key stakeholders from across your organization to participate in the readiness assessment exercise.
2. As a group, review the core competencies from the previous four sections and determine where your organization’s effectiveness lies for each competency. Record your responses in the Exponential Relationships Readiness Assessment tool.

Download the Exponential Relationships Readiness Assessment tool.

Understand your assessment

This step involves the following participants:

• Executive leadership team, including CIO
• Vendor management leader
• Other internal stakeholders of vendor relationships

Activities:

• Create an action plan.

Understand the results of your assessment

Consider the following recommendations based on your readiness assessment scores:

• The chart to the right shows sample results. The bars indicate the recommended scores, and the line indicates the readiness score.
• Three or more categories below the recommended scores, or any categories more than five points below the recommendation: outcome-based relationships are not recommended at this time.
• Two or more categories below the recommended scores: Proceed with caution and limit outcome-based relationships to low-risk areas. Continue to mature capabilities.
• One category below the recommended scores: Evaluate the risks and benefits before engaging in higher-risk vendor relationships. Continue to mature capabilities.
• All categories at or above the recommended scores: You have many of the core capabilities needed to succeed at exponential relationships! Continue to evaluate and refine your vendor relationships strategy, and identify any additional competencies needed based on your industry or region.

Activity 2: Create an action plan

1 hour

1. Gather the stakeholders who participated in the readiness assessment exercise.
2. As a group, review the results of the readiness assessment. Where there any surprise? Do the results reflect your understanding of the organization’s maturity?
3. Determine which areas are likely to limit the organization’s relationship capability, based on lowest scoring areas and relative importance to the organization.
4. Break out into groups and have each group identify three actions the organization could take to mature the lowest scoring areas.
5. Bring the group back together and prioritize the actions. Note who will be accountable for each next step.

Related Info-Tech Research

Jump Start Your Vendor Management Initiative
Create and implement a vendor management framework to begin obtaining measurable results in 90 days.

Elevate Your Vendor Management Initiative
Transform your VMI from tactical to strategic to maximize its impact and value

Evaluate Your Vendor Account Team to Optimize Vendor Relations
Understand the value of knowing your account team’s influence in the organization, and your influence, to drive results.

Related Info-Tech Research

Build an IT Risk Management Program
Mitigate the IT risks that could negatively impact your organization.

Build an IT Budget
Effective IT budgets are more than a spreadsheet. They tell a story.

Adopt an Exponential IT Mindset
Thrive through the next paradigm shift..

Author

Kim Osborne Rodriguez Research Director, CIO Advisory Info-Tech Research Group

Kim is a professional engineer and Registered Communications Distribution Designer (RCDD) with over a decade of experience in management and engineering consulting spanning healthcare, higher education, and commercial sectors. She has worked on some of the largest hospital construction projects in Canada, from early visioning and IT strategy through to design, specifications, and construction administration. She brings a practical and evidence-based approach, with a track record of supporting successful projects. Kim holds a Bachelor’s degree in Honours Mechatronics Engineering and an option in Management Sciences from the University of Waterloo.

Research Contributors and Experts

	Jack Hakimian Senior Vice President Info-Tech Research Group Jack has more than 25 years of technology and management consulting experience. He has served multibillion-dollar organizations in multiple industries including financial services and telecommunications. Jack also served several large public sector institutions. He is a frequent speaker and panelist at technology and innovation conferences and events and holds a Master’s degree in Computer Engineering as well as an MBA from the ESCP-EAP European School of Management.		Michael Tweedie Practice Lead, CIO Strategy Info-Tech Research Group Mike Tweedie brings over 25 years as a technology executive. He’s led several large transformation projects across core infrastructure, application and IT services as the head of Technology at ADP Canada. He was also the Head of Engineering and Service Offerings for a large French IT services firm, focused on cloud adoption and complex ERP deployment and management. Mike holds a Bachelor’s degree in Architecture from Ryerson University.
	Scott Bickley Practice Lead, VCCO Info-Tech Research Group Scott Bickley is a Practice Lead & Principal Research Director at Info-Tech Research Group, focused on Vendor Management and Contract Review. He also has experience in the areas of IT Asset Management (ITAM), Software Asset Management (SAM), and technology procurement along with a deep background in operations, engineering, and quality systems management. Scott holds a B.S. in Justice Studies from Frostburg State University. He also holds active IAITAM certification designations of CSAM and CMAM and is a Certified Scrum Master (SCM).		Donna Bales Principal Research Director Info-Tech Research Group Donna Bales is a Principal Research Director in the CIO Practice at Info-Tech Research Group, specializing in research and advisory services in IT risk, governance, and compliance. She brings over 25 years of experience in strategic consulting and product development and has a history of success in leading complex, multistakeholder industry initiatives. Donna has a bachelor’s degree in economics from the University of Western Ontario.

Research Contributors and Experts

Jennifer Perrier Principal Research Director Info-Tech Research Group Jennifer has 25 years of experience in the information technology and human resources research space, joining Info-Tech in 1998 as the first research analyst with the company. Over the years, she has served as a research analyst and research manager, as well as in a range of roles leading the development and delivery of offerings across Info-Tech’s product and service portfolio, including workshops and the launch of industry roundtables and benchmarking. She was also Research Lead for McLean & Company, the HR advisory division of Info-Tech, during its start-up years. Jennifer’s research expertise spans the areas of IT strategic planning, governance, policy and process management, people management, leadership, organizational change management, performance benchmarking, and cross-industry IT comparative analysis. She has produced and overseen the development of hundreds of publications across the full breadth of both the IT and HR domains in multiple industries. In 2022, Jennifer joined Info-Tech’s IT Financial Management Practice with a focus on developing financial transparency to foster meaningful dialogue between IT and its stakeholders and drive better technology investment decisions.

Phil Bode Principal Research Director Info-Tech Research Group Phil has 30+ years of experience with IT procurement-related topics: contract drafting and review, negotiations, RFXs, procurement processes, and vendor management. Phil has been a frequent speaker at conferences, a contributor to magazine articles in CIO Magazine and ComputerWorld, and quoted in many other magazines. He is a co-author of the book The Art of Creating a Quality RFP. Phil has a Bachelor of Science in Business Administration with a double major of Finance and Entrepreneurship and a Bachelor of Science in Business Administration with a major of Accounting, both from the University of Arizona.

Research Contributors

Erin Morgan Assistant Vice President, IT Administration University of Texas at Arlington

Renee Stanley Assistant Director IT Procurement and Vendor Management University of Texas at Arlington

Note: Additional contributors did not wish to be identified.

Bibliography

Andrea, Dave. “Plante Moran’s 2022 Working Relations Index® (WRI) Study shows supplier relations can improve amid industry crisis.” Plante Moran, 25 Aug 2022. Accessed 18 May 2023.
Andrea, Dave. “Trust between suppliers and OEMs can better prepare you for the next crisis.” Plante Moran, 9 Sept 2020. Accessed 17 May 2023.
Cleary, Shannon, and Carolan McLarney. “Organizational Benefits of an Effective Vendor Management Strategy.” IUP Journal of Supply Chain Management, Vol. 16, Issue 4, Dec 2019.
De Backer, Ruth, and Eileen Kelly Rinaudo. “Improving the management of complex business partnerships.” McKinsey, 21 March 2019. Accessed 9 May 2023 .
Dennean, Kevin et al. “Let's chat about ChatGPT.” UBS, 22 Feb 2023. Accessed 26 May 2023.
F&I Tools. “Nissan Worldwide Vehicle Sales Report.” Factory Warranty List, 2022. Accessed 18 May 2023.
Gomez, Robin. “Adopting ChatGPT and Generative AI in Retail Customer Service.” Radial, 235, April 2023. Accessed 10 May 2023.
Harms, Thomas and Kristina Rogers. “How collaboration can drive value for you, your partners and the planet.” EY, 26 Oct 2021. Accessed 10 May 2023.
Hedge & Co. “Toyota, Honda finish 1-2; General Motors finishes at 3rd in annual Supplier Working Relations Study.” PR Newswire, 23 May 2022. Accessed 17 May 2023.
Henke Jr, John W., and T. Thomas. "Lost supplier trust, lost profits." Supply Chain Management Review, May 2014. Accessed 17 May 2023.
Information Services Group, Inc. “Global Demand for IT and Business Services Continues Upward Surge in Q2, ISG Index™ Finds.” BusinessWire, 7 July 2021. Accessed 8 May 2023.
Kasanoff, Bruce. “New Study Reveals Costs Of Bad Supplier Relationships.” Forbes, 6 Aug 2014. Accessed 17 May 2023.
Macrotrends. “Nissan Motor Gross Profit 2010-2022.” Macrotrends. Accessed 18 May 2023.
Macrotrends. “Toyota Gross Profit 2010-2022.” Macrotrends. Accessed 18 May 2023.
McKinsey. “Mind the [skills] gap.” McKinsey, 27 Jan 2021. Accessed 18 May 2023.
Morgan, Blake. “7 Examples of How Digital Transformation Impacted Business Performance.” Forbes, 21 Jul 2019. Accessed 10 May 2023.
Nissan Motor Corporation. “Nissan reports strong financial results for fiscal year 2022.” Nissan Global Newsroom, 11 May 2023. Accessed 18 May 2023.

Bibliography

“OpenAI and Microsoft extend partnership.” Open AI, 23 Jan 2023. Accessed 26 May 2023.
Pearson, Bryan. “The Apple Of Its Aisles: How Best Buy Lured One Of The Biggest Brands.“ Forbes, 23 Apr 2015. Accessed 23 May 2023.
Perifanis, Nikolaos-Alexandros and Fotis Kitsios. “Investigating the Influence of Artificial Intelligence on Business Value in the Digital Era of Strategy: A Literature Review.” Information, 2 Feb 2023. Accessed 10 May 2023.
Scott, Tim and Nathan Spitse. “Third-party risk is becoming a first priority challenge.” Deloitte. Accessed 18 May 2023.
Stanley, Renee. Interview by Kim Osborne Rodriguez, 17 May 2023.
Statista. “Toyota's retail vehicle sales from 2017 to 2021.” Statista, 27 Jul 2022. Accessed 18 May 2023.
Tlili, Ahmed, et al. “What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education.” Smart Learning Environments, 22 Feb 2023. Accessed 9 May 2023.
Vitasek, Kate. “Outcome-Based Management: What It Is, Why It Matters And How To Make It Happen.” Forbes, 12 Jan 2023. Accessed 9 May 2023.

Identify and Build the Data & Analytics Skills Your Organization Needs

Blending soft skills with deep technical expertise is essential for building successful data & analytics teams.

Analyst Perspective

Blending soft skills with deep technical expertise is essential for building successful data & analytics teams.

In today's changing environment, data & analytics (D&A) teams have become an essential component, and it is critical for organizations to understand the skill and talent makeup of their D&A workforce. Chief data & analytics officers (CDAOs) or other equivalent data leaders can train current data employees or hire proven talent and quickly address skills gaps.

While developing technical skills is critical, soft skills are often left underdeveloped, yet lack of such skills is most likely why the data team would face difficulty moving beyond managing technology and into delivering business value.

Follow Info-Tech's methodology to identify and address skills gaps in today's data workplace. Align D&A skills with your organization's data strategy to ensure that you always have the right skills at the right time.

Ruyi Sun
Research Specialist,
Data & Analytics, and Enterprise Architecture
Info-Tech Research Group

Executive Summary

Your Challenge

The rapid technological evolution in platforms, processes, and applications is leading to gaps in the skills needed to manage and use data. Some critical challenges organizations with skills deficiencies might face include:

• Time loss due to delayed progress and reworking of initiatives
• Poor implementation quality and low productivity
• Reduced credibility of data leader and data initiatives

Common Obstacles

Some common obstacles that could prevent you from identifying and building the data and analytics (D&A) skills your organization needs are:

• Lack of resources and knowledge to secure professionals with the right mixed D&A skills and the right experience/skill level
• Lack of well-formulated and robust data strategy
• Neglecting the value of soft skills and placing all your attention on technical skills

Info-Tech's Approach

Follow Info-Tech's guidance on the roles and skills required to support your D&A strategic growth objectives and how to execute an actionable plan:

• Define skills required for each essential data and analytics role
• Identify roles and skills gap in alignment with your current data strategy
• Establish action plan to close the gaps and reduce risks

Info-Tech Insight

Skills gaps are a frequently named obstacle to realizing corporate goals for D&A. Soft skills and technical skills are complementary, and a D&A team needs both to perform effectively. Identify the essential skills and the gap with current skills required by your organization's data strategy to ensure the right skill is available at the right time and to minimize applicable risks.

The rapidly changing environment is impacting the nature of work

Scarcity of data & analytics (D&A) skills

• Data is one of the most valuable organizational assets, and regardless of your industry, data remains the key to informed decision making. More than 75% of businesses are looking to adopt technologies like big data, cloud computing, and artificial intelligence (AI) in the next five years (World Economic Forum, 2023). As organizations pivot in response to industry disruptions and technological advancements, the nature of work is changing, and the demand for data expertise has grown.
• Despite an increasing need for data expertise, organizations still have trouble securing D&A roles due to inadequate upskilling programs, limited understanding of the skills required, and more (EY, 2022). Notably, scarce D&A skills have been critical. More workers will need at least a base level of D&A skills to adequately perform their jobs.

Organizations struggle to remain competitive when skills gaps aren't addressed

Organizations identify skills gaps as the key barriers preventing industry transformation:

60% of organizations identify skills gaps as the key barriers preventing business transformation (World Economic Forum, 2023)

43% of respondents agree the business area with the greatest need to address potential skills gaps is data analytics (McKinsey & Company, 2020)

Most organizations are not ready to address potential role disruptions and close skills gaps:

87% of surveyed companies say they currently experience skills gaps or expect them within a few years (McKinsey & Company, 2020)

28% say their organizations make effective decisions on how to close skills gaps (McKinsey & Company, 2020)

Neglecting soft skills development impedes CDOs/CDAOs from delivering value

According to BearingPoint's CDO survey, cultural challenges and limited data literacy are the main roadblocks to a CDO's success. To drill further into the problem and understand the root causes of the two main challenges, conduct a root cause analysis (RCA) using the Five Whys technique.

(Source: BearingPoint, 2020)

Five Whys RCA

Problem: Poor data literacy is the top challenge CDOs face when increasing the value of D&A. Why?

• People that lack data literacy find it difficult to embrace and trust the organization's data insights. Why?
• Data workers and the business team don't speak the same language. Why?
• No shared data definition or knowledge is established. Over-extensive data facts do not drive business outcomes. Why?
• Leaders fail to understand that data literacy is more than technical training, it is about encompassing all aspects of business, IT, and data. Why?
• A lack of leadership skills prevents leaders from recognizing these connections and the data team needing to develop soft skills.

Problem: Cultural challenge is one of the biggest obstacles to a CDO's success. Why?

• Decisions are made from gut instinct instead of data-driven insights, thus affecting business performance. Why?
• People within the organization do not believe that data drives operational excellence, so they resist change. Why?
• Companies overestimate the organization's level of data literacy and data maturity. Why?
• A lack of strategies in change management, continuous improvement & data literacy for data initiatives. Why?
• A lack of expertise/leaders possessing these relevant soft skills (e.g. change management, etc.).

As organizations strive to become more data-driven, most conversations around D&A emphasize hard skills. Soft skills like leadership and change management are equally crucial, and deficits there could be the root cause of the data team's inability to demonstrate improved business performance.

Data cannot be fully leveraged without a cohesive data strategy

Business strategy and data strategy are no longer separate entities.

• For any chief data & analytics officer (CDAO) or equivalent data leader, a robust and comprehensive data strategy is the number one tool for generating measurable business value from data. Data leaders should understand what skills are required to achieve these goals, consider the current skills gap, and build development programs to help employees improve those skills.
• Begin your skills development programs by ensuring you have a data strategy plan prepared. A data strategy should never be formulated independently from the business. Organizations with high data maturity will align such efforts to the needs of the business, making data a major part of the business strategy to achieve data centricity.
• Refer to Info-Tech's Build a Robust and Comprehensive Data Strategy blueprint to ensure data can be leveraged as a strategic asset of the organization.

Info-Tech Insight

The process of achieving data centricity requires alignment between the data and business teams, and that requires soft skills.

Follow Info-Tech's methodology to identify the roles and skills needed to execute a data strategy

1. Define Key Roles and Skills

   Digital Leadership Skills, Soft Skills, Technical Skills

   Key Output

   • Defined essential competencies, responsibilities for some common data roles

2. Uncover the Skills Gap

   Data Strategy Alignment, High-Level Data Maturity Assessment, Skills Gap Analysis

   Key Output

   • Data roles and skills aligned with your current data strategy
   • Identified current and target state of data skill sets

3. Build an Actionable Plan

   Initiative Priority, Skills Growth Feasibility, Hiring Feasibility

   Key Output

   • Identified action plan to address the risk of data skills deficiency

Info-Tech Insight

Skills gaps are a frequently named obstacle to realizing corporate goals for D&A. Soft skills and technical skills are complementary, and a D&A team needs both to perform effectively. Identify the essential skills and the gap with current skills that fit your organization's data strategy to ensure the right skill is available at the right time and to minimize applicable risks.

Research benefits

Member benefits

• Reduce time spent defining the target state of skill sets.
• Gain ability to reassess the feasibility of execution on your data strategy, including resources and timeline.
• Increase confidence in the data leader's ability to implement a successful skills development program that is aligned with the organization's data strategy, which correlates directly to successful business outcomes.

Business benefits

• Reduce time and cost spent hiring key data roles.
• Increase chance of retaining high-quality data professionals.
• Reduce time loss for delayed progress and rework of initiatives.
• Optimize quality of data initiative implementation.
• Improve data team productivity.

Insight summary

Overarching insight

Skills gaps are a frequently named obstacle to realizing corporate goals for D&A. Soft skills and technical skills are complementary, and a D&A team needs both to perform effectively. Identify the essential skills and the gap with current skills that fit your organization's data strategy to ensure the right skill is available at the right time and to minimize applicable risks.

Phase 1 insight

Technological advancements will inevitably require new technical skills, but the most in-demand skills go beyond mastering the newest technologies. Soft skills are essential to data roles as the global workforce navigates the changes of the last few years.

Phase 2 insight

Understanding and knowing your organization's data maturity level is a prerequisite to assessing your current skill and determining where you must align in the future.

Phase 3 insight

One of the misconceptions that organizations have includes viewing skills development as a one-time effort. This leads to underinvestment in data team skills, risk of falling behind on technological changes, and failure to connect with business partners. Employees must learn to continuously adapt to the changing circumstances of D&A.

While the program must be agile and dynamic to reflect technological improvements in the development of technical skills, the program should always be anchored in soft skills because data management is fundamentally about interaction, collaboration, and people.

Tactical insight

Seeking input and support across your business units can align stakeholders to focus on the right data analytics skills and build a data learning culture.

Info-Tech offers various levels of support to best suit your needs

Guided Implementation

A Guided Implementation (GI) is a series of calls with an Info-Tech analyst to help implement our best practices in your organization.

A typical GI is four to six calls over the course of two to three months.

What does a typical GI on this topic look like?

Identify and Build the Data & Analytics Skills Your Organization Needs

Phase 1

Define Key Roles and Skills

This phase will walk you through the following activities:

• 1.1 Review D&A Skill & Role List in Data & Analytics Assessment and Planning Tool

This phase involves the following participants:

• Data leads

Key resources for your data strategy: People

Having the right role is a key component for executing effective data strategy.

D&A Common Roles

• Data Steward
• Data Custodian
• Data Owner
• Data Architect
• Data Modeler
• Artificial Intelligence (AI) and Machine Learning (ML) Specialist
• Database Administrator
• Data Quality Analyst
• Security Architect
• Information Architect
• System Architect
• MDM Administrator
• Data Scientist
• Data Engineer
• Data Pipeline Developer
• Data Integration Architect
• Business Intelligence Architect
• Business Intelligence Analyst
• ML Validator

AI and ML Specialist is projected to be the fastest-growing occupation in the next five years (World Economic Forum, 2023).

While tech roles take an average of 62 days to fill, hiring a senior data scientist takes 70.5 days (Workable, 2019). Start your recruitment cycle early for this demand.

D&A Leader Roles

• Chief Data Officer (CDO)/Chief Data & Analytics Officer (CDAO)
• Data Governance Lead
• Data Management Lead
• Information Security Lead
• Data Quality Lead
• Data Product Manager
• Master Data Manager
• Content and Record Manager
• Data Literacy Manager

CDOs act as impactful change agents ensuring that the organization's data management disciplines are running effectively and meeting the business' data needs. Only 12.0% of the surveyed organizations reported having a CDO as of 2012. By 2022, this percentage had increased to 73.7% (NewVantage Partners, 2022).

Sixty-five percent of respondents said lack of data literacy is the top challenge CDOs face today (BearingPoint, 2020). It has become imperative for companies to consider building a data literacy program which will require a dedicated data literacy team.

Key resources for your data strategy: Skill sets

Distinguish between the three skills categories.

• Soft Skills

  Soft skills are described as power skills regarding how you work, such as teamwork, communication, and critical thinking.

• Digital Leadership Skills

  Not everyone working in the D&A field is expected to perform advanced analytical tasks. To thrive in increasingly data-rich environments, however, every data worker, including leaders, requires a basic technological understanding and skill sets such as AI, data literacy, and data ethics. These are digital leadership skills.

• Technical Skills

  Technical skills are the practical skills required to complete a specific task. For example, data scientists and data engineers require programming skills to handle and manage vast amounts of data.

Info-Tech Insight

Technological advancements will inevitably require new technical skills, but the most in-demand skills go beyond mastering the newest technologies. Soft skills are essential to data roles as the global workforce navigates the changes of the last few years.

Soft skills aren't just nice to have

They're a top asset in today's data workplace.

Leadership

• Data leaders with strong leadership abilities can influence the organization's strategic execution and direction, support data initiatives, and foster data cultures. Organizations that build and develop leadership potential are 4.2 times more likely to financially outperform those that do not (Udemy, 2022).

Business Acumen

• The process of deriving conclusions and insights from data is ultimately utilized to improve business decisions and solve business problems. Possessing business acumen helps provide the business context and perspectives for work within data analytics fields.

Critical Thinking

• Critical thinking allows data leaders at every level to objectively assess a problem before making judgment, consider all perspectives and opinions, and be able to make decisions knowing the ultimate impact on results.

Analytical Thinking

• Analytical thinking remains the most important skill for workers in 2023 (World Economic Forum, 2023). Data analytics expertise relies heavily on analytical thinking, which is the process of breaking information into basic principles to analyze and understand the logic and concepts.

Design Thinking & Empathy

• Design thinking skills help D&A professionals understand and prioritize the end-user experience to better inform results and assist the decision-making process. Organizations with high proficiency in design thinking are twice as likely to be high performing (McLean & Company, 2022).

Learning Focused

• The business and data analytics fields continue to evolve rapidly, and the skills, especially technical skills, must keep pace. Learning-focused D&A professionals continuously learn, expanding their knowledge and enhancing their techniques.

Change Management

• Change management is essential, especially for data leaders who act as change agents developing and enabling processes and who assist others with adjusting to changes with cultural and procedural factors. Organizations with high change management proficiency are 2.2 times more likely to be high performing (McLean & Company, 2022).

Resilience

• Being motivated and adaptable is essential when facing challenges and high-pressure situations. Organizations highly proficient in resilience are 1.8 times more likely to be high performing (McLean & Company, 2022).

Managing Risk & Governance Mindset

• Risk management ability is not limited to highly regulated institutions. All data workers must understand risks from the larger organizational perspective and have a holistic governance mindset while achieving their individual goals and making decisions.

Continuous Improvement

• Continuously collecting feedback and reflecting on it is the foundation of continuous improvement. To uncover and track the lessons learned and treat them as opportunities, data workers must be able to discover patterns and connections.

Teamwork & Collaboration

• Value delivery in a data-centric environment is a team effort, requiring collaboration across the business, IT, and data teams. D&A experts with strong collaborative abilities can successfully work with other teams to achieve shared objectives.

Communication & Active Listening

• This includes communicating with relevant stakeholders about timelines and expectations of data projects and associated technology and challenges, paying attention to data consumers, understanding their requirements and needs, and other areas of interest to the organization.

Technical skills for everyday excellence

Digital Leadership Skills

• Technological Literacy
• Data and AI Literacy
• Cloud Computing Literacy
• Data Ethics
• Data Translation

Data & Analytics Technical Competencies

• Data Mining
• Programming Languages (Python, SQL, R, etc.)
• Data Analysis and Statistics
• Computational and Algorithmic Thinking
• AI/ML Skills (Deep Learning, Computer Vision, Natural Language Processing, etc.)
• Data Visualization and Storytelling
• Data Profiling
• Data Modeling & Design
• Data Pipeline (ETL/ELT) Design & Management
• Database Design & Management
• Data Warehouse/Data Lake Design & Management

1.1 Review D&A Skill & Role List in the Data & Analytics Assessment and Planning Tool

Tab 2. Skill & Role List

Objective: Review the library of skills and roles and customize them as needed to align with your organization's language and specific needs.

Download the Data & Analytics Assessment and Planning Tool

Identify and Build the Data & Analytics Skills Your Organization Needs

Phase 2

Uncover the Skills Gap

This phase will walk you through the following activities:

• 2.1 High-level assessment of your present data management maturity
• 2.2 Interview business and data leaders to clarify current skills availability
• 2.3 Use the Data & Analytics Assessment and Planning Tool to Identify your skills gaps

This phase involves the following participants:

• Data leads
• Business leads and subject matter experts (SMEs)
• Key business stakeholders

Identify skills gaps across the organization

Gaps are not just about assigning people to a role, but whether people have the right skill sets to carry out tasks.

• Now that you have identified the essential skills and roles in the data workplace, move to Phase 2. This phase will help you understand the required level of competency, assess where the organization stands today, and identify gaps to close.
• Using the Data & Analytics Assessment and Planning Tool, start with areas that are given the highest priority through a high-level maturity assessment. From there, three levels of gaps will be found: whether people are assigned to a particular position, the right combination of D&A skill sets, and the right competency level for each skill.

• Lack of talent assigned to a position

• Lack of the right combination of D&A skill sets

• Lack of appropriate competency level

Info-Tech Insight

Understanding your organization's data maturity level is a prerequisite to assessing the skill sets you have today and determining where you need to align in the future.

2.1 High-level assessment of your present data management maturity

Identifying and fixing skills gaps takes time, money, and effort. Focus on bridging the gap in high-priority areas.

Input: Current state capabilities, Use cases (if applicable), Data culture diagnostic survey results (if applicable)
Output: High-level maturity assessment, Prioritized list of data management focused area
Materials: Data Management Assessment and Planning Tool (optional), Data & Analytics Assessment and Planning Tool
Participants: Data leads, Business leads and subject matter experts (SMEs), Key business stakeholders

Objectives:

Prioritize these skills and roles based on your current maturity levels and what you intend to accomplish with your data strategy.

Steps:

1. (Optional Step) Refer to the Build a Robust and Comprehensive Data Strategy blueprint. You can assess your data maturity level using the following frameworks and methods:
   • Review current data strategy and craft use cases that represent high-value areas that must be addressed for their teams or functions.
   • Use the data culture assessment survey to determine your organization's data maturity level.
2. (Optional Step) Refer to the Create a Data Management Roadmap blueprint and Data Management Assessment and Planning Tool to dive deep into understanding and assessing capabilities and maturity levels of your organization's data management enablers and understanding your priority areas and specific gaps.
3. If you have completed Data Management Assessment and Planning Tool, fill out your maturity level scores for each of the data management practices within it - Tab 3 (Current-State Assessment). Skip Tab 4 (High-Level Maturity Assessment).
4. If you have not yet completed Data Management Assessment and Planning Tool, skip Tab 3 and continue with Tab 4. Assign values 1 to 3 for each capability and enabler.
5. You can examine your current-state data maturity from a high level in terms of low/mid/high maturity using either Tabs 3 or 4.
6. Suggested focus areas along the data journey:
   • Low Maturity = Data Strategy, Data Governance, Data Architecture
   • Mid Maturity = Data Literacy, Information Management, BI and Reporting, Data Operations Management, Data Quality Management, Data Security/Risk Management
   • High Maturity = MDM, Data Integration, Data Product and Services, Advanced Analytics (ML & AI Management).

Download the Data & Analytics Assessment and Planning Tool

2.2 Interview business and data leaders to clarify current skills availability

1-2 hours per interview

Input: Sample questions targeting the activities, challenges, and opportunities of each unit
Output: Identified skills availability
Materials: Whiteboard/Flip charts, Data & Analytics Assessment and Planning Tool
Participants: Data leads, Business leads and subject matter experts (SMEs), Key business stakeholders

Instruction:

1. Conduct a deep-dive interview with each key data initiative stakeholder (data owners, SMEs, and relevant IT/Business department leads) who can provide insights on the skill sets of their team members, soliciting feedback from business and data leaders about skills and observations of employees as they perform their daily tasks.
2. Populate a current level of competency for each skill in the Data & Analytics Assessment and Planning Tool in Tabs 5 and 6. Having determined your data maturity level, start with the prioritized data management components (e.g. if your organization sits at low data maturity level, start with identifying relevant positions and skills under data governance, data architecture, and data architecture elements).
3. More detailed instructions on how to utilize the workbook are at the next activity.

Key interview questions that will help you :

1. Do you have personnel assigned to the role? What are their primary activities? Do the personnel possess the soft and technical skills noted in the workbook? Are you satisfied with their performance? How would you evaluate their degree of competency on a scale of "vital, important, nice to have, or none"? The following aspects should be considered when making the evaluation:
   • Key Performance Indicators (KPIs): Business unit data will show where the organization is challenged and will help identify potential areas for development.
   • Project Management Office: Look at successful and failed projects for trends in team traits and competencies.
   • Performance Reviews: Look for common themes where employees excel or need to improve.
   • Focus Groups: Speak with a cross section of employees to understand their challenges.
2. What technology is currently used? Are there requirements for new technology to be bought and/or optimized in the future? Will the workforce need to increase their skill level to carry out these activities with the new technology in place?

Download the Data & Analytics Assessment and Planning Tool

2.3 Use the Data & Analytics Assessment and Planning Tool to identify skills gaps

1-3 hours — Not everyone needs the same skill levels.

Input: Current skills competency, Stakeholder interview results and findings
Output: Gap identification and analysis
Materials: Data & Analytics Assessment and Planning Tool
Participants: Data leads

Instruction:

1. Select your organization's data maturity level in terms of Low/Mid/High in cell A6 for both Tab 5 (Soft Skills Assessment) and Tab 6 (Technical Skills Assessment) to reduce irrelevant rows.
2. Bring together key business stakeholders (data owners, SMEs, and relevant IT custodians) to determine whether the data role exists in the organization. If yes, assign a current-state value from “vital, important, nice to have, or none” for each skill in the assessment tool. Info-Tech has specified the desired/required target state of each skill set.
3. Once you've assigned the current-state values, the tool will automatically determine whether there is a gap in skill set.

Download the Data & Analytics Assessment and Planning Tool

Identify and Build the Data & Analytics Skills Your Organization Needs

Phase 3

Build an Actionable Plan

This phase will walk you through the following activities:

• 3.1 Use the Data & Analytics Assessment and Planning Tool to build your actionable roadmap

This phase involves the following participants:

• Data leads
• Business leads and subject matter experts (SMEs)
• Key business stakeholders

Determine next steps and decision points

There are three types of internal skills development strategies

• There are three types of internal skills development strategies organizations can use to ensure the right people with the right abilities are placed in the right roles: reskill, upskill, and new hire.

1. Reskill

   Reskilling involves learning new skills for a different or newly defined position.

2. Upskill

   Upskilling involves building a higher level of competency in skills to improve the worker's performance in their current role.

3. New hire

   New hire involves hiring workers who have the essential skills to fill the open position.

Info-Tech Insight

One of the misconceptions that organizations have includes viewing skills development as a one-time effort. This leads to underinvestment in data team skills, risk of falling behind on technological changes, and failure to connect with business partners. Employees must learn to continuously adapt to the changing circumstances of D&A. While the program must be agile and dynamic to reflect technological improvements in the development of technical skills, the program should always be anchored in soft skills because data management is fundamentally about interaction, collaboration, and people.

How to determine when to upskill, reskill, or hire to meet your skills needs

Reskill

Reskilling often indicates a change in someone's career path, so this decision requires a goal aligned with both individuals and the organization to establish a mutually beneficial situation.

When making reskilling decisions, organizations should also consider the relevance of the skill for different positions. For example, data administrators and data architects have similar skill sets, so reskilling is appropriate for these employees.

Upskill

Upskilling tends to focus more on the soft skills necessary for more advanced positions. A data strategy lead, for example, might require design thinking training, which enables leaders to think from different perspectives.

Skill growth feasibility must also be considered. Some technical skills, particularly those involving cutting-edge technologies, require continual learning to maintain operational excellence. For example, a data scientist may require AI/ML skills training to incorporate use of modern automation technology.

New Hire

For open positions and skills that are too resource-intensive to reskill or upskill, it makes sense to recruit new employees. Consider, however, time and cost feasibility of hiring. Some positions (e.g. senior data scientist) take longer to fill. To minimize risks, coordinate with your HR department and begin recruiting early.

Data & Analytics skills training

There are various learning methods that help employees develop priority competencies to achieve reskilling or upskilling.

Specific training

The data team can collaborate with the human resources department to plan and develop internal training sessions aimed at specific skill sets.

This can also be accomplished through external training providers such as DCAM, which provides training courses on data management and analytics topics.

Formal education program

Colleges and universities can equip students with data analytics skills through formal education programs such as MBAs and undergraduate or graduate degrees in Data Science, Machine Learning, and other fields.

Certification

Investing time and effort to obtain certifications in the data & analytics field allows data workers to develop skills and gain recognition for continuous learning and self-improvement.

AWS Data Analytics and Tableau Data Scientist Certification are two popular data analytics certifications.

Online learning from general providers

Some companies offer online courses in various subjects. Coursera and DataCamp are two examples of popular providers.

Partner with a vendor

The organization can partner with a vendor who brings skills and talents that are not yet available within the organization. Employees can benefit from the collaboration process by familiarizing themselves with the project and enhancing their own skills.

Support from within your business

The data team can engage with other departments that have previously done skills development programs, such as Finance and Change & Communications, who may have relevant resources to help you improve your business acumen and change management skills.

Info-Tech Insight

Seeking input and support across your business units can align stakeholders to focus on the right data analytics skills and build a data learning culture.

Data & Analytics skills reinforcement

Don't assume learners will immediately comprehend new knowledge. Use different methods and approaches to reinforce their development.

Innovation Space

• Skills development is not a one-time event, but a continuous process during which innovation should be encouraged. A key aspect of being innovative is having a “fail fast” mentality, which means collecting feedback, recognizing when something isn't working, encouraging experimentation, and taking a different approach with the goal of achieving operational excellence.
• Human-centered design (HCD) also yields innovative outcomes with a people-first focus. When creating skills development programs for various target groups, organizations should integrate a human-centered approach.

Commercial Lens

• Exposing people to a commercial way of thinking can add long-term value by educating people to act in the business' best interest and raising awareness of what other business functions contribute. This includes concepts such as project management, return on investment (ROI), budget alignment, etc.

Checklists/Rubrics

• Employees should record what they learn so they can take the time to reflect. A checklist is an effective technique for establishing objectives, allowing measurement of skills development and progress.

Buddy Program

• A buddy program helps employees gain and reinforce knowledge and skills they have learned through mutual support and information exchange.

Align HR programs to support skills integration and talent recruitment

With a clear idea of skills needs and an executable strategy for training and reinforcing of concepts, HR programs and processes can help the data team foster a learning environment and establish a recruitment plan. The links below will direct you to blueprints produced by McLean & Company, a division of Info-Tech Research Group.

Workforce Planning

When integrating the skills of the future into workforce planning, determine the best approach for addressing the identified talent gaps – whether to build, buy, or borrow.

Integrate the future skills identified into the organization's workforce plan.

Talent Acquisition

In cases where employee development is not feasible, the organization's talent acquisition strategy must focus more on buying or borrowing talent. This will impact the TA process. For example, sourcing and screening must be updated to reflect new approaches and skills.

If you have a talent acquisition strategy, assess how to integrate the new roles/skills into recruiting.

Competencies/Succession Planning

Review current organizational core competencies to determine if they need to be modified. New skills will help inform critical roles and competencies required in succession talent pools.

If no competency framework exists, use McLean & Company's Develop a Comprehensive Competency Framework blueprint.

Compensation

Evaluate modified and new roles against the organization's compensation structure. Adjust them as necessary. Look at market data to understand compensation for new roles and skills.

Reassess your base pay structure according to market data for new roles and skills.

Learning and Development

L&D plays a huge role in closing the skills gap. Build L&D opportunities to support development of new skills in employees.

Design an Impactful Employee Development Program to build the skills employees need in the future.

3.1 Use the Data & Analytics Assessment and Planning Tool to build an actionable plan

1-3 hours

Input: Roles and skills required, Key decision points
Output: Actionable plan
Materials: Data & Analytics Assessment and Planning Tool
Participants: Data leads, Business leads and subject matter experts (SMEs), Key business stakeholders

Instruction:

1. On Tab 7 (Next Steps & Decision Points), you will find a list of tasks that correspond to roles that where there is a skills gap.
2. Customize this list of tasks initiatives according to your needs.
3. The Gantt chart, which will be generated automatically after assigning start and finish dates for each activity, can be used to structure your plan and guarantee that all the main components of skills development are addressed.

Download the Data & Analytics Assessment and Planning Tool

Related Info-Tech Research

Create a Data Management Roadmap

• This blueprint will help you design a data management practice that will allow your organization to use data as a strategic enabler.

Build a Robust and Comprehensive Data Strategy

• Put a strategy in place to ensure data is available, accessible, well-integrated, secured, of acceptable quality, and suitably visualized to fuel organization-wide decision making. Start treating data as strategic and corporate asset.

Foster Data-Driven Culture With Data Literacy

• By thoughtfully designing a data literacy training program appropriate to the audience's experience, maturity level, and learning style, organizations build a data-driven and engaged culture that helps them unlock their data's full potential and outperform other organizations.

Research Authors and Contributors

Authors:

Contributors:

Bibliography

2022 Workplace Learning Trends Report.” Udemy, 2022. Accessed 20 June 2023.

Agrawal, Sapana, et al. “Beyond hiring: How companies are reskilling to address talent gaps.” McKinsey & Company, 12 Feb. 2020. Accessed 20 June 2023.

Bika, Nikoletta. “Key hiring metrics: Useful benchmarks for tech roles.” Workable, 2019. Accessed 20 June 2023.

Chroust, Tomas. “Chief Data Officer – Leaders of data-driven enterprises.” BearingPoint, 2020. Accessed 20 June 2023.

“Data and AI Leadership Executive Survey 2022.” NewVantage Partners, Jan 2022. Accessed 20 June 2023.

Dondi, Marco, et al. “Defining the skills citizens will need in the future world of work.” McKinsey & Company, June 2021. Accessed 20 June 2023.

Futschek, Gerald. “Algorithmic Thinking: The Key for Understanding Computer Science.” Lecture Notes in Computer Science, vol. 4226, 2006.

Howard, William, et al. “2022 HR Trends Report.” McLean & Company, 2022. Accessed 20 June 2023.

“Future of Jobs Report 2023.” World Economic Forum, May 2023. Accessed 20 June 2023.

Knight, Michelle. “What is Data Ethics?” Dataversity, 19 May 2021. Accessed 20 June 2023.

Little, Jim, et al. “The CIO Imperative: Is your technology moving fast enough to realize your ambitions?” EY, 22 Apr. 2022. Accessed 20 June 2023.

“MDM Roles and Responsibilities.” Profisee, April 2019. Accessed 20 June 2023.

“Reskilling and Upskilling: A Strategic Response to Changing Skill Demands.” TalentGuard, Oct. 2019. Accessed 20 June 2023.

Southekal, Prashanth. “The Five C's: Soft Skills That Every Data Analytics Professional Should Have.” Forbes, 17 Oct. 2022. Accessed 20 June 2023.

Build a Value Measurement Framework

Focus product delivery on business value–driven outcomes.

ANALYST PERSPECTIVE

"A meaningful measurable definition of value is the key to effectively managing the intake, prioritization, and delivery of technology-enabled products and services."

Cole Cioran,

Senior Director, Research – Application Development and Portfolio Management

Info-Tech Research Group

Our understanding of the problem

This Research Is Designed For:

• CIOs who need to understand the value IT creates
• Application leaders who need to make good decisions on what work to prioritize and deliver
• Application and project portfolio managers who need to ensure the portfolio creates business value
• Product owners who are accountable for delivering value

This Research Will Help You:

• Define quality in your organization’s context from both business and IT perspectives.
• Define a repeatable process to understand the value of a product, application, project, initiative, or enhancement.
• Define value sources and metrics.
• Create a tool to make it easier to balance different sources of value.

This Research Will Also Assist:

• Product and application delivery teams who want to make better decisions about what they deliver
• Business analysts who need to make better decisions about how to prioritize their requirements

This Research Will Help Them:

• Create a meaningful relationship with business partners around what creates value for the organization.
• Enable better understanding of your customers and their needs.

Executive summary

Situation

• Measuring the business value provided by IT is critical for improving the relationship between business and IT.
• Rapid changes in today’s market require rapid, value-based decisions.
• Every organization has unique drivers that make it difficult to see the benefits based on time and impact approaches to prioritization.

Complication

• An organization’s lack of a shared definition of value leads to politics and decision making that does not have a firm, quantitative basis.
• Different parts of an organization have different value drivers that must be given balanced consideration.
• Focusing solely on revenue does not necessarily result in the right outcomes.

Resolution

• Standardize your definition of business value. Work with your business partners to define the different sources of business value that are created through technology-enabled products and services.
• Weigh your value drivers. Ensure business and IT understand the relative weight and priority of the different sources of business value you have identified.
• Use a balanced scorecard to understand value. Use the different value drivers to understand and prioritize different products, applications, projects, initiatives, and enhancements.

Info-Tech Insight

1. Business is the authority on business value. While IT can identify some sources of value, business stakeholders must participate in the creation of a definition that is meaningful to the whole organization.
2. It’s about more than profit. Organizations must have a definition that encompasses all of the sources of value, or they risk making short-term decisions with long-term negative impacts.
3. Technology creates business value. Treating IT as a cost center makes for short-sighted decisions in a world where every business process is enabled by technology.

Software is not currently creating the right outcomes

Software products are taking more and more out of IT budgets.

38% of spend on IT employees goes to software roles.

Source: Info-Tech’s Staffing Survey

18% of opex is spent on software licenses.

Source: SoftwareReviews.com

33% of capex is spent on new software.

However, the reception and value of software products do not justify the money invested.

Only 34% of software is rated as both important and effective by users.

Source: Info-Tech’s CIO Business Vision

IT benchmarks do not help or matter to the business. Focus on the metrics that represent business outcomes.

IT departments have a tendency to measure only their own role-based activities and deliverables, which only prove useful for selling practice improvement services. Technology doesn’t exist for technology's sake. It’s in place to generate specific outcomes. IT and the business need to be aligned toward a common goal of enabling business outcomes, and that’s the important measurement.

"In today’s connected world, IT and business must not speak different languages. "

– Cognizant, 2017

CxOs stress the importance of value as the most critical area for IT to improve reporting

N=469 CxOs from Info-Tech’s CEO/CIO Alignment Diagnostic

Key stakeholders want to know how you and your products or services help them realize their goals.

While the basics of value are clear, few take the time to reach a common definition and means to measure and apply value

Often, IT misses the opportunity to become a strategic partner because it doesn’t understand how to communicate and measure its value to the business.

"Price is what you pay. Value is what you get."

– Warren Buffett

Being able to understand the value context will allow IT to articulate where IT spend supports business value and how it enables business goal achievement.

Value is...

Derived from business context

Enabled through governance and strategy

The underlying context for decision making

A measure of achievement

Determine your business context by assessing the goals and defining the unique value drivers in your organization

Competent organizations know that value cannot always be represented by revenue or reduced expenses. However, it is not always apparent how to envision the full spectrum of sources of value. Dissecting value by the benefit type and the value source’s orientation allows you to see the many ways in which a product or service brings value to the organization.

Financial Benefits vs. Improved Capabilities

Inward vs. Outward Orientation

Inward refers to value sources that have an internal impact and improve your organization’s effectiveness and efficiency in performing its operations.Outward refers to value sources that come from your interaction with external factors, such as the market or your customers.

See your strategy through by involving both IT and the business

Buy-in for your IT strategy comes from the ability to showcase value. IT needs to ensure it has an aligned understanding of what is valuable to the organization.

Business value needs to first be established by the business. After that, IT can build a partnership with the business to determine what that value means in the context of IT products and services.

Measure your product or services with Info-Tech’s Value Measurement Framework (VMF) and value scores

The VMF provides a consistent and less subjective approach to generating a value score for an application, product, service, or individual feature, by using business-defined value drivers and product-specific value metrics.

A consistent set of established value drivers, sources, and metrics gives more accurate comparisons of relative value

A balanced and weighted scorecard allows you to measure the various ways products generate value to the business

The Info-Tech approach to measuring value applies the balanced value scorecard approach.

Value Score1 + VS2 + … + VSN = Overall Balance Value Score

Value scores help support decisions. This blueprint looks specifically at four use cases for value scores.

A value score is an input to the following activities:

1. Prioritize Your Product Backlog

Estimate the relative value of different product backlog items (i.e. epics, features, etc.) to ensure the highest value items are completed first.

This blueprint can be used as an input into Info-Tech’s Build a Better Backlog.

2. Prioritize Your Project Backlog

Estimate the relative value of proposed new applications or major changes or enhancements to existing applications to ensure the right projects are selected and completed first.

This blueprint can be used as an input into Info-Tech’s Optimize Project Intake, Approval, and Prioritization.

3. Rationalize Your Applications

Gauge the relative value from the current use of your applications to support strategic decision making such as retirement, consolidation, and further investments.

This blueprint can be used as an input into Info-Tech’s Visualize Your Application Portfolio Strategy With a Business Value-Driven Roadmap.

4. Categorize Application Tiers

Gauge the relative value of your existing applications to distinguish your most to least important systems and build tailored support structures that limit the downtime of key value sources.

This blueprint can be used as an input into Info-Tech’s Streamline Application Maintenance.

The priorities, metrics, and a common understanding of value in your VMF carry over to many other Info-Tech blueprints

Transition to Product Delivery

Build a Product Roadmap

Modernize Your SDLC

Build a Strong Foundation for Quality

Implement Agile Practices That Work

Use Info-Tech’s Value Calculator

The Value Calculator facilitates the activities surrounding defining and measuring the business value of your products and services.

Use this tool to:

• Weigh the importance of each Value Driver based on established organizational priorities.
• Create a repository for Value Sources to provide consistency throughout each measurement.
• Produce an Overall Balanced Value Score for a specific item.

Info-Tech Deliverable

Populate the Value Calculator as you complete the activities and steps on the following slides.

Limitations of the Value Measurement Framework

"All models are wrong, but some are useful."

– George E.P. Box, 1979

Value is tricky: Value can be intangible, ambiguous, and cause all sorts of confusion, with the multiple, and often conflicting, priorities any organization is sure to have. You won’t likely come to a unified understanding of value or an agreement on whether one thing is more valuable than something else. However, this doesn’t mean you shouldn’t try. The VMF provides a means to organize various priorities in a meaningful way and to assess the relative value of a product or service to guide managers and decision makers on the right track and keep alignment with the rest of the organization.

Relative value vs. ROI: This assessment produces a score to determine the value of a product or service relative to other products or services. Its primary function is to prioritize similar items (projects, epics, requirements, etc.) as opposed to producing a monetary value that can directly justify cost and make the case for a positive ROI.

Apply caution with metrics: We live in a metric-crazed era, where everything is believed to be measurable. While there is little debate over recent advances in data, analytics, and our ability to trace business activity, some goals are still quite intangible, and managers stumble trying to link these goals to a quantifiable data source.

In applying the VMF Info-Tech urges you to remember that metrics are not a magical solution. They should be treated as a tool in your toolbox and are sometimes no more than a rough gauge of performance. Carefully assign metrics to your products and services and do not disregard the informed subjective perspective when SMART metrics are unavailable.

"One of the deadly diseases of management is running a company on visible figures alone."

– William Edwards Deming, 1982

Info-Tech’s Build a Value Measurement Framework glossary of terms

This blueprint discusses value in a variety of ways. Use our glossary of terms to understand our specific focus.

Info-Tech offers various levels of support to best suit your needs

DIY Toolkit

“Our team has already made this critical project a priority, and we have the time and capability, but some guidance along the way would be helpful.”

Guided Implementation

“Our team knows that we need to fix a process, but we need assistance to determine where to focus. Some check-ins along the way would help keep us on track.”

Workshop

“We need to hit the ground running and get this project kicked off immediately. Our team has the ability to take this over once we get a framework and strategy in place.”

Consulting

“Our team does not have the time or the knowledge to take this project on. We need assistance through the entirety of this project.”

Diagnostics and consistent frameworks used throughout all four options

Build a Value Measurement Framework – project overview

Phase 1

Define Your Value Drivers

First determine your value drivers and add them to your VMF

One of the main aspects of the VMF is to apply consistent and business-aligned weights to the products or services you will evaluate.

This is why we establish your value drivers first:

• Get the right executive-level “value authorities” to establish the overarching weights.
• Build these into the backbone of the VMF to consistently apply to all your future measurements.

Step 1.1: Identify Value Authorities

Phase 1

1.1: Identify Value Authorities

1.2: Define Value Drivers

Phase 2

2.1: Identify Product or Service SMEs

2.2: Measure Value

This step will walk you through the following activities:

• Identify your authorities on business value.

This step involves the following participants:

• Owners of your value measurement framework

Outcomes of this step

• Your list of targeted individuals to include in Step 2.1

Business value is best defined and measured by the combined effort and perspective of both IT and the business

Buy-in for your IT strategy comes from the ability to showcase value. IT needs to ensure it has an aligned understanding of what is valuable to the organization. First, priorities need to be established by the business. Second, IT can build a partnership with the business to determine what that value means in the context of IT products and services.

Engage key stakeholders to reach a consensus on organizational priorities and value drivers

Engage these key players to create your value drivers:

CEO: Who better holds the vision or mandate of the organization than its leader? Ideally, they are front and center for this discussion.

CIO: IT must ensure that technical/practical considerations are taken into account when determining value.

CFO: The CFO or designated representative will ensure that estimated costs and benefits can be used to manage the budgets.

VPs: Application delivery and mgmt. is designed to generate value for the business. Senior management from business units must help define what that value is.

Evaluators (PMO, PO, APM, etc.): Those primarily responsible for applying the VMF should be present and active in identifying and carefully defining your organization’s value drivers.

Steering Committee: This established body, responsible for the strategic direction of the organization, is really the primary audience.

Identify your authorities of business value to identify, define, and weigh value drivers

1.1 Estimated Time: 15 minutes

The objective of this exercise is to identify key business stakeholders involved in strategic decision making at an organizational level.

1. Review your organization’s governance structure and any related materials.
2. Identify your key business stakeholders. These individuals are the critical business strategic partners.
1. Target those who represent the business at an organizational level and often comprise the organization’s governing bodies.
2. Prioritize a product backlog – include product owners and product managers who are in tune with the specific value drivers of the product in question.

INFO-TECH TIP

If your organization does not have a formal governance structure, your stakeholders would be the key players in devising business strategy. For example:

• CEO
• CFO
• BRMs
• VPs

Leverage your organizational chart, governing charter, and senior management knowledge to better identify key stakeholders.

INPUT

• Key decision maker roles

OUTPUT

• Targeted individuals to define and weigh value drivers

Materials

• N/A

Participants

• Owner of the value measurement framework

Step 1.2: Define Value Drivers

Phase 1

1.1: Identify Value Authorities

1.2: Define Value Drivers

Phase 2

2.1: Identify Product or Service SMEs

2.2: Measure Value

This step will walk you through the following activities:

• Define your value drivers.
• Weigh your value drivers.

This step involves the following participants:

• Owners of your value measurement framework
• Authorities of business value

Outcomes of this step

• A list of your defined and weighted value drivers

Value is based on business needs and vision

Value is subjective. It is defined through the organization’s past achievement and its future objectives.

There must be a consensus view of what is valuable within the organization, and these values need to be shared across the enterprise. Instead of maintaining siloed views and fighting for priorities, all departments must have the same value and purpose in mind. These factors – purpose and mission, past achievement and current state, vision and future state, and culture and leadership – impact what is valuable to the organization.

Value derives from the mission and vision of an organization; therefore, value is unique to each organization

Business value represents what the business needs to do to achieve its target state. Establishing the mission and vision helps identify that target state.

Understand the many types of value your products or services produce

Competent organizations know that value cannot always be represented by revenue or reduced expenses. However, it is not always apparent how to envision the full spectrum of value sources. Dissecting value by the benefit type and the value source’s orientation allows you to see the many ways in which a product or service brings value to the organization.

Financial Benefits vs. Improved Capabilities

Financial Benefits refers to the degree to which the value source can be measured through monetary metrics and is often quite tangible. Human Benefits refers to how a product or service can deliver value through a user’s experience.

Inward vs. Outward Orientation

Inward refers to value sources that have an internal impact and improve your organization’s effectiveness and efficiency in performing its operations. Outward refers to value sources that come from your interaction with external factors, such as the market or your customers.

Expand past Info-Tech’s high-level value quadrants and identify the value drivers specific to your organization

Different industries have a wide range of value drivers. Consider the difference between public and private entities with respect to generating revenue or reaching their customers or other external stakeholders. Even organizations in the same industry may have different values. For example, a mature, well-established manufacturer may view reputation and innovation as its highest-priority values, whereas a struggling manufacturer will see revenue or market share growth as its main drivers.

You do not need to dissect each quadrant into an exhaustive list of value drivers. Info-Tech recommends defining distinct value drivers only for the areas you’ve identified as critical to your organization’s core goals and objectives.

Understand value drivers that enable revenue growth

Understand value drivers that reduce costs

Understand the value drivers that enhance your services

Understand business value drivers that connect the business to your customers

Define your value drivers

1.2 Estimated Time: 1.5 hours

The objective of this exercise is to establish a common understanding of the different values of the organization.

1. Place your business value authorities at the center of this exercise.
2. Collect all the documents your organization has on the mission and vision, strategy, governance, and target state, which may be defined by enterprise architecture.
3. Identify the company mission and vision. Simply transfer the information from the mission and vision document into the appropriate spaces in the business value statement.
4. Determine the organization’s business value drivers. Use the mission and vision, as well as the information from the collected documents, to formulate your own idea of business values.
5. Use value driver template on the next slide to define the value driver, including:

• Value Driver Name
• Description
• Related Business Capabilities – If available, review business architecture materials, such as business capability maps.
• Established KPI and Targets – If available, include any organization-wide established KPIs related to your value driver. These KPIs will likely be used or influence the metrics eventually assigned to your applications.

INPUT

• Mission, vision, value statements

OUTPUT

• List and description of value drivers

Materials

• Whiteboard
• Markers

Participants

• Business value authorities
• Owner of value measurement framework

Example Value Driver

Weigh your value drivers

1.3 Estimated Time: 30 minutes

The objective of this exercise is to prioritize your value drivers based on their relative importance to the business.

1. Again, place the business value authorities at the center of this exercise.
2. In order to determine priority, divide 100% among your value drivers, allocating a percentage to each based on its relative importance to the organization.
3. Normalize those percentages on to a scale of 1 to 10, which will act as the weights for your value drivers.

INPUT

• Mission, vision, value statements

OUTPUT

• Weights for value drivers

Materials

• Whiteboard
• Markers

Participants

• Business value authorities
• Owner of value measurement framework

Weigh your value drivers

1.3 Estimated Time: 30 minutes

Carry results over to the Value Calculator

1.3

Document results of this activity in the “Value Drivers” tab of the Value Calculator.

List your value drivers.

Define or describe your value drivers.

Use this tool to create a repository for value sources to reuse and maintain consistency across your measurements.

Enter the weight of each value driver in terms of importance to the organization.

Phase 2

Measure Value

Step 2.1: Identify Product or Service SMEs

Phase 1

1.1: Identify Value Authorities

1.2: Define Value Drivers

Phase 2

2.1: Identify Product or Service SMEs

2.2: Measure Value

This step will walk you through the following activities:

• Identify your product or service SMEs.
• List your product or services items and components.

This step involves the following participants:

• Owners of your value measurement framework
• Product or service SMEs

Outcomes of this step

• Your list of targeted individuals to include in Step 2.2

Identify the products and services you are evaluating and break down their various components for the VMF

In order to get a full evaluation of a product or service you need to understand its multiple facets, functions, features capabilities, requirements, or any language you use to describe its various components.

Decompose a product or service:

• Get the right subject matter experts in place who know the business and technical aspects of the product or service.
• Decompose the product or service to capture all necessary components.

Before beginning, consider how your use case will impact your value measurement approach

This table looks at how the different use cases of the VMF call for variations of this analysis, is directed at different roles, and relies on participation from different subject matter experts to provide business context.

Identify your product or service SMEs

2.1 Estimated Time: 15 minutes

The objective of this exercise is to identify specific business stakeholders who can speak to the business outcomes of your applications at a functional level.

1. Review your related materials that reference the stakeholders for the scoped products and services (i.e. capability maps, org charts, stakeholder maps).
2. Identify your specific business stakeholders and application SMEs. These individuals represent the business at a functional level and are in tune with the business outcomes of their operations and the applications that support their operations.
1. Use Case 1 – Product Owner, Product Manager
2. Use Case 2 – Project Portfolio Manager, Project Manager, Product Owners, Business Process Owners, Appropriate Business Unit Representatives
3. Use Case 3 – Application Portfolio Manager, Product Owners, Business Analysts, Application SMEs, Business Process Owners, Appropriate Business Unit Representatives
4. Use Case 4 – Application Maintenance Manager, Operations Managers, Application Portfolio Manager, Product Owners, Application SMEs, Business Process Owners, Appropriate Business Unit Representatives

INPUT

• Specific product or service knowledge

OUTPUT

• Targeted individuals to measure specific products or services

Materials

• Whiteboard
• Markers

Participants

• Owner of value measurement framework

Use Case 1: Collect and review all of the product backlog items

Prioritizing your product backlog (epics, features, etc.) requires a consistent method of measuring the value of your product backlog items (PBIs) to continuously compare their value relative to one another. This should be treated as an ongoing initiative as new items are added and existing items change, but an initial introduction of the VMF will require you to collect and analyze all of the items in your backlog.

Regardless of producing a value score for an epic, feature, or user story, your focus should be on identifying their various value sources. Review your product’s artifact documentation, toolsets, or other information sources to extract the business outcomes, impact, benefits, KPIs, or any other description of a value source.

What’s in your backlog?

You may need to create standards for defining and measuring your different PBIs. Traceability can be critical here, as defined business outcomes for features or user stories may be documented at an epic level.

Additional Research

Build a Better Backlog helps you define and organize your product backlog items.

Use Case 2: Review the scope and requirements of the project to determine all of the business outcomes

Depending on where your project is in your intake process, there should be some degree of stated business outcomes or benefits. This may be a less refined description in the form of a project request or business case document, or it could be more defined in a project charter, business requirements document/toolset, or work breakdown structure (WBS). Regardless of the information source, to make proper use of the VMF you need a clear understanding of the various business outcomes to establish the new or improved value sources for the proposed project.

Set Metrics Early

Good project intake documentation begins the discussion of KPIs early on. This alerts teams to the intended value and gives your PMO the ability to integrate it into the workload of other proposed or approved projects.

Additional Research

Optimize Project Intake, Approval, and Prioritization provides templates to define proposed project benefits and outcomes.

Use Cases 3 & 4: Ensure you’ve listed all of each application’s uses (functions, features, capabilities, etc.) and user groups

An application can enable multiple capabilities, perform a variety of functions, and have a range of different user groups. Therefore, a single application can produce multiple value sources, which range in type, impact, and significance to the business’ overarching priorities. In order to effectively measure the overall value of an application you need to determine all of the ways in which that application is used and apply a business-downward view of your applications.

Business Capability

• Sub-capability
• Process
• Task

Application

• Module
• Feature
• Function

Aim for Business Use

Simply listing the business capabilities of an app can be too high level. Regardless of your organization’s terminology, you need to establish all of the different uses and users of an application to properly measure all of the facets of its value.

Additional Research

Discover Your Applications helps you identify and define the business use and features of your applications.

List your product or services items and components

2.2 Estimated Time: 15 minutes

The objective of this exercise is to produce a list of the different items that you are scoring and ensure you have considered all relevant components.

1. List each item you intend to produce a value score for:
1. Use Case 1 – This may be the epics in your product backlog.
2. Use Case 2 – This may be the projects in your project backlog.
3. Use Cases 3 & 4 – This may be the applications in your portfolio. For this approach Info-Tech strongly recommends iteratively assessing the portfolio to produce a list of a subset of applications.
2. For each item list its various components:
1. Use Case 1 – This may be the features or user stories of an epic.
2. Use Case 2 – This may be the business requirements of a project.
3. Use Cases 3 & 4 – This may be the modules, features, functions, capabilities, or subsystems of an application.

INPUT

• Product or service knowledge

OUTPUT

• Detailed list of items and components

Materials

• Whiteboard
• Markers

Participants

• Owner of value measurement framework
• Product or service SMEs

Use Cases 3 & 4: Create a functional view of your applications (optional)

2.3 Estimated Time: 1 hour

The objective of this exercise is to establish the different use cases of an application.

1. Recall the functional requirements and business capabilities for your applications.
2. List the various actors who will be interacting with your applications and list the consumers who will be receiving the information from the applications.
3. Based on your functional requirements, list the use cases that the actors will perform to deliver the necessary information to consumers. Each use case serves as a core function of the application. See the diagram below for an example.
4. Sometimes several use cases are completed before information is sent to consumers. Use arrows to demonstrate the flow of information from one use case to another.

Example: Ordering Products Online

INPUT

• Product or service knowledge

OUTPUT

• Product or service function

Materials

• Whiteboard
• Markers

Participants

• Application architect
• Enterprise architect
• Business and IT stakeholders
• Business analyst
• Development teams

Use Cases 3 & 4: Create a functional view of your applications (optional) (cont’d.)

2.3 Estimated Time: 1 hour

5. Align your application’s use cases to the appropriate business capabilities and stakeholder objectives.

Example:

Step 2.2: Measure Value

Phase 1

1.1: Identify Value Authorities

1.2: Define Value Drivers

Phase 2

2.1: Identify Product or Service SMEs

2.2: Measure Value

This step will walk you through the following activities:

• Identify your value sources.
• Align to a value driver.
• Assign metrics and gauge value fulfillment.

This step involves the following participants:

• Owners of your value measurement framework
• Product or service SMEs

Outcomes of this step

• An initial list of reusable value sources and metrics
• Value scores for your products or services

Use your VMF and a repeatable process to produce value scores for all of your items

With your products or services broken down, you can then determine a list of value sources, as well as their alignment to a value driver and a gauge of their value fulfillment, which in turn indicate the importance and impact of a value source respectively.

Lastly, we produce a value score for all items:

• Determine business outcomes and value sources.
• Align to the appropriate value driver.
• Use metrics as the gauge of value fulfillment.
• Collect your score.
• Repeat.

The business outcome is the impact the product or service has on the intended business activity

Business outcomes are the business-oriented results produced by organization’s capabilities and the applications that support those capabilities. The value source is, in essence, “How does the application impact the outcome?” and this can be either qualitative or quantitative.

Adapted from Agile Coach Journal.

Measure value – Identify your value sources

2.4 Estimated Time: 30 minutes

The objective of this exercise is to establish the different value sources of a product or service.

1. List the items you are producing an overall balance value score for. These can be products, services, projects, applications, product backlog items, epics, etc.
2. For each item, list its various business outcomes in the form of a description that includes:
1. The item being measured
2. Business capability or activity
3. How the item impacts said capability or activity

Consider applying the user story format for future value sources or a variation for current value sources.

As a (user), I want to (activity) so that I get (impact)

INPUT

• Product or service knowledge
• Business process knowledge

OUTPUT

• List of value sources

Materials

• Whiteboard
• Markers

Participants

• Owner of value measurement framework
• Product or service SMEs

Measure value – Align to a value driver

2.5 Estimated Time: 30 minutes

The objective of this exercise is to determine the value driver for each value source.

1. Align each value source to a value driver. Choose between options A and B.
1. Using a whiteboard, draw out a 2 x 2 business value matrix or an adapted version based on your own organizational value drivers. Place each value source in the appropriate quadrant.
1. Increase Revenue
2. Reduce Costs
3. Enhance Services
4. Reach Customers
2. Using a whiteboard or large sticky pads, create a section for each value driver. Place each value source with the appropriate value driver.

INPUT

• Product or service knowledge
• Business process knowledge

OUTPUT

• Value driver weight

Materials

• Whiteboard
• Markers

Participants

• Owner of value measurement framework
• Product or service SMEs

Brainstorm the different sources of business value (cont’d.)

2.5

Example:

Carry results over to the Value Calculator

2.5

Document results of this activity in the Value Calculator in the Item {#} tab.

List your Value Sources

Your Value Driver weights will auto-populate

Aim, but do not reach, for SMART metrics

Creating meaningful metrics

S pecific

M easureable

A chievable

R ealisitic

T ime-based

Follow the SMART framework when adding metrics to the VMF.

The intention of SMART goals and metrics is to make sure you have chosen a gauge that will:

• Reflect the actual business outcome or value source you are measuring.
• Ensure all relevant stakeholders understand the goals or value you are driving towards.
• Ensure you actually have the means to capture the performance.

Info-Tech Insight

Metrics are NOT a magical solution. They should be treated as a tool in your toolbox and are sometimes no more than a rough gauge of performance. Carefully assign metrics to your products and services and do not disregard the informed subjective perspective when SMART metrics are unavailable.

Info-Tech Best Practice

One last critical consideration here is the degree of effort required to collect the metric compared to the value of the analysis you are performing. Assessing whether or not to invest in a project should apply the rigor of carefully selecting and measuring value. However, performing a rationalization of the full app portfolio will likely lead to analysis paralysis. Taking an informed subjective perspective may be the better route.

Measure value – Assign metrics and gauge value fulfillment

2.6 30-60 minutes

The objective of this exercise is to determine an appropriate metric for each value source.

1. For each value source assign a metric that will be the unit of measurement to gauge the value fulfilment of the application.
2. Review the product or services performance with the metric
1. Use case 1&2 (Proposed Applications and/or Features) - You will need to estimate the degree of impact the product or services will have on your selected metric.
2. Use case 3&4 (Existing Applications and/or Features) – You can review historically how the product or service has performed with your selected metric
3. Determine a value fulfillment on a scale of 1 – 10.

10 = The product or service far exceeds expectations and targets on the metric.

5 = the product or service meets expectations on this metric.

1 = the product or service underperforms on this metric.

INPUT

• Product or service knowledge
• Business process knowledge

OUTPUT

• Value driver weight

Materials

• Whiteboard
• Markers

Participants

• Owner of value measurement framework
• Product or service SMEs

Carry results over to the Value Calculator

2.6

Document results of this activity in the Value Calculator in the Item {#} tab.

Assign Metrics.

Consider using current or estimated performance and targets.

Assess the impact on the value source with the value fulfillment.

Collect your Overall Balanced Value Score

Appendix

Bibliography

Brown, Alex. “Calculating Business Value.” Agile 2014 Orlando – July 13, 2014. Scrum Inc. 2014. Web. 20 Nov. 2017.

Brown, Roger. “Defining Business Value.” Scrum Gathering San Diego 2017. Agile Coach Journal. Web.

Curtis, Bill. “The Business Value of Application Internal Quality.” CAST. 6 April 2009. Web. 20 Nov. 2017.

Fleet, Neville, Joan Lasselle, and Paul Zimmerman. “Using a Balance Scorecard to Measure the Productivity and Value of Technical Documentation Organizations.” CIDM. April 2008. Web. 20 Nov. 2017.

Harris, Michael. “Measuring the Business Value of IT.” David Consulting Group. 20 Nov. 2017.

Intrafocus. “What is a Balanced Scorecard?” Intrafocus. Web. 20 Nov. 2017

Kerzner, Harold. Project Management: A Systems Approach to Planning, Scheduling, and Controlling. 12th ed., Wiley, 2017.

Lankhorst, Marc., et al. “Architecture-Based IT Valuation.” Via Nova Architectura. 31 March 2010. Web. 20 Nov. 2017.

Rachlin, Sue, and John Marshall. “Value Measuring Methodology.” Federal CIO Council, Best Practices Committee. October 2002. Web. April 2019.

Thiagarajan, Srinivasan. “Bridging the Gap: Enabling IT to Deliver Better Business Outcomes.” Cognizant. July 2017. Web. April 2019.

Take Action on Service Desk Customer Feedback

Drive up CSAT scores by asking the right questions and effectively responding to user feedback.

EXECUTIVE BRIEF

Analyst Perspective

Collecting feedback is only half the equation.

Executive Summary

Info-Tech Insight

Asking your customers for feedback then doing nothing with it is worse than not asking for feedback at all. Your customers may end up more dissatisfied than they were before, if their opinion is sought out and then ignored. It’s valuable to collect feedback, but the true value for both IT and its customers comes from acting on that feedback and communicating those actions back to your users.

Traditional service desk metrics can be misleading

The watermelon effect

When a service desk appears to hit all its targets according to the metrics it tracks, but service delivery is poor and customer satisfaction is low, this is known as the “watermelon effect”. Service metrics appear green on the outside, but under the surface (unmeasured), they’re red because customers are dissatisfied.

Traditional SLAs and service desk metrics (such as time to respond, average resolution time, percentage of SLAs met) can help you understand service desk performance internally to prioritize your work and identify process improvements. However, they don’t tell you how customers perceive the service or how satisfied they are.

Providing good service to your customers should be your end goal. Failing to measure, monitor, and act on customer feedback means you don’t have the whole picture of how your service desk is performing and whether or where improvements are needed to maximize satisfaction.

There is a shift in ITSM to focus more on customer experience metrics over traditional ones

The Service Desk Institute (SDI) suggests that customer satisfaction is the most important indicator of service desk success, and that traditional metrics around SLA targets – currently the most common way to measure service desk performance – may become less valuable or even obsolete in the future as customer experience-focused targets become more popular. (Service Desk Institute, 2021)

SDI conducted a Customer Experience survey of service desk professionals from a range of organizations, both public and private, from January to March 2018. The majority of respondents said that customer experience is more important than other metrics such as speed of service or adherence to SLAs, and that customer satisfaction is more valuable than traditional metrics. (SDI, 2018).

However, many service desk leaders aren’t effectively measuring customer feedback

Not only is it important to measure customer experience and satisfaction levels, but it’s equally important to act on that data and feed it into a service improvement program. However, many IT leaders are neglecting either one or both of those components.

A strong customer feedback program brings many benefits to IT and the business

Take Action on Service Desk Customer Feedback

Info-Tech’s methodology for measuring and acting on service desk customer feedback

Insight Summary

Key Insight:

Asking your customers for feedback then doing nothing with it is worse than not asking for feedback at all. Your customers may end up more dissatisfied than they were before if they’re asked for their opinion then see nothing done with it. It’s valuable to collect feedback, but the true value for both IT and its customers comes from acting on that feedback and communicating those actions back to your users.

Additional insights:

Understand how to measure customer satisfaction

Phase 1

Understand the main types of customer satisfaction surveys, principles for survey design, and best practices for surveying your users.

Three methods of surveying your customers

Pros and cons of the three methods

Which survey method should you choose?

Only relying on one type of survey will leave gaps in your understanding of customer satisfaction. Include both transactional and relationship surveys to provide a holistic picture of customer satisfaction with the service desk.

If you can only start with one type, choose the type that best aligns with your goals and priorities:

Info-Tech Insight

One-off surveys can be useful to assess whether a specific change has impacted satisfaction, or to inform a planned change/initiative. However, as they aren’t typically part of an on-going customer feedback program, the focus of this research will be on transactional and relationship surveys.

3 common customer satisfaction measures

The three most utilized measures of customer satisfaction include CSAT, CES, and NPS.

When to use each satisfaction measure

How to choose which measure(s) to incorporate in your surveys

The best measures are the ones that align with your specific goals for collecting feedback.

• Most companies will use multiple satisfaction measures. For example, NPS can be tracked to monitor the overall customer sentiment, and CSAT used for more targeted feedback.
• For internal-facing IT departments, CSAT is the most popular of the three methods, and NPS may not be as useful.
• Choose your measure and survey types based on what you are trying to achieve and what kind of information you need to make improvements.
• Remember that one measure alone isn’t going to give you actionable feedback; you’ll need to follow up with additional measures (especially for NPS and CES).
• For CSAT surveys, customize the satisfaction measures in as many ways as you need to target the questions toward the areas you’re most interested in.
• Don’t stick to just these three measures or types of surveys – there are other ways to collect feedback. Experiment to find what works for you.
• If you’re designing your own survey, keep in mind the principles on the next slide.

Info-Tech Insight

While we focus mainly on traditional survey-based approaches to measuring customer satisfaction in this blueprint, there’s no need to limit yourselves to surveys as your only method. Consider multiple techniques to capture a wider audience, including:

• Customer journey mapping
• Focus groups with stakeholders
• Lunch and learns or workshop sessions
• Interviews – phone, chat, in-person
• Kiosks

Principles for survey design

As you design your satisfaction survey – whether transactional or relational – follow these guidelines to ensure the survey delivers value and gets responses.

1. Focus on your goal

Don’t include unnecessary questions that won’t give you actionable information; it will only waste respondents’ time.

2. Be brief

Keep each question as short as possible and limit the total number of survey questions to avoid survey fatigue.

3. Include open-ended questions

Most of your measures will be close-ended, but include at least one comment box to allow for qualitative feedback.

4. Keep questions clear and concise

Ensure that question wording is clear and specific so that all respondents interpret it the same way.

5. Avoid biased or leading questions

You won’t get accurate results if your question leads respondents into thinking or answering a certain way.

6. Avoid double-barreled questions

Don’t ask about two different things in the same question – it will confuse respondents and make your data hard to interpret.

7. Don’t restrict responses

Response options should include all possible opinions (including “don’t know”) to avoid frustrating respondents.

8. Make the survey easy to complete

Pre-populate information where possible (e.g. name, department) and ensure the survey is responsive on mobile devices.

9. Keep questions optional

If every question is mandatory, respondents may leave the survey altogether if they can’t or don’t want to answer one question.

10. Test your survey

Test your survey with your target audience before launching, and incorporate feedback - they may catch issues you didn’t notice.

Prevent survey fatigue to increase response rates

If it takes too much time or effort to complete your survey – whether transactional or relational – your respondents won’t bother. Balance your need to collect relevant data with users’ needs for a simple and worthwhile task in order to get the most value out of your surveys.

There are two types of survey fatigue:

1. Survey response fatigue

Occurs when users are overwhelmed by too many requests for feedback and stop responding.

2. Survey taking fatigue

Occurs when the survey is too long or irrelevant to users, so they grow tired and abandon the survey.

Fight survey fatigue:

• Make it as easy as possible to answer your survey:
  • Keep the survey as short as possible.
  • For transactional surveys, allow respondents to answer directly from email without having to click a separate link if possible.
  • Don’t make all questions mandatory or users may abandon it if they get to a difficult or unapplicable question.
  • Test the survey experience across devices for mobile users.
• Communicate the survey’s value so users will be more likely to donate their time.
• Act on feedback: follow up on both positive and negative responses so users see the value in responding.
• Consider attaching an incentive to responding (e.g. name entered in a monthly draw).

Design and implement transactional surveys

Phase 2

Learn why and how to design a simple survey to assess satisfaction with individual service desk transactions (tickets) and a methodology for survey delivery that will improve response rates.

Use transactional surveys to collect immediate and actionable feedback

Recall the definition of a transactional survey:

• Survey that is tied to a specific customer interaction with the service desk (i.e. a ticket).
• Assesses how satisfied customers are with how the ticket was handled and resolved.
• Sent immediately after ticket is closed.
• Short – usually 1 to 3 questions.

Info-Tech Insight

While feedback on transactional surveys is specific to a single transaction, even one negative experience can impact the overall perception of the service desk. Pair your transactional surveys with an annual relationship survey to capture broader sentiment toward the service desk.

Transactional surveys serve several purposes:

• Gives end users a mechanism to provide feedback when they want to.
• Provides continual insight into customer satisfaction throughout the year to monitor for trends or issues in between broader surveys.
• Provides IT leaders with actionable insights into areas for improvement in their processes, knowledge and skills, or customer service.
• Gives the service desk the opportunity to address any negative experiences or perceptions with customers, to repair the relationship.
• Feeds into individual or team coaching for service desk staff.

Make key decisions ahead of launching your transactional surveys

If you want to get the most of your surveys, you need to do more than just click a button to enable out-of-the-box surveys through your ITSM tool. Make these decisions ahead of time:

Key decisions, continued

Define the goals of your transactional survey program

Every survey should have a goal in mind to ensure only relevant and useful data is collected.

• Your survey program must be backed by clear and actionable goals that will inform all decisions about the survey.
• Survey questions should be structured around that goal, with every question serving a distinct purpose.
• If you don’t have a clear plan for how you will action the data from a particular question, exclude it.
• Don’t run a survey just for the sake of it; wait until you have a clear plan. If customers respond and then see nothing is done with the data, they will learn to avoid your surveys.

Your survey objectives will also determine how often to send the survey:

Info-Tech Insight

Service Managers often get caught up in running a transactional survey program because they think it’s standard practice, or they need to report a satisfaction metric. If that’s your only objective, you will fail to derive value from the data and will only turn customers away from responding.

Design survey content and length

As you design your survey, keep in mind the following principles:

1. Keep it short. Your customers won’t bother responding if they see a survey with multiple questions or long questions that require a lot of reading, effort, or time.
2. Make it simple. This not only makes it easier for your customers to complete, but easier for you to track and monitor.
3. Tie your survey to your goals. Remember that every question should have a clear and actionable purpose.
4. Don’t measure anything you can’t control. If you won’t be able to make changes based on the feedback, there’s no value asking about it.
5. Include an (optional) open-ended question. This will allow customers to provide more detailed feedback or suggestions.

Q: How many questions should the survey contain?

A: Ideally, your survey will have only one mandatory question that captures overall satisfaction with the interaction.

This question can be followed up with an optional open-ended question prompting the respondent for more details. This will provide a lot more context to the overall rating.

If there are additional questions you need to ask based on your goals, clearly make these questions optional so they don’t deter respondents from completing the survey. For example, they can appear only after the respondent has submitted their overall satisfaction response (i.e. on a separate, thank you page).

Additional (optional) measures may include:

• Customer effort score (how easy or difficult was it to get your issue resolved?)
• Customer service skills of the service desk
• Technical skills/knowledge of the agents
• Speed or response or resolution

Design question wording

Tips for writing survey questions:

• Be clear and concise
• Keep questions as short as possible
• Cut out any unnecessary words or phrasing
• Avoid biasing, or leading respondents to select a certain answer
• Don’t attempt to measure multiple constructs in a single question.

Sample question wording:

How satisfied are you with this support experience?

How would you rate your support experience?

Please rate your overall satisfaction with the way your issue was handled.

Choose response options

Once you’ve written your survey question, you need to design the response options for the question. Put careful thought into balancing ease of responding for the user with what will give you the actionable data you need to meet your goals. Keep the following in mind:

Decide how often to send surveys

There are two common choices for when to send ticket satisfaction surveys:

SDI’s 2018 Customer Experience in ITSM survey of service desk professionals found:

Almost two-thirds (65%) send surveys after every ticket.

One-third (33%) send surveys after randomly selected tickets are closed.

Info-Tech Recommendation:

Send a survey after every ticket so that anyone who has feedback gets the opportunity to provide it – and you always get the chance to act on negative feedback. But, limit how often any one customer receives a ticket to avoid over-surveying them – restrict to anywhere between one survey a week to one per month per customer.

Plan detailed survey logistics

Plan detailed survey logistics

Plan detailed survey logistics

Plan detailed survey logistics

Response rates for transactional surveys are typically low…

Most IT organizations see transactional survey response rates of less than 20%.

… but there are steps you can take to maximize your response rate

It is still difficult to achieve high response rates to transactional surveys, but you can at least increase your response rate with these strategies:

1. Reduce frequency

Don’t over-survey any one user or they will start to ignore the surveys.

2. Send immediately

Ask for feedback soon after the ticket was resolved so it’s fresh in the user’s memory.

3. Make it short and simple

Keep the survey short, concise, and simple to respond to.

4. Make it easy to complete

Minimize effort involved as much as possible. Allow users to respond directly from email and from any device.

5. Change email messaging

Experiment with your subject line or email messaging to draw more attention.

6. Respond to feedback

Respond to customers who provide feedback – especially negative – so they know you’re listening.

7. Act on feedback

Demonstrate that you are acting on feedback so users see the value in responding.

Use Info-Tech’s survey template as a starting point

Once you’ve worked through all the decisions in this step, you’re ready to configure your transactional survey in your ITSM solution or survey tool.

As a starting point, you can leverage Info-Tech’s Transactional Service Desk Survey Templatee to design your templates and wording.

Make adjustments to match your decisions or your configuration limitations as needed.

Refer to the key decisions tables on slides 24 and 25 to ensure you’ve made all the configurations necessary as you set up your survey.

Design and implement relationship surveys

Phase 3

Understand why and how to design a survey to assess overall satisfaction with the service desk across your organization, or use Info-Tech’s diagnostic.

How can we evaluate overall Service Desk service quality?

Evaluating service quality in any industry is challenging for both those seeking feedback and those consuming the service: “service quality is more difficult for the consumer to evaluate than goods quality.”

You are in the position of trying to measure something intangible: customer perception, which “result[s] from a comparison of consumer expectations with actual service performance,” which includes both the service outcome and also “the process of service delivery”

(Source: Parasuraman et al, 1985, 42).

Your mission is to design a relationship survey that is:

• Comprehensive but not too long.
• Easy to understand but complex enough to capture enough detail.
• Able to capture satisfaction with both the outcome and the experience of receiving the service.

Use relationship surveys to measure overall service desk service quality

Recall the definition of a relationship survey:

• Survey that is sent periodically (i.e. semi-annually or annually) to the entire customer base to measure the overall relationship with the service desk.
• Shows you where your customer experience is doing well and where it needs improving.
• Asks customers to rate you based on their overall experience rather than on a specific product or interaction.
• Longer and more comprehensive than transactional surveys, covering multiple dimensions/ topics.

Relationship surveys serve several purposes:

• Gives end users an opportunity to provide overall feedback on a wider range of experiences with IT.
• Gives IT the opportunity to respond to feedback and show users their voices are heard.
• Provides insight into year-over-year trends and customer satisfaction.
• Provides IT leaders the opportunity to segment the results by demographic (e.g. by department, location, or seniority) and target improvements where needed most.
• Feeds into strategic planning and annual reports on user experience and satisfaction

Info-Tech Insight

Annual relationship surveys provide great value in the form of year-over-year internal benchmarking data, which you can use to track improvements and validate the impact of your service improvement efforts.

Understand the gaps that decrease service quality

The Service Quality Model (Parasuraman, Zeithaml and Berry, 1985) shows how perceived service quality is negatively impacted by the gap between expectations for quality service and the perceptions of actual service delivery: Gap 1: Consumer expectation – Management perception gap: Are there differences between your assumptions about what users want from a service and what those users expect? Gap 2: Management perception – Service quality specification gap: Do you have challenges translating user expectations for service into standardized processes and guidelines that can meet those expectations? Gap 3: Service quality specifications – Service delivery gap: Do staff members struggle to carry out the service quality processes when delivering service? Gap 4: Service delivery – External communications gap: Have users been led to expect more than you can deliver? Alternatively, are users unaware of how the organization ensures quality service, and therefore unable to appreciate the quality of service they receive? Gap 5: Expected service – Perceived service gap: Is there a discrepancy between users’ expectations and their perception of the service they received (regardless of any user misunderstanding)?

Your survey questions about service and support should provide insight into where these gaps exist in your organization

Make key decisions ahead of launch

Align the relationship survey with your goals

What is motivating you to launch the survey at this time?

Is there a renewed focus on customer service satisfaction? If so, this survey will track the initiative’s success, so its questions must align with the sponsors’ expectations.

Are you surveying customer satisfaction in order to comply with legislation, or directives to measure customer service quality?

What objectives/outcomes will this survey feed into?

What do you need to report on to your stakeholders? Have they communicated any expectations regarding the data they expect to see?

Does the CIO want the annual survey to measure end-user satisfaction with all of IT?

• Or do you only want to measure satisfaction with one set of processes (e.g. Service Desk)?
• Are you seeking feedback on a project (e.g. implementation of new ERP)?
• Are you seeking feedback on the application portfolio?

In 1993 the U.S. president issued an Executive Order requiring executive agencies to “survey customers to determine the kind and quality of services they want and their level of satisfaction with existing services” and “post service standards and measure results against them.” (Clinton, 1993)

Identify what you’re measuring

Info-Tech Insight

As you compose survey questions, decide whether they are intended to capture user satisfaction or effort: this will influence how the question is worded. Include a mix of both.

Determine a framework for your survey

If your relationship survey covers satisfaction with service support, ensure the questions cover the major aspects of service quality. You may wish to align your questions on support with existing frameworks: for example, the SERVQUAL service quality measurement instrument identifies 5 dimensions of service quality: Reliability, Assurance, Tangibles, Empathy, and Responsiveness (see below). As you design the survey, consider if the questions relate to these five dimensions. If you have overlooked any of the dimensions, consider if you need to revise or add questions.

1-5. Arlen, Chris,2022. Paraphrasing Zeithaml, Parasuraman, and Berry, 1990.

Cover logistical details of the survey

Use the Sample Size Calculator to determine your ideal sample size

Info-Tech’s End-User Satisfaction Diagnostics

If you choose to leverage a third-party partner, an Info-Tech satisfaction survey may already be part of your membership. There are two options, depending on your needs:

Design question wording

Choose answer scales that best fit your questions and reporting needs

Likert scale

Respondents select from a range of statements the position with which they most agree:

E.g. How satisfied are you with how long it generally takes to resolve your issue completely?

E.g. Very dissatisfied/Somewhat dissatisfied/ Neutral/ Somewhat satisfied/ Very satisfied/ NA

Frequency scale

How often does the respondent have to do something, or how often do they encounter something?

E.g. How frequently do you need to re-open tickets that have been closed without being satisfactorily resolved?

E.g. Never/ Rarely/ Sometimes/ Often/ Always/ NA

Numeric scale

By asking users to rate their satisfaction on a numeric scale (e.g., 1-5, 1-10), you can facilitate reporting on averages:

E.g. How satisfied are you with IS’s ability to provide services to allow the organization to meet its goals?

E.g. 1 – Not at all Satisfied to 10 – Fully Satisfied / NA

Forced ranking

Learn more about your users’ priorities by asking them to rank answers from most to least important, or selecting their top choices (Sauro, 2018):

E.g. From the following list, drag and drop the 3 aspects of our service that are most important to you into the box on the right.

Info-Tech Insight

Always include an optional open-ended question, which allows customers to provide more feedback or suggestions.

Test the survey before launching

Review your questions for repetition and ask for feedback on your survey draft to discover if readers interpret the questions differently than you intended.

Test the survey with different stakeholder groups:

• IT staff: To discover overlooked topics.
• Representatives of your end-user population: To discover whether they understand the intention of the questions.
• Executives: To validate whether you are capturing the data they are interested in reporting on.

Testing methodology:

• Ask your test subjects to take the survey in your presence so you can monitor their experience as they take it.
• Ask them to narrate their experience as they take the survey.
• Watch for:
  • The time it takes to complete the survey.
  • Moments when they struggle or are uncertain with the survey’s wording.
  • Questions they find repetitive or pointless.

Info-Tech Insight

In the survey testing phase, try to capture at least a few real-time responses to the survey. If you collect survey feedback only once the test is over, you may miss some key insights into the user experience of navigating the survey.

“Follow the golden rule: think of your audience and what they may or may not know. Think about what kinds of outside pressures they may bring to the work you’re giving them. What time constraints do they have?”

– Sally Colwell, Project Officer, Government of Canada Pension Centre

Monitor and maximize your response rate

Ensure success by staying on top of the survey during the period it is open.

• When will your users complete the survey? You know your own organization’s culture best, but SurveyMonkey found that weekday survey responses peaked at mid-morning and mid-afternoon (Wronski). Ensure you send the communication at a time it will not be overlooked. For example, some studies found Mondays to have higher response rates; however, the data is not consistent (Amaresan, 2021). Send the survey at a time you believe your users are least likely to be inundated with other notifications.
• Have a trusted leader send out the first communication informing the end-user base of the survey. Ensure the recipient understands your motivation and how their responses will be used to benefit them (O’Reardon, 2016). Remind them that participating in the survey benefits them: since IT is taking actions based on their feedback, it’s their chance to improve their employee experience of the IT services and tools they use to do their job.
• In the introductory communication, test different email subject lines and email body content to learn which versions increase respondents’ rates of opening the survey link, and “keep it short and clear” (O’Reardon, 2016).
• If your users tend to mistrust emailed links due to security training, tell them how to confirm the legitimacy of the survey.

“[Send] one reminder to those who haven’t completed the survey after a few days. Don’t use the word ‘reminder’ because that’ll go straight in the bin, better to say something like, ‘Another chance to provide your feedback’”

– David O’Reardon, Founder & CEO of Silversix

Analyze and act on feedback

Phase 4

Measure and analyze the results of both surveys and build a plan to act on both positive and negative feedback and communicate the results with the organization.

Leverage the service recovery paradox to improve customer satisfaction

A service failure or a poor experience isn’t what determines customer satisfaction – it’s how you respond to the issue and take steps to fix it that really matters.

This means one poor experience with the service desk doesn’t necessarily lead to an unhappy user; if you quickly and effectively respond to negative feedback to repair the relationship, the customer may be even happier afterwards because you demonstrated that you value them.

“Every complaint becomes an opportunity to turn a bad IT customer experience into a great one.”

– David O’Reardon, Founder & CEO of Silversix

Collecting feedback is only the first step in the customer feedback loop

Closing the feedback loop is one of the most important yet forgotten steps in the process.

1. Collect Feedback

• Send transactional surveys after every ticket is resolved.
• Send a broader annual relationship survey to all users.

• Calculate satisfaction scores.
• Read open-ended comments.
• Analyze for trends, categories, common issues and priorities.

• Respond to users who provided feedback.
• Make improvements based on feedback.

• Communicate feedback results and improvements made to respondents and to service desk staff.
• Summarize results and actions to key stakeholders and business leaders.

Act on feedback to get the true value of your satisfaction program

SDI (2018) survey data shows that the majority of service desk professionals are using their customer satisfaction data to feed into service improvements. However, 30% still aren’t doing anything with the feedback they collect. Collecting feedback is only one half of a good customer feedback program. Acting on that feedback is critical to the success of the program. Using feedback to make improvements not only benefits the service desk but shows users the value of responding and will increase future response rates.

“Your IT service desk’s CSAT survey should be the means of improving your service (and the employee experience), and something that encourages people to provide even more feedback, not just the means for understanding how well it’s doing” – Joe the IT Guy, SysAid

Assign responsibility for acting on feedback

If collecting and analyzing customer feedback is something that happens off the side of your desk, it either won’t get done or won’t get done well.

• Formalize the customer satisfaction program. It’s not a one-time task, but an ongoing initiative that requires significant time and dedication.
• Be clear on who is accountable for the program and who is responsible for all the tasks involved for both transactional and relationship survey data collection, analysis, and communication.

Assign accountability for the customer feedback program to one person (i.e. Service Desk Manager, Service Manager, Infrastructure & Operations Lead, IT Director), who may take on or assign responsibilities such as:

• Designing surveys, including survey questions and response options.
• Configuring survey(s) in ITSM or survey tool.
• Sending relationship surveys and subsequent reminders to the organization.
• Communicating results of both surveys to internal staff, business leaders, and end users.
• Analyzing results.
• Feeding results into improvement plans, coaching, and training.
• Creating reports and dashboards to monitor scores and trends.

Info-Tech Insight

While feedback can feed into internal coaching and training, the goal should never be to place blame or use metrics to punish agents with poor results. The focus should always be on improving the experience for end users.

Determine how and how often to analyze feedback data

• Analyze and report scores from both transactional and relationship surveys to get a more holistic picture of satisfaction across the organization.
• Determine how you will calculate and present satisfaction ratings/scores, both overall and for individual questions. See tips on the right for calculating and presenting NPS and CSAT scores.
• A single satisfaction score doesn’t tell the full story; calculate satisfaction scores at multiple levels to determine where improvements are most needed.
  • For example, satisfaction by service desk tier, team or location, by business department or location, by customer group, etc.
• Analyze survey data regularly to ensure you communicate and act on feedback promptly and avoid further alienating dissatisfied users. Transactional survey feedback should be reviewed at least weekly, but ideally in real time, as resources allow.

Calculating NPS Scores

Categorize respondents into 3 groups:

• 9-10 = Promoters, 7-8 = Neutral, 1-6 = Detractors

Calculate overall NPS score:

• % Promoters - % Detractors

Calculating CSAT Scores

• CSAT is usually presented as a percentage representing the average score.
• To calculate, take the total of all scores, divide by the maximum possible score, then multiply by 100. For example, a satisfaction rating of 80% means on average, users gave a rating of 4/5 or 8/10.
• Note that some organizations present CSAT as the percentage of “satisfied” users, with satisfied being defined as either “yes” on a two-point scale or a score of 4 or 5 on a 5-point scale. Be clear how you are defining your satisfaction rating.

Don’t neglect qualitative feedback

While it may be more difficult and time-consuming to analyze, the reward is also greater in terms of value derived from the data.

Define a process to respond to both negative and positive feedback

Successful customer satisfaction programs respond effectively to both positive and negative outcomes. Late or lack of responses to negative comments may increase customer frustration, while not responding at all to the positive comments may give the perception of indifference.

1. Define what qualifies as a positive vs negative score

E.g. Scores of 1 to 2 out of 5 are negative, scores of 4 to 5 out of 5 are positive.

2. Define process to respond to negative feedback

• Negative responses should go directly to the Service Desk Manager or whoever is accountable for feedback.
• Set an SLO for when the user will be contacted. It should be within 24h but ideally much sooner.
• Investigate the issue to understand exactly what happened and get to the root cause.
• Identify remediation steps to ensure the issue does not occur again.
• Communicate to the customer the action you have taken to improve.

• Positive responses should also be reviewed by the person accountable for feedback, but the timeline to respond may be longer.
• Show respondents that you value their time by thanking them for responding. Showing appreciate helps to build a long-term relationship with the user.
• Share positive results with the team to improve morale, and as a coaching/training mechanism.
• Consider how to use positive feedback as an incentive or reward.

Build a plan to communicate results to various stakeholders

Regular communication about your feedback results and action plan tied to those results is critical to the success of your feedback program. Build your communication plan around these questions:

Communication to your users impacts both response rates and satisfaction

Based on the Customer Communication Cycle by David O’Reardon, 2018

Ask users to provide feedback through transactional and relationship surveys. Thank them for completing the survey – show that you value their time, regardless of the type of feedback they submitted. Be transparent and summarize the results of the survey(s). Make it easy to digest with simple satisfaction scores and a summary of the main insights or priorities revealed. Before asking for feedback, explain how you will use feedback to improve the service. After collecting feedback, share your plan for making improvements based on what the data told you. After you’ve made changes, communicate again to share the results with respondents. Make it clear that their feedback had a direct result on the service they receive. Communicating this before running another survey will also increase the likelihood of respondents providing feedback again.

Info-Tech Insight

Focus your communications to users around them, not you. Demonstrate that you need feedback to improve their experience, not just for you to collect data.

Translate feedback into actionable improvements

Taking action on feedback is arguably the most important step of the whole customer feedback program.

“Without converting feedback into actions, surveys can become just a pointless exercise in number watching.”

– David O’Reardon, Founder & CEO of Silversix

Info-Tech Insight

Outline exactly what you plan to do to address customer feedback in an action plan, and regularly review that action plan to select and prioritize initiatives and monitor progress.

For more guidance on tracking and prioritizing ongoing improvement initiatives, see the blueprints Optimize the Service Desk with a Shift Left Strategy and Build a Continual Improvement Plan for the Service Desk.

Leverage Info-Tech resources to guide your improvement efforts

Map your identified improvements to the relevant resource that can help:

Map process for acting on relationship survey feedback

Use Info-Tech’s Relationship Satisfaction Survey Review Process workflow as a template to define your own process.

Map process for acting on transactional survey feedback

Use Info-Tech’s Transactional Satisfaction Survey Review Process workflow as a template to define your own process.

Related Info-Tech Research

Standardize the Service Desk

This project will help you build and improve essential service desk processes, including incident management, request fulfillment, and knowledge management to create a sustainable service desk.

Optimize the Service Desk With a Shift-Left Strategy

This project will help you build a strategy to shift service support left to optimize your service desk operations and increase end-user satisfaction.

Build a Continual Improvement Plan

This project will help you build a continual improvement plan for the service desk to review key processes and services and manage the progress of improvement initiatives.

Deliver a Customer Service Training Program to Your IT Department

This project will help you deliver a targeted customer service training program to your IT team to enhance their customer service skills when dealing with end users, improve overall service delivery and increase customer satisfaction.

Sources Cited

Amaresan, Swetha. “The best time to send a survey, according to 5 studies.” Hubspot. 15 Jun 2021. Accessed October 2022.
Arlen, Chris. “The 5 Service Dimensions All Customers Care About.” Service Performance Inc. n.d. Accessed October 2022.
Clinton, William Jefferson. “Setting Customer Service Standards.” (1993). Federal Register, 58(176).
“Understanding Confidentiality and Anonymity.” The Evergreen State College. 2022. Accessed October 2022.
"Highlights of the 2017 U.S. PIAAC Results Web Report" (NCES 2020-777). U.S. Department of Education. Institute of Education Sciences, National Center for Education Statistics.
Joe the IT Guy. “Are IT Support’s Customer Satisfaction Surveys Their Own Worst Enemy?” Joe the IT Guy. 29 August 2018. Accessed October 2022.
O’Reardon, David. “10 Ways to Get the Most out of your ITSM Ticket Surveys.” LinkedIn. 2 July 2019. Accessed October 2022.
O'Reardon, David. "13 Ways to increase the response rate of your Service Desk surveys".LinkedIn. 8 June 2016. Accessed October 2022.
O’Reardon, David. “IT Customer Feedback Management – A Why & How Q&A with an Expert.” LinkedIn. 13 March 2018. Accessed October 2022.
Parasuraman, A., Zeithaml, V. A., & Berry, L. L. (1985). "A Conceptual Model of Service Quality and Its Implications for Future Research." Journal of Marketing, 49(4), 41–50.
Quantisoft. "How to Increase IT Help Desk Customer Satisfaction and IT Help Desk Performance.“ Quantisoft. n.d. Accessed November 2022.
Rumberg, Jeff. “Metric of the Month: Customer Effort.” HDI. 26 Mar 2020. Accessed September 2022.
Sauro, Jeff. “15 Common Rating Scales Explained.” MeasuringU. 15 August 2018. Accessed October 2022.
SDI. “Customer Experience in ITSM.” SDI. 2018. Accessed October 2022.
SDI. “CX: Delivering Happiness – The Series, Part 1.” SDI. 12 January 2021. Accessed October 2022.
Wronski, Laura. “Who responds to online surveys at each hour of the day?” SurveyMonkey. n.d. Accessed October 2022.

Research contributors

Sally Colwell

Project Officer

Government of Canada Pension Centre

Develop an IT Infrastructure Services Playbook

Automation, SDI, and DevOps – build a cheat sheet to manage a changing Infrastructure & Operations environment.

Table of contents

Analyst Perspective

Executive Summary

Project Overview

• Phase 1: Define and Prioritize Infrastructure Services
  • 1.1: Define the services you own
  • 1.2: Prioritize infrastructure services
• Phase 2: Build Workflows and an Infrastructure Services Playbook
  • 2.1: Design workflow for service delivery
  • 2.2: Build a services playbook
• Phase 3: Identify Costs and Mature Service Delivery Capabilities
  • 3.1: Eliminate infrastructure service costs
  • 3.2: Mature your I&O organization into a service broker

Summary and Conclusion

ANALYST PERSPECTIVE

Technology is changing how infrastructure services are delivered.

"Managing a hybrid infrastructure environment is challenge enough. Add to this the pressure on IT Operations to deliver services faster and more continuously – it’s a recipe for boondoggle deployments, overcommitted staff, end-user frustration, and operational gridlock.

It’s not every service you provide that causes problems, so prioritize a few in-demand, painful services. Build and maintain durable, flexible processes that enable your team to provide consistent, repeatable services at a standard cost. Identify opportunities to improve service delivery.

You’ll save the business time and money and your own team significant grief." (Andrew Sharp, Research Manager, Infrastructure & Operations, Info-Tech Research Group)

Your infrastructure and operations team is a service provider; standardize, document, and communicate service capabilities

This Research is Designed For:

• CTOs and Infrastructure Managers
• Service Level Managers
• ITSM Managers and Process Owners

This Research Will Help You:

• Inventory services that IT Infrastructure & Operations (I&O) provides to the business (servers, storage, and network).
• Standardize services and track costs.
• Articulate the value of these services to business owners.
• Develop a catalog of infrastructure services.

This Research Will Also Assist:

• CIOs
• Application Development Managers
• Security Managers
• Auditors

This Research Will Help Them:

• Understand the complexities of technical service delivery.
• Make better strategic IT infrastructure decisions.

Executive summary

Situation

• Infrastructure and operations teams are managing deployments on- and off-premises and across multiple infrastructure service providers.
• Though automation tools speed up the delivery process, documentation is always pushed off so the team can meet urgent deadlines.

Complication

• Cloud providers have set the bar high for ease of access to stable infrastructure services.
• Without documented delivery processes, wait times are longer, controls are adequate but ad hoc, builds are non-standard, and errors are more likely to be introduced in production.

Resolution

• Prioritize in-demand services for documentation and standardization.
• Build service workflows and document service requirements in the services playbook.
• Create a costing model and track costs to deliver defined services.
• Leverage data on costs and service requirements to improve service delivery.

Info-Tech Insight

1. Keep it simple. Work through a few in-demand services to get early value from the project.
2. Don’t get lost in automation or tooling. You don’t need a complex tool or back-end automation to get value from standardized services.
3. Do then iterate. With a few completed service processes, it’s much easier to identify opportunities for service automation.

Create an infrastructure services playbook to improve efficiency, support DevOps, and streamline service delivery

Begin building an infrastructure services playbook by defining the services you provide. This will also help your team support changes to service delivery (e.g. more use of cloud services and the shift to DevOps).

In this blueprint, the first step will be to document infrastructure services to:

1. Clarify infrastructure capabilities and achievable service levels.

   Document infrastructure services to clarify achievable service levels with given resources and what you will need to meet service-level requirement gaps. Establishing your ability to meet customer demands is the first step toward becoming a broker of internal or external services.

2. Standardize infrastructure service delivery.

   Sometimes, it’s extremely important to do the exact same thing every time (e.g. server hardening). Sometimes, your team needs room to deviate from the script. Create a playbook that allows you to standardize service delivery as needed.

3. Make good strategic infrastructure decisions.

   Knowledge is power. Defined services and capabilities will help you make important strategic infrastructure decisions around capacity planning and when outsourcing is appropriate.

Review and optimize infrastructure service delivery as you shift to more cloud-based services

If you can’t standardize and streamline how you support cloud services, you risk AppDev and business leaders circumventing the I&O team.

	Example: Create a new server resource in a virtual environment vs. public cloud In a virtualized environment, provisioning processes can still be relatively siloed. In a software-defined environment, many steps require knowledge across the infrastructure stack. Better documentation will help your team deliver services outside their area of specialty.
Identify CPU requirements for a virtual machine (VM) Calculate VM memory requirements Configure the floppy drive for a VM Configure IDE devices for a VM Configure SCSI adapters for a VM Configure network adapters for a VM Configure VM priority for host CPU resources Server is live		Complete SDI code development & review, version control, build status, etc. Identify software and specifications for the instance you want to use Review configuration, storage, and security settings Secure the instance with an existing key pair or create a new key pair Update documentation – public IP address, physical & logical connections, data flows, etc. Launch and connect to instance Server is live