Adopt Generative AI in Solution Delivery

Drive solution quality and team productivity with the right generative AI capabilities.

Analyst Perspective

Build the case for Gen AI with the right opportunities.

Generative AI (Gen AI) presents unique opportunities to address many solution delivery challenges. Code generation can increase productivity, synthetic data generation can produce usable test data, and scanning tools can identify issues before they occur. To be successful, teams must be prepared to embrace the changes that Gen AI brings. Stakeholders must also give teams the opportunity to optimize their own processes and gauge the fit of Gen AI.

Start small with the intent to learn. The right pilot initiative helps you learn the new technology and how it benefits your team without the headache of complex setups and lengthy training and onboarding. Look at your existing solution delivery tools to see what Gen AI capabilities are available and prioritize the use cases where Gen AI can be used out of the box.

Andrew Kum-Seun
Research Director,
Application Delivery and Management
Info-Tech Research Group

Executive Summary

Your Challenge

Delivery teams are under continuous pressure to deliver high-value, high-quality solutions with limited capacity in complex business and technical environments. Common challenges experienced by these teams include:

• Attracting and retaining talent
• Maximizing the return on technology
• Confidently shifting to digital
• Addressing competing priorities
• Fostering a collaborative culture
• Creating high-throughput teams

Generative AI (Gen AI) offers a unique opportunity to address many of these challenges.

Common Obstacles

• Your stakeholders' understanding of what is Gen AI, its value and its application, can be driven by hype and misinterpretation. This confusion can lead to unrealistic expectations and set the wrong precedent for the role Gen AI is intended to play.
• Your solution delivery process is not well documented and is often executed inconsistently. An immature practice will not yield the benefits stakeholders expect.
• The Gen AI marketplace is very broad and diverse. Selecting the appropriate tools and partners is confusing and overwhelming.
• There is a skills gap for what is needed to configure, adopt, and operate Gen AI.

Info-Tech's Approach

• Ground your Gen AI expectations. Set realistic and achievable goals centered on driving business value and efficiency across the entire solution delivery process by enabling Gen AI in key tasks and activities. Propose this process as the ideal pilot for Gen AI.
• Select the right Gen AI opportunities. Discuss how proven Gen AI capabilities can be applied to your solution delivery practice and achieve the outcomes and priorities stakeholders expect. Lessons learned sow the foundation for future Gen AI scaling.
• Assess your Gen AI readiness in your solution delivery teams. Clarify the roles, processes, and tools needed for the implementation, use, and maintenance of Gen AI.

Info-Tech Insight

Position Gen AI as a tooling opportunity to enhance the productivity and depth of your solution delivery practice. Current Gen AI tools are unable to address the various technical and human complexities that commonly occur in solution delivery. Assess the fit of Gen AI by augmenting low-risk, out-of-the-box tools in key areas of your solution delivery process and teams.

Insight Summary

Overarching Info-Tech Insight

Position Gen AI is a tooling opportunity to enhance the productivity and depth of your solution delivery practice. However, current Gen AI tools are unable to address the various technical and human complexities that commonly occur in solution delivery. Assess the fit of Gen AI by augmenting low-risk, out-of-the-box tools in key areas of your solution delivery process and teams.

Understand and optimize first, automate with Gen AI later.
Gen AI magnifies solution delivery inefficiencies and constraints. Adopt a user-centric perspective to understand your solution delivery teams' interactions with solution delivery tools and technologies to better replicate how they complete their tasks and overcome challenges.

Enable before buy. Buy before build.
Your solution delivery vendors see AI as a strategic priority in their product and service offering. Look into your existing toolset and see if you already have the capabilities. Otherwise, prioritize using off-the-shelf solutions with pre-trained Gen AI capabilities and templates.

Innovate but don't experiment.
Do not reinvent the wheel and lower your risk of success. Stick to the proven use cases to understand the value and fit of Gen AI tools and how your teams can transform the way they work. Use your lessons learned to discover scaling opportunities.

Blueprint benefits

What is Gen AI?

Generative AI (Gen AI)
A form of ML whereby, in response to prompts, a Gen AI platform can generate new output based on the data it has been trained on. Depending on its foundational model, a Gen AI platform will provide different modalities and use case applications.

Machine Learning (ML)
The AI system is instructed to search for patterns in a data set and then make predictions based on that set. In this way, the system learns to provide accurate content over time. This requires a supervised intervention if the data is inaccurate. Deep learning is self-supervised and does not require intervention.

Artificial Intelligence (AI)
A field of computer science that focuses on building systems to imitate human behavior. Not all AI systems have learning behavior; many systems (such as customer service chatbots) operate on preset rules.

Info-Tech Insight

Many vendors have jumped on Gen AI as the latest marketing buzzword. When vendors claim to offer Gen AI functionality, pin down what exactly is generative about it. The solution must be able to induce new outputs from inputted data via self-supervision – not trained to produce certain outputs based on certain inputs.

Augment your solution delivery teams with Gen AI

Position Gen AI as a tooling opportunity to enhance the productivity and depth of your solution delivery practice. Current Gen AI tools are unable to address the various technical and human complexities that commonly occur in solution delivery; assess the fit of Gen AI by augmenting low-risk, out-of-the-box tools in key areas of your solution delivery process and teams.

Research deliverable

This research is accompanied by a supporting deliverable to help you accomplish your goals.

Gen AI Solution Delivery Readiness Assessment Tool

Assess the readiness of your solution delivery team for Gen AI. This tool will ask several questions relating to your people, process, and technology, and recommend whether the team is ready to adopt Gen AI practices.

Step 1.1

Set the context

Activities

1.1.1 Understand the challenges of your solution delivery teams.

1.1.2 Outline the value you expect to gain from Gen AI.

This step involves the following participants:

• Applications VP
• Applications Director
• Solution Delivery Manager
• Solution Delivery Team

Outcomes of this step

• SWOT Analysis to help articulate the challenges facing your teams.
• A Gen AI Canvas that will articulate the value you expect to gain.

IT struggles to deliver solutions effectively

• Lack of skills and resources
  Forty-six percent of respondents stated that it was very or somewhat difficult to attract, hire, and retain developers (GitLab, 2023; N=5,010).
• Delayed software delivery
  Code development (37%), monitoring/observability (30%), deploying to non-production environments (30%), and testing (28%) were the top areas where software delivery teams or organizations encountered the most delays (GitLab, 2023, N=5,010).
• Low solution quality and satisfaction
  Only 64% of applications were identified as effective by end users. Effective applications are identified as at least highly important and have high feature and usability satisfaction (Application Portfolio Assessment, August 2021 to July 2022; N=315).
• Burnt out teams
  While workplace flexibility comes with many benefits, longer work hours jeopardize wellbeing. Sixty-two percent of organizations reported increased working hours, while 80% reported an increase in flexibility ("2022 HR Trends Report," McLean & Company, 2022; N=394) .

Creating high-throughput teams is an organizational priority.

CXOs ranked "optimize IT service delivery" as the second highest priority. "Achieve IT business" was ranked first.

(CEO-CIO Alignment Diagnostics, August 2021 to July 2022; n=568)

1.1.1 Understand the challenges of your solution delivery teams

1-3 hours

1. Complete a SWOT analysis of your solution delivery team to discover areas where Gen AI can be applied.
2. Record this information in the Gen AI Solution Delivery Readiness Assessment Tool.

Record the results in the Gen AI Solution Delivery Readiness Assessment Tool

Output

• SWOT analysis of current state of solution delivery practice

Participants

• Applications VP
• Applications Director
• Solution Delivery Manager
• Solution Delivery Team

Gen AI can help solve your solution delivery challenges

Why is software delivery an ideal pilot candidate for Gen AI?

• Many software delivery practices are repeatable and standardized.
• Software delivery roles that are using and implementing Gen AI are technically savvy.
• Automation is a staple in many commonly used tools.
• Change will likely not impact business operations.

Improved productivity

Gen AI jumpstarts the most laborious and mundane parts of software delivery. Delivery teams saved 22 hours (avg) per software use case when using AI in 2022, compared to last year when AI was not used ("Generative AI Speeds Up Software Development," PRNewswire, 2023).

Fungible resources

Teams are transferrable across different frameworks, platforms, and products. Gen AI provides the structure and guidance needed to work across a wider range of projects ("Game changer: The startling power generative AI is bringing to software development," KPMG, 2023).

Improved solution quality

Solution delivery artifacts (e.g. code) are automatically scanned to quickly identify bugs and defects based on recent activities and trends and validate against current system performance and capacity.

Business empowerment

AI enhances the application functionalities workers can build with low- and no-code platforms. In fact, "AI high performers are 1.6 times more likely than other organizations to engage non-technical employees in creating AI applications" ("The state of AI in 2022 — and a half decade in review." McKinsey, 2022, N=1,492).

However, various fears, uncertainties, and doubts challenge Gen AI adoption

Black Box

Little transparency is provided on the tool's rationale behind content creation, decision making, and the use and storage of training data, creating risks for legal, security, intellectual property, and other areas.

Role Replacement

Some workers have job security concerns despite Gen AI being bound to their rule-based logic framework, the quality of their training data, and patterns of consistent behavior.

Skills Gaps

Teams need to gain expertise in AI/ML techniques, training data preparation, and continuous tooling improvements to support effective Gen AI adoption across the delivery practice and ensure reliable operations.

Data Inaccuracy

Significant good quality data is needed to build trust in the applicability and reliability of Gen AI recommendations and outputs. Teams must be able to combine Gen AI insights with human judgment to generate the right outcome.

Slow Delivery of AI Solution

Timelines are sensitive to organizational maturity, experience with Gen AI, and investments in good data management practices. 65% of organizations said it took more than three months to deploy an enterprise-ready AIOps solution (OpsRamp, 2022).

Define the value you want Gen AI to deliver

Well-optimized Gen AI instills stakeholder confidence in ongoing business value delivery and ensures stakeholder buy-in, provided proper expectations are set and met. However, business value is not interpreted or prioritized the same across the organization. Come to a common business value definition to drive change in the right direction by balancing the needs of the individual, team, and organization.

Business value cannot always be represented by revenue or reduced expenses. Dissecting value by the benefit type and the value source's orientation allows you to see the many ways in which Gen AI brings value to the organization.

Financial benefits vs. intrinsic needs

• Financial benefits refers to the degree to which the value source can be measured through monetary metrics, such as revenue generation and cost saving.
• Intrinsic needs refers to how a product, service, or business capability enhanced with Gen AI meets functional, user experience, and existential needs.

Inward vs. outward orientation

• Inward refers to value sources that are internally impacted by Gen AI and improve your employees' and teams' effectiveness in performing their responsibilities.
• Outward refers to value sources that come from your interaction with external stakeholders and customers and were improved from using Gen AI.

See our Build a Value Measurement Framework blueprint for more information about business value definition.

Measure success with the right metrics

Establishing and monitoring metrics are powerful ways to drive behavior and strategic changes in your organization. Determine the right measures that demonstrate the value of your Gen AI implementation by aligning them with your Gen AI objectives, business value drivers, and non-functional requirements.

Select metrics with different views

1. Solution delivery practice effectiveness
   The ability of your practice to deliver, support, and operate solutions with Gen AI
   Examples: Solution quality and throughput, delivery and operational costs, number of defects and issues, and system quality
2. Solution quality and value
   The outcome of your solutions delivered with Gen AI tools
   Examples: Time and money saved, utilization of products and services, speed of process execution, number of errors, and compliance with standards
3. Gen AI journey goals and milestones
   Your organization's position in your Gen AI journey
   Examples: Maturity score, scope of Gen AI adoption, comfort and
   confidence with Gen AI capabilities, and complexity of Gen AI use cases

Leverage Info-Tech's Diagnostics

IT Management & Governance

• Improvement to application development quality and throughput effectiveness
• Increased importance of application delivery and maintenance capabilities across the IT organization
• Delegation of delivery accountability across more IT roles

CIO Business Vision

• Improvements to IT satisfaction and value from delivered solutions
• Changes to the value and importance of IT core services enabled with Gen AI
• The state of business and IT relationships
• Capability to deliver and support Gen AI effectively

1.1.2 Outline the value you expect to gain from Gen AI

1-3 hours

1. Complete the following fields to build your Gen AI canvas:
   1. Problem that Gen AI is intending to solve
   2. List of stakeholders
   3. Desired business and IT outcomes
   4. In-scope solution delivery teams, systems, and capabilities.
2. Record this information in the Gen AI Solution Delivery Readiness Assessment Tool.

Output

• Gen AI Canvas

Participants

• Applications VP
• Applications Director
• Solution Delivery Manager
• Solution Delivery Team

Record the results in the Gen AI Solution Delivery Readiness Assessment Tool

1.1.2 Example

Example of an outline of the value you expect to gain from Gen AI

Problem statements

• Manual testing procedures hinder pace and quality of delivery.
• Inaccurate requirement documentation leads to constant redesigning.

Business and IT outcomes

• Improve code quality and performance.
• Expedite solution delivery cycle.
• Improve collaboration between teams and reduce friction.

List of stakeholders

• Testing team
• Application director
• CIO
• Design team
• Project manager
• Business analysts

In-scope solution delivery teams, system, and capabilities

• Web
• Development
• App development
• Testing
• Quality assurance
• Business analysts
• UI/UX design

Align your objectives to the broader AI strategy

Why is an organizational AI strategy important for Gen AI?

• All Gen AI tactics and capabilities are designed, delivered, and managed to support a consistent interpretation of the broader AI vision and goals.
• An organizational strategy gives clear understanding of the sprawl, criticality, and risks of Gen AI solutions and applications to other IT capabilities dependent on AI.
• Gen AI initiatives are planned, prioritized, and coordinated alongside other software delivery practice optimizations and technology modernization initiatives.
• Resources, skills, and capacities are strategically allocated to meet the needs of Gen AI considering other commitments in the software delivery optimization backlog and roadmap.
• Gen AI expectations and practices uphold the persona, values, and principles of the software delivery team.

What is an AI strategy?

An AI strategy details the direction, activities, and tactics to deliver on the promise of your AI portfolio. It often includes:

• AI vision and goals
• Application, automation, and process portfolio involved or impacted by AI
• Values and principles
• Health of your AI portfolio
• Risks and constraints
• Strategic roadmap

Step 1.2

Evaluate opportunities for Gen AI

Activities

1.2.1 Align Gen AI opportunities with teams and capabilities.

This step involves the following participants:

• Applications VP
• Applications Director
• Solution Delivery Manager
• Solution Delivery Team

Outcomes of this step

• Understand the Gen AI opportunities for your solution delivery practice.

Learn how Gen AI is employed in solution delivery

Other common AI opportunities solutions include test case generation, code translation, use case creation, document generation, and automated testing.

Opportunity 1: Synthetic data generation

Create artificial data that mimics the structure of real-life data.

What are the expected benefits?

• Availability of test data: Creation of large volumes of data compatible for testing multiple systems within the organization.
• Improved privacy: Substituting real data with artificial leads to reduced data leaks.
• Quicker data provisioning: Automated generation of workable datasets aligned to company policies.

What are the notable risks and challenges?

• Generalization and misrepresentations: Data models used in synthetic data generation may not be an accurate representation of production data because of potentially conflicting definitions, omission of dependencies, and multiple sources of truth.
• Lack of accurate representation: It is difficult for synthetic data to fully capture real-world data nuances.
• Legal complexities: Data to build and train the Gen AI tool does not comply with data residency and management standards and regulations.

How should teams prepare for synthetic data generation?

• Understand and align requirements with objectives and involve all stakeholders. Refer to Create a Data Management Roadmap.
• Evaluate quality and validate effectiveness. Refer to Build Your Data Quality Program.
• Build your test data management practices. Refer to Provision and Mask Your Test Data With the Right Tool.

It can be used:

• To train machine learning models when there is not enough real data, or the existing data does not meet specific needs.
• To improve quality of test by using data that closely resembles production without the risk of leveraging sensitive and private information.

"We can simply say that the total addressable market of synthetic data and the total addressable market of data will converge,"
Ofir Zuk, CEO, Datagen (Forbes, 2022)

Opportunity 2: Code generation

Learn patterns and automatically generate code.

What are the expected benefits?

• Increased productivity: It allows developers to generate more code quickly.
• Improved code consistency: Code is generated using a standardized model and lessons learnt from successful projects.
• Rapid prototyping: Expedite development of a working prototype to be verified and validated.

What are the notable risks and challenges?

• Limited contextual understanding: AI may lack domain-specific knowledge or understanding of requirements.
• Dependency: Overreliance on AI generated codes can affect developers' creativity.
• Quality concerns: Generated code is untested and its alignment to coding and quality standards is unclear.

How should teams prepare for code generation?

• Improve the development roles and processes. Refer to Improve Application Development Throughput.
• Build quality into the solution delivery process. Refer to Build a Software Quality Assurance Program.
• Build policies and governance around code generation. Refer to Satisfy Digital End Users With Low- and No-Code.

It can be used to:

• Build solutions without the technical expertise of traditional development.
• Discover different solutions to address coding challenges.
• Kickstart new development projects with prebuilt code.

According to a survey conducted by Microsoft's GitHub, a staggering 92% of programmers were reported as using AI tools in their workflow (GitHub, 2023).

Opportunity 3: Defect forecasting & debugging

Predict and proactively address defects before they occur.

What are the expected benefits?

• Reduced maintenance cost: Find defects earlier in the delivery process, when it's cheaper to fix them.
• Increased efficiency: Testing efforts can remain focused on critical and complex areas of solution.
• Reduced risk: Find critical defects before the product is deployed to production.

What are the notable risks and challenges?

• False positives and negatives: Incorrect interpretation and scope of defect due to inadequate training of the Gen AI model.
• Inadequate training: Training data does not reflect the complexity of the solutions code.
• Not incorporating feedback: Gen AI models are not retrained in concert with solution changes.

How should teams prepare for defect forecasting and debugging?

• Augment Gen AI with software delivery team. Refer to Improve Application Development Throughput and Embed Security into the DevOps Pipeline.
• Gain understanding of past solution performances. Refer to IT Service Management Selection Guide.
• Learn or discover tool sets. Refer to Automate Testing to Get More Doneand Software Quality Assurance.

It can be used to:

• Perform static and dynamic code analysis to find vulnerabilities in the solution source code.
• Forecast potential issues of a solution based on previous projects and industry trends.
• Find root cause and suggest solutions to address found defects.

Using AI technologies, developers can reduce the time taken to debug and test code by up to 70%, allowing them to finish projects faster and with greater accuracy (Aloa, 2023).

Opportunity 4: Requirements documentation & elicitation

Capturing, documenting, and analyzing function and nonfunctional requirements.

What are the expected benefits?

• Improve quality of requirements: Obtain different perspectives and contexts for the problem at hand and help identify ambiguities and misinterpretation of risks and stakeholder expectation.
• Increased savings: Fewer resources are consumed in requirements elicitation activities.
• Increased delivery confidence: Provide sufficient information for the solution delivery team to confidently estimate and commit to the delivery of the requirement.

What are the notable risks and challenges?

• Conflicting bias: Gen AI models may interpret the problem differently than how the stakeholders perceive it.
• Organization-specific interpretation: Inability of the Gen AI models to accommodate unique interpretation of terminologies, standards, trends and scenarios.
• Validation and review: Interpreting extracted insights requires human validation.

How should teams prepare for requirements documentation & elicitation?

• Feed high quality, customizable data for domain-specific Use. Refer to Build Your Data Quality Program.
• Continuously train the model. Refer to Improve Requirements Gathering.
• Refine organization policies, processes, and roles. Refer to Requirements Gathering SOP and BA Playbook.

It can be used to:

• Document requirements in a clear and concise manner that is usable to the solution delivery team.
• Analyze and test requirements against various user, business, and technical scenarios.

91% of top businesses surveyed report having an ongoing investment in AI (NewVantage Partners, 2021).

Opportunity 5: UI design and prototyping

Analyze existing patterns and principles to generate design, layouts, and working solutions.

What are the expected benefits?

• Increased experimentation: Explore different approaches and tactics to solve a solution delivery problem.
• Improved collaboration: Provide quick design layouts that can be reshaped based on stakeholder feedback.
• Ensure design consistency: Enforce a UI/UX design standard for all solutions.

What are the notable risks and challenges?

• Misinterpretation of UX Requirements: Gen AI model incorrectly assumes a specific interpretation of user needs, behaviors, and problem.
• Incorrect or missing requirements: Lead to extensive redesigns and iterations, adding to costs while hampering user experience.
• Design creativity: May lack originality and specific brand aesthetics if not augmented well with human customizability and creativity.

How should teams prepare for UI design and prototyping?

• Understand the solution's end users. Refer to Apply Design Thinking to Build Empathy With the Business.
• Enhance your practice. Refer to Satisfy Digital End Users With Low- and No-Code.
• Implement an iterative delivery process. Refer to Agile & DevOps Resource Center.

It can be used to:

• Visualize the solution through different views and perspectives such as process flows and use-case diagrams.
• Create working prototypes that can be verified and validated by stakeholders and end users.

A study by McKinsey & Company found that companies that invest in AI-driven design outperform their peers in revenue growth and customer experience metrics. They were found to achieve up to two times higher revenue growth than industry peers and up to 10% higher net promoter score (McKinsey & Company, 2018).

Determine the importance of your opportunities by answering these questions

Realizing the complete potential of Gen AI relies on effectively fostering its adoption and resulting changes throughout the entire solution delivery process.

What are the challenges faced by your delivery teams that could be addressed by Gen AI?

• Recognize the precise pain points, bottlenecks, or inefficiencies faced by delivery teams.
• Include all stakeholders' perspectives during problem discovery and root cause analysis.

What's holding back Gen AI adoption in the organization?

• Apart from technical barriers, address cultural and organizational challenges and discuss how organizational change management strategies can mitigate Gen AI adoption risk.

Are your objectives aligned with Gen AI capabilities?

• Identify areas where processes can be modernized and streamlined with automation.
• Evaluate the current capabilities and resources available within the organization to leverage Gen AI technologies effectively.

How can Gen AI improve the entire solution delivery process?

• Investigate and evaluate the improvements Gen AI can reasonably deliver, such as increased accuracy, quickened delivery cycles, improved code quality, or enhanced cross-functional collaboration.

1.2.1 Align Gen AI opportunities to teams and capabilities

1-3 hours

1. Associate the Gen AI opportunities that can be linked to your system capabilities. These opportunities refer to the potential applications of generative AI techniques, such as code generation or synthetic data, to address specific challenges.
   1. Start by analyzing your system's requirements, constraints, and areas where Gen AI techniques can bring value. Identify the potential benefits of integrating Gen AI, such as increased productivity, or enhanced creativity.
   2. Next, discern potential risks or challenges, such as dependency or quality concerns, associated with the opportunity implementation.
2. Record this information in the Gen AI Solution Delivery Readiness Assessment Tool.

Output

• Gen AI opportunity selection

Participants

• Applications VP
• Applications Director
• Solution Delivery Manager
• Solution Delivery Team

Record the results in the Gen AI Solution Delivery Readiness Assessment Tool

Keep an eye out for red flags

Not all Gen AI opportunities are delivered and adopted the same. Some present a bigger risk than others.

• Establishing vague targets and success criteria
• Defining Gen AI as substitution of human capital
• Open-source software not widely adopted or validated
• High level of dependency on automation
• Unadaptable cross-functional training across organization
• Overlooking privacy, security, legal, and ethical implications
• Lack of Gen AI expertise and understanding of good practices

Step 1.3

Assess your readiness for Gen AI

Activities

1.3.1 Assess your readiness for Gen AI.

This step involves the following participants:

• Applications VP
• Applications Director
• Solution Delivery Manager
• Solution Delivery Team

Outcomes of this step

• A completed Gen AI Readiness Assessment to confirm how prepared you are to embrace Gen AI in your solution delivery team.

Prepare your SDLC* to leverage Gen AI

As organizations evolve and adopt more tools and technology, their solution delivery processes become more complex. Process improvement is needed to simplify complex and undocumented software delivery activities and artifacts and prepare it for Gen AI. Gen AI scales process throughput and output quantity, but it multiplies the negative impact of problems the process already has.

When is your process ready for Gen AI?

• Solution value – Ensures the accuracy and alignment of the committed feature and change requests to what the stakeholder truly expects and receives.
• Throughput – Delivers new products, enhancements, and changes at a pace and frequency satisfactory to stakeholder expectations and meets delivery commitments.
• Process governance – Has clear ownership and appropriate standardization. The roles, activities, tasks, and technologies are documented and defined. At each stage of the process someone is responsible and accountable.
• Process management – Follows a set of development frameworks, good practices, and standards to ensure the solution and relevant artifacts are built, tested, and delivered consistently and repeatably.
• Technical quality assurance – Accommodates committed non-functional requirements within the stage's outputs to ensure products meet technical excellence expectations.

*software development lifecycle

To learn more, visit Info-Tech's Modernize Your SDLC blueprint.

To learn more, visit Info-Tech's Build a Winning Business Process Automation Playbook

Assess the impacts from Gen AI changes

Ensure that no stone is left unturned as you evaluate the fit of Gen AI and prepare your adoption and support plans.

By shining a light on considerations that might have otherwise escaped planners and decision makers, an impact analysis is an essential component to Gen AI success. This analysis should answer the following questions on the impact to your solution delivery teams.

1. Will the change impact how our clients/customers receive, consume, or engage with our products/services?
2. Will there be an increase in operational costs, and a change to compensation and/or rewards?
3. Will this change increase the workload and alter staffing levels?
4. Will the vision or mission of the team change?
5. Will a new or different set of skills be needed?
6. Will the change span multiple locations/time zones?
7. Are multiple products/services impacted by this change?
8. Will the workflow and approvals be changed, and will there be a substantial change to scheduling and logistics?
9. Will the tools of the team be substantially different?
10. Will there be a change in reporting relationships?

See our Master Organizational Change Management Practices blueprint for more information.

Brace for impact

A thorough analysis of change impacts will help your software delivery teams and change leaders:

• Bypass avoidable problems.
• Remove non-fixed barriers to success.
• Acknowledge and minimize the impact of unavoidable barriers.
• Identify and leverage potential benefits.
• Measure the success of the change.

Many key IT capabilities are required to successfully leverage Gen AI

Portfolio Management

An accurate and rationalized inventory of all Gen AI tools verifies they support the goals and abide to the usage policies of the broader delivery practice. This becomes critical when tooling is updated frequently and licenses and open- source community principles drastically change (e.g. after an acquisition).

Quality Assurance

Gen AI tools are routinely verified and validated to ensure outcomes are accurate, complete, and aligned to solution delivery quality standards. Models are retrained using lessons learned, new use cases, and updated training data.

Security & Access Management

Externally developed and trained Gen AI models may not include the measures, controls, and tactics you need to prevent vulnerabilities and protect against threats that are critical in your security frameworks, policies, and standards.

Data Management & Governance

All solution delivery data and artifacts can be transformed and consumed in various ways as they transit through solution delivery and Gen AI tools. Data integrations, structures, and definitions must be well-defined, governed, and monitored.

OPERATIONAL SUPPORT

Resources are available to support the ongoing operations of the Gen AI tool, including infrastructure, preparing training data, and managing integration with other tools. They are also prepared to recover backups, roll back, and execute recovery plans at a moment's notice.

Apply Gen AI good practices in your solution delivery practice

1. Keep the human in the loop.
   Gen AI models cannot produce high-quality content with 100% confidence. Keeping the human in the loop allows people to directly give feedback to the model to improve output quality.
2. Strengthen prompt and query engineering.
   The value of the outcome is dependent on what is being asked. Good prompts and queries focus on creating the optimal input by selecting and phrasing the appropriate words, sentence structures, and punctuation to illustrate the focus, scope, problem, and boundaries.
3. Thoughtfully prepare your training data.
   Externally hosted Gen AI tools may store your training data in their systems or use it to train their other models. Intellectual property and sensitive data can leak into third-party systems and AI models if it is not properly masked and sanitized.
4. Build guardrails into your Gen AI models.
   Guardrails can limit the variability of any misleading Gen AI responses by defining the scope and bounds of the response, enforcing the policies of its use, and clarifying the context of its response.
5. Monitor your operational costs.
   The cost breakdown will vary among the types of Gen AI solution and the vendor offerings. Cost per query, consultant fees, infrastructure hosting, and licensing costs are just a few cost factors. Open source can be an attractive cost-saving option, but you must be willing to invest in the roles to assume traditional vendor accountabilities.
6. Check the licenses of your Gen AI tool.
   Each platform has licenses and agreements on how their solution can or cannot be used. They limit your ability to use the tool for commercial purposes or reproductions or may require you to purchase and maintain a specific license to use their solution and materials.

See Build Your Generative AI Roadmap for more information.

Assess your Gen AI readiness

• Solution delivery team
  The team is educated on Gen AI, its use cases, and the tools that enable it. They have the skills and capacity to implement, create, and manage Gen AI.
• Solution delivery process and tools
  The solution delivery process is documented, repeatable, and optimized to use Gen AI effectively. Delivery tools are configured to enable, leverage and manage Gen AI assets to improve their performance and efficiency.
• Solution delivery artifacts
  Delivery artifacts (e.g. code, scripts, documents) that will be used to train and be leveraged by Gen AI tools are discoverable, accurate, complete, standardized, of sufficient quantity, optimized for Gen AI use, and stored in an accessible shared central repository.
• Governance
  Defined policies, role definitions, guidelines, and processes that guide the implementation, development, operations, and management of Gen AI.
• Vision and executive support
  Clear alignment of Gen AI direction, ambition, and objectives with broader business and IT priorities. Stakeholders support the Gen AI initiative and allocate human and financial resources for its implementation within the solution delivery team.
• Operational support
  The capabilities to manage the Gen AI tools and ensure they support the growing needs of the solution delivery practice, such as security management, hosting infrastructure, risk and change management, and data and application integration.

1.3.1 Assess your readiness for Gen AI

1-3 hours

1. Review the current state of your solution delivery teams including their capacity, skills and knowledge, delivery practices, and tools and technologies.
2. Determine the readiness of your team to adopt Gen AI.
3. Discuss the gaps that need to be filled to be successful with Gen AI.
4. Record this information in the Gen AI Solution Delivery Readiness Assessment Tool.

Record the results in the Gen AI Solution Delivery Readiness Assessment Tool

Output

• Gen AI Solution Delivery Readiness Assessment

Participants

• Applications VP
• Applications Director
• Solution Delivery Manager
• Solution Delivery Team

Recognize that Gen AI does not require a fully optimized solution delivery process

To learn more, visit Info-Tech's Develop Your Value-First Business Process Automation (BPA) Strategy.

Be prepared to take the next steps

Deliver Gen AI to your solution delivery teams

Modernize Your SDLC
Efficient and effective SDLC practices are vital, as products need to readily adjust to evolving and changing business needs and technologies.

Adopt Generative AI in Solution Delivery
Generative AI can drive productivity and solution quality gains to your solution delivery teams. Level set expectations with the right use case to demonstrate its value potential.

Select Your AI Vendor & Implementation Partner
The right vendor and partner are critical for success. Build the selection criteria to shortlist the products and services that best meets the current and future needs of your teams.

Drive Business Value With Off-the-Shelf AI
Build a framework that will guide your teams through the selection of an off-the-shelf AI tool with a clear definition of the business case and preparations for successful adoption.

Build Your Enterprise Application Implementation Playbook
Your Gen AI implementation doesn't start with technology, but with an effective plan that your team supports and is aligned to broader stakeholder and sponsor priorities and goals.

Build your Gen AI practice

• Get Started With AI
• AI Strategy & Generative AI Roadmap
• AI Governance

Related Info-Tech Research

Build a Winning Business Process Automation Playbook
Optimize and automate your business processes with a user-centric approach.

Embrace Business Managed Applications
Empower the business to implement their own applications with a trusted business-IT relationship.

Application Portfolio Management Foundations
Ensure your application portfolio delivers the best possible return on investment.

Maximize the Benefits from Enterprise Applications with a Center of Excellence
Optimize your organization's enterprise application capabilities with a refined and scalable methodology.

Create an Architecture for AI
Build your target state architecture from predefined best-practice building blocks.

Deliver on Your Digital Product Vision
Build a product vision your organization can take from strategy through execution.

Enhance Your Solution Architecture Practices
Ensure your software systems solution is architected to reflect stakeholders' short- and long-term needs.

Apply Design Thinking to Build Empathy With the Business
Use design thinking and journey mapping to make IT the business' go-to problem solver.

Modernize Your SDLC
Deliver quality software faster with new tools and practices.

Drive Business Value With Off-the-Shelf AI
A practical guide to ensure return on your off-the-shelf AI investment.

Bibliography

"Altran Helps Developers Write Better Code Faster with Azure AI." Microsoft, 2020.
"Apply Design Thinking to Complex Teams, Problems, and Organizations." IBM, 2021.
Bianca. "Unleashing the Power of AI in Code Generation: 10 Applications You Need to Know — AITechTrend." AITechTrend, 16 May 2023.
Biggs, John. "Deep Code Cleans Your Code with the Power of AI." TechCrunch, 26 Apr 2018.
"Chat GPT as a Tool for Business Analysis — the Brazilian BA." The Brazilian BA, 24 Jan 2023.
Davenport, Thomas, and Randy Bean. "Big Data and AI Executive Survey 2019." New Vantage Partners, 2019.
Davenport, Thomas, and Randy Bean. "Big Data and AI Executive Survey 2021." New Vantage Partners, 2021.
Das, Tamal. "9 Best AI-Powered Code Completion for Productive Development." Geek flare, 5 Apr 2023.
Gondrezick, Ilya. "Council Post: How AI Can Transform the Software Engineering Process." Forbes, 24 Apr 2020.
"Generative AI Speeds up Software Development: Compass UOL Study." PR Newswire, 29 Mar 2023.
"GitLab 2023 Global Develops Report Series." Gitlab, 2023.
"Game Changer: The Startling Power Generative AI Is Bringing to Software Development." KPMG, 30 Jan 2023.
"How AI Can Help with Requirements Analysis Tools." TechTarget, 28 July 2020.
Indra lingam, Ashanta. "How Spotify Is Upleveling Their Entire Design Team." Framer, 2019.
Ingle, Prathamesh. "Top Artificial Intelligence (AI) Tools That Can Generate Code to Help Programmers." Matchcoat, 1 Jan 2023.
Kaur, Jagreet . "AI in Requirements Management | Benefits and Its Processes." Xenon Stack, 13 June 2023.
Lange, Danny. "Game On: How Unity Is Extending the Power of Synthetic Data beyond the Gaming Industry." CIO, 17 Dec 2020.
Lin, Ying. "10 Artificial Intelligence Statistics You Need to Know in 2020." OBERLO, 17 Mar. 2023.
Mauran, Cecily. "Whoops, Samsung Workers Accidentally Leaked Trade Secrets via ChatGPT." Mashable, 6 Apr 2023.