z-Series Modernization and Migration

It seems like anytime there’s a new CIO who is not from the mainframe world there is immediate pressure to get off this platform. However, just as there is a high financial commitment required to stay on System Z, moving off is risky and potentially more costly. You need to truly understand the scale and complexity ahead of the organization.

Under the best of circumstances, mainframe systems are complex, expensive, and difficult to scale. In today’s world, applications written for mainframe legacy systems also present significant operational challenges to customers compounded by the dwindling pool of engineers who specialize in these outdated technologies. Many organizations want to migrate their legacy applications to the cloud, but to do so they need to go through a lengthy migration process that is made more challenging by the complexity of mainframe applications.

The most common tactic is for the organization to better realize its z/Series options and adopt a strategy built on complexity and workload understanding. To make the evident, obvious: the options here for the non-commodity are not as broad as with commodity server platforms and the mainframe is arguably the most widely used and complex non-commodity platform on the market.

Problem statement:

The z/Series remains a vital platform for many businesses and continues to deliver exceptional reliability and performance and play a key role in the enterprise. With the limited and aging resources at hand, CIOs and the like must continually review and understand their migration path with the same regard as any other distributed system roadmap.

This research is designed for:

IT strategic direction decision makers.

IT managers responsible for an existing z/Series platform.

Organizations evaluating platforms for mission critical applications.

This research will help you:

1. Evaluate the future viability of this platform.
2. Assess the fit and purpose, and determine TCO.
3. Develop strategies for overcoming potential challenges.
4. Determine the future of this platform for your organization.

Modernize the mainframe … here we go again.

Prior to 2020, most organizations were muddling around in “year eleven of the four-year plan” to exit the mainframe platform where a medium-term commitment to the platform existed. Since 2020, it appears the appetite for the mainframe platform changed. Again. Discussions mostly seem to be about what the options are beyond hardware outsourcing or re-platforming to “cloud” migration of workloads – mostly planning and strategy topics. A word of caution: it would appear unwise to stand in front of the exit door for fear of being trampled.

Hardware expirations between now and 2025 are motivating hosting deployments. Others are in migration activities, and some have already decommissioned and migrated but now are trying to rehab the operations team now lacking direction and/or structure.

There is little doubt that modernization and “digital transformation” trends will drive more exit traffic, so IT leaders who are still under pressure to get off the platform need to assess their options and decide. Being in a state of perpetually planning to get off the mainframe handcuffs your ability to invest in the mainframe, address deficiencies, and improve cost-effectiveness.

Darin Stahl
Principal Research Advisor, Infrastructure & Operations Research
Info-Tech Research Group

Dating back to 2011, Darin Stahl has been the primary z/Series subject matter expert within the Infrastructure & Operations Research team. Below represents the percentage of calls, per industry, where z/Series advisory has been provided by Darin*:

37% - State Government

19% - Insurance

11% - Municipality

8% - Federal Government

8% - Financial Services

5% - Higher Education

3% - Retail

3% - Hospitality/Resort

3% - Logistics and Transportation

3% - Utility

Based on the Info-Tech call history, there is a consistent cross section of industry members who not only rely upon the mainframe but are also considering migration options.

Note:

Of course, this only represents industries who are Info-Tech members and who called for advisory services about the mainframe.

There may well be more Info-Tech members with mainframes who have no topic to discuss with us about the mainframe specifically. Why do we mention this?

We caution against suggesting things like, ”somewhat less than 50% of mainframes live in state data centers” or any other extrapolated inference from this data.

Our viewpoint and discussion is based on the cases and the calls that we have taken over the years.

*37+ enterprise calls were reviewed and sampled.

For most workloads “scale out" (e.g. virtualized cloud or IaaS ) is going to provide obvious and quantifiable benefits.

However, with some workloads (extremely large analytics or batch processing ) a "scale up" approach is more optimal. But the scale up is really limited to very specific workloads. Despite some assumptions, the gains made when moving from scale up to scale out are not linear.

Obviously, when you scale out from a performance perspective you experience a drop in what a single unit of compute can do. Additionally, there will be latency introduced in the form of network overhead, transactions, and replication into operations that were previously done just bypassing object references within a single frame.

Some applications or use cases will have to be architected or written differently (thinking about the high-demand analytic workloads at large scale). Remember the “grid computing” craze that hit us during the early part of this century? It was advantageous for many to distribute work across a grid of computing devices for applications but the advantage gained was contingent on the workload able to be parsed out as work units and then pulled back together through the application.

There can be some interesting and negative consequences for analytics or batch operations in a large scale as mentioned above. Bottom line, as experienced previously with Microfocus mainframe ports to x86, the batch operations simply take much longer to complete.

Big Data Considerations*:

• Value: Data has no inherent value until it’s used to solve a business problem.
• Variety: The type of data being produced is increasingly diverse and ranges from email and social media to geo-spatial and photographic data. This data may be difficult to process using a structured data model.
• Volume: The sheer size of the datasets is growing exponentially, often ranging from terabytes to petabytes. This is complicating traditional data management strategies.
• Velocity: The increasing speed at which data is being collected and processed is also causing complications. Big data is often time sensitive and needs to be captured in real time as it is streaming into the enterprise.

*Build a Strategy for Big Data Platforms

Below is a summary of concerns regarding core mainframe skills:

1. System Management (System Programmers): This is the most critical and hard-to-replace skill since it requires in-depth low-level knowledge of the mainframe (e.g. at the MVS level). These are skills that are generally not taught anymore, so there is a limited pool of experienced system programmers.
2. Information Management System (IMS) Specialists: Requires a combination of mainframe knowledge and data analysis skills, which makes this a rare skill set. This is becoming more critical as business intelligence takes on an ever-increasing focus in most organizations.
3. Application Development: The primary concern here is a shortage of developers skilled in older languages such as COBOL. It should be noted that this is an application issue; for example, this is not solved by migrating off mainframes.
4. Mainframe Operators: This is an easier skill set to learn, and there are several courses and training programs available. An IT person new to mainframes could learn this position in about six weeks of on-the-job training.
5. DB2 Administration: Advances in database technology have simplified administration (not just for DB2 but also other database products). As a result, as with mainframe operators, this is a skill set that can be learned in a short period of time on the job.

The Challenge

An aging workforce, specialized skills, and high salary expectations

• Mainframe specialists, such as system programmers and IMS specialists, are typically over 50, have a unique skill set, and are tasked with running mission-critical systems.

The In-House Solution:

Build your mentorship program to create a viable succession plan

• Get your money’s worth out of your experienced staff by having them train others.
• Operator skills take about six weeks to learn. However, it takes about two years before a system programmer trainee can become fully independent. This is similar to the learning curve for other platforms; however, this is a more critical issue for mainframes since organizations have far fewer mainframe specialists to fall back on when senior staff retire or move on.

There are several challenges to overcome in a migration project, from finding an appropriate alternative platform to rewriting legacy code. Many organizations have incurred huge costs in the attempt, only to be unsuccessful in the end, so make this decision carefully.

Organizations often have highly sensitive data on their mainframes (e.g. financial data), so many of these organizations are reluctant to have this data live outside of their four walls. However, the convenience of using a hosting provider makes this an attractive option to consider.

The most common tactic is for the organization to adopt some level of outsourcing for the non-commodity platform, retaining the application support/development in-house.

A customer can “re-platform” the non-commodity workload into public cloud offerings or in a few offerings
“re-host.”

If you’re staying with the mainframe and keeping it in-house, it’s important to continue to invest in this platform, keep it current, and look for opportunities to optimize its value.

If this sounds like your organization, it’s time to do the analysis so you can decide and get clarity on the future of the mainframe in your organization.

1. Identify current performance, availability, and security requirements. Assess alternatives based on this criteria.
2. Review and use Info-Tech’s Mainframe TCO Comparison Tool to compare mainframe costs to the potential alternative platform.
3. Assess the business risks and benefits. Can the alternative deliver the same performance, reliability, and security? If not, what are the risks? What do you gain by migrating?
4. If migration is still a go, evaluate the following:

• Do you have the expertise or a reliable third party to perform the migration, including code rewrites?
• How long will the migration take? Can the business function effectively during this transition period?
• How much will the migration cost? Is the value you expect to gain worth the expense?

*3 of the top 4 challenges related to shortfalls of alternative platforms

*Source: Maximize the Value of IBM Mainframes in My Business

The most common tactic is for the organization to adopt some level of outsourcing for the non-commodity platform, retaining the application support/development in-house.

The options here for the non-commodity (z/Series, IBM Power platforms, for example) are not as broad as with commodity server platforms. More confusingly, the term “outsourcing” for these can include:

Traditional/Colocation – A customer transitions their hardware environment to a provider’s data center. The provider can then manage the hardware and “system.”

Onsite Outsourcing – Here a provider will support the hardware/system environment at the client’s site. The provider may acquire the customer’s hardware and provide software licenses. This could also include hiring or “rebadging” staff supporting the platform. This type of arrangement is typically part of a larger services or application transformation. While low risk, it is not as cost-effective as other deployment models.

Managed Hosting – A customer transitions their legacy application environment to an off-prem hosted multi-tenanted environment. It will provide the most cost savings following the transition, stabilization, and disposal of existing environment. Some providers will provide software licensing, and some will also support “Bring Your Own,” as permitted by IBM terms for example.

Info-Tech Insight

Technical debt for non-commodity platforms isn’t only hardware based. Moving an application written for the mainframe onto a “cheaper” hardware platform (or outsourced deployment) leaves the more critical problems and frequently introduces a raft of new ones.

While the majority of the coded functionality (JCLs, programs, etc.) migrate easily, there will be a need to re-code or re-write objects – especially if any object, code, or location references are not exactly the same in the new environment.

Micro Focus has solid experience in this but if consider it within the context of an 80/20 rule (the actual metrics might be much better than that), meaning that some level of rework would have to be accomplished as an overhead to the exercise.

Build that thought into your thinking and business case.

AWS Cloud

• Astadia (an AWS Partner) is re-platforming mainframe workloads to AWS. With its approach you reuse the original application source code and data to AWS services. Consider reviewing Amazon’s “Migrating a Mainframe to AWS in 5 Steps.”

Azure Cloud

• Azure has the Azure Mainframe Migration Center.
• Review Microsoft’s “Mainframe workloads supported on Azure.”

Micro Focus COBOL (Visual COBOL)

• Micro Focus' Visual COBOL also supports running COBOL in Docker containers and managing and orchestrating the containers with Kubernetes. I personally cannot imagine what sort of drunken bender decision would lead me to move COBOL into Docker and then use Kubernetes to run in GCP but there you are...if that's your Jam you can do it.

Yeah, a complication for this situation is the legacy code.

While re-platforming/re-hosting non-COBOL code is not new, we have not had many member observations compared to the re-platforming/re-hosting of COBOL functionality initiatives.

That being said, there are a couple of interesting opportunities to explore.

NTT Data Services (GLOBAL)

• Most intriguing is the re-hosting of a mainframe environment into AWS. Not sure if the AWS target supports NATURAL codebase; it does reference Adabas however (Re-Hosting Mainframe Applications to AWS with NTT DATA Services). Nevertheless, NTT has supported re-platforming and NATURAL codebase environments previously.

ModernSystems (or ModSys) has relevant experience.

• ModSys is the resulting entity following a merger between BluePhoenix and ATERAS a number of years ago. ATERAS is the entity I find references to within my “wayback machine” for member discussions. There are also a number of published case studies still searchable about ATERAS’ successful re-platforming engagements, including the California Public Employees Retirement System (CalPERS) most famously after the Accenture project to rewrite it failed.

ATOS, as a hosting vendor mostly referenced by customers with global locations in a short-term transition posture, could be an option.

Lastly, the other Managed Services vendors with NATURAL and Adabas capabilities:

• HCL
• WIPRO

*92% of organizations that added capacity said TCO is lower than for commodity servers (compared to 50% of those who did not add capacity)

*63% of organizations that added capacity said finding resources is not very difficult (compared to 42% of those who did not add capacity)

• While the application will be replaced and re-platformed, there is the historical VIN data remaining in the VSAM files and access via the application. The challenge is that a bulk conversion can add upfront costs and delay the re-platforming of the application functionality. Some shops will break the historical data migration into a couple of phases.
• While there are technical solutions to accessing VSAM data stores, what I have observed with other members facing a similar scenario is a need to “shrink” the data store over time. The technical accesses to historical VSAM records would also have a lifespan, and rather than kicking the can down the road indefinitely, many have turned to a process-based solution allowing them to shrink the historical data store over time. I have observed three approaches to the handling or digitization of historical records like this:

Temporary workaround. This would align with a technical solution allowing the VASM files to be accessed using platforms other than on mainframe hardware (Micro Focus or other file store trickery). This can be accomplished relatively quickly but does run the risk of technology obsolesce for the workaround at some point in the future.

Bulk conversion. This method would involve the extract/transform/load of the historical records into the new application platform. Often the order of the conversion is completed on work newest to oldest (the idea is that the newest historical records would have the highest likelihood of an access need), but all files would be converted to the new application and the old data store destroyed.

Forward convert, which would have files undergo the extract/transform/load conversion into the new application as they are accessed or reopened. This method would keep historical records indefinitely or until they are converted – or the legal retention schedule allows for their destruction (hopefully no file must be kept forever). This could be a cost-efficient approach since the historical files remaining on the VSAM platform would be shrunk over time based on demand from the district attorney process. The conversion process could be automated and scripted, with a QR step allowing for the records to be deleted from the old platform.

Info-Tech Insight

It is not usual for organizations to leverage options #2 and #3 above to move the functionality forward while containing the scope creep and costs for the data conversions.

• Automatic restart and recovery
• File management
• Integration with security systems such as AD
• Operator alerts
• Ability to control spooling devices
• Cross-platform support
• Cyclical scheduling
• Deadline scheduling
• Event-based scheduling / triggers
• Inter-dependent jobs
• External task monitoring (e.g. under other sub-systems)
• Multiple calendars and time-zones
• Scheduling of packaged applications (such as SAP, Oracle, JD Edwards)
• The ability to schedule web applications (e.g. .net, java-based)
• Workload analysis

• Conditional dependencies
• Critical process monitoring
• Event-based automation (“self-healing” processes in response to common defined error conditions)
• Graphical job stream/workflow visualization
• Alerts (job failure notifications, task thresholds (too long, too quickly, missed windows, too short, etc.) via multiple channels
• API’s supporting programmable scheduler needs
• Virtualization support
• Workload forecasting and workload planning
• Logging and message data supporting auditing capabilities likely to be informed by or compliant with regulatory needs such as Sarbanes, Gramme-Leach
• Historical reporting
• Auditing reports and summaries

✓ Advanced Systems Concepts

✓ BMC

✓ Broadcom

✓ HCL

✓ Fortra

✓ Redwood

✓ SMA Technologies

✓ StoneBranch

✓ Tidal Software

✓ Vinzant Software

Summary

Founded in 1981, ASCs ActiveBatch “provides a central automation hub for scheduling and monitoring so that business-critical systems, like CRM, ERP, Big Data, BI, ETL tools, work order management, project management, and consulting systems, work together seamlessly with minimal human intervention.”*

URL

advsyscon.com

Coverage:

Global

Amazon EC2

Hadoop Ecosystem

IBM Cognos

DataStage

IBM PureData (Netezza)

Informatica Cloud

Microsoft Azure

Microsoft Dynamics AX

Microsoft SharePoint

Microsoft Team Foundation Server

Oracle EBS

Oracle PeopleSoft

SAP

BusinessObjects

ServiceNow

Teradata

VMware

Windows

Linux

Unix

IBM i

Workload Management:

Summary

Founded in 1980, BMCs Control-M product “simplifies application and data workflow orchestration on premises or as a service. It makes it easy to build, define, schedule, manage, and monitor production workflows, ensuring visibility, reliability, and improving SLAs.”*

URL

bmc.com/it-solutions/control-m.html

Coverage:

Global

AWS

Azure

Google Cloud Platform

Cognos

IBM InfoSphere

DataStage

SAP HANA

Oracle EBS

Oracle PeopleSoft

BusinessObjects

ServiceNow

Teradata

VMware

Windows

Linux

Unix

IBM i

IBM z/OS

zLinux

Workload Management:

Summary

Broadcom offers Atomic Automation and Autosys Workload Automation which ”gives you the agility, speed and reliability required for effective digital business automation. From a single unified platform, Atomic centrally provides the orchestration and automation capabilities needed accelerate your digital transformation and support the growth of your company.”*

URL

broadcom.com/products/software/automation/automic-automation

broadcom.com/products/software/automation/autosys

Coverage:

Global

Windows

MacOS

Linux

UNIX

AWS

Azure

Google Cloud Platform

VMware

z/OS

zLinux

System i

OpenVMS

Banner

Ecometry

Hadoop

Oracle EBS

Oracle PeopleSoft

SAP

BusinessObjects

ServiceNow

Teradata

VMware

Windows

Linux

Unix

IBM i

Workload Management:

Summary

“HCL Workload Automation streamlined modelling, advanced AI and open integration for observability. Accelerate the digital transformation of modern enterprises, ensuring business agility and resilience with our latest version of one stop automation platform. Orchestrate unattended and event-driven tasks for IT and business processes from legacy to cloud and kubernetes systems.”*

URL

hcltechsw.com/workload-automation

Coverage:

Global

Windows

MacOS

Linux

UNIX

AWS

Azure

Google Cloud Platform

VMware

z/OS

zLinux

System i

OpenVMS

IBM SoftLayer

IBM BigInsights

IBM Cognos

Hadoop

Microsoft Dynamics 365

Microsoft Dynamics AX

Microsoft SQL Server

Oracle E-Business Suite

PeopleSoft

SAP

ServiceNow

Apache Oozie

Informatica PowerCenter

IBM InfoSphere DataStage

Salesforce

BusinessObjects BI

IBM Sterling Connect:Direct

IBM WebSphere MQ

IBM Cloudant

Apache Spark

Workload Management:

Summary

Fortra’s “JAMS is a centralized workload automation and job scheduling solution that runs, monitors, and manages jobs and workflows that support critical business processes.

JAMS reliably orchestrates the critical IT processes that run your business. Our comprehensive workload automation and job scheduling solution provides a single pane of glass to manage, execute, and monitor jobs—regardless of platforms or applications.”*

URL

jamsscheduler.com

Coverage:

Global

OpenVMS

OS/400

Unix

Windows

z/OS

SAP

Oracle

Microsoft

Infor

Workday

AWS

Azure

Google Cloud Compute

ServiceNow

Salesforce

Micro Focus

Microsoft Dynamics 365

Microsoft Dynamics AX

Microsoft SQL Server

MySQL

NeoBatch

Netezza

Oracle PL/SQL

Oracle E-Business Suite

PeopleSoft

SAP

SAS

Symitar

Workload Management:

Summary

Founded in 1993 and delivered as a SaaS solution, ”Redwood lets you orchestrate securely and reliably across any application, service or server, in the cloud or on-premises, all inside a single platform. Automation solutions are at the core of critical business operations such as forecasting, replenishment, reconciliation, financial close, order to cash, billing, reporting, and more. Enterprises in every industry — from manufacturing, utility, retail, and biotech to healthcare, banking, and aerospace.”*

URL

redwood.com

Coverage:

Global

OpenVMS

OS/400

Unix

Windows

z/OS

SAP

Oracle

Microsoft

Infor

Workday

AWS

Azure

Google Cloud Compute

ServiceNow

Salesforce

Github

Office 365

Slack

Dropbox

Tableau

Informatica

SAP BusinessObjects

Cognos

Microsoft Power BI

Amazon QuickSight

VMware

Xen

Kubernetes

Workload Management:

Summary

“Robot Schedule’s workload automation capabilities allow users to automate everything from simple jobs to complex, event-driven processes on multiple platforms and centralize management from your most reliable system: IBM i. Just create a calendar of when and how jobs should run, and the software will do the rest.”*

URL

fortra.com/products/job-scheduling-software-ibm-i

Coverage:

Global

IBM i (System i, iSeries, AS/400)

AIX/UNIX

Linux

Windows

SQL/Server

Domino

JD Edwards EnterpriseOne

SAP

Automate Schedule (formerly Skybot Scheduler)

Workload Management:

Summary

Founded in1980, SMA offers to “save time, reduce error, and free your IT staff to work on more strategic contributions with OpCon from SMA Technologies. OpCon offers powerful, easy-to-use workload automation and orchestration to eliminate manual tasks and manage workloads across business-critical operations. It's the perfect fit for financial institutions, insurance companies, and other transactional businesses.”*

URL

smatechnologies.com

Coverage:

Global

Windows

Linux

Unix

z/Series

IBM i

Unisys

Oracle

SAP

Microsoft Dynamics AX

Infor M3

Sage

Cegid

Temenos

FICS

Microsoft Azure Data Management

Microsoft Azure VM

Amazon EC2/AWS

Web Services RESTful

Docker

Google Cloud

VMware

ServiceNow

Commvault

Microsoft WSUS

Microsoft Orchestrator

Java

JBoss

Asysco AMT

Tuxedo ART

Nutanix

Corelation

Symitar

Fiserv DNA

Fiserv XP2

Workload Management:

Summary

Founded in 1999, ”the Stonebranch Universal Automation Center (UAC) is an enterprise-grade business automation solution that goes beyond traditional job scheduling. UAC's event-based workload automation solution is designed to automate and orchestrate system jobs and tasks across all mainframe, on-prem, and hybrid IT environments. IT operations teams gain complete visibility and advanced control with a single web-based controller, while removing the need to run individual job schedulers across platforms.”*

URL

stonebranch.com/it-automation-solutions/enterprise-job-scheduling

Coverage:

Global

Windows

Linux

Unix

z/Series

Apache Kafka

AWS

Databricks

Docker

GitHub

Google Cloud

Informatica

Jenkins

Jscape

Kubernetes

Microsoft Azure

Microsoft SQL

Microsoft Teams

PagerDuty

PeopleSoft

Petnaho

RedHat Ansible

Salesforce

SAP

ServiceNow

Slack

SMTP and IMAP

Snowflake

Tableau

VMware

Workload Management:

Summary

Founded in 1979, Tidal’s Workload Automation will “simplify management and execution of end-to-end business processes with our unified automation platform. Orchestrate workflows whether they're running on-prem, in the cloud or hybrid environments.”*

URL

tidalsoftware.com

Coverage:

Global

CentOS

Linux

Microsoft Windows Server

Open VMS

Oracle Cloud

Oracle Enterprise Linux

Red Hat Enterprise Server

Suse Enterprise

Tandem NSK

Ubuntu

UNIX

HPUX (PA-RISC, Itanium)

Solaris (Sparc, X86)

AIX, iSeries

z/Linux

z/OS

Amazon AWS

Microsoft Azure

Oracle OCI

Google Cloud

ServiceNow

Kubernetes

VMware

Cisco UCS

SAP R/3 & SAP S/4HANA

Oracle E-Business

Oracle ERP Cloud

PeopleSoft

JD Edwards

Hadoop

Oracle DB

Microsoft SQL

SAP BusinessObjects

IBM Cognos

FTP/FTPS/SFTP

Informatica

Workload Management:

Summary

Founded in 1987, Global ECS can “simplify operations in all areas of production with the GECS automation framework. Use a single solution to schedule, coordinate and monitor file transfers, database operations, scripts, web services, executables and SAP jobs. Maximize efficiency for all operations across multiple business units intelligently and automatically.”*

URL

vinzantsoftware.com

Coverage:

Global

Windows

Linux

Unix

iSeries

SAP R/3 & SAP S/4HANA

Oracle, SQL/Server

The Scale Up vs. Scale Out TCO Tool provides organizations with a framework for estimating the costs associated with purchasing and licensing for a scale-up and scale-out environment over a multi-year period.

Use this tool to:

• Compare the pre-populated values.
• Insert your own amounts to contrast possible database decisions and determine the TCO of each.

Info-Tech Insight

Watch out for inaccurate financial information. Ensure that the financials for cost match your maintenance and contract terms.

Use the Scale Up vs. Scale Out TCO Tool to determine your TCO options.

Darin Stahl, Principal Research Advisor, Info-Tech Research Group

Darin is a Principal Research Advisor within the Infrastructure Practice, and leveraging 38+ years of experience, his areas of focus include: IT Operations Management, Service Desk, Infrastructure Outsourcing, Managed Services, Cloud Infrastructure, DRP/BCP, Printer Management, Managed Print Services, Application Performance Monitoring/ APM, Managed FTP, non-commodity servers (z/Series, mainframe, IBM i, AIX, Power PC).

Troy Cheeseman, Practice Lead, Info-Tech Research Group

Troy has over 25 years of IT management experience and has championed large enterprise-wide technology transformation programs, remote/home office collaboration and remote work strategies, BCP, IT DRP, IT Operations and expense management programs, international right placement initiatives, and large technology transformation initiatives (M&A). Additionally, he has deep experience working with IT solution providers and technology (cloud) start-ups.