Enterprise Storage Solution Considerations

Enterprise Storage Solution Considerations

Narrow your focus with the right product type and realize efficiencies.

Analyst Perspective

The vendor landscape is continually evolving, as are the solutions they offer. The options and features are increasing and appealing.

To say that the current enterprise storage landscape looks interesting would be an understatement. The solutions offered by vendors continue to grow and evolve. Flash and NVMe are increasing the speed of storage media and reducing latency. Software-defined storage is finding the most efficient use of media to store data where it is best served while managing a variety of vendor storage and older storage area networks and network-attached storage devices. Storage as a service is taking on a new meaning with creative solutions that let you keep the storage appliance on premises or in a colocated data center while administration, management, and support are performed by the vendor for a nominal monthly fee. We cannot discuss enterprise storage without mentioning the cloud. Bring a thermometer because you must understand the difference between hot, warm, and cold storage when discussing the cloud options. Very hot and very cold may also come into play. Storage hardware can assume a higher total cost of ownership with support options that replace the controllers on a regular basis. The options with this type of service are also varied, but the concept of not having to replace all disks and chassis nor go through a data migration is very appealing to many companies. The cloud is growing in popularity when it comes to enterprise storage, but on-premises solutions are still in demand, and whether you choose cloud or on premises, you can be guaranteed an array of features and options to add stability, security, and efficiency to your enterprise storage. P.J. Ryan Research Director, Infrastructure & Operations Info-Tech Research Group

Executive Summary

Info-Tech Insight

Enterprise storage has evolved, with more options than ever

Data is growing, data security will always be a concern, and vendors are providing more and more options for enterprise storage.

“By 2025, it’s estimated that 463 exabytes of data will be created each day globally – that’s the equivalent of 212,765,957 DVDs per day!” (Visual Capitalist)

“Modern criminal groups target not only endpoints and servers, but also central storage systems and their backup infrastructure.” (Continuity Software)

Cloud or on premises? Maybe a hybrid approach with both cloud and on premises is best for you. Do you want to remove the headaches of storage administration, management, and support with a fully managed storage-as-a-service solution? Would you like to upgrade your controllers every three or four years without a major service interruption? The options are increasing and appealing.

High-Level Considerations

1. Understand Your Data

Understand how much data you have and where it is located. This will be crucial when evaluating enterprise storage solutions.

2. Plan for Growth

Your enterprise storage considerations should include your data needs now and in the future.

3. Understand the Mechanics

Take the time to understand the various data storage formats, disk types, and associated technology, as well as the cloud-based and on-premises options. This will help you select the right tool for your needs.

Storage formats, disk drives, and technology

Common data storage formats, technology, and drive types are outlined below. Understanding how data is stored as well as the core building blocks for larger systems will help you decide which solution is best for your storage needs.

Narrow your focus with the right product type

On-premises enterprise storage solutions fit into a few distinct product types.

Cloud storage

• Cloud storage is online storage offered by a cloud provider. Cloud storage is available almost anywhere and is set up with high availability features such as data duplication, redundancy, backup, and power failure protection.
• Cloud storage is very scalable and typically is offered as object storage, block storage, or file storage. Cloud storage vendors may have their own naming scheme for object, block, or file storage.
• Cloud-hosted data is marketed according to the frequency of access and length of time in storage. There are typically three main levels of storage: hot, warm, or cold. Vendors may have their own naming convention for hot, warm, and cold storage. Some may also add more layers such as very hot or very cold.
  • Hot storage is for data that is frequently accessed and modified. It is available on demand and is the most costly of the storage levels.
  • Cold storage is for data that will sit for a long period of time and not need to be accessed. Cold storage is usually only available after several hours or days. Cold storage is very low cost and, in some cases, even free, but retrieval or restoration for the free services can be costly.
  • Warm storage sits in between hot and cold storage. It is for data that is infrequently needed. The cost of warm storage is also in between hot and cold storage costs, and access times are measured in terms of minutes or hours.
  • It is not uncommon for data to start in hot storage and, as it ages, move to warm and eventually cold storage.

“Enterprise cloud storage offers nearly unlimited scalability. Enterprises can add storage quickly and easily as it is needed, eliminating the risk and cost of over-provisioning.”

– Spectrum Enterprise

“Hot data will operate on fresh data. Cold data will operate on less frequent data and [is] used mainly for reporting and planning. Warm data is a balance between the two.”

– TechBlost

Enterprise storage features

The features listed below, while not intended to cover all features offered by all vendors, should be considered and could act as a baseline for discussions with storage providers when evaluating enterprise storage offerings.

What’s new in enterprise storage

• Data warehouses are not a new concept, but the data storage evolution and growth of data means that data lakes and data lakehouses are growing in popularity.
  • “A data lake is a centralized repository that allows you to store all your structured and unstructured data at any scale. You can store your data as-is, without having to first structure the data” (Amazon Web Services).
  • Analytics with a data lake is possible, but manipulation of the data is hindered due to the nature of the data. A data lakehouse adds data management and analytics to a data lake, similar to the data warehouse functionality added to databases.
• Options for on-premises hardware support is changing.
  • Pure Storage was the first to shake up the SAN support model with its Evergreen support option. Evergreen//Forever support allows for storage controller upgrades without having to migrate data or replace your disks or chassis (Pure Storage).
  • In response to the Pure Storage Evergreen offering, Dell, HPE, NetApp, and others have come out with similar programs that offer controller upgrades while maintaining the data, disks, and chassis.
• “As a service” is available as a hybrid solution.
  • Storage as a service (STaaS) originally referred to hosted, fully cloud-based offerings without the need for any on-premises hardware.
  • The latest STaaS offerings provide on-premises or colocated hardware with pay-as-you-go subscription pricing for data consumption. Administration, management, and support are included. The vendor will supply support and manage everything on your behalf.
  • Most of the major storage vendors offer a variation of storage as a service.

“Because data lakes mostly consist of raw unprocessed data, a data scientist with specialized expertise is typically needed to manipulate and translate the data.”

– DevIQ

“A Lakehouse is also a type of centralized data repository, integrated from heterogeneous sources. As can be expected from its name, It shares features with both datawarehouses and data lakes.”

– Cesare

“Storage as a service (STaaS) eliminates Capex, simplifies management and offers extensive flexibility.”

– TechTarget

Major vendors

The current vendor landscape for enterprise storage solutions represents a range of industry veterans and the brands they’ve aggregated along the way, as well as some relative newcomers who have come to the forefront within the past ten years.

Vendors like Dell EMC and HPE are longstanding veterans of storage appliances with established offerings and a back catalogue of acquisitions fueling their growth. Others such as Pure Storage offer creative solutions like all-flash arrays, which are becoming more and more appealing as flash storage becomes more commoditized.

Cloud-based vendors have become popular options in recent years. Cloud storage provides many options and has attracted many other vendors to provide a cloud option in addition to their on-premises solutions. Some software and hardware vendors also partner with cloud vendors to offer a complete solution that includes storage.

Info-Tech Insight

Explore your current vendor’s solutions as a starting point, then use that understanding as a reference point to dive into other players in the market

Key Players

• Amazon
• Cisco
• Dell EMC
• Google
• Hewlett Packard Enterprise
• Hitachi Vantara
• IBM
• Microsoft
• NetApp
• Nutanix
• Pure Storage

Enterprise Storage Use Cases

Block, object, or file storage? NAS, SAN, SDS, or HCI? Cloud or on prem? Hot, warm, or cold?
Which one do you choose?
The following use cases based on actual Info-Tech analyst calls may help you decide.

1. Offsite backup solution
2. Infrastructure consolidation
3. DR/BCP datacenter duplication
4. Expansion of existing storage
5. Complete backup solution
6. Existing storage solution going out of support soon
7. Video storage
8. Classify and offload storage

Offsite backup solution

“Offsite” may make you think of geographical separation or even cloud-based storage, but what is the best option and why?

Use Case: How a manufacturing company dealt with retired applications

• A leading manufacturing company had to preserve older applications no longer in use.
• The company had completed several acquisitions and ended up with multiple legacy applications that had been merged or migrated into replacement solutions. These legacy applications were very important to the original companies, and although the data they held had been migrated to a replacement solution, executives felt they should hold on to these applications for a period of time, just in case.
• A modern archiving solution was considered, but a research advisor from Info-Tech Research joined a call with the manufacturing company and helped the client realize that the solution was a modified backup. The application data had already been preserved through the migration, so data could be accessed in the production environment.
• The data could be exported from the legacy application into a nonsequential database, compressed, and stored in cloud-based cold storage for less than $5 per terabyte per month. The manufacturing company staff realized that they could apply this same approach to several of their legacy applications and save tens of thousands of dollars in the process.
• Cold storage is inexpensive until you start retrieving that data frequently. The manufacturing company knew they did not have a requirement to retrieve the application and data for a very long time, so cloud-based cold storage was ideal.

“Data retrieval from cold storage is harder and slower than it is from hot storage. … Because of the longer retrieval time, online cold storage plans are often much cheaper. … The downside is that you’d incur additional costs when retrieving the data.”

– Ben Stockton, Cloudwards

Infrastructure consolidation

Hyperconverged infrastructure combines storage, virtual infrastructure, and associated management into one piece of equipment.

Use Case: How one company dealt with equipment and storage needs

• One Info-Tech client had recently started in the role of IT director and realized he had inherited aging infrastructure along with a serious data challenge. The storage appliances were old and out of support. The appliances were performing inadequately, and the client was in need of more data due to ongoing growth, but he also realized that the virtual environment was running on very old servers that were no longer supported. The IT director reached out to Info-Tech to find solutions to the virtualization challenge, but the storage problem also came up throughout the course of the conversation with an analyst.
• The analyst quickly realized that the IT director was an ideal candidate for a hyperconverged infrastructure (HCI) storage solution, which would also provide the necessary virtual environment.
• The analyst explained the benefits of having a single appliance that would provide virtualization needs as well as storage needs. The built-in management features would ease the burden of administration, and the software-defined nature of the HCI would allow for the migration of data as well as future expansion options.
• Hyperconverged infrastructure is offered by many vendors under a variety of names. Most are similar but some may have a better interface or other features. The expansion process is simple, and HCI is a good fit for many organizations looking to consolidate virtual infrastructure and storage.

“HCI environments use a hypervisor, usually running on a server that uses direct-attached storage (DAS), to create a data center pool of systems and resources.”

– Samuel Greengard, Datamation

Datacenter duplication

SAN providers offer a varied range of options for their products, and those options are constantly evolving.

Use Case: Independent school district provides better data access using SAN technology

• An independent school district was expanding by adding a second data center in a new school. This new data center would be approximately 20 miles away from the original data center used by the district. The intent was not to replace the original data center but to use both centers to store data and provide services concurrently. The district’s ideal scenario would be that users would not know or care which data center they were reaching, and there would be no difference in the service received from each data center. The school district reached out to Info-Tech when planning discussions reached the topic of data duplication and replication software.
• An Info-Tech analyst joined a call with the school district and guided the conversation toward the existing environment to understand what options might be available. The analyst quickly discovered that all the district’s servers were virtual, and all associated data was stored on a single SAN.
• The analyst informed the school district staff about SAN options, including SAN-to-SAN replication. If the school district had a sufficient link between the two data centers, SAN-to-SAN replication would work for them and provide the two identical copies of data at two locations.
• The analyst continued to offer explanations of other features that some vendors offer with their SANs, such as the ability to turn on or off deduplication and compression, as well as disk options such as flash or NVMe.
• The school district was moving to the request for proposal (RFP) stage but hoped to have SAN-to-SAN replication implemented before the next academic year started.

“SAN-to-SAN replication is a low-cost, highly efficient way to manage mounting quantities of stored data.”

– Secure Infrastructure & Services

Expansion of existing storage

That old storage area network may still have some useful life left in it.

Use Case: Municipality solves data storage aging and growth challenge

• A municipality in the United States reached out to Info-Tech for guidance on its storage challenge. The municipality had accumulated multiple SANs from different vendors over the years. These SANs were running out of storage, and more data storage was needed. The municipality’s data was growing at a rapid pace, thanks to municipal growth and expansion of services. The IT team was also concerned with modernizing their storage and not hindering their long-term growth by making the wrong purchase decision for their current storage needs.
• An analyst from Info-Tech discussed several options with the municipality but in the end advised that software-defined storage may be the best solution.
• Software-defined storage (SDS) would allow the municipality to gain better visibility into existing storage while making more efficient use of existing and new storage. SDS could take over the management of the existing storage from multiple vendors and add additional storage as required. SDS would also be able to integrate cloud-based storage if that was the direction taken by the municipality in the future.
• The municipality moved forward with an SDS solution and added some additional storage capacity. They used some of their existing SANs but retired the more troublesome ones. The SDS system managed all the storage instances and data management. The administration of the storage environment was easier for the storage admins, and long-term savings were achieved through better storage management.

“Often enterprises have added storage on an ad hoc basis as they needed it for various applications. That can result in a mishmash of heterogenous storage hardware from a wide variety of vendors. SDS offers the ability to unify management of these different storage devices, allowing IT to be more efficient.”

– Cynthia Harvey, Enterprise Storage Forum (“What Is Software Defined Storage?”, 2018)

Complete backup solution

Many backup software solutions can provide backups to multiple locations, making two-location backups simple.

Use Case: How an oil refinery modernized its backup solution

• A large oil refinery needed a better solution for the storage of backups. The refinery was replacing its backup software solution but also wanted to improve the backup storage situation and move away from tape-based storage. All other infrastructure was reasonably modern and not in need of replacement at this time.
• A research analyst from Info-Tech helped the client realize that the solution was a modified backup. The general guidance for backups is have a least one copy offsite, so the cloud was the obvious focal point. The analyst also explained that it would be beneficial to have a recent copy of the backup available on site for common restoration requests in addition to having the offsite copy for disaster recovery (DR) purposes.
• The refinery staff conducted a data analysis to determine how much data was being backed up on a daily basis. The solution proposed by the analyst included network-attached storage (NAS) with adequate storage to hold 30 days' worth of on-premises data. The backup software would also simultaneously copy each backup to a cloud-based storage repository. The backup software was smart enough to only back up and transfer data that had changed since the previous backup, so transfer time and capacity was not a factor.
• The NAS would allow for the restoration of any local, on-premises data while the cloud storage would provide a safe location offsite for backup data. It could also serve as the backup location for other cloud-based services that required a backup.

“Data protection demands that enterprises have multiple methods of keeping data safe and replicating it in case of disaster or loss.”

– Drew Robb, Enterprise Storage Forum, 2021

Storage going out of support

SAN solutions have come a long way with improvements in how data is stored and what is used to store the data.

Use Case: How one organization replaced its old storage with a similar solution

• A government organization was looking for a solution for its aging storage area network appliances. The SANs were old and would be no longer supported by the manufacturer within four months. The SANs had slower spinning disks and their individual capacity was at its limit through the addition of extra shelves and disks over the years.
• The organization reached out to Info-Tech for guidance. An analyst arranged a call with them, and they discussed the storage situation in detail, including desired benefits from a storage solution and growth requirements. They also discussed cloud storage, but the government organization was not in a position to move its data to the cloud for a variety of reasons.
• Although the individual SANs were at their storage capacity limit, the total amount of data was well within the limits of many modern on-premises storage solutions. SSD and flash or NVMe storage can store large amounts of data in small footprints and form factors.
• The analyst reviewed several vendors with the client and discussed some advantages and disadvantages of each. They explored the features offered as well as scalability options.
• SANs have been around for a long time but the features and capabilities that come with them has evolved. They are still a very viable solution for many organizations in a variety of scenarios.

“A rapidly growing portion of SAN deployments leverages all-flash storage to gain its high performance, consistent low latency, and lower total cost when compared to spinning disk.”

– NetApp

Video storage

Cloud storage would not be sufficient if you were using a dial up connection, just as on-premises storage solutions would not suffice if they were using floppy disks.

Use Case: Body cams and public cameras in municipalities are driving storage growth

• Municipal law enforcement agencies are wearing body cameras more frequently, for their own protection as well as for the protection of the public. Camera footage can be useful in legal situations as well. Municipalities are also installing more and more public cameras for the purposes of public safety. The recorded video footage from these cameras can result in large data files, which in turn drive data storage requirements.
• Info-Tech analysts are joining calls about video data storage with increasing frequency. The concerns are repetitive, and the guidance is similar on most of these calls.
• The “object” storage format is ideal for video and media data. Most cloud-based storage solutions use object storage, but it is also available with on-premises solutions such as NAS or SAN. The challenges clients are expressing are typically related to inadequate bandwidth for cloud-based storage or other storage formats instead of “object” storage. Cloud-based storage can also grow beyond the budgeted numbers, causing an increase in the monthly cloud cost. Older, slower on-premises hardware sometimes reveals itself as the latency culprit.
• Object storage is well suited for the unstructured data that is video footage. It uses metadata to tag the video file for future retrieval and is easily expandable, which also makes it cost effective.
• Video data stored in a cloud-based repository will work fine as long as the bandwidth is adequate. On-premises storage of video data is also quite adequate on the right storage format, with fast disks and a reasonably up-to-date network infrastructure.

“The captured video is stored for days, weeks, months and sometimes years and consumes a lot of space. Data storage plays a new and important role in these systems. Object storage is ideal to store the video data.”

– Object-Storage.Info

Classify and offload primary storage

Some software products have storage options available as a result of agreements with other storage vendors. Several backup and archive software products fall into this category.

Use Case: Enterprise storage can help reduce data sprawl

• A large engineering firm was trying to manage its data sprawl. The team sampled a small percentage of their data and quickly realized that when they applied their findings on the 1% of data to their entire data estate, the sheer volume of personal files, older files, and unclassified data was going to be a challenge.
• They found a solution in archiving software. The archiving software would tag data based on several factors. The software would move older files away from primary storage to an alternate storage platform but still leave a stub of the moved file in place and maintain limited access to those files. This would reduce primary storage requirements and allow the firm to eliminate multiple file servers
• The engineering firm reached out to Info-Tech and participated in an analyst call. During that call, they laid out their plans, and the analyst made them aware of cloud storage. The positive and negative aspects of cloud storage were discussed, and the firm fully understood that the colder the storage tier, the slower the recovery. The firm's stance was if the files had not been accessed in the past six months, waiting a day or two for retrieval would not be a concern, and the firm was content with cold storage in the cloud.
• The firm had not purchased the archiving software at the time of the analyst call, and the analyst also explained to them that the archiving software may have an existing agreement with a cloud provider for storage options, which could be more cost effective than purchasing cloud storage separately.
• Cold cloud-based storage was the preferred solution for this firm, but this use case also highlights the option that some software products carry regarding storage. Several backup and archive products have a cloud storage option that should be investigated, as they may be cost-effective options.

“Cold storage is perfect for archiving your data. Online backup providers offer low-cost, off-site data backups at the expense of fast speeds and easy access, even though data retrieval often comes at an added cost. If you need to keep your data long-term, but don’t need to access it often, this is the kind of storage you need.”

– Ben Stockton, Cloudwards

Understand your data requirements

Activity

The first step in solving your enterprise storage challenge is identifying your data sources or drivers, data volume size, and growth rates. This information will give you insight into what data sources could be stored on premises or in the cloud, how much storage you will require for the coming five to ten years, and what to consider when exploring enterprise storage solutions.

• Info-Tech’s Modernize Enterprise Storage Workbook can be a valuable asset for determining your current storage drivers and future storage needs, structuring a plan for future storage purchases, and determining timelines and total cost of ownership.
• An example of the Storage Capacity Calculator tab from that workbook is displayed on the right. Using the Storage Capacity Requirements Calculator requires minimal steps.

1. Enter the current date and planning timeline (horizon) in months
2. Identify the top sources of data within the business – the current data drivers. Areas of focus could include business applications, file shares, backup, and archives.
3. For each of these data drivers, include your best estimate of:

• Current data volume
• Growth rate

• Initial data volumes
• Projected growth rates
• Planned implementation date

Download the Modernize Enterprise Storage Workbook and take the first step toward understanding your data requirements.

Download the Modernize Enterprise Storage Workbook

Related Info-Tech Research

Modernize Enterprise Storage

Current and emerging storage technologies are disrupting the status quo – prepare your infrastructure for the exponential rise in data and its storage requirements.

Modernize Enterprise Storage Workbook

This workbook will complement the discussions and activities found in the Modernize Enterprise Storage blueprint. Use this workbook in conjunction with the blueprint to develop a strategy for storage modernization.

Bibliography

Bakkianathan, Raghunathan. “What is the difference between Hot Warm and Cold data storage?” TechBlost, n.d.. Accessed 14 July 2022.
Cesare. “Data warehouse vs Data lake vs Lakehouse… and DeltaLake?“ Medium, 14 June 2021. Accessed 26 July 2022.
Davison, Shawn and Ryan Sappenfield. “Data Lake Vs Lakehouse Vs Data Mesh: The Evolution of Data Transformation.” DevIQ, May 2022. Accessed 23 July 2022.
Desjardins, Jeff. “Infographic: How Much Data is Generated Each Day?” Visual Capitalist, 15 April 2019. Accessed 26 July 2022.
Greengard, Samuel. “Top 10 Hyperconverged Infrastructure (HCI) Solutions.” Datamation, 22 December 2020. Accessed 23 July 2022.
Harvey, Cynthia. “Flash vs. SSD Storage: Is there a Difference?” Enterprise Storage Forum, 10 July 2018. Accessed 23 July 2022.
Harvey, Cynthia. “What Is Software Defined Storage? Features & Benefits.” Enterprise Storage Forum, 22 February 2018. Accessed 23 July 2022.
Hecht, Gil. “4 Predictions for storage and backup security in 2022.” Continuity Software, 09 January 2022. Accessed 22 July 2022.
Jacobi, Jonl. “NVMe SSDs: Everything you need to know about this insanely fast storage.” PCWorld, 10 March 2019. Accessed 22 July 2022
Pritchard, Stephen. “Briefing: Cloud storage performance metrics.” Computer Weekly, 16 July 2021. Accessed 23 July 2022
Robb, Drew. “Best Enterprise Backup Software & Solutions 2022.” Enterprise Storage Forum, 09 April 2021. Accessed 23 July 2022.
Sheldon, Robert. “On-premises STaaS shifts storage buying to Opex model.” TechTarget, 10 August 2020. Accessed 22 July 2022.
“Simplify Your Storage Ownership, Forever.” PureStorage. Accessed 20 July 2022.
Stockton, Ben. “Hot Storage vs Cold Storage in 2022: Instant Access vs Long-Term Archives.” Cloudwards, 29 September 2021. Accessed 22 July 2022.
“The Cost Savings of SAN-to-SAN Replication.” Secure Infrastructure and Services, 31 March 2016. Accessed 16 July 2022.
“Video Surveillance.” Object-Storage.Info, 18 December 2019. Accessed 25 July 2022.
“What is a Data Lake?” Amazon Web Services, n.d. Accessed 17 July 2022.
“What is enterprise cloud storage?” Spectrum Enterprise, n.d. Accessed 28 July 2022.
“What is SAN (Storage Area Network).” NetApp, n.d. Accessed 25 July 2022.
“What is software-defined storage?” RedHat, 08 March 2018. Accessed 16 July 2022.