Do you experience challenges with the following:
- Equipping IT operations processes to manage containers.
- Choosing the right container technology.
- Optimizing your infrastructure strategy for containers.
Our Advice
Critical Insight
- Plan ahead to ensure your container strategy aligns with your infrastructure roadmap. Before deciding between bare metal and cloud, understand the different components of a container management solution and plan for current and future infrastructure services.
- When selecting tools from multiple sources, it is important to understand what each tool should and should not meet. This holistic approach is necessary to avoid gaps and duplication of effort.
Impact and Result
Use the reference architecture to plan for the solution you need and want to deploy. Infrastructure planning and strategy optimizes the container image supply chain, uses your current infrastructure, and reduces costs for compute and image scan time.
Considerations to Optimize Container Management Research & Tools
Besides the small introduction, subscribers and consulting clients within this management domain have access to:
1. Considerations to Optimize Container Management Deck – A document to guide you design your container strategy.
A document that walks you through the components of a container management solution and helps align your business objectives with your current infrastructure services and plan for your future assets.
- Considerations to Optimize Container Management Storyboard
2. Container Reference Architecture – A best-of-breed template to help you build a clear, concise, and compelling strategy document for container management.
Complete the reference architecture tool to strategize your container management.
- Container Reference Architecture
Further reading
Considerations to Optimize Container Management
Design a custom reference architecture that meets your requirements.
Analyst Perspective
Containers have become popular as enterprises use DevOps to develop and deploy applications faster. Containers require managed services because the sheer number of containers can become too complex for IT teams to handle. Orchestration platforms like Kubernetes can be complex, requiring management to automatically deploy container-based applications to operating systems and public clouds. IT operations staff need container management skills and training.
Installing and setting up container orchestration tools can be laborious and error-prone. IT organizations must first implement the right infrastructure setup for containers by having a solid understanding of the scope and scale of containerization projects and developer requirements. IT administrators also need to know how parts of the existing infrastructure connect and communicate to maintain these relationships in a containerized environment. Containers can run on bare metal servers, virtual machines in the cloud, or hybrid configurations, depending on your IT needs
Nitin Mukesh
Senior Research Analyst, Infrastructure and Operations
Info-Tech Research Group
Executive Summary
| Your Challenge | Common Obstacles | Info-Tech’s Approach |
|
The container software market is constantly evolving. Organizations must consider many factors to choose the right container management software for their specific needs and fit their future plans. It's important to consider your organization's current and future infrastructure strategy and how it fits with your container management strategy. The container management platform you choose should be compatible with the existing network infrastructure and storage capabilities available to your organization. |
IT operations staff have not been thinking the same way as developers who have now been using an agile approach for some time. Container image builds are highly automated and have several dependencies including scheduling, testing, and deployment that the IT staff is not trained for or lack the ability to create anything more than a simple image. |
Use the reference architecture to plan for the solution you need and want to deploy. Infrastructure planning and strategy optimizes the container image supply chain and reduces costs for compute and image scan time. Plan ahead to ensure your container strategy aligns with your infrastructure roadmap. Before deciding between bare metal and cloud, understand the different components of a container management solution and plan for current and future infrastructure services. |
Your challenge
Choosing the right container technology: IT is a rapidly changing and evolving market, with startups and seasoned technology vendors maintaining momentum in everything from container platforms to repositories to orchestration tools. The rapid evolution of container platform components such as orchestration, storage, networking, and system services such as load balancing has made the entire stack a moving target.
However, waiting for the industry to be standardized can be a recipe for paralysis, and waiting too long to decide on solutions and approaches can put a company's IT operations in catch-up mode.
Keeping containers secure: Security breaches in containers are almost identical to operating system level breaches in virtual machines in terms of potential application and system vulnerabilities. It is important for any DevOps team working on container and orchestration architecture and management to fully understand the potential vulnerabilities of the platforms they are using.
Optimize your infrastructure strategy for containers: One of the challenges enterprise IT operations management teams face when it comes to containers is the need to rethink the underlying infrastructure to accommodate the technology. While you may not want to embrace the public cloud for your critical applications just yet, IT operations managers will need an on-premises infrastructure so that applications can scale up and down the same way as they are containerized.
Common ways organizations use containers
A Separation of responsibilities
Containerization provides a clear separation of responsibilities as developers can focus on application logic and dependencies, while IT operations teams can focus on deployment and management instead of application details such as specific software versions and configurations.
B Workload portability
Containers can run almost anywhere: physical servers or on-premise data centers on virtual machines or developer machines, as well as public clouds on Linux, Windows, or Mac operating systems, greatly easing development and deployment.
“Lift and shift” existing applications into a modern cloud architecture. Some organizations even use containers to migrate existing applications to more modern environments. While this approach provides some of the basic benefits of operating system virtualization, it does not provide all the benefits of a modular, container-based application architecture.
C Application isolation
Containers virtualize CPU, memory, storage, and network resources at the operating system level, providing developers with a logically isolated view of the operating system from other applications.
Source: TechTarget, 2021
What are containers and why should I containerize?
A container is a partially isolated environment in which an application or parts of an application can run. You can use a single container to run anything from small microservices or software processes to larger applications. Inside the container are all the necessary executable, library, and configuration files. Containers do not contain operating system images. This makes them lighter and more portable with much less overhead. Large application deployments can deploy multiple containers into one or more container clusters (CapitalOne, 2020).
Containers have the following advantages:
- Reduce overhead costs: Because containers do not contain operating system images, they require fewer system resources than traditional or hardware virtual machine environments.
- Enhanced portability: Applications running in containers can be easily deployed on a variety of operating systems and hardware platforms.
- More consistent operations: DevOps teams know that applications in containers run the same no matter where they are deployed.
- Efficiency improvement: Containers allow you to deploy, patch, or scale applications faster.
- Develop better applications: Containers support Agile and DevOps efforts to accelerate development and production cycles.
Source: CapitalOne, 2020
Container on the cloud or on-premise?
| On-premises containers | Public cloud-based containers |
|---|---|
|
Advantages:
Disadvantages:
|
Advantages:
Disadvantages:
|
Info-Tech Insight
Start-ups and small businesses that don't typically need to be closely connected to hardware can easily move (or start) to the cloud. Large (e.g. enterprise-class) companies and companies that need to manage and control local hardware resources are more likely to prefer an on-premises infrastructure. For enterprises, on-premises container deployments can serve as a bridge to full public cloud deployments or hybrid private/public deployments. The answer to the question of public cloud versus on premises depends on the specific needs of your business.
Container management
From container labeling that identifies workloads and ownership to effective reporting that meets the needs of different stakeholders across the organization, it is important that organizations establish an effective framework for container management.
Four key considerations for your container management strategy:
01 Container Image Supply Chain
How containers are built
02 Container Infrastructure and Orchestration
Where and how containers run together
03 Container Runtime Security and Policy Enforcement
How to make sure your containers only do what you want them to do
04 Container Observability
Runtime metrics and debugging
To effectively understand container management solutions, it is useful to define the various components that make up a container management strategy.
1: Container image supply chain
To run a workload as a container, it must first be packaged into a container image. The image supply chain includes all libraries or components that make up a containerized application. This includes CI/CD tools to test and package code into container images, application security testing tools to check for vulnerabilities and logic errors, registries and mirroring tools for hosting container images, and attribution mechanisms such as image signatures for validating images in registries.
Important functions of the supply chain include the ability to:
- Scan container images in registries for security issues and policy compliance.
- Verify in-use image hashes have been scanned and authorized.
- Mirror images from public registries to isolate yourself from outages in these services.
- Attributing images to the team that created them.
Source: Rancher, 2022
Info-Tech Insight
It is important to consider disaster recovery for your image registry. As mentioned above, it is wise to isolate yourself from registry disruptions. However, external registry mirroring is only one part of the equation. You also want to make sure you have a high availability plan for your internal registry as well as proper backup and recovery processes. A highly available, fault-tolerant container management platform is not just a runtime environment.
2: Container infrastructure and orchestration
Orchestration tools
Once you have a container image to run, you need a location to run it. That means both the computer the container runs on and the software that schedules it to run. If you're working with a few containers, you can make manual decisions about where to run container images, what to run with container images, and how best to manage storage and network connectivity. However, at scale, these kinds of decisions should be left to orchestration tools like Kubernetes, Swarm, or Mesos. These platforms can receive workload execution requests, determine where to run based on resource requirements and constraints, and then actually launch that workload on its target. And if a workload fails or resources are low, it can be restarted or moved as needed.
Source: DevOpsCube, 2022
Storage
Storage is another important consideration. This includes both the storage used by the operating system and the storage used by the container itself. First, you need to consider the type of storage you actually need. Can I outsource my storage concerns to a cloud provider using something like Amazon Relational Database Service instead? If not, do you really need block storage (e.g. disk) or can an external object store like AWS S3 meet your needs? If your external object storage service can meet your performance and durability requirements as well as your governance and compliance needs, you're in luck. You may not have to worry about managing the container's persistent storage. Many external storage services can be provisioned on demand, support discrete snapshots, and some even allow dynamic scaling on demand.
Networking
Network connectivity inside and outside the containerized environment is also very important. For example, Kubernetes supports a variety of container networking interfaces (CNIs), each providing different functionality. Questions to consider here are whether you can set traffic control policies (and the OSI layer), how to handle encryption between workloads and between workloads and external entities, and how to manage traffic import for containerized workloads. The impact of these decisions also plays a role on performance.
Backups
Backups are still an important task in containerized environments, but the backup target is changing slightly. An immutable, read-only container file system can be recreated very easily from the original container image and does not need to be backed up. Backups or snapshots on permanent storage should still be considered. If you are using a cloud provider, you should also consider fault domain and geo-recovery scenarios depending on the provider's capabilities. For example, if you're using AWS, you can use S3 replication to ensure that EBS snapshots can be restored in another region in case of a full region outage.
3: Container runtime security and policy enforcement
Ensuring that containers run in a place that meets the resource requirements and constraints set for them is necessary, but not sufficient. It is equally important that your container management solution performs continuous validation and ensures that your workloads comply with all security and other policy requirements of your organization. Runtime security and policy enforcement tools include a function for detecting vulnerabilities in running containers, handling detected vulnerabilities, ensuring that workloads are not running with unnecessary or unintended privileges, and ensuring that only other workloads that need to be allowed can connect.
One of the great benefits of (well implemented) containerized software is reducing the attackable surface of the application. But it doesn't completely remove it. This means you need to think about how to observe running applications to minimize security risks. Scanning as part of the build pipeline is not enough. This is because an image without vulnerabilities at build time can become a vulnerable container because new flaws are discovered in its code or support libraries. Instead, some modern tools focus on detecting unusual behavior at the system call level. As these types of tools mature, they can make a real difference to your workload’s security because they rely on actual observed behavior rather than up-to-date signature files.
4: Container observability
What’s going on in there?
Finally, if your container images are being run somewhere by orchestration tools and well managed by security and policy enforcement tools, you need to know what your containers are doing and how well they are doing it. Orchestration tools will likely have their own logs and metrics, as will networking layers, and security and compliance checking tools; there is a lot to understand in a containerized environment. Container observability covers logging and metrics collection for both your workloads and the tools that run them.
One very important element of observability is the importance of externalizing logs and metrics in a containerized environment. Containers come and go, and in many cases the nodes running on them also come and go, so relying on local storage is not recommended.
The importance of a container management strategy
A container management platform typically consists of a variety of tools from multiple sources. Some container management software vendors or container management services attempt to address all four key components of effective container management. However, many organizations already have tools that provide at least some of the features they need and don't want to waste existing licenses or make significant changes to their entire infrastructure just to run containers.
When choosing tools from multiple sources, it's important to understand what needs each tool meets and what it doesn't. This holistic approach is necessary to avoid gaps and duplication of effort.
For example, scanning an image as part of the build pipeline and then rescanning the image while the container is running is a waste of CPU cycles in the runtime environment. Similarly, using orchestration tools and separate host-based agents to aggregate logs or metrics can waste CPU cycles as well as storage and network resources.
Planning a container management strategy
| 1 | DIY, Managed Services, or Packaged Products Developer satisfaction is important, but it's also wise to consider the team running the container management software. Migrating from bare metal or virtual machine-based deployment methodologies to containers can involve a significant learning curve, so it's a good idea to choose a tool that will help smooth this curve. |
| 2 | Kubernetes In the world of container management, Kubernetes is fast becoming the de facto standard for container orchestration and scheduling. Most of the products that address the other aspects of container management discussed in this post (image supply chain, runtime security and policy enforcement, observability) integrate easily with Kubernetes. Kubernetes is open-source software and using it is possible if your team has the technical skills and the desire to implement it themselves. However, that doesn't mean you should automatically opt to build yourself. |
| 3 | Managed Kubernetes Kubernetes is difficult to implement well. As a result, many solution providers offer packaged products or managed services to facilitate Kubernetes adoption. All major cloud providers now offer Kubernetes services that reduce the operational burden on your teams. Organizations that have invested heavily in the ecosystem of a particular cloud provider may find this route suitable. Other organizations may be able to find a fully managed service that provides container images and lets the service provider worry about running the images which, depending on the cost and capacity of the organization, may be the best option. |
| 4 | Third-Party Orchestration Products A third approach is packaged products from providers that can be installed on the infrastructure (cloud or otherwise). These products can offer several potential advantages over DIY or cloud provider offerings, such as access to additional configuration options or cluster components, enhanced functionality, implementation assistance and training, post-installation product support, and reduced risk of cloud provider lock-in. |
Source: Kubernetes, 2022; Rancher, 2022
Infrastructure considerations
It's important to describe your organization’s current and future infrastructure strategy and how it fits into your container management strategy. It’s all basic for now, but if you plan to move to a virtual machine or cloud provider next year, your container management solution should be able to adapt to your environment now and in the future. Similarly, if you’ve already chosen a public cloud, you may want to make sure that the tool you choose supports some of the cloud options, but full compatibility may not be an important feature.
Infrastructure considerations extend beyond computing. Choosing a container management platform should be compatible with the existing network infrastructure and storage capacity available to your organization. If you have existing policy enforcement, monitoring, and alerting tools, the ideal solution should be able to take advantage of them. Moving to containers can be a game changer for developers and operations teams, so continuing to use existing tools to reduce complexity where possible can save time and money.
Leverage the reference architecture to guide your container management strategy
Using the examples as a guide, complete the tool to strategize your container management
Download the Reference Architecture
Bibliography
Mell, Emily. “What is container management and why is it important?” TechTarget, April 2021.
https://www.techtarget.com/searchitoperations/definition/container-management-software#:~:text=A%20container%20management%20ecosystem%20automates,operator%20to%20keep%20up%20with
Conrad, John. “What is Container Orchestration?” CapitalOne, 24 August 2020.
https://www.capitalone.com/tech/cloud/what-is-container-orchestration/?v=1673357442624
Kubernetes. “Cluster Networking.” Kubernetes, 2022.
https://kubernetes.io/docs/concepts/cluster-administration/networking/
Rancher. “Comparing Kubernetes CNI Providers: Flannel, Calico, Canal, and Weave.” Rancher, 2022.
https://www.suse.com/c/rancher_blog/comparing-kubernetes-cni-providers-flannel-calico-canal-and-weave/
Wilson, Bob. “16 Best Container Orchestration Tools and Services.” DevopsCube, 5 January 2022.
https://devopscube.com/docker-container-clustering-tools/