A multi-tenant architecture is key in reliably hosting applications and business services that will eventually produce value for the customer. Not designing a K8s infrastructure for multi-tenancy will lead to business initiative & project failure. In the first in a two-part blog post, we will discuss the architectural and technical details of building a multitenant k8s platform.
Let’s define Multi-tenant Kubernetes
This defines the Multi-tenancy challenge and this leads to issues around three specific areas – resource sharing, self-service, tenant management from both an API & UI perspective and security.
The Key Technical Challenges to Realizing a Multitenant K8s Architecture
Let us discuss key challenges from an infrastructure planning standpoint.
-
Resource Management – Ensuring that every tenant has ‘fair’ access to the CPU, Network and Memory resources without being starved of them by other tenant workloads running on the same cluster. Further, the management plane should provide the ability to tune resource access per workload to ensure that business SLAs are met or exceeded.
-
Self Service – Enable users across different tenant groups to perform the lifecycle of multi-tenancy – provision k8s clusters, scale up or down clusters across different IaaS providers and to deploy applications using a certified catalog of images including runtimes such as Prometheus,
-
Security – Namespace isolation so that each workload can be isolated such that a security vulnerability or malware in a given tenant doesn’t cause a wider breach resulting in data loss or downtime
Best Practices
A complete multi-tenant K8s environment setup should also address the following key points – which have been covered in other blog posts here, and here.
The right way to design the overall infrastructure should follow the below recommendations –
-
A management plane that enables self-service, integration with backend enterprise systems & hosts a catalog of containerized runtimes that allows developers to deploy and run components certified the O[ps team in a single executable environment.
- Centralize security and auditing
-
Help onboard and support various developer teams with the workflow to checkout code, make changes, produce deployable artifacts and a mechanism to deploy the produced artifacts to the appropriate clusters
-
Automate the management of Day 1, Day 2 and Day 2+ tasks so that the development team can focus on the more important things, such as developing business functionality.
- Achieve better utilization of the underlying hardware as well as provide self-service to tenants
Kubernetes provides various primitives to isolate users and their resources within the data nodes. These include the notions of physical hosts, groups of those (clusters), workloads running in pods and in containers. Using namespaces, tenant workloads can be separated into different parts of the cluster. Policies can then be added to the namespace scope to control access and set limits on resource usage. Kubernetes objects such as CRDs/operators, extensions, etc are shared between members of a cluster.
With the basics in place, the next post will focus on the key technical aspects of building out an enterprise-scale multi-tenant Kubernetes infrastructure.