Before getting started with the Tanzu Kubernetes Grid Service for vSphere, it’s good to have an understanding of the Kubernetes architecture and the underlying technology that makes it possible. Here, we give a high-level introduction to how these technologies are interconnected.

Hello, I'm Kenny Coleman, a Senior Technical Marketing Manager at VMware focused on the Tanzu Kubernetes Grid Service for vSphere.

In this series of videos, we'll be looking at the Tanzu Kubernetes Grid Service for vSphere.

Before getting started and using the service, it's good to have an understanding of the Kubernetes architecture and the underlying technology that is making all this possible.

We will start at the lowest layer and zoom out, thereby painting a picture of how all these are interconnected.

This is an introduction and not a deep dive into the individual components such as networking or storage, but it is meant to be a high-level overview for anyone getting started.

There are multiple layers in this architecture.

First is taking a look at the Kubernetes architecture.

This will not go deep into the services or how storage or networking work, but instead just focus on the core components that satisfy a Kubernetes cluster at the infrastructure level.

Next, we will look at the cluster API architecture and what this technology is doing to automate Kubernetes deployments.

Finally, we will take a look at the vSphere with Kubernetes environment to see how these components all relate to what we've learned thus far.

At the end of the day, what is it that we're trying to accomplish?

Well, it's to get this containerized application running.

If you're watching this, then it's likely you're already familiar with what a container is and as well as a container runtime.

Kubernetes adds a topology and combines features to make it 1 of the best suited container schedulers available.

In the context of Kubernetes, a container is not the lowest level object.

A pod can have 1 or more containers within it.

If you have an application that has multiple services or layers that all have their own container, the pod doesn't have to make up the entire application.

The application can be spread out amongst multiple pods or even across traditional virtual machines as well.

Kubernetes gives you the flexibility to architect the application that best fits your environment and needs.

A pod has to run somewhere, and this is the first part of where the Kubernetes infrastructure layer comes into play with the Kubernetes worker or node.

The Kubernetes infrastructure is made up of only a few pieces, but just like the vSphere ESXi host that are responsible for running your virtualized applications, Kubernetes workers are responsible for running your containerized applications.

The Kubernetes worker can run multiple pods.

And the size and the amount of the pods that can run on a worker are going to depend on the size of the worker itself.

This is analogous to what we experience with vSphere today on how big our ESXi hosts are.

Next is the Kubernetes master or controller.

Like vCenter, this is the brain of the Kubernetes deployment.

It is packed with services that are required for keeping the cluster functioning and carries all the components for deploying applications to the workers themselves.

The API server is the central communication hub.

It provides REST-based services for the components to talk to 1 another, as well as user interaction when deploying applications.

The API server provides the front end for the cluster by exposing the Kubernetes API.

Internal components, such as the scheduler or the nodes, and external components such as kubectl or API-driven systems, can all make calls to the API server.

The Kubernetes Controller Manager is a service that watches the shared state of the cluster through the API server and makes changes attempting to move the current state towards a desired state.

The Controller Manager runs all the Kubernetes Reconciliation State machine services, and each controller provides a control loop that evaluates the current state of the system, compares the current state to the desired state, and then takes action to reconcile the differences.

The scheduler is what will watch for new pods as they are requested and created.

It saves the current state of the cluster.

The control plane of Kubernetes can scale as well.

There is a lot more complex configurations that need to take place that isn't mentioned in this diagram, such as front-ending all these additional master nodes with a load balancer but etcd will replicate changes across the master nodes given in a highly available situation.

The amount of worker nodes needed is based on the resources needed to run the applications.

All of this combined represents a single Kubernetes cluster.

At the infrastructure level, these are all virtual machines running on top of vSphere.

Now that we understand what a cluster is comprised of, we need to know how it's built.

There are a lot of blogs and articles, GitHub repos, and tools available that cover everything from starting at the lowest level by installing individual services on Linux and creating TLS certificates up to the highest point where everything is deployed automatically.

There've been multiple attempts at delivering a single Kubernetes installer experience, but they're either tailored to a specific infrastructure provider, they were developed very early before Kubernetes began maturing, or they're proprietary solutions.

It's a tool that is developed in the Kubernetes open source community under the special interest group of Project Lifecycle.

This means it's an accepted tool by the upstream and larger Kubernetes community for building and deploying Kubernetes clusters in an automated fashion.

It uses the same Kubernetes principles of using the declarative control loop to achieve a desired state and perform all the functions of create, scale, upgrade, and delete clusters themselves.

To roll this back just slightly, we have to understand how to extend Kubernetes.

Natively, Kubernetes knows what a pod is, as well as an application deployment object, such as a stable set or deployment.

So how does Kubernetes know how to use its principles on objects that it doesn't understand?

This is where the custom resource definition, or the CRD, comes into play.

This is an extension of the Kubernetes API, so it knows how to interact with new types of objects.

Cluster API is a set of CRDs that allows Kubernetes to know how to interface with an infrastructure provider and it knows how an object of type machine or cluster is represented.

At a high level, this is how Cluster API works.

As a user, I define a cluster specification.

Within this specification, I define the types of machines that will make up my cluster.

And using the standard kube control command line, I apply this to a Kubernetes cluster that has the cluster API components installed.

The Kubernetes cluster with cluster API is considered now my management cluster.

This cluster is going to be responsible for communicating to the infrastructure provider of my choice and deploying Kubernetes clusters based on the specification I provide.

Cluster API will use that declarative nature of Kubernetes to make sure it achieves my desired state.

And if at a later time I want to add more Kubernetes workers or masters, I simply edit the specification and apply it to the management cluster.

Cluster API has many different providers available, and there is a Cluster API provider for vSphere that allows it to know how to communicate to vCenter and deploy virtual machines based on templates or content catalogs.

VSphere with Kubernetes takes this to another level by introducing an architecture that provides even more features and unique capabilities.

Within vSphere, there is a concept of the supervisor cluster.

This is our management cluster for cluster API.

It's represented as a few virtual machines that are automatically created when enabling the workload platform service on a cluster that satisfies all the requirements.

The supervisor cluster can provision virtual machines, it can run containers that are wrapped at a very lightweight virtual machine called a vSphere pod, and it's also responsible for deploying Kubernetes clusters as a part of the Tanzu Kubernetes Grid Service.

VSphere with Kubernetes has a concept of vSphere namespaces.

Like Kubernetes or Linux namespaces, it defines a boundary or security context.

A vSphere namespace is like a resource pool, but it can run multiple types of Kubernetes objects, such as virtual machines, vSphere pods, and even multiple Tanzu Kubernetes Grid service clusters.

The namespaces also have role-based access control, which is inherited through vSphere single sign-on.

And there's also resource and object limits that can be imposed.

These limits give administrators the control over the namespaces and make sure applications do not take up more resources than allowed.

Within vCenter, we can see that the compute cluster has passed all the tests required to enable the workload service, and the supervisor cluster virtual machines have been created.

These virtual machines represent the Kubernetes and cluster API management cluster.

Each 1 has its own unique IP address, but through a leader election process, only 1 will be the interface needed to interact with any cluster or vSphere pod.

This example shows a Tanzu Kubernetes grid cluster.

Without going through everything in depth, it's easy to understand the cluster specification and the types and amounts of virtual machines we want to represent a new Kubernetes cluster.

After this specification is applied to the supervisor cluster, it will invoke the cluster API components to achieve a desired end state.

The Kubernetes master and worker nodes are represented as virtual machines within the namespace that was defined in the specification.

Thanks for watching this video on the Tanzu Kubernetes Grid Service Architecture.

Make sure you stay tuned for more videos coming up real soon.


Speaker 0: Hello, I'm Kenny Coleman, a Senior Technical Marketing Manager at VMware focused on the Tanzu Kubernetes Grid Service for vSphere. In this series of videos, we'll be looking at the Tanzu Kubernetes Grid Service for vSphere. Before getting started and using the service, it's good to have an understanding of the Kubernetes architecture and the underlying technology that is making all this possible. We will start at the lowest layer and zoom out, thereby painting a picture of how all these are interconnected. This is an introduction and not a deep dive into the individual components such as networking or storage, but it is meant to be a high-level overview for anyone getting started.

There are multiple layers in this architecture. First is taking a look at the Kubernetes architecture. This will not go deep into the services or how storage or networking work, but instead just focus on the core components that satisfy a Kubernetes cluster at the infrastructure level. Next, we will look at the cluster API architecture and what this technology is doing to automate Kubernetes deployments. Finally, we will take a look at the vSphere with Kubernetes environment to see how these components all relate to what we've learned thus far.

All right, let's get started. At the end of the day, what is it that we're trying to accomplish? Well, it's to get this containerized application running. If you're watching this, then it's likely you're already familiar with what a container is and as well as a container runtime. Kubernetes adds a topology and combines features to make it 1 of the best suited container schedulers available.

In the context of Kubernetes, a container is not the lowest level object. Instead, it's the pod. A pod can have 1 or more containers within it. If you have an application that has multiple services or layers that all have their own container, the pod doesn't have to make up the entire application. The application can be spread out amongst multiple pods or even across traditional virtual machines as well.

Kubernetes gives you the flexibility to architect the application that best fits your environment and needs. A pod has to run somewhere, and this is the first part of where the Kubernetes infrastructure layer comes into play with the Kubernetes worker or node. The Kubernetes infrastructure is made up of only a few pieces, but just like the vSphere ESXi host that are responsible for running your virtualized applications, Kubernetes workers are responsible for running your containerized applications. The Kubernetes worker can run multiple pods. And the size and the amount of the pods that can run on a worker are going to depend on the size of the worker itself.

This is analogous to what we experience with vSphere today on how big our ESXi hosts are. Next is the Kubernetes master or controller. Like vCenter, this is the brain of the Kubernetes deployment. It is packed with services that are required for keeping the cluster functioning and carries all the components for deploying applications to the workers themselves. The API server is the central communication hub.

It provides REST-based services for the components to talk to 1 another, as well as user interaction when deploying applications. The API server provides the front end for the cluster by exposing the Kubernetes API. Internal components, such as the scheduler or the nodes, and external components such as kubectl or API-driven systems, can all make calls to the API server. The Kubernetes Controller Manager is a service that watches the shared state of the cluster through the API server and makes changes attempting to move the current state towards a desired state. The Controller Manager runs all the Kubernetes Reconciliation State machine services, and each controller provides a control loop that evaluates the current state of the system, compares the current state to the desired state, and then takes action to reconcile the differences.

The scheduler is what will watch for new pods as they are requested and created. Etcd pretty much becomes our database. It saves the current state of the cluster. The control plane of Kubernetes can scale as well. There is a lot more complex configurations that need to take place that isn't mentioned in this diagram, such as front-ending all these additional master nodes with a load balancer but etcd will replicate changes across the master nodes given in a highly available situation.

The amount of worker nodes needed is based on the resources needed to run the applications. These can scale as needed as well. All of this combined represents a single Kubernetes cluster. At the infrastructure level, these are all virtual machines running on top of vSphere. Now that we understand what a cluster is comprised of, we need to know how it's built.

There are a lot of blogs and articles, GitHub repos, and tools available that cover everything from starting at the lowest level by installing individual services on Linux and creating TLS certificates up to the highest point where everything is deployed automatically. There've been multiple attempts at delivering a single Kubernetes installer experience, but they're either tailored to a specific infrastructure provider, they were developed very early before Kubernetes began maturing, or they're proprietary solutions. However, that's why we have Cluster API. It's a tool that is developed in the Kubernetes open source community under the special interest group of Project Lifecycle. This means it's an accepted tool by the upstream and larger Kubernetes community for building and deploying Kubernetes clusters in an automated fashion.

It uses the same Kubernetes principles of using the declarative control loop to achieve a desired state and perform all the functions of create, scale, upgrade, and delete clusters themselves. To roll this back just slightly, we have to understand how to extend Kubernetes. Natively, Kubernetes knows what a pod is, as well as an application deployment object, such as a stable set or deployment. So how does Kubernetes know how to use its principles on objects that it doesn't understand? This is where the custom resource definition, or the CRD, comes into play.

This is an extension of the Kubernetes API, so it knows how to interact with new types of objects. Cluster API is a set of CRDs that allows Kubernetes to know how to interface with an infrastructure provider and it knows how an object of type machine or cluster is represented. At a high level, this is how Cluster API works. As a user, I define a cluster specification. Within this specification, I define the types of machines that will make up my cluster.

And using the standard kube control command line, I apply this to a Kubernetes cluster that has the cluster API components installed. The Kubernetes cluster with cluster API is considered now my management cluster. This cluster is going to be responsible for communicating to the infrastructure provider of my choice and deploying Kubernetes clusters based on the specification I provide. Cluster API will use that declarative nature of Kubernetes to make sure it achieves my desired state. And if at a later time I want to add more Kubernetes workers or masters, I simply edit the specification and apply it to the management cluster.

Cluster API has many different providers available, and there is a Cluster API provider for vSphere that allows it to know how to communicate to vCenter and deploy virtual machines based on templates or content catalogs. VSphere with Kubernetes takes this to another level by introducing an architecture that provides even more features and unique capabilities. Within vSphere, there is a concept of the supervisor cluster. This is our management cluster for cluster API. It's represented as a few virtual machines that are automatically created when enabling the workload platform service on a cluster that satisfies all the requirements.

The supervisor cluster can provision virtual machines, it can run containers that are wrapped at a very lightweight virtual machine called a vSphere pod, and it's also responsible for deploying Kubernetes clusters as a part of the Tanzu Kubernetes Grid Service. VSphere with Kubernetes has a concept of vSphere namespaces. Like Kubernetes or Linux namespaces, it defines a boundary or security context. A vSphere namespace is like a resource pool, but it can run multiple types of Kubernetes objects, such as virtual machines, vSphere pods, and even multiple Tanzu Kubernetes Grid service clusters. The namespaces also have role-based access control, which is inherited through vSphere single sign-on.

And there's also resource and object limits that can be imposed. These limits give administrators the control over the namespaces and make sure applications do not take up more resources than allowed. Within vCenter, we can see that the compute cluster has passed all the tests required to enable the workload service, and the supervisor cluster virtual machines have been created. These virtual machines represent the Kubernetes and cluster API management cluster. Each 1 has its own unique IP address, but through a leader election process, only 1 will be the interface needed to interact with any cluster or vSphere pod.

This example shows a Tanzu Kubernetes grid cluster. Without going through everything in depth, it's easy to understand the cluster specification and the types and amounts of virtual machines we want to represent a new Kubernetes cluster. After this specification is applied to the supervisor cluster, it will invoke the cluster API components to achieve a desired end state. The Kubernetes master and worker nodes are represented as virtual machines within the namespace that was defined in the specification. Thanks for watching this video on the Tanzu Kubernetes Grid Service Architecture.

Make sure you stay tuned for more videos coming up real soon.

See all VMware Tanzu transcripts on Youtube

An Elevated View of the Tanzu Kubernetes Grid Service Architecture