Zero to production with Azure Kubernetes Service in 15 minutes

. 13 min read

Kubernetes is complex! Yes, I don’t deny that fact, it is. Especially if you have never used an orchestration engine before, it might puzzle you. After all a ton of configurations and setup is required to maintain and deploy an application.

Well to be honest, the amount of benefits it offers deserves a little bit of complexity. Imagine deploying 100s of microservices with a single click. And this is a deployment, where you would never have to come back again for maintenance. You job is to declare what should be done in case of various events.

Kubernetes handles the rest for you. Sudden scale requirements, OK! Hardware or software failures, OK! New releases, blue green deployment, OK!

If you are still at an early stage to understand what value K8s offer, I recommend you read this.

Kubernetes takes away a lot of complexity from the hands of developers and administrators. It creates a cluster of machines that can help you deploy and manage stateful and stateless applications.

Kubernetes High Level Architecture

Essentially from a 10K view, K8s can be broken down into a set of master servers responsible for managing a set of agent nodes. Agent nodes are where you deploy your application containers. K8s master takes care of the rest by scheduling the application containers, providing server-side controllers for deployment and management. It also provides configuration manager and discovery module in case of new services or moving application containers. Users interact with the master API server for any operation.

The biggest advantage of K8s is the managed services which provides location transparent access to application pods. This provides a fault tolerant server, where a microservice pods may have a short lifecycle than the entire application. K8s master scheduler and controllers create advanced networking route tables underneath to enable that. While this is not essentially true for all the deployments. K8s provide extensions to the networking routine so that cloud vendor can integrate with native load balancing and networking routines.

Tackling K8s Complexity

Irrespective of all the value added by K8s, it is complex. But luckily for us, K8s is an open source software backed by Cloud Native Foundation. Here’s what it says:

“The Cloud Native Computing Foundation builds sustainable ecosystems and fosters a community around a constellation of high-quality projects that orchestrate containers as part of a microservices architecture.”

Well that said, it has native support for all the major cloud vendors. It has managed services for Azure, Google, Amazon, Digital Ocean and others.

In this article, I wanted to leverage Azure Kubernetes service to deploy a production quality application. It is not a trivial task, but thanks to Azure – it was not that difficult.

As a starting point, we will deploy the application in a private subnet that does not have any public access. This allows us to control the traffic from public internet and add additional firewall in the front.

Setting up Azure CLI

All Azure resources are managed by Resource Manager. All the deployment requests to Azure goes to the resource manager. It has a REST API exposed, a CLI client for all the major operating systems, portal, power shell modules, resource manager templates and third party tooling like Terraform.

For the purpose of this article we will use Azure CLI, which can be installed across all the major operating system. To install Azure CLI on your operating system, you can follow this installation document.

The very first thing you would need to do is the login using the Azure CLI. I assume that you are going to use your user account for deployment using following commands. If alternatively, you want to use a service account, make sure it has the required contributor/owner rights for provisioning.

az login

Please note that you can also use Azure Cloud Shell from Azure portal. It provides an authenticated remote shell in a Virtual Machine with all the Azure dependencies already installed.

Creating a resource group

“Resource Groups” allows us to group similar resources together. This allows us to co-locate all the resources for management, cost and reporting reasons. I will be creating a resource group in “East US” region. If you choose a different region, make sure it has the Kubernetes cluster available.

  az group create \
    --name ghost-prod-rg \
    --location eastus

Creating a Virtual Network

Azure Kubernetes service is a managed offering. The API server, viz. the master is deployed in the Azure network. But, it allows the agent nodes to be deployed in a Virtual Network for additional security.

If you are familiar with AWS, Virtual Network is equivalent to VPC.

Azure CLI can handle the heavy lifting for us by creating a Virtual Network for us. But, for the purpose of this article, let’s create a Virtual Network manually. This allows us to closely configure all the switches.

az network vnet create \
    --name ghost-prod-vnet \
    --resource-group ghost-prod-rg \
    --address-prefixes 10.1.0.0/16 \
    --subnet-name ghost-aks-private \
    --subnet-prefixes 10.1.0.0/21

We have requested Azure to create a virtual network and a subnet inside the VNet. We have also defined a private address range from 10.1.0.0 to 10.1.255.255. We have also created a subnet "ghost-aks-private" with the address range from 10.1.0.0 to 10.1.7.255 inside our virtual network.

If you are not familiar with the CIDR address notation, please check herefor CIDR range conversions.

Listing the Virtual Network ID

Every resource in Azure has a display name provided by us. But, it also has a complete name or ID which uniquely identifies the resource in entire Azure cloud. We need to fetch the unique ID of the subnet for deploying AKS.

az network vnet subnet list \
    --resource-group ghost-prod-rg \
    --vnet-name ghost-prod-vnet \
    --query '[].{ID: id, Name: name}' \
    --output table

This requests Azure to fetch all the subnets in the given resource group with the name “ghost-prod-vnet”. It returns a detailed JSON. We are using the Azure CLI JSMEPath query syntax for extracting the ID and name in a tabular format.

Listing Virtual Network IDs

Creating an Azure Kubernetes Cluster

Azure Kubernetes Service is a managed service. As of the current version, we pay for the agent nodes and not the master servers. Master servers, viz. the API Server is provisioned and managed by Azure and is accessible over internet.

However, Agent nodes in AKS are deployed in a subnet of a private Virtual Network. In this article, we have opted to choose the advanced networking deployments, so that we can explicitly control all the networking configurations of the deployment.

az aks create \
    --resource-group ghost-prod-rg \
    --name ghost-aks \
    --network-plugin azure \
    --vnet-subnet-id <YOUR_COMPLETE_SUBNET_ID> \
    --docker-bridge-address 172.17.0.1/16 \
    --dns-service-ip 10.2.0.10 \
    --service-cidr 10.2.0.0/24 \
    --node-vm-size Standard_B2s \
    --node-count 2

Above command is going to provision and deploy an AKS cluster named “ghost-aks” in the given resource group. Additionally, we provide the following details:

  1. Network plugin: Kubernetes has its own lean networking layer. This lean layer provides the service naming, discovery and orchestration. Azure Container Networking interfaces creates integrates this networking layer with Azure network. It allows each pod to get an IP address from the underlying Virtual Network. Additionally, as we define K8s services and ingress controllers, these can get assigned Azure load balancers.
  2. VNet Subnet ID: An Azure Kubernetes cluster can only be deployed in a subnet and cannot span more than one subnet. This requires us to provide the complete ID of the subnet for AKS agent nodes. API servers are not deployed on the same subnet.
  3. Docker Bridge Address: An address range for communication among the host servers and the container applications. This address range cannot be overlapping with the host subnet range.
  4. Service Address Range: The IP address range for ingress services, cluster IP and other load balanced service communication. You should plan the service address range to be different from the subnet address range. AKS does not complain if you have overlap, but you may face unpredictable issues in communication.
  5. DNS Service IP: The IP address from the service address range to be used for the Kubernetes DNS server. You should not use the 1st address in the service address range, as it is reserved.
  6. Agent Nodes: Additional information about node size, type, managed disks, count, etc.

Verifying the AKS resources

Let’s query all the resources under your subscription to verify what has been provisioned.

az resource list \
    --query "[].{Name: name, ResourceGroup: resourceGroup}" \
    --output table
Listing resources

For my subscription, I didn’t had any other resources before this deployment. It returned resources in 2 resource group “ghost-prod-rg” and “MC_ghost-prod-rf_ghost-aks-eastus”

While the resource group we created has the Virtual Network and the AKS cluster, all the other resources are created in a different resource group. AKS provisions all the agent nodes and associated K8s resources in a separate resource group for operational reasons. It allows Azure to decommission the entire resource group when the AKS deletion request is issued.

Verifying the AKS Cluster

Let’s connect with the AKS cluster to check the current status of the provisioning. We will be going to use the K8s command line utility ‘kubectl’ – Kube Control.

If you do not have ‘kubectl’ installed, you can find the instructions here.

Kube Control connects with the API server of the cluster for communicating with the agent nodes. AKS API server is deployed in the public domain and not within the Virual Network. This allows kube control to connect with the cluster over internet.

We will first request the credentials for the kube control, to be able to connect with the cluster.

az aks get-credentials \
    --resource-group ghost-prod-rg \
    --name ghost-aks

This will fetch token and client certificates from the AKS server and deploy it to the local context. With that in place, any requests from kube control are authorized.

Let's fetch the cluster info now.

kubectl cluster-info      

This pulls in the cluster information including the dashboard, K8s master, DNS server etc. For me, it returned the following. The internet accessible addresses of the API server components.

Azure Kubernetes Cluster Info

To fetch the information of the agent nodes, we issue:

kubectl get nodes

For my 2 nodes AKS cluster, it returned the following

Azure Kubernetes Node Info

Deploying application to AKS

In this article, I will be deploying a blogging platform called “Ghost” on Kubernetes. It is a simple NodeJS application that uses local file system for static resources. We will be going to use the following docker images for the deployment:

·       Ghost: Ghost blogging application

We will also be adding load balancer to a set of deployed pods in order to expose the service. Here's what we need to do.

Azure Kubernetes Application Deployment

Creating and Managing Pods

Pods sits at the core of the Kubernetes. They represent a logical entity that includes one or more containers sharing the same resources. They share the same physical infrastructure and external file system like volumes. For the ghost application, we have a single standalone container which we will be deploying as a pod.

Kubernetes Pod Communication

The deployment file declares a pod definition for hosting “Ghost” containers. Podsare the smallest entity in the Kubernetes. They encapsulate one or more containers that should coexist in the same physical space. A pod also encapsulates storage resources and associated physical IP addresses.

kubectl run ghost --image=ghost:latest

This requests K8s to create a deployment using “Ghost” docker image. A deploymentis a K8s controller that defines a pod definition along with replica information. This allows us to create a single object that declares a pod and the deployment strategies. With that in place, we can control the replicas count, upgrade strategy, rollback mechanisms and others.

Verifying the deployment

To verify the successful deployment, we will request kube control to fetch all the deployments and pods. The command above creates a deployment with a single replica.

kubectl get deployments
kubectl get pods

For me, it returned a successful deployment and pods running on the K8s cluster. A deployment with 1 requested pod and 1 deployed pod. Pod lists also indicates the running pod.

K8s do not share the pods to the external world. There are other resource types in K8s schema to expose certain pods for consumers. This is because pods are meant to be replaced in case of failures or scaling events. A pod can be restarted at any time or can be moved across a cluster.

While the deployment name should remain same until requested, a pod may be dynamically allocated with a random name. In the deployment definition above, we requested for a single pod. Let’s just access the pod to verify the functionality. We will request K8s to forward a local port to port 2368 of ghost.

kubectl port-forward <Unique pod name> 8080:2368

I requested the application in my local browser at port 8080 to confirm that a successful Ghost application is running.

Please note that in case you encountered any failure in pod deployment, you can request K8s to share the pod logs using the following command.

kubectl logs <Unique pod name>

You can also request K8s to describe the deployment definition with you. It contains detailed deployment information, and event logs to debug any issues with starting pods.

kubectl describe deployments ghost-app

Exposing a load balanced Service

K8s pods as they say are “mortal”. They can be created, removed, scaled up/down, deleted or fail. A K8s Service provides an abstraction on top of a group of pods and policies defining the strategies to access them.

You can practically have thousands of identical pods doing the same job and a single service exposing the pod with a single address. There are various ways a service function, but for this exercise we will use Azure load balancer for it.

To expose the pods as a service, using the following command. It request K8s to deploy an Azure load balancer and expose the pods port 2368 over port 80.

kubectl expose deployment ghost \
    --port=80 \
    --target-port=2368 \
    --type LoadBalancer

Azure load balancer provisioning and assigning IP address might take some time. It may take some time to create a load balancer service and assign it the requested public address. Use the following command to watch the service updates. Wait till you see the external IP address assigned to it.

kubectl get service -w

To validate the deployment, navigate to your public IP address to see the ghost application. You should be able to see your ghost application now.

Scaling the application

K8s deployments provide pod abstractions that are created and deployed as replica sets. These replica sets are deployed as a set. At any point of time, K8s try to maintain the requested number of replicas in the cluster. In case of any failure of a pod, another pod is spin up in the cluster to compensate the failure.

To scale the number of pods in the deployment, use the following scale command:

kubectl scale deployment ghost --replicas=3

This will send instruction to K8s to always maintain 3 replicas of the deployment “Ghost”. K8s will spin up 2 additional replicas of the pod to maintain the desired state. To verify the state of the deployment, use the following command.

kubectl get deployments -wkubectl get pods -w

K8s automatically maintain the desired state. Intentionally delete one pod, and K8s will spin up another pod.

kubectl delete pods <A_POD_ID>kubectl get pods -w

Cleaning up the Resources

To clean up all the resources and create an empty cluster, you can issue the following delete commands.

kubectl delete deployments ghostkubectl delete service ghost

This will also delete the provisioned public IP and associated load balancer routes. You will still see the load balancer and the agent nodes as those are the physical resources associated with the managed AKS.

Using Kubernetes declarative resource configuration

One of the biggest value addition of Docker and Kubernetes is the declarative configuration and immutable infrastructure. While using commands is ok, it is not a recommended practice for a production deployment.

Commands create an immutable infrastructure, where a wrong command of a typo can lead to catastrophic environment. It is also not sustainable and cannot be automated. Docker and K8s both provide declarative template to define the desired production state. These declarative templates are then integrated with tools like Jenkins, HELM, Draft etc. for end to end automation, deployment, testing and incremental upgrades.

While Kubernetes documentation is a perfect source to understand the declarative configuration syntax, it is also available as Kube Control command.

kubectl explain pods

The above template declares a pod resource to be deployed. Pod is configured to look for the latest “Ghost” docker image and expose container port of “2368” to pod. Additionally, we assign couple of labels to the pod for unique identification across large number of cluster pods.

You can always use the explain command to dig deeper into certain elements of the declarative file.

kubectl explain pods.spec.containers

To apply the declarative pod template to the K8s cluster, you can use the create or apply command.

kubectl apply -f <pod_template_file.yaml>

Please note that in case of troubleshooting failed pod deployments, you can use the describe command for the application. You can also use the log command to look into the container logs. You can also connect a remote shell to the pod for troubleshooting. To start an interactive shell in the running pod, you can use the following command.

kubectl exec ghost --stdin --tty -c ghost /bin/sh

Monitoring and Health Checks

A typical distributed application consisting of many microservices can have many failure points. While an orchestration engine like K8s can help with automating deployments, it still must get meaningful information about the health of the service. K8s support 2 modes of monitoring service health.

Readiness probe indicates when the probe is ready to serve the traffic. Imagine an application which serving REST APIs served by a fast HTTP server. While the socket port might be available, but the application is still busy creating connections to the database or initializing scripts. Readiness probe can help identify if the endpoint is ready to serve traffic.

Liveliness probe indicates whether the endpoint is alive. K8s restart the containers with multiple failed liveliness probe. It allows K8s to make sure cluster state is always healthy and any failing pods should be restarted.

The above pod descriptor file creates the “Ghost” pod as above. But it adds additional health checks to monitor the readiness and liveness of the pod. Pods are not marked as running unless pods return successful status of 200 for the health checks. You can check the status of the pod using the describe command.

kubectl describe pod <Unique_pod_name>

The readiness probe does not need to be essentially an HTTP request. It can follow different protocol schemes and endpoints. In the above descriptor, we use HTTP GET with an initial delay, timeout and retry intervals.

Unfortunately for us, we do not have any way right now to force the “Ghost Application” to fail readiness probes. But in case of any failures, K8s will attempt to restart the underlying container.