overview

A Kubernetes "cluster" is made up of at least one "master node" and a few "worker nodes" where each node has a cubelet process running on it called cubelet.

Creating a node simply means:

1. Getting a new bare server with a linux distro installed
2. Installing the master/worker processes (kubectl, docker, etc)
3. Adding it to a cluster

worker nodes

Cubelet makes it possible for the nodes of a cluster to communicate and execute tasks. Each worker node will have containers of different applications running on them depending on how the workload is distributed. This is where the actual work is happening and as a result the worker nodes tend to be larger and more powerful computers compared to the master node.

master node

The master node runs several kubernetes processes which are needed to run and manage the cluster properly. One of such processes is an API server (which itself is a container). Most kubernetes clients will talk to this process.

Another process is the controller manager, which basically keeps an overview of the cluster and keeps track of containers that might need restarted or if a node is missing, etc.

Next, there's the scheduler process which decides which node a new pod should be placed on based on the workload and available resources.

Finally, there's an etcd key value storage. This holds the current state of the cluster at any given time. It has all configuration and status data. It even stores snapshots which can be used to restore the state of a cluster from a previous time.

In production environment you want a backup of your master node so that you can still manage the cluster if the master node goes down.

virtual network

All nodes are contained within a virtual network that allows communication between the nodes and effectively turns all the nodes into a single unified machine with the sum of the resources of all the individual nodes.

pod

A pod is the "lowest level" in kubernetes. It's basically a light weight abstraction over a container. The reason it's abstracted away is so that kubernetes can support more than just docker. This also allows you to interact only with kubernetes without needing to worry about docker or if the pod even is a docker container. In a simple webapp you would perhaps have a pod for your actual application and a second pod for a database. Like containers, a pod generally is meant to run a single application.

Each pod is given it's own IP in the virtual network. This means each pod can communicate using this internal IP, but notably pods are considered ephemeral so they can die very easily. When that happens, such as if the database container crashes or the node it's running on runs out of resources, the pod will die and a new one will be created to replace it. This new pod will have a new IP address, which is inconvenient if you were communicating with the database using it's IP address. You'll need to use services to fix this issue.

services

Services are basically static/permanent ip addresses and load balancers that can be attached to a pod (or several cloned pods). So in the simple web app example both the app pod and database pod will have their own services so that if one of the pods dies it will be replaced, but the same old service ip will stay the same (but point to the new replacement pod).

types

When you create a service you specify the 'type' as either ClusterIP, NodePort, or LoadBalancer.

ClusterIP

Exposes the Service on a cluster-internal IP. Choosing this value makes the Service only reachable from within the cluster. This is the default ServiceType.

Your database for example would probably use a ClusterIP as it shouldn't be accessible from the outside/wider internet. Your webapp however should have an external service so you can use it from a web browser. The rest of the types are different approaches to achieving this.

NodePort

Exposes the Service on each Node's IP at a static port (the NodePort). A ClusterIP Service, to which the NodePort Service routes, is automatically created. You'll be able to contact the NodePort Service, from outside the cluster, by requesting :.

In a single-node cluster this is very straight forward. In a multi-node cluster the internal routing can get more complicated. In that case you might want to introduce an external load balancer so you can spread traffic out across all the nodes and be able to handle failures a bit easier.

NodePort is great, but it has a few limitations. The first is that you need to track which nodes have pods with exposed ports. The second is that it only exposes one service per port. The third is that the ports available to NodePort are in the 30,000 to 32,767 range.

LoadBalancer

Exposes the Service externally using a cloud provider's load balancer. NodePort and ClusterIP Services, to which the external load balancer routes, are automatically created.

This is the default method for many Kubernetes installations in the cloud, and it works great. It supports multiple protocols and multiple ports per service. But by default it uses an IP for every service, and that IP is configured to have its own load balancer configured in the cloud.

ExternalName

Maps the Service to the contents of the externalName field (e.g. foo.bar.example.com), by returning a CNAME record with its value. No proxying of any kind is set up.

Note: You need either kube-dns version 1.7 or CoreDNS version 0.0.8 or higher to use the ExternalName type.

ingress

You can also use Ingress to expose your Service. Ingress is not a Service type, but it acts as the entry point for your cluster. It lets you consolidate your routing rules into a single resource as it can expose multiple services under the same IP address.

config map

For most applications you'll have configuration variables or files which would be used to tell your webapp which url to use for it's database. These config details are often built into the applications image. Meaning if you need to change the config you need to rebuild the image, push it to a repo, and then pull that new image to the kubernetes pod. This is super inconvenient for tiny config tweaks.

A config map solves this issue. It's basically your applications external configuration and it can be presented to your application as environment variables or a config file.

Connect it to a pod to give the pod has access to that information. Then you can just change the data in the config map and all the pods that listen to it will get their new config data. However, the config map is for non-confidential data only. You shouldn't use it for database passwords.

secret

Secret is just like config map, but for storing secret data. However, they're not encrypted out of the box in kubernetes! So anyone with kubernetes API access can read the secrets and you're meant to setup some sort of 3rd party encryption system for managing secrets.

volume

Volumes are used for data storage in kubernetes. Normally all your pods, including those which host a database server, have non-persistent storage. So you loose all your data every time the database pod restarts.

A volume attaches a physical storage on a harddrive to your pod. The storage location could be local (on the same node as the pod) or in many cases it would be a remote storage server or perhaps a cloud storage provider outside of the cluster. It's basically an external harddrive being "plugged in" to the kubernetes cluster.

deployments

In production you generally have multiple "cloned" or replicated nodes all running the same pods. Each pod has a service which acts as dns and a load balancer to determine which node (and thus pod) is actually given the request.

Blueprints are used to achieve this redundancy (rather than creating a second set of pods manually). In the blueprint you can specify how many replicas you want to have. This process of creating and running blueprints is called a deployment. In practice you pretty much always create deployments rather than manually managing pods.

Once the application instances are created, a Kubernetes Deployment Controller continuously monitors those instances. If the Node hosting an instance goes down or is deleted, the Deployment controller replaces the instance with an instance on another Node in the cluster. This provides a self-healing mechanism to address machine failure or maintenance.

replicating databases

Databases can't be replicated using a deployment because the database has state. If you have replicas of your database node they all need to access the same storage volume and so you need a mechanism to determine which pods are currently writing/reading to that storage in order to avoid inconsistencies. This feature, in addition to a replicating feature, is offered by another kubernetes component called statefulset

statefulset

A statefulset is meant for applications like databases (mongo, postgresql, mysql, etc).

Deployment = stateless apps
Statefulset = stateful apps (databases)

It will handle replicating your database pods, much like a deployment, but will also synchronize the reads and writes so that there are no database inconsistencies. Unfortunately, it's also quite a bit more complex to configure and run than a deployment so it's also common practice to simply host a database server outside of your kubernetes cluster and only use kubernetes for stateless applications.