Think back about part 2 of this series: we briefly discussed basic pods. I mentioned there that a pod is the basic execution unit within Kubernetes. A pod can contain multiple containers in it. With those multiple containers, we can build interesting design pattern.<\/p>\n\n\n\n
Brendan Burns has written a stellar book on different of these design patterns, which is available for free<\/a>. The book is an evolution of previous work Brendan had been doing. The same concepts were presented in 2016 at the HotCloud conference<\/a>, both as a paper and as a presentation.<\/p>\n\n\n\n The first part of the book focuses on what we’ll cover in this CKAD study material, namely the side, ambassador and adapter pattern.<\/p>\n\n\n\n These patterns look pretty similar when it comes to the technical implementation. In essence they take an existing container, and add functionality to it by adding another container. There is however a difference between the actual functionality of a sidecar, an ambassador or an adapter:<\/p>\n\n\n\n The exam objective states three patterns (ambassador, adapter and sidecar) which we’ll dive into and we’ll end this topic with init-containers. These aren’t part of the exam objective, but are a useful multi-container pod design pattern as well.<\/p>\n\n\n\n A sidecar adds functionality to an existing application. This functionality could be saving certain files in a certain location, copying files continuously from one location to another or for instance be terminate SSL and have your “legacy” web server run unchanged. <\/p>\n\n\n\n Multiple containers\nin a pod share their network and storage namespace, meaning a sidecar can\nconnect to the main container on localhost; and they can mount the same\nvolumes.<\/p>\n\n\n\n Let’s build an\nexample together that implements an nginx SSL termination in front of an apache\nwebserver.<\/p>\n\n\n\n We’ll start of simply with an apache webserver. We’ll also add a service to this pod; so we can get traffic from the inside into our pod. To build this example I leveraged the nginx documentation<\/a> and a digitalocean tutoria<\/a>l.<\/p>\n\n\n\n This will create a pod on port 80 and also a service on port 80. I created a service of type Let’s now add an nginx in front, that will serve SSL. Bare with me, as this is not a single step process.<\/p>\n\n\n\n To start, we’ll create an SSL certificate (self-signed) (following this tutorial<\/a>)<\/p>\n\n\n\n Then fill in your details to get the certificate.<\/p>\n\n\n\n Next, we’ll create two secrets (remember part 3 of our series?) in kubernetes for each of those files:<\/p>\n\n\n\n In addition to our secrets, we’ll also need an nginx configuration file that will tell nginx where to find the certificate and the key – as well as to enable traffic forwarding (or reverse proxying). This nginx configuration file did the job for me, which I subsequently turned into a configmap:<\/p>\n\n\n\n Then we’ll add an nginx container to our apache container, and mount our two secrets and our configmap in our container.<\/p>\n\n\n\n We can create this by first deleting our existing solo pod with So, now you’ve seen a sidecar at work. We’ve added a nginx SSL terminator in front of our apache web server.<\/p>\n\n\n\n The ambassador pattern is another multi-container pod design pattern. With the ambassador pattern, you pair a second container with your main application to broker outside connections. A good example is to use the ambassador pattern is to shard communication to an external system (e.g. a database). Another good use case is to enable service discovery and configuration to an external system. Imagine running the same application on-prem and in multiple cloud environments; all leveraging platform-specific MySQL databases. An ambassador container could be used to enable the application to connect to the right database, without having to build in logic into the main application to do this service discovery. Let’s build an example where we have a simple pod making outbound http connections through the ambassador. We’ll have the ambassador split traffic between the 2 backends, with a 3:1 ratio; meaning 75% of the traffic goes to server 1 and 25% goes to server 2. <\/p>\n\n\n\n Let’s start off with creating our sample application servers. We’ll start of with creating some “dumb” boilerplate HTML, that we’ll then load into an nginx container.<\/p>\n\n\n\n We’ll create both as a configmap in kubernetes:<\/p>\n\n\n\n You might have noticed we used index1.html and index2.html as files. Nginx only server index.html files by default. As I don’t want to go ahead and start moving files around or editing configfiles for nginx, there is a lazy trick you can use to edit the configmap, to rename the file the actual index.html<\/p>\n\n\n\n This will present you with a vi-alike interface, and you can delete the 1 from index1.html. Same thing for server2 as well.<\/p>\n\n\n\n We’ll then create our pods, including a service for each of them. The reason we create a service, is so Kubernetes will do name resolution for us:<\/p>\n\n\n\n These are our applications, that we can reach through DNS within the default namespace on their service names. Let’s now create our pod with an ambassador in there. First of, we’ll create an nginx config (nginx will be our forward proxy\/ambassador) that will do traffic shaping in 3:1 fashion:<\/p>\n\n\n\n After that, we’ll create our ambassador pod:<\/p>\n\n\n\n We can then exec into our ambassador, and do a couple of curls, and watch the result. We should see a 3:1 traffic pattern:<\/p>\n\n\n\n Adding an adapter\ncontainer to an existing application will function like a physical adapter, it\nwill convert one output to another. In the real world we could use an adapter\nto convert a US power cable to fit in a EU socket; in container world we could use\nan adapter to fit container logging and monitoring into prometheus. <\/p>\n\n\n\n Different applications store logs and metrics in different locations, and potentially in different formats. This could be solved by standardizing coding practices, and it could also be solved by implementing an adapter. This adapter will transform the monitoring interface from the application into the interface that the monitoring system (e.g. prometheus) will expect.<\/p>\n\n\n\n Let’s try this out with a Redis container. We’ll setup a prometheus system, and use that to monitor Redis.<\/p>\n\n\n\n We’ll start off by installing the prometheus operator on our cluster. This will allow us to use the kubernetes API to manage prometheus.<\/p>\n\n\n\n This will take about 2 minutes to setup prometheus end-to-end.<\/p>\n\n\n\n With our prometheus operator all setup – we can go ahead and create our adapter pattern, and then leverage the prometheus operator to monitor our pods. First we need to create a service account, role and rolebinding for prometheus to be able to talk the kubernetes API. Let’s do that first:<\/p>\n\n\n\n With that done, we can create our deployment of Redis, and do the prometheus configuration. <\/p>\n\n\n\n We can then access our prometheus pod, and see prometheus actually do the scraping:<\/p>\n\n\n\n Then take a browser and head to (disclaimer, I made it look really easy to get Prometheus up and running and monitoring Redis. I spent a sizeable amount of time and frustration to get this up and running. I hope to save you some time with my steps)<\/em><\/p>\n\n\n\n I believe init-containers<\/a> deserve a spot in the multi-container pod discussion. An init-container is a special container that runs before the other containers in your pod. This can be useful if you have to wait for an external database to be created, or this can be used to do configuration of your application by keeping the configuration piece out of the actual application container.<\/p>\n\n\n\n As a very small example, let’s create an nginx pod – and have an init-container clone a git repo<\/a> that contains our index.html file. I already had a small repo on Github<\/a> with a very interesting index.html file. <\/p>\n\n\n\n When we create this (using Let’s now exec into our pod ( In this section, we learned all about sidecars, ambassadors and adapters; while also introducing init-containers. I highly recommend Brendan Burns’s book to learn more about the patterns themselves, rather than simply how to implement them as I discussed here.<\/p>\n\n\n\n Up to part 5 now?<\/p>\n","protected":false},"excerpt":{"rendered":" This is part 4 in a multi-series on my CKAD study efforts. You can find previous entries here: Part 1: intro, exam topics and my study plan Part 2: Core concepts Part 3: Configuration Part 5: Observability Part 6: Pod Design Part 7: Networking Think back about part 2 of this series: we briefly discussed […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[],"class_list":["post-224","post","type-post","status-publish","format-standard","hentry","category-ckad"],"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/blog.nillsf.com\/index.php\/wp-json\/wp\/v2\/posts\/224","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.nillsf.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.nillsf.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.nillsf.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.nillsf.com\/index.php\/wp-json\/wp\/v2\/comments?post=224"}],"version-history":[{"count":5,"href":"https:\/\/blog.nillsf.com\/index.php\/wp-json\/wp\/v2\/posts\/224\/revisions"}],"predecessor-version":[{"id":295,"href":"https:\/\/blog.nillsf.com\/index.php\/wp-json\/wp\/v2\/posts\/224\/revisions\/295"}],"wp:attachment":[{"href":"https:\/\/blog.nillsf.com\/index.php\/wp-json\/wp\/v2\/media?parent=224"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.nillsf.com\/index.php\/wp-json\/wp\/v2\/categories?post=224"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.nillsf.com\/index.php\/wp-json\/wp\/v2\/tags?post=224"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}Sidecars<\/h2>\n\n\n\n
apiVersion: v1\nkind: Pod\nmetadata:\n name: sidecar\n labels:\n app: web-server\nspec:\n containers:\n - name: apache-web-server\n image: httpd\n ports:\n - containerPort: 80\n---\napiVersion: v1\nkind: Service\nmetadata:\n name: port-80-service\nspec:\n selector:\n app: web-server\n ports:\n - protocol: TCP\n port: 80\n targetPort: 80\n type: LoadBalancer<\/code><\/pre>\n\n\n\nLoadBalancer<\/code>, which will create an Azure Load Balancer that I can then reach. We can its by doing kubectl get service --watch<\/code> (it takes a couple moments for the IP to become assigned). We can then see it works, but our website is not secure:<\/p>\n\n\n\n![\"\"](\"\/wp-content\/uploads\/2019\/07\/2019-07-22-17_08_21-51.143.102.194-and-7-more-pages-Microsoft-Edge.jpg\")
<\/figure>\n\n\n\nmkdir ssl\ncd ssl\nopenssl req -new -newkey rsa:4096 -x509 -sha256 -days 365 -nodes -out MyCertificate.crt -keyout MyKey.key <\/code><\/pre>\n\n\n\nkubectl create secret generic mycert --from-file MyCertificate.crt\nkubectl create secret generic mykey --from-file MyKey.key<\/code><\/pre>\n\n\n\nserver {\n listen 443 ssl default_server;\n server_name localhost;\n ssl_certificate \/etc\/certstore\/MyCertificate.crt;\n ssl_certificate_key \/etc\/keystore\/MyKey.key;\n ssl_ciphers EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH;\n ssl_protocols TLSv1.1 TLSv1.2;\n\n\n location \/ {\n proxy_pass http:\/\/127.0.0.1:80; \n }\n\n\n error_page 500 502 503 504 \/50x.html;\n location = \/50x.html {\n root \/usr\/share\/nginx\/html;\n }\n\n}<\/code><\/pre>\n\n\n\nkubectl create configmap nginx-config --from-file default.conf<\/code><\/pre>\n\n\n\napiVersion: v1\nkind: Pod\nmetadata:\n name: sidecar\n labels:\n app: web-server\nspec:\n containers:\n - name: apache-web-server\n image: httpd\n ports:\n - containerPort: 80\n - name: nginx-ssl-terminator\n image: nginx\n ports:\n - containerPort: 443\n volumeMounts:\n - name: cert\n mountPath: \/etc\/certstore\/\n - name: key\n mountPath: \/etc\/keystore\/\n - name: config\n mountPath: \/etc\/nginx\/conf.d\/\n volumes:\n - name: cert\n secret:\n secretName: mycert\n - name: key\n secret:\n secretName: mykey\n - name: config\n configMap:\n name: nginx-config\n---\napiVersion: v1\nkind: Service\nmetadata:\n name: port-443-service\nspec:\n selector:\n app: web-server\n ports:\n - protocol: TCP\n port: 443\n targetPort: 443\n type: LoadBalancer<\/code><\/pre>\n\n\n\nkubectl delete pod sidecar<\/code>, and then we create the multi-container pod with a new kubectl create -f simplepod.yaml<\/code>. If we give this a couple of moments, this will create all our containers, setup the networking, and afterwards we can browse the ip. You’ll get a giant security warning which means that it’s working (the security warning is due to our self-signed certificate.)<\/p>\n\n\n\n![\"\"](\"\/wp-content\/uploads\/2019\/07\/2019-07-22-17_45_19-https___51.143.98.12-and-7-more-pages-Microsoft-Edge.jpg\")
<\/figure>\n\n\n\nAmbassador<\/h2>\n\n\n\n
<\/p>\n\n\n\n<!DOCTYPE html>\n<html>\n<head>\n <title>Server 1<\/title>\n<\/head>\n<body>\nServer 1\n<\/body>\n<\/html><\/code><\/pre>\n\n\n\n<!DOCTYPE html>\n<html>\n<head>\n <title>Server 2<\/title>\n<\/head>\n<body>\nServer 2\n<\/body>\n<\/html><\/code><\/pre>\n\n\n\nkubectl create configmap server1 --from-file=index1.html\nkubectl create configmap server2 --from-file=index2.html<\/code><\/pre>\n\n\n\nkubectl edit configmap server1<\/code><\/pre>\n\n\n\napiVersion: v1\nkind: Pod\nmetadata:\n name: server1\n labels:\n app: web-server1\nspec:\n containers:\n - name: nginx-1\n image: nginx\n ports:\n - containerPort: 80\n volumeMounts:\n - name: index\n mountPath: \/usr\/share\/nginx\/html\/\n volumes:\n - name: index\n configMap:\n name: server1\n---\napiVersion: v1\nkind: Pod\nmetadata:\n name: server2\n labels:\n app: web-server2\nspec:\n containers:\n - name: nginx-2\n image: nginx\n ports:\n - containerPort: 80\n volumeMounts:\n - name: index\n mountPath: \/usr\/share\/nginx\/html\/\n volumes:\n - name: index\n configMap:\n name: server2\n---\napiVersion: v1\nkind: Service\nmetadata:\n name: server1\nspec:\n selector:\n app: web-server1\n ports:\n - protocol: TCP\n port: 80\n targetPort: 80\n type: ClusterIP\n---\napiVersion: v1\nkind: Service\nmetadata:\n name: server2\nspec:\n selector:\n app: web-server2\n ports:\n - protocol: TCP\n port: 80\n targetPort: 80\n type: ClusterIP<\/code><\/pre>\n\n\n\n upstream myapp1 {\n server server1 weight=3;\n server server2;\n }\n\n server {\n listen 80;\n\n location \/ {\n proxy_pass http:\/\/myapp1;\n }\n }<\/code><\/pre>\n\n\n\nkubectl create configmap nginxambassador --from-file=nginx.conf<\/code><\/pre>\n\n\n\napiVersion: v1\nkind: Pod\nmetadata:\n name: ambassador\n labels:\n app: ambassador\nspec:\n containers:\n - name: busybox-curl\n image: radial\/busyboxplus:curl\n command:\n - sleep\n - \"3600\"\n - name: nginx-ambassador\n image: nginx\n ports:\n - containerPort: 80\n volumeMounts:\n - name: config\n mountPath: \/etc\/nginx\/conf.d\/\n volumes:\n - name: config\n configMap:\n name: nginxambassador<\/code><\/pre>\n\n\n\nWe can then exec into our ambassador, and do a couple of curls, and watch the result. We should see a 3:1 traffic pattern:\n\nkubectl exec -it ambassador sh\ncurl localhost:80 #repeat to see 3:1 distribution<\/code><\/pre>\n\n\n\nAdapter pattern<\/strong><\/h2>\n\n\n\n
helm install stable\/prometheus-operator --name prometheus-operator --namespace monitoring<\/code><\/pre>\n\n\n\nkubectl create serviceaccount prometheus<\/code><\/pre>\n\n\n\napiVersion: rbac.authorization.k8s.io\/v1beta1\nkind: ClusterRole\nmetadata:\n name: prometheus\nrules:\n- apiGroups: [\"\"]\n resources:\n - nodes\n - services\n - endpoints\n - pods\n verbs: [\"get\", \"list\", \"watch\"]\n- apiGroups: [\"\"]\n resources:\n - configmaps\n verbs: [\"get\"]\n- nonResourceURLs: [\"\/metrics\"]\n verbs: [\"get\"]\n---\napiVersion: rbac.authorization.k8s.io\/v1beta1\nkind: ClusterRoleBinding\nmetadata:\n name: prometheus\nroleRef:\n apiGroup: rbac.authorization.k8s.io\n kind: ClusterRole\n name: prometheus\nsubjects:\n- kind: ServiceAccount\n name: prometheus\n namespace: default<\/code><\/pre>\n\n\n\nkubectl create -f clusterrole.yaml<\/code><\/pre>\n\n\n\napiVersion: apps\/v1\nkind: Deployment\nmetadata:\n name: redis-adapter\nspec:\n replicas: 2\n selector:\n matchLabels:\n app: redis-adapter\n template:\n metadata:\n labels:\n app: redis-adapter\n spec:\n containers:\n - image: redis\n name: redis\n - image: oliver006\/redis_exporter #this is our adapter<\/strong>\n name: adapter\n ports:\n - name: redis-exporter\n containerPort: 9121\n---\nkind: Service #we need to create a service to have prometheus do the discovery<\/strong>\napiVersion: v1\nmetadata:\n name: redis-adapter\n labels:\n app: redis-adapter\nspec:\n selector:\n app: redis-adapter\n ports:\n - name: redis-exporter\n port: 9121\n---\napiVersion: monitoring.coreos.com\/v1\nkind: ServiceMonitor #a servicemonitor tells prometheus which services to monitor\/scrape<\/strong>\nmetadata:\n name: redis-adapter\n labels:\n app: redis-adapter\nspec:\n selector:\n matchLabels:\n app: redis-adapter\n endpoints:\n - targetport: 9121\n---\napiVersion: monitoring.coreos.com\/v1\nkind: Prometheus #this creates an actual prometheus instance that will do the scraping<\/strong>\nmetadata:\n name: prometheus\nspec:\n serviceAccountName: prometheus\n serviceMonitorSelector:\n matchLabels:\n app: redis-adapter\n resources:\n requests:\n memory: 400Mi\n enableAdminAPI: false\n<\/code><\/pre>\n\n\n\nkubectl get pods #get the name of the prometheus pod here\nkubectl port-forward prometheus-prometheus-0 9090<\/code><\/pre>\n\n\n\nlocalhost:9090<\/code>. You can either enter queries here (type in redis <\/code>to have autocomplete help out) or go to Status > Targets<\/code> to see your healthly Redis instances.<\/p>\n\n\n\n![\"\"](\"\/wp-content\/uploads\/2019\/07\/2019-07-28-16_24_40-Window.jpg\")
<\/figure>\n\n\n\nInit-containers<\/h2>\n\n\n\n
apiVersion: v1\nkind: Pod\nmetadata:\n name: init\nspec:\n containers:\n - name: nginx-1\n image: nginx\n ports:\n - containerPort: 80\n volumeMounts:\n - name: html\n mountPath: \/usr\/share\/nginx\/html\/\n initContainers:\n - name: clone-github\n image: alpine\/git\n args:\n - clone\n - --single-branch\n - --\n - https:\/\/github.com\/NillsF\/html-demo.git\n - \/usr\/share\/nginx\/html\/\n volumeMounts:\n - name: html\n mountPath: \/usr\/share\/nginx\/html\/\n volumes:\n - name: html\n emptyDir: {}<\/code><\/pre>\n\n\n\nkubectl create -f init.yaml<\/code>) – it is very insightful to do a kubectl get pods --watch<\/code> to see the pod cycle through the git steps.<\/p>\n\n\n\nNAME READY STATUS RESTARTS AGE\ninit 0\/1 Init:0\/1 0 6s\ninit 0\/1 Init:0\/1 0 7s\ninit 0\/1 PodInitializing 0 8s\ninit 1\/1 Running 0 10s<\/code><\/pre>\n\n\n\nkubectl exec -it init sh<\/code>), to see if we actually pulled in our files from git:<\/p>\n\n\n\n# cd \/usr\/share\/nginx\/html\n# cat index.html\n<!DOCTYPE\u00a0html>\n<html>\n<body>\n\n<h1>This is a nice demo.<\/h1>\n<p>My first paragraph.<\/p>\n\n<\/body>\n<\/html>\u00a0<\/code><\/pre>\n\n\n\nSummary of multicontainer pods<\/h2>\n\n\n\n