{"id":1530,"date":"2021-01-11T13:59:14","date_gmt":"2021-01-11T21:59:14","guid":{"rendered":"http:\/\/blog.nillsf.com\/?p=1530"},"modified":"2021-01-11T13:59:27","modified_gmt":"2021-01-11T21:59:27","slug":"azure-files-nfs-mounted-on-azure-kubernetes-service","status":"publish","type":"post","link":"https:\/\/blog.nillsf.com\/index.php\/2021\/01\/11\/azure-files-nfs-mounted-on-azure-kubernetes-service\/","title":{"rendered":"Azure Files NFS mounted on Azure Kubernetes Service"},"content":{"rendered":"\n

I am doing some work with a customer around mounting Azure Files using NFS in an AKS cluster. In this blog post, I’m summarizing how to achieve this!<\/p>\n\n\n\n

In many applications, you need to have access to some shared files. There’s a couple of ways of setting this up. In Linux a common approach to accessing shared files is using NFS. NFS or Network File System, is a specification on how to access a remote file system of the network.<\/p>\n\n\n\n

Azure Files is a managed file share service in Azure. Initially, the service only supported CIFS\/SMB as a protocol to mount\/access the files. CIFS\/SMB is similar to NFS, but it’s a different protocol. To make Azure Files more friendly to Linux users, recently a preview for Azure Files exposing an NFS endpoint was announced. <\/p>\n\n\n\n

This feature is in preview in a couple of regions right now, with a couple of limitations. An important limitation is that it is (for now) only supported on premium Azure Files and only on new storage accounts. For a little more details on the limitations, please refer to the documentation<\/a>.<\/p>\n\n\n\n

As a final note before we get started, you need to use a private connectivity method to reach your NFS Azure Files. That means you either need to use a private endpoint<\/a> or a service endpoint<\/a>. In this post, we’ll set up Azure Files with NFS using a private endpoint.<\/p>\n\n\n\n

Setting up Azure Files with NFS<\/h2>\n\n\n\n

The script to deploy the Azure Files with NFS the way I created it is also available on GitHub<\/a>. It doesn’t include the feature registration.<\/p>\n\n\n\n

First, we need to register the preview feature:<\/p>\n\n\n\n

az feature register --name AllowNfsFileShares --namespace Microsoft.Storage\naz provider register --namespace Microsoft.Storage<\/code><\/pre>\n\n\n\n
As per usual, it takes a while for this to register. To get the state, use the following. For me, it took about 15 minutes to register.<\/p>\n\n\n\n
az feature show --name AllowNfsFileShares --namespace Microsoft.Storage --query properties.state<\/code><\/pre>\n\n\n\n
When we’re ready,  we can create the network we need for this work:<\/p>\n\n\n\n
RGNAME=nfsaks\nVNETNAME=nfsaks\naz group create -n $RGNAME -l westus2\naz network vnet create -g $RGNAME -n $VNETNAME \\\n  --address-prefixes 10.0.0.0\/16 --subnet-name aks \\\n  --subnet-prefixes 10.0.0.0\/24\naz network vnet subnet create -g $RGNAME --vnet-name $VNETNAME \\\n  -n NFS --address-prefixes 10.0.1.0\/24 <\/code><\/pre>\n\n\n\n
Next, let’s create the storage account, turn of HTTPS only, and create the share:<\/p>\n\n\n\n
STACC=nfnfsaks\naz storage account create \\\n  --name $STACC \\\n  --resource-group $RGNAME \\\n  --location westus2 \\\n  --sku Premium_LRS \\\n  --kind FileStorage\naz storage account update --https-only false \\\n  --name $STACC --resource-group $RGNAME\naz storage share-rm create \\\n  --storage-account $STACC \\\n  --enabled-protocol NFS \\\n  --root-squash RootSquash \\\n  --name \"akstest\" \\\n  --quota 100<\/code><\/pre>\n\n\n\n
Then next, we need to set up a private endpoint for this. We’ll do that this way:<\/p>\n\n\n\n
SUBNETID=`az network vnet subnet show \\\n  --resource-group $RGNAME \\\n  --vnet-name $VNETNAME \\\n  --name NFS \\\n  --query \"id\" -o tsv `\nSTACCID=`az storage account show \\\n  --resource-group $RGNAME \\\n  --name $STACC \\\n  --query \"id\" -o tsv `\naz network vnet subnet update \\\n  --ids $SUBNETID\\\n  --disable-private-endpoint-network-policies \nENDPOINT=`az network private-endpoint create \\\n  --resource-group $RGNAME \\\n  --name \"$STACC-PrivateEndpoint\" \\\n  --location westus2 \\\n  --subnet $SUBNETID \\\n  --private-connection-resource-id $STACCID\\\n  --group-id \"file\" \\\n  --connection-name \"$STACC-Connection\" \\\n  --query \"id\" -o tsv `<\/code><\/pre>\n\n\n\n
Once we have the private endpoint created, we also need to create a DNS zone and set up an A-record for our storage account:<\/p>\n\n\n\n
DNSZONENAME=\"privatelink.file.core.windows.net\"\nVNETID=`az network vnet show \\\n  --resource-group $RGNAME \\\n  --name $VNETNAME \\\n  --query \"id\" -o tsv`\ndnsZone=`az network private-dns zone create \\\n  --resource-group $RGNAME \\\n  --name $DNSZONENAME \\\n  --query \"id\" -o tsv`\naz network private-dns link vnet create \\\n  --resource-group $RGNAME \\\n  --zone-name $DNSZONENAME \\\n  --name \"$VNETNAME-DnsLink\" \\\n  --virtual-network $VNETID \\\n  --registration-enabled false \n\nENDPOINTNIC=`az network private-endpoint show \\\n  --ids $ENDPOINT \\\n  --query \"networkInterfaces[0].id\" -o tsv `\nENDPOINTIP=`az network nic show \\\n  --ids $ENDPOINTNIC \\\n  --query \"ipConfigurations[0].privateIpAddress\" -o tsv `\n\naz network private-dns record-set a create \\\n        --resource-group $RGNAME \\\n        --zone-name $DNSZONENAME \\\n        --name $STACC \naz network private-dns record-set a add-record \\\n        --resource-group $RGNAME \\\n        --zone-name $DNSZONENAME  \\\n        --record-set-name $STACC \\\n        --ipv4-address $ENDPOINTIP <\/code><\/pre>\n\n\n\n
Testing NFS with a VM<\/h2>\n\n\n\n
Before testing things out in AKS and risking issues with Kubernetes, I wanted to test out NFS using a regular VM. I created a small Ubuntu VM in the AKS subnet:<\/p>\n\n\n\n
AKSSUBNETID=`az network vnet subnet show \\\n  --resource-group $RGNAME \\\n  --vnet-name $VNETNAME \\\n  --name AKS\\\n  --query \"id\" -o tsv `\naz vm create -g $RGNAME -n testnfs \\\n  --subnet $AKSSUBNETID \\\n  --nsg-rule SSH \\\n  --ssh-key-values @~\/.ssh\/id_rsa.pub \\\n  --image ubuntuLTS<\/code><\/pre>\n\n\n\n
Once the VM is created, SSH into it and use the following commands to verify that you can mount NFS:<\/p>\n\n\n\n
sudo apt update && sudo apt install nfs-common -y\nsudo mkdir -p \/mount\/nfnfsaks\/akstest\n\nsudo mount -t nfs nfnfsaks.file.core.windows.net:\/nfnfsaks\/akstest \/mount\/nfnfsaks\/akstest -o vers=4,minorversion=1,sec=sys<\/code><\/pre>\n\n\n\n
This works, and to prove it I created a file, unmounted, remounted and the file was back:<\/p>\n\n\n\n
\"\"
Sanity checking that NFS works correctly.<\/figcaption><\/figure>\n\n\n\n
Now that we’re sure it works, let’s get rid of the VM and test this out in AKS.<\/p>\n\n\n\n
az vm delete -g $RGNAME -n testnfs --yes<\/code><\/pre>\n\n\n\n
Mounting NFS in AKS<\/h2>\n\n\n\n
Let’s first create a new cluster in the AKS subnet we pre-created before:<\/p>\n\n\n\n
AKSSUBNETID=`az network vnet subnet show \\\n  --resource-group $RGNAME \\\n  --vnet-name $VNETNAME \\\n  --name AKS\\\n  --query \"id\" -o tsv `\naz aks create -g $RGNAME -n nfstest \\\n --vnet-subnet-id $AKSSUBNETID \\\n --service-cidr 10.1.0.0\/16 \\\n --dns-service-ip 10.1.0.10 \\\n --network-plugin kubenet\naz aks get-credentials -g $RGNAME -n nfstest<\/code><\/pre>\n\n\n\n
Once we have the cluster created, we can go ahead and create a pod that mounts our NFS share. I used a very lazy approach to mounting NFS, just mounting NFS directly in the pod, not by creating a volume. <\/p>\n\n\n\n
apiVersion: v1\nkind: Pod\nmetadata:\n  name: myapp\nspec:\n  containers:\n  - name: myapp\n    image: busybox\n    command: [\"\/bin\/sh\", \"-ec\", \"sleep 1000\"]\n    volumeMounts:\n      - name: nfs\n        mountPath: \/var\/nfs\n  volumes:\n  - name: nfs\n    nfs:\n      server: nfnfsaks.file.core.windows.net\n      path: \"\/nfnfsaks\/akstest\"<\/code><\/pre>\n\n\n\n
We can create this pod using kubectl create -f pod-with-nfs.yaml<\/code>. <\/p>\n\n\n\n
Once we have the pod, we can exec into it, and see what’s available on the file share. As expected, we see that the file we created earlier is there on the mount point. <\/p>\n\n\n\n
\"\"
The file we created in the VM is now present on the file share.<\/figcaption><\/figure>\n\n\n\n
And that’s how you can mount Azure Files over NFS in an Azure Kubernetes cluster.<\/p>\n\n\n\n
Summary<\/h2>\n\n\n\n
In this blog post, we explored mounting Azure Files via NFS in an AKS cluster. That worked pretty seamlessly. Mounting Azure Files took less time than setting it up since it requires a private endpoint which requires a little bit of work. But once the setup was done, this felt very seamless. <\/p>\n\n\n\n
I’m personally looking forward to this being enabled on standard performance Azure Files accounts as well. But while it’s in preview, this worked like a charm for me.<\/p>\n\n\n\n
<\/p>\n","protected":false},"excerpt":{"rendered":"
I am doing some work with a customer around mounting Azure Files using NFS in an AKS cluster. In this blog post, I’m summarizing how to achieve this! In many applications, you need to have access to some shared files. There’s a couple of ways of setting this up. In Linux a common approach to […]<\/p>\n","protected":false},"author":1,"featured_media":1534,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[2,58],"tags":[115,66,122,67],"class_list":["post-1530","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-azure","category-kubernetes","tag-azure-kubernetes-service","tag-private-endpoint","tag-private-link","tag-storage"],"jetpack_featured_media_url":"https:\/\/nillsfblog.blob.core.windows.net\/media\/2021\/01\/2021-01-11-13_55_25-PowerPoint-Slide-Show-Customize-core-dumps-in-Azure-Kubernetes.pptx.png","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/blog.nillsf.com\/index.php\/wp-json\/wp\/v2\/posts\/1530","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.nillsf.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.nillsf.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.nillsf.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.nillsf.com\/index.php\/wp-json\/wp\/v2\/comments?post=1530"}],"version-history":[{"count":1,"href":"https:\/\/blog.nillsf.com\/index.php\/wp-json\/wp\/v2\/posts\/1530\/revisions"}],"predecessor-version":[{"id":1535,"href":"https:\/\/blog.nillsf.com\/index.php\/wp-json\/wp\/v2\/posts\/1530\/revisions\/1535"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blog.nillsf.com\/index.php\/wp-json\/wp\/v2\/media\/1534"}],"wp:attachment":[{"href":"https:\/\/blog.nillsf.com\/index.php\/wp-json\/wp\/v2\/media?parent=1530"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.nillsf.com\/index.php\/wp-json\/wp\/v2\/categories?post=1530"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.nillsf.com\/index.php\/wp-json\/wp\/v2\/tags?post=1530"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}