Azure Kubernetes Service

Author : MD TAREQ HASSAN | Updated : 2021/06/05

Intro

See: Kubernetes Storage Concepts
CSI:
- Container Storage Interface
- K8s extensibily to hook storage services to K8s
- CSI is a unifying effort created by CNCF Storage Working Group, aimed towards defining a standard container storage interface that can enable storage drivers to work on any container orchestrator
- Developers can access storage exposed by a CSI compatible volume driver with the csi volume type on Kubernetes
Storage services i.e. On-premise (SMB) file server, Azure files
- Will be used as K8s persistant volume (PV)
- Pod will use persistant volume claim (PVC) to access K8s PV
- Container will mount (attach) PV to it’s file system will access as if they belong to container’s file system (just like a locak folder/directory)

HostPath

A hostPath volume mounts a file or directory from the host node’s filesystem into your Pod
hostPath volume of type Directory
- A directory must exist at the given path
- From StackOerflow answer
  - hostPath type volumes refer to directories on the Node (VM/machine) where your Pod is scheduled for running (aks-nodepool1-39499429-1 in this case). So you’d need to create this directory at least on that Node
  - To make sure your Pod is consistently scheduled on that specific Node you need to set spec.nodeSelector in the PodTemplate
Not recommended to use. Use only ofr PoC or learning purposes
See: https://kubernetes.io/docs/concepts/storage/volumes/#hostpath

my-hostpath-pv.yaml

kind: PersistentVolume
apiVersion: v1
metadata:
  name: my-hostpath-pv
  labels:
    type: hostpath
spec:
  capacity:
    storage: 2Gi
  accessModes:
    - ReadWriteMany
  persistentVolumeReclaimPolicy: Retain # default
  hostPath:
    type: DirectoryOrCreate
    path: "/data/foo"

my-hostpath-pvc.yaml

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: my-hostpath-pvc
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1Gi
  selector:
    matchLabels:
      type: hostpath

my-app-deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: foo-app-deployment
spec:
  replicas: 3
  selector:
    matchLabels:
      app: foo-app
  template:
    metadata:
      labels:
        app: foo-app
    spec:
      volumes:
      - name: my-hostpath-pv
        persistentVolumeClaim:
          claimName: my-hostpath-pvc
      containers:
      - name: foo-app-container
        image: docker-image-url # i.e. hosted in ACR
        ports:
        - containerPort: 
        volumeMounts:
        - name: foo-app-container-pv-mount
          mountPath: /var/foo
          readOnly: false

On-premise network is connected to Azure VNet (same VNet where AKS belongs) via secured tunnel (i.e. Site-to-Site VPN, ExpressRoute)
There is an on-premise (SMB) file server and apps (pods) need to access that file server or shared folder

Diagram

Using SMB file share as kubenetes volume

Install SMB CSI driver using helm

#
# Check that CSI SMB driver is already installed or not
#
# all release: -a
# all namespace: -A
#
helm list -a -A


#
# If not (already) installed, the add repo and install
#
helm repo add csi-driver-smb https://raw.githubusercontent.com/kubernetes-csi/csi-driver-smb/master/charts
helm install csi-driver-smb csi-driver-smb/csi-driver-smb --namespace kube-system

#
# Search for all available chart versions
#
helm search repo -l csi-driver-smb

#
# To uninstall CSI driver
#
helm uninstall csi-driver-smb -n kube-system

Create secret (on-premise SMB file server user name and password)

Create a secret name “smbcreds” (in demo namespace)
“smbcreds” will be used in Persistant Volume

kubectl create secret generic smbcreds --from-literal username="xxx" --from-literal password="xxx" -n demo

Note: PVs are global objects (non-namespaced) and PVCs belong to a single namespace

Persistant volume (PV): pv-smb.yaml

apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv-smb
spec:
  capacity:
    storage: 100Gi
  accessModes:
    - ReadWriteMany
  persistentVolumeReclaimPolicy: Retain
  mountOptions:
    - dir_mode=0777
    - file_mode=0777
    - vers=3.0
  csi:
    driver: smb.csi.k8s.io
    readOnly: false
    volumeHandle: unique-volumeid  # make sure it's a unique id in the cluster
    volumeAttributes:
      source: "//smb-server-address/sharename"
    nodeStageSecretRef:
      name: smbcreds
      namespace: default

Command: kubectl apply -f pv-smb.yaml (no namespace)

Persistant volume claim (PVC) (static/pre-provisioned): pvc-smb-static.yaml

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: pvc-smb
  namespace: demo
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1Gi
  volumeName: pv-smb
  storageClassName: ""

Command: kubectl apply -f pvc-smb-static.yaml -n demo (namespaced)

Deploy a demo ASP.Net Core MVC App (For MVC App, see below)

apiVersion: v1
kind: Service
metadata:
  name: demo-webapp
  labels:
    app: demo-webapp
#  annotations:
#    service.beta.kubernetes.io/azure-load-balancer-internal: "true"
spec:
  type: LoadBalancer
  ports:
    - port: 80
      targetPort: http
      protocol: TCP
      name: http
  selector:
    app: demo-webapp

---

apiVersion: apps/v1
kind: Deployment
metadata:
  name: demo-webapp
  labels:
    app: demo-webapp
spec:
  replicas: 1
  selector:
    matchLabels:
      app: demo-webapp
  template:
    metadata:
      labels:
        app: demo-webapp
    spec:
      volumes:
        - name: smb
          persistentVolumeClaim:
            claimName: pvc-smb
      containers:
      - name: demo-webapp
        image: myacr.azurecr.io/demoapp:xxxyyyzzz
        ports:
        - name: http
          containerPort: 80
          protocol: TCP
        volumeMounts:
        - name: smb
          mountPath: "/mnt/FooShare"
          readOnly: false

Controllers/HomeController.cs

namespace DemoApp.Controllers
{
    public class SharedFolderModel
    {
        public string MountPath { get; set; } = @"/mnt/FooShare";
        public bool IsAccessible { get; set; } = false;
        public bool HasFiles { get; set; } = false;
        public string[] FileEntries { get; set; }
    }
	
    public class HomeController : Controller
    {
        private readonly ILogger<HomeController> _logger;

        public HomeController(ILogger<HomeController> logger)
        {
            _logger = logger;
        }

        public IActionResult Index()
        {
            var sharedFolder = new SharedFolderModel();

            sharedFolder.IsAccessible = Directory.Exists(sharedFolder.MountPath);

            //Console.WriteLine($"Shared network folder '{mockServerSharedFolderURNPath}' ");
            //Console.WriteLine("IS Accessible");

            if (sharedFolder.IsAccessible)
            {
                sharedFolder.FileEntries = Directory.GetFiles(sharedFolder.MountPath);

                sharedFolder.HasFiles = (sharedFolder.FileEntries?.Length > 0);
            }

            return View(sharedFolder);
        }

        // ... ... ...
    }
}

Views/Home/Index.cshtml

@model DemoApp.Controllers.SharedFolderModel;

@{
    ViewData["Title"] = "Home Page";
}

<div class="text-center">
    <h1 class="display-4">ASP.Net Core MVC App (Count: 12)</h1>
</div>




@if (Model.IsAccessible)
{
    <h4 class="text-success">'@Model.MountPath' is accessible</h4>

    @if (Model.HasFiles)
    {
        <p class="text-primary">Files: </p>

        <ul>
            @foreach (var item in Model.FileEntries)
            {
                <li>@item</li>
            }
        </ul>
    }
    else
    {
        <p class="text-warning">No Files</p>
    }
}
else
{
    <h4 class="text-danger">'@Model.MountPath' is NOT accessible</h4>
}

Links

Author : MD TAREQ HASSAN | Updated : 2021/06/05

Intro

HostPath

SMB File Share

Demo MVC App To Test SMB File Share Access

Azure File Share

NFS File Share