Using Internal Load Balancer

Author : MD TAREQ HASSAN | Updated : 2021/05/19

Prerequisites

Requirements

Make sure the managed identity given to AKS cluster has ‘Contributor’ role to AKS managed resource group
- AKS managed identity: clsutername-agentpool (i.e. my-aks-agentpool)
- AKS managed resource group:
  - Automatically created when AKS is provisioned
  - Name: MC_resourcegroupname_clsutername_region (i.e. MC_my-rg_my-aks_japaneast)
Make sure that managed identity given to AKS cluster has ‘Network Contributor’ role to user (you) managed resource group
- Azure portal > your resource group
- Access control (IAM) > ‘Role assignments’ tab > “+ Add”
- Assign access to: User Managed Identity > search AKS managed identity name > Select
- Add

For New Cluster

Annotation for internal listeners: service.beta.kubernetes.io/azure-load-balancer-internal: "true"
For Strimzi Kafka
- Make sure you installed Strimzi in AKS
- template with annotations will be used to make internal load balancers

K8s LoadBalancer service: my-internal-lb.yaml

apiVersion: v1
kind: Service
metadata:
  name: my-intra-app
  annotations:
    service.beta.kubernetes.io/azure-load-balancer-internal: "true"
spec:
  type: LoadBalancer
  ports:
  - port: 80
  selector:
    app: my-intra-app-pod

Commands:

#
# Login and get credentials
#
az login
az aks get-credentials --resource-group xyz-rg --name xyz-aks-cluster

#
# Create kafka cluster with internal load balancer
# 
kubectl apply -f my-internal-lb.yaml --namespace test

Strimzi Kafka

... ... ...

spec:
  kafka:
    ... ... ...
	
    listeners:
      ... ... ...
	  
    template:
      externalBootstrapService:
        metadata:
          annotations:
            service.beta.kubernetes.io/azure-load-balancer-internal: "true"
      perPodService:
        metadata:
          annotations:
            service.beta.kubernetes.io/azure-load-balancer-internal: "true"
	... ... ...		

kafka-persistent-with-internal-lb.yaml

apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
  name: my-cluster
  namespace: kafka
spec:
  kafka:
    version: 2.7.0
    replicas: 3
    authorization:
      type: simple
      superUsers:
        - CN=strimzi-kafka-admin
    listeners:
      - name: plain
        port: 9092
        type: internal
        tls: false
      - name: tls
        port: 9093
        type: internal
        tls: true
      - name: external
        port: 9094
        type: loadbalancer
        tls: true
        authentication:
          type: tls
    template:
      externalBootstrapService:
        metadata:
          annotations:
            service.beta.kubernetes.io/azure-load-balancer-internal: "true"
      perPodService:
        metadata:
          annotations:
            service.beta.kubernetes.io/azure-load-balancer-internal: "true"
    config:
      offsets.topic.replication.factor: 3
      transaction.state.log.replication.factor: 3
      transaction.state.log.min.isr: 2
      log.message.format.version: "2.7"
      inter.broker.protocol.version: "2.7"
    storage:
      type: jbod
      volumes:
      - id: 0
        type: persistent-claim
        size: 10Gi
        deleteClaim: false
  zookeeper:
    replicas: 3
    storage:
      type: persistent-claim
      size: 10Gi
      deleteClaim: false
  entityOperator:
    topicOperator: {}
    userOperator: {}

Commands:

#
# Login and get credentials
#
az login
az aks get-credentials --resource-group xyz-rg --name xyz-aks-cluster

#
# Create kafka cluster with internal load balancer
# 
kubectl apply -f kafka-persistent-with-internal-lb.yaml --namespace test

For Existing Cluster

Add template section (after listeners section) with nnotation service.beta.kubernetes.io/azure-load-balancer-internal: "true"

... ... ...

spec:
  kafka:
    ... ... ...
	
    listeners:
      ... ... ...
	  
    template:
      externalBootstrapService:
        metadata:
          annotations:
            service.beta.kubernetes.io/azure-load-balancer-internal: "true"
      perPodService:
        metadata:
          annotations:
            service.beta.kubernetes.io/azure-load-balancer-internal: "true"
	... ... ...		

Commands:

#
# Login and get credentials
#
az login
az aks get-credentials --resource-group xyz-rg --name xyz-aks-cluster

#
# Update existing cluster with internal load balancer (load balancer: Public IP -> Private IP)
# 
kubectl apply -f kafka-persistent-with-internal-lb.yaml --namespace test

Author : MD TAREQ HASSAN | Updated : 2021/05/19

Prerequisites

For New Cluster

For Existing Cluster

Links