Author : MD TAREQ HASSAN | Updated : 2021/05/11

AKS Credentials To Execute Kubectl Commands

Login to Azure (PowerShell is being used)

<pre class="highlight"><code><span class="c">#</span><span class="w"> </span><span class="c"># Login to Azure</span><span class="w"> </span><span class="c">#</span><span class="w"> </span><span class="nv">$</span><span class="nn">Env</span><span class="p">:</span><span class="nv">AZ_USERNAME</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"email-here"</span><span class="w"> </span><span class="nv">$</span><span class="nn">Env</span><span class="p">:</span><span class="nv">AZ_PASSWORD</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"pass-here"</span><span class="w">

</span>az login -u $AZ_USERNAME -p $AZ_PASSWORD </code></pre>

Get AKS Credential (PowerShell is being used)

#
# Get AKS cluster credential
#
$Env:RESOURCE_GROUP = "my-poc-rg"
$Env:AKS_CLUSTER_NAME = "my-poc-aks"

az aks get-credentials --resource-group=$RESOURCE_GROUP --name=$AKS_CLUSTER_NAME --overwrite-existing

# Or
az login --use-device-code

Creating Alias Record In Azure DNS Zone

Example: an alias record set will be created in Azure DNS Zone for (static) public IP (which is assigned to Application Gateway)

Using Azure cli command (in PowerShell ISE)

#
# Create a record set of type "A" with "--target-resource" parameter
# "--target-resource" parameter -> to indicate alias record
# Value passed to "--target-resource" parameter is the "Resource ID" of the public IP of Application Gateway
# 
# az network dns record-set a create -g <resource-group-name> -n <record-set-name> -z <dns-zone-name> --target-resource /subscription/***/
#

#
# Required variables
#
$ResourceGroup = "poc-rg"
$AppGatewayPublicIpName = "poc-agic-gateway-pip"
$DnsZoneName = "poc.com"

#
# Get App Gateway public IP's Resource Id
#
$AppGatewayPublicIpResourceId = (az resource show -g $ResourceGroup -n $AppGatewayPublicIpName --resource-type "Microsoft.Network/publicIPAddresses" --query id --output tsv)
#Echo $AppGatewayPublicIpResourceId

#
# Create alias record for App Gateway public IP in DNS zone
#
az network dns record-set a create -g $ResourceGroup -n '@' -z $DnsZoneName --target-resource $AppGatewayPublicIpResourceId

#
# Check record set
# (use jmespath to filter: https://www.azurecitadel.com/cli/jmespath/)
# az network dns record-set list -g $ResourceGroup -z $DnsZoneName
#
az network dns record-set list -g $ResourceGroup -z $DnsZoneName --query "[?type == 'Microsoft.Network/dnszones/A']"

Strimzi Kafka Kubectl Commands

Get AKS credentials first to execute kubectl commands (see above).

Creating Kafka Topic

Method-1: using yaml manifest file

kafka-topic.yaml

apiVersion: kafka.strimzi.io/v1beta1
kind: KafkaTopic
metadata:
  name: my-topic
  namespace: kafka
  labels:
    strimzi.io/cluster: my-cluster
spec:
  replicas: 3
  partitions: 3
#    retention.ms: 7200000
#    segment.bytes: 1073741824

Method-2: Strimzi Github Repo

kubectl --namespace kafka apply -f https://raw.githubusercontent.com/strimzi/strimzi-kafka-operator/main/examples/topic/kafka-topic.yaml

You can also try following (topic will be created by connecting to kafka leader cluster and executing command directly):

kubectl exec -it my-cluster-kafka-0 -c kafka -- bin/kafka-topics.sh --bootstrap-server my-cluster-kafka-bootstrap:9092 --topic my-topic --create --partitions 3 --replication-factor 3

Listing Kafka Topic

List of topics

# Using 'kubectl get ...'
kubectl get kafkatopics # kubectl get kafkatopics.kafka.strimzi.io
kubectl get kafkatopic foo-topic
kubectl get kt foo-topic # kt -> kafkatopic
kubectl get kt foo-topic -o wide
kubectl get kt foo-topic -o yaml


# Using 'kubectl exec ...'
kubectl exec -it my-cluster-kafka-0 -c kafka -- bin/kafka-topics.sh --bootstrap-server my-cluster-kafka-bootstrap:9092 --list

Get details of a topic

# Using 'kubectl get ...'
kubectl get kafkatopic my-topic
kubectl get kt my-topic

# Using 'kubectl exec ...'
kubectl exec -it my-cluster-kafka-0 -c kafka -- bin/kafka-topics.sh --bootstrap-server my-cluster-kafka-bootstrap:9092 --topic my-topic --describe

Deleteing Kafka Topic

Deleteing single user

kubectl delete $(kubectl get kafkatopic -o name)

Deleteing all users

kubectl delete $(kubectl get kafkatopics -o name)

Creating Kafka User

If you are using Azure cloudshell: code foo-user.yaml

foo-user.yaml

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaUser
metadata:
  name: foo-user
  namespace: kafka
  labels:
    strimzi.io/cluster: my-cluster
spec:
  authentication:
    type: tls
  authorization:
    type: simple
    acls:
      - resource:
          type: topic
          name: my-topic
          patternType: literal
        operation: All
      - resource:
          type: topic
          name: test-topic
          patternType: literal
        operation: read

Command: kubectl apply -f foo-user.yaml --namespace kafka

Listing Kafka Users

All users

kubectl get kafkausers
kubectl get kafkausers -o name # name only
kubectl get kafkausers -o wide
kubectl get kafkausers -o yaml # output as yaml
kubectl get kafkausers -o json # output as json

Single user

kubectl get kafkauser foo-user 
kubectl get ku foo-user # kafkauser -> ku
kubectl get ku foo-user -o <output-flag>

Deleting Kafka User

Deleteing single user

kubectl delete $(kubectl get ku foo-user -o name)

Deleteing all users

kubectl delete $(kubectl get kafkausers -o name)

Consumer Group

Create Consumer Group

kafka-console-consumer.bat --bootstrap-server <LOADBALANCER_PUBLIC_IP>:9094 --topic my-topic --consumer.config client-ssl.properties --from-beginning --group cg-xyz

List consumer group

kafka-consumer-groups.bat --bootstrap-server <LOADBALANCER_PUBLIC_IP>:9094 -list --command-config client-ssl.properties
kafka-consumer-groups.bat --bootstrap-server 20.194.x.y:9094 --list --command-config client-ssl.properties

# in local -> ./bin/kafka-consumer-groups.sh --list --bootstrap-server localhost:9092

List consumer group members

kafka-consumer-groups.bat --bootstrap-server 20.194.x.y:9094 --command-config client-ssl.properties --describe --group new-user --members

# local -> ./bin/kafka-consumer-groups.sh --describe --group new-user --members --bootstrap-server localhost:9092

Details: