Deploying LOGIQ PaaS in Azure Kubernetes Service

The following guide takes you through deploying LOGIQ PaaS in an Azure Kubernetes Service cluster. The deployment involves the following steps:
​Creating an Azure Kubernetes Service cluster​
​Configuring the node pool​
​Connecting to the cluster​
​Creating the MinIO blob storage gateway​
​Deploying LOGIQ PaaS​
If you have an AKS cluster that is appropriately sized for deploying LOGIQ and handling your data ingestion rate, you can skip the AKS cluster creation step. However, you must label the nodes as specified in the Node pool configuration table failing which the pods in the cluster will not land in any of the nodes. 
Creating an Azure Kubernetes Cluster
Install Azure CLI.
Connect to your Azure account using Azure CLI by running the following command.
1
az login --tenant “tenant-id”
Copied!
Create a resource group by running the following command.
1
az group create --name <Resource group name> --location <location>
Copied!
Configuring the node pool
Create four node pools as described and labelled in the following table. The table describes the node pool configuration for ingesting 100 GB of data per day.
Important: Ensure that the node pools are all created in the same availability zone.
Size
Node pool
Node labels
Node count
8 Core 16 GB RAM (F8s_v2)
ingest (CPU intensive)
logiq.ai/node=ingest
1
8 Core 16 GB RAM (F8s_v2)
common 
logiq.ai/node=common
1
4 Core 8 GB RAM (F4s_v2)
db
logiq.ai/node=db
1
4 Core 8 GB RAM (F4s_v2)
hauler
logiq.ai/node=hauler
1
Execute the following commands to create the AKS cluster along with the node pools described above.
1
az aks create --kubernetes-version 1.19.11 -g <your resource group> --name <cluster name> -l <location> --node-vm-size Standard_f8s_v2 --zones 1 --nodepool-name ingest --nodepool-labels logiq.ai/node=ingest --node-count 1 --enable-ultra-ssd
2
​
3
az aks nodepool add --cluster-name <cluster name> --name common --resource-group <You resource group> --zones 1 --labels logiq.ai/node=common --node-vm-size standard_f8s_v2 --node-count 1 --enable-ultra-ssd
4
​
5
az aks nodepool add --cluster-name <cluster name> --name db --resource-group <You resource group> --zones 1 --labels logiq.ai/node=db --node-vm-size standard_f4s_v2 --node-count 1 --enable-ultra-ssd
6
​
7
az aks nodepool add --cluster-name <cluster name> --name hauler --resource-group <You resource group> --zones 1 --labels logiq.ai/node=hauler --node-vm-size standard_f4s_v2 --node-count 1 --enable-ultra-ssd
Copied!
The following code block depicts example commands for creating the AKS cluster with the node pool specification provided in the table above.
1
az aks create -g azure-test --name logiqai -l eastus --node-vm-size Standard_f8s_v2 --zones 1 --nodepool-name ingest --node-count 1 --nodepool-labels logiq.ai/node=ingest --enable-ultra-ssd
2
​
3
az aks nodepool add --cluster-name logiqai --name common --resource-group azure-test --zones 1 --labels logiq.ai/node=common --node-vm-size standard_f8s_v2 --node-count 1 --enable-ultra-ssd
4
​
5
az aks nodepool add --cluster-name logiqai --name hauler --resource-group azure-test --zones 1 --labels logiq.ai/node=hauler --node-vm-size standard_f4s_v2 --node-count 1 --enable-ultra-ssd
6
​
7
az aks nodepool add --cluster-name logiqai --name db --resource-group azure-test --zones 1 --labels logiq.ai/node=db --node-vm-size standard_f4s_v2 --node-count 1 --enable-ultra-ssd
Copied!
​
Connecting with your AKS cluster
Connect to the AKS cluster by first visiting the Azure portal, navigating to the AKS cluster you created, and selecting it. Next, click on the Connect icon and follow the instructions displayed on the right panel. Execute the following command and you should see the nodes in your cluster.
1
kubectl get node
Copied!
Creating the MinIO blob storage gateway for S3 compatibility
Follow the instructions on MinIO’s site to create an Azure blog storage account. Once you login, click the “+” button on the right hand corner of the screen to create a bucket named logiq. Note down this bucket name since we'd be using it in later steps. 
Create an Azure ultra disk storage class using the YAML configuration provided below. 
1
kind: StorageClass
2
apiVersion: storage.k8s.io/v1
3
metadata:
4
name: ultra-disk-sc
5
provisioner: kubernetes.io/azure-disk
6
volumeBindingMode: WaitForFirstConsumer # optional, but recommended if you want to wait until the pod that will use this disk is created
7
parameters:
8
skuname: UltraSSD_LRS
9
cachingMode: None
10
kind: managed
11
diskIopsReadWrite: "3000" # minimum value: 2 IOPS/GiB
12
diskMbpsReadWrite: "125" # minimum value: 0.032/GiB
Copied!
Verify that the storage class has been created by running the following command. 
1
kubectl get sc ultra-disk-sc
Copied!
Deploying LOGIQ PaaS
Download the values.yaml file from this location and replace the following variables in the file.
1
awsServiceEndpoint: https://<your-blobstorage-accountname>.az.minio.io
2
s3_bucket with the minio bucketname created at step 5.
3
AWS_ACCESS_KEY_ID: minio service name
4
AWS_SECRET_ACCESS_KEY: access key from the mino.
5
storageClass: ultra-disk-sc
6
creatStorageClass: true
Copied!
Next, follow the instructions on LOGIQ’s Quickstart guide to spin up the LOGIQ stack on this AKS cluster.  
Once you've successfully deployed LOGIQ, you can (optionally) disable monitoring AKS with container insights on your cluster by running the following command. 
1
az aks disable-addons -a monitoring -n <aks cluster name> -g <resource group>
Copied!
​

Deploying Logiq EKS with AWS ALB

Azure Blob Storage Lifecycle Management

Last modified 5mo ago

Export as PDF

Copy link

Edit on GitHub

Contents

Creating an Azure Kubernetes Cluster

Configuring the node pool

Connecting with your AKS cluster

Creating the MinIO blob storage gateway for S3 compatibility

Deploying LOGIQ PaaS