Create a cluster through the Azure Panel.
In order to interact with the cluster from the cli, we need to install Azure CLI tool. You can follow
this tutorial. Basically on a mac it comes to executing these commands:
brew update && brew install azure-cli
Once we have the az tool, we have to login to the azure:
az login
(for CI we need a different way to login, see more here)
To login to a specific tenant / directory we have to list the active subsriptions first:
az account tenant list
[output example:]
[
{
"id": "/tenants/8e07940e-6ef3-40ab-3456-52af33fba4c9",
"tenantId": "8e07940e-6ef3-40ab-3456-52af33fba4c9"
},
{
"id": "/tenants/48e08919-3ab3-2134-8429-21696172c823",
"tenantId": "48e08919-3ab3-2134-8429-21696172c823"
},
{
"id": "/tenants/ac4c473f-2134-4bf3-80a6-02ff111b01ba",
"tenantId": "ac4c473f-2134-4bf3-80a6-02ff111b01ba"
}
]
Now you can login using:
az login --tenant "YOUR-TENANT-HASH-FROM-ABOVE-COMMAND" --allow-no-subscriptions
az group list # check if you can see the proper resource groups
Once successfully logged in, go to the Kubernetes Services and select your cluster. Then if you click on Connect button you will see how to connect to the cluster. Basically we have to run two commands:
az account set --subscription [the-hash-here]
az aks get-credentials --resource-group [resource-group-name-here] --name [cluster-name-here]
To get the resource ID of your resource group
az group show --name [[RESOURCE_GROUP]] --query id --output tsv
To create a Service Principal
az ad sp create-for-rbac --name [[APPNAME-OPTIONAL]] --scope [[ABOVE_GROUP_ID]] --role Contributor --sdk-auth
This will output a JSON object with some IDs.. normally you copy it and save it in eg. Github secrets under AZURE_CREDENTIALS, which will be needed for the login process, eg:
- name: 'Login via Azure CLI'
uses: azure/login@v1
with:
# az ad sp create-for-rbac --name [[APPNAME]] --role contributor --scopes /subscriptions/[[SUBSCRIPTION_ID]]/resourceGroups/[[RESOURCE_GROUP]] --sdk-auth
creds: ${{ secrets.AZURE_CREDENTIALS }}
When while deploying to Containers Services you may get this error:
Error: The subscription is not registered to use namespace 'Microsoft.ContainerInstance'
This means that we have to register this namespace in our subscription:
az account list --output table
az provider register --namespace Microsoft.ContainerInstance
az provider show --namespace Microsoft.ContainerInstance --query "registrationState"
Download the latest deploy.yaml file from: https://kubernetes.github.io/ingress-nginx/deploy/#azure
Also take a look at the config for azure: https://learn.microsoft.com/en-us/azure/aks/ingress-basic?tabs=azure-cli#create-an-ingress-controller
Make sure that externalTrafficPolicy flag is set to Local!
Also if we want to set, eg. - --enable-ssl-passthrough option we can do it now. Simply add this at the end of the arguments array:
Now we can run kubectl apply -f ingress.yaml and this will create a new namespace called ingress-nginx will all relevant resources.
To validate if everything is fine, let's run the below commands:
We need a sample app to test the setup. Let's use the nginx image, so the app.yaml manifest may look like:
run kubectl apply -f app.yaml and you can forward the ports to see if all is as expected:
helm repo add jetstack https://charts.jetstack.io
helm repo update
helm upgrade cert-manager jetstack/cert-manager \
--install \
--create-namespace \
--wait \
--namespace cert-manager \
--set installCRDs=true
Install cert manager: https://cert-manager.io/docs/installation/kubectl/
Verify the installation: kubectl get pods --namespace cert-manager
Now we need the Issuers (prod and stage). Take a look how the issuers may look like: https://cert-manager.io/docs/configuration/acme/
Once we have the stage.yaml and/or prod.yaml issuers simply apply them.
It's time to apply for the certificates. Create an Issuer for your app, eg:
NOTE! Before we can apply the above, we have to redirect the domain(s) to the LoadBalancer IP address
We have to follow steps 1-5 to setup the appriopriate permissions:
https://github.com/Azure/aks-baseline-automation/blob/main/workloads/docs/app-flask-push-dockerbuild.md
The ghToAzAuth.sh variables may look like:
APPNAME=[PROJECT_NAME]
RG=[AKS resource group name, eg. rg]
GHORG=[your github user name]
GHREPO=[github repo name]
GHBRANCH=[git branch name, eg: master]
GHENV=[prod, stage, etc. This only matters if you use the github secrets]
Execute the above script by running: This should give us all necessary values, such as: AZURE_CLIENT_ID, AZURE_TENANT_ID and AZURE_SUBSCRIPTION_ID
Now it will be possible to run a github action step:
...
jobs:
login-example:
runs-on: ubuntu-latest
environment: prod
steps:
...
- uses: azure/login@v1
with:
client-id: ${{ secrets.AZURE_CLIENT_ID }}
tenant-id: ${{ secrets.AZURE_TENANT_ID }}
subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}
enable-AzPSSession: true
Once we are logged in to Azure in github actions we can have a step similar to the following:
In order for kuber to pull images from ACR we need to create a secret with the ACR credentials. Take a look at this file acr.sh. Before executing it you have to change SERVICE_PRINCIPAL_NAME and ACR_NAME values where SERVICE_PRINCIPAL_NAME has to be a unique name.
/bin/bash acr.sh
Copy those values as you won't be able to retrieve them anymore. As an output this script will give us the secrets needed for the below command.
Now we have to create a secret itself (reference):
Basically the manifest (statefulset and service) may look similar to:
In terms of the storageClassName we have to fist check what the provider gives us. Let's run
kubectl get sc
### output for azure:
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
azurefile file.csi.azure.com Delete Immediate true 9d
azurefile-csi file.csi.azure.com Delete Immediate true 9d
azurefile-csi-premium file.csi.azure.com Delete Immediate true 9d
azurefile-premium file.csi.azure.com Delete Immediate true 9d
default (default) disk.csi.azure.com Delete WaitForFirstConsumer true 9d
managed disk.csi.azure.com Delete WaitForFirstConsumer true 9d
managed-csi disk.csi.azure.com Delete WaitForFirstConsumer true 9d
managed-csi-premium disk.csi.azure.com Delete WaitForFirstConsumer true 9d
managed-premium disk.csi.azure.com Delete WaitForFirstConsumer true 9d
As you can see there are two types of the storage in azure (read more here: https://learn.microsoft.com/en-us/azure/aks/concepts-storage#azure-disk) I tried the azurefile but due to the documentation, for the database purposes it's better to use the azuredisk intead, so `managed-csi` in particular. I am not sure though if that's correct...