Retrieving AKS on Azure Local admin credentials
I was asked a question recently how could a customer connect to an Azure Local hosted AKS instance via kubectl, so they could use locally hosted Argo CD pipelines to manage their containerized workloads (e.g. via Helm charts). The reason for the question is that the Microsoft documentation doesn’t make it clear how to do it. E.g. The How To Create a Kubernetes Cluster using Azure CLI shows that once you’ve created a cluster, you use the az connectedk8s proxy azure cli command to create a proxy connection allowing kubectl commands to be run.
That’s fine if you’re doing interactive sessions, such as running the az cli proxy command in one shell, and in another running kubectl, but once the proxy session is closed, the kube config is useless, as it looks something like this:
If you try and run a kubectl when the proxy session is closed, it will fail. Editing the config file with the URL with one of the IP addresses of the control nodes and port 6443 doesn’t work either as the client certificate is not provided.
To get the correct config we have 3 options:
Use Azure CLI to retrieve the admin credentials
SSH to one of the AKS control-plane nodes and retrieve the admin.conf file
Run a debug container on one of the control-plane nodes and retrieve the admin.conf file
I’ll go through all 3 options, starting with the easiest
Using Azure CLI
Assuming you’re already logged in to the correct Azure tenant, you can use a single command to get the admin credentials:
az aksarc get-credentials --resource-group <myResourceGroup> --name <myAKSCluster> --admin --file <myclusterconf>
A word of caution when running the command; if you don’t supply the —file parameter, it merges the config into the default ~/.kube/config file. If you want to use the credentials for automation purposes, it’s best to have them in a separate file, otherwise there could be a security risk.
Here we’re proving that we’re using the accessible URL via the config file that’s just been created to query the API for the cluster info:
Using SSH connecting to control-plane node
The second option is to SSH to one of the control-plane nodes to retrieve the details. This is a bit more involved and assumes you have the Private SSH key file, assuming you provided it at deployment time, or retrieved the one created for you. You will also need to know the IP address of one of the nodes, and there’s no way to get this from the Azure portal, so will have to use kubectl .
az connectedk8s proxy --name <myResourceGroup> --resource-group <myAKSCLuster>
From a new shell, run:
# Prove we're connected to the AKS cluster via the proxy
kubectl cluster-info
# Get the node information and IP addresses
kubectl get nodes -o wide
We can see the cluster is connected via the proxy session and a list of all the cluster nodes. We want to connect to a control-plane node, so select one that has that role and make a note of the internal IP address.
Use the following command to connect to the node:
ssh -i <location of ssh private key file> clouduser@<control-plane ip>
# If you get an error regarding permissions to key file being too open, run:
chmod 400 <location of ssh private key file>
Now we’re in, we can exit the session as we know that works and use the following command to write the output to a file on your local system:
ssh -i <location of ssh private key file> clouduser@<control-plane ip> "sudo cat /etc/kubernetes/admin.conf" > aks-01-admin.conf
Now we can quit our proxy session and test connectivity using the file retrieved from the node:
kubectl --kubeconfig ~/aks-01-admin.conf cluster-info
Running a debug container on one of the control-plane nodes
You can use this method if you don’t have the private SSH key to connect to the node directly.
We need to use make a proxy connection to the cluster first of all to be able to run the kubectl commands
az connectedk8s proxy --name <myResourceGroup> --resource-group <myAKSCLuster>
# Prove we're connected to the AKS cluster via the proxy
kubectl cluster-info
# Get the node information and IP addresses
kubectl get nodes -o wideFrom the node output, we want to get the name of one of the control-plane nodes and use that in the following commands:
kubectl debug node/control-plane-node -it --image=mcr.microsoft.com/cbl-mariner/base/core:2.0
# use chroot to use the host file system
chroot /host
# copy the file to /tmp
cp /etc/kubernetes/admin.conf /tmp/admin.conf
Keep the debug session open and from another shell run the following to copy the file from the debug container to your local machine:
#get the name of the running debug container
kubectl get pods -A | grep node-debugger
#copy the tmp file:
kubectl exec pod/node-debugger-moc-lkopgnn203v-lnb2j -- cat /host/tmp/admin.conf > ./debug-admin.conf
Go back to the debug shell and exit from it.
Once you’ve finished with the debug container, you can remove the pod from the cluster:
# Get the name of debug pods
kubectl --kubeconfig ~/aks-01-admin.conf get pods -A | grep node-debugger
# Delete debug pod
kubectl --kubeconfig ~/aks-01-admin.conf delete pod <node-debugger-pod-name>
You can see in the screenshot above that I had 2 debug sessions listed, one running and the other completed. The running session is because I didn’t exit from the pod, and went off to do something else (oops!). Not to worry, deleting the pod sorts that out.
Conclusion
Clearly, the easiest way to retrieve the admin credentials is to use method one, use the Azure CLI aksarc extension to retrieve them. I wanted to show the other methods, though, as a reference on how to connect to the control-plane pods and retrieve a file.
Azure Monitor Container Insights cost presets (preview)
Microsoft have recently added a new preview feature within the Azure portal to perform some cost optimization on the Container Insights data that is collected to a Log Analytics workspace. This is helpful as going with the default collection could leave you with unnecessary cost, as you don’t care about particular namespaces, or you’ve got a non-production cluster, so don’t care about collecting data every minute, but still want to monitor your cluster and workloads.
Here’s how you can quickly set this up.
From the Azure Portal, open up the blade for your Kubernetes resource (works for both AKS and Arc enabled Kubernetes
Navigate to Monitoring / Insights and then Configure azure monitor (If you’ve not previously enabled it)
If you have configure container insights previously, click on Monitor Settings :
From the blade that’s opened, you’ll need to make sure that Use Managed Identity (preview) is enabled. This appears for Azure Arc for Kubernetes enabled clusters. If this isn’t checked, you will find the Costs presets (preview) list box is grayed out.
From the list box, you can choose from the following profiles:
Standard
Cost-optimized
Custom
None
Standard profile has the following settings:
1 minute collection frequency
No namespace filtering
Syslog collection disabled
Cost-optimized profile has the following settings:
5 minute collection frequency
3 namespaces excluded [kube-system, gatekeeper-system, azure-arc]
Syslog collection disabled
You can use the defined standards as the basis for your own custom collection profile.
Select a profile from the drop down and click on Edit collection settings.
You can now modify the configuration to meet your requirements. For ContainerLog filtering, you can use the link here for more information. The profile will be saved as Custom within the Cost presets list box.
You can also enable Syslog collection for security events on your nodes, but that will increase your Log Analytic costs somewhat, depending on how busy your cluster is!
Azure Arc delivers SQL Databases as Resources in Azure
The SQL extension for Arc-connected servers itself isn’t new, however, it has had a new feature added that is currently in preview; which is the ability to create Azure resources that represent your SQL Database through the Arc-connected Servers.
You can see this will detect objects for SQL instances, named SQL instances, and the databases themselves become objects in Azure
If you already have this installed this is one configuration setting you may want to check and that is “Has NT AUTHORITY\SYSTEM in the sysadmin role.” This is listed in the prerequisites of the install instructions that can be found here View SQL Server databases - SQL Server | Microsoft Learn
It is worth noting this currently works only for the Microsoft.HybridCompute type and not for example the VMware connected Arc systems Microsoft.ConnectedVMwarevSphere.
There are some properties here listed as unknown however, this is a bug that has already been fixed.
This is the next step to bringing SQL databases into the Azure control plane, creating alerts on recovery models, encryption state, or perhaps statistic creation. Enabling more visibility and control into the Hybrid Cloud journey.
This is the first public preview of this feature and I’ll definitely be keeping an eye out for future revisions and enhancements.
Azure Arc Connected vSphere (Preview)
This isn’t a groundbreaking demo in terms of the end result; we deploy some VMs, however, the interesting conversation is about how we got there and what we can do with these VMs. With Azure Arc, we can now connect vSphere to the Azure platform, meaning using Azure we can use cloud methods to deploy servers such as Azure templates. This is a step towards creating an environment with a single-pane-of-glass, at least to view and query your IT estate.
The features today are simple, but Azure is a constantly evolving platform and when you are connected, as new features and services are developed and released, and the ecosystem evolves, you can grow with it.
The goal of this article is to give you some ideas about why connecting your vSphere cluster to Azure via Azure Arc and how having VMWare-connected VMs can start providing a path to a consistent cloud experience. This allows administrators to develop management processes in the cloud without requiring a complete lift and shift of all existing servers. As of this writing, this feature is still in preview.
This is assuming you have deployed the Azure Arc bridge to connect your on-premises resources. This process is evolving and is not meant to be the focus of this article.
Once connected using the Azure portal, you can do a simple VM deployment.
Click, click, VM name, deploy-demo-01 next, next, finish.
We can save this as an ARM template, I created a template spec and saved it to Azure, which I am using through the portal to deploy VM deploy-demo-02 but you could use pipeline tools such as Azure DevOps or other CI/CD tooling or templates from PowerShell command line.
You could use PowerShell to deploy that ARM Template to deploy deploy-demo-03
you could convert this into a Bicep template if that’s your standard.
What I do have now is 3 new vSphere hosted VMs all deployed via Azure ARM
however, I never touched the VMware console, and I now have an Azure Resource object.
You have access to basic controls. You could assign this via Azure RBAC rather than having administrators and users needing to access the vSphere console. There are options for resizing, adding disks, and managing networking, standardizing the experience.
In addition, you can also onboard existing deployed vSphere VMs and enable guest management to help manage things like VMware tools versions from Azure.
While the monitoring agent options are changing, I’ll keep this simple. You could also include VM extension such as the monitoring agent in your template.
which in term allows us to query these servers via Log analytics. This means it can be connected to alerts like low disk space and other services like Azure Policy.
This space is changing, there are a lot of new features coming to Azure Arc which we see as a gateway to connecting your existing IT servers to Azure allowing companies to gain the benefits of Azure without having to execute migrations for every server.
Hopefully, you found this insightful or informative
Azure Arc Connected K8s Billing usage workbook error - 'union' operator: Failed to resolve table expression named 'ContainerLogV2'
Using ‘Container Insights’ for Kubernetes can get expensive, I recently saw a customer creating 19TB of logs a month from logging container insights from an AKS cluster.
You may want to check what logs you are capturing as suggested through Microsoft guides Monitoring cost for Container insights - Azure Monitor | Microsoft Docs which suggests one of the first steps to check billing usage.
As of writing this if you enable container insights and try to view Data Usage you get an error. ‘union' operator: Failed to resolve table expression named 'ContainerLogV2’. ContainerLogV2 is in preview or if you are just aren’t ready to deploy that, here is how to fix the report manually. I also have included the Full report at the end.
However, depending on when you deploy this and how you deployed insight metrics you may hit this error.
As of writing this, ContainerLogV2 is in preview and you can follow this guide to deploy it. Configure the ContainerLogV2 schema (preview) for Container Insights - Azure Monitor | Microsoft Docs. If you want to gain some quick insight into your usage you can edit the workbook. We just need to get through the maze of edits to get to the underlying query.
Select ‘Azure Arc’ services
select ‘Kubernetes clusters’ under infrastructure
select your K8s cluster
select ‘Workbooks’ under Monitoring
select ‘Container Insights Usage’ workbook
you can either edit this one or ‘Save As’ and create a new name. I will create a copy.
Open your chosen workbook
you can now edit the group item.
by Namespace
Editing the Namespace selection Dropdown parameter query
Hopefully, you get the gist of the process. There are a few more queries to edit.
And
This will now work for Data Usage. This is likely a temporary solution until ContainerLogV2 comes out of preview. Configure the ContainerLogV2 schema (preview) for Container Insights - Azure Monitor | Microsoft Docs
Gallery Template code
{
"version": "Notebook/1.0",
"items": [
{
"type": 9,
"content": {
"version": "KqlParameterItem/1.0",
"crossComponentResources": [
"{resource}"
],
"parameters": [
{
"id": "670aac26-0ffe-4f29-81c2-d48911bc64b6",
"version": "KqlParameterItem/1.0",
"name": "timeRange",
"label": "Time Range",
"type": 4,
"description": "Select time-range for data selection",
"isRequired": true,
"value": {
"durationMs": 21600000
},
"typeSettings": {
"selectableValues": [
{
"durationMs": 300000
},
{
"durationMs": 900000
},
{
"durationMs": 3600000
},
{
"durationMs": 14400000
},
{
"durationMs": 43200000
},
{
"durationMs": 86400000
},
{
"durationMs": 172800000
},
{
"durationMs": 259200000
},
{
"durationMs": 604800000
},
{
"durationMs": 1209600000
},
{
"durationMs": 2419200000
},
{
"durationMs": 2592000000
},
{
"durationMs": 5184000000
},
{
"durationMs": 7776000000
}
],
"allowCustom": true
}
},
{
"id": "bfc96857-81df-4f0d-b958-81f96d28ddeb",
"version": "KqlParameterItem/1.0",
"name": "resource",
"type": 5,
"isRequired": true,
"isHiddenWhenLocked": true,
"typeSettings": {
"additionalResourceOptions": [
"value::1"
],
"showDefault": false
},
"timeContext": {
"durationMs": 21600000
},
"timeContextFromParameter": "timeRange",
"defaultValue": "value::1"
},
{
"id": "8d48ec94-fde6-487c-98bf-f1295f5d8b81",
"version": "KqlParameterItem/1.0",
"name": "resourceType",
"type": 7,
"description": "Resource type of resource",
"isRequired": true,
"query": "{\"version\":\"1.0.0\",\"content\":\"\\\"{resource:resourcetype}\\\"\",\"transformers\":null}",
"isHiddenWhenLocked": true,
"typeSettings": {
"additionalResourceOptions": [
"value::1"
],
"showDefault": false
},
"timeContext": {
"durationMs": 21600000
},
"timeContextFromParameter": "timeRange",
"defaultValue": "value::1",
"queryType": 8
},
{
"id": "1b826776-ab99-45ab-86db-cced05e8b36d",
"version": "KqlParameterItem/1.0",
"name": "clusterId",
"type": 1,
"description": "Used to identify the cluster name when the cluster type is AKS Engine",
"isHiddenWhenLocked": true,
"timeContext": {
"durationMs": 0
},
"timeContextFromParameter": "timeRange"
},
{
"id": "67227c35-eab8-4518-9212-1c3c3d564b20",
"version": "KqlParameterItem/1.0",
"name": "masterNodeExists",
"type": 1,
"query": "let MissingTable = view () { print isMissing=1 };\r\nlet masterNodeExists = toscalar(\r\nunion isfuzzy=true MissingTable, (\r\nAzureDiagnostics \r\n| getschema \r\n| summarize c=count() \r\n| project isMissing=iff(c > 0, 0, 1)\r\n) \r\n| top 1 by isMissing asc\r\n);\r\nprint(iif(masterNodeExists == 0, 'yes', 'no'))\r\n",
"crossComponentResources": [
"{resource}"
],
"isHiddenWhenLocked": true,
"timeContext": {
"durationMs": 0
},
"timeContextFromParameter": "timeRange",
"queryType": 0,
"resourceType": "{resourceType}"
}
],
"style": "pills",
"queryType": 0,
"resourceType": "{resourceType}"
},
"name": "pills"
},
{
"type": 1,
"content": {
"json": "Please note that the Container Insights Usage workbook for AKS Engine clusters shows your billable data usage for your cluster's entire workspace ({resource:name}), and not just the cluster itself ({clusterId}). ",
"style": "info"
},
"conditionalVisibility": {
"parameterName": "resourceType",
"comparison": "isEqualTo",
"value": "microsoft.operationalinsights/workspaces"
},
"name": "aks-engine-billable-data-shown-applies-to-entire-workspace-not-just-the-cluster-info-message",
"styleSettings": {
"showBorder": true
}
},
{
"type": 11,
"content": {
"version": "LinkItem/1.0",
"style": "tabs",
"links": [
{
"id": "3b7d39f2-38c5-4586-a155-71e28e333020",
"cellValue": "selectedTab",
"linkTarget": "parameter",
"linkLabel": "Overview",
"subTarget": "overview",
"style": "link"
},
{
"id": "7163b764-7ab2-48b3-b417-41af3eea7ef0",
"cellValue": "selectedTab",
"linkTarget": "parameter",
"linkLabel": "By Table",
"subTarget": "table",
"style": "link"
},
{
"id": "960bbeea-357f-4c07-bdd9-6f3f123d56fd",
"cellValue": "selectedTab",
"linkTarget": "parameter",
"linkLabel": "By Namespace",
"subTarget": "namespace",
"style": "link"
},
{
"id": "7fbcc5bd-62d6-4aeb-b55e-7b1fab65716a",
"cellValue": "selectedTab",
"linkTarget": "parameter",
"linkLabel": "By Log Source",
"subTarget": "logSource",
"style": "link"
},
{
"id": "3434a63b-f3d5-4c11-9620-1e731983041c",
"cellValue": "selectedTab",
"linkTarget": "parameter",
"linkLabel": "By Diagnostic Master Node ",
"subTarget": "diagnosticMasterNode",
"style": "link"
}
]
},
"name": "tabs"
},
{
"type": 1,
"content": {
"json": "Due to querying limitation on Basic Logs, if you are using Basic Logs, this workbook provides partial data on your container log usage.",
"style": "info"
},
"conditionalVisibility": {
"parameterName": "selectedTab",
"comparison": "isEqualTo",
"value": "overview"
},
"name": "basic-logs-info-text"
},
{
"type": 12,
"content": {
"version": "NotebookGroup/1.0",
"groupType": "editable",
"title": "Container Insights Billing Usage",
"items": [
{
"type": 1,
"content": {
"json": "<br/>\r\nThis workbook provides you a summary and source for billable data collected by [Container Insights solution.](https://docs.microsoft.com/en-us/azure/azure-monitor/insights/container-insights-overview)\r\n\r\nThe best way to understand how Container Insights ingests data in Log Analytics workspace is from this [article.](https://medium.com/microsoftazure/azure-monitor-for-containers-optimizing-data-collection-settings-for-cost-ce6f848aca32)\r\n\r\nIn this workbook you can:\r\n\r\n* View billable data ingested by **solution**.\r\n* View billable data ingested by **Container logs (application logs)**\r\n* View billable container logs data ingested segregated by **Kubernetes namespace**\r\n* View billable container logs data ingested segregated by **Cluster name**\r\n* View billable container log data ingested by **logsource entry**\r\n* View billable diagnostic data ingested by **diagnostic master node logs**\r\n\r\nYou can fine tune and control logging by turning off logging on the above mentioned vectors. [Learn how to fine-tune logging](https://docs.microsoft.com/en-us/azure/azure-monitor/insights/container-insights-agent-config)\r\n\r\nYou can control your master node's logs by updating diagnostic settings. [Learn how to update diagnostic settings](https://docs.microsoft.com/en-us/azure/aks/view-master-logs)"
},
"name": "workbooks-explanation-text"
},
{
"type": 1,
"content": {
"json": "`Master node logs` are not enabled. [Learn how to enable](https://docs.microsoft.com/en-us/azure/aks/view-master-logs)"
},
"conditionalVisibility": {
"parameterName": "masterNodeExists",
"comparison": "isEqualTo",
"value": "no"
},
"name": "master-node-logs-are-not-enabled-msg"
}
]
},
"conditionalVisibility": {
"parameterName": "selectedTab",
"comparison": "isEqualTo",
"value": "overview"
},
"name": "workbook-explanation"
},
{
"type": 12,
"content": {
"version": "NotebookGroup/1.0",
"groupType": "editable",
"items": [
{
"type": 3,
"content": {
"version": "KqlItem/1.0",
"query": "union withsource = SourceTable AzureDiagnostics, AzureActivity, AzureMetrics, ContainerLog, Perf, KubePodInventory, ContainerInventory, InsightsMetrics, KubeEvents, KubeServices, KubeNodeInventory, ContainerNodeInventory, KubeMonAgentEvents, ContainerServiceLog, Heartbeat, KubeHealth, ContainerImageInventory\r\n| where _IsBillable == true\r\n| project _BilledSize, TimeGenerated, SourceTable\r\n| summarize BillableDataBytes = sum(_BilledSize) by bin(TimeGenerated, {timeRange:grain}), SourceTable\r\n| render piechart\r\n\r\n\r\n",
"size": 3,
"showAnnotations": true,
"showAnalytics": true,
"title": "Billable Data from Container Insights",
"timeContextFromParameter": "timeRange",
"queryType": 0,
"resourceType": "{resourceType}",
"crossComponentResources": [
"{resource}"
],
"chartSettings": {
"xAxis": "TimeGenerated",
"createOtherGroup": 100,
"seriesLabelSettings": [
{
"seriesName": "LogManagement",
"label": "LogManagementSolution(GB)"
},
{
"seriesName": "ContainerInsights",
"label": "ContainerInsightsSolution(GB)"
}
],
"ySettings": {
"numberFormatSettings": {
"unit": 2,
"options": {
"style": "decimal"
}
}
}
}
},
"customWidth": "50",
"showPin": true,
"name": "billable-data-from-ci"
}
]
},
"conditionalVisibility": {
"parameterName": "selectedTab",
"comparison": "isEqualTo",
"value": "table"
},
"name": "by-datatable-tab"
},
{
"type": 12,
"content": {
"version": "NotebookGroup/1.0",
"groupType": "editable",
"items": [
{
"type": 3,
"content": {
"version": "KqlItem/1.0",
"query": "KubePodInventory\r\n| distinct ContainerID, Namespace\r\n| join kind=innerunique (\r\nContainerLog\r\n| where _IsBillable == true\r\n| summarize BillableDataBytes = sum(_BilledSize) by ContainerID\r\n) on ContainerID\r\n| union (\r\nKubePodInventory\r\n| distinct ContainerID, Namespace\r\n)\r\n| summarize Total=sum(BillableDataBytes) by Namespace\r\n| render piechart\r\n",
"size": 3,
"showAnalytics": true,
"title": "Billable Container Log Data Per Namespace",
"timeContextFromParameter": "timeRange",
"queryType": 0,
"resourceType": "{resourceType}",
"crossComponentResources": [
"{resource}"
],
"tileSettings": {
"showBorder": false,
"titleContent": {
"columnMatch": "Namespace",
"formatter": 1
},
"leftContent": {
"columnMatch": "Total",
"formatter": 12,
"formatOptions": {
"palette": "auto"
},
"numberFormat": {
"unit": 17,
"options": {
"maximumSignificantDigits": 3,
"maximumFractionDigits": 2
}
}
}
},
"graphSettings": {
"type": 0,
"topContent": {
"columnMatch": "Namespace",
"formatter": 1
},
"centerContent": {
"columnMatch": "Total",
"formatter": 1,
"numberFormat": {
"unit": 17,
"options": {
"maximumSignificantDigits": 3,
"maximumFractionDigits": 2
}
}
}
},
"chartSettings": {
"createOtherGroup": 100,
"ySettings": {
"numberFormatSettings": {
"unit": 2,
"options": {
"style": "decimal"
}
}
}
}
},
"name": "billable-data-per-namespace"
}
]
},
"conditionalVisibility": {
"parameterName": "selectedTab",
"comparison": "isEqualTo",
"value": "namespace"
},
"name": "by-namespace-tab"
},
{
"type": 12,
"content": {
"version": "NotebookGroup/1.0",
"groupType": "editable",
"items": [
{
"type": 9,
"content": {
"version": "KqlParameterItem/1.0",
"crossComponentResources": [
"{resource}"
],
"parameters": [
{
"id": "d5701caa-0486-4e6f-adad-6fa5b8496a7d",
"version": "KqlParameterItem/1.0",
"name": "namespace",
"label": "Namespace",
"type": 2,
"description": "Filter data by namespace",
"isRequired": true,
"query": "KubePodInventory\r\n| distinct ContainerID, Namespace\r\n| join kind=innerunique (\r\nContainerLog\r\n| project ContainerID\r\n) on ContainerID\r\n| union (\r\nKubePodInventory\r\n| distinct ContainerID, Namespace\r\n)\r\n| distinct Namespace\r\n| project value = Namespace, label = Namespace, selected = false\r\n| sort by label asc",
"crossComponentResources": [
"{resource}"
],
"typeSettings": {
"additionalResourceOptions": [
"value::1"
],
"showDefault": false
},
"timeContext": {
"durationMs": 0
},
"timeContextFromParameter": "timeRange",
"defaultValue": "value::1",
"queryType": 0,
"resourceType": "{resourceType}"
}
],
"style": "pills",
"queryType": 0,
"resourceType": "{resourceType}"
},
"name": "namespace-pill"
},
{
"type": 3,
"content": {
"version": "KqlItem/1.0",
"query": "KubePodInventory\r\n| where Namespace == '{namespace}'\r\n| distinct ContainerID, Namespace\r\n| join hint.strategy=shuffle (\r\nContainerLog\r\n| where _IsBillable == true\r\n| summarize BillableDataBytes = sum(_BilledSize) by LogEntrySource, ContainerID\r\n) on ContainerID\r\n| union (\r\nKubePodInventory\r\n| where Namespace == '{namespace}'\r\n| distinct ContainerID, Namespace\r\n)\r\n| extend sourceNamespace = strcat(LogEntrySource, \"/\", Namespace)\r\n| summarize Total=sum(BillableDataBytes) by sourceNamespace\r\n| render piechart",
"size": 0,
"showAnnotations": true,
"showAnalytics": true,
"title": "Billable Data Per Log-Source Type for Namespace Selected",
"timeContextFromParameter": "timeRange",
"queryType": 0,
"resourceType": "{resourceType}",
"crossComponentResources": [
"{resource}"
],
"chartSettings": {
"createOtherGroup": 100,
"ySettings": {
"numberFormatSettings": {
"unit": 2,
"options": {
"style": "decimal"
}
}
}
}
},
"customWidth": "35",
"showPin": true,
"name": "billable-data-per-log-source",
"styleSettings": {
"showBorder": true
}
},
{
"type": 3,
"content": {
"version": "KqlItem/1.0",
"query": "KubePodInventory\r\n| where Namespace == '{namespace}'\r\n| distinct ContainerID, Namespace\r\n| project ContainerID\r\n| join hint.strategy=shuffle ( \r\nContainerLog\r\n| project _BilledSize, ContainerID, TimeGenerated, LogEntrySource\r\n) on ContainerID\r\n| union (\r\nKubePodInventory\r\n| where Namespace == '{namespace}'\r\n| distinct ContainerID, Namespace\r\n| project ContainerID\r\n)\r\n| summarize ContainerLogData = sum(_BilledSize) by bin(TimeGenerated, {timeRange:grain}), LogEntrySource\r\n| render linechart\r\n\r\n",
"size": 0,
"showAnnotations": true,
"showAnalytics": true,
"title": "Billable Container Log Data for Namespace Selected",
"timeContextFromParameter": "timeRange",
"queryType": 0,
"resourceType": "{resourceType}",
"crossComponentResources": [
"{resource}"
],
"chartSettings": {
"xSettings": {},
"ySettings": {
"numberFormatSettings": {
"unit": 2,
"options": {
"style": "decimal"
}
}
}
}
},
"customWidth": "65",
"conditionalVisibility": {
"parameterName": "namespace:label",
"comparison": "isNotEqualTo",
"value": "<unset>"
},
"showPin": true,
"name": "billable-container-log-data-for-namespace-selected",
"styleSettings": {
"showBorder": true
}
}
]
},
"conditionalVisibility": {
"parameterName": "selectedTab",
"comparison": "isEqualTo",
"value": "logSource"
},
"name": "by-log-source-tab"
},
{
"type": 12,
"content": {
"version": "NotebookGroup/1.0",
"groupType": "editable",
"items": [
{
"type": 1,
"content": {
"json": "`Master node logs` are not enabled. [Learn how to enable](https://docs.microsoft.com/en-us/azure/aks/view-master-logs)"
},
"conditionalVisibility": {
"parameterName": "masterNodeExists",
"comparison": "isEqualTo",
"value": "no"
},
"name": "master-node-logs-are-not-enabled-msg-under-masternode-tab"
},
{
"type": 3,
"content": {
"version": "KqlItem/1.0",
"query": "AzureDiagnostics\r\n| summarize Total=sum(_BilledSize) by bin(TimeGenerated, {timeRange:grain}), Category\r\n| render barchart\r\n\r\n\r\n",
"size": 0,
"showAnnotations": true,
"showAnalytics": true,
"title": "Billable Diagnostic Master Node Log Data",
"timeContextFromParameter": "timeRange",
"queryType": 0,
"resourceType": "{resourceType}",
"crossComponentResources": [
"{resource}"
],
"chartSettings": {
"ySettings": {
"unit": 2,
"min": null,
"max": null
}
}
},
"customWidth": "75",
"conditionalVisibility": {
"parameterName": "masterNodeExists",
"comparison": "isEqualTo",
"value": "yes"
},
"showPin": true,
"name": "billable-data-from-diagnostic-master-node-logs"
}
]
},
"conditionalVisibility": {
"parameterName": "selectedTab",
"comparison": "isEqualTo",
"value": "diagnosticMasterNode"
},
"name": "by-diagnostic-master-node-tab"
}
],
"fallbackResourceIds": [
"/subscriptions/357e7522-d1ad-433c-9545-7d8b8d72d61a/resourceGroups/ArcResources/providers/Microsoft.Kubernetes/connectedClusters/sv5-k8sdemo"
],
"fromTemplateId": "community-Workbooks/AKS/Billing Usage Divided",
"$schema": "https://github.com/Microsoft/Application-Insights-Workbooks/blob/master/schema/workbook.json"
}Topic Search
-
Securing TLS in WAC (Windows Admin Center) https://t.co/klDc7J7R4G
Posts by Date
- August 2025 1
- March 2025 1
- February 2025 1
- October 2024 1
- August 2024 1
- July 2024 1
- October 2023 1
- September 2023 1
- August 2023 3
- July 2023 1
- June 2023 2
- May 2023 1
- February 2023 3
- January 2023 1
- December 2022 1
- November 2022 3
- October 2022 7
- September 2022 2
- August 2022 4
- July 2022 1
- February 2022 2
- January 2022 1
- October 2021 1
- June 2021 2
- February 2021 1
- December 2020 2
- November 2020 2
- October 2020 1
- September 2020 1
- August 2020 1
- June 2020 1
- May 2020 2
- March 2020 1
- January 2020 2
- December 2019 2
- November 2019 1
- October 2019 7
- June 2019 2
- March 2019 2
- February 2019 1
- December 2018 3
- November 2018 1
- October 2018 4
- September 2018 6
- August 2018 1
- June 2018 1
- April 2018 2
- March 2018 1
- February 2018 3
- January 2018 2
- August 2017 5
- June 2017 2
- May 2017 3
- March 2017 4
- February 2017 4
- December 2016 1
- November 2016 3
- October 2016 3
- September 2016 5
- August 2016 11
- July 2016 13