Skip to content

User Guide for installing KX Insights on Azure Marketplace - Customer Managed Plan

This section details the steps to install the KX Insights Platform on Azure. It will walkthrough the following:

Prerequisites

Before installing the KX Insights Platform on Azure you need the following:

  • Pilot customers in Private Preview stage must share the ID of the subscription where they want to deploy the solution to. The subscription ID needs to be configured in the offer in order to make it available to the customer. Adding a new subscription to the offer can take a few hours, therefore, it is important to configure this in advance.

  • The customer must state which subscription offer they have (i.e. Pay-As-You-Go, Enterprise, ...).

  • The customer individual performing the deployment must have Owner permissions on an empty resource group in the aforementioned subscription, or otherwise Owner permissions on the entire subscription.

  • The following Resource Providers must be registered in the subscription. Consider the Microsoft Guidance to learn how to register resource providers:

    • Microsoft.Compute
    • Microsoft.ContainerService
    • Microsoft.Network
    • Microsoft.Diagnostics
    • Microsoft.OperationalInsights
    • Microsoft.Insights
    • Microsoft.Resources
    • Microsoft.ManagedIdentity
    • Microsoft.Authorization

  • The customer must share which Azure Region they will be interested in deploying the solution to.

  • Sufficient quotas for the planned deployment. The customer subscription must have a quota capacity equal or greater than the following numbers for each quota category in the selected Azure region from the previous pre-requisite. Check this link to learn how to raise resource quotas. The quota numbers below represent a minimum requirement for a successful deployment, however, they might be insufficient depending on the resources created within KX Insights after the deployment, therefore, it is recommended to have extra capacity in the resources below.

    • Compute
      • Total Regional vCPUs >= 18
      • Standard DDSv4 Family vCPUs >= 12
      • Standard DDv4 Family vCPUs >= 6
      • Virtual Machine Scale Sets >= 2
      • Virtual Machines >= 2
      • Standard Storage Managed Disks >= 6
      • StandardSSDStorageDisks >= 6
    • Networking
      • Static Public IP Addresses >= 2
      • Public IP Addresses >= 2
      • Public IP Addresses - Standard >= 2
      • Virtual Networks >= 1
      • Load Balancers >= 1
      • Standard Sku Load Balancers >= 1
      • Network Security Groups >= 1

  • An Azure Region with availability zones.

  • A VM choice that has temp storage available when selecting rook ceph.

Quotas

It is important to ensure that the chosen subscription and intended deployment region have sufficient available resources for a successful deployment.

It is possible to check for available resources prior to commencing the install workflow to ensure that a deployment does not fail, due to a lack of IP or CPU resources as an example.

A KX Insights Platform deployment requires 3 public IP addresses for each external stream (API Gateway, Service Gateway and Ingress Controller). Each stream that accepts data from a KX Insights SDK, requires an additional 3 public IPs.

Please check your quotas and current usage on the portal before the deployment as shown below:

PIP

You can request a quota increase for Azure by clicking on the Request Increase icon or by using the Azure CLI to list your public IPs in a given region as shown below:

az vm list-usage --location "West Europe" --query "[? currentValue != '0']" -o table

The KX Insights Platform requires a minimum of 3 nodes to deploy the system successfully. Using a particular node pool as an example; the deployment includes a 3 node system Node pool of Standard_D4ds_v4 VMs; this consumes 12 cores of your Standard DDSv4 Family vCPUs quota. By default the user Node pool is of the same size as the system Node pool so you would need at least 24 available vCPUs in the region of your choice.

Note

For more resource intensive deployments you might want to select a VM size that has 16 vCores and that would increase the total required vCPUs to 60.

Virtual machine cores have a regional total limits as well.

You can also use Azure CLI to check usage in a given region as follows:

$ az network public-ip list --query "[? location == 'westeurope'].{ipAddress: ipAddress, resourceGroup: resourceGroup, sku: sku.name, publicIpAllocationMethod: publicIpAllocationMethod, location: location}" -o table
IpAddress       ResourceGroup           Sku       PublicIpAllocationMethod    Location
--------------  ----------------------  --------  --------------------------  ----------
20.103.202.177  AWKX10-aks              Standard  Static                      westeurope
20.101.205.189  insights-on-azure1-aks  Standard  Static                      westeurope
20.101.207.38   insights-on-azure1-aks  Standard  Static                      westeurope
20.101.207.47   insights-on-azure1-aks  Standard  Static                      westeurope
20.101.204.20   insights-on-azure1-aks  Standard  Static                      westeurope
20.101.207.44   insights-on-azure1-aks  Standard  Static                      westeurope
20.101.207.45   insights-on-azure1-aks  Standard  Static                      westeurope
13.94.188.195   windowsvm_group         Basic     Static                      westeurope

To obtain a successful deployment, please select an available region with sufficient resources.

Creating and Deploying KX Insights on Azure

Getting Started

  1. Ensure your approved subscription is selected in the global subscription filter in the Azure Portal.

  2. Open the left-hand Portal menu and click Create a resource.

  3. Search for KX Insights in the Azure Marketplace.

  4. Select KX Insights Platform (preview).

  5. Select a plan:

    1. Customer Managed - Standard Support

    2. Customer Managed - Enterprise Support

  6. Click Create and you will be taken to the the 'Basics' section.

Note

KX Insights Core Offer is also available on the Azure Marketplace. This User Guide is specific to the KX Insights Platform (preview) Offer.

Basics

  1. Select your approved subscription from the dropdown.
  2. Create a new resource group. If you wish to deploy multiple instances of KX Insights then each deployment from the Azure Marketplace must have a unique resource group name.
  3. Select an Azure region that has availability zones.
  4. Click Next: General Configuration.

General configuration

  1. Add a valid email address.

  2. Create a Keycloak admin password and make note of it.

Note

This will be required later to access KX Insights post deployment.

  1. Tick Show Advanced Settings.

  2. For security purposes, you can choose to set authorized IP ranges, this is not mandatory at time of deployment. Any IP ranges defined at time of deployment can be updated and edited post deployment.

  3. Select Deploy Rook Ceph if desired. Otherwise, the Storage Class default options can be used.

  4. The default VM size is suitable for deployment purposes. Please refer to the Prerequisites section if changing the VM.

    Note

    You may select an alternative VM size if you have specific workload requirements.

  5. In the Storage Class selector, if you have chosen to Deploy Rook Ceph please select Rook Ceph Block for Read Write Once volumes and Rook CephFS for ReadWriteMany volumes.

  6. You can choose to Deploy Kafka. We recommend ticking this if you intend to build any pipelines for data to stream from or to Kafka.

  7. Click Next.

  8. At this stage you should have a Validation Passed message which brings you to 'Review + Create' final page.

  9. Click Create to deploy.

Note

Please see Potential Deployment Issues for resolutions to errors that may arise if a deployment were to fail.

Create KX Insights Platform roles

When the deployment is successful, locate the deployment outputs; Customer Managed Plan Click on Outputs in the left-hand navigation menu to obtain the keycloak URL

First, you need to add the administrator and standard user roles into the KX Insights Platform. Follow the steps below to log in to Keycloak, which Insights uses as its Identity and Access Management component.

  1. Login to the Keycloak Web UI by navigate to the keycloakUrl, or the insightsUiUrl appended with /auth/, in a web browser.

    Keycloak

  2. Click on Administration Console and log in with the Keycloak admin user giving the username and password you created in General Configuration step of the installation process.

    SignIn

  3. Create the standard user role:

    • Click on Roles in the left-hand menu, then click on the Add Role button.

      AddRole

    • Enter the Role Name: insights.user, making sure you add some description and click Save.

    • Turn Composite Roles ON.

    • Associate it with the following roles:

      insights.builder.*
insights.pipeline.*
insights.query.*

      InsightsUser

  4. Create the administrator role:

    • Click on Roles in the left-hand menu then click on the Add Role button.

    • Enter the Role Name: insights.admin, make sure you add some description and click Save.

    • Turn Composite Roles ON.

    • Associate it with the following roles:

      insights.client.*
insights.user

      InsightsAdmin

Log in to KX Insights

Once the deployment is complete and you have used Keycloak to create a standard user role and an admin role, follow the steps below to log in to the KX Insights Platform:

  1. Locate the deployment outputs.

    1. Customer Managed Plan Click on Outputs in the left-hand navigation menu.
    2. Copy the insightsUiUrl value and navigate to this in a new browser tab.

  2. Log in with the below username and password:

    1. name: demoinsights
    2. auth: (the keycloak admin password you entered in the General configuration step.)

  3. You'll be asked to update your password and log in again with the new credentials.

    KXUi

  4. You will be directed to the KX Insights Platform User Interface.

    KXUi

  5. See KX Insights Overview for an introduction to the User Interface.

Azure Active Directory integration

Follow the instructions provided here to integrate Insights with Azure Active Directory and allow your users to log in to KX Insights with their existing credentials.

Potential deployment issues

Note

If a deployment fails for any reason, once the root cause analysis is done, if you need to redeploy KX Insights, please delete the whole resource group to free up any resources that may be left in the deployment, before starting a new deployment.

Missing Subscription Registration error

If your deployment fails with the below error message this is an indication that you need to register the Microsoft.OperationsManagement resource provider in your Azure Subscription:

{
    "status": "Failed",
    "error": {
        "code": "AddContainerInsightsSolutionError",
        "message": "Code=\"MissingSubscriptionRegistration\" Message=\"The subscription is not registered to use namespace 'Microsoft.OperationsManagement'. See https://aka.ms/rps-not-found for how to register subscriptions.\" Details=[{\"code\":\"MissingSubscriptionRegistration\",\"message\":\"The subscription is not registered to use namespace 'Microsoft.OperationsManagement'. See https://aka.ms/rps-not-found for how to register subscriptions.\",\"target\":\"Microsoft.OperationsManagement\"}]"
    }
}

To resolve this issue:

  1. Go to the subscription.

  2. Select Resource Providers on the left menu .

  3. Search for the Microsoft.OperationsManagement provider.

    SubscriptionResProv

  4. Register it.

    Registered

Quota issues

Please refer to the Prerequisites section. The deployment can fail if the subscription does not have sufficient quota for the selected configuration. Error messages can highlight a need for more IP addresses, CPU etc which you can increase with Azure in order to deploy successfully.