Helm installation

This guide outlines the steps to deploy the DataQI Helm chart into your Kubernetes cluster.

OpenShift users: Review the OpenShift installation guide before proceeding, as additional Security Context Constraints (SCC) and configuration steps are required.

1. Prepare the namespace

Ensure the target namespace exists in your cluster before deploying.

kubectl create namespace <namespace>

2. Provision secrets

As detailed in the Secrets management section, you must provision all required secrets into the namespace before proceeding.

Ensure that your dataqi-image-registry secret is correctly configured to allow Kubernetes to pull the container images.

3. Authenticate Helm

Log in to the DataQI Helm registry using the credentials provided by your account representative.

helm registry login acrdataqi-ayguf2ahfghaenfn.azurecr.io -u <acr-username> -p <acr-password>

4. Pull the chart

Pull the desired version of the DataQI Helm chart(s) to your local machine.

helm pull oci://acrdataqi-ayguf2ahfghaenfn.azurecr.io/helm/dataqi --version <chart-version>

Note: If you’d like to inspect the raw templates and default values.yaml file, you can extract the downloaded archive using tar -zxvf dataqi-<chart-version>.tgz. Alternatively, to see the rendered Kubernetes manifests, you can run helm template dataqi ./dataqi-<chart-version>.tgz.

5. Configure deployment

Create a custom values.yaml file to define your environment-specific configuration.

At a minimum, specify the global registry, telemetry identifiers, and base domain for routing. The packaged chart selects the tested DataQI application image version automatically:

global:
  imageRegistry: "<registry-fqdn>"
  customerId: "example-company"
  environment: "production"
  domain: "dataqi.yourdomain.com"

Key configuration parameters

• global.imageRegistry: The fully qualified domain name (FQDN) of the container registry from which Kubernetes will pull the DataQI images.
• global.customerId: A unique identifier for your organisation. This is used strictly for tagging telemetry to help us when providing support.
• global.environment: The name of the environment you are deploying into (e.g., production, staging). This is also used for telemetry tagging.
• global.domain: The base external domain where the DataQI application will be hosted. This is a critical property; it is used dynamically across the chart to configure all HTTP routes.

The chart embeds the application image version tested with that chart release. Changing the chart version therefore upgrades both the Kubernetes resources and the release-coupled DataQI images.

Example: If you set global.domain: "dataqi.my-company.com", the following endpoints will be automatically configured:

• spa-dataqi.my-company.com (Main application UI)
• api-dataqi.my-company.com (Main application API)
• auth-dataqi.my-company.com (Authentication server)
• mcpsb-dataqi.my-company.com (MCP sandbox — if enabled for support activities)

Configuring Parquet query scratch storage

Every API pod includes a parquet-query-worker sidecar. The API streams authorised spreadsheet-derived Parquet files into a pod-local, disk-backed emptyDir; the worker queries those files and writes a bounded temporary result there. The worker has no Service, ingress, persistent-file-store mount, database configuration, or separate feature flag.

The default workspace ceiling and emptyDir size limit are 6Gi. Ensure the node has enough ephemeral-storage capacity and keep the API and worker ephemeral-storage limits above the workspace ceiling plus their normal writable-layer and log usage. You can tune the ceiling and worker resources together:

apps:
  api:
    parquetQuery:
      maxWorkspaceBytes: 6442450944 # 6 GiB
      worker:
        resources:
          requests:
            cpu: 50m
            memory: 128Mi
            ephemeral-storage: 1Gi
          limits:
            cpu: 1000m
            memory: 1Gi
            ephemeral-storage: 7Gi

The kubelet filesystem backing emptyDir must be encrypted at rest. The chart cannot inspect host encryption, so it does not provide an acknowledgement switch. In a mixed cluster, label verified nodes and add that label to the API selector:

apps:
  api:
    nodeSelector:
      dataqi.ai/ephemeral-storage-encrypted: "true"

This scratch volume does not use a StorageClass and is deleted with the pod. It is not durable application storage and must not be backed up.

Configuring the Kubernetes workload issuer

DataQI uses projected Kubernetes service-account tokens for authenticated service-to-service calls. The chart defaults infra.keycloak.realm.k8sIssuerUrl to https://kubernetes.default.svc.cluster.local, which is the issuer used by typical on-premises clusters.

If your cluster advertises a different service-account issuer, override this value with the exact issuer URL configured by the cluster. Managed Kubernetes services commonly use a public issuer URL; for example:

infra:
  keycloak:
    realm:
      k8sIssuerUrl: "https://example.oic.prod-aks.azure.com/<tenant>/<cluster>/"

The value must match the iss claim in the projected service-account tokens. A mismatch prevents DataQI workloads from obtaining machine-to-machine access tokens and causes the API readiness check to report the Keycloak registration dependency as unhealthy.

Redirecting browser login directly to Microsoft Entra ID

When the dataqi-keycloak-microsoft Secret is configured, Keycloak redirects users directly to Microsoft Entra ID by default. A production environment that uses Microsoft SSO exclusively requires only the identity provider Secret reference:

secrets:
  keycloakMicrosoftIdp: "dataqi-keycloak-microsoft"

This adds the Identity Provider Redirector to the browser authentication flow, selects the chart-managed aad provider, and disables the Keycloak username/password form. When the Secret reference is empty, the redirector is omitted and the local login form remains available.

Local and QA login

Browser-based E2E tests use Keycloak-local accounts and require the username/password form. QA therefore explicitly configures:

infra:
  keycloak:
    realm:
      browserFlow:
        userPasswordFormEnabled: true

Disabling Kubernetes metrics ingestion

By default, the chart deploys a Gateway collector and a DaemonSet agent to ingest Kubernetes cluster-level and node-level metrics (kubeletstats). If you do not want to collect these metrics or deploy the DaemonSet agent, you can disable them in your values.yaml file:

infra:
  otelcollector:
    ingestK8sMetrics: false

This disables the otelcollector-agent DaemonSet and its cluster-level RBAC role/role binding definitions. Note that if you disable both Kubernetes metrics ingestion (ingestK8sMetrics: false) and Postgres database metrics ingestion (ingestCnpgMetrics: false), the otelcollector ServiceAccount is not created, which automatically disables Kubernetes metadata enrichment (via the k8sattributes processor) on all pipeline spans and logs to prevent authentication/authorization failures on startup.

Authenticated browser telemetry

The chart enables telemetry-ingress and authenticated SPA log and application metric export by default. The ingress shares the API hostname at /telemetry, accepts only access tokens issued to the configured SPA client, and forwards accepted OTLP/HTTP traffic to the collector’s private browser receiver. The collector removes unapproved browser attributes before export. Browser tracing remains active only to propagate trace context to API requests.

To stop browser export immediately while retaining the ingress for already-loaded SPA versions, set:

apps:
  spa:
    telemetry:
      enabled: false

Queue size, batch delay, trace-context sampling, and minimum log level are configured under apps.spa.telemetry. The defaults retain all trace contexts and export logs at Warning or above. Set apps.telemetryIngress.enabled: false only when the public route and adapter should also be removed. Additional approved browser client IDs can be added under apps.telemetryIngress.allowedClientIds; the primary apps.spa.clientId is always allowed automatically. Do not expose collector port 4320 through a Gateway, Ingress, or LoadBalancer; it is reserved for traffic from telemetry-ingress.

When infra.keycloak.internalUrl points to a central Keycloak and egress network policies are enabled, configure the Keycloak destinations that telemetry-ingress may reach. Public destinations are allowed only on the configured ports. Private or cross-namespace destinations must also provide their resolved CIDR ranges:

infra:
  keycloak:
    enabled: false
    internalUrl: "https://keycloak.keycloak.svc.cluster.local:8443"

apps:
  telemetryIngress:
    networkPolicy:
      keycloak:
        allowPublic: false
        cidrs:
          - "10.42.16.0/24"
        ports:
          - 8443

For a public Keycloak endpoint, retain allowPublic: true and restrict ports to the endpoint’s HTTPS port. Standard Kubernetes NetworkPolicy cannot select destinations by hostname, so private endpoint CIDRs must be kept aligned with the configured authority.

Sensitive audit data

Sensitive audit payload publication is disabled by default across the API, Extractor, and Transformer. To enable it temporarily for a controlled investigation, set:

auditing:
  enableSensitiveDataLogging: true

Danger

Sensitive audit payloads may contain personal data, user-authored content, queries, raw results, or error details. Enable this setting only in an appropriately controlled environment, limit access to the resulting audit data, and disable it when the investigation is complete.

Configuring Keycloak session timeouts

By default, DataQI configures Keycloak with an 8-hour idle session timeout and a 30-day maximum session lifespan. These defaults suit most production deployments, but can be overridden per environment in your values.yaml:

infra:
  keycloak:
    realm:
      ssoSessionIdleTimeout: 28800    # seconds — how long an inactive session lasts (default: 8 h)
      ssoSessionMaxLifespan: 2592000  # seconds — absolute maximum session length (default: 30 days)
      accessTokenLifespan: 900        # seconds — individual access token lifespan (default: 15 min)

All three values are in seconds. The settings are applied automatically on every helm upgrade — no manual changes in the Keycloak Admin Console are required.

6. Configure TLS

DataQI supports multiple TLS modes to accommodate different certificate management approaches. Set routing.tls.mode in your values file to one of:

• certManager — let cert-manager automatically provision and renew the certificate.
• preProvisioned — supply a TLS secret you have created manually (wildcard cert, Windows CA export, Cloudflare origin cert, etc.).
• external — TLS is terminated upstream (Cloudflare proxy, WAF, load balancer). The chart renders nothing.
• none — plain HTTP; for development use only.

For full instructions and ClusterIssuer examples for each mode, see the TLS certificate configuration guide.

7. Install the chart

Execute the Helm upgrade command to install DataQI.

helm upgrade --install dataqi ./dataqi-<chart-version>.tgz \
  -f <values-file>.yaml \
  -n <namespace> \
  --wait --wait-for-jobs --timeout 10m

The chart configures the Keycloak realm with a Kubernetes Job. --wait-for-jobs ensures Helm reports success only after that configuration has completed. Monitor the deployment using kubectl get pods -n <namespace> to ensure all services start successfully.

PostgreSQL backups

By default, the managed PostgreSQL clusters do not write backups anywhere. To enable WAL archiving and scheduled base backups, configure the databases.backup block in your values file. The chart accepts any CNPG-supported storage backend, including Azure Blob Storage, AWS S3, and on-premises MinIO deployments.

See the PostgreSQL backups guide for full configuration examples.