Skip to main content
Version: Next

Deploying AutoGluon Models with KServe

This guide explains how to deploy AutoGluon models with KServe using the autogluon model format and the kserve-autogluonserver runtime.

The runtime supports:

  • autogluon.tabular.TabularPredictor
  • autogluon.timeseries.TimeSeriesPredictor

Supported Predictor Types and Protocols​

Predictor TypeSupported Inference Protocol
TabularPredictorREST v1, REST v2
TimeSeriesPredictorREST v1 JSON only

Time series v2 tensor payloads are not supported in this release.

Auto-Detection and modelFormat.name​

AutoGluon predictor type is auto-detected from the model artifact in storageUri:

  • The runtime first tries TimeSeriesPredictor.load(...).
  • If that fails, it tries TabularPredictor.load(...).

Use modelFormat.name: autogluon for both tabular and time series models.

Prerequisites​

Before you begin, make sure you have:

  • A Kubernetes cluster with KServe installed.
  • Access to a storage backend reachable by your cluster (for example, GCS, S3, or Azure Blob).
  • A model saved with either TabularPredictor.save(path) or TimeSeriesPredictor.save(path).
Model Artifacts Must Be a Directory

AutoGluon models must be stored as a predictor directory generated by TabularPredictor.save(path) or TimeSeriesPredictor.save(path), not as a single file artifact.

Deploy the Model with REST Endpoint​

Create an InferenceService with explicit runtime selection.

Tabular Example​

apiVersion: "serving.kserve.io/v1beta1"
kind: "InferenceService"
metadata:
  name: "autogluon-titanic"
spec:
  predictor:
    model:
      modelFormat:
        name: autogluon
      protocolVersion: v2
      runtime: kserve-autogluonserver
      storageUri: "gs://your-bucket/autogluon-model/"
      resources:
        requests:
          cpu: "100m"
          memory: "1Gi"
        limits:
          cpu: "1"
          memory: "2Gi"

Time Series Example​

apiVersion: "serving.kserve.io/v1beta1"
kind: "InferenceService"
metadata:
  name: "autogluon-ts-forecast"
spec:
  predictor:
    model:
      modelFormat:
        name: autogluon
      runtime: kserve-autogluonserver
      storageUri: "gs://your-bucket/path/to/timeseries-predictor-save/"
      resources:
        requests:
          cpu: "100m"
          memory: "2Gi"
        limits:
          cpu: "2"
          memory: "4Gi"

Apply your manifest:

kubectl apply -f autogluon.yaml
Runtime Availability

The kserve-autogluonserver runtime may not be installed by default in every release bundle. Verify that the ClusterServingRuntime exists in your cluster before deploying the InferenceService.

Run Inference​

First, determine the ingress IP and ports, then set INGRESS_HOST and INGRESS_PORT.

Tabular REST v1 Example​

Use this for tabular models.

Sample payload:

{
  "instances": [
    {
      "PassengerId": 1,
      "Pclass": 3,
      "Sex": "male"
    },
    {
      "PassengerId": 2,
      "Pclass": 1,
      "Sex": "female"
    }
  ]
}

Before sending the request, determine the ingress IP and ports, then set the INGRESS_HOST and INGRESS_PORT environment variables.

SERVICE_HOSTNAME=$(kubectl get inferenceservice autogluon-titanic -o jsonpath='{.status.url}' | cut -d "/" -f 3)
curl -v \
  -H "Host: ${SERVICE_HOSTNAME}" \
  -H "Content-Type: application/json" \
  -d @./autogluon-input-v1.json \
  http://${INGRESS_HOST}:${INGRESS_PORT}/v1/models/autogluon-titanic:predict

Tabular REST v2 Example​

For v2 requests, provide one input tensor per feature. Each tensor name must match the feature name expected by the model, and all features must have a consistent batch length.

{
  "inputs": [
    { "name": "PassengerId", "shape": [2], "datatype": "INT64", "data": [1, 2] },
    { "name": "Pclass", "shape": [2], "datatype": "INT64", "data": [3, 1] },
    { "name": "Sex", "shape": [2], "datatype": "BYTES", "data": ["male", "female"] }
  ]
}

Before sending the request, determine the ingress IP and ports, then set the INGRESS_HOST and INGRESS_PORT environment variables.

SERVICE_HOSTNAME=$(kubectl get inferenceservice autogluon-titanic -o jsonpath='{.status.url}' | cut -d "/" -f 3)
curl -v \
  -H "Host: ${SERVICE_HOSTNAME}" \
  -H "Content-Type: application/json" \
  -d @./autogluon-input-v2.json \
  http://${INGRESS_HOST}:${INGRESS_PORT}/v2/models/autogluon-titanic/infer

Expected response:

{
  "model_name": "autogluon-titanic",
  "outputs": [
    { "name": "predictions", "datatype": "INT64", "shape": [2], "data": [1, 0] }
  ]
}

Time Series REST v1 Example​

Use this for TimeSeriesPredictor models. Time series requests use JSON payloads with top-level instances and optional known_covariates.

Sample payload:

{
  "instances": [
    { "item_id": "A", "timestamp": "2024-01-01T00:00:00", "target": 12.3 },
    { "item_id": "A", "timestamp": "2024-01-02T00:00:00", "target": 11.1 }
  ],
  "known_covariates": [
    { "item_id": "A", "timestamp": "2024-01-03T00:00:00", "promo": 1 },
    { "item_id": "A", "timestamp": "2024-01-04T00:00:00", "promo": 0 }
  ]
}

Before sending the request, determine the ingress IP and ports, then set the INGRESS_HOST and INGRESS_PORT environment variables.

SERVICE_HOSTNAME=$(kubectl get inferenceservice autogluon-ts-forecast -o jsonpath='{.status.url}' | cut -d "/" -f 3)
curl -v \
  -H "Host: ${SERVICE_HOSTNAME}" \
  -H "Content-Type: application/json" \
  -d @./autogluon-timeseries-input-v1.json \
  http://${INGRESS_HOST}:${INGRESS_PORT}/v1/models/autogluon-ts-forecast:predict

Expected response:

{
  "predictions": [
    {
      "item_id": "A",
      "timestamp": "2024-01-03T00:00:00",
      "mean": 10.87,
      "0.1": 9.95,
      "0.9": 11.66
    }
  ]
}

Prediction Probabilities​

The AutoGluon runtime supports returning probabilities via the PREDICT_PROBA=true environment setting in the runtime container configuration.

note

When probability output is enabled, output schema differs from class prediction output.

Environment Variables​

  • PREDICT_PROBA (tabular): set to true to return class probabilities via predict_proba() instead of predicted labels via predict().
  • AUTOGLUON_TS_ID_COLUMN (time series): overrides the item identifier column used in JSON payloads.
  • AUTOGLUON_TS_TIMESTAMP_COLUMN (time series): overrides the timestamp column used in JSON payloads.

For time series, the target column name always comes from TimeSeriesPredictor.target in the loaded model and is not configurable via environment variable.

Troubleshooting​

  • Ensure storageUri points to a predictor directory created by TabularPredictor.save(path) or TimeSeriesPredictor.save(path).
  • For tabular v2 requests, verify each feature is provided as a separate tensor with matching batch length.
  • For time series requests, ensure column names in instances and known_covariates match model expectations (including id, timestamp, and target).
  • For time series models, use REST v1 JSON (/v1/models/{name}:predict) instead of v2 tensor payloads.
  • If no runtime is selected automatically, set runtime: kserve-autogluonserver explicitly.

References​