Demo

A walkthrough of the current Proof of Concept running on a Gardener-managed cluster in the SAP Cloud Infrastructure.

Stack Deployment

The full Thalamus stack is deployed on a Gardener-managed Kubernetes cluster via a helmfile that installs the platform infrastructure (GPU operator, monitoring, gateway infrastructure) alongside the Thalamus inference workloads, endpoint pickers, and frontend. To deploy the Thalamus stack onto your own Kubernetes cluster, head over to the Getting Started guide.

Model CRD

Inference instances in Thalamus are declared as Kubernetes resources using the thalamus.cloud/v1alpha1 Model CRD. Each Model manifest captures the full lifecycle of an inference workload in a single, version-controlled object. This includes extensive configuration options, such as the inference engine, model weight location, GPU allocation, autoscaling bounds, and scheduler assignment.

Operator Under Development

The Thalamus operator, which reconciles Model resources into running inference workloads, is currently under active development. Until it reaches general availability, model instances are managed through Helm values as described in the Getting Started guide.

See the Model CRD API Reference for the full field specification.

Container Images in Keppel

In the PoC deployment, all container images are stored in and served from SAP's internal OCI registry called Keppel.

Accessing Thalamus

Thalamus exposes two access paths: a simple to access, browser-based chat frontend, and an OpenAI-compatible API endpoint for programmatic access.

Frontend — Open WebUI

Thalamus provides a chat interface which has the option to integrate with an identity provider, allowing for direct access without any additional tooling or credentials setup.

API Endpoint — OpenCode

The inference gateway exposes an OpenAI-compatible API, making it a drop-in replacement for any OpenAI SDK client. The recording below shows OpenCode configured to use the Thalamus PoC endpoint and sending a prompt to the gpt-oss-120b model.