KubeKanvas Logo
  • Features
  • Pricing
  • Templates
    • How KubeKanvas works
    • Docs
    • Downloads
    • Blog
    • E-Book
    • Tutorials
  • FAQs
  • Contact
  • Features
  • Pricing
  • Templates
    • How KubeKanvas works
    • Docs
    • Downloads
    • Blog
    • E-Book
    • Tutorials
  • FAQs
  • Contact
Back to Templates

Deploy Gemma 4 + Ollama on Kubernetes Via Kubekanvas: Local AI Stack

Tags:
GemmaKubernetesKubekanvasLLMK8s
Created by:
Siddiqui
Deploy Gemma 4 + Ollama on Kubernetes Via Kubekanvas: Local AI Stack template preview
0 uses
KubeKanvas Logo
Visual Kubernetes cluster design tool that helps you create, manage, and deploy your applications with ease.
Product
  • Features
  • Pricing
  • Templates
Resources
  • Blog
  • Tutorials
Company
  • About Us
  • Contact
  • Terms of Service
  • Privacy Policy
  • Responsible AI Policy
  • Impressum
XGitHubLinkedIn
© 2026 KubeKanvas. All rights reserved.

Running a local LLM requires a model server, persistent weight storage, and a browser interface behind an ingress. This template deploys Google Gemma 4 via Ollama with Open WebUI on Kubernetes, producing an offline-capable AI stack accessible at a local domain.

What's Included

ComponentTypePortRole
NamespaceNamespace-Isolates all resources under gemma4
ollama-pvPersistentVolume-30 GB hostPath volume for model weights
ollama-pvcPersistentVolumeClaim-Binds Ollama to model storage
webui-pvPersistentVolume-5 GB hostPath volume for WebUI data
webui-pvcPersistentVolumeClaim-Binds WebUI to its data storage
ollama-configConfigMap-Ollama runtime env vars
webui-configConfigMap-Open WebUI env vars
ollamaDeployment11434Ollama inference server running Gemma 4
ollama-serviceService (NodePort)11434 / 30111Exposes Ollama internally and via NodePort
open-webuiDeployment8080Browser chat interface
open-webui-serviceService (NodePort)8080 / 30080Exposes WebUI internally and via NodePort
nginxIngressClass-Declares NGINX as default ingress class
gemma4-ingressIngress80Routes gemma.local to WebUI, ollama.local to API
open-webui-hpaHorizontalPodAutoscaler-Scales WebUI on CPU or memory pressure
gemma4-quotaResourceQuota-Caps CPU, memory, and pod count in namespace
gemma4-limitsLimitRange-Sets per-container resource defaults and ceilings
ollama-network-policyNetworkPolicy11434Restricts Ollama to WebUI pods and ingress controller

Architecture Overview

Ollama runs as a single replica with a 30 GB PersistentVolume for model weights. An init container pulls Gemma 4 on first boot using the Ollama CLI directly. Open WebUI reaches Ollama via internal ClusterIP DNS. NGINX Ingress routes gemma.local to the WebUI and ollama.local to the raw API, with buffering off for token streaming. A NetworkPolicy restricts Ollama to WebUI pods and the ingress controller only.

Prerequisites

  • Docker Desktop with Kubernetes enabled and WSL2 configured with at least 12 GB memory via .wslconfig
  • NGINX Ingress Controller deployed in the ingress-nginx namespace
  • 127.0.0.1 gemma.local ollama.local added to C:\Windows\System32\drivers\etc\hosts
  • KubeKanvas CLI installed and running on your computer (Optional, if you want to use one-click deployment)

How to Deploy

  1. Confirm the NGINX Ingress Controller pod is running in ingress-nginx before applying.
  2. Verify .wslconfig sets memory=12GB and restart Docker Desktop if you changed it.
  3. Update the hostPath.path values in both PersistentVolume specs to match your storage path. Kubernetes creates the directories automatically on first mount.
  4. Deploy the template to your cluster via the Play button in the top right bar. If you prefer to deploy manually, download the YAML and apply it with kubectl apply -f gemma4-k8s-final.yaml.
  5. Wait for all pods to reach Running status. You can monitor progress in the Release Monitor screen.

How to Test

  1. Run kubectl logs -n gemma4 -l app=ollama -c model-puller -f and confirm Pull complete and Init done appear before the main container starts.
  2. Open http://gemma.local and confirm the Open WebUI interface loads.
  3. Send a message and confirm Gemma 4 responds with streamed tokens.
  4. Hit http://ollama.local/api/tags and confirm the JSON response lists gemma4.

Use Cases

  • Offline LLM development: Test prompts against Gemma 4 with no internet or API token after the initial pull.
  • Private code assistance: Point Continue.dev or any OpenAI-compatible client at http://ollama.local for local inference with no data leaving the machine.
  • Kubernetes learning: Study a realistic manifest covering PVs, init containers, Ingress, HPA, NetworkPolicy, ResourceQuota, and LimitRange in one deployable file.
  • Air-gapped environments: Pre-pull weights to the hostPath volume and deploy on a machine with no outbound internet.
  • Multi-user setups: Enable Open WebUI signup so multiple developers share one inference server with isolated chat history.

Summary

This template configures a 17-resource Kubernetes stack that runs Gemma 4 via Ollama, exposes it through a streaming-optimized NGINX Ingress, and enforces namespace-level resource and network controls.