Skip to content

Instantly share code, notes, and snippets.

View nerdalert's full-sized avatar
🐈
🦀 🐿

Brent Salisbury nerdalert

🐈
🦀 🐿
View GitHub Profile

Classifier on GPU deploy support stdout

$ ./deploy/openshift/deploy-to-openshift.sh --kserve --simulator --classifier-gpu
[SUCCESS] Logged in as cluster-admin
[INFO] Creating namespace: vllm-semantic-router-system
namespace/vllm-semantic-router-system configured
[SUCCESS] Namespace ready
[INFO] Installing KServe and LLMInferenceService CRDs...
[INFO] InferenceService CRD already installed.

vSR LlmInferenceServices Kserve GPU Demo

$ ./deploy/openshift/deploy-to-openshift.sh --kserve --no-observability
[SUCCESS] Logged in as cluster-admin
[INFO] Creating namespace: vllm-semantic-router-system
namespace/vllm-semantic-router-system configured
[SUCCESS] Namespace ready
[INFO] Installing KServe and LLMInferenceService CRDs...
[INFO] InferenceService CRD already installed.

vSR LlmInferenceServices Kserve Simulator Demo

$ ./deploy/openshift/deploy-to-openshift.sh --kserve --simulator --no-observability
[SUCCESS] Logged in as cluster-admin
[INFO] Creating namespace: vllm-semantic-router-system
namespace/vllm-semantic-router-system configured
[SUCCESS] Namespace ready
[INFO] Installing KServe and LLMInferenceService CRDs...

KIND vSR Deploy/Validation

# Create a Cluster #

$ kind create cluster --name semantic-router
Creating cluster "semantic-router" ...
 ✓ Ensuring node image (kindest/node:v1.35.0) 🖼
 ✓ Preparing nodes 📦
 ✓ Writing configuration 📜

Deploy and Validation stdout of vSR/Kserve/multi-model sim.

$ ./deploy/openshift/deploy-to-openshift.sh --kserve --simulator
[SUCCESS] Logged in as kube:admin
[INFO] Creating namespace: vllm-semantic-router-system
namespace/vllm-semantic-router-system configured
[SUCCESS] Namespace ready
[INFO] KServe CRD missing; installing KServe dependencies...
[INFO] cert-manager CRDs already present.

stdout testing for opendatahub-io/models-as-a-service#227

brent@ip-172-31-33-128:~/tls/opendatahub-operator$ make install deploy -e VERSION=tls -e IMG='quay.io/bmajsak/opendatahub-operator:tls'
go: downloading go1.25.0 (linux/amd64)
mkdir -p /home/brent/tls/opendatahub-operator/bin
Downloading sigs.k8s.io/kustomize/kustomize/v5@v5.7.0
Downloading sigs.k8s.io/controller-tools/cmd/controller-gen@v0.17.3
/home/brent/tls/opendatahub-operator/bin/controller-gen --load-build-tags=odh rbac:roleName=controller-manager-role crd:ignoreUnexportedFields=true webhook paths="./..." output:crd:artifacts:config=config/crd/bases output:rbac:artifacts:config=config/rbac output:webhook:artifacts:config=config/webhook
/home/brent/tls/opendatahub-operator

Deploy

  • Had to run the script twice to chmod a new install script breaking out the IDP install. No changes to logic, just calling a different backend script.
$ export ENABLE_KEYCLOAK_IDP=true
  ./scripts/deploy-rhoai-stable.sh
## Installing prerequisites
$ ./deploy/openshift/deploy-to-openshift.sh
[SUCCESS] Logged in as kube:admin
[INFO] Creating namespace: vllm-semantic-router-system
namespace/vllm-semantic-router-system configured
[SUCCESS] Namespace ready
[INFO] Checking for llm-katan image...
[INFO] Building llm-katan image...
--> Found container image ce19342 (6 days old) from Docker Hub for "python:3.10-slim"

Deployment of opendatahub-io/models-as-a-service#333 tested on ROSA, requires this PR as well for a simple wait for the ODH namespace readyness: opendatahub-io/models-as-a-service#329

$ ./scripts/deploy-rhoai-stable.sh
## Installing prerequisites

* Installing cert-manager operator...
namespace/cert-manager-operator created
operatorgroup.operators.coreos.com/cert-manager-operator created
$ ./deployment/scripts/deploy-openshift.sh
=========================================
🚀 MaaS Platform OpenShift Deployment
=========================================

📋 Checking prerequisites...

Required tools: