Dedicated Inference Endpoints

This page is for calling and managing dedicated endpoints after they exist (URLs, auth modes, statuses, scale-to-zero, terminate, custom domains). For how deploy works and what Hypervize provisions, see Provisioning.

Dedicated Inference is where you get full model flexibility. Run any Hugging Face model (or your own weights) on private, Hypervize-managed GPU capacity with full control over hardware tier, scaling behavior, and access policies.

How It Works

1. You provide a Hugging Face model ID (and optional HF token for gated models).
2. Hypervize analyzes the model size and provisions appropriate GPU hardware.
3. We provision dedicated capacity running our optimized inference runtime.
4. Auto-scaling is attached (min/max nodes based on traffic).
5. You receive a permanent endpoint ID and a stable URL.

After creation, the endpoint behaves like any other OpenAI-compatible inference endpoint.

Endpoint URL Patterns

Text LLMs (chat):

POST https://hypervize.tech/api/d/{endpoint-id}/chat/completions

Image generation (Diffusers / HF text-to-image):

POST https://hypervize.tech/api/d/{endpoint-id}/images/generations

The path is chosen automatically from the Hugging Face model you deploy (text vs image). The dashboard block page and playground follow the same path.

Pre-Deployment Analysis (Dedicated Form)

The deployment form shows a live "Analysis Summary" and configuration notes derived from Hugging Face metadata. A matching summary appears in the Live Estimate receipt.

• Text models: MoE, speculative decoding, parsers, quant, context — backend tuning applied automatically.
• Image models: detected via pipeline/library (e.g. Diffusers text-to-image); LoRA adapters resolve a base model when metadata provides it; GPU sizing uses inference VRAM estimates (not inflated Hub storage totals).
• High-severity items (rare) still disable "DEPLOY" with guidance to review.

Elastic = curated catalog. Dedicated = your HF ID (text LLM or Diffusers image model).

Examples of supported text LLM types:

• Dense instruct/chat models (Llama-3/3.1/4, Qwen2.5/3, Mistral, Gemma, etc.)
• MoE, speculative/reasoning, tool-calling, long-context variants

Examples of image paths:

• Diffusers full pipelines (e.g. SD Turbo, SDXL) that fit selected GPU class
• LoRA adapters with a resolvable base (hardware sized for the base)

See Provisioning for how sizing and status work after deploy.

Private endpoints require your API key; public endpoints do not.

Public vs Private Endpoints

During creation you choose the auth mode:

• Private (default): Every call must include a valid Authorization: Bearer hvz_live_... header belonging to the owner of the endpoint.
• Public: No Authorization header is required. Useful for demos, client-side apps, or when you want to add your own auth layer on top.

Even public endpoints are still owned by you and appear in your dashboard with full telemetry and billing attribution.

Statuses

Dedicated endpoints have these user-facing states:

Scale to Zero

When you provision with Min Nodes = 0, your endpoint can automatically hibernate after a period of inactivity (usually ~1 hour of no traffic). This stops the underlying compute so you stop paying for it.

• Look for the SLEEPING status in your dashboard and fleet list.
• The first request after sleep will wake it up (you'll briefly see WAKING UP).
• Cold starts are fast thanks to our persistent model cache, but expect a short delay (typically under a minute for hardware allocation + model load). Status usually updates within about a minute while the endpoint is waking.
• While waking, the playground and API will surface a clear message so you know what's happening instead of a confusing error.

You can monitor current active nodes and hourly burn on the endpoint detail page — these update based on actual running capacity.

Scale-to-zero is great for dev, staging, or spiky production workloads.

Calling a Dedicated Endpoint

Text (chat)

{
  "model": "endpt-a1b2c3d4e5f6",
  "messages": [...],
  "max_tokens": 2048,
  "stream": true
}

The dedicated proxy uses the endpoint’s configured model; the model field is ignored when present.

Image generation

{
  "prompt": "a red cube on a table",
  "n": 1,
  "num_inference_steps": 20
}

• prompt (required): text prompt (include any model trigger words from the model card). Max 4000 characters.
• n: number of images (platform caps at 1).
• num_inference_steps / steps: denoising steps (higher = slower, often better quality). Defaults are model-aware (e.g. Z-Image-Turbo uses 9 steps / guidance 0; SDXL-class defaults stay multi-step).
• guidance_scale / cfg_scale: CFG scale; turbo/distilled models often need 0.

Single A10G-class (24GB) is selected for families that fit ~16–22GB peak (including Tongyi-MAI/Z-Image-Turbo). Larger models (Flux-dev class, Qwen-Image) map to multi-GPU. See the image model matrix.

Response is OpenAI-shaped: { "created": ..., "data": [{ "url": "..." }] } (hosted image URL when available) or b64_json.

Billing: dedicated image endpoints use weekly GPU-hours (same as dedicated text). They do not deduct prepaid per-image elastic charges.

Auth: same as dedicated text — private endpoints need your API key (or owner session in the dashboard playground). Public image endpoints let anyone with the URL spend your GPU hours.

Scale-to-zero: optional; off by default for image. Cold starts after hibernation can take several minutes while weights reload.

Monitoring & Logs

Each dedicated endpoint has a detail page in the dashboard with:

• Current status and pricing burn rate
• Live logs and telemetry from the inference runtime
• Telemetry (when enabled)
• In-dashboard playground for quick testing
• Configuration (min/max nodes, scale-to-zero, custom domain, etc.)
• Terminate — permanently remove the endpoint (see below)

Terminate

From the endpoint detail page in the dashboard, use Terminate to tear down dedicated capacity and remove the block from your fleet.

• Capacity stops accruing further weekly charges after terminate; compute hours and add-ons already accrued that week still appear on the next weekly invoice.
• There is no in-place edit of model or min/max after create — terminate and re-provision if you need a different setup.

Custom Domains

Coming soon.

Pricing & Billing (summary)

Hypervize uses dual billing:

Dedicated endpoints are not charged against prepaid for GPU time. Full rules (compute minimums, add-on presence, payment failure / platform blocks) are in Billing & Payments. Tool metering detail: Tool Calls.

You see a live estimate before deploying.

Current Limitations

• Editing an existing endpoint (changing min/max, model, etc.) is not available in the UI. Terminate and re-provision instead.
• Scale-to-zero notifications and advanced autoscaling policies are simpler than always-on endpoints.
• Pre-warmed images for the fastest cold starts are not available for every model family.

When to Use Dedicated vs Elastic

Related Documentation

• Provisioning Dedicated Endpoints — create flow and what happens after Deploy
• Dashboard Guide — Inference — console UI for blocks, logs, playground
• Billing & Payments — weekly dedicated vs prepaid elastic/tools
• Tool Calling — tools work on dedicated; managed tools still bill prepaid
• Supported Models — image model matrix and HF notes