Provisioning Dedicated Endpoints

This page is for how dedicated endpoints are created and what happens after Deploy. For calling endpoints, statuses, scale-to-zero, terminate, and day-to-day management, see Dedicated Inference.

This document explains exactly what happens when you click Deploy on a dedicated endpoint.

High-Level Flow

1. You deploy from the dashboard (authenticated via your session).
2. Hardware sizing — Hypervize analyzes the model using live Hugging Face metadata when available, with intelligent name-based fallbacks.
3. Infrastructure provisioning — We allocate and configure dedicated GPU capacity with auto-scaling policies, container runtime, and networking on your behalf.
4. Endpoint registration — We register the endpoint and its scaling, pricing, and optional custom domain / telemetry settings in your account.
5. Immediate response — You receive the endpointId and stable URL right away (status starts as building).
6. Background completion — Hypervize monitors the process, moves the status to up or failed, and emails you when the endpoint is ready or failed. Failure emails include a plain-English why (capacity, GPU memory, model load, health check, HF access, etc.), optional technical detail, and suggested next steps.

Hardware Selection Logic

Hypervize automatically analyzes the requested model and provisions appropriate dedicated GPU capacity.

• When possible, it queries the Hugging Face Hub for the model's size to estimate required VRAM.
• It then selects a suitable hardware tier (ranging from single A10G-class GPUs up to 8× Blackwell B200 configurations).
• Name-based heuristics act as a fallback when live metadata is unavailable or gated.

Storage volume size is intelligently sized based on the model footprint to ensure reliable loading.

What Hypervize Provisions

For each dedicated endpoint we configure and manage:

• A dedicated container running our high-performance inference runtime, tuned for the specific model and hardware tier.
• Auto-scaling policies that respect your min/max instance settings. Set Min Nodes to 0 to enable scale-to-zero (the endpoint will hibernate when idle and wake on the next request).
• Networking, health checking, and secure isolation boundaries.
• A stable URL at /api/d/{endpoint-id} with the same OpenAI-compatible API as Elastic inference.

Endpoint Records

Hypervize records the endpoint, its autoscaling configuration (min/max nodes and scale-to-zero behavior), pricing details, and any custom domain or telemetry settings. This powers the dashboard view, usage reporting, and the stable endpoint URL.

Model Analysis and Configuration

For HF-sourced models, detailed metadata inspection (augmented by known serving recipes where available) drives model-specific configuration for reliable text LLM serving:

• Detection identifies MoE, speculative support, tool/reasoning parser needs, and other characteristics from HF metadata and recipes.
• Provisioning automatically applies corresponding settings (expert parallel for MoE, speculative decoding with tuned tokens, custom parsers, KV cache options, prefix caching for long context, chunked prefill, etc.).
• Fallbacks and sizing improve with architecture-aware heuristics when live metadata or recipe data is limited.
• The UI surfaces neutral, informative analysis (Analysis Summary + Configuration Notes) rather than scary warnings. High-severity items still gate deployment with guidance.
• The dedicated deployment form and receipt include live Analysis Summary (quantization, context, MoE/speculative, parsers, max lengths, etc.) and notes on what is configured automatically.

Curated Elastic vs Flexible Dedicated: Elastic uses pre-curated, high-performance models with opinionated defaults. Dedicated (HF source) lets you run any text LLM with automatic tuning for compatibility and performance.

Examples of supported text LLM types (with automatic handling):

• Standard dense models (Llama, Qwen, Mistral, Gemma, etc.)
• MoE models (Mixtral, DeepSeek-V3, Qwen2-MoE, etc.) — expert parallelism enabled
• Reasoning/speculative models (Hy3, GLM, certain DeepSeek) — speculative decoding enabled with tuned tokens
• Tool-calling / agentic models — appropriate parsers applied
• Quantized variants (FP8, AWQ, GPTQ, GGUF where supported) — load format and KV cache settings applied
• Long-context models — prefix caching, chunked prefill, conservative max lengths

Non-standard model types trigger Configuration Notes explaining what Hypervize does automatically (custom parsers, specialized tuning) instead of asking you to configure anything.

Status Transitions (Expected)

• building → up (ready)
• For scale-to-zero (Min Nodes = 0): up → hibernated (sleeping after inactivity) → waking (on first request) → up

You'll see clear status badges in the dashboard ("ONLINE", "SLEEPING", "WAKING UP", etc.). The playground surfaces friendly messages when an endpoint is waking from sleep. During wake-ups, status usually updates within about a minute until the instance reports ready.

What You Receive Immediately

{
  "success": true,
  "endpointId": "endpt-a1b2c3d4e5f6",
  "defaultUrl": "https://hypervize.tech/api/d/endpt-a1b2c3d4e5f6/chat/completions",
  "message": "Endpoint provisioning initiated. You will receive an email when it is ready."
}

We email you when the endpoint reaches a ready or failed state.

Custom Domains

Attach a custom domain when you create the endpoint (deployment form), or later via support. Custom domain is billed as a presence add-on on the weekly dedicated invoice (see Billing & Payments).

Terminate

After an endpoint exists, Terminate is available on the dashboard detail page. It removes capacity and drops the block from your fleet; weekly charges already accrued that week still invoice. Details: Dedicated Inference → Terminate.

Current Limitations

• No in-place editing of min/max or model after creation — terminate and re-provision if you need a different setup.

Related

• Dedicated Inference Endpoints — call, statuses, terminate, custom domains
• Dashboard Guide — Inference
• Billing & Payments