- backend_name: Kokoro-FastAPI
  description: 'Kokoro-FastAPI (https://github.com/remsky/Kokoro-FastAPI) is a dedicated
    inference backend for serving the Kokoro TTS model (https://huggingface.co/hexgrad/Kokoro-82M).

    Kokoro is an open-weight text-to-speech model with 82 million parameters, delivering
    quality comparable to much larger models while being significantly faster and
    more cost-efficient.


    Supported model names are strictly limited to:

    - kokoro

    - tts-1

    - tts-1-hd


    The backend image includes a built-in model, so the model selected on the Deployments
    page may be ignored.


    Available voices:

    af_alloy, af_aoede, af_bella, af_heart, af_jadzia, af_jessica, af_kore, af_nicole,
    af_nova, af_river, af_sarah, af_sky, af_v0, af_v0bella, af_v0irulan, af_v0nicole,
    af_v0sarah, af_v0sky,

    am_adam, am_echo, am_eric, am_fenrir, am_liam, am_michael, am_onyx, am_puck, am_santa,
    am_v0adam, am_v0gurney, am_v0michael,

    bf_alice, bf_emma, bf_lily, bf_v0emma, bf_v0isabella,

    bm_daniel, bm_fable, bm_george, bm_lewis, bm_v0george, bm_v0lewis,

    ef_dora, em_alex, em_santa,

    ff_siwis,

    hf_alpha, hf_beta,

    hm_omega, hm_psi,

    if_sara, im_nicola,

    jf_alpha, jf_gongitsune, jf_nezumi, jf_tebukuro,

    jm_kumo,

    pf_dora, pm_alex, pm_santa,

    zf_xiaobei, zf_xiaoni, zf_xiaoxiao, zf_xiaoyi,

    zm_yunjian, zm_yunxi, zm_yunxia, zm_yunyang

    '
  health_check_path: /health
  version_configs:
    latest-gpu:
      image_name: ghcr.io/remsky/kokoro-fastapi-gpu:latest
      custom_framework: cuda
    latest-cpu:
      image_name: ghcr.io/remsky/kokoro-fastapi-cpu:latest
      custom_framework: cpu
  default_run_command: python -m uvicorn api.src.main:app --host 0.0.0.0 --port {{port}}
    --log-level debug
  is_built_in: false
  framework_index_map:
    cuda:
    - latest-gpu
    cpu:
    - latest-cpu
  icon: data:image/png;base64,[encoded data, 18000 characters]
- backend_name: llama.cpp
  description: 'llama.cpp (https://github.com/ggml-org/llama.cpp) is a lightweight,
    high-performance inference engine for running large language models locally.

    It focuses on efficient CPU and edge deployment with minimal dependencies, supporting
    quantization and broad hardware compatibility.

    It enables fast, low-cost LLM inference without requiring GPUs or complex infrastructure.

    Only models in the GGUF format are supported.

    '
  version_configs:
    cuda:
      image_name: ghcr.io/ggml-org/llama.cpp:server-cuda
      custom_framework: cuda
    cpu:
      image_name: ghcr.io/ggml-org/llama.cpp:server
      custom_framework: cpu
    vulkan:
      image_name: ghcr.io/ggml-org/llama.cpp:server-vulkan
      custom_framework: rocm
    musa:
      image_name: ghcr.io/ggml-org/llama.cpp:server-musa
      custom_framework: musa
    rocm:
      image_name: ghcr.io/ggml-org/llama.cpp:server-rocm
      custom_framework: rocm
  default_version: cpu
  default_run_command: -m {{model_path}} --host 0.0.0.0 --port {{port}} --alias {{model_name}}
  is_built_in: false
  health_check_path: /v1/models
  framework_index_map:
    cuda:
    - cuda
    cpu:
    - cpu
    rocm:
    - rocm
    - vulkan
    musa:
    - musa
  icon: data:image/png;base64,[encoded data, 43328 characters]
- backend_name: MinerU
  description: 'MinerU (https://github.com/opendatalab/mineru) is a document parsing
    engine that converts complex PDFs into structured, machine-readable formats such
    as Markdown and JSON.

    It was originally developed during the pre-training of InternLM and is particularly
    strong at handling formulas, symbols, and structured content commonly found in
    scientific literature.


    This inference backend image is built on top of the official MinerU Dockerfile:

    https://github.com/opendatalab/MinerU/blob/master/docker/global/Dockerfile


    The backend image includes a built-in model, so the model selected on the Deployments
    page may be ignored.

    '
  health_check_path: /v1/models
  default_run_command: mineru-vllm-server --port {{port}} --served-model-name {{model_name}}
  version_configs:
    2.7.0:
      image_name: gpustackcommunity/mineru:v2.7.0
      custom_framework: cuda
    3.0.5:
      image_name: gpustackcommunity/mineru:3.0.5
      custom_framework: cuda
    3.0.4-a2:
      image_name: gpustackcommunity/mineru:3.0.4-800I-A2-aarch64
      custom_framework: cann
    3.0.4-a3:
      image_name: gpustackcommunity/mineru:3.0.4-800I-A3-aarch64
      custom_framework: cann
    3.0.4-310p:
      image_name: gpustackcommunity/mineru:3.0.4-300I-Duo-aarch64
      custom_framework: cann
  default_version: 3.0.5
  icon: data:image/png;base64,[encoded data, 24104 characters]
- backend_name: mis-tei
  description: 'mis-tei (https://www.hiascend.com/developer/ascendhub/detail/07a016975cc341f3a5ae131f2b52399d)
    is an inference service for text embedding and reranking models in the Ascend
    ecosystem.

    It provides efficient vector generation and relevance scoring capabilities, commonly
    used in RAG, semantic search, and retrieval systems.

    Based on the Ascend CANN stack, it supports multiple embedding and reranker models
    with optimized performance on Ascend hardware.

    This backend uses a repackaged image to simplify deployment and integration within
    GPUStack environments.

    '
  health_check_path: /health
  default_run_command: --model-id {{model_path}} -p {{port}}
  default_env:
    ENABLE_BOOST: 'True'
    AUTO_TRUNCATE: 'true'
  version_configs:
    7.3.0-a2:
      image_name: gpustackcommunity/mis-tei:7.3.0-800I-A2-aarch64
      custom_framework: cann
    7.3.0-a3:
      image_name: gpustackcommunity/mis-tei:7.3.0-800I-A3-aarch64
      custom_framework: cann
    7.3.0-310p:
      image_name: gpustackcommunity/mis-tei:7.3.0-300I-Duo-aarch64
      custom_framework: cann
  default_version: 7.3.0-a2
  icon: data:image/png;base64,[encoded data, 6780 characters]
- backend_name: PaddleX-GenAI-Server
  description: 'PaddleX-GenAI-Server is a production-ready serving framework within
    the PaddleX ecosystem, designed to package PaddleOCR-VL as standardized, high-performance
    API services.

    By integrating advanced inference backends such as vLLM and SGLang, it enables
    stable, scalable deployment of generative document parsing capabilities in production
    environments.


    For detailed usage instructions, please refer to:

    https://www.paddleocr.ai/latest/version3.x/pipeline_usage/PaddleOCR-VL.html

    '
  version_configs:
    latest-nvidia-gpu:
      image_name: ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlepaddle/paddleocr-genai-vllm-server:latest-nvidia-gpu
      custom_framework: cuda
    latest-nvidia-gpu-sm120:
      image_name: ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlepaddle/paddleocr-genai-vllm-server:latest-nvidia-gpu-sm120
      custom_framework: cuda
    latest-huawei-npu:
      image_name: ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlepaddle/paddleocr-genai-vllm-server:latest-huawei-npu
      custom_framework: cann
    latest-hygon-dcu:
      image_name: ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlepaddle/paddleocr-genai-vllm-server:latest-hygon-dcu
      custom_framework: dtk
    latest-iluvatar-gpu:
      image_name: ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlepaddle/paddleocr-genai-fastdeploy-server:latest-iluvatar-gpu
      run_command: paddleocr genai_server --model_name {{model_name}} --model_dir
        {{model_path}} --host 0.0.0.0 --port {{port}} --backend fastdeploy
      custom_framework: corex
    latest-metax-gpu:
      image_name: ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlepaddle/paddleocr-genai-fastdeploy-server:latest-metax-gpu
      run_command: paddleocr genai_server --model_name {{model_name}} --model_dir
        {{model_path}} --host 0.0.0.0 --port {{port}} --backend fastdeploy
      custom_framework: maca
  default_run_command: paddleocr genai_server --model_name {{MODEL_NAME}} --model_dir
    {{model_path}} --host 0.0.0.0 --port {{port}} --backend vllm
  health_check_path: /v1/models
  default_env:
    PADDLE_PDX_DISABLE_MODEL_SOURCE_CHECK: 'True'
    MODEL_NAME: PaddleOCR-VL-1.5-0.9B
  framework_index_map:
    cuda:
    - latest-nvidia-gpu
    - latest-nvidia-gpu-sm120
    cann:
    - latest-huawei-npu
    dtk:
    - latest-hygon-dcu
    corex:
    - latest-iluvatar-gpu
    maca:
    - latest-metax-gpu
  icon: data:image/png;base64,[encoded data, 51972 characters]
- backend_name: Text-Embeddings-Inference
  description: 'TEI (Text Embeddings Inference) is a high-performance engine by Hugging
    Face specifically optimized for deploying embedding and reranking models on CUDA
    or CPU.


    It is the definitive choice for RAG production environments, offering ultra-low
    latency and superior throughput compared to general LLM frameworks like vLLM or
    llama.cpp.

    '
  health_check_path: /health
  default_run_command: --model-id {{model_path}} --port {{port}}
  is_built_in: false
  version_configs:
    CPU-1.9:
      image_name: ghcr.io/huggingface/text-embeddings-inference:cpu-1.9
      custom_framework: cpu
    CUDA-Turing-1.9:
      image_name: ghcr.io/huggingface/text-embeddings-inference:turing-1.9
      custom_framework: cuda
    CUDA-Ampere-sm80-1.9:
      image_name: ghcr.io/huggingface/text-embeddings-inference:1.9
      custom_framework: cuda
    CUDA-Ampere-sm86-1.9:
      image_name: ghcr.io/huggingface/text-embeddings-inference:86-1.9
      custom_framework: cuda
    CUDA-Ada-sm89-1.9:
      image_name: ghcr.io/huggingface/text-embeddings-inference:89-1.9
      custom_framework: cuda
    CUDA-Hopper-1.9:
      image_name: ghcr.io/huggingface/text-embeddings-inference:hopper-1.9
      custom_framework: cuda
    CUDA-Blackwell-sm100-1.9:
      image_name: ghcr.io/huggingface/text-embeddings-inference:100-1.9
      custom_framework: cuda
    CUDA-Blackwell-sm120-1.9:
      image_name: ghcr.io/huggingface/text-embeddings-inference:120-1.9
      custom_framework: cuda
    CUDA-Blackwell-sm121-1.9:
      image_name: ghcr.io/huggingface/text-embeddings-inference:121-1.9
      custom_framework: cuda
  default_version: CPU-1.9
  framework_index_map:
    cpu:
    - CPU-1.9
    cuda:
    - CUDA-Hopper-1.9
    - CUDA-Turing-1.9
    - CUDA-Ampere-sm80-1.9
    - CUDA-Ampere-sm86-1.9
    - CUDA-Ada-sm89-1.9
    - CUDA-Blackwell-sm100-1.9
    - CUDA-Blackwell-sm120-1.9
    - CUDA-Blackwell-sm121-1.9
  icon: data:image/png;base64,[encoded data, 35696 characters]
