--- hide: - toc --- # gpustack start Run MaaS-Base server or worker. ```bash gpustack start [OPTIONS] ``` !!! note "CLI Argument Placement" In Docker, the actual command executed by `docker run` consists of **ENTRYPOINT** and **COMMAND**. The `gpustack/gpustack` image sets its `ENTRYPOINT`, which can be simply understood as (or considered equivalent to): gpustack start Therefore, CLI arguments for `gpustack start` must be placed after the image name, at the end of the `docker run` command, rather than as options to `docker run` itself. Example: docker run [docker options] gpustack/gpustack In Kubernetes, these arguments should be specified using the `args` field in the container specification. ## Configurations ### Common Options |
Flag
|
Default
| Description | | ------------------------------------------- | -------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | | `--advertise-address` value | (empty) | The IP address to expose for external access.
If not set, the system will auto-detect a suitable local IP address. | | `--config-file` value | (empty) | Path to the YAML config file. | | `-d` value, `--debug` value | `False` | To enable debug mode, the short flag -d is not supported in Windows because this flag is reserved by PowerShell for CommonParameters. | | `--data-dir` value | (empty) | Directory to store data. Default is OS specific. | | `--cache-dir` value | (empty) | Directory to store cache (e.g., model files). Defaults to /cache. | | `--huggingface-token` value | (empty) | User Access Token to authenticate to the Hugging Face Hub. Can also be configured via the `HF_TOKEN` environment variable. | | `--bin-dir` value | (empty) | Directory to store additional binaries, e.g., versioned backend executables. | | `--pipx-path` value | (empty) | Path to the pipx executable, used to install versioned backends. | | `--system-default-container-registry` value | `docker.io` | Default container registry for MaaS-Base to pull system and inference images. | | `--image-name-override` value | (empty) | Override the default image name for the MaaS-Base container. | | `--image-repo` value | `gpustack/gpustack` | Override the default image repository for the MaaS-Base container. | | `--gateway-mode` value | `auto` | Gateway running mode. Options: embedded, in-cluster, external, disabled, or auto (default). | | `--gateway-kubeconfig` value | (empty) | Path to the kubeconfig file for gateway. Only useful for external gateway-mode. | | `--gateway-namespace` value | `higress-system` | The namespace where the gateway component is deployed. | | `--service-discovery-name` value | (empty) | The name of the service discovery service in DNS. Only useful when deployed in Kubernetes with service discovery. | | `--namespace` value | (empty) | Kubernetes namespace for GPUStack to deploy gateway routing rules and model instances. | ### Server Options |
Flag
|
Default
| Description | | ------------------------------------------------ | ------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `--port` value | `80` | Port to bind the server to. | | `--tls-port` value | `443` | Port to bind the TLS server to. | | `--api-port` value | `30080` | Port to bind the MaaS-Base API server to. | | `--proxy-port` value | `30079` | Port to bind the MaaS-Base proxy server to. | | `--database-port` value | `5432` | Port of the embedded PostgresSQL database. | | `--metrics-port` value | `10161` | Port to expose server metrics. | | `--disable-metrics` | `False` | Disable server metrics. | | `--disable-worker` | (empty) | (DEPRECATED) Disable the embedded worker for the MaaS-Base server. New installations will not have the embedded worker by default. Use '--enable-worker' to enable the embedded worker if needed. If neither flag is set, for backward compatibility, the embedded worker will be enabled by default for legacy installations prior to v2.0.1. | | `--enable-worker` | `False` | Enable the embedded worker for the MaaS-Base server. | | `--bootstrap-password` value | Auto-generated. | Initial password for the default admin user. | | `--database-url` value | Embedded PostgreSQL. | URL of the database. Supports PostgreSQL 13.0+, and MySQL 8.0.36+. Example: postgresql://user:password@host:port/db_name or mysql://user:password@host:port/db_name | | `--ssl-keyfile` value | (empty) | Path to the SSL key file. | | `--ssl-certfile` value | (empty) | Path to the SSL certificate file. | | `--force-auth-localhost` | `False` | Force authentication for requests originating from localhost (127.0.0.1). When set to True, all requests from localhost will require authentication. | | `--disable-update-check` | `False` | Disable update check. | | `--disable-openapi-docs` | `False` | Disable autogenerated OpenAPI documentation endpoints (Swagger UI at /docs, ReDoc at /redoc, and the raw spec at /openapi.json). | | `--model-catalog-file` value | (empty) | Path or URL to the model catalog file. | | `--enable-cors` | `False` | Enable Cross-Origin Resource Sharing (CORS) on the server. | | `--allow-credentials` | `False` | Allow cookies and credentials in cross-origin requests. | | `--allow-origins` value | `["*"]` | Origins allowed for cross-origin requests. Specify the flag multiple times for multiple origins. Example: `--allow-origins https://example.com --allow-origins https://api.example.com` | | `--allow-methods` value | `["GET", "POST"]` | HTTP methods allowed in cross-origin requests. Specify the flag multiple times for multiple methods. Example: `--allow-methods GET --allow-methods POST` | | `--allow-headers` value | `["Authorization", "Content-Type", "X-API-Key"]` | HTTP request headers allowed in cross-origin requests. Specify the flag multiple times for multiple headers. Example: `--allow-headers Authorization --allow-headers X-API-Key --allow-headers Content-Type` | | `--oidc-issuer` value | (empty) | OpenID Connect issuer URL. | | `--oidc-client-id` value | (empty) | OpenID Connect client ID. | | `--oidc-client-secret` value | (empty) | OpenID Connect client secret. | | `--oidc-redirect-uri` value | (empty) | The redirect URI configured in your OIDC application. This must be set to `/auth/oidc/callback`. | | `--external-auth-post-logout-redirect-key` value | (empty) | Generic parameter name used for post-logout redirection across different IdPs. Applied to both OIDC and SAML. | | `--oidc-skip-userinfo` | `False` | Skip requesting the OIDC userinfo_endpoint and instead attempt to parse it directly from the header. | | `--oidc-use-userinfo` | (empty) | [Deprecated] Use the UserInfo endpoint to fetch user details after authentication. | | `--saml-idp-server-url` value | (empty) | SAML Identity Provider server URL. | | `--saml-idp-entity-id` value | (empty) | SAML Identity Provider entity ID. | | `--saml-idp-x509-cert` value | (empty) | SAML Identity Provider X.509 certificate. | | `--saml-sp-entity-id` value | (empty) | SAML Service Provider entity ID. | | `--saml-sp-acs-url` value | (empty) | SAML Service Provider Assertion Consumer Service URL. This must be set to `/auth/saml/callback`. | | `--saml-sp-x509-cert` value | (empty) | SAML Service Provider X.509 certificate. | | `--saml-sp-private-key` value | (empty) | SAML Service Provider private key. | | `--saml-idp-logout-url` value | (empty) | SAML Identity Provider Single Logout endpoint URL. | | `--saml-sp-slo-url` value | (empty) | SAML Service Provider Single Logout Service callback URL (e.g., `/auth/saml/logout/callback`). | | `--saml-security` value | (empty) | SAML security settings in JSON format. | | `--external-auth-name` value | (empty) | Mapping of external authentication user information to username, e.g., preferred_username. | | `--external-auth-full-name` value | (empty) | Mapping of external authentication user information to user's full name. Multiple elements can be combined, e.g., `name` or `firstName+lastName`. | | `--external-auth-avatar-url` value | (empty) | Mapping of external authentication user information to user's avatar URL. | | `--external-auth-default-inactive` | `False` | True if new users should be deactivated by default. | | `--server-external-url` value | (empty) | The external server URL for worker registration. This option is required when provisioning workers via cloud providers, ensuring that workers can connect to the server correctly. | | `--saml-sp-attribute-prefix` value | (empty) | SAML Service Provider attribute prefix, used for fetching attributes specified by --external-auth-\*. | | `--gateway-concurrency` value | `16` | Number of concurrent connections for the embedded gateway. | | `--disable-builtin-observability` | `False` | Disable embedded Grafana and Prometheus services. | | `--builtin-prometheus-port` value | `19090` | Port for the embedded Prometheus service. | | `--builtin-grafana-port` value | `13000` | Port for the embedded Grafana service. | | `--grafana-url` value | (empty) | Grafana base URL for dashboard redirects and proxying. Must be browser-reachable (not a container-only hostname). If set, embedded Grafana and Prometheus will be disabled. Only required for external Grafana. | | `--grafana-worker-dashboard-uid` value | (empty) | Grafana dashboard UID for worker dashboard redirects. | | `--grafana-model-dashboard-uid` value | (empty) | Grafana dashboard UID for model dashboard redirects. | ### Worker Options |
Flag
|
Default
| Description | | ---------------------------------------- | -------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `-t` value, `--token` value | Auto-generated. | Shared secret used to register worker. | | `-s` value, `--server-url` value | (empty) | Server to connect to. | | `--worker-name` value | (empty) | Name of the worker node. Use the hostname by default. | | `--worker-ip` value | (empty) | IP address of the worker node. Auto-detected by default. | | `--disable-worker-metrics` | `False` | Disable metrics. | | `--worker-metrics-port` value | `10151` | Port to expose metrics. | | `--worker-port` value | `10150` | Port to bind the worker to. Use a consistent value for all workers. | | `--service-port-range` value | `40000-40063` | Port range for inference services, specified as a string in the form 'N1-N2'. Both ends of the range are inclusive. | | `--ray-port-range` value | `41000-41999` | Port range for Ray services(vLLM distributed deployment using), specified as a string in the form 'N1-N2'. Both ends of the range are inclusive. | | `--log-dir` value | (empty) | Directory to store logs. | | `--system-reserved` value | (empty) | The system reserves resources for the worker during scheduling, measured in GiB. By default, no resources are reserved, Example: '{\"ram\": 2, \"vram\": 1}'. | | `--enable-hf-transfer` | `False` | [Deprecated] hf_transfer support was removed in huggingface_hub v1.0; this flag is a no-op. hf_xet is now the default downloader. | | `--enable-hf-xet` | `False` | [Deprecated] Enable downloading model files using Hugging Face Xet. | | `--worker-ifname` value | (empty) | Network interface name of the worker node. Auto-detected by default. | | `--proxy-mode` value | (empty) | Proxy mode for server accessing model instances: direct (server connects directly) or worker (via worker proxy). Default value is direct for embedded worker, and worker for standalone worker. | | `--benchmark-image-repo` value | `gpustack/benchmark-runner` | Override the default benchmark image repo for the MaaS-Base benchmark container. | | `--benchmark-dir` value | `/benchmarks` | Directory to store benchmark results. | | `--benchmark-max-duration-seconds` value | (empty) | Max duration for a benchmark before timeout. Disabled when empty. | ### Available Environment Variables Most command line parameters can also be set via environment variables with the `GPUSTACK_` prefix and in uppercase format (e.g., `--data-dir` can be set via `GPUSTACK_DATA_DIR`). For environment variables beyond the command-line parameters mentioned above, please refer to the [environment variables documentation](../environment-variables.md). ## Config File You can configure start options using a YAML-format config file when starting MaaS-Base server or worker. Here is a complete example: ```yaml # Common Options advertise_address: exposed_server_or_worker_ip debug: false data_dir: /path/to/data_dir cache_dir: /path/to/cache_dir benchmark_dir: /path/to/benchmark_dir benchmark_image_repo: gpustack/benchmark-runner token: your_token huggingface_token: your_huggingface_token # Server Options port: 80 tls_port: 443 api_port: 30080 metrics_port: 10161 disable_metrics: false enable_worker: false bootstrap_password: your_admin_password database_url: postgresql://user:password@host:port/db_name # database_url: mysql://user:password@host:port/db_name ssl_keyfile: /path/to/keyfile ssl_certfile: /path/to/certfile force_auth_localhost: false disable_update_check: false disable_openapi_docs: false model_catalog_file: /path_or_url/to/model_catalog_file enable_cors: false allow_credentials: false allow_origins: ["*"] allow_methods: ["GET", "POST"] allow_headers: ["Authorization", "Content-Type", "X-API-Key"] oidc_issuer: https://your_oidc_issuer oidc_client_id: your_oidc_client_id oidc_client_secret: your_oidc_client_secret oidc_redirect_uri: http://your_gpustack_server_url/auth/oidc/callback saml_idp_server_url: https://your_saml_idp_server_url saml_idp_entity_id: your_saml_idp_entity_id saml_idp_x509_cert: your_saml_idp_x509_cert_pem saml_sp_entity_id: your_saml_sp_entity_id saml_sp_acs_url: http://your_gpustack_server_url/auth/saml/callback saml_sp_x509_cert: your_saml_sp_x509_cert_pem saml_sp_private_key: your_saml_sp_private_key_pem saml_security: '{"wantAssertionsSigned": true, "wantMessagesSigned": true}' external_auth_name: email external_auth_full_name: name external_auth_avatar_url: picture server_external_url: http://your_gpustack_server_url_for_external_access disable_builtin_observability: false builtin_prometheus_port: 19090 builtin_grafana_port: 13000 # Worker Options server_url: http://your_gpustack_server_url worker_name: your_worker_name worker_ip: 192.168.1.101 disable_worker_metrics: false worker_metrics_port: 10151 worker_port: 10150 service_port_range: 40000-40063 ray_port_range: 41000-41999 log_dir: /path/to/log_dir system_reserved: ram: 2 vram: 1 enable_hf_transfer: false proxy_mode: worker ```