> ## Documentation Index
> Fetch the complete documentation index at: https://instabox.sh/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# GPU machines

> Create a Linux machine with NVIDIA GPUs, CUDA, and PyTorch, on every paid plan.

A GPU machine is a Linux machine on a host with NVIDIA GPUs, so you can run CUDA and PyTorch work,
such as model inference, training, or a simulation, next to your code. To create one, name the GPU
type:

```bash CLI theme={null}
instabox create --name train --gpu l4
```

```bash API theme={null}
curl -X POST https://api.instabox.sh/v1/machines \
  -H "X-API-Key: $INSTABOX_API_KEY" \
  -H "Idempotency-Key: $(uuidgen)" \
  -H "Content-Type: application/json" \
  -d '{"name": "train", "gpu": "l4"}'
```

In the dashboard, open **Create Machine**, pick **GPU (Linux)**, then the GPU type and count.

GPU machines are available on every paid plan, with up to 20 GPUs per workspace. The
free plan cannot create them.

## GPU types

Each GPU brings a fixed share of vCPUs and memory, so the machine's size comes from the GPU type and
count, and you cannot set the vCPUs or memory yourself.

| GPU | Price | Size per GPU | All in, per GPU |
| - | - | - | - |
| NVIDIA L4 (`l4`) | \$0.64 per GPU-hour | 11 vCPU / 36 GiB | \$0.765 per hour |
| NVIDIA H100 PCIe (`h100`) | \$3.00 per GPU-hour | 44 vCPU / 224 GiB | \$3.68 per hour |
| NVIDIA A100-SXM4-40GB (`a100`) | \$2.09 per GPU-hour | 11 vCPU / 70 GiB | \$2.292 per hour |
| Tesla T4 (`t4`) | \$0.49 per GPU-hour | 7 vCPU / 24 GiB | \$0.572 per hour |

To give a machine more than one GPU, add the count after the type. Counts are powers of two, so a
host's GPUs always split evenly between machines: `l4`, `a100`, and `t4` machines take 1, 2, or 4
GPUs, and an `h100` machine takes one.

```bash theme={null}
instabox create --name train --gpu l4:2
```

GPUs are billed per GPU, by the second, only while the machine runs. A paused or stopped GPU machine
holds no GPU, so it does not count toward your workspace's limit ([Pricing](https://instabox.sh/docs/account/pricing.md#gpu)).

When every GPU of a type is in use, a create or start fails with `no_capacity`. Add `--queue` to
wait in line for a GPU instead: the machine starts when one frees up
([Wait for a GPU](https://instabox.sh/docs/machines/gpu-queue.md)).

## What is preinstalled

InstaBox picks the template from the GPU type: `gpu-metal` for `h100`, and `gpu` for the other
types. Both have the CUDA 13.0 toolkit (`nvcc`), PyTorch 2.8 with CUDA in `/opt/venv`, and
`instabox-gpu-smoke`, which checks that the GPU works. On `gpu`, the NVIDIA driver and `nvidia-smi`
come from the host. An `h100` machine gets the whole GPU, so `gpu-metal` brings its own driver.
Check the GPU once the machine runs:

```bash theme={null}
instabox exec train -- nvidia-smi
```

## Model weights

A GPU machine's `/data` disk is 200 GiB at create, whatever its GPU count, and your home is
`/data/home`, so a model you download lands on the disk and is still there after a
stop, a start, a pause, or a snapshot. Keep weights out of `/`: the root disk is the template's
19 GiB, with little room left.

The `hf` CLI is preinstalled, with Hugging Face's Xet transfers in high-performance mode
(`HF_XET_HIGH_PERFORMANCE=1`), so a download uses all the machine's bandwidth:

```bash theme={null}
instabox exec train -- hf download Qwen/Qwen3-8B
```

It goes to `~/.cache/huggingface`, where `transformers`, vLLM, and `diffusers` find it on the next
run without another download. Pass `--local-dir /data/models/qwen3-8b` for a plain directory
instead, for llama.cpp or a tool that wants a path.

When a model needs more room, grow the disk while the machine runs, up to your plan's cap
(200 GiB on Pro, 500 GiB on Builder, 1024 GiB on Team and Enterprise):

```bash theme={null}
instabox resize train --disk 500
```

To download a model once for your whole workspace, [save a snapshot](#start-gpu-machines-from-a-snapshot)
after the download and create the next machines from it. Check that the model fits the machine
before you start: a weight file larger than the GPUs' memory is run from the machine's memory and
disk, which is much slower, and a model that does not fit the machine's memory either needs more
GPUs (`--gpu a100:4`) or a smaller quantization.

## Start GPU machines from a snapshot

Install your models and libraries once, save a [snapshot](https://instabox.sh/docs/machines/snapshots.md), and start new GPU
machines from it. A machine created from a GPU machine's snapshot keeps its GPU type. It gets the
snapshot's GPU count unless you ask for another:

```bash theme={null}
instabox create --name train-2 --snapshot gpu-base --gpu l4:4
```

## What works differently

A GPU machine is a Linux machine, so the pages under Configuration and Access apply to it, except
for these:

* **Fork**: a GPU machine cannot be [forked](https://instabox.sh/docs/machines/fork.md). The fork is refused with
  `fork_unavailable`. To copy one, save a snapshot and create new machines from it, as above.
* **Snapshots**: a running GPU machine cannot be saved live. Stop it first, or add `--restart` to
  `instabox snapshot save` ([Snapshots](https://instabox.sh/docs/machines/snapshots.md)). A GPU machine's snapshot always
  keeps `/data`, so `--root-only` is refused.
* **Size**: the vCPUs and memory come from the GPU type and count, so a resize of the vCPUs or
  memory is refused. You can still grow the disk (`instabox resize train --disk 200`). For more
  GPUs, save a snapshot and create from it with another count (`--gpu l4:4`).
* **Pause on `h100`**: an `h100` machine cannot be paused, because its GPU memory cannot be saved.
  Idle time and the max run time stop it instead, which keeps its files but not its running
  processes ([Auto-pause and max-run](https://instabox.sh/docs/machines/auto-pause.md)).

> [accordion] Memory inside a GPU machine
>
> A GPU machine runs in a sandbox whose `/proc/meminfo` differs from a Linux kernel's in one way that
> matters for large models: file pages you map or read (model weights opened with `mmap`, for
> example) count as used memory, and `MemAvailable` equals `MemFree`. So once the files you map are
> larger than the machine's memory, `MemFree` and `MemAvailable` read 0 and `Cached` can be larger
> than `MemTotal`, even though most of that memory is file cache that can be reclaimed. Tools that
> size buffers from "available" memory (psutil's `virtual_memory().available`, and the CPU offload or
> pinned-memory budgets of some inference servers) then see none.
>
> `MemTotal` is the machine's memory, and `AnonPages` is the memory programs hold (shared memory and
> tmpfs included). To estimate what is free, treat file cache as reclaimable:
>
> ```bash theme={null}
> awk '/^MemTotal|^AnonPages/ {v[$1]=$2} END {printf "%.1f GiB\n", (v["MemTotal:"]-v["AnonPages:"])/1048576}' /proc/meminfo
> ```
>
> Set such budgets from that estimate, or explicitly. A model whose weights do not fit in the
> machine's memory is read from disk again and again while it runs. Give it more GPUs, and so more
> memory: `--gpu l4:2` has twice the memory of `--gpu l4`.

> [card] [GPU workloads](https://instabox.sh/docs/use-cases/gpu)
>
> Serve a model, train a model, generate images, and more, step by step.

> [card] [Wait for a GPU](https://instabox.sh/docs/machines/gpu-queue.md)
>
> Queue a create or start when every GPU of a type is in use.
