> ## Documentation Index
> Fetch the complete documentation index at: https://instabox.sh/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# GPU workloads

> The GPUs you can use on InstaBox, what a GPU machine comes with, and a step-by-step guide for each kind of workload.

A GPU machine is an InstaBox Linux machine with one or more NVIDIA GPUs. You create it with one
command, it runs a few seconds later, and you pay for the GPU by the second while it runs. Use it to
serve an open model, train a model, generate images, or run a model over your own files.

## What you get

| GPU | Price | Size per GPU | All in, per GPU |
| - | - | - | - |
| NVIDIA L4 (`l4`) | \$0.64 per GPU-hour | 11 vCPU / 36 GiB | \$0.765 per hour |
| NVIDIA H100 PCIe (`h100`) | \$3.00 per GPU-hour | 44 vCPU / 224 GiB | \$3.68 per hour |
| NVIDIA A100-SXM4-40GB (`a100`) | \$2.09 per GPU-hour | 11 vCPU / 70 GiB | \$2.292 per hour |
| Tesla T4 (`t4`) | \$0.49 per GPU-hour | 7 vCPU / 24 GiB | \$0.572 per hour |

* **Four GPU types, 1 to 4 GPUs per machine.** `t4`, `l4`, and `a100` machines take 1, 2, or 4
  GPUs, and an `h100` machine takes one. Each GPU brings its own vCPUs and memory.
* **A machine that is ready to use.** CUDA, PyTorch, the Hugging Face `hf` CLI, and `uv` are
  installed, and `nvidia-smi` shows your GPU as soon as the machine runs.
* **A 200 GiB disk for models and data.** Your home, `/data/home`, is on it, so a model that you
  download once is still there after a pause or a stop.
* **Pause and resume with the model loaded.** A pause keeps the machine's processes and the GPU's
  memory, and a paused machine does not pay for its GPU. An `h100` machine cannot be paused.
* **An HTTPS URL for a model server.** Publish a port to call your model from anywhere, or keep the
  URL private behind a token.
* **Available on every paid plan**, with up to 20 running GPUs per workspace.

## Start a GPU machine

Name the GPU type when you create the machine. InstaBox picks the template and the size for that
type.

```bash theme={null}
instabox create --name gpu-1 --gpu l4
```

When the command returns, the machine is running. Check that it sees its GPU.

```bash theme={null}
instabox exec gpu-1 -- nvidia-smi
```

For more than one GPU, add a count after the type, such as `--gpu l4:2`.

## Workloads

Each guide starts from an empty machine and ends with a working result on one `l4` GPU, with the
output of a real run.

> [card] [Serve an open model](https://instabox.sh/docs/use-cases/gpu/run-a-model.md)
>
> Serve an open LLM with vLLM and call it with any OpenAI client.

> [card] [Train a model](https://instabox.sh/docs/use-cases/gpu/train-a-model.md)
>
> Train as a background job, pause without losing the run, and continue from a checkpoint.

> [card] [Generate images](https://instabox.sh/docs/use-cases/gpu/generate-images.md)
>
> Turn prompts into pictures with an open image model.

> [card] [Transcribe audio and describe video](https://instabox.sh/docs/use-cases/gpu/video-inference.md)
>
> Run Whisper and a vision model over your own files.

> [card] [Run batch jobs](https://instabox.sh/docs/use-cases/gpu/batch-jobs.md)
>
> Run a model over a large file of prompts, or split a sweep across many GPUs.

## Pick a GPU

The GPU's own memory decides which models fit.

| GPU | GPU memory | Good for |
| - | - | - |
| `t4` | 16 GiB | Whisper, image models, and LLMs of up to about 4B parameters |
| `l4` | 24 GiB | An 8B-parameter LLM in 16-bit precision, or two small models side by side |
| `a100` | 40 GiB | Larger models, longer contexts, and faster training |
| `h100` | 80 GiB | The largest models that fit one GPU |

When a model does not fit one GPU, give the machine more GPUs and split the model across them.

## Cost

GPUs are billed per GPU, by the second, only while the machine runs. For example, one
NVIDIA L4 costs \$0.765 an hour all in, with its vCPUs and memory. A paused or
stopped machine holds no GPU and pays only for its disk. See [Pricing](https://instabox.sh/docs/account/pricing.md#gpu).

> [!NOTE]
>
> A machine pauses after 30 idle minutes by default. Its processes and its loaded
> model are kept, and a request to a public port wakes it. To keep a machine running, set a longer
> idle time or turn the idle pause off. See [Auto-pause](https://instabox.sh/docs/machines/auto-pause.md).

## Limits

* When every GPU of a type is in use, `create` exits 75 with `no_capacity`. Try again later, or pick
  another GPU type.
* Docker does not run in a GPU machine. Install a model server with `pip` or `uv`, as the guides do.
* A GPU machine cannot be forked. To copy one, save a snapshot and create machines from it.
* [What works differently](https://instabox.sh/docs/machines/gpu.md#what-works-differently) lists the other differences from a
  machine without a GPU.

> [card] [GPU machines](https://instabox.sh/docs/machines/gpu.md)
>
> GPU types and counts, the template, and model weights in full.

> [card] [Pricing](https://instabox.sh/docs/account/pricing.md#gpu)
>
> The price of each GPU type.
