Skip to main content
A GPU machine is a Linux machine on a host with NVIDIA GPUs, so you can run CUDA and PyTorch work, such as model inference, training, or a simulation, next to your code. To create one, name the GPU type:
In the dashboard, open Create Machine, pick GPU (Linux), then the GPU type and count. GPU machines are available on every paid plan, with up to GPUs per workspace. The free plan cannot create them.

GPU types

Each GPU brings a fixed share of vCPUs and memory, so the machine’s size comes from the GPU type and count, and you cannot set the vCPUs or memory yourself. To give a machine more than one GPU, add the count after the type. Counts are powers of two, so a host’s GPUs always split evenly between machines: l4, a100, and t4 machines take 1, 2, or 4 GPUs, and an h100 machine takes one.
GPUs are billed per GPU, by the second, only while the machine runs. A paused or stopped GPU machine holds no GPU, so it does not count toward your workspace’s limit (Pricing). When every GPU of a type is in use, a create or start fails with no_capacity. Add --queue to wait in line for a GPU instead: the machine starts when one frees up (Wait for a GPU).

What is preinstalled

InstaBox picks the template from the GPU type: gpu-metal for h100, and gpu for the other types. Both have the CUDA 13.0 toolkit (nvcc), PyTorch 2.8 with CUDA in /opt/venv, and instabox-gpu-smoke, which checks that the GPU works. On gpu, the NVIDIA driver and nvidia-smi come from the host. An h100 machine gets the whole GPU, so gpu-metal brings its own driver. Check the GPU once the machine runs:

Model weights

A GPU machine’s /data disk is 200 GiB at create, whatever its GPU count, and your home is /data/home, so a model you download lands on the disk and is still there after a stop, a start, a pause, or a snapshot. Keep weights out of /: the root disk is the template’s 19 GiB, with little room left. The hf CLI is preinstalled, with Hugging Face’s Xet transfers in high-performance mode (HF_XET_HIGH_PERFORMANCE=1), so a download uses all the machine’s bandwidth:
It goes to ~/.cache/huggingface, where transformers, vLLM, and diffusers find it on the next run without another download. Pass --local-dir /data/models/qwen3-8b for a plain directory instead, for llama.cpp or a tool that wants a path. When a model needs more room, grow the disk while the machine runs, up to your plan’s cap ():
To download a model once for your whole workspace, save a snapshot after the download and create the next machines from it. Check that the model fits the machine before you start: a weight file larger than the GPUs’ memory is run from the machine’s memory and disk, which is much slower, and a model that does not fit the machine’s memory either needs more GPUs (--gpu a100:4) or a smaller quantization.

Start GPU machines from a snapshot

Install your models and libraries once, save a snapshot, and start new GPU machines from it. A machine created from a GPU machine’s snapshot keeps its GPU type. It gets the snapshot’s GPU count unless you ask for another:

What works differently

A GPU machine is a Linux machine, so the pages under Configuration and Access apply to it, except for these:
  • Fork: a GPU machine cannot be forked. The fork is refused with fork_unavailable. To copy one, save a snapshot and create new machines from it, as above.
  • Snapshots: a running GPU machine cannot be saved live. Stop it first, or add --restart to instabox snapshot save (Snapshots). A GPU machine’s snapshot always keeps /data, so --root-only is refused.
  • Size: the vCPUs and memory come from the GPU type and count, so a resize of the vCPUs or memory is refused. You can still grow the disk (instabox resize train --disk 200). For more GPUs, save a snapshot and create from it with another count (--gpu l4:4).
  • Pause on h100: an h100 machine cannot be paused, because its GPU memory cannot be saved. Idle time and the max run time stop it instead, which keeps its files but not its running processes (Auto-pause and max-run).
A GPU machine runs in a sandbox whose /proc/meminfo differs from a Linux kernel’s in one way that matters for large models: file pages you map or read (model weights opened with mmap, for example) count as used memory, and MemAvailable equals MemFree. So once the files you map are larger than the machine’s memory, MemFree and MemAvailable read 0 and Cached can be larger than MemTotal, even though most of that memory is file cache that can be reclaimed. Tools that size buffers from “available” memory (psutil’s virtual_memory().available, and the CPU offload or pinned-memory budgets of some inference servers) then see none.MemTotal is the machine’s memory, and AnonPages is the memory programs hold (shared memory and tmpfs included). To estimate what is free, treat file cache as reclaimable:
Set such budgets from that estimate, or explicitly. A model whose weights do not fit in the machine’s memory is read from disk again and again while it runs. Give it more GPUs, and so more memory: --gpu l4:2 has twice the memory of --gpu l4.

GPU workloads

Serve a model, train a model, generate images, and more, step by step.

Wait for a GPU

Queue a create or start when every GPU of a type is in use.