Skip to main content
When every GPU of a type is in use, a create or start of a GPU machine fails with no_capacity. Add --queue and the request waits in line for a GPU of that type instead, so you do not have to retry by hand. The machine starts as soon as a GPU frees up, and you are not billed while it waits.
The CLI shows the machine’s place in the line and waits until it runs:
If a GPU is free and nobody is waiting for that type, the create runs at once, as it does without --queue. While others wait, it joins the line. Ctrl-C stops the CLI from waiting but leaves the request in the line.

Queue a create or a start

You can queue a create from a template, a create from a GPU snapshot, and a start of a stopped GPU machine. A stopped GPU machine needs a free GPU to start, so a start is where the queue helps most: your disks are already there, and the machine runs as soon as a GPU frees up.
A request waits up to by default. Set a longer wait for a training job that can start later, up to , or a shorter one, down to , for an agent that needs an answer soon. --queue-timeout implies --queue:
To queue the request and return at once, add --no-wait. Then wait for the machine later:
instabox wait prints the machine’s position as it changes and returns once the machine runs. In the API, add a queue object to the create or start request. maxWaitSeconds is to , and an empty object waits seconds:
queue works only on a request for a GPU machine, and not together with failFast (--fail-fast in the CLI), so either one answers 400.

See where you are in the line

While a machine waits, its state is stopped and it has a queue object. instabox ls shows it as Stopped (queued 3), where 3 is its position. Once it has a GPU and is starting, ls shows Stopped (queue: starting), and Stopped (queue: cancelling) after you cancel it then. GET /v1/machines/{id} returns the same queue object:
state is queued while the request waits, and dispatched once it has a GPU and the machine is starting. When the machine runs, or the request ends any other way, the machine has no queue object any more. To list your workspace’s requests, run:
Add --state to list requests that ended (placed, expired, cancelled, or failed), or --state all. For ended requests the table shows the STATE, the REASON each one ended, and how long it WAITED. The API is GET /v1/gpu-queue, newest first, which takes state too.

Cancel a request

What happens to the machine depends on how it was queued:
  • A start, or a create from a snapshot: the machine stays stopped with its disks, so you can start it later.
  • A create from a template: the machine is deleted, because it never had any data.
If the machine already has its GPU and is starting, the cancel answers cancelling: the start runs to its end, and then the machine is stopped, or deleted for a create from a template. The seconds it ran are billed as any run is. The API is DELETE /v1/gpu-queue/{id}.
Do not cancel with instabox rm. It deletes the machine and its disks, which for a queued start loses the data you queued to get back to.

How the line is ordered

Requests for each GPU type form one line, and each type’s line is separate. Workspaces that hold fewer GPUs go first, counting the GPUs of every type their running machines hold, so a workspace that has nothing running is not stuck behind one that already runs many machines. Within a workspace, the oldest request goes first. Your position can therefore move both ways: up as requests ahead of you start, and down when another workspace that holds fewer GPUs joins the line. A request for several GPUs is not starved by smaller requests. A smaller request can take a free GPU that the request at the head of the line cannot use yet. But once the head has waited , smaller requests of that type stop going ahead of it, and freed GPUs are kept for it until a host has enough. While a GPU type has a line, a create or start of that type without --queue fails at once with no_capacity, and the message tells you to add --queue. This keeps requests that retry from taking the GPUs the line is waiting for.

Billing and limits

Nothing is billed while a request waits: the machine holds no host and no GPU. Billing starts when the machine runs, as for any start. A queued start, or a create from a snapshot, keeps its disks exactly as a stopped machine does. A create’s lifetime limits, such as --delete-after, do not count while it waits, so a machine is not deleted while it is in the line. They start when the machine first runs. A create from a snapshot that never runs, because its wait ran out or you cancelled it, stays stopped, and its limits start counting when the request ends. A start keeps the deadlines the machine already had. Who can queue is who can create the GPU type: GPU machines are on every paid plan. A workspace can have requests waiting at once, and the GPUs it waits for plus the GPUs it runs can be at most twice its GPU limit. A request over either limit fails with 429 queue_limit, and the CLI exits 5. Your workspace’s GPU limit and plan limits still apply when the machine starts. When the machine would pass them then, the request keeps its place and waits until you stop another GPU machine or its wait runs out.

How a request ends

A request that already has its GPU when its wait runs out still starts.
A webhook receives sandbox.queued when a request joins the line and sandbox.queue_ended when it expires, is cancelled, or fails, with the reason. A request that starts sends the usual sandbox.started.
Only creates and starts of GPU machines wait in the line. A restore or a resize of a GPU machine, and CPU machines, still fail with no_capacity when there is no room. A request waits for one GPU type: it does not fall back to another type.

GPU machines

GPU types, what is preinstalled, and what works differently.

Pricing

The price of each GPU type.