no_capacity. Add --queue and the request waits in line for a GPU of that type instead, so you do
not have to retry by hand. The machine starts as soon as a GPU frees up, and you are not billed while
it waits.
--queue. While others wait, it joins the line. Ctrl-C stops the CLI from waiting but leaves the
request in the line.
Queue a create or a start
You can queue a create from a template, a create from a GPU snapshot, and a start of a stopped GPU machine. A stopped GPU machine needs a free GPU to start, so a start is where the queue helps most: your disks are already there, and the machine runs as soon as a GPU frees up.--queue-timeout implies --queue:
--no-wait. Then wait for the machine later:
instabox wait prints the machine’s position as it changes and returns once the machine runs.
In the API, add a queue object to the create or start request. maxWaitSeconds is to ,
and an empty object waits seconds:
queue works only on a request for a GPU machine, and not together with failFast (--fail-fast
in the CLI), so either one answers 400.
See where you are in the line
While a machine waits, its state isstopped and it has a queue object. instabox ls shows it as
Stopped (queued 3), where 3 is its position. Once it has a GPU and is starting, ls shows
Stopped (queue: starting), and Stopped (queue: cancelling) after you cancel it then. GET /v1/machines/{id} returns the same queue object:
state is queued while the request waits, and dispatched once it has a GPU and the machine is
starting. When the machine runs, or the request ends any other way, the machine has no queue
object any more.
To list your workspace’s requests, run:
--state to list requests that ended (placed, expired, cancelled, or failed), or
--state all. For ended requests the table shows the STATE, the REASON each one ended, and how
long it WAITED. The API is GET /v1/gpu-queue, newest first, which takes state too.
Cancel a request
- A start, or a create from a snapshot: the machine stays
stoppedwith its disks, so you can start it later. - A create from a template: the machine is deleted, because it never had any data.
cancelling: the start runs
to its end, and then the machine is stopped, or deleted for a create from a template. The seconds
it ran are billed as any run is.
The API is DELETE /v1/gpu-queue/{id}.
How the line is ordered
Requests for each GPU type form one line, and each type’s line is separate. Workspaces that hold fewer GPUs go first, counting the GPUs of every type their running machines hold, so a workspace that has nothing running is not stuck behind one that already runs many machines. Within a workspace, the oldest request goes first. Your position can therefore move both ways: up as requests ahead of you start, and down when another workspace that holds fewer GPUs joins the line. A request for several GPUs is not starved by smaller requests. A smaller request can take a free GPU that the request at the head of the line cannot use yet. But once the head has waited , smaller requests of that type stop going ahead of it, and freed GPUs are kept for it until a host has enough. While a GPU type has a line, a create or start of that type without--queue fails at once with
no_capacity, and the message tells you to add --queue. This keeps requests that retry from
taking the GPUs the line is waiting for.
Billing and limits
Nothing is billed while a request waits: the machine holds no host and no GPU. Billing starts when the machine runs, as for any start. A queued start, or a create from a snapshot, keeps its disks exactly as a stopped machine does. A create’s lifetime limits, such as--delete-after, do not count while it waits, so a machine is
not deleted while it is in the line. They start when the machine first runs. A create from a
snapshot that never runs, because its wait ran out or you cancelled it, stays stopped, and its
limits start counting when the request ends. A start keeps the deadlines the machine already had.
Who can queue is who can create the GPU type: GPU machines are on every paid plan.
A workspace can have requests waiting at once, and the GPUs it waits for plus the GPUs it runs can
be at most twice its GPU limit. A request over either limit fails with 429 queue_limit, and the CLI exits 5. Your
workspace’s GPU limit and plan limits still apply when the machine starts. When the machine would
pass them then, the request keeps its place and waits until you stop another GPU machine or its
wait runs out.
How a request ends
A request that already has its GPU when its wait runs out still starts.
Webhooks
Webhooks
A webhook receives
sandbox.queued when a request joins the line and
sandbox.queue_ended when it expires, is cancelled, or fails, with the reason. A request
that starts sends the usual sandbox.started.What the queue does not cover
What the queue does not cover
Only creates and starts of GPU machines wait in the line. A restore or a resize of a GPU
machine, and CPU machines, still fail with
no_capacity when there is no room. A request waits
for one GPU type: it does not fall back to another type.GPU machines
GPU types, what is preinstalled, and what works differently.
Pricing
The price of each GPU type.