Computer use

At the end of this page, a model drives a browser on a Linux desktop in a machine, and you watch the same screen live in the dashboard or through a link you share. An agent that uses a computer needs a screen to look at and a mouse and keyboard to drive. Every agent-browser machine (the default template) has a Linux desktop with a browser. Your code takes screenshots and sends clicks and keys.

Install the SDK

npm install @insforge/instabox-sdk
export INSTABOX_API_KEY=ibk_...

Keep the key on your server, never in a browser.

Start the desktop

import { Sandbox } from '@insforge/instabox-sdk'

const sbx = await Sandbox.create('agent-browser')
await sbx.desktop.start({ width: 1280, height: 800 })   // safe to call again
await sbx.desktop.open('https://example.com')           // opens the browser

The machine’s browser now shows https://example.com on the desktop.

Let the model drive

for (let step = 0; step < 20; step++) {
  const png = await sbx.desktop.screenshot()           // PNG bytes for a vision model
  const action = await yourModel.nextAction(png)      // { type: 'click', x, y } | { type: 'type', text } | { type: 'done' }
  if (action.type === 'done') break
  if (action.type === 'click') await sbx.desktop.click(action.x, action.y)
  if (action.type === 'type') await sbx.desktop.type(action.text)
}

yourModel.nextAction stands for your own call to a vision model. Each pass sends it a screenshot and carries out the action it returns, until it answers done or 20 steps have run.The desktop also takes press('Enter') or a chord such as press(['ctrl', 'l']), scroll({ direction: 'down', amount: 5 }), move(x, y), drag([x1, y1], [x2, y2]), and click(x, y, { button: 'right' }) or { double: true }. cursorPosition() and status() read its state.

Watch the desktop

In the dashboard, open the machine and choose the Computer tab. You see the desktop as the model drives it. Click the desktop to take control. Control goes back to the agent after a minute without input, or 30 seconds after you close the tab.

Share a link

const { url, expiresAt } = await sbx.desktop.streamUrl({
  viewOnly: true,      // the default: watch, no typing or clicking
  ttlSeconds: 3600,    // 60 to 86400; one hour by default
})

<iframe src="LINK_URL" width="1280" height="800" style="border:0"></iframe>

streamUrl returns the link as url, and when it expires as expiresAt. Put url in place of LINK_URL. The link opens a full-screen viewer of this machine’s desktop with no sign-in. It is not an API key. sbx.desktop.revokeStreamUrls() ends every link and closes open viewers within a few seconds.

Links belong to the machine’s creator. Only the person who created the machine, signed in or with their personal key, can make or revoke them. A workspace API key gets 403 permission_denied. It opens the desktop over GET /v1/machines/{id}/vnc instead (below).

Desktops for your own users

Make a fresh link from your server each time a user opens their desktop. Revoke the links when their access ends:

app.get('/api/desktop-link', async (req, res) => {
  const machineId = await machineOf(req.user)       // your own user-to-machine mapping
  const sbx = await Sandbox.connect(machineId)
  res.json(await sbx.desktop.streamUrl({ viewOnly: !req.user.canControl, ttlSeconds: 600 }))
})

This server runs on the personal key of the account that creates the machines. Tag machines with metadata: { customer: 'acme' } at create to find them later with Sandbox.list({ metadata }).

On a workspace key: relay VNC

Your server opens GET /v1/machines/{id}/vnc with any key that has machines:write. The endpoint is a WebSocket carrying VNC, for a viewer such as noVNC. Your server passes the stream to your user’s viewer, or sends them sbx.desktop.screenshot() images.Each user has at most 8 open terminals and desktops at once, and a workspace key counts as its workspace’s owner. A ninth connection gets a retryable 429 quota_exceeded, so close a viewer’s connection when they leave.

Over HTTP

H="X-API-Key: $INSTABOX_API_KEY"
curl -s -H "$H" -H 'Content-Type: application/json' \
  -d '{"action": "start", "width": 1280, "height": 800}' "$INSTABOX_API_URL/v1/machines/m_.../desktop"
curl -s -H "$H" -H 'Content-Type: application/json' \
  -d '{"viewOnly": true, "ttlSeconds": 3600}' "$INSTABOX_API_URL/v1/machines/m_.../desktop/stream"
curl -s -X DELETE -H "$H" "$INSTABOX_API_URL/v1/machines/m_.../desktop/stream"   # revoke

Desktop

Every desktop call, viewers, and link rules.

SDK reference

Sandbox and sbx.desktop in full.

Get started

Use cases

Machines

Access

Security

Account

Reference

Desktop

SDK reference