l4 GPU: it transcribes speech
with Whisper, and it describes a video with
Qwen2.5-VL.
Transcribe speech
Whisper turns speech into text, with a timestamp for each part.1
Create a GPU machine
2
Install ffmpeg and Transformers
Install Then install Hugging Face Transformers in
ffmpeg with apt. This takes about a minute./opt/venv.3
Copy the script
Save this script as Copy it to the machine.
transcribe.py on your computer. It loads Whisper on the GPU, transcribes
the file that you name, and prints each part with its start time.transcribe.py
4
Transcribe a recording
Download a sample recording, a part of a speech by Martin Luther King Jr.Run the script on it. The first run downloads Whisper.The output is the speech as text, one line for each part, with its start time in seconds. This
13-second recording is one part.The first run took 28 seconds with the download, and the second run took 11 seconds.
5
Transcribe the sound of a video
To transcribe a video of your own, copy it to the machine.Take the sound out of the video as a WAV file first, because Whisper cannot read every MP4
directly.Then transcribe the WAV file.
Describe a video
A vision-language model answers a question about a video. vLLM serves the model with the OpenAI API, so you send the video in a chat request. These steps use the same machine.1
Install vLLM
Install vLLM in a virtual environment of its own under Follow the job’s output until it ends, with the job ID that the command printed.
/data/home. The install takes about a
minute, so run it as a background job.2
Put a video on the machine
This guide uses a 10-second clip of a jellyfish. Download it to the machine.
3
Start the vision model
Start vLLM with the model as a background job. Wait until the server answers on port 8000. The first start takes 3 to 4 minutes.
--allowed-local-media-path lets a request name
a file under /data/home, and --gpu-memory-utilization 0.6 leaves GPU memory free for Whisper.4
Ask about the video
Save this request as Send the request to the server with The answer comes back in a few seconds.To ask about another file, change the
describe.json on your computer. It sends the video as a video_url part,
next to the question.describe.json
--stdin, and print the answer with jq.url to its path under /data/home.What it uses
Both models fit onel4 at the same time: together they used 14.4 GB of its 24 GiB at their peak.
For a large library, split the files across several GPU machines, as in
Run batch jobs.
Stop the machine
Stop the machine when you are done. The models,ffmpeg, and your files stay on its disk.
instabox start video, start the vision model again with the same vllm serve command.
Serve an open model
Call a model on the machine from your computer through a private port.
Run batch jobs
Split a large job across many GPU machines.