OpenShot Video Editor — Python Manipulation Guide

This skill teaches setup-gen (create project files, prepare media, launch GUI) and reward-gen (verify project structure, exported video/audio) how to work with OpenShot using Python.

• Libraries: json, subprocess, cv2, numpy, Pillow, imagehash, librosa
• Install: pip3 install opencv-python numpy Pillow imagehash librosa fastdtw scipy scikit-image
• System: sudo apt install openshot-qt python3-openshot ffmpeg (VM)
• Project format: .osp (plain JSON)
• Config path (Linux): ~/.openshot_qt/

---

0. GUI Startup on VM (for setup-gen)

After preparing the .osp project file and media assets, setup-gen should launch OpenShot with the project loaded for the GUI agent.

CRITICAL VM LIMIT: GUI launches must set DISPLAY=:0.

import os
import shlex
import subprocess
import time

def launch_gui(command: str, delay_sec: float = 1.0):
    env = os.environ.copy()
    env["DISPLAY"] = ":0"
    subprocess.Popen(
        shlex.split(command),
        stdout=subprocess.DEVNULL,
        stderr=subprocess.DEVNULL,
        env=env,
    )
    time.sleep(delay_sec)

# Launch OpenShot with a pre-built project
launch_gui('openshot-qt "/home/user/Desktop/project.osp"', delay_sec=3.0)

# Or launch OpenShot and auto-import media files (no project)
launch_gui('openshot-qt "/home/user/Desktop/video1.mp4" "/home/user/Desktop/image.png"', delay_sec=3.0)

Guidelines:

• OpenShot auto-loads .osp files passed as arguments, or imports non-.osp files as media.
• Use non-blocking launch (Popen) so script exits cleanly.
• OpenShot is heavier than VLC/GIMP — use delay_sec=3.0 or higher.
• Open initial project, never golden project.

---

---

0.5. Media Asset Library (setup-gen)

Use the real video/image library at assets/media/openshot/ instead of generating synthetic media.

Available Categories

Category	Count	Description	Good for
clips	~30	Real video clips: interview, sports, craft, abstract, aerial, dance	Timeline editing, trimming, effects, transitions
overlays	~10	Photos for overlays: transparent backgrounds, light effects	Logo overlays, title cards, compositing

Audio can be extracted from clips on the VM via FFmpeg. Also share assets from assets/media/vlc/ for additional variety.

Upload Pattern

# Upload multiple clips for a multi-track editing task
python3 scripts/env_cli.py -c "<workdir>/env_config_initial.json" upload \
    "assets/media/openshot/clips/interview_talking_person_001.mp4" "/home/user/Desktop/clip_a.mp4"
python3 scripts/env_cli.py -c "<workdir>/env_config_initial.json" upload \
    "assets/media/openshot/clips/drone_aerial_landscape_016.mp4" "/home/user/Desktop/clip_b.mp4"
python3 scripts/env_cli.py -c "<workdir>/env_config_initial.json" upload \
    "assets/media/openshot/overlays/particle_light_effect_006.jpg" "/home/user/Desktop/overlay.jpg"
# Repeat for golden_env

Then initial_setup.py creates the .osp project referencing these uploaded media files.

Picking Assets from Manifest

import json, random
manifest = json.load(open("assets/media/manifest.json"))
openshot_clips = [a for a in manifest["assets"] if a["domain"] == "openshot" and a["category"] == "clips"]
# Pick clips with different themes for a multi-clip project
interview = random.choice([c for c in openshot_clips if "interview" in c.get("tags", [])])
broll = random.choice([c for c in openshot_clips if "aerial" in c.get("tags", []) or "nature" in c.get("tags", [])])

---

1. Project File (.osp) — JSON Structure (setup-gen & reward-gen)

OpenShot project files are plain JSON with .osp extension. This is the primary mechanism for both setup and verification.

Complete .osp Schema

import json
import uuid
import os

def new_id():
    return str(uuid.uuid4().hex[:10])

def create_empty_project(
    width=1920, height=1080,
    fps_num=30, fps_den=1,
    sample_rate=44100, channels=2,
    channel_layout=3,  # LAYOUT_STEREO
    profile="HD 1080p 30 fps"
):
    """Create a minimal valid .osp project."""
    return {
        "id": new_id(),
        "version": {
            "openshot-qt": "3.4.0",
            "libopenshot": "0.4.0"
        },
        "profile": profile,
        "width": width,
        "height": height,
        "fps": {"num": fps_num, "den": fps_den},
        "display_ratio": {"num": 16, "den": 9},
        "pixel_ratio": {"num": 1, "den": 1},
        "sample_rate": sample_rate,
        "channels": channels,
        "channel_layout": channel_layout,
        "files": [],
        "clips": [],
        "effects": [],
        "transitions": [],
        "markers": [],
        "history": {"undo": [], "redo": []}
    }

def save_project(project: dict, path: str):
    with open(path, "w") as f:
        json.dump(project, f, indent=2)

def load_project(path: str) -> dict:
    with open(path, "r") as f:
        return json.load(f)

Adding Files (Media References)

def add_file(project: dict, file_path: str, media_type: str = "video",
             width: int = 1920, height: int = 1080,
             has_audio: bool = True, has_video: bool = True,
             duration: float = None) -> str:
    """Register a media file in the project. Returns file ID.
    duration: media duration in seconds. If None, must be set manually or OpenShot infers it.
    """
    file_id = new_id()
    file_entry = {
        "id": file_id,
        "path": os.path.abspath(file_path),
        "media_type": media_type,  # "video", "audio", "image"
        "has_audio": has_audio,
        "has_video": has_video,
        "width": width,
        "height": height,
        "display_ratio": {"num": 16, "den": 9},
    }
    if duration is not None:
        file_entry["duration"] = duration
    project["files"].append(file_entry)
    return file_id

# Examples
file_id = add_file(project, "/home/user/Desktop/interview.mp4")
img_id = add_file(project, "/home/user/Desktop/logo.png",
                  media_type="image", width=800, height=600,
                  has_audio=False, has_video=True)
audio_id = add_file(project, "/home/user/Desktop/bgm.mp3",
                    media_type="audio", has_audio=True, has_video=False)

Keyframe System

All animatable properties use this structure:

def make_keyframe(value, frame=1, interpolation=2):
    """Create a single-point keyframe.
    interpolation: 0=Bezier, 1=Linear, 2=Constant
    """
    return {
        "Points": [{
            "co": {"X": float(frame), "Y": float(value)},
            "handle_left": {"X": float(frame) - 0.5, "Y": float(value)},
            "handle_right": {"X": float(frame) + 0.5, "Y": float(value)},
            "interpolation": interpolation
        }]
    }

def make_keyframe_animated(points):
    """Create a multi-point animated keyframe.
    points: list of (frame, value, interpolation) tuples
    """
    return {
        "Points": [{
            "co": {"X": float(f), "Y": float(v)},
            "handle_left": {"X": float(f) - 0.5, "Y": float(v)},
            "handle_right": {"X": float(f) + 0.5, "Y": float(v)},
            "interpolation": interp
        } for f, v, interp in points]
    }

# Constant value (opacity = 1.0 always)
kf_full_alpha = make_keyframe(1.0)

# Fade in: 0.0 at frame 1 → 1.0 at frame 30 (linear)
kf_fade_in = make_keyframe_animated([(1, 0.0, 1), (30, 1.0, 1)])

# Fade out: 1.0 at frame 1 → 0.0 at frame 30 (linear)
kf_fade_out = make_keyframe_animated([(1, 1.0, 1), (30, 0.0, 1)])

# Smooth zoom using Bezier interpolation
kf_zoom = make_keyframe_animated([(1, 0.5, 0), (60, 1.0, 0)])

Adding Clips to Timeline

def add_clip(project: dict, file_id: str, file_path: str,
             position: float = 0.0, start: float = 0.0, end: float = 10.0,
             layer: int = 0, volume: float = 1.0, alpha: float = 1.0,
             scale_x: float = 1.0, scale_y: float = 1.0,
             location_x: float = 0.0, location_y: float = 0.0,
             rotation: float = 0.0) -> str:
    """Add a clip to the timeline. Returns clip ID.

    position: where clip starts on timeline (seconds)
    start/end: trim points within source media (seconds)
    layer: track number (0 = bottom, higher = on top)
    """
    clip_id = new_id()
    project["clips"].append({
        "id": clip_id,
        "file_id": file_id,
        "title": os.path.basename(file_path),
        "position": position,
        "start": start,
        "end": end,
        "layer": layer,
        "reader": {"path": os.path.abspath(file_path)},
        # Keyframe properties
        "alpha": make_keyframe(alpha),
        "volume": make_keyframe(volume),
        "scale_x": make_keyframe(scale_x),
        "scale_y": make_keyframe(scale_y),
        "location_x": make_keyframe(location_x),
        "location_y": make_keyframe(location_y),
        "rotation": make_keyframe(rotation),
        "origin_x": make_keyframe(0.5),
        "origin_y": make_keyframe(0.5),
        "shear_x": make_keyframe(0.0),
        "shear_y": make_keyframe(0.0),
        "time": make_keyframe(1.0),
        "channel_filter": make_keyframe(-1),
        "channel_mapping": make_keyframe(-1),
        "has_audio": make_keyframe(-1),
        "has_video": make_keyframe(-1),
        "effects": [],
        # Scale/gravity/anchor enums
        "scale": 0,       # CROP (0=Crop, 1=BestFit, 2=Stretch, 3=None)
        "gravity": 4,     # CENTER (0-8 = TL,T,TR,L,C,R,BL,B,BR)
        "anchor": 0,
        "display": 0,
        "mixing": 0,
        "wave_color": {"red": make_keyframe(0), "green": make_keyframe(123), "blue": make_keyframe(255), "alpha": make_keyframe(255)},
    })
    return clip_id

# Example: place a 10-second clip at the start of track 0
clip1_id = add_clip(project, file_id, "/home/user/Desktop/interview.mp4",
                    position=0.0, start=0.0, end=10.0, layer=0)

# Example: overlay a logo on track 1 with 70% opacity, scaled down
logo_id = add_clip(project, img_id, "/home/user/Desktop/logo.png",
                   position=0.0, start=0.0, end=10.0, layer=1,
                   alpha=0.7, scale_x=0.2, scale_y=0.2,
                   location_x=0.35, location_y=0.35)

Adding Effects to Clips

# Available effects (name → class):
# Bars, Blur, Brightness, Caption, ChromaKey, ColorMap, ColorShift, Crop,
# Deinterlace, Hue, LensFlare, Mask, Negate, Noise, Pixelate, Saturation,
# Sharpen, Shift, SphericalProjection, Wave
# OpenCV-dependent: ObjectDetection, Outline, Stabilizer, Tracker

def add_effect_to_clip(project: dict, clip_id: str,
                       effect_name: str, effect_params: dict) -> str:
    """Add an effect to a specific clip. Returns effect ID."""
    effect_id = new_id()
    effect = {
        "id": effect_id,
        "name": effect_name,
        **effect_params
    }
    for clip in project["clips"]:
        if clip["id"] == clip_id:
            clip["effects"].append(effect)
            break
    return effect_id

# --- Effect parameter examples ---

# Blur effect
blur_params = {
    "horizontal_radius": make_keyframe(10),    # 0-100
    "vertical_radius": make_keyframe(10),      # 0-100
    "sigma": make_keyframe(3),                 # 0-100
    "iterations": make_keyframe(3),            # 1-100
}

# Brightness effect
brightness_params = {
    "brightness": make_keyframe(1.2),          # 0.0-4.0 (1.0 = no change)
}

# ChromaKey (green screen removal)
chromakey_params = {
    "color": {"red": make_keyframe(0), "green": make_keyframe(255),
              "blue": make_keyframe(0), "alpha": make_keyframe(255)},
    "fuzz": make_keyframe(25),                 # 0-100 tolerance
}

# Saturation effect
saturation_params = {
    "saturation": make_keyframe(1.5),          # 0.0-4.0 (1.0 = no change)
    "saturation_R": make_keyframe(1.0),
    "saturation_G": make_keyframe(1.0),
    "saturation_B": make_keyframe(1.0),
}

# Hue effect
hue_params = {
    "hue": make_keyframe(0.0),                 # 0-360 degrees of hue shift
}

# Crop effect
crop_params = {
    "left": make_keyframe(0.1),                # 0.0-1.0 (fraction of width)
    "right": make_keyframe(0.1),
    "top": make_keyframe(0.1),
    "bottom": make_keyframe(0.1),
}

# Negate (invert colors)
negate_params = {}  # no parameters needed

# Pixelate
pixelate_params = {
    "pixelization": make_keyframe(20),         # 0-100
    "left": make_keyframe(0.0),
    "right": make_keyframe(0.0),
    "top": make_keyframe(0.0),
    "bottom": make_keyframe(0.0),
}

# Wave
wave_params = {
    "wavelength": make_keyframe(30),           # 0-200
    "amplitude": make_keyframe(10),            # 0-100
    "multiplier": make_keyframe(0.02),
    "shift_x": make_keyframe(0),
    "speed_y": make_keyframe(0.2),
}

# ColorShift
colorshift_params = {
    "red_x": make_keyframe(5),                 # pixel shift per channel
    "red_y": make_keyframe(0),
    "green_x": make_keyframe(-5),
    "green_y": make_keyframe(0),
    "blue_x": make_keyframe(0),
    "blue_y": make_keyframe(5),
}

# Apply blur to clip1
add_effect_to_clip(project, clip1_id, "Blur", blur_params)

Adding Transitions

def add_transition(project: dict,
                   position: float, start: float = 0.0, end: float = 2.0,
                   layer: int = 0,
                   brightness_start: float = -1.0, brightness_end: float = 1.0,
                   transition_type: str = "Mask",
                   resource: str = "") -> str:
    """Add a transition between clips.

    position: where the transition starts on the timeline (seconds)
    start/end: trim points within the transition resource (seconds).
        NOTE: end is NOT the timeline end position — it is the duration of the transition
        effect relative to its own start. For a 2-second cross-dissolve, use end=2.0.
    brightness_start/end: -1.0 (fully visible) to 1.0 (fully transparent)
    resource: path to wipe/mask image (common transitions use gradients)
    """
    trans_id = new_id()
    project["transitions"].append({
        "id": trans_id,
        "title": "Transition",
        "type": transition_type,
        "position": position,
        "start": start,
        "end": end,
        "layer": layer,
        "brightness": make_keyframe_animated([
            (1, brightness_start, 1),
            (int(end * 30), brightness_end, 1)  # end frame = end * fps
        ]),
        "contrast": make_keyframe(3.0),
        "reader": {"path": resource} if resource else {},
        "replace_image": False,
    })
    return trans_id

# Common OpenShot transition wipe images are in:
# /usr/share/openshot-qt/transitions/ (or /usr/lib/python3/dist-packages/openshot_qt/transitions/)
# Examples: common/fade.svg, extra/wipe_down.svg, extra/wipe_right.svg

# Fade transition at 5 seconds (2 second duration)
add_transition(project, position=5.0, end=2.0, layer=0)

Adding Markers

def add_marker(project: dict, position: float, title: str = "Marker") -> str:
    """Add a timeline marker."""
    marker_id = new_id()
    project["markers"].append({
        "id": marker_id,
        "position": position,
        "title": title,
    })
    return marker_id

add_marker(project, 0.0, "Intro Start")
add_marker(project, 15.0, "Chapter 2")

Complete Setup Example

# Full example: create a project with 2 clips and a transition
project = create_empty_project(width=1920, height=1080, fps_num=30, fps_den=1)

# Register media
vid_id = add_file(project, "/home/user/Desktop/clip_a.mp4")
vid2_id = add_file(project, "/home/user/Desktop/clip_b.mp4")
music_id = add_file(project, "/home/user/Desktop/bgm.mp3",
                    media_type="audio", has_video=False)

# Place clips on timeline
add_clip(project, vid_id, "/home/user/Desktop/clip_a.mp4",
         position=0.0, start=0.0, end=8.0, layer=0)
add_clip(project, vid2_id, "/home/user/Desktop/clip_b.mp4",
         position=6.0, start=0.0, end=8.0, layer=0)  # 2-second overlap
add_clip(project, music_id, "/home/user/Desktop/bgm.mp3",
         position=0.0, start=0.0, end=14.0, layer=1, volume=0.3)

# Cross-dissolve transition in the overlap region
add_transition(project, position=6.0, end=2.0, layer=0)

# Add marker
add_marker(project, 6.0, "Transition Point")

# Save
save_project(project, "/home/user/Desktop/project.osp")

---

2. Creating Media Assets (setup-gen)

Generating Test Videos with FFmpeg

import subprocess

def create_test_video(output_path: str, duration: int = 10,
                      width: int = 1920, height: int = 1080,
                      fps: int = 30, pattern: str = "testsrc"):
    """Generate a test video. pattern: testsrc, testsrc2, smptebars, color=c=blue"""
    subprocess.run([
        "ffmpeg", "-y", "-f", "lavfi",
        "-i", f"{pattern}=duration={duration}:size={width}x{height}:rate={fps}",
        "-pix_fmt", "yuv420p",
        "-c:v", "libx264", "-preset", "fast",
        output_path
    ], check=True, capture_output=True)

def create_video_with_audio(output_path: str, duration: int = 10,
                            width: int = 1920, height: int = 1080, fps: int = 30):
    """Generate test video with sine wave audio."""
    subprocess.run([
        "ffmpeg", "-y",
        "-f", "lavfi", "-i", f"testsrc2=duration={duration}:size={width}x{height}:rate={fps}",
        "-f", "lavfi", "-i", f"sine=frequency=440:duration={duration}",
        "-pix_fmt", "yuv420p",
        "-c:v", "libx264", "-preset", "fast",
        "-c:a", "aac", "-b:a", "128k",
        "-shortest", output_path
    ], check=True, capture_output=True)

def create_color_video(output_path: str, color: str = "blue",
                       duration: int = 5, width: int = 1920, height: int = 1080):
    """Generate a solid color video."""
    subprocess.run([
        "ffmpeg", "-y", "-f", "lavfi",
        "-i", f"color=c={color}:duration={duration}:size={width}x{height}:rate=30",
        "-pix_fmt", "yuv420p", "-c:v", "libx264", output_path
    ], check=True, capture_output=True)

def create_text_video(output_path: str, text: str = "Hello World",
                      duration: int = 5, fontsize: int = 72):
    """Generate a video with text overlay on black background."""
    subprocess.run([
        "ffmpeg", "-y", "-f", "lavfi",
        "-i", f"color=c=black:duration={duration}:size=1920x1080:rate=30",
        "-vf", f"drawtext=text='{text}':fontsize={fontsize}:fontcolor=white:x=(w-text_w)/2:y=(h-text_h)/2",
        "-pix_fmt", "yuv420p", "-c:v", "libx264", output_path
    ], check=True, capture_output=True)

# Create test media files
create_test_video("/home/user/Desktop/clip_a.mp4", duration=10)
create_video_with_audio("/home/user/Desktop/clip_b.mp4", duration=8)
create_color_video("/home/user/Desktop/title_bg.mp4", color="darkblue", duration=3)

Generating Test Audio

def create_test_audio(output_path: str, duration: int = 10,
                      frequency: int = 440, format: str = "mp3"):
    """Generate a sine wave audio file."""
    subprocess.run([
        "ffmpeg", "-y", "-f", "lavfi",
        "-i", f"sine=frequency={frequency}:duration={duration}",
        "-c:a", "libmp3lame" if format == "mp3" else "aac",
        output_path
    ], check=True, capture_output=True)

def create_silence(output_path: str, duration: int = 10):
    """Generate a silent audio file."""
    subprocess.run([
        "ffmpeg", "-y", "-f", "lavfi",
        "-i", f"anullsrc=r=44100:cl=stereo",
        "-t", str(duration), "-c:a", "libmp3lame",
        output_path
    ], check=True, capture_output=True)

create_test_audio("/home/user/Desktop/bgm.mp3", duration=30)

Generating Test Images

from PIL import Image, ImageDraw, ImageFont

def create_test_image(output_path: str, width: int = 1920, height: int = 1080,
                      color: str = "white", text: str = None):
    img = Image.new("RGB", (width, height), color=color)
    if text:
        draw = ImageDraw.Draw(img)
        try:
            font = ImageFont.truetype("/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf", 72)
        except OSError:
            font = ImageFont.load_default()
        bbox = draw.textbbox((0, 0), text, font=font)
        tw, th = bbox[2] - bbox[0], bbox[3] - bbox[1]
        draw.text(((width - tw) / 2, (height - th) / 2), text,
                  fill="black" if color == "white" else "white", font=font)
    img.save(output_path)

create_test_image("/home/user/Desktop/logo.png", 400, 300, "yellow", "LOGO")
create_test_image("/home/user/Desktop/title_card.png", 1920, 1080, "black", "My Video")

Video Manipulation with FFmpeg

def trim_video(input_path: str, output_path: str,
               start_time: str = "00:00:00", duration: str = "00:00:10"):
    subprocess.run([
        "ffmpeg", "-y", "-i", input_path,
        "-ss", start_time, "-t", duration,
        "-c", "copy", output_path
    ], check=True, capture_output=True)

def concat_videos(input_paths: list, output_path: str):
    """Concatenate videos using FFmpeg concat demuxer."""
    list_file = "/tmp/concat_list.txt"
    with open(list_file, "w") as f:
        for p in input_paths:
            f.write(f"file '{p}'\n")
    subprocess.run([
        "ffmpeg", "-y", "-f", "concat", "-safe", "0",
        "-i", list_file, "-c", "copy", output_path
    ], check=True, capture_output=True)

def extract_frame(input_path: str, output_path: str, timestamp: str = "00:00:01"):
    subprocess.run([
        "ffmpeg", "-y", "-i", input_path,
        "-ss", timestamp, "-frames:v", "1", output_path
    ], check=True, capture_output=True)

def extract_audio(input_path: str, output_path: str):
    subprocess.run([
        "ffmpeg", "-y", "-i", input_path,
        "-vn", "-c:a", "libmp3lame", "-q:a", "2", output_path
    ], check=True, capture_output=True)

def add_audio_to_video(video_path: str, audio_path: str, output_path: str):
    subprocess.run([
        "ffmpeg", "-y", "-i", video_path, "-i", audio_path,
        "-c:v", "copy", "-c:a", "aac", "-shortest", output_path
    ], check=True, capture_output=True)

Golden File Pattern

import shutil

# For project-level golden files: save the expected .osp state
save_project(golden_project, "/home/user/Desktop/golden_project.osp")

# For export-level golden files: render expected output
# (use libopenshot or FFmpeg to produce the expected video)
shutil.copy("/home/user/Desktop/expected_output.mp4",
            "/home/user/Desktop/golden_output.mp4")

---

3. libopenshot Python API (setup-gen & reward-gen) — Quick Reference

The python3-openshot package provides SWIG bindings for headless rendering and verification. Install: sudo apt install python3-openshot (not on PyPI).

Timeline Export (Headless Render)

import openshot, json

# From scratch
t = openshot.Timeline(1920, 1080, openshot.Fraction(30, 1), 44100, 2, openshot.LAYOUT_STEREO)
clip = openshot.Clip("/home/user/Desktop/video.mp4")
clip.Position(0.0); clip.Start(0.0); clip.End(10.0); clip.Layer(0)
t.AddClip(clip)
t.Open()

w = openshot.FFmpegWriter("/home/user/Desktop/output.mp4")
w.SetVideoOptions(True, "libx264", openshot.Fraction(30, 1), 1920, 1080,
                  openshot.Fraction(1, 1), False, False, 5000000)
w.SetAudioOptions(True, "aac", 44100, 2, openshot.LAYOUT_STEREO, 192000)
w.Open()
for frame_num in range(1, t.GetMaxFrame() + 1):
    w.WriteFrame(t.GetFrame(frame_num))
w.Close(); t.Close()

# From .osp file — all media paths must be absolute and exist on disk
with open("/home/user/Desktop/project.osp") as f:
    project_json = f.read()
project = json.loads(project_json)
fps = openshot.Fraction(project["fps"]["num"], project["fps"]["den"])
t = openshot.Timeline(project["width"], project["height"], fps,
                      project["sample_rate"], project["channels"], project["channel_layout"])
t.SetJson(project_json); t.Open()
# ... export same as above