watch-skill

# Watch Skill **Give every AI agent eyes for video — and a way to check its own work.** [](https://github.com/oxbshw/watch-skill/actions/workflows/ci.yml) [](https://github.com/oxbshw/watch-skill/actions/workflows/install.yml) [](https://pypi.org/project/watch-skill/) [](https://pypi.org/project/watch-skill/) [](pyproject.toml) [](LICENSE) [Install](#install) · [Documentation](docs/README.md) · [Examples](examples/README.md) · [Comparison](docs/comparison.md) · [Roadmap](docs/ROADMAP.md)

Watch Skill turns videos, live streams, meetings, and screen recordings into a searchable, timestamped index. An agent can ask what happened, get an answer that cites the exact moment behind it, and ask again tomorrow without processing the video a second time.

When the video is the agent's own browser or desktop session, THE LOOP closes the circle: record the work, judge it against plain-language criteria, and prove the fix.

uvx --from "watch-skill[standard]" watch-skill setup

_{THE LOOP catching a $NaN total that an end-state screenshot misses, then proving the fix.}

What it does

Available as Claude Code skills, 23 MCP tools, a CLI, a REST API, and native adapters for LangChain/LangGraph, CrewAI, the OpenAI Agents SDK, LlamaIndex, and AutoGen.

Install

Two pieces, and you want both. The engine does the work; the skills teach your agent when to reach for it.

# 1. the engine — installs, wires up every AI agent on the machine, backs up each config
uvx --from "watch-skill[standard]" watch-skill setup

# 2. the skills — into Claude Code, Codex, Cursor, Copilot, Gemini CLI, and 20+ more
npx skills add oxbshw/watch-skill -g

Neither needs a clone, and the engine command works the same on macOS, Linux, and Windows — CI runs it on all three on every push.

Prefer a permanent install to uvx fetching on demand?

pipx install "watch-skill[standard]"     # or: pip install "watch-skill[standard]"
watch-skill setup

/plugin marketplace add oxbshw/watch-skill
/plugin install watch-skill@watch-skill
/watch-skill:setup-watch-skill

docker run --rm -i -v watch-skill-data:/data ghcr.io/oxbshw/watch-skill serve

curl -fsSL https://raw.githubusercontent.com/oxbshw/watch-skill/main/scripts/install.sh | sh

powershell -ExecutionPolicy Bypass -c "irm https://raw.githubusercontent.com/oxbshw/watch-skill/main/scripts/install.ps1 | iex"

{ "mcpServers": { "watch-skill": {
    "command": "uvx",
    "args": ["--from", "watch-skill[standard]", "watch-skill", "serve"] } } }

standard is frames, retrieval, and MCP — about 200 MB. watch-skill[all] adds OCR, local Whisper, REST, and the browser THE LOOP drives. watch-skill doctor names anything missing and prints the one command that installs it, so starting small is safe.

Coming from claude-video? Your /watch commands and flags work unchanged — see the migration guide.

First run

watch-skill watch "https://youtu.be/..." "Summarize the important moments."

That prints a report and an id. Everything after it is a lookup against the index, not a second download:

watch-skill ask <video_id> "when does the demo first fail?"
watch-skill search "pricing decision"        # across every video you've watched
watch-skill library ask "what did the team decide about auth?"

Useful flags on watch:

And the rest of the surface:

watch-skill serve                            # MCP over stdio — what agents connect to
watch-skill api                              # REST, port 8748
watch-skill doctor                           # check and repair the setup
watch-skill viewer <video_id> --out r.html   # one self-contained page to share
watch-skill loop viewer <loop_id>            # a run's iterations, compared
watch-skill bench providers                  # compare every provider you have a key for

Transcription, OCR, and search run locally and need no API key. Visual question answering uses whichever provider you already pay for — Anthropic, OpenAI, Gemini, OpenRouter, Groq, Together, Fireworks, DeepSeek, xAI, Mistral, MiniMax, Moonshot, Z.ai, or Qwen — or nothing at all with a local Ollama model. Anything else that speaks the OpenAI format (vLLM, LM Studio, llama.cpp, LiteLLM, Azure OpenAI, a company gateway) works through the custom provider:

watch-skill setup-vision --provider groq            # or any of the above
watch-skill setup-vision --provider custom \
  --base-url http://127.0.0.1:8000/v1               # your own server

See Getting started for manual installation and Configuration for provider and privacy settings.

Why use it

• Evidence instead of frame dumps. Scene detection and perceptual deduplication spend the frame budget on distinct moments. Answers include timestamps, confidence, and the evidence used to support them.
• Persistent video memory. Analyze once, ask again without downloading or transcribing the same video. Hybrid full-text and vector retrieval works within one video or across the entire library.
• Local-first processing. Original-language captions are preferred, local Whisper is the default fallback, and cloud speech-to-text is opt-in. An Ollama configuration keeps the complete pipeline on the machine.
• Flow verification. THE LOOP records an agent's browser, screen, or window; checks the result against plain-language criteria; and produces before/after proof after a fix.
• Corrections that persist. report_mistake stores a local lesson, applies it to related questions, and turns it into a replayable evaluation.
• Measured cost controls. Text-first answers, semantic caching, configurable token budgets, and explicit cheapest, quality_first, and offline_only policies keep the trade-offs visible.
• Multilingual retrieval. Script-aware OCR routing, Arabic normalization, CJK substring matching, and multilingual embeddings support questions across languages.

Sixteen providers is a menu, not an answer, so there is a benchmark that settles it on your own keys: watch-skill bench providers reads the same committed frames with every provider you have configured and prints char-hit rate, latency, and cost from each one's reported tokens — see method and results.

The repository includes reproducible cost and perception benchmarks. Product claims in this README link to the relevant implementation notes or testable example rather than relying on unqualified marketing numbers.

Works with your agent

The setup command detects supported clients and updates their configuration with a backup. Manual guides are available for every entry below.

Claude Code	Claude Desktop	Cursor	Codex CLI
Cline	Windsurf	Gemini CLI	VS Code
GitHub Copilot CLI	Kimi Code	Qwen Code	OpenCode
Goose	OpenHands	Kilo Code	Qodo
Agent Zero	OpenClaw	Pi	Hermes
Zed	Roo Code	Continue	JetBrains
Amp	Aider

Native tools are also available for LangChain/LangGraph, CrewAI, OpenAI Agents SDK, LlamaIndex, and AutoGen; any other framework can use REST or MCP.

Why both skills and MCP

MCP gives an agent 23 tools. Skills give it the judgement about when to use them — that a screen recording in the conversation is worth watching, that a follow-up question should hit the index instead of re-processing, that a UI change deserves a verification pass. An agent with only the tools waits to be told; an agent with the skills reaches for them.

That is why npx skills add oxbshw/watch-skill -g is step two of the install and not an optional extra. Pick individual ones with --skill <name>, or list them first:

npx skills add oxbshw/watch-skill --list

Skills and MCP complement each other: skills carry the judgement about when video is worth watching, MCP carries the 23 tools. Installing both is the full setup, and watch-skill setup does it.

The full compatibility matrix separates machine-tested, machine-configured, and documentation-verified integrations. If your agent is missing, the adapter template provides a short contribution path.

Common workflows

Build a searchable video library

watch-skill batch ./recordings --limit 50
watch-skill library overview
watch-skill library ask "What did the team decide about authentication?"

library ask synthesizes evidence across videos and retains per-video timestamp provenance. The library example demonstrates a question whose answer is distributed across four clips.

Verify an agent's browser work

watch-skill loop start \
  --source "browser:http://127.0.0.1:3000" \
  --criteria "Checkout completes and the total is always a valid currency amount"

The loop captures the full interaction, critiques failures, and records proof after the agent applies a fix — the run shown at the top of this page. Example 14 walks through that transient $NaN bug.

Export an offline report

watch-skill viewer <video_id> --out video-report.html

The generated page contains its frames, transcript, OCR, cached answers, and cited evidence. It has no external runtime dependencies and can be opened without a server.

Examples

The examples progress from a first watch to agent integration, cross-video memory, and self-verification.

See the example catalog for prerequisites, expected output, and a recommended path through all 16 examples.

Architecture

All interfaces call the same Python core. Skills and agent adapters decide when to use Watch Skill; acquisition, perception, transcription, indexing, answering, and verification remain in src/watch_skill.

flowchart LR
    A["Agents and frameworks"] --> S["Skills · MCP · CLI · REST"]
    S --> AC["Acquire"]
    AC --> P["Scenes · OCR · transcript"]
    P --> I[("Persistent index")]
    I --> Q["Answers · extraction · library"]
    I --> L["Lessons and evaluations"]
    V["Browser · screen · stream capture"] --> C["Loop critic"]
    C --> I

Read Architecture for the data model, provider boundaries, and extension points.

Documentation

Development

git clone https://github.com/oxbshw/watch-skill
cd watch-skill
uv sync --extra all
uv run pytest
uv run ruff check .

See CONTRIBUTING.md for test tiers, documentation standards, and the agent-adapter checklist. Security and privacy reports are covered by SECURITY.md.

Listed on

Independent directories that index Watch Skill. They are maintained by their operators, so the details there can lag a release.

• Agent Skills Hub
• Neuralbox