The claude-video repository is a Python-based tool that extends Claude's capabilities to analyze video content by downloading, extracting frames, transcribing audio, and passing the results to Claude for analysis. The core command is `/watch`, which accepts either a URL (supporting YouTube, Loom, TikTok, X, Instagram, and hundreds of other sources via yt-dlp) or a local file path in formats like MP4, MOV, MKV, or WebM. Users pair the video with a question, and Claude responds based on what it has actually seen and heard rather than relying on titles or missing transcripts.
The tool prioritizes efficiency by checking for captions first before downloading video. When captions are available through yt-dlp, the transcript is retrieved instantly at no cost. If captions are unavailable or when audio analysis is needed, the script extracts a mono 16 kHz 64 kbps MP3 audio clip and sends it to either Groq's whisper-large-v3 or OpenAI's whisper-1 API for transcription. Frame extraction happens through ffmpeg and operates in three detail modes: efficient (keyframe-only extraction, fastest at ~0.5 seconds), balanced (scene-change detection), and token-burner (uncapped scene-change detection). The script automatically adjusts frame budgets based on video duration, ranging from approximately 30 frames for videos under 30 seconds to a capped 100 frames for videos over 10 minutes, with warnings when sparse scanning might miss important details.
Frame deduplication runs by default across all extraction modes to eliminate near-identical frames that would waste tokens. The process compares each frame against the last kept frame using mean absolute difference on 16×16 grayscale thumbnails, dropping frames where brightness change falls at or below a threshold of 2.0. This approach catches slow fades and handles static content efficiently while preserving distinct visual changes like code diffs, terminal scrolling, or color changes. The deduplication happens before the frame budget cap is applied, ensuring the budget is spent on distinct frames only.
Real-world use cases documented in the repository include analyzing content hooks and structure from viral videos, diagnosing bugs from screen recordings by identifying the exact frame where issues appear, summarizing long videos to extract key moments and substance, stripping marketing hype from product launch videos to identify actual new features, and converting video playlists into searchable note collections. The tool handles focused analysis when users specify time ranges like "around 2:30" or "from 0:45 to 1:00", automatically increasing frame density for those windows.
Installation is available across multiple platforms. Claude Code users can install via the marketplace with auto-updates. For Codex, Cursor, Copilot, Gemini CLI, and over 50 other agent hosts, the Agent Skills CLI handles installation globally or per-project. Claude.ai web users can download the skill file and add it through Settings. Manual installation involves cloning the repository and symlinking the skills directory into the host's skills folder. The tool requires zero configuration to start, with yt-dlp and ffmpeg installing automatically on first run via brew on macOS, with exact commands printed for Linux and Windows users. A Whisper API key is only necessary when videos lack captions.
