Technical Deep Dive

AI Video Production Backend: Claude, FFmpeg & Hormozi Subtitles

The engineering behind a headless video factory — prompt architecture, hybrid visual strategy, server-side FFmpeg assembly, and word-level subtitle sync that took production from $60 to $0.68.

Claude 3.5

FFmpeg

ElevenLabs

Kling AI

TL;DR

Three engineering systems that make autonomous video possible

This is the backend architecture behind our AI video pipeline — the parts that can’t be replicated with no-code tools. Three systems working together: Claude 3.5 Sonnet as Creative Director (Dual-Hook Logic + Seamless Loop grammar), an Asset Tiering System that maximizes ROI by mixing Kling AI animation with Flux static images, and a custom Node.js + FFmpeg engine that synthesizes sound effects on-the-fly, generates Hormozi-style word-level subtitles, and mixes 4 independent audio layers — all server-side, no timeline, no editor.

Prompt Architecture

1. Semantic Architecture: Claude 3.5 Sonnet as Creative Director

We don’t treat LLMs as simple chat interfaces. In this pipeline, Claude 3.5 Sonnet enforces two high-level psychological algorithms that make every video designed to retain viewers:

Dual-Hook Logic

The system generates two independent hooks: a Visual Hook (on-screen text) and an Audio Hook (narrator voice), engineered to complement each other and double scroll-stop probability.

Seamless Loop Grammar

The AI analyzes syntax to ensure the final phrase acts as a grammatical prefix to the first word. This creates an “infinity loop” that spikes retention rates — the video never feels like it ends.

Claude also performs semantic splitting of long-form articles into 5 viral angles (Discovery, Extreme, Dark Side, Science, Controversy), each with a mathematically optimized 22-second duration.

Cost Optimization

2. Hybrid Visual Strategy: Maximizing ROI

Generating 100% AI video is cost-prohibitive. To solve this, we implemented an Asset Tiering System that allocates rendering budget based on visual impact:

Tier 1 — Hero Scenes

Kling 2.5 Turbo — Cinematic Animation

Hook scene and finale are animated using Kling 2.5 Turbo for fluid cinematic motion. These are the highest-impact moments where visual quality directly affects watch time. Budget allocated here.

Tier 2 — Context Scenes

Flux Schnell + Programmatic Motion ($0.003/image)

Context scenes are generated via Flux Schnell at $0.003/image, then animated programmatically via Ken Burns paths in FFmpeg (zoompan filter). Visually rich, fraction of the cost of video generation.

This hybrid approach brings cost per video to $0.68 while maintaining broadcast-quality visual variety throughout.

Server-Side Assembly

3. Headless Production: The FFmpeg Engine

The “impossible” part of the pipeline. Instead of Adobe Premiere, our n8n workflow executes native FFmpeg commands server-side. Three capabilities that no standard no-code tool can replicate:

• 1Automatic Sound Design — Synthesizes sound effects (Whooshes, Booms, Impacts) on-the-fly using FFmpeg lavfi filters. No sound library, no manual sync, zero cost.

• 2Word-Level Subtitle Sync — ElevenLabs returns per-character timestamps. We generate custom .ass subtitle files with active word highlighting at precise millisecond alignment — the “Hormozi style” effect.

• 34-Layer Audio Mix — Voice, background music, sound FX layer 1, sound FX layer 2 mixed into a single output via amix filter with independent volume control per layer.

Ken Burns motion on static image

ffmpeg -loop 1 -i scene2.jpg -vf "zoompan=z='pzoom+0.0015':d=72" -t 3 context_clip.mp4

4-layer audio mix

ffmpeg -i voice.mp3 -i music.mp3 -i sfx1.wav -i sfx2.wav

-filter_complex "amix=inputs=4:weights=1 0.3 0.6 0.6" final.mp4

Production Metrics

Results

7 min

Render Time

Was 3–5 hours of manual editing

$0.68

Cost per Short

Was $25–$60 with professional editor

4

Audio Layers Mixed

Voice + music + 2x SFX, all server-side

∞

Scalability

No human bottleneck in production

Tech Stack

AI

Claude 3.5 SonnetCreative Director: Dual-Hook Logic, Seamless Loop grammar, semantic angle splitting, visual prompt generation

K

Kling 2.5 TurboCinematic video generation for hero scenes — hook and finale frames where visual quality matters most

EL

ElevenLabs Multilingual v2Voice synthesis with per-character timestamps — enables word-level subtitle highlighting

FF

FFmpeg (Node.js)Headless video assembly: zoompan motion, lavfi SFX synthesis, .ass subtitle burn, amix audio, MP4 output

Fx

Flux Schnell$0.003/image for context scenes — animated programmatically to match cinematic look at minimal cost

n8n

n8n (self-hosted)Orchestration of the full pipeline — triggers, API calls, FFmpeg execution, Telegram approval, YouTube publish

Stop editing. Start orchestrating.

This content factory is one example of how we convert manual chaos into engineered precision. We build and deploy custom video pipelines for any niche — ready in 2–3 weeks.

Get a Free Assessment →