Founder case study · AI video production
Idea to Motion is a prompt-to-video production system, designed and built solo across 462 commits: AI plans the script, verifies its claims, voices it, writes real motion-graphics code and renders the video, while a human approves the two moments that matter. This page explains how it works in plain language, then shows the actual code that keeps it honest.
The product, from the creator's seat
Choose the format, voice, and visual language. The system turns that intent into a reviewable video project, not one opaque AI request.
Five reader-facing phases. One project moves forward without hiding the decisions.
The product streams a scene-based script, verifies claims, generates voice and visual direction, reveals scene previews progressively, then lets the creator edit and approve the composition before rendering.
The worker is temporary. The project state is not.
When a code-generation worker fails at stage eight, PostgreSQL retains the current stage and completed artifacts. BullMQ retries that stage, and buffered events help a reconnecting browser catch up before returning to live progress.
Every scene crosses explicit gates before it joins the video.
All generated scene code is compiled with esbuild and inspected for known hazards. MCP client scene submissions additionally render the first, middle, and final frame in an isolated worker thread with a five-second timeout.
One MCP entry in the client you already use. Your assistant supplies the reasoning; the server keeps the contracts, and every script, direction and scene it submits passes the same gates as the web app.
From one prompt — typed in the studio, or spoken to your own agent — to an editable, recoverable production workspace.
Open live productThe chapters above compress what is really eleven stages on the narrated route. These are the actual stage names from the code, in the order the state machine allows them to run.
Researches the topic and writes a scene-by-scene script, streamed to the browser token by token.
script_generationVerifies claims against live Google Search. Contradicted lines are rewritten against the evidence; if the checker itself fails, the script continues to review marked unverified.
fact_checkThe creator edits and approves the script before any money is spent producing it.
script_reviewElevenLabs or Cartesia — on the creator's own API key — or a local GPU voice sidecar with the identical contract, speaks the script and returns word-level timing.
tts_generationFacecam footage is transcribed with word timestamps; faceless videos skip the call because voice timing already exists.
transcriptionEvery script line is pinned to exact start and end times on the voice track.
timestamp_mappingTimed lines become visual direction: beats, layouts, brand assets and motion for each scene.
direction_generationAn agent writes real React/Remotion code per scene. Every file faces the validation gates before composition.
code_generationThe creator works the real composition in a scene-based editor with a timeline, tweaks it by chat or direct manipulation, and approves the render.
previewHeadless Chromium renders h264 at 30fps, at the resolution tier the account allows; audio is loudness-normalized to the -14 LUFS social standard.
renderingMP4, a midpoint-frame thumbnail and generated social metadata land in object storage.
doneThe stage enum holds eighteen values, not eleven: facecam projects insert facecam_editing and facecam_review after Voice, and uploaded-footage projects take a separate creator route through creator_ingest, creator_edit_direction, creator_composite, creator_preview and creator_render. Silent motion graphics skip voice and transcription entirely. The enum is append-only so database indices stay stable across migrations: the transition map, not the enum order, is the real graph.
Four excerpts quoted from the private repository, with paths, so the claims on this page stay checkable rather than atmospheric.
if (!this.props.stage.canTransitionTo(targetStage)) {
return Result.fail(
new ValidationError(
`Cannot transition from "${this.props.stage.value}"
to "${targetStageValue}"`,
"INVALID_TRANSITION",
),
);
}/* abridged: per-stage BullMQ policy */
script_generation: { attempts: 3, delay: 2000 },
direction_generation: { attempts: 3, delay: 5000 },
transcription: { attempts: 1, delay: 1000 },
rendering: { attempts: 1, delay: 1000 },// BullMQ defaults: lockDuration=30s, stalledInterval=30s. // Our facecam editing stage (Scribe transcription) can // take 2-5 minutes for long videos, so a default lock // would mark a healthy worker as stalled. lockDuration: 600_000, // 10 minutes stalledInterval: 300_000 // 5 minutes
const SMOKE_RENDER_TIMEOUT_MS = 5000;
const frames = [0, Math.floor(durationInFrames / 2),
durationInFrames - 1];
for (const frame of frames) {
currentFrame = frame;
renderToStaticMarkup(
React.createElement(Main, { scene, scenePlan }));
}Technical depth
Exact architecture, the reliability mechanisms, and the decision record with tradeoffs stated, closing with the boundary of what this page does and does not claim.
Next.js feature modules and Express controllers translate product actions into use-case requests. Server-sent events stream progress with sequence ids and 15-second heartbeats, and interactive MCP Apps render the same state inside an agent's chat.
presentation/controllers · SSE helpers · MCP routes86 use-cases coordinate repositories, queues, model ports, storage and streaming, returning explicit Result values instead of using exceptions for control flow.
application/use-cases · Result<T, E> everywherePipelineJob owns legal stage transitions, stage-guarded artifact setters, progress state and failure state. Infrastructure cannot redefine the workflow's business rules.
PipelineJob.transitionTo() · setGeneratedCode() guardsPrisma, 14 BullMQ workers, Redis, five routable model providers, two TTS backends, object storage, Remotion and FFmpeg sit behind interfaces, wired at explicit composition roots. Rendering itself is a swappable provider: in-process, or a separate HTTP render service.
repositories · workers · service adapters · ModelRegistryOne Postgres row per job stores the script, fact-check report, audio path, transcript, scene directions, generated code and rendered asset paths. Every worker saves immediately after each legal transition, so the database advances in lockstep with the state machine and any job can be understood from durable state alone.
PostgreSQL · Prisma upsertPer-stage BullMQ policy: three attempts with exponential backoff where models are flaky, exactly one where output is expensive or billable. Failed jobs are cleared and resumed from their checkpointed stage rather than restarted.
BullMQ · per-stage policyProgress events publish live over Redis and append to a durable list. A reconnecting browser replays the buffer in order, then rejoins the live channel; buffers expire an hour after completion. Once a stage is durable, Postgres, not the buffer, is the source of truth, so a refresh can never resurrect pre-edit content.
SSE · Redis buffer · seq idsA worker-level failure listener catches even out-of-memory kills, persists the failure and emits a terminal progress event. Locks run ten minutes with five-minute stall checks, and long stages extend their lock mid-flight.
Failure listener · 10-min lockEvery scene faces eight numbered checks, learned from real failures and beginning with an esbuild compile: lowercase img tags, dead logo CDNs, frame arithmetic bugs, unknown icon names, hallucinated identifiers and more. Errors return to the model as targeted repair hints; in preview, an agentic autofixer isolates the crashing scene and patches it surgically.
esbuild · 8 static gates · autofixerClient-submitted scenes additionally render three frames inside an isolated worker thread with mocked Remotion globals and a five-second timeout, then pass a layout audit for unclamped animations and under-built scenes.
worker_threads · 5s ceilingA hundred and thirty-one days from empty repository to a launched product with billing, accounts and an open agent protocol, in seven phases.
Turborepo foundation, infrastructure services and a Clean Architecture scaffold.
Core faceless flow, SSE progress, browser preview, job retry, code autofix and reconnect replay.
Langfuse and OpenTelemetry tracing, code validation, render timeouts, live render progress and manual editing.
Uploaded-video and facecam workflows, retake control, background removal and creator-edit direction.
Client-driven creation, OAuth 2.1, per-scene briefs, parity gates and interactive MCP Apps, then Chromium and memory fixes, retryable exports and script fact checking.
Accounts and access tiers, a credits paywall with Dodo payments, an admin dashboard, bring-your-own voice keys, 4K export with per-resolution caching, and rendering split out into its own service.
The ideatomotion.com cutover, a server-rendered marketing site with one-click Google sign-in, silent motion graphics, recorded-video workflows, bring-your-own music, and a scene-based timeline editor.