Two pipelines
Everything on CuratedAI comes out of two systems. One reads the day's AI news and publishes a briefing from it; the other turns long video into short. Both run unattended, and both are described here stage by stage, including the decisions that are not obvious from the outside.
Built and maintained by Gaurav
The briefing pipeline
DAILY · PYTHON · ~74,000 LINES
Reads 61 AI news sources every morning and publishes one Top 10 with twelve platform deliverables built from it. 195 days of output so far.
SHAPE A funnel, then a fan. Hundreds of stories converge to ten, and those ten explode into twelve deliverables.
-
IN
61 sources
A balanced source list read every morning, typically a couple of hundred stories.
WHY Breadth first. A narrow list produces a confident, wrong picture of the day, and the filtering downstream only works if there was something to filter.
-
01
Score
Each story gets an AI-relevance score and a composite quality score out of ten. Nothing is discarded yet.
WHY Scoring and cutting are separate steps on purpose. A story can score badly on relevance and still matter, and that judgement belongs after the numbers, not inside them.
-
02
Rank three ways
Three independent selections: raw score, a composite, and one tuned for spread. Passes two and three may not reuse pass one’s picks.
WHY One ranking is one opinion. Forcing the passes to disagree surfaces thirty distinct stories instead of the same ten sorted three ways, and the disagreement is the useful part.
-
03
Thirty-day dedupe
Blocks an exact URL, then the same company plus the same key terms, then the same company plus five overlapping words.
WHY Yesterday’s news is the hardest thing to filter. Monday’s "Cursor Raises $2.3B" has to kill Thursday’s "Cursor $2.3B Valuation", and no exact-match check catches that.
-
04
Reconcile
The three rankings are reconciled into a single Top 10.
WHY Reconciled rather than averaged. Averaging three rankings returns the blandest ten; resolving them keeps a story that one pass felt strongly about.
-
05
One per company
A hard cap of one story per company.
WHY On a big funding day every source leads with the same firm. Without the cap the briefing becomes that company’s newsletter.
-
06
Write twelve
One curation fans out into the briefing, a narration script, Slack, Shorts titles, LinkedIn, a Twitter thread, TikTok and per-story social cards.
WHY Written from the same curation rather than re-decided per platform, so the day says the same thing everywhere it appears.
-
07
Render
9x16 news image and 4:5 Instagram cards composited locally from the same story set.
WHY Local rendering keeps the per-day cost at zero and the turnaround under the time it takes to review the copy.
-
08
Validate
Completeness, duplicate titles and dead links checked before the day is called done. Failure sends it back to be rewritten.
WHY Twelve outputs from one source is twelve chances to publish something malformed at four in the morning. The gate is cheaper than the correction.
-
OUT
Published
One dated page with every story linked to where it came from.
WHY The link is the product. A digest nobody can check is an opinion.
The clipper pipeline
PRIVATE · PYTHON · ~7,500 LINES
Takes a YouTube URL and returns finished vertical clips with burnt-in word-level captions, a written hook, a description and a publish slot. It runs unattended.
SHAPE A line with a tournament in the middle and a hard stop at the end.
-
IN
Transcript
Cache first, then yt-dlp subtitles, then the transcript API.
WHY Three sources in falling order of cost. Long episodes are the norm, so nothing is fetched twice.
-
01
Keyword score
A sliding window over the transcript with filler stripped out, scoring every candidate span.
WHY Cheap scoring before any model call. Sending an entire transcript to a model to find fifteen candidates costs more than finding them arithmetically first.
-
02
Model re-rank
The top fifteen candidates go to the local Claude CLI, which re-orders them and writes a title and description for each.
WHY The model is used where judgement is needed and not where counting will do.
-
03
Cut segments
Segments pulled at 1080p, preferring an already-downloaded source file.
WHY Re-downloading the same two-hour episode for each of six clips is the obvious way to do this and the wrong one.
-
04
Transcribe
faster-whisper runs locally for word-level timings.
WHY Local means no per-minute bill and nothing leaves the machine, which matters when the source is unpublished.
-
05
Burn captions
Word-by-word captions in Bebas Neue, burnt in rather than sidecar.
WHY Every platform renders sidecar captions differently. Burnt in, the clip looks the same everywhere.
-
06
Hook tournament
Eight hooks generated, validated, judged against each other, run off, then trimmed to twelve words. A clip that falls back to its own title is retried automatically.
WHY The hook decides whether the clip is watched, so it is the one place worth spending eight generations and a judge. The retry exists because falling back to the title is the failure case and should not ship quietly.
-
07
Render 9:16
1080x1920, black field, the 16:9 clip centred with the winning hook set above it.
WHY Letterboxing with the hook in the dead space turns wasted pixels into the thing that stops the scroll.
-
08
Quota gate
YouTube allows 10,000 units a day and an upload costs 1,600. Three go out, 4,800 is spent, state is written, the run stops.
WHY The quota is arithmetic, so the scheduler does the arithmetic instead of discovering the limit by hitting it.
-
OUT
Scheduled
Posted to a US-optimised slot. Anything touching an already-live video clears a confirmation first.
WHY A y/N prompt is not a safety gate: a piped yes or a reflexive return key defeats it. Everything reachable from a terminal eventually gets run by accident.
These are the systems behind the daily briefing, the directory and the radar. The ratio they produce is the point: across 148 days where both the intake and the published briefing exist, 31,727 stories went in and 1,483 came out.