Automated YouTube Shorts from Tech News — End to End

The Idea: Turn Tech News into Daily Shorts, Automatically

Tech moves fast. A lot of people want quick video summaries of what happened today in the industry — but producing even a 45-second video manually takes 20–30 minutes when you factor in scripting, recording, editing, and uploading. I wanted to close that gap entirely.

The pipeline runs as two independent npm scripts. npm run fetch is the data pipeline: it pulls headlines from three sources, summarises them with AI, and writes a dated JSON "queue" file. npm run upload is the media pipeline: it drains that queue — producing and uploading one Short per news item. Or run npm run all to fire both in sequence. Keeping them decoupled means a failed upload never blocks new news from being queued.

You can see the output of this pipeline live on the ▶ The Project Sandbox - new tech news Shorts published automatically every few hours.

Project Structure

tech-news-shorts/
├── src/
│   ├── index.js                  # CLI entry point
│   ├── news/
│   │   ├── fetcher.js            # NewsAPI + GNews + HackerNews fetcher
│   │   ├── fetch-runner.js       # Local test runner for news fetching
│   │   └── summarizer.js         # HuggingFace Inference API (Llama)
│   ├── reels/
│   │   ├── processor.js          # Orchestrates the full shorts pipeline
│   │   ├── tts.js                # TTS: node-edge-tts → ElevenLabs → Google
│   │   ├── pexels.js             # Pexels video downloader (title-based query)
│   │   └── ffmpeg.js             # Audio + video merge + title overlay
│   ├── youtube/
│   │   ├── auth.js               # One-time OAuth2 setup (localhost callback)
│   │   └── uploader.js           # YouTube Data API v3 upload
│   └── utils/
│       ├── db.js                 # JSON file database (news_YYYY-MM-DD.json)
│       └── logger.js             # Colored timestamped logger
├── db/                           # Auto-created: daily news JSON files
├── output/                       # Auto-created: final .mp4 shorts
├── .env                          # Your API keys (never commit)
└── package.json

Phase 1 — Fetch, Summarise & Store

npm run fetch kicks off src/news/fetcher.js. It calls three sources: NewsAPI, GNews, and HackerNews (no API key needed for HN). Articles are deduplicated by URL and the full article text is fetched before summarisation — not just the headline snippet.

Each article is then sent to the Hugging Face Inference Router using meta-llama/Llama-3.3-70B-Instruct:fastest to produce a concise plain-English summary. Results are persisted to db/news_YYYY-MM-DD.json, each record tagged "status": "unprocessed". That file is the handoff point to Phase 2.

// src/news/summarizer.js — HuggingFace Inference Router
const HF_API =
  "https://router.huggingface.co/v1/chat/completions";

async function summarise(text) {
  const res = await fetch(HF_API, {
    method: "POST",
    headers: {
      Authorization: `Bearer ${process.env.HUGGINGFACE_API_KEY}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      model: process.env.HF_MODEL ||
             "meta-llama/Llama-3.3-70B-Instruct:fastest",
      messages: [
        { role: "system", content: "Summarise in 2-3 sentences." },
        { role: "user",   content: text },
      ],
    }),
  });
  const data = await res.json();
  return data.choices[0].message.content.trim();
}

Phase 2 — Produce & Upload the Short

npm run upload runs src/reels/processor.js. It scans db/ for any "status": "unprocessed" record and pipes it through a three-step media pipeline: TTS → Pexels video → FFmpeg merge → YouTube upload.

Step A — Text-to-Speech: The summary is converted to speech using node-edge-tts, which taps Microsoft Edge's neural voices for free with no API key. The default voice is en-US-AriaNeural; you can swap it in .env with EDGE_TTS_VOICE. ElevenLabs and Google Cloud TTS are available as fallbacks if those keys are present.

// src/reels/tts.js — node-edge-tts (primary)
import { EdgeTTS } from "node-edge-tts";

export async function synthesise(text, outPath) {
  const tts = new EdgeTTS();
  await tts.ttsPromise(text, outPath, {
    voice: process.env.EDGE_TTS_VOICE || "en-US-AriaNeural",
  });
  return outPath; // .mp3
}

Step B — Fetch a Pexels Stock Video: src/reels/pexels.js derives a search keyword from the article title (e.g. "AI chip ban" → "technology AI") and queries the Pexels Videos API with orientation=portrait. The clip is downloaded and cached locally so the same query doesn't re-download on retry.

// src/reels/pexels.js (simplified)
export async function fetchVideo(query, destPath) {
  const res = await fetch(
    `https://api.pexels.com/videos/search?query=${encodeURIComponent(query)}&orientation=portrait&per_page=5&page=${randomPage()}`,
    { headers: { Authorization: process.env.PEXELS_API_KEY } }
  );
  const { videos } = await res.json();
  const link = videos[0].video_files[0].link;
  await downloadFile(link, destPath);
  return destPath;
}

Step C — FFmpeg Merge & Upload: src/reels/ffmpeg.js uses the bundled ffmpeg-static binary (no system install needed) to scale the clip to 1080×1920 (9:16), loop it to match the exact TTS audio duration, overlay the headline as a white caption near the bottom, and merge the audio track. The YouTube Data API v3 then uploads the .mp4 as a Short; on success the record is flipped to "processed".

// src/reels/ffmpeg.js — scale + loop + overlay + merge
import ffmpegPath from "ffmpeg-static";
import { execFile } from "child_process";

export function buildShort({ videoPath, audioPath, headline, outPath }) {
  return new Promise((resolve, reject) => {
    execFile(ffmpegPath, [
      "-stream_loop", "-1", "-i", videoPath,
      "-i", audioPath,
      "-vf", [
        "scale=1080:1920:force_original_aspect_ratio=decrease",
        "pad=1080:1920:(ow-iw)/2:(oh-ih)/2",
        `drawtext=text='${headline}':fontcolor=white:fontsize=48:` +
          "x=(w-text_w)/2:y=h*0.75:line_spacing=8",
      ].join(","),
      "-c:v", "libx264", "-c:a", "aac",
      "-shortest", outPath,
    ], (err) => (err ? reject(err) : resolve(outPath)));
  });
}

// hot take The hardest part wasn't the AI, the video editing, or the Pexels integration — it was YouTube's OAuth flow. Thirty lines of business logic, two hundred lines of auth wrangling. Google's API client libraries desperately need a "just let me upload a video" mode. — Shweta Suryavanshi

API Keys & Environment

Copy .env.example to .env and fill in:

# Required
NEWS_API_KEY=        # newsapi.org — 100 req/day free
GNEWS_API_KEY=       # gnews.io — 100 req/day free
HUGGINGFACE_API_KEY= # huggingface.co/settings/tokens — free
PEXELS_API_KEY=      # pexels.com/api — free

# TTS (optional overrides — node-edge-tts needs no key)
EDGE_TTS_VOICE=en-US-AriaNeural
ELEVENLABS_API_KEY=
GOOGLE_TTS_KEY=

# YouTube OAuth
YOUTUBE_CLIENT_ID=
YOUTUBE_CLIENT_SECRET=
YOUTUBE_REFRESH_TOKEN=  # generated by: npm run auth

HackerNews requires no key and is always active. If YouTube credentials are absent, the finished .mp4 is saved to output/ for manual upload to the channel.

YouTube OAuth2 Setup

1. Go to Google Cloud Console → Credentials and create an OAuth 2.0 Client ID (Desktop app).
2. Add http://localhost:3000/oauth2callback to Authorized redirect URIs.
3. Paste YOUTUBE_CLIENT_ID and YOUTUBE_CLIENT_SECRET into .env.
4. Set the OAuth consent screen to External and add your Gmail as a test user.
5. Run npm run auth, open the printed URL, authorise, then copy the printed YOUTUBE_REFRESH_TOKEN back into .env.

Running the Pipeline

# Install (ffmpeg + ffprobe bundled — no system install needed)
npm install

# One-time YouTube OAuth
npm run auth

# Fetch news + summarise → writes db/news_YYYY-MM-DD.json
npm run fetch

# Generate videos + upload → drains unprocessed records
npm run upload

# Or run both in sequence
npm run all

# Test individual stages without uploading
npm run test:fetch   # news fetching + summarisation
npm run test:tts     # TTS audio → output/tts-test/
npm run test:vid     # full video → output/vid-test/

What's Next

Right now the caption is a static drawtext overlay. The next step is word-by-word animated captions synced to the TTS audio timestamps — which dramatically improves watch-time on Shorts. I also want to add a Pillow-based thumbnail generator: render the headline on a branded background and push it via the YouTube thumbnail API.

The same two-phase architecture maps directly onto Instagram Reels and TikTok: swap src/youtube/uploader.js, adjust the output resolution, done. The news fetch, Llama summarisation, Edge TTS, and Pexels steps stay identical.