Sreekar Palla — ClipTube

Overview

Some of our best gaming nights, conversations, funny moments, and other recordings ended up scattered across my friends' and my PCs. I wanted to find a particular game, phrase, or moment without scrubbing through years of video, so I built Python indexing utilities, a native macOS library, and ClipTube, a web app where we can search and stream the same personal archive.

ClipTube macOS app in dark mode showing the All Clips library in a two-column thumbnail grid, a sidebar with All Clips, Favorites, On This Day, and Unwatched, and an empty clip preview pane. — The native macOS app organizes years of recordings into a searchable clip library.

ClipTube web library showing a grid of video thumbnails, search controls, and a sidebar of browsing, watch history, date, and status filters. — The browser library brings searchable clips and rediscovery filters into one view.

What it is

ClipTube is the web companion to a native macOS video-library app I built for my friends and me. Together, the apps make it possible to browse and replay years of PC recordings, from gaming clips to conversations and funny moments we would otherwise have to stumble across by chance.

A shared clip catalog connects each recording to the context that helps us remember it: its date, tags, on-screen text, transcript, and technical details. The original videos stay in the library, ready to play in either app.

Why I built it

My friends and I kept saving videos over the years, but a growing folder of recordings is hard to explore when the moment you remember has no useful filename. We remember who was playing, which game was on screen, or something somebody said—not where a particular file landed.

I built ClipTube to put that context to work, turning the archive into something we could search together and return to whenever an old moment came up in conversation.

Indexing the archive

I wrote Python utilities to scan the video library, interpret dates in filenames and folders, and apply configurable keyword tags. OpenCV samples video frames and Tesseract OCR reads text from game screens and other desktop content. A separate Whisper pass transcribes speech and adds that text to the catalog, so visual and spoken details can help identify a clip.

The web indexer enriches the catalog with media information from ffprobe and generates cached thumbnails with FFmpeg. Keeping searchable metadata and generated assets alongside the original files lets the apps reopen the library without rescanning every video just to display it.

Diagram showing PC video recordings flowing through a Python indexing pipeline with sampled-frame OCR and Whisper transcription into a shared clip catalog, then into a SwiftUI macOS app and ClipTube's browser app with SQLite search and video streaming. — OCR, tags, and speech transcripts enrich the shared clip catalog for both apps; ClipTube also builds searchable indexes and streams the original videos.

One library, two apps

The macOS app uses SwiftUI for the library and AVKit for playback. It reads clips directly from the collection and caches generated thumbnails, while keeping the browsing and playback experience close to the videos themselves.

The web companion pairs a React and TypeScript interface with a Node.js and Express server. The server builds SQLite full-text and semantic search indexes, caches media metadata and thumbnails, and streams video on demand so we can seek through clips in the browser.

Search and rediscovery

On macOS, I built fuzzy, semantic, and exact transcript search over clip filenames, tags, and spoken text. Sentence embeddings help surface relevant clips even when a query describes the moment in different words; filters for people, games, dates, and times narrow the results when we remember more.

The web app combines weighted SQLite full-text results with semantic matches from clip details and overlapping transcript chunks. Favorites, watch history, calendar and On This Day views, session browsing, statistics, and duplicate detection offer more ways to find a clip than typing a search query.

What I worked through

A single search technique was not enough for an archive with messy filenames, visible game names, personal tags, and long conversations. I built the desktop and web apps to support both precise lookups and looser descriptions, while keeping the results connected to dates, people, and games.

Long transcripts raised a separate indexing challenge: splitting speech into independent passages could lose the context around a relevant phrase. The web indexer carries words across transcript chunks, then combines metadata, keyword, and semantic results so specific terms and broader descriptions can both lead back to a useful clip.

Lessons learned

The most important part of a personal media library is the context attached to each file. Dates, screen text, tags, and transcripts give us different ways to remember the same moment, and combining those clues makes older recordings far more useful than a folder full of videos.

Building both apps also taught me to design the catalog around the people using it, then let each client handle search and playback in a way that fits its platform. The result is software my friends and I can keep using as the archive—and the memories in it—grows.

Highlights

Created a Python indexing pipeline with OpenCV, Tesseract OCR, contextual tagging, and separate Whisper transcription to make a personal video archive searchable
Built a native SwiftUI macOS app with fuzzy, semantic, and exact transcript search, plus filters for people, games, dates, and times
Developed a React and TypeScript web app backed by Node.js, Express, and SQLite to search the library and stream clips in the browser
Designed semantic transcript search for long recordings using 400-word chunks with 80-word overlaps
Added calendar and On This Day rediscovery, favorites, watch history, session browsing, statistics, and duplicate detection

Code

Making on-screen text searchable with batched OCR

python

    def extract_screen_text(self, video_path):
        cap = cv2.VideoCapture(video_path)
        total_frames = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))
        sampled = sorted(random.sample(
            range(total_frames), min(OCR_FRAMES_COUNT, total_frames)
        ))

        frames = []
        for frame_index in sampled:
            cap.set(cv2.CAP_PROP_POS_FRAMES, frame_index)
            ok, frame = cap.read()
            if not ok:
                continue

            gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
            if gray.shape[1] < 1000:
                gray = cv2.resize(gray, None, fx=2, fy=2,
                                  interpolation=cv2.INTER_LINEAR)
            gray = cv2.threshold(gray, 0, 255,
                                 cv2.THRESH_BINARY + cv2.THRESH_OTSU)[1]
            if cv2.countNonZero(gray) < gray.size / 2:
                gray = cv2.bitwise_not(gray)
            frames.append(gray)
        cap.release()

        recognized_text = []
        for start in range(0, len(frames), 5):
            batch = frames[start:start + 5]
            separator = np.full((20, batch[0].shape[1]), 255, dtype=np.uint8)
            stack = np.vstack([part for frame in batch for part in (frame, separator)][:-1])
            text = pytesseract.image_to_string(stack, config=self.ocr_config)
            recognized_text.extend(text.split())

        return ' '.join(recognized_text)

ClipUtilities samples up to 80 frames from each recording, preprocesses them for OCR, then stacks five frames into each Tesseract pass. Batching cuts down on OCR calls while capturing game titles and other text shown in the video, giving the library more context for finding a clip.

Skills & Technologies

SwiftSwiftUImacOS DevelopmentPythonOpenCVTesseract OCRWhisperTypeScriptReactJavaScriptNode.jsExpressSemantic SearchFull-Text SearchFFmpegSQLiteDocker