Build Your Own Semantic Reddit Search for Lead Discovery

Keyword search misses the customer who describes your exact problem in words you never thought to search for. This is the worked example: pull a subreddit, embed it, rank it against your product, and read the top of the list.

Keyword search misses a predictable kind of thread: the one where someone describes your exact problem without your product's words for it. “I keep losing track of what I promised people” is a task-manager lead no keyword alert would catch. Semantic search — match by meaning, not string overlap — fixes that, and it's a small project on raw API access. Worked example: pull a month of a subreddit, embed it, rank it against your product.

1. How do I pull a month of Reddit posts and comments?

after= accepts a relative window ("30d") directly, then the cursor takes over for the rest of the pagination:

pull.py

import requests

API_KEY = "YOUR_KEY"
BASE = "https://api.threadsnoop.com/v1"
HEADERS = {"x-api-key": API_KEY}

def fetch_window(subreddit, kind, window="30d"):
    cursor, items = window, []
    while True:
        params = {"subreddit": subreddit, "sort": "asc", "after": cursor, "limit": 100}
        resp = requests.get(f"{BASE}/{kind}", headers=HEADERS, params=params)
        resp.raise_for_status()
        payload = resp.json()
        items.extend(payload["data"])
        meta = payload["_threadsnoop"]
        if not meta["has_more"]:
            return items
        cursor = meta["cursor"]

posts = fetch_window("productivity", "posts")
comments = fetch_window("productivity", "comments")

A month of a mid-traffic subreddit is a few thousand items — a few dozen credits per kind at 100/page. Pull comments, not just posts: buyer pain often shows up three replies into someone else's thread.

2. How do I embed Reddit posts for semantic search?

Any embeddings provider works — this uses a generic OpenAI-compatible one. Embed the Reddit text in batches, plus one more string: your problem, in the customer's voice, not your category name.

embed_and_rank.py

EMBEDDINGS_URL = "https://api.openai.com/v1/embeddings"  # swap in your provider

def embed(texts, batch_size=100):
    out = []
    for i in range(0, len(texts), batch_size):
        resp = requests.post(
            EMBEDDINGS_URL,
            headers={"Authorization": "Bearer YOUR_EMBEDDINGS_KEY"},
            json={"model": "text-embedding-3-small", "input": texts[i : i + batch_size]},
        )
        out.extend(item["embedding"] for item in resp.json()["data"])
    return out

def text_of(item):
    return item.get("selftext") or item.get("body") or item.get("title", "")

candidates = posts + comments
vectors = embed([text_of(c) for c in candidates])
problem_vector = embed(["I keep forgetting things I agreed to and don't notice until someone chases me"])[0]

def cosine(a, b):
    dot = sum(x * y for x, y in zip(a, b))
    return dot / ((sum(x * x for x in a) ** 0.5) * (sum(y * y for y in b) ** 0.5))

ranked = sorted(zip(candidates, vectors), key=lambda p: cosine(problem_vector, p[1]), reverse=True)
for item, vec in ranked[:20]:
    print(f"{cosine(problem_vector, vec):.3f}  r/{item['subreddit']}  {text_of(item)[:90]}")

Read the top twenty by hand. Embedding your category name ("task management software") instead of the pain matches people discussing that category, a smaller and less useful set than people who actually have the problem — the wording of the problem string is what most of your result quality rides on.

Don't want to write this much code?

There's a plain-prompt version of the same idea that skips the embeddings pipeline entirely: how to leverage Claude to find your first SaaS customers. Once you have matches — from this pipeline or that simpler one — the rules for engaging without getting banned are the same: how to find Reddit users who need your product.

Frequently asked questions

What does semantic search catch that keyword search doesn't?

Threads where someone describes your problem without using your product's vocabulary for it — 'I keep losing track of what I promised people' instead of 'task manager.' Matching by meaning (via embeddings) catches these; a keyword list structurally can't.

Which embeddings provider should I use?

Any of them — the example in this post uses a generic OpenAI-compatible embeddings endpoint, but the pipeline works with whichever provider you already have a key for. The Reddit data pull is the same regardless.

How much does pulling a month of a subreddit cost?

It depends on subreddit traffic, but a mid-traffic community's month of posts and comments is usually a few thousand items — a few dozen credits at 100 results per page, well within the 1,000 free Reddit searches you get on signup for a first test.

Do I have to build and run this myself?

Yes, today — this post is the build-it-yourself path: pull the data, embed it, rank it against your product, on a schedule you control. The API is free to start, so the fastest way to see if it's worth building is to run the worked example above against your own product.

Build it yourself

Get a free API key and build the pipeline above — the worked example uses real endpoints, not pseudocode.

Get your free API key→

1,000 free Reddit searches on signup, no card required.

keep reading