Skip to main content

What Is Focal Harvest?

Focal Harvest is a lightweight, open-source Python CLI tool for automated web research and OSINT collection. It queries search engines, scrapes and sanitizes target URLs concurrently, and synthesizes findings into structured Markdown reports — using Gemini, Claude, or GPT-4o-mini, with a zero-config offline fallback for keyless environments.

Two Primary Use Cases​

Research Pipeline​

Developers, analysts, and researchers repeat the same manual workflow every week:

Search → Open 20 tabs → Strip ads → Copy-paste → Synthesize in LLM → Save report

Focal Harvest runs that entire cycle as a single terminal command. You get a structured Markdown report — and the tabs stay closed.

OSINT Collection Framework​

Security practitioners and intelligence analysts need to monitor topics across multiple platforms continuously. Focal Harvest covers Reddit, Hacker News, Stack Overflow, arXiv, GitHub, and 17+ pre-configured platforms, with daemon scheduling for continuous monitoring, Wayback Machine fallbacks for bot-protected pages, and Discord or Telegram alerts when a sweep completes.

See OSINT Use Cases → for specific workflows.

What You Get Out of the Box​

  • Structured Markdown reports saved locally to reports/
  • Raw JSON exports for downstream processing
  • Local caching so repeat queries load from disk instantly
  • Multi-LLM failover across Gemini, OpenAI, and Claude
  • Offline mode using a built-in statistical keyword ranker — no API keys needed

Design Constraints​

Focal Harvest enforces five non-negotiable architectural boundaries that govern every feature decision:

  1. Under 5 MB install size — no bloat, installs in seconds
  2. Terminal-only — no web frontends, no background REST servers
  3. Runs on low-end hardware — no GPU required
  4. No local AI downloads — remote APIs only; no Ollama or Llama
  5. No SQL databases — all data stored in plain Markdown and JSON files

Where to Go From Here​