← Projects

Job Scraper 02

2026 — ongoing

v0.3.0 on PyPI & GHCR · improving toward a 1.0

A FastAPI scraper and a Streamlit dashboard in two containers, driven by a REST API the UI just happens to call. Paste your CV and a few keywords, hit Start, and it searches multiple job sites and ranks each result 0–10 against you. I built it while job hunting and it found me the job. It's a real package on PyPI now, with a release pipeline that does more than tag a commit.

Role — Sole developer — the tool, the packaging, and the release pipeline

On PyPI
pip install job-scraper02
Version
v0.3.0
Tests
26 · CI runs the real app
Container images
3 (scraper · ui · all-in-one)
AI providers
3 + keyless Lite Mode
PyPI downloads/mo
138
GitHub release downloads
0
FastAPIStreamlitPlaywrightSQLitepandasGroq / Anthropic / GeminiDocker Composen8n (optional)PyPI Trusted PublisherGHCR

The constraint that shaped it

It runs with no API key at all. Bring a Groq, Anthropic or Gemini key for AI-scored relevance, or stay in Lite Mode — keyword matching, no key, no signup. Everything lands in a local SQLite file you own, and nothing leaves the machine unless you set up an AI key or a webhook. A tool that demands a paid credential before it shows you anything is a tool nobody tries.

What it looks like

3 screens · click to enlarge
The dashboard. Every listing carries a 0–10 score, colour-coded, and — this is the part that makes it useful — matched_keywords with the exact terms that hit, not just a percentage. Status, running/paused state and job count sit up top; a Refresh button rather than a socket, because a scrape is a job you check on, not a stream you watch.

The decision I'd defend to anyone

AI scoring is optional, not required. With no key it drops to keyword matching, still ranks the jobs and still works. That's the whole pitch: a job tool that makes you sign up for a paid API before it will show you anything is a job tool you close.

Everything the dashboard does is also a REST endpoint and a CLI command, so it wires into your own automation. There's an optional n8n profile if you want a full automation engine, and one-click configs for Railway and Render. The default is still docker compose up on your own machine, with a SQLite file you keep.

v0.3.0 — a release that had to prove the app runs

Lite Mode had been scoring every job zero since it shipped. A regex bug meant no listing ever matched, however well it fit the CV. No error, nothing in the logs — the failure mode that makes a feature look present and be useless. Fixed, with regression tests covering match, no-match, word-boundary and score-cap.

Notifications were dead by default — the alert threshold was 70 on a scale that only goes to 10. Results weren't sorted by score, despite the README claiming high scorers bubble to the top. Two scrapes could race if triggered close together; the run lock is atomic now. The plain `Dockerfile` most platforms auto-detect started only the UI with no backend attached, so it was silently unusable.

CI now proves the app works, not that it compiles. It builds the wheel and checks nothing leaked out of the package, installs and runs the real job-scraper console script, and drives a genuine scrape against the live containers, asserting a real scored job comes back. Same lesson as every other project of mine: a green pipeline that only ran the type checker means nothing.

Packaged properly. The flat top-level db / app / scraper modules moved into a real job_scraper package, so installing it can't collide with someone else's db module in their environment.

How it was built

  • 11 February 2026

    Job Scraper 02

    Built for myself because I was tired of checking ten job boards every morning. It runs with no API key at all — a tool that demands a paid credential to demonstrate itself is a tool nobody tries.

  • 24 April 2026

    Job Scraper published to PyPI

    Via a GitHub Actions Trusted Publisher pipeline, with two container images alongside it.

Problems that came out of this project1

Related notes