A running log of the project: the bench, the gallery, the Tor exit study, the solver shootout, the production pipeline — and an honest record of the one self-inflicted incident, so the numbers on this site stay trustworthy.
2026-06-16
Verified every "missing" product by hand. Opened all 36 URLs that day-2 didn't scrape, one by one, in a normal browser — every single one is a discontinued/delisted toy whose old URL now 301-redirects to a category page, the brand page, "all items", or the region homepage. Zero were live products the parser missed. Tell-tale: the same SKU (Schleich "Steintriceratops 70828") is dead in CH, DE and AT — a parser bug fails randomly, a gone product fails everywhere. Added a pop-up report on Live runs (closes with ✕ / Esc) listing each one and where its dead URL now lands.
Reframed the home-page summary into "What we've gathered — across every run": products captured, pages fetched, Tor exits used, time spent, and captchas met → solved — totalled across all runs. Dropped "failed"-style framing, because a discontinued product didn't fail — there was simply nothing on the page to scrape. The scarier "random-exit hostility %" stays on the exit-study tab, clearly labelled as a different measurement.
The Live runs table now shows every production day (not just the latest), each with its real captcha-met→solved and pace numbers, pulled from the saved per-pass reports + exit logs.
Fixed a misleading roll-up: it counted every retry attempt as an error ("104 errors" when only 26 URLs actually ended unscraped). Now it counts unique, truly-unscraped URLs only.
Day 3 — a third 3,000-URL production day — kicked off in the Linux container.
2026-06-14
⚠ Incident — the day-1 run wedged (and the dashboard briefly believed it)
The first production day (launched 06-13) froze for ~11 h at 627/3,000: headed Chrome on this Mac hung in the retry pass, while a separate status publisher kept writing "running" — so the dashboard reported progress that wasn't happening. Root cause: one giant long-lived browser with no watchdog. Fix: killed the zombie, freed the Chrome profile lock, and rebuilt the run to be crash-proof — short capped passes (250 URLs) each recomputing what's left from the saved file, with a per-pass watchdog, so a hang now costs ≤25 min instead of the whole night. Lesson on the record: never trust a "running" flag — verify the output file is actually growing.
Day 1 finished — 2,974/3,000 (99.1%) once the run was crash-proof; the 26 it couldn't get were all discontinued products, with zero captcha or block losses.
Built + proved the Linux worker image. The Docker image (Tor + Xvfb virtual display + real Chrome) runs the scraper with no on-screen window to wedge — exactly how it'll run on the production worker. Verified on this Mac: Tor bootstraps inside the container, exit-rotation auth checks out, and it scrapes + solves doodles end-to-end.
Day 2 ran entirely inside that container — 2,964/3,000 (98.8%) — with a fresh disposable container per batch, so a hang can never carry over to the next one.
2026-06-13
Honesty pass over the whole site. Recomputed every model's cost to one consistent basis — $ per 1,000 captcha-solves (real bench tokens × real OpenRouter prices), replacing a field that mixed total project bills with stale estimates; color-cv's bogus $2.12 dropped to ~$0.08. Swapped the cost "battery" (a full battery read as good on a cost axis) for a $-tier chip that goes green→red as it gets pricier. Added an Economics tab (real $/month, the free-key question, Tor machine specs). Killed the "3-day study" wording where only day 1 has run, relabelled "Product success" → "pages parsed OK", and reconciled the captcha gallery (a cumulative corpus) against the ~1% live exit rate.
First full production day went live on the dashboard, streaming scrape progress as it ran.
Scoped the displayed exit-study metrics to exclude the rate-limit window from the night before (see the 06-12 incident) — so "hostile exits" reflects real hostility, not a dev mistake.
Benched more cheap models/combos on the same 37 puzzles and surfaced them on the leaderboard + shootout.
Merged the captcha-image views into one Real captchas gallery (filter by model + verdict) and ingested fresh captures.
2026-06-12
⚠ Incident (~20:44, my fault — Philippos)
Started too many qwen-based scrapers at once — the 3-day exit study plus a 4-arm shootout at 3,000 URLs each plus the production run, ~5 concurrent solvers all hitting the same OpenRouter qwen endpoint. OpenRouter rate-limited qwen (HTTP 429), which spiralled every qwen-based run into fail-and-rotate. Caught it, stopped the over-concurrent runs, and set a rule: never run more than ~2 qwen solvers at once. The metrics on this site are scoped to exclude that window, so they reflect the scraper — not the mistake.
Shipped the production reliability pipeline: scrape every product, or log exactly why it couldn't be fetched. PR #3
Shipped human-like solving: give up after a few puzzles + scroll/fidget instead of grinding 30 in a row. PR #4
Added a daily production runner + live status, surfaced as the Production tab.
Ran the solver shootout — model "arms" scraping in parallel through separate Tor circuits — with a live comparison table.
Added gemini-3.1-flash-lite as a 4th contestant (a Tamás request) across the leaderboard, gallery and shootout; its live arm hit 93.6% product success over 500 fetches (the exits it drew were clean, so it met 0 captchas to solve).
Turned on real captcha capture during the exit study + per-day counts.
Launched the 3-day Tor exit study with per-exit logging and a new dashboard tab.
Added the Real captchas gallery and rebuilt the click-overlay gallery as a model×puzzle matrix.
Corrected paid-model costs from real OpenRouter spend (the flat estimate was ~13× too low for thinking models).
Merged a 13-config benchmark: gemini-3-flash topped solve rate ~97%, qwen3-vl-32b ~92% as the value pick.
Added the ordered-solve metric (right icons in the right order) — exposing that the free CV baselines order near-randomly.
Prototyped the name-free shape-match locator + a cropper + labeling tool. PR #2
Tuned the colour doodle detector — recall 16% → 71%.
Stood up the first model-bench dashboard from hand-labeled real puzzles.
2026-06-09 — 06-11
Made Gemini 2.5's "thinking" mode off-by-default for speed (env-toggle).