The Tor exit study runs qwen3-vl-32b live (a cheap vision model). Its challengers — two free computer-vision methods (shape-match, color-cv) and gemini-3.1-flash-lite (Tamás's paid entry) — are scored on the same real Smyths puzzles. Here's how all four do, and exactly where the free ones break.
qwen3-vl-32b — a vision AI. It reads the legend strip ("click these icons, in this order"), finds each doodle in the photo, and clicks them in sequence. Costs a fraction of a cent per puzzle.
gemini-3.1-flash-lite — Tamás's paid vision AI. Same job as qwen, just as accurate, but ~6× the per-solve cost (it burns more tokens) — though both are still pennies a month.
shape-match — free. Cuts each icon out of the legend and looks for a matching shape in the photo. No AI, no cost.
color-cv — free. Finds the hand-drawn doodles by their ink colour standing out from the photo. Also no AI.
A puzzle only counts as solved if every doodle is clicked in the right order. "Found" = located the drawings at all. The gap between the two is where the free methods lose.
Three rungs, hardest last: ① hit each icon = of every individual doodle, how many it clicked near · ② found all icons / puzzle = puzzles where it located every doodle (order ignored) · ③ in the right order = solved, the only one that actually passes the captcha.
Full ranking of all benched configs on the leaderboard ›
Both free methods find roughly half the doodles but solve far fewer — because finding a drawing isn't enough, you have to click them in the legend's order. That ordering step is what the vision model gets and the CV methods don't.
The same five puzzles through each method. Green = the correct doodle; red = where the method clicked. Overlapping = a hit; scattered red = a miss.
The 37 above are a fixed lab set. This is the real-world run: every method scrapes the same URL sample, each on its own Tor (its own exits), at the same time. It's not a same-captcha test — each arm meets different exits — but it shows how each method actually performs end-to-end: how many captchas it cleared, how many products it got, how fast, how reliably.