Potluck
Every recipe you've ever saved, extracted into one shareable family recipe box
An AI recipe extractor: your Instagram saved posts, any recipe URL on the web, and scanned cookbook PDFs — pulled apart by Claude into structured recipes, gated by an eval harness, and served to your family through links that need no accounts. Built with five agents in parallel worktrees. This guide tells the whole story, including the part where the build method broke down and we folded back to one agent — because that turned out to be the most valuable lesson in the series.
Read the post-mortem. This build produced Tommy's verdict, recorded verbatim in the field notes: "I consider the multi-agent my first breakdown. I do not consider it a success." The app shipped and the family uses it every week — but the build burned 92% of a session allowance in ten minutes of agent fan-out, and seventeen hours of wall clock delivered two hours of work. The companion page, the Potluck post-mortem, catalogs everything that went wrong and traces each failure to its root cause. Most AI tutorials only publish the wins; the failures are where the transferable lessons live.
- What you're building
- The Instagram wall, and the honest way around it
- Contracts before code
- The data model: four ideas
- The extraction waterfall
- The eval harness — treating the prompt like code
- PDFs: segment first, extract second
- Sharing without accounts
- What reality did to the plan
- The pivot: five agents, then one
- Where it lives now
What you're building
Potluck fills itself from three doors. A one-time Instagram export backfills every recipe you've ever saved. A paste-a-URL box ingests any recipe page on the web. And a PDF upload takes a single printed recipe or a 63-page scan of Mom's handwritten cookbook and finds every recipe inside it. All three converge on one import queue; Claude is the extraction engine at the end of it; and the readable side is a family link in the group chat — no accounts, no logins, a print stylesheet, and a screen that stays awake while you cook.
The stack is deliberately boring: Next.js App Router, Postgres with Drizzle, Zod contracts, and claude-sonnet-5 with structured outputs. What's not boring is the discipline around it: a fixture library of real captions and real recipe pages, an eval harness that gates the extraction pipeline's merge at 90%, and a queue design whose first rule — never silently drop a save — paid for itself half a dozen separate times before launch.
| Numbers at the end | |
|---|---|
| Recipes in the box after one afternoon | 89 real + 4 seeds (now 160) |
| Extraction eval headline | 96.3% over 20 fixtures (caption / HTML / PDF) |
| Mom's 63-page scanned cookbook | 55 segments → 46 recipes, ~10s a page, nearly all auto-saved |
| Test suite | 303 tests, six queue invariants, a static write-scan on the share zone |
| API cost, entire library | ≈ $6 |
| Lines that collided across five parallel agents | 0 |
The Instagram wall, and the honest way around it
Kill the dream first: there is no API for your saved posts. Meta retired the API that could read personal accounts in December 2024; what remains is built for businesses managing their own published content. Any tutorial telling you otherwise is teaching you to scrape, which breaks monthly and violates the terms of service besides. And scraping isn't quietly viable either — the field notes record fetching a creator's public reels page with a browser user-agent: HTTP 200, 660 KB of HTML, zero post links, four "log in" prompts.
What actually works is the export. Instagram's Download Your Information tool hands you your entire saved history — and here the plan met its first big reality correction, which turned out to be a gift. The plan assumed the JSON export format and designed a politely throttled scraper to fetch each post's caption. The real export defaults to HTML, and the HTML carries the captions — 2,089 of Tommy's 2,143 saved posts arrived with their full caption embedded, plus the poster's handle. The throttled scraper the plan was built around became the exception path for 54 posts instead of the main road for two thousand. The backfill went from "an afternoon of polite scraping" to "however fast Claude answers the prompts."
A saved feed is not a recipe list. The first five captioned posts through the pipeline all came back no_recipe — correctly. They were photography reels. Tommy's 2,143 saved posts sorted into twenty collections named things like College, Dirtbikes, and Tinfoil Hat; exactly one was named Recipes. The import became two-step on the spot: a preview that parses the export and lists collections with counts (writing nothing), then a real import that queues only what you tick. Nothing is ticked by default — the safe default for a queue that costs API calls is "queue nothing until told."
Even inside the Recipes collection, the hit rate was about 40%: 102 posts became 43 recipes and 59 no_recipe rows. The misses are reels whose caption is a hook — "you NEED this 🔥" — with the recipe only spoken in the video. Those 59 rows sit in a "couldn't parse" list with their captions attached: a concrete, measured backlog for a future video-transcription feature, and the payoff of never silently dropping a save.
Contracts before code
This is the series' core pattern and it did its job here better than anywhere: agents never talk to each other; they talk to the types. Before any lane wrote a line, the Orchestrator committed contracts/types.ts and contracts/api.md — the recipe shape, the queue row, the extraction result, every API route, and the const arrays (SOURCE_TYPES, UNITS, COURSES) that drive Drizzle, Zod, and the UI from one list.
That last sentence earned its keep immediately. The pushback round caught, before any agent started: claude-sonnet-5 rejects temperature (the plan said "temperature 0"; determinism now comes from structured outputs against the Zod schema — the better tool anyway); Drizzle has no native int4range (page ranges became two int columns, because the house rule is never hand-write migration SQL); and package.json was a collision waiting to happen — four lane prompts each promised npm scripts, so every script was pre-declared in the contracts commit and the file made Orchestrator-owned.
The receipts, from the merge log: five agents, roughly 7,000 lines including tests, and not one line of TypeScript collided. Every merge conflicted only on the append-only field-notes file, and every resolution was "delete the three conflict markers." Two agents even wrote the admin cookie helper independently from the same three lines of spec — and each one's cookies verified in the other's checker on the first try.
The data model: four ideas
Everything else is bookkeeping around four decisions:
- Raw text is sacred. Every caption, page, and ingredient line is stored as written, next to the structured version. When the extraction prompt improves, re-extracting the whole library is a batch job against data you already have — and Potluck used exactly that, twice, to backfill fields that didn't exist on day one (course, nutrition) without re-fetching anything.
- Quantities stay text. "1½", "a splash", "2–3" — normalize those to floats and you've destroyed information a cook wanted. Display-time is the only time to normalize.
- Every recipe knows where it came from. Source type, URL, handle, and for PDFs the page range — so a card can say "Moms Recipes, page 24" and the review queue sits next to its evidence.
- One queue to rule the inputs. All flows converge on
pending_imports, with forward-only status transitions and an invariant the tests enforce: failed rows carry their reason, no-recipe rows keep their raw text, and nothing is ever deleted by the pipeline.
The extraction waterfall
Cheapest method first; Claude is the closer, not the whole pipeline:
URL → fetch page ├─ 1. JSON-LD present? (schema.org/Recipe) → parse, normalize, done └─ 2. Claude fallback → extract from raw text
Roughly seventy percent of recipe pages on the web embed schema.org JSON-LD — machine-readable for free, no tokens spent. Only what's left reaches Claude: Instagram captions, unmarked-up blogs, and cookbook pages. (The original plan had a third stage, the Python recipe-scrapers library, between these two. It was cut at contracts time: a second runtime wasn't worth ~15% coverage when Claude backstops everything anyway. Field-noted as a real "the plan met the stack" decision.)
The Claude stage uses structured outputs — client.messages.parse with a Zod schema — so there is no JSON scraping, and the two clauses doing the heaviest lifting are prompt rules: recipe_found: false when the text merely references a recipe ("link in bio!") — never invent — and a 0–1 confidence that routes every result: high auto-saves, middling lands in a review queue, low joins the couldn't-parse list with its raw text intact.
The eval harness — treating the prompt like code
Most AI-app tutorials ship their prompts untested. This one gated the extraction lane's merge on a number: a fixture library of 20 real inputs (hashtag walls, emoji dividers, bilingual captions, giveaway posts, JSON-LD in four flavors, a hand-built PDF) each with a known-good expected output, scored by a documented rubric — ingredient overlap as fuzzy F1, step and title terms, routing agreement. The gate was ≥90%; the first real run scored 93.6%, and after the browse-features round, 96.3%.
What the eval taught, that eyeballing never would: the model splits "mix, then bake" into more steps than a human fixture author does (style, not error — the rubric deliberately doesn't score step wording); it keeps a caption's full flowery title where the fixture wanted the dish name; and one genuinely useful miss — a family chili with vague quantities scored confidence 0.45 and fell to no_recipe where a human would send it to review. That's a threshold question, found by measurement, logged with data attached.
PDFs: segment first, extract second
One file might hold one recipe or forty, so PDFs get two passes. Pass one is a skim: pull per-page text if there's a text layer, render pages to images if it's a scan, and ask one cheap question — list every distinct recipe and its page range. Each answer becomes a queue row. Pass two is nothing new: each row runs the same extraction prompt as a caption, fed only its pages. Adding an entire input format never moved the agent contracts — the fan-out just feeds the same queue.
The showcase moment of the whole build: Mom's cookbook, 63 scanned pages of handwritten index cards on lined paper. One segmentation call over 63 page images found 55 recipe boundaries — including two-page recipes and paired entries like "Cola Cake" followed by "Icing for Cola Cake" — and the vision pass read Cola Cake, Italian Cream Cake, 125-Year-Old Walnut Pound Cake, Buttermilk Fried Chicken, Chicken and Dumplings out of the handwriting at confidences 0.72–0.90. The plan predicted handwritten scans would be the low-confidence tail; it was right about the band and wrong about the outcome — nearly all auto-saved.
It did not go that smoothly the first time. The first run rendered 63 blank pages and reported "no recipes" with zero errors — a silent JBIG2 decode failure that vision answered honestly. And the file itself wasn't a PDF by its magic bytes: macOS TextEdit had wrapped it in an RTFD container with %PDF- buried at byte 16,384. Both stories, with root causes, are in the post-mortem — they're the best material in it.
Sharing without accounts
Nobody in a family wants another login. The readable side has no accounts at all: a long crypto-random token in the URL, resolved against a share_links table with a scope (whole collection, or one recipe), a label, and a revocation flag. Be precise about what this is: unguessable, not encrypted. The link is a password; anyone it's forwarded to can view; you revoke links, not people.
Two rules keep it safe, and both are enforced by tests, not comments: share-link creation lives only in the admin zone, and nothing reachable from a token can write to the queue or trigger a Claude call. That second rule is a static scan that walks the import graph from every page in the share zone and fails the suite if any reachable module contains a database write — which meant the layer boundary ("read helpers live apart from write helpers") was a design decision made on day one instead of a merge-day surprise. A leaked link can never run up your API bill.
What reality did to the plan
A running list, because the pattern matters more than any item: every external system the plan made an assumption about corrected that assumption on contact.
| The plan assumed | Reality delivered |
|---|---|
| JSON export with post links | HTML export with full captions — better, and it deleted the scraper |
| "Your saved posts" as the unit | Collections are the unit; one of twenty was recipes |
| temperature 0 for determinism | Sonnet 5 rejects sampling params; structured outputs instead |
| int4range page ranges | Drizzle has no range type; two int columns |
| A PDF upload is a PDF | An RTFD container with the PDF at byte 16,384 |
| Rendering a scan produces the scan | 63 silently blank pages (JBIG2 WASM needs a path in Node) |
| Five windows driven by hand | Tommy: "I do not want to try to control 5 agents" — subagents instead |
The design survived every one of these because the seams were in the right places: one queue everything feeds, contracts as the single decision log, raw text kept everywhere. When the Instagram format flipped, the parser changed and nothing else did. That's what "the contracts were drawn in the right place" buys.
The pivot: five agents, then one
The build method changed twice, and the guide would be dishonest to smooth that over.
1The plan: five Claude Code windows in five git worktrees, Tommy orchestrating by hand. Dead on arrival — "I do not want to try to control 5 agents."
2The build: one Orchestrator window spawning each lane as a background subagent pointed at its worktree. The contract-first structure made this swap free — nothing in the plan depended on humans per lane. Five lanes built the MVP: ~2 hours of actual agent work, zero code conflicts, every gate green.
3The breakdown: those 2 hours took ~17 hours of wall clock, nearly all of it agents parked behind one shared session rate limit. Then came the tuning loop — Tommy testing on his phone, findings dispatched back to lane agents — and every resume reloaded 150–300K tokens of context. 92% of a session allowance went in about ten minutes of fan-out. Three agents were dispatched to chase two "bugs" that turned out to be recipes Tommy had deleted on purpose.
4The pivot: "your agents are fighting more than we are accomplishing." Everything after that — nutrition estimation, photo fetching, source labels, the full restyle, all the fixes — was one agent, small commits, Tommy testing between them. It shipped more per session token than the fan-out by an order of magnitude.
The one-line lesson, straight from the field notes: multi-agent is a build tool, not an iteration tool. Contracts-first parallelism is real — five agents, 7K lines, zero collisions proves it. But the moment a human is testing and reacting, fold back to one agent: the tuning loop is sequential by nature, and parallel context reloads are pure cost. The full accounting — with the token math, the counterfactual estimate, and Tommy's verdict unedited — is in the post-mortem.
Where it lives now
Potluck ended as a lesson in deployment honesty too. The standalone repo froze as this guide's artifact, and the live app folded into the framelogic.ai platform as a self-contained module: its own Postgres schema (one additive migration, audited by eye), its tables and routes under one path prefix, files in R2 behind a storage adapter that keeps disk for local dev, background work through the platform's cron, and a ship-dark flag so rollback is "unset one env var." The family link and the admin password work exactly as before — the family never noticed the move, which is the point.
Since moving in it has grown the features a real household asks for once they use a thing daily: a grocery list built by tapping + on recipes (per-device, printable, emailable — the share zone stays read-only), star ratings (one adjustable vote per device through a single deliberate write endpoint), and QR codes for handing the family link to someone in person. Each one arrived as a conversation, shipped in an evening, on the seams the original contracts drew.