Blog
My AI knows what's in my pantry
Playbooks

My AI knows what's in my pantry

· 9 min read · DustinFounder

I'm a dad of four. Three school-age kids, one toddler. My wife and I both work. And every Sunday morning, the same question hits: "What are we eating this week?"

It shouldn't be this hard. But when you're feeding six people with different preferences — one kid who'll eat anything, another who wants the exact same sandwich every single day, a toddler who's basically a chaos agent at the table — meal planning becomes a logistics problem. And I build systems for a living.

So I built one for dinner.

The Problem Nobody Talks About

Every meal planning app I've tried makes the same mistake: they assume you're starting from scratch. Pick a recipe. Generate a list. Go shop.

But that's not how feeding a family actually works.

Real meal planning means knowing that your fridge still has half a cantaloupe from Tuesday. That your garage overstock has three boxes of ramen and two cans of Bush's beans. That school serves pizza on Tuesdays (and your kid actually likes it, so you don't need to pack lunch).

Meet the System

I've been building TeamYou — a mobile-first AI productivity platform with a knowledge graph at its core. The idea is simple: your AI assistant should know you. Not just respond to commands, but understand your context — your family, your preferences, your patterns — and use that understanding to actually help.

Meal planning became the perfect testbed.

Here's what the system looks like today:

The Knowledge Layer

Everything lives in TeamYou as topics — structured knowledge containers that link to each other through a graph:

  • Family: Meal Preferences — Who likes what. My second-grader's sandwich spec (turkey, swiss, Oroweat, no condiments). My oldest's smoothie preferences. My middle schooler's school lunch opinions.
  • Meal Rotations — Breakfast, packed lunch, and dinner rotations. Each one maps days of the week to meals (Monday cereal, Tuesday smoothies, Wednesday poached eggs…), with every meal linked to its own recipe topic containing ingredients, prep notes, and kid ratings.
  • Food: Smoothie Recipe, Food: Pancake Recipe, Food: Taco Night, etc. — Individual recipes as standalone topics, connected to the rotations through graph edges.
  • Home: Inside Fridge, Home: Garage Fridge, Home: Freezer, Home: Pantry, Home: Overstock — Live inventory of what we actually have on hand.
  • Family: School Calendar — School year dates, holidays, early dismissals, staff dev days.
  • Food: Shopping Prefs — Which stores we use, when we typically shop.

These aren't static documents. They're a living knowledge graph that my AI manages and queries at runtime.

The Sunday Routine

Every Sunday at 8:30am, a cron job kicks off the weekly meal plan. But it doesn't just dump a template. It thinks:

  1. Seasonality check — Queries the school calendar topic to determine what kind of week this is. Normal school week? Partial (Monday holiday)? Winter break? Summer? The entire plan structure adapts. During school weeks, it pulls the school lunch menu. During breaks, it shifts to all-home meals.
  2. School lunch integration — Hits the district's public lunch menu API to pull the actual menu for the week. Same menu across all schools in the district, so it works for both the elementary and middle schooler. It filters out the daily staples (protein pack, PB&J, salad bar) and surfaces the main entrees: nachos Monday, pizza Tuesday, french toast Wednesday.
  3. Meal rotations — Pulls the active breakfast, packed lunch, and dinner rotations from TeamYou, follows the graph edges to each recipe topic, and lays out the week. Tuesday breakfast? Smoothies (linked recipe: banana, berries, spinach, yogurt, honey). Wednesday dinner? Taco night (linked recipe: ground turkey, shells, shredded cheese, salsa).
  4. Posts a living document — Drops the kickoff into our Slack meal planning channel as a structured post. Breakfast filled in. School lunch menu displayed. Dinner and packed lunch sections stubbed with next steps. Then we refine through thread discussion — and the top-level post gets edited as decisions are made.

The whole thing runs automatically. I wake up Sunday morning with a meal planning kickoff already waiting for me.

The Pantry Camera Trick

Smart fridges promise to solve this. They don't.

And he would know — his company builds computer vision into hardware products.

So instead of waiting for fridge cameras to get smarter, I just started taking photos myself.

I snap photos of my fridge shelves. My AI looks at them and tells me what it sees: "Top shelf — leftover rotisserie chicken in a glass container, two cucumbers, a bag of organic blueberries. Door — Clover milk (looks about 2/3 full), two jars of pickles, sriracha, yellow mustard."

Then it catalogs everything into the corresponding TeamYou inventory topic, tagged by location within the fridge ([Top Shelf], [Drawer], [Right Door Mid]).

We did the full kitchen fridge — 47 items identified from four photos. Then moved to the garage: overstock shelves, freezer, the works. Photos of steel shelving units full of ramen boxes, canned tomatoes, cleaning supplies. The AI sorted each item into the right inventory topic. Cleaning supplies and paper goods? Overstock. Frozen berries? Freezer. Bread in the kitchen drawer? Pantry.

It even learned our storage conventions over time. Spindrift goes in the garage overstock, not the kitchen pantry. The counter basket is for fruits, melons, potatoes, and onions.

The inventory isn't perfect — it can't read every label through a closed container, and quantities are approximate. But it's way better than trying to remember what you have when you're standing in the grocery store aisle.

The Receipt Pipeline

After a grocery run, I can share the receipt and the AI auto-sorts items into the right inventory topics:

  • Produce → Inside Fridge or Pantry (counter basket for fruits/melons)
  • Deli meats → Inside Fridge
  • Frozen berries → Freezer
  • Spindrift → Overstock
  • Paper goods → Overstock

It knows the mapping because it learned from corrections. The first time it put Spindrift in the pantry, I moved it to overstock. Now it routes beverages to the garage automatically.

How It All Connects

Here's where the knowledge graph earns its keep. When the Sunday routine runs:

  1. It checks the school calendar topic → determines it's a normal school week
  2. Pulls the school lunch menu → knows Tuesday is pizza day
  3. Checks meal preferences → my second-grader always buys school lunch on pizza day, my middle schooler prefers packed
  4. Pulls meal rotations → Tuesday breakfast is smoothies, dinner is pasta night
  5. Follows the edges to Smoothie Recipe and Pasta Night → needs bananas, berries, spinach, yogurt, penne, marinara
  6. Checks Inside Fridge inventory → we have yogurt and spinach
  7. Checks Pantry inventory → bananas in the counter basket
  8. Checks Freezer inventory → triple berry mix available
  9. No berries on the grocery list. Bananas covered. Penne in the pantry. Just need honey and marinara.

The AI knows what we have, what we need, what the kids will eat, and what the school is serving. It can reason across all of it because everything is connected in the graph.

Why This Matters Beyond My Kitchen

I built this for my family. But the pattern is universal.

The core insight: AI gets dramatically more useful when it has structured knowledge about your life. Not chat history. Not a giant prompt. A knowledge graph that represents what you know, what you have, what you prefer — and that the AI can query, update, and reason across.

Meal planning is just one application. The same architecture powers:

  • Morning agendas — daily briefings that pull from your calendar, todos, and project context
  • Smart todos — tasks that understand dependencies and can partially execute themselves
  • Project coordination — multiple AI agents sharing structured context through the graph

The missing piece in most AI assistants isn't intelligence — it's structured knowledge about your life. Not chat history. Not retrieval-augmented search over old conversations. A living knowledge graph that connects your kid's sandwich order to your pantry inventory to the school calendar — and lets the AI reason across all of it.

That's what we're building with TeamYou.

The Setup (For the Curious)

The whole system runs on:

  • TeamYou — knowledge graph platform for topics, details, edges, and semantic search; also available as the TeamYou skill for OpenClaw
  • OpenClaw — AI agent framework that runs the cron jobs and handles multi-channel messaging (TeamYou Pro)
  • Slack — the interface (a dedicated #meal-planning channel)
  • School lunch API — public district menu data (most US school districts publish these)
  • School calendar scraper — pulls the district's published calendar (start/end dates, holidays, early dismissals) into the Family: School Calendar topic
  • Vision models — for the photo-to-inventory pipeline
  • Google Calendar — vacation/trip detection for meal plan adaptation

The Sunday routine is a markdown file that describes the process. The AI reads it, queries the APIs, and executes the plan. No custom code for the meal planning logic itself — just structured instructions and a knowledge graph.

Total development time: Initial setup in the first hour, but refinement over a week of iterative building, mostly in conversation with the AI itself. We'd try something, see what worked, refine, repeat. The meal rotations were the first thing we built. Then school lunches. Then the photo inventory. Each piece made the next one more useful because the graph kept growing.

What's Next

We're still building. The grocery list generation needs to get smarter about store routing (Costco for bulk, Safeway for produce, Trader Joe's for snacks). The inventory needs an expiration-awareness layer. And the rotation library keeps growing — we're constantly adding new recipes and refining the graph connections.

But the foundation is solid. Every week, the system gets a little bit smarter because the knowledge graph gets a little bit richer. That's the compound effect of persistent context.

My AI knows what's in my pantry. And that changes everything about how it can help.


Dustin Preisler is co-founder of TeamYou. He previously led data platform engineering at Amazon Music for a decade. He lives in the Bay Area with his wife, four kids, and an AI that has opinions about sandwich bread.