About PantriBot

Why PantriBot visits

Every visit begins when someone actively asks Pantri to find a recipe. A search provider suggests relevant pages, and PantriBot may retrieve a small, bounded set of those candidates to inspect the structured recipe information they publish.

The person may have searched for a named dish, such as “Mediterranean farro bowl,” or asked for help using an ingredient, such as “use up the spinach.” They did not necessarily name your site.

What it reads and shows

PantriBot looks for structured Recipe data already published in the page, such as the recipe name, image, cooking time, yield, ingredients, instructions and nutrition. It may show those fields inside Pantri so the person can decide what to cook.

Every discovered recipe keeps a visible, clickable link to its original publisher. Pages explicitly marked isAccessibleForFree: false in their structured data are skipped.

What it does not do

• It does not run a background or scheduled crawl.
• It does not follow links on a page to discover more pages.
• Pantri does not use fetched recipe content to train machine-learning models.
• It does not build a public search index or public recipe database.
• It does not attempt to bypass a login, paywall or bot-protection challenge.

How to control access

PantriBot checks /robots.txt before every page fetch. A specific PantriBot group takes priority; if none exists, it follows the rules for User-agent: *.

To refuse PantriBot entirely, add:

User-agent: PantriBot
Disallow: /

To allow one section while refusing everything else:

User-agent: PantriBot
Disallow: /
Allow: /recipes/

Rules written only for other named agents—such as GPTBot, ClaudeBot or CCBot—do not apply to PantriBot. PantriBot does not train models or conduct an open-ended crawl, so it identifies and evaluates itself separately.

Robots policy details

• Valid policies are normally refreshed within 24 hours.
• An HTTP 4xx response for /robots.txt is treated as unavailable rules, in line with RFC 9309, so the fetch may proceed.
• A network error or server error fails closed when no earlier valid policy exists. A recently cached valid policy may be used briefly if refreshing it fails.
• Published Crawl-delay values are honoured. If the requested delay does not fit within the page-fetch budget, the source is skipped.

Verifying requests

User-agent strings can be copied by anyone. Pantri does not currently publish a verified outbound IP list or reverse-DNS scheme, so the user-agent by itself is not proof that a request came from Pantri. This page will be updated if a verification method is introduced.