Build a Recipe Recommendation Engine with Bright Data MCP & OpenAI

Input a recipe search URL → Bright Data MCP paginated scraping → GPT-4o-mini structured extraction → loop over single pages → output JSON files and Webhook notifications

Who

Content creators, nutritionists, recipe-aggregator developers, or automation engineers who want to batch-scrape and structure recipe data in n8n using MCP and LLMs.

You get

Automatically convert any public recipe search page into structured JSON (title, URL, category, rating, serves, dietary options, etc.) with pagination traversal, per-page looped scraping, and delivery via Webhook or local file.

You need

n8n · local disk

Steps

1. Configure inputs and credentials

   After importing the template into n8n, open the 'Set the Recipe Extract URL' node (Edit Fields/Set) and update three fields: url (target recipe search page, e.g. https://realfood.tesco.com/search.html?search=tomato%20pasta&sortby=Rating), base_url (root of the search page), and webhook_url (your notification endpoint). Also confirm that the Bright Data MCP Client Header Auth credential is set (Value = Bearer + Web Unlocker Token) and the OpenAI API credential is configured.

   Setup guidance · Editorial suggestion。Setup assistant starter; guides user through credential and input configuration

   Copy prompt

   Please complete the following setup in n8n:
   1. After importing the template, locate the 'Set the Recipe Extract URL' node and set url, base_url, and webhook_url.
   2. In Credentials, configure mcpClientApi: choose Header Auth, and set Value to Bearer YOUR_BRIGHT_DATA_WEB_UNLOCKER_TOKEN.
   3. Configure the openAiApi credential.
   4. Verify that the local path in the 'Write the structured content to disk' node (e.g. d:\Recipe-Structured-Data-{{ $('Loop Over Items').item.json.page_number }}.json) is writable on your system.

2. Scrape search page and extract paginated links with Bright Data MCP

   Once triggered, the 'Bright Data MCP Client For Recipe Extract' node calls scrape_as_html to fetch the search-page HTML. Then the 'Paginated Data Extract' LLM Chain node invokes GPT-4o-mini, guided by the 'Structured Output Parser for Paginated Data' JSON Schema, to extract an array of pagination links (page_number and link). The system message is "You are an expert data miner".

   Execution prompt · Source text。Paginated Data Extract node text field (LLM Chain user input)

   Copy prompt

   =Extract the paginated data from the provided content. 
   
   Here's the content : {{ $json.result.content[0].text }}
   
   Append the link with the Base URL as {{ $('Set the Recipe Extract URL').item.json.base_url }}

   Execution prompt · Source text。Paginated Data Extract node messages.messageValues[0].message (system prompt)

   Copy prompt

   You are an expert data miner

   Execution prompt · Source text。Structured Output Parser for Paginated Data node jsonSchemaExample field

   Copy prompt

   [{
   	"page_number": "page number in int",
   	"link": "link to the web content"
   }]

3. Convert output and loop over paginated links

   The 'Code to output the array of paginated info' node runs `return $input.first().json.output` to normalize the LLM parse result into a standard array. Then the 'Loop Over Items' node emits each pagination link individually for downstream processing.

   Execution prompt · Source text。Code to output the array of paginated info node jsCode field

   Copy prompt

   return $input.first().json.output

4. Per-page scrape and wait inside the loop

   Each link enters the 'Wait' node (10-second delay), then the 'Bright Data MCP Client For Recipe Extract Within The Loop' node calls scrape_as_html again to fetch the individual recipe-page HTML. The node uses the expression `{{ $json.link }}` to pass the URL dynamically.

   Execution prompt · Source text。Bright Data MCP Client For Recipe Extract Within The Loop node toolParameters field

   Copy prompt

   ={
      "url": "{{ $json.link }}"
   }

5. Structured recipe extraction with GPT-4o-mini

   The 'Structured Recipe Data Extract' LLM Chain node calls GPT-4o-mini with the same system message "You are an expert data miner" and user input "=Extract search data. Here's the content : Output the data in JSON". The output is parsed by the 'Structured Output Parser for Recipe Data Extract' against a full JSON Schema into structured fields: pageTitle, searchTerms, sortBy, numberOfResults, numberOfPages, numberOfFirstResult, numberOfLastResult, and a recipes array (title, url, category, rating, serves, dietaryOptions, isHealthy).

   Execution prompt · Source text。Structured Recipe Data Extract node text field (LLM Chain user input)

   Copy prompt

   =Extract search data. Here's the content : 
   Output the data in JSON

   Execution prompt · Source text。Structured Recipe Data Extract node messages.messageValues[0].message (system prompt)

   Copy prompt

   You are an expert data miner

   Execution prompt · Source text。Structured Output Parser for Recipe Data Extract node inputSchema field

   Copy prompt

   {
     "$schema": "http://json-schema.org/schema#",
     "title": "TescoRecipeSearchResults",
     "type": "object",
     "properties": {
       "pageTitle": { "type": "string" },
       "searchTerms": { "type": "string" },
       "sortBy": { "type": "string" },
       "numberOfResults": { "type": "integer" },
       "numberOfPages": { "type": "integer" },
       "numberOfFirstResult": { "type": "integer" },
       "numberOfLastResult": { "type": "integer" },
       "recipes": {
         "type": "array",
         "items": {
           "type": "object",
           "properties": {
             "title": { "type": "string" },
             "url": { "type": "string" },
             "category": { "type": "string" },
             "rating": { "type": "number" },
             "serves": { "type": "integer" },
             "dietaryOptions": {
               "type": "array",
               "items": { "type": "string" }
             },
             "isHealthy": { "type": "boolean" }
           },
           "required": [
             "title",
             "url",
             "category",
             "rating",
             "serves",
             "dietaryOptions"
           ],
           "additionalProperties": false
         }
       }
     },
     "required": [
       "pageTitle",
       "searchTerms",
       "sortBy",
       "numberOfResults",
       "numberOfPages",
       "numberOfFirstResult",
       "numberOfLastResult",
       "recipes"
     ],
     "additionalProperties": false
   }

6. Webhook push of structured results

   The 'Webhook Notification for Data Extract Within the Loop' node POSTs `{{ $json.output.toJsonString() }}` as the recipe_content field in multipart/form-data format to the configured webhook_url. This can feed Slack, Discord, or internal APIs.

7. Save JSON to local disk

   The 'Create a binary data' node converts the JSON output to base64 binary, then the 'Write the structured content to disk' node writes it to a local file whose name includes the page number variable: `=d:\Recipe-Structured-Data-{{ $('Loop Over Items').item.json.page_number }}.json`. Adjust the path for your operating system.

   Execution prompt · Source text。Create a binary data node functionCode field

   Copy prompt

   items[0].binary = {
     data: {
       data: new Buffer(JSON.stringify(items[0].json, null, 2)).toString('base64')
     }
   };
   return items;

Please complete the following setup in n8n:
1. After importing the template, locate the 'Set the Recipe Extract URL' node and set url, base_url, and webhook_url.
2. In Credentials, configure mcpClientApi: choose Header Auth, and set Value to Bearer YOUR_BRIGHT_DATA_WEB_UNLOCKER_TOKEN.
3. Configure the openAiApi credential.
4. Verify that the local path in the 'Write the structured content to disk' node (e.g. d:\Recipe-Structured-Data-{{ $('Loop Over Items').item.json.page_number }}.json) is writable on your system.