{
  "name": "AI-Powered Self-Healing Data Pipeline with Anomaly Detection and Auto-Remediation",
  "nodes": [
    {
      "id": "d9dbb356-61f3-4caf-a35b-d533cce5833b",
      "name": "Sticky Note",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        0,
        0
      ],
      "parameters": {
        "width": 480,
        "height": 896,
        "content": "## Untitled workflow\n\n### How it works\n\nThis workflow monitors data pipelines for anomalies using statistical and ML-based methods.\nIt triggers every 15 minutes to fetch and analyze the latest data batch.\nAnomalies are classified and remediation plans are generated using AI.\nRemediation actions are executed and validated for success.\nCritical issues trigger PagerDuty alerts, while failures escalate to human review.\nSuccessful remediations and incidents are logged for further analysis.\n\n### Setup steps\n\n- [ ] Configure PostgreSQL credentials for fetching and logging data.\n- [ ] Set up Anthropic API credentials for anomaly classification and remediation planning.\n- [ ] Configure Slack credentials for notifications and human review escalation.\n- [ ] Set up PagerDuty API credentials for critical alerts.\n- [ ] Configure DataDog API credentials for metrics and baseline updates.\n\n### Customization\n\nAdjust the `nullThreshold`, `outlierIQRMultiplier`, and `slackChannel` in the Workflow Configuration node based on pipeline requirements. Modify the SQL queries in PostgreSQL nodes to match your schema."
      },
      "typeVersion": 1
    },
    {
      "id": "cae273a4-8934-4f75-992b-533691af3789",
      "name": "Sticky Note1",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        560,
        368
      ],
      "parameters": {
        "color": 7,
        "width": 416,
        "height": 320,
        "content": "## Trigger and workflow configuration\n\nInitializes the workflow schedule and sets configuration parameters for thresholds, channels, and database tables."
      },
      "typeVersion": 1
    },
    {
      "id": "8e6641e1-e324-457c-81f4-49e82e96bf47",
      "name": "Sticky Note2",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        1008,
        368
      ],
      "parameters": {
        "color": 7,
        "width": 416,
        "height": 384,
        "content": "## Fetch and prepare data batch\n\nRetrieves the latest data batch from PostgreSQL and fetches historical baseline metrics for comparison."
      },
      "typeVersion": 1
    },
    {
      "id": "d644bc3b-4da8-4e48-8489-9d0004afb31d",
      "name": "Sticky Note3",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        1456,
        464
      ],
      "parameters": {
        "color": 7,
        "width": 864,
        "height": 304,
        "content": "## Baseline comparison and trend analysis\n\nCompares current data with historical baselines and calculates trends using ML-based anomaly scoring."
      },
      "typeVersion": 1
    },
    {
      "id": "977ff1af-5034-46d9-a467-053af31a7b66",
      "name": "Sticky Note4",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        2352,
        320
      ],
      "parameters": {
        "color": 7,
        "height": 384,
        "content": "## Statistical quality checks\n\nPerforms statistical quality checks on the fetched data batch to identify potential issues."
      },
      "typeVersion": 1
    },
    {
      "id": "1b546004-ad5e-42c6-9ca7-4ecd53c43258",
      "name": "Sticky Note5",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        2624,
        416
      ],
      "parameters": {
        "color": 7,
        "width": 992,
        "height": 416,
        "content": "## Anomaly detection and severity assessment\n\nDetects anomalies and assesses their severity. Routes critical issues to PagerDuty for immediate attention."
      },
      "typeVersion": 1
    },
    {
      "id": "02a7b9bd-7bd8-4dd3-b4d3-205c9b1f7b63",
      "name": "Sticky Note6",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        2800,
        112
      ],
      "parameters": {
        "color": 7,
        "width": 992,
        "height": 272,
        "content": "## AI-based anomaly classification and planning\n\nClassifies detected anomalies and generates remediation plans using AI models."
      },
      "typeVersion": 1
    },
    {
      "id": "8a0633cd-6b5c-4781-a8b9-9a3923302671",
      "name": "Sticky Note7",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        3952,
        400
      ],
      "parameters": {
        "color": 7,
        "width": 416,
        "height": 352,
        "content": "## Remediation history and success rate calculation\n\nQueries historical remediation data to calculate success rates and inform remediation actions."
      },
      "typeVersion": 1
    },
    {
      "id": "3a6f91a9-2dc1-4e16-83f6-19bfd5d1bc08",
      "name": "Sticky Note8",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        4400,
        288
      ],
      "parameters": {
        "color": 7,
        "height": 448,
        "content": "## Remediation action routing\n\nRoutes remediation actions based on the AI-generated plan and historical success rates."
      },
      "typeVersion": 1
    },
    {
      "id": "012a6714-4d0f-4e98-a105-5fbd767e4f63",
      "name": "Sticky Note9",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        4672,
        0
      ],
      "parameters": {
        "color": 7,
        "height": 976,
        "content": "## Execute remediation actions\n\nExecutes specific remediation actions such as filling nulls, capping outliers, reformatting schemas, or retrying upstream sources."
      },
      "typeVersion": 1
    },
    {
      "id": "79bfaacb-de3b-4bea-8067-bfbce387d741",
      "name": "Sticky Note10",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        4944,
        368
      ],
      "parameters": {
        "color": 7,
        "width": 416,
        "height": 320,
        "content": "## Merge and validate remediated data\n\nMerges remediated data from all branches and re-runs quality checks to validate remediation success."
      },
      "typeVersion": 1
    },
    {
      "id": "301e9a62-8cc3-4416-a141-d611aae565e0",
      "name": "Sticky Note11",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        5392,
        256
      ],
      "parameters": {
        "color": 7,
        "width": 640,
        "height": 608,
        "content": "## Remediation outcome handling\n\nHandles the outcome of remediation attempts, either sending success notifications or escalating failures for human review."
      },
      "typeVersion": 1
    },
    {
      "id": "49e95793-c995-41e5-a8d9-89ac42f49205",
      "name": "Sticky Note12",
      "type": "n8n-nodes-base.stickyNote",
      "position": [
        6064,
        272
      ],
      "parameters": {
        "color": 7,
        "width": 1088,
        "height": 512,
        "content": "## Incident logging and reporting\n\nLogs remediation outcomes to PostgreSQL and generates comprehensive reports. Sends metrics to DataDog for monitoring."
      },
      "typeVersion": 1
    },
    {
      "id": "1ef7f20b-a647-4005-8edb-57ab592d450d",
      "name": "When Scheduled Run Starts",
      "type": "n8n-nodes-base.scheduleTrigger",
      "position": [
        608,
        528
      ],
      "parameters": {
        "rule": {
          "interval": [
            {
              "field": "minutes",
              "minutesInterval": 15
            }
          ]
        }
      },
      "typeVersion": 1.3
    },
    {
      "id": "c04084df-c8c0-470c-a441-209e84cf092f",
      "name": "Set Pipeline Parameters",
      "type": "n8n-nodes-base.set",
      "position": [
        832,
        528
      ],
      "parameters": {
        "options": {},
        "assignments": {
          "assignments": [
            {
              "id": "id-1",
              "name": "nullThreshold",
              "type": "number",
              "value": 0.1
            },
            {
              "id": "id-2",
              "name": "outlierIQRMultiplier",
              "type": "number",
              "value": 1.5
            },
            {
              "id": "id-3",
              "name": "slackChannel",
              "type": "string",
              "value": "<__PLACEHOLDER_VALUE__Slack channel ID for notifications__>"
            },
            {
              "id": "id-4",
              "name": "postgresTable",
              "type": "string",
              "value": "data_pipeline_incidents"
            },
            {
              "id": "id-5",
              "name": "pagerDutyApiKey",
              "type": "string",
              "value": "<__PLACEHOLDER_VALUE__PagerDuty API key__>"
            },
            {
              "id": "id-6",
              "name": "datadogApiKey",
              "type": "string",
              "value": "<__PLACEHOLDER_VALUE__DataDog API key__>"
            },
            {
              "id": "id-7",
              "name": "baselineTable",
              "type": "string",
              "value": "data_pipeline_baseline"
            },
            {
              "id": "id-8",
              "name": "anomalyScoreThreshold",
              "type": "number",
              "value": 0.75
            }
          ]
        },
        "includeOtherFields": true
      },
      "typeVersion": 3.4
    },
    {
      "id": "166ad159-ed18-43c8-a790-fd4aaefd8384",
      "name": "Fetch Latest Data from PostgreSQL",
      "type": "n8n-nodes-base.postgres",
      "position": [
        1056,
        528
      ],
      "parameters": {
        "query": "SELECT * FROM <__PLACEHOLDER_VALUE__source_table_name__> WHERE created_at > NOW() - INTERVAL '15 minutes' ORDER BY created_at DESC",
        "options": {},
        "operation": "executeQuery"
      },
      "typeVersion": 2.6
    },
    {
      "id": "851922a3-e811-4d61-937e-eb8e5e1735bc",
      "name": "Run Statistical Quality Checks",
      "type": "n8n-nodes-base.code",
      "position": [
        2400,
        528
      ],
      "parameters": {
        "jsCode": "// Statistical Quality Checks for Data Pipeline\n// Performs comprehensive data quality analysis\n\nconst items = $input.all();\nconst config = $('Set Pipeline Parameters').first().json;\n\n// Default thresholds (can be overridden by config)\nconst thresholds = {\n  nullRateThreshold: config.nullRateThreshold || 0.1,\n  outlierThreshold: config.outlierThreshold || 1.5,\n  minRecords: config.minRecords || 10\n};\n\nconst results = {\n  anomalyDetected: false,\n  anomalyType: [],\n  affectedColumns: [],\n  statistics: {},\n  totalRecords: items.length,\n  timestamp: new Date().toISOString()\n};\n\n// Check minimum records\nif (items.length < thresholds.minRecords) {\n  results.anomalyDetected = true;\n  results.anomalyType.push('INSUFFICIENT_DATA');\n  return [results];\n}\n\n// Get all column names from first item\nconst columns = Object.keys(items[0].json);\n\n// Analyze each column\nfor (const column of columns) {\n  const values = items.map(item => item.json[column]);\n  const nonNullValues = values.filter(v => v !== null && v !== undefined && v !== '');\n  \n  const stats = {\n    totalCount: values.length,\n    nullCount: values.length - nonNullValues.length,\n    nullRate: (values.length - nonNullValues.length) / values.length,\n    dataType: typeof nonNullValues[0]\n  };\n  \n  // Null rate check\n  if (stats.nullRate > thresholds.nullRateThreshold) {\n    results.anomalyDetected = true;\n    if (!results.anomalyType.includes('HIGH_NULL_RATE')) {\n      results.anomalyType.push('HIGH_NULL_RATE');\n    }\n    results.affectedColumns.push(column);\n  }\n  \n  // Numeric column analysis\n  if (stats.dataType === 'number' && nonNullValues.length > 0) {\n    const numericValues = nonNullValues.filter(v => typeof v === 'number').sort((a, b) => a - b);\n    \n    if (numericValues.length > 0) {\n      // Calculate quartiles for IQR method\n      const q1Index = Math.floor(numericValues.length * 0.25);\n      const q3Index = Math.floor(numericValues.length * 0.75);\n      const q1 = numericValues[q1Index];\n      const q3 = numericValues[q3Index];\n      const iqr = q3 - q1;\n      \n      const lowerBound = q1 - (thresholds.outlierThreshold * iqr);\n      const upperBound = q3 + (thresholds.outlierThreshold * iqr);\n      \n      const outliers = numericValues.filter(v => v < lowerBound || v > upperBound);\n      \n      stats.min = Math.min(...numericValues);\n      stats.max = Math.max(...numericValues);\n      stats.mean = numericValues.reduce((a, b) => a + b, 0) / numericValues.length;\n      stats.median = numericValues[Math.floor(numericValues.length / 2)];\n      stats.q1 = q1;\n      stats.q3 = q3;\n      stats.iqr = iqr;\n      stats.outlierCount = outliers.length;\n      stats.outlierRate = outliers.length / numericValues.length;\n      \n      // Outlier detection\n      if (stats.outlierRate > 0.05) { // More than 5% outliers\n        results.anomalyDetected = true;\n        if (!results.anomalyType.includes('OUTLIERS_DETECTED')) {\n          results.anomalyType.push('OUTLIERS_DETECTED');\n        }\n        if (!results.affectedColumns.includes(column)) {\n          results.affectedColumns.push(column);\n        }\n      }\n    }\n  }\n  \n  // String column analysis\n  if (stats.dataType === 'string' && nonNullValues.length > 0) {\n    const uniqueValues = [...new Set(nonNullValues)];\n    stats.uniqueCount = uniqueValues.length;\n    stats.uniqueRate = uniqueValues.length / nonNullValues.length;\n    \n    // Value distribution\n    const distribution = {};\n    nonNullValues.forEach(v => {\n      distribution[v] = (distribution[v] || 0) + 1;\n    });\n    stats.topValues = Object.entries(distribution)\n      .sort((a, b) => b[1] - a[1])\n      .slice(0, 5)\n      .map(([value, count]) => ({ value, count, percentage: count / nonNullValues.length }));\n  }\n  \n  results.statistics[column] = stats;\n}\n\n// Schema validation - check for unexpected schema changes\nif (config.expectedSchema) {\n  const expectedColumns = Object.keys(config.expectedSchema);\n  const actualColumns = columns;\n  \n  const missingColumns = expectedColumns.filter(col => !actualColumns.includes(col));\n  const extraColumns = actualColumns.filter(col => !expectedColumns.includes(col));\n  \n  if (missingColumns.length > 0 || extraColumns.length > 0) {\n    results.anomalyDetected = true;\n    if (!results.anomalyType.includes('SCHEMA_MISMATCH')) {\n      results.anomalyType.push('SCHEMA_MISMATCH');\n    }\n    results.schemaMismatch = {\n      missingColumns,\n      extraColumns\n    };\n  }\n}\n\nreturn [results];"
      },
      "typeVersion": 2
    },
    {
      "id": "cc04f670-4e5e-411f-911c-8a4aca3aa6bb",
      "name": "Check If Anomaly Detected",
      "type": "n8n-nodes-base.if",
      "position": [
        2672,
        528
      ],
      "parameters": {
        "options": {},
        "conditions": {
          "options": {
            "leftValue": "",
            "caseSensitive": true,
            "typeValidation": "strict"
          },
          "combinator": "and",
          "conditions": [
            {
              "id": "id-1",
              "operator": {
                "type": "boolean",
                "operation": "true"
              },
              "leftValue": "={{ $('Run Statistical Quality Checks').item.json.anomalyDetected }}"
            }
          ]
        }
      },
      "typeVersion": 2.3
    },
    {
      "id": "149b2c4a-33e7-4988-aa5a-05b77f3a6756",
      "name": "AI Anomaly Classifier Agent",
      "type": "@n8n/n8n-nodes-langchain.anthropic",
      "position": [
        2848,
        224
      ],
      "parameters": {
        "modelId": {
          "__rl": true,
          "mode": "id",
          "value": "claude-3-5-sonnet-20241022"
        },
        "options": {
          "system": "You are a data quality expert specializing in anomaly detection and classification. Analyze data quality issues and provide accurate classifications with severity assessments."
        },
        "messages": {
          "values": [
            {
              "content": "=Analyze this data quality anomaly and classify it:\n\nAnomaly Type: {{ $('Run Statistical Quality Checks').first().json.anomalyType }}\nAffected Columns: {{ $('Run Statistical Quality Checks').first().json.affectedColumns }}\nStatistics: {{ JSON.stringify($('Run Statistical Quality Checks').first().json.statistics) }}\nTotal Records: {{ $('Run Statistical Quality Checks').first().json.totalRecords }}\n\nClassify this anomaly into one of these categories:\n- schema_drift: Unexpected changes in data structure or types\n- missing_data: High rates of null or missing values\n- outliers: Statistical anomalies in numeric data\n- upstream_failure: Issues with data source or pipeline\n\nProvide your response as JSON with:\n{\n  \"classification\": \"category_name\",\n  \"severity\": \"low|medium|high|critical\",\n  \"confidence\": 0-100,\n  \"explanation\": \"detailed reasoning\"\n}"
            }
          ]
        }
      },
      "typeVersion": 1
    },
    {
      "id": "95ff1c6e-1df8-4a6c-b889-7a8b06d56e64",
      "name": "AI Remediation Planner Agent",
      "type": "@n8n/n8n-nodes-langchain.anthropic",
      "position": [
        3648,
        224
      ],
      "parameters": {
        "modelId": {
          "__rl": true,
          "mode": "id",
          "value": "claude-3-5-sonnet-20241022"
        },
        "options": {
          "system": "You are a data engineering expert specializing in automated remediation strategies. Suggest practical, safe remediation actions for data quality issues based on anomaly classification and historical patterns."
        },
        "messages": {
          "values": [
            {
              "content": "=Based on this classified anomaly, suggest a specific remediation action:\n\nClassification: {{ $json.classification }}\nSeverity: {{ $json.severity }}\nAnomaly Details: {{ $json.explanation }}\nAffected Columns: {{ $('Run Statistical Quality Checks').first().json.affectedColumns }}\nStatistics: {{ JSON.stringify($('Run Statistical Quality Checks').first().json.statistics) }}\n\nSuggest ONE of these remediation actions:\n- fill_nulls: Fill missing values with appropriate defaults\n- cap_outliers: Cap extreme values to acceptable ranges\n- reformat_schema: Fix schema mismatches or type issues\n- retry_upstream: Retry fetching data from source\n\nProvide:\n1. Action (one of the above)\n2. Parameters needed for the action\n3. Expected success probability (0-100)\n4. Reasoning\n\nReturn as JSON: {\"action\": \"...\", \"parameters\": {...}, \"successProbability\": ..., \"reasoning\": \"...\"}"
            }
          ]
        }
      },
      "typeVersion": 1
    },
    {
      "id": "417fec55-ce0c-4f94-941e-761c08a1cd6b",
      "name": "Route by Remediation Type",
      "type": "n8n-nodes-base.switch",
      "position": [
        4448,
        496
      ],
      "parameters": {
        "rules": {
          "values": [
            {
              "outputKey": "fill_nulls",
              "conditions": {
                "options": {
                  "leftValue": "",
                  "caseSensitive": true,
                  "typeValidation": "strict"
                },
                "combinator": "and",
                "conditions": [
                  {
                    "operator": {
                      "type": "string",
                      "operation": "equals"
                    },
                    "leftValue": "={{ $json.action }}",
                    "rightValue": "fill_nulls"
                  }
                ]
              },
              "renameOutput": true
            },
            {
              "outputKey": "cap_outliers",
              "conditions": {
                "options": {
                  "leftValue": "",
                  "caseSensitive": true,
                  "typeValidation": "strict"
                },
                "combinator": "and",
                "conditions": [
                  {
                    "operator": {
                      "type": "string",
                      "operation": "equals"
                    },
                    "leftValue": "={{ $json.action }}",
                    "rightValue": "cap_outliers"
                  }
                ]
              },
              "renameOutput": true
            },
            {
              "outputKey": "reformat_schema",
              "conditions": {
                "options": {
                  "leftValue": "",
                  "caseSensitive": true,
                  "typeValidation": "strict"
                },
                "combinator": "and",
                "conditions": [
                  {
                    "operator": {
                      "type": "string",
                      "operation": "equals"
                    },
                    "leftValue": "={{ $json.action }}",
                    "rightValue": "reformat_schema"
                  }
                ]
              },
              "renameOutput": true
            },
            {
              "outputKey": "retry_upstream",
              "conditions": {
                "options": {
                  "leftValue": "",
                  "caseSensitive": true,
                  "typeValidation": "strict"
                },
                "combinator": "and",
                "conditions": [
                  {
                    "operator": {
                      "type": "string",
                      "operation": "equals"
                    },
                    "leftValue": "={{ $json.action }}",
                    "rightValue": "retry_upstream"
                  }
                ]
              },
              "renameOutput": true
            }
          ]
        },
        "options": {}
      },
      "typeVersion": 3.4
    },
    {
      "id": "8ec27c7c-d895-4055-aef6-57af8abdd50b",
      "name": "Remediate Missing Values",
      "type": "n8n-nodes-base.code",
      "position": [
        4720,
        240
      ],
      "parameters": {
        "jsCode": "// Get the original data from Statistical Quality Checks node\nconst originalData = $('Run Statistical Quality Checks').all();\n\n// Function to calculate median for numeric arrays\nfunction calculateMedian(arr) {\n  const sorted = arr.filter(val => val !== null && val !== undefined).sort((a, b) => a - b);\n  if (sorted.length === 0) return 0;\n  const mid = Math.floor(sorted.length / 2);\n  return sorted.length % 2 !== 0 ? sorted[mid] : (sorted[mid - 1] + sorted[mid]) / 2;\n}\n\n// Function to calculate mode for categorical data\nfunction calculateMode(arr) {\n  const filtered = arr.filter(val => val !== null && val !== undefined);\n  if (filtered.length === 0) return '';\n  const frequency = {};\n  let maxFreq = 0;\n  let mode = filtered[0];\n  \n  filtered.forEach(val => {\n    frequency[val] = (frequency[val] || 0) + 1;\n    if (frequency[val] > maxFreq) {\n      maxFreq = frequency[val];\n      mode = val;\n    }\n  });\n  \n  return mode;\n}\n\n// Function to determine if a value is numeric\nfunction isNumeric(val) {\n  return typeof val === 'number' && !isNaN(val);\n}\n\n// Analyze data structure and identify columns with nulls\nconst columnsWithNulls = [];\nconst columnTypes = {};\nconst columnValues = {};\n\nif (originalData.length > 0) {\n  const sampleItem = originalData[0].json;\n  const columns = Object.keys(sampleItem);\n  \n  // Collect all values for each column\n  columns.forEach(col => {\n    columnValues[col] = originalData.map(item => item.json[col]);\n    \n    // Check if column has nulls\n    const hasNulls = columnValues[col].some(val => val === null || val === undefined);\n    if (hasNulls) {\n      columnsWithNulls.push(col);\n    }\n    \n    // Determine column type based on non-null values\n    const nonNullValues = columnValues[col].filter(val => val !== null && val !== undefined);\n    if (nonNullValues.length > 0) {\n      if (nonNullValues.every(val => isNumeric(val))) {\n        columnTypes[col] = 'numeric';\n      } else if (typeof nonNullValues[0] === 'string') {\n        columnTypes[col] = 'text';\n      } else {\n        columnTypes[col] = 'categorical';\n      }\n    }\n  });\n}\n\n// Calculate fill values for each column with nulls\nconst fillValues = {};\nconst remediationDetails = [];\n\ncolumnsWithNulls.forEach(col => {\n  if (columnTypes[col] === 'numeric') {\n    fillValues[col] = calculateMedian(columnValues[col]);\n    remediationDetails.push({\n      column: col,\n      type: 'numeric',\n      fillMethod: 'median',\n      fillValue: fillValues[col]\n    });\n  } else if (columnTypes[col] === 'categorical') {\n    fillValues[col] = calculateMode(columnValues[col]);\n    remediationDetails.push({\n      column: col,\n      type: 'categorical',\n      fillMethod: 'mode',\n      fillValue: fillValues[col]\n    });\n  } else {\n    fillValues[col] = '';\n    remediationDetails.push({\n      column: col,\n      type: 'text',\n      fillMethod: 'empty_string',\n      fillValue: fillValues[col]\n    });\n  }\n});\n\n// Apply remediation to the data\nconst remediatedData = originalData.map(item => {\n  const newItem = { ...item.json };\n  let nullsFixed = 0;\n  \n  columnsWithNulls.forEach(col => {\n    if (newItem[col] === null || newItem[col] === undefined) {\n      newItem[col] = fillValues[col];\n      nullsFixed++;\n    }\n  });\n  \n  return {\n    json: {\n      ...newItem,\n      _remediationApplied: true,\n      _remediationType: 'fill_nulls',\n      _nullsFixed: nullsFixed\n    }\n  };\n});\n\n// Return remediated data with metadata\nreturn [\n  {\n    json: {\n      remediatedData: remediatedData.map(item => item.json),\n      remediationApplied: true,\n      remediationType: 'fill_nulls',\n      columnsRemediated: columnsWithNulls,\n      remediationDetails: remediationDetails,\n      totalRecords: remediatedData.length,\n      timestamp: new Date().toISOString()\n    }\n  }\n];"
      },
      "typeVersion": 2
    },
    {
      "id": "eca247e2-c66e-48e8-9f6c-0979dedf2aef",
      "name": "Remediate Outliers",
      "type": "n8n-nodes-base.code",
      "position": [
        4720,
        432
      ],
      "parameters": {
        "jsCode": "// Get the original data from Statistical Quality Checks node\nconst originalData = $('Run Statistical Quality Checks').all();\nconst remediatedItems = [];\nconst cappingDetails = [];\n\n// Process each item\nfor (const item of originalData) {\n  const data = item.json;\n  const remediatedData = { ...data };\n  \n  // Identify numeric columns\n  const numericColumns = [];\n  for (const key in data) {\n    if (typeof data[key] === 'number' && !isNaN(data[key])) {\n      numericColumns.push(key);\n    }\n  }\n  \n  // Calculate statistics for each numeric column across all items\n  const columnStats = {};\n  \n  for (const column of numericColumns) {\n    const values = originalData\n      .map(item => item.json[column])\n      .filter(val => typeof val === 'number' && !isNaN(val))\n      .sort((a, b) => a - b);\n    \n    if (values.length > 0) {\n      // Calculate Q1, Q3, and IQR\n      const q1Index = Math.floor(values.length * 0.25);\n      const q3Index = Math.floor(values.length * 0.75);\n      \n      const q1 = values[q1Index];\n      const q3 = values[q3Index];\n      const iqr = q3 - q1;\n      \n      const lowerBound = q1 - 1.5 * iqr;\n      const upperBound = q3 + 1.5 * iqr;\n      \n      columnStats[column] = {\n        q1,\n        q3,\n        iqr,\n        lowerBound,\n        upperBound\n      };\n    }\n  }\n  \n  // Cap outliers in the current item\n  for (const column of numericColumns) {\n    if (columnStats[column]) {\n      const originalValue = remediatedData[column];\n      const { lowerBound, upperBound } = columnStats[column];\n      \n      if (originalValue < lowerBound) {\n        remediatedData[column] = lowerBound;\n        cappingDetails.push({\n          column,\n          originalValue,\n          cappedValue: lowerBound,\n          reason: 'below_lower_bound'\n        });\n      } else if (originalValue > upperBound) {\n        remediatedData[column] = upperBound;\n        cappingDetails.push({\n          column,\n          originalValue,\n          cappedValue: upperBound,\n          reason: 'above_upper_bound'\n        });\n      }\n    }\n  }\n  \n  remediatedItems.push({\n    json: remediatedData\n  });\n}\n\n// Return remediated data with metadata\nreturn [{\n  json: {\n    remediatedData: remediatedItems.map(item => item.json),\n    remediationApplied: true,\n    remediationType: 'cap_outliers',\n    cappingDetails: cappingDetails,\n    totalItemsProcessed: remediatedItems.length,\n    totalValuesCapped: cappingDetails.length,\n    timestamp: new Date().toISOString()\n  }\n}];"
      },
      "typeVersion": 2
    },
    {
      "id": "571ffdbd-387c-4e35-9c42-67badf38e3f8",
      "name": "Remediate Schema Format",
      "type": "n8n-nodes-base.code",
      "position": [
        4720,
        624
      ],
      "parameters": {
        "jsCode": "// Get original data from Statistical Quality Checks node\nconst originalData = $('Run Statistical Quality Checks').all();\nconst remediationPlan = $json;\n\n// Helper function to convert value to appropriate type\nfunction convertToType(value, targetType) {\n  if (value === null || value === undefined) {\n    return getDefaultForType(targetType);\n  }\n  \n  switch(targetType) {\n    case 'number':\n      const num = Number(value);\n      return isNaN(num) ? 0 : num;\n    \n    case 'string':\n      return String(value);\n    \n    case 'boolean':\n      if (typeof value === 'boolean') return value;\n      if (typeof value === 'string') {\n        return value.toLowerCase() === 'true' || value === '1';\n      }\n      return Boolean(value);\n    \n    case 'date':\n      const date = new Date(value);\n      return isNaN(date.getTime()) ? new Date().toISOString() : date.toISOString();\n    \n    default:\n      return value;\n  }\n}\n\n// Helper function to get default value for type\nfunction getDefaultForType(type) {\n  switch(type) {\n    case 'number': return 0;\n    case 'string': return '';\n    case 'boolean': return false;\n    case 'date': return new Date().toISOString();\n    default: return null;\n  }\n}\n\n// Expected schema definition (can be customized based on your data)\nconst expectedSchema = {\n  id: 'number',\n  timestamp: 'date',\n  value: 'number',\n  status: 'string',\n  is_active: 'boolean',\n  metric_name: 'string',\n  count: 'number'\n};\n\n// Process each item and reformat according to schema\nconst remediatedItems = originalData.map(item => {\n  const data = item.json;\n  const reformattedData = {};\n  const schemaChanges = [];\n  \n  // Apply expected schema\n  for (const [field, expectedType] of Object.entries(expectedSchema)) {\n    if (!(field in data)) {\n      // Missing column - add with default\n      reformattedData[field] = getDefaultForType(expectedType);\n      schemaChanges.push(`Added missing field '${field}' with default value`);\n    } else {\n      const originalValue = data[field];\n      const originalType = typeof originalValue;\n      \n      // Convert to expected type\n      reformattedData[field] = convertToType(originalValue, expectedType);\n      \n      // Track if conversion happened\n      if (originalType !== expectedType && originalValue !== null) {\n        schemaChanges.push(`Converted '${field}' from ${originalType} to ${expectedType}`);\n      }\n    }\n  }\n  \n  // Preserve any additional fields not in schema\n  for (const [field, value] of Object.entries(data)) {\n    if (!(field in expectedSchema)) {\n      reformattedData[field] = value;\n    }\n  }\n  \n  return {\n    json: {\n      ...reformattedData,\n      remediationApplied: true,\n      remediationType: 'schema_reformat',\n      schemaChanges: schemaChanges,\n      originalData: data,\n      remediatedAt: new Date().toISOString()\n    }\n  };\n});\n\nreturn remediatedItems;"
      },
      "typeVersion": 2
    },
    {
      "id": "8870b64c-7959-4abd-8640-e6db01dbd4bb",
      "name": "Retry Upstream Data Fetch",
      "type": "n8n-nodes-base.code",
      "position": [
        4720,
        816
      ],
      "parameters": {
        "jsCode": "// Retry fetching data from upstream source\n// This would re-execute the Fetch Latest Data Batch query or call the upstream API\n\nconst items = $input.all();\nconst remediationPlan = items[0].json;\n\n// Simulate retry logic - in production, this would actually re-execute the upstream fetch\n// For now, we'll create a placeholder structure showing the retry attempt\n\nconst retryResult = {\n  remediationApplied: true,\n  remediationType: 'retry_upstream',\n  retryDetails: {\n    attemptNumber: 1,\n    maxRetries: 3,\n    retryTimestamp: new Date().toISOString(),\n    upstreamSource: 'Fetch Latest Data Batch',\n    status: 'retry_initiated'\n  },\n  originalAnomalies: remediationPlan.anomalies || [],\n  message: 'Upstream data fetch retry initiated. In production, this would re-execute the data fetch operation.'\n};\n\n// In a real implementation, you would:\n// 1. Re-execute the upstream query/API call\n// 2. Validate the new data\n// 3. Compare with previous results\n// 4. Return the fresh data with retry metadata\n\nreturn [retryResult];"
      },
      "typeVersion": 2
    },
    {
      "id": "c9adc82f-5bb0-43ea-b23e-9d7de1cf2fdd",
      "name": "Merge Remediated Data Streams",
      "type": "n8n-nodes-base.code",
      "position": [
        4992,
        528
      ],
      "parameters": {
        "jsCode": "// Merge all remediated data from different remediation branches\n// Combine with original anomaly classification and remediation plan\n// Preserve all metadata including action taken, parameters used, and original statistics\n\nconst remediatedItems = [];\n\n// Process all input items from different remediation branches\nfor (const item of $input.all()) {\n  const mergedData = {\n    // Original data from the item\n    remediatedData: item.json.remediatedData || item.json.data || item.json,\n    \n    // Preserve anomaly classification from earlier in the workflow\n    anomalyClassification: item.json.anomalyClassification || {\n      type: item.json.anomalyType,\n      severity: item.json.severity,\n      confidence: item.json.confidence,\n      description: item.json.anomalyDescription\n    },\n    \n    // Preserve remediation plan\n    remediationPlan: item.json.remediationPlan || {\n      action: item.json.action,\n      strategy: item.json.strategy,\n      reasoning: item.json.reasoning\n    },\n    \n    // Metadata about the remediation execution\n    metadata: {\n      actionTaken: item.json.actionTaken || item.json.action,\n      parametersUsed: item.json.parametersUsed || item.json.parameters || {},\n      originalStatistics: item.json.originalStatistics || item.json.statistics || {},\n      remediationTimestamp: new Date().toISOString(),\n      remediationBranch: item.json.remediationBranch || 'unknown',\n      recordsProcessed: item.json.recordsProcessed || 0,\n      recordsModified: item.json.recordsModified || 0\n    },\n    \n    // Preserve any additional fields\n    ...item.json\n  };\n  \n  remediatedItems.push(mergedData);\n}\n\n// Return merged results\nreturn remediatedItems.map(item => ({ json: item }));"
      },
      "typeVersion": 2
    },
    {
      "id": "000e0c73-8cb8-4e0c-a296-376afb91c814",
      "name": "Re-run Quality Checks on Remediated Data",
      "type": "n8n-nodes-base.code",
      "position": [
        5216,
        528
      ],
      "parameters": {
        "jsCode": "// Re-run Quality Checks on Remediated Data\n// This validates whether the remediation was successful\n\nconst items = $input.all();\nconst results = [];\n\nfor (const item of items) {\n  const data = item.json.remediatedData || item.json.data || item.json;\n  \n  // Initialize quality check results\n  const qualityChecks = {\n    nullCheck: { passed: true, issues: [] },\n    outlierCheck: { passed: true, issues: [] },\n    schemaCheck: { passed: true, issues: [] },\n    rangeCheck: { passed: true, issues: [] }\n  };\n  \n  let totalIssues = 0;\n  \n  // Check for null/missing values\n  if (Array.isArray(data)) {\n    data.forEach((record, idx) => {\n      Object.entries(record).forEach(([key, value]) => {\n        if (value === null || value === undefined || value === '') {\n          qualityChecks.nullCheck.passed = false;\n          qualityChecks.nullCheck.issues.push(`Null value in record ${idx}, field: ${key}`);\n          totalIssues++;\n        }\n      });\n    });\n  } else {\n    Object.entries(data).forEach(([key, value]) => {\n      if (value === null || value === undefined || value === '') {\n        qualityChecks.nullCheck.passed = false;\n        qualityChecks.nullCheck.issues.push(`Null value in field: ${key}`);\n        totalIssues++;\n      }\n    });\n  }\n  \n  // Check for outliers in numeric fields\n  const numericFields = [];\n  const dataArray = Array.isArray(data) ? data : [data];\n  \n  dataArray.forEach(record => {\n    Object.entries(record).forEach(([key, value]) => {\n      if (typeof value === 'number' && !isNaN(value)) {\n        if (!numericFields[key]) numericFields[key] = [];\n        numericFields[key].push(value);\n      }\n    });\n  });\n  \n  Object.entries(numericFields).forEach(([field, values]) => {\n    if (values.length > 2) {\n      const mean = values.reduce((a, b) => a + b, 0) / values.length;\n      const variance = values.reduce((sum, val) => sum + Math.pow(val - mean, 2), 0) / values.length;\n      const stdDev = Math.sqrt(variance);\n      \n      values.forEach((value, idx) => {\n        const zScore = Math.abs((value - mean) / stdDev);\n        if (zScore > 3) {\n          qualityChecks.outlierCheck.passed = false;\n          qualityChecks.outlierCheck.issues.push(`Outlier detected in ${field}: ${value} (z-score: ${zScore.toFixed(2)})`);\n          totalIssues++;\n        }\n      });\n    }\n  });\n  \n  // Check for schema consistency\n  if (Array.isArray(data) && data.length > 1) {\n    const firstKeys = Object.keys(data[0]).sort();\n    data.forEach((record, idx) => {\n      const currentKeys = Object.keys(record).sort();\n      if (JSON.stringify(firstKeys) !== JSON.stringify(currentKeys)) {\n        qualityChecks.schemaCheck.passed = false;\n        qualityChecks.schemaCheck.issues.push(`Schema mismatch in record ${idx}`);\n        totalIssues++;\n      }\n    });\n  }\n  \n  // Determine if remediation was successful\n  const remediationSuccessful = totalIssues === 0 && \n    qualityChecks.nullCheck.passed && \n    qualityChecks.outlierCheck.passed && \n    qualityChecks.schemaCheck.passed;\n  \n  results.push({\n    json: {\n      ...item.json,\n      qualityCheckResults: qualityChecks,\n      totalIssuesFound: totalIssues,\n      remediationSuccessful: remediationSuccessful,\n      checkTimestamp: new Date().toISOString(),\n      status: remediationSuccessful ? 'PASSED' : 'FAILED'\n    }\n  });\n}\n\nreturn results;"
      },
      "typeVersion": 2
    },
    {
      "id": "88946fb7-c399-4cb3-a89c-d85b661f3ed3",
      "name": "Check If Remediation Successful",
      "type": "n8n-nodes-base.if",
      "position": [
        5440,
        528
      ],
      "parameters": {
        "options": {},
        "conditions": {
          "options": {
            "leftValue": "",
            "caseSensitive": false,
            "typeValidation": "loose"
          },
          "combinator": "and",
          "conditions": [
            {
              "id": "id-1",
              "operator": {
                "type": "boolean",
                "operation": "true"
              },
              "leftValue": "={{ $('Re-run Quality Checks on Remediated Data').item.json.remediationSuccessful }}"
            }
          ]
        }
      },
      "typeVersion": 2.3
    },
    {
      "id": "3433ed04-2930-47d9-852f-b76211a0dca2",
      "name": "Send Success Notification to Slack",
      "type": "n8n-nodes-base.slack",
      "position": [
        5888,
        384
      ],
      "parameters": {
        "text": "=✅ *Data Pipeline Auto-Remediation Successful*\n\n*Anomaly Type:* {{ $('AI Anomaly Classifier Agent').first().json.classification }}\n*Severity:* {{ $('AI Anomaly Classifier Agent').first().json.severity }}\n*Action Taken:* {{ $('AI Remediation Planner Agent').first().json.action }}\n*Affected Columns:* {{ $('Run Statistical Quality Checks').first().json.affectedColumns }}\n\n*Remediation Details:*\n{{ $json.remediationDetails }}\n\n*Timestamp:* {{ new Date().toISOString() }}",
        "select": "channel",
        "channelId": {
          "__rl": true,
          "mode": "id",
          "value": "={{ $('Set Pipeline Parameters').first().json.slackChannel }}"
        },
        "otherOptions": {}
      },
      "typeVersion": 2.4
    },
    {
      "id": "56bc6275-cb66-4e99-94c8-30ac3beb0d85",
      "name": "Escalate to Human Review in Slack",
      "type": "n8n-nodes-base.slack",
      "position": [
        5664,
        624
      ],
      "parameters": {
        "text": "=🚨 *Data Pipeline Remediation Failed - Human Review Required*\n\n*Anomaly Type:* {{ $('AI Anomaly Classifier Agent').first().json.classification }}\n*Severity:* {{ $('AI Anomaly Classifier Agent').first().json.severity }}\n*Confidence:* {{ $('AI Anomaly Classifier Agent').first().json.confidence }}%\n\n*Attempted Action:* {{ $('AI Remediation Planner Agent').first().json.action }}\n*Reasoning:* {{ $('AI Remediation Planner Agent').first().json.reasoning }}\n\n*Affected Columns:* {{ $('Run Statistical Quality Checks').first().json.affectedColumns }}\n\n*Data Sample:*\n```{{ JSON.stringify($('Run Statistical Quality Checks').first().json.dataSample, null, 2) }}```\n\n*Statistics:*\n```{{ JSON.stringify($('Run Statistical Quality Checks').first().json.statistics, null, 2) }}```\n\n*Timestamp:* {{ new Date().toISOString() }}\n\n@data-team Please investigate and resolve manually.",
        "select": "channel",
        "channelId": {
          "__rl": true,
          "mode": "id",
          "value": "={{ $('Set Pipeline Parameters').first().json.slackChannel }}"
        },
        "otherOptions": {}
      },
      "typeVersion": 2.4
    },
    {
      "id": "073bb465-95b7-45d2-8df3-3a887dfd3147",
      "name": "Log Successful Remediation Incident",
      "type": "n8n-nodes-base.postgres",
      "position": [
        6112,
        384
      ],
      "parameters": {
        "query": "INSERT INTO {{ $('Set Pipeline Parameters').first().json.postgresTable }} (timestamp, anomaly_type, severity, classification_confidence, affected_columns, action_taken, remediation_successful, statistics, data_sample) VALUES (NOW(), '{{ $('AI Anomaly Classifier Agent').first().json.classification }}', '{{ $('AI Anomaly Classifier Agent').first().json.severity }}', {{ $('AI Anomaly Classifier Agent').first().json.confidence }}, '{{ $('Run Statistical Quality Checks').first().json.affectedColumns }}', '{{ $('AI Remediation Planner Agent').first().json.action }}', true, '{{ JSON.stringify($('Run Statistical Quality Checks').first().json.statistics) }}', '{{ JSON.stringify($('Run Statistical Quality Checks').first().json.dataSample) }}')",
        "options": {},
        "operation": "executeQuery"
      },
      "typeVersion": 2.6
    },
    {
      "id": "21a70c96-bdaa-4c9c-b074-4e28ba6e9210",
      "name": "Log Failed Remediation Incident",
      "type": "n8n-nodes-base.postgres",
      "position": [
        6112,
        624
      ],
      "parameters": {
        "query": "INSERT INTO {{ $('Set Pipeline Parameters').first().json.postgresTable }} (timestamp, anomaly_type, severity, classification_confidence, affected_columns, action_taken, remediation_successful, statistics, data_sample) VALUES (NOW(), '{{ $('AI Anomaly Classifier Agent').first().json.classification }}', '{{ $('AI Anomaly Classifier Agent').first().json.severity }}', {{ $('AI Anomaly Classifier Agent').first().json.confidence }}, '{{ $('Run Statistical Quality Checks').first().json.affectedColumns }}', '{{ $('AI Remediation Planner Agent').first().json.action }}', false, '{{ JSON.stringify($('Run Statistical Quality Checks').first().json.statistics) }}', '{{ JSON.stringify($('Run Statistical Quality Checks').first().json.dataSample) }}')",
        "options": {},
        "operation": "executeQuery"
      },
      "typeVersion": 2.6
    },
    {
      "id": "d3e68f3f-0560-4f07-b7fc-b2d051372bde",
      "name": "Fetch Historical Baseline Data",
      "type": "n8n-nodes-base.postgres",
      "position": [
        1280,
        608
      ],
      "parameters": {
        "query": "SELECT * FROM {{ $('Set Pipeline Parameters').first().json.baselineTable }} WHERE metric_date >= NOW() - INTERVAL '7 days' ORDER BY metric_date DESC",
        "options": {},
        "operation": "executeQuery"
      },
      "typeVersion": 2.6
    },
    {
      "id": "752fdba5-4246-424c-a718-4991b2b31a04",
      "name": "Compare with Historical Baseline",
      "type": "n8n-nodes-base.code",
      "position": [
        1504,
        608
      ],
      "parameters": {
        "jsCode": "// Compare current data batch with historical baseline metrics\n// Calculate deviation percentages for key metrics\n// Flag significant changes that may indicate data quality issues\n\nconst currentData = $('Run Statistical Quality Checks').first().json;\nconst historicalBaseline = $('Fetch Historical Baseline Data').all();\n\n// Initialize comparison results\nconst comparisonResults = {\n  timestamp: new Date().toISOString(),\n  significantDeviations: false,\n  deviationFlags: [],\n  metrics: {}\n};\n\n// Helper function to calculate percentage deviation\nfunction calculateDeviation(current, baseline) {\n  if (baseline === 0) return current === 0 ? 0 : 100;\n  return ((current - baseline) / baseline) * 100;\n}\n\n// Helper function to determine if deviation is significant\nfunction isSignificantDeviation(deviation, threshold = 20) {\n  return Math.abs(deviation) > threshold;\n}\n\n// Extract baseline metrics from historical data\nlet baselineMetrics = {};\nif (historicalBaseline.length > 0) {\n  // Calculate average baseline metrics from historical data\n  const historicalStats = historicalBaseline.map(item => item.json);\n  \n  // Average null rates across historical data\n  const avgNullRates = {};\n  const avgOutlierCounts = {};\n  const avgRecordVolumes = [];\n  \n  historicalStats.forEach(stat => {\n    if (stat.statistics) {\n      Object.entries(stat.statistics).forEach(([column, stats]) => {\n        if (!avgNullRates[column]) avgNullRates[column] = [];\n        if (!avgOutlierCounts[column]) avgOutlierCounts[column] = [];\n        \n        if (stats.nullRate !== undefined) {\n          avgNullRates[column].push(stats.nullRate);\n        }\n        if (stats.outlierCount !== undefined) {\n          avgOutlierCounts[column].push(stats.outlierCount);\n        }\n      });\n    }\n    if (stat.totalRecords) {\n      avgRecordVolumes.push(stat.totalRecords);\n    }\n  });\n  \n  // Calculate baseline averages\n  baselineMetrics.nullRates = {};\n  Object.entries(avgNullRates).forEach(([column, rates]) => {\n    baselineMetrics.nullRates[column] = rates.reduce((a, b) => a + b, 0) / rates.length;\n  });\n  \n  baselineMetrics.outlierCounts = {};\n  Object.entries(avgOutlierCounts).forEach(([column, counts]) => {\n    baselineMetrics.outlierCounts[column] = counts.reduce((a, b) => a + b, 0) / counts.length;\n  });\n  \n  baselineMetrics.avgRecordVolume = avgRecordVolumes.length > 0 \n    ? avgRecordVolumes.reduce((a, b) => a + b, 0) / avgRecordVolumes.length \n    : 0;\n} else {\n  // No historical baseline available - use current as baseline\n  baselineMetrics.nullRates = {};\n  baselineMetrics.outlierCounts = {};\n  baselineMetrics.avgRecordVolume = currentData.totalRecords || 0;\n  \n  if (currentData.statistics) {\n    Object.entries(currentData.statistics).forEach(([column, stats]) => {\n      baselineMetrics.nullRates[column] = stats.nullRate || 0;\n      baselineMetrics.outlierCounts[column] = stats.outlierCount || 0;\n    });\n  }\n}\n\n// Compare record volume\nconst currentRecordVolume = currentData.totalRecords || 0;\nconst volumeDeviation = calculateDeviation(currentRecordVolume, baselineMetrics.avgRecordVolume);\n\ncomparisonResults.metrics.recordVolume = {\n  current: currentRecordVolume,\n  baseline: baselineMetrics.avgRecordVolume,\n  deviation: volumeDeviation,\n  deviationPercentage: `${volumeDeviation.toFixed(2)}%`,\n  isSignificant: isSignificantDeviation(volumeDeviation)\n};\n\nif (isSignificantDeviation(volumeDeviation)) {\n  comparisonResults.significantDeviations = true;\n  comparisonResults.deviationFlags.push({\n    metric: 'record_volume',\n    deviation: volumeDeviation,\n    message: `Record volume deviated by ${volumeDeviation.toFixed(2)}% from baseline`\n  });\n}\n\n// Compare null rates per column\ncomparisonResults.metrics.nullRates = {};\nif (currentData.statistics) {\n  Object.entries(currentData.statistics).forEach(([column, stats]) => {\n    const currentNullRate = stats.nullRate || 0;\n    const baselineNullRate = baselineMetrics.nullRates[column] || 0;\n    const nullRateDeviation = calculateDeviation(currentNullRate, baselineNullRate);\n    \n    comparisonResults.metrics.nullRates[column] = {\n      current: currentNullRate,\n      baseline: baselineNullRate,\n      deviation: nullRateDeviation,\n      deviationPercentage: `${nullRateDeviation.toFixed(2)}%`,\n      isSignificant: isSignificantDeviation(nullRateDeviation, 30) // 30% threshold for null rates\n    };\n    \n    if (isSignificantDeviation(nullRateDeviation, 30)) {\n      comparisonResults.significantDeviations = true;\n      comparisonResults.deviationFlags.push({\n        metric: 'null_rate',\n        column: column,\n        deviation: nullRateDeviation,\n        message: `Null rate in column '${column}' deviated by ${nullRateDeviation.toFixed(2)}% from baseline`\n      });\n    }\n  });\n}\n\n// Compare outlier counts per column\ncomparisonResults.metrics.outlierCounts = {};\nif (currentData.statistics) {\n  Object.entries(currentData.statistics).forEach(([column, stats]) => {\n    if (stats.outlierCount !== undefined) {\n      const currentOutlierCount = stats.outlierCount || 0;\n      const baselineOutlierCount = baselineMetrics.outlierCounts[column] || 0;\n      const outlierDeviation = calculateDeviation(currentOutlierCount, baselineOutlierCount);\n      \n      comparisonResults.metrics.outlierCounts[column] = {\n        current: currentOutlierCount,\n        baseline: baselineOutlierCount,\n        deviation: outlierDeviation,\n        deviationPercentage: `${outlierDeviation.toFixed(2)}%`,\n        isSignificant: isSignificantDeviation(outlierDeviation, 50) // 50% threshold for outliers\n      };\n      \n      if (isSignificantDeviation(outlierDeviation, 50)) {\n        comparisonResults.significantDeviations = true;\n        comparisonResults.deviationFlags.push({\n          metric: 'outlier_count',\n          column: column,\n          deviation: outlierDeviation,\n          message: `Outlier count in column '${column}' deviated by ${outlierDeviation.toFixed(2)}% from baseline`\n        });\n      }\n    }\n  });\n}\n\n// Add summary statistics\ncomparisonResults.summary = {\n  totalDeviations: comparisonResults.deviationFlags.length,\n  hasSignificantChanges: comparisonResults.significantDeviations,\n  baselineDataPoints: historicalBaseline.length,\n  comparisonTimestamp: new Date().toISOString()\n};\n\nreturn [{ json: comparisonResults }];"
      },
      "typeVersion": 2
    },
    {
      "id": "f7302d26-e7b7-4882-bb38-e7f81146d7f0",
      "name": "Calculate Trend Analysis Metrics",
      "type": "n8n-nodes-base.code",
      "position": [
        1728,
        608
      ],
      "parameters": {
        "jsCode": "// Trend Analysis on Historical Data\n// Calculates moving averages, trend direction, seasonal patterns, and velocity of change\n\nconst currentData = $('Run Statistical Quality Checks').first().json;\nconst historicalData = $('Fetch Historical Baseline Data').all();\nconst comparisonData = $('Compare with Historical Baseline').first().json;\n\n// Helper function to calculate moving average\nfunction calculateMovingAverage(data, windowSize) {\n  const result = [];\n  for (let i = 0; i < data.length; i++) {\n    if (i < windowSize - 1) {\n      result.push(null);\n      continue;\n    }\n    const window = data.slice(i - windowSize + 1, i + 1);\n    const avg = window.reduce((sum, val) => sum + val, 0) / windowSize;\n    result.push(avg);\n  }\n  return result;\n}\n\n// Helper function to detect trend direction\nfunction detectTrendDirection(values) {\n  if (values.length < 2) return 'insufficient_data';\n  \n  const recentValues = values.slice(-10); // Last 10 data points\n  const firstHalf = recentValues.slice(0, Math.floor(recentValues.length / 2));\n  const secondHalf = recentValues.slice(Math.floor(recentValues.length / 2));\n  \n  const firstAvg = firstHalf.reduce((sum, val) => sum + val, 0) / firstHalf.length;\n  const secondAvg = secondHalf.reduce((sum, val) => sum + val, 0) / secondHalf.length;\n  \n  const percentChange = ((secondAvg - firstAvg) / firstAvg) * 100;\n  \n  if (Math.abs(percentChange) < 5) return 'stable';\n  if (percentChange > 0) return 'improving';\n  return 'degrading';\n}\n\n// Helper function to calculate velocity of change\nfunction calculateVelocity(values) {\n  if (values.length < 2) return 0;\n  \n  const changes = [];\n  for (let i = 1; i < values.length; i++) {\n    changes.push(values[i] - values[i - 1]);\n  }\n  \n  const avgChange = changes.reduce((sum, val) => sum + val, 0) / changes.length;\n  return avgChange;\n}\n\n// Helper function to detect seasonal patterns\nfunction detectSeasonalPatterns(values, period = 7) {\n  if (values.length < period * 2) return { hasPattern: false, strength: 0 };\n  \n  const seasonalAverages = [];\n  for (let i = 0; i < period; i++) {\n    const periodValues = [];\n    for (let j = i; j < values.length; j += period) {\n      periodValues.push(values[j]);\n    }\n    const avg = periodValues.reduce((sum, val) => sum + val, 0) / periodValues.length;\n    seasonalAverages.push(avg);\n  }\n  \n  const overallAvg = values.reduce((sum, val) => sum + val, 0) / values.length;\n  const variance = seasonalAverages.reduce((sum, val) => sum + Math.pow(val - overallAvg, 2), 0) / seasonalAverages.length;\n  const strength = Math.sqrt(variance) / overallAvg;\n  \n  return {\n    hasPattern: strength > 0.1,\n    strength: strength,\n    pattern: seasonalAverages\n  };\n}\n\n// Extract time series data from historical baseline\nconst timeSeriesData = {};\nconst affectedColumns = currentData.affectedColumns || [];\n\nif (historicalData && historicalData.length > 0) {\n  // Process each affected column\n  affectedColumns.forEach(column => {\n    const values = historicalData\n      .map(item => item.json[column])\n      .filter(val => typeof val === 'number' && !isNaN(val));\n    \n    if (values.length > 0) {\n      timeSeriesData[column] = values;\n    }\n  });\n}\n\n// Perform trend analysis for each column\nconst trendAnalysis = {};\n\nObject.entries(timeSeriesData).forEach(([column, values]) => {\n  const ma7 = calculateMovingAverage(values, 7);\n  const ma30 = calculateMovingAverage(values, 30);\n  const trendDirection = detectTrendDirection(values);\n  const velocity = calculateVelocity(values);\n  const seasonality = detectSeasonalPatterns(values, 7);\n  \n  // Calculate prediction for next value\n  const recentValues = values.slice(-5);\n  const recentAvg = recentValues.reduce((sum, val) => sum + val, 0) / recentValues.length;\n  const prediction = recentAvg + velocity;\n  \n  trendAnalysis[column] = {\n    currentValue: values[values.length - 1],\n    movingAverage7: ma7[ma7.length - 1],\n    movingAverage30: ma30[ma30.length - 1],\n    trendDirection: trendDirection,\n    velocity: velocity,\n    velocityPercentage: (velocity / recentAvg) * 100,\n    seasonality: seasonality,\n    prediction: prediction,\n    predictionConfidence: seasonality.hasPattern ? 'medium' : 'low',\n    dataPoints: values.length\n  };\n});\n\n// Overall trend assessment\nconst trendDirections = Object.values(trendAnalysis).map(t => t.trendDirection);\nconst degradingCount = trendDirections.filter(d => d === 'degrading').length;\nconst improvingCount = trendDirections.filter(d => d === 'improving').length;\n\nlet overallTrend = 'stable';\nif (degradingCount > improvingCount) {\n  overallTrend = 'degrading';\n} else if (improvingCount > degradingCount) {\n  overallTrend = 'improving';\n}\n\nconst result = {\n  trendAnalysis: trendAnalysis,\n  overallTrend: overallTrend,\n  summary: {\n    totalColumnsAnalyzed: Object.keys(trendAnalysis).length,\n    degradingColumns: degradingCount,\n    improvingColumns: improvingCount,\n    stableColumns: trendDirections.filter(d => d === 'stable').length\n  },\n  recommendations: [],\n  timestamp: new Date().toISOString()\n};\n\n// Generate recommendations based on trends\nif (overallTrend === 'degrading') {\n  result.recommendations.push('Data quality is degrading. Consider investigating upstream data sources.');\n}\n\nObject.entries(trendAnalysis).forEach(([column, analysis]) => {\n  if (analysis.trendDirection === 'degrading' && Math.abs(analysis.velocityPercentage) > 10) {\n    result.recommendations.push(`Column '${column}' is degrading rapidly (${analysis.velocityPercentage.toFixed(2)}% change rate). Immediate attention required.`);\n  }\n  \n  if (analysis.seasonality.hasPattern && analysis.seasonality.strength > 0.2) {\n    result.recommendations.push(`Column '${column}' shows strong seasonal patterns. Consider time-based adjustments.`);\n  }\n});\n\nreturn [result];"
      },
      "typeVersion": 2
    },
    {
      "id": "f9ac24bc-8571-4135-88ea-a1866f80797c",
      "name": "Post Critical Alert to PagerDuty",
      "type": "n8n-nodes-base.httpRequest",
      "position": [
        3472,
        672
      ],
      "parameters": {
        "url": "https://api.pagerduty.com/incidents",
        "method": "POST",
        "options": {},
        "jsonBody": "={\n  \"incident\": {\n    \"type\": \"incident\",\n    \"title\": \"Critical Data Pipeline Anomaly: {{ $('AI Anomaly Classifier Agent').first().json.classification }}\",\n    \"service\": {\n      \"id\": \"{{ $('Set Pipeline Parameters').first().json.pagerDutyServiceId }}\",\n      \"type\": \"service_reference\"\n    },\n    \"urgency\": \"high\",\n    \"body\": {\n      \"type\": \"incident_body\",\n      \"details\": \"Anomaly Type: {{ $('AI Anomaly Classifier Agent').first().json.classification }}\\nSeverity: {{ $('AI Anomaly Classifier Agent').first().json.severity }}\\nConfidence: {{ $('AI Anomaly Classifier Agent').first().json.confidence }}%\\nAffected Columns: {{ $('Run Statistical Quality Checks').first().json.affectedColumns }}\\nExplanation: {{ $('AI Anomaly Classifier Agent').first().json.explanation }}\"\n    }\n  }\n}",
        "sendBody": true,
        "sendHeaders": true,
        "specifyBody": "json",
        "headerParameters": {
          "parameters": [
            {
              "name": "Authorization",
              "value": "={{ 'Token token=' + $('Set Pipeline Parameters').first().json.pagerDutyApiKey }}"
            },
            {
              "name": "Content-Type",
              "value": "application/json"
            }
          ]
        }
      },
      "typeVersion": 4.3
    },
    {
      "id": "4e962c77-baa0-45b3-9474-df13b2553939",
      "name": "Check If Critical Severity",
      "type": "n8n-nodes-base.if",
      "position": [
        3248,
        608
      ],
      "parameters": {
        "options": {},
        "conditions": {
          "options": {
            "leftValue": "",
            "caseSensitive": false,
            "typeValidation": "loose"
          },
          "combinator": "and",
          "conditions": [
            {
              "id": "id-1",
              "operator": {
                "type": "string",
                "operation": "equals"
              },
              "leftValue": "={{ $('AI Anomaly Classifier Agent').item.json.severity }}",
              "rightValue": "critical"
            }
          ]
        }
      },
      "typeVersion": 2.3
    },
    {
      "id": "9780decc-57e4-4e9a-95de-c18f02a2dc90",
      "name": "Update Historical Baseline Metrics",
      "type": "n8n-nodes-base.postgres",
      "position": [
        7008,
        528
      ],
      "parameters": {
        "query": "INSERT INTO {{ $('Set Pipeline Parameters').first().json.baselineTable }} (metric_date, total_records, null_rate_avg, outlier_rate_avg, anomaly_count, remediation_success_rate) VALUES (NOW(), {{ $json.totalRecords }}, {{ $json.avgNullRate }}, {{ $json.avgOutlierRate }}, {{ $json.anomalyCount }}, {{ $json.successRate }}) ON CONFLICT (metric_date) DO UPDATE SET total_records = EXCLUDED.total_records, null_rate_avg = EXCLUDED.null_rate_avg",
        "options": {},
        "operation": "executeQuery"
      },
      "typeVersion": 2.6
    },
    {
      "id": "c365a7cd-a100-47e3-a285-05511b9b7b94",
      "name": "Generate Remediation Report",
      "type": "n8n-nodes-base.code",
      "position": [
        6560,
        528
      ],
      "parameters": {
        "jsCode": "// Generate Comprehensive Remediation Report\n// Aggregates incident data and produces structured analytics\n\nconst items = $input.all();\n\n// Initialize report structure\nconst report = {\n  reportMetadata: {\n    generatedAt: new Date().toISOString(),\n    reportPeriod: 'last_15_minutes',\n    totalIncidents: items.length\n  },\n  summaryStatistics: {\n    totalIncidents: items.length,\n    successfulRemediations: 0,\n    failedRemediations: 0,\n    successRate: 0,\n    averageConfidence: 0,\n    criticalIncidents: 0,\n    highSeverityIncidents: 0,\n    mediumSeverityIncidents: 0,\n    lowSeverityIncidents: 0\n  },\n  breakdownByAnomalyType: {},\n  breakdownBySeverity: {},\n  breakdownByAction: {},\n  topIssues: [],\n  affectedColumnsFrequency: {},\n  trendAnalysis: {\n    anomalyRateChange: 0,\n    successRateChange: 0,\n    mostCommonAnomalyType: '',\n    emergingPatterns: []\n  },\n  recommendations: []\n};\n\n// Process each incident\nlet totalConfidence = 0;\nconst anomalyTypeCounts = {};\nconst severityCounts = { critical: 0, high: 0, medium: 0, low: 0 };\nconst actionCounts = {};\nconst columnFrequency = {};\nconst issueDetails = [];\n\nfor (const item of items) {\n  const data = item.json;\n  \n  // Count successes and failures\n  if (data.remediation_successful === true || data.remediationSuccessful === true) {\n    report.summaryStatistics.successfulRemediations++;\n  } else {\n    report.summaryStatistics.failedRemediations++;\n  }\n  \n  // Aggregate confidence scores\n  const confidence = data.classification_confidence || data.confidence || 0;\n  totalConfidence += confidence;\n  \n  // Count by anomaly type\n  const anomalyType = data.anomaly_type || data.classification || 'unknown';\n  anomalyTypeCounts[anomalyType] = (anomalyTypeCounts[anomalyType] || 0) + 1;\n  \n  // Count by severity\n  const severity = (data.severity || 'medium').toLowerCase();\n  severityCounts[severity] = (severityCounts[severity] || 0) + 1;\n  \n  if (severity === 'critical') report.summaryStatistics.criticalIncidents++;\n  if (severity === 'high') report.summaryStatistics.highSeverityIncidents++;\n  if (severity === 'medium') report.summaryStatistics.mediumSeverityIncidents++;\n  if (severity === 'low') report.summaryStatistics.lowSeverityIncidents++;\n  \n  // Count by action taken\n  const action = data.action_taken || data.action || 'unknown';\n  actionCounts[action] = (actionCounts[action] || 0) + 1;\n  \n  // Track affected columns\n  let affectedColumns = [];\n  if (typeof data.affected_columns === 'string') {\n    try {\n      affectedColumns = JSON.parse(data.affected_columns);\n    } catch (e) {\n      affectedColumns = data.affected_columns.split(',').map(c => c.trim());\n    }\n  } else if (Array.isArray(data.affected_columns)) {\n    affectedColumns = data.affected_columns;\n  }\n  \n  affectedColumns.forEach(col => {\n    columnFrequency[col] = (columnFrequency[col] || 0) + 1;\n  });\n  \n  // Collect issue details for top issues analysis\n  issueDetails.push({\n    anomalyType,\n    severity,\n    action,\n    affectedColumns,\n    successful: data.remediation_successful || data.remediationSuccessful || false,\n    confidence,\n    timestamp: data.timestamp || new Date().toISOString()\n  });\n}\n\n// Calculate summary statistics\nreport.summaryStatistics.successRate = items.length > 0 \n  ? (report.summaryStatistics.successfulRemediations / items.length * 100).toFixed(2) \n  : 0;\nreport.summaryStatistics.averageConfidence = items.length > 0 \n  ? (totalConfidence / items.length).toFixed(2) \n  : 0;\n\n// Build breakdown by anomaly type\nfor (const [type, count] of Object.entries(anomalyTypeCounts)) {\n  const typeItems = issueDetails.filter(i => i.anomalyType === type);\n  const successCount = typeItems.filter(i => i.successful).length;\n  \n  report.breakdownByAnomalyType[type] = {\n    count,\n    percentage: ((count / items.length) * 100).toFixed(2),\n    successRate: typeItems.length > 0 ? ((successCount / typeItems.length) * 100).toFixed(2) : 0,\n    successCount,\n    failureCount: typeItems.length - successCount\n  };\n}\n\n// Build breakdown by severity\nfor (const [severity, count] of Object.entries(severityCounts)) {\n  if (count > 0) {\n    const severityItems = issueDetails.filter(i => i.severity.toLowerCase() === severity);\n    const successCount = severityItems.filter(i => i.successful).length;\n    \n    report.breakdownBySeverity[severity] = {\n      count,\n      percentage: ((count / items.length) * 100).toFixed(2),\n      successRate: severityItems.length > 0 ? ((successCount / severityItems.length) * 100).toFixed(2) : 0\n    };\n  }\n}\n\n// Build breakdown by action\nfor (const [action, count] of Object.entries(actionCounts)) {\n  const actionItems = issueDetails.filter(i => i.action === action);\n  const successCount = actionItems.filter(i => i.successful).length;\n  \n  report.breakdownByAction[action] = {\n    count,\n    percentage: ((count / items.length) * 100).toFixed(2),\n    successRate: actionItems.length > 0 ? ((successCount / actionItems.length) * 100).toFixed(2) : 0,\n    successCount,\n    failureCount: actionItems.length - successCount\n  };\n}\n\n// Identify top issues (most frequent affected columns)\nreport.topIssues = Object.entries(columnFrequency)\n  .sort((a, b) => b[1] - a[1])\n  .slice(0, 10)\n  .map(([column, frequency]) => ({\n    column,\n    frequency,\n    percentage: ((frequency / items.length) * 100).toFixed(2)\n  }));\n\nreport.affectedColumnsFrequency = columnFrequency;\n\n// Trend analysis\nconst mostCommonType = Object.entries(anomalyTypeCounts)\n  .sort((a, b) => b[1] - a[1])[0];\nreport.trendAnalysis.mostCommonAnomalyType = mostCommonType ? mostCommonType[0] : 'none';\n\n// Identify emerging patterns\nif (report.summaryStatistics.criticalIncidents > 0) {\n  report.trendAnalysis.emergingPatterns.push('Critical incidents detected - immediate attention required');\n}\nif (parseFloat(report.summaryStatistics.successRate) < 70) {\n  report.trendAnalysis.emergingPatterns.push('Low remediation success rate - review remediation strategies');\n}\nif (report.topIssues.length > 0 && report.topIssues[0].frequency > items.length * 0.5) {\n  report.trendAnalysis.emergingPatterns.push(`High concentration of issues in column: ${report.topIssues[0].column}`);\n}\n\n// Generate recommendations\nif (parseFloat(report.summaryStatistics.successRate) < 80) {\n  report.recommendations.push({\n    priority: 'high',\n    category: 'remediation_strategy',\n    recommendation: 'Review and optimize remediation strategies - success rate below 80%',\n    actionItems: [\n      'Analyze failed remediation attempts',\n      'Consider adding new remediation patterns',\n      'Review threshold configurations'\n    ]\n  });\n}\n\nif (report.summaryStatistics.criticalIncidents > 0) {\n  report.recommendations.push({\n    priority: 'critical',\n    category: 'incident_response',\n    recommendation: `${report.summaryStatistics.criticalIncidents} critical incidents require immediate attention`,\n    actionItems: [\n      'Review critical incidents immediately',\n      'Escalate to senior data engineers',\n      'Consider implementing additional monitoring'\n    ]\n  });\n}\n\nif (report.topIssues.length > 0) {\n  const topColumn = report.topIssues[0];\n  if (topColumn.frequency > 3) {\n    report.recommendations.push({\n      priority: 'medium',\n      category: 'data_quality',\n      recommendation: `Column '${topColumn.column}' frequently affected - investigate upstream data source`,\n      actionItems: [\n        `Review data source for column: ${topColumn.column}`,\n        'Check for schema changes or data type issues',\n        'Consider adding validation at ingestion layer'\n      ]\n    });\n  }\n}\n\nconst failedActions = Object.entries(report.breakdownByAction)\n  .filter(([action, stats]) => parseFloat(stats.successRate) < 50);\n\nif (failedActions.length > 0) {\n  failedActions.forEach(([action, stats]) => {\n    report.recommendations.push({\n      priority: 'medium',\n      category: 'remediation_action',\n      recommendation: `Action '${action}' has low success rate (${stats.successRate}%) - needs improvement`,\n      actionItems: [\n        `Review implementation of ${action} remediation`,\n        'Analyze failure patterns',\n        'Consider alternative remediation approaches'\n      ]\n    });\n  });\n}\n\nif (parseFloat(report.summaryStatistics.averageConfidence) < 70) {\n  report.recommendations.push({\n    priority: 'low',\n    category: 'classification',\n    recommendation: 'Low average classification confidence - consider model retraining or threshold adjustment',\n    actionItems: [\n      'Review classification model performance',\n      'Collect more training data for edge cases',\n      'Adjust confidence thresholds'\n    ]\n  });\n}\n\n// Add metrics export format\nreport.metricsExport = {\n  timestamp: new Date().toISOString(),\n  metrics: [\n    { name: 'data_pipeline.incidents.total', value: items.length, type: 'count' },\n    { name: 'data_pipeline.remediation.success_rate', value: parseFloat(report.summaryStatistics.successRate), type: 'percentage' },\n    { name: 'data_pipeline.remediation.successful', value: report.summaryStatistics.successfulRemediations, type: 'count' },\n    { name: 'data_pipeline.remediation.failed', value: report.summaryStatistics.failedRemediations, type: 'count' },\n    { name: 'data_pipeline.incidents.critical', value: report.summaryStatistics.criticalIncidents, type: 'count' },\n    { name: 'data_pipeline.incidents.high', value: report.summaryStatistics.highSeverityIncidents, type: 'count' },\n    { name: 'data_pipeline.classification.avg_confidence', value: parseFloat(report.summaryStatistics.averageConfidence), type: 'percentage' }\n  ]\n};\n\n// Add anomaly type metrics\nfor (const [type, stats] of Object.entries(report.breakdownByAnomalyType)) {\n  report.metricsExport.metrics.push({\n    name: `data_pipeline.anomaly.${type}.count`,\n    value: stats.count,\n    type: 'count'\n  });\n  report.metricsExport.metrics.push({\n    name: `data_pipeline.anomaly.${type}.success_rate`,\n    value: parseFloat(stats.successRate),\n    type: 'percentage'\n  });\n}\n\nreturn [{ json: report }];"
      },
      "typeVersion": 2
    },
    {
      "id": "54ec3eb2-4ac0-425c-90bd-f2a32d6b7927",
      "name": "Aggregate Incident Logs",
      "type": "n8n-nodes-base.aggregate",
      "position": [
        6336,
        528
      ],
      "parameters": {
        "options": {},
        "aggregate": "aggregateAllItemData"
      },
      "typeVersion": 1
    },
    {
      "id": "1cf633a1-26c8-4fec-b474-37ac5b5dc074",
      "name": "Run ML-Based Anomaly Scoring",
      "type": "n8n-nodes-base.code",
      "position": [
        1952,
        608
      ],
      "parameters": {
        "jsCode": "// ML-Based Anomaly Scoring using Isolation Forest Algorithm\n// Combines multiple features: deviation from baseline, trend velocity, historical patterns, and statistical outliers\n\nconst currentData = $('Run Statistical Quality Checks').first().json;\nconst baselineData = $('Compare with Historical Baseline').first().json;\nconst trendData = $('Calculate Trend Analysis Metrics').first().json;\n\n// Helper function to normalize values to 0-1 range\nfunction normalize(value, min, max) {\n  if (max === min) return 0;\n  return (value - min) / (max - min);\n}\n\n// Helper function to calculate isolation score for a single feature\nfunction calculateIsolationScore(value, dataset) {\n  if (!dataset || dataset.length === 0) return 0;\n  \n  const sorted = [...dataset].sort((a, b) => a - b);\n  const n = sorted.length;\n  \n  // Find position of value in sorted array\n  let position = 0;\n  for (let i = 0; i < n; i++) {\n    if (sorted[i] >= value) {\n      position = i;\n      break;\n    }\n  }\n  \n  // Calculate isolation score based on position\n  // Values at extremes get higher scores\n  const distanceFromMedian = Math.abs(position - n / 2);\n  const maxDistance = n / 2;\n  return distanceFromMedian / maxDistance;\n}\n\n// Feature 1: Deviation from Baseline Score\nfunction calculateBaselineDeviationScore() {\n  if (!baselineData || !baselineData.baseline) return 0;\n  \n  const deviations = [];\n  const stats = currentData.statistics || {};\n  const baseline = baselineData.baseline || {};\n  \n  for (const column in stats) {\n    if (stats[column].mean !== undefined && baseline[column]?.mean !== undefined) {\n      const deviation = Math.abs(stats[column].mean - baseline[column].mean);\n      const baselineStdDev = baseline[column].stdDev || 1;\n      const normalizedDeviation = deviation / baselineStdDev;\n      deviations.push(normalizedDeviation);\n    }\n  }\n  \n  if (deviations.length === 0) return 0;\n  \n  // Average deviation score\n  const avgDeviation = deviations.reduce((a, b) => a + b, 0) / deviations.length;\n  return Math.min(avgDeviation / 3, 1); // Normalize to 0-1, 3 std devs = max\n}\n\n// Feature 2: Trend Velocity Score\nfunction calculateTrendVelocityScore() {\n  if (!trendData || !trendData.velocity) return 0;\n  \n  const velocity = trendData.velocity || 0;\n  const historicalVelocities = trendData.historicalVelocities || [velocity];\n  \n  // Calculate how unusual the current velocity is\n  const isolationScore = calculateIsolationScore(Math.abs(velocity), historicalVelocities.map(Math.abs));\n  \n  return isolationScore;\n}\n\n// Feature 3: Historical Pattern Score\nfunction calculateHistoricalPatternScore() {\n  if (!baselineData || !baselineData.historicalPatterns) return 0;\n  \n  const patterns = baselineData.historicalPatterns || {};\n  const currentPattern = currentData.pattern || 'unknown';\n  \n  // Check if current pattern matches historical patterns\n  const patternFrequency = patterns[currentPattern] || 0;\n  const totalPatterns = Object.values(patterns).reduce((a, b) => a + b, 0) || 1;\n  const patternProbability = patternFrequency / totalPatterns;\n  \n  // Lower probability = higher anomaly score\n  return 1 - patternProbability;\n}\n\n// Feature 4: Statistical Outlier Score\nfunction calculateStatisticalOutlierScore() {\n  const stats = currentData.statistics || {};\n  const outlierScores = [];\n  \n  for (const column in stats) {\n    if (stats[column].outlierRate !== undefined) {\n      outlierScores.push(stats[column].outlierRate);\n    }\n  }\n  \n  if (outlierScores.length === 0) return 0;\n  \n  // Average outlier rate across all columns\n  const avgOutlierRate = outlierScores.reduce((a, b) => a + b, 0) / outlierScores.length;\n  return Math.min(avgOutlierRate * 10, 1); // Scale up and cap at 1\n}\n\n// Feature 5: Null Rate Score\nfunction calculateNullRateScore() {\n  const stats = currentData.statistics || {};\n  const nullRates = [];\n  \n  for (const column in stats) {\n    if (stats[column].nullRate !== undefined) {\n      nullRates.push(stats[column].nullRate);\n    }\n  }\n  \n  if (nullRates.length === 0) return 0;\n  \n  // Average null rate across all columns\n  const avgNullRate = nullRates.reduce((a, b) => a + b, 0) / nullRates.length;\n  return Math.min(avgNullRate * 5, 1); // Scale up and cap at 1\n}\n\n// Calculate individual feature scores\nconst featureScores = {\n  baselineDeviation: calculateBaselineDeviationScore(),\n  trendVelocity: calculateTrendVelocityScore(),\n  historicalPattern: calculateHistoricalPatternScore(),\n  statisticalOutlier: calculateStatisticalOutlierScore(),\n  nullRate: calculateNullRateScore()\n};\n\n// Feature weights (can be tuned based on domain knowledge)\nconst weights = {\n  baselineDeviation: 0.25,\n  trendVelocity: 0.20,\n  historicalPattern: 0.20,\n  statisticalOutlier: 0.20,\n  nullRate: 0.15\n};\n\n// Calculate weighted anomaly score\nlet anomalyScore = 0;\nfor (const feature in featureScores) {\n  anomalyScore += featureScores[feature] * weights[feature];\n}\n\n// Ensure score is between 0 and 1\nanomalyScore = Math.max(0, Math.min(1, anomalyScore));\n\n// Identify top contributing factors\nconst contributingFactors = Object.entries(featureScores)\n  .map(([feature, score]) => ({\n    feature,\n    score: score,\n    weightedContribution: score * weights[feature],\n    percentage: (score * weights[feature] / anomalyScore * 100).toFixed(2)\n  }))\n  .sort((a, b) => b.weightedContribution - a.weightedContribution);\n\n// Determine anomaly severity based on score\nlet severity = 'low';\nif (anomalyScore >= 0.75) {\n  severity = 'critical';\n} else if (anomalyScore >= 0.5) {\n  severity = 'high';\n} else if (anomalyScore >= 0.25) {\n  severity = 'medium';\n}\n\n// Return results\nreturn [{\n  json: {\n    anomalyScore: parseFloat(anomalyScore.toFixed(4)),\n    severity: severity,\n    featureScores: featureScores,\n    contributingFactors: contributingFactors,\n    topContributor: contributingFactors[0]?.feature || 'none',\n    weights: weights,\n    timestamp: new Date().toISOString(),\n    metadata: {\n      totalRecords: currentData.totalRecords || 0,\n      affectedColumns: currentData.affectedColumns || [],\n      anomalyTypes: currentData.anomalyType || []\n    }\n  }\n}];"
      },
      "typeVersion": 2
    },
    {
      "id": "1e302d8a-a95f-4569-ab4b-3c7ddefde971",
      "name": "Check If High Anomaly Score",
      "type": "n8n-nodes-base.if",
      "position": [
        2176,
        608
      ],
      "parameters": {
        "options": {},
        "conditions": {
          "options": {
            "leftValue": "",
            "caseSensitive": false,
            "typeValidation": "loose"
          },
          "combinator": "and",
          "conditions": [
            {
              "id": "id-1",
              "operator": {
                "type": "number",
                "operation": "gt"
              },
              "leftValue": "={{ $json.anomalyScore }}",
              "rightValue": "={{ $('Set Pipeline Parameters').first().json.anomalyScoreThreshold }}"
            }
          ]
        }
      },
      "typeVersion": 2.3
    },
    {
      "id": "4d9d7aa5-57af-4cb3-8d06-3c4f570caee0",
      "name": "Query Remediation History from PostgreSQL",
      "type": "n8n-nodes-base.postgres",
      "position": [
        4000,
        608
      ],
      "parameters": {
        "query": "SELECT action_taken, remediation_successful, COUNT(*) as count FROM {{ $('Set Pipeline Parameters').first().json.postgresTable }} WHERE timestamp >= NOW() - INTERVAL '30 days' AND anomaly_type = '{{ $('AI Anomaly Classifier Agent').first().json.classification }}' GROUP BY action_taken, remediation_successful",
        "options": {},
        "operation": "executeQuery"
      },
      "typeVersion": 2.6
    },
    {
      "id": "87a6c69a-d199-4a4a-80a0-0543060f66e9",
      "name": "Calculate Remediation Success Rate",
      "type": "n8n-nodes-base.code",
      "position": [
        4224,
        608
      ],
      "parameters": {
        "jsCode": "// Calculate Remediation Success Rate from Historical Data\n// Analyzes past remediation attempts to inform current decision making\n\nconst historicalData = $('Query Remediation History from PostgreSQL').all();\nconst currentRemediationPlan = $('AI Remediation Planner Agent').first().json;\n\n// Initialize success rate analysis\nconst successRateAnalysis = {\n  overallSuccessRate: 0,\n  actionTypePerformance: {},\n  bestPerformingStrategies: [],\n  recommendations: [],\n  totalAttempts: historicalData.length,\n  timestamp: new Date().toISOString()\n};\n\n// If no historical data, return default analysis\nif (historicalData.length === 0) {\n  return [{\n    json: {\n      ...currentRemediationPlan,\n      successRateMetadata: {\n        overallSuccessRate: 0,\n        confidence: 'low',\n        historicalDataPoints: 0,\n        recommendation: 'Insufficient historical data - proceeding with AI recommendation'\n      }\n    }\n  }];\n}\n\n// Group historical attempts by action type\nconst actionGroups = {};\nlet totalSuccesses = 0;\n\nfor (const item of historicalData) {\n  const data = item.json;\n  const actionType = data.action_taken || data.actionTaken || 'unknown';\n  const wasSuccessful = data.remediation_successful || data.remediationSuccessful || false;\n  \n  if (!actionGroups[actionType]) {\n    actionGroups[actionType] = {\n      total: 0,\n      successful: 0,\n      failed: 0,\n      avgConfidence: 0,\n      severityBreakdown: {}\n    };\n  }\n  \n  actionGroups[actionType].total++;\n  \n  if (wasSuccessful) {\n    actionGroups[actionType].successful++;\n    totalSuccesses++;\n  } else {\n    actionGroups[actionType].failed++;\n  }\n  \n  // Track confidence scores\n  if (data.classification_confidence || data.confidence) {\n    const confidence = data.classification_confidence || data.confidence;\n    actionGroups[actionType].avgConfidence += confidence;\n  }\n  \n  // Track severity breakdown\n  const severity = data.severity || 'unknown';\n  if (!actionGroups[actionType].severityBreakdown[severity]) {\n    actionGroups[actionType].severityBreakdown[severity] = { total: 0, successful: 0 };\n  }\n  actionGroups[actionType].severityBreakdown[severity].total++;\n  if (wasSuccessful) {\n    actionGroups[actionType].severityBreakdown[severity].successful++;\n  }\n}\n\n// Calculate success rates for each action type\nfor (const [actionType, stats] of Object.entries(actionGroups)) {\n  const successRate = (stats.successful / stats.total) * 100;\n  const avgConfidence = stats.total > 0 ? stats.avgConfidence / stats.total : 0;\n  \n  successRateAnalysis.actionTypePerformance[actionType] = {\n    successRate: successRate.toFixed(2),\n    totalAttempts: stats.total,\n    successful: stats.successful,\n    failed: stats.failed,\n    avgConfidence: avgConfidence.toFixed(2),\n    severityBreakdown: stats.severityBreakdown\n  };\n}\n\n// Calculate overall success rate\nsuccessRateAnalysis.overallSuccessRate = ((totalSuccesses / historicalData.length) * 100).toFixed(2);\n\n// Identify best performing strategies\nconst sortedActions = Object.entries(successRateAnalysis.actionTypePerformance)\n  .sort((a, b) => parseFloat(b[1].successRate) - parseFloat(a[1].successRate))\n  .filter(([_, stats]) => stats.totalAttempts >= 3); // Only consider actions with at least 3 attempts\n\nsuccessRateAnalysis.bestPerformingStrategies = sortedActions.slice(0, 3).map(([action, stats]) => ({\n  action,\n  successRate: stats.successRate,\n  totalAttempts: stats.totalAttempts\n}));\n\n// Generate recommendations based on current remediation plan\nconst currentAction = currentRemediationPlan.action;\nconst currentActionStats = successRateAnalysis.actionTypePerformance[currentAction];\n\nif (currentActionStats) {\n  const successRate = parseFloat(currentActionStats.successRate);\n  \n  if (successRate >= 80) {\n    successRateAnalysis.recommendations.push({\n      type: 'high_confidence',\n      message: `Action '${currentAction}' has a ${successRate}% historical success rate - proceed with confidence`,\n      confidence: 'high'\n    });\n  } else if (successRate >= 50) {\n    successRateAnalysis.recommendations.push({\n      type: 'moderate_confidence',\n      message: `Action '${currentAction}' has a ${successRate}% historical success rate - proceed with caution`,\n      confidence: 'moderate'\n    });\n  } else {\n    successRateAnalysis.recommendations.push({\n      type: 'low_confidence',\n      message: `Action '${currentAction}' has a ${successRate}% historical success rate - consider alternative approach`,\n      confidence: 'low',\n      alternatives: successRateAnalysis.bestPerformingStrategies.filter(s => s.action !== currentAction)\n    });\n  }\n} else {\n  successRateAnalysis.recommendations.push({\n    type: 'no_historical_data',\n    message: `No historical data for action '${currentAction}' - proceeding based on AI recommendation`,\n    confidence: 'unknown'\n  });\n}\n\n// Add success rate metadata to remediation plan\nconst enhancedRemediationPlan = {\n  ...currentRemediationPlan,\n  successRateMetadata: {\n    overallSuccessRate: successRateAnalysis.overallSuccessRate,\n    actionSuccessRate: currentActionStats ? currentActionStats.successRate : 'N/A',\n    historicalDataPoints: historicalData.length,\n    confidence: currentActionStats ? \n      (parseFloat(currentActionStats.successRate) >= 80 ? 'high' : \n       parseFloat(currentActionStats.successRate) >= 50 ? 'moderate' : 'low') : 'unknown',\n    recommendations: successRateAnalysis.recommendations,\n    bestPerformingStrategies: successRateAnalysis.bestPerformingStrategies,\n    fullAnalysis: successRateAnalysis\n  }\n};\n\nreturn [{ json: enhancedRemediationPlan }];"
      },
      "typeVersion": 2
    },
    {
      "id": "6a3d8dc7-d69c-4090-b3cd-53756f1807f5",
      "name": "Send Metrics to DataDog",
      "type": "n8n-nodes-base.httpRequest",
      "position": [
        6784,
        528
      ],
      "parameters": {
        "url": "https://api.datadoghq.com/api/v1/series",
        "method": "POST",
        "options": {},
        "jsonBody": "={\n  \"series\": [\n    {\n      \"metric\": \"data_pipeline.anomaly.count\",\n      \"points\": [[{{ Math.floor(Date.now() / 1000) }}, {{ $('Run Statistical Quality Checks').first().json.anomalyDetected ? 1 : 0 }}]],\n      \"type\": \"count\",\n      \"tags\": [\n        \"pipeline:self_healing\",\n        \"anomaly_type:{{ $('AI Anomaly Classifier Agent').first().json.classification }}\",\n        \"severity:{{ $('AI Anomaly Classifier Agent').first().json.severity }}\"\n      ]\n    },\n    {\n      \"metric\": \"data_pipeline.remediation.success_rate\",\n      \"points\": [[{{ Math.floor(Date.now() / 1000) }}, {{ $('Re-run Quality Checks on Remediated Data').first().json.remediationSuccessful ? 100 : 0 }}]],\n      \"type\": \"gauge\",\n      \"tags\": [\n        \"pipeline:self_healing\",\n        \"action:{{ $('AI Remediation Planner Agent').first().json.action }}\"\n      ]\n    },\n    {\n      \"metric\": \"data_pipeline.health.score\",\n      \"points\": [[{{ Math.floor(Date.now() / 1000) }}, {{ $('Re-run Quality Checks on Remediated Data').first().json.remediationSuccessful ? 100 : 50 }}]],\n      \"type\": \"gauge\",\n      \"tags\": [\n        \"pipeline:self_healing\",\n        \"status:{{ $('Re-run Quality Checks on Remediated Data').first().json.status }}\"\n      ]\n    }\n  ]\n}",
        "sendBody": true,
        "sendHeaders": true,
        "specifyBody": "json",
        "headerParameters": {
          "parameters": [
            {
              "name": "DD-API-KEY",
              "value": "={{ $('Set Pipeline Parameters').first().json.datadogApiKey }}"
            },
            {
              "name": "Content-Type",
              "value": "application/json"
            }
          ]
        }
      },
      "typeVersion": 4.3
    },
    {
      "id": "63b4014d-b9b4-4398-ac1f-823af4aee357",
      "name": "Prepare Data Sample for Review",
      "type": "n8n-nodes-base.code",
      "position": [
        5888,
        704
      ],
      "parameters": {
        "jsCode": "// Data Sampling for Human Review\n// Creates a representative sample of data showing the anomaly\n// Includes diverse examples, edge cases, and clear context\n\nconst items = $input.all();\nconst anomalyData = $('Run Statistical Quality Checks').first().json;\nconst classification = $('AI Anomaly Classifier Agent').first().json;\nconst remediationPlan = $('AI Remediation Planner Agent').first().json;\nconst originalData = $('Fetch Latest Data from PostgreSQL').all();\n\n// Configuration\nconst MAX_SAMPLES = 15;\nconst EDGE_CASE_COUNT = 5;\n\n// Helper function to calculate statistical measures\nfunction getStatistics(values) {\n  const sorted = values.filter(v => v !== null && v !== undefined).sort((a, b) => a - b);\n  if (sorted.length === 0) return null;\n  \n  return {\n    min: sorted[0],\n    max: sorted[sorted.length - 1],\n    median: sorted[Math.floor(sorted.length / 2)],\n    mean: sorted.reduce((a, b) => a + b, 0) / sorted.length\n  };\n}\n\n// Identify affected columns from anomaly data\nconst affectedColumns = anomalyData.affectedColumns || [];\n\n// Sample selection strategy based on anomaly type\nlet selectedSamples = [];\nconst anomalyTypes = anomalyData.anomalyType || [];\n\nif (anomalyTypes.includes('HIGH_NULL_RATE')) {\n  // Sample records with nulls in affected columns\n  const recordsWithNulls = originalData.filter(item => {\n    return affectedColumns.some(col => {\n      const value = item.json[col];\n      return value === null || value === undefined || value === '';\n    });\n  });\n  \n  // Take diverse samples\n  const step = Math.max(1, Math.floor(recordsWithNulls.length / MAX_SAMPLES));\n  for (let i = 0; i < Math.min(MAX_SAMPLES, recordsWithNulls.length); i += step) {\n    selectedSamples.push({\n      record: recordsWithNulls[i].json,\n      reason: 'Contains null values in affected columns',\n      affectedFields: affectedColumns.filter(col => {\n        const value = recordsWithNulls[i].json[col];\n        return value === null || value === undefined || value === '';\n      })\n    });\n  }\n}\n\nif (anomalyTypes.includes('OUTLIERS_DETECTED')) {\n  // Sample records with outlier values\n  for (const column of affectedColumns) {\n    const values = originalData.map(item => item.json[column]).filter(v => typeof v === 'number');\n    \n    if (values.length > 0) {\n      const stats = getStatistics(values);\n      const sorted = values.sort((a, b) => a - b);\n      const q1 = sorted[Math.floor(sorted.length * 0.25)];\n      const q3 = sorted[Math.floor(sorted.length * 0.75)];\n      const iqr = q3 - q1;\n      const lowerBound = q1 - 1.5 * iqr;\n      const upperBound = q3 + 1.5 * iqr;\n      \n      // Find outlier records\n      const outlierRecords = originalData.filter(item => {\n        const value = item.json[column];\n        return typeof value === 'number' && (value < lowerBound || value > upperBound);\n      });\n      \n      // Sample extreme outliers (top and bottom)\n      const extremeOutliers = outlierRecords\n        .sort((a, b) => Math.abs(b.json[column] - stats.median) - Math.abs(a.json[column] - stats.median))\n        .slice(0, EDGE_CASE_COUNT);\n      \n      extremeOutliers.forEach(item => {\n        selectedSamples.push({\n          record: item.json,\n          reason: `Outlier detected in ${column}`,\n          affectedFields: [column],\n          outlierDetails: {\n            value: item.json[column],\n            median: stats.median,\n            deviation: Math.abs(item.json[column] - stats.median),\n            bounds: { lower: lowerBound, upper: upperBound }\n          }\n        });\n      });\n    }\n  }\n}\n\nif (anomalyTypes.includes('SCHEMA_MISMATCH')) {\n  // Sample records showing schema issues\n  const schemaMismatch = anomalyData.schemaMismatch || {};\n  \n  if (schemaMismatch.missingColumns && schemaMismatch.missingColumns.length > 0) {\n    selectedSamples.push({\n      record: originalData[0]?.json || {},\n      reason: 'Schema mismatch - missing expected columns',\n      affectedFields: schemaMismatch.missingColumns,\n      schemaIssue: 'Missing columns: ' + schemaMismatch.missingColumns.join(', ')\n    });\n  }\n  \n  if (schemaMismatch.extraColumns && schemaMismatch.extraColumns.length > 0) {\n    selectedSamples.push({\n      record: originalData[0]?.json || {},\n      reason: 'Schema mismatch - unexpected columns present',\n      affectedFields: schemaMismatch.extraColumns,\n      schemaIssue: 'Extra columns: ' + schemaMismatch.extraColumns.join(', ')\n    });\n  }\n}\n\n// If no specific samples selected, take random diverse samples\nif (selectedSamples.length === 0 && originalData.length > 0) {\n  const step = Math.max(1, Math.floor(originalData.length / MAX_SAMPLES));\n  for (let i = 0; i < Math.min(MAX_SAMPLES, originalData.length); i += step) {\n    selectedSamples.push({\n      record: originalData[i].json,\n      reason: 'Representative sample from dataset',\n      affectedFields: affectedColumns\n    });\n  }\n}\n\n// Limit to MAX_SAMPLES\nselectedSamples = selectedSamples.slice(0, MAX_SAMPLES);\n\n// Create formatted output for human review\nconst reviewReport = {\n  summary: {\n    anomalyType: classification.classification || anomalyTypes.join(', '),\n    severity: classification.severity || 'unknown',\n    confidence: classification.confidence || 0,\n    totalRecords: originalData.length,\n    samplesIncluded: selectedSamples.length,\n    affectedColumns: affectedColumns,\n    timestamp: new Date().toISOString()\n  },\n  \n  context: {\n    explanation: classification.explanation || 'No explanation provided',\n    attemptedRemediation: remediationPlan.action || 'unknown',\n    remediationReasoning: remediationPlan.reasoning || 'No reasoning provided'\n  },\n  \n  statistics: anomalyData.statistics || {},\n  \n  dataSamples: selectedSamples.map((sample, idx) => ({\n    sampleNumber: idx + 1,\n    reason: sample.reason,\n    affectedFields: sample.affectedFields,\n    outlierDetails: sample.outlierDetails || null,\n    schemaIssue: sample.schemaIssue || null,\n    data: sample.record\n  })),\n  \n  recommendations: [\n    'Review the data samples above to understand the anomaly pattern',\n    'Check if the affected columns require schema changes or validation rules',\n    'Verify if the anomaly is due to upstream data source issues',\n    'Consider updating quality check thresholds if this is expected behavior',\n    'Implement additional validation rules to prevent similar issues'\n  ]\n};\n\nreturn [{ json: reviewReport }];"
      },
      "typeVersion": 2
    }
  ],
  "connections": {
    "Remediate Outliers": {
      "main": [
        [
          {
            "node": "Merge Remediated Data Streams",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Aggregate Incident Logs": {
      "main": [
        [
          {
            "node": "Generate Remediation Report",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Remediate Schema Format": {
      "main": [
        [
          {
            "node": "Merge Remediated Data Streams",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Send Metrics to DataDog": {
      "main": [
        [
          {
            "node": "Update Historical Baseline Metrics",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Set Pipeline Parameters": {
      "main": [
        [
          {
            "node": "Fetch Latest Data from PostgreSQL",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Remediate Missing Values": {
      "main": [
        [
          {
            "node": "Merge Remediated Data Streams",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Check If Anomaly Detected": {
      "main": [
        [
          {
            "node": "AI Anomaly Classifier Agent",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Retry Upstream Data Fetch": {
      "main": [
        [
          {
            "node": "Merge Remediated Data Streams",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Route by Remediation Type": {
      "main": [
        [
          {
            "node": "Remediate Missing Values",
            "type": "main",
            "index": 0
          }
        ],
        [
          {
            "node": "Remediate Outliers",
            "type": "main",
            "index": 0
          }
        ],
        [
          {
            "node": "Remediate Schema Format",
            "type": "main",
            "index": 0
          }
        ],
        [
          {
            "node": "Retry Upstream Data Fetch",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "When Scheduled Run Starts": {
      "main": [
        [
          {
            "node": "Set Pipeline Parameters",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Check If Critical Severity": {
      "main": [
        [
          {
            "node": "Post Critical Alert to PagerDuty",
            "type": "main",
            "index": 0
          }
        ],
        [
          {
            "node": "AI Remediation Planner Agent",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "AI Anomaly Classifier Agent": {
      "main": [
        [
          {
            "node": "AI Remediation Planner Agent",
            "type": "main",
            "index": 0
          },
          {
            "node": "Check If Critical Severity",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Check If High Anomaly Score": {
      "main": [
        [
          {
            "node": "Run Statistical Quality Checks",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Generate Remediation Report": {
      "main": [
        [
          {
            "node": "Send Metrics to DataDog",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "AI Remediation Planner Agent": {
      "main": [
        [
          {
            "node": "Route by Remediation Type",
            "type": "main",
            "index": 0
          },
          {
            "node": "Query Remediation History from PostgreSQL",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Run ML-Based Anomaly Scoring": {
      "main": [
        [
          {
            "node": "Check If High Anomaly Score",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Merge Remediated Data Streams": {
      "main": [
        [
          {
            "node": "Re-run Quality Checks on Remediated Data",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Fetch Historical Baseline Data": {
      "main": [
        [
          {
            "node": "Compare with Historical Baseline",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Prepare Data Sample for Review": {
      "main": [
        [
          {
            "node": "Log Failed Remediation Incident",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Run Statistical Quality Checks": {
      "main": [
        [
          {
            "node": "Check If Anomaly Detected",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Check If Remediation Successful": {
      "main": [
        [
          {
            "node": "Send Success Notification to Slack",
            "type": "main",
            "index": 0
          }
        ],
        [
          {
            "node": "Escalate to Human Review in Slack",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Log Failed Remediation Incident": {
      "main": [
        [
          {
            "node": "Aggregate Incident Logs",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Calculate Trend Analysis Metrics": {
      "main": [
        [
          {
            "node": "Run ML-Based Anomaly Scoring",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Compare with Historical Baseline": {
      "main": [
        [
          {
            "node": "Calculate Trend Analysis Metrics",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Post Critical Alert to PagerDuty": {
      "main": [
        [
          {
            "node": "AI Remediation Planner Agent",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Escalate to Human Review in Slack": {
      "main": [
        [
          {
            "node": "Log Failed Remediation Incident",
            "type": "main",
            "index": 0
          },
          {
            "node": "Prepare Data Sample for Review",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Fetch Latest Data from PostgreSQL": {
      "main": [
        [
          {
            "node": "Run Statistical Quality Checks",
            "type": "main",
            "index": 0
          },
          {
            "node": "Fetch Historical Baseline Data",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Calculate Remediation Success Rate": {
      "main": [
        [
          {
            "node": "Route by Remediation Type",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Send Success Notification to Slack": {
      "main": [
        [
          {
            "node": "Log Successful Remediation Incident",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Log Successful Remediation Incident": {
      "main": [
        [
          {
            "node": "Aggregate Incident Logs",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Re-run Quality Checks on Remediated Data": {
      "main": [
        [
          {
            "node": "Check If Remediation Successful",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Query Remediation History from PostgreSQL": {
      "main": [
        [
          {
            "node": "Calculate Remediation Success Rate",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  },
  "settings": {
    "executionOrder": "v1"
  },
  "active": false
}