{
  "name": "Domain-Scoped Site Crawler for Markdown Datasets (long runs)",
  "nodes": [
    {
      "id": "2e35db0e-e5ad-4deb-abf2-14d65e736dc3",
      "name": "Run by hand",
      "type": "n8n-nodes-base.manualTrigger",
      "typeVersion": 1,
      "position": [
        0,
        0
      ],
      "parameters": {}
    },
    {
      "id": "9bc83376-2afa-4225-a71b-4ab00596eb23",
      "name": "Every day",
      "type": "n8n-nodes-base.scheduleTrigger",
      "typeVersion": 1.2,
      "position": [
        0,
        200
      ],
      "parameters": {
        "rule": {
          "interval": [
            {
              "triggerAtHour": 8
            }
          ]
        }
      },
      "notes": "Activate the workflow to run on this schedule."
    },
    {
      "id": "080200da-c8d5-4e1c-a890-42e1c72a8093",
      "name": "Start Apify run",
      "type": "n8n-nodes-base.httpRequest",
      "typeVersion": 4.2,
      "position": [
        260,
        100
      ],
      "parameters": {
        "method": "POST",
        "url": "https://api.apify.com/v2/actors/vexaloom-labs~domain-scoped-site-crawler-for-markdown-datasets/runs?waitForFinish=60",
        "authentication": "genericCredentialType",
        "genericAuthType": "httpHeaderAuth",
        "sendBody": true,
        "specifyBody": "json",
        "jsonBody": "{\n  \"startUrls\": [\n    {\n      \"url\": \"https://docs.apify.com\"\n    }\n  ],\n  \"maxCrawlDepth\": 1,\n  \"maxCrawlPages\": 3,\n  \"useSitemaps\": false,\n  \"useLlmsTxt\": false,\n  \"respectRobotsTxtFile\": true,\n  \"keepUrlFragments\": false,\n  \"ignoreCanonicalUrl\": false,\n  \"outputFormat\": \"markdown\",\n  \"htmlTransformer\": \"auto\",\n  \"includeImages\": false,\n  \"chunkMarkdown\": false,\n  \"chunkMaxChars\": 2000,\n  \"chunkOverlapChars\": 200,\n  \"parsePdfText\": true,\n  \"takeScreenshots\": false,\n  \"storeSkippedUrls\": false\n}",
        "options": {
          "timeout": 90000
        }
      },
      "notes": "Actor: vexaloom-labs/domain-scoped-site-crawler-for-markdown-datasets. Starts the run and waits up to 60 s for it. Header Auth credential: name Authorization, value Bearer + your Apify token."
    },
    {
      "id": "7670e50e-0f5b-4fcd-a3bd-36644a4d2893",
      "name": "Run finished?",
      "type": "n8n-nodes-base.if",
      "typeVersion": 2.2,
      "position": [
        520,
        100
      ],
      "parameters": {
        "conditions": {
          "options": {
            "caseSensitive": true,
            "leftValue": "",
            "typeValidation": "strict",
            "version": 2
          },
          "conditions": [
            {
              "id": "b2fe4ca6-9d93-4343-af2c-113293d8e7fb",
              "leftValue": "={{ ['SUCCEEDED', 'FAILED', 'ABORTED', 'TIMED-OUT'].includes($json.data.status) }}",
              "rightValue": "",
              "operator": {
                "type": "boolean",
                "operation": "true",
                "singleValue": true
              }
            }
          ],
          "combinator": "and"
        },
        "options": {}
      }
    },
    {
      "id": "e0b6ff92-8a53-44f8-aa40-afc987f7489f",
      "name": "Check the run again",
      "type": "n8n-nodes-base.httpRequest",
      "typeVersion": 4.2,
      "position": [
        520,
        320
      ],
      "parameters": {
        "method": "GET",
        "url": "=https://api.apify.com/v2/actor-runs/{{ $json.data.id }}?waitForFinish=60",
        "authentication": "genericCredentialType",
        "genericAuthType": "httpHeaderAuth",
        "options": {
          "timeout": 90000
        }
      },
      "notes": "Waits up to 60 s per check, then loops back."
    },
    {
      "id": "5d738dae-3ac4-4787-ac8e-507ef87913eb",
      "name": "Run succeeded?",
      "type": "n8n-nodes-base.if",
      "typeVersion": 2.2,
      "position": [
        780,
        0
      ],
      "parameters": {
        "conditions": {
          "options": {
            "caseSensitive": true,
            "leftValue": "",
            "typeValidation": "strict",
            "version": 2
          },
          "conditions": [
            {
              "id": "48937fa6-ffb5-4f09-a778-e368b40754e3",
              "leftValue": "={{ $json.data.status === 'SUCCEEDED' }}",
              "rightValue": "",
              "operator": {
                "type": "boolean",
                "operation": "true",
                "singleValue": true
              }
            }
          ],
          "combinator": "and"
        },
        "options": {}
      }
    },
    {
      "id": "389d3572-eb38-4724-a236-4ff17e9a2637",
      "name": "Get the results",
      "type": "n8n-nodes-base.httpRequest",
      "typeVersion": 4.2,
      "position": [
        1040,
        -100
      ],
      "parameters": {
        "method": "GET",
        "url": "=https://api.apify.com/v2/datasets/{{ $json.data.defaultDatasetId }}/items?clean=true",
        "authentication": "genericCredentialType",
        "genericAuthType": "httpHeaderAuth",
        "options": {
          "timeout": 120000
        }
      },
      "notes": "Each record of the run's dataset becomes one n8n item. Connect your destination after this node."
    },
    {
      "id": "ff8a6976-bc63-4e85-a248-d450c302af5b",
      "name": "Run did not succeed",
      "type": "n8n-nodes-base.stopAndError",
      "typeVersion": 1,
      "position": [
        1040,
        100
      ],
      "parameters": {
        "errorMessage": "={{ 'Apify run ' + $json.data.id + ' ended with status ' + $json.data.status }}"
      }
    },
    {
      "id": "e39a9ae6-1d84-421d-ab3c-908ced721d33",
      "name": "Setup",
      "type": "n8n-nodes-base.stickyNote",
      "typeVersion": 1,
      "position": [
        0,
        -400
      ],
      "parameters": {
        "content": "## Domain-Scoped Site Crawler for Markdown Datasets: long runs\n\nSource: [vexaloom-labs/domain-scoped-site-crawler-for-markdown-datasets](https://apify.com/vexaloom-labs/domain-scoped-site-crawler-for-markdown-datasets) on Apify.\n\nFor runs that need more than 5 minutes (the synchronous endpoint answers HTTP 408 after 300 s).\n1. Select the same Header Auth credential (name `Authorization`, value `Bearer YOUR_APIFY_TOKEN`) in the three HTTP Request nodes.\n2. Edit the JSON body of **Start Apify run** to change the input.\n3. Connect your destination after **Get the results**.\n\nGenerated by recipeforagents.com. Validated, not executed: runs are billed to your Apify account.",
        "height": 360,
        "width": 500,
        "color": 6
      }
    }
  ],
  "connections": {
    "Run by hand": {
      "main": [
        [
          {
            "node": "Start Apify run",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Every day": {
      "main": [
        [
          {
            "node": "Start Apify run",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Start Apify run": {
      "main": [
        [
          {
            "node": "Run finished?",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Run finished?": {
      "main": [
        [
          {
            "node": "Run succeeded?",
            "type": "main",
            "index": 0
          }
        ],
        [
          {
            "node": "Check the run again",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Check the run again": {
      "main": [
        [
          {
            "node": "Run finished?",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Run succeeded?": {
      "main": [
        [
          {
            "node": "Get the results",
            "type": "main",
            "index": 0
          }
        ],
        [
          {
            "node": "Run did not succeed",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  },
  "pinData": {},
  "active": false,
  "settings": {
    "executionOrder": "v1"
  },
  "meta": {
    "templateCredsSetupCompleted": false
  },
  "tags": []
}