What it does

Web scraping for AI agents: scrape, map, crawl and batch public pages into Markdown, tables and JSON, each with an Evidence Record (final URL, HTTP status, robots.txt decision, content hashes); a blocked or empty page comes back with the reason. Hosted at https://mcp.octocrawl.dev/mcp or local via npx -y @octocrawl/mcp. Setup.

Both connectionPricing not confirmedv0.8.1

Connection manifest

Public connection metadata from the source. Follow the provider’s setup guide for authentication, permissions, and required variables.

{
  "name": "io.github.77777R7/octocrawl",
  "version": "0.8.1",
  "repository": {
    "url": "https://github.com/77777R7/Octocrawl"
  },
  "remotes": [
    {
      "type": "streamable-http",
      "url": "https://mcp.octocrawl.dev/mcp"
    },
    {
      "type": "streamable-http",
      "url": "https://mcp.octocrawl.dev/mcp-oauth"
    }
  ],
  "packages": [
    {
      "registryType": "npm",
      "identifier": "@octocrawl/mcp",
      "version": "0.8.1",
      "transport": {
        "type": "stdio"
      },
      "environmentVariables": [
        {
          "name": "W2L_API_URL",
          "isRequired": false
        },
        {
          "name": "W2L_API_TOKEN",
          "isRequired": false
        },
        {
          "name": "W2L_MCP_TOOLSETS",
          "isRequired": false
        },
        {
          "name": "W2L_MCP_READ_ONLY",
          "isRequired": false
        }
      ]
    }
  ]
}

Metadata reviewed for listing. Server code, permissions, and runtime behavior have not been security-audited. Directory access is free; individual providers may charge.