> ## Documentation Index
> Fetch the complete documentation index at: https://docs.bland.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Discover Sitemap URLs

> Discovers URLs from a website's sitemap for web scraping.

Analyzes a website to discover available URLs from its sitemap. This is useful for finding all the pages available on a website before creating a web scraping knowledge base item.

### Headers

**authorization** `string` *required*

Your API key for authentication.

**content-type** `string` *required*

Must be `application/json`.

### Body Parameters

**url** `string` *required*

The base URL of the website to discover sitemap URLs from.

### Response

**data** `object`

Discovered URLs and sitemap information.

**data.urls** `string[]`

Array of discovered URLs from the website's sitemap.

**data.sitemap\_url** `string`

The sitemap URL that was found and processed (if any).

**errors** `null`

Will be `null` on successful discovery.

```bash cURL theme={null}
curl -X POST https://api.bland.ai/v1/knowledge/crawl \
  -H "authorization: Bearer YOUR_API_KEY" \
  -H "content-type: application/json" \
  -d '{
    "url": "https://example.com"
  }'
```

```json Request Body theme={null}
{
  "url": "https://docs.example.com"
}
```

```json Success Response theme={null}
{
  "data": {
    "urls": [
      "https://example.com/",
      "https://example.com/about",
      "https://example.com/products",
      "https://example.com/contact",
      "https://example.com/docs/getting-started",
      "https://example.com/docs/api-reference",
      "https://example.com/docs/tutorials"
    ],
    "sitemap_url": "https://example.com/sitemap.xml"
  },
  "errors": null
}
```

```json No Sitemap Found theme={null}
{
  "data": {
    "urls": [],
    "sitemap_url": null
  },
  "errors": null
}
```

```json Error Response theme={null}
{
  "data": null,
  "errors": [
    {
      "error": "INVALID_INPUT",
      "message": "Invalid URL provided"
    }
  ]
}
```

***

Docs for agents: [llms.txt](https://docs.bland.ai/llms.txt)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.